2024-09-04T17:32:28.497 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-04T17:32:28.520 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-04T17:32:28.603 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/dis-2024-09-04_17:03:08-rbd-wip-dis-testing-distro-default-smithi/7889281 branch: wip-dis-testing description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_fsx_rate_limit} email: idryomov@gmail.com first_in_suite: false job_id: '7889281' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: dis-2024-09-04_17:03:08-rbd-wip-dis-testing-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 - machine: cpus: 1 disk: 40 ram: 8000 volumes: count: 4 size: 30 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-dis-testing ceph: conf: client: rbd qos iops burst: 100 rbd qos iops limit: 50 rbd qos schedule tick min: 100 global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - SLOW_OPS - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum - noscrub - nodeep-scrub sha1: 3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: live-migration-nbd sha1: 250c2b7303ac79643ee967bc9c5b14e5a0dbbf52 owner: scheduled_dis@teuthology priority: 70 repo: https://github.com/ceph/ceph-ci roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 6930 sha1: 3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab sleep_before_teardown: 0 subset: 4/100 suite: rbd suite_branch: live-migration-nbd suite_path: /home/teuthworker/src/github.com_effi-ofer_ceph_250c2b7303ac79643ee967bc9c5b14e5a0dbbf52/qa suite_relpath: qa suite_repo: https://github.com/effi-ofer/ceph suite_sha1: 250c2b7303ac79643ee967bc9c5b14e5a0dbbf52 targets: smithi092.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEwEsJaiBW8otX9/ecRoIhVM4N79EMJJCpGCoieqhT81V3quwVZkCcIa5CNExYFVSOCc818jHJclc/bvKk3oa+Y= smithi190.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBACsZpD3Vefj1a51cbkMHbuTP2LQnWBs+GbywGhmgZlPy+EqPqpCmDKPoOVh1KQq0JcY3EaJl/KAZ1BqcH3hto0= tasks: - install: null - ceph: null - thrashosds: timeout: 1200 - rbd_fsx: clients: - client.0 ops: 6000 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-04_17:03:08 tube: smithi user: dis verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-04T17:32:28.603 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_effi-ofer_ceph_250c2b7303ac79643ee967bc9c5b14e5a0dbbf52/qa; will attempt to use it 2024-09-04T17:32:28.604 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_effi-ofer_ceph_250c2b7303ac79643ee967bc9c5b14e5a0dbbf52/qa/tasks 2024-09-04T17:32:28.604 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-04T17:32:28.606 INFO:teuthology.task.internal:Checking packages... 2024-09-04T17:32:28.628 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab' 2024-09-04T17:32:28.628 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-04T17:32:28.628 INFO:teuthology.packaging:ref: None 2024-09-04T17:32:28.628 INFO:teuthology.packaging:tag: None 2024-09-04T17:32:28.628 INFO:teuthology.packaging:branch: wip-dis-testing 2024-09-04T17:32:28.628 INFO:teuthology.packaging:sha1: 3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab 2024-09-04T17:32:28.629 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-dis-testing 2024-09-04T17:32:28.782 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4703-g3e4a7f4c-1jammy 2024-09-04T17:32:28.784 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-04T17:32:28.791 INFO:teuthology.task.internal:no buildpackages task found 2024-09-04T17:32:28.791 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-04T17:32:28.817 INFO:teuthology.task.internal:Saving configuration 2024-09-04T17:32:28.831 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-04T17:32:28.843 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-04T17:32:28.868 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi092.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dis-2024-09-04_17:03:08-rbd-wip-dis-testing-distro-default-smithi/7889281', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-04 17:26:49.654249', 'locked_by': 'scheduled_dis@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAICNYSZh5uxyigWM2B6KjKagHY8kXc9gJ4EoIrneLylcM'} 2024-09-04T17:32:28.893 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi190.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dis-2024-09-04_17:03:08-rbd-wip-dis-testing-distro-default-smithi/7889281', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-04 17:26:49.652654', 'locked_by': 'scheduled_dis@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBBcg/h6jGNdfQrx5rWlUAMqgzto+32AtphLTv9tq30I'} 2024-09-04T17:32:28.893 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-04T17:32:28.902 INFO:teuthology.task.internal:roles: ubuntu@smithi092.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-09-04T17:32:28.902 INFO:teuthology.task.internal:roles: ubuntu@smithi190.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-09-04T17:32:28.902 INFO:teuthology.run_tasks:Running task console_log... 2024-09-04T17:32:28.976 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f6c57f8bac0>, signals=[15]) 2024-09-04T17:32:28.976 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-04T17:32:28.986 INFO:teuthology.task.internal:Opening connections... 2024-09-04T17:32:28.987 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi092.front.sepia.ceph.com 2024-09-04T17:32:28.988 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-04T17:32:29.070 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi190.front.sepia.ceph.com 2024-09-04T17:32:29.071 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-04T17:32:29.150 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-04T17:32:29.161 DEBUG:teuthology.orchestra.run.smithi092:> uname -m 2024-09-04T17:32:29.166 INFO:teuthology.orchestra.run.smithi092.stdout:x86_64 2024-09-04T17:32:29.166 DEBUG:teuthology.orchestra.run.smithi092:> cat /etc/os-release 2024-09-04T17:32:29.214 INFO:teuthology.orchestra.run.smithi092.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-04T17:32:29.214 INFO:teuthology.orchestra.run.smithi092.stdout:NAME="Ubuntu" 2024-09-04T17:32:29.214 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION_ID="22.04" 2024-09-04T17:32:29.214 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-04T17:32:29.214 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION_CODENAME=jammy 2024-09-04T17:32:29.214 INFO:teuthology.orchestra.run.smithi092.stdout:ID=ubuntu 2024-09-04T17:32:29.214 INFO:teuthology.orchestra.run.smithi092.stdout:ID_LIKE=debian 2024-09-04T17:32:29.214 INFO:teuthology.orchestra.run.smithi092.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-04T17:32:29.214 INFO:teuthology.orchestra.run.smithi092.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-04T17:32:29.215 INFO:teuthology.orchestra.run.smithi092.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-04T17:32:29.215 INFO:teuthology.orchestra.run.smithi092.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-04T17:32:29.215 INFO:teuthology.orchestra.run.smithi092.stdout:UBUNTU_CODENAME=jammy 2024-09-04T17:32:29.215 INFO:teuthology.lock.ops:Updating smithi092.front.sepia.ceph.com on lock server 2024-09-04T17:32:29.237 DEBUG:teuthology.orchestra.run.smithi190:> uname -m 2024-09-04T17:32:29.243 INFO:teuthology.orchestra.run.smithi190.stdout:x86_64 2024-09-04T17:32:29.243 DEBUG:teuthology.orchestra.run.smithi190:> cat /etc/os-release 2024-09-04T17:32:29.290 INFO:teuthology.orchestra.run.smithi190.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-04T17:32:29.291 INFO:teuthology.orchestra.run.smithi190.stdout:NAME="Ubuntu" 2024-09-04T17:32:29.291 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION_ID="22.04" 2024-09-04T17:32:29.291 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-04T17:32:29.291 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION_CODENAME=jammy 2024-09-04T17:32:29.291 INFO:teuthology.orchestra.run.smithi190.stdout:ID=ubuntu 2024-09-04T17:32:29.291 INFO:teuthology.orchestra.run.smithi190.stdout:ID_LIKE=debian 2024-09-04T17:32:29.291 INFO:teuthology.orchestra.run.smithi190.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-04T17:32:29.291 INFO:teuthology.orchestra.run.smithi190.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-04T17:32:29.291 INFO:teuthology.orchestra.run.smithi190.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-04T17:32:29.291 INFO:teuthology.orchestra.run.smithi190.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-04T17:32:29.291 INFO:teuthology.orchestra.run.smithi190.stdout:UBUNTU_CODENAME=jammy 2024-09-04T17:32:29.292 INFO:teuthology.lock.ops:Updating smithi190.front.sepia.ceph.com on lock server 2024-09-04T17:32:29.312 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-04T17:32:29.326 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-04T17:32:29.389 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-04T17:32:29.390 DEBUG:teuthology.orchestra.run.smithi092:> test '!' -e /home/ubuntu/cephtest 2024-09-04T17:32:29.392 DEBUG:teuthology.orchestra.run.smithi190:> test '!' -e /home/ubuntu/cephtest 2024-09-04T17:32:29.397 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-04T17:32:29.405 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-04T17:32:29.405 DEBUG:teuthology.orchestra.run.smithi092:> test -z $(ls -A /var/lib/ceph) 2024-09-04T17:32:29.440 DEBUG:teuthology.orchestra.run.smithi190:> test -z $(ls -A /var/lib/ceph) 2024-09-04T17:32:29.450 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-04T17:32:29.509 INFO:teuthology.run_tasks:Running task kernel... 2024-09-04T17:32:29.526 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-04T17:32:29.526 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-04T17:32:29.526 DEBUG:teuthology.orchestra.run.smithi092:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-04T17:32:29.527 DEBUG:teuthology.orchestra.run.smithi190:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-04T17:32:29.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-04T17:32:29.531 DEBUG:teuthology.orchestra.run.smithi092:> uname -r 2024-09-04T17:32:29.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-04T17:32:29.532 DEBUG:teuthology.orchestra.run.smithi190:> uname -r 2024-09-04T17:32:29.578 INFO:teuthology.orchestra.run.smithi092.stdout:5.15.0-56-generic 2024-09-04T17:32:29.578 INFO:teuthology.task.kernel:Running kernel on smithi092: 5.15.0-56-generic 2024-09-04T17:32:29.578 DEBUG:teuthology.orchestra.run.smithi092:> sudo apt-get clean 2024-09-04T17:32:29.579 INFO:teuthology.orchestra.run.smithi190.stdout:5.15.0-56-generic 2024-09-04T17:32:29.580 INFO:teuthology.task.kernel:Running kernel on smithi190: 5.15.0-56-generic 2024-09-04T17:32:29.580 DEBUG:teuthology.orchestra.run.smithi190:> sudo apt-get clean 2024-09-04T17:32:29.719 DEBUG:teuthology.orchestra.run.smithi092:> sudo apt-get update 2024-09-04T17:32:29.720 DEBUG:teuthology.orchestra.run.smithi190:> sudo apt-get update 2024-09-04T17:32:30.017 INFO:teuthology.orchestra.run.smithi092.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-04T17:32:30.018 INFO:teuthology.orchestra.run.smithi092.stdout:Get:2 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-04T17:32:30.039 INFO:teuthology.orchestra.run.smithi190.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-04T17:32:30.039 INFO:teuthology.orchestra.run.smithi190.stdout:Get:2 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-04T17:32:30.071 INFO:teuthology.orchestra.run.smithi092.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-04T17:32:30.075 INFO:teuthology.orchestra.run.smithi190.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-04T17:32:30.425 INFO:teuthology.orchestra.run.smithi092.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-04T17:32:30.431 INFO:teuthology.orchestra.run.smithi190.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-04T17:32:30.439 INFO:teuthology.orchestra.run.smithi092.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,771 kB] 2024-09-04T17:32:30.445 INFO:teuthology.orchestra.run.smithi190.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [533 kB] 2024-09-04T17:32:30.565 INFO:teuthology.orchestra.run.smithi092.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,988 kB] 2024-09-04T17:32:30.571 INFO:teuthology.orchestra.run.smithi190.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [690 kB] 2024-09-04T17:32:30.740 INFO:teuthology.orchestra.run.smithi190.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,771 kB] 2024-09-04T17:32:30.765 INFO:teuthology.orchestra.run.smithi190.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,988 kB] 2024-09-04T17:32:30.870 INFO:teuthology.orchestra.run.smithi092.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [533 kB] 2024-09-04T17:32:30.879 INFO:teuthology.orchestra.run.smithi092.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [690 kB] 2024-09-04T17:32:30.887 INFO:teuthology.orchestra.run.smithi092.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [291 kB] 2024-09-04T17:32:30.895 INFO:teuthology.orchestra.run.smithi092.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-04T17:32:30.896 INFO:teuthology.orchestra.run.smithi092.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-04T17:32:30.897 INFO:teuthology.orchestra.run.smithi092.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,327 kB] 2024-09-04T17:32:30.913 INFO:teuthology.orchestra.run.smithi190.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [291 kB] 2024-09-04T17:32:30.917 INFO:teuthology.orchestra.run.smithi092.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [349 kB] 2024-09-04T17:32:30.922 INFO:teuthology.orchestra.run.smithi190.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [349 kB] 2024-09-04T17:32:30.925 INFO:teuthology.orchestra.run.smithi190.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-04T17:32:30.926 INFO:teuthology.orchestra.run.smithi190.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-04T17:32:30.928 INFO:teuthology.orchestra.run.smithi190.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,327 kB] 2024-09-04T17:32:30.931 INFO:teuthology.orchestra.run.smithi092.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-04T17:32:30.932 INFO:teuthology.orchestra.run.smithi092.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-04T17:32:30.933 INFO:teuthology.orchestra.run.smithi092.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,386 kB] 2024-09-04T17:32:30.937 INFO:teuthology.orchestra.run.smithi190.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-04T17:32:30.937 INFO:teuthology.orchestra.run.smithi190.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-04T17:32:30.938 INFO:teuthology.orchestra.run.smithi190.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,386 kB] 2024-09-04T17:32:30.993 INFO:teuthology.orchestra.run.smithi092.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [400 kB] 2024-09-04T17:32:31.016 INFO:teuthology.orchestra.run.smithi190.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [410 kB] 2024-09-04T17:32:31.016 INFO:teuthology.orchestra.run.smithi190.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [400 kB] 2024-09-04T17:32:31.021 INFO:teuthology.orchestra.run.smithi190.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-04T17:32:31.021 INFO:teuthology.orchestra.run.smithi190.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,121 kB] 2024-09-04T17:32:31.024 INFO:teuthology.orchestra.run.smithi190.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-04T17:32:31.024 INFO:teuthology.orchestra.run.smithi190.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [899 kB] 2024-09-04T17:32:31.035 INFO:teuthology.orchestra.run.smithi092.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [410 kB] 2024-09-04T17:32:31.036 INFO:teuthology.orchestra.run.smithi092.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-04T17:32:31.036 INFO:teuthology.orchestra.run.smithi092.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [899 kB] 2024-09-04T17:32:31.043 INFO:teuthology.orchestra.run.smithi092.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-04T17:32:31.043 INFO:teuthology.orchestra.run.smithi092.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,121 kB] 2024-09-04T17:32:31.048 INFO:teuthology.orchestra.run.smithi190.stdout:Get:23 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [626 kB] 2024-09-04T17:32:31.051 INFO:teuthology.orchestra.run.smithi092.stdout:Get:23 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [626 kB] 2024-09-04T17:32:31.056 INFO:teuthology.orchestra.run.smithi190.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [730 kB] 2024-09-04T17:32:31.062 INFO:teuthology.orchestra.run.smithi092.stdout:Get:24 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [175 kB] 2024-09-04T17:32:31.063 INFO:teuthology.orchestra.run.smithi092.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [730 kB] 2024-09-04T17:32:31.065 INFO:teuthology.orchestra.run.smithi092.stdout:Get:26 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.2 kB] 2024-09-04T17:32:31.066 INFO:teuthology.orchestra.run.smithi092.stdout:Get:27 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-04T17:32:31.066 INFO:teuthology.orchestra.run.smithi092.stdout:Get:28 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-04T17:32:31.067 INFO:teuthology.orchestra.run.smithi092.stdout:Get:29 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-04T17:32:31.070 INFO:teuthology.orchestra.run.smithi190.stdout:Get:25 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [175 kB] 2024-09-04T17:32:31.074 INFO:teuthology.orchestra.run.smithi190.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [260 kB] 2024-09-04T17:32:31.075 INFO:teuthology.orchestra.run.smithi092.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [260 kB] 2024-09-04T17:32:31.075 INFO:teuthology.orchestra.run.smithi190.stdout:Get:27 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.2 kB] 2024-09-04T17:32:31.076 INFO:teuthology.orchestra.run.smithi190.stdout:Get:28 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-04T17:32:31.077 INFO:teuthology.orchestra.run.smithi190.stdout:Get:29 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-04T17:32:31.077 INFO:teuthology.orchestra.run.smithi190.stdout:Get:30 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-04T17:32:31.079 INFO:teuthology.orchestra.run.smithi092.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-04T17:32:31.079 INFO:teuthology.orchestra.run.smithi092.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-04T17:32:31.079 INFO:teuthology.orchestra.run.smithi092.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-04T17:32:31.081 INFO:teuthology.orchestra.run.smithi190.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-04T17:32:31.081 INFO:teuthology.orchestra.run.smithi092.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-04T17:32:31.086 INFO:teuthology.orchestra.run.smithi190.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-04T17:32:31.087 INFO:teuthology.orchestra.run.smithi190.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-04T17:32:31.087 INFO:teuthology.orchestra.run.smithi190.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-04T17:32:31.125 INFO:teuthology.orchestra.run.smithi092.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-04T17:32:31.132 INFO:teuthology.orchestra.run.smithi092.stdout:Get:36 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-04T17:32:31.158 INFO:teuthology.orchestra.run.smithi190.stdout:Get:35 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-04T17:32:31.162 INFO:teuthology.orchestra.run.smithi092.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-04T17:32:31.163 INFO:teuthology.orchestra.run.smithi190.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-04T17:32:31.163 INFO:teuthology.orchestra.run.smithi190.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-04T17:32:31.164 INFO:teuthology.orchestra.run.smithi092.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-04T17:32:31.164 INFO:teuthology.orchestra.run.smithi190.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-04T17:32:31.165 INFO:teuthology.orchestra.run.smithi190.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-04T17:32:31.165 INFO:teuthology.orchestra.run.smithi190.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-04T17:32:31.166 INFO:teuthology.orchestra.run.smithi092.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-04T17:32:31.166 INFO:teuthology.orchestra.run.smithi092.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-04T17:32:31.166 INFO:teuthology.orchestra.run.smithi092.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-04T17:32:31.167 INFO:teuthology.orchestra.run.smithi190.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-04T17:32:31.168 INFO:teuthology.orchestra.run.smithi190.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-04T17:32:31.207 INFO:teuthology.orchestra.run.smithi092.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-04T17:32:31.208 INFO:teuthology.orchestra.run.smithi092.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-04T17:32:31.208 INFO:teuthology.orchestra.run.smithi092.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-04T17:32:31.249 INFO:teuthology.orchestra.run.smithi190.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-04T17:32:31.249 INFO:teuthology.orchestra.run.smithi190.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-04T17:32:37.156 INFO:teuthology.orchestra.run.smithi190.stdout:Fetched 15.8 MB in 3s (5,806 kB/s) 2024-09-04T17:32:37.207 INFO:teuthology.orchestra.run.smithi092.stdout:Fetched 15.8 MB in 3s (5,769 kB/s) 2024-09-04T17:32:38.682 INFO:teuthology.orchestra.run.smithi190.stdout:Reading package lists... 2024-09-04T17:32:38.701 DEBUG:teuthology.orchestra.run.smithi190:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-04T17:32:38.764 INFO:teuthology.orchestra.run.smithi190.stdout:Reading package lists... 2024-09-04T17:32:38.798 INFO:teuthology.orchestra.run.smithi092.stdout:Reading package lists... 2024-09-04T17:32:38.817 DEBUG:teuthology.orchestra.run.smithi092:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-04T17:32:38.881 INFO:teuthology.orchestra.run.smithi092.stdout:Reading package lists... 2024-09-04T17:32:38.949 INFO:teuthology.orchestra.run.smithi190.stdout:Building dependency tree... 2024-09-04T17:32:38.950 INFO:teuthology.orchestra.run.smithi190.stdout:Reading state information... 2024-09-04T17:32:39.099 INFO:teuthology.orchestra.run.smithi092.stdout:Building dependency tree... 2024-09-04T17:32:39.099 INFO:teuthology.orchestra.run.smithi092.stdout:Reading state information... 2024-09-04T17:32:39.112 INFO:teuthology.orchestra.run.smithi190.stdout:The following packages were automatically installed and are no longer required: 2024-09-04T17:32:39.112 INFO:teuthology.orchestra.run.smithi190.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-04T17:32:39.112 INFO:teuthology.orchestra.run.smithi190.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-04T17:32:39.113 INFO:teuthology.orchestra.run.smithi190.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-04T17:32:39.113 INFO:teuthology.orchestra.run.smithi190.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-04T17:32:39.113 INFO:teuthology.orchestra.run.smithi190.stdout: python2.7-minimal 2024-09-04T17:32:39.114 INFO:teuthology.orchestra.run.smithi190.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-04T17:32:39.115 INFO:teuthology.orchestra.run.smithi190.stdout:The following additional packages will be installed: 2024-09-04T17:32:39.115 INFO:teuthology.orchestra.run.smithi190.stdout: linux-generic linux-headers-5.15.0-119 linux-headers-5.15.0-119-generic 2024-09-04T17:32:39.115 INFO:teuthology.orchestra.run.smithi190.stdout: linux-headers-generic linux-image-5.15.0-119-generic 2024-09-04T17:32:39.115 INFO:teuthology.orchestra.run.smithi190.stdout: linux-modules-5.15.0-119-generic linux-modules-extra-5.15.0-119-generic 2024-09-04T17:32:39.116 INFO:teuthology.orchestra.run.smithi190.stdout:Suggested packages: 2024-09-04T17:32:39.116 INFO:teuthology.orchestra.run.smithi190.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-04T17:32:39.117 INFO:teuthology.orchestra.run.smithi190.stdout:Recommended packages: 2024-09-04T17:32:39.117 INFO:teuthology.orchestra.run.smithi190.stdout: thermald 2024-09-04T17:32:39.172 INFO:teuthology.orchestra.run.smithi190.stdout:The following NEW packages will be installed: 2024-09-04T17:32:39.172 INFO:teuthology.orchestra.run.smithi190.stdout: linux-headers-5.15.0-119 linux-headers-5.15.0-119-generic 2024-09-04T17:32:39.172 INFO:teuthology.orchestra.run.smithi190.stdout: linux-image-5.15.0-119-generic linux-modules-5.15.0-119-generic 2024-09-04T17:32:39.172 INFO:teuthology.orchestra.run.smithi190.stdout: linux-modules-extra-5.15.0-119-generic 2024-09-04T17:32:39.174 INFO:teuthology.orchestra.run.smithi190.stdout:The following packages will be upgraded: 2024-09-04T17:32:39.174 INFO:teuthology.orchestra.run.smithi190.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-04T17:32:39.225 INFO:teuthology.orchestra.run.smithi190.stdout:3 upgraded, 5 newly installed, 0 to remove and 344 not upgraded. 2024-09-04T17:32:39.225 INFO:teuthology.orchestra.run.smithi190.stdout:Need to get 113 MB of archives. 2024-09-04T17:32:39.225 INFO:teuthology.orchestra.run.smithi190.stdout:After this operation, 583 MB of additional disk space will be used. 2024-09-04T17:32:39.225 INFO:teuthology.orchestra.run.smithi190.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-119-generic amd64 5.15.0-119.129 [22.6 MB] 2024-09-04T17:32:39.274 INFO:teuthology.orchestra.run.smithi092.stdout:The following packages were automatically installed and are no longer required: 2024-09-04T17:32:39.274 INFO:teuthology.orchestra.run.smithi092.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-04T17:32:39.274 INFO:teuthology.orchestra.run.smithi092.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-04T17:32:39.275 INFO:teuthology.orchestra.run.smithi092.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-04T17:32:39.275 INFO:teuthology.orchestra.run.smithi092.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-04T17:32:39.275 INFO:teuthology.orchestra.run.smithi092.stdout: python2.7-minimal 2024-09-04T17:32:39.275 INFO:teuthology.orchestra.run.smithi092.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-04T17:32:39.276 INFO:teuthology.orchestra.run.smithi092.stdout:The following additional packages will be installed: 2024-09-04T17:32:39.276 INFO:teuthology.orchestra.run.smithi092.stdout: linux-generic linux-headers-5.15.0-119 linux-headers-5.15.0-119-generic 2024-09-04T17:32:39.276 INFO:teuthology.orchestra.run.smithi092.stdout: linux-headers-generic linux-image-5.15.0-119-generic 2024-09-04T17:32:39.277 INFO:teuthology.orchestra.run.smithi092.stdout: linux-modules-5.15.0-119-generic linux-modules-extra-5.15.0-119-generic 2024-09-04T17:32:39.278 INFO:teuthology.orchestra.run.smithi092.stdout:Suggested packages: 2024-09-04T17:32:39.278 INFO:teuthology.orchestra.run.smithi092.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-04T17:32:39.278 INFO:teuthology.orchestra.run.smithi092.stdout:Recommended packages: 2024-09-04T17:32:39.278 INFO:teuthology.orchestra.run.smithi092.stdout: thermald 2024-09-04T17:32:39.329 INFO:teuthology.orchestra.run.smithi092.stdout:The following NEW packages will be installed: 2024-09-04T17:32:39.329 INFO:teuthology.orchestra.run.smithi092.stdout: linux-headers-5.15.0-119 linux-headers-5.15.0-119-generic 2024-09-04T17:32:39.329 INFO:teuthology.orchestra.run.smithi092.stdout: linux-image-5.15.0-119-generic linux-modules-5.15.0-119-generic 2024-09-04T17:32:39.329 INFO:teuthology.orchestra.run.smithi092.stdout: linux-modules-extra-5.15.0-119-generic 2024-09-04T17:32:39.330 INFO:teuthology.orchestra.run.smithi092.stdout:The following packages will be upgraded: 2024-09-04T17:32:39.330 INFO:teuthology.orchestra.run.smithi092.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-04T17:32:39.510 INFO:teuthology.orchestra.run.smithi092.stdout:3 upgraded, 5 newly installed, 0 to remove and 344 not upgraded. 2024-09-04T17:32:39.511 INFO:teuthology.orchestra.run.smithi092.stdout:Need to get 113 MB of archives. 2024-09-04T17:32:39.511 INFO:teuthology.orchestra.run.smithi092.stdout:After this operation, 583 MB of additional disk space will be used. 2024-09-04T17:32:39.511 INFO:teuthology.orchestra.run.smithi092.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-119-generic amd64 5.15.0-119.129 [22.6 MB] 2024-09-04T17:32:39.642 INFO:teuthology.orchestra.run.smithi190.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-119-generic amd64 5.15.0-119.129 [11.5 MB] 2024-09-04T17:32:39.731 INFO:teuthology.orchestra.run.smithi190.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-119-generic amd64 5.15.0-119.129 [63.9 MB] 2024-09-04T17:32:40.505 INFO:teuthology.orchestra.run.smithi190.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.119.119 [1,694 B] 2024-09-04T17:32:40.505 INFO:teuthology.orchestra.run.smithi190.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.119.119 [2,502 B] 2024-09-04T17:32:40.505 INFO:teuthology.orchestra.run.smithi190.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-119 all 5.15.0-119.129 [12.3 MB] 2024-09-04T17:32:40.628 INFO:teuthology.orchestra.run.smithi190.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-119-generic amd64 5.15.0-119.129 [2,879 kB] 2024-09-04T17:32:40.724 INFO:teuthology.orchestra.run.smithi190.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.119.119 [2,350 B] 2024-09-04T17:32:40.909 INFO:teuthology.orchestra.run.smithi092.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-119-generic amd64 5.15.0-119.129 [11.5 MB] 2024-09-04T17:32:41.117 INFO:teuthology.orchestra.run.smithi190.stdout:Fetched 113 MB in 2s (73.4 MB/s) 2024-09-04T17:32:41.257 INFO:teuthology.orchestra.run.smithi092.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-119-generic amd64 5.15.0-119.129 [63.9 MB] 2024-09-04T17:32:41.327 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package linux-modules-5.15.0-119-generic. 2024-09-04T17:32:43.291 INFO:teuthology.orchestra.run.smithi190.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-04T17:32:43.296 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../0-linux-modules-5.15.0-119-generic_5.15.0-119.129_amd64.deb ... 2024-09-04T17:32:43.415 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking linux-modules-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-04T17:32:43.450 INFO:teuthology.orchestra.run.smithi092.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.119.119 [1,694 B] 2024-09-04T17:32:43.450 INFO:teuthology.orchestra.run.smithi092.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.119.119 [2,502 B] 2024-09-04T17:32:43.537 INFO:teuthology.orchestra.run.smithi092.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-119 all 5.15.0-119.129 [12.3 MB] 2024-09-04T17:32:43.885 INFO:teuthology.orchestra.run.smithi092.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-119-generic amd64 5.15.0-119.129 [2,879 kB] 2024-09-04T17:32:43.964 INFO:teuthology.orchestra.run.smithi092.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.119.119 [2,350 B] 2024-09-04T17:32:44.374 INFO:teuthology.orchestra.run.smithi092.stdout:Fetched 113 MB in 5s (24.5 MB/s) 2024-09-04T17:32:44.588 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package linux-modules-5.15.0-119-generic. 2024-09-04T17:32:45.574 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package linux-image-5.15.0-119-generic. 2024-09-04T17:32:45.595 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../1-linux-image-5.15.0-119-generic_5.15.0-119.129_amd64.deb ... 2024-09-04T17:32:45.683 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking linux-image-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-04T17:32:46.519 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-119-generic. 2024-09-04T17:32:46.540 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-119-generic_5.15.0-119.129_amd64.deb ... 2024-09-04T17:32:46.568 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking linux-modules-extra-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-04T17:32:46.606 INFO:teuthology.orchestra.run.smithi092.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-04T17:32:46.611 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../0-linux-modules-5.15.0-119-generic_5.15.0-119.129_amd64.deb ... 2024-09-04T17:32:46.735 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking linux-modules-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-04T17:32:49.251 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package linux-image-5.15.0-119-generic. 2024-09-04T17:32:49.272 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../1-linux-image-5.15.0-119-generic_5.15.0-119.129_amd64.deb ... 2024-09-04T17:32:49.362 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking linux-image-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-04T17:32:49.856 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-119-generic. 2024-09-04T17:32:49.876 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-119-generic_5.15.0-119.129_amd64.deb ... 2024-09-04T17:32:49.914 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking linux-modules-extra-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-04T17:32:53.117 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../3-linux-generic_5.15.0.119.119_amd64.deb ... 2024-09-04T17:32:53.234 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking linux-generic (5.15.0.119.119) over (5.15.0.56.54) ... 2024-09-04T17:32:53.575 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.119.119_amd64.deb ... 2024-09-04T17:32:53.696 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking linux-image-generic (5.15.0.119.119) over (5.15.0.56.54) ... 2024-09-04T17:32:54.024 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package linux-headers-5.15.0-119. 2024-09-04T17:32:54.047 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../5-linux-headers-5.15.0-119_5.15.0-119.129_all.deb ... 2024-09-04T17:32:54.091 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking linux-headers-5.15.0-119 (5.15.0-119.129) ... 2024-09-04T17:32:56.723 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../3-linux-generic_5.15.0.119.119_amd64.deb ... 2024-09-04T17:32:56.859 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking linux-generic (5.15.0.119.119) over (5.15.0.56.54) ... 2024-09-04T17:32:57.243 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.119.119_amd64.deb ... 2024-09-04T17:32:57.429 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking linux-image-generic (5.15.0.119.119) over (5.15.0.56.54) ... 2024-09-04T17:32:57.749 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package linux-headers-5.15.0-119. 2024-09-04T17:32:57.771 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../5-linux-headers-5.15.0-119_5.15.0-119.129_all.deb ... 2024-09-04T17:32:57.832 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking linux-headers-5.15.0-119 (5.15.0-119.129) ... 2024-09-04T17:32:59.857 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package linux-headers-5.15.0-119-generic. 2024-09-04T17:32:59.895 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../6-linux-headers-5.15.0-119-generic_5.15.0-119.129_amd64.deb ... 2024-09-04T17:32:59.932 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking linux-headers-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-04T17:33:03.261 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.119.119_amd64.deb ... 2024-09-04T17:33:03.386 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking linux-headers-generic (5.15.0.119.119) over (5.15.0.56.54) ... 2024-09-04T17:33:03.753 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up linux-headers-5.15.0-119 (5.15.0-119.129) ... 2024-09-04T17:33:03.874 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up linux-headers-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-04T17:33:03.937 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package linux-headers-5.15.0-119-generic. 2024-09-04T17:33:03.973 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../6-linux-headers-5.15.0-119-generic_5.15.0-119.129_amd64.deb ... 2024-09-04T17:33:04.012 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking linux-headers-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-04T17:33:04.017 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up linux-headers-generic (5.15.0.119.119) ... 2024-09-04T17:33:04.129 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up linux-image-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-04T17:33:05.591 INFO:teuthology.orchestra.run.smithi190.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-04T17:33:05.591 INFO:teuthology.orchestra.run.smithi190.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-04T17:33:05.591 INFO:teuthology.orchestra.run.smithi190.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-119-generic 2024-09-04T17:33:05.592 INFO:teuthology.orchestra.run.smithi190.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-119-generic 2024-09-04T17:33:06.340 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up linux-modules-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-04T17:33:07.733 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up linux-modules-extra-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-04T17:33:07.843 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.119.119_amd64.deb ... 2024-09-04T17:33:07.965 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking linux-headers-generic (5.15.0.119.119) over (5.15.0.56.54) ... 2024-09-04T17:33:08.340 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up linux-headers-5.15.0-119 (5.15.0-119.129) ... 2024-09-04T17:33:08.479 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up linux-headers-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-04T17:33:08.630 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up linux-headers-generic (5.15.0.119.119) ... 2024-09-04T17:33:08.759 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up linux-image-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-04T17:33:09.126 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up linux-image-generic (5.15.0.119.119) ... 2024-09-04T17:33:09.252 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up linux-generic (5.15.0.119.119) ... 2024-09-04T17:33:09.411 INFO:teuthology.orchestra.run.smithi190.stdout:Processing triggers for linux-image-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-04T17:33:09.481 INFO:teuthology.orchestra.run.smithi190.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-04T17:33:09.481 INFO:teuthology.orchestra.run.smithi190.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-119-generic 2024-09-04T17:33:10.066 INFO:teuthology.orchestra.run.smithi092.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-04T17:33:10.066 INFO:teuthology.orchestra.run.smithi092.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-04T17:33:10.066 INFO:teuthology.orchestra.run.smithi092.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-119-generic 2024-09-04T17:33:10.066 INFO:teuthology.orchestra.run.smithi092.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-119-generic 2024-09-04T17:33:10.263 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up linux-modules-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-04T17:33:12.506 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up linux-modules-extra-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-04T17:33:13.932 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up linux-image-generic (5.15.0.119.119) ... 2024-09-04T17:33:14.074 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up linux-generic (5.15.0.119.119) ... 2024-09-04T17:33:14.201 INFO:teuthology.orchestra.run.smithi092.stdout:Processing triggers for linux-image-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-04T17:33:14.308 INFO:teuthology.orchestra.run.smithi092.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-04T17:33:14.308 INFO:teuthology.orchestra.run.smithi092.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-119-generic 2024-09-04T17:33:23.605 INFO:teuthology.orchestra.run.smithi190.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-04T17:33:23.605 INFO:teuthology.orchestra.run.smithi190.stdout:Sourcing file `/etc/default/grub' 2024-09-04T17:33:23.641 INFO:teuthology.orchestra.run.smithi190.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-04T17:33:23.667 INFO:teuthology.orchestra.run.smithi190.stdout:Generating grub configuration file ... 2024-09-04T17:33:23.940 INFO:teuthology.orchestra.run.smithi190.stdout:Found linux image: /boot/vmlinuz-5.15.0-119-generic 2024-09-04T17:33:23.954 INFO:teuthology.orchestra.run.smithi190.stdout:Found initrd image: /boot/initrd.img-5.15.0-119-generic 2024-09-04T17:33:24.258 INFO:teuthology.orchestra.run.smithi190.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-04T17:33:24.266 INFO:teuthology.orchestra.run.smithi190.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-04T17:33:24.345 INFO:teuthology.orchestra.run.smithi190.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-04T17:33:24.351 INFO:teuthology.orchestra.run.smithi190.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-04T17:33:24.515 INFO:teuthology.orchestra.run.smithi190.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-04T17:33:24.515 INFO:teuthology.orchestra.run.smithi190.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-04T17:33:24.515 INFO:teuthology.orchestra.run.smithi190.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-04T17:33:24.570 INFO:teuthology.orchestra.run.smithi190.stdout:done 2024-09-04T17:33:25.090 DEBUG:teuthology.orchestra.run.smithi190:> dpkg -s linux-image-generic 2024-09-04T17:33:25.114 INFO:teuthology.orchestra.run.smithi190.stdout:Package: linux-image-generic 2024-09-04T17:33:25.114 INFO:teuthology.orchestra.run.smithi190.stdout:Status: install ok installed 2024-09-04T17:33:25.114 INFO:teuthology.orchestra.run.smithi190.stdout:Priority: optional 2024-09-04T17:33:25.114 INFO:teuthology.orchestra.run.smithi190.stdout:Section: kernel 2024-09-04T17:33:25.114 INFO:teuthology.orchestra.run.smithi190.stdout:Installed-Size: 21 2024-09-04T17:33:25.114 INFO:teuthology.orchestra.run.smithi190.stdout:Maintainer: Ubuntu Kernel Team 2024-09-04T17:33:25.115 INFO:teuthology.orchestra.run.smithi190.stdout:Architecture: amd64 2024-09-04T17:33:25.115 INFO:teuthology.orchestra.run.smithi190.stdout:Source: linux-meta 2024-09-04T17:33:25.115 INFO:teuthology.orchestra.run.smithi190.stdout:Version: 5.15.0.119.119 2024-09-04T17:33:25.115 INFO:teuthology.orchestra.run.smithi190.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-119), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-04T17:33:25.115 INFO:teuthology.orchestra.run.smithi190.stdout:Depends: linux-image-5.15.0-119-generic, linux-modules-extra-5.15.0-119-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-04T17:33:25.115 INFO:teuthology.orchestra.run.smithi190.stdout:Recommends: thermald 2024-09-04T17:33:25.115 INFO:teuthology.orchestra.run.smithi190.stdout:Description: Generic Linux kernel image 2024-09-04T17:33:25.115 INFO:teuthology.orchestra.run.smithi190.stdout: This package will always depend on the latest generic kernel image 2024-09-04T17:33:25.115 INFO:teuthology.orchestra.run.smithi190.stdout: available. 2024-09-04T17:33:25.115 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-119-generic 2024-09-04T17:33:25.115 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-04T17:33:25.116 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-04T17:33:25.116 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-04T17:33:25.116 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi190.front.sepia.ceph.com, path=None, version=distro) 2024-09-04T17:33:25.116 DEBUG:teuthology.orchestra.run.smithi190:> sudo apt-get clean 2024-09-04T17:33:25.281 DEBUG:teuthology.orchestra.run.smithi190:> sudo apt-get update 2024-09-04T17:33:25.537 INFO:teuthology.orchestra.run.smithi190.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-04T17:33:25.537 INFO:teuthology.orchestra.run.smithi190.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-04T17:33:25.609 INFO:teuthology.orchestra.run.smithi190.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-04T17:33:25.690 INFO:teuthology.orchestra.run.smithi190.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-04T17:33:27.014 INFO:teuthology.orchestra.run.smithi190.stdout:Reading package lists... 2024-09-04T17:33:27.033 DEBUG:teuthology.orchestra.run.smithi190:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-04T17:33:27.095 INFO:teuthology.orchestra.run.smithi190.stdout:Reading package lists... 2024-09-04T17:33:27.283 INFO:teuthology.orchestra.run.smithi190.stdout:Building dependency tree... 2024-09-04T17:33:27.284 INFO:teuthology.orchestra.run.smithi190.stdout:Reading state information... 2024-09-04T17:33:27.447 INFO:teuthology.orchestra.run.smithi190.stdout:linux-image-generic is already the newest version (5.15.0.119.119). 2024-09-04T17:33:27.447 INFO:teuthology.orchestra.run.smithi190.stdout:The following packages were automatically installed and are no longer required: 2024-09-04T17:33:27.447 INFO:teuthology.orchestra.run.smithi190.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-04T17:33:27.448 INFO:teuthology.orchestra.run.smithi190.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-04T17:33:27.448 INFO:teuthology.orchestra.run.smithi190.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-04T17:33:27.448 INFO:teuthology.orchestra.run.smithi190.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-04T17:33:27.448 INFO:teuthology.orchestra.run.smithi190.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-04T17:33:27.448 INFO:teuthology.orchestra.run.smithi190.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-04T17:33:27.488 INFO:teuthology.orchestra.run.smithi190.stdout:0 upgraded, 0 newly installed, 0 to remove and 344 not upgraded. 2024-09-04T17:33:27.490 DEBUG:teuthology.orchestra.run.smithi190:> dpkg -s linux-image-generic 2024-09-04T17:33:27.502 INFO:teuthology.orchestra.run.smithi190.stdout:Package: linux-image-generic 2024-09-04T17:33:27.502 INFO:teuthology.orchestra.run.smithi190.stdout:Status: install ok installed 2024-09-04T17:33:27.502 INFO:teuthology.orchestra.run.smithi190.stdout:Priority: optional 2024-09-04T17:33:27.502 INFO:teuthology.orchestra.run.smithi190.stdout:Section: kernel 2024-09-04T17:33:27.503 INFO:teuthology.orchestra.run.smithi190.stdout:Installed-Size: 21 2024-09-04T17:33:27.503 INFO:teuthology.orchestra.run.smithi190.stdout:Maintainer: Ubuntu Kernel Team 2024-09-04T17:33:27.503 INFO:teuthology.orchestra.run.smithi190.stdout:Architecture: amd64 2024-09-04T17:33:27.503 INFO:teuthology.orchestra.run.smithi190.stdout:Source: linux-meta 2024-09-04T17:33:27.503 INFO:teuthology.orchestra.run.smithi190.stdout:Version: 5.15.0.119.119 2024-09-04T17:33:27.503 INFO:teuthology.orchestra.run.smithi190.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-119), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-04T17:33:27.503 INFO:teuthology.orchestra.run.smithi190.stdout:Depends: linux-image-5.15.0-119-generic, linux-modules-extra-5.15.0-119-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-04T17:33:27.503 INFO:teuthology.orchestra.run.smithi190.stdout:Recommends: thermald 2024-09-04T17:33:27.503 INFO:teuthology.orchestra.run.smithi190.stdout:Description: Generic Linux kernel image 2024-09-04T17:33:27.503 INFO:teuthology.orchestra.run.smithi190.stdout: This package will always depend on the latest generic kernel image 2024-09-04T17:33:27.503 INFO:teuthology.orchestra.run.smithi190.stdout: available. 2024-09-04T17:33:27.504 DEBUG:teuthology.orchestra.run.smithi190:> mktemp 2024-09-04T17:33:27.553 INFO:teuthology.orchestra.run.smithi190.stdout:/tmp/tmp.bB2Ij4ZJqG 2024-09-04T17:33:27.553 DEBUG:teuthology.orchestra.run.smithi190:> sudo cp /boot/grub/grub.cfg /tmp/tmp.bB2Ij4ZJqG 2024-09-04T17:33:27.605 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 0666 /tmp/tmp.bB2Ij4ZJqG 2024-09-04T17:33:27.727 DEBUG:teuthology.orchestra.remote:smithi190:/tmp/tmp.bB2Ij4ZJqG is 10KB 2024-09-04T17:33:27.739 DEBUG:teuthology.orchestra.run.smithi190:> rm -fr /tmp/tmp.bB2Ij4ZJqG 2024-09-04T17:33:27.745 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-04T17:33:27.798 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-04T17:33:27.798 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-04T17:33:27.798 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-04T17:33:27.864 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-119-generic 2024-09-04T17:33:27.865 DEBUG:teuthology.orchestra.run.smithi190:> sudo update-grub 2024-09-04T17:33:28.201 INFO:teuthology.orchestra.run.smithi092.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-04T17:33:28.201 INFO:teuthology.orchestra.run.smithi092.stdout:Sourcing file `/etc/default/grub' 2024-09-04T17:33:28.231 INFO:teuthology.orchestra.run.smithi092.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-04T17:33:28.256 INFO:teuthology.orchestra.run.smithi092.stdout:Generating grub configuration file ... 2024-09-04T17:33:28.597 INFO:teuthology.orchestra.run.smithi092.stdout:Found linux image: /boot/vmlinuz-5.15.0-119-generic 2024-09-04T17:33:28.612 INFO:teuthology.orchestra.run.smithi092.stdout:Found initrd image: /boot/initrd.img-5.15.0-119-generic 2024-09-04T17:33:28.973 INFO:teuthology.orchestra.run.smithi092.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-04T17:33:28.979 INFO:teuthology.orchestra.run.smithi092.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-04T17:33:29.049 INFO:teuthology.orchestra.run.smithi092.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-04T17:33:29.054 INFO:teuthology.orchestra.run.smithi092.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-04T17:33:29.204 INFO:teuthology.orchestra.run.smithi190.stderr:Sourcing file `/etc/default/grub' 2024-09-04T17:33:29.205 INFO:teuthology.orchestra.run.smithi190.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-04T17:33:29.207 INFO:teuthology.orchestra.run.smithi190.stderr:Generating grub configuration file ... 2024-09-04T17:33:29.212 INFO:teuthology.orchestra.run.smithi092.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-04T17:33:29.212 INFO:teuthology.orchestra.run.smithi092.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-04T17:33:29.212 INFO:teuthology.orchestra.run.smithi092.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-04T17:33:29.247 INFO:teuthology.orchestra.run.smithi092.stdout:done 2024-09-04T17:33:29.436 INFO:teuthology.orchestra.run.smithi190.stderr:Found linux image: /boot/vmlinuz-5.15.0-119-generic 2024-09-04T17:33:29.442 INFO:teuthology.orchestra.run.smithi190.stderr:Found initrd image: /boot/initrd.img-5.15.0-119-generic 2024-09-04T17:33:29.715 INFO:teuthology.orchestra.run.smithi190.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-04T17:33:29.720 INFO:teuthology.orchestra.run.smithi190.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-04T17:33:29.758 INFO:teuthology.orchestra.run.smithi190.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-04T17:33:29.763 INFO:teuthology.orchestra.run.smithi190.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-04T17:33:29.800 DEBUG:teuthology.orchestra.run.smithi092:> dpkg -s linux-image-generic 2024-09-04T17:33:29.815 INFO:teuthology.orchestra.run.smithi092.stdout:Package: linux-image-generic 2024-09-04T17:33:29.815 INFO:teuthology.orchestra.run.smithi092.stdout:Status: install ok installed 2024-09-04T17:33:29.815 INFO:teuthology.orchestra.run.smithi092.stdout:Priority: optional 2024-09-04T17:33:29.816 INFO:teuthology.orchestra.run.smithi092.stdout:Section: kernel 2024-09-04T17:33:29.816 INFO:teuthology.orchestra.run.smithi092.stdout:Installed-Size: 21 2024-09-04T17:33:29.816 INFO:teuthology.orchestra.run.smithi092.stdout:Maintainer: Ubuntu Kernel Team 2024-09-04T17:33:29.816 INFO:teuthology.orchestra.run.smithi092.stdout:Architecture: amd64 2024-09-04T17:33:29.816 INFO:teuthology.orchestra.run.smithi092.stdout:Source: linux-meta 2024-09-04T17:33:29.816 INFO:teuthology.orchestra.run.smithi092.stdout:Version: 5.15.0.119.119 2024-09-04T17:33:29.816 INFO:teuthology.orchestra.run.smithi092.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-119), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-04T17:33:29.816 INFO:teuthology.orchestra.run.smithi092.stdout:Depends: linux-image-5.15.0-119-generic, linux-modules-extra-5.15.0-119-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-04T17:33:29.816 INFO:teuthology.orchestra.run.smithi092.stdout:Recommends: thermald 2024-09-04T17:33:29.816 INFO:teuthology.orchestra.run.smithi092.stdout:Description: Generic Linux kernel image 2024-09-04T17:33:29.816 INFO:teuthology.orchestra.run.smithi092.stdout: This package will always depend on the latest generic kernel image 2024-09-04T17:33:29.816 INFO:teuthology.orchestra.run.smithi092.stdout: available. 2024-09-04T17:33:29.817 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-119-generic 2024-09-04T17:33:29.817 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-04T17:33:29.817 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-04T17:33:29.817 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-04T17:33:29.817 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi092.front.sepia.ceph.com, path=None, version=distro) 2024-09-04T17:33:29.817 DEBUG:teuthology.orchestra.run.smithi092:> sudo apt-get clean 2024-09-04T17:33:29.873 INFO:teuthology.orchestra.run.smithi190.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-04T17:33:29.873 INFO:teuthology.orchestra.run.smithi190.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-04T17:33:29.873 INFO:teuthology.orchestra.run.smithi190.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-04T17:33:29.883 INFO:teuthology.orchestra.run.smithi190.stderr:done 2024-09-04T17:33:29.885 DEBUG:teuthology.orchestra.run.smithi190:> sudo shutdown -r now 2024-09-04T17:33:29.993 DEBUG:teuthology.orchestra.run.smithi092:> sudo apt-get update 2024-09-04T17:33:30.122 INFO:teuthology.orchestra.run.smithi092.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-04T17:33:30.133 INFO:teuthology.orchestra.run.smithi092.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-04T17:33:30.133 INFO:teuthology.orchestra.run.smithi092.stdout:Hit:3 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-04T17:33:30.152 INFO:teuthology.orchestra.run.smithi092.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-04T17:33:31.557 INFO:teuthology.orchestra.run.smithi092.stdout:Reading package lists... 2024-09-04T17:33:31.575 DEBUG:teuthology.orchestra.run.smithi092:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-04T17:33:31.629 INFO:teuthology.orchestra.run.smithi092.stdout:Reading package lists... 2024-09-04T17:33:31.814 INFO:teuthology.orchestra.run.smithi092.stdout:Building dependency tree... 2024-09-04T17:33:31.814 INFO:teuthology.orchestra.run.smithi092.stdout:Reading state information... 2024-09-04T17:33:31.983 INFO:teuthology.orchestra.run.smithi092.stdout:linux-image-generic is already the newest version (5.15.0.119.119). 2024-09-04T17:33:31.983 INFO:teuthology.orchestra.run.smithi092.stdout:The following packages were automatically installed and are no longer required: 2024-09-04T17:33:31.983 INFO:teuthology.orchestra.run.smithi092.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-04T17:33:31.983 INFO:teuthology.orchestra.run.smithi092.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-04T17:33:31.983 INFO:teuthology.orchestra.run.smithi092.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-04T17:33:31.984 INFO:teuthology.orchestra.run.smithi092.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-04T17:33:31.984 INFO:teuthology.orchestra.run.smithi092.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-04T17:33:31.984 INFO:teuthology.orchestra.run.smithi092.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-04T17:33:32.023 INFO:teuthology.orchestra.run.smithi092.stdout:0 upgraded, 0 newly installed, 0 to remove and 344 not upgraded. 2024-09-04T17:33:32.024 DEBUG:teuthology.orchestra.run.smithi092:> dpkg -s linux-image-generic 2024-09-04T17:33:32.084 INFO:teuthology.orchestra.run.smithi092.stdout:Package: linux-image-generic 2024-09-04T17:33:32.084 INFO:teuthology.orchestra.run.smithi092.stdout:Status: install ok installed 2024-09-04T17:33:32.084 INFO:teuthology.orchestra.run.smithi092.stdout:Priority: optional 2024-09-04T17:33:32.084 INFO:teuthology.orchestra.run.smithi092.stdout:Section: kernel 2024-09-04T17:33:32.084 INFO:teuthology.orchestra.run.smithi092.stdout:Installed-Size: 21 2024-09-04T17:33:32.084 INFO:teuthology.orchestra.run.smithi092.stdout:Maintainer: Ubuntu Kernel Team 2024-09-04T17:33:32.084 INFO:teuthology.orchestra.run.smithi092.stdout:Architecture: amd64 2024-09-04T17:33:32.085 INFO:teuthology.orchestra.run.smithi092.stdout:Source: linux-meta 2024-09-04T17:33:32.085 INFO:teuthology.orchestra.run.smithi092.stdout:Version: 5.15.0.119.119 2024-09-04T17:33:32.085 INFO:teuthology.orchestra.run.smithi092.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-119), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-04T17:33:32.085 INFO:teuthology.orchestra.run.smithi092.stdout:Depends: linux-image-5.15.0-119-generic, linux-modules-extra-5.15.0-119-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-04T17:33:32.085 INFO:teuthology.orchestra.run.smithi092.stdout:Recommends: thermald 2024-09-04T17:33:32.085 INFO:teuthology.orchestra.run.smithi092.stdout:Description: Generic Linux kernel image 2024-09-04T17:33:32.085 INFO:teuthology.orchestra.run.smithi092.stdout: This package will always depend on the latest generic kernel image 2024-09-04T17:33:32.085 INFO:teuthology.orchestra.run.smithi092.stdout: available. 2024-09-04T17:33:32.085 DEBUG:teuthology.orchestra.run.smithi092:> mktemp 2024-09-04T17:33:32.129 INFO:teuthology.orchestra.run.smithi092.stdout:/tmp/tmp.2DdpyMdCJb 2024-09-04T17:33:32.129 DEBUG:teuthology.orchestra.run.smithi092:> sudo cp /boot/grub/grub.cfg /tmp/tmp.2DdpyMdCJb 2024-09-04T17:33:32.193 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0666 /tmp/tmp.2DdpyMdCJb 2024-09-04T17:33:32.321 DEBUG:teuthology.orchestra.remote:smithi092:/tmp/tmp.2DdpyMdCJb is 10KB 2024-09-04T17:33:32.335 DEBUG:teuthology.orchestra.run.smithi092:> rm -fr /tmp/tmp.2DdpyMdCJb 2024-09-04T17:33:32.340 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-04T17:33:32.393 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-04T17:33:32.393 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-04T17:33:32.393 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-04T17:33:32.478 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-119-generic 2024-09-04T17:33:32.478 DEBUG:teuthology.orchestra.run.smithi092:> sudo update-grub 2024-09-04T17:33:33.933 INFO:teuthology.orchestra.run.smithi092.stderr:Sourcing file `/etc/default/grub' 2024-09-04T17:33:33.935 INFO:teuthology.orchestra.run.smithi092.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-04T17:33:33.938 INFO:teuthology.orchestra.run.smithi092.stderr:Generating grub configuration file ... 2024-09-04T17:33:34.142 INFO:teuthology.orchestra.run.smithi092.stderr:Found linux image: /boot/vmlinuz-5.15.0-119-generic 2024-09-04T17:33:34.148 INFO:teuthology.orchestra.run.smithi092.stderr:Found initrd image: /boot/initrd.img-5.15.0-119-generic 2024-09-04T17:33:34.375 INFO:teuthology.orchestra.run.smithi092.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-04T17:33:34.381 INFO:teuthology.orchestra.run.smithi092.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-04T17:33:34.422 INFO:teuthology.orchestra.run.smithi092.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-04T17:33:34.427 INFO:teuthology.orchestra.run.smithi092.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-04T17:33:34.546 INFO:teuthology.orchestra.run.smithi092.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-04T17:33:34.546 INFO:teuthology.orchestra.run.smithi092.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-04T17:33:34.546 INFO:teuthology.orchestra.run.smithi092.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-04T17:33:34.566 INFO:teuthology.orchestra.run.smithi092.stderr:done 2024-09-04T17:33:34.575 DEBUG:teuthology.orchestra.run.smithi092:> sudo shutdown -r now 2024-09-04T17:33:59.909 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-04T17:33:59.909 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-09-04T17:33:59.910 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-04T17:34:04.623 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-04T17:34:04.624 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-09-04T17:34:04.624 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-04T17:34:23.037 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.92 2024-09-04T17:34:32.045 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-09-04T17:34:32.045 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-04T17:34:34.557 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.190 2024-09-04T17:34:35.101 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.92 2024-09-04T17:34:43.564 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-09-04T17:34:43.565 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-04T17:34:47.105 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-09-04T17:34:47.106 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-04T17:35:01.949 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.190 2024-09-04T17:35:05.537 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.92 2024-09-04T17:35:13.955 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-09-04T17:35:13.956 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-04T17:35:14.611 DEBUG:teuthology.orchestra.run.smithi190:> true 2024-09-04T17:35:15.292 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-09-04T17:35:15.293 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-119-generic"... 2024-09-04T17:35:15.293 DEBUG:teuthology.orchestra.run.smithi190:> uname -r 2024-09-04T17:35:15.338 INFO:teuthology.orchestra.run.smithi190.stdout:5.15.0-119-generic 2024-09-04T17:35:15.339 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-119-generic vs 5.15.0-119-generic 2024-09-04T17:35:15.339 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-04T17:35:15.339 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-04T17:35:16.339 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-04T17:35:16.339 DEBUG:teuthology.orchestra.run.smithi190:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-04T17:35:16.408 INFO:teuthology.orchestra.run.smithi190.stdout:ttyS1 2024-09-04T17:35:16.436 DEBUG:teuthology.parallel:result is None 2024-09-04T17:35:20.540 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-09-04T17:35:20.540 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-04T17:35:20.867 DEBUG:teuthology.orchestra.run.smithi092:> true 2024-09-04T17:35:21.448 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-09-04T17:35:21.448 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-119-generic"... 2024-09-04T17:35:21.448 DEBUG:teuthology.orchestra.run.smithi092:> uname -r 2024-09-04T17:35:21.494 INFO:teuthology.orchestra.run.smithi092.stdout:5.15.0-119-generic 2024-09-04T17:35:21.494 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-119-generic vs 5.15.0-119-generic 2024-09-04T17:35:21.495 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-04T17:35:21.495 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-04T17:35:22.495 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-04T17:35:22.495 DEBUG:teuthology.orchestra.run.smithi092:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-04T17:35:22.572 INFO:teuthology.orchestra.run.smithi092.stdout:ttyS1 2024-09-04T17:35:22.591 DEBUG:teuthology.parallel:result is None 2024-09-04T17:35:22.592 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-04T17:35:22.598 INFO:teuthology.task.internal:Creating test directory... 2024-09-04T17:35:22.599 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-04T17:35:22.601 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-04T17:35:22.607 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-04T17:35:22.613 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-04T17:35:22.619 INFO:teuthology.task.internal:Creating archive directory... 2024-09-04T17:35:22.619 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-04T17:35:22.647 DEBUG:teuthology.orchestra.run.smithi190:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-04T17:35:22.914 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-04T17:35:22.921 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-04T17:35:22.921 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-04T17:35:22.924 DEBUG:teuthology.orchestra.run.smithi190:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-04T17:35:22.942 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-04T17:35:22.945 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-04T17:35:22.950 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-04T17:35:22.955 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-04T17:35:22.957 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-04T17:35:22.966 INFO:teuthology.task.internal:Configuring sudo... 2024-09-04T17:35:22.967 DEBUG:teuthology.orchestra.run.smithi092:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-04T17:35:22.999 DEBUG:teuthology.orchestra.run.smithi190:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-04T17:35:23.017 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-04T17:35:23.025 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-04T17:35:23.026 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-04T17:35:23.063 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-04T17:35:23.068 DEBUG:teuthology.orchestra.run.smithi092:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-04T17:35:23.115 DEBUG:teuthology.orchestra.run.smithi092:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-04T17:35:23.163 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-04T17:35:23.163 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-04T17:35:23.228 DEBUG:teuthology.orchestra.run.smithi190:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-04T17:35:23.234 DEBUG:teuthology.orchestra.run.smithi190:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-04T17:35:23.283 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-04T17:35:23.283 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-04T17:35:23.359 DEBUG:teuthology.orchestra.run.smithi092:> sudo service rsyslog restart 2024-09-04T17:35:23.361 DEBUG:teuthology.orchestra.run.smithi190:> sudo service rsyslog restart 2024-09-04T17:35:23.438 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-04T17:35:23.445 INFO:teuthology.task.internal:Starting timer... 2024-09-04T17:35:23.445 INFO:teuthology.run_tasks:Running task pcp... 2024-09-04T17:35:23.454 INFO:teuthology.run_tasks:Running task selinux... 2024-09-04T17:35:23.463 DEBUG:teuthology.task.selinux:Excluding smithi092: OS 'ubuntu' does not support SELinux 2024-09-04T17:35:23.463 DEBUG:teuthology.task.selinux:Excluding smithi190: OS 'ubuntu' does not support SELinux 2024-09-04T17:35:23.463 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-04T17:35:23.463 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-04T17:35:23.463 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-04T17:35:23.463 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-04T17:35:23.471 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-04T17:35:23.474 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-04T17:35:23.577 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-04T17:35:23.589 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-04T17:35:23.590 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi092.front.sepia.ceph.com,smithi190.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-04T17:42:48.079 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi092.front.sepia.ceph.com'), Remote(name='ubuntu@smithi190.front.sepia.ceph.com')] 2024-09-04T17:42:48.080 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-09-04T17:42:48.081 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-04T17:42:48.161 DEBUG:teuthology.orchestra.run.smithi092:> true 2024-09-04T17:42:48.237 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-09-04T17:42:48.237 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-09-04T17:42:48.237 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-04T17:42:48.318 DEBUG:teuthology.orchestra.run.smithi190:> true 2024-09-04T17:42:48.393 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-09-04T17:42:48.393 INFO:teuthology.run_tasks:Running task clock... 2024-09-04T17:42:48.403 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-04T17:42:48.403 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-04T17:42:48.404 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-04T17:42:48.406 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-04T17:42:48.406 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-04T17:42:48.433 INFO:teuthology.orchestra.run.smithi092.stdout: 4 Sep 17:42:48 ntpd[18422]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-04T17:42:48.433 INFO:teuthology.orchestra.run.smithi092.stdout: 4 Sep 17:42:48 ntpd[18422]: Command line: ntpd -gq 2024-09-04T17:42:48.433 INFO:teuthology.orchestra.run.smithi092.stdout: 4 Sep 17:42:48 ntpd[18422]: ---------------------------------------------------- 2024-09-04T17:42:48.433 INFO:teuthology.orchestra.run.smithi092.stdout: 4 Sep 17:42:48 ntpd[18422]: ntp-4 is maintained by Network Time Foundation, 2024-09-04T17:42:48.433 INFO:teuthology.orchestra.run.smithi092.stdout: 4 Sep 17:42:48 ntpd[18422]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-04T17:42:48.433 INFO:teuthology.orchestra.run.smithi092.stdout: 4 Sep 17:42:48 ntpd[18422]: corporation. Support and training for ntp-4 are 2024-09-04T17:42:48.433 INFO:teuthology.orchestra.run.smithi092.stdout: 4 Sep 17:42:48 ntpd[18422]: available at https://www.nwtime.org/support 2024-09-04T17:42:48.433 INFO:teuthology.orchestra.run.smithi092.stdout: 4 Sep 17:42:48 ntpd[18422]: ---------------------------------------------------- 2024-09-04T17:42:48.434 INFO:teuthology.orchestra.run.smithi092.stdout: 4 Sep 17:42:48 ntpd[18422]: proto: precision = 0.044 usec (-24) 2024-09-04T17:42:48.434 INFO:teuthology.orchestra.run.smithi092.stdout: 4 Sep 17:42:48 ntpd[18422]: basedate set to 2022-02-04 2024-09-04T17:42:48.434 INFO:teuthology.orchestra.run.smithi092.stdout: 4 Sep 17:42:48 ntpd[18422]: gps base set to 2022-02-06 (week 2196) 2024-09-04T17:42:48.434 INFO:teuthology.orchestra.run.smithi092.stdout: 4 Sep 17:42:48 ntpd[18422]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-04T17:42:48.435 INFO:teuthology.orchestra.run.smithi092.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-04T17:42:48.435 INFO:teuthology.orchestra.run.smithi092.stdout: 4 Sep 17:42:48 ntpd[18422]: restrict ::: KOD does nothing without LIMITED. 2024-09-04T17:42:48.435 INFO:teuthology.orchestra.run.smithi092.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-04T17:42:48.436 INFO:teuthology.orchestra.run.smithi092.stdout: 4 Sep 17:42:48 ntpd[18422]: Listen and drop on 0 v6wildcard [::]:123 2024-09-04T17:42:48.436 INFO:teuthology.orchestra.run.smithi092.stdout: 4 Sep 17:42:48 ntpd[18422]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-04T17:42:48.437 INFO:teuthology.orchestra.run.smithi092.stdout: 4 Sep 17:42:48 ntpd[18422]: Listen normally on 2 lo 127.0.0.1:123 2024-09-04T17:42:48.437 INFO:teuthology.orchestra.run.smithi092.stdout: 4 Sep 17:42:48 ntpd[18422]: Listen normally on 3 enp3s0f1 172.21.15.92:123 2024-09-04T17:42:48.437 INFO:teuthology.orchestra.run.smithi092.stdout: 4 Sep 17:42:48 ntpd[18422]: Listen normally on 4 lo [::1]:123 2024-09-04T17:42:48.437 INFO:teuthology.orchestra.run.smithi092.stdout: 4 Sep 17:42:48 ntpd[18422]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:cd45%5]:123 2024-09-04T17:42:48.437 INFO:teuthology.orchestra.run.smithi092.stdout: 4 Sep 17:42:48 ntpd[18422]: Listening on routing socket on fd #22 for interface updates 2024-09-04T17:42:48.463 INFO:teuthology.orchestra.run.smithi190.stdout: 4 Sep 17:42:48 ntpd[18425]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-04T17:42:48.464 INFO:teuthology.orchestra.run.smithi190.stdout: 4 Sep 17:42:48 ntpd[18425]: Command line: ntpd -gq 2024-09-04T17:42:48.464 INFO:teuthology.orchestra.run.smithi190.stdout: 4 Sep 17:42:48 ntpd[18425]: ---------------------------------------------------- 2024-09-04T17:42:48.464 INFO:teuthology.orchestra.run.smithi190.stdout: 4 Sep 17:42:48 ntpd[18425]: ntp-4 is maintained by Network Time Foundation, 2024-09-04T17:42:48.464 INFO:teuthology.orchestra.run.smithi190.stdout: 4 Sep 17:42:48 ntpd[18425]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-04T17:42:48.464 INFO:teuthology.orchestra.run.smithi190.stdout: 4 Sep 17:42:48 ntpd[18425]: corporation. Support and training for ntp-4 are 2024-09-04T17:42:48.464 INFO:teuthology.orchestra.run.smithi190.stdout: 4 Sep 17:42:48 ntpd[18425]: available at https://www.nwtime.org/support 2024-09-04T17:42:48.464 INFO:teuthology.orchestra.run.smithi190.stdout: 4 Sep 17:42:48 ntpd[18425]: ---------------------------------------------------- 2024-09-04T17:42:48.465 INFO:teuthology.orchestra.run.smithi190.stdout: 4 Sep 17:42:48 ntpd[18425]: proto: precision = 0.060 usec (-24) 2024-09-04T17:42:48.465 INFO:teuthology.orchestra.run.smithi190.stdout: 4 Sep 17:42:48 ntpd[18425]: basedate set to 2022-02-04 2024-09-04T17:42:48.465 INFO:teuthology.orchestra.run.smithi190.stdout: 4 Sep 17:42:48 ntpd[18425]: gps base set to 2022-02-06 (week 2196) 2024-09-04T17:42:48.465 INFO:teuthology.orchestra.run.smithi190.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-04T17:42:48.466 INFO:teuthology.orchestra.run.smithi190.stdout: 4 Sep 17:42:48 ntpd[18425]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-04T17:42:48.466 INFO:teuthology.orchestra.run.smithi190.stdout: 4 Sep 17:42:48 ntpd[18425]: restrict ::: KOD does nothing without LIMITED. 2024-09-04T17:42:48.466 INFO:teuthology.orchestra.run.smithi190.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-04T17:42:48.467 INFO:teuthology.orchestra.run.smithi190.stdout: 4 Sep 17:42:48 ntpd[18425]: Listen and drop on 0 v6wildcard [::]:123 2024-09-04T17:42:48.467 INFO:teuthology.orchestra.run.smithi190.stdout: 4 Sep 17:42:48 ntpd[18425]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-04T17:42:48.467 INFO:teuthology.orchestra.run.smithi190.stdout: 4 Sep 17:42:48 ntpd[18425]: Listen normally on 2 lo 127.0.0.1:123 2024-09-04T17:42:48.467 INFO:teuthology.orchestra.run.smithi190.stdout: 4 Sep 17:42:48 ntpd[18425]: Listen normally on 3 enp3s0f1 172.21.15.190:123 2024-09-04T17:42:48.467 INFO:teuthology.orchestra.run.smithi190.stdout: 4 Sep 17:42:48 ntpd[18425]: Listen normally on 4 lo [::1]:123 2024-09-04T17:42:48.467 INFO:teuthology.orchestra.run.smithi190.stdout: 4 Sep 17:42:48 ntpd[18425]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7763%5]:123 2024-09-04T17:42:48.467 INFO:teuthology.orchestra.run.smithi190.stdout: 4 Sep 17:42:48 ntpd[18425]: Listening on routing socket on fd #22 for interface updates 2024-09-04T17:42:49.434 INFO:teuthology.orchestra.run.smithi092.stderr: 4 Sep 17:42:49 ntpd[18422]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-04T17:42:49.434 INFO:teuthology.orchestra.run.smithi092.stderr: 4 Sep 17:42:49 ntpd[18422]: can't open /var/log/ntpstats/rawstats.20240904: Permission denied 2024-09-04T17:42:49.434 INFO:teuthology.orchestra.run.smithi092.stderr: 4 Sep 17:42:49 ntpd[18422]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-04T17:42:49.434 INFO:teuthology.orchestra.run.smithi092.stderr: 4 Sep 17:42:49 ntpd[18422]: can't open /var/log/ntpstats/rawstats.20240904: Permission denied 2024-09-04T17:42:49.434 INFO:teuthology.orchestra.run.smithi092.stderr: 4 Sep 17:42:49 ntpd[18422]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-04T17:42:49.434 INFO:teuthology.orchestra.run.smithi092.stderr: 4 Sep 17:42:49 ntpd[18422]: can't open /var/log/ntpstats/rawstats.20240904: Permission denied 2024-09-04T17:42:49.434 INFO:teuthology.orchestra.run.smithi092.stderr: 4 Sep 17:42:49 ntpd[18422]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-04T17:42:49.434 INFO:teuthology.orchestra.run.smithi092.stderr: 4 Sep 17:42:49 ntpd[18422]: can't open /var/log/ntpstats/peerstats.20240904: Permission denied 2024-09-04T17:42:49.464 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:49 ntpd[18425]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-04T17:42:49.464 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:49 ntpd[18425]: can't open /var/log/ntpstats/rawstats.20240904: Permission denied 2024-09-04T17:42:49.465 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:49 ntpd[18425]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-04T17:42:49.465 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:49 ntpd[18425]: can't open /var/log/ntpstats/rawstats.20240904: Permission denied 2024-09-04T17:42:49.465 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:49 ntpd[18425]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-04T17:42:49.465 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:49 ntpd[18425]: can't open /var/log/ntpstats/rawstats.20240904: Permission denied 2024-09-04T17:42:49.465 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:49 ntpd[18425]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-04T17:42:49.465 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:49 ntpd[18425]: can't open /var/log/ntpstats/rawstats.20240904: Permission denied 2024-09-04T17:42:49.465 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:49 ntpd[18425]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-04T17:42:49.465 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:49 ntpd[18425]: can't open /var/log/ntpstats/peerstats.20240904: Permission denied 2024-09-04T17:42:49.465 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:49 ntpd[18425]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-04T17:42:49.465 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:49 ntpd[18425]: can't open /var/log/ntpstats/rawstats.20240904: Permission denied 2024-09-04T17:42:49.465 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:49 ntpd[18425]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-04T17:42:49.465 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:49 ntpd[18425]: can't open /var/log/ntpstats/peerstats.20240904: Permission denied 2024-09-04T17:42:50.464 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:50 ntpd[18425]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-04T17:42:50.464 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:50 ntpd[18425]: can't open /var/log/ntpstats/rawstats.20240904: Permission denied 2024-09-04T17:42:50.464 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:50 ntpd[18425]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-04T17:42:50.464 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:50 ntpd[18425]: can't open /var/log/ntpstats/rawstats.20240904: Permission denied 2024-09-04T17:42:50.465 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:50 ntpd[18425]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-04T17:42:50.465 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:50 ntpd[18425]: can't open /var/log/ntpstats/peerstats.20240904: Permission denied 2024-09-04T17:42:51.434 INFO:teuthology.orchestra.run.smithi092.stderr: 4 Sep 17:42:51 ntpd[18422]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-04T17:42:51.434 INFO:teuthology.orchestra.run.smithi092.stderr: 4 Sep 17:42:51 ntpd[18422]: can't open /var/log/ntpstats/rawstats.20240904: Permission denied 2024-09-04T17:42:51.434 INFO:teuthology.orchestra.run.smithi092.stderr: 4 Sep 17:42:51 ntpd[18422]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-04T17:42:51.434 INFO:teuthology.orchestra.run.smithi092.stderr: 4 Sep 17:42:51 ntpd[18422]: can't open /var/log/ntpstats/rawstats.20240904: Permission denied 2024-09-04T17:42:51.434 INFO:teuthology.orchestra.run.smithi092.stderr: 4 Sep 17:42:51 ntpd[18422]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-04T17:42:51.434 INFO:teuthology.orchestra.run.smithi092.stderr: 4 Sep 17:42:51 ntpd[18422]: can't open /var/log/ntpstats/rawstats.20240904: Permission denied 2024-09-04T17:42:51.434 INFO:teuthology.orchestra.run.smithi092.stderr: 4 Sep 17:42:51 ntpd[18422]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-04T17:42:51.435 INFO:teuthology.orchestra.run.smithi092.stderr: 4 Sep 17:42:51 ntpd[18422]: can't open /var/log/ntpstats/peerstats.20240904: Permission denied 2024-09-04T17:42:51.435 INFO:teuthology.orchestra.run.smithi092.stderr: 4 Sep 17:42:51 ntpd[18422]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-04T17:42:51.435 INFO:teuthology.orchestra.run.smithi092.stderr: 4 Sep 17:42:51 ntpd[18422]: can't open /var/log/ntpstats/rawstats.20240904: Permission denied 2024-09-04T17:42:51.435 INFO:teuthology.orchestra.run.smithi092.stderr: 4 Sep 17:42:51 ntpd[18422]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-04T17:42:51.435 INFO:teuthology.orchestra.run.smithi092.stderr: 4 Sep 17:42:51 ntpd[18422]: can't open /var/log/ntpstats/peerstats.20240904: Permission denied 2024-09-04T17:42:51.465 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:51 ntpd[18425]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-04T17:42:51.465 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:51 ntpd[18425]: can't open /var/log/ntpstats/rawstats.20240904: Permission denied 2024-09-04T17:42:51.465 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:51 ntpd[18425]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-04T17:42:51.465 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:51 ntpd[18425]: can't open /var/log/ntpstats/rawstats.20240904: Permission denied 2024-09-04T17:42:51.465 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:51 ntpd[18425]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-04T17:42:51.465 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:51 ntpd[18425]: can't open /var/log/ntpstats/rawstats.20240904: Permission denied 2024-09-04T17:42:51.465 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:51 ntpd[18425]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-04T17:42:51.466 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:51 ntpd[18425]: can't open /var/log/ntpstats/peerstats.20240904: Permission denied 2024-09-04T17:42:51.466 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:51 ntpd[18425]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-04T17:42:51.466 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:51 ntpd[18425]: can't open /var/log/ntpstats/rawstats.20240904: Permission denied 2024-09-04T17:42:51.466 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:51 ntpd[18425]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-04T17:42:51.466 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:51 ntpd[18425]: can't open /var/log/ntpstats/peerstats.20240904: Permission denied 2024-09-04T17:42:52.433 INFO:teuthology.orchestra.run.smithi092.stderr: 4 Sep 17:42:52 ntpd[18422]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-04T17:42:52.434 INFO:teuthology.orchestra.run.smithi092.stderr: 4 Sep 17:42:52 ntpd[18422]: can't open /var/log/ntpstats/rawstats.20240904: Permission denied 2024-09-04T17:42:52.434 INFO:teuthology.orchestra.run.smithi092.stderr: 4 Sep 17:42:52 ntpd[18422]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-04T17:42:52.434 INFO:teuthology.orchestra.run.smithi092.stderr: 4 Sep 17:42:52 ntpd[18422]: can't open /var/log/ntpstats/rawstats.20240904: Permission denied 2024-09-04T17:42:52.434 INFO:teuthology.orchestra.run.smithi092.stderr: 4 Sep 17:42:52 ntpd[18422]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-04T17:42:52.434 INFO:teuthology.orchestra.run.smithi092.stderr: 4 Sep 17:42:52 ntpd[18422]: can't open /var/log/ntpstats/peerstats.20240904: Permission denied 2024-09-04T17:42:52.464 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:52 ntpd[18425]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-04T17:42:52.464 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:52 ntpd[18425]: can't open /var/log/ntpstats/rawstats.20240904: Permission denied 2024-09-04T17:42:52.464 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:52 ntpd[18425]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-04T17:42:52.465 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:52 ntpd[18425]: can't open /var/log/ntpstats/rawstats.20240904: Permission denied 2024-09-04T17:42:52.465 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:52 ntpd[18425]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-04T17:42:52.465 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:52 ntpd[18425]: can't open /var/log/ntpstats/peerstats.20240904: Permission denied 2024-09-04T17:42:53.434 INFO:teuthology.orchestra.run.smithi092.stderr: 4 Sep 17:42:53 ntpd[18422]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-04T17:42:53.434 INFO:teuthology.orchestra.run.smithi092.stderr: 4 Sep 17:42:53 ntpd[18422]: can't open /var/log/ntpstats/rawstats.20240904: Permission denied 2024-09-04T17:42:53.434 INFO:teuthology.orchestra.run.smithi092.stderr: 4 Sep 17:42:53 ntpd[18422]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-04T17:42:53.434 INFO:teuthology.orchestra.run.smithi092.stderr: 4 Sep 17:42:53 ntpd[18422]: can't open /var/log/ntpstats/rawstats.20240904: Permission denied 2024-09-04T17:42:53.434 INFO:teuthology.orchestra.run.smithi092.stderr: 4 Sep 17:42:53 ntpd[18422]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-04T17:42:53.434 INFO:teuthology.orchestra.run.smithi092.stderr: 4 Sep 17:42:53 ntpd[18422]: can't open /var/log/ntpstats/rawstats.20240904: Permission denied 2024-09-04T17:42:53.434 INFO:teuthology.orchestra.run.smithi092.stderr: 4 Sep 17:42:53 ntpd[18422]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-04T17:42:53.435 INFO:teuthology.orchestra.run.smithi092.stderr: 4 Sep 17:42:53 ntpd[18422]: can't open /var/log/ntpstats/peerstats.20240904: Permission denied 2024-09-04T17:42:53.435 INFO:teuthology.orchestra.run.smithi092.stderr: 4 Sep 17:42:53 ntpd[18422]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-04T17:42:53.435 INFO:teuthology.orchestra.run.smithi092.stderr: 4 Sep 17:42:53 ntpd[18422]: can't open /var/log/ntpstats/rawstats.20240904: Permission denied 2024-09-04T17:42:53.435 INFO:teuthology.orchestra.run.smithi092.stderr: 4 Sep 17:42:53 ntpd[18422]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-04T17:42:53.435 INFO:teuthology.orchestra.run.smithi092.stderr: 4 Sep 17:42:53 ntpd[18422]: can't open /var/log/ntpstats/peerstats.20240904: Permission denied 2024-09-04T17:42:53.465 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:53 ntpd[18425]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-04T17:42:53.465 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:53 ntpd[18425]: can't open /var/log/ntpstats/rawstats.20240904: Permission denied 2024-09-04T17:42:53.465 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:53 ntpd[18425]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-04T17:42:53.465 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:53 ntpd[18425]: can't open /var/log/ntpstats/rawstats.20240904: Permission denied 2024-09-04T17:42:53.465 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:53 ntpd[18425]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-04T17:42:53.465 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:53 ntpd[18425]: can't open /var/log/ntpstats/rawstats.20240904: Permission denied 2024-09-04T17:42:53.465 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:53 ntpd[18425]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-04T17:42:53.465 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:53 ntpd[18425]: can't open /var/log/ntpstats/peerstats.20240904: Permission denied 2024-09-04T17:42:53.465 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:53 ntpd[18425]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-04T17:42:53.465 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:53 ntpd[18425]: can't open /var/log/ntpstats/rawstats.20240904: Permission denied 2024-09-04T17:42:53.465 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:53 ntpd[18425]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-04T17:42:53.466 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:53 ntpd[18425]: can't open /var/log/ntpstats/peerstats.20240904: Permission denied 2024-09-04T17:42:54.433 INFO:teuthology.orchestra.run.smithi092.stderr: 4 Sep 17:42:54 ntpd[18422]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-04T17:42:54.433 INFO:teuthology.orchestra.run.smithi092.stderr: 4 Sep 17:42:54 ntpd[18422]: can't open /var/log/ntpstats/rawstats.20240904: Permission denied 2024-09-04T17:42:54.434 INFO:teuthology.orchestra.run.smithi092.stderr: 4 Sep 17:42:54 ntpd[18422]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-04T17:42:54.434 INFO:teuthology.orchestra.run.smithi092.stderr: 4 Sep 17:42:54 ntpd[18422]: can't open /var/log/ntpstats/rawstats.20240904: Permission denied 2024-09-04T17:42:54.434 INFO:teuthology.orchestra.run.smithi092.stderr: 4 Sep 17:42:54 ntpd[18422]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-04T17:42:54.434 INFO:teuthology.orchestra.run.smithi092.stderr: 4 Sep 17:42:54 ntpd[18422]: can't open /var/log/ntpstats/peerstats.20240904: Permission denied 2024-09-04T17:42:54.464 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:54 ntpd[18425]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-04T17:42:54.464 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:54 ntpd[18425]: can't open /var/log/ntpstats/rawstats.20240904: Permission denied 2024-09-04T17:42:54.464 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:54 ntpd[18425]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-04T17:42:54.464 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:54 ntpd[18425]: can't open /var/log/ntpstats/rawstats.20240904: Permission denied 2024-09-04T17:42:54.464 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:54 ntpd[18425]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-04T17:42:54.464 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:54 ntpd[18425]: can't open /var/log/ntpstats/peerstats.20240904: Permission denied 2024-09-04T17:42:55.434 INFO:teuthology.orchestra.run.smithi092.stderr: 4 Sep 17:42:55 ntpd[18422]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-04T17:42:55.434 INFO:teuthology.orchestra.run.smithi092.stderr: 4 Sep 17:42:55 ntpd[18422]: can't open /var/log/ntpstats/rawstats.20240904: Permission denied 2024-09-04T17:42:55.434 INFO:teuthology.orchestra.run.smithi092.stderr: 4 Sep 17:42:55 ntpd[18422]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-04T17:42:55.434 INFO:teuthology.orchestra.run.smithi092.stderr: 4 Sep 17:42:55 ntpd[18422]: can't open /var/log/ntpstats/rawstats.20240904: Permission denied 2024-09-04T17:42:55.434 INFO:teuthology.orchestra.run.smithi092.stderr: 4 Sep 17:42:55 ntpd[18422]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-04T17:42:55.434 INFO:teuthology.orchestra.run.smithi092.stderr: 4 Sep 17:42:55 ntpd[18422]: can't open /var/log/ntpstats/rawstats.20240904: Permission denied 2024-09-04T17:42:55.434 INFO:teuthology.orchestra.run.smithi092.stderr: 4 Sep 17:42:55 ntpd[18422]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-04T17:42:55.435 INFO:teuthology.orchestra.run.smithi092.stderr: 4 Sep 17:42:55 ntpd[18422]: can't open /var/log/ntpstats/peerstats.20240904: Permission denied 2024-09-04T17:42:55.435 INFO:teuthology.orchestra.run.smithi092.stderr: 4 Sep 17:42:55 ntpd[18422]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-04T17:42:55.435 INFO:teuthology.orchestra.run.smithi092.stderr: 4 Sep 17:42:55 ntpd[18422]: can't open /var/log/ntpstats/rawstats.20240904: Permission denied 2024-09-04T17:42:55.435 INFO:teuthology.orchestra.run.smithi092.stderr: 4 Sep 17:42:55 ntpd[18422]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-04T17:42:55.435 INFO:teuthology.orchestra.run.smithi092.stderr: 4 Sep 17:42:55 ntpd[18422]: can't open /var/log/ntpstats/peerstats.20240904: Permission denied 2024-09-04T17:42:55.435 INFO:teuthology.orchestra.run.smithi092.stdout: 4 Sep 17:42:55 ntpd[18422]: ntpd: time slew -0.001869 s 2024-09-04T17:42:55.435 INFO:teuthology.orchestra.run.smithi092.stdout:ntpd: time slew -0.001869s 2024-09-04T17:42:55.436 INFO:teuthology.orchestra.run.smithi092.stderr: 4 Sep 17:42:55 ntpd[18422]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-04T17:42:55.436 INFO:teuthology.orchestra.run.smithi092.stderr: 4 Sep 17:42:55 ntpd[18422]: can't open /var/log/ntpstats/loopstats.20240904: Permission denied 2024-09-04T17:42:55.464 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:55 ntpd[18425]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-04T17:42:55.464 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:55 ntpd[18425]: can't open /var/log/ntpstats/rawstats.20240904: Permission denied 2024-09-04T17:42:55.464 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:55 ntpd[18425]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-04T17:42:55.464 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:55 ntpd[18425]: can't open /var/log/ntpstats/rawstats.20240904: Permission denied 2024-09-04T17:42:55.464 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:55 ntpd[18425]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-04T17:42:55.465 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:55 ntpd[18425]: can't open /var/log/ntpstats/rawstats.20240904: Permission denied 2024-09-04T17:42:55.465 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:55 ntpd[18425]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-04T17:42:55.465 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:55 ntpd[18425]: can't open /var/log/ntpstats/peerstats.20240904: Permission denied 2024-09-04T17:42:55.465 INFO:teuthology.orchestra.run.smithi190.stdout: 4 Sep 17:42:55 ntpd[18425]: ntpd: time slew -0.002218 s 2024-09-04T17:42:55.465 INFO:teuthology.orchestra.run.smithi190.stdout:ntpd: time slew -0.002218s 2024-09-04T17:42:55.465 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:55 ntpd[18425]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-04T17:42:55.466 INFO:teuthology.orchestra.run.smithi190.stderr: 4 Sep 17:42:55 ntpd[18425]: can't open /var/log/ntpstats/loopstats.20240904: Permission denied 2024-09-04T17:42:55.492 INFO:teuthology.orchestra.run.smithi092.stdout: remote refid st t when poll reach delay offset jitter 2024-09-04T17:42:55.492 INFO:teuthology.orchestra.run.smithi092.stdout:============================================================================== 2024-09-04T17:42:55.492 INFO:teuthology.orchestra.run.smithi092.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-04T17:42:55.492 INFO:teuthology.orchestra.run.smithi092.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-04T17:42:55.492 INFO:teuthology.orchestra.run.smithi092.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-04T17:42:55.492 INFO:teuthology.orchestra.run.smithi092.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-04T17:42:55.519 INFO:teuthology.orchestra.run.smithi190.stdout: remote refid st t when poll reach delay offset jitter 2024-09-04T17:42:55.519 INFO:teuthology.orchestra.run.smithi190.stdout:============================================================================== 2024-09-04T17:42:55.519 INFO:teuthology.orchestra.run.smithi190.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-04T17:42:55.519 INFO:teuthology.orchestra.run.smithi190.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-04T17:42:55.519 INFO:teuthology.orchestra.run.smithi190.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-04T17:42:55.519 INFO:teuthology.orchestra.run.smithi190.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-04T17:42:55.520 INFO:teuthology.run_tasks:Running task install... 2024-09-04T17:42:55.529 DEBUG:teuthology.task.install:project ceph 2024-09-04T17:42:55.529 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab'}} 2024-09-04T17:42:55.529 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab'} 2024-09-04T17:42:55.529 INFO:teuthology.task.install:Using flavor: default 2024-09-04T17:42:55.534 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-04T17:42:55.535 INFO:teuthology.task.install:extra packages: [] 2024-09-04T17:42:55.535 DEBUG:teuthology.orchestra.run.smithi092:> sudo apt-key list | grep Ceph 2024-09-04T17:42:55.535 DEBUG:teuthology.orchestra.run.smithi190:> sudo apt-key list | grep Ceph 2024-09-04T17:42:55.593 INFO:teuthology.orchestra.run.smithi092.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-04T17:42:55.620 INFO:teuthology.orchestra.run.smithi190.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-04T17:42:55.621 INFO:teuthology.orchestra.run.smithi092.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-04T17:42:55.621 INFO:teuthology.orchestra.run.smithi092.stdout:uid [ unknown] Ceph.com (release key) 2024-09-04T17:42:55.622 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-04T17:42:55.622 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab 2024-09-04T17:42:55.648 INFO:teuthology.orchestra.run.smithi190.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-04T17:42:55.648 INFO:teuthology.orchestra.run.smithi190.stdout:uid [ unknown] Ceph.com (release key) 2024-09-04T17:42:55.649 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-04T17:42:55.649 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab 2024-09-04T17:42:55.766 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default/ 2024-09-04T17:42:55.766 INFO:teuthology.task.install.deb:Package version is 19.3.0-4703-g3e4a7f4c-1jammy 2024-09-04T17:42:55.787 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default/ 2024-09-04T17:42:55.787 INFO:teuthology.task.install.deb:Package version is 19.3.0-4703-g3e4a7f4c-1jammy 2024-09-04T17:42:55.874 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-04T17:42:55.874 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-04T17:42:55.888 DEBUG:teuthology.orchestra.run.smithi092:> sudo apt-get update 2024-09-04T17:42:55.890 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-04T17:42:55.891 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-04T17:42:55.904 DEBUG:teuthology.orchestra.run.smithi190:> sudo apt-get update 2024-09-04T17:42:56.058 INFO:teuthology.orchestra.run.smithi092.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-04T17:42:56.059 INFO:teuthology.orchestra.run.smithi092.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-04T17:42:56.069 INFO:teuthology.orchestra.run.smithi092.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-04T17:42:56.071 INFO:teuthology.orchestra.run.smithi190.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-04T17:42:56.082 INFO:teuthology.orchestra.run.smithi190.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-04T17:42:56.086 INFO:teuthology.orchestra.run.smithi092.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-04T17:42:56.099 INFO:teuthology.orchestra.run.smithi190.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-04T17:42:56.115 INFO:teuthology.orchestra.run.smithi092.stdout:Ign:5 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy InRelease 2024-09-04T17:42:56.126 INFO:teuthology.orchestra.run.smithi190.stdout:Ign:4 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy InRelease 2024-09-04T17:42:56.138 INFO:teuthology.orchestra.run.smithi092.stdout:Get:6 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy Release [7,005 B] 2024-09-04T17:42:56.147 INFO:teuthology.orchestra.run.smithi190.stdout:Get:5 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy Release [7,005 B] 2024-09-04T17:42:56.160 INFO:teuthology.orchestra.run.smithi092.stdout:Ign:7 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-04T17:42:56.172 INFO:teuthology.orchestra.run.smithi190.stdout:Ign:6 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-04T17:42:56.181 INFO:teuthology.orchestra.run.smithi092.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.2 kB] 2024-09-04T17:42:56.193 INFO:teuthology.orchestra.run.smithi190.stdout:Hit:7 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-04T17:42:56.194 INFO:teuthology.orchestra.run.smithi190.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.2 kB] 2024-09-04T17:42:56.209 INFO:teuthology.orchestra.run.smithi092.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,424 B] 2024-09-04T17:42:56.221 INFO:teuthology.orchestra.run.smithi190.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,424 B] 2024-09-04T17:42:56.401 INFO:teuthology.orchestra.run.smithi092.stdout:Fetched 28.6 kB in 0s (85.2 kB/s) 2024-09-04T17:42:56.424 INFO:teuthology.orchestra.run.smithi190.stdout:Fetched 28.6 kB in 0s (86.0 kB/s) 2024-09-04T17:42:57.434 INFO:teuthology.orchestra.run.smithi092.stdout:Reading package lists... 2024-09-04T17:42:57.450 DEBUG:teuthology.orchestra.run.smithi092:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.3.0-4703-g3e4a7f4c-1jammy cephadm=19.3.0-4703-g3e4a7f4c-1jammy ceph-mds=19.3.0-4703-g3e4a7f4c-1jammy ceph-mgr=19.3.0-4703-g3e4a7f4c-1jammy ceph-common=19.3.0-4703-g3e4a7f4c-1jammy ceph-fuse=19.3.0-4703-g3e4a7f4c-1jammy ceph-test=19.3.0-4703-g3e4a7f4c-1jammy ceph-volume=19.3.0-4703-g3e4a7f4c-1jammy radosgw=19.3.0-4703-g3e4a7f4c-1jammy python3-rados=19.3.0-4703-g3e4a7f4c-1jammy python3-rgw=19.3.0-4703-g3e4a7f4c-1jammy python3-cephfs=19.3.0-4703-g3e4a7f4c-1jammy python3-rbd=19.3.0-4703-g3e4a7f4c-1jammy libcephfs2=19.3.0-4703-g3e4a7f4c-1jammy libcephfs-dev=19.3.0-4703-g3e4a7f4c-1jammy librados2=19.3.0-4703-g3e4a7f4c-1jammy librbd1=19.3.0-4703-g3e4a7f4c-1jammy rbd-fuse=19.3.0-4703-g3e4a7f4c-1jammy 2024-09-04T17:42:57.473 INFO:teuthology.orchestra.run.smithi190.stdout:Reading package lists... 2024-09-04T17:42:57.490 DEBUG:teuthology.orchestra.run.smithi190:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.3.0-4703-g3e4a7f4c-1jammy cephadm=19.3.0-4703-g3e4a7f4c-1jammy ceph-mds=19.3.0-4703-g3e4a7f4c-1jammy ceph-mgr=19.3.0-4703-g3e4a7f4c-1jammy ceph-common=19.3.0-4703-g3e4a7f4c-1jammy ceph-fuse=19.3.0-4703-g3e4a7f4c-1jammy ceph-test=19.3.0-4703-g3e4a7f4c-1jammy ceph-volume=19.3.0-4703-g3e4a7f4c-1jammy radosgw=19.3.0-4703-g3e4a7f4c-1jammy python3-rados=19.3.0-4703-g3e4a7f4c-1jammy python3-rgw=19.3.0-4703-g3e4a7f4c-1jammy python3-cephfs=19.3.0-4703-g3e4a7f4c-1jammy python3-rbd=19.3.0-4703-g3e4a7f4c-1jammy libcephfs2=19.3.0-4703-g3e4a7f4c-1jammy libcephfs-dev=19.3.0-4703-g3e4a7f4c-1jammy librados2=19.3.0-4703-g3e4a7f4c-1jammy librbd1=19.3.0-4703-g3e4a7f4c-1jammy rbd-fuse=19.3.0-4703-g3e4a7f4c-1jammy 2024-09-04T17:42:57.512 INFO:teuthology.orchestra.run.smithi092.stdout:Reading package lists... 2024-09-04T17:42:57.552 INFO:teuthology.orchestra.run.smithi190.stdout:Reading package lists... 2024-09-04T17:42:57.721 INFO:teuthology.orchestra.run.smithi092.stdout:Building dependency tree... 2024-09-04T17:42:57.721 INFO:teuthology.orchestra.run.smithi092.stdout:Reading state information... 2024-09-04T17:42:57.762 INFO:teuthology.orchestra.run.smithi190.stdout:Building dependency tree... 2024-09-04T17:42:57.762 INFO:teuthology.orchestra.run.smithi190.stdout:Reading state information... 2024-09-04T17:42:57.912 INFO:teuthology.orchestra.run.smithi092.stdout:The following packages were automatically installed and are no longer required: 2024-09-04T17:42:57.912 INFO:teuthology.orchestra.run.smithi092.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-04T17:42:57.913 INFO:teuthology.orchestra.run.smithi092.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-04T17:42:57.913 INFO:teuthology.orchestra.run.smithi092.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-04T17:42:57.913 INFO:teuthology.orchestra.run.smithi092.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-04T17:42:57.913 INFO:teuthology.orchestra.run.smithi092.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-04T17:42:57.913 INFO:teuthology.orchestra.run.smithi092.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-04T17:42:57.914 INFO:teuthology.orchestra.run.smithi092.stdout:The following additional packages will be installed: 2024-09-04T17:42:57.914 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-04T17:42:57.914 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-04T17:42:57.914 INFO:teuthology.orchestra.run.smithi092.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-04T17:42:57.914 INFO:teuthology.orchestra.run.smithi092.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-04T17:42:57.915 INFO:teuthology.orchestra.run.smithi092.stdout: libradosstriper1 librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph 2024-09-04T17:42:57.915 INFO:teuthology.orchestra.run.smithi092.stdout: libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli 2024-09-04T17:42:57.916 INFO:teuthology.orchestra.run.smithi092.stdout: pkg-config python-asyncssh-doc python-babel-localedata 2024-09-04T17:42:57.916 INFO:teuthology.orchestra.run.smithi092.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-04T17:42:57.916 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-certifi 2024-09-04T17:42:57.916 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cheroot python3-cherrypy3 python3-google-auth python3-idna 2024-09-04T17:42:57.916 INFO:teuthology.orchestra.run.smithi092.stdout: python3-iniconfig python3-jaraco.classes python3-jaraco.collections 2024-09-04T17:42:57.916 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco.functools python3-jaraco.text python3-jinja2 python3-joblib 2024-09-04T17:42:57.916 INFO:teuthology.orchestra.run.smithi092.stdout: python3-kubernetes python3-logutils python3-mako python3-markupsafe 2024-09-04T17:42:57.916 INFO:teuthology.orchestra.run.smithi092.stdout: python3-natsort python3-openssl python3-paste python3-pastedeploy 2024-09-04T17:42:57.916 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pastescript python3-pecan python3-pluggy python3-portend 2024-09-04T17:42:57.916 INFO:teuthology.orchestra.run.smithi092.stdout: python3-prettytable python3-psutil python3-py python3-pyasn1 2024-09-04T17:42:57.916 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-modules python3-pygments python3-pyinotify python3-pytest 2024-09-04T17:42:57.916 INFO:teuthology.orchestra.run.smithi092.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-routes 2024-09-04T17:42:57.916 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rsa python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-04T17:42:57.917 INFO:teuthology.orchestra.run.smithi092.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-04T17:42:57.917 INFO:teuthology.orchestra.run.smithi092.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-04T17:42:57.917 INFO:teuthology.orchestra.run.smithi092.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-04T17:42:57.917 INFO:teuthology.orchestra.run.smithi092.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-09-04T17:42:57.917 INFO:teuthology.orchestra.run.smithi092.stdout: xmlstarlet zip 2024-09-04T17:42:57.917 INFO:teuthology.orchestra.run.smithi092.stdout:Suggested packages: 2024-09-04T17:42:57.917 INFO:teuthology.orchestra.run.smithi092.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-04T17:42:57.917 INFO:teuthology.orchestra.run.smithi092.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-04T17:42:57.917 INFO:teuthology.orchestra.run.smithi092.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-04T17:42:57.918 INFO:teuthology.orchestra.run.smithi092.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-04T17:42:57.918 INFO:teuthology.orchestra.run.smithi092.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-04T17:42:57.918 INFO:teuthology.orchestra.run.smithi092.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-04T17:42:57.918 INFO:teuthology.orchestra.run.smithi092.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-04T17:42:57.918 INFO:teuthology.orchestra.run.smithi092.stdout:Recommended packages: 2024-09-04T17:42:57.918 INFO:teuthology.orchestra.run.smithi092.stdout: btrfs-tools 2024-09-04T17:42:57.962 INFO:teuthology.orchestra.run.smithi190.stdout:The following packages were automatically installed and are no longer required: 2024-09-04T17:42:57.962 INFO:teuthology.orchestra.run.smithi190.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-04T17:42:57.962 INFO:teuthology.orchestra.run.smithi190.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-04T17:42:57.962 INFO:teuthology.orchestra.run.smithi190.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-04T17:42:57.963 INFO:teuthology.orchestra.run.smithi190.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-04T17:42:57.963 INFO:teuthology.orchestra.run.smithi190.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-04T17:42:57.963 INFO:teuthology.orchestra.run.smithi190.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-04T17:42:57.964 INFO:teuthology.orchestra.run.smithi190.stdout:The following additional packages will be installed: 2024-09-04T17:42:57.964 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-04T17:42:57.964 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-04T17:42:57.964 INFO:teuthology.orchestra.run.smithi190.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-04T17:42:57.964 INFO:teuthology.orchestra.run.smithi190.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-04T17:42:57.964 INFO:teuthology.orchestra.run.smithi190.stdout: libradosstriper1 librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph 2024-09-04T17:42:57.965 INFO:teuthology.orchestra.run.smithi190.stdout: libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli 2024-09-04T17:42:57.965 INFO:teuthology.orchestra.run.smithi190.stdout: pkg-config python-asyncssh-doc python-babel-localedata 2024-09-04T17:42:57.965 INFO:teuthology.orchestra.run.smithi190.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-04T17:42:57.965 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-certifi 2024-09-04T17:42:57.965 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cheroot python3-cherrypy3 python3-google-auth python3-idna 2024-09-04T17:42:57.965 INFO:teuthology.orchestra.run.smithi190.stdout: python3-iniconfig python3-jaraco.classes python3-jaraco.collections 2024-09-04T17:42:57.966 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco.functools python3-jaraco.text python3-jinja2 python3-joblib 2024-09-04T17:42:57.966 INFO:teuthology.orchestra.run.smithi190.stdout: python3-kubernetes python3-logutils python3-mako python3-markupsafe 2024-09-04T17:42:57.966 INFO:teuthology.orchestra.run.smithi190.stdout: python3-natsort python3-openssl python3-paste python3-pastedeploy 2024-09-04T17:42:57.966 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pastescript python3-pecan python3-pluggy python3-portend 2024-09-04T17:42:57.966 INFO:teuthology.orchestra.run.smithi190.stdout: python3-prettytable python3-psutil python3-py python3-pyasn1 2024-09-04T17:42:57.966 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1-modules python3-pygments python3-pyinotify python3-pytest 2024-09-04T17:42:57.966 INFO:teuthology.orchestra.run.smithi190.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-routes 2024-09-04T17:42:57.966 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rsa python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-04T17:42:57.966 INFO:teuthology.orchestra.run.smithi190.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-04T17:42:57.966 INFO:teuthology.orchestra.run.smithi190.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-04T17:42:57.966 INFO:teuthology.orchestra.run.smithi190.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-04T17:42:57.966 INFO:teuthology.orchestra.run.smithi190.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-09-04T17:42:57.966 INFO:teuthology.orchestra.run.smithi190.stdout: xmlstarlet zip 2024-09-04T17:42:57.967 INFO:teuthology.orchestra.run.smithi190.stdout:Suggested packages: 2024-09-04T17:42:57.967 INFO:teuthology.orchestra.run.smithi190.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-04T17:42:57.967 INFO:teuthology.orchestra.run.smithi190.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-04T17:42:57.967 INFO:teuthology.orchestra.run.smithi190.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-04T17:42:57.967 INFO:teuthology.orchestra.run.smithi190.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-04T17:42:57.967 INFO:teuthology.orchestra.run.smithi190.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-04T17:42:57.967 INFO:teuthology.orchestra.run.smithi190.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-04T17:42:57.967 INFO:teuthology.orchestra.run.smithi190.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-04T17:42:57.967 INFO:teuthology.orchestra.run.smithi190.stdout:Recommended packages: 2024-09-04T17:42:57.967 INFO:teuthology.orchestra.run.smithi190.stdout: btrfs-tools 2024-09-04T17:42:58.019 INFO:teuthology.orchestra.run.smithi092.stdout:The following NEW packages will be installed: 2024-09-04T17:42:58.019 INFO:teuthology.orchestra.run.smithi092.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-04T17:42:58.019 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-04T17:42:58.019 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-04T17:42:58.021 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-09-04T17:42:58.021 INFO:teuthology.orchestra.run.smithi092.stdout: liblua5.3-dev libnbd0 liboath0 libonig5 libpcre2-16-0 libqt5core5a 2024-09-04T17:42:58.021 INFO:teuthology.orchestra.run.smithi092.stdout: libqt5dbus5 libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-09-04T17:42:58.021 INFO:teuthology.orchestra.run.smithi092.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-09-04T17:42:58.021 INFO:teuthology.orchestra.run.smithi092.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-09-04T17:42:58.021 INFO:teuthology.orchestra.run.smithi092.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-09-04T17:42:58.021 INFO:teuthology.orchestra.run.smithi092.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-09-04T17:42:58.021 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-09-04T17:42:58.021 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-04T17:42:58.021 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-04T17:42:58.022 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-04T17:42:58.022 INFO:teuthology.orchestra.run.smithi092.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-04T17:42:58.022 INFO:teuthology.orchestra.run.smithi092.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-04T17:42:58.022 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-04T17:42:58.022 INFO:teuthology.orchestra.run.smithi092.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-04T17:42:58.022 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-04T17:42:58.022 INFO:teuthology.orchestra.run.smithi092.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-09-04T17:42:58.022 INFO:teuthology.orchestra.run.smithi092.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-09-04T17:42:58.022 INFO:teuthology.orchestra.run.smithi092.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-09-04T17:42:58.022 INFO:teuthology.orchestra.run.smithi092.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-09-04T17:42:58.022 INFO:teuthology.orchestra.run.smithi092.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-09-04T17:42:58.022 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-09-04T17:42:58.022 INFO:teuthology.orchestra.run.smithi092.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-09-04T17:42:58.069 INFO:teuthology.orchestra.run.smithi190.stdout:The following NEW packages will be installed: 2024-09-04T17:42:58.070 INFO:teuthology.orchestra.run.smithi190.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-04T17:42:58.070 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-04T17:42:58.070 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-04T17:42:58.070 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-09-04T17:42:58.070 INFO:teuthology.orchestra.run.smithi190.stdout: liblua5.3-dev libnbd0 liboath0 libonig5 libpcre2-16-0 libqt5core5a 2024-09-04T17:42:58.070 INFO:teuthology.orchestra.run.smithi190.stdout: libqt5dbus5 libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-09-04T17:42:58.071 INFO:teuthology.orchestra.run.smithi190.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-09-04T17:42:58.071 INFO:teuthology.orchestra.run.smithi190.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-09-04T17:42:58.071 INFO:teuthology.orchestra.run.smithi190.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-09-04T17:42:58.071 INFO:teuthology.orchestra.run.smithi190.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-09-04T17:42:58.071 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-09-04T17:42:58.071 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-04T17:42:58.071 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-04T17:42:58.071 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-04T17:42:58.071 INFO:teuthology.orchestra.run.smithi190.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-04T17:42:58.072 INFO:teuthology.orchestra.run.smithi190.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-04T17:42:58.072 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-04T17:42:58.072 INFO:teuthology.orchestra.run.smithi190.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-04T17:42:58.072 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-04T17:42:58.072 INFO:teuthology.orchestra.run.smithi190.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-09-04T17:42:58.072 INFO:teuthology.orchestra.run.smithi190.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-09-04T17:42:58.072 INFO:teuthology.orchestra.run.smithi190.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-09-04T17:42:58.072 INFO:teuthology.orchestra.run.smithi190.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-09-04T17:42:58.072 INFO:teuthology.orchestra.run.smithi190.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-09-04T17:42:58.072 INFO:teuthology.orchestra.run.smithi190.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-09-04T17:42:58.072 INFO:teuthology.orchestra.run.smithi190.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-09-04T17:42:58.130 INFO:teuthology.orchestra.run.smithi190.stdout:0 upgraded, 119 newly installed, 0 to remove and 331 not upgraded. 2024-09-04T17:42:58.130 INFO:teuthology.orchestra.run.smithi190.stdout:Need to get 251 MB of archives. 2024-09-04T17:42:58.130 INFO:teuthology.orchestra.run.smithi190.stdout:After this operation, 933 MB of additional disk space will be used. 2024-09-04T17:42:58.130 INFO:teuthology.orchestra.run.smithi190.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-04T17:42:58.163 INFO:teuthology.orchestra.run.smithi092.stdout:0 upgraded, 119 newly installed, 0 to remove and 331 not upgraded. 2024-09-04T17:42:58.163 INFO:teuthology.orchestra.run.smithi092.stdout:Need to get 251 MB of archives. 2024-09-04T17:42:58.163 INFO:teuthology.orchestra.run.smithi092.stdout:After this operation, 933 MB of additional disk space will be used. 2024-09-04T17:42:58.163 INFO:teuthology.orchestra.run.smithi092.stdout:Get:1 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.3.0-4703-g3e4a7f4c-1jammy [3,631 kB] 2024-09-04T17:42:58.210 INFO:teuthology.orchestra.run.smithi092.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-04T17:42:58.212 INFO:teuthology.orchestra.run.smithi190.stdout:Get:2 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.3.0-4703-g3e4a7f4c-1jammy [3,631 kB] 2024-09-04T17:42:58.217 INFO:teuthology.orchestra.run.smithi190.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-04T17:42:58.220 INFO:teuthology.orchestra.run.smithi190.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-04T17:42:58.237 INFO:teuthology.orchestra.run.smithi190.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-04T17:42:58.295 INFO:teuthology.orchestra.run.smithi190.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-04T17:42:58.300 INFO:teuthology.orchestra.run.smithi190.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-04T17:42:58.312 INFO:teuthology.orchestra.run.smithi190.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-04T17:42:58.317 INFO:teuthology.orchestra.run.smithi190.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2024-09-04T17:42:58.317 INFO:teuthology.orchestra.run.smithi190.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-04T17:42:58.318 INFO:teuthology.orchestra.run.smithi190.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-04T17:42:58.319 INFO:teuthology.orchestra.run.smithi190.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-04T17:42:58.327 INFO:teuthology.orchestra.run.smithi190.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-04T17:42:58.329 INFO:teuthology.orchestra.run.smithi190.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-04T17:42:58.331 INFO:teuthology.orchestra.run.smithi190.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-04T17:42:58.333 INFO:teuthology.orchestra.run.smithi190.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-04T17:42:58.334 INFO:teuthology.orchestra.run.smithi190.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-04T17:42:58.350 INFO:teuthology.orchestra.run.smithi092.stdout:Get:3 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.3.0-4703-g3e4a7f4c-1jammy [3,335 kB] 2024-09-04T17:42:58.350 INFO:teuthology.orchestra.run.smithi190.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-04T17:42:58.353 INFO:teuthology.orchestra.run.smithi190.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-04T17:42:58.355 INFO:teuthology.orchestra.run.smithi190.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-04T17:42:58.355 INFO:teuthology.orchestra.run.smithi190.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-04T17:42:58.355 INFO:teuthology.orchestra.run.smithi190.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-04T17:42:58.357 INFO:teuthology.orchestra.run.smithi190.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-04T17:42:58.357 INFO:teuthology.orchestra.run.smithi190.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-04T17:42:58.357 INFO:teuthology.orchestra.run.smithi190.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-04T17:42:58.368 INFO:teuthology.orchestra.run.smithi190.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-04T17:42:58.368 INFO:teuthology.orchestra.run.smithi190.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-04T17:42:58.384 INFO:teuthology.orchestra.run.smithi190.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-04T17:42:58.385 INFO:teuthology.orchestra.run.smithi190.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-04T17:42:58.387 INFO:teuthology.orchestra.run.smithi190.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-04T17:42:58.388 INFO:teuthology.orchestra.run.smithi190.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-04T17:42:58.388 INFO:teuthology.orchestra.run.smithi190.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-04T17:42:58.389 INFO:teuthology.orchestra.run.smithi190.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-04T17:42:58.390 INFO:teuthology.orchestra.run.smithi190.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-04T17:42:58.390 INFO:teuthology.orchestra.run.smithi190.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-04T17:42:58.402 INFO:teuthology.orchestra.run.smithi190.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-04T17:42:58.403 INFO:teuthology.orchestra.run.smithi190.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-04T17:42:58.418 INFO:teuthology.orchestra.run.smithi190.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-04T17:42:58.419 INFO:teuthology.orchestra.run.smithi190.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-04T17:42:58.424 INFO:teuthology.orchestra.run.smithi190.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-04T17:42:58.425 INFO:teuthology.orchestra.run.smithi190.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-04T17:42:58.425 INFO:teuthology.orchestra.run.smithi190.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-04T17:42:58.425 INFO:teuthology.orchestra.run.smithi190.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-04T17:42:58.427 INFO:teuthology.orchestra.run.smithi190.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-04T17:42:58.429 INFO:teuthology.orchestra.run.smithi190.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-04T17:42:58.430 INFO:teuthology.orchestra.run.smithi092.stdout:Get:4 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.3.0-4703-g3e4a7f4c-1jammy [722 kB] 2024-09-04T17:42:58.435 INFO:teuthology.orchestra.run.smithi190.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-04T17:42:58.437 INFO:teuthology.orchestra.run.smithi190.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-04T17:42:58.442 INFO:teuthology.orchestra.run.smithi092.stdout:Get:5 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.3.0-4703-g3e4a7f4c-1jammy [356 kB] 2024-09-04T17:42:58.447 INFO:teuthology.orchestra.run.smithi190.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-04T17:42:58.448 INFO:teuthology.orchestra.run.smithi092.stdout:Get:6 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.3.0-4703-g3e4a7f4c-1jammy [32.8 kB] 2024-09-04T17:42:58.449 INFO:teuthology.orchestra.run.smithi092.stdout:Get:7 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.3.0-4703-g3e4a7f4c-1jammy [180 kB] 2024-09-04T17:42:58.449 INFO:teuthology.orchestra.run.smithi190.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-04T17:42:58.450 INFO:teuthology.orchestra.run.smithi190.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-04T17:42:58.450 INFO:teuthology.orchestra.run.smithi190.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-04T17:42:58.452 INFO:teuthology.orchestra.run.smithi092.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.3.0-4703-g3e4a7f4c-1jammy [71.2 kB] 2024-09-04T17:42:58.453 INFO:teuthology.orchestra.run.smithi190.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-04T17:42:58.453 INFO:teuthology.orchestra.run.smithi092.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.3.0-4703-g3e4a7f4c-1jammy [338 kB] 2024-09-04T17:42:58.458 INFO:teuthology.orchestra.run.smithi092.stdout:Get:10 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.3.0-4703-g3e4a7f4c-1jammy [7,979 kB] 2024-09-04T17:42:58.463 INFO:teuthology.orchestra.run.smithi190.stdout:Get:53 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.3.0-4703-g3e4a7f4c-1jammy [3,335 kB] 2024-09-04T17:42:58.506 INFO:teuthology.orchestra.run.smithi190.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-04T17:42:58.507 INFO:teuthology.orchestra.run.smithi190.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-04T17:42:58.508 INFO:teuthology.orchestra.run.smithi190.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-04T17:42:58.508 INFO:teuthology.orchestra.run.smithi190.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-04T17:42:58.509 INFO:teuthology.orchestra.run.smithi190.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-04T17:42:58.526 INFO:teuthology.orchestra.run.smithi190.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-04T17:42:58.528 INFO:teuthology.orchestra.run.smithi190.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-04T17:42:58.528 INFO:teuthology.orchestra.run.smithi190.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-04T17:42:58.542 INFO:teuthology.orchestra.run.smithi190.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-04T17:42:58.542 INFO:teuthology.orchestra.run.smithi190.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-04T17:42:58.543 INFO:teuthology.orchestra.run.smithi190.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-04T17:42:58.543 INFO:teuthology.orchestra.run.smithi190.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-04T17:42:58.544 INFO:teuthology.orchestra.run.smithi190.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-04T17:42:58.544 INFO:teuthology.orchestra.run.smithi190.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-04T17:42:58.559 INFO:teuthology.orchestra.run.smithi190.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-04T17:42:58.560 INFO:teuthology.orchestra.run.smithi190.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-04T17:42:58.563 INFO:teuthology.orchestra.run.smithi190.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-04T17:42:58.576 INFO:teuthology.orchestra.run.smithi190.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-04T17:42:58.578 INFO:teuthology.orchestra.run.smithi190.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-04T17:42:58.578 INFO:teuthology.orchestra.run.smithi190.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-04T17:42:58.581 INFO:teuthology.orchestra.run.smithi190.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-04T17:42:58.583 INFO:teuthology.orchestra.run.smithi190.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-04T17:42:58.584 INFO:teuthology.orchestra.run.smithi190.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-04T17:42:58.584 INFO:teuthology.orchestra.run.smithi190.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.2 [473 kB] 2024-09-04T17:42:58.593 INFO:teuthology.orchestra.run.smithi190.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-04T17:42:58.594 INFO:teuthology.orchestra.run.smithi190.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-04T17:42:58.596 INFO:teuthology.orchestra.run.smithi190.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-04T17:42:58.610 INFO:teuthology.orchestra.run.smithi190.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-04T17:42:58.610 INFO:teuthology.orchestra.run.smithi190.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-04T17:42:58.611 INFO:teuthology.orchestra.run.smithi190.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-04T17:42:58.612 INFO:teuthology.orchestra.run.smithi190.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-04T17:42:58.613 INFO:teuthology.orchestra.run.smithi190.stdout:Get:85 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.3.0-4703-g3e4a7f4c-1jammy [722 kB] 2024-09-04T17:42:58.613 INFO:teuthology.orchestra.run.smithi190.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-04T17:42:58.619 INFO:teuthology.orchestra.run.smithi190.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-04T17:42:58.619 INFO:teuthology.orchestra.run.smithi190.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-04T17:42:58.627 INFO:teuthology.orchestra.run.smithi190.stdout:Get:89 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.3.0-4703-g3e4a7f4c-1jammy [356 kB] 2024-09-04T17:42:58.629 INFO:teuthology.orchestra.run.smithi190.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-04T17:42:58.631 INFO:teuthology.orchestra.run.smithi190.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-04T17:42:58.632 INFO:teuthology.orchestra.run.smithi190.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-04T17:42:58.634 INFO:teuthology.orchestra.run.smithi190.stdout:Get:93 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.3.0-4703-g3e4a7f4c-1jammy [32.8 kB] 2024-09-04T17:42:58.635 INFO:teuthology.orchestra.run.smithi190.stdout:Get:94 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.3.0-4703-g3e4a7f4c-1jammy [180 kB] 2024-09-04T17:42:58.636 INFO:teuthology.orchestra.run.smithi092.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-04T17:42:58.638 INFO:teuthology.orchestra.run.smithi190.stdout:Get:95 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.3.0-4703-g3e4a7f4c-1jammy [71.2 kB] 2024-09-04T17:42:58.639 INFO:teuthology.orchestra.run.smithi190.stdout:Get:96 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.3.0-4703-g3e4a7f4c-1jammy [338 kB] 2024-09-04T17:42:58.645 INFO:teuthology.orchestra.run.smithi190.stdout:Get:97 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.3.0-4703-g3e4a7f4c-1jammy [7,979 kB] 2024-09-04T17:42:58.646 INFO:teuthology.orchestra.run.smithi092.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-04T17:42:58.730 INFO:teuthology.orchestra.run.smithi092.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-04T17:42:58.750 INFO:teuthology.orchestra.run.smithi092.stdout:Get:14 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.3.0-4703-g3e4a7f4c-1jammy [112 kB] 2024-09-04T17:42:58.755 INFO:teuthology.orchestra.run.smithi092.stdout:Get:15 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.3.0-4703-g3e4a7f4c-1jammy [473 kB] 2024-09-04T17:42:58.767 INFO:teuthology.orchestra.run.smithi092.stdout:Get:16 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.3.0-4703-g3e4a7f4c-1jammy [28.2 MB] 2024-09-04T17:42:58.851 INFO:teuthology.orchestra.run.smithi190.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.3.0-4703-g3e4a7f4c-1jammy [112 kB] 2024-09-04T17:42:58.853 INFO:teuthology.orchestra.run.smithi190.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.3.0-4703-g3e4a7f4c-1jammy [473 kB] 2024-09-04T17:42:58.862 INFO:teuthology.orchestra.run.smithi190.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.3.0-4703-g3e4a7f4c-1jammy [28.2 MB] 2024-09-04T17:42:58.961 INFO:teuthology.orchestra.run.smithi092.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-04T17:42:58.971 INFO:teuthology.orchestra.run.smithi092.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-04T17:42:59.001 INFO:teuthology.orchestra.run.smithi092.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-04T17:42:59.010 INFO:teuthology.orchestra.run.smithi092.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2024-09-04T17:42:59.012 INFO:teuthology.orchestra.run.smithi092.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-04T17:42:59.013 INFO:teuthology.orchestra.run.smithi092.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-04T17:42:59.014 INFO:teuthology.orchestra.run.smithi092.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-04T17:42:59.032 INFO:teuthology.orchestra.run.smithi092.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-04T17:42:59.037 INFO:teuthology.orchestra.run.smithi092.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-04T17:42:59.044 INFO:teuthology.orchestra.run.smithi092.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-04T17:42:59.124 INFO:teuthology.orchestra.run.smithi092.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-04T17:42:59.124 INFO:teuthology.orchestra.run.smithi092.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-04T17:42:59.128 INFO:teuthology.orchestra.run.smithi092.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-04T17:42:59.133 INFO:teuthology.orchestra.run.smithi092.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-04T17:42:59.137 INFO:teuthology.orchestra.run.smithi092.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-04T17:42:59.138 INFO:teuthology.orchestra.run.smithi092.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-04T17:42:59.139 INFO:teuthology.orchestra.run.smithi092.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-04T17:42:59.141 INFO:teuthology.orchestra.run.smithi092.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-04T17:42:59.141 INFO:teuthology.orchestra.run.smithi092.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-04T17:42:59.208 INFO:teuthology.orchestra.run.smithi092.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-04T17:42:59.224 INFO:teuthology.orchestra.run.smithi092.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-04T17:42:59.225 INFO:teuthology.orchestra.run.smithi092.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-04T17:42:59.225 INFO:teuthology.orchestra.run.smithi092.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-04T17:42:59.226 INFO:teuthology.orchestra.run.smithi092.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-04T17:42:59.231 INFO:teuthology.orchestra.run.smithi092.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-04T17:42:59.232 INFO:teuthology.orchestra.run.smithi092.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-04T17:42:59.290 INFO:teuthology.orchestra.run.smithi092.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-04T17:42:59.291 INFO:teuthology.orchestra.run.smithi092.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-04T17:42:59.293 INFO:teuthology.orchestra.run.smithi092.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-04T17:42:59.371 INFO:teuthology.orchestra.run.smithi092.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-04T17:42:59.371 INFO:teuthology.orchestra.run.smithi092.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-04T17:42:59.374 INFO:teuthology.orchestra.run.smithi092.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-04T17:42:59.376 INFO:teuthology.orchestra.run.smithi092.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-04T17:42:59.377 INFO:teuthology.orchestra.run.smithi092.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-04T17:42:59.388 INFO:teuthology.orchestra.run.smithi092.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-04T17:42:59.452 INFO:teuthology.orchestra.run.smithi092.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-04T17:42:59.453 INFO:teuthology.orchestra.run.smithi092.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-04T17:42:59.454 INFO:teuthology.orchestra.run.smithi092.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-04T17:42:59.457 INFO:teuthology.orchestra.run.smithi092.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-04T17:42:59.534 INFO:teuthology.orchestra.run.smithi092.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-04T17:42:59.535 INFO:teuthology.orchestra.run.smithi092.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-04T17:42:59.538 INFO:teuthology.orchestra.run.smithi092.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-04T17:42:59.539 INFO:teuthology.orchestra.run.smithi092.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-04T17:42:59.545 INFO:teuthology.orchestra.run.smithi092.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-04T17:42:59.546 INFO:teuthology.orchestra.run.smithi092.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-04T17:42:59.615 INFO:teuthology.orchestra.run.smithi092.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-04T17:42:59.619 INFO:teuthology.orchestra.run.smithi092.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-04T17:42:59.828 INFO:teuthology.orchestra.run.smithi092.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-04T17:42:59.843 INFO:teuthology.orchestra.run.smithi190.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.3.0-4703-g3e4a7f4c-1jammy [5,203 kB] 2024-09-04T17:42:59.886 INFO:teuthology.orchestra.run.smithi092.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-04T17:42:59.888 INFO:teuthology.orchestra.run.smithi092.stdout:Get:66 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.3.0-4703-g3e4a7f4c-1jammy [5,203 kB] 2024-09-04T17:42:59.889 INFO:teuthology.orchestra.run.smithi092.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-04T17:42:59.889 INFO:teuthology.orchestra.run.smithi092.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-04T17:42:59.892 INFO:teuthology.orchestra.run.smithi092.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-04T17:42:59.922 INFO:teuthology.orchestra.run.smithi190.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.3.0-4703-g3e4a7f4c-1jammy [251 kB] 2024-09-04T17:42:59.924 INFO:teuthology.orchestra.run.smithi190.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.3.0-4703-g3e4a7f4c-1jammy [124 kB] 2024-09-04T17:42:59.926 INFO:teuthology.orchestra.run.smithi190.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.3.0-4703-g3e4a7f4c-1jammy [1,561 kB] 2024-09-04T17:42:59.937 INFO:teuthology.orchestra.run.smithi092.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-04T17:42:59.939 INFO:teuthology.orchestra.run.smithi190.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.3.0-4703-g3e4a7f4c-1jammy [6,309 kB] 2024-09-04T17:42:59.941 INFO:teuthology.orchestra.run.smithi092.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-04T17:42:59.941 INFO:teuthology.orchestra.run.smithi092.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-04T17:42:59.966 INFO:teuthology.orchestra.run.smithi092.stdout:Get:73 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.3.0-4703-g3e4a7f4c-1jammy [251 kB] 2024-09-04T17:42:59.973 INFO:teuthology.orchestra.run.smithi092.stdout:Get:74 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.3.0-4703-g3e4a7f4c-1jammy [124 kB] 2024-09-04T17:42:59.975 INFO:teuthology.orchestra.run.smithi092.stdout:Get:75 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.3.0-4703-g3e4a7f4c-1jammy [1,561 kB] 2024-09-04T17:42:59.980 INFO:teuthology.orchestra.run.smithi092.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-04T17:42:59.981 INFO:teuthology.orchestra.run.smithi092.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-04T17:43:00.030 INFO:teuthology.orchestra.run.smithi092.stdout:Get:78 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.3.0-4703-g3e4a7f4c-1jammy [6,309 kB] 2024-09-04T17:43:00.050 INFO:teuthology.orchestra.run.smithi092.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-04T17:43:00.051 INFO:teuthology.orchestra.run.smithi092.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-04T17:43:00.052 INFO:teuthology.orchestra.run.smithi092.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-04T17:43:00.054 INFO:teuthology.orchestra.run.smithi092.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-04T17:43:00.054 INFO:teuthology.orchestra.run.smithi092.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-04T17:43:00.055 INFO:teuthology.orchestra.run.smithi092.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-04T17:43:00.063 INFO:teuthology.orchestra.run.smithi092.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-04T17:43:00.125 INFO:teuthology.orchestra.run.smithi190.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.3.0-4703-g3e4a7f4c-1jammy [23.1 MB] 2024-09-04T17:43:00.132 INFO:teuthology.orchestra.run.smithi092.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-04T17:43:00.135 INFO:teuthology.orchestra.run.smithi092.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-04T17:43:00.137 INFO:teuthology.orchestra.run.smithi092.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-04T17:43:00.213 INFO:teuthology.orchestra.run.smithi092.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-04T17:43:00.219 INFO:teuthology.orchestra.run.smithi092.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-04T17:43:00.222 INFO:teuthology.orchestra.run.smithi092.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-04T17:43:00.223 INFO:teuthology.orchestra.run.smithi092.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.2 [473 kB] 2024-09-04T17:43:00.235 INFO:teuthology.orchestra.run.smithi092.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-04T17:43:00.237 INFO:teuthology.orchestra.run.smithi092.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-04T17:43:00.244 INFO:teuthology.orchestra.run.smithi092.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-04T17:43:00.255 INFO:teuthology.orchestra.run.smithi092.stdout:Get:96 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.3.0-4703-g3e4a7f4c-1jammy [23.1 MB] 2024-09-04T17:43:00.294 INFO:teuthology.orchestra.run.smithi092.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-04T17:43:00.296 INFO:teuthology.orchestra.run.smithi092.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-04T17:43:00.297 INFO:teuthology.orchestra.run.smithi092.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-04T17:43:00.376 INFO:teuthology.orchestra.run.smithi092.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-04T17:43:00.378 INFO:teuthology.orchestra.run.smithi092.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-04T17:43:00.395 INFO:teuthology.orchestra.run.smithi092.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-04T17:43:00.396 INFO:teuthology.orchestra.run.smithi092.stdout:Get:103 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-04T17:43:00.397 INFO:teuthology.orchestra.run.smithi092.stdout:Get:104 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-04T17:43:00.402 INFO:teuthology.orchestra.run.smithi092.stdout:Get:105 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-04T17:43:00.404 INFO:teuthology.orchestra.run.smithi092.stdout:Get:106 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-04T17:43:00.951 INFO:teuthology.orchestra.run.smithi190.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.3.0-4703-g3e4a7f4c-1jammy [14.0 kB] 2024-09-04T17:43:00.952 INFO:teuthology.orchestra.run.smithi190.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.3.0-4703-g3e4a7f4c-1jammy [880 kB] 2024-09-04T17:43:00.969 INFO:teuthology.orchestra.run.smithi190.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.3.0-4703-g3e4a7f4c-1jammy [2,433 kB] 2024-09-04T17:43:01.009 INFO:teuthology.orchestra.run.smithi190.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.3.0-4703-g3e4a7f4c-1jammy [812 kB] 2024-09-04T17:43:01.029 INFO:teuthology.orchestra.run.smithi190.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.3.0-4703-g3e4a7f4c-1jammy [160 kB] 2024-09-04T17:43:01.031 INFO:teuthology.orchestra.run.smithi190.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.3.0-4703-g3e4a7f4c-1jammy [61.4 MB] 2024-09-04T17:43:01.114 INFO:teuthology.orchestra.run.smithi092.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.3.0-4703-g3e4a7f4c-1jammy [14.0 kB] 2024-09-04T17:43:01.115 INFO:teuthology.orchestra.run.smithi092.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.3.0-4703-g3e4a7f4c-1jammy [880 kB] 2024-09-04T17:43:01.132 INFO:teuthology.orchestra.run.smithi092.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.3.0-4703-g3e4a7f4c-1jammy [2,433 kB] 2024-09-04T17:43:01.173 INFO:teuthology.orchestra.run.smithi092.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.3.0-4703-g3e4a7f4c-1jammy [812 kB] 2024-09-04T17:43:01.183 INFO:teuthology.orchestra.run.smithi092.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.3.0-4703-g3e4a7f4c-1jammy [160 kB] 2024-09-04T17:43:01.185 INFO:teuthology.orchestra.run.smithi092.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.3.0-4703-g3e4a7f4c-1jammy [61.4 MB] 2024-09-04T17:43:03.279 INFO:teuthology.orchestra.run.smithi190.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.3.0-4703-g3e4a7f4c-1jammy [8,625 kB] 2024-09-04T17:43:03.387 INFO:teuthology.orchestra.run.smithi190.stdout:Get:114 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.3.0-4703-g3e4a7f4c-1jammy [14.2 kB] 2024-09-04T17:43:03.387 INFO:teuthology.orchestra.run.smithi190.stdout:Get:115 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.3.0-4703-g3e4a7f4c-1jammy [55.7 MB] 2024-09-04T17:43:03.490 INFO:teuthology.orchestra.run.smithi092.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.3.0-4703-g3e4a7f4c-1jammy [8,625 kB] 2024-09-04T17:43:03.698 INFO:teuthology.orchestra.run.smithi092.stdout:Get:114 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.3.0-4703-g3e4a7f4c-1jammy [14.2 kB] 2024-09-04T17:43:03.699 INFO:teuthology.orchestra.run.smithi092.stdout:Get:115 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.3.0-4703-g3e4a7f4c-1jammy [55.7 MB] 2024-09-04T17:43:05.224 INFO:teuthology.orchestra.run.smithi190.stdout:Get:116 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.3.0-4703-g3e4a7f4c-1jammy [130 kB] 2024-09-04T17:43:05.226 INFO:teuthology.orchestra.run.smithi190.stdout:Get:117 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.3.0-4703-g3e4a7f4c-1jammy [38.9 kB] 2024-09-04T17:43:05.228 INFO:teuthology.orchestra.run.smithi190.stdout:Get:118 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.3.0-4703-g3e4a7f4c-1jammy [15.9 MB] 2024-09-04T17:43:05.695 INFO:teuthology.orchestra.run.smithi190.stdout:Get:119 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.3.0-4703-g3e4a7f4c-1jammy [91.0 kB] 2024-09-04T17:43:05.700 INFO:teuthology.orchestra.run.smithi092.stdout:Get:116 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.3.0-4703-g3e4a7f4c-1jammy [130 kB] 2024-09-04T17:43:05.703 INFO:teuthology.orchestra.run.smithi092.stdout:Get:117 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.3.0-4703-g3e4a7f4c-1jammy [38.9 kB] 2024-09-04T17:43:05.766 INFO:teuthology.orchestra.run.smithi092.stdout:Get:118 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.3.0-4703-g3e4a7f4c-1jammy [15.9 MB] 2024-09-04T17:43:06.068 INFO:teuthology.orchestra.run.smithi092.stdout:Get:119 https://4.chacra.ceph.com/r/ceph/wip-dis-testing/3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.3.0-4703-g3e4a7f4c-1jammy [91.0 kB] 2024-09-04T17:43:06.269 INFO:teuthology.orchestra.run.smithi190.stdout:Fetched 251 MB in 8s (32.9 MB/s) 2024-09-04T17:43:06.388 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-04T17:43:06.477 INFO:teuthology.orchestra.run.smithi190.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165988 files and directories currently installed.) 2024-09-04T17:43:06.482 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-04T17:43:06.562 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-04T17:43:06.634 INFO:teuthology.orchestra.run.smithi092.stdout:Fetched 251 MB in 8s (31.2 MB/s) 2024-09-04T17:43:06.794 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-04T17:43:06.873 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-04T17:43:06.894 INFO:teuthology.orchestra.run.smithi092.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165988 files and directories currently installed.) 2024-09-04T17:43:06.899 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-04T17:43:06.900 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-04T17:43:06.940 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-04T17:43:06.985 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-04T17:43:07.310 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-04T17:43:07.313 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-04T17:43:07.336 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-04T17:43:07.340 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-04T17:43:07.377 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-04T17:43:07.380 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-04T17:43:07.717 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-04T17:43:07.743 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-04T17:43:07.779 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-04T17:43:07.782 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-04T17:43:07.806 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-04T17:43:07.855 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-04T17:43:08.228 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-04T17:43:08.241 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-04T17:43:08.254 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-04T17:43:08.267 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-04T17:43:08.302 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-04T17:43:08.307 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-04T17:43:08.610 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-04T17:43:08.637 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-04T17:43:08.668 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-04T17:43:08.714 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-04T17:43:08.740 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-04T17:43:08.781 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-04T17:43:09.030 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-04T17:43:09.056 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-04T17:43:09.084 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-04T17:43:09.096 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-04T17:43:09.110 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-04T17:43:09.142 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-04T17:43:09.424 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package librados2. 2024-09-04T17:43:09.450 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../007-librados2_19.3.0-4703-g3e4a7f4c-1jammy_amd64.deb ... 2024-09-04T17:43:09.491 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking librados2 (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:43:09.495 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-04T17:43:09.522 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-04T17:43:09.562 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-04T17:43:09.890 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package librados2. 2024-09-04T17:43:09.916 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../007-librados2_19.3.0-4703-g3e4a7f4c-1jammy_amd64.deb ... 2024-09-04T17:43:09.943 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libnbd0. 2024-09-04T17:43:09.956 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking librados2 (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:43:09.958 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../008-libnbd0_1.10.5-1_amd64.deb ... 2024-09-04T17:43:09.993 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libnbd0 (1.10.5-1) ... 2024-09-04T17:43:10.279 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package librbd1. 2024-09-04T17:43:10.306 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../009-librbd1_19.3.0-4703-g3e4a7f4c-1jammy_amd64.deb ... 2024-09-04T17:43:10.379 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking librbd1 (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:43:10.434 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libnbd0. 2024-09-04T17:43:10.461 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../008-libnbd0_1.10.5-1_amd64.deb ... 2024-09-04T17:43:10.501 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libnbd0 (1.10.5-1) ... 2024-09-04T17:43:10.824 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libcephfs2. 2024-09-04T17:43:10.837 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package librbd1. 2024-09-04T17:43:10.851 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../010-libcephfs2_19.3.0-4703-g3e4a7f4c-1jammy_amd64.deb ... 2024-09-04T17:43:10.864 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../009-librbd1_19.3.0-4703-g3e4a7f4c-1jammy_amd64.deb ... 2024-09-04T17:43:10.890 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libcephfs2 (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:43:10.904 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking librbd1 (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:43:11.176 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-rados. 2024-09-04T17:43:11.192 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../011-python3-rados_19.3.0-4703-g3e4a7f4c-1jammy_amd64.deb ... 2024-09-04T17:43:11.234 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-rados (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:43:11.340 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libcephfs2. 2024-09-04T17:43:11.366 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../010-libcephfs2_19.3.0-4703-g3e4a7f4c-1jammy_amd64.deb ... 2024-09-04T17:43:11.406 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libcephfs2 (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:43:11.562 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-04T17:43:11.588 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../012-python3-ceph-argparse_19.3.0-4703-g3e4a7f4c-1jammy_all.deb ... 2024-09-04T17:43:11.629 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-ceph-argparse (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:43:11.692 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-rados. 2024-09-04T17:43:11.707 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../011-python3-rados_19.3.0-4703-g3e4a7f4c-1jammy_amd64.deb ... 2024-09-04T17:43:11.750 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-rados (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:43:11.864 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-cephfs. 2024-09-04T17:43:11.879 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../013-python3-cephfs_19.3.0-4703-g3e4a7f4c-1jammy_amd64.deb ... 2024-09-04T17:43:11.931 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-cephfs (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:43:12.045 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-04T17:43:12.060 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../012-python3-ceph-argparse_19.3.0-4703-g3e4a7f4c-1jammy_all.deb ... 2024-09-04T17:43:12.103 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-ceph-argparse (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:43:12.251 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-04T17:43:12.276 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../014-python3-ceph-common_19.3.0-4703-g3e4a7f4c-1jammy_all.deb ... 2024-09-04T17:43:12.317 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-ceph-common (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:43:12.339 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-cephfs. 2024-09-04T17:43:12.354 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../013-python3-cephfs_19.3.0-4703-g3e4a7f4c-1jammy_amd64.deb ... 2024-09-04T17:43:12.397 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-cephfs (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:43:12.653 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-04T17:43:12.680 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../015-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-04T17:43:12.700 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-04T17:43:12.720 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-04T17:43:12.726 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../014-python3-ceph-common_19.3.0-4703-g3e4a7f4c-1jammy_all.deb ... 2024-09-04T17:43:12.766 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-ceph-common (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:43:13.022 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-prettytable. 2024-09-04T17:43:13.049 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../016-python3-prettytable_2.5.0-2_all.deb ... 2024-09-04T17:43:13.089 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-04T17:43:13.103 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-04T17:43:13.129 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../015-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-04T17:43:13.169 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-04T17:43:13.325 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-rbd. 2024-09-04T17:43:13.340 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../017-python3-rbd_19.3.0-4703-g3e4a7f4c-1jammy_amd64.deb ... 2024-09-04T17:43:13.375 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-rbd (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:43:13.464 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-prettytable. 2024-09-04T17:43:13.490 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../016-python3-prettytable_2.5.0-2_all.deb ... 2024-09-04T17:43:13.522 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-04T17:43:13.728 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-04T17:43:13.754 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../018-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-04T17:43:13.783 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-rbd. 2024-09-04T17:43:13.794 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-04T17:43:13.809 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../017-python3-rbd_19.3.0-4703-g3e4a7f4c-1jammy_amd64.deb ... 2024-09-04T17:43:13.850 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-rbd (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:43:14.072 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-04T17:43:14.087 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../019-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-04T17:43:14.130 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-04T17:43:14.203 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-04T17:43:14.229 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../018-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-04T17:43:14.269 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-04T17:43:14.450 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-04T17:43:14.476 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../020-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-04T17:43:14.516 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-04T17:43:14.564 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-04T17:43:14.590 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../019-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-04T17:43:14.639 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-04T17:43:14.844 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package lua5.1. 2024-09-04T17:43:14.871 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../021-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-04T17:43:14.902 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-04T17:43:14.991 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-04T17:43:15.006 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../020-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-04T17:43:15.050 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-04T17:43:15.222 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package lua-any. 2024-09-04T17:43:15.237 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../022-lua-any_27ubuntu1_all.deb ... 2024-09-04T17:43:15.280 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-04T17:43:15.370 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package lua5.1. 2024-09-04T17:43:15.396 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../021-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-04T17:43:15.445 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-04T17:43:15.533 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package zip. 2024-09-04T17:43:15.560 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../023-zip_3.0-12build2_amd64.deb ... 2024-09-04T17:43:15.599 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking zip (3.0-12build2) ... 2024-09-04T17:43:15.781 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package lua-any. 2024-09-04T17:43:15.808 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../022-lua-any_27ubuntu1_all.deb ... 2024-09-04T17:43:15.856 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-04T17:43:15.928 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package unzip. 2024-09-04T17:43:15.954 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../024-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-04T17:43:15.994 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-04T17:43:16.142 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package zip. 2024-09-04T17:43:16.168 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../023-zip_3.0-12build2_amd64.deb ... 2024-09-04T17:43:16.209 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking zip (3.0-12build2) ... 2024-09-04T17:43:16.339 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package luarocks. 2024-09-04T17:43:16.366 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../025-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-04T17:43:16.397 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-04T17:43:16.554 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package unzip. 2024-09-04T17:43:16.580 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../024-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-04T17:43:16.622 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-04T17:43:16.780 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package librgw2. 2024-09-04T17:43:16.807 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../026-librgw2_19.3.0-4703-g3e4a7f4c-1jammy_amd64.deb ... 2024-09-04T17:43:16.846 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking librgw2 (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:43:17.466 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-rgw. 2024-09-04T17:43:17.492 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../027-python3-rgw_19.3.0-4703-g3e4a7f4c-1jammy_amd64.deb ... 2024-09-04T17:43:17.532 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-rgw (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:43:17.574 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package luarocks. 2024-09-04T17:43:17.601 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../025-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-04T17:43:17.653 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-04T17:43:17.877 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-04T17:43:17.903 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../028-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-04T17:43:17.944 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-04T17:43:18.016 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package librgw2. 2024-09-04T17:43:18.042 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../026-librgw2_19.3.0-4703-g3e4a7f4c-1jammy_amd64.deb ... 2024-09-04T17:43:18.082 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking librgw2 (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:43:18.196 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libradosstriper1. 2024-09-04T17:43:18.222 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../029-libradosstriper1_19.3.0-4703-g3e4a7f4c-1jammy_amd64.deb ... 2024-09-04T17:43:18.263 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libradosstriper1 (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:43:18.676 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-rgw. 2024-09-04T17:43:18.699 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-common. 2024-09-04T17:43:18.702 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../027-python3-rgw_19.3.0-4703-g3e4a7f4c-1jammy_amd64.deb ... 2024-09-04T17:43:18.725 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../030-ceph-common_19.3.0-4703-g3e4a7f4c-1jammy_amd64.deb ... 2024-09-04T17:43:18.743 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-rgw (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:43:18.766 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-common (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:43:19.029 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-04T17:43:19.055 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../028-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-04T17:43:19.087 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-04T17:43:19.424 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libradosstriper1. 2024-09-04T17:43:19.450 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../029-libradosstriper1_19.3.0-4703-g3e4a7f4c-1jammy_amd64.deb ... 2024-09-04T17:43:19.490 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libradosstriper1 (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:43:19.919 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-common. 2024-09-04T17:43:19.945 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../030-ceph-common_19.3.0-4703-g3e4a7f4c-1jammy_amd64.deb ... 2024-09-04T17:43:19.985 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-common (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:43:20.254 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-base. 2024-09-04T17:43:20.280 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../031-ceph-base_19.3.0-4703-g3e4a7f4c-1jammy_amd64.deb ... 2024-09-04T17:43:20.355 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-base (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:43:20.919 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-04T17:43:20.945 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../032-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-04T17:43:20.985 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-04T17:43:21.288 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-cheroot. 2024-09-04T17:43:21.314 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../033-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-04T17:43:21.354 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-04T17:43:21.465 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-base. 2024-09-04T17:43:21.492 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../031-ceph-base_19.3.0-4703-g3e4a7f4c-1jammy_amd64.deb ... 2024-09-04T17:43:21.549 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-base (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:43:21.666 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-04T17:43:21.692 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../034-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-04T17:43:21.732 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-04T17:43:22.018 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-04T17:43:22.045 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../035-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-04T17:43:22.085 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-04T17:43:22.203 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-04T17:43:22.230 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../032-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-04T17:43:22.270 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-04T17:43:22.488 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-04T17:43:22.514 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../036-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-04T17:43:22.554 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-04T17:43:22.556 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-cheroot. 2024-09-04T17:43:22.583 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../033-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-04T17:43:22.622 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-04T17:43:22.857 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-tempora. 2024-09-04T17:43:22.883 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../037-python3-tempora_4.1.2-1_all.deb ... 2024-09-04T17:43:22.923 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-04T17:43:22.950 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-04T17:43:22.977 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../034-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-04T17:43:23.017 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-04T17:43:23.235 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-portend. 2024-09-04T17:43:23.261 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../038-python3-portend_3.0.0-1_all.deb ... 2024-09-04T17:43:23.301 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-04T17:43:23.395 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-04T17:43:23.422 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../035-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-04T17:43:23.470 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-04T17:43:23.579 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-04T17:43:23.605 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../039-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-04T17:43:23.671 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-04T17:43:23.781 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-04T17:43:23.808 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../036-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-04T17:43:23.848 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-04T17:43:23.990 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-04T17:43:24.016 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../040-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-04T17:43:24.057 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-04T17:43:24.151 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-tempora. 2024-09-04T17:43:24.178 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../037-python3-tempora_4.1.2-1_all.deb ... 2024-09-04T17:43:24.217 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-04T17:43:24.419 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-natsort. 2024-09-04T17:43:24.445 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../041-python3-natsort_8.0.2-1_all.deb ... 2024-09-04T17:43:24.485 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-04T17:43:24.571 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-portend. 2024-09-04T17:43:24.597 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../038-python3-portend_3.0.0-1_all.deb ... 2024-09-04T17:43:24.645 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-04T17:43:24.796 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-logutils. 2024-09-04T17:43:24.822 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../042-python3-logutils_0.3.3-8_all.deb ... 2024-09-04T17:43:24.863 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-04T17:43:25.007 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-04T17:43:25.033 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../039-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-04T17:43:25.073 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-04T17:43:25.132 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-04T17:43:25.158 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../043-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-04T17:43:25.207 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-04T17:43:25.376 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-04T17:43:25.403 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../040-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-04T17:43:25.443 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-04T17:43:25.477 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-mako. 2024-09-04T17:43:25.503 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../044-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-04T17:43:25.543 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-04T17:43:25.805 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-natsort. 2024-09-04T17:43:25.832 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../041-python3-natsort_8.0.2-1_all.deb ... 2024-09-04T17:43:25.871 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-04T17:43:25.872 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-04T17:43:25.898 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../045-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-04T17:43:25.938 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-04T17:43:26.166 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-logutils. 2024-09-04T17:43:26.193 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../042-python3-logutils_0.3.3-8_all.deb ... 2024-09-04T17:43:26.233 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-04T17:43:26.249 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-04T17:43:26.275 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../046-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-04T17:43:26.315 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-04T17:43:26.477 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-04T17:43:26.503 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../043-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-04T17:43:26.544 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-04T17:43:26.593 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-webob. 2024-09-04T17:43:26.620 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../047-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-04T17:43:26.660 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-04T17:43:26.855 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-mako. 2024-09-04T17:43:26.882 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../044-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-04T17:43:26.922 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-04T17:43:27.005 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-waitress. 2024-09-04T17:43:27.032 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../048-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-04T17:43:27.399 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-04T17:43:27.458 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-04T17:43:27.485 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../045-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-04T17:43:27.525 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-04T17:43:27.673 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-tempita. 2024-09-04T17:43:27.690 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../049-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-04T17:43:27.732 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-04T17:43:27.803 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-04T17:43:27.827 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../046-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-04T17:43:27.861 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-04T17:43:28.043 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-paste. 2024-09-04T17:43:28.070 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../050-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-04T17:43:28.110 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-04T17:43:28.147 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-webob. 2024-09-04T17:43:28.174 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../047-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-04T17:43:28.213 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-04T17:43:28.438 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-04T17:43:28.464 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../051-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-04T17:43:28.504 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-04T17:43:28.558 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-waitress. 2024-09-04T17:43:28.585 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../048-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-04T17:43:28.628 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-04T17:43:28.790 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-04T17:43:28.816 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../052-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-04T17:43:28.857 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-04T17:43:28.936 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-tempita. 2024-09-04T17:43:28.963 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../049-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-04T17:43:29.004 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-04T17:43:29.168 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-webtest. 2024-09-04T17:43:29.194 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../053-python3-webtest_2.0.35-1_all.deb ... 2024-09-04T17:43:29.234 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-04T17:43:29.314 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-paste. 2024-09-04T17:43:29.341 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../050-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-04T17:43:29.381 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-04T17:43:29.537 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-pecan. 2024-09-04T17:43:29.564 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../054-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-04T17:43:29.604 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-04T17:43:29.700 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-04T17:43:29.727 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../051-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-04T17:43:29.767 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-04T17:43:29.925 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-certifi. 2024-09-04T17:43:29.952 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../055-python3-certifi_2020.6.20-1_all.deb ... 2024-09-04T17:43:29.991 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-04T17:43:30.053 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-04T17:43:30.080 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../052-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-04T17:43:30.128 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-04T17:43:30.320 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-idna. 2024-09-04T17:43:30.346 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../056-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-04T17:43:30.381 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-webtest. 2024-09-04T17:43:30.386 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-04T17:43:30.408 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../053-python3-webtest_2.0.35-1_all.deb ... 2024-09-04T17:43:30.448 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-04T17:43:30.706 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-urllib3. 2024-09-04T17:43:30.732 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../057-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-04T17:43:30.772 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-04T17:43:30.776 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-pecan. 2024-09-04T17:43:30.802 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../054-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-04T17:43:30.842 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-04T17:43:31.092 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-requests. 2024-09-04T17:43:31.118 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../058-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-04T17:43:31.158 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-04T17:43:31.197 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-certifi. 2024-09-04T17:43:31.226 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../055-python3-certifi_2020.6.20-1_all.deb ... 2024-09-04T17:43:31.706 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-04T17:43:32.345 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-04T17:43:32.372 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../059-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-04T17:43:32.411 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-04T17:43:32.542 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-idna. 2024-09-04T17:43:32.569 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../056-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-04T17:43:32.608 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-04T17:43:32.706 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-04T17:43:32.733 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../060-ceph-mgr-modules-core_19.3.0-4703-g3e4a7f4c-1jammy_all.deb ... 2024-09-04T17:43:32.772 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-mgr-modules-core (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:43:32.887 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-urllib3. 2024-09-04T17:43:32.914 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../057-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-04T17:43:32.945 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-04T17:43:33.094 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-04T17:43:33.121 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../061-libsqlite3-mod-ceph_19.3.0-4703-g3e4a7f4c-1jammy_amd64.deb ... 2024-09-04T17:43:33.152 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libsqlite3-mod-ceph (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:43:33.223 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-requests. 2024-09-04T17:43:33.250 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../058-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-04T17:43:33.315 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-04T17:43:33.463 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-04T17:43:33.490 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../062-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-04T17:43:33.530 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-04T17:43:33.601 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-04T17:43:33.628 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../059-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-04T17:43:33.667 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-04T17:43:33.866 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-openssl. 2024-09-04T17:43:33.892 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../063-python3-openssl_21.0.0-1_all.deb ... 2024-09-04T17:43:33.932 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-04T17:43:33.954 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-04T17:43:33.981 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../060-ceph-mgr-modules-core_19.3.0-4703-g3e4a7f4c-1jammy_all.deb ... 2024-09-04T17:43:34.020 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-mgr-modules-core (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:43:34.261 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-mgr. 2024-09-04T17:43:34.287 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../064-ceph-mgr_19.3.0-4703-g3e4a7f4c-1jammy_amd64.deb ... 2024-09-04T17:43:34.327 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-mgr (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:43:34.401 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-04T17:43:34.428 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../061-libsqlite3-mod-ceph_19.3.0-4703-g3e4a7f4c-1jammy_amd64.deb ... 2024-09-04T17:43:34.467 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libsqlite3-mod-ceph (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:43:34.663 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-mon. 2024-09-04T17:43:34.690 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../065-ceph-mon_19.3.0-4703-g3e4a7f4c-1jammy_amd64.deb ... 2024-09-04T17:43:34.730 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-mon (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:43:34.778 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-04T17:43:34.805 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../062-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-04T17:43:34.862 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-04T17:43:35.223 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-openssl. 2024-09-04T17:43:35.250 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../063-python3-openssl_21.0.0-1_all.deb ... 2024-09-04T17:43:35.298 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-04T17:43:35.341 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-osd. 2024-09-04T17:43:35.368 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../066-ceph-osd_19.3.0-4703-g3e4a7f4c-1jammy_amd64.deb ... 2024-09-04T17:43:35.408 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-osd (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:43:35.618 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-mgr. 2024-09-04T17:43:35.645 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../064-ceph-mgr_19.3.0-4703-g3e4a7f4c-1jammy_amd64.deb ... 2024-09-04T17:43:35.676 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-mgr (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:43:36.046 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-mon. 2024-09-04T17:43:36.073 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../065-ceph-mon_19.3.0-4703-g3e4a7f4c-1jammy_amd64.deb ... 2024-09-04T17:43:36.145 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-mon (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:43:36.453 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph. 2024-09-04T17:43:36.480 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../067-ceph_19.3.0-4703-g3e4a7f4c-1jammy_amd64.deb ... 2024-09-04T17:43:36.519 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:43:36.792 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-fuse. 2024-09-04T17:43:36.815 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../068-ceph-fuse_19.3.0-4703-g3e4a7f4c-1jammy_amd64.deb ... 2024-09-04T17:43:36.847 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-fuse (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:43:36.866 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-osd. 2024-09-04T17:43:36.893 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../066-ceph-osd_19.3.0-4703-g3e4a7f4c-1jammy_amd64.deb ... 2024-09-04T17:43:36.932 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-osd (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:43:37.275 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-mds. 2024-09-04T17:43:37.301 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../069-ceph-mds_19.3.0-4703-g3e4a7f4c-1jammy_amd64.deb ... 2024-09-04T17:43:37.341 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-mds (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:43:37.753 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package cephadm. 2024-09-04T17:43:37.774 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../070-cephadm_19.3.0-4703-g3e4a7f4c-1jammy_amd64.deb ... 2024-09-04T17:43:37.811 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking cephadm (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:43:37.977 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph. 2024-09-04T17:43:37.992 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../067-ceph_19.3.0-4703-g3e4a7f4c-1jammy_amd64.deb ... 2024-09-04T17:43:38.035 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:43:38.130 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-04T17:43:38.157 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../071-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-04T17:43:38.196 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-04T17:43:38.355 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-fuse. 2024-09-04T17:43:38.381 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../068-ceph-fuse_19.3.0-4703-g3e4a7f4c-1jammy_amd64.deb ... 2024-09-04T17:43:38.421 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-fuse (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:43:38.508 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-04T17:43:38.534 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../072-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-04T17:43:38.574 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-04T17:43:38.791 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-mds. 2024-09-04T17:43:38.818 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../069-ceph-mds_19.3.0-4703-g3e4a7f4c-1jammy_amd64.deb ... 2024-09-04T17:43:38.857 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-mds (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:43:39.277 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package cephadm. 2024-09-04T17:43:39.304 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../070-cephadm_19.3.0-4703-g3e4a7f4c-1jammy_amd64.deb ... 2024-09-04T17:43:39.344 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking cephadm (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:43:39.612 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-babel. 2024-09-04T17:43:39.640 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../073-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-04T17:43:39.663 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-04T17:43:39.678 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-04T17:43:39.690 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../071-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-04T17:43:39.738 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-04T17:43:40.015 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-jinja2. 2024-09-04T17:43:40.041 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-04T17:43:40.042 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../074-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-04T17:43:40.057 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../072-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-04T17:43:40.081 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-04T17:43:40.099 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-04T17:43:40.443 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-04T17:43:40.469 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../075-ceph-mgr-cephadm_19.3.0-4703-g3e4a7f4c-1jammy_all.deb ... 2024-09-04T17:43:40.518 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-mgr-cephadm (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:43:40.863 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-04T17:43:40.889 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../076-python3-repoze.lru_0.7-2_all.deb ... 2024-09-04T17:43:40.929 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-04T17:43:41.080 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-babel. 2024-09-04T17:43:41.107 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../073-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-04T17:43:41.146 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-04T17:43:41.223 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-routes. 2024-09-04T17:43:41.250 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../077-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-04T17:43:41.307 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-04T17:43:41.457 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-jinja2. 2024-09-04T17:43:41.484 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../074-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-04T17:43:41.523 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-04T17:43:41.651 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-04T17:43:41.678 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../078-ceph-mgr-dashboard_19.3.0-4703-g3e4a7f4c-1jammy_all.deb ... 2024-09-04T17:43:41.718 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-mgr-dashboard (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:43:41.902 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-04T17:43:41.929 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../075-ceph-mgr-cephadm_19.3.0-4703-g3e4a7f4c-1jammy_all.deb ... 2024-09-04T17:43:41.968 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-mgr-cephadm (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:43:42.297 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-04T17:43:42.324 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../076-python3-repoze.lru_0.7-2_all.deb ... 2024-09-04T17:43:42.363 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-04T17:43:42.858 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-routes. 2024-09-04T17:43:42.885 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../077-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-04T17:43:42.924 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-04T17:43:43.278 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-04T17:43:43.305 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../078-ceph-mgr-dashboard_19.3.0-4703-g3e4a7f4c-1jammy_all.deb ... 2024-09-04T17:43:43.344 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-mgr-dashboard (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:43:44.707 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-04T17:43:44.724 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../079-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-04T17:43:44.764 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-04T17:43:45.267 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-joblib. 2024-09-04T17:43:45.295 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../080-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-04T17:43:45.333 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-04T17:43:45.696 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-04T17:43:45.723 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../081-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-04T17:43:45.762 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-04T17:43:46.073 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-sklearn. 2024-09-04T17:43:46.100 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../082-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-04T17:43:46.108 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-04T17:43:46.138 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../079-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-04T17:43:46.182 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-04T17:43:46.233 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-04T17:43:46.703 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-joblib. 2024-09-04T17:43:46.730 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../080-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-04T17:43:46.769 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-04T17:43:46.959 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-04T17:43:46.987 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../083-ceph-mgr-diskprediction-local_19.3.0-4703-g3e4a7f4c-1jammy_all.deb ... 2024-09-04T17:43:47.017 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-mgr-diskprediction-local (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:43:47.123 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-04T17:43:47.150 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../081-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-04T17:43:47.189 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-04T17:43:47.517 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-sklearn. 2024-09-04T17:43:47.545 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../082-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-04T17:43:47.669 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-04T17:43:47.995 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-cachetools. 2024-09-04T17:43:48.023 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../084-python3-cachetools_5.0.0-1_all.deb ... 2024-09-04T17:43:48.062 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-04T17:43:48.348 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-04T17:43:48.376 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../085-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-04T17:43:48.390 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-04T17:43:48.414 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-04T17:43:48.418 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../083-ceph-mgr-diskprediction-local_19.3.0-4703-g3e4a7f4c-1jammy_all.deb ... 2024-09-04T17:43:48.457 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-mgr-diskprediction-local (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:43:48.734 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-04T17:43:48.750 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../086-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-04T17:43:48.784 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-04T17:43:49.062 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-rsa. 2024-09-04T17:43:49.078 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../087-python3-rsa_4.8-1_all.deb ... 2024-09-04T17:43:49.112 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-04T17:43:49.365 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-google-auth. 2024-09-04T17:43:49.394 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../088-python3-google-auth_1.5.1-3_all.deb ... 2024-09-04T17:43:49.432 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-04T17:43:49.442 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-cachetools. 2024-09-04T17:43:49.469 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../084-python3-cachetools_5.0.0-1_all.deb ... 2024-09-04T17:43:49.508 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-04T17:43:49.718 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-04T17:43:49.734 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../089-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-04T17:43:49.776 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-04T17:43:49.811 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-04T17:43:49.839 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../085-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-04T17:43:49.877 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-04T17:43:50.080 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-websocket. 2024-09-04T17:43:50.093 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../090-python3-websocket_1.2.3-1_all.deb ... 2024-09-04T17:43:50.138 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-04T17:43:50.248 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-04T17:43:50.275 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../086-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-04T17:43:50.314 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-04T17:43:50.474 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-04T17:43:50.502 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../091-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-04T17:43:50.541 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-04T17:43:50.626 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-rsa. 2024-09-04T17:43:50.653 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../087-python3-rsa_4.8-1_all.deb ... 2024-09-04T17:43:50.700 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-04T17:43:51.029 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-google-auth. 2024-09-04T17:43:51.056 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../088-python3-google-auth_1.5.1-3_all.deb ... 2024-09-04T17:43:51.104 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-04T17:43:51.221 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-04T17:43:51.237 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../092-ceph-mgr-k8sevents_19.3.0-4703-g3e4a7f4c-1jammy_all.deb ... 2024-09-04T17:43:51.271 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-mgr-k8sevents (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:43:51.491 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-04T17:43:51.515 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-04T17:43:51.520 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../089-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-04T17:43:51.531 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../093-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-04T17:43:51.565 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-04T17:43:51.566 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-04T17:43:51.885 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-websocket. 2024-09-04T17:43:51.901 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-04T17:43:51.912 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../090-python3-websocket_1.2.3-1_all.deb ... 2024-09-04T17:43:51.929 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../094-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-04T17:43:51.952 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-04T17:43:51.968 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-04T17:43:52.237 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package jq. 2024-09-04T17:43:52.265 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../095-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-04T17:43:52.271 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-04T17:43:52.299 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../091-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-04T17:43:52.303 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-04T17:43:52.338 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-04T17:43:52.539 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package socat. 2024-09-04T17:43:52.555 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../096-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-04T17:43:52.589 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-04T17:43:52.867 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package xmlstarlet. 2024-09-04T17:43:52.883 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../097-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-04T17:43:52.917 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-04T17:43:53.238 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-test. 2024-09-04T17:43:53.255 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../098-ceph-test_19.3.0-4703-g3e4a7f4c-1jammy_amd64.deb ... 2024-09-04T17:43:53.288 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-test (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:43:53.377 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-04T17:43:53.393 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../092-ceph-mgr-k8sevents_19.3.0-4703-g3e4a7f4c-1jammy_all.deb ... 2024-09-04T17:43:53.435 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-mgr-k8sevents (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:43:53.704 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-04T17:43:53.732 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../093-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-04T17:43:53.762 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-04T17:43:54.099 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-04T17:43:54.126 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../094-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-04T17:43:54.165 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-04T17:43:54.418 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package jq. 2024-09-04T17:43:54.446 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../095-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-04T17:43:54.476 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-04T17:43:54.679 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package socat. 2024-09-04T17:43:54.707 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../096-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-04T17:43:54.745 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-04T17:43:55.016 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package xmlstarlet. 2024-09-04T17:43:55.044 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../097-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-04T17:43:55.073 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-04T17:43:55.411 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-test. 2024-09-04T17:43:55.427 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../098-ceph-test_19.3.0-4703-g3e4a7f4c-1jammy_amd64.deb ... 2024-09-04T17:43:55.460 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-test (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:43:55.767 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-volume. 2024-09-04T17:43:55.795 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../099-ceph-volume_19.3.0-4703-g3e4a7f4c-1jammy_all.deb ... 2024-09-04T17:43:55.833 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-volume (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:43:56.129 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-04T17:43:56.146 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../100-libcephfs-dev_19.3.0-4703-g3e4a7f4c-1jammy_amd64.deb ... 2024-09-04T17:43:56.187 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libcephfs-dev (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:43:56.507 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-04T17:43:56.535 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../101-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-04T17:43:56.573 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-04T17:43:56.928 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-04T17:43:56.956 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../102-lua-sec_1.0.2-1_amd64.deb ... 2024-09-04T17:43:56.994 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-04T17:43:57.323 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package nvme-cli. 2024-09-04T17:43:57.351 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../103-nvme-cli_1.16-3ubuntu0.2_amd64.deb ... 2024-09-04T17:43:57.390 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking nvme-cli (1.16-3ubuntu0.2) ... 2024-09-04T17:43:57.803 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package pkg-config. 2024-09-04T17:43:57.831 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../104-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-04T17:43:57.869 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-04T17:43:57.965 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-volume. 2024-09-04T17:43:57.993 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../099-ceph-volume_19.3.0-4703-g3e4a7f4c-1jammy_all.deb ... 2024-09-04T17:43:58.031 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-volume (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:43:58.131 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-04T17:43:58.159 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../105-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-04T17:43:58.197 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-04T17:43:58.352 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-04T17:43:58.368 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../100-libcephfs-dev_19.3.0-4703-g3e4a7f4c-1jammy_amd64.deb ... 2024-09-04T17:43:58.410 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libcephfs-dev (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:43:58.642 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-04T17:43:58.670 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../106-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-04T17:43:58.708 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-04T17:43:58.747 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-04T17:43:58.775 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../101-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-04T17:43:58.813 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-04T17:43:59.011 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-pastescript. 2024-09-04T17:43:59.039 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../107-python3-pastescript_2.0.2-4_all.deb ... 2024-09-04T17:43:59.077 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-04T17:43:59.151 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-04T17:43:59.179 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../102-lua-sec_1.0.2-1_amd64.deb ... 2024-09-04T17:43:59.217 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-04T17:43:59.414 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-pluggy. 2024-09-04T17:43:59.442 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../108-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-04T17:43:59.480 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-04T17:43:59.596 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package nvme-cli. 2024-09-04T17:43:59.624 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../103-nvme-cli_1.16-3ubuntu0.2_amd64.deb ... 2024-09-04T17:43:59.663 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking nvme-cli (1.16-3ubuntu0.2) ... 2024-09-04T17:43:59.750 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-psutil. 2024-09-04T17:43:59.766 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../109-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-04T17:43:59.800 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-04T17:44:00.076 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package pkg-config. 2024-09-04T17:44:00.092 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../104-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-04T17:44:00.111 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-py. 2024-09-04T17:44:00.134 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-04T17:44:00.140 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../110-python3-py_1.10.0-1_all.deb ... 2024-09-04T17:44:00.177 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-04T17:44:00.362 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-04T17:44:00.390 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../105-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-04T17:44:00.428 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-04T17:44:00.513 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-pygments. 2024-09-04T17:44:00.542 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../111-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-04T17:44:00.580 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-04T17:44:00.815 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-04T17:44:00.844 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../106-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-04T17:44:00.873 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-04T17:44:01.035 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-04T17:44:01.063 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../112-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-04T17:44:01.101 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-04T17:44:01.159 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-pastescript. 2024-09-04T17:44:01.187 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../107-python3-pastescript_2.0.2-4_all.deb ... 2024-09-04T17:44:01.226 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-04T17:44:01.414 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-toml. 2024-09-04T17:44:01.442 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../113-python3-toml_0.10.2-1_all.deb ... 2024-09-04T17:44:01.480 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-04T17:44:01.562 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-pluggy. 2024-09-04T17:44:01.590 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../108-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-04T17:44:01.629 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-04T17:44:01.741 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-pytest. 2024-09-04T17:44:01.758 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../114-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-04T17:44:01.791 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-04T17:44:01.949 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-psutil. 2024-09-04T17:44:01.977 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../109-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-04T17:44:02.015 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-04T17:44:02.120 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-simplejson. 2024-09-04T17:44:02.148 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../115-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-04T17:44:02.186 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-04T17:44:02.368 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-py. 2024-09-04T17:44:02.396 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../110-python3-py_1.10.0-1_all.deb ... 2024-09-04T17:44:02.434 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-04T17:44:02.463 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-04T17:44:02.480 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../116-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-04T17:44:02.513 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-04T17:44:02.763 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-pygments. 2024-09-04T17:44:02.791 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../111-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-04T17:44:02.837 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-04T17:44:03.234 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package radosgw. 2024-09-04T17:44:03.263 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../117-radosgw_19.3.0-4703-g3e4a7f4c-1jammy_amd64.deb ... 2024-09-04T17:44:03.283 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-04T17:44:03.300 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking radosgw (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:03.311 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../112-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-04T17:44:03.350 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-04T17:44:03.911 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-toml. 2024-09-04T17:44:03.939 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../113-python3-toml_0.10.2-1_all.deb ... 2024-09-04T17:44:03.977 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-04T17:44:04.236 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package rbd-fuse. 2024-09-04T17:44:04.253 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../118-rbd-fuse_19.3.0-4703-g3e4a7f4c-1jammy_amd64.deb ... 2024-09-04T17:44:04.280 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-pytest. 2024-09-04T17:44:04.294 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking rbd-fuse (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:04.308 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../114-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-04T17:44:04.347 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-04T17:44:04.608 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-04T17:44:04.684 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-simplejson. 2024-09-04T17:44:04.700 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../115-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-04T17:44:04.741 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-04T17:44:04.858 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-04T17:44:04.959 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up nvme-cli (1.16-3ubuntu0.2) ... 2024-09-04T17:44:05.036 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-04T17:44:05.052 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../116-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-04T17:44:05.094 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-04T17:44:05.213 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-04T17:44:05.550 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-04T17:44:05.856 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package radosgw. 2024-09-04T17:44:05.885 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../117-radosgw_19.3.0-4703-g3e4a7f4c-1jammy_amd64.deb ... 2024-09-04T17:44:05.923 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking radosgw (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:05.963 INFO:teuthology.orchestra.run.smithi190.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-04T17:44:06.027 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up cephadm (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:06.527 INFO:teuthology.orchestra.run.smithi190.stdout:Adding system user cephadm....done 2024-09-04T17:44:06.658 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-04T17:44:06.842 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package rbd-fuse. 2024-09-04T17:44:06.859 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../118-rbd-fuse_19.3.0-4703-g3e4a7f4c-1jammy_amd64.deb ... 2024-09-04T17:44:06.900 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking rbd-fuse (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:06.934 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-04T17:44:07.169 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-04T17:44:07.213 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-04T17:44:07.270 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-04T17:44:07.447 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-04T17:44:07.521 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-04T17:44:07.573 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up nvme-cli (1.16-3ubuntu0.2) ... 2024-09-04T17:44:07.780 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-04T17:44:07.836 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-04T17:44:07.906 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-04T17:44:08.185 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-04T17:44:08.225 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-04T17:44:08.593 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-04T17:44:08.596 INFO:teuthology.orchestra.run.smithi092.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-04T17:44:08.666 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up cephadm (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:08.836 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-04T17:44:09.088 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-04T17:44:09.158 INFO:teuthology.orchestra.run.smithi092.stdout:Adding system user cephadm....done 2024-09-04T17:44:09.314 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-04T17:44:09.347 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-04T17:44:09.598 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-04T17:44:09.599 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-04T17:44:09.858 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-04T17:44:09.866 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-04T17:44:10.009 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-04T17:44:10.108 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-ceph-argparse (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:10.260 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-04T17:44:10.368 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-04T17:44:10.469 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libnbd0 (1.10.5-1) ... 2024-09-04T17:44:10.503 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-04T17:44:10.570 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-04T17:44:10.637 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-04T17:44:10.688 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-04T17:44:10.830 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-04T17:44:10.939 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-04T17:44:11.089 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-04T17:44:11.299 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-04T17:44:11.468 INFO:teuthology.orchestra.run.smithi190.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-04T17:44:11.495 INFO:teuthology.orchestra.run.smithi190.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-04T17:44:11.559 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-04T17:44:11.559 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-04T17:44:11.685 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-04T17:44:11.843 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-04T17:44:12.045 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-04T17:44:12.070 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-04T17:44:12.154 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-04T17:44:12.312 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-04T17:44:12.422 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-04T17:44:12.572 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-04T17:44:12.698 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-04T17:44:12.823 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-ceph-argparse (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:12.999 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up zip (3.0-12build2) ... 2024-09-04T17:44:13.090 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-04T17:44:13.125 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-04T17:44:13.225 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libnbd0 (1.10.5-1) ... 2024-09-04T17:44:13.351 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-04T17:44:13.452 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-04T17:44:13.553 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-04T17:44:13.760 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-04T17:44:13.788 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-04T17:44:14.011 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-04T17:44:14.154 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-04T17:44:14.164 INFO:teuthology.orchestra.run.smithi092.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-04T17:44:14.194 INFO:teuthology.orchestra.run.smithi092.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-04T17:44:14.280 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-04T17:44:14.282 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-04T17:44:14.409 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-04T17:44:14.572 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-04T17:44:14.768 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-04T17:44:14.894 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-04T17:44:15.154 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-04T17:44:15.173 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-04T17:44:15.413 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-04T17:44:15.416 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-04T17:44:15.756 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up zip (3.0-12build2) ... 2024-09-04T17:44:15.776 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-04T17:44:15.882 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-04T17:44:16.060 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-04T17:44:16.404 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-04T17:44:16.534 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-04T17:44:16.663 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-04T17:44:16.768 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-04T17:44:16.789 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-ceph-common (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:16.869 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-04T17:44:16.979 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-04T17:44:17.098 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-04T17:44:17.263 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-04T17:44:17.408 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-04T17:44:17.648 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-04T17:44:17.899 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-04T17:44:18.275 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-04T17:44:18.602 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-04T17:44:18.614 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-04T17:44:18.873 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-04T17:44:18.928 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-04T17:44:19.008 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-04T17:44:19.179 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-04T17:44:19.267 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-04T17:44:19.305 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-ceph-common (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:19.393 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-04T17:44:19.519 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-04T17:44:19.631 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-04T17:44:19.804 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-04T17:44:19.949 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-04T17:44:20.063 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-04T17:44:20.314 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-04T17:44:20.440 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-04T17:44:20.767 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-04T17:44:21.018 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-04T17:44:21.113 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-04T17:44:21.268 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-04T17:44:21.448 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-04T17:44:21.528 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-04T17:44:21.566 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-04T17:44:21.779 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-04T17:44:21.808 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-04T17:44:21.918 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-04T17:44:22.019 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-04T17:44:22.055 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-04T17:44:22.181 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-04T17:44:22.287 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-04T17:44:22.457 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-04T17:44:22.538 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-04T17:44:22.583 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-04T17:44:22.789 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-04T17:44:22.884 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-04T17:44:22.907 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-04T17:44:23.177 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-04T17:44:23.191 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-04T17:44:23.434 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-04T17:44:23.436 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-04T17:44:23.570 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-04T17:44:23.685 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-04T17:44:23.696 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-04T17:44:23.786 INFO:teuthology.orchestra.run.smithi190.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-04T17:44:23.928 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-04T17:44:24.098 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-04T17:44:24.212 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-04T17:44:24.224 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-04T17:44:24.511 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-04T17:44:24.649 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-04T17:44:24.656 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-04T17:44:24.897 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-04T17:44:24.932 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-04T17:44:25.019 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-04T17:44:25.061 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-04T17:44:25.295 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-04T17:44:25.343 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-04T17:44:25.421 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-04T17:44:25.636 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-04T17:44:25.747 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-04T17:44:25.879 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-04T17:44:26.005 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-04T17:44:26.048 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-04T17:44:26.130 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-04T17:44:26.220 INFO:teuthology.orchestra.run.smithi092.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-04T17:44:26.324 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-04T17:44:26.425 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-04T17:44:26.465 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-04T17:44:26.591 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-04T17:44:26.701 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-04T17:44:26.976 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-04T17:44:27.046 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-04T17:44:27.244 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-04T17:44:27.305 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-04T17:44:27.354 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-04T17:44:27.605 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-04T17:44:27.707 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-04T17:44:27.731 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-04T17:44:27.984 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-04T17:44:28.048 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-04T17:44:28.311 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-04T17:44:28.341 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-04T17:44:28.446 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up librados2 (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:28.572 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libsqlite3-mod-ceph (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:28.609 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-04T17:44:28.698 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-04T17:44:28.710 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-04T17:44:28.970 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-04T17:44:28.984 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libcephfs2 (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:29.101 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libradosstriper1 (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:29.227 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-04T17:44:29.331 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-04T17:44:29.504 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up librbd1 (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:29.627 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-04T17:44:29.630 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-mgr-modules-core (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:29.756 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-fuse (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:29.968 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-04T17:44:29.968 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-04T17:44:29.992 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-04T17:44:30.277 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-04T17:44:30.465 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libcephfs-dev (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:30.566 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-rados (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:30.613 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-04T17:44:30.667 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up librgw2 (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:30.739 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up librados2 (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:30.818 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-04T17:44:30.840 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libsqlite3-mod-ceph (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:30.941 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-04T17:44:31.268 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libcephfs2 (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:31.403 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libradosstriper1 (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:31.546 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-04T17:44:31.822 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up librbd1 (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:31.940 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-mgr-modules-core (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:32.041 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-fuse (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:32.080 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-rbd (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:32.206 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up rbd-fuse (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:32.237 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-04T17:44:32.237 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-04T17:44:32.332 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-rgw (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:32.433 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-cephfs (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:32.534 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-common (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:32.776 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libcephfs-dev (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:32.877 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-rados (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:33.043 INFO:teuthology.orchestra.run.smithi190.stdout:Adding group ceph....done 2024-09-04T17:44:33.353 INFO:teuthology.orchestra.run.smithi190.stdout:Adding system user ceph....done 2024-09-04T17:44:33.355 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up librgw2 (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:33.481 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-04T17:44:33.505 INFO:teuthology.orchestra.run.smithi190.stdout:Setting system user ceph properties....done 2024-09-04T17:44:33.519 INFO:teuthology.orchestra.run.smithi190.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-04T17:44:33.653 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-04T17:44:33.995 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-04T17:44:34.468 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-test (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:34.594 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up radosgw (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:34.945 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-rbd (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:35.054 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-04T17:44:35.055 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-04T17:44:35.069 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up rbd-fuse (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:35.195 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-rgw (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:35.338 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-cephfs (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:35.464 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-common (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:35.529 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-base (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:35.877 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-04T17:44:35.897 INFO:teuthology.orchestra.run.smithi092.stdout:Adding group ceph....done 2024-09-04T17:44:36.200 INFO:teuthology.orchestra.run.smithi092.stdout:Adding system user ceph....done 2024-09-04T17:44:36.318 INFO:teuthology.orchestra.run.smithi092.stdout:Setting system user ceph properties....done 2024-09-04T17:44:36.335 INFO:teuthology.orchestra.run.smithi092.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-04T17:44:36.356 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-mds (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:36.477 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-04T17:44:36.575 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-04T17:44:36.575 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-04T17:44:36.772 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-04T17:44:37.008 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-mgr (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:37.208 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-04T17:44:37.208 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-04T17:44:37.290 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-test (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:37.533 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up radosgw (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:37.676 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-osd (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:37.947 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-04T17:44:37.947 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-04T17:44:37.968 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-04T17:44:37.968 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-04T17:44:38.426 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-base (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:38.435 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-mgr-k8sevents (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:38.561 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-mgr-diskprediction-local (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:38.721 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-mon (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:38.760 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-04T17:44:38.938 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-04T17:44:38.938 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-04T17:44:39.211 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-mds (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:39.405 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-mgr-cephadm (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:39.437 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-04T17:44:39.437 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-04T17:44:39.531 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:39.632 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-mgr-dashboard (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:39.758 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-volume (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:39.921 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-mgr (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:40.094 INFO:teuthology.orchestra.run.smithi190.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-04T17:44:40.127 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-04T17:44:40.127 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-04T17:44:40.203 INFO:teuthology.orchestra.run.smithi190.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-04T17:44:40.356 INFO:teuthology.orchestra.run.smithi190.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-04T17:44:40.623 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-osd (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:40.889 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-04T17:44:40.889 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-04T17:44:41.307 INFO:teuthology.orchestra.run.smithi190.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-04T17:44:41.349 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-mgr-k8sevents (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:41.475 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-mgr-diskprediction-local (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:41.626 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-mon (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:41.845 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-04T17:44:41.845 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-04T17:44:41.960 INFO:teuthology.orchestra.run.smithi190.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-04T17:44:41.966 DEBUG:teuthology.parallel:result is None 2024-09-04T17:44:42.319 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-mgr-cephadm (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:42.479 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:42.605 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-mgr-dashboard (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:42.747 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-volume (19.3.0-4703-g3e4a7f4c-1jammy) ... 2024-09-04T17:44:43.108 INFO:teuthology.orchestra.run.smithi092.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-04T17:44:43.218 INFO:teuthology.orchestra.run.smithi092.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-04T17:44:43.361 INFO:teuthology.orchestra.run.smithi092.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-04T17:44:44.479 INFO:teuthology.orchestra.run.smithi092.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-04T17:44:45.129 INFO:teuthology.orchestra.run.smithi092.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-04T17:44:45.134 DEBUG:teuthology.parallel:result is None 2024-09-04T17:44:45.134 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab 2024-09-04T17:44:45.280 DEBUG:teuthology.orchestra.run.smithi092:> dpkg-query -W -f '${Version}' ceph 2024-09-04T17:44:45.298 INFO:teuthology.orchestra.run.smithi092.stdout:19.3.0-4703-g3e4a7f4c-1jammy 2024-09-04T17:44:45.299 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4703-g3e4a7f4c-1jammy 2024-09-04T17:44:45.299 INFO:teuthology.task.install:The correct ceph version 19.3.0-4703-g3e4a7f4c-1jammy is installed. 2024-09-04T17:44:45.300 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab 2024-09-04T17:44:45.438 DEBUG:teuthology.orchestra.run.smithi190:> dpkg-query -W -f '${Version}' ceph 2024-09-04T17:44:45.457 INFO:teuthology.orchestra.run.smithi190.stdout:19.3.0-4703-g3e4a7f4c-1jammy 2024-09-04T17:44:45.458 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4703-g3e4a7f4c-1jammy 2024-09-04T17:44:45.458 INFO:teuthology.task.install:The correct ceph version 19.3.0-4703-g3e4a7f4c-1jammy is installed. 2024-09-04T17:44:45.460 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-04T17:44:45.460 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-04T17:44:45.461 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-04T17:44:45.476 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-04T17:44:45.477 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-04T17:44:45.522 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-04T17:44:45.522 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-04T17:44:45.522 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/daemon-helper 2024-09-04T17:44:45.538 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-04T17:44:45.591 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-04T17:44:45.591 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/usr/bin/daemon-helper 2024-09-04T17:44:45.604 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-04T17:44:45.660 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-04T17:44:45.661 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-04T17:44:45.661 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-04T17:44:45.673 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-04T17:44:45.727 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-04T17:44:45.728 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-04T17:44:45.742 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-04T17:44:45.797 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-04T17:44:45.797 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-04T17:44:45.797 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/stdin-killer 2024-09-04T17:44:45.809 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-04T17:44:45.863 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-04T17:44:45.863 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/usr/bin/stdin-killer 2024-09-04T17:44:45.878 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-04T17:44:45.933 INFO:teuthology.run_tasks:Running task ceph... 2024-09-04T17:44:46.019 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-04T17:44:46.019 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 777 /var/log/ceph 2024-09-04T17:44:46.021 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 777 /var/log/ceph 2024-09-04T17:44:46.034 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-04T17:44:46.034 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-04T17:44:46.077 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-04T17:44:46.090 INFO:tasks.ceph:Creating extra log directories... 2024-09-04T17:44:46.090 DEBUG:teuthology.orchestra.run.smithi092:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-04T17:44:46.139 DEBUG:teuthology.orchestra.run.smithi190:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-04T17:44:46.161 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-04T17:44:46.161 INFO:tasks.ceph:config {'conf': {'client': {'rbd qos iops burst': 100, 'rbd qos iops limit': 50, 'rbd qos schedule tick min': 100}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-04T17:44:46.161 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/dis-2024-09-04_17:03:08-rbd-wip-dis-testing-distro-default-smithi/7889281', 'branch': 'wip-dis-testing', 'description': 'rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_fsx_rate_limit}', 'email': 'idryomov@gmail.com', 'first_in_suite': False, 'job_id': '7889281', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'dis-2024-09-04_17:03:08-rbd-wip-dis-testing-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'machine': {'cpus': 1, 'disk': 40, 'ram': 8000}, 'volumes': {'count': 4, 'size': 30}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-dis-testing'}, 'ceph': {'conf': {'client': {'rbd qos iops burst': 100, 'rbd qos iops limit': 50, 'rbd qos schedule tick min': 100}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': '3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'live-migration-nbd', 'sha1': '250c2b7303ac79643ee967bc9c5b14e5a0dbbf52'}}, 'owner': 'scheduled_dis@teuthology', 'priority': 70, 'repo': 'https://github.com/ceph/ceph-ci', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 6930, 'sha1': '3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab', 'sleep_before_teardown': 0, 'subset': '4/100', 'suite': 'rbd', 'suite_branch': 'live-migration-nbd', 'suite_path': '/home/teuthworker/src/github.com_effi-ofer_ceph_250c2b7303ac79643ee967bc9c5b14e5a0dbbf52/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/effi-ofer/ceph', 'suite_sha1': '250c2b7303ac79643ee967bc9c5b14e5a0dbbf52', 'targets': {'smithi092.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEwEsJaiBW8otX9/ecRoIhVM4N79EMJJCpGCoieqhT81V3quwVZkCcIa5CNExYFVSOCc818jHJclc/bvKk3oa+Y=', 'smithi190.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBACsZpD3Vefj1a51cbkMHbuTP2LQnWBs+GbywGhmgZlPy+EqPqpCmDKPoOVh1KQq0JcY3EaJl/KAZ1BqcH3hto0='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'thrashosds': {'timeout': 1200}}, {'rbd_fsx': {'clients': ['client.0'], 'ops': 6000}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-04_17:03:08', 'tube': 'smithi', 'user': 'dis', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-04T17:44:46.162 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-04T17:44:46.193 DEBUG:teuthology.orchestra.run.smithi190:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-04T17:44:46.199 DEBUG:teuthology.orchestra.run.smithi092:> sudo install -d -m0777 -- /var/run/ceph 2024-09-04T17:44:46.241 DEBUG:teuthology.orchestra.run.smithi190:> sudo install -d -m0777 -- /var/run/ceph 2024-09-04T17:44:46.254 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-04T17:44:46.254 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/scratch_devs of=/dev/stdout 2024-09-04T17:44:46.301 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-04T17:44:46.301 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_1 2024-09-04T17:44:46.348 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-04T17:44:46.349 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-04T17:44:46.349 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 876 Links: 1 2024-09-04T17:44:46.349 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-04T17:44:46.349 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-09-04 17:39:18.767087330 +0000 2024-09-04T17:44:46.349 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-09-04 17:39:18.507089652 +0000 2024-09-04T17:44:46.349 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-09-04 17:39:18.507089652 +0000 2024-09-04T17:44:46.349 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: - 2024-09-04T17:44:46.349 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-04T17:44:46.402 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-09-04T17:44:46.402 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-09-04T17:44:46.402 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000457043 s, 1.1 MB/s 2024-09-04T17:44:46.404 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-04T17:44:46.453 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_2 2024-09-04T17:44:46.500 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-04T17:44:46.500 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-04T17:44:46.500 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 886 Links: 1 2024-09-04T17:44:46.500 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-04T17:44:46.501 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-09-04 17:39:19.251083006 +0000 2024-09-04T17:44:46.501 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-09-04 17:39:19.023085042 +0000 2024-09-04T17:44:46.501 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-09-04 17:39:19.023085042 +0000 2024-09-04T17:44:46.501 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: - 2024-09-04T17:44:46.501 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-04T17:44:46.556 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-09-04T17:44:46.556 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-09-04T17:44:46.556 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000340963 s, 1.5 MB/s 2024-09-04T17:44:46.558 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-04T17:44:46.605 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_3 2024-09-04T17:44:46.652 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-04T17:44:46.652 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-04T17:44:46.652 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 908 Links: 1 2024-09-04T17:44:46.652 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-04T17:44:46.653 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-09-04 17:39:19.719078826 +0000 2024-09-04T17:44:46.653 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-09-04 17:39:19.495080827 +0000 2024-09-04T17:44:46.653 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-09-04 17:39:19.495080827 +0000 2024-09-04T17:44:46.653 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: - 2024-09-04T17:44:46.653 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-04T17:44:46.706 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-09-04T17:44:46.706 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-09-04T17:44:46.706 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000399182 s, 1.3 MB/s 2024-09-04T17:44:46.707 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-04T17:44:46.760 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_4 2024-09-04T17:44:46.808 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-04T17:44:46.808 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-04T17:44:46.808 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 920 Links: 1 2024-09-04T17:44:46.808 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-04T17:44:46.808 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-09-04 17:39:20.243074146 +0000 2024-09-04T17:44:46.808 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-09-04 17:39:19.975076538 +0000 2024-09-04T17:44:46.809 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-09-04 17:39:19.975076538 +0000 2024-09-04T17:44:46.809 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: - 2024-09-04T17:44:46.809 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-04T17:44:46.862 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-09-04T17:44:46.862 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-09-04T17:44:46.862 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000345065 s, 1.5 MB/s 2024-09-04T17:44:46.863 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-04T17:44:46.913 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-04T17:44:46.914 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-04T17:44:46.914 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/scratch_devs of=/dev/stdout 2024-09-04T17:44:46.919 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-04T17:44:46.920 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_1 2024-09-04T17:44:46.965 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-04T17:44:46.965 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-04T17:44:46.966 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 867 Links: 1 2024-09-04T17:44:46.966 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-04T17:44:46.966 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-09-04 17:39:18.717154118 +0000 2024-09-04T17:44:46.966 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-09-04 17:39:18.485156190 +0000 2024-09-04T17:44:46.966 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-09-04 17:39:18.485156190 +0000 2024-09-04T17:44:46.966 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: - 2024-09-04T17:44:46.966 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-04T17:44:47.020 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-09-04T17:44:47.020 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-09-04T17:44:47.020 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000449706 s, 1.1 MB/s 2024-09-04T17:44:47.021 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-04T17:44:47.070 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_2 2024-09-04T17:44:47.117 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-04T17:44:47.117 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-04T17:44:47.117 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 881 Links: 1 2024-09-04T17:44:47.117 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-04T17:44:47.117 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-09-04 17:39:19.189149902 +0000 2024-09-04T17:44:47.117 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-09-04 17:39:18.949152046 +0000 2024-09-04T17:44:47.117 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-09-04 17:39:18.949152046 +0000 2024-09-04T17:44:47.117 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: - 2024-09-04T17:44:47.118 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-04T17:44:47.172 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-09-04T17:44:47.172 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-09-04T17:44:47.172 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000340156 s, 1.5 MB/s 2024-09-04T17:44:47.173 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-04T17:44:47.222 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_3 2024-09-04T17:44:47.269 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-04T17:44:47.269 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-04T17:44:47.269 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 905 Links: 1 2024-09-04T17:44:47.270 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-04T17:44:47.270 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-09-04 17:39:19.665145650 +0000 2024-09-04T17:44:47.270 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-09-04 17:39:19.417147866 +0000 2024-09-04T17:44:47.270 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-09-04 17:39:19.417147866 +0000 2024-09-04T17:44:47.270 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: - 2024-09-04T17:44:47.270 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-04T17:44:47.323 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-09-04T17:44:47.323 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-09-04T17:44:47.324 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000431426 s, 1.2 MB/s 2024-09-04T17:44:47.325 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-04T17:44:47.374 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_4 2024-09-04T17:44:47.421 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-04T17:44:47.421 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-04T17:44:47.421 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 923 Links: 1 2024-09-04T17:44:47.421 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-04T17:44:47.421 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-09-04 17:39:20.137141434 +0000 2024-09-04T17:44:47.421 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-09-04 17:39:19.893143613 +0000 2024-09-04T17:44:47.421 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-09-04 17:39:19.893143613 +0000 2024-09-04T17:44:47.421 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: - 2024-09-04T17:44:47.421 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-04T17:44:47.476 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-09-04T17:44:47.476 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-09-04T17:44:47.476 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000387773 s, 1.3 MB/s 2024-09-04T17:44:47.477 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-04T17:44:47.526 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-04T17:44:47.527 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi092.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi190.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-04T17:44:47.527 INFO:tasks.ceph:Generating config... 2024-09-04T17:44:47.528 INFO:tasks.ceph:[client] rbd qos iops burst = 100 2024-09-04T17:44:47.528 INFO:tasks.ceph:[client] rbd qos iops limit = 50 2024-09-04T17:44:47.528 INFO:tasks.ceph:[client] rbd qos schedule tick min = 100 2024-09-04T17:44:47.528 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-04T17:44:47.528 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-09-04T17:44:47.528 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-04T17:44:47.528 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-04T17:44:47.529 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-04T17:44:47.529 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-04T17:44:47.529 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-04T17:44:47.529 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-04T17:44:47.529 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-04T17:44:47.529 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-04T17:44:47.529 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-09-04T17:44:47.529 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-04T17:44:47.529 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-04T17:44:47.529 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-04T17:44:47.529 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-04T17:44:47.529 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-04T17:44:47.529 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-04T17:44:47.530 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-04T17:44:47.530 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-04T17:44:47.530 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-04T17:44:47.530 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-04T17:44:47.530 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-04T17:44:47.530 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-04T17:44:47.530 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-04T17:44:47.530 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-04T17:44:47.530 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-09-04T17:44:47.530 INFO:tasks.ceph:Setting up mon.a... 2024-09-04T17:44:47.531 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-04T17:44:47.565 INFO:teuthology.orchestra.run.smithi092.stdout:creating /etc/ceph/ceph.keyring 2024-09-04T17:44:47.582 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-04T17:44:47.658 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-04T17:44:47.711 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.92'), ('mon.c', '[v2:172.21.15.92:3301,v1:172.21.15.92:6790]'), ('mon.b', '172.21.15.190')] 2024-09-04T17:44:47.711 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.92,[v2:172.21.15.92:3301,v1:172.21.15.92:6790],172.21.15.190', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'rbd qos iops burst': 100, 'rbd qos iops limit': 50, 'rbd qos schedule tick min': 100}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-09-04T17:44:47.712 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-04T17:44:47.712 DEBUG:teuthology.orchestra.run.smithi092:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-04T17:44:47.757 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.92 --addv c '[v2:172.21.15.92:3301,v1:172.21.15.92:6790]' --add b 172.21.15.190 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-04T17:44:47.843 INFO:teuthology.orchestra.run.smithi092.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-04T17:44:47.843 INFO:teuthology.orchestra.run.smithi092.stdout:monmaptool: generated fsid f36b6ce2-8c72-4c99-8607-5c53c38ef67b 2024-09-04T17:44:47.843 INFO:teuthology.orchestra.run.smithi092.stdout:setting min_mon_release = quincy 2024-09-04T17:44:47.843 INFO:teuthology.orchestra.run.smithi092.stdout:epoch 0 2024-09-04T17:44:47.843 INFO:teuthology.orchestra.run.smithi092.stdout:fsid f36b6ce2-8c72-4c99-8607-5c53c38ef67b 2024-09-04T17:44:47.843 INFO:teuthology.orchestra.run.smithi092.stdout:last_changed 2024-09-04T17:44:47.845311+0000 2024-09-04T17:44:47.843 INFO:teuthology.orchestra.run.smithi092.stdout:created 2024-09-04T17:44:47.845311+0000 2024-09-04T17:44:47.844 INFO:teuthology.orchestra.run.smithi092.stdout:min_mon_release 17 (quincy) 2024-09-04T17:44:47.844 INFO:teuthology.orchestra.run.smithi092.stdout:election_strategy: 1 2024-09-04T17:44:47.844 INFO:teuthology.orchestra.run.smithi092.stdout:0: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.a 2024-09-04T17:44:47.844 INFO:teuthology.orchestra.run.smithi092.stdout:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.b 2024-09-04T17:44:47.844 INFO:teuthology.orchestra.run.smithi092.stdout:2: [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] mon.c 2024-09-04T17:44:47.844 INFO:teuthology.orchestra.run.smithi092.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-04T17:44:47.845 DEBUG:teuthology.orchestra.run.smithi092:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-04T17:44:47.892 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID f36b6ce2-8c72-4c99-8607-5c53c38ef67b... 2024-09-04T17:44:47.893 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-04T17:44:47.937 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-04T17:44:47.966 INFO:teuthology.orchestra.run.smithi092.stdout:[global] 2024-09-04T17:44:47.966 INFO:teuthology.orchestra.run.smithi092.stdout: chdir = "" 2024-09-04T17:44:47.966 INFO:teuthology.orchestra.run.smithi092.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-04T17:44:47.966 INFO:teuthology.orchestra.run.smithi092.stdout: auth supported = cephx 2024-09-04T17:44:47.967 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:44:47.967 INFO:teuthology.orchestra.run.smithi092.stdout: filestore xattr use omap = true 2024-09-04T17:44:47.967 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:44:47.967 INFO:teuthology.orchestra.run.smithi092.stdout: mon clock drift allowed = 1.000 2024-09-04T17:44:47.967 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:44:47.967 INFO:teuthology.orchestra.run.smithi092.stdout: osd crush chooseleaf type = 0 2024-09-04T17:44:47.967 INFO:teuthology.orchestra.run.smithi092.stdout: auth debug = true 2024-09-04T17:44:47.967 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:44:47.968 INFO:teuthology.orchestra.run.smithi092.stdout: ms die on old message = true 2024-09-04T17:44:47.968 INFO:teuthology.orchestra.run.smithi092.stdout: ms die on bug = true 2024-09-04T17:44:47.968 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:44:47.968 INFO:teuthology.orchestra.run.smithi092.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-04T17:44:47.968 INFO:teuthology.orchestra.run.smithi092.stdout: mon pg warn max object skew = 0 2024-09-04T17:44:47.968 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:44:47.968 INFO:teuthology.orchestra.run.smithi092.stdout: # disable pg_autoscaler by default for new pools 2024-09-04T17:44:47.968 INFO:teuthology.orchestra.run.smithi092.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-04T17:44:47.969 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:44:47.969 INFO:teuthology.orchestra.run.smithi092.stdout: osd pool default size = 2 2024-09-04T17:44:47.969 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:44:47.969 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd allow primary affinity = true 2024-09-04T17:44:47.969 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd allow pg remap = true 2024-09-04T17:44:47.969 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on legacy crush tunables = false 2024-09-04T17:44:47.969 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on crush straw calc version zero = false 2024-09-04T17:44:47.969 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on no sortbitwise = false 2024-09-04T17:44:47.969 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on osd down out interval zero = false 2024-09-04T17:44:47.970 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on too few osds = false 2024-09-04T17:44:47.970 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-04T17:44:47.970 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-04T17:44:47.970 INFO:teuthology.orchestra.run.smithi092.stdout: mon_allow_pool_size_one = true 2024-09-04T17:44:47.970 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:44:47.970 INFO:teuthology.orchestra.run.smithi092.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-04T17:44:47.970 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:44:47.970 INFO:teuthology.orchestra.run.smithi092.stdout: osd default data pool replay window = 5 2024-09-04T17:44:47.970 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:44:47.971 INFO:teuthology.orchestra.run.smithi092.stdout: mon allow pool delete = true 2024-09-04T17:44:47.971 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:44:47.971 INFO:teuthology.orchestra.run.smithi092.stdout: mon cluster log file level = debug 2024-09-04T17:44:47.971 INFO:teuthology.orchestra.run.smithi092.stdout: debug asserts on shutdown = true 2024-09-04T17:44:47.971 INFO:teuthology.orchestra.run.smithi092.stdout: mon health detail to clog = false 2024-09-04T17:44:47.971 INFO:teuthology.orchestra.run.smithi092.stdout: mon host = "172.21.15.92,[v2:172.21.15.92:3301,v1:172.21.15.92:6790],172.21.15.190" 2024-09-04T17:44:47.971 INFO:teuthology.orchestra.run.smithi092.stdout: mon client directed command retry = 5 2024-09-04T17:44:47.971 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on pool no app = False 2024-09-04T17:44:47.971 INFO:teuthology.orchestra.run.smithi092.stdout: ms inject socket failures = 5000 2024-09-04T17:44:47.971 INFO:teuthology.orchestra.run.smithi092.stdout: fsid = f36b6ce2-8c72-4c99-8607-5c53c38ef67b 2024-09-04T17:44:47.971 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:44:47.972 INFO:teuthology.orchestra.run.smithi092.stdout:[osd] 2024-09-04T17:44:47.972 INFO:teuthology.orchestra.run.smithi092.stdout: osd journal size = 100 2024-09-04T17:44:47.972 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:44:47.972 INFO:teuthology.orchestra.run.smithi092.stdout: osd scrub load threshold = 5.0 2024-09-04T17:44:47.972 INFO:teuthology.orchestra.run.smithi092.stdout: osd scrub max interval = 600 2024-09-04T17:44:47.972 INFO:teuthology.orchestra.run.smithi092.stdout: osd mclock profile = high_recovery_ops 2024-09-04T17:44:47.972 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:44:47.972 INFO:teuthology.orchestra.run.smithi092.stdout: osd recover clone overlap = true 2024-09-04T17:44:47.973 INFO:teuthology.orchestra.run.smithi092.stdout: osd recovery max chunk = 1048576 2024-09-04T17:44:47.973 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:44:47.973 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug shutdown = true 2024-09-04T17:44:47.973 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug op order = true 2024-09-04T17:44:47.973 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug verify stray on activate = true 2024-09-04T17:44:47.973 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug trim objects = true 2024-09-04T17:44:47.973 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:44:47.973 INFO:teuthology.orchestra.run.smithi092.stdout: osd open classes on start = true 2024-09-04T17:44:47.973 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug pg log writeout = true 2024-09-04T17:44:47.973 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:44:47.973 INFO:teuthology.orchestra.run.smithi092.stdout: osd deep scrub update digest min age = 30 2024-09-04T17:44:47.974 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:44:47.974 INFO:teuthology.orchestra.run.smithi092.stdout: osd map max advance = 10 2024-09-04T17:44:47.974 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:44:47.974 INFO:teuthology.orchestra.run.smithi092.stdout: journal zero on create = true 2024-09-04T17:44:47.974 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:44:47.974 INFO:teuthology.orchestra.run.smithi092.stdout: filestore ondisk finisher threads = 3 2024-09-04T17:44:47.974 INFO:teuthology.orchestra.run.smithi092.stdout: filestore apply finisher threads = 3 2024-09-04T17:44:47.974 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:44:47.974 INFO:teuthology.orchestra.run.smithi092.stdout: bdev debug aio = true 2024-09-04T17:44:47.975 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug misdirected ops = true 2024-09-04T17:44:47.975 INFO:teuthology.orchestra.run.smithi092.stdout: bdev async discard = True 2024-09-04T17:44:47.975 INFO:teuthology.orchestra.run.smithi092.stdout: bdev enable discard = True 2024-09-04T17:44:47.975 INFO:teuthology.orchestra.run.smithi092.stdout: bluestore allocator = bitmap 2024-09-04T17:44:47.975 INFO:teuthology.orchestra.run.smithi092.stdout: bluestore block size = 96636764160 2024-09-04T17:44:47.975 INFO:teuthology.orchestra.run.smithi092.stdout: bluestore fsck on mount = True 2024-09-04T17:44:47.975 INFO:teuthology.orchestra.run.smithi092.stdout: debug bluefs = 1/20 2024-09-04T17:44:47.975 INFO:teuthology.orchestra.run.smithi092.stdout: debug bluestore = 1/20 2024-09-04T17:44:47.975 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2024-09-04T17:44:47.975 INFO:teuthology.orchestra.run.smithi092.stdout: debug osd = 20 2024-09-04T17:44:47.975 INFO:teuthology.orchestra.run.smithi092.stdout: debug rocksdb = 4/10 2024-09-04T17:44:47.976 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd backfillfull_ratio = 0.85 2024-09-04T17:44:47.976 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd full ratio = 0.9 2024-09-04T17:44:47.976 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd nearfull ratio = 0.8 2024-09-04T17:44:47.976 INFO:teuthology.orchestra.run.smithi092.stdout: osd blocked scrub grace period = 3600 2024-09-04T17:44:47.976 INFO:teuthology.orchestra.run.smithi092.stdout: osd failsafe full ratio = 0.95 2024-09-04T17:44:47.976 INFO:teuthology.orchestra.run.smithi092.stdout: osd max markdown count = 1000 2024-09-04T17:44:47.976 INFO:teuthology.orchestra.run.smithi092.stdout: osd objectstore = bluestore 2024-09-04T17:44:47.977 INFO:teuthology.orchestra.run.smithi092.stdout: osd shutdown pgref assert = True 2024-09-04T17:44:47.977 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:44:47.977 INFO:teuthology.orchestra.run.smithi092.stdout:[mgr] 2024-09-04T17:44:47.977 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2024-09-04T17:44:47.977 INFO:teuthology.orchestra.run.smithi092.stdout: debug mgr = 20 2024-09-04T17:44:47.977 INFO:teuthology.orchestra.run.smithi092.stdout: debug mon = 20 2024-09-04T17:44:47.977 INFO:teuthology.orchestra.run.smithi092.stdout: debug auth = 20 2024-09-04T17:44:47.977 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min pgs per osd = 4 2024-09-04T17:44:47.977 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min bytes per osd = 10 2024-09-04T17:44:47.977 INFO:teuthology.orchestra.run.smithi092.stdout: mgr/telemetry/nag = false 2024-09-04T17:44:47.981 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:44:47.981 INFO:teuthology.orchestra.run.smithi092.stdout:[mon] 2024-09-04T17:44:47.981 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2024-09-04T17:44:47.981 INFO:teuthology.orchestra.run.smithi092.stdout: debug mon = 20 2024-09-04T17:44:47.981 INFO:teuthology.orchestra.run.smithi092.stdout: debug paxos = 20 2024-09-04T17:44:47.981 INFO:teuthology.orchestra.run.smithi092.stdout: debug auth = 20 2024-09-04T17:44:47.981 INFO:teuthology.orchestra.run.smithi092.stdout: mon data avail warn = 5 2024-09-04T17:44:47.981 INFO:teuthology.orchestra.run.smithi092.stdout: mon mgr mkfs grace = 240 2024-09-04T17:44:47.982 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min pgs per osd = 4 2024-09-04T17:44:47.982 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd reporter subtree level = osd 2024-09-04T17:44:47.982 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd prime pg temp = true 2024-09-04T17:44:47.982 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min bytes per osd = 10 2024-09-04T17:44:47.982 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:44:47.982 INFO:teuthology.orchestra.run.smithi092.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-04T17:44:47.982 INFO:teuthology.orchestra.run.smithi092.stdout: auth mon ticket ttl = 660 # 11m 2024-09-04T17:44:47.982 INFO:teuthology.orchestra.run.smithi092.stdout: auth service ticket ttl = 240 # 4m 2024-09-04T17:44:47.982 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:44:47.982 INFO:teuthology.orchestra.run.smithi092.stdout: # don't complain about insecure global_id in the test suite 2024-09-04T17:44:47.983 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-04T17:44:47.983 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-04T17:44:47.983 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:44:47.983 INFO:teuthology.orchestra.run.smithi092.stdout: # 1m isn't quite enough 2024-09-04T17:44:47.983 INFO:teuthology.orchestra.run.smithi092.stdout: mon_down_mkfs_grace = 2m 2024-09-04T17:44:47.983 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:44:47.983 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_filestore_osds = false 2024-09-04T17:44:47.983 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:44:47.984 INFO:teuthology.orchestra.run.smithi092.stdout:[client] 2024-09-04T17:44:47.984 INFO:teuthology.orchestra.run.smithi092.stdout: rgw cache enabled = true 2024-09-04T17:44:47.984 INFO:teuthology.orchestra.run.smithi092.stdout: rgw enable ops log = true 2024-09-04T17:44:47.984 INFO:teuthology.orchestra.run.smithi092.stdout: rgw enable usage log = true 2024-09-04T17:44:47.984 INFO:teuthology.orchestra.run.smithi092.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-04T17:44:47.984 INFO:teuthology.orchestra.run.smithi092.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-04T17:44:47.984 INFO:teuthology.orchestra.run.smithi092.stdout: rbd qos iops burst = 100 2024-09-04T17:44:47.985 INFO:teuthology.orchestra.run.smithi092.stdout: rbd qos iops limit = 50 2024-09-04T17:44:47.985 INFO:teuthology.orchestra.run.smithi092.stdout: rbd qos schedule tick min = 100 2024-09-04T17:44:47.985 INFO:teuthology.orchestra.run.smithi092.stdout:[mon.a] 2024-09-04T17:44:47.985 INFO:teuthology.orchestra.run.smithi092.stdout:[mon.c] 2024-09-04T17:44:47.985 INFO:teuthology.orchestra.run.smithi092.stdout:[mon.b] 2024-09-04T17:44:47.987 INFO:teuthology.orchestra.run.smithi190.stdout:[global] 2024-09-04T17:44:47.987 INFO:teuthology.orchestra.run.smithi190.stdout: chdir = "" 2024-09-04T17:44:47.987 INFO:teuthology.orchestra.run.smithi190.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-04T17:44:47.987 INFO:teuthology.orchestra.run.smithi190.stdout: auth supported = cephx 2024-09-04T17:44:47.987 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-04T17:44:47.987 INFO:teuthology.orchestra.run.smithi190.stdout: filestore xattr use omap = true 2024-09-04T17:44:47.987 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-04T17:44:47.988 INFO:teuthology.orchestra.run.smithi190.stdout: mon clock drift allowed = 1.000 2024-09-04T17:44:47.988 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-04T17:44:47.988 INFO:teuthology.orchestra.run.smithi190.stdout: osd crush chooseleaf type = 0 2024-09-04T17:44:47.988 INFO:teuthology.orchestra.run.smithi190.stdout: auth debug = true 2024-09-04T17:44:47.988 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-04T17:44:47.988 INFO:teuthology.orchestra.run.smithi190.stdout: ms die on old message = true 2024-09-04T17:44:47.988 INFO:teuthology.orchestra.run.smithi190.stdout: ms die on bug = true 2024-09-04T17:44:47.988 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-04T17:44:47.988 INFO:teuthology.orchestra.run.smithi190.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-04T17:44:47.988 INFO:teuthology.orchestra.run.smithi190.stdout: mon pg warn max object skew = 0 2024-09-04T17:44:47.988 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-04T17:44:47.989 INFO:teuthology.orchestra.run.smithi190.stdout: # disable pg_autoscaler by default for new pools 2024-09-04T17:44:47.989 INFO:teuthology.orchestra.run.smithi190.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-04T17:44:47.989 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-04T17:44:47.989 INFO:teuthology.orchestra.run.smithi190.stdout: osd pool default size = 2 2024-09-04T17:44:47.989 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-04T17:44:47.989 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd allow primary affinity = true 2024-09-04T17:44:47.989 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd allow pg remap = true 2024-09-04T17:44:47.989 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on legacy crush tunables = false 2024-09-04T17:44:47.990 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on crush straw calc version zero = false 2024-09-04T17:44:47.990 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on no sortbitwise = false 2024-09-04T17:44:47.990 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on osd down out interval zero = false 2024-09-04T17:44:47.990 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on too few osds = false 2024-09-04T17:44:47.990 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-04T17:44:47.990 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-04T17:44:47.990 INFO:teuthology.orchestra.run.smithi190.stdout: mon_allow_pool_size_one = true 2024-09-04T17:44:47.990 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-04T17:44:47.990 INFO:teuthology.orchestra.run.smithi190.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-04T17:44:47.991 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-04T17:44:47.991 INFO:teuthology.orchestra.run.smithi190.stdout: osd default data pool replay window = 5 2024-09-04T17:44:47.991 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-04T17:44:47.991 INFO:teuthology.orchestra.run.smithi190.stdout: mon allow pool delete = true 2024-09-04T17:44:47.991 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-04T17:44:47.991 INFO:teuthology.orchestra.run.smithi190.stdout: mon cluster log file level = debug 2024-09-04T17:44:47.991 INFO:teuthology.orchestra.run.smithi190.stdout: debug asserts on shutdown = true 2024-09-04T17:44:47.991 INFO:teuthology.orchestra.run.smithi190.stdout: mon health detail to clog = false 2024-09-04T17:44:47.991 INFO:teuthology.orchestra.run.smithi190.stdout: mon host = "172.21.15.92,[v2:172.21.15.92:3301,v1:172.21.15.92:6790],172.21.15.190" 2024-09-04T17:44:47.991 INFO:teuthology.orchestra.run.smithi190.stdout: mon client directed command retry = 5 2024-09-04T17:44:47.991 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on pool no app = False 2024-09-04T17:44:47.992 INFO:teuthology.orchestra.run.smithi190.stdout: ms inject socket failures = 5000 2024-09-04T17:44:47.992 INFO:teuthology.orchestra.run.smithi190.stdout: fsid = f36b6ce2-8c72-4c99-8607-5c53c38ef67b 2024-09-04T17:44:47.992 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-04T17:44:47.992 INFO:teuthology.orchestra.run.smithi190.stdout:[osd] 2024-09-04T17:44:47.992 INFO:teuthology.orchestra.run.smithi190.stdout: osd journal size = 100 2024-09-04T17:44:47.992 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-04T17:44:47.992 INFO:teuthology.orchestra.run.smithi190.stdout: osd scrub load threshold = 5.0 2024-09-04T17:44:47.993 INFO:teuthology.orchestra.run.smithi190.stdout: osd scrub max interval = 600 2024-09-04T17:44:47.993 INFO:teuthology.orchestra.run.smithi190.stdout: osd mclock profile = high_recovery_ops 2024-09-04T17:44:47.993 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-04T17:44:47.993 INFO:teuthology.orchestra.run.smithi190.stdout: osd recover clone overlap = true 2024-09-04T17:44:47.993 INFO:teuthology.orchestra.run.smithi190.stdout: osd recovery max chunk = 1048576 2024-09-04T17:44:47.993 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-04T17:44:47.993 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug shutdown = true 2024-09-04T17:44:47.993 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug op order = true 2024-09-04T17:44:47.994 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug verify stray on activate = true 2024-09-04T17:44:47.994 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug trim objects = true 2024-09-04T17:44:47.994 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-04T17:44:47.994 INFO:teuthology.orchestra.run.smithi190.stdout: osd open classes on start = true 2024-09-04T17:44:47.994 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug pg log writeout = true 2024-09-04T17:44:47.994 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-04T17:44:47.994 INFO:teuthology.orchestra.run.smithi190.stdout: osd deep scrub update digest min age = 30 2024-09-04T17:44:47.995 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-04T17:44:47.995 INFO:teuthology.orchestra.run.smithi190.stdout: osd map max advance = 10 2024-09-04T17:44:47.995 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-04T17:44:47.995 INFO:teuthology.orchestra.run.smithi190.stdout: journal zero on create = true 2024-09-04T17:44:47.995 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-04T17:44:47.995 INFO:teuthology.orchestra.run.smithi190.stdout: filestore ondisk finisher threads = 3 2024-09-04T17:44:47.995 INFO:teuthology.orchestra.run.smithi190.stdout: filestore apply finisher threads = 3 2024-09-04T17:44:47.995 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-04T17:44:47.995 INFO:teuthology.orchestra.run.smithi190.stdout: bdev debug aio = true 2024-09-04T17:44:47.995 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug misdirected ops = true 2024-09-04T17:44:47.996 INFO:teuthology.orchestra.run.smithi190.stdout: bdev async discard = True 2024-09-04T17:44:47.996 INFO:teuthology.orchestra.run.smithi190.stdout: bdev enable discard = True 2024-09-04T17:44:47.996 INFO:teuthology.orchestra.run.smithi190.stdout: bluestore allocator = bitmap 2024-09-04T17:44:47.996 INFO:teuthology.orchestra.run.smithi190.stdout: bluestore block size = 96636764160 2024-09-04T17:44:47.996 INFO:teuthology.orchestra.run.smithi190.stdout: bluestore fsck on mount = True 2024-09-04T17:44:47.996 INFO:teuthology.orchestra.run.smithi190.stdout: debug bluefs = 1/20 2024-09-04T17:44:47.996 INFO:teuthology.orchestra.run.smithi190.stdout: debug bluestore = 1/20 2024-09-04T17:44:47.996 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2024-09-04T17:44:47.996 INFO:teuthology.orchestra.run.smithi190.stdout: debug osd = 20 2024-09-04T17:44:47.997 INFO:teuthology.orchestra.run.smithi190.stdout: debug rocksdb = 4/10 2024-09-04T17:44:47.997 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd backfillfull_ratio = 0.85 2024-09-04T17:44:47.997 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd full ratio = 0.9 2024-09-04T17:44:47.997 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd nearfull ratio = 0.8 2024-09-04T17:44:47.997 INFO:teuthology.orchestra.run.smithi190.stdout: osd blocked scrub grace period = 3600 2024-09-04T17:44:47.997 INFO:teuthology.orchestra.run.smithi190.stdout: osd failsafe full ratio = 0.95 2024-09-04T17:44:47.997 INFO:teuthology.orchestra.run.smithi190.stdout: osd max markdown count = 1000 2024-09-04T17:44:47.997 INFO:teuthology.orchestra.run.smithi190.stdout: osd objectstore = bluestore 2024-09-04T17:44:47.997 INFO:teuthology.orchestra.run.smithi190.stdout: osd shutdown pgref assert = True 2024-09-04T17:44:47.998 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-04T17:44:47.998 INFO:teuthology.orchestra.run.smithi190.stdout:[mgr] 2024-09-04T17:44:47.998 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2024-09-04T17:44:47.998 INFO:teuthology.orchestra.run.smithi190.stdout: debug mgr = 20 2024-09-04T17:44:47.998 INFO:teuthology.orchestra.run.smithi190.stdout: debug mon = 20 2024-09-04T17:44:47.998 INFO:teuthology.orchestra.run.smithi190.stdout: debug auth = 20 2024-09-04T17:44:47.998 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min pgs per osd = 4 2024-09-04T17:44:47.998 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min bytes per osd = 10 2024-09-04T17:44:47.998 INFO:teuthology.orchestra.run.smithi190.stdout: mgr/telemetry/nag = false 2024-09-04T17:44:47.999 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-04T17:44:47.999 INFO:teuthology.orchestra.run.smithi190.stdout:[mon] 2024-09-04T17:44:47.999 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2024-09-04T17:44:47.999 INFO:teuthology.orchestra.run.smithi190.stdout: debug mon = 20 2024-09-04T17:44:47.999 INFO:teuthology.orchestra.run.smithi190.stdout: debug paxos = 20 2024-09-04T17:44:47.999 INFO:teuthology.orchestra.run.smithi190.stdout: debug auth = 20 2024-09-04T17:44:47.999 INFO:teuthology.orchestra.run.smithi190.stdout: mon data avail warn = 5 2024-09-04T17:44:47.999 INFO:teuthology.orchestra.run.smithi190.stdout: mon mgr mkfs grace = 240 2024-09-04T17:44:47.999 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min pgs per osd = 4 2024-09-04T17:44:47.999 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd reporter subtree level = osd 2024-09-04T17:44:47.999 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd prime pg temp = true 2024-09-04T17:44:48.000 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min bytes per osd = 10 2024-09-04T17:44:48.000 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-04T17:44:48.000 INFO:teuthology.orchestra.run.smithi190.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-04T17:44:48.000 INFO:teuthology.orchestra.run.smithi190.stdout: auth mon ticket ttl = 660 # 11m 2024-09-04T17:44:48.000 INFO:teuthology.orchestra.run.smithi190.stdout: auth service ticket ttl = 240 # 4m 2024-09-04T17:44:48.000 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-04T17:44:48.000 INFO:teuthology.orchestra.run.smithi190.stdout: # don't complain about insecure global_id in the test suite 2024-09-04T17:44:48.000 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-04T17:44:48.000 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-04T17:44:48.001 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-04T17:44:48.001 INFO:teuthology.orchestra.run.smithi190.stdout: # 1m isn't quite enough 2024-09-04T17:44:48.001 INFO:teuthology.orchestra.run.smithi190.stdout: mon_down_mkfs_grace = 2m 2024-09-04T17:44:48.001 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-04T17:44:48.001 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_filestore_osds = false 2024-09-04T17:44:48.001 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-04T17:44:48.001 INFO:teuthology.orchestra.run.smithi190.stdout:[client] 2024-09-04T17:44:48.001 INFO:teuthology.orchestra.run.smithi190.stdout: rgw cache enabled = true 2024-09-04T17:44:48.001 INFO:teuthology.orchestra.run.smithi190.stdout: rgw enable ops log = true 2024-09-04T17:44:48.002 INFO:teuthology.orchestra.run.smithi190.stdout: rgw enable usage log = true 2024-09-04T17:44:48.002 INFO:teuthology.orchestra.run.smithi190.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-04T17:44:48.002 INFO:teuthology.orchestra.run.smithi190.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-04T17:44:48.002 INFO:teuthology.orchestra.run.smithi190.stdout: rbd qos iops burst = 100 2024-09-04T17:44:48.002 INFO:teuthology.orchestra.run.smithi190.stdout: rbd qos iops limit = 50 2024-09-04T17:44:48.002 INFO:teuthology.orchestra.run.smithi190.stdout: rbd qos schedule tick min = 100 2024-09-04T17:44:48.002 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.a] 2024-09-04T17:44:48.002 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.c] 2024-09-04T17:44:48.002 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.b] 2024-09-04T17:44:48.003 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-04T17:44:48.003 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-04T17:44:48.075 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-04T17:44:48.075 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-04T17:44:48.076 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-04T17:44:48.124 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-04T17:44:48.124 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-04T17:44:48.172 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi092.front.sepia.ceph.com 2024-09-04T17:44:48.172 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-04T17:44:48.172 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-04T17:44:48.172 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-04T17:44:48.238 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-04T17:44:48.238 DEBUG:teuthology.orchestra.run.smithi092:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-04T17:44:48.285 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi190.front.sepia.ceph.com 2024-09-04T17:44:48.285 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-04T17:44:48.285 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-04T17:44:48.285 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-04T17:44:48.309 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-04T17:44:48.309 DEBUG:teuthology.orchestra.run.smithi190:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-04T17:44:48.357 INFO:tasks.ceph:Setting up mon nodes... 2024-09-04T17:44:48.357 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-04T17:44:48.358 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-04T17:44:48.401 INFO:teuthology.orchestra.run.smithi092.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-04T17:44:48.404 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-04T17:44:48.447 INFO:teuthology.orchestra.run.smithi190.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-04T17:44:48.463 INFO:tasks.ceph:Setting up mds nodes... 2024-09-04T17:44:48.463 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-04T17:44:48.463 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-04T17:44:48.497 INFO:teuthology.orchestra.run.smithi092.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-04T17:44:48.507 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-04T17:44:48.542 INFO:teuthology.orchestra.run.smithi190.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-04T17:44:48.553 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-04T17:44:48.553 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi092.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi190.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-04T17:44:48.554 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-04T17:44:48.564 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-04T17:44:48.564 INFO:tasks.ceph:role: osd.0 2024-09-04T17:44:48.565 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi092.front.sepia.ceph.com 2024-09-04T17:44:48.565 DEBUG:teuthology.orchestra.run.smithi092:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-04T17:44:48.628 INFO:teuthology.orchestra.run.smithi092.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-04T17:44:48.628 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-04T17:44:48.628 INFO:teuthology.orchestra.run.smithi092.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-04T17:44:48.628 INFO:teuthology.orchestra.run.smithi092.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-04T17:44:48.628 INFO:teuthology.orchestra.run.smithi092.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-04T17:44:48.629 INFO:teuthology.orchestra.run.smithi092.stdout: = sunit=0 swidth=0 blks 2024-09-04T17:44:48.629 INFO:teuthology.orchestra.run.smithi092.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-04T17:44:48.629 INFO:teuthology.orchestra.run.smithi092.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-04T17:44:48.629 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-04T17:44:48.629 INFO:teuthology.orchestra.run.smithi092.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-04T17:44:49.103 INFO:teuthology.orchestra.run.smithi092.stdout:Discarding blocks...Done. 2024-09-04T17:44:49.105 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi092.front.sepia.ceph.com -o noatime 2024-09-04T17:44:49.105 DEBUG:teuthology.orchestra.run.smithi092:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-04T17:44:49.196 DEBUG:teuthology.orchestra.run.smithi092:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-04T17:44:49.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-04T17:44:49.208 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: /sbin/restorecon: command not found 2024-09-04T17:44:49.208 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-04T17:44:49.264 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-04T17:44:49.264 INFO:tasks.ceph:role: osd.1 2024-09-04T17:44:49.264 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi092.front.sepia.ceph.com 2024-09-04T17:44:49.264 DEBUG:teuthology.orchestra.run.smithi092:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-04T17:44:49.328 INFO:teuthology.orchestra.run.smithi092.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-04T17:44:49.328 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-04T17:44:49.328 INFO:teuthology.orchestra.run.smithi092.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-04T17:44:49.328 INFO:teuthology.orchestra.run.smithi092.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-04T17:44:49.328 INFO:teuthology.orchestra.run.smithi092.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-04T17:44:49.329 INFO:teuthology.orchestra.run.smithi092.stdout: = sunit=0 swidth=0 blks 2024-09-04T17:44:49.329 INFO:teuthology.orchestra.run.smithi092.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-04T17:44:49.329 INFO:teuthology.orchestra.run.smithi092.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-04T17:44:49.329 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-04T17:44:49.329 INFO:teuthology.orchestra.run.smithi092.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-04T17:44:49.801 INFO:teuthology.orchestra.run.smithi092.stdout:Discarding blocks...Done. 2024-09-04T17:44:49.804 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi092.front.sepia.ceph.com -o noatime 2024-09-04T17:44:49.804 DEBUG:teuthology.orchestra.run.smithi092:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-04T17:44:49.961 DEBUG:teuthology.orchestra.run.smithi092:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-04T17:44:49.973 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: /sbin/restorecon: command not found 2024-09-04T17:44:49.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-04T17:44:49.974 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-04T17:44:50.028 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-04T17:44:50.028 INFO:tasks.ceph:role: osd.2 2024-09-04T17:44:50.028 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi092.front.sepia.ceph.com 2024-09-04T17:44:50.029 DEBUG:teuthology.orchestra.run.smithi092:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-04T17:44:50.090 INFO:teuthology.orchestra.run.smithi092.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-04T17:44:50.090 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-04T17:44:50.091 INFO:teuthology.orchestra.run.smithi092.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-04T17:44:50.091 INFO:teuthology.orchestra.run.smithi092.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-04T17:44:50.091 INFO:teuthology.orchestra.run.smithi092.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-04T17:44:50.091 INFO:teuthology.orchestra.run.smithi092.stdout: = sunit=0 swidth=0 blks 2024-09-04T17:44:50.091 INFO:teuthology.orchestra.run.smithi092.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-04T17:44:50.091 INFO:teuthology.orchestra.run.smithi092.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-04T17:44:50.091 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-04T17:44:50.091 INFO:teuthology.orchestra.run.smithi092.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-04T17:44:50.576 INFO:teuthology.orchestra.run.smithi092.stdout:Discarding blocks...Done. 2024-09-04T17:44:50.578 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi092.front.sepia.ceph.com -o noatime 2024-09-04T17:44:50.578 DEBUG:teuthology.orchestra.run.smithi092:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-04T17:44:50.716 DEBUG:teuthology.orchestra.run.smithi092:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-04T17:44:50.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-04T17:44:50.728 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: /sbin/restorecon: command not found 2024-09-04T17:44:50.728 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-04T17:44:50.787 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-04T17:44:50.787 INFO:tasks.ceph:role: osd.3 2024-09-04T17:44:50.788 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi092.front.sepia.ceph.com 2024-09-04T17:44:50.788 DEBUG:teuthology.orchestra.run.smithi092:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-04T17:44:50.852 INFO:teuthology.orchestra.run.smithi092.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-04T17:44:50.852 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-04T17:44:50.852 INFO:teuthology.orchestra.run.smithi092.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-04T17:44:50.852 INFO:teuthology.orchestra.run.smithi092.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-04T17:44:50.852 INFO:teuthology.orchestra.run.smithi092.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-04T17:44:50.852 INFO:teuthology.orchestra.run.smithi092.stdout: = sunit=0 swidth=0 blks 2024-09-04T17:44:50.853 INFO:teuthology.orchestra.run.smithi092.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-04T17:44:50.853 INFO:teuthology.orchestra.run.smithi092.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-04T17:44:50.853 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-04T17:44:50.853 INFO:teuthology.orchestra.run.smithi092.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-04T17:44:51.278 INFO:teuthology.orchestra.run.smithi092.stdout:Discarding blocks...Done. 2024-09-04T17:44:51.280 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi092.front.sepia.ceph.com -o noatime 2024-09-04T17:44:51.280 DEBUG:teuthology.orchestra.run.smithi092:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-04T17:44:51.379 DEBUG:teuthology.orchestra.run.smithi092:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-04T17:44:51.389 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: /sbin/restorecon: command not found 2024-09-04T17:44:51.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-04T17:44:51.390 DEBUG:teuthology.orchestra.run.smithi092:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-04T17:44:51.491 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-04T17:44:51.487+0000 7ff1fcbb77c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-04T17:44:51.493 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-04T17:44:51.491+0000 7ff1fcbb77c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-04T17:44:51.494 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-04T17:44:51.491+0000 7ff1fcbb77c0 -1 bdev(0x5638b997b800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-04T17:44:51.494 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-04T17:44:51.491+0000 7ff1fcbb77c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-04T17:44:54.655 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-04T17:44:54.669 DEBUG:teuthology.orchestra.run.smithi092:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-04T17:44:54.750 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-04T17:44:54.747+0000 7fc30abe47c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-04T17:44:54.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-04T17:44:54.747+0000 7fc30abe47c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-04T17:44:54.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-04T17:44:54.747+0000 7fc30abe47c0 -1 bdev(0x5646f71a9800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-04T17:44:54.751 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-04T17:44:54.747+0000 7fc30abe47c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-04T17:44:57.904 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-04T17:44:57.961 DEBUG:teuthology.orchestra.run.smithi092:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-04T17:44:58.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-04T17:44:58.039+0000 7ff8ff0387c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-04T17:44:58.043 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-04T17:44:58.039+0000 7ff8ff0387c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-04T17:44:58.043 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-04T17:44:58.039+0000 7ff8ff0387c0 -1 bdev(0x55856d051800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-04T17:44:58.043 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-04T17:44:58.039+0000 7ff8ff0387c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-04T17:45:01.230 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-04T17:45:01.284 DEBUG:teuthology.orchestra.run.smithi092:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-04T17:45:01.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-04T17:45:01.367+0000 7ff29db997c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-04T17:45:01.371 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-04T17:45:01.367+0000 7ff29db997c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-04T17:45:01.371 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-04T17:45:01.367+0000 7ff29db997c0 -1 bdev(0x556f1a66b800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-04T17:45:01.371 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-04T17:45:01.367+0000 7ff29db997c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-04T17:45:04.501 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-04T17:45:04.561 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-04T17:45:04.577 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-04T17:45:04.577 INFO:tasks.ceph:role: osd.4 2024-09-04T17:45:04.577 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi190.front.sepia.ceph.com 2024-09-04T17:45:04.577 DEBUG:teuthology.orchestra.run.smithi190:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-04T17:45:04.640 INFO:teuthology.orchestra.run.smithi190.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-04T17:45:04.640 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-04T17:45:04.640 INFO:teuthology.orchestra.run.smithi190.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-04T17:45:04.640 INFO:teuthology.orchestra.run.smithi190.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-04T17:45:04.640 INFO:teuthology.orchestra.run.smithi190.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-04T17:45:04.640 INFO:teuthology.orchestra.run.smithi190.stdout: = sunit=0 swidth=0 blks 2024-09-04T17:45:04.640 INFO:teuthology.orchestra.run.smithi190.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-04T17:45:04.641 INFO:teuthology.orchestra.run.smithi190.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-04T17:45:04.641 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-04T17:45:04.641 INFO:teuthology.orchestra.run.smithi190.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-04T17:45:05.095 INFO:teuthology.orchestra.run.smithi190.stdout:Discarding blocks...Done. 2024-09-04T17:45:05.097 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi190.front.sepia.ceph.com -o noatime 2024-09-04T17:45:05.097 DEBUG:teuthology.orchestra.run.smithi190:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-04T17:45:05.193 DEBUG:teuthology.orchestra.run.smithi190:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-04T17:45:05.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-04T17:45:05.205 INFO:teuthology.orchestra.run.smithi190.stderr:sudo: /sbin/restorecon: command not found 2024-09-04T17:45:05.205 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-04T17:45:05.259 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-04T17:45:05.260 INFO:tasks.ceph:role: osd.5 2024-09-04T17:45:05.260 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi190.front.sepia.ceph.com 2024-09-04T17:45:05.260 DEBUG:teuthology.orchestra.run.smithi190:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-04T17:45:05.324 INFO:teuthology.orchestra.run.smithi190.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-04T17:45:05.324 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-04T17:45:05.324 INFO:teuthology.orchestra.run.smithi190.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-04T17:45:05.325 INFO:teuthology.orchestra.run.smithi190.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-04T17:45:05.325 INFO:teuthology.orchestra.run.smithi190.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-04T17:45:05.325 INFO:teuthology.orchestra.run.smithi190.stdout: = sunit=0 swidth=0 blks 2024-09-04T17:45:05.325 INFO:teuthology.orchestra.run.smithi190.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-04T17:45:05.325 INFO:teuthology.orchestra.run.smithi190.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-04T17:45:05.325 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-04T17:45:05.325 INFO:teuthology.orchestra.run.smithi190.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-04T17:45:05.793 INFO:teuthology.orchestra.run.smithi190.stdout:Discarding blocks...Done. 2024-09-04T17:45:05.795 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi190.front.sepia.ceph.com -o noatime 2024-09-04T17:45:05.796 DEBUG:teuthology.orchestra.run.smithi190:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-04T17:45:05.945 DEBUG:teuthology.orchestra.run.smithi190:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-04T17:45:05.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-04T17:45:05.957 INFO:teuthology.orchestra.run.smithi190.stderr:sudo: /sbin/restorecon: command not found 2024-09-04T17:45:05.958 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-04T17:45:06.012 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-04T17:45:06.012 INFO:tasks.ceph:role: osd.6 2024-09-04T17:45:06.012 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi190.front.sepia.ceph.com 2024-09-04T17:45:06.012 DEBUG:teuthology.orchestra.run.smithi190:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-04T17:45:06.080 INFO:teuthology.orchestra.run.smithi190.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-04T17:45:06.080 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-04T17:45:06.081 INFO:teuthology.orchestra.run.smithi190.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-04T17:45:06.081 INFO:teuthology.orchestra.run.smithi190.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-04T17:45:06.081 INFO:teuthology.orchestra.run.smithi190.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-04T17:45:06.081 INFO:teuthology.orchestra.run.smithi190.stdout: = sunit=0 swidth=0 blks 2024-09-04T17:45:06.081 INFO:teuthology.orchestra.run.smithi190.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-04T17:45:06.081 INFO:teuthology.orchestra.run.smithi190.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-04T17:45:06.081 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-04T17:45:06.081 INFO:teuthology.orchestra.run.smithi190.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-04T17:45:06.577 INFO:teuthology.orchestra.run.smithi190.stdout:Discarding blocks...Done. 2024-09-04T17:45:06.579 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi190.front.sepia.ceph.com -o noatime 2024-09-04T17:45:06.580 DEBUG:teuthology.orchestra.run.smithi190:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-04T17:45:06.721 DEBUG:teuthology.orchestra.run.smithi190:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-04T17:45:06.732 INFO:teuthology.orchestra.run.smithi190.stderr:sudo: /sbin/restorecon: command not found 2024-09-04T17:45:06.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-04T17:45:06.734 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-04T17:45:06.787 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-04T17:45:06.788 INFO:tasks.ceph:role: osd.7 2024-09-04T17:45:06.788 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi190.front.sepia.ceph.com 2024-09-04T17:45:06.788 DEBUG:teuthology.orchestra.run.smithi190:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-04T17:45:06.848 INFO:teuthology.orchestra.run.smithi190.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-04T17:45:06.849 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-04T17:45:06.849 INFO:teuthology.orchestra.run.smithi190.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-04T17:45:06.849 INFO:teuthology.orchestra.run.smithi190.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-04T17:45:06.849 INFO:teuthology.orchestra.run.smithi190.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-04T17:45:06.849 INFO:teuthology.orchestra.run.smithi190.stdout: = sunit=0 swidth=0 blks 2024-09-04T17:45:06.849 INFO:teuthology.orchestra.run.smithi190.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-04T17:45:06.849 INFO:teuthology.orchestra.run.smithi190.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-04T17:45:06.849 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-04T17:45:06.850 INFO:teuthology.orchestra.run.smithi190.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-04T17:45:07.276 INFO:teuthology.orchestra.run.smithi190.stdout:Discarding blocks...Done. 2024-09-04T17:45:07.278 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi190.front.sepia.ceph.com -o noatime 2024-09-04T17:45:07.278 DEBUG:teuthology.orchestra.run.smithi190:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-04T17:45:07.371 DEBUG:teuthology.orchestra.run.smithi190:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-04T17:45:07.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-04T17:45:07.381 INFO:teuthology.orchestra.run.smithi190.stderr:sudo: /sbin/restorecon: command not found 2024-09-04T17:45:07.381 DEBUG:teuthology.orchestra.run.smithi190:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-04T17:45:07.495 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-04T17:45:07.492+0000 7fe30fd817c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-04T17:45:07.497 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-04T17:45:07.496+0000 7fe30fd817c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-04T17:45:07.497 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-04T17:45:07.496+0000 7fe30fd817c0 -1 bdev(0x5632deb5d800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-04T17:45:07.497 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-04T17:45:07.496+0000 7fe30fd817c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-04T17:45:10.649 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-04T17:45:10.664 DEBUG:teuthology.orchestra.run.smithi190:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-04T17:45:10.746 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-04T17:45:10.744+0000 7fbc91cdd7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-04T17:45:10.747 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-04T17:45:10.744+0000 7fbc91cdd7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-04T17:45:10.747 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-04T17:45:10.744+0000 7fbc91cdd7c0 -1 bdev(0x556fc54bd800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-04T17:45:10.747 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-04T17:45:10.744+0000 7fbc91cdd7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-04T17:45:13.904 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-04T17:45:13.966 DEBUG:teuthology.orchestra.run.smithi190:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-04T17:45:14.052 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-04T17:45:14.048+0000 7f359939e7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-04T17:45:14.053 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-04T17:45:14.052+0000 7f359939e7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-04T17:45:14.053 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-04T17:45:14.052+0000 7f359939e7c0 -1 bdev(0x5618532ad800 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-04T17:45:14.053 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-04T17:45:14.052+0000 7f359939e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-04T17:45:17.203 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-04T17:45:17.256 DEBUG:teuthology.orchestra.run.smithi190:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-04T17:45:17.338 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-04T17:45:17.336+0000 7f2e87ea87c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-04T17:45:17.339 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-04T17:45:17.336+0000 7f2e87ea87c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-04T17:45:17.339 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-04T17:45:17.336+0000 7f2e87ea87c0 -1 bdev(0x562c8e5cf800 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-04T17:45:17.339 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-04T17:45:17.336+0000 7f2e87ea87c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-04T17:45:20.495 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-04T17:45:20.553 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-04T17:45:20.553 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-04T17:45:20.553 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-04T17:45:20.568 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-04T17:45:20.568 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-04T17:45:20.623 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-04T17:45:20.623 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-04T17:45:20.679 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-04T17:45:20.679 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-04T17:45:20.734 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-04T17:45:20.734 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-04T17:45:20.787 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-04T17:45:20.787 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-04T17:45:20.801 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-04T17:45:20.801 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-04T17:45:20.857 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-04T17:45:20.857 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-04T17:45:20.911 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-04T17:45:20.911 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-04T17:45:20.967 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-04T17:45:20.967 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-04T17:45:21.019 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-04T17:45:21.019 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-04T17:45:21.025 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-04T17:45:21.025 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-04T17:45:21.064 INFO:tasks.ceph:Adding keys to all mons... 2024-09-04T17:45:21.064 DEBUG:teuthology.orchestra.run.smithi092:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-04T17:45:21.068 DEBUG:teuthology.orchestra.run.smithi190:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-04T17:45:21.110 INFO:teuthology.orchestra.run.smithi092.stdout:[mgr.y] 2024-09-04T17:45:21.110 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCQnNhmGeMPGBAAYbes5mwlE2Jos5ElJIdWUg== 2024-09-04T17:45:21.110 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.0] 2024-09-04T17:45:21.110 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCTnNhmZ/NxHRAAXW960XihzC1UzODS2VCY1w== 2024-09-04T17:45:21.111 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.1] 2024-09-04T17:45:21.111 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCWnNhm0F7iLBAAfb0hfhQJKBQ50rKm0o4BXw== 2024-09-04T17:45:21.111 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.2] 2024-09-04T17:45:21.111 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCanNhmAcOmAhAAKOpQOOEH/ZLpyC9w6v6Okg== 2024-09-04T17:45:21.111 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.3] 2024-09-04T17:45:21.111 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCdnNhmglwzFhAAG4TSmwXKiio2gfsw+w+xyw== 2024-09-04T17:45:21.111 INFO:teuthology.orchestra.run.smithi092.stdout:[mgr.x] 2024-09-04T17:45:21.111 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCQnNhm6hTIGhAA5kbrYxqSuNlobkGVQ6Vj7w== 2024-09-04T17:45:21.111 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.4] 2024-09-04T17:45:21.111 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCjnNhm0AioHRAAz0rHtv15UZMORGVhCs30dg== 2024-09-04T17:45:21.111 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.5] 2024-09-04T17:45:21.111 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCmnNhmj5CdLBAAe2snJWAUoIDy/XF+PM2tHg== 2024-09-04T17:45:21.112 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.6] 2024-09-04T17:45:21.112 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCqnNhmTxA3AxAAneEdjyaYn/gDyj7Pz6vItw== 2024-09-04T17:45:21.112 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.7] 2024-09-04T17:45:21.112 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCtnNhm72NNFBAAIdDhw2KzH1u6iUjhGOM3Kw== 2024-09-04T17:45:21.112 INFO:teuthology.orchestra.run.smithi092.stdout:[client.0] 2024-09-04T17:45:21.112 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCQnNhm7SnIHRAA0TfYnXWjV+agRcRQynsMJw== 2024-09-04T17:45:21.112 INFO:teuthology.orchestra.run.smithi092.stdout:[client.1] 2024-09-04T17:45:21.112 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCQnNhmbkRpIBAAVQZECP4gHt3eaQ5BXpevvQ== 2024-09-04T17:45:21.117 INFO:teuthology.orchestra.run.smithi190.stdout:[mgr.y] 2024-09-04T17:45:21.117 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQCQnNhmGeMPGBAAYbes5mwlE2Jos5ElJIdWUg== 2024-09-04T17:45:21.118 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.0] 2024-09-04T17:45:21.118 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQCTnNhmZ/NxHRAAXW960XihzC1UzODS2VCY1w== 2024-09-04T17:45:21.118 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.1] 2024-09-04T17:45:21.118 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQCWnNhm0F7iLBAAfb0hfhQJKBQ50rKm0o4BXw== 2024-09-04T17:45:21.118 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.2] 2024-09-04T17:45:21.118 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQCanNhmAcOmAhAAKOpQOOEH/ZLpyC9w6v6Okg== 2024-09-04T17:45:21.118 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.3] 2024-09-04T17:45:21.118 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQCdnNhmglwzFhAAG4TSmwXKiio2gfsw+w+xyw== 2024-09-04T17:45:21.118 INFO:teuthology.orchestra.run.smithi190.stdout:[mgr.x] 2024-09-04T17:45:21.118 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQCQnNhm6hTIGhAA5kbrYxqSuNlobkGVQ6Vj7w== 2024-09-04T17:45:21.118 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.4] 2024-09-04T17:45:21.118 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQCjnNhm0AioHRAAz0rHtv15UZMORGVhCs30dg== 2024-09-04T17:45:21.118 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.5] 2024-09-04T17:45:21.119 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQCmnNhmj5CdLBAAe2snJWAUoIDy/XF+PM2tHg== 2024-09-04T17:45:21.119 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.6] 2024-09-04T17:45:21.119 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQCqnNhmTxA3AxAAneEdjyaYn/gDyj7Pz6vItw== 2024-09-04T17:45:21.119 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.7] 2024-09-04T17:45:21.119 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQCtnNhm72NNFBAAIdDhw2KzH1u6iUjhGOM3Kw== 2024-09-04T17:45:21.119 INFO:teuthology.orchestra.run.smithi190.stdout:[client.0] 2024-09-04T17:45:21.119 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQCQnNhm7SnIHRAA0TfYnXWjV+agRcRQynsMJw== 2024-09-04T17:45:21.119 INFO:teuthology.orchestra.run.smithi190.stdout:[client.1] 2024-09-04T17:45:21.119 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQCQnNhmbkRpIBAAVQZECP4gHt3eaQ5BXpevvQ== 2024-09-04T17:45:21.120 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-04T17:45:21.157 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-04T17:45:21.209 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-04T17:45:21.211 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-04T17:45:21.254 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-04T17:45:21.467 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-04T17:45:21.514 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-04T17:45:21.557 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-04T17:45:21.604 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-04T17:45:21.607 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-04T17:45:21.692 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-04T17:45:21.695 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-04T17:45:21.780 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-04T17:45:21.783 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-04T17:45:21.869 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-04T17:45:21.873 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-04T17:45:21.916 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-04T17:45:21.961 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-04T17:45:22.004 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-04T17:45:22.045 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-04T17:45:22.093 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-04T17:45:22.095 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-04T17:45:22.140 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-04T17:45:22.186 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-04T17:45:22.230 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-04T17:45:22.230 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-04T17:45:22.282 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-04T17:45:22.374 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-04T17:45:22.389 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-04T17:45:22.437 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-04T17:45:22.527 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-04T17:45:22.538 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-04T17:45:22.552 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-04T17:45:22.650 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-04T17:45:22.660 DEBUG:teuthology.orchestra.run.smithi092:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-04T17:45:22.662 DEBUG:teuthology.orchestra.run.smithi190:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-04T17:45:22.707 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-04T17:45:22.708 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-04T17:45:22.708 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-04T17:45:22.710 INFO:tasks.ceph.mon.a:Started 2024-09-04T17:45:22.710 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-04T17:45:22.710 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-04T17:45:22.713 INFO:tasks.ceph.mon.c:Started 2024-09-04T17:45:22.713 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-04T17:45:22.713 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-04T17:45:22.753 INFO:tasks.ceph.mon.b:Started 2024-09-04T17:45:22.753 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-04T17:45:22.753 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-04T17:45:22.753 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-04T17:45:22.756 INFO:tasks.ceph.mgr.y:Started 2024-09-04T17:45:22.756 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-04T17:45:22.756 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-04T17:45:22.758 INFO:tasks.ceph.mgr.x:Started 2024-09-04T17:45:22.758 DEBUG:tasks.ceph:set 0 configs 2024-09-04T17:45:22.758 INFO:tasks.ceph:Setting crush tunables to default 2024-09-04T17:45:22.758 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd crush tunables default 2024-09-04T17:45:28.284 INFO:teuthology.orchestra.run.smithi092.stderr:adjusted tunables profile to default 2024-09-04T17:45:28.306 INFO:tasks.ceph:check_enable_crimson: False 2024-09-04T17:45:28.306 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-04T17:45:28.306 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-04T17:45:28.306 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-04T17:45:28.320 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-04T17:45:28.320 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-04T17:45:28.375 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-04T17:45:28.376 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-04T17:45:28.435 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-04T17:45:28.436 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-04T17:45:28.491 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-04T17:45:28.491 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-04T17:45:28.505 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-04T17:45:28.505 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-04T17:45:28.559 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-04T17:45:28.559 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-04T17:45:28.615 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-04T17:45:28.615 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-04T17:45:28.671 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new e58efca2-bc5a-4bc4-b3d7-4036ebd9dc63 0 2024-09-04T17:45:28.918 INFO:teuthology.orchestra.run.smithi190.stdout:0 2024-09-04T17:45:28.935 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new 36e03b64-2602-4143-bdd5-f7355de255fe 1 2024-09-04T17:45:29.145 INFO:teuthology.orchestra.run.smithi190.stdout:1 2024-09-04T17:45:29.159 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new 581ab71e-be04-4031-a0c4-f7e40b74b7c7 2 2024-09-04T17:45:29.349 INFO:teuthology.orchestra.run.smithi190.stdout:2 2024-09-04T17:45:29.365 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new 5f5d096e-fab2-41fe-b8dd-943d1c40894c 3 2024-09-04T17:45:29.556 INFO:teuthology.orchestra.run.smithi190.stdout:3 2024-09-04T17:45:29.572 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new 840951a5-e13b-462c-8c8e-2ca61a53fe3c 4 2024-09-04T17:45:29.767 INFO:teuthology.orchestra.run.smithi190.stdout:4 2024-09-04T17:45:29.784 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new 82e556e5-00c3-46fc-aab5-3544bba279d1 5 2024-09-04T17:45:29.987 INFO:teuthology.orchestra.run.smithi190.stdout:5 2024-09-04T17:45:30.003 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new 4e1e3d67-3020-470d-98b7-891fdc966238 6 2024-09-04T17:45:30.206 INFO:teuthology.orchestra.run.smithi190.stdout:6 2024-09-04T17:45:30.223 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new bd9c9d3e-27db-4bc2-b998-7f446576a543 7 2024-09-04T17:45:30.412 INFO:teuthology.orchestra.run.smithi190.stdout:7 2024-09-04T17:45:30.428 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-04T17:45:30.429 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-04T17:45:30.431 INFO:tasks.ceph.osd.0:Started 2024-09-04T17:45:30.431 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-04T17:45:30.431 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-04T17:45:30.433 INFO:tasks.ceph.osd.1:Started 2024-09-04T17:45:30.433 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-04T17:45:30.434 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-04T17:45:30.436 INFO:tasks.ceph.osd.2:Started 2024-09-04T17:45:30.436 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-04T17:45:30.436 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-04T17:45:30.439 INFO:tasks.ceph.osd.3:Started 2024-09-04T17:45:30.439 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-04T17:45:30.440 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-04T17:45:30.442 INFO:tasks.ceph.osd.4:Started 2024-09-04T17:45:30.442 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-04T17:45:30.442 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-04T17:45:30.443 INFO:tasks.ceph.osd.5:Started 2024-09-04T17:45:30.444 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-04T17:45:30.444 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-04T17:45:30.445 INFO:tasks.ceph.osd.6:Started 2024-09-04T17:45:30.446 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-04T17:45:30.446 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-04T17:45:30.448 INFO:tasks.ceph.osd.7:Started 2024-09-04T17:45:30.448 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-04T17:45:30.641 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:45:30.641 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":10,"fsid":"f36b6ce2-8c72-4c99-8607-5c53c38ef67b","created":"2024-09-04T17:45:28.072591+0000","modified":"2024-09-04T17:45:30.411007+0000","last_up_change":"0.000000","last_in_change":"2024-09-04T17:45:30.411007+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"e58efca2-bc5a-4bc4-b3d7-4036ebd9dc63","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"36e03b64-2602-4143-bdd5-f7355de255fe","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"581ab71e-be04-4031-a0c4-f7e40b74b7c7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"5f5d096e-fab2-41fe-b8dd-943d1c40894c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"840951a5-e13b-462c-8c8e-2ca61a53fe3c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"82e556e5-00c3-46fc-aab5-3544bba279d1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"4e1e3d67-3020-470d-98b7-891fdc966238","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"bd9c9d3e-27db-4bc2-b998-7f446576a543","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-04T17:45:30.656 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-04T17:45:30.657 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-04T17:45:31.136 INFO:tasks.ceph.mgr.y.smithi092.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-04T17:45:31.137 INFO:tasks.ceph.mgr.y.smithi092.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-04T17:45:31.137 INFO:tasks.ceph.mgr.y.smithi092.stderr: from numpy import show_config as show_numpy_config 2024-09-04T17:45:31.194 INFO:tasks.ceph.mgr.x.smithi190.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-04T17:45:31.194 INFO:tasks.ceph.mgr.x.smithi190.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-04T17:45:31.195 INFO:tasks.ceph.mgr.x.smithi190.stderr: from numpy import show_config as show_numpy_config 2024-09-04T17:45:31.463 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-04T17:45:31.565 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:45:31.564+0000 7fb8fe2477c0 -1 Falling back to public interface 2024-09-04T17:45:31.636 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:45:31.634+0000 7f676502d7c0 -1 Falling back to public interface 2024-09-04T17:45:31.644 INFO:teuthology.misc.health.smithi092.stdout: 2024-09-04T17:45:31.645 INFO:teuthology.misc.health.smithi092.stdout:{"epoch":10,"fsid":"f36b6ce2-8c72-4c99-8607-5c53c38ef67b","created":"2024-09-04T17:45:28.072591+0000","modified":"2024-09-04T17:45:30.411007+0000","last_up_change":"0.000000","last_in_change":"2024-09-04T17:45:30.411007+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"e58efca2-bc5a-4bc4-b3d7-4036ebd9dc63","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"36e03b64-2602-4143-bdd5-f7355de255fe","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"581ab71e-be04-4031-a0c4-f7e40b74b7c7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"5f5d096e-fab2-41fe-b8dd-943d1c40894c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"840951a5-e13b-462c-8c8e-2ca61a53fe3c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"82e556e5-00c3-46fc-aab5-3544bba279d1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"4e1e3d67-3020-470d-98b7-891fdc966238","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"bd9c9d3e-27db-4bc2-b998-7f446576a543","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-04T17:45:31.660 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-04T17:45:32.114 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:45:32.112+0000 7feca42937c0 -1 Falling back to public interface 2024-09-04T17:45:32.125 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:45:32.124+0000 7f23545557c0 -1 Falling back to public interface 2024-09-04T17:45:32.148 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:45:32.136+0000 7f50606867c0 -1 Falling back to public interface 2024-09-04T17:45:32.152 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:45:32.150+0000 7ff58d8db7c0 -1 Falling back to public interface 2024-09-04T17:45:32.160 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:45:32.158+0000 7feeb53e87c0 -1 Falling back to public interface 2024-09-04T17:45:32.364 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:45:32.362+0000 7f31095327c0 -1 Falling back to public interface 2024-09-04T17:45:34.081 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-09-04T17:45:34.080+0000 7fc85ed10640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-04T17:45:34.083 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-09-04T17:45:34.080+0000 7fc85ed10640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-04T17:45:35.565 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:45:35.564+0000 7fb8fe2477c0 -1 osd.7 0 log_to_monitors true 2024-09-04T17:45:35.949 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:45:35.946+0000 7f676502d7c0 -1 osd.2 0 log_to_monitors true 2024-09-04T17:45:36.216 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:45:36.216+0000 7feca42937c0 -1 osd.5 0 log_to_monitors true 2024-09-04T17:45:36.274 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:45:36.270+0000 7feeb53e87c0 -1 osd.3 0 log_to_monitors true 2024-09-04T17:45:36.289 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:45:36.286+0000 7ff58d8db7c0 -1 osd.0 0 log_to_monitors true 2024-09-04T17:45:36.440 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:45:36.440+0000 7f50606867c0 -1 osd.6 0 log_to_monitors true 2024-09-04T17:45:36.668 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:45:36.668+0000 7f23545557c0 -1 osd.4 0 log_to_monitors true 2024-09-04T17:45:36.885 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:45:36.882+0000 7f31095327c0 -1 osd.1 0 log_to_monitors true 2024-09-04T17:45:38.130 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:45:38.126+0000 7f67617e8640 -1 osd.2 0 waiting for initial osdmap 2024-09-04T17:45:38.275 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:45:38.276+0000 7fb8fa1f0640 -1 osd.7 0 waiting for initial osdmap 2024-09-04T17:45:38.319 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:45:38.318+0000 7f6758e08640 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-04T17:45:38.336 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:45:38.336+0000 7fb8f2022640 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-04T17:45:38.465 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-04T17:45:38.788 INFO:teuthology.misc.health.smithi092.stdout: 2024-09-04T17:45:38.788 INFO:teuthology.misc.health.smithi092.stdout:{"epoch":13,"fsid":"f36b6ce2-8c72-4c99-8607-5c53c38ef67b","created":"2024-09-04T17:45:28.072591+0000","modified":"2024-09-04T17:45:38.110706+0000","last_up_change":"0.000000","last_in_change":"2024-09-04T17:45:30.411007+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"e58efca2-bc5a-4bc4-b3d7-4036ebd9dc63","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"36e03b64-2602-4143-bdd5-f7355de255fe","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"581ab71e-be04-4031-a0c4-f7e40b74b7c7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"5f5d096e-fab2-41fe-b8dd-943d1c40894c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"840951a5-e13b-462c-8c8e-2ca61a53fe3c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"82e556e5-00c3-46fc-aab5-3544bba279d1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"4e1e3d67-3020-470d-98b7-891fdc966238","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"bd9c9d3e-27db-4bc2-b998-7f446576a543","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-04T17:45:38.808 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-04T17:45:39.787 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:45:39.788+0000 7f23504fe640 -1 osd.4 0 waiting for initial osdmap 2024-09-04T17:45:39.808 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:45:39.808+0000 7f2348330640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-04T17:45:39.822 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:45:39.820+0000 7feca0a4e640 -1 osd.5 0 waiting for initial osdmap 2024-09-04T17:45:39.836 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:45:39.834+0000 7ff589884640 -1 osd.0 0 waiting for initial osdmap 2024-09-04T17:45:39.844 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:45:39.844+0000 7fec9806e640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-04T17:45:39.859 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:45:39.858+0000 7ff5816b6640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-04T17:45:39.866 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:45:39.864+0000 7f505ce41640 -1 osd.6 0 waiting for initial osdmap 2024-09-04T17:45:39.871 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:45:39.870+0000 7feeb1391640 -1 osd.3 0 waiting for initial osdmap 2024-09-04T17:45:39.879 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:45:39.878+0000 7f31054db640 -1 osd.1 0 waiting for initial osdmap 2024-09-04T17:45:39.888 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:45:39.888+0000 7f5054461640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-04T17:45:39.899 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:45:39.898+0000 7feea91c3640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-04T17:45:39.915 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:45:39.914+0000 7f30fd30d640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-04T17:45:45.615 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-04T17:45:45.926 INFO:teuthology.misc.health.smithi092.stdout: 2024-09-04T17:45:45.926 INFO:teuthology.misc.health.smithi092.stdout:{"epoch":18,"fsid":"f36b6ce2-8c72-4c99-8607-5c53c38ef67b","created":"2024-09-04T17:45:28.072591+0000","modified":"2024-09-04T17:45:43.323983+0000","last_up_change":"2024-09-04T17:45:40.299829+0000","last_in_change":"2024-09-04T17:45:30.411007+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-04T17:45:41.111814+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58efca2-bc5a-4bc4-b3d7-4036ebd9dc63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2135042466},{"type":"v1","addr":"172.21.15.92:6809","nonce":2135042466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":2135042466},{"type":"v1","addr":"172.21.15.92:6811","nonce":2135042466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":2135042466},{"type":"v1","addr":"172.21.15.92:6815","nonce":2135042466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":2135042466},{"type":"v1","addr":"172.21.15.92:6813","nonce":2135042466}]},"public_addr":"172.21.15.92:6809/2135042466","cluster_addr":"172.21.15.92:6811/2135042466","heartbeat_back_addr":"172.21.15.92:6815/2135042466","heartbeat_front_addr":"172.21.15.92:6813/2135042466","state":["exists","up"]},{"osd":1,"uuid":"36e03b64-2602-4143-bdd5-f7355de255fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":1306159585},{"type":"v1","addr":"172.21.15.92:6825","nonce":1306159585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1306159585},{"type":"v1","addr":"172.21.15.92:6827","nonce":1306159585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1306159585},{"type":"v1","addr":"172.21.15.92:6831","nonce":1306159585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1306159585},{"type":"v1","addr":"172.21.15.92:6829","nonce":1306159585}]},"public_addr":"172.21.15.92:6825/1306159585","cluster_addr":"172.21.15.92:6827/1306159585","heartbeat_back_addr":"172.21.15.92:6831/1306159585","heartbeat_front_addr":"172.21.15.92:6829/1306159585","state":["exists","up"]},{"osd":2,"uuid":"581ab71e-be04-4031-a0c4-f7e40b74b7c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":205409741},{"type":"v1","addr":"172.21.15.92:6801","nonce":205409741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":205409741},{"type":"v1","addr":"172.21.15.92:6803","nonce":205409741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":205409741},{"type":"v1","addr":"172.21.15.92:6807","nonce":205409741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":205409741},{"type":"v1","addr":"172.21.15.92:6805","nonce":205409741}]},"public_addr":"172.21.15.92:6801/205409741","cluster_addr":"172.21.15.92:6803/205409741","heartbeat_back_addr":"172.21.15.92:6807/205409741","heartbeat_front_addr":"172.21.15.92:6805/205409741","state":["exists","up"]},{"osd":3,"uuid":"5f5d096e-fab2-41fe-b8dd-943d1c40894c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":3416917175},{"type":"v1","addr":"172.21.15.92:6817","nonce":3416917175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":3416917175},{"type":"v1","addr":"172.21.15.92:6819","nonce":3416917175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":3416917175},{"type":"v1","addr":"172.21.15.92:6823","nonce":3416917175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":3416917175},{"type":"v1","addr":"172.21.15.92:6821","nonce":3416917175}]},"public_addr":"172.21.15.92:6817/3416917175","cluster_addr":"172.21.15.92:6819/3416917175","heartbeat_back_addr":"172.21.15.92:6823/3416917175","heartbeat_front_addr":"172.21.15.92:6821/3416917175","state":["exists","up"]},{"osd":4,"uuid":"840951a5-e13b-462c-8c8e-2ca61a53fe3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4272750248},{"type":"v1","addr":"172.21.15.190:6817","nonce":4272750248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4272750248},{"type":"v1","addr":"172.21.15.190:6819","nonce":4272750248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4272750248},{"type":"v1","addr":"172.21.15.190:6823","nonce":4272750248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":4272750248},{"type":"v1","addr":"172.21.15.190:6821","nonce":4272750248}]},"public_addr":"172.21.15.190:6817/4272750248","cluster_addr":"172.21.15.190:6819/4272750248","heartbeat_back_addr":"172.21.15.190:6823/4272750248","heartbeat_front_addr":"172.21.15.190:6821/4272750248","state":["exists","up"]},{"osd":5,"uuid":"82e556e5-00c3-46fc-aab5-3544bba279d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":1688427632},{"type":"v1","addr":"172.21.15.190:6809","nonce":1688427632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":1688427632},{"type":"v1","addr":"172.21.15.190:6811","nonce":1688427632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":1688427632},{"type":"v1","addr":"172.21.15.190:6815","nonce":1688427632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1688427632},{"type":"v1","addr":"172.21.15.190:6813","nonce":1688427632}]},"public_addr":"172.21.15.190:6809/1688427632","cluster_addr":"172.21.15.190:6811/1688427632","heartbeat_back_addr":"172.21.15.190:6815/1688427632","heartbeat_front_addr":"172.21.15.190:6813/1688427632","state":["exists","up"]},{"osd":6,"uuid":"4e1e3d67-3020-470d-98b7-891fdc966238","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":2117327904},{"type":"v1","addr":"172.21.15.190:6825","nonce":2117327904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":2117327904},{"type":"v1","addr":"172.21.15.190:6827","nonce":2117327904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2117327904},{"type":"v1","addr":"172.21.15.190:6831","nonce":2117327904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2117327904},{"type":"v1","addr":"172.21.15.190:6829","nonce":2117327904}]},"public_addr":"172.21.15.190:6825/2117327904","cluster_addr":"172.21.15.190:6827/2117327904","heartbeat_back_addr":"172.21.15.190:6831/2117327904","heartbeat_front_addr":"172.21.15.190:6829/2117327904","state":["exists","up"]},{"osd":7,"uuid":"bd9c9d3e-27db-4bc2-b998-7f446576a543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":4006671087},{"type":"v1","addr":"172.21.15.190:6801","nonce":4006671087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":4006671087},{"type":"v1","addr":"172.21.15.190:6803","nonce":4006671087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":4006671087},{"type":"v1","addr":"172.21.15.190:6807","nonce":4006671087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":4006671087},{"type":"v1","addr":"172.21.15.190:6805","nonce":4006671087}]},"public_addr":"172.21.15.190:6801/4006671087","cluster_addr":"172.21.15.190:6803/4006671087","heartbeat_back_addr":"172.21.15.190:6807/4006671087","heartbeat_front_addr":"172.21.15.190:6805/4006671087","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-04T17:45:37.324122+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-04T17:45:37.856638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-04T17:45:36.966684+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-04T17:45:37.266271+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-04T17:45:37.627069+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-04T17:45:37.241115+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-04T17:45:37.397375+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-04T17:45:36.619385+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-04T17:45:45.944 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-04T17:45:45.944 INFO:tasks.ceph:Creating RBD pool 2024-09-04T17:45:45.944 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-04T17:45:46.349 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'rbd' created 2024-09-04T17:45:46.370 DEBUG:teuthology.orchestra.run.smithi092:> rbd --cluster ceph pool init rbd 2024-09-04T17:45:49.433 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-04T17:45:49.434 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-04T17:45:49.435 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-04T17:45:49.758 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:45:49.777 INFO:teuthology.orchestra.run.smithi092.stdout:[{"version":9,"timestamp":"2024-09-04T17:45:39.884230+0000","name":"","changes":[{"name":"osd.1/osd_mclock_max_capacity_iops_ssd","new_value":"5343.096401"}]}] 2024-09-04T17:45:49.777 INFO:tasks.ceph_manager:config epoch is 9 2024-09-04T17:45:49.777 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-04T17:45:49.777 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-04T17:45:49.777 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-04T17:45:50.131 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:45:50.147 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":5,"flags":0,"active_gid":4108,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3923854027},{"type":"v1","addr":"172.21.15.190:6833","nonce":3923854027}]},"active_addr":"172.21.15.190:6833/3923854027","active_change":"2024-09-04T17:45:33.057148+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":4104,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.190:0","nonce":2190254873}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.190:0","nonce":3705085728}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.190:0","nonce":1665058527}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.190:0","nonce":3710538976}]}]} 2024-09-04T17:45:50.152 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-04T17:45:50.152 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-04T17:45:50.152 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-04T17:45:50.460 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:45:50.460 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":22,"fsid":"f36b6ce2-8c72-4c99-8607-5c53c38ef67b","created":"2024-09-04T17:45:28.072591+0000","modified":"2024-09-04T17:45:49.384196+0000","last_up_change":"2024-09-04T17:45:40.299829+0000","last_in_change":"2024-09-04T17:45:30.411007+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-04T17:45:41.111814+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-04T17:45:46.265395+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58efca2-bc5a-4bc4-b3d7-4036ebd9dc63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2135042466},{"type":"v1","addr":"172.21.15.92:6809","nonce":2135042466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":2135042466},{"type":"v1","addr":"172.21.15.92:6811","nonce":2135042466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":2135042466},{"type":"v1","addr":"172.21.15.92:6815","nonce":2135042466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":2135042466},{"type":"v1","addr":"172.21.15.92:6813","nonce":2135042466}]},"public_addr":"172.21.15.92:6809/2135042466","cluster_addr":"172.21.15.92:6811/2135042466","heartbeat_back_addr":"172.21.15.92:6815/2135042466","heartbeat_front_addr":"172.21.15.92:6813/2135042466","state":["exists","up"]},{"osd":1,"uuid":"36e03b64-2602-4143-bdd5-f7355de255fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":1306159585},{"type":"v1","addr":"172.21.15.92:6825","nonce":1306159585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1306159585},{"type":"v1","addr":"172.21.15.92:6827","nonce":1306159585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1306159585},{"type":"v1","addr":"172.21.15.92:6831","nonce":1306159585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1306159585},{"type":"v1","addr":"172.21.15.92:6829","nonce":1306159585}]},"public_addr":"172.21.15.92:6825/1306159585","cluster_addr":"172.21.15.92:6827/1306159585","heartbeat_back_addr":"172.21.15.92:6831/1306159585","heartbeat_front_addr":"172.21.15.92:6829/1306159585","state":["exists","up"]},{"osd":2,"uuid":"581ab71e-be04-4031-a0c4-f7e40b74b7c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":205409741},{"type":"v1","addr":"172.21.15.92:6801","nonce":205409741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":205409741},{"type":"v1","addr":"172.21.15.92:6803","nonce":205409741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":205409741},{"type":"v1","addr":"172.21.15.92:6807","nonce":205409741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":205409741},{"type":"v1","addr":"172.21.15.92:6805","nonce":205409741}]},"public_addr":"172.21.15.92:6801/205409741","cluster_addr":"172.21.15.92:6803/205409741","heartbeat_back_addr":"172.21.15.92:6807/205409741","heartbeat_front_addr":"172.21.15.92:6805/205409741","state":["exists","up"]},{"osd":3,"uuid":"5f5d096e-fab2-41fe-b8dd-943d1c40894c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":3416917175},{"type":"v1","addr":"172.21.15.92:6817","nonce":3416917175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":3416917175},{"type":"v1","addr":"172.21.15.92:6819","nonce":3416917175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":3416917175},{"type":"v1","addr":"172.21.15.92:6823","nonce":3416917175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":3416917175},{"type":"v1","addr":"172.21.15.92:6821","nonce":3416917175}]},"public_addr":"172.21.15.92:6817/3416917175","cluster_addr":"172.21.15.92:6819/3416917175","heartbeat_back_addr":"172.21.15.92:6823/3416917175","heartbeat_front_addr":"172.21.15.92:6821/3416917175","state":["exists","up"]},{"osd":4,"uuid":"840951a5-e13b-462c-8c8e-2ca61a53fe3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4272750248},{"type":"v1","addr":"172.21.15.190:6817","nonce":4272750248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4272750248},{"type":"v1","addr":"172.21.15.190:6819","nonce":4272750248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4272750248},{"type":"v1","addr":"172.21.15.190:6823","nonce":4272750248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":4272750248},{"type":"v1","addr":"172.21.15.190:6821","nonce":4272750248}]},"public_addr":"172.21.15.190:6817/4272750248","cluster_addr":"172.21.15.190:6819/4272750248","heartbeat_back_addr":"172.21.15.190:6823/4272750248","heartbeat_front_addr":"172.21.15.190:6821/4272750248","state":["exists","up"]},{"osd":5,"uuid":"82e556e5-00c3-46fc-aab5-3544bba279d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":1688427632},{"type":"v1","addr":"172.21.15.190:6809","nonce":1688427632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":1688427632},{"type":"v1","addr":"172.21.15.190:6811","nonce":1688427632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":1688427632},{"type":"v1","addr":"172.21.15.190:6815","nonce":1688427632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1688427632},{"type":"v1","addr":"172.21.15.190:6813","nonce":1688427632}]},"public_addr":"172.21.15.190:6809/1688427632","cluster_addr":"172.21.15.190:6811/1688427632","heartbeat_back_addr":"172.21.15.190:6815/1688427632","heartbeat_front_addr":"172.21.15.190:6813/1688427632","state":["exists","up"]},{"osd":6,"uuid":"4e1e3d67-3020-470d-98b7-891fdc966238","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":2117327904},{"type":"v1","addr":"172.21.15.190:6825","nonce":2117327904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":2117327904},{"type":"v1","addr":"172.21.15.190:6827","nonce":2117327904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2117327904},{"type":"v1","addr":"172.21.15.190:6831","nonce":2117327904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2117327904},{"type":"v1","addr":"172.21.15.190:6829","nonce":2117327904}]},"public_addr":"172.21.15.190:6825/2117327904","cluster_addr":"172.21.15.190:6827/2117327904","heartbeat_back_addr":"172.21.15.190:6831/2117327904","heartbeat_front_addr":"172.21.15.190:6829/2117327904","state":["exists","up"]},{"osd":7,"uuid":"bd9c9d3e-27db-4bc2-b998-7f446576a543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":4006671087},{"type":"v1","addr":"172.21.15.190:6801","nonce":4006671087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":4006671087},{"type":"v1","addr":"172.21.15.190:6803","nonce":4006671087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":4006671087},{"type":"v1","addr":"172.21.15.190:6807","nonce":4006671087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":4006671087},{"type":"v1","addr":"172.21.15.190:6805","nonce":4006671087}]},"public_addr":"172.21.15.190:6801/4006671087","cluster_addr":"172.21.15.190:6803/4006671087","heartbeat_back_addr":"172.21.15.190:6807/4006671087","heartbeat_front_addr":"172.21.15.190:6805/4006671087","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-04T17:45:37.324122+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-04T17:45:37.856638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-04T17:45:36.966684+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-04T17:45:37.266271+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-04T17:45:37.627069+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-04T17:45:37.241115+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-04T17:45:37.397375+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-04T17:45:36.619385+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-04T17:45:50.477 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-04T17:45:50.478 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-04T17:45:50.766 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:45:50.766 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":22,"fsid":"f36b6ce2-8c72-4c99-8607-5c53c38ef67b","created":"2024-09-04T17:45:28.072591+0000","modified":"2024-09-04T17:45:49.384196+0000","last_up_change":"2024-09-04T17:45:40.299829+0000","last_in_change":"2024-09-04T17:45:30.411007+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-04T17:45:41.111814+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-04T17:45:46.265395+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58efca2-bc5a-4bc4-b3d7-4036ebd9dc63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2135042466},{"type":"v1","addr":"172.21.15.92:6809","nonce":2135042466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":2135042466},{"type":"v1","addr":"172.21.15.92:6811","nonce":2135042466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":2135042466},{"type":"v1","addr":"172.21.15.92:6815","nonce":2135042466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":2135042466},{"type":"v1","addr":"172.21.15.92:6813","nonce":2135042466}]},"public_addr":"172.21.15.92:6809/2135042466","cluster_addr":"172.21.15.92:6811/2135042466","heartbeat_back_addr":"172.21.15.92:6815/2135042466","heartbeat_front_addr":"172.21.15.92:6813/2135042466","state":["exists","up"]},{"osd":1,"uuid":"36e03b64-2602-4143-bdd5-f7355de255fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":1306159585},{"type":"v1","addr":"172.21.15.92:6825","nonce":1306159585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1306159585},{"type":"v1","addr":"172.21.15.92:6827","nonce":1306159585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1306159585},{"type":"v1","addr":"172.21.15.92:6831","nonce":1306159585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1306159585},{"type":"v1","addr":"172.21.15.92:6829","nonce":1306159585}]},"public_addr":"172.21.15.92:6825/1306159585","cluster_addr":"172.21.15.92:6827/1306159585","heartbeat_back_addr":"172.21.15.92:6831/1306159585","heartbeat_front_addr":"172.21.15.92:6829/1306159585","state":["exists","up"]},{"osd":2,"uuid":"581ab71e-be04-4031-a0c4-f7e40b74b7c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":205409741},{"type":"v1","addr":"172.21.15.92:6801","nonce":205409741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":205409741},{"type":"v1","addr":"172.21.15.92:6803","nonce":205409741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":205409741},{"type":"v1","addr":"172.21.15.92:6807","nonce":205409741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":205409741},{"type":"v1","addr":"172.21.15.92:6805","nonce":205409741}]},"public_addr":"172.21.15.92:6801/205409741","cluster_addr":"172.21.15.92:6803/205409741","heartbeat_back_addr":"172.21.15.92:6807/205409741","heartbeat_front_addr":"172.21.15.92:6805/205409741","state":["exists","up"]},{"osd":3,"uuid":"5f5d096e-fab2-41fe-b8dd-943d1c40894c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":3416917175},{"type":"v1","addr":"172.21.15.92:6817","nonce":3416917175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":3416917175},{"type":"v1","addr":"172.21.15.92:6819","nonce":3416917175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":3416917175},{"type":"v1","addr":"172.21.15.92:6823","nonce":3416917175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":3416917175},{"type":"v1","addr":"172.21.15.92:6821","nonce":3416917175}]},"public_addr":"172.21.15.92:6817/3416917175","cluster_addr":"172.21.15.92:6819/3416917175","heartbeat_back_addr":"172.21.15.92:6823/3416917175","heartbeat_front_addr":"172.21.15.92:6821/3416917175","state":["exists","up"]},{"osd":4,"uuid":"840951a5-e13b-462c-8c8e-2ca61a53fe3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4272750248},{"type":"v1","addr":"172.21.15.190:6817","nonce":4272750248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4272750248},{"type":"v1","addr":"172.21.15.190:6819","nonce":4272750248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4272750248},{"type":"v1","addr":"172.21.15.190:6823","nonce":4272750248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":4272750248},{"type":"v1","addr":"172.21.15.190:6821","nonce":4272750248}]},"public_addr":"172.21.15.190:6817/4272750248","cluster_addr":"172.21.15.190:6819/4272750248","heartbeat_back_addr":"172.21.15.190:6823/4272750248","heartbeat_front_addr":"172.21.15.190:6821/4272750248","state":["exists","up"]},{"osd":5,"uuid":"82e556e5-00c3-46fc-aab5-3544bba279d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":1688427632},{"type":"v1","addr":"172.21.15.190:6809","nonce":1688427632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":1688427632},{"type":"v1","addr":"172.21.15.190:6811","nonce":1688427632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":1688427632},{"type":"v1","addr":"172.21.15.190:6815","nonce":1688427632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1688427632},{"type":"v1","addr":"172.21.15.190:6813","nonce":1688427632}]},"public_addr":"172.21.15.190:6809/1688427632","cluster_addr":"172.21.15.190:6811/1688427632","heartbeat_back_addr":"172.21.15.190:6815/1688427632","heartbeat_front_addr":"172.21.15.190:6813/1688427632","state":["exists","up"]},{"osd":6,"uuid":"4e1e3d67-3020-470d-98b7-891fdc966238","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":2117327904},{"type":"v1","addr":"172.21.15.190:6825","nonce":2117327904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":2117327904},{"type":"v1","addr":"172.21.15.190:6827","nonce":2117327904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2117327904},{"type":"v1","addr":"172.21.15.190:6831","nonce":2117327904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2117327904},{"type":"v1","addr":"172.21.15.190:6829","nonce":2117327904}]},"public_addr":"172.21.15.190:6825/2117327904","cluster_addr":"172.21.15.190:6827/2117327904","heartbeat_back_addr":"172.21.15.190:6831/2117327904","heartbeat_front_addr":"172.21.15.190:6829/2117327904","state":["exists","up"]},{"osd":7,"uuid":"bd9c9d3e-27db-4bc2-b998-7f446576a543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":4006671087},{"type":"v1","addr":"172.21.15.190:6801","nonce":4006671087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":4006671087},{"type":"v1","addr":"172.21.15.190:6803","nonce":4006671087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":4006671087},{"type":"v1","addr":"172.21.15.190:6807","nonce":4006671087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":4006671087},{"type":"v1","addr":"172.21.15.190:6805","nonce":4006671087}]},"public_addr":"172.21.15.190:6801/4006671087","cluster_addr":"172.21.15.190:6803/4006671087","heartbeat_back_addr":"172.21.15.190:6807/4006671087","heartbeat_front_addr":"172.21.15.190:6805/4006671087","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-04T17:45:37.324122+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-04T17:45:37.856638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-04T17:45:36.966684+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-04T17:45:37.266271+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-04T17:45:37.627069+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-04T17:45:37.241115+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-04T17:45:37.397375+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-04T17:45:36.619385+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-04T17:45:50.784 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-04T17:45:50.784 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-04T17:45:50.785 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-04T17:45:50.785 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-04T17:45:50.785 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-04T17:45:50.786 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-04T17:45:50.786 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-04T17:45:50.787 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-04T17:45:51.044 INFO:teuthology.orchestra.run.smithi092.stdout:64424509445 2024-09-04T17:45:51.045 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-04T17:45:51.046 INFO:teuthology.orchestra.run.smithi092.stdout:60129542149 2024-09-04T17:45:51.046 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-04T17:45:51.047 INFO:teuthology.orchestra.run.smithi092.stdout:64424509444 2024-09-04T17:45:51.048 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-04T17:45:51.052 INFO:teuthology.orchestra.run.smithi092.stdout:60129542150 2024-09-04T17:45:51.052 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-04T17:45:51.054 INFO:teuthology.orchestra.run.smithi092.stdout:64424509444 2024-09-04T17:45:51.054 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-04T17:45:51.056 INFO:teuthology.orchestra.run.smithi092.stdout:64424509444 2024-09-04T17:45:51.056 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-04T17:45:51.058 INFO:teuthology.orchestra.run.smithi092.stdout:64424509444 2024-09-04T17:45:51.058 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-04T17:45:51.067 INFO:teuthology.orchestra.run.smithi092.stdout:64424509444 2024-09-04T17:45:51.067 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-04T17:45:51.565 INFO:teuthology.orchestra.run.smithi092.stdout:64424509445 2024-09-04T17:45:51.589 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509445 for osd.4 2024-09-04T17:45:51.589 DEBUG:teuthology.parallel:result is None 2024-09-04T17:45:51.589 INFO:teuthology.orchestra.run.smithi092.stdout:64424509444 2024-09-04T17:45:51.596 INFO:teuthology.orchestra.run.smithi092.stdout:64424509444 2024-09-04T17:45:51.604 INFO:teuthology.orchestra.run.smithi092.stdout:64424509444 2024-09-04T17:45:51.610 INFO:teuthology.orchestra.run.smithi092.stdout:60129542149 2024-09-04T17:45:51.610 INFO:teuthology.orchestra.run.smithi092.stdout:60129542150 2024-09-04T17:45:51.621 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509444 for osd.1 2024-09-04T17:45:51.622 DEBUG:teuthology.parallel:result is None 2024-09-04T17:45:51.631 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509444 for osd.6 2024-09-04T17:45:51.631 DEBUG:teuthology.parallel:result is None 2024-09-04T17:45:51.633 INFO:teuthology.orchestra.run.smithi092.stdout:64424509444 2024-09-04T17:45:51.634 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.7 2024-09-04T17:45:51.634 DEBUG:teuthology.parallel:result is None 2024-09-04T17:45:51.637 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509444 for osd.0 2024-09-04T17:45:51.637 DEBUG:teuthology.parallel:result is None 2024-09-04T17:45:51.643 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.2 2024-09-04T17:45:51.643 DEBUG:teuthology.parallel:result is None 2024-09-04T17:45:51.643 INFO:teuthology.orchestra.run.smithi092.stdout:64424509444 2024-09-04T17:45:51.652 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509444 for osd.3 2024-09-04T17:45:51.652 DEBUG:teuthology.parallel:result is None 2024-09-04T17:45:51.661 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509444 for osd.5 2024-09-04T17:45:51.661 DEBUG:teuthology.parallel:result is None 2024-09-04T17:45:51.661 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-04T17:45:51.662 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-04T17:45:51.974 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:45:51.974 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-09-04T17:45:51.991 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-04T17:45:51.068821+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":223312,"kb_used_data":2896,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754751408,"statfs":{"total":773094113280,"available":772865441792,"internally_reserved":0,"allocated":2965504,"data_stored":1779334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.715436"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-04T17:45:49.598853+0000","last_change":"2024-09-04T17:45:49.598853+0000","last_active":"2024-09-04T17:45:49.598853+0000","last_peered":"2024-09-04T17:45:49.598853+0000","last_clean":"2024-09-04T17:45:49.598853+0000","last_became_active":"2024-09-04T17:45:47.371940+0000","last_became_peered":"2024-09-04T17:45:47.371940+0000","last_unstale":"2024-09-04T17:45:49.598853+0000","last_undegraded":"2024-09-04T17:45:49.598853+0000","last_fullsized":"2024-09-04T17:45:49.598853+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:47:59.861359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001226074,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-04T17:45:49.398776+0000","last_change":"2024-09-04T17:45:49.398776+0000","last_active":"2024-09-04T17:45:49.398776+0000","last_peered":"2024-09-04T17:45:49.398776+0000","last_clean":"2024-09-04T17:45:49.398776+0000","last_became_active":"2024-09-04T17:45:47.373242+0000","last_became_peered":"2024-09-04T17:45:47.373242+0000","last_unstale":"2024-09-04T17:45:49.398776+0000","last_undegraded":"2024-09-04T17:45:49.398776+0000","last_fullsized":"2024-09-04T17:45:49.398776+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:15:20.867375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011364179999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-04T17:45:49.476607+0000","last_change":"2024-09-04T17:45:49.476607+0000","last_active":"2024-09-04T17:45:49.476607+0000","last_peered":"2024-09-04T17:45:49.476607+0000","last_clean":"2024-09-04T17:45:49.476607+0000","last_became_active":"2024-09-04T17:45:47.369308+0000","last_became_peered":"2024-09-04T17:45:47.369308+0000","last_unstale":"2024-09-04T17:45:49.476607+0000","last_undegraded":"2024-09-04T17:45:49.476607+0000","last_fullsized":"2024-09-04T17:45:49.476607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:17:13.378553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001029826,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-04T17:45:49.398055+0000","last_change":"2024-09-04T17:45:49.398055+0000","last_active":"2024-09-04T17:45:49.398055+0000","last_peered":"2024-09-04T17:45:49.398055+0000","last_clean":"2024-09-04T17:45:49.398055+0000","last_became_active":"2024-09-04T17:45:47.370234+0000","last_became_peered":"2024-09-04T17:45:47.370234+0000","last_unstale":"2024-09-04T17:45:49.398055+0000","last_undegraded":"2024-09-04T17:45:49.398055+0000","last_fullsized":"2024-09-04T17:45:49.398055+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T18:23:34.154001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012026370000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-04T17:45:49.413277+0000","last_change":"2024-09-04T17:45:49.411482+0000","last_active":"2024-09-04T17:45:49.413277+0000","last_peered":"2024-09-04T17:45:49.413277+0000","last_clean":"2024-09-04T17:45:49.413277+0000","last_became_active":"2024-09-04T17:45:47.370178+0000","last_became_peered":"2024-09-04T17:45:47.370178+0000","last_unstale":"2024-09-04T17:45:49.413277+0000","last_undegraded":"2024-09-04T17:45:49.413277+0000","last_fullsized":"2024-09-04T17:45:49.413277+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T17:50:19.387994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026316069999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-04T17:45:49.977652+0000","last_change":"2024-09-04T17:45:49.977652+0000","last_active":"2024-09-04T17:45:49.977652+0000","last_peered":"2024-09-04T17:45:49.977652+0000","last_clean":"2024-09-04T17:45:49.977652+0000","last_became_active":"2024-09-04T17:45:47.369860+0000","last_became_peered":"2024-09-04T17:45:47.369860+0000","last_unstale":"2024-09-04T17:45:49.977652+0000","last_undegraded":"2024-09-04T17:45:49.977652+0000","last_fullsized":"2024-09-04T17:45:49.977652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T22:23:00.044282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001025195,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-04T17:45:49.476793+0000","last_change":"2024-09-04T17:45:49.476793+0000","last_active":"2024-09-04T17:45:49.476793+0000","last_peered":"2024-09-04T17:45:49.476793+0000","last_clean":"2024-09-04T17:45:49.476793+0000","last_became_active":"2024-09-04T17:45:47.371162+0000","last_became_peered":"2024-09-04T17:45:47.371162+0000","last_unstale":"2024-09-04T17:45:49.476793+0000","last_undegraded":"2024-09-04T17:45:49.476793+0000","last_fullsized":"2024-09-04T17:45:49.476793+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T23:44:01.197720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010553590000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-04T17:45:49.409680+0000","last_change":"2024-09-04T17:45:49.409680+0000","last_active":"2024-09-04T17:45:49.409680+0000","last_peered":"2024-09-04T17:45:49.409680+0000","last_clean":"2024-09-04T17:45:49.409680+0000","last_became_active":"2024-09-04T17:45:47.373348+0000","last_became_peered":"2024-09-04T17:45:47.373348+0000","last_unstale":"2024-09-04T17:45:49.409680+0000","last_undegraded":"2024-09-04T17:45:49.409680+0000","last_fullsized":"2024-09-04T17:45:49.409680+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:28:27.174757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086618299999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-04T17:45:49.475341+0000","last_change":"2024-09-04T17:45:42.341149+0000","last_active":"2024-09-04T17:45:49.475341+0000","last_peered":"2024-09-04T17:45:49.475341+0000","last_clean":"2024-09-04T17:45:49.475341+0000","last_became_active":"2024-09-04T17:45:42.339286+0000","last_became_peered":"2024-09-04T17:45:42.339286+0000","last_unstale":"2024-09-04T17:45:49.475341+0000","last_undegraded":"2024-09-04T17:45:49.475341+0000","last_fullsized":"2024-09-04T17:45:49.475341+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:41.305128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:41.305128+0000","last_clean_scrub_stamp":"2024-09-04T17:45:41.305128+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:52:24.860049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509444,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":74820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":225280,"data_stored":74839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":74820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":74820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509444,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":225280,"data_stored":74839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509444,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":815104,"data_stored":665188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31992,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339848,"statfs":{"total":96636764160,"available":96604004352,"internally_reserved":0,"allocated":815104,"data_stored":665188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":221184,"data_stored":74820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-04T17:45:51.992 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-04T17:45:52.269 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:45:52.269 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-09-04T17:45:52.286 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-04T17:45:51.068821+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":223312,"kb_used_data":2896,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754751408,"statfs":{"total":773094113280,"available":772865441792,"internally_reserved":0,"allocated":2965504,"data_stored":1779334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.715436"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-04T17:45:49.598853+0000","last_change":"2024-09-04T17:45:49.598853+0000","last_active":"2024-09-04T17:45:49.598853+0000","last_peered":"2024-09-04T17:45:49.598853+0000","last_clean":"2024-09-04T17:45:49.598853+0000","last_became_active":"2024-09-04T17:45:47.371940+0000","last_became_peered":"2024-09-04T17:45:47.371940+0000","last_unstale":"2024-09-04T17:45:49.598853+0000","last_undegraded":"2024-09-04T17:45:49.598853+0000","last_fullsized":"2024-09-04T17:45:49.598853+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:47:59.861359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001226074,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-04T17:45:49.398776+0000","last_change":"2024-09-04T17:45:49.398776+0000","last_active":"2024-09-04T17:45:49.398776+0000","last_peered":"2024-09-04T17:45:49.398776+0000","last_clean":"2024-09-04T17:45:49.398776+0000","last_became_active":"2024-09-04T17:45:47.373242+0000","last_became_peered":"2024-09-04T17:45:47.373242+0000","last_unstale":"2024-09-04T17:45:49.398776+0000","last_undegraded":"2024-09-04T17:45:49.398776+0000","last_fullsized":"2024-09-04T17:45:49.398776+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:15:20.867375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011364179999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-04T17:45:49.476607+0000","last_change":"2024-09-04T17:45:49.476607+0000","last_active":"2024-09-04T17:45:49.476607+0000","last_peered":"2024-09-04T17:45:49.476607+0000","last_clean":"2024-09-04T17:45:49.476607+0000","last_became_active":"2024-09-04T17:45:47.369308+0000","last_became_peered":"2024-09-04T17:45:47.369308+0000","last_unstale":"2024-09-04T17:45:49.476607+0000","last_undegraded":"2024-09-04T17:45:49.476607+0000","last_fullsized":"2024-09-04T17:45:49.476607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:17:13.378553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001029826,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-04T17:45:49.398055+0000","last_change":"2024-09-04T17:45:49.398055+0000","last_active":"2024-09-04T17:45:49.398055+0000","last_peered":"2024-09-04T17:45:49.398055+0000","last_clean":"2024-09-04T17:45:49.398055+0000","last_became_active":"2024-09-04T17:45:47.370234+0000","last_became_peered":"2024-09-04T17:45:47.370234+0000","last_unstale":"2024-09-04T17:45:49.398055+0000","last_undegraded":"2024-09-04T17:45:49.398055+0000","last_fullsized":"2024-09-04T17:45:49.398055+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T18:23:34.154001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012026370000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-04T17:45:49.413277+0000","last_change":"2024-09-04T17:45:49.411482+0000","last_active":"2024-09-04T17:45:49.413277+0000","last_peered":"2024-09-04T17:45:49.413277+0000","last_clean":"2024-09-04T17:45:49.413277+0000","last_became_active":"2024-09-04T17:45:47.370178+0000","last_became_peered":"2024-09-04T17:45:47.370178+0000","last_unstale":"2024-09-04T17:45:49.413277+0000","last_undegraded":"2024-09-04T17:45:49.413277+0000","last_fullsized":"2024-09-04T17:45:49.413277+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T17:50:19.387994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026316069999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-04T17:45:49.977652+0000","last_change":"2024-09-04T17:45:49.977652+0000","last_active":"2024-09-04T17:45:49.977652+0000","last_peered":"2024-09-04T17:45:49.977652+0000","last_clean":"2024-09-04T17:45:49.977652+0000","last_became_active":"2024-09-04T17:45:47.369860+0000","last_became_peered":"2024-09-04T17:45:47.369860+0000","last_unstale":"2024-09-04T17:45:49.977652+0000","last_undegraded":"2024-09-04T17:45:49.977652+0000","last_fullsized":"2024-09-04T17:45:49.977652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T22:23:00.044282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001025195,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-04T17:45:49.476793+0000","last_change":"2024-09-04T17:45:49.476793+0000","last_active":"2024-09-04T17:45:49.476793+0000","last_peered":"2024-09-04T17:45:49.476793+0000","last_clean":"2024-09-04T17:45:49.476793+0000","last_became_active":"2024-09-04T17:45:47.371162+0000","last_became_peered":"2024-09-04T17:45:47.371162+0000","last_unstale":"2024-09-04T17:45:49.476793+0000","last_undegraded":"2024-09-04T17:45:49.476793+0000","last_fullsized":"2024-09-04T17:45:49.476793+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T23:44:01.197720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010553590000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-04T17:45:49.409680+0000","last_change":"2024-09-04T17:45:49.409680+0000","last_active":"2024-09-04T17:45:49.409680+0000","last_peered":"2024-09-04T17:45:49.409680+0000","last_clean":"2024-09-04T17:45:49.409680+0000","last_became_active":"2024-09-04T17:45:47.373348+0000","last_became_peered":"2024-09-04T17:45:47.373348+0000","last_unstale":"2024-09-04T17:45:49.409680+0000","last_undegraded":"2024-09-04T17:45:49.409680+0000","last_fullsized":"2024-09-04T17:45:49.409680+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:28:27.174757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086618299999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-04T17:45:49.475341+0000","last_change":"2024-09-04T17:45:42.341149+0000","last_active":"2024-09-04T17:45:49.475341+0000","last_peered":"2024-09-04T17:45:49.475341+0000","last_clean":"2024-09-04T17:45:49.475341+0000","last_became_active":"2024-09-04T17:45:42.339286+0000","last_became_peered":"2024-09-04T17:45:42.339286+0000","last_unstale":"2024-09-04T17:45:49.475341+0000","last_undegraded":"2024-09-04T17:45:49.475341+0000","last_fullsized":"2024-09-04T17:45:49.475341+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:41.305128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:41.305128+0000","last_clean_scrub_stamp":"2024-09-04T17:45:41.305128+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:52:24.860049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509444,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":74820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":225280,"data_stored":74839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":74820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":74820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509444,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":225280,"data_stored":74839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509444,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":815104,"data_stored":665188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31992,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339848,"statfs":{"total":96636764160,"available":96604004352,"internally_reserved":0,"allocated":815104,"data_stored":665188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":221184,"data_stored":74820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-04T17:45:52.287 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-04T17:45:52.287 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-04T17:45:52.287 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-04T17:45:52.287 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-04T17:45:52.600 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:45:52.600 INFO:teuthology.orchestra.run.smithi092.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-04T17:45:52.618 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-04T17:45:52.618 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-04T17:45:52.630 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-04T17:45:52.630 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'wip-dis-testing'}, 'ceph': {'conf': {'client': {'rbd qos iops burst': 100, 'rbd qos iops limit': 50, 'rbd qos schedule tick min': 100}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': '3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '3e4a7f4c58c7c6fe936e6fda96ba06459c94d8ab'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'live-migration-nbd', 'sha1': '250c2b7303ac79643ee967bc9c5b14e5a0dbbf52'}} 2024-09-04T17:45:52.630 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-04T17:45:52.631 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-04T17:45:52.940 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-04T17:45:52.940 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-04T17:45:52.940 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-04T17:45:53.232 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:45:53.232 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-09-04T17:45:53.248 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-04T17:45:53.069400+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":223312,"kb_used_data":2896,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754751408,"statfs":{"total":773094113280,"available":772865441792,"internally_reserved":0,"allocated":2965504,"data_stored":1779334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.713409"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-04T17:45:49.598853+0000","last_change":"2024-09-04T17:45:49.598853+0000","last_active":"2024-09-04T17:45:49.598853+0000","last_peered":"2024-09-04T17:45:49.598853+0000","last_clean":"2024-09-04T17:45:49.598853+0000","last_became_active":"2024-09-04T17:45:47.371940+0000","last_became_peered":"2024-09-04T17:45:47.371940+0000","last_unstale":"2024-09-04T17:45:49.598853+0000","last_undegraded":"2024-09-04T17:45:49.598853+0000","last_fullsized":"2024-09-04T17:45:49.598853+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:47:59.861359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001226074,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-04T17:45:49.398776+0000","last_change":"2024-09-04T17:45:49.398776+0000","last_active":"2024-09-04T17:45:49.398776+0000","last_peered":"2024-09-04T17:45:49.398776+0000","last_clean":"2024-09-04T17:45:49.398776+0000","last_became_active":"2024-09-04T17:45:47.373242+0000","last_became_peered":"2024-09-04T17:45:47.373242+0000","last_unstale":"2024-09-04T17:45:49.398776+0000","last_undegraded":"2024-09-04T17:45:49.398776+0000","last_fullsized":"2024-09-04T17:45:49.398776+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:15:20.867375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011364179999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-04T17:45:49.476607+0000","last_change":"2024-09-04T17:45:49.476607+0000","last_active":"2024-09-04T17:45:49.476607+0000","last_peered":"2024-09-04T17:45:49.476607+0000","last_clean":"2024-09-04T17:45:49.476607+0000","last_became_active":"2024-09-04T17:45:47.369308+0000","last_became_peered":"2024-09-04T17:45:47.369308+0000","last_unstale":"2024-09-04T17:45:49.476607+0000","last_undegraded":"2024-09-04T17:45:49.476607+0000","last_fullsized":"2024-09-04T17:45:49.476607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:17:13.378553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001029826,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-04T17:45:49.398055+0000","last_change":"2024-09-04T17:45:49.398055+0000","last_active":"2024-09-04T17:45:49.398055+0000","last_peered":"2024-09-04T17:45:49.398055+0000","last_clean":"2024-09-04T17:45:49.398055+0000","last_became_active":"2024-09-04T17:45:47.370234+0000","last_became_peered":"2024-09-04T17:45:47.370234+0000","last_unstale":"2024-09-04T17:45:49.398055+0000","last_undegraded":"2024-09-04T17:45:49.398055+0000","last_fullsized":"2024-09-04T17:45:49.398055+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T18:23:34.154001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012026370000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-04T17:45:49.413277+0000","last_change":"2024-09-04T17:45:49.411482+0000","last_active":"2024-09-04T17:45:49.413277+0000","last_peered":"2024-09-04T17:45:49.413277+0000","last_clean":"2024-09-04T17:45:49.413277+0000","last_became_active":"2024-09-04T17:45:47.370178+0000","last_became_peered":"2024-09-04T17:45:47.370178+0000","last_unstale":"2024-09-04T17:45:49.413277+0000","last_undegraded":"2024-09-04T17:45:49.413277+0000","last_fullsized":"2024-09-04T17:45:49.413277+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T17:50:19.387994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026316069999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-04T17:45:49.977652+0000","last_change":"2024-09-04T17:45:49.977652+0000","last_active":"2024-09-04T17:45:49.977652+0000","last_peered":"2024-09-04T17:45:49.977652+0000","last_clean":"2024-09-04T17:45:49.977652+0000","last_became_active":"2024-09-04T17:45:47.369860+0000","last_became_peered":"2024-09-04T17:45:47.369860+0000","last_unstale":"2024-09-04T17:45:49.977652+0000","last_undegraded":"2024-09-04T17:45:49.977652+0000","last_fullsized":"2024-09-04T17:45:49.977652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T22:23:00.044282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001025195,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-04T17:45:49.476793+0000","last_change":"2024-09-04T17:45:49.476793+0000","last_active":"2024-09-04T17:45:49.476793+0000","last_peered":"2024-09-04T17:45:49.476793+0000","last_clean":"2024-09-04T17:45:49.476793+0000","last_became_active":"2024-09-04T17:45:47.371162+0000","last_became_peered":"2024-09-04T17:45:47.371162+0000","last_unstale":"2024-09-04T17:45:49.476793+0000","last_undegraded":"2024-09-04T17:45:49.476793+0000","last_fullsized":"2024-09-04T17:45:49.476793+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T23:44:01.197720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010553590000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-04T17:45:49.409680+0000","last_change":"2024-09-04T17:45:49.409680+0000","last_active":"2024-09-04T17:45:49.409680+0000","last_peered":"2024-09-04T17:45:49.409680+0000","last_clean":"2024-09-04T17:45:49.409680+0000","last_became_active":"2024-09-04T17:45:47.373348+0000","last_became_peered":"2024-09-04T17:45:47.373348+0000","last_unstale":"2024-09-04T17:45:49.409680+0000","last_undegraded":"2024-09-04T17:45:49.409680+0000","last_fullsized":"2024-09-04T17:45:49.409680+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:28:27.174757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086618299999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-04T17:45:49.475341+0000","last_change":"2024-09-04T17:45:42.341149+0000","last_active":"2024-09-04T17:45:49.475341+0000","last_peered":"2024-09-04T17:45:49.475341+0000","last_clean":"2024-09-04T17:45:49.475341+0000","last_became_active":"2024-09-04T17:45:42.339286+0000","last_became_peered":"2024-09-04T17:45:42.339286+0000","last_unstale":"2024-09-04T17:45:49.475341+0000","last_undegraded":"2024-09-04T17:45:49.475341+0000","last_fullsized":"2024-09-04T17:45:49.475341+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:41.305128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:41.305128+0000","last_clean_scrub_stamp":"2024-09-04T17:45:41.305128+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:52:24.860049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":74820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":225280,"data_stored":74839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":74820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":74820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":225280,"data_stored":74839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":815104,"data_stored":665188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31992,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339848,"statfs":{"total":96636764160,"available":96604004352,"internally_reserved":0,"allocated":815104,"data_stored":665188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":221184,"data_stored":74820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-04T17:45:53.249 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-04T17:45:53.534 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:45:53.535 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-09-04T17:45:53.550 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-04T17:45:53.069400+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":223312,"kb_used_data":2896,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754751408,"statfs":{"total":773094113280,"available":772865441792,"internally_reserved":0,"allocated":2965504,"data_stored":1779334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.713409"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-04T17:45:49.598853+0000","last_change":"2024-09-04T17:45:49.598853+0000","last_active":"2024-09-04T17:45:49.598853+0000","last_peered":"2024-09-04T17:45:49.598853+0000","last_clean":"2024-09-04T17:45:49.598853+0000","last_became_active":"2024-09-04T17:45:47.371940+0000","last_became_peered":"2024-09-04T17:45:47.371940+0000","last_unstale":"2024-09-04T17:45:49.598853+0000","last_undegraded":"2024-09-04T17:45:49.598853+0000","last_fullsized":"2024-09-04T17:45:49.598853+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:47:59.861359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001226074,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-04T17:45:49.398776+0000","last_change":"2024-09-04T17:45:49.398776+0000","last_active":"2024-09-04T17:45:49.398776+0000","last_peered":"2024-09-04T17:45:49.398776+0000","last_clean":"2024-09-04T17:45:49.398776+0000","last_became_active":"2024-09-04T17:45:47.373242+0000","last_became_peered":"2024-09-04T17:45:47.373242+0000","last_unstale":"2024-09-04T17:45:49.398776+0000","last_undegraded":"2024-09-04T17:45:49.398776+0000","last_fullsized":"2024-09-04T17:45:49.398776+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:15:20.867375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011364179999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-04T17:45:49.476607+0000","last_change":"2024-09-04T17:45:49.476607+0000","last_active":"2024-09-04T17:45:49.476607+0000","last_peered":"2024-09-04T17:45:49.476607+0000","last_clean":"2024-09-04T17:45:49.476607+0000","last_became_active":"2024-09-04T17:45:47.369308+0000","last_became_peered":"2024-09-04T17:45:47.369308+0000","last_unstale":"2024-09-04T17:45:49.476607+0000","last_undegraded":"2024-09-04T17:45:49.476607+0000","last_fullsized":"2024-09-04T17:45:49.476607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:17:13.378553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001029826,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-04T17:45:49.398055+0000","last_change":"2024-09-04T17:45:49.398055+0000","last_active":"2024-09-04T17:45:49.398055+0000","last_peered":"2024-09-04T17:45:49.398055+0000","last_clean":"2024-09-04T17:45:49.398055+0000","last_became_active":"2024-09-04T17:45:47.370234+0000","last_became_peered":"2024-09-04T17:45:47.370234+0000","last_unstale":"2024-09-04T17:45:49.398055+0000","last_undegraded":"2024-09-04T17:45:49.398055+0000","last_fullsized":"2024-09-04T17:45:49.398055+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T18:23:34.154001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012026370000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-04T17:45:49.413277+0000","last_change":"2024-09-04T17:45:49.411482+0000","last_active":"2024-09-04T17:45:49.413277+0000","last_peered":"2024-09-04T17:45:49.413277+0000","last_clean":"2024-09-04T17:45:49.413277+0000","last_became_active":"2024-09-04T17:45:47.370178+0000","last_became_peered":"2024-09-04T17:45:47.370178+0000","last_unstale":"2024-09-04T17:45:49.413277+0000","last_undegraded":"2024-09-04T17:45:49.413277+0000","last_fullsized":"2024-09-04T17:45:49.413277+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T17:50:19.387994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026316069999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-04T17:45:49.977652+0000","last_change":"2024-09-04T17:45:49.977652+0000","last_active":"2024-09-04T17:45:49.977652+0000","last_peered":"2024-09-04T17:45:49.977652+0000","last_clean":"2024-09-04T17:45:49.977652+0000","last_became_active":"2024-09-04T17:45:47.369860+0000","last_became_peered":"2024-09-04T17:45:47.369860+0000","last_unstale":"2024-09-04T17:45:49.977652+0000","last_undegraded":"2024-09-04T17:45:49.977652+0000","last_fullsized":"2024-09-04T17:45:49.977652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T22:23:00.044282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001025195,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-04T17:45:49.476793+0000","last_change":"2024-09-04T17:45:49.476793+0000","last_active":"2024-09-04T17:45:49.476793+0000","last_peered":"2024-09-04T17:45:49.476793+0000","last_clean":"2024-09-04T17:45:49.476793+0000","last_became_active":"2024-09-04T17:45:47.371162+0000","last_became_peered":"2024-09-04T17:45:47.371162+0000","last_unstale":"2024-09-04T17:45:49.476793+0000","last_undegraded":"2024-09-04T17:45:49.476793+0000","last_fullsized":"2024-09-04T17:45:49.476793+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T23:44:01.197720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010553590000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-04T17:45:49.409680+0000","last_change":"2024-09-04T17:45:49.409680+0000","last_active":"2024-09-04T17:45:49.409680+0000","last_peered":"2024-09-04T17:45:49.409680+0000","last_clean":"2024-09-04T17:45:49.409680+0000","last_became_active":"2024-09-04T17:45:47.373348+0000","last_became_peered":"2024-09-04T17:45:47.373348+0000","last_unstale":"2024-09-04T17:45:49.409680+0000","last_undegraded":"2024-09-04T17:45:49.409680+0000","last_fullsized":"2024-09-04T17:45:49.409680+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:28:27.174757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086618299999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-04T17:45:49.475341+0000","last_change":"2024-09-04T17:45:42.341149+0000","last_active":"2024-09-04T17:45:49.475341+0000","last_peered":"2024-09-04T17:45:49.475341+0000","last_clean":"2024-09-04T17:45:49.475341+0000","last_became_active":"2024-09-04T17:45:42.339286+0000","last_became_peered":"2024-09-04T17:45:42.339286+0000","last_unstale":"2024-09-04T17:45:49.475341+0000","last_undegraded":"2024-09-04T17:45:49.475341+0000","last_fullsized":"2024-09-04T17:45:49.475341+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:41.305128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:41.305128+0000","last_clean_scrub_stamp":"2024-09-04T17:45:41.305128+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:52:24.860049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":74820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":225280,"data_stored":74839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":74820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":74820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":225280,"data_stored":74839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":815104,"data_stored":665188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31992,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339848,"statfs":{"total":96636764160,"available":96604004352,"internally_reserved":0,"allocated":815104,"data_stored":665188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":221184,"data_stored":74820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-04T17:45:53.551 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-04T17:45:53.551 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-04T17:45:53.831 INFO:teuthology.orchestra.run.smithi092.stdout:epoch 23 2024-09-04T17:45:53.831 INFO:teuthology.orchestra.run.smithi092.stdout:fsid f36b6ce2-8c72-4c99-8607-5c53c38ef67b 2024-09-04T17:45:53.831 INFO:teuthology.orchestra.run.smithi092.stdout:created 2024-09-04T17:45:28.072591+0000 2024-09-04T17:45:53.831 INFO:teuthology.orchestra.run.smithi092.stdout:modified 2024-09-04T17:45:52.795548+0000 2024-09-04T17:45:53.831 INFO:teuthology.orchestra.run.smithi092.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-04T17:45:53.831 INFO:teuthology.orchestra.run.smithi092.stdout:crush_version 5 2024-09-04T17:45:53.831 INFO:teuthology.orchestra.run.smithi092.stdout:full_ratio 0.95 2024-09-04T17:45:53.831 INFO:teuthology.orchestra.run.smithi092.stdout:backfillfull_ratio 0.9 2024-09-04T17:45:53.831 INFO:teuthology.orchestra.run.smithi092.stdout:nearfull_ratio 0.85 2024-09-04T17:45:53.831 INFO:teuthology.orchestra.run.smithi092.stdout:require_min_compat_client luminous 2024-09-04T17:45:53.831 INFO:teuthology.orchestra.run.smithi092.stdout:min_compat_client jewel 2024-09-04T17:45:53.831 INFO:teuthology.orchestra.run.smithi092.stdout:require_osd_release squid 2024-09-04T17:45:53.832 INFO:teuthology.orchestra.run.smithi092.stdout:stretch_mode_enabled false 2024-09-04T17:45:53.832 INFO:teuthology.orchestra.run.smithi092.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 18 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 8.00 2024-09-04T17:45:53.832 INFO:teuthology.orchestra.run.smithi092.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 22 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 1.99 2024-09-04T17:45:53.832 INFO:teuthology.orchestra.run.smithi092.stdout:max_osd 8 2024-09-04T17:45:53.832 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6808/2135042466,v1:172.21.15.92:6809/2135042466] [v2:172.21.15.92:6810/2135042466,v1:172.21.15.92:6811/2135042466] exists,up e58efca2-bc5a-4bc4-b3d7-4036ebd9dc63 2024-09-04T17:45:53.832 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1 up in weight 1 up_from 15 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6824/1306159585,v1:172.21.15.92:6825/1306159585] [v2:172.21.15.92:6826/1306159585,v1:172.21.15.92:6827/1306159585] exists,up 36e03b64-2602-4143-bdd5-f7355de255fe 2024-09-04T17:45:53.832 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6800/205409741,v1:172.21.15.92:6801/205409741] [v2:172.21.15.92:6802/205409741,v1:172.21.15.92:6803/205409741] exists,up 581ab71e-be04-4031-a0c4-f7e40b74b7c7 2024-09-04T17:45:53.832 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6816/3416917175,v1:172.21.15.92:6817/3416917175] [v2:172.21.15.92:6818/3416917175,v1:172.21.15.92:6819/3416917175] exists,up 5f5d096e-fab2-41fe-b8dd-943d1c40894c 2024-09-04T17:45:53.832 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.190:6816/4272750248,v1:172.21.15.190:6817/4272750248] [v2:172.21.15.190:6818/4272750248,v1:172.21.15.190:6819/4272750248] exists,up 840951a5-e13b-462c-8c8e-2ca61a53fe3c 2024-09-04T17:45:53.832 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5 up in weight 1 up_from 15 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.190:6808/1688427632,v1:172.21.15.190:6809/1688427632] [v2:172.21.15.190:6810/1688427632,v1:172.21.15.190:6811/1688427632] exists,up 82e556e5-00c3-46fc-aab5-3544bba279d1 2024-09-04T17:45:53.832 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6 up in weight 1 up_from 15 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.190:6824/2117327904,v1:172.21.15.190:6825/2117327904] [v2:172.21.15.190:6826/2117327904,v1:172.21.15.190:6827/2117327904] exists,up 4e1e3d67-3020-470d-98b7-891fdc966238 2024-09-04T17:45:53.832 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.190:6800/4006671087,v1:172.21.15.190:6801/4006671087] [v2:172.21.15.190:6802/4006671087,v1:172.21.15.190:6803/4006671087] exists,up bd9c9d3e-27db-4bc2-b998-7f446576a543 2024-09-04T17:45:53.848 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6808/2135042466,v1:172.21.15.92:6809/2135042466] [v2:172.21.15.92:6810/2135042466,v1:172.21.15.92:6811/2135042466] exists,up e58efca2-bc5a-4bc4-b3d7-4036ebd9dc63', 'osd.1 up in weight 1 up_from 15 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6824/1306159585,v1:172.21.15.92:6825/1306159585] [v2:172.21.15.92:6826/1306159585,v1:172.21.15.92:6827/1306159585] exists,up 36e03b64-2602-4143-bdd5-f7355de255fe', 'osd.2 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6800/205409741,v1:172.21.15.92:6801/205409741] [v2:172.21.15.92:6802/205409741,v1:172.21.15.92:6803/205409741] exists,up 581ab71e-be04-4031-a0c4-f7e40b74b7c7', 'osd.3 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6816/3416917175,v1:172.21.15.92:6817/3416917175] [v2:172.21.15.92:6818/3416917175,v1:172.21.15.92:6819/3416917175] exists,up 5f5d096e-fab2-41fe-b8dd-943d1c40894c', 'osd.4 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.190:6816/4272750248,v1:172.21.15.190:6817/4272750248] [v2:172.21.15.190:6818/4272750248,v1:172.21.15.190:6819/4272750248] exists,up 840951a5-e13b-462c-8c8e-2ca61a53fe3c', 'osd.5 up in weight 1 up_from 15 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.190:6808/1688427632,v1:172.21.15.190:6809/1688427632] [v2:172.21.15.190:6810/1688427632,v1:172.21.15.190:6811/1688427632] exists,up 82e556e5-00c3-46fc-aab5-3544bba279d1', 'osd.6 up in weight 1 up_from 15 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.190:6824/2117327904,v1:172.21.15.190:6825/2117327904] [v2:172.21.15.190:6826/2117327904,v1:172.21.15.190:6827/2117327904] exists,up 4e1e3d67-3020-470d-98b7-891fdc966238', 'osd.7 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.190:6800/4006671087,v1:172.21.15.190:6801/4006671087] [v2:172.21.15.190:6802/4006671087,v1:172.21.15.190:6803/4006671087] exists,up bd9c9d3e-27db-4bc2-b998-7f446576a543'] 2024-09-04T17:45:53.848 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-04T17:45:53.964 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:45:53.964 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "mon.a", 2024-09-04T17:45:53.964 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster": "ceph", 2024-09-04T17:45:53.965 INFO:teuthology.orchestra.run.smithi092.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-04T17:45:53.965 INFO:teuthology.orchestra.run.smithi092.stdout: "admin_socket_mode": "", 2024-09-04T17:45:53.965 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-04T17:45:53.965 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_client_required": "cephx, none", 2024-09-04T17:45:53.965 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_cluster_required": "cephx", 2024-09-04T17:45:53.965 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_debug": "true", 2024-09-04T17:45:53.965 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-04T17:45:53.965 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-04T17:45:53.965 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_service_required": "cephx", 2024-09-04T17:45:53.965 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-04T17:45:53.965 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_supported": "cephx", 2024-09-04T17:45:53.965 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_aio": "true", 2024-09-04T17:45:53.965 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-04T17:45:53.966 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_aio_poll_ms": "250", 2024-09-04T17:45:53.966 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_aio_reap_max": "16", 2024-09-04T17:45:53.966 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_async_discard_threads": "0", 2024-09-04T17:45:53.966 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_block_size": "4096", 2024-09-04T17:45:53.966 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_debug_aio": "false", 2024-09-04T17:45:53.966 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-04T17:45:53.966 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-04T17:45:53.966 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_debug_inflight_ios": "false", 2024-09-04T17:45:53.966 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_enable_discard": "false", 2024-09-04T17:45:53.966 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_flock_retry": "3", 2024-09-04T17:45:53.966 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-04T17:45:53.966 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_inject_crash": "0", 2024-09-04T17:45:53.966 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-04T17:45:53.966 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_ioring": "false", 2024-09-04T17:45:53.967 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_ioring_hipri": "false", 2024-09-04T17:45:53.967 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-04T17:45:53.967 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-04T17:45:53.967 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-04T17:45:53.967 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-04T17:45:53.967 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_stalled_read_warn_lifetime": "86400", 2024-09-04T17:45:53.967 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_stalled_read_warn_threshold": "1", 2024-09-04T17:45:53.967 INFO:teuthology.orchestra.run.smithi092.stdout: "bdev_type": "", 2024-09-04T17:45:53.967 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_alloc_size": "1048576", 2024-09-04T17:45:53.967 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_allocator": "hybrid", 2024-09-04T17:45:53.967 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_buffered_io": "true", 2024-09-04T17:45:53.967 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_check_for_zeros": "false", 2024-09-04T17:45:53.967 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-04T17:45:53.968 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-04T17:45:53.968 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_compact_log_sync": "false", 2024-09-04T17:45:53.968 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-04T17:45:53.968 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-04T17:45:53.968 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-04T17:45:53.968 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-04T17:45:53.968 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_max_log_runway": "4194304", 2024-09-04T17:45:53.968 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_max_prefetch": "1048576", 2024-09-04T17:45:53.968 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_min_flush_size": "524288", 2024-09-04T17:45:53.968 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_min_log_runway": "1048576", 2024-09-04T17:45:53.968 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_replay_recovery": "false", 2024-09-04T17:45:53.968 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-04T17:45:53.969 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-04T17:45:53.969 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_sync_write": "false", 2024-09-04T17:45:53.969 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-04T17:45:53.969 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-04T17:45:53.969 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-04T17:45:53.969 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_allocation_from_file": "true", 2024-09-04T17:45:53.969 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_allocator": "hybrid", 2024-09-04T17:45:53.969 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_async_db_compaction": "true", 2024-09-04T17:45:53.969 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-04T17:45:53.969 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-04T17:45:53.969 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-04T17:45:53.969 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-04T17:45:53.969 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_bdev_label_multi": "true", 2024-09-04T17:45:53.969 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_bdev_label_multi_upgrade": "false", 2024-09-04T17:45:53.970 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_bdev_label_require_all": "true", 2024-09-04T17:45:53.970 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-04T17:45:53.970 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-04T17:45:53.970 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-04T17:45:53.970 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_block_create": "true", 2024-09-04T17:45:53.970 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_block_db_create": "false", 2024-09-04T17:45:53.970 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_block_db_path": "", 2024-09-04T17:45:53.970 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_block_db_size": "0", 2024-09-04T17:45:53.970 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_block_path": "", 2024-09-04T17:45:53.970 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_block_preallocate_file": "false", 2024-09-04T17:45:53.970 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_block_size": "107374182400", 2024-09-04T17:45:53.970 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_block_wal_create": "false", 2024-09-04T17:45:53.970 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_block_wal_path": "", 2024-09-04T17:45:53.970 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_block_wal_size": "100663296", 2024-09-04T17:45:53.971 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_bluefs": "true", 2024-09-04T17:45:53.971 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-04T17:45:53.971 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-04T17:45:53.971 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-04T17:45:53.971 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_btree2_alloc_weight_factor": "2.000000", 2024-09-04T17:45:53.971 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-04T17:45:53.971 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-04T17:45:53.971 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-04T17:45:53.971 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-04T17:45:53.971 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-04T17:45:53.971 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_autotune": "true", 2024-09-04T17:45:53.971 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-04T17:45:53.971 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-04T17:45:53.971 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-04T17:45:53.972 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-04T17:45:53.972 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_size": "0", 2024-09-04T17:45:53.972 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-04T17:45:53.972 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-04T17:45:53.972 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-04T17:45:53.972 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-04T17:45:53.972 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_type": "2q", 2024-09-04T17:45:53.972 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-04T17:45:53.972 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_clone_cow": "true", 2024-09-04T17:45:53.972 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-04T17:45:53.972 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-04T17:45:53.972 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-04T17:45:53.973 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-04T17:45:53.973 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-04T17:45:53.973 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-04T17:45:53.973 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-04T17:45:53.973 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_compression_mode": "none", 2024-09-04T17:45:53.973 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-04T17:45:53.973 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_csum_type": "crc32c", 2024-09-04T17:45:53.973 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_enforce_min_alloc_size": "0", 2024-09-04T17:45:53.973 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-04T17:45:53.973 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_freelist": "false", 2024-09-04T17:45:53.974 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-04T17:45:53.974 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-04T17:45:53.974 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-04T17:45:53.974 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-04T17:45:53.974 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-04T17:45:53.974 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-04T17:45:53.974 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-04T17:45:53.974 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-04T17:45:53.974 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-04T17:45:53.974 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-04T17:45:53.974 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-04T17:45:53.974 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-04T17:45:53.974 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_small_allocations": "0", 2024-09-04T17:45:53.974 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-04T17:45:53.975 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_default_buffered_read": "true", 2024-09-04T17:45:53.975 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_default_buffered_write": "false", 2024-09-04T17:45:53.975 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-04T17:45:53.975 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-04T17:45:53.975 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-04T17:45:53.975 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_elastic_shared_blobs": "true", 2024-09-04T17:45:53.975 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-04T17:45:53.975 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-04T17:45:53.975 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-04T17:45:53.975 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-04T17:45:53.975 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-04T17:45:53.975 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fail_eio": "false", 2024-09-04T17:45:53.975 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-04T17:45:53.976 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-04T17:45:53.976 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-04T17:45:53.976 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-04T17:45:53.976 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-04T17:45:53.976 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-04T17:45:53.976 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_on_mount": "false", 2024-09-04T17:45:53.976 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-04T17:45:53.976 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_on_umount": "false", 2024-09-04T17:45:53.976 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-04T17:45:53.976 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-04T17:45:53.976 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-04T17:45:53.976 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-04T17:45:53.976 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-04T17:45:53.976 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-04T17:45:53.977 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-04T17:45:53.977 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-04T17:45:53.977 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_ignore_data_csum": "false", 2024-09-04T17:45:53.977 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-04T17:45:53.977 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-04T17:45:53.977 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-04T17:45:53.977 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-04T17:45:53.977 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_log_op_age": "5.000000", 2024-09-04T17:45:53.977 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_max_alloc_size": "0", 2024-09-04T17:45:53.977 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_max_blob_size": "0", 2024-09-04T17:45:53.977 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-04T17:45:53.977 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-04T17:45:53.977 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-04T17:45:53.977 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_max_deferred_txc": "32", 2024-09-04T17:45:53.978 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_min_alloc_size": "0", 2024-09-04T17:45:53.978 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-04T17:45:53.978 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-04T17:45:53.978 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_nid_prealloc": "1024", 2024-09-04T17:45:53.978 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-04T17:45:53.978 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-04T17:45:53.978 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-04T17:45:53.978 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-04T17:45:53.978 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_retry_disk_reads": "3", 2024-09-04T17:45:53.978 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_rocksdb_cf": "true", 2024-09-04T17:45:53.978 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-04T17:45:53.978 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-04T17:45:53.979 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-04T17:45:53.979 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_slow_ops_warn_lifetime": "86400", 2024-09-04T17:45:53.979 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_slow_ops_warn_threshold": "1", 2024-09-04T17:45:53.979 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-04T17:45:53.979 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-04T17:45:53.979 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-04T17:45:53.980 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_spdk_mem": "512", 2024-09-04T17:45:53.980 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-04T17:45:53.980 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-04T17:45:53.980 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-04T17:45:53.980 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-04T17:45:53.980 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-04T17:45:53.980 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-04T17:45:53.980 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-04T17:45:53.980 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_tracing": "false", 2024-09-04T17:45:53.980 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-04T17:45:53.980 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-04T17:45:53.980 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-04T17:45:53.980 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-04T17:45:53.981 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-04T17:45:53.981 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-04T17:45:53.981 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-04T17:45:53.981 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-04T17:45:53.981 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-04T17:45:53.981 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_write_v2": "false", 2024-09-04T17:45:53.981 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_write_v2_random": "false", 2024-09-04T17:45:53.981 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_zero_block_detection": "false", 2024-09-04T17:45:53.981 INFO:teuthology.orchestra.run.smithi092.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-04T17:45:53.981 INFO:teuthology.orchestra.run.smithi092.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-04T17:45:53.981 INFO:teuthology.orchestra.run.smithi092.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-04T17:45:53.981 INFO:teuthology.orchestra.run.smithi092.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-04T17:45:53.981 INFO:teuthology.orchestra.run.smithi092.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-04T17:45:53.981 INFO:teuthology.orchestra.run.smithi092.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-04T17:45:53.982 INFO:teuthology.orchestra.run.smithi092.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-04T17:45:53.982 INFO:teuthology.orchestra.run.smithi092.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-04T17:45:53.982 INFO:teuthology.orchestra.run.smithi092.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-04T17:45:53.982 INFO:teuthology.orchestra.run.smithi092.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-04T17:45:53.982 INFO:teuthology.orchestra.run.smithi092.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-04T17:45:53.982 INFO:teuthology.orchestra.run.smithi092.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-04T17:45:53.982 INFO:teuthology.orchestra.run.smithi092.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-04T17:45:53.982 INFO:teuthology.orchestra.run.smithi092.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-04T17:45:53.982 INFO:teuthology.orchestra.run.smithi092.stdout: "cephx_cluster_require_signatures": "false", 2024-09-04T17:45:53.982 INFO:teuthology.orchestra.run.smithi092.stdout: "cephx_cluster_require_version": "2", 2024-09-04T17:45:53.982 INFO:teuthology.orchestra.run.smithi092.stdout: "cephx_require_signatures": "false", 2024-09-04T17:45:53.982 INFO:teuthology.orchestra.run.smithi092.stdout: "cephx_require_version": "2", 2024-09-04T17:45:53.982 INFO:teuthology.orchestra.run.smithi092.stdout: "cephx_service_require_signatures": "false", 2024-09-04T17:45:53.983 INFO:teuthology.orchestra.run.smithi092.stdout: "cephx_service_require_version": "2", 2024-09-04T17:45:53.983 INFO:teuthology.orchestra.run.smithi092.stdout: "cephx_sign_messages": "true", 2024-09-04T17:45:53.983 INFO:teuthology.orchestra.run.smithi092.stdout: "chdir": "", 2024-09-04T17:45:53.983 INFO:teuthology.orchestra.run.smithi092.stdout: "client_acl_type": "", 2024-09-04T17:45:53.983 INFO:teuthology.orchestra.run.smithi092.stdout: "client_asio_thread_count": "2", 2024-09-04T17:45:53.983 INFO:teuthology.orchestra.run.smithi092.stdout: "client_cache_mid": "0.750000", 2024-09-04T17:45:53.983 INFO:teuthology.orchestra.run.smithi092.stdout: "client_cache_size": "16384", 2024-09-04T17:45:53.983 INFO:teuthology.orchestra.run.smithi092.stdout: "client_caps_release_delay": "5", 2024-09-04T17:45:53.983 INFO:teuthology.orchestra.run.smithi092.stdout: "client_check_pool_perm": "true", 2024-09-04T17:45:53.983 INFO:teuthology.orchestra.run.smithi092.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-04T17:45:53.983 INFO:teuthology.orchestra.run.smithi092.stdout: "client_debug_force_sync_read": "false", 2024-09-04T17:45:53.983 INFO:teuthology.orchestra.run.smithi092.stdout: "client_debug_getattr_caps": "false", 2024-09-04T17:45:53.983 INFO:teuthology.orchestra.run.smithi092.stdout: "client_debug_inject_features": "", 2024-09-04T17:45:53.984 INFO:teuthology.orchestra.run.smithi092.stdout: "client_debug_inject_tick_delay": "0", 2024-09-04T17:45:53.984 INFO:teuthology.orchestra.run.smithi092.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-04T17:45:53.984 INFO:teuthology.orchestra.run.smithi092.stdout: "client_die_on_failed_remount": "false", 2024-09-04T17:45:53.984 INFO:teuthology.orchestra.run.smithi092.stdout: "client_dirsize_rbytes": "true", 2024-09-04T17:45:53.984 INFO:teuthology.orchestra.run.smithi092.stdout: "client_force_lazyio": "false", 2024-09-04T17:45:53.984 INFO:teuthology.orchestra.run.smithi092.stdout: "client_fs": "", 2024-09-04T17:45:53.984 INFO:teuthology.orchestra.run.smithi092.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-04T17:45:53.984 INFO:teuthology.orchestra.run.smithi092.stdout: "client_inject_release_failure": "false", 2024-09-04T17:45:53.984 INFO:teuthology.orchestra.run.smithi092.stdout: "client_max_inline_size": "4096", 2024-09-04T17:45:53.984 INFO:teuthology.orchestra.run.smithi092.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-04T17:45:53.984 INFO:teuthology.orchestra.run.smithi092.stdout: "client_mds_namespace": "", 2024-09-04T17:45:53.984 INFO:teuthology.orchestra.run.smithi092.stdout: "client_metadata": "", 2024-09-04T17:45:53.984 INFO:teuthology.orchestra.run.smithi092.stdout: "client_mount_gid": "-1", 2024-09-04T17:45:53.985 INFO:teuthology.orchestra.run.smithi092.stdout: "client_mount_timeout": "300", 2024-09-04T17:45:53.985 INFO:teuthology.orchestra.run.smithi092.stdout: "client_mount_uid": "-1", 2024-09-04T17:45:53.985 INFO:teuthology.orchestra.run.smithi092.stdout: "client_mountpoint": "/", 2024-09-04T17:45:53.985 INFO:teuthology.orchestra.run.smithi092.stdout: "client_notify_timeout": "10", 2024-09-04T17:45:53.985 INFO:teuthology.orchestra.run.smithi092.stdout: "client_oc": "true", 2024-09-04T17:45:53.985 INFO:teuthology.orchestra.run.smithi092.stdout: "client_oc_max_dirty": "104857600", 2024-09-04T17:45:53.985 INFO:teuthology.orchestra.run.smithi092.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-04T17:45:53.985 INFO:teuthology.orchestra.run.smithi092.stdout: "client_oc_max_objects": "1000", 2024-09-04T17:45:53.985 INFO:teuthology.orchestra.run.smithi092.stdout: "client_oc_size": "209715200", 2024-09-04T17:45:53.985 INFO:teuthology.orchestra.run.smithi092.stdout: "client_oc_target_dirty": "8388608", 2024-09-04T17:45:53.985 INFO:teuthology.orchestra.run.smithi092.stdout: "client_permissions": "true", 2024-09-04T17:45:53.985 INFO:teuthology.orchestra.run.smithi092.stdout: "client_quota": "true", 2024-09-04T17:45:53.985 INFO:teuthology.orchestra.run.smithi092.stdout: "client_quota_df": "true", 2024-09-04T17:45:53.986 INFO:teuthology.orchestra.run.smithi092.stdout: "client_readahead_max_bytes": "0", 2024-09-04T17:45:53.986 INFO:teuthology.orchestra.run.smithi092.stdout: "client_readahead_max_periods": "4", 2024-09-04T17:45:53.986 INFO:teuthology.orchestra.run.smithi092.stdout: "client_readahead_min": "131072", 2024-09-04T17:45:53.986 INFO:teuthology.orchestra.run.smithi092.stdout: "client_reconnect_stale": "false", 2024-09-04T17:45:53.986 INFO:teuthology.orchestra.run.smithi092.stdout: "client_shutdown_timeout": "30", 2024-09-04T17:45:53.986 INFO:teuthology.orchestra.run.smithi092.stdout: "client_snapdir": ".snap", 2024-09-04T17:45:53.986 INFO:teuthology.orchestra.run.smithi092.stdout: "client_tick_interval": "1", 2024-09-04T17:45:53.986 INFO:teuthology.orchestra.run.smithi092.stdout: "client_trace": "", 2024-09-04T17:45:53.986 INFO:teuthology.orchestra.run.smithi092.stdout: "client_try_dentry_invalidate": "false", 2024-09-04T17:45:53.986 INFO:teuthology.orchestra.run.smithi092.stdout: "client_use_faked_inos": "false", 2024-09-04T17:45:53.986 INFO:teuthology.orchestra.run.smithi092.stdout: "client_use_random_mds": "false", 2024-09-04T17:45:53.986 INFO:teuthology.orchestra.run.smithi092.stdout: "clog_to_graylog": "false", 2024-09-04T17:45:53.986 INFO:teuthology.orchestra.run.smithi092.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-04T17:45:53.987 INFO:teuthology.orchestra.run.smithi092.stdout: "clog_to_graylog_port": "12201", 2024-09-04T17:45:53.987 INFO:teuthology.orchestra.run.smithi092.stdout: "clog_to_monitors": "default=true", 2024-09-04T17:45:53.987 INFO:teuthology.orchestra.run.smithi092.stdout: "clog_to_syslog": "false", 2024-09-04T17:45:53.987 INFO:teuthology.orchestra.run.smithi092.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-04T17:45:53.987 INFO:teuthology.orchestra.run.smithi092.stdout: "clog_to_syslog_level": "info", 2024-09-04T17:45:53.987 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "-", 2024-09-04T17:45:53.987 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_network": "", 2024-09-04T17:45:53.987 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_network_interface": "", 2024-09-04T17:45:53.987 INFO:teuthology.orchestra.run.smithi092.stdout: "compressor_zlib_isal": "false", 2024-09-04T17:45:53.987 INFO:teuthology.orchestra.run.smithi092.stdout: "compressor_zlib_level": "5", 2024-09-04T17:45:53.987 INFO:teuthology.orchestra.run.smithi092.stdout: "compressor_zlib_winsize": "-15", 2024-09-04T17:45:53.987 INFO:teuthology.orchestra.run.smithi092.stdout: "compressor_zstd_level": "1", 2024-09-04T17:45:53.987 INFO:teuthology.orchestra.run.smithi092.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-04T17:45:53.987 INFO:teuthology.orchestra.run.smithi092.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-04T17:45:53.988 INFO:teuthology.orchestra.run.smithi092.stdout: "crimson_alien_op_num_threads": "6", 2024-09-04T17:45:53.988 INFO:teuthology.orchestra.run.smithi092.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-04T17:45:53.988 INFO:teuthology.orchestra.run.smithi092.stdout: "crimson_osd_obc_lru_size": "512", 2024-09-04T17:45:53.988 INFO:teuthology.orchestra.run.smithi092.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-04T17:45:53.988 INFO:teuthology.orchestra.run.smithi092.stdout: "crimson_osd_stat_interval": "0", 2024-09-04T17:45:53.988 INFO:teuthology.orchestra.run.smithi092.stdout: "crimson_seastar_cpu_cores": "", 2024-09-04T17:45:53.988 INFO:teuthology.orchestra.run.smithi092.stdout: "crimson_seastar_num_threads": "0", 2024-09-04T17:45:53.988 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_location": "", 2024-09-04T17:45:53.988 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_location_hook": "", 2024-09-04T17:45:53.988 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_location_hook_timeout": "10", 2024-09-04T17:45:53.988 INFO:teuthology.orchestra.run.smithi092.stdout: "daemonize": "false", 2024-09-04T17:45:53.988 INFO:teuthology.orchestra.run.smithi092.stdout: "daos_pool": "tank", 2024-09-04T17:45:53.988 INFO:teuthology.orchestra.run.smithi092.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-04T17:45:53.989 INFO:teuthology.orchestra.run.smithi092.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-04T17:45:53.989 INFO:teuthology.orchestra.run.smithi092.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-04T17:45:53.989 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_alienstore": "0/5", 2024-09-04T17:45:53.989 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_allow_any_pool_priority": "false", 2024-09-04T17:45:53.989 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_asok": "1/5", 2024-09-04T17:45:53.989 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_asok_assert_abort": "false", 2024-09-04T17:45:53.989 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_asserts_on_shutdown": "true", 2024-09-04T17:45:53.989 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_auth": "20/20", 2024-09-04T17:45:53.989 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_bdev": "1/3", 2024-09-04T17:45:53.989 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_bluefs": "1/5", 2024-09-04T17:45:53.989 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_bluestore": "1/5", 2024-09-04T17:45:53.989 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_buffer": "0/1", 2024-09-04T17:45:53.989 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_ceph_dedup": "0/5", 2024-09-04T17:45:53.989 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_ceph_exporter": "1/5", 2024-09-04T17:45:53.990 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_cephfs_mirror": "0/5", 2024-09-04T17:45:53.990 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_cephsqlite": "0/5", 2024-09-04T17:45:53.990 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_client": "0/5", 2024-09-04T17:45:53.990 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_compressor": "1/5", 2024-09-04T17:45:53.990 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_context": "0/1", 2024-09-04T17:45:53.990 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_crimson_interrupt": "0/5", 2024-09-04T17:45:53.990 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_crush": "1/1", 2024-09-04T17:45:53.990 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_crypto": "1/5", 2024-09-04T17:45:53.990 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_cyanstore": "0/5", 2024-09-04T17:45:53.990 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_deliberately_leak_memory": "false", 2024-09-04T17:45:53.990 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_disable_randomized_ping": "false", 2024-09-04T17:45:53.990 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_dpdk": "1/5", 2024-09-04T17:45:53.990 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_eventtrace": "1/5", 2024-09-04T17:45:53.991 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_filer": "0/1", 2024-09-04T17:45:53.991 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_filestore": "1/3", 2024-09-04T17:45:53.991 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_finisher": "1/1", 2024-09-04T17:45:53.991 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_fuse": "1/5", 2024-09-04T17:45:53.991 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_heartbeat_testing_span": "0", 2024-09-04T17:45:53.991 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_heartbeatmap": "1/5", 2024-09-04T17:45:53.991 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-04T17:45:53.991 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_javaclient": "1/5", 2024-09-04T17:45:53.991 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_journal": "1/3", 2024-09-04T17:45:53.991 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_journaler": "0/5", 2024-09-04T17:45:53.991 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_kstore": "1/5", 2024-09-04T17:45:53.991 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_lockdep": "0/1", 2024-09-04T17:45:53.991 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_mclock": "1/5", 2024-09-04T17:45:53.991 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_mds": "1/5", 2024-09-04T17:45:53.992 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_mds_balancer": "1/5", 2024-09-04T17:45:53.992 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_mds_locker": "1/5", 2024-09-04T17:45:53.992 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_mds_log": "1/5", 2024-09-04T17:45:53.992 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_mds_log_expire": "1/5", 2024-09-04T17:45:53.992 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_mds_migrator": "1/5", 2024-09-04T17:45:53.992 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_mds_quiesce": "3/5", 2024-09-04T17:45:53.992 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_memstore": "1/5", 2024-09-04T17:45:53.992 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_mgr": "2/5", 2024-09-04T17:45:53.992 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_mgrc": "1/5", 2024-09-04T17:45:53.992 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_mon": "20/20", 2024-09-04T17:45:53.992 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_monc": "0/10", 2024-09-04T17:45:53.992 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_ms": "1/1", 2024-09-04T17:45:53.993 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_none": "0/5", 2024-09-04T17:45:53.993 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_objclass": "0/5", 2024-09-04T17:45:53.993 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_objectcacher": "0/5", 2024-09-04T17:45:53.993 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_objecter": "0/1", 2024-09-04T17:45:53.993 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_optracker": "0/5", 2024-09-04T17:45:53.993 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_osd": "1/5", 2024-09-04T17:45:53.993 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_paxos": "20/20", 2024-09-04T17:45:53.993 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_perfcounter": "1/5", 2024-09-04T17:45:53.993 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_prioritycache": "1/5", 2024-09-04T17:45:53.993 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_rados": "0/5", 2024-09-04T17:45:53.993 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_rbd": "0/5", 2024-09-04T17:45:53.993 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_rbd_mirror": "0/5", 2024-09-04T17:45:53.993 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_rbd_pwl": "0/5", 2024-09-04T17:45:53.993 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_rbd_replay": "0/5", 2024-09-04T17:45:53.994 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_refs": "0/0", 2024-09-04T17:45:53.994 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_reserver": "1/1", 2024-09-04T17:45:53.994 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_rgw": "1/5", 2024-09-04T17:45:53.994 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_rgw_access": "1/5", 2024-09-04T17:45:53.994 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_rgw_datacache": "1/5", 2024-09-04T17:45:53.994 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_rgw_dbstore": "1/5", 2024-09-04T17:45:53.994 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_rgw_flight": "1/5", 2024-09-04T17:45:53.994 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_rgw_lifecycle": "1/5", 2024-09-04T17:45:53.994 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_rgw_notification": "1/5", 2024-09-04T17:45:53.994 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_rgw_sync": "1/5", 2024-09-04T17:45:53.994 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_rocksdb": "4/5", 2024-09-04T17:45:53.994 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_seastore": "0/5", 2024-09-04T17:45:53.994 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_seastore_backref": "0/5", 2024-09-04T17:45:53.995 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_seastore_cache": "0/5", 2024-09-04T17:45:53.995 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_seastore_cleaner": "0/5", 2024-09-04T17:45:53.995 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_seastore_device": "0/5", 2024-09-04T17:45:53.995 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_seastore_epm": "0/5", 2024-09-04T17:45:53.995 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-04T17:45:53.995 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_seastore_journal": "0/5", 2024-09-04T17:45:53.995 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_seastore_lba": "0/5", 2024-09-04T17:45:53.995 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_seastore_odata": "0/5", 2024-09-04T17:45:53.995 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_seastore_omap": "0/5", 2024-09-04T17:45:53.995 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_seastore_onode": "0/5", 2024-09-04T17:45:53.995 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_seastore_t": "0/5", 2024-09-04T17:45:53.995 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_seastore_tm": "0/5", 2024-09-04T17:45:53.995 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_striper": "0/1", 2024-09-04T17:45:53.996 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_test": "0/5", 2024-09-04T17:45:53.996 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_throttle": "1/1", 2024-09-04T17:45:53.996 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_timer": "0/1", 2024-09-04T17:45:53.996 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_tp": "0/5", 2024-09-04T17:45:53.996 INFO:teuthology.orchestra.run.smithi092.stdout: "debug_trace": "1/5", 2024-09-04T17:45:53.996 INFO:teuthology.orchestra.run.smithi092.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-04T17:45:53.996 INFO:teuthology.orchestra.run.smithi092.stdout: "device_failure_prediction_mode": "none", 2024-09-04T17:45:53.996 INFO:teuthology.orchestra.run.smithi092.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-04T17:45:53.996 INFO:teuthology.orchestra.run.smithi092.stdout: "erasure_code_dir": "/usr/lib/ceph/erasure-code", 2024-09-04T17:45:53.996 INFO:teuthology.orchestra.run.smithi092.stdout: "err_to_graylog": "false", 2024-09-04T17:45:53.996 INFO:teuthology.orchestra.run.smithi092.stdout: "err_to_journald": "false", 2024-09-04T17:45:53.996 INFO:teuthology.orchestra.run.smithi092.stdout: "err_to_stderr": "true", 2024-09-04T17:45:53.996 INFO:teuthology.orchestra.run.smithi092.stdout: "err_to_syslog": "false", 2024-09-04T17:45:53.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event_tracing": "false", 2024-09-04T17:45:53.997 INFO:teuthology.orchestra.run.smithi092.stdout: "exporter_addr": "0.0.0.0", 2024-09-04T17:45:53.997 INFO:teuthology.orchestra.run.smithi092.stdout: "exporter_cert_file": "", 2024-09-04T17:45:53.997 INFO:teuthology.orchestra.run.smithi092.stdout: "exporter_http_port": "9926", 2024-09-04T17:45:53.997 INFO:teuthology.orchestra.run.smithi092.stdout: "exporter_key_file": "", 2024-09-04T17:45:53.997 INFO:teuthology.orchestra.run.smithi092.stdout: "exporter_prio_limit": "5", 2024-09-04T17:45:53.997 INFO:teuthology.orchestra.run.smithi092.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-04T17:45:53.997 INFO:teuthology.orchestra.run.smithi092.stdout: "exporter_sort_metrics": "true", 2024-09-04T17:45:53.997 INFO:teuthology.orchestra.run.smithi092.stdout: "exporter_stats_period": "5", 2024-09-04T17:45:53.997 INFO:teuthology.orchestra.run.smithi092.stdout: "fake_statfs_for_testing": "0", 2024-09-04T17:45:53.997 INFO:teuthology.orchestra.run.smithi092.stdout: "fatal_signal_handlers": "true", 2024-09-04T17:45:53.997 INFO:teuthology.orchestra.run.smithi092.stdout: "filer_max_purge_ops": "10", 2024-09-04T17:45:53.997 INFO:teuthology.orchestra.run.smithi092.stdout: "filer_max_truncate_ops": "128", 2024-09-04T17:45:53.998 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_apply_finisher_threads": "1", 2024-09-04T17:45:53.998 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_blackhole": "false", 2024-09-04T17:45:53.998 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_btrfs_clone_range": "true", 2024-09-04T17:45:53.998 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_btrfs_snap": "true", 2024-09-04T17:45:53.998 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_caller_concurrency": "10", 2024-09-04T17:45:53.998 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_collect_device_partition_information": "true", 2024-09-04T17:45:53.998 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_commit_timeout": "600.000000", 2024-09-04T17:45:53.998 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_debug_inject_read_err": "false", 2024-09-04T17:45:53.998 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_debug_omap_check": "false", 2024-09-04T17:45:53.998 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_debug_verify_split": "false", 2024-09-04T17:45:53.998 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_dump_file": "", 2024-09-04T17:45:53.998 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-04T17:45:53.998 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-04T17:45:53.998 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_fadvise": "true", 2024-09-04T17:45:53.999 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_fail_eio": "true", 2024-09-04T17:45:53.999 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_fd_cache_shards": "16", 2024-09-04T17:45:53.999 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_fd_cache_size": "128", 2024-09-04T17:45:53.999 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_fiemap": "false", 2024-09-04T17:45:53.999 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_fiemap_threshold": "4096", 2024-09-04T17:45:53.999 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-04T17:45:53.999 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-04T17:45:53.999 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_inject_stall": "0", 2024-09-04T17:45:53.999 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_journal_parallel": "false", 2024-09-04T17:45:53.999 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_journal_trailing": "false", 2024-09-04T17:45:53.999 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_journal_writeahead": "false", 2024-09-04T17:45:53.999 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_kill_at": "0", 2024-09-04T17:45:53.999 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-04T17:45:53.999 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-04T17:45:54.000 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-04T17:45:54.000 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-04T17:45:54.000 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-04T17:45:54.000 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_max_inline_xattrs": "0", 2024-09-04T17:45:54.000 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-04T17:45:54.000 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-04T17:45:54.000 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-04T17:45:54.000 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-04T17:45:54.000 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_max_xattr_value_size": "0", 2024-09-04T17:45:54.000 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-04T17:45:54.000 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-04T17:45:54.000 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-04T17:45:54.001 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_merge_threshold": "-10", 2024-09-04T17:45:54.001 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-04T17:45:54.001 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_odsync_write": "false", 2024-09-04T17:45:54.001 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_omap_backend": "rocksdb", 2024-09-04T17:45:54.001 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_omap_backend_path": "", 2024-09-04T17:45:54.001 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-04T17:45:54.001 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-04T17:45:54.001 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-04T17:45:54.001 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_op_thread_timeout": "60", 2024-09-04T17:45:54.001 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_op_threads": "2", 2024-09-04T17:45:54.001 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_punch_hole": "false", 2024-09-04T17:45:54.001 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-04T17:45:54.001 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-04T17:45:54.002 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-04T17:45:54.002 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-04T17:45:54.002 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-04T17:45:54.002 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-04T17:45:54.002 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-04T17:45:54.002 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-04T17:45:54.002 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-04T17:45:54.002 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_queue_max_ops": "50", 2024-09-04T17:45:54.002 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-04T17:45:54.002 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_seek_data_hole": "false", 2024-09-04T17:45:54.002 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_sloppy_crc": "false", 2024-09-04T17:45:54.002 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-04T17:45:54.003 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_splice": "false", 2024-09-04T17:45:54.003 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_split_multiple": "2", 2024-09-04T17:45:54.003 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_split_rand_factor": "20", 2024-09-04T17:45:54.003 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_update_to": "1000", 2024-09-04T17:45:54.003 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-04T17:45:54.003 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-04T17:45:54.003 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-04T17:45:54.003 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-04T17:45:54.003 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-04T17:45:54.003 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-04T17:45:54.003 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_wbthrottle_enable": "true", 2024-09-04T17:45:54.003 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-04T17:45:54.003 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-04T17:45:54.003 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-04T17:45:54.004 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-04T17:45:54.004 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-04T17:45:54.004 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-04T17:45:54.004 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_xfs_extsize": "false", 2024-09-04T17:45:54.004 INFO:teuthology.orchestra.run.smithi092.stdout: "filestore_zfs_snap": "false", 2024-09-04T17:45:54.004 INFO:teuthology.orchestra.run.smithi092.stdout: "fio_dir": "/tmp/fio", 2024-09-04T17:45:54.004 INFO:teuthology.orchestra.run.smithi092.stdout: "fsid": "f36b6ce2-8c72-4c99-8607-5c53c38ef67b", 2024-09-04T17:45:54.004 INFO:teuthology.orchestra.run.smithi092.stdout: "fuse_allow_other": "true", 2024-09-04T17:45:54.004 INFO:teuthology.orchestra.run.smithi092.stdout: "fuse_atomic_o_trunc": "true", 2024-09-04T17:45:54.004 INFO:teuthology.orchestra.run.smithi092.stdout: "fuse_big_writes": "true", 2024-09-04T17:45:54.004 INFO:teuthology.orchestra.run.smithi092.stdout: "fuse_debug": "false", 2024-09-04T17:45:54.004 INFO:teuthology.orchestra.run.smithi092.stdout: "fuse_default_permissions": "false", 2024-09-04T17:45:54.005 INFO:teuthology.orchestra.run.smithi092.stdout: "fuse_disable_pagecache": "false", 2024-09-04T17:45:54.005 INFO:teuthology.orchestra.run.smithi092.stdout: "fuse_max_write": "0", 2024-09-04T17:45:54.005 INFO:teuthology.orchestra.run.smithi092.stdout: "fuse_multithreaded": "true", 2024-09-04T17:45:54.005 INFO:teuthology.orchestra.run.smithi092.stdout: "fuse_require_active_mds": "true", 2024-09-04T17:45:54.005 INFO:teuthology.orchestra.run.smithi092.stdout: "fuse_set_user_groups": "true", 2024-09-04T17:45:54.005 INFO:teuthology.orchestra.run.smithi092.stdout: "fuse_splice_move": "true", 2024-09-04T17:45:54.005 INFO:teuthology.orchestra.run.smithi092.stdout: "fuse_splice_read": "true", 2024-09-04T17:45:54.005 INFO:teuthology.orchestra.run.smithi092.stdout: "fuse_splice_write": "true", 2024-09-04T17:45:54.005 INFO:teuthology.orchestra.run.smithi092.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-04T17:45:54.005 INFO:teuthology.orchestra.run.smithi092.stdout: "fuse_use_invalidate_cb": "true", 2024-09-04T17:45:54.005 INFO:teuthology.orchestra.run.smithi092.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-04T17:45:54.005 INFO:teuthology.orchestra.run.smithi092.stdout: "gss_target_name": "ceph", 2024-09-04T17:45:54.005 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_file": "", 2024-09-04T17:45:54.006 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_inject_failure": "0", 2024-09-04T17:45:54.006 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_interval": "5", 2024-09-04T17:45:54.006 INFO:teuthology.orchestra.run.smithi092.stdout: "host": "smithi092", 2024-09-04T17:45:54.006 INFO:teuthology.orchestra.run.smithi092.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-04T17:45:54.006 INFO:teuthology.orchestra.run.smithi092.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-04T17:45:54.006 INFO:teuthology.orchestra.run.smithi092.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-04T17:45:54.006 INFO:teuthology.orchestra.run.smithi092.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-04T17:45:54.006 INFO:teuthology.orchestra.run.smithi092.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-04T17:45:54.006 INFO:teuthology.orchestra.run.smithi092.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-04T17:45:54.006 INFO:teuthology.orchestra.run.smithi092.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-04T17:45:54.006 INFO:teuthology.orchestra.run.smithi092.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-04T17:45:54.006 INFO:teuthology.orchestra.run.smithi092.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-04T17:45:54.006 INFO:teuthology.orchestra.run.smithi092.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-04T17:45:54.007 INFO:teuthology.orchestra.run.smithi092.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-04T17:45:54.007 INFO:teuthology.orchestra.run.smithi092.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-04T17:45:54.007 INFO:teuthology.orchestra.run.smithi092.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-04T17:45:54.007 INFO:teuthology.orchestra.run.smithi092.stdout: "inject_early_sigterm": "false", 2024-09-04T17:45:54.007 INFO:teuthology.orchestra.run.smithi092.stdout: "jaeger_agent_port": "6799", 2024-09-04T17:45:54.007 INFO:teuthology.orchestra.run.smithi092.stdout: "jaeger_tracing_enable": "false", 2024-09-04T17:45:54.007 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_aio": "true", 2024-09-04T17:45:54.007 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_align_min_size": "65536", 2024-09-04T17:45:54.007 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_block_align": "true", 2024-09-04T17:45:54.007 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_block_size": "4096", 2024-09-04T17:45:54.007 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_dio": "true", 2024-09-04T17:45:54.007 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_discard": "false", 2024-09-04T17:45:54.007 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_force_aio": "false", 2024-09-04T17:45:54.007 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_ignore_corruption": "false", 2024-09-04T17:45:54.008 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_max_write_bytes": "10485760", 2024-09-04T17:45:54.008 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_max_write_entries": "100", 2024-09-04T17:45:54.008 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_replay_from": "0", 2024-09-04T17:45:54.008 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-04T17:45:54.008 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-04T17:45:54.008 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-04T17:45:54.008 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-04T17:45:54.008 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_write_header_frequency": "0", 2024-09-04T17:45:54.008 INFO:teuthology.orchestra.run.smithi092.stdout: "journal_zero_on_create": "false", 2024-09-04T17:45:54.008 INFO:teuthology.orchestra.run.smithi092.stdout: "journaler_prefetch_periods": "10", 2024-09-04T17:45:54.008 INFO:teuthology.orchestra.run.smithi092.stdout: "journaler_prezero_periods": "5", 2024-09-04T17:45:54.008 INFO:teuthology.orchestra.run.smithi092.stdout: "journaler_write_head_interval": "15", 2024-09-04T17:45:54.008 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "", 2024-09-04T17:45:54.008 INFO:teuthology.orchestra.run.smithi092.stdout: "keyfile": "", 2024-09-04T17:45:54.008 INFO:teuthology.orchestra.run.smithi092.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-04T17:45:54.009 INFO:teuthology.orchestra.run.smithi092.stdout: "kstore_backend": "rocksdb", 2024-09-04T17:45:54.009 INFO:teuthology.orchestra.run.smithi092.stdout: "kstore_default_stripe_size": "65536", 2024-09-04T17:45:54.009 INFO:teuthology.orchestra.run.smithi092.stdout: "kstore_fsck_on_mount": "false", 2024-09-04T17:45:54.009 INFO:teuthology.orchestra.run.smithi092.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-04T17:45:54.009 INFO:teuthology.orchestra.run.smithi092.stdout: "kstore_max_bytes": "67108864", 2024-09-04T17:45:54.009 INFO:teuthology.orchestra.run.smithi092.stdout: "kstore_max_ops": "512", 2024-09-04T17:45:54.009 INFO:teuthology.orchestra.run.smithi092.stdout: "kstore_nid_prealloc": "1024", 2024-09-04T17:45:54.009 INFO:teuthology.orchestra.run.smithi092.stdout: "kstore_onode_map_size": "1024", 2024-09-04T17:45:54.009 INFO:teuthology.orchestra.run.smithi092.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-04T17:45:54.009 INFO:teuthology.orchestra.run.smithi092.stdout: "kstore_sync_submit_transaction": "false", 2024-09-04T17:45:54.009 INFO:teuthology.orchestra.run.smithi092.stdout: "kstore_sync_transaction": "false", 2024-09-04T17:45:54.009 INFO:teuthology.orchestra.run.smithi092.stdout: "librados_thread_count": "2", 2024-09-04T17:45:54.010 INFO:teuthology.orchestra.run.smithi092.stdout: "lockdep": "false", 2024-09-04T17:45:54.010 INFO:teuthology.orchestra.run.smithi092.stdout: "lockdep_force_backtrace": "false", 2024-09-04T17:45:54.010 INFO:teuthology.orchestra.run.smithi092.stdout: "log_coarse_timestamps": "true", 2024-09-04T17:45:54.010 INFO:teuthology.orchestra.run.smithi092.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-04T17:45:54.010 INFO:teuthology.orchestra.run.smithi092.stdout: "log_flush_on_exit": "false", 2024-09-04T17:45:54.010 INFO:teuthology.orchestra.run.smithi092.stdout: "log_graylog_host": "127.0.0.1", 2024-09-04T17:45:54.010 INFO:teuthology.orchestra.run.smithi092.stdout: "log_graylog_port": "12201", 2024-09-04T17:45:54.010 INFO:teuthology.orchestra.run.smithi092.stdout: "log_max_new": "1000", 2024-09-04T17:45:54.010 INFO:teuthology.orchestra.run.smithi092.stdout: "log_max_recent": "10000", 2024-09-04T17:45:54.010 INFO:teuthology.orchestra.run.smithi092.stdout: "log_stderr_prefix": "", 2024-09-04T17:45:54.010 INFO:teuthology.orchestra.run.smithi092.stdout: "log_stop_at_utilization": "0.970000", 2024-09-04T17:45:54.010 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_file": "true", 2024-09-04T17:45:54.010 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_graylog": "false", 2024-09-04T17:45:54.010 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_journald": "false", 2024-09-04T17:45:54.011 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_stderr": "false", 2024-09-04T17:45:54.011 INFO:teuthology.orchestra.run.smithi092.stdout: "log_to_syslog": "false", 2024-09-04T17:45:54.011 INFO:teuthology.orchestra.run.smithi092.stdout: "max_rotating_auth_attempts": "10", 2024-09-04T17:45:54.011 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-04T17:45:54.011 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_action_on_write_error": "1", 2024-09-04T17:45:54.011 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_alternate_name_max": "8192", 2024-09-04T17:45:54.011 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_asio_thread_count": "2", 2024-09-04T17:45:54.011 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_export_pin": "true", 2024-09-04T17:45:54.011 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_fragment_dirs": "true", 2024-09-04T17:45:54.011 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-04T17:45:54.011 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_fragment_interval": "5", 2024-09-04T17:45:54.011 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-04T17:45:54.011 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-04T17:45:54.011 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_interval": "10", 2024-09-04T17:45:54.011 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_max": "-1", 2024-09-04T17:45:54.012 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_max_until": "-1", 2024-09-04T17:45:54.012 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_merge_size": "50", 2024-09-04T17:45:54.012 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_midchunk": "0.300000", 2024-09-04T17:45:54.012 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-04T17:45:54.012 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_min_start": "0.200000", 2024-09-04T17:45:54.012 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_minchunk": "0.001000", 2024-09-04T17:45:54.012 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_mode": "0", 2024-09-04T17:45:54.012 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_need_max": "1.200000", 2024-09-04T17:45:54.012 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_need_min": "0.800000", 2024-09-04T17:45:54.012 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_overload_epochs": "2", 2024-09-04T17:45:54.012 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-04T17:45:54.012 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-04T17:45:54.012 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_split_bits": "3", 2024-09-04T17:45:54.013 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-04T17:45:54.013 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_split_size": "10000", 2024-09-04T17:45:54.013 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-04T17:45:54.013 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_target_decay": "10.000000", 2024-09-04T17:45:54.013 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-04T17:45:54.013 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_beacon_grace": "15.000000", 2024-09-04T17:45:54.013 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_beacon_interval": "4.000000", 2024-09-04T17:45:54.013 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-04T17:45:54.013 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-04T17:45:54.013 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache_mid": "0.700000", 2024-09-04T17:45:54.013 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-09-04T17:45:54.013 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache_quiesce_delay": "0", 2024-09-04T17:45:54.014 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache_quiesce_sleep": "200", 2024-09-04T17:45:54.014 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache_quiesce_splitauth": "true", 2024-09-04T17:45:54.014 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache_quiesce_threshold": "524288", 2024-09-04T17:45:54.014 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache_release_free_interval": "10", 2024-09-04T17:45:54.014 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache_reservation": "0.050000", 2024-09-04T17:45:54.014 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-04T17:45:54.014 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache_trim_interval": "1", 2024-09-04T17:45:54.014 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache_trim_threshold": "262144", 2024-09-04T17:45:54.014 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-04T17:45:54.014 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-04T17:45:54.014 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-04T17:45:54.014 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_client_prealloc_inos": "1000", 2024-09-04T17:45:54.014 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-04T17:45:54.014 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_connect_bootstrapping": "false", 2024-09-04T17:45:54.015 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_damage_table_max_entries": "10000", 2024-09-04T17:45:54.015 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-04T17:45:54.015 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_debug_auth_pins": "false", 2024-09-04T17:45:54.015 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_debug_frag": "false", 2024-09-04T17:45:54.015 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_debug_scatterstat": "false", 2024-09-04T17:45:54.015 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_debug_subtrees": "false", 2024-09-04T17:45:54.015 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_decay_halflife": "5.000000", 2024-09-04T17:45:54.015 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_default_dir_hash": "2", 2024-09-04T17:45:54.015 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_defer_session_stale": "true", 2024-09-04T17:45:54.015 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_deny_all_reconnect": "false", 2024-09-04T17:45:54.015 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_dir_keys_per_op": "16384", 2024-09-04T17:45:54.015 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_dir_max_commit_size": "10", 2024-09-04T17:45:54.015 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_dir_max_entries": "0", 2024-09-04T17:45:54.015 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_dir_prefetch": "true", 2024-09-04T17:45:54.016 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-04T17:45:54.016 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-04T17:45:54.016 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_dump_cache_on_map": "false", 2024-09-04T17:45:54.016 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-04T17:45:54.016 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-04T17:45:54.016 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_early_reply": "true", 2024-09-04T17:45:54.016 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_enable_op_tracker": "true", 2024-09-04T17:45:54.016 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_enforce_unique_name": "true", 2024-09-04T17:45:54.016 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-04T17:45:54.016 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-04T17:45:54.016 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_export_ephemeral_random": "true", 2024-09-04T17:45:54.016 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-04T17:45:54.017 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-04T17:45:54.017 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-04T17:45:54.017 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-04T17:45:54.017 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-04T17:45:54.017 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-04T17:45:54.017 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-04T17:45:54.017 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-04T17:45:54.017 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_health_summarize_threshold": "10", 2024-09-04T17:45:54.017 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-04T17:45:54.017 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-04T17:45:54.017 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_inject_health_dummy": "false", 2024-09-04T17:45:54.017 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-04T17:45:54.017 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_inject_migrator_session_race": "false", 2024-09-04T17:45:54.018 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-04T17:45:54.018 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-04T17:45:54.018 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-04T17:45:54.018 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_join_fs": "", 2024-09-04T17:45:54.018 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_journal_format": "1", 2024-09-04T17:45:54.018 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-04T17:45:54.018 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_kill_create_at": "0", 2024-09-04T17:45:54.018 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_kill_dirfrag_at": "0", 2024-09-04T17:45:54.018 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_kill_export_at": "0", 2024-09-04T17:45:54.018 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_kill_import_at": "0", 2024-09-04T17:45:54.018 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_kill_journal_at": "0", 2024-09-04T17:45:54.018 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_kill_journal_expire_at": "0", 2024-09-04T17:45:54.018 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_kill_journal_replay_at": "0", 2024-09-04T17:45:54.019 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_kill_link_at": "0", 2024-09-04T17:45:54.019 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_kill_mdstable_at": "0", 2024-09-04T17:45:54.019 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_kill_openc_at": "0", 2024-09-04T17:45:54.019 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_kill_rename_at": "0", 2024-09-04T17:45:54.019 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_kill_shutdown_at": "0", 2024-09-04T17:45:54.019 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log_event_large_threshold": "524288", 2024-09-04T17:45:54.019 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log_events_per_segment": "1024", 2024-09-04T17:45:54.019 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log_major_segment_event_ratio": "12", 2024-09-04T17:45:54.019 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log_max_events": "-1", 2024-09-04T17:45:54.019 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log_max_segments": "128", 2024-09-04T17:45:54.019 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log_pause": "false", 2024-09-04T17:45:54.019 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log_segment_size": "0", 2024-09-04T17:45:54.019 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-04T17:45:54.019 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log_skip_unbounded_events": "false", 2024-09-04T17:45:54.020 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-09-04T17:45:54.020 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log_trim_threshold": "128", 2024-09-04T17:45:54.020 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-09-04T17:45:54.020 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log_warn_factor": "2.000000", 2024-09-04T17:45:54.020 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_max_caps_per_client": "1048576", 2024-09-04T17:45:54.020 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_max_completed_flushes": "100000", 2024-09-04T17:45:54.020 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_max_completed_requests": "100000", 2024-09-04T17:45:54.020 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_max_export_size": "20971520", 2024-09-04T17:45:54.020 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_max_file_recover": "32", 2024-09-04T17:45:54.020 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_max_purge_files": "64", 2024-09-04T17:45:54.020 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_max_purge_ops": "8192", 2024-09-04T17:45:54.020 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-04T17:45:54.020 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-04T17:45:54.021 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_max_snaps_per_dir": "100", 2024-09-04T17:45:54.021 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_metrics_update_interval": "2", 2024-09-04T17:45:54.021 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_min_caps_per_client": "100", 2024-09-04T17:45:54.021 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_min_caps_working_set": "10000", 2024-09-04T17:45:54.021 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-04T17:45:54.021 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_numa_node": "-1", 2024-09-04T17:45:54.021 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-04T17:45:54.021 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_op_complaint_time": "30.000000", 2024-09-04T17:45:54.021 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_op_history_duration": "600", 2024-09-04T17:45:54.021 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_op_history_size": "20", 2024-09-04T17:45:54.021 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_op_history_slow_op_size": "20", 2024-09-04T17:45:54.021 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-04T17:45:54.021 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_op_log_threshold": "5", 2024-09-04T17:45:54.022 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_ping_grace": "15", 2024-09-04T17:45:54.022 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_ping_interval": "5", 2024-09-04T17:45:54.022 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-04T17:45:54.022 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-04T17:45:54.022 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_recall_max_caps": "30000", 2024-09-04T17:45:54.022 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-04T17:45:54.022 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-04T17:45:54.022 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-04T17:45:54.022 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_recall_warning_threshold": "262144", 2024-09-04T17:45:54.022 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-04T17:45:54.022 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_replay_interval": "1.000000", 2024-09-04T17:45:54.022 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-04T17:45:54.022 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-04T17:45:54.022 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_root_ino_gid": "0", 2024-09-04T17:45:54.023 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_root_ino_uid": "0", 2024-09-04T17:45:54.023 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-04T17:45:54.023 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_server_dispatch_client_request_delay": "0", 2024-09-04T17:45:54.023 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_server_dispatch_killpoint_random": "0.000000", 2024-09-04T17:45:54.023 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-04T17:45:54.023 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-04T17:45:54.023 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-04T17:45:54.023 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-04T17:45:54.023 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-04T17:45:54.023 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-04T17:45:54.023 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-04T17:45:54.023 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-04T17:45:54.023 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-04T17:45:54.024 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_shutdown_check": "0", 2024-09-04T17:45:54.024 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_skip_ino": "0", 2024-09-04T17:45:54.024 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-04T17:45:54.024 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_snap_max_uid": "4294967294", 2024-09-04T17:45:54.024 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_snap_min_uid": "0", 2024-09-04T17:45:54.024 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_snap_rstat": "false", 2024-09-04T17:45:54.024 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_standby_replay_damaged": "false", 2024-09-04T17:45:54.024 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_symlink_recovery": "true", 2024-09-04T17:45:54.024 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-04T17:45:54.024 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_thrash_exports": "0", 2024-09-04T17:45:54.024 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_thrash_fragments": "0", 2024-09-04T17:45:54.024 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_tick_interval": "5.000000", 2024-09-04T17:45:54.024 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_valgrind_exit": "false", 2024-09-04T17:45:54.025 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_verify_backtrace": "1", 2024-09-04T17:45:54.025 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_verify_scatter": "false", 2024-09-04T17:45:54.025 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-04T17:45:54.025 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_wipe_sessions": "false", 2024-09-04T17:45:54.025 INFO:teuthology.orchestra.run.smithi092.stdout: "mempool_debug": "false", 2024-09-04T17:45:54.025 INFO:teuthology.orchestra.run.smithi092.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-04T17:45:54.025 INFO:teuthology.orchestra.run.smithi092.stdout: "memstore_device_bytes": "1073741824", 2024-09-04T17:45:54.025 INFO:teuthology.orchestra.run.smithi092.stdout: "memstore_page_set": "false", 2024-09-04T17:45:54.025 INFO:teuthology.orchestra.run.smithi092.stdout: "memstore_page_size": "65536", 2024-09-04T17:45:54.025 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_client_bytes": "134217728", 2024-09-04T17:45:54.025 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_client_messages": "512", 2024-09-04T17:45:54.025 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-04T17:45:54.025 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-04T17:45:54.025 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-04T17:45:54.026 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-04T17:45:54.026 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_disabled_modules": "", 2024-09-04T17:45:54.026 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_enable_op_tracker": "true", 2024-09-04T17:45:54.026 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-04T17:45:54.026 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_max_pg_creating": "1024", 2024-09-04T17:45:54.026 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_max_pg_num_change": "128", 2024-09-04T17:45:54.026 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_mds_bytes": "134217728", 2024-09-04T17:45:54.026 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_mds_messages": "128", 2024-09-04T17:45:54.026 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-04T17:45:54.026 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_mon_bytes": "134217728", 2024-09-04T17:45:54.026 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_mon_messages": "128", 2024-09-04T17:45:54.026 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_num_op_tracker_shard": "32", 2024-09-04T17:45:54.026 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_op_complaint_time": "30.000000", 2024-09-04T17:45:54.026 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_op_history_duration": "600", 2024-09-04T17:45:54.027 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_op_history_size": "20", 2024-09-04T17:45:54.027 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_op_history_slow_op_size": "20", 2024-09-04T17:45:54.027 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_op_history_slow_op_threshold": "10.000000", 2024-09-04T17:45:54.027 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_op_log_threshold": "5", 2024-09-04T17:45:54.027 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_osd_bytes": "536870912", 2024-09-04T17:45:54.027 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_osd_messages": "8192", 2024-09-04T17:45:54.027 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_pool": "true", 2024-09-04T17:45:54.027 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-04T17:45:54.027 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_standby_modules": "true", 2024-09-04T17:45:54.027 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_stats_period": "5", 2024-09-04T17:45:54.027 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_stats_threshold": "5", 2024-09-04T17:45:54.027 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_tick_period": "2", 2024-09-04T17:45:54.027 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-04T17:45:54.028 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-04T17:45:54.028 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_allow_pool_delete": "true", 2024-09-04T17:45:54.028 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_allow_pool_size_one": "true", 2024-09-04T17:45:54.028 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_auth_validate_all_caps": "true", 2024-09-04T17:45:54.028 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-04T17:45:54.028 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-04T17:45:54.028 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_client_bytes": "104857600", 2024-09-04T17:45:54.028 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_client_directed_command_retry": "5", 2024-09-04T17:45:54.028 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-04T17:45:54.028 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-04T17:45:54.028 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-04T17:45:54.028 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-04T17:45:54.028 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_client_hunt_parallel": "3", 2024-09-04T17:45:54.029 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_client_log_interval": "1.000000", 2024-09-04T17:45:54.029 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-04T17:45:54.029 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_client_ping_interval": "10.000000", 2024-09-04T17:45:54.029 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-04T17:45:54.029 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_client_target_rank": "-1", 2024-09-04T17:45:54.029 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-04T17:45:54.029 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-04T17:45:54.029 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-04T17:45:54.029 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_cluster_log_level": "debug", 2024-09-04T17:45:54.029 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_cluster_log_to_file": "true", 2024-09-04T17:45:54.029 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-04T17:45:54.029 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-04T17:45:54.029 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-04T17:45:54.030 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_cluster_log_to_journald": "false", 2024-09-04T17:45:54.030 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-04T17:45:54.030 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-04T17:45:54.030 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-04T17:45:54.030 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_compact_on_bootstrap": "false", 2024-09-04T17:45:54.030 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_compact_on_start": "false", 2024-09-04T17:45:54.030 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_compact_on_trim": "true", 2024-09-04T17:45:54.030 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-04T17:45:54.030 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-04T17:45:54.030 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-04T17:45:54.030 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_cpu_threads": "4", 2024-09-04T17:45:54.030 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_crush_min_required_version": "hammer", 2024-09-04T17:45:54.030 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_daemon_bytes": "419430400", 2024-09-04T17:45:54.030 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-04T17:45:54.030 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_data_avail_crit": "5", 2024-09-04T17:45:54.031 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_data_avail_warn": "5", 2024-09-04T17:45:54.031 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_data_size_warn": "16106127360", 2024-09-04T17:45:54.031 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-04T17:45:54.031 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-04T17:45:54.031 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_debug_dump_json": "false", 2024-09-04T17:45:54.031 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-04T17:45:54.031 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_debug_dump_transactions": "false", 2024-09-04T17:45:54.031 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_debug_extra_checks": "false", 2024-09-04T17:45:54.031 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-04T17:45:54.031 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-04T17:45:54.031 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_debug_no_require_reef": "false", 2024-09-04T17:45:54.031 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_debug_no_require_squid": "false", 2024-09-04T17:45:54.031 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-04T17:45:54.032 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-04T17:45:54.032 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-04T17:45:54.032 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_down_mkfs_grace": "120", 2024-09-04T17:45:54.032 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_down_uptime_grace": "60", 2024-09-04T17:45:54.032 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_election_default_strategy": "1", 2024-09-04T17:45:54.032 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_election_timeout": "5.000000", 2024-09-04T17:45:54.032 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-04T17:45:54.032 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_elector_ping_divisor": "2", 2024-09-04T17:45:54.032 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-04T17:45:54.032 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_enable_op_tracker": "true", 2024-09-04T17:45:54.032 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_fake_pool_delete": "false", 2024-09-04T17:45:54.032 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_force_quorum_join": "false", 2024-09-04T17:45:54.032 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_fsmap_prune_threshold": "300", 2024-09-04T17:45:54.033 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_globalid_prealloc": "10000", 2024-09-04T17:45:54.033 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_health_detail_to_clog": "false", 2024-09-04T17:45:54.033 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_health_log_update_period": "5", 2024-09-04T17:45:54.033 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_health_max_detail": "50", 2024-09-04T17:45:54.033 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_health_to_clog": "true", 2024-09-04T17:45:54.033 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_health_to_clog_interval": "600", 2024-09-04T17:45:54.033 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-04T17:45:54.033 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_host": "172.21.15.92,[v2:172.21.15.92:3301,v1:172.21.15.92:6790],172.21.15.190", 2024-09-04T17:45:54.033 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_host_override": "", 2024-09-04T17:45:54.033 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_initial_members": "", 2024-09-04T17:45:54.033 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-04T17:45:54.033 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-04T17:45:54.033 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-04T17:45:54.034 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-04T17:45:54.034 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-04T17:45:54.034 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_lease": "5.000000", 2024-09-04T17:45:54.034 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-04T17:45:54.034 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-04T17:45:54.034 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_log_full_interval": "50", 2024-09-04T17:45:54.034 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_log_max": "10000", 2024-09-04T17:45:54.034 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_log_max_summary": "50", 2024-09-04T17:45:54.034 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-04T17:45:54.034 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_max_log_epochs": "500", 2024-09-04T17:45:54.034 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-04T17:45:54.034 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-04T17:45:54.034 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_max_nvmeof_epochs": "500", 2024-09-04T17:45:54.034 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_max_osd": "10000", 2024-09-04T17:45:54.035 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_max_pg_per_osd": "10000", 2024-09-04T17:45:54.035 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_max_pool_pg_num": "65536", 2024-09-04T17:45:54.035 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-04T17:45:54.035 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-04T17:45:54.035 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_mds_force_trim_to": "0", 2024-09-04T17:45:54.035 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_mds_skip_sanity": "false", 2024-09-04T17:45:54.035 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_memory_autotune": "true", 2024-09-04T17:45:54.035 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_memory_target": "2147483648", 2024-09-04T17:45:54.035 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_mgr_beacon_grace": "30", 2024-09-04T17:45:54.035 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-04T17:45:54.035 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_mgr_digest_period": "5", 2024-09-04T17:45:54.035 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_mgr_inactive_grace": "60", 2024-09-04T17:45:54.035 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-04T17:45:54.036 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-04T17:45:54.036 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_min_osdmap_epochs": "500", 2024-09-04T17:45:54.036 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_nvmeofgw_beacon_grace": "10", 2024-09-04T17:45:54.036 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_nvmeofgw_set_group_id_retry": "1000", 2024-09-04T17:45:54.036 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_op_complaint_time": "30", 2024-09-04T17:45:54.036 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_op_history_duration": "600", 2024-09-04T17:45:54.036 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_op_history_size": "20", 2024-09-04T17:45:54.036 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_op_history_slow_op_size": "20", 2024-09-04T17:45:54.036 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-04T17:45:54.036 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_op_log_threshold": "5", 2024-09-04T17:45:54.036 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-04T17:45:54.036 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-04T17:45:54.036 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-04T17:45:54.036 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_auto_mark_in": "false", 2024-09-04T17:45:54.037 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-04T17:45:54.037 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-04T17:45:54.037 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-04T17:45:54.037 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_cache_size": "500", 2024-09-04T17:45:54.037 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-04T17:45:54.037 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-04T17:45:54.037 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-04T17:45:54.037 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_down_out_interval": "600", 2024-09-04T17:45:54.037 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-04T17:45:54.037 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-04T17:45:54.037 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_force_trim_to": "0", 2024-09-04T17:45:54.037 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-04T17:45:54.037 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-04T17:45:54.038 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-04T17:45:54.038 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-04T17:45:54.038 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-04T17:45:54.038 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-04T17:45:54.038 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-04T17:45:54.038 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_min_down_reporters": "2", 2024-09-04T17:45:54.038 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-04T17:45:54.038 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-04T17:45:54.038 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-04T17:45:54.038 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-04T17:45:54.038 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-04T17:45:54.038 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-04T17:45:54.038 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_report_timeout": "900", 2024-09-04T17:45:54.039 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-04T17:45:54.039 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-04T17:45:54.039 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-04T17:45:54.039 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-04T17:45:54.039 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-04T17:45:54.039 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-04T17:45:54.039 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-04T17:45:54.039 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-04T17:45:54.039 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-04T17:45:54.039 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_pg_stuck_threshold": "60", 2024-09-04T17:45:54.039 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-04T17:45:54.039 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-04T17:45:54.039 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-04T17:45:54.040 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-04T17:45:54.040 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-04T17:45:54.040 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-04T17:45:54.040 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_probe_timeout": "2.000000", 2024-09-04T17:45:54.040 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_reweight_max_change": "0.050000", 2024-09-04T17:45:54.040 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_reweight_max_osds": "4", 2024-09-04T17:45:54.040 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-04T17:45:54.040 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-04T17:45:54.040 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-04T17:45:54.040 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-04T17:45:54.040 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-04T17:45:54.040 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_scrub_interval": "86400", 2024-09-04T17:45:54.040 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_scrub_max_keys": "100", 2024-09-04T17:45:54.040 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_scrub_timeout": "300", 2024-09-04T17:45:54.041 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_session_timeout": "300", 2024-09-04T17:45:54.041 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_smart_report_timeout": "5", 2024-09-04T17:45:54.041 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_stat_smooth_intervals": "6", 2024-09-04T17:45:54.041 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-04T17:45:54.041 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_stretch_pool_min_size": "2", 2024-09-04T17:45:54.041 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_stretch_pool_size": "4", 2024-09-04T17:45:54.041 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-04T17:45:54.041 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-04T17:45:54.041 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_sync_debug": "false", 2024-09-04T17:45:54.041 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-04T17:45:54.041 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-04T17:45:54.041 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_sync_provider_kill_at": "0", 2024-09-04T17:45:54.041 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_sync_requester_kill_at": "0", 2024-09-04T17:45:54.042 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_sync_timeout": "60.000000", 2024-09-04T17:45:54.042 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_target_pg_per_osd": "100", 2024-09-04T17:45:54.042 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_tick_interval": "5", 2024-09-04T17:45:54.042 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_timecheck_interval": "300.000000", 2024-09-04T17:45:54.042 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-04T17:45:54.042 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_use_min_delay_socket": "false", 2024-09-04T17:45:54.042 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_older_version_delay": "604800", 2024-09-04T17:45:54.042 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-04T17:45:54.042 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-04T17:45:54.042 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-04T17:45:54.042 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-04T17:45:54.042 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-04T17:45:54.042 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-04T17:45:54.042 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-04T17:45:54.042 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_misplaced": "false", 2024-09-04T17:45:54.043 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-04T17:45:54.043 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_older_version": "true", 2024-09-04T17:45:54.043 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-04T17:45:54.043 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_pool_no_app": "false", 2024-09-04T17:45:54.043 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-04T17:45:54.043 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-04T17:45:54.043 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-04T17:45:54.043 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-04T17:45:54.043 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-04T17:45:54.043 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-04T17:45:54.043 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-04T17:45:54.043 INFO:teuthology.orchestra.run.smithi092.stdout: "monmap": "", 2024-09-04T17:45:54.043 INFO:teuthology.orchestra.run.smithi092.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-04T17:45:54.043 INFO:teuthology.orchestra.run.smithi092.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-04T17:45:54.044 INFO:teuthology.orchestra.run.smithi092.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-04T17:45:54.044 INFO:teuthology.orchestra.run.smithi092.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-04T17:45:54.044 INFO:teuthology.orchestra.run.smithi092.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-04T17:45:54.044 INFO:teuthology.orchestra.run.smithi092.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-04T17:45:54.044 INFO:teuthology.orchestra.run.smithi092.stdout: "motr_tracing_enabled": "false", 2024-09-04T17:45:54.044 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_op_threads": "3", 2024-09-04T17:45:54.044 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-04T17:45:54.044 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_rdma_cm": "false", 2024-09-04T17:45:54.044 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_rdma_device_name": "", 2024-09-04T17:45:54.044 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_rdma_dscp": "96", 2024-09-04T17:45:54.044 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-04T17:45:54.044 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-04T17:45:54.044 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_rdma_local_gid": "", 2024-09-04T17:45:54.045 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-04T17:45:54.045 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_rdma_port_num": "1", 2024-09-04T17:45:54.045 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-04T17:45:54.045 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-04T17:45:54.045 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-04T17:45:54.045 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-04T17:45:54.045 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_rdma_sl": "3", 2024-09-04T17:45:54.045 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_rdma_support_srq": "true", 2024-09-04T17:45:54.045 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_rdma_type": "ib", 2024-09-04T17:45:54.045 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_async_reap_threshold": "5", 2024-09-04T17:45:54.045 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_bind_before_connect": "false", 2024-09-04T17:45:54.045 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_bind_ipv4": "true", 2024-09-04T17:45:54.045 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_bind_ipv6": "false", 2024-09-04T17:45:54.046 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_bind_msgr1": "true", 2024-09-04T17:45:54.046 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_bind_msgr2": "true", 2024-09-04T17:45:54.046 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_bind_port_max": "7568", 2024-09-04T17:45:54.046 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_bind_port_min": "6800", 2024-09-04T17:45:54.046 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-04T17:45:54.046 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_bind_retry_count": "3", 2024-09-04T17:45:54.046 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_bind_retry_delay": "5", 2024-09-04T17:45:54.046 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_blackhole_client": "false", 2024-09-04T17:45:54.046 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_blackhole_mds": "false", 2024-09-04T17:45:54.046 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_blackhole_mgr": "false", 2024-09-04T17:45:54.046 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_blackhole_mon": "false", 2024-09-04T17:45:54.046 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_blackhole_osd": "false", 2024-09-04T17:45:54.046 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_client_mode": "crc secure", 2024-09-04T17:45:54.046 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_client_throttle_retry_time_interval": "5000", 2024-09-04T17:45:54.047 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_cluster_mode": "crc secure", 2024-09-04T17:45:54.047 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_cluster_type": "", 2024-09-04T17:45:54.047 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_compress_secure": "false", 2024-09-04T17:45:54.047 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_connection_idle_timeout": "900", 2024-09-04T17:45:54.047 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_connection_ready_timeout": "10", 2024-09-04T17:45:54.047 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_crc_data": "true", 2024-09-04T17:45:54.047 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_crc_header": "true", 2024-09-04T17:45:54.047 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_die_on_bad_msg": "false", 2024-09-04T17:45:54.047 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_die_on_bug": "true", 2024-09-04T17:45:54.047 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_die_on_old_message": "true", 2024-09-04T17:45:54.047 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_die_on_skipped_message": "false", 2024-09-04T17:45:54.047 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-04T17:45:54.047 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-04T17:45:54.048 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dpdk_coremask": "0xF", 2024-09-04T17:45:54.048 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-04T17:45:54.048 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-04T17:45:54.048 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dpdk_enable_tso": "true", 2024-09-04T17:45:54.048 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-04T17:45:54.048 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-04T17:45:54.048 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dpdk_hugepages": "", 2024-09-04T17:45:54.048 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-04T17:45:54.048 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-04T17:45:54.048 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dpdk_lro": "true", 2024-09-04T17:45:54.048 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dpdk_memory_channel": "4", 2024-09-04T17:45:54.048 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-04T17:45:54.048 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dpdk_pmd": "", 2024-09-04T17:45:54.049 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dpdk_port_id": "0", 2024-09-04T17:45:54.049 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-04T17:45:54.049 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-04T17:45:54.049 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_dump_on_send": "false", 2024-09-04T17:45:54.049 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_initial_backoff": "0.200000", 2024-09-04T17:45:54.049 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_inject_delay_max": "1.000000", 2024-09-04T17:45:54.049 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-04T17:45:54.049 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_inject_delay_type": "", 2024-09-04T17:45:54.049 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-04T17:45:54.049 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_inject_network_congestion": "0", 2024-09-04T17:45:54.049 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_inject_socket_failures": "5000", 2024-09-04T17:45:54.049 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_learn_addr_from_peer": "true", 2024-09-04T17:45:54.049 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_max_accept_failures": "4", 2024-09-04T17:45:54.050 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_max_backoff": "15.000000", 2024-09-04T17:45:54.050 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_mon_client_mode": "secure crc", 2024-09-04T17:45:54.050 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-04T17:45:54.050 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_mon_service_mode": "secure crc", 2024-09-04T17:45:54.050 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_osd_compress_min_size": "1024", 2024-09-04T17:45:54.050 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_osd_compress_mode": "none", 2024-09-04T17:45:54.050 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-04T17:45:54.050 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-04T17:45:54.050 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_pq_min_cost": "65536", 2024-09-04T17:45:54.050 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_public_type": "", 2024-09-04T17:45:54.050 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_service_mode": "crc secure", 2024-09-04T17:45:54.050 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_tcp_listen_backlog": "512", 2024-09-04T17:45:54.050 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_tcp_nodelay": "true", 2024-09-04T17:45:54.050 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-04T17:45:54.051 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_tcp_rcvbuf": "0", 2024-09-04T17:45:54.051 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_time_events_min_wait_interval": "1000", 2024-09-04T17:45:54.051 INFO:teuthology.orchestra.run.smithi092.stdout: "ms_type": "async+posix", 2024-09-04T17:45:54.051 INFO:teuthology.orchestra.run.smithi092.stdout: "no_config_file": "false", 2024-09-04T17:45:54.051 INFO:teuthology.orchestra.run.smithi092.stdout: "nvmeof_mon_client_disconnect_panic": "100", 2024-09-04T17:45:54.051 INFO:teuthology.orchestra.run.smithi092.stdout: "nvmeof_mon_client_tick_period": "2", 2024-09-04T17:45:54.051 INFO:teuthology.orchestra.run.smithi092.stdout: "objecter_completion_locks_per_session": "32", 2024-09-04T17:45:54.051 INFO:teuthology.orchestra.run.smithi092.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-04T17:45:54.051 INFO:teuthology.orchestra.run.smithi092.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-04T17:45:54.051 INFO:teuthology.orchestra.run.smithi092.stdout: "objecter_inflight_ops": "1024", 2024-09-04T17:45:54.051 INFO:teuthology.orchestra.run.smithi092.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-04T17:45:54.051 INFO:teuthology.orchestra.run.smithi092.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-04T17:45:54.051 INFO:teuthology.orchestra.run.smithi092.stdout: "objecter_tick_interval": "5.000000", 2024-09-04T17:45:54.051 INFO:teuthology.orchestra.run.smithi092.stdout: "objecter_timeout": "10.000000", 2024-09-04T17:45:54.052 INFO:teuthology.orchestra.run.smithi092.stdout: "objectstore_blackhole": "false", 2024-09-04T17:45:54.052 INFO:teuthology.orchestra.run.smithi092.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-09-04T17:45:54.052 INFO:teuthology.orchestra.run.smithi092.stdout: "openssl_engine_opts": "", 2024-09-04T17:45:54.052 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_agent_delay_time": "5.000000", 2024-09-04T17:45:54.052 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_agent_hist_halflife": "1000", 2024-09-04T17:45:54.052 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_agent_max_low_ops": "2", 2024-09-04T17:45:54.052 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_agent_max_ops": "4", 2024-09-04T17:45:54.052 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-04T17:45:54.052 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-04T17:45:54.052 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_agent_slop": "0.020000", 2024-09-04T17:45:54.052 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-04T17:45:54.052 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-04T17:45:54.052 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_asio_thread_count": "2", 2024-09-04T17:45:54.053 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_async_recovery_min_cost": "100", 2024-09-04T17:45:54.053 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-04T17:45:54.053 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-04T17:45:54.053 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_backfill_scan_max": "512", 2024-09-04T17:45:54.053 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_backfill_scan_min": "64", 2024-09-04T17:45:54.053 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_backoff_on_degraded": "false", 2024-09-04T17:45:54.053 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_backoff_on_peering": "false", 2024-09-04T17:45:54.053 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_backoff_on_unfound": "true", 2024-09-04T17:45:54.053 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_beacon_report_interval": "300", 2024-09-04T17:45:54.053 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_bench_duration": "30", 2024-09-04T17:45:54.053 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-04T17:45:54.053 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_bench_max_block_size": "67108864", 2024-09-04T17:45:54.053 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-04T17:45:54.054 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_blkin_trace_all": "false", 2024-09-04T17:45:54.054 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-04T17:45:54.054 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-04T17:45:54.054 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-04T17:45:54.054 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-04T17:45:54.054 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_check_for_log_corruption": "false", 2024-09-04T17:45:54.054 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-04T17:45:54.054 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-04T17:45:54.054 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_class_dir": "/usr/lib/rados-classes", 2024-09-04T17:45:54.054 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-04T17:45:54.054 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_class_update_on_start": "true", 2024-09-04T17:45:54.054 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_client_message_cap": "256", 2024-09-04T17:45:54.054 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_client_message_size_cap": "524288000", 2024-09-04T17:45:54.055 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_client_op_priority": "63", 2024-09-04T17:45:54.055 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_client_watch_timeout": "30", 2024-09-04T17:45:54.055 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_command_max_records": "256", 2024-09-04T17:45:54.055 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_compact_on_start": "false", 2024-09-04T17:45:54.055 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-04T17:45:54.055 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-04T17:45:54.055 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-04T17:45:54.055 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_crush_update_on_start": "true", 2024-09-04T17:45:54.055 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_crush_update_weight_set": "true", 2024-09-04T17:45:54.055 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-04T17:45:54.055 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-04T17:45:54.055 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-04T17:45:54.055 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-04T17:45:54.055 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-04T17:45:54.056 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_feed_pullee": "-1", 2024-09-04T17:45:54.056 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-04T17:45:54.056 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-04T17:45:54.056 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-04T17:45:54.056 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_misdirected_ops": "false", 2024-09-04T17:45:54.056 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_no_acting_change": "false", 2024-09-04T17:45:54.056 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_no_purge_strays": "false", 2024-09-04T17:45:54.056 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_op_order": "false", 2024-09-04T17:45:54.056 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-04T17:45:54.056 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-04T17:45:54.056 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-04T17:45:54.056 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-04T17:45:54.056 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_shutdown": "false", 2024-09-04T17:45:54.057 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-04T17:45:54.057 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-04T17:45:54.057 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_trim_objects": "false", 2024-09-04T17:45:54.057 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-04T17:45:54.057 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-04T17:45:54.057 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_verify_snaps": "false", 2024-09-04T17:45:54.057 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-04T17:45:54.057 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-04T17:45:54.057 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_deep_scrub_keys": "1024", 2024-09-04T17:45:54.057 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-04T17:45:54.057 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-04T17:45:54.057 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-04T17:45:54.057 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_deep_scrub_stride": "524288", 2024-09-04T17:45:54.058 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-04T17:45:54.058 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-04T17:45:54.058 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_default_notify_timeout": "30", 2024-09-04T17:45:54.058 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_delete_sleep": "0.000000", 2024-09-04T17:45:54.058 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-04T17:45:54.058 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-04T17:45:54.058 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-04T17:45:54.058 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_discard_disconnected_ops": "true", 2024-09-04T17:45:54.058 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_ec_partial_reads": "true", 2024-09-04T17:45:54.058 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_enable_op_tracker": "true", 2024-09-04T17:45:54.058 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-04T17:45:54.058 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-04T17:45:54.058 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-04T17:45:54.058 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-04T17:45:54.059 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_fast_info": "true", 2024-09-04T17:45:54.059 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_fast_shutdown": "true", 2024-09-04T17:45:54.059 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-04T17:45:54.059 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-04T17:45:54.059 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-04T17:45:54.059 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-04T17:45:54.059 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-04T17:45:54.059 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_function_tracing": "false", 2024-09-04T17:45:54.059 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_heartbeat_grace": "20", 2024-09-04T17:45:54.059 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_heartbeat_interval": "6", 2024-09-04T17:45:54.059 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-04T17:45:54.059 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_heartbeat_min_peers": "10", 2024-09-04T17:45:54.059 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_heartbeat_min_size": "2000", 2024-09-04T17:45:54.059 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_heartbeat_stale": "600", 2024-09-04T17:45:54.060 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-04T17:45:54.060 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_hit_set_max_size": "100000", 2024-09-04T17:45:54.060 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_hit_set_min_size": "1000", 2024-09-04T17:45:54.060 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-04T17:45:54.060 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-04T17:45:54.060 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-04T17:45:54.060 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-04T17:45:54.060 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-04T17:45:54.060 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-04T17:45:54.060 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_journal_size": "5120", 2024-09-04T17:45:54.060 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_kill_backfill_at": "0", 2024-09-04T17:45:54.060 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-04T17:45:54.060 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_map_cache_size": "50", 2024-09-04T17:45:54.060 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_map_dedup": "true", 2024-09-04T17:45:54.061 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_map_message_max": "40", 2024-09-04T17:45:54.061 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-04T17:45:54.061 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_map_share_max_epochs": "40", 2024-09-04T17:45:54.061 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_attr_name_len": "100", 2024-09-04T17:45:54.061 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_attr_size": "0", 2024-09-04T17:45:54.061 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_backfills": "1", 2024-09-04T17:45:54.061 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_markdown_count": "5", 2024-09-04T17:45:54.061 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_markdown_period": "600", 2024-09-04T17:45:54.061 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_object_name_len": "2048", 2024-09-04T17:45:54.061 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_object_namespace_len": "256", 2024-09-04T17:45:54.061 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_object_size": "134217728", 2024-09-04T17:45:54.061 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-04T17:45:54.061 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-04T17:45:54.062 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_pg_blocked_by": "16", 2024-09-04T17:45:54.062 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_pg_log_entries": "10000", 2024-09-04T17:45:54.062 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-04T17:45:54.062 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_pgls": "1024", 2024-09-04T17:45:54.062 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_push_cost": "8388608", 2024-09-04T17:45:54.062 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_push_objects": "10", 2024-09-04T17:45:54.062 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_scrubs": "3", 2024-09-04T17:45:54.062 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-04T17:45:54.062 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_trimming_pgs": "2", 2024-09-04T17:45:54.062 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_write_op_reply_len": "64", 2024-09-04T17:45:54.062 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_max_write_size": "90", 2024-09-04T17:45:54.062 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-04T17:45:54.062 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-04T17:45:54.063 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-04T17:45:54.063 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-04T17:45:54.063 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-04T17:45:54.063 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-04T17:45:54.063 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-04T17:45:54.063 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-04T17:45:54.063 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_profile": "balanced", 2024-09-04T17:45:54.063 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-04T17:45:54.063 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-04T17:45:54.063 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-04T17:45:54.063 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-04T17:45:54.063 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-04T17:45:54.063 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-04T17:45:54.064 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-04T17:45:54.064 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-04T17:45:54.064 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-04T17:45:54.064 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-04T17:45:54.064 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-04T17:45:54.064 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_memory_base": "805306368", 2024-09-04T17:45:54.064 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_memory_cache_min": "134217728", 2024-09-04T17:45:54.064 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-04T17:45:54.064 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-04T17:45:54.064 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_memory_target": "4294967296", 2024-09-04T17:45:54.064 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_memory_target_autotune": "false", 2024-09-04T17:45:54.064 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-04T17:45:54.064 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_min_pg_log_entries": "250", 2024-09-04T17:45:54.065 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_min_recovery_priority": "0", 2024-09-04T17:45:54.065 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-04T17:45:54.065 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-04T17:45:54.065 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mon_report_interval": "5", 2024-09-04T17:45:54.065 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-04T17:45:54.065 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-04T17:45:54.065 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_num_cache_shards": "32", 2024-09-04T17:45:54.065 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_num_op_tracker_shard": "32", 2024-09-04T17:45:54.065 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_numa_auto_affinity": "true", 2024-09-04T17:45:54.065 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_numa_node": "-1", 2024-09-04T17:45:54.065 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_numa_prefer_iface": "true", 2024-09-04T17:45:54.065 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-04T17:45:54.065 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_objecter_finishers": "1", 2024-09-04T17:45:54.065 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_objectstore": "bluestore", 2024-09-04T17:45:54.066 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_objectstore_fuse": "false", 2024-09-04T17:45:54.066 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_objectstore_ideal_list_max": "64", 2024-09-04T17:45:54.066 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_objectstore_tracing": "false", 2024-09-04T17:45:54.066 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_complaint_time": "30.000000", 2024-09-04T17:45:54.066 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_history_duration": "600", 2024-09-04T17:45:54.066 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_history_size": "20", 2024-09-04T17:45:54.066 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_history_slow_op_size": "20", 2024-09-04T17:45:54.066 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-04T17:45:54.066 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_log_threshold": "5", 2024-09-04T17:45:54.066 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_num_shards": "0", 2024-09-04T17:45:54.066 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_num_shards_hdd": "5", 2024-09-04T17:45:54.066 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_num_shards_ssd": "8", 2024-09-04T17:45:54.066 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-04T17:45:54.066 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-04T17:45:54.067 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-04T17:45:54.067 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-04T17:45:54.067 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_pq_min_cost": "65536", 2024-09-04T17:45:54.067 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-04T17:45:54.067 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_queue_cut_off": "high", 2024-09-04T17:45:54.067 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-04T17:45:54.067 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_op_thread_timeout": "15", 2024-09-04T17:45:54.067 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_open_classes_on_start": "true", 2024-09-04T17:45:54.067 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_os_flags": "0", 2024-09-04T17:45:54.067 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_peering_op_priority": "255", 2024-09-04T17:45:54.067 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pg_delete_cost": "1048576", 2024-09-04T17:45:54.067 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pg_delete_priority": "5", 2024-09-04T17:45:54.067 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-04T17:45:54.068 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-04T17:45:54.068 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-04T17:45:54.068 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pg_log_trim_max": "10000", 2024-09-04T17:45:54.068 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pg_log_trim_min": "100", 2024-09-04T17:45:54.068 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-04T17:45:54.068 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-04T17:45:54.068 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-09-04T17:45:54.068 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-09-04T17:45:54.068 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-04T17:45:54.068 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-04T17:45:54.068 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-04T17:45:54.068 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-04T17:45:54.068 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-04T17:45:54.069 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-04T17:45:54.069 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_crimson": "false", 2024-09-04T17:45:54.069 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-04T17:45:54.069 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-04T17:45:54.069 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-04T17:45:54.069 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-04T17:45:54.069 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-04T17:45:54.069 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-04T17:45:54.069 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-04T17:45:54.069 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-04T17:45:54.069 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_flags": "0", 2024-09-04T17:45:54.069 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-04T17:45:54.069 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_min_size": "0", 2024-09-04T17:45:54.070 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-04T17:45:54.070 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_pg_num": "32", 2024-09-04T17:45:54.070 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_pgp_num": "0", 2024-09-04T17:45:54.070 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-04T17:45:54.070 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_read_ratio": "70", 2024-09-04T17:45:54.070 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_size": "2", 2024-09-04T17:45:54.070 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_default_type": "replicated", 2024-09-04T17:45:54.070 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-04T17:45:54.070 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-04T17:45:54.070 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_push_per_object_cost": "1000", 2024-09-04T17:45:54.070 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-04T17:45:54.070 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recover_clone_overlap": "true", 2024-09-04T17:45:54.070 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-04T17:45:54.070 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recovery_cost": "20971520", 2024-09-04T17:45:54.071 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-04T17:45:54.071 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recovery_max_active": "0", 2024-09-04T17:45:54.071 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-04T17:45:54.071 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-04T17:45:54.071 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-04T17:45:54.071 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-04T17:45:54.071 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recovery_max_single_start": "1", 2024-09-04T17:45:54.071 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recovery_op_priority": "3", 2024-09-04T17:45:54.071 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-04T17:45:54.071 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recovery_priority": "5", 2024-09-04T17:45:54.071 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-04T17:45:54.071 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recovery_sleep": "0.000000", 2024-09-04T17:45:54.071 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-04T17:45:54.071 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-04T17:45:54.071 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-04T17:45:54.072 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_repair_during_recovery": "false", 2024-09-04T17:45:54.072 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_requested_scrub_priority": "120", 2024-09-04T17:45:54.072 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-04T17:45:54.072 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-04T17:45:54.072 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_auto_repair": "false", 2024-09-04T17:45:54.072 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-04T17:45:54.072 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-04T17:45:54.072 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_begin_hour": "0", 2024-09-04T17:45:54.072 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_begin_week_day": "0", 2024-09-04T17:45:54.072 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_chunk_max": "25", 2024-09-04T17:45:54.072 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_chunk_min": "5", 2024-09-04T17:45:54.072 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_cost": "52428800", 2024-09-04T17:45:54.072 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_disable_reservation_queuing": "false", 2024-09-04T17:45:54.073 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_during_recovery": "false", 2024-09-04T17:45:54.073 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_end_hour": "0", 2024-09-04T17:45:54.073 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_end_week_day": "0", 2024-09-04T17:45:54.073 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_event_cost": "4096", 2024-09-04T17:45:54.073 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-04T17:45:54.073 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-04T17:45:54.073 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_invalid_stats": "true", 2024-09-04T17:45:54.073 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-04T17:45:54.073 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-04T17:45:54.073 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_max_preemptions": "5", 2024-09-04T17:45:54.073 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-04T17:45:54.073 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_priority": "5", 2024-09-04T17:45:54.073 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_scrub_sleep": "0.000000", 2024-09-04T17:45:54.074 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-04T17:45:54.074 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-04T17:45:54.074 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-04T17:45:54.074 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_skip_check_past_interval_bounds": "false", 2024-09-04T17:45:54.074 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_skip_data_digest": "false", 2024-09-04T17:45:54.074 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_smart_report_timeout": "5", 2024-09-04T17:45:54.074 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_snap_trim_cost": "1048576", 2024-09-04T17:45:54.074 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_snap_trim_priority": "5", 2024-09-04T17:45:54.074 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-04T17:45:54.074 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-04T17:45:54.074 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-04T17:45:54.074 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-04T17:45:54.074 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-04T17:45:54.074 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-04T17:45:54.075 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-04T17:45:54.075 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_target_transaction_size": "30", 2024-09-04T17:45:54.075 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-04T17:45:54.075 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-04T17:45:54.075 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-04T17:45:54.075 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-04T17:45:54.075 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-04T17:45:54.075 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-04T17:45:54.075 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-04T17:45:54.075 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-04T17:45:54.075 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-04T17:45:54.075 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-04T17:45:54.075 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_tracing": "false", 2024-09-04T17:45:54.076 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_use_stale_snap": "false", 2024-09-04T17:45:54.076 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-04T17:45:54.076 INFO:teuthology.orchestra.run.smithi092.stdout: "osdc_blkin_trace_all": "false", 2024-09-04T17:45:54.076 INFO:teuthology.orchestra.run.smithi092.stdout: "paxos_kill_at": "0", 2024-09-04T17:45:54.076 INFO:teuthology.orchestra.run.smithi092.stdout: "paxos_max_join_drift": "10", 2024-09-04T17:45:54.076 INFO:teuthology.orchestra.run.smithi092.stdout: "paxos_min": "500", 2024-09-04T17:45:54.076 INFO:teuthology.orchestra.run.smithi092.stdout: "paxos_min_wait": "0.050000", 2024-09-04T17:45:54.076 INFO:teuthology.orchestra.run.smithi092.stdout: "paxos_propose_interval": "1.000000", 2024-09-04T17:45:54.076 INFO:teuthology.orchestra.run.smithi092.stdout: "paxos_service_trim_max": "500", 2024-09-04T17:45:54.076 INFO:teuthology.orchestra.run.smithi092.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-04T17:45:54.076 INFO:teuthology.orchestra.run.smithi092.stdout: "paxos_service_trim_min": "250", 2024-09-04T17:45:54.076 INFO:teuthology.orchestra.run.smithi092.stdout: "paxos_stash_full_interval": "25", 2024-09-04T17:45:54.076 INFO:teuthology.orchestra.run.smithi092.stdout: "paxos_trim_max": "500", 2024-09-04T17:45:54.077 INFO:teuthology.orchestra.run.smithi092.stdout: "paxos_trim_min": "250", 2024-09-04T17:45:54.077 INFO:teuthology.orchestra.run.smithi092.stdout: "perf": "true", 2024-09-04T17:45:54.077 INFO:teuthology.orchestra.run.smithi092.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-04T17:45:54.077 INFO:teuthology.orchestra.run.smithi092.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-04T17:45:54.077 INFO:teuthology.orchestra.run.smithi092.stdout: "plugin_dir": "/usr/lib/ceph", 2024-09-04T17:45:54.077 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "-", 2024-09-04T17:45:54.077 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrv": "", 2024-09-04T17:45:54.077 INFO:teuthology.orchestra.run.smithi092.stdout: "public_bind_addr": "-", 2024-09-04T17:45:54.077 INFO:teuthology.orchestra.run.smithi092.stdout: "public_network": "", 2024-09-04T17:45:54.077 INFO:teuthology.orchestra.run.smithi092.stdout: "public_network_interface": "", 2024-09-04T17:45:54.077 INFO:teuthology.orchestra.run.smithi092.stdout: "qat_compressor_busy_polling": "false", 2024-09-04T17:45:54.077 INFO:teuthology.orchestra.run.smithi092.stdout: "qat_compressor_enabled": "false", 2024-09-04T17:45:54.077 INFO:teuthology.orchestra.run.smithi092.stdout: "qat_compressor_session_max_number": "256", 2024-09-04T17:45:54.078 INFO:teuthology.orchestra.run.smithi092.stdout: "rados_mon_op_timeout": "0", 2024-09-04T17:45:54.078 INFO:teuthology.orchestra.run.smithi092.stdout: "rados_osd_op_timeout": "0", 2024-09-04T17:45:54.078 INFO:teuthology.orchestra.run.smithi092.stdout: "rados_tracing": "false", 2024-09-04T17:45:54.078 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_atime_update_interval": "60", 2024-09-04T17:45:54.078 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-04T17:45:54.078 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_balance_parent_reads": "false", 2024-09-04T17:45:54.078 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_balance_snap_reads": "false", 2024-09-04T17:45:54.078 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_blkin_trace_all": "false", 2024-09-04T17:45:54.078 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-04T17:45:54.078 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-04T17:45:54.078 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_cache": "true", 2024-09-04T17:45:54.078 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-04T17:45:54.078 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-04T17:45:54.079 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-04T17:45:54.079 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-04T17:45:54.079 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_cache_policy": "writearound", 2024-09-04T17:45:54.079 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_cache_size": "33554432", 2024-09-04T17:45:54.079 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-04T17:45:54.079 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-04T17:45:54.079 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_clone_copy_on_read": "false", 2024-09-04T17:45:54.079 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_compression_hint": "none", 2024-09-04T17:45:54.079 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_concurrent_management_ops": "10", 2024-09-04T17:45:54.080 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-04T17:45:54.080 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_default_clone_format": "auto", 2024-09-04T17:45:54.080 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_default_data_pool": "", 2024-09-04T17:45:54.080 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_default_features": "61", 2024-09-04T17:45:54.080 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_default_format": "2", 2024-09-04T17:45:54.080 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_default_map_options": "", 2024-09-04T17:45:54.080 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_default_order": "22", 2024-09-04T17:45:54.080 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_default_pool": "rbd", 2024-09-04T17:45:54.080 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-04T17:45:54.080 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_default_stripe_count": "0", 2024-09-04T17:45:54.080 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_default_stripe_unit": "0", 2024-09-04T17:45:54.080 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-04T17:45:54.081 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-04T17:45:54.081 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-04T17:45:54.081 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_enable_alloc_hint": "true", 2024-09-04T17:45:54.081 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-04T17:45:54.081 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_io_scheduler": "simple", 2024-09-04T17:45:54.081 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-04T17:45:54.081 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-04T17:45:54.081 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-04T17:45:54.081 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-04T17:45:54.081 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-04T17:45:54.081 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-04T17:45:54.081 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-04T17:45:54.081 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-04T17:45:54.081 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-04T17:45:54.082 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_journal_order": "24", 2024-09-04T17:45:54.082 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_journal_pool": "", 2024-09-04T17:45:54.082 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_journal_splay_width": "4", 2024-09-04T17:45:54.082 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_localize_parent_reads": "false", 2024-09-04T17:45:54.082 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_localize_snap_reads": "false", 2024-09-04T17:45:54.082 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-04T17:45:54.082 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-04T17:45:54.082 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-04T17:45:54.082 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-04T17:45:54.082 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-04T17:45:54.082 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-04T17:45:54.082 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-04T17:45:54.082 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-04T17:45:54.083 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-04T17:45:54.083 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-04T17:45:54.083 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-04T17:45:54.083 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-04T17:45:54.083 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-04T17:45:54.083 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-04T17:45:54.083 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-04T17:45:54.083 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-04T17:45:54.083 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-04T17:45:54.083 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-04T17:45:54.083 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-04T17:45:54.083 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-04T17:45:54.084 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-04T17:45:54.084 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-04T17:45:54.084 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-04T17:45:54.084 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-04T17:45:54.084 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-04T17:45:54.084 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-04T17:45:54.084 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-04T17:45:54.084 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-04T17:45:54.084 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-04T17:45:54.084 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-04T17:45:54.084 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-04T17:45:54.084 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_mtime_update_interval": "60", 2024-09-04T17:45:54.084 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_non_blocking_aio": "true", 2024-09-04T17:45:54.085 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_op_thread_timeout": "60", 2024-09-04T17:45:54.085 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_op_threads": "1", 2024-09-04T17:45:54.085 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_parent_cache_enabled": "false", 2024-09-04T17:45:54.085 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-04T17:45:54.085 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-04T17:45:54.085 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-04T17:45:54.085 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_plugins": "", 2024-09-04T17:45:54.085 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_bps_burst": "0", 2024-09-04T17:45:54.085 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-04T17:45:54.085 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_bps_limit": "0", 2024-09-04T17:45:54.085 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_exclude_ops": "0", 2024-09-04T17:45:54.085 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_iops_burst": "0", 2024-09-04T17:45:54.085 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-04T17:45:54.085 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_iops_limit": "0", 2024-09-04T17:45:54.086 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-04T17:45:54.086 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-04T17:45:54.086 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-04T17:45:54.086 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-04T17:45:54.086 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-04T17:45:54.086 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-04T17:45:54.086 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-04T17:45:54.086 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-04T17:45:54.086 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-04T17:45:54.086 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-04T17:45:54.086 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-04T17:45:54.086 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-04T17:45:54.086 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-04T17:45:54.086 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-04T17:45:54.087 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_read_from_replica_policy": "default", 2024-09-04T17:45:54.087 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-04T17:45:54.087 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-04T17:45:54.087 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-04T17:45:54.087 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-04T17:45:54.087 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_skip_partial_discard": "true", 2024-09-04T17:45:54.087 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-04T17:45:54.087 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_tracing": "false", 2024-09-04T17:45:54.087 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_validate_names": "true", 2024-09-04T17:45:54.087 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd_validate_pool": "true", 2024-09-04T17:45:54.087 INFO:teuthology.orchestra.run.smithi092.stdout: "restapi_base_url": "", 2024-09-04T17:45:54.087 INFO:teuthology.orchestra.run.smithi092.stdout: "restapi_log_level": "", 2024-09-04T17:45:54.087 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_account_default_quota_max_objects": "-1", 2024-09-04T17:45:54.088 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_account_default_quota_max_size": "-1", 2024-09-04T17:45:54.088 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_acl_grants_max_num": "100", 2024-09-04T17:45:54.088 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_admin_entry": "admin", 2024-09-04T17:45:54.088 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-04T17:45:54.088 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_asio_assert_yielding": "false", 2024-09-04T17:45:54.088 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_backend_store": "rados", 2024-09-04T17:45:54.088 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_barbican_url": "", 2024-09-04T17:45:54.088 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_beast_enable_async": "true", 2024-09-04T17:45:54.088 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_bucket_counters_cache": "false", 2024-09-04T17:45:54.088 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-09-04T17:45:54.088 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-04T17:45:54.088 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-04T17:45:54.088 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-04T17:45:54.089 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-04T17:45:54.089 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-04T17:45:54.089 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-04T17:45:54.089 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-04T17:45:54.089 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_cache_enabled": "true", 2024-09-04T17:45:54.089 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_cache_expiry_interval": "900", 2024-09-04T17:45:54.089 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_cache_lru_size": "25000", 2024-09-04T17:45:54.089 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_config_store": "rados", 2024-09-04T17:45:54.089 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_content_length_compat": "false", 2024-09-04T17:45:54.089 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_copy_obj_progress": "true", 2024-09-04T17:45:54.089 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-04T17:45:54.089 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_cors_rules_max_num": "100", 2024-09-04T17:45:54.089 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_cross_domain_policy": "", 2024-09-04T17:45:54.090 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-04T17:45:54.090 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_kmip_addr": "", 2024-09-04T17:45:54.090 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-04T17:45:54.090 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-04T17:45:54.090 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-04T17:45:54.090 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-04T17:45:54.090 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_kmip_password": "", 2024-09-04T17:45:54.090 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-04T17:45:54.090 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_kmip_username": "", 2024-09-04T17:45:54.090 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_require_ssl": "true", 2024-09-04T17:45:54.090 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-04T17:45:54.090 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-04T17:45:54.090 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-04T17:45:54.090 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-04T17:45:54.091 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-04T17:45:54.091 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-04T17:45:54.091 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-04T17:45:54.091 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-04T17:45:54.091 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-04T17:45:54.091 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-04T17:45:54.091 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-04T17:45:54.091 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-04T17:45:54.091 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-04T17:45:54.091 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-04T17:45:54.091 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-04T17:45:54.091 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_vault_addr": "", 2024-09-04T17:45:54.091 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_vault_auth": "token", 2024-09-04T17:45:54.092 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_vault_namespace": "", 2024-09-04T17:45:54.092 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_vault_prefix": "", 2024-09-04T17:45:54.092 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-04T17:45:54.092 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-04T17:45:54.092 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-04T17:45:54.092 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-04T17:45:54.092 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_vault_token_file": "", 2024-09-04T17:45:54.092 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-04T17:45:54.092 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_curl_buffersize": "524288", 2024-09-04T17:45:54.092 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-04T17:45:54.092 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_curl_low_speed_time": "300", 2024-09-04T17:45:54.092 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-04T17:45:54.092 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-04T17:45:54.093 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-04T17:45:54.093 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-04T17:45:54.093 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-04T17:45:54.093 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-04T17:45:54.093 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-04T17:45:54.093 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-04T17:45:54.093 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-04T17:45:54.093 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-04T17:45:54.093 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_d4n_address": "127.0.0.1:6379", 2024-09-04T17:45:54.093 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_d4n_l1_datacache_address": "127.0.0.1:6379", 2024-09-04T17:45:54.093 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_d4n_l1_datacache_persistent_path": "/tmp/rgw_d4n_datacache/", 2024-09-04T17:45:54.093 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_d4n_l1_datacache_size": "1073741824", 2024-09-04T17:45:54.093 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_d4n_l1_evict_cache_on_start": "true", 2024-09-04T17:45:54.094 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_d4n_l1_fadvise": "4", 2024-09-04T17:45:54.094 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_d4n_libaio_aio_num": "64", 2024-09-04T17:45:54.094 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_d4n_libaio_aio_threads": "20", 2024-09-04T17:45:54.094 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-04T17:45:54.094 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_data_log_changes_size": "1000", 2024-09-04T17:45:54.094 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_data_log_num_shards": "128", 2024-09-04T17:45:54.094 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-04T17:45:54.094 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_data_log_window": "30", 2024-09-04T17:45:54.094 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-04T17:45:54.094 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-04T17:45:54.094 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-04T17:45:54.094 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-09-04T17:45:54.094 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-04T17:45:54.094 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-04T17:45:54.095 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-04T17:45:54.095 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-04T17:45:54.095 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-04T17:45:54.095 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-04T17:45:54.095 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-04T17:45:54.095 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-04T17:45:54.095 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-04T17:45:54.095 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_disable_s3select": "false", 2024-09-04T17:45:54.095 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-04T17:45:54.095 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-04T17:45:54.095 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-04T17:45:54.095 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-04T17:45:54.095 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-04T17:45:54.095 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-04T17:45:54.096 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-04T17:45:54.096 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-04T17:45:54.096 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-04T17:45:54.096 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-04T17:45:54.096 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-04T17:45:54.096 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-04T17:45:54.096 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_dns_name": "", 2024-09-04T17:45:54.096 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_dns_s3website_name": "", 2024-09-04T17:45:54.096 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_dynamic_resharding": "true", 2024-09-04T17:45:54.096 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_dynamic_resharding_may_reduce": "true", 2024-09-04T17:45:54.096 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_dynamic_resharding_reduction_wait": "120", 2024-09-04T17:45:54.096 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-04T17:45:54.096 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_enable_gc_threads": "true", 2024-09-04T17:45:54.097 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_enable_lc_threads": "true", 2024-09-04T17:45:54.097 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_enable_mdsearch": "true", 2024-09-04T17:45:54.097 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_enable_ops_log": "false", 2024-09-04T17:45:54.097 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_enable_quota_threads": "true", 2024-09-04T17:45:54.097 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_enable_static_website": "false", 2024-09-04T17:45:54.097 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_enable_usage_log": "false", 2024-09-04T17:45:54.097 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_enforce_swift_acls": "true", 2024-09-04T17:45:54.097 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_exit_timeout_secs": "120", 2024-09-04T17:45:54.097 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_expose_bucket": "false", 2024-09-04T17:45:54.097 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_extended_http_attrs": "", 2024-09-04T17:45:54.097 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_filter": "none", 2024-09-04T17:45:54.097 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-04T17:45:54.098 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_frontends": "beast port=7480", 2024-09-04T17:45:54.098 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-04T17:45:54.098 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_gc_max_deferred": "50", 2024-09-04T17:45:54.098 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-04T17:45:54.098 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_gc_max_objs": "32", 2024-09-04T17:45:54.098 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-04T17:45:54.098 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-04T17:45:54.098 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-04T17:45:54.098 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-04T17:45:54.098 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_gc_processor_period": "3600", 2024-09-04T17:45:54.098 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-04T17:45:54.098 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-04T17:45:54.098 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-04T17:45:54.099 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-04T17:45:54.099 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_init_timeout": "300", 2024-09-04T17:45:54.099 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-04T17:45:54.099 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-04T17:45:54.099 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_kafka_connection_idle": "300", 2024-09-04T17:45:54.099 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_kafka_message_timeout": "5000", 2024-09-04T17:45:54.099 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_kafka_sleep_timeout": "10", 2024-09-04T17:45:54.099 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-04T17:45:54.099 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-09-04T17:45:54.099 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-04T17:45:54.099 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_admin_domain": "", 2024-09-04T17:45:54.099 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_admin_password": "", 2024-09-04T17:45:54.099 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_admin_password_path": "", 2024-09-04T17:45:54.100 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_admin_project": "", 2024-09-04T17:45:54.100 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_admin_tenant": "", 2024-09-04T17:45:54.100 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_admin_token": "", 2024-09-04T17:45:54.100 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_admin_token_path": "", 2024-09-04T17:45:54.100 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_admin_user": "", 2024-09-04T17:45:54.100 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_barbican_domain": "", 2024-09-04T17:45:54.100 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_barbican_password": "", 2024-09-04T17:45:54.100 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_barbican_project": "", 2024-09-04T17:45:54.100 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-04T17:45:54.100 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_barbican_user": "", 2024-09-04T17:45:54.100 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-04T17:45:54.100 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-04T17:45:54.100 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-04T17:45:54.101 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-04T17:45:54.101 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-04T17:45:54.101 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_url": "", 2024-09-04T17:45:54.101 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-04T17:45:54.101 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_lc_debug_interval": "-1", 2024-09-04T17:45:54.101 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_lc_lock_max_time": "90", 2024-09-04T17:45:54.101 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_lc_max_objs": "32", 2024-09-04T17:45:54.101 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_lc_max_rules": "1000", 2024-09-04T17:45:54.101 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_lc_max_worker": "3", 2024-09-04T17:45:54.101 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-04T17:45:54.101 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_lc_thread_delay": "0", 2024-09-04T17:45:54.101 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-04T17:45:54.101 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_ldap_dnattr": "uid", 2024-09-04T17:45:54.102 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-04T17:45:54.102 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_ldap_searchfilter": "", 2024-09-04T17:45:54.102 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-04T17:45:54.102 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_ldap_uri": "", 2024-09-04T17:45:54.102 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_lfuda_sync_frequency": "60", 2024-09-04T17:45:54.102 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-04T17:45:54.102 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-04T17:45:54.103 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-04T17:45:54.104 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_log_http_headers": "", 2024-09-04T17:45:54.104 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-04T17:45:54.104 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-04T17:45:54.104 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_log_object_name_utc": "false", 2024-09-04T17:45:54.104 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-09-04T17:45:54.104 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-09-04T17:45:54.104 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_max_attr_name_len": "0", 2024-09-04T17:45:54.104 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_max_attr_size": "0", 2024-09-04T17:45:54.104 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-04T17:45:54.104 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_max_chunk_size": "4194304", 2024-09-04T17:45:54.104 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-04T17:45:54.104 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-04T17:45:54.104 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-04T17:45:54.104 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_max_listing_results": "1000", 2024-09-04T17:45:54.105 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_max_notify_retries": "10", 2024-09-04T17:45:54.105 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-04T17:45:54.105 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_max_put_param_size": "1048576", 2024-09-04T17:45:54.105 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_max_put_size": "5368709120", 2024-09-04T17:45:54.105 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_max_slo_entries": "1000", 2024-09-04T17:45:54.105 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_md_log_max_shards": "64", 2024-09-04T17:45:54.105 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-04T17:45:54.105 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-04T17:45:54.105 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-04T17:45:54.105 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-04T17:45:54.105 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_mp_lock_max_time": "600", 2024-09-04T17:45:54.105 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-04T17:45:54.105 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-04T17:45:54.106 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-04T17:45:54.106 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-04T17:45:54.106 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-04T17:45:54.106 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-04T17:45:54.106 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-04T17:45:54.106 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-04T17:45:54.106 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_nfs_max_gc": "300", 2024-09-04T17:45:54.106 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-04T17:45:54.106 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-04T17:45:54.106 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-04T17:45:54.106 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-04T17:45:54.106 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-04T17:45:54.106 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-04T17:45:54.107 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-04T17:45:54.107 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_num_async_rados_threads": "32", 2024-09-04T17:45:54.107 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_num_control_oids": "8", 2024-09-04T17:45:54.107 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_numa_node": "-1", 2024-09-04T17:45:54.107 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-04T17:45:54.107 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-04T17:45:54.107 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_objexp_chunk_size": "100", 2024-09-04T17:45:54.107 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_objexp_gc_interval": "600", 2024-09-04T17:45:54.107 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-04T17:45:54.107 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-04T17:45:54.107 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-04T17:45:54.107 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_op_thread_timeout": "600", 2024-09-04T17:45:54.107 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_op_tracing": "false", 2024-09-04T17:45:54.107 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_opa_token": "", 2024-09-04T17:45:54.108 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_opa_url": "", 2024-09-04T17:45:54.108 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_opa_verify_ssl": "true", 2024-09-04T17:45:54.108 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-04T17:45:54.108 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-04T17:45:54.108 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_ops_log_rados": "false", 2024-09-04T17:45:54.108 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_ops_log_socket_path": "", 2024-09-04T17:45:54.108 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-04T17:45:54.108 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-04T17:45:54.108 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-04T17:45:54.108 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_period_push_interval": "2.000000", 2024-09-04T17:45:54.108 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-04T17:45:54.108 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-04T17:45:54.108 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-04T17:45:54.109 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-09-04T17:45:54.109 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_posix_cache_lanes": "3", 2024-09-04T17:45:54.109 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-09-04T17:45:54.109 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_posix_cache_max_buckets": "100", 2024-09-04T17:45:54.109 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_posix_cache_partitions": "3", 2024-09-04T17:45:54.109 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-09-04T17:45:54.109 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_print_continue": "true", 2024-09-04T17:45:54.109 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-04T17:45:54.109 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-04T17:45:54.109 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-04T17:45:54.109 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-04T17:45:54.109 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-04T17:45:54.109 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_rados_tracing": "false", 2024-09-04T17:45:54.109 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_realm": "", 2024-09-04T17:45:54.110 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_realm_id": "", 2024-09-04T17:45:54.110 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-04T17:45:54.110 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_region": "", 2024-09-04T17:45:54.110 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-04T17:45:54.110 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-04T17:45:54.110 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-04T17:45:54.110 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-04T17:45:54.110 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_request_uri": "", 2024-09-04T17:45:54.110 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_reshard_batch_size": "64", 2024-09-04T17:45:54.110 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-04T17:45:54.110 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_reshard_debug_interval": "-1", 2024-09-04T17:45:54.110 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_reshard_max_aio": "128", 2024-09-04T17:45:54.110 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_reshard_num_logs": "16", 2024-09-04T17:45:54.111 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_reshard_thread_interval": "600", 2024-09-04T17:45:54.111 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_resolve_cname": "false", 2024-09-04T17:45:54.111 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-04T17:45:54.111 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_run_sync_thread": "true", 2024-09-04T17:45:54.111 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-09-04T17:45:54.111 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-04T17:45:54.111 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-04T17:45:54.111 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-04T17:45:54.111 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-04T17:45:54.111 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-04T17:45:54.111 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-04T17:45:54.111 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-04T17:45:54.111 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-04T17:45:54.111 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_scheduler_type": "throttler", 2024-09-04T17:45:54.112 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_script_uri": "", 2024-09-04T17:45:54.112 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_service_provider_name": "", 2024-09-04T17:45:54.112 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-04T17:45:54.112 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sts_client_id": "", 2024-09-04T17:45:54.112 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sts_client_secret": "", 2024-09-04T17:45:54.112 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sts_entry": "sts", 2024-09-04T17:45:54.112 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sts_key": "sts", 2024-09-04T17:45:54.112 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-04T17:45:54.112 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sts_min_session_duration": "900", 2024-09-04T17:45:54.112 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sts_token_introspection_url": "", 2024-09-04T17:45:54.112 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_swift_account_in_url": "false", 2024-09-04T17:45:54.112 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_swift_auth_entry": "auth", 2024-09-04T17:45:54.112 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_swift_auth_url": "", 2024-09-04T17:45:54.113 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_swift_custom_header": "", 2024-09-04T17:45:54.113 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-04T17:45:54.113 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_swift_need_stats": "true", 2024-09-04T17:45:54.113 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_swift_tenant_name": "", 2024-09-04T17:45:54.113 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_swift_token_expiration": "86400", 2024-09-04T17:45:54.113 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_swift_url": "", 2024-09-04T17:45:54.113 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_swift_url_prefix": "swift", 2024-09-04T17:45:54.113 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-04T17:45:54.113 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-04T17:45:54.113 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-04T17:45:54.113 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sync_lease_period": "120", 2024-09-04T17:45:54.113 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-04T17:45:54.113 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-04T17:45:54.114 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-04T17:45:54.114 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-04T17:45:54.114 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-04T17:45:54.114 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-04T17:45:54.114 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-04T17:45:54.114 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-04T17:45:54.114 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-04T17:45:54.114 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_thread_pool_size": "512", 2024-09-04T17:45:54.114 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_topic_persistency_max_retries": "0", 2024-09-04T17:45:54.114 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-09-04T17:45:54.114 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-09-04T17:45:54.114 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_topic_require_publish_policy": "false", 2024-09-04T17:45:54.114 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_torrent_comment": "", 2024-09-04T17:45:54.115 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_torrent_createby": "", 2024-09-04T17:45:54.115 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_torrent_encoding": "", 2024-09-04T17:45:54.115 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_torrent_flag": "false", 2024-09-04T17:45:54.115 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_torrent_max_size": "5368709120", 2024-09-04T17:45:54.115 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_torrent_origin": "", 2024-09-04T17:45:54.115 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-04T17:45:54.115 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_torrent_tracker": "", 2024-09-04T17:45:54.115 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_trust_forwarded_https": "false", 2024-09-04T17:45:54.115 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-04T17:45:54.115 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-04T17:45:54.115 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_usage_max_shards": "32", 2024-09-04T17:45:54.115 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_usage_max_user_shards": "1", 2024-09-04T17:45:54.115 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_use_opa_authz": "false", 2024-09-04T17:45:54.115 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_user_counters_cache": "false", 2024-09-04T17:45:54.116 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_user_counters_cache_size": "10000", 2024-09-04T17:45:54.116 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-04T17:45:54.116 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-04T17:45:54.116 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_user_max_buckets": "1000", 2024-09-04T17:45:54.116 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_user_policies_max_num": "100", 2024-09-04T17:45:54.116 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-04T17:45:54.116 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-04T17:45:54.116 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-04T17:45:54.116 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-04T17:45:54.116 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_user_unique_email": "true", 2024-09-04T17:45:54.116 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_verify_ssl": "true", 2024-09-04T17:45:54.116 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-04T17:45:54.116 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_zone": "", 2024-09-04T17:45:54.117 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_zone_id": "", 2024-09-04T17:45:54.117 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-04T17:45:54.117 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_zonegroup": "", 2024-09-04T17:45:54.117 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_zonegroup_id": "", 2024-09-04T17:45:54.117 INFO:teuthology.orchestra.run.smithi092.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-04T17:45:54.117 INFO:teuthology.orchestra.run.smithi092.stdout: "rgwlc_auto_session_clear": "true", 2024-09-04T17:45:54.117 INFO:teuthology.orchestra.run.smithi092.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-04T17:45:54.117 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_block_size": "4096", 2024-09-04T17:45:54.117 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-04T17:45:54.117 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-04T17:45:54.117 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-04T17:45:54.117 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-04T17:45:54.117 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-04T17:45:54.117 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_cache_size": "536870912", 2024-09-04T17:45:54.118 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-04T17:45:54.118 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-04T17:45:54.118 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-04T17:45:54.118 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-04T17:45:54.118 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-04T17:45:54.118 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-04T17:45:54.118 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-04T17:45:54.118 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-04T17:45:54.118 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_index_type": "binary_search", 2024-09-04T17:45:54.118 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-04T17:45:54.118 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-04T17:45:54.118 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_partition_filters": "false", 2024-09-04T17:45:54.119 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_perf": "false", 2024-09-04T17:45:54.119 INFO:teuthology.orchestra.run.smithi092.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-04T17:45:54.119 INFO:teuthology.orchestra.run.smithi092.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-04T17:45:54.119 INFO:teuthology.orchestra.run.smithi092.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-04T17:45:54.119 INFO:teuthology.orchestra.run.smithi092.stdout: "run_dir": "/var/run/ceph", 2024-09-04T17:45:54.119 INFO:teuthology.orchestra.run.smithi092.stdout: "seastore_block_create": "true", 2024-09-04T17:45:54.119 INFO:teuthology.orchestra.run.smithi092.stdout: "seastore_cache_lru_size": "67108864", 2024-09-04T17:45:54.119 INFO:teuthology.orchestra.run.smithi092.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-04T17:45:54.119 INFO:teuthology.orchestra.run.smithi092.stdout: "seastore_data_delta_based_overwrite": "0", 2024-09-04T17:45:54.119 INFO:teuthology.orchestra.run.smithi092.stdout: "seastore_default_max_object_size": "16777216", 2024-09-04T17:45:54.119 INFO:teuthology.orchestra.run.smithi092.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-04T17:45:54.119 INFO:teuthology.orchestra.run.smithi092.stdout: "seastore_device_size": "53687091200", 2024-09-04T17:45:54.119 INFO:teuthology.orchestra.run.smithi092.stdout: "seastore_disable_end_to_end_data_protection": "true", 2024-09-04T17:45:54.119 INFO:teuthology.orchestra.run.smithi092.stdout: "seastore_full_integrity_check": "false", 2024-09-04T17:45:54.120 INFO:teuthology.orchestra.run.smithi092.stdout: "seastore_journal_batch_capacity": "16", 2024-09-04T17:45:54.120 INFO:teuthology.orchestra.run.smithi092.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-04T17:45:54.120 INFO:teuthology.orchestra.run.smithi092.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-04T17:45:54.120 INFO:teuthology.orchestra.run.smithi092.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-04T17:45:54.120 INFO:teuthology.orchestra.run.smithi092.stdout: "seastore_main_device_type": "SSD", 2024-09-04T17:45:54.120 INFO:teuthology.orchestra.run.smithi092.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-04T17:45:54.120 INFO:teuthology.orchestra.run.smithi092.stdout: "seastore_max_data_allocation_size": "32768", 2024-09-04T17:45:54.120 INFO:teuthology.orchestra.run.smithi092.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-04T17:45:54.120 INFO:teuthology.orchestra.run.smithi092.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-04T17:45:54.120 INFO:teuthology.orchestra.run.smithi092.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-04T17:45:54.120 INFO:teuthology.orchestra.run.smithi092.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-04T17:45:54.120 INFO:teuthology.orchestra.run.smithi092.stdout: "seastore_segment_size": "67108864", 2024-09-04T17:45:54.120 INFO:teuthology.orchestra.run.smithi092.stdout: "set_keepcaps": "false", 2024-09-04T17:45:54.121 INFO:teuthology.orchestra.run.smithi092.stdout: "setgroup": "", 2024-09-04T17:45:54.121 INFO:teuthology.orchestra.run.smithi092.stdout: "setuser": "", 2024-09-04T17:45:54.121 INFO:teuthology.orchestra.run.smithi092.stdout: "setuser_match_path": "", 2024-09-04T17:45:54.121 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-04T17:45:54.121 INFO:teuthology.orchestra.run.smithi092.stdout: "thp": "false", 2024-09-04T17:45:54.121 INFO:teuthology.orchestra.run.smithi092.stdout: "threadpool_default_timeout": "60", 2024-09-04T17:45:54.121 INFO:teuthology.orchestra.run.smithi092.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-04T17:45:54.121 INFO:teuthology.orchestra.run.smithi092.stdout: "throttler_perf_counter": "true", 2024-09-04T17:45:54.121 INFO:teuthology.orchestra.run.smithi092.stdout: "tmp_dir": "/tmp", 2024-09-04T17:45:54.121 INFO:teuthology.orchestra.run.smithi092.stdout: "tmp_file_template": "/tmp/ceph-mon.a.XXXXXX", 2024-09-04T17:45:54.121 INFO:teuthology.orchestra.run.smithi092.stdout: "uadk_compressor_enabled": "false", 2024-09-04T17:45:54.121 INFO:teuthology.orchestra.run.smithi092.stdout: "uadk_wd_sync_ctx_num": "2" 2024-09-04T17:45:54.121 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:45:54.123 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-04T17:45:54.262 INFO:teuthology.orchestra.run.smithi092.stdout:mon.a: {} 2024-09-04T17:45:54.262 INFO:teuthology.orchestra.run.smithi092.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-04T17:45:54.276 INFO:teuthology.orchestra.run.smithi092.stdout:mon.b: {} 2024-09-04T17:45:54.276 INFO:teuthology.orchestra.run.smithi092.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-04T17:45:54.285 INFO:teuthology.orchestra.run.smithi092.stdout:mon.c: {} 2024-09-04T17:45:54.285 INFO:teuthology.orchestra.run.smithi092.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-04T17:45:54.300 DEBUG:teuthology.orchestra.run.smithi190:> type ceph-objectstore-tool 2024-09-04T17:45:54.308 INFO:teuthology.orchestra.run.smithi190.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-04T17:45:54.308 DEBUG:teuthology.orchestra.run.smithi092:> type ceph-objectstore-tool 2024-09-04T17:45:54.312 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-04T17:45:54.313 INFO:teuthology.run_tasks:Running task rbd_fsx... 2024-09-04T17:45:54.323 INFO:tasks.rbd_fsx:starting rbd_fsx... 2024-09-04T17:45:54.323 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-04T17:45:54.323 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-04T17:45:54.323 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-04T17:45:54.323 INFO:tasks.thrashosds.thrasher:Killing osd 3, live_osds are [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-04T17:45:54.324 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-04T17:45:54.324 INFO:teuthology.orchestra.run:waiting for 300 2024-09-04T17:45:54.324 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-04T17:45:54.324 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-04T17:45:54.324 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-04T17:45:54.325 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-04T17:45:54.325 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-04T17:45:54.325 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-04T17:45:54.325 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-04T17:45:54.326 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 1 -w 1 -t 1 -h 1 -l 250000000 -S 0 -N 6000 pool_client.0 image_client.0 2024-09-04T17:45:54.352 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:45:54.352+0000 7f505d642640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26446) UID: 0 2024-09-04T17:45:54.371 INFO:teuthology.orchestra.run.smithi092.stdout:mapped writes DISABLED 2024-09-04T17:45:54.372 INFO:teuthology.orchestra.run.smithi092.stdout:mapped reads DISABLED 2024-09-04T17:45:54.372 INFO:teuthology.orchestra.run.smithi092.stdout:Seed set to 5082 2024-09-04T17:45:54.426 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:45:54.426+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:45:54.470 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:45:54.470 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:45:54.470 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:45:54.470 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:45:54.486 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-04T17:45:54.492 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:45:54.493 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:45:54.539 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:45:54.538+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:45:54.597 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:45:54.598 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:45:54.610 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-04T17:45:54.655 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:45:54.654+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:45:54.725 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:45:54.726 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:45:54.729 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:45:54.728+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:45:54.742 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-04T17:45:54.743 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-04T17:45:54.775 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:45:54.776 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:45:54.788 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-04T17:45:54.830 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:45:54.828+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:45:54.909 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:45:54.909 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:45:54.926 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-04T17:45:54.926 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is set 2024-09-04T17:45:54.950 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:45:54.946+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:45:55.031 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:45:55.030+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:45:55.048 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:45:55.048 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:45:55.062 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-04T17:45:55.133 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:45:55.132+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:45:55.223 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:45:55.223 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:45:55.234 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:45:55.230+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:45:55.235 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-04T17:45:55.335 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:45:55.334+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:45:55.400 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:45:55.400 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:45:55.414 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-04T17:45:55.436 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:45:55.436+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:45:55.537 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:45:55.536+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:45:55.580 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:45:55.580 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:45:55.594 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-04T17:45:55.638 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:45:55.636+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:45:55.717 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:45:55.717 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:45:55.732 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-04T17:45:55.779 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:45:55.780+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:45:55.842 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:45:55.838+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:45:55.863 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:45:55.863 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:45:55.863 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:45:55.863 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:45:55.877 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-04T17:45:55.963 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:45:55.964+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:45:56.044 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:45:56.044+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:45:56.068 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:45:56.068 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:45:56.068 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:45:56.068 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:45:56.069 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:45:56.082 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-04T17:45:56.167 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:45:56.168+0000 7f505d642640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26446) UID: 0 2024-09-04T17:45:56.246 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:45:56.244+0000 7f505d642640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26446) UID: 0 2024-09-04T17:45:56.246 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:45:56.247 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:45:56.247 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:45:56.247 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:45:56.247 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.247 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-04T17:45:56.248 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:45:55.967014+0000", 2024-09-04T17:45:56.248 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.28245540499999999, 2024-09-04T17:45:56.248 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020068199999999999, 2024-09-04T17:45:56.248 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:45:56.249 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:45:56.249 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:45:56.249 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:45:56.249 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:45:56.249 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:45:56.250 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4 2024-09-04T17:45:56.250 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.250 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:45:56.250 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.250 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:45:56.250 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:55.967014+0000", 2024-09-04T17:45:56.250 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:45:56.250 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.250 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.250 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:45:56.250 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:55.967014+0000", 2024-09-04T17:45:56.251 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:45:56.251 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.251 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.251 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:45:56.251 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:55.967015+0000", 2024-09-04T17:45:56.251 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.79e-06 2024-09-04T17:45:56.251 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.251 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.251 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:45:56.251 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:55.967022+0000", 2024-09-04T17:45:56.251 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.4500000000000001e-06 2024-09-04T17:45:56.251 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.251 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.252 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:45:56.252 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:55.967032+0000", 2024-09-04T17:45:56.252 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0560999999999999e-05 2024-09-04T17:45:56.252 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.252 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.252 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:45:56.252 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:55.967042+0000", 2024-09-04T17:45:56.252 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5910000000000002e-06 2024-09-04T17:45:56.252 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.252 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.252 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:45:56.252 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:55.967105+0000", 2024-09-04T17:45:56.253 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.2533999999999995e-05 2024-09-04T17:45:56.253 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.253 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.253 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:45:56.253 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:55.967214+0000", 2024-09-04T17:45:56.253 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000109756 2024-09-04T17:45:56.253 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:45:56.253 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:45:56.253 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:45:56.253 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.254 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.254 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:5 3.1a 3:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-04T17:45:56.254 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:45:55.967529+0000", 2024-09-04T17:45:56.254 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.28193998300000001, 2024-09-04T17:45:56.254 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000150798, 2024-09-04T17:45:56.254 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:45:56.254 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:45:56.254 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:45:56.254 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:45:56.254 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:45:56.254 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:45:56.254 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 5 2024-09-04T17:45:56.255 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.255 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:45:56.255 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.255 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:45:56.255 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:55.967529+0000", 2024-09-04T17:45:56.255 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:45:56.255 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.255 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.255 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:45:56.255 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:55.967529+0000", 2024-09-04T17:45:56.255 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:45:56.255 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.255 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.255 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:45:56.256 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:55.967530+0000", 2024-09-04T17:45:56.256 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.16e-07 2024-09-04T17:45:56.256 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.256 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.256 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:45:56.256 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:55.967533+0000", 2024-09-04T17:45:56.256 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.1930000000000002e-06 2024-09-04T17:45:56.256 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.256 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.256 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:45:56.256 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:55.967542+0000", 2024-09-04T17:45:56.257 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.456e-06 2024-09-04T17:45:56.257 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.257 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.257 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:45:56.257 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:55.967546+0000", 2024-09-04T17:45:56.257 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7709999999999997e-06 2024-09-04T17:45:56.257 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.257 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.257 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:45:56.257 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:55.967589+0000", 2024-09-04T17:45:56.258 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2663999999999997e-05 2024-09-04T17:45:56.258 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.258 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.258 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:45:56.258 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:55.967680+0000", 2024-09-04T17:45:56.258 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.0797999999999994e-05 2024-09-04T17:45:56.258 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:45:56.258 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:45:56.258 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:45:56.258 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:45:56.258 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:45:56.258 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:45:56.261 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-04T17:45:56.347 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:45:56.346+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:45:56.421 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:45:56.421 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:45:56.421 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:45:56.421 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:45:56.434 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-04T17:45:56.448 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:45:56.450+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:45:56.549 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:45:56.548+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:45:56.555 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:45:56.555 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:45:56.555 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:45:56.555 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:45:56.555 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:45:56.569 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-04T17:45:56.655 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:45:56.656+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:45:56.737 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:45:56.737 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:45:56.737 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:45:56.737 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:45:56.737 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.737 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4208.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-04T17:45:56.737 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:45:47.375913+0000", 2024-09-04T17:45:56.738 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 9.3624808559999995, 2024-09-04T17:45:56.738 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0032363050000000001, 2024-09-04T17:45:56.738 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:45:56.738 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:45:56.738 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:45:56.738 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:45:56.738 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4208", 2024-09-04T17:45:56.738 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/3944925361", 2024-09-04T17:45:56.738 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 1 2024-09-04T17:45:56.738 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.738 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:45:56.738 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.739 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:45:56.739 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:47.375913+0000", 2024-09-04T17:45:56.739 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:45:56.739 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.739 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.739 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:45:56.739 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:47.375913+0000", 2024-09-04T17:45:56.739 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:45:56.739 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.739 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.739 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:45:56.740 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:47.375927+0000", 2024-09-04T17:45:56.740 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3733e-05 2024-09-04T17:45:56.740 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.740 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.740 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:45:56.740 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:47.375944+0000", 2024-09-04T17:45:56.740 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.73e-05 2024-09-04T17:45:56.740 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.740 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.740 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:45:56.740 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:47.375979+0000", 2024-09-04T17:45:56.740 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4422999999999999e-05 2024-09-04T17:45:56.740 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.741 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.741 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:45:56.741 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:47.376004+0000", 2024-09-04T17:45:56.741 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5417000000000001e-05 2024-09-04T17:45:56.741 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.741 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.741 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:45:56.741 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:47.376188+0000", 2024-09-04T17:45:56.741 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00018414500000000001 2024-09-04T17:45:56.741 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.741 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.741 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:45:56.741 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:47.376564+0000", 2024-09-04T17:45:56.741 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00037576999999999999 2024-09-04T17:45:56.742 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.742 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.742 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 2", 2024-09-04T17:45:56.742 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:47.376921+0000", 2024-09-04T17:45:56.742 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00035701100000000001 2024-09-04T17:45:56.742 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.742 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.742 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:45:56.742 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:47.378733+0000", 2024-09-04T17:45:56.742 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0018124129999999999 2024-09-04T17:45:56.742 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.742 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.742 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:45:56.743 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:47.378973+0000", 2024-09-04T17:45:56.743 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000239521 2024-09-04T17:45:56.743 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.743 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.743 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:45:56.743 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:47.379082+0000", 2024-09-04T17:45:56.743 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000109212 2024-09-04T17:45:56.743 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.743 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.743 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:45:56.743 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:47.379149+0000", 2024-09-04T17:45:56.743 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7360000000000006e-05 2024-09-04T17:45:56.743 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:45:56.743 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:45:56.744 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:45:56.744 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.744 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.744 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop_reply(client.4208.0:1 2.3 e20/19 ondisk, result = 0)", 2024-09-04T17:45:56.744 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:45:47.378474+0000", 2024-09-04T17:45:56.744 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 9.3599200220000007, 2024-09-04T17:45:56.744 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000285407, 2024-09-04T17:45:56.744 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:45:56.744 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:45:56.744 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:45:56.744 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:45:56.744 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.744 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:45:56.745 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:47.378474+0000", 2024-09-04T17:45:56.745 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:45:56.745 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.745 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.745 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:45:56.745 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:47.378474+0000", 2024-09-04T17:45:56.745 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:45:56.745 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.745 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.745 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:45:56.745 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:47.378476+0000", 2024-09-04T17:45:56.746 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.046e-06 2024-09-04T17:45:56.746 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.746 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.746 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:45:56.746 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:47.378487+0000", 2024-09-04T17:45:56.746 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0603e-05 2024-09-04T17:45:56.746 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.746 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.746 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:45:56.746 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:47.378523+0000", 2024-09-04T17:45:56.746 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.6452999999999998e-05 2024-09-04T17:45:56.746 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.746 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.747 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:45:56.747 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:47.378545+0000", 2024-09-04T17:45:56.747 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1860000000000001e-05 2024-09-04T17:45:56.747 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.747 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.747 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:45:56.747 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:47.378685+0000", 2024-09-04T17:45:56.747 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000139732 2024-09-04T17:45:56.747 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.747 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.747 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:45:56.747 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:47.378716+0000", 2024-09-04T17:45:56.748 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.1300000000000002e-05 2024-09-04T17:45:56.748 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.748 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.748 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:45:56.748 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:47.378759+0000", 2024-09-04T17:45:56.748 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.3412999999999999e-05 2024-09-04T17:45:56.748 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:45:56.748 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:45:56.748 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:45:56.748 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.748 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.748 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4208.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-04T17:45:56.749 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:45:47.382815+0000", 2024-09-04T17:45:56.749 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 9.3555788159999995, 2024-09-04T17:45:56.749 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00036012300000000001, 2024-09-04T17:45:56.749 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:45:56.749 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:45:56.749 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:45:56.749 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:45:56.749 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4208", 2024-09-04T17:45:56.749 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/3944925361", 2024-09-04T17:45:56.749 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 2 2024-09-04T17:45:56.749 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.749 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:45:56.750 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.750 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:45:56.750 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:47.382815+0000", 2024-09-04T17:45:56.750 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:45:56.750 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.750 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.750 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:45:56.750 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:47.382815+0000", 2024-09-04T17:45:56.750 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:45:56.750 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.750 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.750 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:45:56.751 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:47.382817+0000", 2024-09-04T17:45:56.751 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4030000000000001e-06 2024-09-04T17:45:56.751 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.751 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.751 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:45:56.751 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:47.382830+0000", 2024-09-04T17:45:56.751 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.224e-05 2024-09-04T17:45:56.751 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.751 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.751 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:45:56.751 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:47.382860+0000", 2024-09-04T17:45:56.752 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0380000000000001e-05 2024-09-04T17:45:56.752 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.752 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.752 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:45:56.752 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:47.382876+0000", 2024-09-04T17:45:56.752 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5452999999999999e-05 2024-09-04T17:45:56.752 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.752 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.752 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:45:56.752 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:47.382969+0000", 2024-09-04T17:45:56.752 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.3128999999999996e-05 2024-09-04T17:45:56.752 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.753 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.753 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:45:56.753 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:47.383175+0000", 2024-09-04T17:45:56.753 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020651799999999999 2024-09-04T17:45:56.753 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:45:56.753 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:45:56.753 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:45:56.753 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.753 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.753 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4208.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-04T17:45:56.754 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:45:47.383486+0000", 2024-09-04T17:45:56.754 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 9.3549077450000002, 2024-09-04T17:45:56.754 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00026112800000000001, 2024-09-04T17:45:56.754 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:45:56.754 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:45:56.754 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:45:56.754 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:45:56.754 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4208", 2024-09-04T17:45:56.754 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/3944925361", 2024-09-04T17:45:56.754 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 3 2024-09-04T17:45:56.755 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.755 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:45:56.755 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.755 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:45:56.755 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:47.383486+0000", 2024-09-04T17:45:56.755 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:45:56.755 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.755 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.755 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:45:56.755 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:47.383486+0000", 2024-09-04T17:45:56.755 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:45:56.755 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.755 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.756 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:45:56.756 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:47.383488+0000", 2024-09-04T17:45:56.756 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7770000000000001e-06 2024-09-04T17:45:56.756 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.756 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.756 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:45:56.756 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:47.383496+0000", 2024-09-04T17:45:56.756 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.3000000000000002e-06 2024-09-04T17:45:56.756 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.756 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.756 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:45:56.757 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:47.383518+0000", 2024-09-04T17:45:56.757 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1586999999999998e-05 2024-09-04T17:45:56.757 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.757 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.757 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:45:56.757 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:47.383531+0000", 2024-09-04T17:45:56.757 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2748999999999999e-05 2024-09-04T17:45:56.757 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.757 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.757 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:45:56.757 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:47.383613+0000", 2024-09-04T17:45:56.757 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.2565999999999995e-05 2024-09-04T17:45:56.758 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.758 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.758 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:45:56.758 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:47.383747+0000", 2024-09-04T17:45:56.758 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013414899999999999 2024-09-04T17:45:56.758 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:45:56.758 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:45:56.758 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:45:56.758 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.758 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.758 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4208.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-04T17:45:56.758 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:45:48.381707+0000", 2024-09-04T17:45:56.759 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 8.3566871220000003, 2024-09-04T17:45:56.759 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0077482999999999996, 2024-09-04T17:45:56.759 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:45:56.759 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:45:56.759 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:45:56.759 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:45:56.759 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4208", 2024-09-04T17:45:56.759 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/3944925361", 2024-09-04T17:45:56.759 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 5 2024-09-04T17:45:56.759 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.759 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:45:56.759 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.759 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:45:56.760 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:48.381707+0000", 2024-09-04T17:45:56.760 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:45:56.760 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.760 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.760 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:45:56.760 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:48.381707+0000", 2024-09-04T17:45:56.760 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:45:56.760 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.760 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.760 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:45:56.760 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:48.381710+0000", 2024-09-04T17:45:56.760 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4470000000000002e-06 2024-09-04T17:45:56.760 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.761 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.761 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:45:56.761 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:48.381724+0000", 2024-09-04T17:45:56.761 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3475999999999999e-05 2024-09-04T17:45:56.761 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.761 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.761 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:45:56.761 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:48.381757+0000", 2024-09-04T17:45:56.761 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.3652999999999998e-05 2024-09-04T17:45:56.762 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.762 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.762 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:45:56.762 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:48.381774+0000", 2024-09-04T17:45:56.762 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6820000000000002e-05 2024-09-04T17:45:56.762 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.762 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.762 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:45:56.762 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:48.381942+0000", 2024-09-04T17:45:56.762 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000167622 2024-09-04T17:45:56.762 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.762 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.762 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op must wait for map", 2024-09-04T17:45:56.763 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:48.381954+0000", 2024-09-04T17:45:56.763 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1790000000000001e-05 2024-09-04T17:45:56.763 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.763 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.763 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:45:56.763 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:48.386203+0000", 2024-09-04T17:45:56.763 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0042497469999999999 2024-09-04T17:45:56.763 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.763 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.763 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:45:56.763 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:48.386533+0000", 2024-09-04T17:45:56.763 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000329387 2024-09-04T17:45:56.763 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.763 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.764 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 1", 2024-09-04T17:45:56.764 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:48.386872+0000", 2024-09-04T17:45:56.764 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00033965800000000001 2024-09-04T17:45:56.764 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.764 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.764 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:45:56.764 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:48.388202+0000", 2024-09-04T17:45:56.764 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.001329383 2024-09-04T17:45:56.764 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.764 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.764 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:45:56.764 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:48.389165+0000", 2024-09-04T17:45:56.764 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00096350600000000004 2024-09-04T17:45:56.764 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.765 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.765 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:45:56.765 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:48.389324+0000", 2024-09-04T17:45:56.765 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00015896899999999999 2024-09-04T17:45:56.765 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.765 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.765 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:45:56.765 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:48.389455+0000", 2024-09-04T17:45:56.765 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013084199999999999 2024-09-04T17:45:56.765 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:45:56.765 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:45:56.765 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:45:56.765 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.766 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.766 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop_reply(client.4208.0:5 2.2 e21/19 ondisk, result = 0)", 2024-09-04T17:45:56.766 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:45:48.388935+0000", 2024-09-04T17:45:56.766 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 8.3494587780000007, 2024-09-04T17:45:56.766 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00054329200000000002, 2024-09-04T17:45:56.766 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:45:56.766 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:45:56.766 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:45:56.766 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:45:56.766 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.766 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:45:56.766 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:48.388935+0000", 2024-09-04T17:45:56.766 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:45:56.766 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.767 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.767 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:45:56.767 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:48.388935+0000", 2024-09-04T17:45:56.767 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:45:56.767 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.767 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.767 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:45:56.767 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:48.388937+0000", 2024-09-04T17:45:56.767 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1969999999999999e-06 2024-09-04T17:45:56.767 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.767 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.767 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:45:56.767 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:48.388947+0000", 2024-09-04T17:45:56.767 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4800000000000007e-06 2024-09-04T17:45:56.768 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.768 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.768 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:45:56.768 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:48.388972+0000", 2024-09-04T17:45:56.768 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5596e-05 2024-09-04T17:45:56.768 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.768 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.768 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:45:56.768 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:48.388988+0000", 2024-09-04T17:45:56.768 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5605999999999999e-05 2024-09-04T17:45:56.768 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.768 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.768 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:45:56.769 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:48.389118+0000", 2024-09-04T17:45:56.769 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00012952600000000001 2024-09-04T17:45:56.769 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.769 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.769 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:45:56.769 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:48.389143+0000", 2024-09-04T17:45:56.769 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5913000000000001e-05 2024-09-04T17:45:56.769 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.769 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.769 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:45:56.769 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:48.389478+0000", 2024-09-04T17:45:56.770 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00033497399999999998 2024-09-04T17:45:56.770 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:45:56.770 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:45:56.770 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:45:56.770 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.770 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.770 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4208.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-04T17:45:56.770 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:45:49.401690+0000", 2024-09-04T17:45:56.770 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 7.3367036140000002, 2024-09-04T17:45:56.770 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0116665, 2024-09-04T17:45:56.770 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:45:56.770 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:45:56.771 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:45:56.771 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:45:56.771 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4208", 2024-09-04T17:45:56.771 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/3944925361", 2024-09-04T17:45:56.771 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 7 2024-09-04T17:45:56.771 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.771 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:45:56.771 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.771 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:45:56.771 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:49.401690+0000", 2024-09-04T17:45:56.771 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:45:56.772 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.772 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.772 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:45:56.772 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:49.401690+0000", 2024-09-04T17:45:56.772 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:45:56.772 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.772 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.772 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:45:56.772 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:49.401694+0000", 2024-09-04T17:45:56.772 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4070000000000002e-06 2024-09-04T17:45:56.772 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.773 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.773 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:45:56.773 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:49.401709+0000", 2024-09-04T17:45:56.773 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.522e-05 2024-09-04T17:45:56.773 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.773 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.773 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:45:56.773 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:49.401744+0000", 2024-09-04T17:45:56.773 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.5490000000000001e-05 2024-09-04T17:45:56.773 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.773 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.773 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:45:56.773 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:49.401772+0000", 2024-09-04T17:45:56.774 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.7903e-05 2024-09-04T17:45:56.774 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.774 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.774 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:45:56.774 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:49.401950+0000", 2024-09-04T17:45:56.774 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00017753800000000001 2024-09-04T17:45:56.774 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.774 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.774 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op must wait for map", 2024-09-04T17:45:56.774 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:49.401971+0000", 2024-09-04T17:45:56.774 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1307000000000001e-05 2024-09-04T17:45:56.774 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.774 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.775 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:45:56.775 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:49.409203+0000", 2024-09-04T17:45:56.775 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0072319669999999997 2024-09-04T17:45:56.775 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.775 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.775 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:45:56.775 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:49.409456+0000", 2024-09-04T17:45:56.775 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00025300499999999999 2024-09-04T17:45:56.775 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.775 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.775 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 1", 2024-09-04T17:45:56.775 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:49.409851+0000", 2024-09-04T17:45:56.775 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000394597 2024-09-04T17:45:56.775 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.776 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.776 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:45:56.776 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:49.412198+0000", 2024-09-04T17:45:56.776 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0023476880000000001 2024-09-04T17:45:56.776 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.776 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.776 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:45:56.776 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:49.413181+0000", 2024-09-04T17:45:56.776 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00098244999999999995 2024-09-04T17:45:56.776 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.776 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.777 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:45:56.777 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:49.413273+0000", 2024-09-04T17:45:56.777 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.2158999999999996e-05 2024-09-04T17:45:56.777 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.777 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.777 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:45:56.777 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:49.413357+0000", 2024-09-04T17:45:56.777 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.3769000000000004e-05 2024-09-04T17:45:56.777 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:45:56.777 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:45:56.777 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:45:56.777 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.778 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.778 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop_reply(client.4208.0:7 2.2 e22/19 ondisk, result = 0)", 2024-09-04T17:45:56.778 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:45:49.411989+0000", 2024-09-04T17:45:56.778 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 7.3264050530000002, 2024-09-04T17:45:56.778 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024245500000000001, 2024-09-04T17:45:56.778 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:45:56.778 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:45:56.778 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:45:56.778 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:45:56.778 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.778 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:45:56.778 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:49.411989+0000", 2024-09-04T17:45:56.778 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:45:56.779 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.779 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.779 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:45:56.779 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:49.411989+0000", 2024-09-04T17:45:56.779 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:45:56.779 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.779 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.779 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:45:56.779 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:49.411991+0000", 2024-09-04T17:45:56.779 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.883e-06 2024-09-04T17:45:56.779 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.779 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.779 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:45:56.779 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:49.411997+0000", 2024-09-04T17:45:56.780 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6630000000000004e-06 2024-09-04T17:45:56.780 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.780 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.780 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:45:56.780 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:49.412028+0000", 2024-09-04T17:45:56.780 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0283000000000001e-05 2024-09-04T17:45:56.780 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.780 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.780 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:45:56.780 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:49.412042+0000", 2024-09-04T17:45:56.780 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4233000000000001e-05 2024-09-04T17:45:56.781 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.781 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.781 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:45:56.781 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:49.412160+0000", 2024-09-04T17:45:56.781 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000118183 2024-09-04T17:45:56.781 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.781 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.781 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:45:56.781 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:49.412181+0000", 2024-09-04T17:45:56.781 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1129999999999999e-05 2024-09-04T17:45:56.781 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:56.782 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:56.782 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:45:56.782 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:49.412231+0000", 2024-09-04T17:45:56.782 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0080000000000002e-05 2024-09-04T17:45:56.782 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:45:56.782 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:45:56.782 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:45:56.782 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:45:56.782 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:45:56.782 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:45:56.784 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-04T17:45:56.784 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:45:56.784+0000 7f505d642640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26446) UID: 0 2024-09-04T17:45:56.884 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:45:56.884+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:45:56.951 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-04T17:45:56.958 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:45:56.958 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:45:56.959 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:45:56.959 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:45:56.973 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-04T17:45:56.985 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-09-04T17:45:56.998 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:45:56.998+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:45:57.086 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:45:57.084+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:45:57.139 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:45:57.140 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:45:57.140 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:45:57.140 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:45:57.140 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:45:57.153 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-04T17:45:57.199 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:45:57.200+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:45:57.288 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:45:57.288+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:45:57.321 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:45:57.321 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:45:57.321 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:45:57.322 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [] 2024-09-04T17:45:57.322 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:45:57.337 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-04T17:45:57.423 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:45:57.424+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:45:57.490 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:45:57.488+0000 7f505d642640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26446) UID: 0 2024-09-04T17:45:57.498 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:45:57.499 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:45:57.499 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:45:57.499 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:45:57.514 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-04T17:45:57.591 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:45:57.590+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:45:57.676 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:45:57.676 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:45:57.677 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:45:57.677 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:45:57.677 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:45:57.690 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-04T17:45:57.692 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:45:57.692+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:45:57.793 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:45:57.792+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:45:57.818 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:45:57.818 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:45:57.818 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:45:57.818 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:45:57.818 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.818 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4219.0:25 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-04T17:45:57.819 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:45:42.382664+0000", 2024-09-04T17:45:57.819 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 15.434287012, 2024-09-04T17:45:57.819 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.011398609000000001, 2024-09-04T17:45:57.819 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:45:57.819 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:45:57.819 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:45:57.819 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:45:57.819 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4219", 2024-09-04T17:45:57.819 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.190:0/3705085728", 2024-09-04T17:45:57.819 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 25 2024-09-04T17:45:57.819 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.819 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:45:57.819 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.819 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:45:57.820 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.382664+0000", 2024-09-04T17:45:57.820 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:45:57.820 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.820 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.820 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:45:57.820 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.382664+0000", 2024-09-04T17:45:57.820 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:45:57.820 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.820 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.820 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:45:57.820 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.382665+0000", 2024-09-04T17:45:57.820 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.097e-06 2024-09-04T17:45:57.820 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.821 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.821 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:45:57.821 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.382673+0000", 2024-09-04T17:45:57.821 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.0330000000000008e-06 2024-09-04T17:45:57.821 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.821 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.821 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:45:57.821 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.382687+0000", 2024-09-04T17:45:57.821 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4293e-05 2024-09-04T17:45:57.821 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.821 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.821 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:45:57.821 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.382712+0000", 2024-09-04T17:45:57.822 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4467000000000001e-05 2024-09-04T17:45:57.822 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.822 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.822 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:45:57.822 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.391178+0000", 2024-09-04T17:45:57.822 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0084663380000000003 2024-09-04T17:45:57.822 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.823 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.823 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:45:57.823 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.391408+0000", 2024-09-04T17:45:57.823 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000230311 2024-09-04T17:45:57.823 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.823 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.823 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 0", 2024-09-04T17:45:57.823 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.391790+0000", 2024-09-04T17:45:57.823 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00038148700000000002 2024-09-04T17:45:57.823 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.823 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.823 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:45:57.823 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.392854+0000", 2024-09-04T17:45:57.824 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.001064349 2024-09-04T17:45:57.824 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.824 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.824 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:45:57.824 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.393790+0000", 2024-09-04T17:45:57.824 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00093565 2024-09-04T17:45:57.824 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.824 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.824 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:45:57.824 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.393954+0000", 2024-09-04T17:45:57.824 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00016424500000000001 2024-09-04T17:45:57.824 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.824 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.825 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:45:57.825 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.394062+0000", 2024-09-04T17:45:57.825 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00010833899999999999 2024-09-04T17:45:57.825 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:45:57.825 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:45:57.825 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:45:57.825 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.826 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.826 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4219.0:43 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-04T17:45:57.826 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:45:42.486528+0000", 2024-09-04T17:45:57.826 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 15.330422733000001, 2024-09-04T17:45:57.826 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.010711543, 2024-09-04T17:45:57.826 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:45:57.826 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:45:57.826 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:45:57.826 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:45:57.826 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4219", 2024-09-04T17:45:57.826 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.190:0/3705085728", 2024-09-04T17:45:57.826 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 43 2024-09-04T17:45:57.827 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.827 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:45:57.827 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.827 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:45:57.827 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.486528+0000", 2024-09-04T17:45:57.827 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:45:57.827 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.827 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.827 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:45:57.827 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.486528+0000", 2024-09-04T17:45:57.827 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:45:57.827 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.828 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.828 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:45:57.828 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.486529+0000", 2024-09-04T17:45:57.828 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3400000000000001e-06 2024-09-04T17:45:57.828 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.828 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.828 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:45:57.828 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.486537+0000", 2024-09-04T17:45:57.828 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.0129999999999993e-06 2024-09-04T17:45:57.828 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.828 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.828 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:45:57.828 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.486553+0000", 2024-09-04T17:45:57.829 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5206e-05 2024-09-04T17:45:57.829 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.829 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.829 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:45:57.829 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.486562+0000", 2024-09-04T17:45:57.829 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1770000000000004e-06 2024-09-04T17:45:57.829 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.829 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.829 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:45:57.829 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.492074+0000", 2024-09-04T17:45:57.829 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0055120840000000004 2024-09-04T17:45:57.829 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.829 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.829 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:45:57.830 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.492222+0000", 2024-09-04T17:45:57.830 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000148116 2024-09-04T17:45:57.830 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.830 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.830 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 0", 2024-09-04T17:45:57.830 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.492428+0000", 2024-09-04T17:45:57.830 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020638500000000001 2024-09-04T17:45:57.830 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.830 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.830 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:45:57.830 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.495537+0000", 2024-09-04T17:45:57.830 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0031089860000000002 2024-09-04T17:45:57.830 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.831 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.831 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:45:57.831 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.496948+0000", 2024-09-04T17:45:57.831 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.001410419 2024-09-04T17:45:57.831 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.831 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.831 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:45:57.831 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.497119+0000", 2024-09-04T17:45:57.831 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00017159200000000001 2024-09-04T17:45:57.831 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.831 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.831 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:45:57.831 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.497240+0000", 2024-09-04T17:45:57.832 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00012022499999999999 2024-09-04T17:45:57.833 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:45:57.833 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:45:57.833 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:45:57.833 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.833 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.833 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4219.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-04T17:45:57.833 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:45:42.486655+0000", 2024-09-04T17:45:57.833 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 15.330295788000001, 2024-09-04T17:45:57.833 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.012257815, 2024-09-04T17:45:57.834 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:45:57.834 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:45:57.834 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:45:57.834 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:45:57.834 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4219", 2024-09-04T17:45:57.834 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.190:0/3705085728", 2024-09-04T17:45:57.834 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44 2024-09-04T17:45:57.834 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.834 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:45:57.834 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.834 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:45:57.834 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.486655+0000", 2024-09-04T17:45:57.834 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:45:57.834 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.835 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.835 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:45:57.835 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.486655+0000", 2024-09-04T17:45:57.835 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:45:57.835 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.835 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.835 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:45:57.835 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.486656+0000", 2024-09-04T17:45:57.835 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.017e-06 2024-09-04T17:45:57.835 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.835 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.835 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:45:57.835 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.486813+0000", 2024-09-04T17:45:57.836 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00015743199999999999 2024-09-04T17:45:57.836 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.836 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.836 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:45:57.836 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.486830+0000", 2024-09-04T17:45:57.836 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6399999999999999e-05 2024-09-04T17:45:57.836 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.836 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.836 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:45:57.836 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.486840+0000", 2024-09-04T17:45:57.836 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.8360000000000004e-06 2024-09-04T17:45:57.836 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.836 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.837 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:45:57.837 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.494148+0000", 2024-09-04T17:45:57.837 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0073080209999999996 2024-09-04T17:45:57.837 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.837 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.837 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:45:57.837 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.494318+0000", 2024-09-04T17:45:57.837 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00017074200000000001 2024-09-04T17:45:57.837 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.837 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.837 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 0", 2024-09-04T17:45:57.837 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.494529+0000", 2024-09-04T17:45:57.837 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000210471 2024-09-04T17:45:57.837 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.838 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.838 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:45:57.838 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.498630+0000", 2024-09-04T17:45:57.838 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0041006519999999998 2024-09-04T17:45:57.838 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.838 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.838 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:45:57.838 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.498673+0000", 2024-09-04T17:45:57.838 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.3229999999999998e-05 2024-09-04T17:45:57.838 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.838 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.838 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:45:57.838 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.498805+0000", 2024-09-04T17:45:57.838 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013176900000000001 2024-09-04T17:45:57.839 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.839 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.839 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:45:57.839 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.498913+0000", 2024-09-04T17:45:57.839 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000108245 2024-09-04T17:45:57.839 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:45:57.839 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:45:57.839 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:45:57.839 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.839 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.839 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4219.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-04T17:45:57.839 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:45:42.486862+0000", 2024-09-04T17:45:57.839 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 15.330089176, 2024-09-04T17:45:57.839 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.013920495999999999, 2024-09-04T17:45:57.840 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:45:57.840 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:45:57.840 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:45:57.840 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:45:57.840 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4219", 2024-09-04T17:45:57.840 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.190:0/3705085728", 2024-09-04T17:45:57.840 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 45 2024-09-04T17:45:57.840 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.840 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:45:57.840 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.840 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:45:57.840 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.486862+0000", 2024-09-04T17:45:57.840 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:45:57.841 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.841 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.841 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:45:57.841 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.486862+0000", 2024-09-04T17:45:57.841 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:45:57.841 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.841 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.841 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:45:57.841 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.486863+0000", 2024-09-04T17:45:57.841 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.6299999999999993e-07 2024-09-04T17:45:57.841 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.841 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.841 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:45:57.841 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.486869+0000", 2024-09-04T17:45:57.842 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.5930000000000002e-06 2024-09-04T17:45:57.842 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.842 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.842 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:45:57.842 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.486879+0000", 2024-09-04T17:45:57.842 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.6840000000000008e-06 2024-09-04T17:45:57.842 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.842 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.842 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:45:57.842 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.486886+0000", 2024-09-04T17:45:57.843 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.6359999999999994e-06 2024-09-04T17:45:57.843 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.843 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.843 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:45:57.843 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.495684+0000", 2024-09-04T17:45:57.843 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0087973960000000007 2024-09-04T17:45:57.843 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.843 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.843 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:45:57.843 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.495904+0000", 2024-09-04T17:45:57.843 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000219772 2024-09-04T17:45:57.843 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.843 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.843 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 0", 2024-09-04T17:45:57.844 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.496257+0000", 2024-09-04T17:45:57.844 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00035375400000000002 2024-09-04T17:45:57.844 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.844 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.844 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:45:57.844 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.499061+0000", 2024-09-04T17:45:57.844 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0028034940000000001 2024-09-04T17:45:57.844 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.844 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.844 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:45:57.844 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.500468+0000", 2024-09-04T17:45:57.844 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0014067260000000001 2024-09-04T17:45:57.845 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.845 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.845 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:45:57.845 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.500660+0000", 2024-09-04T17:45:57.845 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000192339 2024-09-04T17:45:57.845 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.845 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.845 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:45:57.845 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.500782+0000", 2024-09-04T17:45:57.845 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000122139 2024-09-04T17:45:57.845 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:45:57.845 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:45:57.846 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:45:57.846 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.846 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.846 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4219.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-04T17:45:57.846 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:45:42.486924+0000", 2024-09-04T17:45:57.846 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 15.330026303, 2024-09-04T17:45:57.846 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.01546777, 2024-09-04T17:45:57.846 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:45:57.846 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:45:57.846 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:45:57.846 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:45:57.846 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4219", 2024-09-04T17:45:57.846 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.190:0/3705085728", 2024-09-04T17:45:57.846 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 46 2024-09-04T17:45:57.847 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.847 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:45:57.847 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.847 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:45:57.847 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.486924+0000", 2024-09-04T17:45:57.847 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:45:57.847 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.847 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.847 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:45:57.847 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.486924+0000", 2024-09-04T17:45:57.847 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:45:57.847 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.847 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.847 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:45:57.848 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.486925+0000", 2024-09-04T17:45:57.848 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.2699999999999998e-07 2024-09-04T17:45:57.848 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.848 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.848 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:45:57.848 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.486932+0000", 2024-09-04T17:45:57.848 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.9460000000000001e-06 2024-09-04T17:45:57.848 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.848 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.848 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:45:57.848 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.486944+0000", 2024-09-04T17:45:57.848 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1683e-05 2024-09-04T17:45:57.848 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.848 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.849 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:45:57.849 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.486952+0000", 2024-09-04T17:45:57.849 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.7600000000000002e-06 2024-09-04T17:45:57.849 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.849 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.849 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:45:57.849 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.497360+0000", 2024-09-04T17:45:57.849 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.010408373 2024-09-04T17:45:57.849 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.849 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.849 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:45:57.849 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.497648+0000", 2024-09-04T17:45:57.849 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00028803800000000001 2024-09-04T17:45:57.850 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.850 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.850 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 0", 2024-09-04T17:45:57.850 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.497898+0000", 2024-09-04T17:45:57.850 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024940800000000001 2024-09-04T17:45:57.850 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.850 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.850 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:45:57.850 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.502098+0000", 2024-09-04T17:45:57.850 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0042000709999999997 2024-09-04T17:45:57.850 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.850 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.850 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:45:57.851 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.502136+0000", 2024-09-04T17:45:57.851 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.8788999999999997e-05 2024-09-04T17:45:57.851 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.851 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.851 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:45:57.851 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.502286+0000", 2024-09-04T17:45:57.851 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014945599999999999 2024-09-04T17:45:57.851 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.851 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.851 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:45:57.851 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.502392+0000", 2024-09-04T17:45:57.851 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000106319 2024-09-04T17:45:57.851 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:45:57.851 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:45:57.852 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:45:57.852 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.852 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.852 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4219.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-04T17:45:57.852 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:45:42.486966+0000", 2024-09-04T17:45:57.852 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 15.329984489999999, 2024-09-04T17:45:57.852 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.016951681999999999, 2024-09-04T17:45:57.852 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:45:57.852 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:45:57.852 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:45:57.852 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:45:57.852 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4219", 2024-09-04T17:45:57.852 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.190:0/3705085728", 2024-09-04T17:45:57.853 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 47 2024-09-04T17:45:57.853 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.853 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:45:57.853 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.853 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:45:57.853 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.486966+0000", 2024-09-04T17:45:57.853 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:45:57.853 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.853 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.853 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:45:57.853 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.486966+0000", 2024-09-04T17:45:57.853 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:45:57.853 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.854 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.854 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:45:57.854 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.486967+0000", 2024-09-04T17:45:57.854 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.5600000000000005e-07 2024-09-04T17:45:57.854 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.854 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.854 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:45:57.854 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487043+0000", 2024-09-04T17:45:57.854 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.6329999999999996e-05 2024-09-04T17:45:57.854 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.854 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.854 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:45:57.854 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487055+0000", 2024-09-04T17:45:57.854 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.164e-05 2024-09-04T17:45:57.855 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.855 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.855 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:45:57.855 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487065+0000", 2024-09-04T17:45:57.855 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0073e-05 2024-09-04T17:45:57.855 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.855 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.855 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:45:57.855 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.499186+0000", 2024-09-04T17:45:57.855 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.012121226000000001 2024-09-04T17:45:57.855 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.855 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.855 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:45:57.856 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.499387+0000", 2024-09-04T17:45:57.856 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000200972 2024-09-04T17:45:57.856 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.856 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.856 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 0", 2024-09-04T17:45:57.856 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.499590+0000", 2024-09-04T17:45:57.856 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000203329 2024-09-04T17:45:57.856 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.856 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.856 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:45:57.857 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.502535+0000", 2024-09-04T17:45:57.857 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0029447269999999998 2024-09-04T17:45:57.857 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.857 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.857 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:45:57.857 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.503604+0000", 2024-09-04T17:45:57.857 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0010688049999999999 2024-09-04T17:45:57.857 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.857 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.857 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:45:57.857 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.503799+0000", 2024-09-04T17:45:57.857 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019521799999999999 2024-09-04T17:45:57.857 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.858 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.858 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:45:57.858 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.503918+0000", 2024-09-04T17:45:57.858 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000118706 2024-09-04T17:45:57.858 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:45:57.858 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:45:57.858 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:45:57.858 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.858 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.858 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4219.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-04T17:45:57.858 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:45:42.487079+0000", 2024-09-04T17:45:57.858 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 15.329871224, 2024-09-04T17:45:57.858 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.018831018000000001, 2024-09-04T17:45:57.858 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:45:57.859 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:45:57.859 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:45:57.859 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:45:57.859 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4219", 2024-09-04T17:45:57.859 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.190:0/3705085728", 2024-09-04T17:45:57.859 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 48 2024-09-04T17:45:57.859 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.859 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:45:57.859 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.859 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:45:57.859 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487079+0000", 2024-09-04T17:45:57.859 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:45:57.859 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.859 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.860 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:45:57.860 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487079+0000", 2024-09-04T17:45:57.860 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:45:57.860 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.860 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.860 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:45:57.860 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487080+0000", 2024-09-04T17:45:57.860 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.23e-07 2024-09-04T17:45:57.860 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.860 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.860 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:45:57.860 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487086+0000", 2024-09-04T17:45:57.860 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.9030000000000001e-06 2024-09-04T17:45:57.861 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.861 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.861 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:45:57.861 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487098+0000", 2024-09-04T17:45:57.861 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1610000000000001e-05 2024-09-04T17:45:57.861 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.861 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.861 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:45:57.861 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487111+0000", 2024-09-04T17:45:57.861 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3162999999999999e-05 2024-09-04T17:45:57.861 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.861 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.861 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:45:57.861 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.500899+0000", 2024-09-04T17:45:57.862 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.013788181 2024-09-04T17:45:57.862 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.862 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.862 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:45:57.862 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.501038+0000", 2024-09-04T17:45:57.862 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000138758 2024-09-04T17:45:57.862 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.862 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.862 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 0", 2024-09-04T17:45:57.862 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.501288+0000", 2024-09-04T17:45:57.862 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024998499999999999 2024-09-04T17:45:57.862 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.862 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.863 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:45:57.863 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.505619+0000", 2024-09-04T17:45:57.863 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0043308369999999997 2024-09-04T17:45:57.863 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.863 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.863 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:45:57.863 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.505662+0000", 2024-09-04T17:45:57.863 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.3433000000000002e-05 2024-09-04T17:45:57.863 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.863 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.863 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:45:57.863 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.505800+0000", 2024-09-04T17:45:57.863 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013755199999999999 2024-09-04T17:45:57.864 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.864 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.864 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:45:57.864 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.505911+0000", 2024-09-04T17:45:57.864 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000110773 2024-09-04T17:45:57.864 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:45:57.864 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:45:57.864 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:45:57.864 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.864 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.864 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4219.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-04T17:45:57.864 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:45:42.487139+0000", 2024-09-04T17:45:57.864 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 15.329811415, 2024-09-04T17:45:57.865 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.019193162, 2024-09-04T17:45:57.865 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:45:57.865 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:45:57.865 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:45:57.865 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:45:57.865 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4219", 2024-09-04T17:45:57.865 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.190:0/3705085728", 2024-09-04T17:45:57.865 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 49 2024-09-04T17:45:57.865 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.865 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:45:57.865 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.865 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:45:57.865 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487139+0000", 2024-09-04T17:45:57.865 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:45:57.866 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.866 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.866 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:45:57.866 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487139+0000", 2024-09-04T17:45:57.866 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:45:57.866 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.866 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.866 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:45:57.866 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487140+0000", 2024-09-04T17:45:57.866 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1629999999999999e-06 2024-09-04T17:45:57.866 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.866 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.866 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:45:57.867 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487152+0000", 2024-09-04T17:45:57.867 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1914e-05 2024-09-04T17:45:57.867 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.867 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.867 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:45:57.867 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487165+0000", 2024-09-04T17:45:57.867 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.293e-05 2024-09-04T17:45:57.867 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.867 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.867 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:45:57.867 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487174+0000", 2024-09-04T17:45:57.867 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1330000000000005e-06 2024-09-04T17:45:57.867 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.867 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.868 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:45:57.868 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.502694+0000", 2024-09-04T17:45:57.868 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.015519593 2024-09-04T17:45:57.868 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.868 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.868 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:45:57.868 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.502839+0000", 2024-09-04T17:45:57.868 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000144993 2024-09-04T17:45:57.868 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.868 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.868 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 0", 2024-09-04T17:45:57.868 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.503034+0000", 2024-09-04T17:45:57.868 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019492399999999999 2024-09-04T17:45:57.869 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.869 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.869 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:45:57.869 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.506043+0000", 2024-09-04T17:45:57.869 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0030094710000000001 2024-09-04T17:45:57.869 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.869 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.869 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:45:57.869 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.506084+0000", 2024-09-04T17:45:57.869 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.057e-05 2024-09-04T17:45:57.869 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.869 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.869 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:45:57.869 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.506211+0000", 2024-09-04T17:45:57.870 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00012676199999999999 2024-09-04T17:45:57.870 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.870 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.870 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:45:57.870 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.506332+0000", 2024-09-04T17:45:57.870 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000121709 2024-09-04T17:45:57.870 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:45:57.870 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:45:57.870 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:45:57.870 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.870 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.870 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4219.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-04T17:45:57.871 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:45:42.487196+0000", 2024-09-04T17:45:57.871 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 15.329754585, 2024-09-04T17:45:57.871 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.021893024000000001, 2024-09-04T17:45:57.871 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:45:57.871 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:45:57.871 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:45:57.871 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:45:57.871 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4219", 2024-09-04T17:45:57.871 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.190:0/3705085728", 2024-09-04T17:45:57.871 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 50 2024-09-04T17:45:57.871 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.871 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:45:57.871 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.872 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:45:57.872 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487196+0000", 2024-09-04T17:45:57.872 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:45:57.872 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.872 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.872 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:45:57.872 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487196+0000", 2024-09-04T17:45:57.872 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:45:57.872 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.872 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.872 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:45:57.873 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487197+0000", 2024-09-04T17:45:57.873 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.06e-07 2024-09-04T17:45:57.873 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.873 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.873 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:45:57.873 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487270+0000", 2024-09-04T17:45:57.873 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2992999999999997e-05 2024-09-04T17:45:57.873 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.873 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.873 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:45:57.873 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487281+0000", 2024-09-04T17:45:57.873 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.13e-05 2024-09-04T17:45:57.873 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.874 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.874 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:45:57.874 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487289+0000", 2024-09-04T17:45:57.874 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.8429999999999993e-06 2024-09-04T17:45:57.874 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.874 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.874 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:45:57.874 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.504017+0000", 2024-09-04T17:45:57.874 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.016728481 2024-09-04T17:45:57.874 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.874 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.874 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:45:57.874 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.504153+0000", 2024-09-04T17:45:57.874 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000135396 2024-09-04T17:45:57.875 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.875 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.875 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 0", 2024-09-04T17:45:57.875 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.504378+0000", 2024-09-04T17:45:57.875 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000225255 2024-09-04T17:45:57.875 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.875 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.875 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:45:57.875 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.508835+0000", 2024-09-04T17:45:57.875 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0044573759999999999 2024-09-04T17:45:57.875 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.875 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.875 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:45:57.876 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.508936+0000", 2024-09-04T17:45:57.876 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000100555 2024-09-04T17:45:57.876 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.876 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.876 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:45:57.876 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.509021+0000", 2024-09-04T17:45:57.876 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.4546e-05 2024-09-04T17:45:57.876 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.876 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.876 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:45:57.876 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.509089+0000", 2024-09-04T17:45:57.876 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.8572999999999996e-05 2024-09-04T17:45:57.876 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:45:57.877 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:45:57.877 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:45:57.877 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.877 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.877 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4219.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-04T17:45:57.877 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:45:42.487303+0000", 2024-09-04T17:45:57.877 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 15.329647756, 2024-09-04T17:45:57.877 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.022763057, 2024-09-04T17:45:57.877 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:45:57.877 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:45:57.877 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:45:57.877 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:45:57.877 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4219", 2024-09-04T17:45:57.878 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.190:0/3705085728", 2024-09-04T17:45:57.878 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 51 2024-09-04T17:45:57.878 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.878 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:45:57.878 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.878 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:45:57.878 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487303+0000", 2024-09-04T17:45:57.878 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:45:57.878 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.878 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.878 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:45:57.878 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487303+0000", 2024-09-04T17:45:57.878 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:45:57.878 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.879 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.879 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:45:57.879 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487304+0000", 2024-09-04T17:45:57.879 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.7000000000000004e-07 2024-09-04T17:45:57.879 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.879 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.879 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:45:57.879 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487313+0000", 2024-09-04T17:45:57.879 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8200000000000003e-06 2024-09-04T17:45:57.879 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.879 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.879 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:45:57.879 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487322+0000", 2024-09-04T17:45:57.880 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4199999999999996e-06 2024-09-04T17:45:57.880 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.880 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.880 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:45:57.880 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487329+0000", 2024-09-04T17:45:57.880 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2599999999999999e-06 2024-09-04T17:45:57.880 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.880 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.880 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:45:57.880 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.506432+0000", 2024-09-04T17:45:57.880 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.019102592000000002 2024-09-04T17:45:57.880 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.880 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.881 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:45:57.881 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.506570+0000", 2024-09-04T17:45:57.881 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013837300000000001 2024-09-04T17:45:57.881 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.881 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.881 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 0", 2024-09-04T17:45:57.881 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.506771+0000", 2024-09-04T17:45:57.881 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000200748 2024-09-04T17:45:57.881 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.881 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.881 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:45:57.881 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.508859+0000", 2024-09-04T17:45:57.881 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0020881010000000002 2024-09-04T17:45:57.881 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.882 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.882 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:45:57.882 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.509919+0000", 2024-09-04T17:45:57.882 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0010601580000000001 2024-09-04T17:45:57.882 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.882 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.882 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:45:57.882 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.510002+0000", 2024-09-04T17:45:57.882 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.2719999999999994e-05 2024-09-04T17:45:57.882 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.882 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.882 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:45:57.882 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.510066+0000", 2024-09-04T17:45:57.883 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.4095000000000002e-05 2024-09-04T17:45:57.883 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:45:57.883 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:45:57.883 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:45:57.883 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.883 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.883 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4219.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-04T17:45:57.883 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:45:42.487345+0000", 2024-09-04T17:45:57.883 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 15.329605823, 2024-09-04T17:45:57.883 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.025395999999999998, 2024-09-04T17:45:57.883 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:45:57.883 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:45:57.883 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:45:57.883 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:45:57.884 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4219", 2024-09-04T17:45:57.884 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.190:0/3705085728", 2024-09-04T17:45:57.884 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 52 2024-09-04T17:45:57.884 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.884 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:45:57.884 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.884 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:45:57.884 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487345+0000", 2024-09-04T17:45:57.884 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:45:57.884 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.884 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.884 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:45:57.884 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487345+0000", 2024-09-04T17:45:57.884 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:45:57.885 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.885 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.885 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:45:57.885 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487346+0000", 2024-09-04T17:45:57.885 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.4600000000000004e-07 2024-09-04T17:45:57.885 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.885 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.885 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:45:57.885 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487351+0000", 2024-09-04T17:45:57.885 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.9399999999999999e-06 2024-09-04T17:45:57.885 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.885 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.885 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:45:57.886 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487361+0000", 2024-09-04T17:45:57.886 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.3840000000000003e-06 2024-09-04T17:45:57.886 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.886 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.886 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:45:57.886 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487368+0000", 2024-09-04T17:45:57.886 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1729999999999998e-06 2024-09-04T17:45:57.886 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.886 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.886 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:45:57.886 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.507626+0000", 2024-09-04T17:45:57.886 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.020257547000000001 2024-09-04T17:45:57.886 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.886 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.887 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:45:57.887 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.507861+0000", 2024-09-04T17:45:57.887 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000235708 2024-09-04T17:45:57.887 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.887 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.887 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 0", 2024-09-04T17:45:57.887 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.508188+0000", 2024-09-04T17:45:57.887 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00032624800000000001 2024-09-04T17:45:57.887 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.887 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.887 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:45:57.887 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.509521+0000", 2024-09-04T17:45:57.887 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.001333433 2024-09-04T17:45:57.887 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.888 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.888 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:45:57.888 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.512613+0000", 2024-09-04T17:45:57.888 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0030923919999999998 2024-09-04T17:45:57.888 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.888 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.888 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:45:57.888 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.512686+0000", 2024-09-04T17:45:57.888 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3107000000000003e-05 2024-09-04T17:45:57.888 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.888 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.888 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:45:57.888 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.512741+0000", 2024-09-04T17:45:57.888 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4422000000000002e-05 2024-09-04T17:45:57.889 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:45:57.889 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:45:57.889 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:45:57.889 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.889 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.889 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4219.0:69 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-04T17:45:57.889 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:45:42.733113+0000", 2024-09-04T17:45:57.889 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 15.083837234000001, 2024-09-04T17:45:57.889 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.010816174, 2024-09-04T17:45:57.889 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:45:57.889 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:45:57.889 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:45:57.889 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:45:57.889 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4219", 2024-09-04T17:45:57.890 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.190:0/3705085728", 2024-09-04T17:45:57.890 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 69 2024-09-04T17:45:57.890 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.890 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:45:57.890 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.890 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:45:57.890 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.733113+0000", 2024-09-04T17:45:57.890 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:45:57.890 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.890 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.890 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:45:57.890 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.733113+0000", 2024-09-04T17:45:57.890 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:45:57.890 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.891 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.891 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:45:57.891 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.733114+0000", 2024-09-04T17:45:57.891 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1500000000000004e-07 2024-09-04T17:45:57.891 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.891 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.891 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:45:57.891 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.733180+0000", 2024-09-04T17:45:57.891 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.5382999999999994e-05 2024-09-04T17:45:57.891 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.891 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.891 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:45:57.891 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.733188+0000", 2024-09-04T17:45:57.891 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.4819999999999994e-06 2024-09-04T17:45:57.892 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.892 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.892 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:45:57.892 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.733194+0000", 2024-09-04T17:45:57.892 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.541e-06 2024-09-04T17:45:57.892 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.892 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.892 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:45:57.892 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.736018+0000", 2024-09-04T17:45:57.892 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0028243130000000002 2024-09-04T17:45:57.892 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.892 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.892 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:45:57.893 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.736097+0000", 2024-09-04T17:45:57.893 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.9288000000000004e-05 2024-09-04T17:45:57.893 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.893 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.893 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 0", 2024-09-04T17:45:57.893 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.736195+0000", 2024-09-04T17:45:57.893 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.7831000000000006e-05 2024-09-04T17:45:57.893 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.893 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.893 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:45:57.893 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.740389+0000", 2024-09-04T17:45:57.893 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0041941119999999998 2024-09-04T17:45:57.894 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.894 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.894 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:45:57.894 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.743834+0000", 2024-09-04T17:45:57.894 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0034443770000000002 2024-09-04T17:45:57.894 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.894 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.894 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:45:57.894 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.743876+0000", 2024-09-04T17:45:57.894 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2243999999999997e-05 2024-09-04T17:45:57.894 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.894 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.894 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:45:57.895 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.743930+0000", 2024-09-04T17:45:57.895 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.3888000000000003e-05 2024-09-04T17:45:57.895 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:45:57.895 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:45:57.895 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:45:57.895 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.895 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.895 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4219.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-04T17:45:57.895 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:45:42.733210+0000", 2024-09-04T17:45:57.895 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 15.083740897, 2024-09-04T17:45:57.895 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.012565447, 2024-09-04T17:45:57.895 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:45:57.895 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:45:57.895 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:45:57.896 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:45:57.896 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4219", 2024-09-04T17:45:57.896 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.190:0/3705085728", 2024-09-04T17:45:57.896 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 70 2024-09-04T17:45:57.896 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.896 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:45:57.896 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.896 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:45:57.896 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.733210+0000", 2024-09-04T17:45:57.896 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:45:57.896 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.896 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.896 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:45:57.897 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.733210+0000", 2024-09-04T17:45:57.897 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:45:57.897 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.897 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.897 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:45:57.897 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.733211+0000", 2024-09-04T17:45:57.897 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.5700000000000001e-07 2024-09-04T17:45:57.897 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.897 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.897 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:45:57.897 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.733217+0000", 2024-09-04T17:45:57.897 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.0839999999999997e-06 2024-09-04T17:45:57.897 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.898 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.898 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:45:57.898 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.733226+0000", 2024-09-04T17:45:57.898 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5740000000000002e-06 2024-09-04T17:45:57.898 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.898 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.898 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:45:57.898 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.733231+0000", 2024-09-04T17:45:57.898 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.3420000000000001e-06 2024-09-04T17:45:57.898 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.898 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.898 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:45:57.898 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.737079+0000", 2024-09-04T17:45:57.898 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.003847825 2024-09-04T17:45:57.899 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.899 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.899 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:45:57.899 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.737160+0000", 2024-09-04T17:45:57.899 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.1917999999999997e-05 2024-09-04T17:45:57.899 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.899 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.899 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 0", 2024-09-04T17:45:57.899 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.737261+0000", 2024-09-04T17:45:57.899 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000100556 2024-09-04T17:45:57.899 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.899 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.899 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:45:57.899 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.740399+0000", 2024-09-04T17:45:57.900 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0031382519999999998 2024-09-04T17:45:57.900 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.900 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.900 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:45:57.900 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.745679+0000", 2024-09-04T17:45:57.900 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0052797169999999997 2024-09-04T17:45:57.900 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.900 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.900 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:45:57.900 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.745718+0000", 2024-09-04T17:45:57.900 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.9134000000000002e-05 2024-09-04T17:45:57.900 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.900 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.900 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:45:57.901 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.745775+0000", 2024-09-04T17:45:57.901 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7188000000000002e-05 2024-09-04T17:45:57.901 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:45:57.901 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:45:57.901 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:45:57.901 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.901 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.901 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4219.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-04T17:45:57.901 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:45:42.733259+0000", 2024-09-04T17:45:57.901 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 15.083691846000001, 2024-09-04T17:45:57.901 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.01272336, 2024-09-04T17:45:57.901 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:45:57.901 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:45:57.902 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:45:57.902 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:45:57.902 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4219", 2024-09-04T17:45:57.902 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.190:0/3705085728", 2024-09-04T17:45:57.902 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 71 2024-09-04T17:45:57.902 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.902 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:45:57.902 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.902 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:45:57.902 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.733259+0000", 2024-09-04T17:45:57.902 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:45:57.902 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.902 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.903 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:45:57.903 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.733259+0000", 2024-09-04T17:45:57.903 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:45:57.903 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.903 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.903 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:45:57.903 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.733260+0000", 2024-09-04T17:45:57.903 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.2699999999999998e-07 2024-09-04T17:45:57.903 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.903 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.903 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:45:57.903 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.733266+0000", 2024-09-04T17:45:57.903 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.3969999999999999e-06 2024-09-04T17:45:57.903 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.904 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.904 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:45:57.904 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.733277+0000", 2024-09-04T17:45:57.904 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0998999999999999e-05 2024-09-04T17:45:57.904 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.904 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.904 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:45:57.904 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.733282+0000", 2024-09-04T17:45:57.904 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6110000000000004e-06 2024-09-04T17:45:57.904 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.904 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.904 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:45:57.904 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.737734+0000", 2024-09-04T17:45:57.905 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0044519859999999998 2024-09-04T17:45:57.905 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.905 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.905 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:45:57.905 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.737856+0000", 2024-09-04T17:45:57.905 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00012253599999999999 2024-09-04T17:45:57.905 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.905 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.905 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 0", 2024-09-04T17:45:57.905 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.738048+0000", 2024-09-04T17:45:57.905 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019119300000000001 2024-09-04T17:45:57.905 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.905 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.906 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:45:57.906 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.740760+0000", 2024-09-04T17:45:57.906 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0027129400000000001 2024-09-04T17:45:57.906 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.906 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.906 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:45:57.906 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.745845+0000", 2024-09-04T17:45:57.906 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0050843909999999997 2024-09-04T17:45:57.906 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.906 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.906 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:45:57.906 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.745945+0000", 2024-09-04T17:45:57.906 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00010063799999999999 2024-09-04T17:45:57.906 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.906 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.907 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:45:57.907 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.745982+0000", 2024-09-04T17:45:57.907 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.6742000000000002e-05 2024-09-04T17:45:57.907 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:45:57.907 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:45:57.907 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:45:57.907 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.907 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.907 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4219.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-04T17:45:57.907 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:45:42.733306+0000", 2024-09-04T17:45:57.907 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 15.083644694, 2024-09-04T17:45:57.907 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.012829217, 2024-09-04T17:45:57.907 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:45:57.908 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:45:57.908 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:45:57.908 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:45:57.908 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4219", 2024-09-04T17:45:57.908 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.190:0/3705085728", 2024-09-04T17:45:57.908 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 72 2024-09-04T17:45:57.908 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.908 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:45:57.908 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.908 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:45:57.908 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.733306+0000", 2024-09-04T17:45:57.908 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:45:57.908 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.909 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.909 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:45:57.909 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.733306+0000", 2024-09-04T17:45:57.909 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:45:57.909 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.909 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.909 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:45:57.909 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.733307+0000", 2024-09-04T17:45:57.909 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.5599999999999995e-07 2024-09-04T17:45:57.909 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.909 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.909 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:45:57.909 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.733398+0000", 2024-09-04T17:45:57.909 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.0982000000000004e-05 2024-09-04T17:45:57.910 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.910 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.910 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:45:57.910 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.733406+0000", 2024-09-04T17:45:57.910 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.1769999999999998e-06 2024-09-04T17:45:57.910 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.910 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.910 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:45:57.910 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.733414+0000", 2024-09-04T17:45:57.910 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.1259999999999998e-06 2024-09-04T17:45:57.910 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.910 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.910 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:45:57.910 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.738652+0000", 2024-09-04T17:45:57.911 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0052379790000000002 2024-09-04T17:45:57.911 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.911 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.911 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:45:57.911 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.738717+0000", 2024-09-04T17:45:57.911 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.5078000000000003e-05 2024-09-04T17:45:57.911 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.911 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.911 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 0", 2024-09-04T17:45:57.911 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.738818+0000", 2024-09-04T17:45:57.911 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00010142 2024-09-04T17:45:57.911 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.911 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.912 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:45:57.912 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.740899+0000", 2024-09-04T17:45:57.912 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0020805760000000002 2024-09-04T17:45:57.912 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.912 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.912 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:45:57.912 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.746036+0000", 2024-09-04T17:45:57.912 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0051371899999999998 2024-09-04T17:45:57.912 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.912 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.912 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:45:57.912 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.746083+0000", 2024-09-04T17:45:57.913 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7298e-05 2024-09-04T17:45:57.913 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.913 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.913 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:45:57.913 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.746135+0000", 2024-09-04T17:45:57.913 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1835000000000001e-05 2024-09-04T17:45:57.913 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:45:57.913 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:45:57.913 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:45:57.913 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.913 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.913 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4219.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-04T17:45:57.913 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:45:42.733430+0000", 2024-09-04T17:45:57.913 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 15.083520614999999, 2024-09-04T17:45:57.914 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.013004422, 2024-09-04T17:45:57.914 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:45:57.914 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:45:57.914 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:45:57.914 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:45:57.914 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4219", 2024-09-04T17:45:57.914 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.190:0/3705085728", 2024-09-04T17:45:57.914 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 73 2024-09-04T17:45:57.914 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.914 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:45:57.914 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.914 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:45:57.914 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.733430+0000", 2024-09-04T17:45:57.915 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:45:57.915 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.915 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.915 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:45:57.915 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.733430+0000", 2024-09-04T17:45:57.915 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:45:57.915 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.915 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.915 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:45:57.915 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.733431+0000", 2024-09-04T17:45:57.915 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.4200000000000005e-07 2024-09-04T17:45:57.915 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.915 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.916 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:45:57.916 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.733439+0000", 2024-09-04T17:45:57.916 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.6899999999999992e-06 2024-09-04T17:45:57.916 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.916 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.916 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:45:57.916 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.733451+0000", 2024-09-04T17:45:57.916 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2163e-05 2024-09-04T17:45:57.916 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.916 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.916 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:45:57.916 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.733456+0000", 2024-09-04T17:45:57.916 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9269999999999996e-06 2024-09-04T17:45:57.916 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.917 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.917 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:45:57.917 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.739605+0000", 2024-09-04T17:45:57.917 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0061489650000000002 2024-09-04T17:45:57.917 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.917 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.917 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:45:57.917 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.739718+0000", 2024-09-04T17:45:57.917 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00011315 2024-09-04T17:45:57.917 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.917 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.917 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 0", 2024-09-04T17:45:57.917 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.739900+0000", 2024-09-04T17:45:57.918 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00018205399999999999 2024-09-04T17:45:57.918 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.918 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.918 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:45:57.918 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.743294+0000", 2024-09-04T17:45:57.918 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0033936050000000001 2024-09-04T17:45:57.918 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.918 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.918 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:45:57.918 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.746243+0000", 2024-09-04T17:45:57.918 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0029493140000000002 2024-09-04T17:45:57.918 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.918 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.919 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:45:57.919 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.746363+0000", 2024-09-04T17:45:57.919 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000120599 2024-09-04T17:45:57.919 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.919 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.919 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:45:57.919 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.746435+0000", 2024-09-04T17:45:57.919 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1112999999999995e-05 2024-09-04T17:45:57.919 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:45:57.919 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:45:57.919 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:45:57.919 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.920 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.920 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4219.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-04T17:45:57.920 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:45:42.733482+0000", 2024-09-04T17:45:57.920 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 15.083468322, 2024-09-04T17:45:57.920 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.013108754, 2024-09-04T17:45:57.920 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:45:57.920 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:45:57.920 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:45:57.920 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:45:57.920 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4219", 2024-09-04T17:45:57.920 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.190:0/3705085728", 2024-09-04T17:45:57.920 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 74 2024-09-04T17:45:57.920 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.921 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:45:57.921 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.921 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:45:57.921 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.733482+0000", 2024-09-04T17:45:57.921 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:45:57.921 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.921 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.921 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:45:57.921 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.733482+0000", 2024-09-04T17:45:57.921 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:45:57.921 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.921 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.921 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:45:57.922 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.733483+0000", 2024-09-04T17:45:57.922 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.5600000000000005e-07 2024-09-04T17:45:57.922 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.922 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.922 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:45:57.922 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.733487+0000", 2024-09-04T17:45:57.922 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.8380000000000004e-06 2024-09-04T17:45:57.922 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.922 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.922 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:45:57.922 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.733496+0000", 2024-09-04T17:45:57.922 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1030000000000008e-06 2024-09-04T17:45:57.922 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.922 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.923 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:45:57.923 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.733507+0000", 2024-09-04T17:45:57.923 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1447999999999999e-05 2024-09-04T17:45:57.923 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.923 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.923 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:45:57.923 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.741100+0000", 2024-09-04T17:45:57.923 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0075924989999999999 2024-09-04T17:45:57.923 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.923 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.923 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:45:57.923 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.741175+0000", 2024-09-04T17:45:57.923 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.5432999999999994e-05 2024-09-04T17:45:57.923 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.924 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.924 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 0", 2024-09-04T17:45:57.924 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.741280+0000", 2024-09-04T17:45:57.924 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000104653 2024-09-04T17:45:57.924 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.924 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.924 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:45:57.924 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.743506+0000", 2024-09-04T17:45:57.924 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0022260169999999998 2024-09-04T17:45:57.924 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.924 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.925 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:45:57.925 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.746492+0000", 2024-09-04T17:45:57.925 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.002986027 2024-09-04T17:45:57.925 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.925 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.925 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:45:57.925 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.746533+0000", 2024-09-04T17:45:57.925 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.1278e-05 2024-09-04T17:45:57.925 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.925 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.925 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:45:57.925 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.746591+0000", 2024-09-04T17:45:57.925 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7802e-05 2024-09-04T17:45:57.926 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:45:57.926 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:45:57.926 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:45:57.926 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.926 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.926 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4219.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-04T17:45:57.926 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:45:42.733592+0000", 2024-09-04T17:45:57.926 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 15.083358608999999, 2024-09-04T17:45:57.926 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.013216525999999999, 2024-09-04T17:45:57.926 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:45:57.926 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:45:57.926 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:45:57.926 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:45:57.927 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4219", 2024-09-04T17:45:57.927 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.190:0/3705085728", 2024-09-04T17:45:57.927 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 75 2024-09-04T17:45:57.927 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.927 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:45:57.927 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.927 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:45:57.927 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.733592+0000", 2024-09-04T17:45:57.927 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:45:57.927 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.927 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.927 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:45:57.927 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.733592+0000", 2024-09-04T17:45:57.927 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:45:57.928 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.928 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.928 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:45:57.928 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.733593+0000", 2024-09-04T17:45:57.928 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5499999999999996e-07 2024-09-04T17:45:57.928 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.928 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.928 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:45:57.928 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.733677+0000", 2024-09-04T17:45:57.928 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.3725000000000002e-05 2024-09-04T17:45:57.928 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.928 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.928 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:45:57.929 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.733687+0000", 2024-09-04T17:45:57.929 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0234000000000001e-05 2024-09-04T17:45:57.929 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.929 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.929 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:45:57.929 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.733693+0000", 2024-09-04T17:45:57.929 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.4180000000000002e-06 2024-09-04T17:45:57.929 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.929 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.929 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:45:57.929 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.741648+0000", 2024-09-04T17:45:57.929 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0079542150000000006 2024-09-04T17:45:57.929 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.929 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.930 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:45:57.930 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.741724+0000", 2024-09-04T17:45:57.930 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.6508000000000007e-05 2024-09-04T17:45:57.930 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.930 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.930 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 0", 2024-09-04T17:45:57.930 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.741892+0000", 2024-09-04T17:45:57.930 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000168133 2024-09-04T17:45:57.930 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.930 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.930 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:45:57.930 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.743515+0000", 2024-09-04T17:45:57.930 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.001622903 2024-09-04T17:45:57.931 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.931 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.931 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:45:57.931 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.746655+0000", 2024-09-04T17:45:57.931 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0031402130000000002 2024-09-04T17:45:57.931 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.931 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.931 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:45:57.931 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.746749+0000", 2024-09-04T17:45:57.931 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.3256000000000003e-05 2024-09-04T17:45:57.931 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.931 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.931 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:45:57.931 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.746809+0000", 2024-09-04T17:45:57.932 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.9966000000000002e-05 2024-09-04T17:45:57.932 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:45:57.932 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:45:57.932 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:45:57.932 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.932 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.932 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4219.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-04T17:45:57.932 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:45:42.733705+0000", 2024-09-04T17:45:57.932 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 15.083245515, 2024-09-04T17:45:57.932 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.01366711, 2024-09-04T17:45:57.932 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:45:57.932 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:45:57.932 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:45:57.932 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:45:57.933 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4219", 2024-09-04T17:45:57.933 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.190:0/3705085728", 2024-09-04T17:45:57.933 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 76 2024-09-04T17:45:57.933 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.933 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:45:57.933 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.933 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:45:57.933 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.733705+0000", 2024-09-04T17:45:57.933 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:45:57.933 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.933 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.933 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:45:57.933 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.733705+0000", 2024-09-04T17:45:57.934 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:45:57.934 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.934 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.934 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:45:57.934 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.733706+0000", 2024-09-04T17:45:57.934 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.0100000000000005e-07 2024-09-04T17:45:57.934 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.934 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.934 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:45:57.935 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.733709+0000", 2024-09-04T17:45:57.935 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.2799999999999999e-06 2024-09-04T17:45:57.935 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.935 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.935 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:45:57.935 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.733714+0000", 2024-09-04T17:45:57.935 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.096e-06 2024-09-04T17:45:57.935 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.935 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.935 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:45:57.935 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.733720+0000", 2024-09-04T17:45:57.935 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6729999999999997e-06 2024-09-04T17:45:57.935 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.935 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.935 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:45:57.936 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.742736+0000", 2024-09-04T17:45:57.936 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0090160350000000004 2024-09-04T17:45:57.936 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.936 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.936 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:45:57.936 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.742827+0000", 2024-09-04T17:45:57.936 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.0616000000000001e-05 2024-09-04T17:45:57.936 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.936 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.936 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 0", 2024-09-04T17:45:57.936 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.742936+0000", 2024-09-04T17:45:57.936 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000109948 2024-09-04T17:45:57.936 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.937 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.937 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:45:57.937 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.743946+0000", 2024-09-04T17:45:57.937 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.001009249 2024-09-04T17:45:57.937 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.937 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.937 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:45:57.937 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.747280+0000", 2024-09-04T17:45:57.937 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0033343140000000001 2024-09-04T17:45:57.937 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.937 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.937 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:45:57.937 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.747318+0000", 2024-09-04T17:45:57.938 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.8167000000000002e-05 2024-09-04T17:45:57.938 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.938 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.938 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:45:57.938 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.747372+0000", 2024-09-04T17:45:57.938 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4131000000000001e-05 2024-09-04T17:45:57.938 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:45:57.938 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:45:57.938 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:45:57.938 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.938 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.938 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4219.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-04T17:45:57.938 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:45:43.085719+0000", 2024-09-04T17:45:57.938 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 14.731231888, 2024-09-04T17:45:57.939 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.010475935, 2024-09-04T17:45:57.939 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:45:57.939 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:45:57.939 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:45:57.939 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:45:57.939 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4219", 2024-09-04T17:45:57.939 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.190:0/3705085728", 2024-09-04T17:45:57.939 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 94 2024-09-04T17:45:57.939 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.939 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:45:57.939 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.939 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:45:57.939 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:43.085719+0000", 2024-09-04T17:45:57.940 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:45:57.940 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.940 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.940 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:45:57.940 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:43.085719+0000", 2024-09-04T17:45:57.940 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:45:57.940 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.940 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.940 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:45:57.940 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:43.085720+0000", 2024-09-04T17:45:57.940 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.3600000000000002e-07 2024-09-04T17:45:57.940 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.940 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.941 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:45:57.941 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:43.085726+0000", 2024-09-04T17:45:57.941 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.8239999999999996e-06 2024-09-04T17:45:57.941 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.941 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.941 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:45:57.941 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:43.085735+0000", 2024-09-04T17:45:57.941 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.7929999999999992e-06 2024-09-04T17:45:57.941 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.941 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.941 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:45:57.941 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:43.085743+0000", 2024-09-04T17:45:57.941 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1169999999999998e-06 2024-09-04T17:45:57.941 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.942 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.942 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:45:57.942 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:43.093710+0000", 2024-09-04T17:45:57.942 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0079678709999999996 2024-09-04T17:45:57.942 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.942 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.942 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:45:57.942 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:43.093943+0000", 2024-09-04T17:45:57.942 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000232375 2024-09-04T17:45:57.942 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.942 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.942 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 0", 2024-09-04T17:45:57.942 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:43.094341+0000", 2024-09-04T17:45:57.942 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00039790700000000002 2024-09-04T17:45:57.943 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.943 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.943 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:45:57.943 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:43.095904+0000", 2024-09-04T17:45:57.943 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0015635110000000001 2024-09-04T17:45:57.943 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.943 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.943 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:45:57.943 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:43.095944+0000", 2024-09-04T17:45:57.943 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0157000000000002e-05 2024-09-04T17:45:57.943 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.943 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.943 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:45:57.944 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:43.096073+0000", 2024-09-04T17:45:57.944 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00012902500000000001 2024-09-04T17:45:57.944 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:45:57.944 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:45:57.944 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:45:57.944 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:43.096195+0000", 2024-09-04T17:45:57.944 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000121419 2024-09-04T17:45:57.944 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:45:57.944 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:45:57.944 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:45:57.944 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:45:57.944 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:45:57.944 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:45:57.946 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:45:57.944+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:45:57.946 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-04T17:45:57.949 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is set 2024-09-04T17:45:58.047 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:45:58.046+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:45:58.076 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:45:58.077 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:45:58.091 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-04T17:45:58.147 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:45:58.148+0000 7f505d642640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26446) UID: 0 2024-09-04T17:45:58.248 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:45:58.248+0000 7f505d642640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26446) UID: 0 2024-09-04T17:45:58.262 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:45:58.262 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:45:58.277 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-04T17:45:58.349 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:45:58.348+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:45:58.394 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:45:58.395 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:45:58.408 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-04T17:45:58.454 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:45:58.454+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:45:58.526 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:45:58.526 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:45:58.541 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-04T17:45:58.583 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:45:58.582+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:45:58.652 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:45:58.650+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:45:58.698 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:45:58.698 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:45:58.711 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-04T17:45:58.754 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:45:58.754+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:45:58.854 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:45:58.852+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:45:58.879 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:45:58.879 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:45:58.893 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-04T17:45:58.955 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:45:58.954+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:45:59.011 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:45:59.011 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:45:59.024 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-04T17:45:59.066 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:45:59.066+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:45:59.156 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:45:59.156+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:45:59.185 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:45:59.185 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:45:59.198 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-04T17:45:59.257 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:45:59.254+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:45:59.312 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:45:59.312 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:45:59.325 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-04T17:45:59.371 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:45:59.372+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:45:59.457 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:45:59.457 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:45:59.457 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:45:59.457 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:45:59.460 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:45:59.458+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:45:59.472 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-04T17:45:59.475 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-04T17:45:59.503 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:45:59.504 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:45:59.539 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:45:59.540 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:45:59.560 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:45:59.560+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:45:59.574 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-04T17:45:59.575 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-04T17:45:59.593 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:45:59.593 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:45:59.597 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-04T17:45:59.597 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-04T17:45:59.639 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:45:59.640 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:45:59.654 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-04T17:45:59.695 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:45:59.696+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:45:59.762 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:45:59.760+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:45:59.820 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:45:59.820 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:45:59.835 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-04T17:45:59.856 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:45:59.879 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:45:59.880+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:45:59.882 INFO:teuthology.orchestra.run.smithi092.stdout:3 write 0xca54cae thru 0xca5d41f (0x8772 bytes) 2024-09-04T17:45:59.888 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:45:59.888 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:45:59.889 INFO:teuthology.orchestra.run.smithi092.stdout:9 punch from 0x570870f to 0x5713e43, (0xb734 bytes) 2024-09-04T17:45:59.889 INFO:teuthology.orchestra.run.smithi092.stdout:10 write 0x22a4ede thru 0x22b43bb (0xf4de bytes) 2024-09-04T17:45:59.895 INFO:teuthology.orchestra.run.smithi092.stdout:11 punch from 0xc7e6b30 to 0xc7f3321, (0xc7f1 bytes) 2024-09-04T17:45:59.895 INFO:teuthology.orchestra.run.smithi092.stdout:12 write 0x51eca57 thru 0x51f7b71 (0xb11b bytes) 2024-09-04T17:45:59.900 INFO:teuthology.orchestra.run.smithi092.stdout:13 read 0x53c968e thru 0x53d8dac (0xf71f bytes) 2024-09-04T17:45:59.901 INFO:teuthology.orchestra.run.smithi092.stdout:15 write 0xbc6176f thru 0xbc683aa (0x6c3c bytes) 2024-09-04T17:45:59.906 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:45:59.906 INFO:teuthology.orchestra.run.smithi092.stdout:18 write 0xb591e2b thru 0xb5984b7 (0x668d bytes) 2024-09-04T17:45:59.911 INFO:teuthology.orchestra.run.smithi092.stdout:19 punch from 0xbdc0d7e to 0xbdce86f, (0xdaf1 bytes) 2024-09-04T17:45:59.911 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:45:59.911 INFO:teuthology.orchestra.run.smithi092.stdout:21 punch from 0xc90092d to 0xc90ad11, (0xa3e4 bytes) 2024-09-04T17:45:59.911 INFO:teuthology.orchestra.run.smithi092.stdout:23 writesame 0xa912442 thru 0xa91c476 data_size 0x1f(0xa035 bytes) 2024-09-04T17:45:59.917 INFO:teuthology.orchestra.run.smithi092.stdout:24 punch from 0x783dc41 to 0x78472e1, (0x96a0 bytes) 2024-09-04T17:45:59.917 INFO:teuthology.orchestra.run.smithi092.stdout:25 read 0x2d0ef8 thru 0x2d941c (0x8525 bytes) 2024-09-04T17:45:59.929 INFO:teuthology.orchestra.run.smithi092.stdout:26 writesame 0xd4585cd thru 0xd45ecac data_size 0x10(0x66e0 bytes) 2024-09-04T17:45:59.929 INFO:teuthology.orchestra.run.smithi092.stdout:28 write 0x94e77cf thru 0x94f758b (0xfdbd bytes) 2024-09-04T17:45:59.935 INFO:teuthology.orchestra.run.smithi092.stdout:truncating to largest ever: 0x24371d8 2024-09-04T17:45:59.958 INFO:teuthology.orchestra.run.smithi092.stdout:29 trunc from 0xd45ecad to 0x24371d8 2024-09-04T17:45:59.964 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:45:59.964+0000 7f505d642640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26446) UID: 0 2024-09-04T17:45:59.978 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-04T17:45:59.998 INFO:teuthology.orchestra.run.smithi092.stdout:32 writesame 0x5a54c37 thru 0x5a562c9 data_size 0x1(0x1693 bytes) 2024-09-04T17:45:59.998 INFO:teuthology.orchestra.run.smithi092.stdout:33 punch from 0x805eb5 to 0x815dae, (0xfef9 bytes) 2024-09-04T17:45:59.998 INFO:teuthology.orchestra.run.smithi092.stdout:34 write 0x3b19f03 thru 0x3b26b83 (0xcc81 bytes) 2024-09-04T17:46:00.004 INFO:teuthology.orchestra.run.smithi092.stdout:35 read 0x3baff26 thru 0x3bb3674 (0x374f bytes) 2024-09-04T17:46:00.005 INFO:teuthology.orchestra.run.smithi092.stdout:36 write 0x3aed5c5 thru 0x3afb019 (0xda55 bytes) 2024-09-04T17:46:00.008 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:00.014 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:00.014 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:00.025 INFO:teuthology.orchestra.run.smithi092.stdout:39 write 0xa653ea5 thru 0xa65dffc (0xa158 bytes) 2024-09-04T17:46:00.033 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-04T17:46:00.033 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:00.033 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:00.033 INFO:teuthology.orchestra.run.smithi092.stdout:42 write 0x52692a thru 0x52979f (0x2e76 bytes) 2024-09-04T17:46:00.036 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:00.036 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:00.036 INFO:teuthology.orchestra.run.smithi092.stdout:45 write 0x416c216 thru 0x41793b4 (0xd19f bytes) 2024-09-04T17:46:00.039 INFO:teuthology.orchestra.run.smithi092.stdout:48 write 0x303ba5 thru 0x30d3f5 (0x9851 bytes) 2024-09-04T17:46:00.043 INFO:teuthology.orchestra.run.smithi092.stdout:truncating to largest ever: 0x9626ed8 2024-09-04T17:46:00.045 INFO:teuthology.orchestra.run.smithi092.stdout:50 trunc from 0xa65dffd to 0x9626ed8 2024-09-04T17:46:00.061 INFO:teuthology.orchestra.run.smithi092.stdout:51 read 0x60eac01 thru 0x60f588d (0xac8d bytes) 2024-09-04T17:46:00.061 INFO:teuthology.orchestra.run.smithi092.stdout:52 read 0x5f266ab thru 0x5f2c3a6 (0x5cfc bytes) 2024-09-04T17:46:00.061 INFO:teuthology.orchestra.run.smithi092.stdout:truncating to largest ever: 0xc01281a 2024-09-04T17:46:00.065 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:00.062+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:00.066 INFO:teuthology.orchestra.run.smithi092.stdout:54 trunc from 0x9626ed8 to 0xc01281a 2024-09-04T17:46:00.084 INFO:teuthology.orchestra.run.smithi092.stdout:55 trunc from 0xc01281a to 0x6d43476 2024-09-04T17:46:00.092 INFO:teuthology.orchestra.run.smithi092.stdout:58 read 0x1b9170b thru 0x1b97ea1 (0x6797 bytes) 2024-09-04T17:46:00.103 INFO:teuthology.orchestra.run.smithi092.stdout:59 writesame 0x9462461 thru 0x94626da data_size 0x1(0x27a bytes) 2024-09-04T17:46:00.111 INFO:teuthology.orchestra.run.smithi092.stdout:61 write 0xae27dc0 thru 0xae2af27 (0x3168 bytes) 2024-09-04T17:46:00.115 INFO:teuthology.orchestra.run.smithi092.stdout:62 write 0x630b756 thru 0x631a597 (0xee42 bytes) 2024-09-04T17:46:00.119 INFO:teuthology.orchestra.run.smithi092.stdout:63 write 0x45171fd thru 0x4522131 (0xaf35 bytes) 2024-09-04T17:46:00.124 INFO:teuthology.orchestra.run.smithi092.stdout:64 read 0x982a062 thru 0x983298e (0x892d bytes) 2024-09-04T17:46:00.124 INFO:teuthology.orchestra.run.smithi092.stdout:65 read 0x2312e5f thru 0x2315ff3 (0x3195 bytes) 2024-09-04T17:46:00.126 INFO:teuthology.orchestra.run.smithi092.stdout:67 trunc from 0xae2af28 to 0x8fd7bd5 2024-09-04T17:46:00.142 INFO:teuthology.orchestra.run.smithi092.stdout:68 write 0x935ff19 thru 0x936a2ef (0xa3d7 bytes) 2024-09-04T17:46:00.146 INFO:teuthology.orchestra.run.smithi092.stdout:69 read 0x475ccfb thru 0x4760dd5 (0x40db bytes) 2024-09-04T17:46:00.146 INFO:teuthology.orchestra.run.smithi092.stdout:70 write 0x3f1aaa9 thru 0x3f1e6e9 (0x3c41 bytes) 2024-09-04T17:46:00.150 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:00.151 INFO:teuthology.orchestra.run.smithi092.stdout:72 read 0x7ec5357 thru 0x7ecfe69 (0xab13 bytes) 2024-09-04T17:46:00.152 INFO:teuthology.orchestra.run.smithi092.stdout:73 trunc from 0x936a2f0 to 0x84b3b4c 2024-09-04T17:46:00.163 INFO:teuthology.orchestra.run.smithi092.stdout:75 punch from 0xb5f016 to 0xb61565, (0x254f bytes) 2024-09-04T17:46:00.163 INFO:teuthology.orchestra.run.smithi092.stdout:76 read 0x15348f3 thru 0x1543b52 (0xf260 bytes) 2024-09-04T17:46:00.163 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:00.163 INFO:teuthology.orchestra.run.smithi092.stdout:78 write 0x28c8b86 thru 0x28c8f99 (0x414 bytes) 2024-09-04T17:46:00.165 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:00.164+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:00.166 INFO:teuthology.orchestra.run.smithi092.stdout:80 punch from 0x693ff82 to 0x6949527, (0x95a5 bytes) 2024-09-04T17:46:00.166 INFO:teuthology.orchestra.run.smithi092.stdout:81 read 0x28f765 thru 0x29c0e4 (0xc980 bytes) 2024-09-04T17:46:00.166 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:00.167 INFO:teuthology.orchestra.run.smithi092.stdout:85 punch from 0x58b87d5 to 0x58c682e, (0xe059 bytes) 2024-09-04T17:46:00.167 INFO:teuthology.orchestra.run.smithi092.stdout:86 read 0x58bff6a thru 0x58c27ee (0x2885 bytes) 2024-09-04T17:46:00.167 INFO:teuthology.orchestra.run.smithi092.stdout:88 punch from 0x4940912 to 0x49473b7, (0x6aa5 bytes) 2024-09-04T17:46:00.167 INFO:teuthology.orchestra.run.smithi092.stdout:truncating to largest ever: 0xc8312dd 2024-09-04T17:46:00.171 INFO:teuthology.orchestra.run.smithi092.stdout:89 trunc from 0x84b3b4c to 0xc8312dd 2024-09-04T17:46:00.179 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:00.179 INFO:teuthology.orchestra.run.smithi092.stdout:91 read 0xffd460 thru 0x10015e4 (0x4185 bytes) 2024-09-04T17:46:00.179 INFO:teuthology.orchestra.run.smithi092.stdout:92 write 0x1910eaa thru 0x1917121 (0x6278 bytes) 2024-09-04T17:46:00.187 INFO:teuthology.orchestra.run.smithi092.stdout:94 trunc from 0xc8312dd to 0x9fcb894 2024-09-04T17:46:00.195 INFO:teuthology.orchestra.run.smithi092.stdout:96 write 0xdc9ce thru 0xe7101 (0xa734 bytes) 2024-09-04T17:46:00.196 INFO:teuthology.orchestra.run.smithi092.stdout:98 writesame 0x6fb95e9 thru 0x6fc35af data_size 0x1(0x9fc7 bytes) 2024-09-04T17:46:00.205 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:00.206 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:00.221 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-04T17:46:00.240 INFO:teuthology.orchestra.run.smithi092.stdout:100 read 0x2c82ab5 thru 0x2c896b3 (0x6bff bytes) 2024-09-04T17:46:00.246 INFO:teuthology.orchestra.run.smithi092.stdout:101 trunc from 0x9fcb894 to 0x6f4b907 2024-09-04T17:46:00.267 INFO:teuthology.orchestra.run.smithi092.stdout:102 trunc from 0x6f4b907 to 0xac901ba 2024-09-04T17:46:00.278 INFO:teuthology.orchestra.run.smithi092.stdout:103 punch from 0x6435cce to 0x643d948, (0x7c7a bytes) 2024-09-04T17:46:00.278 INFO:teuthology.orchestra.run.smithi092.stdout:104 write 0x185bed thru 0x18da13 (0x7e27 bytes) 2024-09-04T17:46:00.280 INFO:teuthology.orchestra.run.smithi092.stdout:106 read 0x18adabd thru 0x18bc493 (0xe9d7 bytes) 2024-09-04T17:46:00.300 INFO:teuthology.orchestra.run.smithi092.stdout:107 write 0xdfc5484 thru 0xdfccc94 (0x7811 bytes) 2024-09-04T17:46:00.305 INFO:teuthology.orchestra.run.smithi092.stdout:109 punch from 0xa4c75ae to 0xa4ce139, (0x6b8b bytes) 2024-09-04T17:46:00.306 INFO:teuthology.orchestra.run.smithi092.stdout:111 write 0xca962da thru 0xcaa02ec (0xa013 bytes) 2024-09-04T17:46:00.310 INFO:teuthology.orchestra.run.smithi092.stdout:112 read 0x92df120 thru 0x92e775e (0x863f bytes) 2024-09-04T17:46:00.311 INFO:teuthology.orchestra.run.smithi092.stdout:113 read 0x8652ec4 thru 0x865c367 (0x94a4 bytes) 2024-09-04T17:46:00.311 INFO:teuthology.orchestra.run.smithi092.stdout:114 write 0xadd2b48 thru 0xaddc6b2 (0x9b6b bytes) 2024-09-04T17:46:00.311 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:00.312+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:00.315 INFO:teuthology.orchestra.run.smithi092.stdout:115 punch from 0x7d2341d to 0x7d261bd, (0x2da0 bytes) 2024-09-04T17:46:00.315 INFO:teuthology.orchestra.run.smithi092.stdout:116 read 0x102ec9f thru 0x103bdc2 (0xd124 bytes) 2024-09-04T17:46:00.315 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:00.315 INFO:teuthology.orchestra.run.smithi092.stdout:119 write 0x30c0c44 thru 0x30ce857 (0xdc14 bytes) 2024-09-04T17:46:00.319 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:00.319 INFO:teuthology.orchestra.run.smithi092.stdout:123 punch from 0xf8d619 to 0xf9c1a4, (0xeb8b bytes) 2024-09-04T17:46:00.319 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:00.319 INFO:teuthology.orchestra.run.smithi092.stdout:125 writesame 0xdeee6a6 thru 0xdefce2f data_size 0xb2(0xe78a bytes) 2024-09-04T17:46:00.321 INFO:teuthology.orchestra.run.smithi092.stdout:126 read 0xdb5fc39 thru 0xdb627d0 (0x2b98 bytes) 2024-09-04T17:46:00.321 INFO:teuthology.orchestra.run.smithi092.stdout:127 read 0x427cd6a thru 0x4280c4a (0x3ee1 bytes) 2024-09-04T17:46:00.322 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:00.322 INFO:teuthology.orchestra.run.smithi092.stdout:129 writesame 0x6e0790d thru 0x6e14a25 data_size 0x7(0xd119 bytes) 2024-09-04T17:46:00.351 INFO:teuthology.orchestra.run.smithi092.stdout:130 trunc from 0xdfccc95 to 0x216f066 2024-09-04T17:46:00.367 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:46:00.364+0000 7f505d642640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26446) UID: 0 2024-09-04T17:46:00.382 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:00.395 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:00.395 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:00.409 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-04T17:46:00.416 INFO:teuthology.orchestra.run.smithi092.stdout:132 writesame 0xbb45303 thru 0xbb4d62b data_size 0x1(0x8329 bytes) 2024-09-04T17:46:00.425 INFO:tasks.ceph.osd.3:Stopped 2024-09-04T17:46:00.425 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.3 2024-09-04T17:46:00.425 INFO:tasks.thrashosds.thrasher:remote for osd 3 is ubuntu@smithi092.front.sepia.ceph.com 2024-09-04T17:46:00.425 DEBUG:teuthology.orchestra.run.smithi092:> type ceph-objectstore-tool 2024-09-04T17:46:00.431 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-04T17:46:00.431 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-3 --journal-path /var/lib/ceph/osd/ceph-3/journal --op list-pgs 2024-09-04T17:46:00.436 INFO:teuthology.orchestra.run.smithi092.stdout:133 write 0xee34398 thru 0xee3947b (0x50e4 bytes) 2024-09-04T17:46:00.441 INFO:teuthology.orchestra.run.smithi092.stdout:134 writesame 0x1b4146a thru 0x1b4bd19 data_size 0x8(0xa8b0 bytes) 2024-09-04T17:46:00.450 INFO:teuthology.orchestra.run.smithi092.stdout:135 write 0x4eec094 thru 0x4eec9d4 (0x941 bytes) 2024-09-04T17:46:00.457 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:00.457 INFO:teuthology.orchestra.run.smithi092.stdout:138 read 0x381f2d3 thru 0x382bdeb (0xcb19 bytes) 2024-09-04T17:46:00.457 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:00.457 INFO:teuthology.orchestra.run.smithi092.stdout:140 punch from 0xc90f2e to 0xc980c0, (0x7192 bytes) 2024-09-04T17:46:00.457 INFO:teuthology.orchestra.run.smithi092.stdout:141 write 0x8e5c862 thru 0x8e6108e (0x482d bytes) 2024-09-04T17:46:00.462 INFO:teuthology.orchestra.run.smithi092.stdout:142 writesame 0x3e523d7 thru 0x3e5a326 data_size 0x61(0x7f50 bytes) 2024-09-04T17:46:00.466 INFO:teuthology.orchestra.run.smithi092.stdout:143 write 0xcaee490 thru 0xcaeee3f (0x9b0 bytes) 2024-09-04T17:46:00.471 INFO:teuthology.orchestra.run.smithi092.stdout:144 write 0x14f3eaf thru 0x14f8f60 (0x50b2 bytes) 2024-09-04T17:46:00.475 INFO:teuthology.orchestra.run.smithi092.stdout:145 write 0xcdc313 thru 0xceb280 (0xef6e bytes) 2024-09-04T17:46:00.480 INFO:teuthology.orchestra.run.smithi092.stdout:146 punch from 0x60d462d to 0x60e3ad1, (0xf4a4 bytes) 2024-09-04T17:46:00.480 INFO:teuthology.orchestra.run.smithi092.stdout:147 punch from 0x42776f0 to 0x42851d5, (0xdae5 bytes) 2024-09-04T17:46:00.480 INFO:teuthology.orchestra.run.smithi092.stdout:148 clone 1 order 19 su 131072 sc 7 2024-09-04T17:46:00.499 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:00.500+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:00.568 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:00.566+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:00.581 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:00.581 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:00.595 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-04T17:46:00.683 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:00.684+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:00.761 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:00.761 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:00.770 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:00.766+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:00.775 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-04T17:46:00.807 INFO:teuthology.orchestra.run.smithi092.stdout:2.1 2024-09-04T17:46:00.871 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:00.870+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:00.943 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:00.943 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:00.958 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-04T17:46:00.973 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is unset 2024-09-04T17:46:01.003 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:01.004+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:01.073 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:01.070+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:01.082 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:01.083 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:01.097 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-04T17:46:01.183 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:01.184+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:01.225 INFO:teuthology.orchestra.run.smithi092.stdout:149 read 0x602e1e5 thru 0x602e3fd (0x219 bytes) 2024-09-04T17:46:01.225 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:01.225 INFO:teuthology.orchestra.run.smithi092.stdout:151 write 0x51e8320 thru 0x51efab0 (0x7791 bytes) 2024-09-04T17:46:01.230 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-3 --journal-path /var/lib/ceph/osd/ceph-3/journal --op export --pgid 2.1 --file /var/log/ceph/exp.2.1.3 2024-09-04T17:46:01.235 INFO:teuthology.orchestra.run.smithi092.stdout:152 write 0x184d271 thru 0x184eff0 (0x1d80 bytes) 2024-09-04T17:46:01.243 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:01.243 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:01.243 INFO:teuthology.orchestra.run.smithi092.stdout:156 punch from 0x24c8220 to 0x24cdb82, (0x5962 bytes) 2024-09-04T17:46:01.243 INFO:teuthology.orchestra.run.smithi092.stdout:157 read 0xbd7ffc8 thru 0xbd8a091 (0xa0ca bytes) 2024-09-04T17:46:01.243 INFO:teuthology.orchestra.run.smithi092.stdout:158 read 0xc18fab7 thru 0xc193344 (0x388e bytes) 2024-09-04T17:46:01.244 INFO:teuthology.orchestra.run.smithi092.stdout:159 writesame 0x410b3bf thru 0x411a167 data_size 0x1(0xeda9 bytes) 2024-09-04T17:46:01.266 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:01.266 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:01.270 INFO:teuthology.orchestra.run.smithi092.stdout:162 write 0xee2756a thru 0xee3230d (0xada4 bytes) 2024-09-04T17:46:01.274 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:01.272+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:01.276 INFO:teuthology.orchestra.run.smithi092.stdout:163 read 0xa4f9a4a thru 0xa4fe086 (0x463d bytes) 2024-09-04T17:46:01.280 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-04T17:46:01.285 INFO:teuthology.orchestra.run.smithi092.stdout:164 trunc from 0xee3947c to 0x7f507d1 2024-09-04T17:46:01.314 INFO:teuthology.orchestra.run.smithi092.stdout:166 read 0xa15cea thru 0xa16944 (0xc5b bytes) 2024-09-04T17:46:01.314 INFO:teuthology.orchestra.run.smithi092.stdout:truncating to largest ever: 0xd0d0420 2024-09-04T17:46:01.322 INFO:teuthology.orchestra.run.smithi092.stdout:167 trunc from 0x7f507d1 to 0xd0d0420 2024-09-04T17:46:01.329 INFO:teuthology.orchestra.run.smithi092.stdout:168 read 0x7293abe thru 0x72a2497 (0xe9da bytes) 2024-09-04T17:46:01.330 INFO:teuthology.orchestra.run.smithi092.stdout:169 read 0x43adb2e thru 0x43afd01 (0x21d4 bytes) 2024-09-04T17:46:01.330 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:01.330 INFO:teuthology.orchestra.run.smithi092.stdout:171 trunc from 0xd0d0420 to 0xccbcc42 2024-09-04T17:46:01.336 INFO:teuthology.orchestra.run.smithi092.stdout:173 read 0x36904fc thru 0x369a489 (0x9f8e bytes) 2024-09-04T17:46:01.353 INFO:teuthology.orchestra.run.smithi092.stdout:175 trunc from 0xccbcc42 to 0x4d8e2ca 2024-09-04T17:46:01.374 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:01.374+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:01.379 INFO:teuthology.orchestra.run.smithi092.stdout:176 read 0x12cacf thru 0x136173 (0x96a5 bytes) 2024-09-04T17:46:01.379 INFO:teuthology.orchestra.run.smithi092.stdout:177 write 0x3e293fb thru 0x3e2b7ed (0x23f3 bytes) 2024-09-04T17:46:01.386 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:01.386 INFO:teuthology.orchestra.run.smithi092.stdout:180 write 0xc4864c thru 0xc48b9e (0x553 bytes) 2024-09-04T17:46:01.422 INFO:teuthology.orchestra.run.smithi092.stdout:182 write 0xd3cc810 thru 0xd3d6945 (0xa136 bytes) 2024-09-04T17:46:01.430 INFO:teuthology.orchestra.run.smithi092.stdout:183 read 0x2f6f65c thru 0x2f72220 (0x2bc5 bytes) 2024-09-04T17:46:01.430 INFO:teuthology.orchestra.run.smithi092.stdout:189 punch from 0x34f1be to 0x35bbd8, (0xca1a bytes) 2024-09-04T17:46:01.453 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:01.453 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:01.455 INFO:teuthology.orchestra.run.smithi092.stdout:190 trunc from 0xd3d6946 to 0x2544886 2024-09-04T17:46:01.470 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-04T17:46:01.501 INFO:teuthology.orchestra.run.smithi092.stdout:191 write 0x3623e17 thru 0x3625c5c (0x1e46 bytes) 2024-09-04T17:46:01.501 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:01.498+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:01.517 INFO:teuthology.orchestra.run.smithi092.stdout:192 writesame 0x628cd6a thru 0x62947b1 data_size 0x68(0x7a48 bytes) 2024-09-04T17:46:01.517 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:01.517 INFO:teuthology.orchestra.run.smithi092.stdout:194 write 0x1cfc815 thru 0x1d0814d (0xb939 bytes) 2024-09-04T17:46:01.523 INFO:teuthology.orchestra.run.smithi092.stdout:195 read 0x356f763 thru 0x357f305 (0xfba3 bytes) 2024-09-04T17:46:01.523 INFO:teuthology.orchestra.run.smithi092.stdout:196 read 0x59260b8 thru 0x59273e1 (0x132a bytes) 2024-09-04T17:46:01.523 INFO:teuthology.orchestra.run.smithi092.stdout:197 write 0x5537833 thru 0x5543c1d (0xc3eb bytes) 2024-09-04T17:46:01.529 INFO:teuthology.orchestra.run.smithi092.stdout:198 punch from 0x14f46a7 to 0x14fb2f3, (0x6c4c bytes) 2024-09-04T17:46:01.529 INFO:teuthology.orchestra.run.smithi092.stdout:200 read 0x5458209 thru 0x5459c2a (0x1a22 bytes) 2024-09-04T17:46:01.530 INFO:teuthology.orchestra.run.smithi092.stdout:201 read 0x2a8fa76 thru 0x2a962f2 (0x687d bytes) 2024-09-04T17:46:01.530 INFO:teuthology.orchestra.run.smithi092.stdout:202 writesame 0xabfdf7 thru 0xacaaa3 data_size 0x23(0xacad bytes) 2024-09-04T17:46:01.538 INFO:teuthology.orchestra.run.smithi092.stdout:204 punch from 0x24a0b4c to 0x24a298b, (0x1e3f bytes) 2024-09-04T17:46:01.553 INFO:teuthology.orchestra.run.smithi092.stdout:206 writesame 0x8d1d4a8 thru 0x8d1e545 data_size 0x3(0x109e bytes) 2024-09-04T17:46:01.558 INFO:teuthology.orchestra.run.smithi092.stdout:207 trunc from 0x8d1e546 to 0xae3fc4a 2024-09-04T17:46:01.569 INFO:teuthology.orchestra.run.smithi092.stdout:208 trunc from 0xae3fc4a to 0xc2a6e7b 2024-09-04T17:46:01.576 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:01.576+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:01.578 INFO:teuthology.orchestra.run.smithi092.stdout:209 punch from 0x2fd98dd to 0x2fdd34a, (0x3a6d bytes) 2024-09-04T17:46:01.578 INFO:teuthology.orchestra.run.smithi092.stdout:210 read 0x1198f4f thru 0x11a3a45 (0xaaf7 bytes) 2024-09-04T17:46:01.578 INFO:teuthology.orchestra.run.smithi092.stdout:211 read 0x202e408 thru 0x2031377 (0x2f70 bytes) 2024-09-04T17:46:01.579 INFO:teuthology.orchestra.run.smithi092.stdout:212 punch from 0x13a2dab to 0x13ad3bf, (0xa614 bytes) 2024-09-04T17:46:01.579 INFO:teuthology.orchestra.run.smithi092.stdout:214 write 0x8aebc38 thru 0x8af06bd (0x4a86 bytes) 2024-09-04T17:46:01.585 INFO:teuthology.orchestra.run.smithi092.stdout:215 punch from 0x77a2f98 to 0x77ae095, (0xb0fd bytes) 2024-09-04T17:46:01.585 INFO:teuthology.orchestra.run.smithi092.stdout:217 write 0x6d78cf5 thru 0x6d8860a (0xf916 bytes) 2024-09-04T17:46:01.594 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:01.594 INFO:teuthology.orchestra.run.smithi092.stdout:219 punch from 0x4e4508e to 0x4e53c40, (0xebb2 bytes) 2024-09-04T17:46:01.594 INFO:teuthology.orchestra.run.smithi092.stdout:220 writesame 0xb0547ae thru 0xb060671 data_size 0x4(0xbec4 bytes) 2024-09-04T17:46:01.599 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-04T17:46:01.610 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:01.610 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:01.615 INFO:teuthology.orchestra.run.smithi092.stdout:225 trunc from 0xc2a6e7b to 0x8db1c17 2024-09-04T17:46:01.620 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:01.620 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:01.631 INFO:teuthology.orchestra.run.smithi092.stdout:226 punch from 0x3eeefae to 0x3efdaa5, (0xeaf7 bytes) 2024-09-04T17:46:01.631 INFO:teuthology.orchestra.run.smithi092.stdout:228 punch from 0x15d0fcb to 0x15db7e7, (0xa81c bytes) 2024-09-04T17:46:01.631 INFO:teuthology.orchestra.run.smithi092.stdout:229 read 0x72676c3 thru 0x726fbfc (0x853a bytes) 2024-09-04T17:46:01.631 INFO:teuthology.orchestra.run.smithi092.stdout:230 read 0x2473f5e thru 0x24757c8 (0x186b bytes) 2024-09-04T17:46:01.637 INFO:teuthology.orchestra.run.smithi092.stdout:232 trunc from 0x8db1c17 to 0xc2eadb8 2024-09-04T17:46:01.639 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-04T17:46:01.642 INFO:teuthology.orchestra.run.smithi092.stderr:Exporting 2.1 info 2.1( empty local-lis/les=19/20 n=0 ec=19/19 lis/c=19/19 les/c/f=20/20/0 sis=19) 2024-09-04T17:46:01.642 INFO:teuthology.orchestra.run.smithi092.stderr:Export successful 2024-09-04T17:46:01.644 INFO:teuthology.orchestra.run.smithi092.stdout:233 read 0x144affa thru 0x145a82e (0xf835 bytes) 2024-09-04T17:46:01.645 INFO:teuthology.orchestra.run.smithi092.stdout:234 read 0x6a8f45c thru 0x6a96f9b (0x7b40 bytes) 2024-09-04T17:46:01.645 INFO:teuthology.orchestra.run.smithi092.stdout:236 writesame 0x56fb4c thru 0x572d24 data_size 0x1(0x31d9 bytes) 2024-09-04T17:46:01.656 INFO:teuthology.orchestra.run.smithi092.stdout:237 punch from 0x98ec41a to 0x98ecd06, (0x8ec bytes) 2024-09-04T17:46:01.666 INFO:teuthology.orchestra.run.smithi092.stdout:238 trunc from 0xc2eadb8 to 0x593d5dd 2024-09-04T17:46:01.677 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:01.674+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:01.684 INFO:teuthology.orchestra.run.smithi092.stdout:239 read 0x233b1fa thru 0x234638c (0xb193 bytes) 2024-09-04T17:46:01.691 INFO:teuthology.orchestra.run.smithi092.stdout:240 trunc from 0x593d5dd to 0x17c9d93 2024-09-04T17:46:01.740 INFO:teuthology.orchestra.run.smithi092.stdout:241 trunc from 0x17c9d93 to 0x54ce166 2024-09-04T17:46:01.757 INFO:teuthology.orchestra.run.smithi092.stdout:242 write 0x76f138f thru 0x76f8295 (0x6f07 bytes) 2024-09-04T17:46:01.763 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:46:01.764 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:46:01.764 INFO:teuthology.orchestra.run.smithi092.stdout:243 punch from 0x6cfb238 to 0x6d05b05, (0xa8cd bytes) 2024-09-04T17:46:01.764 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:01.772 INFO:teuthology.orchestra.run.smithi092.stdout:245 write 0x83edd6f thru 0x83f3189 (0x541b bytes) 2024-09-04T17:46:01.778 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:01.778 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:01.778+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:01.779 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:01.779 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:46:01.779 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:46:01.779 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:46:01.779 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:01.782 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:46:01.782 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:46:01.791 INFO:teuthology.orchestra.run.smithi092.stdout:248 write 0xc6dd3a5 thru 0xc6e030e (0x2f6a bytes) 2024-09-04T17:46:01.798 INFO:teuthology.orchestra.run.smithi092.stdout:249 read 0x1ab3e04 thru 0x1abd5f8 (0x97f5 bytes) 2024-09-04T17:46:01.798 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:01.798 INFO:teuthology.orchestra.run.smithi092.stdout:252 read 0xc7c9fc thru 0xc8abe8 (0xe1ed bytes) 2024-09-04T17:46:01.802 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-04T17:46:01.808 INFO:teuthology.orchestra.run.smithi092.stdout:254 trunc from 0xc6e030f to 0x6677ae9 2024-09-04T17:46:01.809 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-04T17:46:01.809 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-04T17:46:01.810 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-04T17:46:01.825 INFO:teuthology.orchestra.run.smithi092.stdout:255 punch from 0x13230b4 to 0x132b439, (0x8385 bytes) 2024-09-04T17:46:01.825 INFO:teuthology.orchestra.run.smithi092.stdout:256 write 0x30b9a44 thru 0x30bbfc7 (0x2584 bytes) 2024-09-04T17:46:01.831 INFO:teuthology.orchestra.run.smithi092.stdout:258 read 0x45fdb32 thru 0x4600d6b (0x323a bytes) 2024-09-04T17:46:01.831 INFO:teuthology.orchestra.run.smithi092.stdout:259 write 0x109cde1 thru 0x10a3903 (0x6b23 bytes) 2024-09-04T17:46:01.833 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:46:01.833 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:46:01.842 INFO:teuthology.orchestra.run.smithi092.stdout:261 write 0x3f09f9b thru 0x3f0cb8d (0x2bf3 bytes) 2024-09-04T17:46:01.847 INFO:teuthology.orchestra.run.smithi092.stdout:263 write 0x2c09ba thru 0x2c452b (0x3b72 bytes) 2024-09-04T17:46:01.855 INFO:teuthology.orchestra.run.smithi092.stdout:265 read 0x349a968 thru 0x34a5667 (0xad00 bytes) 2024-09-04T17:46:01.856 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:46:01.856 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:46:01.868 INFO:teuthology.orchestra.run.smithi092.stdout:266 write 0xa9d6741 thru 0xa9d7f3f (0x17ff bytes) 2024-09-04T17:46:01.875 INFO:teuthology.orchestra.run.smithi092.stdout:268 trunc from 0xa9d7f40 to 0xc32b5b8 2024-09-04T17:46:01.878 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-04T17:46:01.879 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-04T17:46:01.879 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:01.878+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:01.882 INFO:teuthology.orchestra.run.smithi092.stdout:269 writesame 0x45b4d81 thru 0x45b762f data_size 0x1(0x28af bytes) 2024-09-04T17:46:01.891 INFO:teuthology.orchestra.run.smithi092.stdout:270 read 0x73f6dc4 thru 0x74003af (0x95ec bytes) 2024-09-04T17:46:01.891 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:01.891 INFO:teuthology.orchestra.run.smithi092.stdout:273 punch from 0x8910703 to 0x891d92a, (0xd227 bytes) 2024-09-04T17:46:01.891 INFO:teuthology.orchestra.run.smithi092.stdout:274 write 0x80fd1f thru 0x8113a7 (0x1689 bytes) 2024-09-04T17:46:01.898 INFO:teuthology.orchestra.run.smithi092.stdout:275 write 0x52de3e2 thru 0x52edf46 (0xfb65 bytes) 2024-09-04T17:46:01.900 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:46:01.900 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:46:01.903 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-04T17:46:01.904 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-04T17:46:01.906 INFO:teuthology.orchestra.run.smithi092.stdout:truncating to largest ever: 0xdeb1df0 2024-09-04T17:46:01.908 INFO:teuthology.orchestra.run.smithi092.stdout:276 trunc from 0xc32b5b8 to 0xdeb1df0 2024-09-04T17:46:01.916 INFO:teuthology.orchestra.run.smithi092.stdout:277 punch from 0x792ce7b to 0x7936985, (0x9b0a bytes) 2024-09-04T17:46:01.916 INFO:teuthology.orchestra.run.smithi092.stdout:278 write 0xa6ad600 thru 0xa6b2e20 (0x5821 bytes) 2024-09-04T17:46:01.924 INFO:teuthology.orchestra.run.smithi092.stdout:280 read 0x38d0d49 thru 0x38dd53b (0xc7f3 bytes) 2024-09-04T17:46:01.928 INFO:teuthology.orchestra.run.smithi092.stdout:281 trunc from 0xdeb1df0 to 0xbde9beb 2024-09-04T17:46:01.933 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:01.934 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:46:01.934 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:46:01.934 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:46:01.934 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.934 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4219.0:1 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'1, mlcod=0'0)", 2024-09-04T17:46:01.934 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:45:42.341035+0000", 2024-09-04T17:46:01.934 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.594612345000002, 2024-09-04T17:46:01.934 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0032434780000000002, 2024-09-04T17:46:01.934 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:01.934 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:01.934 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:01.934 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:01.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.935 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:01.935 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.341035+0000", 2024-09-04T17:46:01.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:01.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.935 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:01.935 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.341035+0000", 2024-09-04T17:46:01.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:01.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.936 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:01.936 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.341036+0000", 2024-09-04T17:46:01.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0299999999999999e-07 2024-09-04T17:46:01.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.936 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:01.936 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.341044+0000", 2024-09-04T17:46:01.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.157e-06 2024-09-04T17:46:01.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.936 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:01.936 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.341069+0000", 2024-09-04T17:46:01.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4799000000000001e-05 2024-09-04T17:46:01.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.937 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:01.937 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.341084+0000", 2024-09-04T17:46:01.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5557e-05 2024-09-04T17:46:01.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.937 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:01.937 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.341209+0000", 2024-09-04T17:46:01.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012469900000000001 2024-09-04T17:46:01.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.938 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.938 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:01.938 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.341260+0000", 2024-09-04T17:46:01.938 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0797000000000002e-05 2024-09-04T17:46:01.938 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.938 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.938 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:01.938 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.344184+0000", 2024-09-04T17:46:01.938 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029241800000000002 2024-09-04T17:46:01.938 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.938 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.938 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:01.938 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.344279+0000", 2024-09-04T17:46:01.939 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4785999999999997e-05 2024-09-04T17:46:01.939 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:01.939 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:01.939 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:01.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.939 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4219.0:20 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'10, mlcod=17'8)", 2024-09-04T17:46:01.939 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:45:42.383069+0000", 2024-09-04T17:46:01.939 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.5525789, 2024-09-04T17:46:01.939 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028321119999999999, 2024-09-04T17:46:01.939 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:01.939 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:01.940 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:01.940 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:01.940 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.940 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:01.940 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.383069+0000", 2024-09-04T17:46:01.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:01.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.940 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.940 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:01.940 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.383069+0000", 2024-09-04T17:46:01.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:01.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.940 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:01.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.383071+0000", 2024-09-04T17:46:01.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9999999999999999e-06 2024-09-04T17:46:01.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:01.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.383361+0000", 2024-09-04T17:46:01.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00029085800000000001 2024-09-04T17:46:01.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:01.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.383390+0000", 2024-09-04T17:46:01.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8989999999999999e-05 2024-09-04T17:46:01.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.942 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.942 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:01.942 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.383404+0000", 2024-09-04T17:46:01.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3499999999999999e-05 2024-09-04T17:46:01.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.942 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.942 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:01.942 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.383531+0000", 2024-09-04T17:46:01.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012723199999999999 2024-09-04T17:46:01.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.942 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.943 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:01.943 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.383566+0000", 2024-09-04T17:46:01.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5089999999999998e-05 2024-09-04T17:46:01.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.943 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:01.943 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.385816+0000", 2024-09-04T17:46:01.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0022497559999999999 2024-09-04T17:46:01.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.943 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:01.943 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.385901+0000", 2024-09-04T17:46:01.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4685999999999995e-05 2024-09-04T17:46:01.944 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:01.944 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:01.944 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:01.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.944 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4219.0:40 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'22, mlcod=17'19)", 2024-09-04T17:46:01.944 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:45:42.488420+0000", 2024-09-04T17:46:01.944 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.447227742999999, 2024-09-04T17:46:01.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0036461570000000001, 2024-09-04T17:46:01.944 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:01.944 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:01.944 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:01.945 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:01.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:01.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.488420+0000", 2024-09-04T17:46:01.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:01.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:01.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.488420+0000", 2024-09-04T17:46:01.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:01.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:01.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.488421+0000", 2024-09-04T17:46:01.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8400000000000002e-07 2024-09-04T17:46:01.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:01.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.488464+0000", 2024-09-04T17:46:01.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3699000000000003e-05 2024-09-04T17:46:01.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:01.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.488493+0000", 2024-09-04T17:46:01.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8229999999999999e-05 2024-09-04T17:46:01.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:01.947 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.488512+0000", 2024-09-04T17:46:01.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9570000000000001e-05 2024-09-04T17:46:01.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:01.947 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.488646+0000", 2024-09-04T17:46:01.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013357899999999999 2024-09-04T17:46:01.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:01.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.488685+0000", 2024-09-04T17:46:01.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8840000000000001e-05 2024-09-04T17:46:01.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:01.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.492019+0000", 2024-09-04T17:46:01.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0033346539999999998 2024-09-04T17:46:01.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:01.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.492066+0000", 2024-09-04T17:46:01.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6601000000000003e-05 2024-09-04T17:46:01.948 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:01.948 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:01.948 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:01.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.949 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4219.0:52 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'34, mlcod=17'30)", 2024-09-04T17:46:01.949 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:45:42.508805+0000", 2024-09-04T17:46:01.949 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.426842605000001, 2024-09-04T17:46:01.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0038787629999999999, 2024-09-04T17:46:01.950 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:01.950 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:01.950 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:01.950 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:01.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:01.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.508805+0000", 2024-09-04T17:46:01.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:01.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:01.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.508805+0000", 2024-09-04T17:46:01.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:01.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:01.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.508805+0000", 2024-09-04T17:46:01.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9999999999999998e-07 2024-09-04T17:46:01.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:01.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.508809+0000", 2024-09-04T17:46:01.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9750000000000001e-06 2024-09-04T17:46:01.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:01.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.508819+0000", 2024-09-04T17:46:01.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1870000000000003e-06 2024-09-04T17:46:01.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:01.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.508824+0000", 2024-09-04T17:46:01.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0989999999999998e-06 2024-09-04T17:46:01.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:01.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.508872+0000", 2024-09-04T17:46:01.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8253e-05 2024-09-04T17:46:01.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:01.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.508887+0000", 2024-09-04T17:46:01.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4793999999999999e-05 2024-09-04T17:46:01.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:01.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.512627+0000", 2024-09-04T17:46:01.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0037407859999999999 2024-09-04T17:46:01.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.954 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:01.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.512684+0000", 2024-09-04T17:46:01.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6168999999999997e-05 2024-09-04T17:46:01.954 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:01.954 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:01.954 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:01.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.954 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4219.0:64 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'42, mlcod=17'40)", 2024-09-04T17:46:01.954 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:45:42.733254+0000", 2024-09-04T17:46:01.954 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.202393022999999, 2024-09-04T17:46:01.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0060170980000000002, 2024-09-04T17:46:01.954 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:01.955 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:01.955 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:01.955 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:01.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:01.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.733254+0000", 2024-09-04T17:46:01.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:01.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:01.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.733254+0000", 2024-09-04T17:46:01.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:01.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.956 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:01.956 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.733256+0000", 2024-09-04T17:46:01.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.95e-06 2024-09-04T17:46:01.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.956 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:01.956 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.733268+0000", 2024-09-04T17:46:01.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2054e-05 2024-09-04T17:46:01.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.957 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:01.957 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.733297+0000", 2024-09-04T17:46:01.957 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8033000000000001e-05 2024-09-04T17:46:01.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.957 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:01.957 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.733312+0000", 2024-09-04T17:46:01.957 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5682999999999999e-05 2024-09-04T17:46:01.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.957 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:01.957 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.733458+0000", 2024-09-04T17:46:01.957 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014544199999999999 2024-09-04T17:46:01.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:01.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.733495+0000", 2024-09-04T17:46:01.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7089999999999999e-05 2024-09-04T17:46:01.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:01.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.739177+0000", 2024-09-04T17:46:01.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0056820070000000002 2024-09-04T17:46:01.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:01.959 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.739272+0000", 2024-09-04T17:46:01.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4839000000000005e-05 2024-09-04T17:46:01.959 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:01.959 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:01.959 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:01.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.959 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4219.0:65 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'43, mlcod=17'40)", 2024-09-04T17:46:01.959 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:45:42.733844+0000", 2024-09-04T17:46:01.959 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.201803950999999, 2024-09-04T17:46:01.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054788689999999999, 2024-09-04T17:46:01.959 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:01.959 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:01.960 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:01.960 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:01.960 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.960 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:01.960 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.733844+0000", 2024-09-04T17:46:01.960 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:01.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.960 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.960 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:01.960 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.733844+0000", 2024-09-04T17:46:01.960 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:01.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:01.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.733845+0000", 2024-09-04T17:46:01.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.844e-06 2024-09-04T17:46:01.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:01.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.733857+0000", 2024-09-04T17:46:01.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2069000000000001e-05 2024-09-04T17:46:01.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:01.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.733882+0000", 2024-09-04T17:46:01.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.438e-05 2024-09-04T17:46:01.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:01.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.733901+0000", 2024-09-04T17:46:01.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9564000000000001e-05 2024-09-04T17:46:01.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:01.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.734434+0000", 2024-09-04T17:46:01.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00053266200000000004 2024-09-04T17:46:01.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:01.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.734494+0000", 2024-09-04T17:46:01.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9939999999999999e-05 2024-09-04T17:46:01.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:01.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.739277+0000", 2024-09-04T17:46:01.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047834729999999999 2024-09-04T17:46:01.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:01.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.739322+0000", 2024-09-04T17:46:01.964 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4937e-05 2024-09-04T17:46:01.964 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:01.964 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:01.964 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:01.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.964 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.964 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4219.0:66 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'44, mlcod=17'40)", 2024-09-04T17:46:01.964 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:45:42.734547+0000", 2024-09-04T17:46:01.964 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.201100016000002, 2024-09-04T17:46:01.964 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0060660940000000002, 2024-09-04T17:46:01.964 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:01.964 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:01.964 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:01.965 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:01.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.965 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:01.965 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.734547+0000", 2024-09-04T17:46:01.965 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:01.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.965 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:01.965 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.734547+0000", 2024-09-04T17:46:01.965 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:01.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.965 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:01.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.734549+0000", 2024-09-04T17:46:01.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6700000000000001e-06 2024-09-04T17:46:01.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:01.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.734601+0000", 2024-09-04T17:46:01.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1663000000000003e-05 2024-09-04T17:46:01.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:01.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.734623+0000", 2024-09-04T17:46:01.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2215999999999999e-05 2024-09-04T17:46:01.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:01.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.734634+0000", 2024-09-04T17:46:01.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1209999999999999e-05 2024-09-04T17:46:01.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:01.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.735556+0000", 2024-09-04T17:46:01.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00092180699999999996 2024-09-04T17:46:01.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.968 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:01.968 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.735616+0000", 2024-09-04T17:46:01.968 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9669000000000001e-05 2024-09-04T17:46:01.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.968 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:01.968 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.740541+0000", 2024-09-04T17:46:01.968 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0049256259999999998 2024-09-04T17:46:01.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.968 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:01.969 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.740614+0000", 2024-09-04T17:46:01.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2232999999999998e-05 2024-09-04T17:46:01.969 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:01.969 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:01.969 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:01.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.969 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4219.0:67 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'45, mlcod=17'40)", 2024-09-04T17:46:01.969 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:45:42.735489+0000", 2024-09-04T17:46:01.969 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.200158593000001, 2024-09-04T17:46:01.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051833030000000002, 2024-09-04T17:46:01.970 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:01.970 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:01.970 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:01.970 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:01.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.970 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:01.970 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.735489+0000", 2024-09-04T17:46:01.970 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:01.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.970 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:01.970 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.735489+0000", 2024-09-04T17:46:01.970 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:01.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.971 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:01.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.735490+0000", 2024-09-04T17:46:01.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5200000000000001e-06 2024-09-04T17:46:01.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.971 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:01.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.735502+0000", 2024-09-04T17:46:01.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1922999999999999e-05 2024-09-04T17:46:01.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.971 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:01.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.735519+0000", 2024-09-04T17:46:01.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6736999999999999e-05 2024-09-04T17:46:01.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:01.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.735530+0000", 2024-09-04T17:46:01.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0873000000000001e-05 2024-09-04T17:46:01.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:01.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.737774+0000", 2024-09-04T17:46:01.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0022443900000000002 2024-09-04T17:46:01.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:01.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.737814+0000", 2024-09-04T17:46:01.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9412000000000001e-05 2024-09-04T17:46:01.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.973 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.973 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:01.973 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.740619+0000", 2024-09-04T17:46:01.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002805316 2024-09-04T17:46:01.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.973 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.973 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:01.973 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.740672+0000", 2024-09-04T17:46:01.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3131999999999999e-05 2024-09-04T17:46:01.973 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:01.973 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:01.973 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:01.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.974 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4219.0:68 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'46, mlcod=17'40)", 2024-09-04T17:46:01.974 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:45:42.736110+0000", 2024-09-04T17:46:01.974 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.199537200000002, 2024-09-04T17:46:01.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059532159999999999, 2024-09-04T17:46:01.974 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:01.974 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:01.974 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:01.974 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:01.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.974 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:01.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.736110+0000", 2024-09-04T17:46:01.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:01.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:01.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.736110+0000", 2024-09-04T17:46:01.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:01.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:01.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.736112+0000", 2024-09-04T17:46:01.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.251e-06 2024-09-04T17:46:01.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:01.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.736119+0000", 2024-09-04T17:46:01.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9770000000000003e-06 2024-09-04T17:46:01.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:01.976 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.736141+0000", 2024-09-04T17:46:01.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2172999999999999e-05 2024-09-04T17:46:01.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:01.976 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.736150+0000", 2024-09-04T17:46:01.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2459999999999992e-06 2024-09-04T17:46:01.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.977 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:01.977 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.738590+0000", 2024-09-04T17:46:01.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024402460000000001 2024-09-04T17:46:01.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.977 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:01.977 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.738623+0000", 2024-09-04T17:46:01.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3037000000000002e-05 2024-09-04T17:46:01.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.977 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:01.977 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.741995+0000", 2024-09-04T17:46:01.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003371364 2024-09-04T17:46:01.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.978 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.978 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:01.978 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.742064+0000", 2024-09-04T17:46:01.978 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8922000000000002e-05 2024-09-04T17:46:01.978 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:01.978 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:01.978 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:01.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.978 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.978 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4219.0:69 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'47, mlcod=17'40)", 2024-09-04T17:46:01.978 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:45:42.736825+0000", 2024-09-04T17:46:01.978 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.198822607, 2024-09-04T17:46:01.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0072074649999999997, 2024-09-04T17:46:01.979 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:01.979 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:01.979 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:01.979 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:01.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:01.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.736825+0000", 2024-09-04T17:46:01.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:01.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:01.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.736825+0000", 2024-09-04T17:46:01.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:01.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:01.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.736826+0000", 2024-09-04T17:46:01.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.164e-06 2024-09-04T17:46:01.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:01.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.736864+0000", 2024-09-04T17:46:01.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7817000000000001e-05 2024-09-04T17:46:01.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:01.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.736881+0000", 2024-09-04T17:46:01.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7105999999999999e-05 2024-09-04T17:46:01.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:01.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.736892+0000", 2024-09-04T17:46:01.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1127e-05 2024-09-04T17:46:01.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:01.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.739397+0000", 2024-09-04T17:46:01.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025048029999999999 2024-09-04T17:46:01.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.982 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.982 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:01.982 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.739447+0000", 2024-09-04T17:46:01.982 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.994e-05 2024-09-04T17:46:01.982 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.982 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.982 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:01.982 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.743960+0000", 2024-09-04T17:46:01.982 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045136450000000002 2024-09-04T17:46:01.982 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.982 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.982 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:01.982 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.744032+0000", 2024-09-04T17:46:01.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1863e-05 2024-09-04T17:46:01.983 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:01.983 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:01.983 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:01.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.983 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4219.0:70 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'48, mlcod=17'40)", 2024-09-04T17:46:01.983 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:45:42.737775+0000", 2024-09-04T17:46:01.983 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.19787243, 2024-09-04T17:46:01.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0081053870000000004, 2024-09-04T17:46:01.983 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:01.983 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:01.983 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:01.983 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:01.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:01.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.737775+0000", 2024-09-04T17:46:01.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:01.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:01.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.737775+0000", 2024-09-04T17:46:01.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:01.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:01.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.737777+0000", 2024-09-04T17:46:01.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.733e-06 2024-09-04T17:46:01.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:01.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.737789+0000", 2024-09-04T17:46:01.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2057e-05 2024-09-04T17:46:01.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:01.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.737817+0000", 2024-09-04T17:46:01.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7801999999999999e-05 2024-09-04T17:46:01.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.986 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:01.986 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.737837+0000", 2024-09-04T17:46:01.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9904000000000001e-05 2024-09-04T17:46:01.986 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.986 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.986 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:01.986 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.740774+0000", 2024-09-04T17:46:01.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029378899999999999 2024-09-04T17:46:01.986 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.986 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.986 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:01.986 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.740824+0000", 2024-09-04T17:46:01.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9963000000000003e-05 2024-09-04T17:46:01.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.987 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.987 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:01.987 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.745795+0000", 2024-09-04T17:46:01.987 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004970515 2024-09-04T17:46:01.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.987 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.987 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:01.987 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.745880+0000", 2024-09-04T17:46:01.987 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5523000000000001e-05 2024-09-04T17:46:01.987 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:01.987 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:01.987 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:01.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.988 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4219.0:71 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'49, mlcod=17'40)", 2024-09-04T17:46:01.988 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:45:42.738579+0000", 2024-09-04T17:46:01.988 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.197068367, 2024-09-04T17:46:01.988 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0073648999999999997, 2024-09-04T17:46:01.988 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:01.988 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:01.988 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:01.988 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:01.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.988 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:01.988 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.738579+0000", 2024-09-04T17:46:01.988 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:01.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.989 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.989 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:01.989 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.738579+0000", 2024-09-04T17:46:01.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:01.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.989 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.989 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:01.989 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.738581+0000", 2024-09-04T17:46:01.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.967e-06 2024-09-04T17:46:01.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.989 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.989 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:01.990 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.738593+0000", 2024-09-04T17:46:01.990 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1866e-05 2024-09-04T17:46:01.990 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.990 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.990 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:01.990 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.738615+0000", 2024-09-04T17:46:01.990 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1963999999999998e-05 2024-09-04T17:46:01.990 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.990 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.990 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:01.990 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.738631+0000", 2024-09-04T17:46:01.990 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5800000000000001e-05 2024-09-04T17:46:01.990 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.991 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.991 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:01.991 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.741437+0000", 2024-09-04T17:46:01.991 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028058240000000002 2024-09-04T17:46:01.991 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.991 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.991 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:01.991 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.741477+0000", 2024-09-04T17:46:01.991 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0856999999999999e-05 2024-09-04T17:46:01.991 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.991 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.991 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:01.991 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.745886+0000", 2024-09-04T17:46:01.991 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004409039 2024-09-04T17:46:01.991 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.992 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:01.992 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.745944+0000", 2024-09-04T17:46:01.992 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7583000000000001e-05 2024-09-04T17:46:01.992 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:01.992 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:01.992 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:01.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.992 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4219.0:72 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'50, mlcod=17'40)", 2024-09-04T17:46:01.992 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:45:42.739461+0000", 2024-09-04T17:46:01.992 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.196186279999999, 2024-09-04T17:46:01.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0065627890000000003, 2024-09-04T17:46:01.993 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:01.993 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:01.993 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:01.993 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:01.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:01.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.739461+0000", 2024-09-04T17:46:01.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:01.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:01.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.739461+0000", 2024-09-04T17:46:01.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:01.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:01.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.739463+0000", 2024-09-04T17:46:01.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0999999999999998e-06 2024-09-04T17:46:01.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:01.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.739641+0000", 2024-09-04T17:46:01.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000177765 2024-09-04T17:46:01.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.995 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:01.995 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.739667+0000", 2024-09-04T17:46:01.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5452999999999998e-05 2024-09-04T17:46:01.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.995 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:01.995 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.739683+0000", 2024-09-04T17:46:01.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6787e-05 2024-09-04T17:46:01.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.995 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:01.995 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.742156+0000", 2024-09-04T17:46:01.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024726269999999998 2024-09-04T17:46:01.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.996 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:01.996 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.742225+0000", 2024-09-04T17:46:01.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8899999999999994e-05 2024-09-04T17:46:01.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.996 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:01.996 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.745954+0000", 2024-09-04T17:46:01.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0037289710000000002 2024-09-04T17:46:01.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:01.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.746024+0000", 2024-09-04T17:46:01.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0185999999999995e-05 2024-09-04T17:46:01.997 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:01.997 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:01.997 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:01.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.997 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4219.0:73 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'51, mlcod=17'40)", 2024-09-04T17:46:01.997 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:45:42.740426+0000", 2024-09-04T17:46:01.997 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.195221184000001, 2024-09-04T17:46:01.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0056659659999999997, 2024-09-04T17:46:01.998 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:01.998 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:01.998 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:01.998 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:01.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:01.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.740426+0000", 2024-09-04T17:46:01.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:01.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:01.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.740426+0000", 2024-09-04T17:46:01.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:01.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:01.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.740428+0000", 2024-09-04T17:46:01.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4160000000000001e-06 2024-09-04T17:46:01.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:01.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:01.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.740436+0000", 2024-09-04T17:46:01.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0539999999999995e-06 2024-09-04T17:46:01.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:01.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:02.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.740455+0000", 2024-09-04T17:46:02.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9009999999999999e-05 2024-09-04T17:46:02.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:02.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.740469+0000", 2024-09-04T17:46:02.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4686e-05 2024-09-04T17:46:02.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:02.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.743374+0000", 2024-09-04T17:46:02.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029045410000000001 2024-09-04T17:46:02.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:02.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.743420+0000", 2024-09-04T17:46:02.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6372999999999998e-05 2024-09-04T17:46:02.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:02.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.746029+0000", 2024-09-04T17:46:02.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0026082829999999999 2024-09-04T17:46:02.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:02.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.746092+0000", 2024-09-04T17:46:02.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3602999999999994e-05 2024-09-04T17:46:02.002 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:02.002 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:02.002 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:02.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.002 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4219.0:74 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'52, mlcod=17'44)", 2024-09-04T17:46:02.002 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:45:42.741798+0000", 2024-09-04T17:46:02.002 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.193849105000002, 2024-09-04T17:46:02.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0043572130000000004, 2024-09-04T17:46:02.002 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:02.003 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:02.003 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:02.003 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:02.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:02.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.741798+0000", 2024-09-04T17:46:02.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:02.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:02.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.741798+0000", 2024-09-04T17:46:02.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:02.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:02.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.741800+0000", 2024-09-04T17:46:02.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7600000000000001e-06 2024-09-04T17:46:02.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:02.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.741811+0000", 2024-09-04T17:46:02.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0616e-05 2024-09-04T17:46:02.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:02.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.741837+0000", 2024-09-04T17:46:02.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6356999999999999e-05 2024-09-04T17:46:02.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:02.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.741852+0000", 2024-09-04T17:46:02.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4800000000000001e-05 2024-09-04T17:46:02.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:02.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.744113+0000", 2024-09-04T17:46:02.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0022609489999999999 2024-09-04T17:46:02.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:02.006 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.744169+0000", 2024-09-04T17:46:02.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.647e-05 2024-09-04T17:46:02.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.006 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:02.006 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.746096+0000", 2024-09-04T17:46:02.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019269809999999999 2024-09-04T17:46:02.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.006 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:02.006 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.746156+0000", 2024-09-04T17:46:02.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9280000000000002e-05 2024-09-04T17:46:02.006 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:02.007 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:02.007 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:02.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.007 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4219.0:75 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'53, mlcod=17'44)", 2024-09-04T17:46:02.007 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:45:42.742428+0000", 2024-09-04T17:46:02.007 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.193219023000001, 2024-09-04T17:46:02.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0044071520000000001, 2024-09-04T17:46:02.007 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:02.007 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:02.007 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:02.007 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:02.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:02.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.742428+0000", 2024-09-04T17:46:02.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:02.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:02.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.742428+0000", 2024-09-04T17:46:02.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:02.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:02.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.742430+0000", 2024-09-04T17:46:02.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.513e-06 2024-09-04T17:46:02.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:02.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.742580+0000", 2024-09-04T17:46:02.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014970200000000001 2024-09-04T17:46:02.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:02.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.742607+0000", 2024-09-04T17:46:02.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7650000000000001e-05 2024-09-04T17:46:02.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:02.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.742622+0000", 2024-09-04T17:46:02.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5126999999999999e-05 2024-09-04T17:46:02.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:02.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.744700+0000", 2024-09-04T17:46:02.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020779069999999999 2024-09-04T17:46:02.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:02.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.744731+0000", 2024-09-04T17:46:02.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0433000000000002e-05 2024-09-04T17:46:02.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:02.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.746772+0000", 2024-09-04T17:46:02.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020407440000000002 2024-09-04T17:46:02.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:02.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.746836+0000", 2024-09-04T17:46:02.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4076000000000001e-05 2024-09-04T17:46:02.011 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:02.011 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:02.011 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:02.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.011 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4219.0:76 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'54, mlcod=17'44)", 2024-09-04T17:46:02.011 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:45:42.743459+0000", 2024-09-04T17:46:02.012 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.192188347999998, 2024-09-04T17:46:02.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040473490000000004, 2024-09-04T17:46:02.012 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:02.012 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:02.012 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:02.012 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:02.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:02.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.743459+0000", 2024-09-04T17:46:02.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:02.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:02.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.743459+0000", 2024-09-04T17:46:02.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:02.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:02.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.743461+0000", 2024-09-04T17:46:02.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.637e-06 2024-09-04T17:46:02.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:02.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.743473+0000", 2024-09-04T17:46:02.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2529e-05 2024-09-04T17:46:02.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:02.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.743494+0000", 2024-09-04T17:46:02.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1036999999999999e-05 2024-09-04T17:46:02.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:02.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.743510+0000", 2024-09-04T17:46:02.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5590000000000002e-05 2024-09-04T17:46:02.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:02.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.746236+0000", 2024-09-04T17:46:02.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002725866 2024-09-04T17:46:02.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:02.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.746267+0000", 2024-09-04T17:46:02.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0775999999999998e-05 2024-09-04T17:46:02.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:02.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.747443+0000", 2024-09-04T17:46:02.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011767069999999999 2024-09-04T17:46:02.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:02.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.747506+0000", 2024-09-04T17:46:02.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3207000000000007e-05 2024-09-04T17:46:02.016 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:02.016 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:02.016 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:02.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.016 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4219.0:78 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'56, mlcod=17'53)", 2024-09-04T17:46:02.016 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:45:42.749440+0000", 2024-09-04T17:46:02.016 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.186207683999999, 2024-09-04T17:46:02.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0038032610000000001, 2024-09-04T17:46:02.016 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:02.016 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:02.016 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:02.017 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:02.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:02.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.749440+0000", 2024-09-04T17:46:02.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:02.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:02.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.749440+0000", 2024-09-04T17:46:02.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:02.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:02.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.749441+0000", 2024-09-04T17:46:02.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.37e-06 2024-09-04T17:46:02.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:02.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.749579+0000", 2024-09-04T17:46:02.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013783900000000001 2024-09-04T17:46:02.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:02.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.749607+0000", 2024-09-04T17:46:02.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8045999999999999e-05 2024-09-04T17:46:02.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:02.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.749622+0000", 2024-09-04T17:46:02.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4843e-05 2024-09-04T17:46:02.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:02.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.749834+0000", 2024-09-04T17:46:02.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021182899999999999 2024-09-04T17:46:02.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:02.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.749882+0000", 2024-09-04T17:46:02.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7926000000000002e-05 2024-09-04T17:46:02.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:02.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.753166+0000", 2024-09-04T17:46:02.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0032841250000000002 2024-09-04T17:46:02.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:02.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.753243+0000", 2024-09-04T17:46:02.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7282999999999999e-05 2024-09-04T17:46:02.020 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:02.021 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:02.021 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:02.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.021 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4219.0:79 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'57, mlcod=17'53)", 2024-09-04T17:46:02.021 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:45:42.750198+0000", 2024-09-04T17:46:02.021 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.185449617, 2024-09-04T17:46:02.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0038080940000000001, 2024-09-04T17:46:02.021 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:02.021 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:02.021 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:02.021 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:02.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:02.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.750198+0000", 2024-09-04T17:46:02.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:02.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:02.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.750198+0000", 2024-09-04T17:46:02.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:02.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:02.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.750200+0000", 2024-09-04T17:46:02.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.967e-06 2024-09-04T17:46:02.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:02.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.750392+0000", 2024-09-04T17:46:02.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000192222 2024-09-04T17:46:02.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:02.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.750417+0000", 2024-09-04T17:46:02.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5329000000000001e-05 2024-09-04T17:46:02.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:02.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.750432+0000", 2024-09-04T17:46:02.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4847e-05 2024-09-04T17:46:02.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:02.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.750693+0000", 2024-09-04T17:46:02.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026076400000000003 2024-09-04T17:46:02.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:02.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.750745+0000", 2024-09-04T17:46:02.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1783999999999998e-05 2024-09-04T17:46:02.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:02.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.753918+0000", 2024-09-04T17:46:02.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031730019999999999 2024-09-04T17:46:02.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:02.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.754006+0000", 2024-09-04T17:46:02.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8178999999999997e-05 2024-09-04T17:46:02.025 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:02.025 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:02.025 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:02.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.026 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4219.0:80 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'58, mlcod=17'53)", 2024-09-04T17:46:02.026 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:45:42.750934+0000", 2024-09-04T17:46:02.026 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.184713759000001, 2024-09-04T17:46:02.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031122400000000001, 2024-09-04T17:46:02.026 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:02.026 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:02.026 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:02.026 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:02.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:02.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.750934+0000", 2024-09-04T17:46:02.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:02.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:02.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.750934+0000", 2024-09-04T17:46:02.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:02.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:02.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.750935+0000", 2024-09-04T17:46:02.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2729999999999999e-06 2024-09-04T17:46:02.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:02.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.751024+0000", 2024-09-04T17:46:02.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.852e-05 2024-09-04T17:46:02.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:02.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.751047+0000", 2024-09-04T17:46:02.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3730000000000001e-05 2024-09-04T17:46:02.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:02.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.751065+0000", 2024-09-04T17:46:02.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7518999999999999e-05 2024-09-04T17:46:02.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:02.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.751419+0000", 2024-09-04T17:46:02.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000353994 2024-09-04T17:46:02.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:02.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.751451+0000", 2024-09-04T17:46:02.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2530000000000002e-05 2024-09-04T17:46:02.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:02.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.754010+0000", 2024-09-04T17:46:02.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025591870000000001 2024-09-04T17:46:02.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:02.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:42.754046+0000", 2024-09-04T17:46:02.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5487000000000001e-05 2024-09-04T17:46:02.030 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:02.030 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:02.030 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:02.030 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:02.030 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:02.030 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:02.031 INFO:teuthology.orchestra.run.smithi092.stdout:282 writesame 0xafb9950 thru 0xafc3597 data_size 0xc(0x9c48 bytes) 2024-09-04T17:46:02.032 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-04T17:46:02.033 INFO:teuthology.orchestra.run.smithi092.stdout:283 read 0x3dea5e3 thru 0x3dee1a3 (0x3bc1 bytes) 2024-09-04T17:46:02.033 INFO:teuthology.orchestra.run.smithi092.stdout:285 trunc from 0xbde9beb to 0xd2d64dd 2024-09-04T17:46:02.033 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:02.034 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-3 --journal-path /var/lib/ceph/osd/ceph-3/journal --force --op remove --pgid 2.1 2024-09-04T17:46:02.034 INFO:teuthology.orchestra.run.smithi092.stdout:288 trunc from 0xd2d64dd to 0x5af648a 2024-09-04T17:46:02.034 INFO:teuthology.orchestra.run.smithi092.stdout:289 write 0x2e37736 thru 0x2e38a5b (0x1326 bytes) 2024-09-04T17:46:02.034 INFO:teuthology.orchestra.run.smithi092.stdout:290 write 0xbd8397a thru 0xbd9253c (0xebc3 bytes) 2024-09-04T17:46:02.035 INFO:teuthology.orchestra.run.smithi092.stdout:294 write 0x2ec766b thru 0x2ed369c (0xc032 bytes) 2024-09-04T17:46:02.035 INFO:teuthology.orchestra.run.smithi092.stdout:295 write 0x9b39798 thru 0x9b451de (0xba47 bytes) 2024-09-04T17:46:02.035 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:02.030+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:02.036 INFO:teuthology.orchestra.run.smithi092.stdout:297 write 0x8b3fdde thru 0x8b41247 (0x146a bytes) 2024-09-04T17:46:02.044 INFO:teuthology.orchestra.run.smithi092.stdout:298 read 0x806a666 thru 0x806fe49 (0x57e4 bytes) 2024-09-04T17:46:02.044 INFO:teuthology.orchestra.run.smithi092.stdout:299 read 0x1c38941 thru 0x1c39d90 (0x1450 bytes) 2024-09-04T17:46:02.052 INFO:teuthology.orchestra.run.smithi092.stdout:300 trunc from 0xbd9253d to 0x54a545f 2024-09-04T17:46:02.072 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:02.072 INFO:teuthology.orchestra.run.smithi092.stdout:302 write 0x199f691 thru 0x19a38ea (0x425a bytes) 2024-09-04T17:46:02.081 INFO:teuthology.orchestra.run.smithi092.stdout:303 write 0xdd7b57 thru 0xde6172 (0xe61c bytes) 2024-09-04T17:46:02.093 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:02.093 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:02.094 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:02.094 INFO:teuthology.orchestra.run.smithi092.stdout:310 writesame 0x36eedac thru 0x36f8f41 data_size 0x2b(0xa196 bytes) 2024-09-04T17:46:02.099 INFO:teuthology.orchestra.run.smithi092.stdout:311 write 0x1602cb thru 0x16fb00 (0xf836 bytes) 2024-09-04T17:46:02.106 INFO:teuthology.orchestra.run.smithi092.stdout:312 read 0x50bb318 thru 0x50c5f1c (0xac05 bytes) 2024-09-04T17:46:02.106 INFO:teuthology.orchestra.run.smithi092.stdout:313 read 0x24fc981 thru 0x25098be (0xcf3e bytes) 2024-09-04T17:46:02.107 INFO:teuthology.orchestra.run.smithi092.stdout:314 punch from 0x29a3cc0 to 0x29a8a75, (0x4db5 bytes) 2024-09-04T17:46:02.107 INFO:teuthology.orchestra.run.smithi092.stdout:315 read 0x2bd5751 thru 0x2be4ede (0xf78e bytes) 2024-09-04T17:46:02.110 INFO:teuthology.orchestra.run.smithi092.stdout:316 trunc from 0x54a545f to 0x8b8bc76 2024-09-04T17:46:02.118 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:02.118 INFO:teuthology.orchestra.run.smithi092.stdout:319 write 0x31dd5c6 thru 0x31e4165 (0x6ba0 bytes) 2024-09-04T17:46:02.126 INFO:teuthology.orchestra.run.smithi092.stdout:320 writesame 0x34ad420 thru 0x34b283b data_size 0x7(0x541c bytes) 2024-09-04T17:46:02.132 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:02.130+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:02.134 INFO:teuthology.orchestra.run.smithi092.stdout:321 trunc from 0x8b8bc76 to 0x8ce850d 2024-09-04T17:46:02.142 INFO:teuthology.orchestra.run.smithi092.stdout:322 read 0x7a754bb thru 0x7a82d3f (0xd885 bytes) 2024-09-04T17:46:02.142 INFO:teuthology.orchestra.run.smithi092.stdout:323 write 0x86c771 thru 0x87221d (0x5aad bytes) 2024-09-04T17:46:02.148 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:02.158 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:02.158 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:46:02.158 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:46:02.158 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:02.159 INFO:teuthology.orchestra.run.smithi092.stdout:325 write 0x9955be6 thru 0x995ed34 (0x914f bytes) 2024-09-04T17:46:02.164 INFO:teuthology.orchestra.run.smithi092.stdout:326 write 0x760038c thru 0x760bb87 (0xb7fc bytes) 2024-09-04T17:46:02.171 INFO:teuthology.orchestra.run.smithi092.stdout:327 write 0xf18aad thru 0xf1b351 (0x28a5 bytes) 2024-09-04T17:46:02.172 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-04T17:46:02.179 INFO:teuthology.orchestra.run.smithi092.stdout:328 read 0x8bcfa4e thru 0x8bd2f3c (0x34ef bytes) 2024-09-04T17:46:02.179 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:02.179 INFO:teuthology.orchestra.run.smithi092.stdout:330 punch from 0x9421818 to 0x9427388, (0x5b70 bytes) 2024-09-04T17:46:02.179 INFO:teuthology.orchestra.run.smithi092.stdout:331 read 0x177abe8 thru 0x178971d (0xeb36 bytes) 2024-09-04T17:46:02.194 INFO:teuthology.orchestra.run.smithi092.stdout:332 writesame 0xcb34166 thru 0xcb3568f data_size 0x15(0x152a bytes) 2024-09-04T17:46:02.194 INFO:teuthology.orchestra.run.smithi092.stdout:333 read 0x13a77ee thru 0x13ae873 (0x7086 bytes) 2024-09-04T17:46:02.196 INFO:teuthology.orchestra.run.smithi092.stdout:334 trunc from 0xcb35690 to 0xde97c38 2024-09-04T17:46:02.204 INFO:teuthology.orchestra.run.smithi092.stdout:335 read 0xb64f99b thru 0xb658b42 (0x91a8 bytes) 2024-09-04T17:46:02.204 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:02.204 INFO:teuthology.orchestra.run.smithi092.stdout:338 write 0xd922c09 thru 0xd92cb52 (0x9f4a bytes) 2024-09-04T17:46:02.210 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:02.210 INFO:teuthology.orchestra.run.smithi092.stdout:340 read 0x19eac06 thru 0x19f80ed (0xd4e8 bytes) 2024-09-04T17:46:02.210 INFO:teuthology.orchestra.run.smithi092.stdout:341 write 0x9d43761 thru 0x9d496c4 (0x5f64 bytes) 2024-09-04T17:46:02.216 INFO:teuthology.orchestra.run.smithi092.stdout:342 writesame 0x5ee229c thru 0x5ee6382 data_size 0x1(0x40e7 bytes) 2024-09-04T17:46:02.232 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:02.232+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:02.242 INFO:teuthology.orchestra.run.smithi092.stdout:343 writesame 0x36f1ca7 thru 0x36ff673 data_size 0x1(0xd9cd bytes) 2024-09-04T17:46:02.267 INFO:teuthology.orchestra.run.smithi092.stdout:344 read 0x1725a2f thru 0x1725f32 (0x504 bytes) 2024-09-04T17:46:02.269 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:02.269 INFO:teuthology.orchestra.run.smithi092.stdout:346 read 0x1db8432 thru 0x1db9b34 (0x1703 bytes) 2024-09-04T17:46:02.269 INFO:teuthology.orchestra.run.smithi092.stdout:347 punch from 0x8104adc to 0x8108f87, (0x44ab bytes) 2024-09-04T17:46:02.269 INFO:teuthology.orchestra.run.smithi092.stdout:348 writesame 0x28ab85e thru 0x28b9f35 data_size 0xa6(0xe6d8 bytes) 2024-09-04T17:46:02.276 INFO:teuthology.orchestra.run.smithi092.stdout:349 write 0x51ea8a9 thru 0x51ebb0d (0x1265 bytes) 2024-09-04T17:46:02.279 INFO:teuthology.orchestra.run.smithi092.stdout:350 write 0xa72ca48 thru 0xa7372f8 (0xa8b1 bytes) 2024-09-04T17:46:02.286 INFO:teuthology.orchestra.run.smithi092.stdout:351 trunc from 0xde97c38 to 0xd371737 2024-09-04T17:46:02.297 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:02.297 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:46:02.297 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:46:02.297 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:46:02.297 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:02.304 INFO:teuthology.orchestra.run.smithi092.stdout:352 write 0xad1ce8 thru 0xadb0ae (0x93c7 bytes) 2024-09-04T17:46:02.307 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:02.307 INFO:teuthology.orchestra.run.smithi092.stdout:354 write 0x81f1b8d thru 0x81f6e43 (0x52b7 bytes) 2024-09-04T17:46:02.311 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-04T17:46:02.314 INFO:teuthology.orchestra.run.smithi092.stdout:355 punch from 0x6adf29e to 0x6ae5b37, (0x6899 bytes) 2024-09-04T17:46:02.314 INFO:teuthology.orchestra.run.smithi092.stdout:358 writesame 0x6f500b7 thru 0x6f5fd32 data_size 0x16(0xfc7c bytes) 2024-09-04T17:46:02.322 INFO:teuthology.orchestra.run.smithi092.stdout:359 read 0x5cb80c thru 0x5d4e21 (0x9616 bytes) 2024-09-04T17:46:02.323 INFO:teuthology.orchestra.run.smithi092.stdout:360 punch from 0xca8b6d5 to 0xca9678f, (0xb0ba bytes) 2024-09-04T17:46:02.324 INFO:teuthology.orchestra.run.smithi092.stdout:361 write 0x3e71670 thru 0x3e7daca (0xc45b bytes) 2024-09-04T17:46:02.329 INFO:teuthology.orchestra.run.smithi092.stdout:362 write 0xb371983 thru 0xb37743a (0x5ab8 bytes) 2024-09-04T17:46:02.333 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:02.330+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:02.333 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:02.344 INFO:teuthology.orchestra.run.smithi092.stdout:364 trunc from 0xd371737 to 0x525197c 2024-09-04T17:46:02.367 INFO:teuthology.orchestra.run.smithi092.stdout:365 punch from 0x3df7a7e to 0x3e03562, (0xbae4 bytes) 2024-09-04T17:46:02.367 INFO:teuthology.orchestra.run.smithi092.stdout:366 read 0x2be0efc thru 0x2beca0d (0xbb12 bytes) 2024-09-04T17:46:02.367 INFO:teuthology.orchestra.run.smithi092.stdout:367 read 0x15128a2 thru 0x151fabf (0xd21e bytes) 2024-09-04T17:46:02.367 INFO:teuthology.orchestra.run.smithi092.stdout:368 write 0x174052b thru 0x174ceee (0xc9c4 bytes) 2024-09-04T17:46:02.370 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:02.370 INFO:teuthology.orchestra.run.smithi092.stdout:373 punch from 0x20b85c9 to 0x20c10cc, (0x8b03 bytes) 2024-09-04T17:46:02.372 INFO:teuthology.orchestra.run.smithi092.stdout:374 trunc from 0x525197c to 0x3782d3d 2024-09-04T17:46:02.391 INFO:teuthology.orchestra.run.smithi092.stdout:375 punch from 0x11ab36c to 0x11b29f7, (0x768b bytes) 2024-09-04T17:46:02.393 INFO:teuthology.orchestra.run.smithi092.stdout:376 trunc from 0x3782d3d to 0x26d4c1b 2024-09-04T17:46:02.433 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:02.433 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:46:02.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:46:02.433 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:46:02.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.434 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4208.0:5 2.2 e21/19 2:5cea7035:::rbd_info:head v 21'1, mlcod=0'0)", 2024-09-04T17:46:02.434 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:45:48.387588+0000", 2024-09-04T17:46:02.434 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.047688487, 2024-09-04T17:46:02.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0016157809999999999, 2024-09-04T17:46:02.434 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:02.434 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:02.434 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:02.434 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:02.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.434 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:02.434 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:48.387588+0000", 2024-09-04T17:46:02.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:02.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:02.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:48.387588+0000", 2024-09-04T17:46:02.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:02.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:02.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:48.387588+0000", 2024-09-04T17:46:02.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0299999999999998e-07 2024-09-04T17:46:02.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:02.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:48.387601+0000", 2024-09-04T17:46:02.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2653000000000001e-05 2024-09-04T17:46:02.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:02.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:48.387643+0000", 2024-09-04T17:46:02.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2179999999999999e-05 2024-09-04T17:46:02.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:02.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:48.387669+0000", 2024-09-04T17:46:02.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6149999999999999e-05 2024-09-04T17:46:02.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:02.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:48.387783+0000", 2024-09-04T17:46:02.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000113522 2024-09-04T17:46:02.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:02.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:48.387829+0000", 2024-09-04T17:46:02.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6069999999999998e-05 2024-09-04T17:46:02.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:02.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:48.389083+0000", 2024-09-04T17:46:02.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001253964 2024-09-04T17:46:02.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:02.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:48.389203+0000", 2024-09-04T17:46:02.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012053900000000001 2024-09-04T17:46:02.438 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:02.438 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:02.438 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:02.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.438 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4208.0:7 2.2 e22/19 2:5cea7035:::rbd_info:head v 22'2, mlcod=0'0)", 2024-09-04T17:46:02.438 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:45:49.410521+0000", 2024-09-04T17:46:02.438 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.024755177999999, 2024-09-04T17:46:02.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001715741, 2024-09-04T17:46:02.438 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:02.439 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:02.439 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:02.439 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:02.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.439 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:02.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:49.410521+0000", 2024-09-04T17:46:02.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:02.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.439 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:02.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:49.410521+0000", 2024-09-04T17:46:02.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:02.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:02.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:49.410522+0000", 2024-09-04T17:46:02.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0300000000000001e-06 2024-09-04T17:46:02.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:02.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:49.410535+0000", 2024-09-04T17:46:02.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.291e-05 2024-09-04T17:46:02.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:02.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:49.410570+0000", 2024-09-04T17:46:02.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5305999999999997e-05 2024-09-04T17:46:02.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:02.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:49.410599+0000", 2024-09-04T17:46:02.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8833999999999999e-05 2024-09-04T17:46:02.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:02.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:49.410729+0000", 2024-09-04T17:46:02.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013043500000000001 2024-09-04T17:46:02.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:02.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:49.410770+0000", 2024-09-04T17:46:02.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0266999999999999e-05 2024-09-04T17:46:02.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:02.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:49.412129+0000", 2024-09-04T17:46:02.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001359119 2024-09-04T17:46:02.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:02.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:49.412237+0000", 2024-09-04T17:46:02.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010784 2024-09-04T17:46:02.442 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:02.443 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:02.443 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:02.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.443 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:849 3.6 e28/24 3:61ca0580:::rbd_data.11346251f094.000000000000018a:head v 28'4, mlcod=28'2)", 2024-09-04T17:46:02.443 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:01.800320+0000", 2024-09-04T17:46:02.443 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.63495646400000005, 2024-09-04T17:46:02.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00074074099999999995, 2024-09-04T17:46:02.443 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:02.443 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:02.443 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:02.443 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:02.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:02.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.800320+0000", 2024-09-04T17:46:02.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:02.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:02.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.800320+0000", 2024-09-04T17:46:02.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:02.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:02.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.800321+0000", 2024-09-04T17:46:02.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4750000000000001e-06 2024-09-04T17:46:02.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:02.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.800332+0000", 2024-09-04T17:46:02.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.101e-05 2024-09-04T17:46:02.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:02.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.800350+0000", 2024-09-04T17:46:02.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7425e-05 2024-09-04T17:46:02.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:02.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.800359+0000", 2024-09-04T17:46:02.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4439999999999997e-06 2024-09-04T17:46:02.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:02.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.800426+0000", 2024-09-04T17:46:02.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6557999999999995e-05 2024-09-04T17:46:02.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:02.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.800442+0000", 2024-09-04T17:46:02.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6741000000000001e-05 2024-09-04T17:46:02.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:02.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.801016+0000", 2024-09-04T17:46:02.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000573527 2024-09-04T17:46:02.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:02.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.801060+0000", 2024-09-04T17:46:02.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4561000000000002e-05 2024-09-04T17:46:02.447 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:02.447 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:02.447 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:02.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.448 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:850 3.0 3:07d21144:::rbd_data.11346251f094.0000000000000016:head [read 379388~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-04T17:46:02.448 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:01.801838+0000", 2024-09-04T17:46:02.448 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.63343851699999998, 2024-09-04T17:46:02.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000217903, 2024-09-04T17:46:02.448 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:02.448 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:02.448 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:02.448 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:02.448 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:02.448 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:02.448 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 850 2024-09-04T17:46:02.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.449 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:02.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:02.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.801838+0000", 2024-09-04T17:46:02.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:02.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:02.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.801838+0000", 2024-09-04T17:46:02.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:02.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.450 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:02.450 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.801839+0000", 2024-09-04T17:46:02.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.703e-06 2024-09-04T17:46:02.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.450 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:02.450 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.801845+0000", 2024-09-04T17:46:02.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4450000000000004e-06 2024-09-04T17:46:02.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.450 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:02.451 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.801858+0000", 2024-09-04T17:46:02.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2992e-05 2024-09-04T17:46:02.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.451 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:02.451 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.801866+0000", 2024-09-04T17:46:02.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3969999999999994e-06 2024-09-04T17:46:02.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:02.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.801931+0000", 2024-09-04T17:46:02.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5263000000000002e-05 2024-09-04T17:46:02.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:02.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.801998+0000", 2024-09-04T17:46:02.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6315000000000005e-05 2024-09-04T17:46:02.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:02.453 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.802055+0000", 2024-09-04T17:46:02.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7788000000000003e-05 2024-09-04T17:46:02.453 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:02.453 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:02.453 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:02.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.453 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:856 3.6 e28/24 3:61ca0580:::rbd_data.11346251f094.000000000000018a:head v 28'5, mlcod=28'3)", 2024-09-04T17:46:02.453 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:01.815842+0000", 2024-09-04T17:46:02.453 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.61943417000000001, 2024-09-04T17:46:02.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00060989900000000001, 2024-09-04T17:46:02.453 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:02.453 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:02.454 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:02.454 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:02.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.454 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:02.454 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.815842+0000", 2024-09-04T17:46:02.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:02.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.454 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:02.454 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.815842+0000", 2024-09-04T17:46:02.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:02.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.454 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:02.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.815844+0000", 2024-09-04T17:46:02.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.781e-06 2024-09-04T17:46:02.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:02.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.815854+0000", 2024-09-04T17:46:02.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0644e-05 2024-09-04T17:46:02.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:02.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.815875+0000", 2024-09-04T17:46:02.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1007999999999999e-05 2024-09-04T17:46:02.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:02.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.815887+0000", 2024-09-04T17:46:02.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1878e-05 2024-09-04T17:46:02.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:02.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.815956+0000", 2024-09-04T17:46:02.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8931999999999997e-05 2024-09-04T17:46:02.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:02.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.815979+0000", 2024-09-04T17:46:02.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2450999999999999e-05 2024-09-04T17:46:02.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:02.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.816404+0000", 2024-09-04T17:46:02.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00042527599999999999 2024-09-04T17:46:02.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:02.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.816452+0000", 2024-09-04T17:46:02.458 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7929000000000001e-05 2024-09-04T17:46:02.458 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:02.458 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:02.458 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:02.458 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.458 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.458 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:876 3.6 e28/24 3:6268fb61:::rbd_data.11346251f094.000000000000001c:head v 28'6, mlcod=28'4)", 2024-09-04T17:46:02.458 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:01.841605+0000", 2024-09-04T17:46:02.458 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.59367099199999995, 2024-09-04T17:46:02.458 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00083743200000000004, 2024-09-04T17:46:02.458 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:02.458 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:02.458 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:02.458 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:02.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:02.459 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.841605+0000", 2024-09-04T17:46:02.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:02.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:02.459 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.841605+0000", 2024-09-04T17:46:02.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:02.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:02.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.841607+0000", 2024-09-04T17:46:02.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.922e-06 2024-09-04T17:46:02.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:02.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.841651+0000", 2024-09-04T17:46:02.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4119000000000002e-05 2024-09-04T17:46:02.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:02.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.841671+0000", 2024-09-04T17:46:02.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9398e-05 2024-09-04T17:46:02.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:02.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.841681+0000", 2024-09-04T17:46:02.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0832000000000001e-05 2024-09-04T17:46:02.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:02.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.841753+0000", 2024-09-04T17:46:02.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1845000000000001e-05 2024-09-04T17:46:02.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:02.462 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.841782+0000", 2024-09-04T17:46:02.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8963999999999999e-05 2024-09-04T17:46:02.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.462 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:02.462 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.842409+0000", 2024-09-04T17:46:02.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000626748 2024-09-04T17:46:02.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.462 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:02.462 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.842443+0000", 2024-09-04T17:46:02.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3603999999999999e-05 2024-09-04T17:46:02.463 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:02.463 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:02.463 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:02.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.463 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:937 3.0 3:02897c52:::rbd_data.11346251f094.0000000000000163:head [set-alloc-hint object_size 524288 write_size 524288,write 104784~26288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-04T17:46:02.463 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:01.944645+0000", 2024-09-04T17:46:02.463 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.49063159200000001, 2024-09-04T17:46:02.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001556492, 2024-09-04T17:46:02.464 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:02.464 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:02.464 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:02.464 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:02.464 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:02.464 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:02.464 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 937 2024-09-04T17:46:02.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.464 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:02.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.464 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:02.465 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.944645+0000", 2024-09-04T17:46:02.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:02.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:02.465 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.944645+0000", 2024-09-04T17:46:02.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:02.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:02.465 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.944647+0000", 2024-09-04T17:46:02.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4650000000000001e-06 2024-09-04T17:46:02.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:02.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.944682+0000", 2024-09-04T17:46:02.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5209999999999997e-05 2024-09-04T17:46:02.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:02.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.944703+0000", 2024-09-04T17:46:02.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1027e-05 2024-09-04T17:46:02.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:02.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.944712+0000", 2024-09-04T17:46:02.467 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8750000000000006e-06 2024-09-04T17:46:02.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.467 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.467 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:02.467 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.944781+0000", 2024-09-04T17:46:02.467 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9023999999999995e-05 2024-09-04T17:46:02.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.467 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.467 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:02.468 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.944889+0000", 2024-09-04T17:46:02.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107732 2024-09-04T17:46:02.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.468 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2024-09-04T17:46:02.468 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.944988+0000", 2024-09-04T17:46:02.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8947e-05 2024-09-04T17:46:02.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.468 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:46:02.468 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.945775+0000", 2024-09-04T17:46:02.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000787238 2024-09-04T17:46:02.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.469 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:02.469 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.946114+0000", 2024-09-04T17:46:02.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00033877400000000002 2024-09-04T17:46:02.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.469 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:02.469 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.946150+0000", 2024-09-04T17:46:02.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5967000000000003e-05 2024-09-04T17:46:02.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.470 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:02.470 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.946201+0000", 2024-09-04T17:46:02.470 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1233000000000002e-05 2024-09-04T17:46:02.470 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:02.470 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:02.470 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:02.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.470 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop_reply(client.4404.0:937 3.0 e28/24 ondisk, result = 0)", 2024-09-04T17:46:02.470 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:01.946030+0000", 2024-09-04T17:46:02.470 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.48924658500000001, 2024-09-04T17:46:02.470 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017761099999999999, 2024-09-04T17:46:02.471 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:02.471 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:02.471 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:02.471 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:02.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:02.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.946030+0000", 2024-09-04T17:46:02.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:02.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:02.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.946030+0000", 2024-09-04T17:46:02.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:02.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.472 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:02.472 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.946030+0000", 2024-09-04T17:46:02.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8000000000000004e-07 2024-09-04T17:46:02.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.472 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:02.472 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.946034+0000", 2024-09-04T17:46:02.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.07e-06 2024-09-04T17:46:02.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.472 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:02.472 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.946045+0000", 2024-09-04T17:46:02.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0079e-05 2024-09-04T17:46:02.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.473 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:02.473 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.946052+0000", 2024-09-04T17:46:02.473 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7530000000000001e-06 2024-09-04T17:46:02.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.473 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:02.473 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.946100+0000", 2024-09-04T17:46:02.473 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8162999999999999e-05 2024-09-04T17:46:02.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.474 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:02.474 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.946110+0000", 2024-09-04T17:46:02.474 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9110000000000005e-06 2024-09-04T17:46:02.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.474 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:02.474 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.946207+0000", 2024-09-04T17:46:02.474 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6755000000000005e-05 2024-09-04T17:46:02.474 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:02.474 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:02.474 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:02.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.475 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:951 3.0 3:02897c52:::rbd_data.11346251f094.0000000000000163:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-04T17:46:02.475 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:01.975793+0000", 2024-09-04T17:46:02.475 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.45948292200000002, 2024-09-04T17:46:02.475 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001258278, 2024-09-04T17:46:02.475 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:02.475 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:02.475 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:02.475 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:02.475 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:02.475 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:02.475 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 951 2024-09-04T17:46:02.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.475 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:02.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:02.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.975793+0000", 2024-09-04T17:46:02.476 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:02.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:02.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.975793+0000", 2024-09-04T17:46:02.476 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:02.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:02.477 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.975795+0000", 2024-09-04T17:46:02.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.046e-06 2024-09-04T17:46:02.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.477 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:02.477 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.975802+0000", 2024-09-04T17:46:02.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3629999999999999e-06 2024-09-04T17:46:02.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.477 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:02.477 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.975820+0000", 2024-09-04T17:46:02.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7985000000000001e-05 2024-09-04T17:46:02.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.478 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.478 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:02.478 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.975832+0000", 2024-09-04T17:46:02.478 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2302e-05 2024-09-04T17:46:02.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.478 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.478 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:02.478 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.975910+0000", 2024-09-04T17:46:02.478 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8312999999999994e-05 2024-09-04T17:46:02.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.479 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:02.479 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.975976+0000", 2024-09-04T17:46:02.479 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5671000000000002e-05 2024-09-04T17:46:02.479 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.479 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2024-09-04T17:46:02.479 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.976052+0000", 2024-09-04T17:46:02.479 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5815000000000005e-05 2024-09-04T17:46:02.479 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.479 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:46:02.479 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.976652+0000", 2024-09-04T17:46:02.480 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00060068000000000003 2024-09-04T17:46:02.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.480 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.480 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:02.480 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.976940+0000", 2024-09-04T17:46:02.480 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028739100000000001 2024-09-04T17:46:02.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.480 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.480 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:02.480 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.976998+0000", 2024-09-04T17:46:02.480 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8514999999999998e-05 2024-09-04T17:46:02.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.481 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:02.481 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.977051+0000", 2024-09-04T17:46:02.481 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3196999999999999e-05 2024-09-04T17:46:02.481 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:02.481 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:02.481 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:02.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.481 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop_reply(client.4404.0:951 3.0 e28/24 ondisk, result = 0)", 2024-09-04T17:46:02.481 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:01.976859+0000", 2024-09-04T17:46:02.481 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.45841664599999998, 2024-09-04T17:46:02.481 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019796599999999999, 2024-09-04T17:46:02.482 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:02.482 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:02.482 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:02.482 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:02.482 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.482 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:02.482 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.976859+0000", 2024-09-04T17:46:02.482 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:02.482 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.482 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.482 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:02.482 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.976859+0000", 2024-09-04T17:46:02.482 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:02.483 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.483 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.483 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:02.483 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.976860+0000", 2024-09-04T17:46:02.483 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.16e-07 2024-09-04T17:46:02.483 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.483 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.483 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:02.483 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.976864+0000", 2024-09-04T17:46:02.483 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6859999999999999e-06 2024-09-04T17:46:02.483 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.483 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.483 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:02.484 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.976873+0000", 2024-09-04T17:46:02.484 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0389999999999993e-06 2024-09-04T17:46:02.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.484 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.484 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:02.484 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.976879+0000", 2024-09-04T17:46:02.484 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6269999999999997e-06 2024-09-04T17:46:02.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.484 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.484 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:02.485 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.976924+0000", 2024-09-04T17:46:02.485 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5837999999999997e-05 2024-09-04T17:46:02.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.485 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.485 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:02.485 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.976933+0000", 2024-09-04T17:46:02.485 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3990000000000004e-06 2024-09-04T17:46:02.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.485 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.485 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:02.485 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.977057+0000", 2024-09-04T17:46:02.485 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000124561 2024-09-04T17:46:02.485 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:02.486 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:02.486 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:02.486 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.486 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.487 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:1050 3.4 3:237c6c21:::rbd_data.11346251f094.0000000000000012:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 182129~23213] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-04T17:46:02.487 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:02.149667+0000", 2024-09-04T17:46:02.488 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.285608904, 2024-09-04T17:46:02.488 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0027256060000000002, 2024-09-04T17:46:02.488 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:02.488 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:02.488 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:02.488 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:02.488 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:02.488 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:02.488 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 1050 2024-09-04T17:46:02.488 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.488 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:02.488 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.489 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:02.489 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.149667+0000", 2024-09-04T17:46:02.489 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:02.489 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.489 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.489 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:02.489 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.149667+0000", 2024-09-04T17:46:02.489 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:02.489 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.489 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.489 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:02.490 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.149670+0000", 2024-09-04T17:46:02.490 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5299999999999999e-06 2024-09-04T17:46:02.490 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.490 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.490 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:02.490 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.149711+0000", 2024-09-04T17:46:02.490 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0997000000000001e-05 2024-09-04T17:46:02.490 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.490 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.490 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:02.490 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.149737+0000", 2024-09-04T17:46:02.490 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5848000000000001e-05 2024-09-04T17:46:02.491 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.491 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.491 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:02.491 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.149749+0000", 2024-09-04T17:46:02.491 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2439000000000001e-05 2024-09-04T17:46:02.491 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.491 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.491 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:02.491 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.149897+0000", 2024-09-04T17:46:02.491 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014796300000000001 2024-09-04T17:46:02.491 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.491 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.492 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:02.492 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.150171+0000", 2024-09-04T17:46:02.492 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027427499999999999 2024-09-04T17:46:02.492 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.492 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.492 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2024-09-04T17:46:02.492 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.150533+0000", 2024-09-04T17:46:02.492 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00036135300000000002 2024-09-04T17:46:02.492 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.492 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.492 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:46:02.492 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.151935+0000", 2024-09-04T17:46:02.493 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014021159999999999 2024-09-04T17:46:02.493 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.493 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.493 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:02.493 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.152222+0000", 2024-09-04T17:46:02.493 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028762500000000001 2024-09-04T17:46:02.494 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.494 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.494 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:02.494 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.152300+0000", 2024-09-04T17:46:02.494 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7459999999999994e-05 2024-09-04T17:46:02.495 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.495 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.495 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:02.495 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.152393+0000", 2024-09-04T17:46:02.495 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2999999999999997e-05 2024-09-04T17:46:02.495 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:02.495 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:02.496 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:02.496 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.496 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.496 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop_reply(client.4404.0:1050 3.4 e28/24 ondisk, result = 0)", 2024-09-04T17:46:02.496 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:02.152097+0000", 2024-09-04T17:46:02.496 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.283179185, 2024-09-04T17:46:02.497 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030905599999999999, 2024-09-04T17:46:02.497 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:02.497 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:02.497 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:02.497 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:02.497 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.497 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:02.497 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.152097+0000", 2024-09-04T17:46:02.497 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:02.498 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.498 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.498 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:02.498 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.152097+0000", 2024-09-04T17:46:02.498 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:02.498 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.498 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.498 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:02.498 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.152098+0000", 2024-09-04T17:46:02.498 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7000000000000003e-07 2024-09-04T17:46:02.498 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.498 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.498 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:02.498 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.152102+0000", 2024-09-04T17:46:02.499 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6630000000000001e-06 2024-09-04T17:46:02.499 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.499 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.499 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:02.499 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.152114+0000", 2024-09-04T17:46:02.499 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1504e-05 2024-09-04T17:46:02.499 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.499 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.499 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:02.499 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.152128+0000", 2024-09-04T17:46:02.499 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3587999999999999e-05 2024-09-04T17:46:02.499 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.499 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.499 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:02.500 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.152204+0000", 2024-09-04T17:46:02.500 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6437e-05 2024-09-04T17:46:02.500 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.500 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.500 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:02.500 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.152215+0000", 2024-09-04T17:46:02.500 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1216000000000001e-05 2024-09-04T17:46:02.500 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.500 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.500 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:02.500 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.152406+0000", 2024-09-04T17:46:02.501 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019077800000000001 2024-09-04T17:46:02.501 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:02.501 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:02.501 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:02.501 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.501 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.501 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:1068 3.8 3:16c95858:::rbd_data.11346251f094.000000000000001d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 232109~10405] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-04T17:46:02.501 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:02.179957+0000", 2024-09-04T17:46:02.501 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.25531865999999998, 2024-09-04T17:46:02.501 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023318670000000001, 2024-09-04T17:46:02.501 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:02.501 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:02.502 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:02.502 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:02.502 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:02.502 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:02.502 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 1068 2024-09-04T17:46:02.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.502 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:02.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.502 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:02.502 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.179957+0000", 2024-09-04T17:46:02.502 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:02.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.503 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:02.503 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.179957+0000", 2024-09-04T17:46:02.503 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:02.503 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.503 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.503 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:02.503 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.179961+0000", 2024-09-04T17:46:02.503 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.388e-06 2024-09-04T17:46:02.503 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.503 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.503 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:02.504 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.180005+0000", 2024-09-04T17:46:02.504 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4549000000000003e-05 2024-09-04T17:46:02.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.504 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.504 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:02.504 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.180038+0000", 2024-09-04T17:46:02.504 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2521999999999998e-05 2024-09-04T17:46:02.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.504 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.505 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:02.505 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.180054+0000", 2024-09-04T17:46:02.505 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5773999999999998e-05 2024-09-04T17:46:02.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.505 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:02.505 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.180183+0000", 2024-09-04T17:46:02.505 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012960099999999999 2024-09-04T17:46:02.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.505 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:02.505 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.180419+0000", 2024-09-04T17:46:02.505 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023581200000000001 2024-09-04T17:46:02.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.506 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.506 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:46:02.506 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.182019+0000", 2024-09-04T17:46:02.506 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001600249 2024-09-04T17:46:02.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.506 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.506 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:02.506 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.182111+0000", 2024-09-04T17:46:02.506 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1685000000000001e-05 2024-09-04T17:46:02.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.506 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.506 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:02.506 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.182289+0000", 2024-09-04T17:46:02.506 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017828700000000001 2024-09-04T17:46:02.507 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:02.507 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:02.507 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:02.507 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.507 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.507 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:1070 3.0 3:053e1812:::rbd_data.11346251f094.0000000000000030:head [read 262144~38686] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-04T17:46:02.507 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:02.182443+0000", 2024-09-04T17:46:02.507 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.25283325699999998, 2024-09-04T17:46:02.507 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00031210199999999997, 2024-09-04T17:46:02.507 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:02.507 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:02.508 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:02.508 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:02.508 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:02.508 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:02.508 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 1070 2024-09-04T17:46:02.508 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.508 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:02.508 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.508 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:02.508 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.182443+0000", 2024-09-04T17:46:02.508 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:02.509 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.509 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.509 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:02.509 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.182443+0000", 2024-09-04T17:46:02.509 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:02.509 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.509 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.509 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:02.509 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.182445+0000", 2024-09-04T17:46:02.509 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6750000000000001e-06 2024-09-04T17:46:02.509 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.509 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.510 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:02.510 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.182452+0000", 2024-09-04T17:46:02.510 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3849999999999996e-06 2024-09-04T17:46:02.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.510 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:02.510 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.182472+0000", 2024-09-04T17:46:02.510 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0094000000000001e-05 2024-09-04T17:46:02.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.510 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:02.510 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.182483+0000", 2024-09-04T17:46:02.510 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1114999999999999e-05 2024-09-04T17:46:02.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.511 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.511 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:02.511 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.182591+0000", 2024-09-04T17:46:02.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010770899999999999 2024-09-04T17:46:02.511 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.511 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.511 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:02.511 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.182664+0000", 2024-09-04T17:46:02.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3104999999999999e-05 2024-09-04T17:46:02.511 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.511 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.511 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:02.511 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.182755+0000", 2024-09-04T17:46:02.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1019000000000004e-05 2024-09-04T17:46:02.512 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:02.512 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:02.512 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:02.512 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.512 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.512 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:1114 3.6 e28/24 3:635a7fe7:::rbd_data.11346251f094.0000000000000017:head v 28'7, mlcod=28'5)", 2024-09-04T17:46:02.512 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:02.309095+0000", 2024-09-04T17:46:02.512 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.12618106900000001, 2024-09-04T17:46:02.512 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0016026879999999999, 2024-09-04T17:46:02.512 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:02.512 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:02.513 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:02.513 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:02.513 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.513 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:02.513 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.309095+0000", 2024-09-04T17:46:02.513 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:02.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.513 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.513 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:02.514 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.309095+0000", 2024-09-04T17:46:02.514 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:02.514 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.514 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.514 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:02.514 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.309097+0000", 2024-09-04T17:46:02.515 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.697e-06 2024-09-04T17:46:02.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.515 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:02.515 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.309143+0000", 2024-09-04T17:46:02.515 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5853000000000003e-05 2024-09-04T17:46:02.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.516 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:02.516 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.309160+0000", 2024-09-04T17:46:02.516 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7441000000000001e-05 2024-09-04T17:46:02.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.516 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.516 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:02.516 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.309169+0000", 2024-09-04T17:46:02.516 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.721e-06 2024-09-04T17:46:02.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.516 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.516 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:02.516 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.309288+0000", 2024-09-04T17:46:02.516 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000119683 2024-09-04T17:46:02.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.516 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.517 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:02.517 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.309329+0000", 2024-09-04T17:46:02.517 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0303999999999999e-05 2024-09-04T17:46:02.517 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.517 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.517 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:02.517 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.310621+0000", 2024-09-04T17:46:02.517 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0012919299999999999 2024-09-04T17:46:02.517 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.517 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.517 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:02.517 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.310698+0000", 2024-09-04T17:46:02.517 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7058999999999995e-05 2024-09-04T17:46:02.518 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:02.518 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:02.518 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:02.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.518 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.518 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:1123 3.8 3:16e607df:::rbd_data.11346251f094.0000000000000165:head [set-alloc-hint object_size 524288 write_size 524288,write 203139~23224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-04T17:46:02.518 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:02.336206+0000", 2024-09-04T17:46:02.518 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.099070188000000003, 2024-09-04T17:46:02.518 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0012244409999999999, 2024-09-04T17:46:02.518 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:02.518 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:02.518 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:02.518 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:02.518 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:02.518 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:02.519 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 1123 2024-09-04T17:46:02.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.519 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:02.519 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.519 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:02.519 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.336206+0000", 2024-09-04T17:46:02.519 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:02.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.519 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.519 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:02.520 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.336206+0000", 2024-09-04T17:46:02.520 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:02.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.520 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.520 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:02.520 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.336210+0000", 2024-09-04T17:46:02.520 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8630000000000001e-06 2024-09-04T17:46:02.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.520 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.520 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:02.520 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.336266+0000", 2024-09-04T17:46:02.520 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6419999999999999e-05 2024-09-04T17:46:02.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.521 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.521 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:02.521 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.336303+0000", 2024-09-04T17:46:02.521 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6443000000000003e-05 2024-09-04T17:46:02.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.521 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.521 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:02.521 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.336320+0000", 2024-09-04T17:46:02.521 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7319000000000001e-05 2024-09-04T17:46:02.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.521 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.521 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:02.522 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.336460+0000", 2024-09-04T17:46:02.522 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000140247 2024-09-04T17:46:02.522 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.522 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.522 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:02.522 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.336568+0000", 2024-09-04T17:46:02.522 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107664 2024-09-04T17:46:02.522 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.522 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.522 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:46:02.522 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.337313+0000", 2024-09-04T17:46:02.522 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000745048 2024-09-04T17:46:02.522 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.522 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.523 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:02.523 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.337346+0000", 2024-09-04T17:46:02.523 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3433999999999999e-05 2024-09-04T17:46:02.523 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.523 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.523 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:02.523 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.337430+0000", 2024-09-04T17:46:02.523 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4003000000000002e-05 2024-09-04T17:46:02.523 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:02.523 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:02.523 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:02.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.524 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.524 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:1132 3.8 3:16e607df:::rbd_data.11346251f094.0000000000000165:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-04T17:46:02.524 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:02.351797+0000", 2024-09-04T17:46:02.524 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.083479553999999997, 2024-09-04T17:46:02.524 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00120622, 2024-09-04T17:46:02.524 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:02.524 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:02.524 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:02.525 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:02.525 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:02.525 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:02.525 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 1132 2024-09-04T17:46:02.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.525 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:02.526 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.526 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:02.526 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.351797+0000", 2024-09-04T17:46:02.526 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:02.526 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.526 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.527 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:02.527 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.351797+0000", 2024-09-04T17:46:02.527 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:02.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.527 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.528 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:02.528 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.351799+0000", 2024-09-04T17:46:02.528 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0540000000000002e-06 2024-09-04T17:46:02.528 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.528 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.528 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:02.528 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.351806+0000", 2024-09-04T17:46:02.529 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9700000000000002e-06 2024-09-04T17:46:02.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.529 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.529 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:02.529 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.351822+0000", 2024-09-04T17:46:02.530 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6336000000000001e-05 2024-09-04T17:46:02.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.530 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.530 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:02.530 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.351831+0000", 2024-09-04T17:46:02.530 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8249999999999994e-06 2024-09-04T17:46:02.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.531 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.531 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:02.531 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.351912+0000", 2024-09-04T17:46:02.531 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1087999999999993e-05 2024-09-04T17:46:02.532 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.532 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.532 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:02.532 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.352034+0000", 2024-09-04T17:46:02.532 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012223900000000001 2024-09-04T17:46:02.532 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.533 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.533 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:46:02.533 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.352908+0000", 2024-09-04T17:46:02.533 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00087437300000000001 2024-09-04T17:46:02.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.533 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.534 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:02.534 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.352942+0000", 2024-09-04T17:46:02.534 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3673000000000001e-05 2024-09-04T17:46:02.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.534 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:02.535 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.353003+0000", 2024-09-04T17:46:02.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0662000000000003e-05 2024-09-04T17:46:02.535 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:02.535 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:02.535 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:02.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.536 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.536 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:1145 3.8 3:1172812b:::rbd_data.11345f88eb9a.0000000000000005:3 [read 1124514~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-04T17:46:02.536 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:02.370203+0000", 2024-09-04T17:46:02.536 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.065072978000000004, 2024-09-04T17:46:02.537 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028402899999999999, 2024-09-04T17:46:02.537 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:02.537 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:02.537 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:02.537 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:02.537 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:02.538 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:02.538 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 1145 2024-09-04T17:46:02.538 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.538 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:02.538 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.538 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:02.539 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.370203+0000", 2024-09-04T17:46:02.539 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:02.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.539 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:02.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.370203+0000", 2024-09-04T17:46:02.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:02.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:02.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.370205+0000", 2024-09-04T17:46:02.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8840000000000001e-06 2024-09-04T17:46:02.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.541 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:02.541 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.370210+0000", 2024-09-04T17:46:02.542 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3600000000000004e-06 2024-09-04T17:46:02.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.542 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.542 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:02.542 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.370225+0000", 2024-09-04T17:46:02.542 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5041e-05 2024-09-04T17:46:02.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.543 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.543 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:02.543 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.370235+0000", 2024-09-04T17:46:02.543 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9380000000000004e-06 2024-09-04T17:46:02.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.544 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.544 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:02.544 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.370311+0000", 2024-09-04T17:46:02.544 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6093000000000005e-05 2024-09-04T17:46:02.544 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.545 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.545 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:02.545 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.370389+0000", 2024-09-04T17:46:02.545 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7828000000000001e-05 2024-09-04T17:46:02.545 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.545 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.546 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:02.546 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.370487+0000", 2024-09-04T17:46:02.546 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7885000000000002e-05 2024-09-04T17:46:02.546 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:02.546 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:02.546 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:02.547 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:02.547 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:02.547 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:02.548 INFO:teuthology.orchestra.run.smithi092.stdout:377 write 0x541aaf3 thru 0x541e97d (0x3e8b bytes) 2024-09-04T17:46:02.548 INFO:teuthology.orchestra.run.smithi092.stdout:378 write 0x389f619 thru 0x38a1aa6 (0x248e bytes) 2024-09-04T17:46:02.550 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-04T17:46:02.552 INFO:teuthology.orchestra.run.smithi092.stdout:379 flatten 2024-09-04T17:46:02.552 INFO:teuthology.orchestra.run.smithi092.stdout:380 punch from 0x19dc2ce to 0x19eb9fd, (0xf72f bytes) 2024-09-04T17:46:02.552 INFO:teuthology.orchestra.run.smithi092.stdout:381 read 0x1399372 thru 0x1399811 (0x4a0 bytes) 2024-09-04T17:46:02.552 INFO:teuthology.orchestra.run.smithi092.stdout:382 write 0xb56d193 thru 0xb57ad9b (0xdc09 bytes) 2024-09-04T17:46:02.553 INFO:teuthology.orchestra.run.smithi092.stdout:383 read 0x4642664 thru 0x464b900 (0x929d bytes) 2024-09-04T17:46:02.553 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:02.553 INFO:teuthology.orchestra.run.smithi092.stdout:386 punch from 0x5d23be3 to 0x5d2bfd8, (0x83f5 bytes) 2024-09-04T17:46:02.553 INFO:teuthology.orchestra.run.smithi092.stdout:387 trunc from 0xb57ad9c to 0xa82a0d7 2024-09-04T17:46:02.553 INFO:teuthology.orchestra.run.smithi092.stdout:388 writesame 0x36670be thru 0x3667ac5 data_size 0x8(0xa08 bytes) 2024-09-04T17:46:02.553 INFO:teuthology.orchestra.run.smithi092.stdout:389 punch from 0x5c01d84 to 0x5c08a06, (0x6c82 bytes) 2024-09-04T17:46:02.553 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:02.553 INFO:teuthology.orchestra.run.smithi092.stdout:391 trunc from 0xa82a0d7 to 0x5d43f82 2024-09-04T17:46:02.555 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:02.556+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:02.557 INFO:teuthology.orchestra.run.smithi092.stdout:392 read 0x1c46e35 thru 0x1c53fed (0xd1b9 bytes) 2024-09-04T17:46:02.587 INFO:teuthology.orchestra.run.smithi092.stdout:393 write 0xee0b53a thru 0xee1aa51 (0xf518 bytes) 2024-09-04T17:46:02.594 INFO:teuthology.orchestra.run.smithi092.stdout:394 write 0xdf92086 thru 0xdf9a35f (0x82da bytes) 2024-09-04T17:46:02.600 INFO:teuthology.orchestra.run.smithi092.stdout:395 writesame 0x4fbfd54 thru 0x4fcd9ca data_size 0x3(0xdc77 bytes) 2024-09-04T17:46:02.633 INFO:teuthology.orchestra.run.smithi092.stdout:396 write 0x598ff3e thru 0x599a2b5 (0xa378 bytes) 2024-09-04T17:46:02.641 INFO:teuthology.orchestra.run.smithi092.stdout:397 punch from 0x2467135 to 0x247406d, (0xcf38 bytes) 2024-09-04T17:46:02.641 INFO:teuthology.orchestra.run.smithi092.stdout:truncating to largest ever: 0xe652280 2024-09-04T17:46:02.641 INFO:teuthology.orchestra.run.smithi092.stdout:399 trunc from 0xee1aa52 to 0xe652280 2024-09-04T17:46:02.656 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:02.656+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:02.658 INFO:teuthology.orchestra.run.smithi092.stdout:400 read 0xd40e6ea thru 0xd412fc2 (0x48d9 bytes) 2024-09-04T17:46:02.658 INFO:teuthology.orchestra.run.smithi092.stdout:401 write 0x26c9fe thru 0x26f6bb (0x2cbe bytes) 2024-09-04T17:46:02.661 INFO:teuthology.orchestra.run.smithi092.stdout:402 write 0xc61fff7 thru 0xc62c6a9 (0xc6b3 bytes) 2024-09-04T17:46:02.670 INFO:teuthology.orchestra.run.smithi092.stdout:404 writesame 0xbc5ae30 thru 0xbc60fed data_size 0x1(0x61be bytes) 2024-09-04T17:46:02.684 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:02.684 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:46:02.684 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.685 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:1370 3.4 e28/24 3:22db5e3d:::rbd_data.11346251f094.0000000000000174:head v 28'3, mlcod=28'1)", 2024-09-04T17:46:02.685 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:02.686754+0000", 2024-09-04T17:46:02.685 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.00097628500000000002, 2024-09-04T17:46:02.685 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00099863800000000004, 2024-09-04T17:46:02.685 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:02.685 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:02.685 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:02.685 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:02.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.685 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:02.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.686754+0000", 2024-09-04T17:46:02.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:02.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:02.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.686754+0000", 2024-09-04T17:46:02.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:02.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:02.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.686757+0000", 2024-09-04T17:46:02.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5330000000000001e-06 2024-09-04T17:46:02.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:02.687 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.686820+0000", 2024-09-04T17:46:02.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3526000000000001e-05 2024-09-04T17:46:02.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:02.687 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.686859+0000", 2024-09-04T17:46:02.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8260000000000003e-05 2024-09-04T17:46:02.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:02.687 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.686881+0000", 2024-09-04T17:46:02.688 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2549999999999999e-05 2024-09-04T17:46:02.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.688 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.688 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:02.688 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.686990+0000", 2024-09-04T17:46:02.688 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010919 2024-09-04T17:46:02.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.688 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.688 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:02.688 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.687017+0000", 2024-09-04T17:46:02.688 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6137999999999999e-05 2024-09-04T17:46:02.688 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:02.688 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:02.689 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:02.689 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:02.689 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-04T17:46:02.689 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 1 2024-09-04T17:46:02.689 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:02.690 INFO:teuthology.orchestra.run.smithi092.stdout:405 write 0x43f5e22 thru 0x440398d (0xdb6c bytes) 2024-09-04T17:46:02.700 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-04T17:46:02.707 INFO:teuthology.orchestra.run.smithi092.stdout:407 trunc from 0xe652280 to 0x7c988f9 2024-09-04T17:46:02.738 INFO:teuthology.orchestra.run.smithi092.stdout:408 writesame 0x932b045 thru 0x9337b57 data_size 0x27(0xcb13 bytes) 2024-09-04T17:46:02.738 INFO:teuthology.orchestra.run.smithi092.stdout:409 read 0x5004c57 thru 0x50097c2 (0x4b6c bytes) 2024-09-04T17:46:02.738 INFO:teuthology.orchestra.run.smithi092.stdout:411 trunc from 0x9337b58 to 0x9ce9804 2024-09-04T17:46:02.745 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:02.758 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:02.758+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:02.759 INFO:teuthology.orchestra.run.smithi092.stdout:413 writesame 0xde6fd58 thru 0xde6ff2a data_size 0x1(0x1d3 bytes) 2024-09-04T17:46:02.759 INFO:teuthology.orchestra.run.smithi092.stdout:414 read 0xae93932 thru 0xae9e6ea (0xadb9 bytes) 2024-09-04T17:46:02.759 INFO:teuthology.orchestra.run.smithi092.stdout:415 punch from 0x57c321e to 0x57c9b5f, (0x6941 bytes) 2024-09-04T17:46:02.768 INFO:teuthology.orchestra.run.smithi092.stdout:416 write 0xee063b8 thru 0xee10ff3 (0xac3c bytes) 2024-09-04T17:46:02.775 INFO:teuthology.orchestra.run.smithi092.stdout:417 punch from 0x8021213 to 0x802b27e, (0xa06b bytes) 2024-09-04T17:46:02.775 INFO:teuthology.orchestra.run.smithi092.stdout:418 writesame 0x3a4b434 thru 0x3a55c4d data_size 0x1(0xa81a bytes) 2024-09-04T17:46:02.828 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:02.828 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:46:02.829 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:46:02.829 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:46:02.829 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:02.834 INFO:teuthology.orchestra.run.smithi092.stdout:421 punch from 0xb2df49c to 0xb2ee5dd, (0xf141 bytes) 2024-09-04T17:46:02.834 INFO:teuthology.orchestra.run.smithi092.stdout:423 read 0xc6ff151 thru 0xc706c55 (0x7b05 bytes) 2024-09-04T17:46:02.834 INFO:teuthology.orchestra.run.smithi092.stdout:424 read 0xb86875 thru 0xb8d275 (0x6a01 bytes) 2024-09-04T17:46:02.835 INFO:teuthology.orchestra.run.smithi092.stdout:425 write 0x141fca1 thru 0x142cf15 (0xd275 bytes) 2024-09-04T17:46:02.842 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-04T17:46:02.843 INFO:teuthology.orchestra.run.smithi092.stdout:truncating to largest ever: 0xe7949d6 2024-09-04T17:46:02.843 INFO:teuthology.orchestra.run.smithi092.stdout:426 trunc from 0xee10ff4 to 0xe7949d6 2024-09-04T17:46:02.858 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:02.856+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:02.880 INFO:teuthology.orchestra.run.smithi092.stdout:427 trunc from 0xe7949d6 to 0x40a0ea2 2024-09-04T17:46:02.900 INFO:teuthology.orchestra.run.smithi092.stdout:431 write 0x403fdc0 thru 0x4043e25 (0x4066 bytes) 2024-09-04T17:46:02.908 INFO:teuthology.orchestra.run.smithi092.stdout:432 read 0xa37c52 thru 0xa44530 (0xc8df bytes) 2024-09-04T17:46:02.908 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:02.919 INFO:teuthology.orchestra.run.smithi092.stdout:434 write 0x5c800fa thru 0x5c8092e (0x835 bytes) 2024-09-04T17:46:02.941 INFO:teuthology.orchestra.run.smithi092.stdout:435 write 0x926d64b thru 0x9270e9a (0x3850 bytes) 2024-09-04T17:46:02.947 INFO:teuthology.orchestra.run.smithi092.stdout:436 writesame 0x873873b thru 0x87421b5 data_size 0x1(0x9a7b bytes) 2024-09-04T17:46:02.958 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:02.958+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:02.981 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:02.981 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:46:02.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:46:02.981 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:46:02.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.981 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4208.0:1 2.3 e20/19 2:c4c92e5a:::rbd_trash:head v 20'1, mlcod=0'0)", 2024-09-04T17:46:02.981 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:45:47.377621+0000", 2024-09-04T17:46:02.981 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.605920556000001, 2024-09-04T17:46:02.982 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0010679260000000001, 2024-09-04T17:46:02.982 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:02.982 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:02.982 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:02.982 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:02.982 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.982 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:02.982 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:47.377621+0000", 2024-09-04T17:46:02.982 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:02.982 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.982 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.982 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:02.982 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:47.377621+0000", 2024-09-04T17:46:02.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:02.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.983 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:02.983 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:47.377622+0000", 2024-09-04T17:46:02.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0299999999999998e-07 2024-09-04T17:46:02.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.983 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:02.983 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:47.377633+0000", 2024-09-04T17:46:02.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1528999999999999e-05 2024-09-04T17:46:02.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:02.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:47.377645+0000", 2024-09-04T17:46:02.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1779e-05 2024-09-04T17:46:02.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:02.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:47.377655+0000", 2024-09-04T17:46:02.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8989999999999996e-06 2024-09-04T17:46:02.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:02.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:47.377726+0000", 2024-09-04T17:46:02.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1237999999999998e-05 2024-09-04T17:46:02.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:02.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:47.377760+0000", 2024-09-04T17:46:02.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3958000000000003e-05 2024-09-04T17:46:02.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:02.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:47.378637+0000", 2024-09-04T17:46:02.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00087744799999999996 2024-09-04T17:46:02.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:02.986 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:47.378689+0000", 2024-09-04T17:46:02.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1372000000000002e-05 2024-09-04T17:46:02.986 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:02.986 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:02.986 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:02.986 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.986 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.986 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:937 3.0 e28/24 3:02897c52:::rbd_data.11346251f094.0000000000000163:head v 28'4, mlcod=28'2)", 2024-09-04T17:46:02.986 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:01.945087+0000", 2024-09-04T17:46:02.986 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.038454502, 2024-09-04T17:46:02.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000898294, 2024-09-04T17:46:02.986 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:02.986 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:02.987 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:02.987 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:02.987 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.987 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:02.987 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.945087+0000", 2024-09-04T17:46:02.987 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:02.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.987 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.987 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:02.987 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.945087+0000", 2024-09-04T17:46:02.987 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:02.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.988 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:02.988 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.945089+0000", 2024-09-04T17:46:02.988 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.784e-06 2024-09-04T17:46:02.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.988 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:02.988 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.945109+0000", 2024-09-04T17:46:02.988 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0088000000000001e-05 2024-09-04T17:46:02.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.989 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.989 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:02.989 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.945130+0000", 2024-09-04T17:46:02.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1345e-05 2024-09-04T17:46:02.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.989 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.989 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:02.989 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.945140+0000", 2024-09-04T17:46:02.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.499e-06 2024-09-04T17:46:02.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.989 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.989 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:02.989 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.945208+0000", 2024-09-04T17:46:02.990 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8038000000000001e-05 2024-09-04T17:46:02.990 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.990 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.990 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:02.990 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.945234+0000", 2024-09-04T17:46:02.990 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5979e-05 2024-09-04T17:46:02.990 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.990 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.990 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:02.990 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.945941+0000", 2024-09-04T17:46:02.990 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00070737999999999997 2024-09-04T17:46:02.990 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.990 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.991 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:02.991 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.945985+0000", 2024-09-04T17:46:02.991 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4181000000000003e-05 2024-09-04T17:46:02.991 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:02.991 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:02.991 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:02.991 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.991 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.991 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:951 3.0 e28/24 3:02897c52:::rbd_data.11346251f094.0000000000000163:head v 28'5, mlcod=28'3)", 2024-09-04T17:46:02.991 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:01.976152+0000", 2024-09-04T17:46:02.992 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.0073890080000001, 2024-09-04T17:46:02.992 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00066695600000000002, 2024-09-04T17:46:02.992 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:02.992 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:02.992 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:02.992 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:02.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.992 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:02.992 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.976152+0000", 2024-09-04T17:46:02.992 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:02.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:02.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.976152+0000", 2024-09-04T17:46:02.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:02.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:02.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.976154+0000", 2024-09-04T17:46:02.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.635e-06 2024-09-04T17:46:02.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:02.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.976165+0000", 2024-09-04T17:46:02.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1389000000000001e-05 2024-09-04T17:46:02.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:02.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.976187+0000", 2024-09-04T17:46:02.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1361000000000001e-05 2024-09-04T17:46:02.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:02.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.976197+0000", 2024-09-04T17:46:02.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8689999999999999e-06 2024-09-04T17:46:02.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.995 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:02.995 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.976268+0000", 2024-09-04T17:46:02.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0900000000000002e-05 2024-09-04T17:46:02.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.995 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:02.995 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.976292+0000", 2024-09-04T17:46:02.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4494e-05 2024-09-04T17:46:02.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.995 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:02.995 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.976788+0000", 2024-09-04T17:46:02.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00049621699999999997 2024-09-04T17:46:02.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.996 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:02.996 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.976819+0000", 2024-09-04T17:46:02.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1090999999999998e-05 2024-09-04T17:46:02.996 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:02.996 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:02.996 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:02.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.997 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:1050 3.4 e28/24 3:237c6c21:::rbd_data.11346251f094.0000000000000012:head v 28'2, mlcod=0'0)", 2024-09-04T17:46:02.997 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:02.150709+0000", 2024-09-04T17:46:02.997 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.83283281399999998, 2024-09-04T17:46:02.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0013452329999999999, 2024-09-04T17:46:02.997 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:02.997 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:02.997 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:02.997 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:02.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:02.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.150709+0000", 2024-09-04T17:46:02.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:02.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:02.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.150709+0000", 2024-09-04T17:46:02.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:02.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:02.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.150710+0000", 2024-09-04T17:46:02.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8169999999999999e-06 2024-09-04T17:46:02.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:02.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.150753+0000", 2024-09-04T17:46:02.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2710000000000003e-05 2024-09-04T17:46:02.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:02.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:02.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:02.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.150772+0000", 2024-09-04T17:46:02.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9119000000000001e-05 2024-09-04T17:46:03.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:03.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.150785+0000", 2024-09-04T17:46:03.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2744e-05 2024-09-04T17:46:03.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:03.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.150853+0000", 2024-09-04T17:46:03.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8005999999999999e-05 2024-09-04T17:46:03.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:03.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.150875+0000", 2024-09-04T17:46:03.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2004000000000001e-05 2024-09-04T17:46:03.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:03.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.151966+0000", 2024-09-04T17:46:03.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0010910340000000001 2024-09-04T17:46:03.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:03.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.152054+0000", 2024-09-04T17:46:03.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7799000000000004e-05 2024-09-04T17:46:03.002 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:03.002 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:03.002 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:03.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.002 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:1098 3.9 e28/24 3:91683f21:::rbd_data.11346251f094.0000000000000050:head v 28'78, mlcod=28'76)", 2024-09-04T17:46:03.002 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:02.277164+0000", 2024-09-04T17:46:03.002 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.70637753299999995, 2024-09-04T17:46:03.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019986840000000001, 2024-09-04T17:46:03.002 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:03.002 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:03.003 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:03.003 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:03.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:03.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.277164+0000", 2024-09-04T17:46:03.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:03.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:03.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.277164+0000", 2024-09-04T17:46:03.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:03.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:03.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.277166+0000", 2024-09-04T17:46:03.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.962e-06 2024-09-04T17:46:03.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:03.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.277403+0000", 2024-09-04T17:46:03.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0002376 2024-09-04T17:46:03.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:03.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.277426+0000", 2024-09-04T17:46:03.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2478000000000001e-05 2024-09-04T17:46:03.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:03.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.277440+0000", 2024-09-04T17:46:03.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3725e-05 2024-09-04T17:46:03.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:03.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.277561+0000", 2024-09-04T17:46:03.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000120988 2024-09-04T17:46:03.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.006 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:03.006 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.277602+0000", 2024-09-04T17:46:03.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1130000000000001e-05 2024-09-04T17:46:03.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.006 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:03.006 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.279056+0000", 2024-09-04T17:46:03.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014545350000000001 2024-09-04T17:46:03.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:03.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.279163+0000", 2024-09-04T17:46:03.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106266 2024-09-04T17:46:03.007 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:03.007 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:03.007 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:03.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.007 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:1146 3.9 e28/24 3:97509b58:::rbd_data.11346251f094.000000000000002e:head v 28'79, mlcod=28'77)", 2024-09-04T17:46:03.007 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:02.371990+0000", 2024-09-04T17:46:03.007 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.61155192199999997, 2024-09-04T17:46:03.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0010613860000000001, 2024-09-04T17:46:03.007 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:03.008 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:03.008 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:03.008 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:03.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:03.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.371990+0000", 2024-09-04T17:46:03.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:03.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:03.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.371990+0000", 2024-09-04T17:46:03.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:03.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:03.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.371991+0000", 2024-09-04T17:46:03.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4980000000000001e-06 2024-09-04T17:46:03.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:03.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.372075+0000", 2024-09-04T17:46:03.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4268e-05 2024-09-04T17:46:03.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:03.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.372091+0000", 2024-09-04T17:46:03.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5554e-05 2024-09-04T17:46:03.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:03.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.372098+0000", 2024-09-04T17:46:03.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8950000000000001e-06 2024-09-04T17:46:03.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:03.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.372166+0000", 2024-09-04T17:46:03.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8298000000000002e-05 2024-09-04T17:46:03.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:03.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.372188+0000", 2024-09-04T17:46:03.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1708e-05 2024-09-04T17:46:03.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:03.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.372998+0000", 2024-09-04T17:46:03.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00081050399999999998 2024-09-04T17:46:03.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:03.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.373051+0000", 2024-09-04T17:46:03.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2661000000000003e-05 2024-09-04T17:46:03.012 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:03.012 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:03.012 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:03.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.012 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:1174 3.9 e28/24 3:91683f21:::rbd_data.11346251f094.0000000000000050:head v 28'80, mlcod=28'78)", 2024-09-04T17:46:03.012 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:02.412870+0000", 2024-09-04T17:46:03.012 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.57067137700000004, 2024-09-04T17:46:03.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00101965, 2024-09-04T17:46:03.012 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:03.013 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:03.013 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:03.013 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:03.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:03.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.412870+0000", 2024-09-04T17:46:03.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:03.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:03.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.412870+0000", 2024-09-04T17:46:03.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:03.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:03.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.412872+0000", 2024-09-04T17:46:03.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.59e-06 2024-09-04T17:46:03.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:03.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.412882+0000", 2024-09-04T17:46:03.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0647e-05 2024-09-04T17:46:03.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:03.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.412901+0000", 2024-09-04T17:46:03.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8461000000000002e-05 2024-09-04T17:46:03.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.016 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:03.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.412909+0000", 2024-09-04T17:46:03.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2779999999999993e-06 2024-09-04T17:46:03.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.016 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:03.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.412978+0000", 2024-09-04T17:46:03.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8573999999999998e-05 2024-09-04T17:46:03.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:03.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.412997+0000", 2024-09-04T17:46:03.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9173000000000001e-05 2024-09-04T17:46:03.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:03.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.413836+0000", 2024-09-04T17:46:03.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00083885899999999996 2024-09-04T17:46:03.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:03.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.413890+0000", 2024-09-04T17:46:03.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4067999999999998e-05 2024-09-04T17:46:03.018 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:03.018 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:03.018 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:03.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.018 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:1199 3.0 e28/24 3:00cc0f40:::rbd_data.11346251f094.0000000000000075:head v 28'6, mlcod=28'4)", 2024-09-04T17:46:03.018 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:02.448872+0000", 2024-09-04T17:46:03.018 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.53466922500000003, 2024-09-04T17:46:03.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001200069, 2024-09-04T17:46:03.018 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:03.018 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:03.018 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:03.019 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:03.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:03.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.448872+0000", 2024-09-04T17:46:03.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:03.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:03.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.448872+0000", 2024-09-04T17:46:03.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:03.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:03.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.448874+0000", 2024-09-04T17:46:03.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5910000000000001e-06 2024-09-04T17:46:03.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:03.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.448896+0000", 2024-09-04T17:46:03.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2543000000000001e-05 2024-09-04T17:46:03.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:03.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.448914+0000", 2024-09-04T17:46:03.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7368999999999999e-05 2024-09-04T17:46:03.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.021 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:03.021 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.448922+0000", 2024-09-04T17:46:03.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8019999999999992e-06 2024-09-04T17:46:03.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.021 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:03.021 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.449048+0000", 2024-09-04T17:46:03.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012616299999999999 2024-09-04T17:46:03.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.021 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:03.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.449124+0000", 2024-09-04T17:46:03.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6468e-05 2024-09-04T17:46:03.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:03.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.450035+0000", 2024-09-04T17:46:03.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00091072600000000005 2024-09-04T17:46:03.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:03.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.450072+0000", 2024-09-04T17:46:03.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7407000000000003e-05 2024-09-04T17:46:03.023 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:03.023 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:03.023 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:03.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.023 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:1370 3.4 e28/24 3:22db5e3d:::rbd_data.11346251f094.0000000000000174:head v 28'3, mlcod=28'1)", 2024-09-04T17:46:03.023 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:02.686754+0000", 2024-09-04T17:46:03.023 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.29678699400000003, 2024-09-04T17:46:03.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001330258, 2024-09-04T17:46:03.023 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:03.023 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:03.023 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:03.023 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:03.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:03.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.686754+0000", 2024-09-04T17:46:03.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:03.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:03.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.686754+0000", 2024-09-04T17:46:03.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:03.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:03.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.686757+0000", 2024-09-04T17:46:03.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5330000000000001e-06 2024-09-04T17:46:03.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:03.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.686820+0000", 2024-09-04T17:46:03.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3526000000000001e-05 2024-09-04T17:46:03.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:03.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.686859+0000", 2024-09-04T17:46:03.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8260000000000003e-05 2024-09-04T17:46:03.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:03.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.686881+0000", 2024-09-04T17:46:03.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2549999999999999e-05 2024-09-04T17:46:03.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:03.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.686990+0000", 2024-09-04T17:46:03.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010919 2024-09-04T17:46:03.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:03.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.687017+0000", 2024-09-04T17:46:03.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6137999999999999e-05 2024-09-04T17:46:03.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:03.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.688024+0000", 2024-09-04T17:46:03.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0010077720000000001 2024-09-04T17:46:03.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:03.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.688085+0000", 2024-09-04T17:46:03.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0288999999999998e-05 2024-09-04T17:46:03.027 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:03.027 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:03.027 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:03.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.028 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:1377 3.4 e28/24 3:22db5e3d:::rbd_data.11346251f094.0000000000000174:head v 28'4, mlcod=28'2)", 2024-09-04T17:46:03.028 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:02.716364+0000", 2024-09-04T17:46:03.028 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.26717776700000001, 2024-09-04T17:46:03.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00083559200000000002, 2024-09-04T17:46:03.028 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:03.028 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:03.028 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:03.028 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:03.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:03.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.716364+0000", 2024-09-04T17:46:03.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:03.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:03.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.716364+0000", 2024-09-04T17:46:03.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:03.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:03.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.716365+0000", 2024-09-04T17:46:03.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5969999999999999e-06 2024-09-04T17:46:03.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:03.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.716375+0000", 2024-09-04T17:46:03.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5419999999999994e-06 2024-09-04T17:46:03.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:03.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.716401+0000", 2024-09-04T17:46:03.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6380000000000002e-05 2024-09-04T17:46:03.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:03.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.716412+0000", 2024-09-04T17:46:03.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0793e-05 2024-09-04T17:46:03.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:03.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.716489+0000", 2024-09-04T17:46:03.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6884000000000005e-05 2024-09-04T17:46:03.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:03.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.716515+0000", 2024-09-04T17:46:03.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.639e-05 2024-09-04T17:46:03.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:03.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.717131+0000", 2024-09-04T17:46:03.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00061580700000000001 2024-09-04T17:46:03.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:03.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:03.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:03.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.717199+0000", 2024-09-04T17:46:03.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8199000000000002e-05 2024-09-04T17:46:03.032 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:03.032 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:03.032 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:03.032 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:03.032 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:03.032 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:03.033 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-04T17:46:03.034 INFO:teuthology.orchestra.run.smithi092.stdout:438 writesame 0xbbb4672 thru 0xbbb893b data_size 0x53(0x42ca bytes) 2024-09-04T17:46:03.034 INFO:teuthology.orchestra.run.smithi092.stdout:439 read 0xa0b288b thru 0xa0b33d6 (0xb4c bytes) 2024-09-04T17:46:03.034 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:03.034 INFO:teuthology.orchestra.run.smithi092.stdout:441 write 0xdea5c5e thru 0xdeabcef (0x6092 bytes) 2024-09-04T17:46:03.035 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-04T17:46:03.036 INFO:teuthology.orchestra.run.smithi092.stdout:442 read 0x80df84d thru 0x80e81dc (0x8990 bytes) 2024-09-04T17:46:03.036 INFO:teuthology.orchestra.run.smithi092.stdout:446 trunc from 0xdeabcf0 to 0xca7bcfb 2024-09-04T17:46:03.036 INFO:teuthology.orchestra.run.smithi092.stdout:447 write 0x7819897 thru 0x781b6e1 (0x1e4b bytes) 2024-09-04T17:46:03.036 INFO:teuthology.orchestra.run.smithi092.stdout:449 read 0x1823dc0 thru 0x182a703 (0x6944 bytes) 2024-09-04T17:46:03.036 INFO:teuthology.orchestra.run.smithi092.stdout:450 write 0x939cd93 thru 0x93a08b3 (0x3b21 bytes) 2024-09-04T17:46:03.041 INFO:teuthology.orchestra.run.smithi092.stdout:451 writesame 0x8f62588 thru 0x8f6eeed data_size 0x27(0xc966 bytes) 2024-09-04T17:46:03.058 INFO:teuthology.orchestra.run.smithi092.stdout:452 trunc from 0xca7bcfb to 0x837cf09 2024-09-04T17:46:03.059 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:03.058+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:03.086 INFO:teuthology.orchestra.run.smithi092.stdout:453 write 0xb5e261a thru 0xb5efd71 (0xd758 bytes) 2024-09-04T17:46:03.095 INFO:teuthology.orchestra.run.smithi092.stdout:455 trunc from 0xb5efd72 to 0xe51262e 2024-09-04T17:46:03.103 INFO:teuthology.orchestra.run.smithi092.stdout:456 write 0x4159d84 thru 0x4162f7c (0x91f9 bytes) 2024-09-04T17:46:03.109 INFO:teuthology.orchestra.run.smithi092.stdout:458 write 0x72d6d69 thru 0x72e69cc (0xfc64 bytes) 2024-09-04T17:46:03.114 INFO:teuthology.orchestra.run.smithi092.stdout:459 write 0x972d8a3 thru 0x973a58b (0xcce9 bytes) 2024-09-04T17:46:03.118 INFO:teuthology.orchestra.run.smithi092.stdout:461 write 0x311b1ff thru 0x311dc05 (0x2a07 bytes) 2024-09-04T17:46:03.121 INFO:teuthology.orchestra.run.smithi092.stdout:464 write 0xbb244da thru 0xbb2615a (0x1c81 bytes) 2024-09-04T17:46:03.125 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:03.125 INFO:teuthology.orchestra.run.smithi092.stdout:466 write 0xd468de4 thru 0xd46f863 (0x6a80 bytes) 2024-09-04T17:46:03.129 INFO:teuthology.orchestra.run.smithi092.stdout:467 write 0x61b02ab thru 0x61b0afc (0x852 bytes) 2024-09-04T17:46:03.133 INFO:teuthology.orchestra.run.smithi092.stdout:468 read 0x372cf64 thru 0x373241b (0x54b8 bytes) 2024-09-04T17:46:03.133 INFO:teuthology.orchestra.run.smithi092.stdout:469 writesame 0x4c3252f thru 0x4c33714 data_size 0x2(0x11e6 bytes) 2024-09-04T17:46:03.137 INFO:teuthology.orchestra.run.smithi092.stdout:470 read 0x11e1812 thru 0x11eb49d (0x9c8c bytes) 2024-09-04T17:46:03.137 INFO:teuthology.orchestra.run.smithi092.stdout:471 writesame 0x8ac9517 thru 0x8acf3e7 data_size 0x57(0x5ed1 bytes) 2024-09-04T17:46:03.142 INFO:teuthology.orchestra.run.smithi092.stdout:472 read 0xbe5c220 thru 0xbe61b89 (0x596a bytes) 2024-09-04T17:46:03.142 INFO:teuthology.orchestra.run.smithi092.stdout:473 read 0x8fd7e63 thru 0x8fe28f4 (0xaa92 bytes) 2024-09-04T17:46:03.151 INFO:teuthology.orchestra.run.smithi092.stdout:474 trunc from 0xe51262e to 0x796350e 2024-09-04T17:46:03.160 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:03.160+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:03.166 INFO:teuthology.orchestra.run.smithi092.stdout:475 writesame 0x37b673d thru 0x37c2063 data_size 0x1f(0xb927 bytes) 2024-09-04T17:46:03.172 INFO:teuthology.orchestra.run.smithi092.stdout:476 read 0x39476b7 thru 0x394bfee (0x4938 bytes) 2024-09-04T17:46:03.172 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:03.180 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:03.180 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:46:03.180 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:46:03.180 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:03.186 INFO:teuthology.orchestra.run.smithi092.stdout:478 write 0xc8bab6f thru 0xc8c7620 (0xcab2 bytes) 2024-09-04T17:46:03.195 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-04T17:46:03.200 INFO:teuthology.orchestra.run.smithi092.stdout:479 writesame 0xe68f1e8 thru 0xe69b725 data_size 0x1(0xc53e bytes) 2024-09-04T17:46:03.200 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:03.201 INFO:teuthology.orchestra.run.smithi092.stdout:483 read 0x92cf25 thru 0x93685e (0x993a bytes) 2024-09-04T17:46:03.206 INFO:teuthology.orchestra.run.smithi092.stdout:484 trunc from 0xe69b726 to 0xa0ba343 2024-09-04T17:46:03.217 INFO:teuthology.orchestra.run.smithi092.stdout:486 clone 2 order 25 su 4194304 sc 7 2024-09-04T17:46:03.283 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:03.284+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:03.361 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-09-04T17:46:03.362 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:03.362+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:03.364 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:03.364 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:46:03.364 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:46:03.365 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:46:03.365 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:03.379 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-04T17:46:03.471 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:03.472+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:03.550 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:03.550 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:46:03.550 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:46:03.550 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:46:03.550 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.550 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:7 3.1a 3:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-04T17:46:03.550 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:45:56.949499+0000", 2024-09-04T17:46:03.550 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 6.5996242919999997, 2024-09-04T17:46:03.550 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0062683419999999997, 2024-09-04T17:46:03.550 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:03.550 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:03.550 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:03.550 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:03.551 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:03.551 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:03.551 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 7 2024-09-04T17:46:03.551 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.551 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:03.551 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.551 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:03.551 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:56.949499+0000", 2024-09-04T17:46:03.551 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:03.551 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.551 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.551 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:03.551 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:56.949499+0000", 2024-09-04T17:46:03.551 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:03.552 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.552 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.552 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:03.552 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:56.949503+0000", 2024-09-04T17:46:03.552 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.8369999999999996e-06 2024-09-04T17:46:03.552 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.552 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.552 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:03.552 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:56.949519+0000", 2024-09-04T17:46:03.552 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6133e-05 2024-09-04T17:46:03.552 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.553 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.553 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:03.553 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:56.949561+0000", 2024-09-04T17:46:03.553 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.1489999999999997e-05 2024-09-04T17:46:03.553 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.553 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.553 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:03.553 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:56.949588+0000", 2024-09-04T17:46:03.553 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.7083000000000001e-05 2024-09-04T17:46:03.553 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.553 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.553 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:03.554 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:56.949727+0000", 2024-09-04T17:46:03.554 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013893600000000001 2024-09-04T17:46:03.554 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.554 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.554 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op must wait for map", 2024-09-04T17:46:03.554 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:56.949741+0000", 2024-09-04T17:46:03.554 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4239e-05 2024-09-04T17:46:03.554 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.554 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.554 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:03.554 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:56.952095+0000", 2024-09-04T17:46:03.554 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.002353946 2024-09-04T17:46:03.554 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.554 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.555 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:03.555 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:56.952263+0000", 2024-09-04T17:46:03.555 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00016754899999999999 2024-09-04T17:46:03.555 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.555 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.555 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 1", 2024-09-04T17:46:03.555 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:56.952471+0000", 2024-09-04T17:46:03.555 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020808400000000001 2024-09-04T17:46:03.555 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.555 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.555 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:03.555 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:56.953704+0000", 2024-09-04T17:46:03.556 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.001233311 2024-09-04T17:46:03.556 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.556 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.556 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:03.556 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:56.955566+0000", 2024-09-04T17:46:03.556 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0018616290000000001 2024-09-04T17:46:03.556 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.556 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.556 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:03.556 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:56.955652+0000", 2024-09-04T17:46:03.556 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.6149000000000005e-05 2024-09-04T17:46:03.556 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.557 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.557 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:03.557 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:56.955767+0000", 2024-09-04T17:46:03.557 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000114956 2024-09-04T17:46:03.557 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:03.557 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:03.557 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:03.557 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.557 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.557 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:9 3.1a 3:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-04T17:46:03.557 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:45:57.965066+0000", 2024-09-04T17:46:03.557 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 5.5840570960000004, 2024-09-04T17:46:03.557 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0074753450000000004, 2024-09-04T17:46:03.558 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:03.558 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:03.558 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:03.558 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:03.558 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:03.558 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:03.558 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 9 2024-09-04T17:46:03.558 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.558 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:03.558 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.558 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:03.558 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:57.965066+0000", 2024-09-04T17:46:03.558 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:03.558 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.559 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.559 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:03.559 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:57.965066+0000", 2024-09-04T17:46:03.559 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:03.559 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.559 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.559 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:03.559 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:57.965069+0000", 2024-09-04T17:46:03.559 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.2100000000000002e-06 2024-09-04T17:46:03.559 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.559 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.560 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:03.560 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:57.965092+0000", 2024-09-04T17:46:03.560 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2665999999999999e-05 2024-09-04T17:46:03.560 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.560 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.560 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:03.560 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:57.965123+0000", 2024-09-04T17:46:03.560 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.1127000000000002e-05 2024-09-04T17:46:03.560 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.560 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.560 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:03.560 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:57.965141+0000", 2024-09-04T17:46:03.560 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7782999999999999e-05 2024-09-04T17:46:03.561 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.561 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.561 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:03.561 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:57.965259+0000", 2024-09-04T17:46:03.561 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000118466 2024-09-04T17:46:03.561 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.561 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.561 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op must wait for map", 2024-09-04T17:46:03.561 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:57.965270+0000", 2024-09-04T17:46:03.561 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0693000000000001e-05 2024-09-04T17:46:03.561 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.561 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.562 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:03.562 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:57.969607+0000", 2024-09-04T17:46:03.562 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0043376129999999997 2024-09-04T17:46:03.562 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.562 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.562 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:03.562 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:57.969742+0000", 2024-09-04T17:46:03.562 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013452900000000001 2024-09-04T17:46:03.562 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.562 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.562 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 1", 2024-09-04T17:46:03.562 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:57.969978+0000", 2024-09-04T17:46:03.562 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000235701 2024-09-04T17:46:03.562 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.563 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.563 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:03.563 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:57.971977+0000", 2024-09-04T17:46:03.563 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0019995389999999998 2024-09-04T17:46:03.563 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.563 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.563 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:03.563 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:57.972229+0000", 2024-09-04T17:46:03.564 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00025209800000000002 2024-09-04T17:46:03.564 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.564 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.564 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:03.564 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:57.972421+0000", 2024-09-04T17:46:03.564 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019196799999999999 2024-09-04T17:46:03.564 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.564 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.564 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:03.564 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:57.972541+0000", 2024-09-04T17:46:03.564 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00011995200000000001 2024-09-04T17:46:03.564 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:03.564 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:03.565 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:03.565 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.565 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.565 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:980 3.3 3:c3527a1c:::rbd_data.11346251f094.0000000000000136:head [set-alloc-hint object_size 524288 write_size 524288,write 235416~26728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-04T17:46:03.565 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:02.035572+0000", 2024-09-04T17:46:03.565 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 1.5135509819999999, 2024-09-04T17:46:03.565 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0029568799999999998, 2024-09-04T17:46:03.565 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:03.565 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:03.566 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:03.566 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:03.566 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:03.566 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:03.566 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 980 2024-09-04T17:46:03.566 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.566 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:03.566 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.566 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:03.566 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.035572+0000", 2024-09-04T17:46:03.566 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:03.566 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.566 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.567 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:03.567 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.035572+0000", 2024-09-04T17:46:03.567 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:03.567 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.567 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.567 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:03.567 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.035575+0000", 2024-09-04T17:46:03.567 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.7360000000000001e-06 2024-09-04T17:46:03.567 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.567 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.567 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:03.567 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.035611+0000", 2024-09-04T17:46:03.567 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.6497e-05 2024-09-04T17:46:03.567 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.568 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.568 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:03.568 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.035643+0000", 2024-09-04T17:46:03.568 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.2413000000000003e-05 2024-09-04T17:46:03.568 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.568 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.568 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:03.568 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.035665+0000", 2024-09-04T17:46:03.568 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1552999999999998e-05 2024-09-04T17:46:03.568 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.568 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.568 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:03.568 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.035802+0000", 2024-09-04T17:46:03.569 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013688900000000001 2024-09-04T17:46:03.569 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.569 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.569 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:03.569 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.036049+0000", 2024-09-04T17:46:03.569 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024672499999999999 2024-09-04T17:46:03.569 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.569 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.569 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 0", 2024-09-04T17:46:03.569 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.036274+0000", 2024-09-04T17:46:03.569 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00022568100000000001 2024-09-04T17:46:03.570 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.570 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.570 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:03.570 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.038121+0000", 2024-09-04T17:46:03.570 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.001846386 2024-09-04T17:46:03.570 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.570 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.570 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:03.570 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.038226+0000", 2024-09-04T17:46:03.570 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000104929 2024-09-04T17:46:03.570 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.570 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.570 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:03.571 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.038401+0000", 2024-09-04T17:46:03.571 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00017493200000000001 2024-09-04T17:46:03.571 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.571 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.571 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:03.571 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.038529+0000", 2024-09-04T17:46:03.571 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00012813899999999999 2024-09-04T17:46:03.571 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:03.571 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:03.571 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:03.571 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.571 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.571 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:984 3.14 3:2ebf0f02:::rbd_data.11346251f094.0000000000000111:head [set-alloc-hint object_size 524288 write_size 524288,write 523742~546] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-04T17:46:03.572 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:02.043525+0000", 2024-09-04T17:46:03.572 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 1.5055974569999999, 2024-09-04T17:46:03.572 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0037627139999999999, 2024-09-04T17:46:03.572 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:03.572 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:03.572 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:03.572 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:03.572 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:03.572 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:03.572 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 984 2024-09-04T17:46:03.572 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.572 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:03.572 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.573 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:03.573 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.043525+0000", 2024-09-04T17:46:03.573 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:03.573 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.573 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.573 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:03.573 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.043525+0000", 2024-09-04T17:46:03.573 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:03.573 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.573 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.573 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:03.573 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.043528+0000", 2024-09-04T17:46:03.574 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.3659999999999999e-06 2024-09-04T17:46:03.574 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.574 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.574 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:03.574 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.043539+0000", 2024-09-04T17:46:03.574 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1117e-05 2024-09-04T17:46:03.574 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.574 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.574 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:03.574 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.043568+0000", 2024-09-04T17:46:03.574 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.9062999999999999e-05 2024-09-04T17:46:03.574 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.575 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.575 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:03.575 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.043583+0000", 2024-09-04T17:46:03.575 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5106000000000001e-05 2024-09-04T17:46:03.575 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.575 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.575 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:03.575 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.043711+0000", 2024-09-04T17:46:03.575 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000127873 2024-09-04T17:46:03.575 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.576 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.576 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:03.576 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.044071+0000", 2024-09-04T17:46:03.576 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00036043700000000002 2024-09-04T17:46:03.576 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.576 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.576 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 7", 2024-09-04T17:46:03.576 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.044341+0000", 2024-09-04T17:46:03.576 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00026988099999999997 2024-09-04T17:46:03.576 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.576 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.576 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:03.577 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.045650+0000", 2024-09-04T17:46:03.577 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.001308783 2024-09-04T17:46:03.577 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.577 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.577 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:03.577 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.047109+0000", 2024-09-04T17:46:03.577 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0014586130000000001 2024-09-04T17:46:03.577 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.577 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.577 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:03.577 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.047217+0000", 2024-09-04T17:46:03.577 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000108105 2024-09-04T17:46:03.578 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.578 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.578 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:03.578 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.047288+0000", 2024-09-04T17:46:03.578 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1370000000000003e-05 2024-09-04T17:46:03.578 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:03.578 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:03.578 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:03.578 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.578 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.578 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:987 3.14 3:2ebf0f02:::rbd_data.11346251f094.0000000000000111:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-04T17:46:03.578 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:02.060353+0000", 2024-09-04T17:46:03.578 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 1.488769464, 2024-09-04T17:46:03.578 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0039364339999999999, 2024-09-04T17:46:03.579 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:03.579 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:03.579 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:03.579 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:03.579 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:03.579 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:03.579 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 987 2024-09-04T17:46:03.579 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.579 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:03.579 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.579 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:03.579 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.060353+0000", 2024-09-04T17:46:03.579 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:03.580 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.580 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.580 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:03.580 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.060353+0000", 2024-09-04T17:46:03.580 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:03.580 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.580 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.580 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:03.580 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.060356+0000", 2024-09-04T17:46:03.580 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.667e-06 2024-09-04T17:46:03.580 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.581 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.581 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:03.581 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.060367+0000", 2024-09-04T17:46:03.581 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1277000000000001e-05 2024-09-04T17:46:03.581 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.581 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.581 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:03.581 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.060399+0000", 2024-09-04T17:46:03.581 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.2033000000000003e-05 2024-09-04T17:46:03.581 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.581 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.581 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:03.581 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.060415+0000", 2024-09-04T17:46:03.582 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5305999999999999e-05 2024-09-04T17:46:03.582 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.582 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.582 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:03.582 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.060570+0000", 2024-09-04T17:46:03.582 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000155515 2024-09-04T17:46:03.582 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.582 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.582 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:03.582 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.060783+0000", 2024-09-04T17:46:03.582 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021328300000000001 2024-09-04T17:46:03.582 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.582 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.582 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 7", 2024-09-04T17:46:03.583 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.061079+0000", 2024-09-04T17:46:03.583 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00029512400000000002 2024-09-04T17:46:03.583 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.583 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.583 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:03.583 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.063081+0000", 2024-09-04T17:46:03.583 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0020024539999999999 2024-09-04T17:46:03.583 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.583 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.583 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:03.583 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.064134+0000", 2024-09-04T17:46:03.583 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.001052622 2024-09-04T17:46:03.583 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.583 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.584 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:03.584 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.064220+0000", 2024-09-04T17:46:03.584 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.6716000000000001e-05 2024-09-04T17:46:03.584 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.584 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.584 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:03.584 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.064290+0000", 2024-09-04T17:46:03.584 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.9437000000000006e-05 2024-09-04T17:46:03.584 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:03.584 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:03.584 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:03.584 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.584 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.585 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:989 3.3 3:c3527a1c:::rbd_data.11346251f094.0000000000000136:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-04T17:46:03.585 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:02.060449+0000", 2024-09-04T17:46:03.585 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 1.4886741050000001, 2024-09-04T17:46:03.585 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.004621895, 2024-09-04T17:46:03.585 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:03.585 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:03.585 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:03.585 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:03.585 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:03.585 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:03.585 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 989 2024-09-04T17:46:03.585 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.585 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:03.586 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.586 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:03.586 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.060449+0000", 2024-09-04T17:46:03.586 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:03.586 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.586 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.586 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:03.586 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.060449+0000", 2024-09-04T17:46:03.586 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:03.586 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.586 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.586 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:03.586 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.060450+0000", 2024-09-04T17:46:03.587 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.004e-06 2024-09-04T17:46:03.587 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.587 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.587 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:03.587 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.060454+0000", 2024-09-04T17:46:03.587 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.5730000000000001e-06 2024-09-04T17:46:03.587 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.587 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.587 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:03.587 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.060466+0000", 2024-09-04T17:46:03.587 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1912999999999999e-05 2024-09-04T17:46:03.587 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.587 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.587 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:03.588 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.060475+0000", 2024-09-04T17:46:03.588 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8440000000000004e-06 2024-09-04T17:46:03.588 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.588 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.588 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:03.588 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.060590+0000", 2024-09-04T17:46:03.588 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00011479199999999999 2024-09-04T17:46:03.588 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.588 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.588 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:03.588 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.060737+0000", 2024-09-04T17:46:03.588 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014715900000000001 2024-09-04T17:46:03.588 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.588 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.588 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 0", 2024-09-04T17:46:03.589 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.060935+0000", 2024-09-04T17:46:03.589 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019801199999999999 2024-09-04T17:46:03.589 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.589 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.589 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:03.589 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.062378+0000", 2024-09-04T17:46:03.589 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0014430160000000001 2024-09-04T17:46:03.589 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.589 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.589 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:03.589 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.064874+0000", 2024-09-04T17:46:03.589 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0024955569999999998 2024-09-04T17:46:03.589 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.590 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.590 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:03.590 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.064959+0000", 2024-09-04T17:46:03.590 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.5488999999999994e-05 2024-09-04T17:46:03.590 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.590 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.590 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:03.590 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.065071+0000", 2024-09-04T17:46:03.590 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000111536 2024-09-04T17:46:03.590 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:03.590 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:03.590 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:03.591 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.591 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.591 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:1014 3.13 e28/24 3:cc76826e:::rbd_data.11346251f094.000000000000001a:head v 28'6, mlcod=28'4)", 2024-09-04T17:46:03.591 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:02.093683+0000", 2024-09-04T17:46:03.591 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 1.4554396709999999, 2024-09-04T17:46:03.591 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0025949229999999998, 2024-09-04T17:46:03.591 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:03.591 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:03.591 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:03.591 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:03.591 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.591 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:03.591 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.093683+0000", 2024-09-04T17:46:03.591 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:03.592 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.592 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.592 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:03.592 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.093683+0000", 2024-09-04T17:46:03.592 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:03.592 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.592 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.592 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:03.592 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.093685+0000", 2024-09-04T17:46:03.592 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0540000000000002e-06 2024-09-04T17:46:03.592 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.592 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.592 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:03.593 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.093848+0000", 2024-09-04T17:46:03.593 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00016261800000000001 2024-09-04T17:46:03.593 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.593 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.593 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:03.593 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.093889+0000", 2024-09-04T17:46:03.593 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.1529999999999997e-05 2024-09-04T17:46:03.593 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.593 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.593 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:03.593 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.093909+0000", 2024-09-04T17:46:03.593 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9423e-05 2024-09-04T17:46:03.593 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.594 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.594 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:03.594 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.094072+0000", 2024-09-04T17:46:03.594 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00016274599999999999 2024-09-04T17:46:03.594 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.594 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.594 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:03.594 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.094137+0000", 2024-09-04T17:46:03.594 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.5499000000000004e-05 2024-09-04T17:46:03.594 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.594 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.594 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:03.595 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.096167+0000", 2024-09-04T17:46:03.595 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0020303809999999999 2024-09-04T17:46:03.595 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.595 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.595 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:03.595 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.096278+0000", 2024-09-04T17:46:03.595 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000110672 2024-09-04T17:46:03.595 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:03.595 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:03.595 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:03.595 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.595 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.595 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:1090 3.3 3:c7450f4b:::rbd_data.11346251f094.00000000000001b3:head [set-alloc-hint object_size 524288 write_size 524288,write 11273~40778] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-04T17:46:03.596 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:02.210413+0000", 2024-09-04T17:46:03.596 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 1.3387098989999999, 2024-09-04T17:46:03.596 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0032587520000000002, 2024-09-04T17:46:03.596 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:03.596 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:03.596 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:03.596 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:03.596 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:03.596 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:03.596 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 1090 2024-09-04T17:46:03.596 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.596 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:03.597 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.597 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:03.597 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.210413+0000", 2024-09-04T17:46:03.597 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:03.597 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.597 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.597 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:03.597 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.210413+0000", 2024-09-04T17:46:03.597 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:03.597 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.597 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.597 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:03.598 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.210415+0000", 2024-09-04T17:46:03.598 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5500000000000001e-06 2024-09-04T17:46:03.598 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.598 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.598 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:03.598 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.210470+0000", 2024-09-04T17:46:03.598 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4450000000000002e-05 2024-09-04T17:46:03.598 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.598 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.598 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:03.598 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.210502+0000", 2024-09-04T17:46:03.598 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.2103000000000001e-05 2024-09-04T17:46:03.598 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.598 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.599 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:03.599 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.210517+0000", 2024-09-04T17:46:03.599 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.52e-05 2024-09-04T17:46:03.599 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.599 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.599 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:03.599 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.210664+0000", 2024-09-04T17:46:03.599 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014691899999999999 2024-09-04T17:46:03.599 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.599 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.599 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:03.599 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.211038+0000", 2024-09-04T17:46:03.599 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00037426000000000001 2024-09-04T17:46:03.600 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.600 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.600 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 0", 2024-09-04T17:46:03.600 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.211456+0000", 2024-09-04T17:46:03.600 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00041775999999999998 2024-09-04T17:46:03.600 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.600 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.600 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:03.600 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.212916+0000", 2024-09-04T17:46:03.600 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0014595260000000001 2024-09-04T17:46:03.600 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.600 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.600 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:03.601 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.213499+0000", 2024-09-04T17:46:03.601 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000583009 2024-09-04T17:46:03.601 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.601 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.601 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:03.601 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.213590+0000", 2024-09-04T17:46:03.601 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1075999999999993e-05 2024-09-04T17:46:03.601 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.601 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.601 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:03.601 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.213672+0000", 2024-09-04T17:46:03.601 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.1898999999999996e-05 2024-09-04T17:46:03.601 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:03.602 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:03.602 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:03.602 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.602 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.602 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:1095 3.1 e28/24 3:86c08994:::rbd_data.11346251f094.000000000000006e:head v 28'8, mlcod=28'6)", 2024-09-04T17:46:03.602 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:02.255028+0000", 2024-09-04T17:46:03.602 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 1.294095089, 2024-09-04T17:46:03.602 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0041319540000000002, 2024-09-04T17:46:03.602 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:03.602 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:03.602 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:03.602 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:03.602 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.603 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:03.603 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.255028+0000", 2024-09-04T17:46:03.603 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:03.603 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.603 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.603 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:03.603 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.255028+0000", 2024-09-04T17:46:03.603 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:03.603 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.603 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.603 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:03.603 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.255030+0000", 2024-09-04T17:46:03.603 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9970000000000001e-06 2024-09-04T17:46:03.603 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.604 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.604 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:03.604 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.256825+0000", 2024-09-04T17:46:03.604 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.001794882 2024-09-04T17:46:03.604 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.604 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.604 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:03.604 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.256853+0000", 2024-09-04T17:46:03.604 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.8753000000000001e-05 2024-09-04T17:46:03.604 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.604 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.604 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:03.604 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.256867+0000", 2024-09-04T17:46:03.604 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.343e-05 2024-09-04T17:46:03.605 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.605 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.605 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:03.605 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.257046+0000", 2024-09-04T17:46:03.605 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000179085 2024-09-04T17:46:03.605 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.605 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.605 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:03.605 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.257087+0000", 2024-09-04T17:46:03.605 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.1063999999999999e-05 2024-09-04T17:46:03.605 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.605 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.605 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:03.606 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.259073+0000", 2024-09-04T17:46:03.606 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0019860070000000001 2024-09-04T17:46:03.606 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.606 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.606 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:03.606 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.259160+0000", 2024-09-04T17:46:03.606 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.6736000000000004e-05 2024-09-04T17:46:03.606 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:03.606 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:03.606 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:03.606 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.606 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.606 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:1098 3.9 3:91683f21:::rbd_data.11346251f094.0000000000000050:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-04T17:46:03.607 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:02.275602+0000", 2024-09-04T17:46:03.607 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 1.2735204580000001, 2024-09-04T17:46:03.607 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0036002289999999999, 2024-09-04T17:46:03.607 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:03.607 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:03.607 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:03.607 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:03.607 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:03.607 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:03.607 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 1098 2024-09-04T17:46:03.607 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.607 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:03.607 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.607 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:03.608 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.275602+0000", 2024-09-04T17:46:03.608 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:03.608 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.608 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.608 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:03.608 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.275602+0000", 2024-09-04T17:46:03.608 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:03.608 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.608 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.608 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:03.608 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.275606+0000", 2024-09-04T17:46:03.608 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.2329999999999998e-06 2024-09-04T17:46:03.608 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.609 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.609 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:03.609 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.275618+0000", 2024-09-04T17:46:03.609 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2480000000000001e-05 2024-09-04T17:46:03.609 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.609 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.609 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:03.609 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.275652+0000", 2024-09-04T17:46:03.609 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.3482999999999998e-05 2024-09-04T17:46:03.609 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.609 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.609 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:03.609 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.275674+0000", 2024-09-04T17:46:03.610 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2053e-05 2024-09-04T17:46:03.610 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.610 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.610 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:03.610 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.275861+0000", 2024-09-04T17:46:03.610 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00018788499999999999 2024-09-04T17:46:03.610 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.610 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.610 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:03.610 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.276094+0000", 2024-09-04T17:46:03.610 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00023213899999999999 2024-09-04T17:46:03.610 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.610 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.611 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 2", 2024-09-04T17:46:03.611 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.276412+0000", 2024-09-04T17:46:03.611 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00031810999999999997 2024-09-04T17:46:03.611 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.611 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.611 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:03.611 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.278432+0000", 2024-09-04T17:46:03.611 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0020204009999999998 2024-09-04T17:46:03.611 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.611 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.611 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:03.611 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.279050+0000", 2024-09-04T17:46:03.611 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00061746900000000005 2024-09-04T17:46:03.612 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.612 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.612 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:03.612 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.279136+0000", 2024-09-04T17:46:03.612 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.6142000000000003e-05 2024-09-04T17:46:03.612 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.612 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.612 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:03.612 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.279203+0000", 2024-09-04T17:46:03.612 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6834000000000004e-05 2024-09-04T17:46:03.612 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:03.613 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:03.613 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:03.613 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.613 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.613 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:1103 3.3 3:c7450f4b:::rbd_data.11346251f094.00000000000001b3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-04T17:46:03.613 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:02.293056+0000", 2024-09-04T17:46:03.613 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 1.2560668770000001, 2024-09-04T17:46:03.613 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0045977919999999999, 2024-09-04T17:46:03.613 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:03.614 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:03.614 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:03.614 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:03.614 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:03.614 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:03.614 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 1103 2024-09-04T17:46:03.615 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.615 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:03.615 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.615 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:03.615 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.293056+0000", 2024-09-04T17:46:03.615 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:03.615 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.615 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.615 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:03.615 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.293056+0000", 2024-09-04T17:46:03.616 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:03.616 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.616 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.616 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:03.616 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.293058+0000", 2024-09-04T17:46:03.616 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.357e-06 2024-09-04T17:46:03.616 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.616 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.616 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:03.616 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.293069+0000", 2024-09-04T17:46:03.616 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0783e-05 2024-09-04T17:46:03.616 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.617 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.617 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:03.617 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.293100+0000", 2024-09-04T17:46:03.617 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.082e-05 2024-09-04T17:46:03.617 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.617 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.617 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:03.617 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.293122+0000", 2024-09-04T17:46:03.617 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1650000000000001e-05 2024-09-04T17:46:03.617 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.617 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.617 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:03.617 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.293247+0000", 2024-09-04T17:46:03.618 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000125575 2024-09-04T17:46:03.618 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.618 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.618 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:03.618 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.293389+0000", 2024-09-04T17:46:03.618 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014207299999999999 2024-09-04T17:46:03.618 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.618 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.618 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 0", 2024-09-04T17:46:03.618 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.293561+0000", 2024-09-04T17:46:03.618 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00017203499999999999 2024-09-04T17:46:03.618 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.619 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.619 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:03.619 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.294692+0000", 2024-09-04T17:46:03.619 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.001130268 2024-09-04T17:46:03.619 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.619 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.619 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:03.619 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.297502+0000", 2024-09-04T17:46:03.619 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.002810959 2024-09-04T17:46:03.620 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.620 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.620 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:03.620 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.297588+0000", 2024-09-04T17:46:03.620 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.5096e-05 2024-09-04T17:46:03.620 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.620 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.620 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:03.620 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.297654+0000", 2024-09-04T17:46:03.620 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6175999999999998e-05 2024-09-04T17:46:03.620 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:03.620 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:03.620 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:03.620 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.621 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.621 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:1125 3.1c e28/24 3:3fef72cb:::rbd_data.11346251f094.00000000000000c0:head v 28'12, mlcod=28'10)", 2024-09-04T17:46:03.621 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:02.352419+0000", 2024-09-04T17:46:03.621 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 1.1967041110000001, 2024-09-04T17:46:03.621 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.002665246, 2024-09-04T17:46:03.621 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:03.621 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:03.621 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:03.621 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:03.621 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.621 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:03.621 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.352419+0000", 2024-09-04T17:46:03.621 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:03.622 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.622 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.622 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:03.622 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.352419+0000", 2024-09-04T17:46:03.622 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:03.622 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.622 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.622 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:03.622 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.352421+0000", 2024-09-04T17:46:03.622 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1069999999999999e-06 2024-09-04T17:46:03.622 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.622 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.622 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:03.622 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.352439+0000", 2024-09-04T17:46:03.623 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7733000000000001e-05 2024-09-04T17:46:03.623 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.623 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.623 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:03.623 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.352469+0000", 2024-09-04T17:46:03.623 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0908999999999998e-05 2024-09-04T17:46:03.623 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.623 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.623 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:03.623 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.352491+0000", 2024-09-04T17:46:03.623 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1622999999999999e-05 2024-09-04T17:46:03.623 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.623 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.623 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:03.624 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.352647+0000", 2024-09-04T17:46:03.624 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00015623899999999999 2024-09-04T17:46:03.624 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.624 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.624 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:03.624 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.352708+0000", 2024-09-04T17:46:03.624 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.0256e-05 2024-09-04T17:46:03.624 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.624 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.624 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:03.624 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.355001+0000", 2024-09-04T17:46:03.624 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.002293233 2024-09-04T17:46:03.624 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.624 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.625 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:03.625 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.355084+0000", 2024-09-04T17:46:03.625 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.3146000000000007e-05 2024-09-04T17:46:03.625 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:03.625 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:03.625 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:03.625 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.625 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.625 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:1146 3.9 3:97509b58:::rbd_data.11346251f094.000000000000002e:head [stat,write 263467~51652] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-04T17:46:03.625 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:02.370491+0000", 2024-09-04T17:46:03.625 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 1.178631548, 2024-09-04T17:46:03.625 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0030776200000000001, 2024-09-04T17:46:03.625 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:03.626 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:03.626 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:03.626 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:03.626 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:03.626 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:03.626 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 1146 2024-09-04T17:46:03.626 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.626 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:03.626 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.626 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:03.626 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.370491+0000", 2024-09-04T17:46:03.626 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:03.627 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.627 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.627 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:03.627 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.370491+0000", 2024-09-04T17:46:03.627 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:03.627 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.627 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.627 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:03.627 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.370494+0000", 2024-09-04T17:46:03.627 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4870000000000001e-06 2024-09-04T17:46:03.627 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.627 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.627 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:03.627 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.370551+0000", 2024-09-04T17:46:03.627 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6879999999999998e-05 2024-09-04T17:46:03.628 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.628 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.628 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:03.628 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.370574+0000", 2024-09-04T17:46:03.628 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.3810000000000001e-05 2024-09-04T17:46:03.628 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.628 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.628 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:03.628 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.370587+0000", 2024-09-04T17:46:03.628 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2203e-05 2024-09-04T17:46:03.628 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.628 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.629 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:03.629 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.370742+0000", 2024-09-04T17:46:03.629 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000155172 2024-09-04T17:46:03.629 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.629 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.629 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:03.629 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.371002+0000", 2024-09-04T17:46:03.629 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000260411 2024-09-04T17:46:03.629 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.629 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.629 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 2", 2024-09-04T17:46:03.629 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.371422+0000", 2024-09-04T17:46:03.629 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00041986999999999998 2024-09-04T17:46:03.629 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.630 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.630 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:03.630 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.372978+0000", 2024-09-04T17:46:03.630 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0015559219999999999 2024-09-04T17:46:03.630 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.630 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.630 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:03.630 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.373296+0000", 2024-09-04T17:46:03.630 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00031798699999999999 2024-09-04T17:46:03.630 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.630 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.630 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:03.631 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.373443+0000", 2024-09-04T17:46:03.631 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014718500000000001 2024-09-04T17:46:03.631 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.631 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.631 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:03.631 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.373569+0000", 2024-09-04T17:46:03.631 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00012569299999999999 2024-09-04T17:46:03.631 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:03.631 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:03.631 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:03.631 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.631 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.631 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:1171 3.1 e28/24 3:86c08994:::rbd_data.11346251f094.000000000000006e:head v 28'11, mlcod=28'9)", 2024-09-04T17:46:03.631 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:02.400278+0000", 2024-09-04T17:46:03.632 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 1.148844464, 2024-09-04T17:46:03.632 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0026677659999999998, 2024-09-04T17:46:03.632 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:03.632 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:03.632 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:03.632 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:03.632 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.632 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:03.632 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.400278+0000", 2024-09-04T17:46:03.632 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:03.632 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.632 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.632 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:03.632 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.400278+0000", 2024-09-04T17:46:03.633 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:03.633 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.633 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.633 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:03.633 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.400280+0000", 2024-09-04T17:46:03.633 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.176e-06 2024-09-04T17:46:03.633 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.633 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.633 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:03.633 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.400290+0000", 2024-09-04T17:46:03.633 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0509999999999999e-05 2024-09-04T17:46:03.634 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.634 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.634 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:03.634 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.400310+0000", 2024-09-04T17:46:03.634 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9870000000000001e-05 2024-09-04T17:46:03.634 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.634 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.634 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:03.634 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.400327+0000", 2024-09-04T17:46:03.635 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6674e-05 2024-09-04T17:46:03.635 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.635 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.635 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:03.635 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.400527+0000", 2024-09-04T17:46:03.635 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020035800000000001 2024-09-04T17:46:03.635 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.635 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.635 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:03.635 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.400580+0000", 2024-09-04T17:46:03.635 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2913e-05 2024-09-04T17:46:03.635 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.635 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.636 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:03.636 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.402809+0000", 2024-09-04T17:46:03.636 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0022296059999999999 2024-09-04T17:46:03.636 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.636 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.636 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:03.636 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.402946+0000", 2024-09-04T17:46:03.636 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013665899999999999 2024-09-04T17:46:03.636 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:03.636 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:03.636 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:03.636 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.636 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.637 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:1167 3.e e28/24 3:72a8dc85:::rbd_data.11346251f094.0000000000000060:head v 28'17, mlcod=28'15)", 2024-09-04T17:46:03.637 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:02.401260+0000", 2024-09-04T17:46:03.637 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 1.1478623450000001, 2024-09-04T17:46:03.637 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0043508169999999999, 2024-09-04T17:46:03.637 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:03.637 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:03.637 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:03.637 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:03.637 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.637 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:03.637 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.401260+0000", 2024-09-04T17:46:03.637 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:03.638 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.638 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.638 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:03.638 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.401260+0000", 2024-09-04T17:46:03.638 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:03.638 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.638 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.638 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:03.638 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.401262+0000", 2024-09-04T17:46:03.638 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.973e-06 2024-09-04T17:46:03.638 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.638 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.638 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:03.639 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.401278+0000", 2024-09-04T17:46:03.639 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5274e-05 2024-09-04T17:46:03.639 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.639 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.639 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:03.639 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.401306+0000", 2024-09-04T17:46:03.639 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.8192e-05 2024-09-04T17:46:03.639 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.639 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.639 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:03.639 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.401324+0000", 2024-09-04T17:46:03.640 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7844000000000001e-05 2024-09-04T17:46:03.640 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.640 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.640 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:03.640 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.401484+0000", 2024-09-04T17:46:03.640 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00015974899999999999 2024-09-04T17:46:03.640 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.640 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.640 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:03.640 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.401550+0000", 2024-09-04T17:46:03.640 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6588999999999996e-05 2024-09-04T17:46:03.640 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.640 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.641 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:03.641 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.405483+0000", 2024-09-04T17:46:03.641 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0039331929999999998 2024-09-04T17:46:03.641 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.641 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.641 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:03.641 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.405611+0000", 2024-09-04T17:46:03.641 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00012800299999999999 2024-09-04T17:46:03.641 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:03.641 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:03.641 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:03.641 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.641 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.642 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:1166 3.1c e28/24 3:3ce0fe61:::rbd_data.11346251f094.000000000000005f:head v 28'14, mlcod=28'12)", 2024-09-04T17:46:03.642 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:02.401725+0000", 2024-09-04T17:46:03.642 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 1.147398055, 2024-09-04T17:46:03.642 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0038971169999999999, 2024-09-04T17:46:03.642 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:03.642 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:03.642 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:03.642 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:03.642 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.642 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:03.642 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.401725+0000", 2024-09-04T17:46:03.642 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:03.642 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.643 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.643 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:03.643 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.401725+0000", 2024-09-04T17:46:03.643 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:03.643 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.643 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.643 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:03.643 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.401727+0000", 2024-09-04T17:46:03.643 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4700000000000001e-06 2024-09-04T17:46:03.643 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.643 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.643 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:03.643 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.401742+0000", 2024-09-04T17:46:03.643 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4846e-05 2024-09-04T17:46:03.644 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.644 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.644 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:03.644 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.401774+0000", 2024-09-04T17:46:03.644 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.1609999999999997e-05 2024-09-04T17:46:03.644 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.644 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.644 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:03.644 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.401793+0000", 2024-09-04T17:46:03.644 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9176000000000001e-05 2024-09-04T17:46:03.644 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.644 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.644 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:03.645 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.402000+0000", 2024-09-04T17:46:03.645 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020703599999999999 2024-09-04T17:46:03.645 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.645 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.645 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:03.645 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.402055+0000", 2024-09-04T17:46:03.645 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.5132e-05 2024-09-04T17:46:03.645 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.645 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.645 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:03.645 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.405531+0000", 2024-09-04T17:46:03.646 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.003476397 2024-09-04T17:46:03.646 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.646 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.646 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:03.646 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.405622+0000", 2024-09-04T17:46:03.646 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.0450000000000003e-05 2024-09-04T17:46:03.646 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:03.646 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:03.646 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:03.646 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.647 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.647 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:1257 3.e e28/24 3:7532413d:::rbd_data.11346251f094.0000000000000005:head v 28'18, mlcod=28'16)", 2024-09-04T17:46:03.647 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:02.463267+0000", 2024-09-04T17:46:03.647 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 1.085855843, 2024-09-04T17:46:03.647 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0027257750000000002, 2024-09-04T17:46:03.647 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:03.647 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:03.647 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:03.647 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:03.647 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.647 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:03.647 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.463267+0000", 2024-09-04T17:46:03.647 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:03.648 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.648 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.648 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:03.648 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.463267+0000", 2024-09-04T17:46:03.648 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:03.648 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.648 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.648 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:03.648 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.463269+0000", 2024-09-04T17:46:03.649 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.306e-06 2024-09-04T17:46:03.649 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.649 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.649 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:03.649 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.463312+0000", 2024-09-04T17:46:03.650 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.3066999999999999e-05 2024-09-04T17:46:03.650 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.650 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.650 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:03.650 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.463357+0000", 2024-09-04T17:46:03.650 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4815999999999999e-05 2024-09-04T17:46:03.651 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.651 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.651 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:03.651 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.463378+0000", 2024-09-04T17:46:03.651 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1126999999999999e-05 2024-09-04T17:46:03.651 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.651 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.652 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:03.652 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.463527+0000", 2024-09-04T17:46:03.652 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000148932 2024-09-04T17:46:03.652 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.652 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.652 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:03.652 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.463584+0000", 2024-09-04T17:46:03.653 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6999000000000001e-05 2024-09-04T17:46:03.653 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.653 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.653 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:03.653 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.465852+0000", 2024-09-04T17:46:03.653 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0022680529999999999 2024-09-04T17:46:03.653 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.654 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.654 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:03.654 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.465993+0000", 2024-09-04T17:46:03.654 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014047500000000001 2024-09-04T17:46:03.654 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:03.654 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:03.654 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:03.655 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.655 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.655 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:1430 3.f e28/24 3:f58475cb:::rbd_data.11346251f094.0000000000000074:head v 28'8, mlcod=28'6)", 2024-09-04T17:46:03.655 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:02.805728+0000", 2024-09-04T17:46:03.655 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.74339502300000004, 2024-09-04T17:46:03.655 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0025320009999999999, 2024-09-04T17:46:03.656 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:03.656 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:03.656 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:03.656 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:03.656 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.656 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:03.657 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.805728+0000", 2024-09-04T17:46:03.657 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:03.657 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.657 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.657 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:03.657 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.805728+0000", 2024-09-04T17:46:03.657 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:03.657 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.657 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.657 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:03.657 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.805730+0000", 2024-09-04T17:46:03.657 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5069999999999999e-06 2024-09-04T17:46:03.658 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.658 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.658 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:03.658 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.805847+0000", 2024-09-04T17:46:03.658 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000116816 2024-09-04T17:46:03.658 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.658 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.658 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:03.658 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.805879+0000", 2024-09-04T17:46:03.658 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.1946000000000002e-05 2024-09-04T17:46:03.658 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.658 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.658 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:03.659 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.805895+0000", 2024-09-04T17:46:03.659 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5962999999999999e-05 2024-09-04T17:46:03.659 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.659 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.659 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:03.659 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.806039+0000", 2024-09-04T17:46:03.659 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014353599999999999 2024-09-04T17:46:03.659 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.659 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.659 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:03.659 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.806099+0000", 2024-09-04T17:46:03.659 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.0235999999999997e-05 2024-09-04T17:46:03.660 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.660 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.660 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:03.660 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.808134+0000", 2024-09-04T17:46:03.660 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0020347540000000002 2024-09-04T17:46:03.660 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.660 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.660 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:03.660 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.808260+0000", 2024-09-04T17:46:03.660 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000126243 2024-09-04T17:46:03.660 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:03.660 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:03.661 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:03.661 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.661 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.661 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:1479 3.14 3:2c84d8a7:::rbd_data.11346251f094.00000000000000bb:head [set-alloc-hint object_size 524288 write_size 524288,write 131322~2101] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-04T17:46:03.661 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:02.926950+0000", 2024-09-04T17:46:03.661 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.62217268199999998, 2024-09-04T17:46:03.661 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0031905990000000001, 2024-09-04T17:46:03.661 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:03.661 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:03.662 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:03.662 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:03.662 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:03.662 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:03.662 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 1479 2024-09-04T17:46:03.662 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.662 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:03.662 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.662 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:03.663 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.926950+0000", 2024-09-04T17:46:03.663 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:03.663 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.663 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.663 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:03.663 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.926950+0000", 2024-09-04T17:46:03.663 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:03.663 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.663 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.663 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:03.664 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.926953+0000", 2024-09-04T17:46:03.664 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.7530000000000002e-06 2024-09-04T17:46:03.664 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.664 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.664 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:03.664 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.926968+0000", 2024-09-04T17:46:03.664 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5583e-05 2024-09-04T17:46:03.664 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.664 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.664 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:03.664 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.927000+0000", 2024-09-04T17:46:03.664 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.1303000000000002e-05 2024-09-04T17:46:03.665 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.665 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.665 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:03.665 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.927016+0000", 2024-09-04T17:46:03.665 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6010000000000001e-05 2024-09-04T17:46:03.665 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.665 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.665 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:03.665 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.927180+0000", 2024-09-04T17:46:03.665 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00016427600000000001 2024-09-04T17:46:03.665 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.665 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.666 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:03.666 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.927547+0000", 2024-09-04T17:46:03.666 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00036696999999999999 2024-09-04T17:46:03.666 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.666 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.666 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 7", 2024-09-04T17:46:03.666 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.927983+0000", 2024-09-04T17:46:03.666 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00043572000000000001 2024-09-04T17:46:03.666 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.666 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.666 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:03.666 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.929550+0000", 2024-09-04T17:46:03.667 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0015669550000000001 2024-09-04T17:46:03.667 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.667 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.667 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:03.667 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.929971+0000", 2024-09-04T17:46:03.667 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00042084400000000002 2024-09-04T17:46:03.667 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.667 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.667 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:03.667 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.930069+0000", 2024-09-04T17:46:03.667 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.8899000000000003e-05 2024-09-04T17:46:03.667 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.667 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.667 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:03.668 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.930141+0000", 2024-09-04T17:46:03.668 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1285999999999995e-05 2024-09-04T17:46:03.668 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:03.668 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:03.668 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:03.668 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.668 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.668 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:1490 3.1a 3:5bb407e0:::rbd_data.11346251f094.0000000000000121:head [set-alloc-hint object_size 524288 write_size 524288,write 448075~14416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-04T17:46:03.668 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:02.947759+0000", 2024-09-04T17:46:03.668 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.60136408299999999, 2024-09-04T17:46:03.668 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.002524231, 2024-09-04T17:46:03.668 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:03.669 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:03.669 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:03.669 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:03.669 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:03.669 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:03.669 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 1490 2024-09-04T17:46:03.669 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.669 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:03.669 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.669 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:03.669 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.947759+0000", 2024-09-04T17:46:03.670 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:03.670 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.670 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.670 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:03.670 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.947759+0000", 2024-09-04T17:46:03.670 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:03.670 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.670 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.670 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:03.670 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.947761+0000", 2024-09-04T17:46:03.670 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5270000000000001e-06 2024-09-04T17:46:03.670 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.670 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.670 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:03.671 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.947788+0000", 2024-09-04T17:46:03.671 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6780000000000001e-05 2024-09-04T17:46:03.671 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.671 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.671 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:03.671 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.947819+0000", 2024-09-04T17:46:03.671 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.1043000000000001e-05 2024-09-04T17:46:03.671 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.671 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.671 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:03.671 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.947835+0000", 2024-09-04T17:46:03.671 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.575e-05 2024-09-04T17:46:03.671 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.671 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.671 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:03.672 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.947945+0000", 2024-09-04T17:46:03.672 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000109796 2024-09-04T17:46:03.672 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.672 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.672 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:03.672 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.948167+0000", 2024-09-04T17:46:03.672 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00022202499999999999 2024-09-04T17:46:03.672 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.672 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.672 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 1", 2024-09-04T17:46:03.672 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.948396+0000", 2024-09-04T17:46:03.672 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00022950100000000001 2024-09-04T17:46:03.672 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.673 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.673 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:03.673 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.950092+0000", 2024-09-04T17:46:03.673 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0016957739999999999 2024-09-04T17:46:03.673 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.673 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.673 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:03.673 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.950117+0000", 2024-09-04T17:46:03.673 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4862999999999999e-05 2024-09-04T17:46:03.673 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.673 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.673 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:03.673 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.950197+0000", 2024-09-04T17:46:03.674 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.0536000000000003e-05 2024-09-04T17:46:03.674 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:03.674 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:03.674 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:03.674 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.950283+0000", 2024-09-04T17:46:03.674 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.5636000000000005e-05 2024-09-04T17:46:03.674 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:03.674 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:03.674 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:03.674 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:03.674 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:03.674 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:03.676 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-04T17:46:03.676 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:03.674+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:03.777 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:03.774+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:03.847 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:03.848 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:46:03.848 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:46:03.848 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:03.862 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-04T17:46:03.877 INFO:teuthology.orchestra.run.smithi092.stdout: marking collection for removal 2024-09-04T17:46:03.877 INFO:teuthology.orchestra.run.smithi092.stdout:setting '_remove' omap key 2024-09-04T17:46:03.877 INFO:teuthology.orchestra.run.smithi092.stdout:finish_remove_pgs 2.1_head removing 2.1 2024-09-04T17:46:03.882 INFO:teuthology.orchestra.run.smithi092.stdout:Remove successful 2024-09-04T17:46:03.904 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-04T17:46:03.911 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:03.912+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:03.979 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:03.978+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:03.986 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:03.986 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:46:03.986 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:46:03.986 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:46:03.986 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:04.000 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-04T17:46:04.051 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:46:04.051 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:46:04.067 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:46:04.067 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:46:04.084 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-04T17:46:04.084 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-04T17:46:04.084 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-04T17:46:04.087 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:04.088+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:04.104 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:46:04.104 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:46:04.124 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:46:04.124 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:46:04.142 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-04T17:46:04.142 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-04T17:46:04.163 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:46:04.163 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:46:04.167 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-04T17:46:04.168 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-04T17:46:04.170 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:04.170 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:04.180 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is unset 2024-09-04T17:46:04.180 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:04.180+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:04.186 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-04T17:46:04.281 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:04.278+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:04.352 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:04.353 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:04.367 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-04T17:46:04.410 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #0, image image_client.0 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent1 2024-09-04T17:46:04.411 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:04.412+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:04.424 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-3 --journal-path /var/lib/ceph/osd/ceph-3/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.2.1.3 2024-09-04T17:46:04.483 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:04.484+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:04.489 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:04.489 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:04.503 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-04T17:46:04.583 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:04.582+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:04.674 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:04.674 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:04.685 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:46:04.684+0000 7f505d642640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26446) UID: 0 2024-09-04T17:46:04.688 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-04T17:46:04.786 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:04.784+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:04.846 INFO:teuthology.orchestra.run.smithi092.stdout:487 write 0xe4df4f2 thru 0xe4eb458 (0xbf67 bytes) 2024-09-04T17:46:04.857 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:04.857 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:04.866 INFO:teuthology.orchestra.run.smithi092.stdout:489 trunc from 0xe4eb459 to 0x56452b2 2024-09-04T17:46:04.872 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-04T17:46:04.887 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:04.886+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:04.913 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:04.934 INFO:teuthology.orchestra.run.smithi092.stdout:492 writesame 0xa5fcf11 thru 0xa6078a2 data_size 0xf(0xa992 bytes) 2024-09-04T17:46:04.935 INFO:teuthology.orchestra.run.smithi092.stdout:494 write 0x421b2e9 thru 0x421cb2b (0x1843 bytes) 2024-09-04T17:46:04.952 INFO:teuthology.orchestra.run.smithi092.stdout:495 writesame 0xb201a84 thru 0xb202926 data_size 0x1(0xea3 bytes) 2024-09-04T17:46:04.954 INFO:teuthology.orchestra.run.smithi092.stdout:496 read 0x6638d95 thru 0x6640ce5 (0x7f51 bytes) 2024-09-04T17:46:04.955 INFO:teuthology.orchestra.run.smithi092.stdout:497 read 0x8db71ea thru 0x8db8660 (0x1477 bytes) 2024-09-04T17:46:04.956 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:04.956 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:04.956 INFO:teuthology.orchestra.run.smithi092.stdout:501 read 0xa79e5 thru 0xa823c (0x858 bytes) 2024-09-04T17:46:04.957 INFO:teuthology.orchestra.run.smithi092.stdout:503 read 0x1113b8f thru 0x111dac5 (0x9f37 bytes) 2024-09-04T17:46:04.958 INFO:teuthology.orchestra.run.smithi092.stdout:504 write 0x533809f thru 0x534195c (0x98be bytes) 2024-09-04T17:46:04.966 INFO:teuthology.orchestra.run.smithi092.stdout:505 trunc from 0xb202927 to 0xe42d1ad 2024-09-04T17:46:04.979 INFO:teuthology.orchestra.run.smithi092.stdout:506 read 0xd46063a thru 0xd46648f (0x5e56 bytes) 2024-09-04T17:46:04.980 INFO:teuthology.orchestra.run.smithi092.stdout:507 write 0x1e78a24 thru 0x1e7dc6b (0x5248 bytes) 2024-09-04T17:46:04.988 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:04.986+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:05.004 INFO:teuthology.orchestra.run.smithi092.stdout:509 trunc from 0xe42d1ad to 0x4d887f6 2024-09-04T17:46:05.047 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:05.047 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:05.047 INFO:teuthology.orchestra.run.smithi092.stdout:510 writesame 0xa2b3b56 thru 0xa2bba01 data_size 0x43(0x7eac bytes) 2024-09-04T17:46:05.049 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:05.050 INFO:teuthology.orchestra.run.smithi092.stdout:514 writesame 0x5be1e8 thru 0x5c0e54 data_size 0x33(0x2c6d bytes) 2024-09-04T17:46:05.052 INFO:teuthology.orchestra.run.smithi092.stdout:515 punch from 0x743a36c to 0x74470d6, (0xcd6a bytes) 2024-09-04T17:46:05.052 INFO:teuthology.orchestra.run.smithi092.stdout:516 read 0x6e116b4 thru 0x6e1670f (0x505c bytes) 2024-09-04T17:46:05.053 INFO:teuthology.orchestra.run.smithi092.stdout:517 read 0xbbac5f thru 0xbbaf25 (0x2c7 bytes) 2024-09-04T17:46:05.054 INFO:teuthology.orchestra.run.smithi092.stdout:519 read 0x1e50485 thru 0x1e515d8 (0x1154 bytes) 2024-09-04T17:46:05.055 INFO:teuthology.orchestra.run.smithi092.stdout:521 write 0x53e495d thru 0x53e8340 (0x39e4 bytes) 2024-09-04T17:46:05.059 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:05.059 INFO:teuthology.orchestra.run.smithi092.stdout:523 write 0x5df8005 thru 0x5dfd026 (0x5022 bytes) 2024-09-04T17:46:05.062 INFO:teuthology.orchestra.run.smithi092.stdout:524 read 0x19a0de thru 0x1a12a1 (0x71c4 bytes) 2024-09-04T17:46:05.063 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-04T17:46:05.064 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:05.079 INFO:teuthology.orchestra.run.smithi092.stdout:527 writesame 0xc2f4576 thru 0xc2f6f5c data_size 0x1(0x29e7 bytes) 2024-09-04T17:46:05.088 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:05.086+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:05.103 INFO:teuthology.orchestra.run.smithi092.stdout:529 trunc from 0xc2f6f5d to 0x16f2e6d 2024-09-04T17:46:05.119 INFO:teuthology.orchestra.run.smithi092.stdout:530 read 0x28c63 thru 0x2bc2e (0x2fcc bytes) 2024-09-04T17:46:05.156 INFO:teuthology.orchestra.run.smithi092.stdout:531 write 0xe139bf7 thru 0xe145ed1 (0xc2db bytes) 2024-09-04T17:46:05.161 INFO:teuthology.orchestra.run.smithi092.stdout:533 punch from 0x7e19485 to 0x7e1cb4b, (0x36c6 bytes) 2024-09-04T17:46:05.162 INFO:teuthology.orchestra.run.smithi092.stdout:534 write 0x94cd321 thru 0x94d953b (0xc21b bytes) 2024-09-04T17:46:05.164 INFO:teuthology.orchestra.run.smithi092.stdout:535 read 0xab5f5a4 thru 0xab64cc1 (0x571e bytes) 2024-09-04T17:46:05.165 INFO:teuthology.orchestra.run.smithi092.stdout:536 read 0x7af98c5 thru 0x7b03eed (0xa629 bytes) 2024-09-04T17:46:05.165 INFO:teuthology.orchestra.run.smithi092.stdout:537 read 0xccaa52e thru 0xccb57f7 (0xb2ca bytes) 2024-09-04T17:46:05.166 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:05.166 INFO:teuthology.orchestra.run.smithi092.stdout:540 writesame 0xaacf009 thru 0xaad9cb0 data_size 0xaa(0xaca8 bytes) 2024-09-04T17:46:05.169 INFO:teuthology.orchestra.run.smithi092.stdout:541 read 0xc1db733 thru 0xc1e566a (0x9f38 bytes) 2024-09-04T17:46:05.169 INFO:teuthology.orchestra.run.smithi092.stdout:542 write 0xd108efa thru 0xd117824 (0xe92b bytes) 2024-09-04T17:46:05.173 INFO:teuthology.orchestra.run.smithi092.stdout:545 punch from 0xb938263 to 0xb93e78a, (0x6527 bytes) 2024-09-04T17:46:05.173 INFO:teuthology.orchestra.run.smithi092.stdout:546 read 0xb27344b thru 0xb2777e3 (0x4399 bytes) 2024-09-04T17:46:05.187 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:05.187 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:05.189 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:05.186+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:05.192 INFO:teuthology.orchestra.run.smithi092.stdout:548 trunc from 0xe145ed2 to 0x26be70b 2024-09-04T17:46:05.202 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-04T17:46:05.216 INFO:teuthology.orchestra.run.smithi092.stdout:550 trunc from 0x26be70b to 0x47d2402 2024-09-04T17:46:05.226 INFO:teuthology.orchestra.run.smithi092.stdout:551 write 0x14e69c8 thru 0x14f49bb (0xdff4 bytes) 2024-09-04T17:46:05.228 INFO:teuthology.orchestra.run.smithi092.stdout:552 write 0x2a4c3b3 thru 0x2a4c42b (0x79 bytes) 2024-09-04T17:46:05.230 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:05.242 INFO:teuthology.orchestra.run.smithi092.stdout:555 trunc from 0x47d2402 to 0xc6cc59a 2024-09-04T17:46:05.254 INFO:teuthology.orchestra.run.smithi092.stdout:556 writesame 0x9aeb631 thru 0x9af9208 data_size 0x8c(0xdbd8 bytes) 2024-09-04T17:46:05.257 INFO:teuthology.orchestra.run.smithi092.stdout:557 read 0x1012eff thru 0x1022176 (0xf278 bytes) 2024-09-04T17:46:05.258 INFO:teuthology.orchestra.run.smithi092.stdout:558 write 0x214e1f4 thru 0x215b96c (0xd779 bytes) 2024-09-04T17:46:05.261 INFO:teuthology.orchestra.run.smithi092.stdout:560 punch from 0xb9a654b to 0xb9abc57, (0x570c bytes) 2024-09-04T17:46:05.261 INFO:teuthology.orchestra.run.smithi092.stdout:562 writesame 0x107b9bf thru 0x107dd3d data_size 0xd(0x237f bytes) 2024-09-04T17:46:05.266 INFO:teuthology.orchestra.run.smithi092.stdout:563 trunc from 0xc6cc59a to 0xabdd435 2024-09-04T17:46:05.282 INFO:teuthology.orchestra.run.smithi092.stdout:564 write 0xa42dfd6 thru 0xa4305d8 (0x2603 bytes) 2024-09-04T17:46:05.290 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:46:05.288+0000 7f505d642640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26446) UID: 0 2024-09-04T17:46:05.298 INFO:teuthology.orchestra.run.smithi092.stdout:565 trunc from 0xabdd435 to 0x44a088a 2024-09-04T17:46:05.316 INFO:teuthology.orchestra.run.smithi092.stdout:566 punch from 0x3b6e5dd to 0x3b7625b, (0x7c7e bytes) 2024-09-04T17:46:05.316 INFO:teuthology.orchestra.run.smithi092.stdout:567 write 0x11d5d2d thru 0x11e3537 (0xd80b bytes) 2024-09-04T17:46:05.324 INFO:teuthology.orchestra.run.smithi092.stdout:568 trunc from 0x44a088a to 0x1aa218a 2024-09-04T17:46:05.327 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:05.328 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:05.343 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-04T17:46:05.346 INFO:teuthology.orchestra.run.smithi092.stdout:569 read 0x560199 thru 0x5623d5 (0x223d bytes) 2024-09-04T17:46:05.362 INFO:teuthology.orchestra.run.smithi092.stdout:572 write 0x3fc72d6 thru 0x3fc8e77 (0x1ba2 bytes) 2024-09-04T17:46:05.365 INFO:teuthology.orchestra.run.smithi092.stdout:573 read 0x396ccd3 thru 0x397376a (0x6a98 bytes) 2024-09-04T17:46:05.365 INFO:teuthology.orchestra.run.smithi092.stdout:575 read 0x55263a thru 0x55f23a (0xcc01 bytes) 2024-09-04T17:46:05.381 INFO:teuthology.orchestra.run.smithi092.stdout:576 trunc from 0x3fc8e78 to 0xb8c372c 2024-09-04T17:46:05.390 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:05.388+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:05.391 INFO:teuthology.orchestra.run.smithi092.stdout:577 clone 3 order 21 su 32768 sc 13 2024-09-04T17:46:05.478 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:05.478 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:05.491 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:46:05.492+0000 7f505d642640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26446) UID: 0 2024-09-04T17:46:05.495 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-04T17:46:05.592 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:05.590+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:05.663 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:05.663 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:05.677 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-04T17:46:05.718 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:05.718+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:05.793 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:05.792+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:05.840 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:05.840 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:05.854 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-04T17:46:05.894 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:05.892+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:05.976 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:05.976 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:05.990 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-04T17:46:05.995 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:05.996+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:06.096 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:06.096+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:06.115 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:06.115 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:06.130 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-04T17:46:06.169 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-04T17:46:06.202 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-04T17:46:06.222 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:06.222+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:06.261 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:06.261 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:06.278 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-04T17:46:06.300 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone2 from 0xb8c372c (overlap 0x16f2e6d) to 0x125bebd 2024-09-04T17:46:06.300 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:06.298+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:06.340 INFO:teuthology.orchestra.run.smithi092.stdout:get_pg_num_history pg_num_history pg_num_history(e23 pg_nums {1={16=1},2={19=8}} deleted_pools ) 2024-09-04T17:46:06.340 INFO:teuthology.orchestra.run.smithi092.stdout:Importing pgid 2.1 2024-09-04T17:46:06.340 INFO:teuthology.orchestra.run.smithi092.stdout:write_pg epoch 23 info 2.1( empty local-lis/les=19/20 n=0 ec=19/19 lis/c=19/19 les/c/f=20/20/0 sis=19) 2024-09-04T17:46:06.341 INFO:teuthology.orchestra.run.smithi092.stdout:Import successful 2024-09-04T17:46:06.346 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:46:06.346 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:46:06.379 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:46:06.379 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:46:06.399 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #1, image image_client.0-clone1 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent2 2024-09-04T17:46:06.399 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:06.400+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:06.404 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-04T17:46:06.404 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-04T17:46:06.404 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-04T17:46:06.425 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:46:06.425 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:46:06.446 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:46:06.446 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:46:06.450 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:06.450 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:46:06.450 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:46:06.451 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:46:06.451 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:06.467 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-04T17:46:06.468 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-04T17:46:06.468 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-04T17:46:06.501 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:06.500+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:06.508 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:46:06.508 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:46:06.512 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-04T17:46:06.513 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-04T17:46:06.593 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:06.593 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:46:06.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:46:06.593 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:46:06.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.594 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4208.0:1 2.3 e20/19 2:c4c92e5a:::rbd_trash:head v 20'1, mlcod=0'0)", 2024-09-04T17:46:06.594 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:45:47.377621+0000", 2024-09-04T17:46:06.594 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.218476798000001, 2024-09-04T17:46:06.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0010679260000000001, 2024-09-04T17:46:06.594 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:06.594 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:06.594 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:06.594 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:06.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.594 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:06.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:47.377621+0000", 2024-09-04T17:46:06.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:06.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:06.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:47.377621+0000", 2024-09-04T17:46:06.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:06.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:06.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:47.377622+0000", 2024-09-04T17:46:06.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0299999999999998e-07 2024-09-04T17:46:06.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.596 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:06.596 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:47.377633+0000", 2024-09-04T17:46:06.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1528999999999999e-05 2024-09-04T17:46:06.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.596 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:06.596 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:47.377645+0000", 2024-09-04T17:46:06.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1779e-05 2024-09-04T17:46:06.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.596 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:06.597 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:47.377655+0000", 2024-09-04T17:46:06.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8989999999999996e-06 2024-09-04T17:46:06.597 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.597 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:06.597 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:47.377726+0000", 2024-09-04T17:46:06.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1237999999999998e-05 2024-09-04T17:46:06.597 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.597 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:06.597 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:47.377760+0000", 2024-09-04T17:46:06.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3958000000000003e-05 2024-09-04T17:46:06.597 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:06.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:47.378637+0000", 2024-09-04T17:46:06.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00087744799999999996 2024-09-04T17:46:06.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:06.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:45:47.378689+0000", 2024-09-04T17:46:06.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1372000000000002e-05 2024-09-04T17:46:06.598 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:06.598 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:06.598 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:06.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.599 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:937 3.0 e28/24 3:02897c52:::rbd_data.11346251f094.0000000000000163:head v 28'4, mlcod=28'2)", 2024-09-04T17:46:06.599 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:01.945087+0000", 2024-09-04T17:46:06.599 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.6510107439999997, 2024-09-04T17:46:06.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000898294, 2024-09-04T17:46:06.599 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:06.599 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:06.599 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:06.599 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:06.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:06.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.945087+0000", 2024-09-04T17:46:06.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:06.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:06.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.945087+0000", 2024-09-04T17:46:06.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:06.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:06.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.945089+0000", 2024-09-04T17:46:06.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.784e-06 2024-09-04T17:46:06.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:06.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.945109+0000", 2024-09-04T17:46:06.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0088000000000001e-05 2024-09-04T17:46:06.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:06.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.945130+0000", 2024-09-04T17:46:06.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1345e-05 2024-09-04T17:46:06.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:06.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.945140+0000", 2024-09-04T17:46:06.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.499e-06 2024-09-04T17:46:06.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:06.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.945208+0000", 2024-09-04T17:46:06.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8038000000000001e-05 2024-09-04T17:46:06.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:06.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.945234+0000", 2024-09-04T17:46:06.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5979e-05 2024-09-04T17:46:06.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:06.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.945941+0000", 2024-09-04T17:46:06.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00070737999999999997 2024-09-04T17:46:06.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:06.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.945985+0000", 2024-09-04T17:46:06.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4181000000000003e-05 2024-09-04T17:46:06.603 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:06.603 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:06.603 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:06.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.603 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:951 3.0 e28/24 3:02897c52:::rbd_data.11346251f094.0000000000000163:head v 28'5, mlcod=28'3)", 2024-09-04T17:46:06.603 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:01.976152+0000", 2024-09-04T17:46:06.603 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.6199452499999998, 2024-09-04T17:46:06.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00066695600000000002, 2024-09-04T17:46:06.604 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:06.604 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:06.604 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:06.604 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:06.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:06.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.976152+0000", 2024-09-04T17:46:06.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:06.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:06.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.976152+0000", 2024-09-04T17:46:06.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:06.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:06.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.976154+0000", 2024-09-04T17:46:06.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.635e-06 2024-09-04T17:46:06.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:06.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.976165+0000", 2024-09-04T17:46:06.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1389000000000001e-05 2024-09-04T17:46:06.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:06.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.976187+0000", 2024-09-04T17:46:06.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1361000000000001e-05 2024-09-04T17:46:06.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:06.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.976197+0000", 2024-09-04T17:46:06.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8689999999999999e-06 2024-09-04T17:46:06.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:06.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.976268+0000", 2024-09-04T17:46:06.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0900000000000002e-05 2024-09-04T17:46:06.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:06.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.976292+0000", 2024-09-04T17:46:06.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4494e-05 2024-09-04T17:46:06.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:06.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.976788+0000", 2024-09-04T17:46:06.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00049621699999999997 2024-09-04T17:46:06.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:06.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:01.976819+0000", 2024-09-04T17:46:06.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1090999999999998e-05 2024-09-04T17:46:06.608 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:06.608 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:06.608 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:06.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.608 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:1050 3.4 e28/24 3:237c6c21:::rbd_data.11346251f094.0000000000000012:head v 28'2, mlcod=0'0)", 2024-09-04T17:46:06.608 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:02.150709+0000", 2024-09-04T17:46:06.608 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.4453890559999998, 2024-09-04T17:46:06.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0013452329999999999, 2024-09-04T17:46:06.608 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:06.608 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:06.608 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:06.609 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:06.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:06.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.150709+0000", 2024-09-04T17:46:06.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:06.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:06.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.150709+0000", 2024-09-04T17:46:06.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:06.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:06.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.150710+0000", 2024-09-04T17:46:06.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8169999999999999e-06 2024-09-04T17:46:06.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:06.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.150753+0000", 2024-09-04T17:46:06.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2710000000000003e-05 2024-09-04T17:46:06.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:06.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.150772+0000", 2024-09-04T17:46:06.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9119000000000001e-05 2024-09-04T17:46:06.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:06.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.150785+0000", 2024-09-04T17:46:06.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2744e-05 2024-09-04T17:46:06.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:06.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.150853+0000", 2024-09-04T17:46:06.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8005999999999999e-05 2024-09-04T17:46:06.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:06.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.150875+0000", 2024-09-04T17:46:06.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2004000000000001e-05 2024-09-04T17:46:06.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:06.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.151966+0000", 2024-09-04T17:46:06.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0010910340000000001 2024-09-04T17:46:06.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:06.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.152054+0000", 2024-09-04T17:46:06.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7799000000000004e-05 2024-09-04T17:46:06.612 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:06.612 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:06.613 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:06.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.613 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:1098 3.9 e28/24 3:91683f21:::rbd_data.11346251f094.0000000000000050:head v 28'78, mlcod=28'76)", 2024-09-04T17:46:06.613 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:02.277164+0000", 2024-09-04T17:46:06.613 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.3189337749999996, 2024-09-04T17:46:06.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019986840000000001, 2024-09-04T17:46:06.613 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:06.613 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:06.613 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:06.613 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:06.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.613 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:06.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.277164+0000", 2024-09-04T17:46:06.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:06.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:06.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.277164+0000", 2024-09-04T17:46:06.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:06.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:06.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.277166+0000", 2024-09-04T17:46:06.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.962e-06 2024-09-04T17:46:06.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:06.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.277403+0000", 2024-09-04T17:46:06.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0002376 2024-09-04T17:46:06.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:06.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.277426+0000", 2024-09-04T17:46:06.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2478000000000001e-05 2024-09-04T17:46:06.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:06.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.277440+0000", 2024-09-04T17:46:06.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3725e-05 2024-09-04T17:46:06.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:06.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.277561+0000", 2024-09-04T17:46:06.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000120988 2024-09-04T17:46:06.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:06.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.277602+0000", 2024-09-04T17:46:06.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1130000000000001e-05 2024-09-04T17:46:06.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:06.617 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.279056+0000", 2024-09-04T17:46:06.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014545350000000001 2024-09-04T17:46:06.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.617 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:06.617 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.279163+0000", 2024-09-04T17:46:06.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106266 2024-09-04T17:46:06.617 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:06.617 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:06.617 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:06.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.617 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:1146 3.9 e28/24 3:97509b58:::rbd_data.11346251f094.000000000000002e:head v 28'79, mlcod=28'77)", 2024-09-04T17:46:06.617 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:02.371990+0000", 2024-09-04T17:46:06.618 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.2241081640000004, 2024-09-04T17:46:06.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0010613860000000001, 2024-09-04T17:46:06.618 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:06.618 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:06.618 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:06.618 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:06.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.618 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:06.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.371990+0000", 2024-09-04T17:46:06.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:06.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.618 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:06.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.371990+0000", 2024-09-04T17:46:06.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:06.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:06.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.371991+0000", 2024-09-04T17:46:06.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4980000000000001e-06 2024-09-04T17:46:06.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:06.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.372075+0000", 2024-09-04T17:46:06.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4268e-05 2024-09-04T17:46:06.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:06.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.372091+0000", 2024-09-04T17:46:06.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5554e-05 2024-09-04T17:46:06.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:06.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.372098+0000", 2024-09-04T17:46:06.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8950000000000001e-06 2024-09-04T17:46:06.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:06.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.372166+0000", 2024-09-04T17:46:06.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8298000000000002e-05 2024-09-04T17:46:06.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:06.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.372188+0000", 2024-09-04T17:46:06.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1708e-05 2024-09-04T17:46:06.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:06.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.372998+0000", 2024-09-04T17:46:06.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00081050399999999998 2024-09-04T17:46:06.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:06.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.373051+0000", 2024-09-04T17:46:06.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2661000000000003e-05 2024-09-04T17:46:06.622 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:06.622 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:06.622 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:06.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.622 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:1174 3.9 e28/24 3:91683f21:::rbd_data.11346251f094.0000000000000050:head v 28'80, mlcod=28'78)", 2024-09-04T17:46:06.622 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:02.412870+0000", 2024-09-04T17:46:06.622 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.1832276190000002, 2024-09-04T17:46:06.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00101965, 2024-09-04T17:46:06.622 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:06.622 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:06.622 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:06.623 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:06.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.623 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:06.623 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.412870+0000", 2024-09-04T17:46:06.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:06.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.623 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:06.623 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.412870+0000", 2024-09-04T17:46:06.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:06.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.623 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:06.623 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.412872+0000", 2024-09-04T17:46:06.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.59e-06 2024-09-04T17:46:06.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:06.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.412882+0000", 2024-09-04T17:46:06.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0647e-05 2024-09-04T17:46:06.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:06.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.412901+0000", 2024-09-04T17:46:06.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8461000000000002e-05 2024-09-04T17:46:06.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:06.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.412909+0000", 2024-09-04T17:46:06.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2779999999999993e-06 2024-09-04T17:46:06.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:06.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.412978+0000", 2024-09-04T17:46:06.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8573999999999998e-05 2024-09-04T17:46:06.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:06.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.412997+0000", 2024-09-04T17:46:06.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9173000000000001e-05 2024-09-04T17:46:06.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:06.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.413836+0000", 2024-09-04T17:46:06.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00083885899999999996 2024-09-04T17:46:06.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:06.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.413890+0000", 2024-09-04T17:46:06.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4067999999999998e-05 2024-09-04T17:46:06.626 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:06.626 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:06.626 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:06.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.627 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:1199 3.0 e28/24 3:00cc0f40:::rbd_data.11346251f094.0000000000000075:head v 28'6, mlcod=28'4)", 2024-09-04T17:46:06.627 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:02.448872+0000", 2024-09-04T17:46:06.627 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.1472254670000002, 2024-09-04T17:46:06.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001200069, 2024-09-04T17:46:06.627 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:06.627 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:06.627 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:06.627 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:06.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:06.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.448872+0000", 2024-09-04T17:46:06.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:06.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:06.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.448872+0000", 2024-09-04T17:46:06.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:06.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:06.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.448874+0000", 2024-09-04T17:46:06.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5910000000000001e-06 2024-09-04T17:46:06.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:06.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.448896+0000", 2024-09-04T17:46:06.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2543000000000001e-05 2024-09-04T17:46:06.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:06.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.448914+0000", 2024-09-04T17:46:06.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7368999999999999e-05 2024-09-04T17:46:06.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:06.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.448922+0000", 2024-09-04T17:46:06.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8019999999999992e-06 2024-09-04T17:46:06.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:06.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.449048+0000", 2024-09-04T17:46:06.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012616299999999999 2024-09-04T17:46:06.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:06.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.449124+0000", 2024-09-04T17:46:06.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6468e-05 2024-09-04T17:46:06.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:06.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.450035+0000", 2024-09-04T17:46:06.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00091072600000000005 2024-09-04T17:46:06.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:06.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.450072+0000", 2024-09-04T17:46:06.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7407000000000003e-05 2024-09-04T17:46:06.631 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:06.631 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:06.632 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:06.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.632 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:1370 3.4 e28/24 3:22db5e3d:::rbd_data.11346251f094.0000000000000174:head v 28'3, mlcod=28'1)", 2024-09-04T17:46:06.632 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:02.686754+0000", 2024-09-04T17:46:06.632 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.9093432359999998, 2024-09-04T17:46:06.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001330258, 2024-09-04T17:46:06.632 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:06.632 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:06.632 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:06.632 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:06.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:06.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.686754+0000", 2024-09-04T17:46:06.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:06.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:06.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.686754+0000", 2024-09-04T17:46:06.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:06.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:06.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.686757+0000", 2024-09-04T17:46:06.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5330000000000001e-06 2024-09-04T17:46:06.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:06.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.686820+0000", 2024-09-04T17:46:06.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3526000000000001e-05 2024-09-04T17:46:06.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:06.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.686859+0000", 2024-09-04T17:46:06.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8260000000000003e-05 2024-09-04T17:46:06.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:06.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.686881+0000", 2024-09-04T17:46:06.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2549999999999999e-05 2024-09-04T17:46:06.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:06.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.686990+0000", 2024-09-04T17:46:06.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010919 2024-09-04T17:46:06.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:06.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.687017+0000", 2024-09-04T17:46:06.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6137999999999999e-05 2024-09-04T17:46:06.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:06.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.688024+0000", 2024-09-04T17:46:06.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0010077720000000001 2024-09-04T17:46:06.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:06.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.688085+0000", 2024-09-04T17:46:06.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0288999999999998e-05 2024-09-04T17:46:06.636 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:06.636 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:06.636 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:06.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.637 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:1377 3.4 e28/24 3:22db5e3d:::rbd_data.11346251f094.0000000000000174:head v 28'4, mlcod=28'2)", 2024-09-04T17:46:06.637 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:02.716364+0000", 2024-09-04T17:46:06.637 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.8797340089999999, 2024-09-04T17:46:06.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00083559200000000002, 2024-09-04T17:46:06.637 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:06.637 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:06.637 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:06.637 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:06.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:06.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.716364+0000", 2024-09-04T17:46:06.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:06.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:06.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.716364+0000", 2024-09-04T17:46:06.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:06.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:06.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.716365+0000", 2024-09-04T17:46:06.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5969999999999999e-06 2024-09-04T17:46:06.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:06.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.716375+0000", 2024-09-04T17:46:06.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5419999999999994e-06 2024-09-04T17:46:06.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:06.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.716401+0000", 2024-09-04T17:46:06.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6380000000000002e-05 2024-09-04T17:46:06.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:06.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.716412+0000", 2024-09-04T17:46:06.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0793e-05 2024-09-04T17:46:06.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:06.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.716489+0000", 2024-09-04T17:46:06.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6884000000000005e-05 2024-09-04T17:46:06.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:06.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.716515+0000", 2024-09-04T17:46:06.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.639e-05 2024-09-04T17:46:06.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:06.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.717131+0000", 2024-09-04T17:46:06.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00061580700000000001 2024-09-04T17:46:06.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:06.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.717199+0000", 2024-09-04T17:46:06.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8199000000000002e-05 2024-09-04T17:46:06.641 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:06.641 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:06.641 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:06.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.641 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:1630 3.4 e28/24 3:26bc90e3:::rbd_data.11346251f094.0000000000000191:head v 28'5, mlcod=28'3)", 2024-09-04T17:46:06.641 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:03.193421+0000", 2024-09-04T17:46:06.642 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.4026769479999999, 2024-09-04T17:46:06.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000880666, 2024-09-04T17:46:06.642 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:06.642 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:06.642 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:06.642 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:06.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:06.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:03.193421+0000", 2024-09-04T17:46:06.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:06.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:06.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:03.193421+0000", 2024-09-04T17:46:06.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:06.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:06.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:03.193423+0000", 2024-09-04T17:46:06.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.475e-06 2024-09-04T17:46:06.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:06.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:03.193443+0000", 2024-09-04T17:46:06.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9717000000000001e-05 2024-09-04T17:46:06.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.644 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:06.644 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:03.193463+0000", 2024-09-04T17:46:06.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0173000000000002e-05 2024-09-04T17:46:06.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.644 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:06.644 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:03.193473+0000", 2024-09-04T17:46:06.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0185e-05 2024-09-04T17:46:06.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:06.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:03.193549+0000", 2024-09-04T17:46:06.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5606999999999996e-05 2024-09-04T17:46:06.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:06.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:03.193577+0000", 2024-09-04T17:46:06.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8558999999999999e-05 2024-09-04T17:46:06.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:06.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:03.194269+0000", 2024-09-04T17:46:06.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00069163099999999995 2024-09-04T17:46:06.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:06.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:03.194301+0000", 2024-09-04T17:46:06.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2319e-05 2024-09-04T17:46:06.646 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:06.646 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:06.646 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:06.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.646 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:1642 3.4 e28/24 3:26bc90e3:::rbd_data.11346251f094.0000000000000191:head v 28'6, mlcod=28'4)", 2024-09-04T17:46:06.646 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:03.212981+0000", 2024-09-04T17:46:06.646 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.3831163289999999, 2024-09-04T17:46:06.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00072205499999999998, 2024-09-04T17:46:06.647 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:06.647 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:06.647 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:06.647 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:06.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.647 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:06.647 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:03.212981+0000", 2024-09-04T17:46:06.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:06.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.647 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:06.647 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:03.212981+0000", 2024-09-04T17:46:06.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:06.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.648 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:06.648 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:03.212983+0000", 2024-09-04T17:46:06.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.787e-06 2024-09-04T17:46:06.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.648 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:06.648 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:03.212994+0000", 2024-09-04T17:46:06.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0879e-05 2024-09-04T17:46:06.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.649 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:06.649 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:03.213013+0000", 2024-09-04T17:46:06.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9127000000000002e-05 2024-09-04T17:46:06.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.649 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:06.649 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:03.213025+0000", 2024-09-04T17:46:06.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2041e-05 2024-09-04T17:46:06.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.649 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:06.649 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:03.213097+0000", 2024-09-04T17:46:06.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1404999999999998e-05 2024-09-04T17:46:06.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:06.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:03.213131+0000", 2024-09-04T17:46:06.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4e-05 2024-09-04T17:46:06.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:06.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:03.213674+0000", 2024-09-04T17:46:06.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00054374999999999996 2024-09-04T17:46:06.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:06.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:06.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:06.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:03.213703+0000", 2024-09-04T17:46:06.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9065999999999999e-05 2024-09-04T17:46:06.651 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:06.651 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:06.651 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:06.651 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:06.651 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:06.651 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:06.653 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-04T17:46:06.653 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:06.650+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:06.704 INFO:teuthology.orchestra.run.smithi092.stdout:578 trunc from 0xb8c372c to 0xca081aa 2024-09-04T17:46:06.719 INFO:teuthology.orchestra.run.smithi092.stdout:579 punch from 0x2042412 to 0x2052409, (0xfff7 bytes) 2024-09-04T17:46:06.719 INFO:teuthology.orchestra.run.smithi092.stdout:580 write 0x7db5292 thru 0x7dc31bb (0xdf2a bytes) 2024-09-04T17:46:06.749 INFO:teuthology.orchestra.run.smithi092.stdout:581 read 0xc7f36a4 thru 0xc7fc848 (0x91a5 bytes) 2024-09-04T17:46:06.750 INFO:teuthology.orchestra.run.smithi092.stdout:582 read 0xc7c36b8 thru 0xc7c5461 (0x1daa bytes) 2024-09-04T17:46:06.750 INFO:teuthology.orchestra.run.smithi092.stdout:584 write 0x709f87e thru 0x70a62b9 (0x6a3c bytes) 2024-09-04T17:46:06.753 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:06.750+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:06.773 INFO:teuthology.orchestra.run.smithi092.stdout:586 read 0x34f262a thru 0x34f47f4 (0x21cb bytes) 2024-09-04T17:46:06.774 INFO:teuthology.orchestra.run.smithi092.stdout:587 writesame 0xe96de1 thru 0xea34fe data_size 0x7(0xc71e bytes) 2024-09-04T17:46:06.810 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:06.810 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:46:06.810 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:46:06.810 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:06.817 INFO:teuthology.orchestra.run.smithi092.stdout:588 write 0x238fcfc thru 0x2391d4c (0x2051 bytes) 2024-09-04T17:46:06.830 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-04T17:46:06.833 INFO:teuthology.orchestra.run.smithi092.stdout:591 read 0x2a77ec8 thru 0x2a7bccf (0x3e08 bytes) 2024-09-04T17:46:06.834 INFO:teuthology.orchestra.run.smithi092.stdout:592 punch from 0x25bd1eb to 0x25c2efd, (0x5d12 bytes) 2024-09-04T17:46:06.834 INFO:teuthology.orchestra.run.smithi092.stdout:593 read 0xa1a3716 thru 0xa1a7518 (0x3e03 bytes) 2024-09-04T17:46:06.834 INFO:teuthology.orchestra.run.smithi092.stdout:594 punch from 0x5a037f6 to 0x5a0bd00, (0x850a bytes) 2024-09-04T17:46:06.835 INFO:teuthology.orchestra.run.smithi092.stdout:595 read 0x1be7083 thru 0x1bed723 (0x66a1 bytes) 2024-09-04T17:46:06.835 INFO:teuthology.orchestra.run.smithi092.stdout:596 write 0x52c0866 thru 0x52c1382 (0xb1d bytes) 2024-09-04T17:46:06.853 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:06.850+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:06.868 INFO:teuthology.orchestra.run.smithi092.stdout:597 trunc from 0xca081aa to 0x26c5b47 2024-09-04T17:46:06.897 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f /var/log/ceph/exp.2.1.3 2024-09-04T17:46:06.947 INFO:teuthology.orchestra.run.smithi092.stdout:598 trunc from 0x26c5b47 to 0x27b3fd4 2024-09-04T17:46:06.954 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:06.952+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:06.968 INFO:teuthology.orchestra.run.smithi092.stdout:599 writesame 0x5117ddd thru 0x512794a data_size 0x1(0xfb6e bytes) 2024-09-04T17:46:06.972 INFO:teuthology.orchestra.run.smithi092.stdout:600 trunc from 0x512794b to 0x74a24c7 2024-09-04T17:46:06.973 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:06.973 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:46:06.973 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:46:06.973 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:46:06.973 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:06.989 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-04T17:46:07.004 INFO:teuthology.orchestra.run.smithi092.stdout:603 write 0xe94103e thru 0xe94422d (0x31f0 bytes) 2024-09-04T17:46:07.008 INFO:teuthology.orchestra.run.smithi092.stdout:604 read 0xcf628cf thru 0xcf68d47 (0x6479 bytes) 2024-09-04T17:46:07.008 INFO:teuthology.orchestra.run.smithi092.stdout:605 read 0xd498eaa thru 0xd49a6a6 (0x17fd bytes) 2024-09-04T17:46:07.008 INFO:teuthology.orchestra.run.smithi092.stdout:607 write 0x13ce069 thru 0x13ce2cf (0x267 bytes) 2024-09-04T17:46:07.055 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:07.054+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:07.057 INFO:teuthology.orchestra.run.smithi092.stdout:608 trunc from 0xe94422e to 0x256c735 2024-09-04T17:46:07.078 INFO:teuthology.orchestra.run.smithi092.stdout:609 clone 4 order 24 su 524288 sc 9 2024-09-04T17:46:07.155 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:07.158+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:07.158 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:07.158 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:46:07.158 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:46:07.158 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:46:07.158 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.158 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:7 3.1a 3:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-04T17:46:07.159 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:45:56.949499+0000", 2024-09-04T17:46:07.159 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 10.208439017, 2024-09-04T17:46:07.159 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0062683419999999997, 2024-09-04T17:46:07.159 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:07.159 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:07.159 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:07.159 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:07.159 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:07.159 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:07.159 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 7 2024-09-04T17:46:07.159 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.159 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:07.159 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.160 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:07.160 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:56.949499+0000", 2024-09-04T17:46:07.160 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.160 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.160 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.160 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:07.160 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:56.949499+0000", 2024-09-04T17:46:07.160 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.160 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.160 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.160 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:07.160 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:56.949503+0000", 2024-09-04T17:46:07.161 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.8369999999999996e-06 2024-09-04T17:46:07.161 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.161 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.161 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:07.161 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:56.949519+0000", 2024-09-04T17:46:07.161 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6133e-05 2024-09-04T17:46:07.161 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.161 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.161 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:07.161 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:56.949561+0000", 2024-09-04T17:46:07.161 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.1489999999999997e-05 2024-09-04T17:46:07.161 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.161 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.161 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:07.162 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:56.949588+0000", 2024-09-04T17:46:07.162 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.7083000000000001e-05 2024-09-04T17:46:07.162 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.162 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.162 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:07.162 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:56.949727+0000", 2024-09-04T17:46:07.162 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013893600000000001 2024-09-04T17:46:07.162 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.162 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.162 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op must wait for map", 2024-09-04T17:46:07.162 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:56.949741+0000", 2024-09-04T17:46:07.162 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4239e-05 2024-09-04T17:46:07.162 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.162 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.163 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:07.163 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:56.952095+0000", 2024-09-04T17:46:07.163 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.002353946 2024-09-04T17:46:07.163 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.163 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.163 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:07.163 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:56.952263+0000", 2024-09-04T17:46:07.163 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00016754899999999999 2024-09-04T17:46:07.163 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.163 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.163 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 1", 2024-09-04T17:46:07.163 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:56.952471+0000", 2024-09-04T17:46:07.163 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020808400000000001 2024-09-04T17:46:07.163 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.163 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.164 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:07.164 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:56.953704+0000", 2024-09-04T17:46:07.164 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.001233311 2024-09-04T17:46:07.164 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.164 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.164 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:07.164 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:56.955566+0000", 2024-09-04T17:46:07.164 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0018616290000000001 2024-09-04T17:46:07.164 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.164 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.164 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:07.164 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:56.955652+0000", 2024-09-04T17:46:07.164 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.6149000000000005e-05 2024-09-04T17:46:07.164 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.165 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.165 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:07.165 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:56.955767+0000", 2024-09-04T17:46:07.165 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000114956 2024-09-04T17:46:07.165 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.165 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:07.165 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.165 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.165 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.165 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:9 3.1a 3:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-04T17:46:07.165 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:45:57.965066+0000", 2024-09-04T17:46:07.165 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 9.1928718210000007, 2024-09-04T17:46:07.165 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0074753450000000004, 2024-09-04T17:46:07.166 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:07.166 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:07.166 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:07.166 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:07.166 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:07.166 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:07.166 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 9 2024-09-04T17:46:07.166 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.166 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:07.166 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.166 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:07.166 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:57.965066+0000", 2024-09-04T17:46:07.166 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.167 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.167 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.167 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:07.167 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:57.965066+0000", 2024-09-04T17:46:07.167 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.167 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.167 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.167 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:07.167 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:57.965069+0000", 2024-09-04T17:46:07.167 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.2100000000000002e-06 2024-09-04T17:46:07.167 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.167 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.167 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:07.167 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:57.965092+0000", 2024-09-04T17:46:07.168 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2665999999999999e-05 2024-09-04T17:46:07.168 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.168 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.168 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:07.168 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:57.965123+0000", 2024-09-04T17:46:07.168 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.1127000000000002e-05 2024-09-04T17:46:07.168 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.168 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.168 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:07.168 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:57.965141+0000", 2024-09-04T17:46:07.168 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7782999999999999e-05 2024-09-04T17:46:07.168 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.168 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.169 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:07.169 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:57.965259+0000", 2024-09-04T17:46:07.169 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000118466 2024-09-04T17:46:07.169 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.169 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.169 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op must wait for map", 2024-09-04T17:46:07.169 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:57.965270+0000", 2024-09-04T17:46:07.169 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0693000000000001e-05 2024-09-04T17:46:07.169 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.169 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.169 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:07.169 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:57.969607+0000", 2024-09-04T17:46:07.170 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0043376129999999997 2024-09-04T17:46:07.170 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.170 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.170 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:07.170 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:57.969742+0000", 2024-09-04T17:46:07.170 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013452900000000001 2024-09-04T17:46:07.170 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.170 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.170 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 1", 2024-09-04T17:46:07.170 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:57.969978+0000", 2024-09-04T17:46:07.170 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000235701 2024-09-04T17:46:07.170 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.170 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.170 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:07.171 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:57.971977+0000", 2024-09-04T17:46:07.171 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0019995389999999998 2024-09-04T17:46:07.171 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.171 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.171 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:07.171 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:57.972229+0000", 2024-09-04T17:46:07.171 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00025209800000000002 2024-09-04T17:46:07.171 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.171 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.171 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:07.171 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:57.972421+0000", 2024-09-04T17:46:07.171 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019196799999999999 2024-09-04T17:46:07.171 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.171 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.172 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:07.172 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:57.972541+0000", 2024-09-04T17:46:07.172 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00011995200000000001 2024-09-04T17:46:07.172 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.172 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:07.172 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.172 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.172 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.172 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:3167 3.3 3:c43d9109:::rbd_data.113417f3df25.0000000000000002:5 [read 2228224~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-04T17:46:07.172 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:06.781233+0000", 2024-09-04T17:46:07.172 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.37670492100000003, 2024-09-04T17:46:07.173 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0058832980000000003, 2024-09-04T17:46:07.173 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:07.173 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:07.173 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:07.173 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:07.173 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:07.173 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:07.173 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 3167 2024-09-04T17:46:07.173 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.173 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:07.173 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.173 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:07.173 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.781233+0000", 2024-09-04T17:46:07.174 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.174 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.174 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.174 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:07.174 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.781233+0000", 2024-09-04T17:46:07.174 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.174 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.174 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.174 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:07.174 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.781233+0000", 2024-09-04T17:46:07.174 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0100000000000001e-07 2024-09-04T17:46:07.174 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.174 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.175 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:07.175 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.781234+0000", 2024-09-04T17:46:07.175 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.207e-06 2024-09-04T17:46:07.175 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.175 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.175 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:07.175 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.781241+0000", 2024-09-04T17:46:07.175 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3250000000000002e-06 2024-09-04T17:46:07.175 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.175 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.175 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:07.175 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.781245+0000", 2024-09-04T17:46:07.175 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.382e-06 2024-09-04T17:46:07.175 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.176 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.176 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:07.176 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.786923+0000", 2024-09-04T17:46:07.176 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0056780759999999998 2024-09-04T17:46:07.176 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.176 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.176 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:07.176 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.787011+0000", 2024-09-04T17:46:07.176 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8446e-05 2024-09-04T17:46:07.176 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.176 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.176 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:07.176 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.787116+0000", 2024-09-04T17:46:07.177 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000104561 2024-09-04T17:46:07.177 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.177 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:07.177 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.177 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.177 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.177 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:3168 3.3 3:c43d9109:::rbd_data.113417f3df25.0000000000000002:5 [read 2654208~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-04T17:46:07.177 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:06.781252+0000", 2024-09-04T17:46:07.177 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.37668541100000003, 2024-09-04T17:46:07.177 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0061115809999999996, 2024-09-04T17:46:07.177 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:07.177 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:07.177 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:07.178 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:07.178 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:07.178 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:07.178 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 3168 2024-09-04T17:46:07.178 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.178 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:07.178 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.178 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:07.178 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.781252+0000", 2024-09-04T17:46:07.178 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.178 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.178 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.178 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:07.179 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.781252+0000", 2024-09-04T17:46:07.179 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.179 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.179 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.179 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:07.179 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.781252+0000", 2024-09-04T17:46:07.179 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4499999999999998e-07 2024-09-04T17:46:07.179 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.179 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.179 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:07.179 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.781254+0000", 2024-09-04T17:46:07.179 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6869999999999999e-06 2024-09-04T17:46:07.179 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.180 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.180 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:07.180 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.781258+0000", 2024-09-04T17:46:07.180 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2899999999999996e-06 2024-09-04T17:46:07.180 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.180 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.180 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:07.180 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.781264+0000", 2024-09-04T17:46:07.180 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1050000000000003e-06 2024-09-04T17:46:07.180 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.180 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.180 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:07.180 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.787158+0000", 2024-09-04T17:46:07.180 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0058945550000000001 2024-09-04T17:46:07.181 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.181 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.181 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:07.181 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.787248+0000", 2024-09-04T17:46:07.181 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9995999999999997e-05 2024-09-04T17:46:07.181 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.181 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.181 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:07.181 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.787364+0000", 2024-09-04T17:46:07.181 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000115603 2024-09-04T17:46:07.181 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.181 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:07.181 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.182 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.182 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.182 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:3169 3.3 3:c43d9109:::rbd_data.113417f3df25.0000000000000002:5 [read 3080192~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-04T17:46:07.182 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:06.781269+0000", 2024-09-04T17:46:07.182 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.37666818000000002, 2024-09-04T17:46:07.182 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0063353300000000001, 2024-09-04T17:46:07.182 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:07.182 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:07.182 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:07.183 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:07.183 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:07.183 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:07.183 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 3169 2024-09-04T17:46:07.183 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.183 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:07.183 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.183 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:07.183 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.781269+0000", 2024-09-04T17:46:07.183 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.183 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.183 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.183 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:07.183 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.781269+0000", 2024-09-04T17:46:07.184 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.184 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.184 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.184 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:07.184 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.781270+0000", 2024-09-04T17:46:07.184 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.34e-07 2024-09-04T17:46:07.184 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.184 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.184 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:07.184 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.781273+0000", 2024-09-04T17:46:07.184 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4000000000000001e-06 2024-09-04T17:46:07.184 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.184 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.185 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:07.185 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.781290+0000", 2024-09-04T17:46:07.185 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7136999999999999e-05 2024-09-04T17:46:07.185 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.185 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.185 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:07.185 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.781297+0000", 2024-09-04T17:46:07.185 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.5089999999999999e-06 2024-09-04T17:46:07.185 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.185 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.185 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:07.185 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.787401+0000", 2024-09-04T17:46:07.185 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0061036930000000003 2024-09-04T17:46:07.186 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.186 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.186 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:07.186 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.787494+0000", 2024-09-04T17:46:07.186 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.3455000000000006e-05 2024-09-04T17:46:07.186 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.186 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.186 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:07.186 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.787605+0000", 2024-09-04T17:46:07.186 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000110702 2024-09-04T17:46:07.186 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.186 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:07.186 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.186 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.187 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.187 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:3170 3.3 3:c43d9109:::rbd_data.113417f3df25.0000000000000002:5 [read 3506176~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-04T17:46:07.187 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:06.781306+0000", 2024-09-04T17:46:07.187 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.37663186199999998, 2024-09-04T17:46:07.187 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0065387739999999998, 2024-09-04T17:46:07.187 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:07.187 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:07.187 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:07.187 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:07.187 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:07.187 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:07.187 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 3170 2024-09-04T17:46:07.187 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.187 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:07.188 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.188 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:07.188 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.781306+0000", 2024-09-04T17:46:07.188 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.188 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.188 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.188 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:07.188 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.781306+0000", 2024-09-04T17:46:07.188 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.188 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.188 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.188 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:07.188 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.781306+0000", 2024-09-04T17:46:07.188 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1699999999999998e-07 2024-09-04T17:46:07.189 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.189 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.189 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:07.189 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.781309+0000", 2024-09-04T17:46:07.189 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.8490000000000002e-06 2024-09-04T17:46:07.189 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.189 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.189 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:07.189 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.781317+0000", 2024-09-04T17:46:07.189 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.7360000000000001e-06 2024-09-04T17:46:07.189 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.189 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.189 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:07.190 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.781323+0000", 2024-09-04T17:46:07.190 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.319e-06 2024-09-04T17:46:07.190 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.190 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.190 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:07.190 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.787642+0000", 2024-09-04T17:46:07.190 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0063193650000000004 2024-09-04T17:46:07.190 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.190 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.192 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:07.192 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.787737+0000", 2024-09-04T17:46:07.192 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4437000000000004e-05 2024-09-04T17:46:07.192 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.192 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.192 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:07.192 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.787844+0000", 2024-09-04T17:46:07.192 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000107551 2024-09-04T17:46:07.192 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.192 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:07.192 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.192 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.193 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.193 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:3171 3.3 3:c43d9109:::rbd_data.113417f3df25.0000000000000002:5 [read 3932160~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-04T17:46:07.193 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:06.781332+0000", 2024-09-04T17:46:07.193 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.37660564200000002, 2024-09-04T17:46:07.193 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0081394099999999997, 2024-09-04T17:46:07.193 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:07.193 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:07.193 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:07.193 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:07.193 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:07.193 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:07.193 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 3171 2024-09-04T17:46:07.193 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.194 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:07.194 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.194 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:07.194 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.781332+0000", 2024-09-04T17:46:07.194 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.194 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.194 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.194 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:07.194 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.781332+0000", 2024-09-04T17:46:07.194 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.194 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.194 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.194 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:07.194 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.781332+0000", 2024-09-04T17:46:07.195 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2899999999999999e-07 2024-09-04T17:46:07.195 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.195 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.195 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:07.195 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.781335+0000", 2024-09-04T17:46:07.195 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4449999999999999e-06 2024-09-04T17:46:07.195 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.195 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.195 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:07.195 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.781342+0000", 2024-09-04T17:46:07.195 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.5859999999999999e-06 2024-09-04T17:46:07.195 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.195 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.196 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:07.196 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.781348+0000", 2024-09-04T17:46:07.196 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.083e-06 2024-09-04T17:46:07.196 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.196 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.196 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:07.196 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.789238+0000", 2024-09-04T17:46:07.196 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0078893980000000006 2024-09-04T17:46:07.196 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.196 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.196 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:07.196 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.789344+0000", 2024-09-04T17:46:07.197 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00010611399999999999 2024-09-04T17:46:07.197 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.197 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.197 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:07.197 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.789471+0000", 2024-09-04T17:46:07.197 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000127355 2024-09-04T17:46:07.197 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.197 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:07.197 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.197 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.197 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.197 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:3209 3.1 e30/24 3:83661abe:::rbd_data.1134cfbce83d.0000000000000000:head v 30'68, mlcod=30'66)", 2024-09-04T17:46:07.197 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:06.802199+0000", 2024-09-04T17:46:07.198 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.35573861200000001, 2024-09-04T17:46:07.198 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0073018989999999997, 2024-09-04T17:46:07.198 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:07.198 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:07.198 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:07.198 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:07.198 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.198 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:07.198 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.802199+0000", 2024-09-04T17:46:07.198 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.198 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.198 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.198 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:07.198 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.802199+0000", 2024-09-04T17:46:07.199 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.199 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.199 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.199 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:07.199 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.802200+0000", 2024-09-04T17:46:07.199 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2470000000000001e-06 2024-09-04T17:46:07.199 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.199 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.199 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:07.199 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.805454+0000", 2024-09-04T17:46:07.199 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0032535730000000001 2024-09-04T17:46:07.199 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.199 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.199 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:07.199 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.805476+0000", 2024-09-04T17:46:07.200 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2086000000000002e-05 2024-09-04T17:46:07.200 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.200 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.200 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:07.200 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.805486+0000", 2024-09-04T17:46:07.200 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0366e-05 2024-09-04T17:46:07.200 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.200 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.200 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:07.200 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.805559+0000", 2024-09-04T17:46:07.200 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2997999999999995e-05 2024-09-04T17:46:07.200 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.200 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.201 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:07.201 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.805576+0000", 2024-09-04T17:46:07.201 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6665000000000001e-05 2024-09-04T17:46:07.201 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.201 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.201 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:07.201 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.809425+0000", 2024-09-04T17:46:07.201 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00384915 2024-09-04T17:46:07.201 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.201 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.201 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:07.201 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.809501+0000", 2024-09-04T17:46:07.201 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.5814000000000003e-05 2024-09-04T17:46:07.201 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.202 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:07.202 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.202 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.202 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.202 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:3901 3.3 3:c43d9109:::rbd_data.113417f3df25.0000000000000002:5 [read 7634944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-04T17:46:07.202 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:06.891233+0000", 2024-09-04T17:46:07.202 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.266704881, 2024-09-04T17:46:07.202 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0057881399999999998, 2024-09-04T17:46:07.202 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:07.202 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:07.202 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:07.202 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:07.202 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:07.203 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:07.203 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 3901 2024-09-04T17:46:07.203 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.203 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:07.203 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.203 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:07.203 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.891233+0000", 2024-09-04T17:46:07.203 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.203 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.203 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.203 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:07.203 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.891233+0000", 2024-09-04T17:46:07.203 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.204 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.204 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.204 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:07.204 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.891233+0000", 2024-09-04T17:46:07.204 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4900000000000002e-07 2024-09-04T17:46:07.204 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.204 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.204 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:07.204 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.891234+0000", 2024-09-04T17:46:07.204 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2470000000000001e-06 2024-09-04T17:46:07.204 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.204 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.204 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:07.205 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.891238+0000", 2024-09-04T17:46:07.205 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.9580000000000001e-06 2024-09-04T17:46:07.205 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.205 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.205 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:07.205 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.891241+0000", 2024-09-04T17:46:07.205 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.2049999999999998e-06 2024-09-04T17:46:07.205 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.205 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.205 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:07.205 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.896871+0000", 2024-09-04T17:46:07.205 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0056294789999999997 2024-09-04T17:46:07.205 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.206 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.206 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:07.206 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.896945+0000", 2024-09-04T17:46:07.206 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4071000000000003e-05 2024-09-04T17:46:07.206 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.206 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.206 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:07.206 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.897021+0000", 2024-09-04T17:46:07.206 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.5931000000000002e-05 2024-09-04T17:46:07.206 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.206 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:07.206 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.206 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.206 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.207 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:3902 3.3 3:c43d9109:::rbd_data.113417f3df25.0000000000000002:5 [read 8060928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-04T17:46:07.207 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:06.891245+0000", 2024-09-04T17:46:07.207 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.26669219399999999, 2024-09-04T17:46:07.207 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0059582359999999996, 2024-09-04T17:46:07.207 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:07.207 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:07.207 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:07.207 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:07.207 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:07.207 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:07.207 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 3902 2024-09-04T17:46:07.207 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.207 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:07.208 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.208 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:07.208 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.891245+0000", 2024-09-04T17:46:07.208 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.208 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.208 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.208 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:07.208 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.891245+0000", 2024-09-04T17:46:07.208 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.208 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.208 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.208 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:07.208 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.891246+0000", 2024-09-04T17:46:07.209 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6899999999999999e-07 2024-09-04T17:46:07.209 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.209 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.209 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:07.209 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.891248+0000", 2024-09-04T17:46:07.209 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.903e-06 2024-09-04T17:46:07.209 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.209 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.209 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:07.209 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.891251+0000", 2024-09-04T17:46:07.209 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.9369999999999997e-06 2024-09-04T17:46:07.209 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.209 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.209 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:07.210 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.891255+0000", 2024-09-04T17:46:07.210 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.8010000000000001e-06 2024-09-04T17:46:07.210 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.210 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.210 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:07.210 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.897052+0000", 2024-09-04T17:46:07.210 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0057971129999999996 2024-09-04T17:46:07.210 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.210 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.210 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:07.210 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.897127+0000", 2024-09-04T17:46:07.210 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4540000000000001e-05 2024-09-04T17:46:07.210 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.211 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.211 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:07.211 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.897204+0000", 2024-09-04T17:46:07.211 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.6673000000000003e-05 2024-09-04T17:46:07.211 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.211 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:07.211 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.211 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.211 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.211 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:3963 3.3 3:c43d9109:::rbd_data.113417f3df25.0000000000000002:5 [read 6815744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-04T17:46:07.211 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:06.892991+0000", 2024-09-04T17:46:07.211 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.26494640899999999, 2024-09-04T17:46:07.211 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0056712139999999999, 2024-09-04T17:46:07.211 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:07.212 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:07.212 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:07.212 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:07.212 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:07.212 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:07.212 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 3963 2024-09-04T17:46:07.212 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.212 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:07.212 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.212 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:07.212 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.892991+0000", 2024-09-04T17:46:07.212 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.213 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.213 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.213 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:07.213 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.892991+0000", 2024-09-04T17:46:07.213 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.213 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.213 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.213 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:07.213 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.892992+0000", 2024-09-04T17:46:07.213 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.8700000000000005e-07 2024-09-04T17:46:07.213 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.213 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.214 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:07.214 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.892995+0000", 2024-09-04T17:46:07.214 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.7520000000000001e-06 2024-09-04T17:46:07.214 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.214 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.214 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:07.214 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.893003+0000", 2024-09-04T17:46:07.214 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.0949999999999996e-06 2024-09-04T17:46:07.214 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.214 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.214 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:07.214 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.893009+0000", 2024-09-04T17:46:07.214 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.088e-06 2024-09-04T17:46:07.215 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.215 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.215 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:07.215 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.898517+0000", 2024-09-04T17:46:07.215 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0055085430000000003 2024-09-04T17:46:07.215 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.215 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.215 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:07.215 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.898591+0000", 2024-09-04T17:46:07.215 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3559999999999994e-05 2024-09-04T17:46:07.215 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.215 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.215 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:07.215 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.898662+0000", 2024-09-04T17:46:07.216 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1489000000000006e-05 2024-09-04T17:46:07.216 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.216 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:07.216 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.216 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.216 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.216 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:3964 3.3 3:c43d9109:::rbd_data.113417f3df25.0000000000000002:5 [read 7241728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-04T17:46:07.216 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:06.893044+0000", 2024-09-04T17:46:07.216 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.26489326499999999, 2024-09-04T17:46:07.216 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0058055290000000002, 2024-09-04T17:46:07.216 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:07.216 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:07.216 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:07.216 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:07.217 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:07.217 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:07.217 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 3964 2024-09-04T17:46:07.217 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.217 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:07.217 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.217 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:07.217 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.893044+0000", 2024-09-04T17:46:07.217 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.217 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.217 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.217 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:07.217 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.893044+0000", 2024-09-04T17:46:07.218 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.218 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.218 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.218 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:07.218 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.893045+0000", 2024-09-04T17:46:07.218 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.5300000000000003e-07 2024-09-04T17:46:07.218 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.218 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.218 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:07.218 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.893048+0000", 2024-09-04T17:46:07.218 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.9550000000000001e-06 2024-09-04T17:46:07.218 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.218 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.218 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:07.219 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.893057+0000", 2024-09-04T17:46:07.219 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.3780000000000007e-06 2024-09-04T17:46:07.219 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.219 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.219 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:07.219 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.893064+0000", 2024-09-04T17:46:07.219 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.8240000000000002e-06 2024-09-04T17:46:07.219 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.219 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.219 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:07.219 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.898693+0000", 2024-09-04T17:46:07.219 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0056285040000000003 2024-09-04T17:46:07.220 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.220 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.220 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:07.220 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.898775+0000", 2024-09-04T17:46:07.220 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.2210000000000001e-05 2024-09-04T17:46:07.220 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.220 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.220 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:07.220 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.898850+0000", 2024-09-04T17:46:07.220 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4805e-05 2024-09-04T17:46:07.220 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.220 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:07.220 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.221 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.221 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.221 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:3965 3.3 3:c43d9109:::rbd_data.113417f3df25.0000000000000002:5 [read 7667712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-04T17:46:07.221 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:06.893096+0000", 2024-09-04T17:46:07.221 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.264841204, 2024-09-04T17:46:07.221 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0059163669999999996, 2024-09-04T17:46:07.221 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:07.221 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:07.221 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:07.221 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:07.221 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:07.221 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:07.221 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 3965 2024-09-04T17:46:07.222 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.222 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:07.222 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.222 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:07.222 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.893096+0000", 2024-09-04T17:46:07.222 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.222 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.222 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.222 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:07.222 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.893096+0000", 2024-09-04T17:46:07.222 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.222 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.222 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.222 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:07.223 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.893097+0000", 2024-09-04T17:46:07.223 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.9299999999999997e-07 2024-09-04T17:46:07.223 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.223 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.223 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:07.223 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.893100+0000", 2024-09-04T17:46:07.223 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.9500000000000001e-06 2024-09-04T17:46:07.223 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.223 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.223 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:07.223 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.893109+0000", 2024-09-04T17:46:07.223 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8540000000000003e-06 2024-09-04T17:46:07.223 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.224 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.224 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:07.224 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.893118+0000", 2024-09-04T17:46:07.224 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7560000000000005e-06 2024-09-04T17:46:07.224 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.224 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.224 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:07.224 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.898880+0000", 2024-09-04T17:46:07.224 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0057624249999999998 2024-09-04T17:46:07.224 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.224 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.224 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:07.224 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.898948+0000", 2024-09-04T17:46:07.224 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7895e-05 2024-09-04T17:46:07.225 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.225 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.225 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:07.225 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.899013+0000", 2024-09-04T17:46:07.225 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.4694000000000001e-05 2024-09-04T17:46:07.225 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.225 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:07.225 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.225 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.225 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.225 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:3966 3.3 3:c43d9109:::rbd_data.113417f3df25.0000000000000002:5 [read 8093696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-04T17:46:07.225 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:06.893128+0000", 2024-09-04T17:46:07.225 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.264810025, 2024-09-04T17:46:07.226 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0060456520000000003, 2024-09-04T17:46:07.226 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:07.226 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:07.226 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:07.226 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:07.226 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:07.226 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:07.226 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 3966 2024-09-04T17:46:07.226 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.226 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:07.226 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.226 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:07.226 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.893128+0000", 2024-09-04T17:46:07.226 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.227 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.227 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.227 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:07.227 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.893128+0000", 2024-09-04T17:46:07.227 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.227 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.227 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.227 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:07.227 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.893128+0000", 2024-09-04T17:46:07.227 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6899999999999997e-07 2024-09-04T17:46:07.227 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.227 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.227 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:07.228 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.893131+0000", 2024-09-04T17:46:07.228 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4559999999999999e-06 2024-09-04T17:46:07.228 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.228 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.228 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:07.228 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.893138+0000", 2024-09-04T17:46:07.228 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.7759999999999997e-06 2024-09-04T17:46:07.228 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.228 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.228 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:07.228 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.893144+0000", 2024-09-04T17:46:07.228 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.818e-06 2024-09-04T17:46:07.228 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.229 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.229 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:07.229 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.899041+0000", 2024-09-04T17:46:07.229 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0058967359999999996 2024-09-04T17:46:07.229 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.229 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.229 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:07.229 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.899108+0000", 2024-09-04T17:46:07.229 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7452000000000004e-05 2024-09-04T17:46:07.229 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.229 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.229 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:07.229 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.899173+0000", 2024-09-04T17:46:07.229 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.4844999999999993e-05 2024-09-04T17:46:07.230 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.230 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:07.230 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.230 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.230 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.230 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4028 3.3 3:c43d9109:::rbd_data.113417f3df25.0000000000000002:5 [read 7274496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-04T17:46:07.230 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:06.894780+0000", 2024-09-04T17:46:07.230 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.26315781799999999, 2024-09-04T17:46:07.230 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0057744709999999998, 2024-09-04T17:46:07.230 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:07.230 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:07.230 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:07.230 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:07.231 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:07.231 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:07.231 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4028 2024-09-04T17:46:07.231 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.231 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:07.231 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.231 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:07.231 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.894780+0000", 2024-09-04T17:46:07.231 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.231 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.231 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.231 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:07.231 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.894780+0000", 2024-09-04T17:46:07.232 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.232 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.232 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.232 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:07.232 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.894780+0000", 2024-09-04T17:46:07.232 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.8400000000000005e-07 2024-09-04T17:46:07.232 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.232 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.232 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:07.232 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.894783+0000", 2024-09-04T17:46:07.232 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5019999999999999e-06 2024-09-04T17:46:07.232 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.232 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.233 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:07.233 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.894790+0000", 2024-09-04T17:46:07.233 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3930000000000002e-06 2024-09-04T17:46:07.233 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.233 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.233 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:07.233 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.894798+0000", 2024-09-04T17:46:07.233 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.5789999999999998e-06 2024-09-04T17:46:07.233 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.233 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.233 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:07.233 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.900426+0000", 2024-09-04T17:46:07.234 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.005628386 2024-09-04T17:46:07.234 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.234 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.234 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:07.234 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.900493+0000", 2024-09-04T17:46:07.234 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7309000000000002e-05 2024-09-04T17:46:07.234 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.234 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.234 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:07.234 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.900554+0000", 2024-09-04T17:46:07.234 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.0817999999999999e-05 2024-09-04T17:46:07.234 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.234 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:07.235 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.235 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.235 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.235 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4029 3.3 3:c43d9109:::rbd_data.113417f3df25.0000000000000002:5 [read 7700480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-04T17:46:07.235 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:06.894806+0000", 2024-09-04T17:46:07.235 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.26313125599999998, 2024-09-04T17:46:07.235 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0059059469999999999, 2024-09-04T17:46:07.235 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:07.235 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:07.235 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:07.235 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:07.235 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:07.235 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:07.236 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4029 2024-09-04T17:46:07.236 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.236 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:07.236 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.236 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:07.236 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.894806+0000", 2024-09-04T17:46:07.236 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.236 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.236 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.236 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:07.236 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.894806+0000", 2024-09-04T17:46:07.236 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.236 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.236 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.236 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:07.237 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.894807+0000", 2024-09-04T17:46:07.237 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9800000000000004e-07 2024-09-04T17:46:07.237 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.237 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.237 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:07.237 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.894809+0000", 2024-09-04T17:46:07.237 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5210000000000001e-06 2024-09-04T17:46:07.237 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.237 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.237 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:07.237 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.894818+0000", 2024-09-04T17:46:07.237 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.3979999999999999e-06 2024-09-04T17:46:07.237 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.238 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.238 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:07.238 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.894824+0000", 2024-09-04T17:46:07.238 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6150000000000002e-06 2024-09-04T17:46:07.238 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.238 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.238 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:07.238 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.900582+0000", 2024-09-04T17:46:07.238 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0057579950000000001 2024-09-04T17:46:07.238 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.238 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.238 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:07.238 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.900648+0000", 2024-09-04T17:46:07.238 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.5982000000000006e-05 2024-09-04T17:46:07.239 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.239 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.239 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:07.239 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.900712+0000", 2024-09-04T17:46:07.239 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.3937999999999997e-05 2024-09-04T17:46:07.239 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.239 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:07.239 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.239 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.239 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.239 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4030 3.3 3:c43d9109:::rbd_data.113417f3df25.0000000000000002:5 [read 8126464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-04T17:46:07.239 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:06.895174+0000", 2024-09-04T17:46:07.239 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.26276333600000001, 2024-09-04T17:46:07.239 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0057327560000000003, 2024-09-04T17:46:07.239 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:07.240 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:07.240 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:07.240 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:07.240 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:07.240 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:07.240 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4030 2024-09-04T17:46:07.240 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.240 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:07.240 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.240 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:07.240 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.895174+0000", 2024-09-04T17:46:07.240 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.240 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.241 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.241 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:07.241 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.895174+0000", 2024-09-04T17:46:07.241 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.241 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.241 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.241 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:07.241 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.895175+0000", 2024-09-04T17:46:07.241 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.0800000000000003e-07 2024-09-04T17:46:07.241 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.241 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.241 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:07.241 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.895179+0000", 2024-09-04T17:46:07.241 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0080000000000004e-06 2024-09-04T17:46:07.242 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.242 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.242 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:07.242 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.895191+0000", 2024-09-04T17:46:07.242 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1446e-05 2024-09-04T17:46:07.242 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.242 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.242 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:07.242 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.895199+0000", 2024-09-04T17:46:07.242 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.2290000000000003e-06 2024-09-04T17:46:07.242 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.242 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.242 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:07.243 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.900740+0000", 2024-09-04T17:46:07.243 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0055411530000000001 2024-09-04T17:46:07.243 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.243 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.243 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:07.243 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.900807+0000", 2024-09-04T17:46:07.243 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7311999999999995e-05 2024-09-04T17:46:07.243 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.243 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.243 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:07.243 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.900907+0000", 2024-09-04T17:46:07.243 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.9699999999999998e-05 2024-09-04T17:46:07.243 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.243 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:07.244 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.244 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.244 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.244 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4094 3.3 3:c43d9109:::rbd_data.113417f3df25.0000000000000002:5 [read 7733248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-04T17:46:07.244 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:06.896813+0000", 2024-09-04T17:46:07.244 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.261124358, 2024-09-04T17:46:07.244 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0056949449999999999, 2024-09-04T17:46:07.244 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:07.244 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:07.244 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:07.244 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:07.244 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:07.245 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:07.245 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4094 2024-09-04T17:46:07.245 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.245 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:07.245 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.245 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:07.245 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.896813+0000", 2024-09-04T17:46:07.245 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.245 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.245 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.245 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:07.245 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.896813+0000", 2024-09-04T17:46:07.245 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.245 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.246 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.246 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:07.246 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.896814+0000", 2024-09-04T17:46:07.246 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2e-07 2024-09-04T17:46:07.246 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.246 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.246 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:07.246 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.896816+0000", 2024-09-04T17:46:07.246 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6479999999999999e-06 2024-09-04T17:46:07.246 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.246 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.246 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:07.246 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.896824+0000", 2024-09-04T17:46:07.247 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4139999999999997e-06 2024-09-04T17:46:07.247 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.247 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.247 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:07.247 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.896829+0000", 2024-09-04T17:46:07.247 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6810000000000003e-06 2024-09-04T17:46:07.247 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.247 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.247 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:07.247 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.902372+0000", 2024-09-04T17:46:07.247 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0055423299999999998 2024-09-04T17:46:07.247 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.247 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.248 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:07.248 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.902440+0000", 2024-09-04T17:46:07.248 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.8570000000000002e-05 2024-09-04T17:46:07.248 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.248 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.248 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:07.248 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.902508+0000", 2024-09-04T17:46:07.248 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7781999999999996e-05 2024-09-04T17:46:07.248 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.248 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:07.248 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.248 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.248 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.248 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4095 3.3 3:c43d9109:::rbd_data.113417f3df25.0000000000000002:5 [read 8159232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-04T17:46:07.249 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:06.896865+0000", 2024-09-04T17:46:07.249 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.26107217300000002, 2024-09-04T17:46:07.249 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0080564920000000002, 2024-09-04T17:46:07.249 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:07.249 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:07.249 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:07.249 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:07.249 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:07.249 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:07.249 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4095 2024-09-04T17:46:07.249 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.249 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:07.249 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.250 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:07.250 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.896865+0000", 2024-09-04T17:46:07.250 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.250 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.250 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.250 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:07.250 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.896865+0000", 2024-09-04T17:46:07.250 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.250 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.250 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.250 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:07.250 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.896866+0000", 2024-09-04T17:46:07.250 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.9599999999999998e-07 2024-09-04T17:46:07.251 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.251 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.251 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:07.251 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.896869+0000", 2024-09-04T17:46:07.251 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.1300000000000001e-06 2024-09-04T17:46:07.251 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.251 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.251 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:07.251 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.896880+0000", 2024-09-04T17:46:07.251 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.046e-05 2024-09-04T17:46:07.251 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.251 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.251 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:07.251 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.896887+0000", 2024-09-04T17:46:07.252 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.9410000000000001e-06 2024-09-04T17:46:07.252 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.252 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.252 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:07.252 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.904717+0000", 2024-09-04T17:46:07.252 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0078303009999999996 2024-09-04T17:46:07.252 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.252 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.252 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:07.252 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.904797+0000", 2024-09-04T17:46:07.252 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.0348999999999999e-05 2024-09-04T17:46:07.252 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.252 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.253 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:07.253 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.904922+0000", 2024-09-04T17:46:07.253 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00012451500000000001 2024-09-04T17:46:07.253 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.253 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:07.253 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.253 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.253 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.253 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4264 3.9 3:970e51a0:::rbd_data.1134cfbce83d.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 1597545~615] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-04T17:46:07.253 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.021843+0000", 2024-09-04T17:46:07.253 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.136094135, 2024-09-04T17:46:07.253 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.015077662, 2024-09-04T17:46:07.254 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:07.254 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:07.254 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:07.254 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:07.254 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:07.254 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:07.254 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4264 2024-09-04T17:46:07.254 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.254 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:07.254 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.254 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:07.254 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.021843+0000", 2024-09-04T17:46:07.254 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.255 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.255 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.255 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:07.255 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.021843+0000", 2024-09-04T17:46:07.255 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.255 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.255 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.255 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:07.255 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.021845+0000", 2024-09-04T17:46:07.255 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.328e-06 2024-09-04T17:46:07.255 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.255 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.255 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:07.255 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.026201+0000", 2024-09-04T17:46:07.255 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.004355935 2024-09-04T17:46:07.256 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.256 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.256 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:07.256 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.026214+0000", 2024-09-04T17:46:07.256 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3315999999999999e-05 2024-09-04T17:46:07.256 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.256 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.256 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:07.256 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.026221+0000", 2024-09-04T17:46:07.256 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7070000000000003e-06 2024-09-04T17:46:07.256 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.256 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.256 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:07.256 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.026266+0000", 2024-09-04T17:46:07.257 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.5120999999999997e-05 2024-09-04T17:46:07.257 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.257 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.257 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:07.257 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.026375+0000", 2024-09-04T17:46:07.257 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00010935 2024-09-04T17:46:07.257 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.257 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.257 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 2", 2024-09-04T17:46:07.257 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.026610+0000", 2024-09-04T17:46:07.257 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00023530400000000001 2024-09-04T17:46:07.257 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.258 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.258 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:07.258 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.032096+0000", 2024-09-04T17:46:07.258 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0054859050000000001 2024-09-04T17:46:07.258 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.258 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.258 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:07.258 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.036851+0000", 2024-09-04T17:46:07.258 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0047543409999999996 2024-09-04T17:46:07.258 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.258 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.258 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:07.258 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.036890+0000", 2024-09-04T17:46:07.259 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.9128999999999998e-05 2024-09-04T17:46:07.259 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.259 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.259 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:07.259 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.036921+0000", 2024-09-04T17:46:07.259 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.1226000000000002e-05 2024-09-04T17:46:07.259 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.259 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:07.259 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.259 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.259 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:07.259 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:07.261 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is set 2024-09-04T17:46:07.261 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-04T17:46:07.262 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:07.260+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:07.335 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone3 from 0x256c735 (overlap 0x256c735) to 0x0 2024-09-04T17:46:07.364 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:07.360+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:07.462 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:07.460+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:07.470 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:07.470 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:46:07.470 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:46:07.470 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:07.484 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-04T17:46:07.559 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #2, image image_client.0-clone2 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent3 2024-09-04T17:46:07.571 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:07.572+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:07.649 INFO:teuthology.orchestra.run.smithi092.stdout:611 read 0x259ce6 thru 0x25e024 (0x433f bytes) 2024-09-04T17:46:07.651 INFO:teuthology.orchestra.run.smithi092.stdout:612 read 0x21cf852 thru 0x21d9ebe (0xa66d bytes) 2024-09-04T17:46:07.653 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:07.654 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:46:07.654 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:46:07.654 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:46:07.654 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:07.654 INFO:teuthology.orchestra.run.smithi092.stdout:613 read 0x1d0bfe2 thru 0x1d0e85b (0x287a bytes) 2024-09-04T17:46:07.656 INFO:teuthology.orchestra.run.smithi092.stdout:614 read 0xaf794a thru 0xb02830 (0xaee7 bytes) 2024-09-04T17:46:07.658 INFO:teuthology.orchestra.run.smithi092.stdout:615 punch from 0x1daacf8 to 0x1db6de4, (0xc0ec bytes) 2024-09-04T17:46:07.663 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:46:07.664+0000 7f505d642640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26446) UID: 0 2024-09-04T17:46:07.669 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-04T17:46:07.674 INFO:teuthology.orchestra.run.smithi092.stdout:616 trunc from 0x256c735 to 0x551f4df 2024-09-04T17:46:07.686 INFO:teuthology.orchestra.run.smithi092.stdout:617 write 0x4cf5039 thru 0x4d0171c (0xc6e4 bytes) 2024-09-04T17:46:07.742 INFO:teuthology.orchestra.run.smithi092.stdout:618 read 0x3421891 thru 0x3431055 (0xf7c5 bytes) 2024-09-04T17:46:07.743 INFO:teuthology.orchestra.run.smithi092.stdout:619 writesame 0x177d9ce thru 0x1783d8c data_size 0x1(0x63bf bytes) 2024-09-04T17:46:07.764 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:07.762+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:07.797 INFO:teuthology.orchestra.run.smithi092.stdout:623 read 0x502f4a thru 0x505c51 (0x2d08 bytes) 2024-09-04T17:46:07.798 INFO:teuthology.orchestra.run.smithi092.stdout:624 punch from 0x2b803f1 to 0x2b80735, (0x344 bytes) 2024-09-04T17:46:07.809 INFO:teuthology.orchestra.run.smithi092.stdout:626 trunc from 0x551f4df to 0x959bc03 2024-09-04T17:46:07.826 INFO:teuthology.orchestra.run.smithi092.stdout:627 read 0xc90c3f thru 0xc91626 (0x9e8 bytes) 2024-09-04T17:46:07.843 INFO:teuthology.orchestra.run.smithi092.stdout:628 write 0x9b037f6 thru 0x9b0fc31 (0xc43c bytes) 2024-09-04T17:46:07.845 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:07.845 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:46:07.845 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:46:07.845 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:46:07.845 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.846 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4713 3.c 3:370edc5a:::rbd_data.113417f3df25.0000000000000000:5 [read 3047424~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:07.846 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.357178+0000", 2024-09-04T17:46:07.846 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.48773265300000002, 2024-09-04T17:46:07.846 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.028824949999999998, 2024-09-04T17:46:07.846 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:07.846 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:07.846 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:07.846 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:07.846 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:07.847 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:07.847 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4713 2024-09-04T17:46:07.847 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.847 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:07.847 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.847 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:07.847 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357178+0000", 2024-09-04T17:46:07.847 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.847 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.847 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.848 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:07.848 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357178+0000", 2024-09-04T17:46:07.848 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.848 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.848 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.848 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:07.848 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357179+0000", 2024-09-04T17:46:07.848 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.0299999999999998e-07 2024-09-04T17:46:07.849 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.849 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.849 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:07.849 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357182+0000", 2024-09-04T17:46:07.849 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.8140000000000001e-06 2024-09-04T17:46:07.849 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.849 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.849 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:07.850 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357192+0000", 2024-09-04T17:46:07.850 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5670000000000001e-06 2024-09-04T17:46:07.850 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.850 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.850 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:07.850 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357198+0000", 2024-09-04T17:46:07.850 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.9659999999999999e-06 2024-09-04T17:46:07.851 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.851 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.851 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:07.851 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.385796+0000", 2024-09-04T17:46:07.851 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.028597305999999999 2024-09-04T17:46:07.851 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.852 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.852 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:07.852 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.385871+0000", 2024-09-04T17:46:07.852 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.5469999999999994e-05 2024-09-04T17:46:07.852 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.852 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.853 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:07.853 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.386003+0000", 2024-09-04T17:46:07.853 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013202400000000001 2024-09-04T17:46:07.853 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.853 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:07.854 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.854 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.854 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.855 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4715 3.c 3:370edc5a:::rbd_data.113417f3df25.0000000000000000:5 [read 3473408~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:07.855 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.357229+0000", 2024-09-04T17:46:07.855 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.48768225199999998, 2024-09-04T17:46:07.855 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.029072164000000001, 2024-09-04T17:46:07.855 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:07.856 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:07.856 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:07.856 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:07.856 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:07.856 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:07.857 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4715 2024-09-04T17:46:07.857 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.857 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:07.858 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.858 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:07.859 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357229+0000", 2024-09-04T17:46:07.859 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.859 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.860 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.860 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:07.860 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357229+0000", 2024-09-04T17:46:07.860 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.860 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.860 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.860 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:07.861 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357229+0000", 2024-09-04T17:46:07.861 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1299999999999999e-07 2024-09-04T17:46:07.861 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.861 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.861 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:07.862 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357233+0000", 2024-09-04T17:46:07.862 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.754e-06 2024-09-04T17:46:07.862 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.862 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.862 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:07.862 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357245+0000", 2024-09-04T17:46:07.862 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1588e-05 2024-09-04T17:46:07.863 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.863 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.863 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:07.863 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357253+0000", 2024-09-04T17:46:07.863 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.0369999999999995e-06 2024-09-04T17:46:07.863 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.863 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.864 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:07.864 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.386036+0000", 2024-09-04T17:46:07.864 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.028783534 2024-09-04T17:46:07.864 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.864 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.864 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:07.864 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.386132+0000", 2024-09-04T17:46:07.865 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5561000000000002e-05 2024-09-04T17:46:07.865 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.865 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.865 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:07.865 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.386301+0000", 2024-09-04T17:46:07.865 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00016897700000000001 2024-09-04T17:46:07.865 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.866 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:07.866 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.866 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.866 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.866 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4716 3.c 3:370edc5a:::rbd_data.113417f3df25.0000000000000000:5 [read 3899392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:07.866 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.357265+0000", 2024-09-04T17:46:07.866 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.48764586799999998, 2024-09-04T17:46:07.866 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.029266948000000001, 2024-09-04T17:46:07.866 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:07.866 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:07.867 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:07.867 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:07.867 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:07.867 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:07.867 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4716 2024-09-04T17:46:07.867 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.867 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:07.867 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.867 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:07.867 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357265+0000", 2024-09-04T17:46:07.867 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.868 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.868 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.868 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:07.868 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357265+0000", 2024-09-04T17:46:07.868 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.868 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.868 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.868 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:07.868 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357266+0000", 2024-09-04T17:46:07.868 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1200000000000001e-07 2024-09-04T17:46:07.868 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.868 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.868 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:07.869 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357276+0000", 2024-09-04T17:46:07.869 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0208e-05 2024-09-04T17:46:07.869 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.869 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.869 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:07.869 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357285+0000", 2024-09-04T17:46:07.869 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9959999999999999e-06 2024-09-04T17:46:07.869 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.869 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.869 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:07.869 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357293+0000", 2024-09-04T17:46:07.869 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.311e-06 2024-09-04T17:46:07.869 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.870 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.870 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:07.870 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.386333+0000", 2024-09-04T17:46:07.870 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.029040442999999999 2024-09-04T17:46:07.870 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.870 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.870 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:07.870 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.386413+0000", 2024-09-04T17:46:07.870 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.9716999999999996e-05 2024-09-04T17:46:07.870 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.870 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.870 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:07.870 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.386532+0000", 2024-09-04T17:46:07.871 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000119361 2024-09-04T17:46:07.871 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.871 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:07.871 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.871 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.871 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.871 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4739 3.c 3:37f94306:::rbd_data.113417f3df25.0000000000000006:5 [read 229376~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:07.871 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.357303+0000", 2024-09-04T17:46:07.871 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.48760776099999997, 2024-09-04T17:46:07.871 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.029554939999999998, 2024-09-04T17:46:07.871 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:07.872 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:07.872 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:07.872 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:07.872 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:07.872 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:07.872 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4739 2024-09-04T17:46:07.872 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.872 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:07.872 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.872 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:07.872 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357303+0000", 2024-09-04T17:46:07.872 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.873 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.873 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.873 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:07.873 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357303+0000", 2024-09-04T17:46:07.873 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.873 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.873 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.873 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:07.873 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357304+0000", 2024-09-04T17:46:07.873 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.13e-07 2024-09-04T17:46:07.873 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.873 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.873 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:07.874 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357307+0000", 2024-09-04T17:46:07.874 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.8660000000000002e-06 2024-09-04T17:46:07.874 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.874 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.874 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:07.874 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357315+0000", 2024-09-04T17:46:07.874 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.4530000000000002e-06 2024-09-04T17:46:07.874 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.874 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.874 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:07.874 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357322+0000", 2024-09-04T17:46:07.874 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6470000000000001e-06 2024-09-04T17:46:07.874 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.875 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.875 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:07.875 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.386582+0000", 2024-09-04T17:46:07.875 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.029259721999999998 2024-09-04T17:46:07.875 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.875 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.875 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:07.875 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.386703+0000", 2024-09-04T17:46:07.875 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00012124400000000001 2024-09-04T17:46:07.875 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.875 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.875 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:07.875 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.386858+0000", 2024-09-04T17:46:07.876 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00015539500000000001 2024-09-04T17:46:07.876 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.876 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:07.876 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.876 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.876 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.876 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4742 3.c 3:37f94306:::rbd_data.113417f3df25.0000000000000006:5 [read 655360~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:07.876 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.357331+0000", 2024-09-04T17:46:07.876 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.487579808, 2024-09-04T17:46:07.876 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.029685888000000001, 2024-09-04T17:46:07.876 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:07.876 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:07.876 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:07.877 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:07.877 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:07.877 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:07.877 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4742 2024-09-04T17:46:07.877 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.877 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:07.877 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.877 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:07.877 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357331+0000", 2024-09-04T17:46:07.877 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.877 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.877 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.878 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:07.878 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357331+0000", 2024-09-04T17:46:07.878 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.878 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.878 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.878 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:07.878 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357332+0000", 2024-09-04T17:46:07.878 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.0299999999999999e-07 2024-09-04T17:46:07.878 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.878 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.878 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:07.878 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357335+0000", 2024-09-04T17:46:07.878 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.791e-06 2024-09-04T17:46:07.879 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.879 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.879 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:07.879 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357343+0000", 2024-09-04T17:46:07.879 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.3610000000000001e-06 2024-09-04T17:46:07.879 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.879 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.879 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:07.879 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357349+0000", 2024-09-04T17:46:07.879 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.2879999999999998e-06 2024-09-04T17:46:07.879 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.879 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.879 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:07.879 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.386890+0000", 2024-09-04T17:46:07.880 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.029540789000000001 2024-09-04T17:46:07.880 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.880 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.880 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:07.880 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.386938+0000", 2024-09-04T17:46:07.880 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.8140000000000003e-05 2024-09-04T17:46:07.880 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.880 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.880 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:07.880 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.387017+0000", 2024-09-04T17:46:07.880 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.8916000000000001e-05 2024-09-04T17:46:07.880 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.880 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:07.881 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.881 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.881 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.881 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4747 3.c 3:37f94306:::rbd_data.113417f3df25.0000000000000006:5 [read 1081344~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:07.881 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.357360+0000", 2024-09-04T17:46:07.881 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.48755125500000002, 2024-09-04T17:46:07.881 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.029971241999999999, 2024-09-04T17:46:07.881 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:07.881 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:07.881 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:07.881 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:07.881 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:07.881 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:07.881 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4747 2024-09-04T17:46:07.882 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.882 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:07.882 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.882 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:07.882 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357360+0000", 2024-09-04T17:46:07.882 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.882 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.882 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.882 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:07.882 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357360+0000", 2024-09-04T17:46:07.882 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.882 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.882 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.883 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:07.883 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357360+0000", 2024-09-04T17:46:07.883 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.75e-07 2024-09-04T17:46:07.883 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.883 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.883 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:07.883 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357363+0000", 2024-09-04T17:46:07.883 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.683e-06 2024-09-04T17:46:07.883 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.883 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.883 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:07.883 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357371+0000", 2024-09-04T17:46:07.883 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.9289999999999998e-06 2024-09-04T17:46:07.883 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.884 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.884 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:07.884 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.358831+0000", 2024-09-04T17:46:07.884 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0014602969999999999 2024-09-04T17:46:07.884 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.884 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.884 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:07.884 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.387067+0000", 2024-09-04T17:46:07.884 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.028235577000000001 2024-09-04T17:46:07.884 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.884 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.884 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:07.884 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.387167+0000", 2024-09-04T17:46:07.885 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.9773999999999997e-05 2024-09-04T17:46:07.885 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.885 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.885 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:07.885 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.387331+0000", 2024-09-04T17:46:07.885 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00016440700000000001 2024-09-04T17:46:07.885 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.885 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:07.885 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.885 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.885 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.885 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4786 3.c 3:370edc5a:::rbd_data.113417f3df25.0000000000000000:5 [read 983040~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:07.885 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.361528+0000", 2024-09-04T17:46:07.885 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.48338277099999999, 2024-09-04T17:46:07.886 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.028764359999999999, 2024-09-04T17:46:07.886 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:07.886 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:07.886 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:07.886 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:07.886 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:07.886 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:07.886 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4786 2024-09-04T17:46:07.886 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.886 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:07.886 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.886 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:07.886 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361528+0000", 2024-09-04T17:46:07.887 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.887 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.887 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.887 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:07.887 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361528+0000", 2024-09-04T17:46:07.887 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.887 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.887 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.887 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:07.887 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361529+0000", 2024-09-04T17:46:07.887 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9800000000000004e-07 2024-09-04T17:46:07.887 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.887 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.888 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:07.888 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361538+0000", 2024-09-04T17:46:07.888 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1789999999999997e-06 2024-09-04T17:46:07.888 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.888 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.888 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:07.888 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361547+0000", 2024-09-04T17:46:07.888 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9190000000000005e-06 2024-09-04T17:46:07.888 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.888 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.888 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:07.888 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361554+0000", 2024-09-04T17:46:07.889 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1269999999999997e-06 2024-09-04T17:46:07.889 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.889 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.889 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:07.889 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.390031+0000", 2024-09-04T17:46:07.889 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.028477308 2024-09-04T17:46:07.889 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.889 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.889 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:07.889 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.390131+0000", 2024-09-04T17:46:07.889 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.9776000000000002e-05 2024-09-04T17:46:07.890 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.890 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.890 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:07.890 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.390293+0000", 2024-09-04T17:46:07.890 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00016155299999999999 2024-09-04T17:46:07.890 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.890 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:07.890 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.890 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.890 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.890 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4787 3.c 3:370edc5a:::rbd_data.113417f3df25.0000000000000000:5 [read 1409024~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:07.890 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.361564+0000", 2024-09-04T17:46:07.891 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.48334695500000002, 2024-09-04T17:46:07.891 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.028991560999999999, 2024-09-04T17:46:07.891 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:07.891 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:07.891 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:07.891 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:07.891 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:07.891 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:07.891 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4787 2024-09-04T17:46:07.891 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.891 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:07.891 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.892 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:07.892 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361564+0000", 2024-09-04T17:46:07.892 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.892 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.892 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.892 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:07.892 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361564+0000", 2024-09-04T17:46:07.892 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.892 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.892 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.892 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:07.892 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361565+0000", 2024-09-04T17:46:07.893 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.3e-07 2024-09-04T17:46:07.893 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.893 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.893 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:07.893 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361568+0000", 2024-09-04T17:46:07.893 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.8839999999999998e-06 2024-09-04T17:46:07.893 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.893 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.893 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:07.893 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361578+0000", 2024-09-04T17:46:07.894 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0406e-05 2024-09-04T17:46:07.894 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.894 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.894 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:07.894 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361584+0000", 2024-09-04T17:46:07.894 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.3940000000000001e-06 2024-09-04T17:46:07.894 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.894 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.894 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:07.895 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.390343+0000", 2024-09-04T17:46:07.895 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.028758224999999998 2024-09-04T17:46:07.895 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.895 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.895 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:07.896 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.390422+0000", 2024-09-04T17:46:07.896 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.9636999999999997e-05 2024-09-04T17:46:07.896 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.896 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.896 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:07.896 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.390556+0000", 2024-09-04T17:46:07.896 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000133385 2024-09-04T17:46:07.897 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.897 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:07.897 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.897 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.897 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.897 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4788 3.c 3:370edc5a:::rbd_data.113417f3df25.0000000000000000:5 [read 1835008~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:07.897 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.361593+0000", 2024-09-04T17:46:07.898 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.48331761400000001, 2024-09-04T17:46:07.898 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.029263407000000002, 2024-09-04T17:46:07.898 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:07.898 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:07.898 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:07.898 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:07.898 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:07.898 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:07.899 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4788 2024-09-04T17:46:07.899 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.899 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:07.899 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.899 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:07.899 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361593+0000", 2024-09-04T17:46:07.899 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.900 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.900 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.900 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:07.900 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361593+0000", 2024-09-04T17:46:07.900 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.900 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.900 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.900 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:07.900 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361594+0000", 2024-09-04T17:46:07.900 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7299999999999996e-07 2024-09-04T17:46:07.901 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.901 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.901 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:07.901 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361597+0000", 2024-09-04T17:46:07.901 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.8700000000000001e-06 2024-09-04T17:46:07.901 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.901 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.901 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:07.901 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361605+0000", 2024-09-04T17:46:07.901 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.3259999999999996e-06 2024-09-04T17:46:07.901 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.901 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.901 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:07.901 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361612+0000", 2024-09-04T17:46:07.902 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.3810000000000004e-06 2024-09-04T17:46:07.902 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.902 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.902 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:07.902 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.390608+0000", 2024-09-04T17:46:07.902 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.028996688999999999 2024-09-04T17:46:07.902 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.902 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.902 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:07.902 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.390703+0000", 2024-09-04T17:46:07.902 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4394999999999993e-05 2024-09-04T17:46:07.902 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.902 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.903 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:07.903 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.390857+0000", 2024-09-04T17:46:07.903 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00015417299999999999 2024-09-04T17:46:07.903 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.903 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:07.903 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.903 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.903 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.903 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4789 3.c 3:370edc5a:::rbd_data.113417f3df25.0000000000000000:5 [read 2260992~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:07.903 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.361620+0000", 2024-09-04T17:46:07.903 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.48329049600000001, 2024-09-04T17:46:07.903 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.029448236999999999, 2024-09-04T17:46:07.903 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:07.903 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:07.904 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:07.904 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:07.904 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:07.904 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:07.904 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4789 2024-09-04T17:46:07.904 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.904 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:07.904 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.904 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:07.904 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361620+0000", 2024-09-04T17:46:07.904 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.904 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.904 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.905 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:07.905 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361620+0000", 2024-09-04T17:46:07.905 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.905 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.905 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.905 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:07.905 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361621+0000", 2024-09-04T17:46:07.905 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7000000000000005e-07 2024-09-04T17:46:07.905 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.905 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.905 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:07.905 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361624+0000", 2024-09-04T17:46:07.906 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.8789999999999999e-06 2024-09-04T17:46:07.906 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.906 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.906 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:07.906 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361632+0000", 2024-09-04T17:46:07.906 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.3680000000000002e-06 2024-09-04T17:46:07.906 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.906 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.906 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:07.906 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361639+0000", 2024-09-04T17:46:07.906 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.3450000000000002e-06 2024-09-04T17:46:07.906 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.907 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.907 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:07.907 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.390900+0000", 2024-09-04T17:46:07.907 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.02926161 2024-09-04T17:46:07.907 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.907 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.907 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:07.907 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.390968+0000", 2024-09-04T17:46:07.907 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7941999999999994e-05 2024-09-04T17:46:07.907 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.907 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.908 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:07.908 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.391069+0000", 2024-09-04T17:46:07.908 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000100523 2024-09-04T17:46:07.908 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.908 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:07.908 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.908 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.908 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.908 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4790 3.c 3:370edc5a:::rbd_data.113417f3df25.0000000000000000:5 [read 2686976~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:07.908 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.361648+0000", 2024-09-04T17:46:07.908 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.48326322999999999, 2024-09-04T17:46:07.908 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.029697429000000001, 2024-09-04T17:46:07.908 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:07.909 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:07.909 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:07.909 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:07.909 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:07.909 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:07.909 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4790 2024-09-04T17:46:07.909 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.909 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:07.909 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.909 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:07.909 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361648+0000", 2024-09-04T17:46:07.909 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.909 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.910 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.910 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:07.910 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361648+0000", 2024-09-04T17:46:07.910 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.910 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.910 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.910 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:07.910 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361648+0000", 2024-09-04T17:46:07.910 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.1799999999999995e-07 2024-09-04T17:46:07.910 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.910 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.910 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:07.910 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361651+0000", 2024-09-04T17:46:07.911 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.706e-06 2024-09-04T17:46:07.911 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.911 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.911 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:07.911 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361659+0000", 2024-09-04T17:46:07.911 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.0439999999999996e-06 2024-09-04T17:46:07.911 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.911 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.911 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:07.911 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361666+0000", 2024-09-04T17:46:07.911 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.5069999999999997e-06 2024-09-04T17:46:07.911 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.911 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.912 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:07.912 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.391109+0000", 2024-09-04T17:46:07.912 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.029443039000000001 2024-09-04T17:46:07.912 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.912 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.912 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:07.912 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.391197+0000", 2024-09-04T17:46:07.912 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7991999999999994e-05 2024-09-04T17:46:07.912 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.912 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.912 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:07.912 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.391345+0000", 2024-09-04T17:46:07.912 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014852299999999999 2024-09-04T17:46:07.913 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.913 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:07.913 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.913 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.913 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.913 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4791 3.c 3:370edc5a:::rbd_data.113417f3df25.0000000000000000:5 [read 3112960~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:07.913 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.361675+0000", 2024-09-04T17:46:07.913 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.48323644599999999, 2024-09-04T17:46:07.913 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.029981876000000001, 2024-09-04T17:46:07.913 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:07.913 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:07.913 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:07.913 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:07.914 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:07.914 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:07.914 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4791 2024-09-04T17:46:07.914 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.914 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:07.914 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.914 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:07.914 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361675+0000", 2024-09-04T17:46:07.914 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.914 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.914 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.914 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:07.914 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361675+0000", 2024-09-04T17:46:07.915 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.915 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.915 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.915 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:07.915 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361675+0000", 2024-09-04T17:46:07.915 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6700000000000003e-07 2024-09-04T17:46:07.915 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.915 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.915 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:07.915 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361678+0000", 2024-09-04T17:46:07.915 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.954e-06 2024-09-04T17:46:07.915 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.915 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.916 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:07.916 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361686+0000", 2024-09-04T17:46:07.916 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.259e-06 2024-09-04T17:46:07.916 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.916 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.916 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:07.916 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361693+0000", 2024-09-04T17:46:07.916 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.5400000000000001e-06 2024-09-04T17:46:07.916 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.916 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.916 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:07.917 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.391401+0000", 2024-09-04T17:46:07.917 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.029708427999999999 2024-09-04T17:46:07.917 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.917 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.917 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:07.917 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.391500+0000", 2024-09-04T17:46:07.917 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.8329e-05 2024-09-04T17:46:07.917 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.917 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.917 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:07.917 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.391656+0000", 2024-09-04T17:46:07.918 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000156799 2024-09-04T17:46:07.918 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.918 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:07.918 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.918 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.918 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.918 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4792 3.c 3:370edc5a:::rbd_data.113417f3df25.0000000000000000:5 [read 3538944~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:07.918 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.361701+0000", 2024-09-04T17:46:07.918 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.48320949499999999, 2024-09-04T17:46:07.919 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.030264729000000001, 2024-09-04T17:46:07.919 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:07.919 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:07.919 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:07.919 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:07.919 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:07.919 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:07.919 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4792 2024-09-04T17:46:07.919 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.919 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:07.919 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.919 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:07.919 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361701+0000", 2024-09-04T17:46:07.920 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.920 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.920 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.920 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:07.920 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361701+0000", 2024-09-04T17:46:07.920 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.920 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.920 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.920 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:07.920 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361702+0000", 2024-09-04T17:46:07.920 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1200000000000003e-07 2024-09-04T17:46:07.920 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.920 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.920 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:07.921 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361705+0000", 2024-09-04T17:46:07.921 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.7870000000000002e-06 2024-09-04T17:46:07.921 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.921 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.921 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:07.921 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361713+0000", 2024-09-04T17:46:07.921 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.3829999999999992e-06 2024-09-04T17:46:07.922 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.922 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.922 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:07.922 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361720+0000", 2024-09-04T17:46:07.922 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7909999999999999e-06 2024-09-04T17:46:07.922 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.922 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.922 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:07.922 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.391734+0000", 2024-09-04T17:46:07.922 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.030014320000000001 2024-09-04T17:46:07.923 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.923 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.923 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:07.923 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.391821+0000", 2024-09-04T17:46:07.923 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.6546999999999997e-05 2024-09-04T17:46:07.923 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.923 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.923 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:07.923 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.391966+0000", 2024-09-04T17:46:07.923 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000145389 2024-09-04T17:46:07.923 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.924 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:07.924 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.924 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.924 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.924 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4793 3.c 3:370edc5a:::rbd_data.113417f3df25.0000000000000000:5 [read 3964928~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:07.924 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.361728+0000", 2024-09-04T17:46:07.924 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.48318258200000003, 2024-09-04T17:46:07.924 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.030481419999999999, 2024-09-04T17:46:07.924 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:07.924 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:07.924 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:07.924 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:07.925 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:07.925 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:07.925 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4793 2024-09-04T17:46:07.925 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.925 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:07.925 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.925 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:07.925 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361728+0000", 2024-09-04T17:46:07.925 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.925 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.926 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.926 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:07.926 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361728+0000", 2024-09-04T17:46:07.926 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.926 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.926 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.926 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:07.926 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361729+0000", 2024-09-04T17:46:07.926 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.1699999999999997e-07 2024-09-04T17:46:07.926 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.926 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.926 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:07.926 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361732+0000", 2024-09-04T17:46:07.927 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4159999999999998e-06 2024-09-04T17:46:07.927 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.927 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.927 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:07.927 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361739+0000", 2024-09-04T17:46:07.927 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.6149999999999999e-06 2024-09-04T17:46:07.927 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.927 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.927 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:07.927 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361746+0000", 2024-09-04T17:46:07.927 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.3160000000000002e-06 2024-09-04T17:46:07.927 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.928 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.928 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:07.928 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.392002+0000", 2024-09-04T17:46:07.928 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.030256555000000001 2024-09-04T17:46:07.928 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.928 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.928 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:07.928 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.392076+0000", 2024-09-04T17:46:07.928 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3382999999999999e-05 2024-09-04T17:46:07.928 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.928 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.928 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:07.928 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.392210+0000", 2024-09-04T17:46:07.929 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000134318 2024-09-04T17:46:07.929 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.929 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:07.929 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.929 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.929 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.929 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4843 3.c 3:37f94306:::rbd_data.113417f3df25.0000000000000006:5 [read 98304~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:07.929 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.362055+0000", 2024-09-04T17:46:07.929 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.48285594199999998, 2024-09-04T17:46:07.929 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.030407970999999999, 2024-09-04T17:46:07.929 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:07.929 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:07.929 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:07.930 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:07.930 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:07.930 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:07.930 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4843 2024-09-04T17:46:07.930 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.930 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:07.930 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.930 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:07.930 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.362055+0000", 2024-09-04T17:46:07.930 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.930 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.930 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.930 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:07.930 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.362055+0000", 2024-09-04T17:46:07.931 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.931 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.931 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.931 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:07.931 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.362056+0000", 2024-09-04T17:46:07.931 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6400000000000002e-07 2024-09-04T17:46:07.931 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.931 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.931 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:07.931 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.362059+0000", 2024-09-04T17:46:07.932 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.8759999999999998e-06 2024-09-04T17:46:07.932 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.932 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.932 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:07.932 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.362067+0000", 2024-09-04T17:46:07.932 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.9969999999999998e-06 2024-09-04T17:46:07.932 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.932 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.932 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:07.932 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.362074+0000", 2024-09-04T17:46:07.933 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7120000000000003e-06 2024-09-04T17:46:07.933 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.933 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.933 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:07.933 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.392246+0000", 2024-09-04T17:46:07.933 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.030171678 2024-09-04T17:46:07.933 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.933 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.933 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:07.933 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.392334+0000", 2024-09-04T17:46:07.933 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7887999999999996e-05 2024-09-04T17:46:07.934 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.934 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.934 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:07.934 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.392463+0000", 2024-09-04T17:46:07.934 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00012925600000000001 2024-09-04T17:46:07.934 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.934 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:07.934 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.934 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.934 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.934 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4844 3.c 3:37f94306:::rbd_data.113417f3df25.0000000000000006:5 [read 524288~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:07.935 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.362083+0000", 2024-09-04T17:46:07.935 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.48282758199999998, 2024-09-04T17:46:07.935 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.030622511000000002, 2024-09-04T17:46:07.935 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:07.935 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:07.935 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:07.935 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:07.935 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:07.935 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:07.935 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4844 2024-09-04T17:46:07.935 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.935 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:07.936 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.936 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:07.936 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.362083+0000", 2024-09-04T17:46:07.936 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.936 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.936 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.936 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:07.936 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.362083+0000", 2024-09-04T17:46:07.936 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.936 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.937 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.937 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:07.937 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.362084+0000", 2024-09-04T17:46:07.937 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.8599999999999998e-07 2024-09-04T17:46:07.937 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.937 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.937 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:07.937 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.362087+0000", 2024-09-04T17:46:07.938 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.9040000000000001e-06 2024-09-04T17:46:07.938 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.938 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.938 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:07.938 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.362097+0000", 2024-09-04T17:46:07.938 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.6129999999999993e-06 2024-09-04T17:46:07.938 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.938 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.938 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:07.939 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.362103+0000", 2024-09-04T17:46:07.939 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6510000000000004e-06 2024-09-04T17:46:07.939 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.939 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.939 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:07.939 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.392496+0000", 2024-09-04T17:46:07.939 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.030393311999999999 2024-09-04T17:46:07.939 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.939 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.939 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:07.939 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.392581+0000", 2024-09-04T17:46:07.939 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.4668999999999999e-05 2024-09-04T17:46:07.940 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.940 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.940 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:07.940 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.392706+0000", 2024-09-04T17:46:07.940 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000124776 2024-09-04T17:46:07.940 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.940 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:07.940 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.940 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.940 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.940 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4845 3.c 3:37f94306:::rbd_data.113417f3df25.0000000000000006:5 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:07.941 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.362112+0000", 2024-09-04T17:46:07.941 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.482798907, 2024-09-04T17:46:07.941 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.030831418999999999, 2024-09-04T17:46:07.941 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:07.941 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:07.941 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:07.941 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:07.941 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:07.941 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:07.941 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4845 2024-09-04T17:46:07.941 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.941 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:07.942 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.942 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:07.942 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.362112+0000", 2024-09-04T17:46:07.942 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.942 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.942 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.942 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:07.942 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.362112+0000", 2024-09-04T17:46:07.942 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.942 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.942 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.943 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:07.943 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.362113+0000", 2024-09-04T17:46:07.943 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.5199999999999996e-07 2024-09-04T17:46:07.943 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.943 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.943 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:07.943 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.362116+0000", 2024-09-04T17:46:07.943 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.8700000000000001e-06 2024-09-04T17:46:07.943 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.944 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.944 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:07.944 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.362124+0000", 2024-09-04T17:46:07.944 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.4780000000000008e-06 2024-09-04T17:46:07.944 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.944 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.944 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:07.944 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.362131+0000", 2024-09-04T17:46:07.944 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.489e-06 2024-09-04T17:46:07.944 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.944 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.945 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:07.945 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.392736+0000", 2024-09-04T17:46:07.945 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.030605842000000001 2024-09-04T17:46:07.945 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.945 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.945 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:07.945 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.392818+0000", 2024-09-04T17:46:07.945 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.1877000000000002e-05 2024-09-04T17:46:07.945 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.945 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.945 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:07.946 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.392944+0000", 2024-09-04T17:46:07.946 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00012521099999999999 2024-09-04T17:46:07.946 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.946 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:07.946 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.946 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.946 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.946 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4846 3.c 3:37f94306:::rbd_data.113417f3df25.0000000000000006:5 [read 1376256~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:07.946 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.362139+0000", 2024-09-04T17:46:07.946 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.48277163099999998, 2024-09-04T17:46:07.946 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.031054723999999999, 2024-09-04T17:46:07.946 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:07.946 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:07.947 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:07.947 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:07.947 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:07.947 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:07.947 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4846 2024-09-04T17:46:07.947 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.947 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:07.947 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.947 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:07.947 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.362139+0000", 2024-09-04T17:46:07.947 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.948 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.948 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.948 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:07.948 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.362139+0000", 2024-09-04T17:46:07.948 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.948 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.948 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.948 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:07.948 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.362140+0000", 2024-09-04T17:46:07.948 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.9100000000000004e-07 2024-09-04T17:46:07.949 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.949 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.949 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:07.949 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.362143+0000", 2024-09-04T17:46:07.949 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.615e-06 2024-09-04T17:46:07.949 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.949 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.949 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:07.949 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.362151+0000", 2024-09-04T17:46:07.949 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.9480000000000008e-06 2024-09-04T17:46:07.949 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.949 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.950 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:07.950 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.362157+0000", 2024-09-04T17:46:07.950 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.1489999999999999e-06 2024-09-04T17:46:07.950 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.950 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.950 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:07.950 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.392975+0000", 2024-09-04T17:46:07.950 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.030818808999999999 2024-09-04T17:46:07.950 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.950 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.950 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:07.951 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.393059+0000", 2024-09-04T17:46:07.951 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.3657000000000002e-05 2024-09-04T17:46:07.951 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.951 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.951 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:07.951 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.393194+0000", 2024-09-04T17:46:07.951 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000134955 2024-09-04T17:46:07.951 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.951 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:07.951 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.951 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.951 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.952 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4847 3.c 3:37f94306:::rbd_data.113417f3df25.0000000000000006:5 [read 1802240~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:07.952 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.362168+0000", 2024-09-04T17:46:07.952 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.48274294099999998, 2024-09-04T17:46:07.952 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.031266395000000002, 2024-09-04T17:46:07.952 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:07.952 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:07.952 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:07.952 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:07.952 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:07.952 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:07.953 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4847 2024-09-04T17:46:07.953 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.953 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:07.953 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.953 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:07.953 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.362168+0000", 2024-09-04T17:46:07.953 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.953 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.953 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.953 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:07.953 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.362168+0000", 2024-09-04T17:46:07.953 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.953 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.954 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.954 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:07.954 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.362169+0000", 2024-09-04T17:46:07.954 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.2799999999999996e-07 2024-09-04T17:46:07.954 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.954 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.954 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:07.954 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.362171+0000", 2024-09-04T17:46:07.954 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.8049999999999998e-06 2024-09-04T17:46:07.954 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.954 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.954 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:07.954 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.362180+0000", 2024-09-04T17:46:07.955 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.2379999999999997e-06 2024-09-04T17:46:07.955 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.955 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.955 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:07.955 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.362186+0000", 2024-09-04T17:46:07.955 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6109999999999999e-06 2024-09-04T17:46:07.955 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.955 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.955 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:07.955 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.393227+0000", 2024-09-04T17:46:07.956 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.031040221999999999 2024-09-04T17:46:07.956 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.956 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.956 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:07.956 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.393312+0000", 2024-09-04T17:46:07.956 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.5173999999999995e-05 2024-09-04T17:46:07.956 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.956 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.956 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:07.956 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.393434+0000", 2024-09-04T17:46:07.956 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000122717 2024-09-04T17:46:07.956 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.957 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:07.957 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.957 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.957 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.957 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:5506 3.5 3:a2d24818:::rbd_data.113431586065.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:07.957 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.765716+0000", 2024-09-04T17:46:07.957 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.079195209000000003, 2024-09-04T17:46:07.957 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.034908609, 2024-09-04T17:46:07.957 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:07.957 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:07.957 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:07.957 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:07.958 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:07.958 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:07.958 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 5506 2024-09-04T17:46:07.958 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.958 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:07.958 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.958 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:07.958 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.765716+0000", 2024-09-04T17:46:07.958 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.958 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.958 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.958 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:07.958 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.765716+0000", 2024-09-04T17:46:07.959 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:07.959 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.959 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.959 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:07.959 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.765718+0000", 2024-09-04T17:46:07.959 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.3860000000000001e-06 2024-09-04T17:46:07.959 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.959 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.959 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:07.959 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.770060+0000", 2024-09-04T17:46:07.959 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0043418170000000004 2024-09-04T17:46:07.959 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.959 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.960 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:07.960 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.770084+0000", 2024-09-04T17:46:07.960 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.3748999999999999e-05 2024-09-04T17:46:07.960 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.960 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.960 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:07.960 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.770099+0000", 2024-09-04T17:46:07.960 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4993e-05 2024-09-04T17:46:07.960 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.960 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.961 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:07.961 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.770236+0000", 2024-09-04T17:46:07.961 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013729700000000001 2024-09-04T17:46:07.961 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.961 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.961 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:07.961 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.770491+0000", 2024-09-04T17:46:07.961 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000255467 2024-09-04T17:46:07.961 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.961 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.961 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:07.961 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.800375+0000", 2024-09-04T17:46:07.962 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.029883542999999999 2024-09-04T17:46:07.962 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.962 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.962 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:07.962 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.800520+0000", 2024-09-04T17:46:07.962 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014463400000000001 2024-09-04T17:46:07.962 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:07.962 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:07.962 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:07.962 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.800624+0000", 2024-09-04T17:46:07.962 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00010472299999999999 2024-09-04T17:46:07.962 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.963 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:07.963 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.963 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:07.963 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:07.963 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:07.964 INFO:teuthology.orchestra.run.smithi092.stdout:629 writesame 0xc9c4ea0 thru 0xc9d1fff data_size 0x50(0xd160 bytes) 2024-09-04T17:46:07.964 INFO:teuthology.orchestra.run.smithi092.stdout:630 write 0xa107343 thru 0xa10df18 (0x6bd6 bytes) 2024-09-04T17:46:07.965 INFO:teuthology.orchestra.run.smithi092.stdout:631 write 0x6f32d51 thru 0x6f3334f (0x5ff bytes) 2024-09-04T17:46:07.965 INFO:teuthology.orchestra.run.smithi092.stdout:632 writesame 0x4997886 thru 0x49a35ec data_size 0x1(0xbd67 bytes) 2024-09-04T17:46:07.965 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:07.962+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:07.965 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-04T17:46:07.982 INFO:teuthology.orchestra.run.smithi092.stdout:633 read 0x5ee141a thru 0x5ee6497 (0x507e bytes) 2024-09-04T17:46:07.983 INFO:teuthology.orchestra.run.smithi092.stdout:634 punch from 0x7b4b06a to 0x7b57ce7, (0xcc7d bytes) 2024-09-04T17:46:07.983 INFO:teuthology.orchestra.run.smithi092.stdout:635 read 0xc511472 thru 0xc518e80 (0x7a0f bytes) 2024-09-04T17:46:07.983 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:07.983 INFO:teuthology.orchestra.run.smithi092.stdout:640 write 0x22b1c4e thru 0x22b40f3 (0x24a6 bytes) 2024-09-04T17:46:07.986 INFO:teuthology.orchestra.run.smithi092.stdout:642 clone 5 order 19 su 262144 sc 2 2024-09-04T17:46:08.066 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:08.062+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:08.092 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:08.092 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:46:08.092 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:46:08.092 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:08.106 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-04T17:46:08.166 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:08.164+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:08.232 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:08.232 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:46:08.232 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:46:08.232 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:46:08.232 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:08.247 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-04T17:46:08.291 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:08.292+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:08.368 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:08.368+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:08.389 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone4 from 0xc9d2000 (overlap 0x256c735) to 0xef82e2 2024-09-04T17:46:08.435 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:08.436 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:46:08.436 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:46:08.436 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:46:08.436 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.436 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:1348 3.10 3:0afd7a80:::rbd_data.11346251f094.000000000000009b:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-04T17:46:08.436 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:02.609239+0000", 2024-09-04T17:46:08.436 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 5.827670254, 2024-09-04T17:46:08.436 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.026961731999999999, 2024-09-04T17:46:08.436 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:08.436 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:08.436 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:08.437 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:08.437 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:08.437 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:08.437 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 1348 2024-09-04T17:46:08.437 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.437 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:08.437 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.437 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:08.437 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.609239+0000", 2024-09-04T17:46:08.437 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:08.437 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.437 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.437 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:08.438 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.609239+0000", 2024-09-04T17:46:08.438 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:08.438 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.438 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.438 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:08.438 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.609242+0000", 2024-09-04T17:46:08.438 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.3269999999999999e-06 2024-09-04T17:46:08.438 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.438 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.438 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:08.438 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.609252+0000", 2024-09-04T17:46:08.438 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0689e-05 2024-09-04T17:46:08.438 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.439 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.439 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:08.439 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.609282+0000", 2024-09-04T17:46:08.439 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.932e-05 2024-09-04T17:46:08.439 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.439 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.439 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:08.439 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.609296+0000", 2024-09-04T17:46:08.439 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4197e-05 2024-09-04T17:46:08.439 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.439 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.439 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:08.440 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.609408+0000", 2024-09-04T17:46:08.440 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000111962 2024-09-04T17:46:08.440 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.440 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.440 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:08.440 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.609544+0000", 2024-09-04T17:46:08.440 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000136706 2024-09-04T17:46:08.440 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.440 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.440 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 0", 2024-09-04T17:46:08.440 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.613984+0000", 2024-09-04T17:46:08.440 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0044397890000000004 2024-09-04T17:46:08.440 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.441 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.441 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:08.441 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.635928+0000", 2024-09-04T17:46:08.441 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.021943547000000001 2024-09-04T17:46:08.441 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.441 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.441 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:08.441 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.636030+0000", 2024-09-04T17:46:08.441 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00010199700000000001 2024-09-04T17:46:08.441 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.441 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.441 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:08.441 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.636123+0000", 2024-09-04T17:46:08.442 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.3628999999999994e-05 2024-09-04T17:46:08.442 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.442 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.442 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:08.442 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.636201+0000", 2024-09-04T17:46:08.442 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.7569000000000002e-05 2024-09-04T17:46:08.442 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:08.442 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:08.442 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:08.442 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.442 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.442 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4019 3.1e 3:7817bfb4:::rbd_data.113417f3df25.0000000000000001:5 [read 7634944~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-04T17:46:08.442 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:06.909047+0000", 2024-09-04T17:46:08.442 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 1.5278622990000001, 2024-09-04T17:46:08.443 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.026506477000000001, 2024-09-04T17:46:08.443 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:08.443 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:08.443 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:08.443 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:08.443 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:08.443 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:08.443 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4019 2024-09-04T17:46:08.443 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.443 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:08.443 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.443 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:08.443 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909047+0000", 2024-09-04T17:46:08.443 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:08.444 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.444 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.444 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:08.444 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909047+0000", 2024-09-04T17:46:08.444 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:08.444 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.444 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.444 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:08.444 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909048+0000", 2024-09-04T17:46:08.444 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0800000000000005e-07 2024-09-04T17:46:08.444 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.444 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.444 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:08.444 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909050+0000", 2024-09-04T17:46:08.445 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.587e-06 2024-09-04T17:46:08.445 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.445 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.445 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:08.445 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909057+0000", 2024-09-04T17:46:08.445 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.9269999999999999e-06 2024-09-04T17:46:08.445 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.445 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.445 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:08.445 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909062+0000", 2024-09-04T17:46:08.445 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2730000000000002e-06 2024-09-04T17:46:08.445 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.446 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.446 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:08.446 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.935341+0000", 2024-09-04T17:46:08.446 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.026278526 2024-09-04T17:46:08.446 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.446 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.446 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:08.446 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.935423+0000", 2024-09-04T17:46:08.446 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.2158999999999997e-05 2024-09-04T17:46:08.446 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.446 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.446 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:08.446 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.935554+0000", 2024-09-04T17:46:08.446 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013049700000000001 2024-09-04T17:46:08.447 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:08.447 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:08.447 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:08.447 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.447 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.447 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4020 3.1e 3:7817bfb4:::rbd_data.113417f3df25.0000000000000001:5 [read 8060928~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-04T17:46:08.447 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:06.909070+0000", 2024-09-04T17:46:08.447 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 1.5278391140000001, 2024-09-04T17:46:08.447 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.026770842999999999, 2024-09-04T17:46:08.447 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:08.447 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:08.447 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:08.447 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:08.448 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:08.448 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:08.448 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4020 2024-09-04T17:46:08.448 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.448 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:08.448 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.448 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:08.448 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909070+0000", 2024-09-04T17:46:08.448 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:08.448 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.448 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.448 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:08.448 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909070+0000", 2024-09-04T17:46:08.449 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:08.449 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.449 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.449 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:08.449 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909071+0000", 2024-09-04T17:46:08.449 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9599999999999999e-07 2024-09-04T17:46:08.449 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.449 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.449 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:08.449 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909073+0000", 2024-09-04T17:46:08.449 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2010000000000002e-06 2024-09-04T17:46:08.449 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.449 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.449 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:08.450 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909080+0000", 2024-09-04T17:46:08.450 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.391e-06 2024-09-04T17:46:08.450 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.450 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.450 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:08.450 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909086+0000", 2024-09-04T17:46:08.450 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1499999999999998e-06 2024-09-04T17:46:08.450 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.450 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.450 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:08.450 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.935601+0000", 2024-09-04T17:46:08.450 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.026515318999999999 2024-09-04T17:46:08.450 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.451 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.451 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:08.451 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.935699+0000", 2024-09-04T17:46:08.451 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.7956999999999997e-05 2024-09-04T17:46:08.451 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.451 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.451 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:08.451 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.935841+0000", 2024-09-04T17:46:08.451 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014232900000000001 2024-09-04T17:46:08.451 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:08.451 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:08.451 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:08.451 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.452 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.452 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4076 3.1e 3:7817bfb4:::rbd_data.113417f3df25.0000000000000001:5 [read 4259840~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-04T17:46:08.452 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:06.909093+0000", 2024-09-04T17:46:08.452 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 1.5278166049999999, 2024-09-04T17:46:08.452 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.026930059999999999, 2024-09-04T17:46:08.452 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:08.452 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:08.452 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:08.452 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:08.452 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:08.452 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:08.452 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4076 2024-09-04T17:46:08.452 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.452 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:08.453 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.453 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:08.453 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909093+0000", 2024-09-04T17:46:08.453 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:08.453 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.453 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.453 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:08.453 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909093+0000", 2024-09-04T17:46:08.453 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:08.453 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.453 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.453 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:08.453 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909094+0000", 2024-09-04T17:46:08.454 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7999999999999995e-07 2024-09-04T17:46:08.454 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.454 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.454 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:08.454 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909096+0000", 2024-09-04T17:46:08.454 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1959999999999998e-06 2024-09-04T17:46:08.454 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.454 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.454 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:08.454 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909103+0000", 2024-09-04T17:46:08.454 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.6499999999999996e-06 2024-09-04T17:46:08.454 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.454 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.454 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:08.455 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909108+0000", 2024-09-04T17:46:08.455 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0860000000000001e-06 2024-09-04T17:46:08.455 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.455 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.455 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:08.455 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.935885+0000", 2024-09-04T17:46:08.455 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.026776687 2024-09-04T17:46:08.455 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.455 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.455 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:08.455 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.935946+0000", 2024-09-04T17:46:08.455 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.0473000000000002e-05 2024-09-04T17:46:08.455 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.455 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.456 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:08.456 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.936023+0000", 2024-09-04T17:46:08.456 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.7287999999999996e-05 2024-09-04T17:46:08.456 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:08.456 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:08.456 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:08.456 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.456 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.456 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4077 3.1e 3:7817bfb4:::rbd_data.113417f3df25.0000000000000001:5 [read 4685824~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-04T17:46:08.456 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:06.909117+0000", 2024-09-04T17:46:08.456 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 1.527792885, 2024-09-04T17:46:08.456 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.027219975, 2024-09-04T17:46:08.456 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:08.457 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:08.457 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:08.457 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:08.457 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:08.457 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:08.457 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4077 2024-09-04T17:46:08.457 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.457 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:08.457 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.457 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:08.457 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909117+0000", 2024-09-04T17:46:08.457 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:08.457 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.457 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.458 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:08.458 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909117+0000", 2024-09-04T17:46:08.458 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:08.458 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.458 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.458 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:08.458 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909117+0000", 2024-09-04T17:46:08.458 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.8299999999999997e-07 2024-09-04T17:46:08.458 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.458 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.458 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:08.458 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909119+0000", 2024-09-04T17:46:08.458 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2680000000000001e-06 2024-09-04T17:46:08.459 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.459 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.459 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:08.459 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909127+0000", 2024-09-04T17:46:08.459 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.9670000000000001e-06 2024-09-04T17:46:08.459 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.459 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.459 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:08.459 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909133+0000", 2024-09-04T17:46:08.459 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.215e-06 2024-09-04T17:46:08.459 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.459 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.459 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:08.459 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.936076+0000", 2024-09-04T17:46:08.460 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.026943292000000001 2024-09-04T17:46:08.460 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.460 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.460 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:08.460 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.936171+0000", 2024-09-04T17:46:08.460 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4933999999999996e-05 2024-09-04T17:46:08.460 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.460 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.460 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:08.460 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.936337+0000", 2024-09-04T17:46:08.460 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000165816 2024-09-04T17:46:08.460 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:08.460 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:08.461 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:08.461 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.461 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.461 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4078 3.1e 3:7817bfb4:::rbd_data.113417f3df25.0000000000000001:5 [read 5111808~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-04T17:46:08.461 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:06.909140+0000", 2024-09-04T17:46:08.461 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 1.5277692599999999, 2024-09-04T17:46:08.461 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.027371784, 2024-09-04T17:46:08.461 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:08.461 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:08.461 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:08.461 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:08.461 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:08.461 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:08.461 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4078 2024-09-04T17:46:08.462 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.462 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:08.462 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.462 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:08.462 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909140+0000", 2024-09-04T17:46:08.462 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:08.462 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.462 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.462 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:08.462 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909140+0000", 2024-09-04T17:46:08.462 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:08.462 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.462 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.463 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:08.463 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909141+0000", 2024-09-04T17:46:08.463 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6800000000000001e-07 2024-09-04T17:46:08.463 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.463 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.463 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:08.463 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909153+0000", 2024-09-04T17:46:08.463 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2542e-05 2024-09-04T17:46:08.463 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.463 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.463 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:08.463 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909162+0000", 2024-09-04T17:46:08.463 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.833e-06 2024-09-04T17:46:08.464 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.464 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.464 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:08.464 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909174+0000", 2024-09-04T17:46:08.464 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2429000000000001e-05 2024-09-04T17:46:08.464 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.464 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.464 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:08.464 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.936371+0000", 2024-09-04T17:46:08.464 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.027196708999999999 2024-09-04T17:46:08.465 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.465 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.465 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:08.465 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.936432+0000", 2024-09-04T17:46:08.465 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.0707e-05 2024-09-04T17:46:08.465 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.465 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.465 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:08.465 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.936512+0000", 2024-09-04T17:46:08.465 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.0096e-05 2024-09-04T17:46:08.465 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:08.466 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:08.466 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:08.466 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.466 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.466 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4079 3.1e 3:7817bfb4:::rbd_data.113417f3df25.0000000000000001:5 [read 5537792~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-04T17:46:08.466 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:06.909184+0000", 2024-09-04T17:46:08.466 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 1.5277259399999998, 2024-09-04T17:46:08.466 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.027638882, 2024-09-04T17:46:08.466 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:08.466 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:08.466 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:08.466 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:08.466 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:08.467 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:08.467 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4079 2024-09-04T17:46:08.467 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.467 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:08.467 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.467 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:08.467 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909184+0000", 2024-09-04T17:46:08.467 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:08.467 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.467 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.467 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:08.467 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909184+0000", 2024-09-04T17:46:08.467 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:08.468 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.468 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.468 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:08.468 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909184+0000", 2024-09-04T17:46:08.468 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2e-07 2024-09-04T17:46:08.468 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.468 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.468 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:08.468 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909187+0000", 2024-09-04T17:46:08.468 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.576e-06 2024-09-04T17:46:08.468 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.468 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.468 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:08.469 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909193+0000", 2024-09-04T17:46:08.469 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6449999999999999e-06 2024-09-04T17:46:08.469 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.469 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.469 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:08.469 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909199+0000", 2024-09-04T17:46:08.469 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2939999999999996e-06 2024-09-04T17:46:08.469 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.469 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.469 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:08.469 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.936548+0000", 2024-09-04T17:46:08.469 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.027349605999999999 2024-09-04T17:46:08.469 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.470 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.470 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:08.470 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.936640+0000", 2024-09-04T17:46:08.470 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.2194000000000005e-05 2024-09-04T17:46:08.470 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.470 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.470 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:08.470 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.936822+0000", 2024-09-04T17:46:08.470 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00018204699999999999 2024-09-04T17:46:08.470 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:08.470 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:08.470 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:08.470 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.471 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.471 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4080 3.1e 3:7817bfb4:::rbd_data.113417f3df25.0000000000000001:5 [read 5963776~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-04T17:46:08.471 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:06.909207+0000", 2024-09-04T17:46:08.471 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 1.5277029639999999, 2024-09-04T17:46:08.471 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.027874711, 2024-09-04T17:46:08.471 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:08.471 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:08.471 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:08.471 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:08.471 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:08.471 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:08.471 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4080 2024-09-04T17:46:08.471 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.472 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:08.472 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.472 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:08.472 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909207+0000", 2024-09-04T17:46:08.472 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:08.472 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.472 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.472 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:08.472 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909207+0000", 2024-09-04T17:46:08.472 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:08.472 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.472 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.472 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:08.473 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909207+0000", 2024-09-04T17:46:08.473 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9900000000000001e-07 2024-09-04T17:46:08.473 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.473 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.473 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:08.473 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909209+0000", 2024-09-04T17:46:08.473 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1639999999999999e-06 2024-09-04T17:46:08.473 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.473 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.473 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:08.473 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909217+0000", 2024-09-04T17:46:08.473 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4869999999999996e-06 2024-09-04T17:46:08.474 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.474 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.474 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:08.474 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909222+0000", 2024-09-04T17:46:08.474 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1780000000000002e-06 2024-09-04T17:46:08.474 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.474 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.474 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:08.474 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.936856+0000", 2024-09-04T17:46:08.474 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.027634394999999999 2024-09-04T17:46:08.474 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.474 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.474 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:08.475 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.936940+0000", 2024-09-04T17:46:08.475 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.4241999999999997e-05 2024-09-04T17:46:08.475 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.475 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.475 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:08.475 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.937081+0000", 2024-09-04T17:46:08.475 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000140746 2024-09-04T17:46:08.475 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:08.475 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:08.475 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:08.475 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.475 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.475 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4081 3.1e 3:7817bfb4:::rbd_data.113417f3df25.0000000000000001:5 [read 6389760~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-04T17:46:08.475 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:06.909230+0000", 2024-09-04T17:46:08.475 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 1.527679888, 2024-09-04T17:46:08.476 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.028121515, 2024-09-04T17:46:08.476 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:08.476 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:08.476 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:08.476 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:08.476 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:08.476 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:08.476 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4081 2024-09-04T17:46:08.476 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.476 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:08.476 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.476 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:08.476 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909230+0000", 2024-09-04T17:46:08.476 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:08.477 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.477 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.477 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:08.477 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909230+0000", 2024-09-04T17:46:08.477 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:08.477 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.477 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.477 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:08.477 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909230+0000", 2024-09-04T17:46:08.477 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4700000000000002e-07 2024-09-04T17:46:08.477 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.477 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.477 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:08.477 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909232+0000", 2024-09-04T17:46:08.478 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2110000000000001e-06 2024-09-04T17:46:08.478 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.478 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.478 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:08.478 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909240+0000", 2024-09-04T17:46:08.478 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.5460000000000003e-06 2024-09-04T17:46:08.478 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.478 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.478 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:08.478 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909245+0000", 2024-09-04T17:46:08.478 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1900000000000003e-06 2024-09-04T17:46:08.478 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.478 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.479 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:08.479 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.937119+0000", 2024-09-04T17:46:08.479 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.027874448999999999 2024-09-04T17:46:08.479 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.479 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.479 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:08.479 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.937209+0000", 2024-09-04T17:46:08.479 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9423000000000001e-05 2024-09-04T17:46:08.479 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.479 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.479 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:08.479 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.937351+0000", 2024-09-04T17:46:08.479 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000142249 2024-09-04T17:46:08.480 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:08.480 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:08.480 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:08.480 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.480 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.480 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4082 3.1e 3:7817bfb4:::rbd_data.113417f3df25.0000000000000001:5 [read 6815744~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-04T17:46:08.480 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:06.909253+0000", 2024-09-04T17:46:08.480 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 1.5276568740000001, 2024-09-04T17:46:08.480 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.028352584, 2024-09-04T17:46:08.480 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:08.480 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:08.480 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:08.481 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:08.481 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:08.481 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:08.481 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4082 2024-09-04T17:46:08.481 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.481 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:08.481 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.481 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:08.481 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909253+0000", 2024-09-04T17:46:08.481 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:08.481 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.481 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.481 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:08.482 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909253+0000", 2024-09-04T17:46:08.482 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:08.482 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.482 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.482 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:08.482 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909253+0000", 2024-09-04T17:46:08.482 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0999999999999999e-07 2024-09-04T17:46:08.482 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.482 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.482 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:08.482 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909255+0000", 2024-09-04T17:46:08.482 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1840000000000002e-06 2024-09-04T17:46:08.483 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.483 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.483 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:08.483 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909264+0000", 2024-09-04T17:46:08.483 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.422e-06 2024-09-04T17:46:08.483 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.483 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.483 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:08.483 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909277+0000", 2024-09-04T17:46:08.483 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3723000000000001e-05 2024-09-04T17:46:08.483 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.483 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.483 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:08.484 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.937384+0000", 2024-09-04T17:46:08.484 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.028106973 2024-09-04T17:46:08.484 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.484 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.484 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:08.484 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.937471+0000", 2024-09-04T17:46:08.484 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.6366999999999995e-05 2024-09-04T17:46:08.484 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.484 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.484 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:08.484 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.937605+0000", 2024-09-04T17:46:08.484 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013440500000000001 2024-09-04T17:46:08.485 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:08.485 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:08.485 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:08.485 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.485 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.485 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4083 3.1e 3:7817bfb4:::rbd_data.113417f3df25.0000000000000001:5 [read 7241728~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-04T17:46:08.485 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:06.909286+0000", 2024-09-04T17:46:08.485 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 1.5276234280000001, 2024-09-04T17:46:08.485 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.028595905000000001, 2024-09-04T17:46:08.485 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:08.485 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:08.485 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:08.486 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:08.486 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:08.486 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:08.486 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4083 2024-09-04T17:46:08.486 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.486 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:08.486 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.486 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:08.486 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909286+0000", 2024-09-04T17:46:08.486 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:08.486 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.486 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.486 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:08.486 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909286+0000", 2024-09-04T17:46:08.487 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:08.487 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.487 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.487 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:08.487 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909287+0000", 2024-09-04T17:46:08.487 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2399999999999998e-07 2024-09-04T17:46:08.487 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.487 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.487 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:08.487 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909295+0000", 2024-09-04T17:46:08.487 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.2339999999999994e-06 2024-09-04T17:46:08.487 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.487 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.488 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:08.488 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909301+0000", 2024-09-04T17:46:08.488 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.8340000000000004e-06 2024-09-04T17:46:08.488 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.488 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.488 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:08.488 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909306+0000", 2024-09-04T17:46:08.488 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.3159999999999996e-06 2024-09-04T17:46:08.488 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.488 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.488 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:08.488 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.937638+0000", 2024-09-04T17:46:08.489 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.028332267000000001 2024-09-04T17:46:08.489 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.489 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.489 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:08.489 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.937741+0000", 2024-09-04T17:46:08.489 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000102756 2024-09-04T17:46:08.489 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.489 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.489 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:08.489 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.937882+0000", 2024-09-04T17:46:08.489 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014097400000000001 2024-09-04T17:46:08.489 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:08.489 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:08.490 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:08.490 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.490 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.490 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4084 3.1e 3:7817bfb4:::rbd_data.113417f3df25.0000000000000001:5 [read 7667712~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-04T17:46:08.490 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:06.909314+0000", 2024-09-04T17:46:08.490 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 1.52759522, 2024-09-04T17:46:08.490 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.028838895, 2024-09-04T17:46:08.490 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:08.490 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:08.490 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:08.490 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:08.490 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:08.490 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:08.491 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4084 2024-09-04T17:46:08.491 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.491 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:08.491 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.491 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:08.491 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909314+0000", 2024-09-04T17:46:08.491 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:08.491 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.491 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.491 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:08.491 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909314+0000", 2024-09-04T17:46:08.491 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:08.491 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.491 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.492 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:08.492 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909315+0000", 2024-09-04T17:46:08.492 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7999999999999995e-07 2024-09-04T17:46:08.492 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.492 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.492 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:08.492 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909317+0000", 2024-09-04T17:46:08.492 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2529999999999999e-06 2024-09-04T17:46:08.492 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.492 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.492 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:08.492 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909325+0000", 2024-09-04T17:46:08.493 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.6279999999999996e-06 2024-09-04T17:46:08.493 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.493 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.493 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:08.493 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909330+0000", 2024-09-04T17:46:08.493 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.3040000000000004e-06 2024-09-04T17:46:08.493 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.493 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.493 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:08.493 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.937915+0000", 2024-09-04T17:46:08.493 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.028584766000000001 2024-09-04T17:46:08.493 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.493 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.494 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:08.494 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.938002+0000", 2024-09-04T17:46:08.494 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7590999999999995e-05 2024-09-04T17:46:08.494 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.494 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.494 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:08.494 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.938153+0000", 2024-09-04T17:46:08.494 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00015067300000000001 2024-09-04T17:46:08.494 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:08.494 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:08.494 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:08.494 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.494 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.494 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4085 3.1e 3:7817bfb4:::rbd_data.113417f3df25.0000000000000001:5 [read 8093696~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-04T17:46:08.495 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:06.909338+0000", 2024-09-04T17:46:08.495 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 1.527571386, 2024-09-04T17:46:08.495 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.029098151999999999, 2024-09-04T17:46:08.495 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:08.495 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:08.495 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:08.495 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:08.495 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:08.495 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:08.495 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4085 2024-09-04T17:46:08.495 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.495 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:08.495 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.495 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:08.496 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909338+0000", 2024-09-04T17:46:08.496 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:08.496 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.496 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.496 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:08.496 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909338+0000", 2024-09-04T17:46:08.496 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:08.496 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.496 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.496 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:08.496 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909339+0000", 2024-09-04T17:46:08.496 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6100000000000001e-07 2024-09-04T17:46:08.496 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.497 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.497 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:08.497 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909356+0000", 2024-09-04T17:46:08.497 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7147e-05 2024-09-04T17:46:08.497 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.497 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.497 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:08.497 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909362+0000", 2024-09-04T17:46:08.497 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.657e-06 2024-09-04T17:46:08.497 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.497 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.497 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:08.497 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909368+0000", 2024-09-04T17:46:08.498 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6459999999999998e-06 2024-09-04T17:46:08.498 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.498 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.498 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:08.498 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.938194+0000", 2024-09-04T17:46:08.498 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.028826424 2024-09-04T17:46:08.498 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.498 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.498 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:08.498 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.938296+0000", 2024-09-04T17:46:08.498 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000101087 2024-09-04T17:46:08.498 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.498 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.498 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:08.499 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.938436+0000", 2024-09-04T17:46:08.499 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014072999999999999 2024-09-04T17:46:08.499 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:08.499 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:08.499 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:08.499 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.499 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.499 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4797 3.1e 3:7817bfb4:::rbd_data.113417f3df25.0000000000000001:5 [read 1474560~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:08.499 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.350022+0000", 2024-09-04T17:46:08.499 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 1.0868878660000001, 2024-09-04T17:46:08.499 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.026399108000000001, 2024-09-04T17:46:08.499 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:08.499 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:08.499 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:08.500 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:08.500 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:08.500 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:08.500 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4797 2024-09-04T17:46:08.500 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.500 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:08.500 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.500 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:08.500 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350022+0000", 2024-09-04T17:46:08.500 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:08.500 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.500 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.500 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:08.501 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350022+0000", 2024-09-04T17:46:08.501 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:08.501 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.501 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.501 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:08.501 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350023+0000", 2024-09-04T17:46:08.501 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0160000000000001e-06 2024-09-04T17:46:08.501 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.501 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.501 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:08.501 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350038+0000", 2024-09-04T17:46:08.501 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5671999999999998e-05 2024-09-04T17:46:08.501 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.502 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.502 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:08.502 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350049+0000", 2024-09-04T17:46:08.502 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0266e-05 2024-09-04T17:46:08.502 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.502 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.502 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:08.502 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350056+0000", 2024-09-04T17:46:08.502 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4900000000000003e-06 2024-09-04T17:46:08.502 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.502 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.502 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:08.502 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.374015+0000", 2024-09-04T17:46:08.502 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.023959081 2024-09-04T17:46:08.503 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.503 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.503 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:08.503 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.374112+0000", 2024-09-04T17:46:08.503 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.7322000000000001e-05 2024-09-04T17:46:08.503 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.503 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.503 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:08.503 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.376421+0000", 2024-09-04T17:46:08.503 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0023082609999999998 2024-09-04T17:46:08.503 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:08.503 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:08.503 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:08.504 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.504 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.504 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4798 3.1e 3:7817bfb4:::rbd_data.113417f3df25.0000000000000001:5 [read 1900544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:08.504 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.350077+0000", 2024-09-04T17:46:08.504 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 1.0868321569999999, 2024-09-04T17:46:08.504 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.026604033999999999, 2024-09-04T17:46:08.504 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:08.504 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:08.504 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:08.504 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:08.504 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:08.504 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:08.504 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4798 2024-09-04T17:46:08.504 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.505 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:08.505 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.505 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:08.505 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350077+0000", 2024-09-04T17:46:08.505 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:08.505 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.505 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.505 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:08.505 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350077+0000", 2024-09-04T17:46:08.505 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:08.505 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.505 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.505 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:08.506 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350078+0000", 2024-09-04T17:46:08.506 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.4300000000000003e-07 2024-09-04T17:46:08.506 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.506 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.506 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:08.506 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350082+0000", 2024-09-04T17:46:08.506 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.9829999999999998e-06 2024-09-04T17:46:08.506 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.506 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.506 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:08.506 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350091+0000", 2024-09-04T17:46:08.506 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.3500000000000003e-06 2024-09-04T17:46:08.506 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.506 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.506 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:08.507 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350099+0000", 2024-09-04T17:46:08.507 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.7500000000000003e-06 2024-09-04T17:46:08.507 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.507 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.507 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:08.507 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.376457+0000", 2024-09-04T17:46:08.507 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.026358320000000001 2024-09-04T17:46:08.507 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.507 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.507 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:08.507 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.376552+0000", 2024-09-04T17:46:08.507 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5074000000000005e-05 2024-09-04T17:46:08.507 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.507 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.507 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:08.508 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.376681+0000", 2024-09-04T17:46:08.508 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00012891399999999999 2024-09-04T17:46:08.508 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:08.508 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:08.508 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:08.508 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.508 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.508 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4799 3.1e 3:7817bfb4:::rbd_data.113417f3df25.0000000000000001:5 [read 2326528~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:08.508 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.350111+0000", 2024-09-04T17:46:08.508 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 1.0867987130000001, 2024-09-04T17:46:08.508 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.029614627000000001, 2024-09-04T17:46:08.508 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:08.508 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:08.508 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:08.509 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:08.509 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:08.509 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:08.509 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4799 2024-09-04T17:46:08.509 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.509 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:08.509 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.509 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:08.509 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350111+0000", 2024-09-04T17:46:08.509 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:08.509 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.510 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.510 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:08.510 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350111+0000", 2024-09-04T17:46:08.510 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:08.510 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.510 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.510 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:08.510 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350111+0000", 2024-09-04T17:46:08.510 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.9599999999999999e-07 2024-09-04T17:46:08.510 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.510 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.510 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:08.511 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350115+0000", 2024-09-04T17:46:08.511 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.506e-06 2024-09-04T17:46:08.511 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.511 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.511 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:08.511 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350126+0000", 2024-09-04T17:46:08.511 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0658e-05 2024-09-04T17:46:08.511 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.511 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.511 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:08.511 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350133+0000", 2024-09-04T17:46:08.511 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2130000000000002e-06 2024-09-04T17:46:08.511 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.511 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.512 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:08.512 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.376712+0000", 2024-09-04T17:46:08.512 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.026579479 2024-09-04T17:46:08.512 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.512 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.512 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:08.512 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.379533+0000", 2024-09-04T17:46:08.512 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0028206780000000001 2024-09-04T17:46:08.512 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.512 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.512 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:08.512 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.379725+0000", 2024-09-04T17:46:08.512 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019239699999999999 2024-09-04T17:46:08.512 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:08.513 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:08.513 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:08.513 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.513 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.513 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4800 3.1e 3:7817bfb4:::rbd_data.113417f3df25.0000000000000001:5 [read 2752512~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:08.513 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.350151+0000", 2024-09-04T17:46:08.513 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 1.0867580489999999, 2024-09-04T17:46:08.513 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.029858974999999999, 2024-09-04T17:46:08.513 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:08.513 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:08.513 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:08.513 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:08.513 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:08.513 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:08.514 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4800 2024-09-04T17:46:08.514 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.514 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:08.514 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.514 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:08.514 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350151+0000", 2024-09-04T17:46:08.514 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:08.514 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.514 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.514 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:08.514 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350151+0000", 2024-09-04T17:46:08.514 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:08.514 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.514 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.515 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:08.515 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350152+0000", 2024-09-04T17:46:08.515 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.4899999999999995e-07 2024-09-04T17:46:08.515 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.515 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.515 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:08.515 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350156+0000", 2024-09-04T17:46:08.515 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.6160000000000002e-06 2024-09-04T17:46:08.515 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.515 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.515 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:08.515 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350166+0000", 2024-09-04T17:46:08.515 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0168e-05 2024-09-04T17:46:08.516 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.516 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.516 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:08.516 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350173+0000", 2024-09-04T17:46:08.516 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6830000000000002e-06 2024-09-04T17:46:08.516 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.516 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.516 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:08.516 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.379767+0000", 2024-09-04T17:46:08.516 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.029594915999999999 2024-09-04T17:46:08.516 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.516 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.516 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:08.517 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.379864+0000", 2024-09-04T17:46:08.517 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.6164999999999998e-05 2024-09-04T17:46:08.517 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.517 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.517 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:08.517 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.380010+0000", 2024-09-04T17:46:08.517 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000146778 2024-09-04T17:46:08.517 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:08.517 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:08.517 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:08.517 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.517 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.517 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4801 3.1e 3:7817bfb4:::rbd_data.113417f3df25.0000000000000001:5 [read 3178496~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:08.518 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.350183+0000", 2024-09-04T17:46:08.518 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 1.0867262, 2024-09-04T17:46:08.518 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.030089563999999999, 2024-09-04T17:46:08.518 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:08.518 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:08.518 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:08.518 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:08.518 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:08.518 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:08.518 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4801 2024-09-04T17:46:08.518 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.518 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:08.518 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.518 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:08.519 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350183+0000", 2024-09-04T17:46:08.519 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:08.519 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.519 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.519 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:08.519 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350183+0000", 2024-09-04T17:46:08.519 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:08.519 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.519 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.519 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:08.519 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350184+0000", 2024-09-04T17:46:08.519 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3399999999999998e-07 2024-09-04T17:46:08.519 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.520 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.520 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:08.520 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350187+0000", 2024-09-04T17:46:08.520 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4470000000000002e-06 2024-09-04T17:46:08.520 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.520 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.520 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:08.520 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350196+0000", 2024-09-04T17:46:08.520 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9530000000000005e-06 2024-09-04T17:46:08.520 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.520 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.520 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:08.520 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350203+0000", 2024-09-04T17:46:08.521 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.4160000000000001e-06 2024-09-04T17:46:08.521 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.521 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.521 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:08.521 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.380042+0000", 2024-09-04T17:46:08.521 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.029839099000000001 2024-09-04T17:46:08.521 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.521 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.521 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:08.521 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.380131+0000", 2024-09-04T17:46:08.521 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8857999999999995e-05 2024-09-04T17:46:08.521 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.522 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.522 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:08.522 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.380273+0000", 2024-09-04T17:46:08.522 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014205700000000001 2024-09-04T17:46:08.522 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:08.522 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:08.522 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:08.522 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.522 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.522 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4802 3.1e 3:7817bfb4:::rbd_data.113417f3df25.0000000000000001:5 [read 3604480~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:08.522 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.350221+0000", 2024-09-04T17:46:08.522 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 1.0866880189999999, 2024-09-04T17:46:08.522 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.030306293000000002, 2024-09-04T17:46:08.522 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:08.523 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:08.523 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:08.523 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:08.523 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:08.523 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:08.523 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4802 2024-09-04T17:46:08.523 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.523 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:08.523 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.523 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:08.523 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350221+0000", 2024-09-04T17:46:08.523 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:08.523 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.523 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.524 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:08.524 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350221+0000", 2024-09-04T17:46:08.524 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:08.524 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.524 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.524 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:08.524 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350223+0000", 2024-09-04T17:46:08.524 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1009999999999999e-06 2024-09-04T17:46:08.524 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.524 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.524 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:08.524 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350226+0000", 2024-09-04T17:46:08.525 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7000000000000002e-06 2024-09-04T17:46:08.525 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.525 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.525 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:08.525 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350236+0000", 2024-09-04T17:46:08.525 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5899999999999997e-06 2024-09-04T17:46:08.525 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.525 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.525 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:08.525 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350243+0000", 2024-09-04T17:46:08.525 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.5299999999999999e-06 2024-09-04T17:46:08.525 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.525 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.526 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:08.526 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.380303+0000", 2024-09-04T17:46:08.526 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.030060104000000001 2024-09-04T17:46:08.526 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.526 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.526 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:08.526 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.380390+0000", 2024-09-04T17:46:08.526 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.6420000000000003e-05 2024-09-04T17:46:08.526 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.526 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.526 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:08.526 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.380528+0000", 2024-09-04T17:46:08.526 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013784800000000001 2024-09-04T17:46:08.527 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:08.527 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:08.527 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:08.527 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.527 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.527 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4803 3.1e 3:7817bfb4:::rbd_data.113417f3df25.0000000000000001:5 [read 4030464~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:08.527 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.350254+0000", 2024-09-04T17:46:08.527 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 1.086655167, 2024-09-04T17:46:08.527 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.030523386999999999, 2024-09-04T17:46:08.527 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:08.527 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:08.527 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:08.527 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:08.527 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:08.528 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:08.528 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4803 2024-09-04T17:46:08.528 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.528 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:08.528 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.528 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:08.528 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350254+0000", 2024-09-04T17:46:08.528 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:08.528 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.528 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.528 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:08.528 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350254+0000", 2024-09-04T17:46:08.528 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:08.529 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.529 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.529 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:08.529 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350255+0000", 2024-09-04T17:46:08.529 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.2799999999999996e-07 2024-09-04T17:46:08.529 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.529 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.529 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:08.529 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350258+0000", 2024-09-04T17:46:08.529 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.1300000000000001e-06 2024-09-04T17:46:08.529 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.529 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.529 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:08.530 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350278+0000", 2024-09-04T17:46:08.530 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9584000000000001e-05 2024-09-04T17:46:08.530 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.530 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.530 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:08.530 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350285+0000", 2024-09-04T17:46:08.530 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3270000000000003e-06 2024-09-04T17:46:08.530 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.530 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.530 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:08.530 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.380558+0000", 2024-09-04T17:46:08.530 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.030273220999999999 2024-09-04T17:46:08.530 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.531 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.531 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:08.531 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.380640+0000", 2024-09-04T17:46:08.531 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.1772000000000002e-05 2024-09-04T17:46:08.531 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.531 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.531 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:08.531 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.380778+0000", 2024-09-04T17:46:08.531 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000137725 2024-09-04T17:46:08.531 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:08.531 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:08.531 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:08.531 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:08.531 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:08.532 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:08.532 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-04T17:46:08.534 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:08.530+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:08.534 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-04T17:46:08.634 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:08.634+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:08.634 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #3, image image_client.0-clone3 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent4 2024-09-04T17:46:08.637 INFO:teuthology.orchestra.run.smithi092.stdout:643 write 0xcf8480 thru 0xcf91e6 (0xd67 bytes) 2024-09-04T17:46:08.650 INFO:teuthology.orchestra.run.smithi092.stdout:644 read 0x53a62d0 thru 0x53a844a (0x217b bytes) 2024-09-04T17:46:08.662 INFO:teuthology.orchestra.run.smithi092.stdout:646 trunc from 0xc9d2000 to 0x7a34d54 2024-09-04T17:46:08.691 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:46:08.692 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:46:08.692 INFO:teuthology.orchestra.run.smithi092.stdout:648 trunc from 0x7a34d54 to 0xcb5f0f2 2024-09-04T17:46:08.702 INFO:teuthology.orchestra.run.smithi092.stdout:649 write 0x49b3f52 thru 0x49bdbde (0x9c8d bytes) 2024-09-04T17:46:08.709 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:46:08.709 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:46:08.711 INFO:teuthology.orchestra.run.smithi092.stdout:650 writesame 0xc4bb14f thru 0xc4cadee data_size 0x5e(0xfca0 bytes) 2024-09-04T17:46:08.718 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:08.718 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:46:08.718 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.718 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:6447 3.11 3:8b078d7b:::rbd_object_map.11343d768295:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:08.718 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:08.714810+0000", 2024-09-04T17:46:08.718 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.0060519329999999998, 2024-09-04T17:46:08.718 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0060968319999999999, 2024-09-04T17:46:08.718 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:08.718 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:08.718 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "waiting for sub ops", 2024-09-04T17:46:08.718 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:08.718 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:08.718 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:08.719 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 6447 2024-09-04T17:46:08.719 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.719 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:08.719 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.719 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:08.719 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.714810+0000", 2024-09-04T17:46:08.719 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:08.719 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.719 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.719 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:08.719 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.714810+0000", 2024-09-04T17:46:08.719 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:08.719 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.719 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.719 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:08.720 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.714811+0000", 2024-09-04T17:46:08.720 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.37e-07 2024-09-04T17:46:08.720 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.720 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.720 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:08.720 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.714818+0000", 2024-09-04T17:46:08.720 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1029999999999996e-06 2024-09-04T17:46:08.720 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.720 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.720 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:08.720 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.714825+0000", 2024-09-04T17:46:08.720 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6250000000000001e-06 2024-09-04T17:46:08.720 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.720 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.721 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:08.721 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.714829+0000", 2024-09-04T17:46:08.721 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.442e-06 2024-09-04T17:46:08.721 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.721 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.721 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:08.721 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.716652+0000", 2024-09-04T17:46:08.721 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.001823039 2024-09-04T17:46:08.721 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.721 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.721 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:08.722 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.716905+0000", 2024-09-04T17:46:08.722 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000252654 2024-09-04T17:46:08.722 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.722 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.722 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 4", 2024-09-04T17:46:08.722 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.717846+0000", 2024-09-04T17:46:08.722 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00094094299999999997 2024-09-04T17:46:08.722 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:08.722 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:08.722 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:08.722 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.720235+0000", 2024-09-04T17:46:08.722 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0023890249999999999 2024-09-04T17:46:08.722 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:08.722 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:08.722 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:08.723 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:08.723 INFO:teuthology.orchestra.run.smithi190.stdout: ], 2024-09-04T17:46:08.723 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 1 2024-09-04T17:46:08.723 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:08.723 INFO:teuthology.orchestra.run.smithi092.stdout:651 write 0x3816bb6 thru 0x3820d89 (0xa1d4 bytes) 2024-09-04T17:46:08.726 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-04T17:46:08.727 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-04T17:46:08.727 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-04T17:46:08.730 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:08.731 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:08.731 INFO:teuthology.orchestra.run.smithi092.stdout:655 read 0x9328fe1 thru 0x9332eef (0x9f0f bytes) 2024-09-04T17:46:08.731 INFO:teuthology.orchestra.run.smithi092.stdout:656 read 0x793dc13 thru 0x7945852 (0x7c40 bytes) 2024-09-04T17:46:08.735 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-04T17:46:08.736 INFO:teuthology.orchestra.run.smithi092.stdout:658 trunc from 0xcb5f0f2 to 0xb646352 2024-09-04T17:46:08.737 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:46:08.736+0000 7f505d642640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26446) UID: 0 2024-09-04T17:46:08.750 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:46:08.750 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:46:08.761 INFO:teuthology.orchestra.run.smithi092.stdout:659 trunc from 0xb646352 to 0x81e9bd4 2024-09-04T17:46:08.768 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:46:08.769 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:46:08.771 INFO:teuthology.orchestra.run.smithi092.stdout:661 write 0x2fea80c thru 0x2ff8d64 (0xe559 bytes) 2024-09-04T17:46:08.788 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-04T17:46:08.788 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-04T17:46:08.793 INFO:teuthology.orchestra.run.smithi092.stdout:664 trunc from 0x81e9bd4 to 0x2402c31 2024-09-04T17:46:08.807 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:46:08.807 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:46:08.811 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-04T17:46:08.811 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-04T17:46:08.842 INFO:teuthology.orchestra.run.smithi092.stdout:665 writesame 0x9ea4f2e thru 0x9eb195d data_size 0x28(0xca30 bytes) 2024-09-04T17:46:08.843 INFO:teuthology.orchestra.run.smithi092.stdout:666 read 0x118963f thru 0x1196df3 (0xd7b5 bytes) 2024-09-04T17:46:08.843 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:08.842+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:08.844 INFO:teuthology.orchestra.run.smithi092.stdout:667 punch from 0x5915a37 to 0x5924eaa, (0xf473 bytes) 2024-09-04T17:46:08.844 INFO:teuthology.orchestra.run.smithi092.stdout:668 write 0x11d1a03 thru 0x11d20e7 (0x6e5 bytes) 2024-09-04T17:46:08.866 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:08.866 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:08.870 INFO:teuthology.orchestra.run.smithi092.stdout:670 trunc from 0x9eb195e to 0x17983b 2024-09-04T17:46:08.882 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-04T17:46:08.913 INFO:teuthology.orchestra.run.smithi092.stdout:671 writesame 0x458c1dd thru 0x45928c7 data_size 0x1(0x66eb bytes) 2024-09-04T17:46:08.913 INFO:teuthology.orchestra.run.smithi092.stdout:674 write 0x1ffae4b thru 0x2000e4a (0x6000 bytes) 2024-09-04T17:46:08.938 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:08.934+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:08.951 INFO:teuthology.orchestra.run.smithi092.stdout:675 write 0xe3c2130 thru 0xe3c8757 (0x6628 bytes) 2024-09-04T17:46:08.957 INFO:teuthology.orchestra.run.smithi092.stdout:677 write 0xbef696a thru 0xbf0590c (0xefa3 bytes) 2024-09-04T17:46:08.964 INFO:teuthology.orchestra.run.smithi092.stdout:679 read 0xca3a0ed thru 0xca41b1e (0x7a32 bytes) 2024-09-04T17:46:08.971 INFO:teuthology.orchestra.run.smithi092.stdout:681 trunc from 0xe3c8758 to 0xbd1448d 2024-09-04T17:46:08.993 INFO:teuthology.orchestra.run.smithi092.stdout:682 read 0x7d020c8 thru 0x7d0c836 (0xa76f bytes) 2024-09-04T17:46:08.994 INFO:teuthology.orchestra.run.smithi092.stdout:685 punch from 0x11731e2 to 0x11820b0, (0xeece bytes) 2024-09-04T17:46:08.994 INFO:teuthology.orchestra.run.smithi092.stdout:686 punch from 0xb5a3fff to 0xb5aee92, (0xae93 bytes) 2024-09-04T17:46:08.994 INFO:teuthology.orchestra.run.smithi092.stdout:688 read 0xaecbc21 thru 0xaed9bc1 (0xdfa1 bytes) 2024-09-04T17:46:08.994 INFO:teuthology.orchestra.run.smithi092.stdout:689 write 0x6363d70 thru 0x6366581 (0x2812 bytes) 2024-09-04T17:46:09.013 INFO:teuthology.orchestra.run.smithi092.stdout:690 write 0xc04e76b thru 0xc05e6e4 (0xff7a bytes) 2024-09-04T17:46:09.017 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:09.018 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:09.020 INFO:teuthology.orchestra.run.smithi092.stdout:693 writesame 0x1816c1a thru 0x18242ed data_size 0x6(0xd6d4 bytes) 2024-09-04T17:46:09.033 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-04T17:46:09.040 INFO:teuthology.orchestra.run.smithi092.stdout:694 read 0x3ebb2a2 thru 0x3eca425 (0xf184 bytes) 2024-09-04T17:46:09.041 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:09.036+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:09.052 INFO:teuthology.orchestra.run.smithi092.stdout:696 write 0xe33f296 thru 0xe343b56 (0x48c1 bytes) 2024-09-04T17:46:09.072 INFO:teuthology.orchestra.run.smithi092.stdout:697 trunc from 0xe343b57 to 0x94b50a0 2024-09-04T17:46:09.091 INFO:teuthology.orchestra.run.smithi092.stdout:698 writesame 0x8486586 thru 0x849296e data_size 0x1(0xc3e9 bytes) 2024-09-04T17:46:09.124 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:09.124 INFO:teuthology.orchestra.run.smithi092.stdout:700 read 0x5130a0b thru 0x5137cf1 (0x72e7 bytes) 2024-09-04T17:46:09.138 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:09.136+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:09.140 INFO:teuthology.orchestra.run.smithi092.stdout:701 write 0xa9b0021 thru 0xa9b15e0 (0x15c0 bytes) 2024-09-04T17:46:09.147 INFO:teuthology.orchestra.run.smithi092.stdout:702 read 0x5b49a64 thru 0x5b4c084 (0x2621 bytes) 2024-09-04T17:46:09.147 INFO:teuthology.orchestra.run.smithi092.stdout:703 punch from 0x657dbb7 to 0x658a947, (0xcd90 bytes) 2024-09-04T17:46:09.147 INFO:teuthology.orchestra.run.smithi092.stdout:704 clone 6 order 20 su 16384 sc 6 2024-09-04T17:46:09.164 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:09.164 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:09.178 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-04T17:46:09.262 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-04T17:46:09.266 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:09.266+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:09.340 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:09.340+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:09.353 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:09.353 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:09.369 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-04T17:46:09.390 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone5 from 0xa9b15e1 (overlap 0x17983b) to 0x108429 2024-09-04T17:46:09.440 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:09.440+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:09.496 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:09.496 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:09.511 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-04T17:46:09.554 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:09.554+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:09.572 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #4, image image_client.0-clone4 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent5 2024-09-04T17:46:09.623 INFO:teuthology.orchestra.run.smithi092.stdout:706 read 0x9cb4497 thru 0x9cbc55b (0x80c5 bytes) 2024-09-04T17:46:09.624 INFO:teuthology.orchestra.run.smithi092.stdout:707 write 0xa542c16 thru 0xa54b566 (0x8951 bytes) 2024-09-04T17:46:09.641 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:09.638+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:09.654 INFO:teuthology.orchestra.run.smithi092.stdout:708 write 0xb14f916 thru 0xb15b4af (0xbb9a bytes) 2024-09-04T17:46:09.665 INFO:teuthology.orchestra.run.smithi092.stdout:709 punch from 0x23047bc to 0x2306d77, (0x25bb bytes) 2024-09-04T17:46:09.665 INFO:teuthology.orchestra.run.smithi092.stdout:710 writesame 0xeb4ec6 thru 0xebc0c2 data_size 0x3(0x71fd bytes) 2024-09-04T17:46:09.680 INFO:teuthology.orchestra.run.smithi092.stdout:711 trunc from 0xb15b4b0 to 0xb724efb 2024-09-04T17:46:09.681 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:09.681 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:09.695 INFO:teuthology.orchestra.run.smithi092.stdout:713 read 0x7afd512 thru 0x7b0abcd (0xd6bc bytes) 2024-09-04T17:46:09.695 INFO:teuthology.orchestra.run.smithi092.stdout:714 writesame 0x7602371 thru 0x76111db data_size 0x41(0xee6b bytes) 2024-09-04T17:46:09.698 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-04T17:46:09.707 INFO:teuthology.orchestra.run.smithi092.stdout:716 writesame 0x4924360 thru 0x4927581 data_size 0x45(0x3222 bytes) 2024-09-04T17:46:09.731 INFO:teuthology.orchestra.run.smithi092.stdout:717 write 0xe9ccf10 thru 0xe9d95e3 (0xc6d4 bytes) 2024-09-04T17:46:09.742 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:46:09.740+0000 7f505d642640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26446) UID: 0 2024-09-04T17:46:09.743 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:09.765 INFO:teuthology.orchestra.run.smithi092.stdout:719 trunc from 0xe9d95e4 to 0x4830373 2024-09-04T17:46:09.805 INFO:teuthology.orchestra.run.smithi092.stdout:720 writesame 0x1b1e994 thru 0x1b2325f data_size 0x6(0x48cc bytes) 2024-09-04T17:46:09.818 INFO:teuthology.orchestra.run.smithi092.stdout:721 read 0x2e03399 thru 0x2e0d920 (0xa588 bytes) 2024-09-04T17:46:09.819 INFO:teuthology.orchestra.run.smithi092.stdout:722 read 0x3a5fea3 thru 0x3a6800b (0x8169 bytes) 2024-09-04T17:46:09.819 INFO:teuthology.orchestra.run.smithi092.stdout:723 read 0x1b2e3ac thru 0x1b3cf18 (0xeb6d bytes) 2024-09-04T17:46:09.835 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:09.835 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:09.838 INFO:teuthology.orchestra.run.smithi092.stdout:725 writesame 0x806c63c thru 0x8076a6b data_size 0x8e(0xa430 bytes) 2024-09-04T17:46:09.838 INFO:teuthology.orchestra.run.smithi092.stdout:727 write 0x1c2e2aa thru 0x1c3c32e (0xe085 bytes) 2024-09-04T17:46:09.843 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:46:09.844+0000 7f505d642640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26446) UID: 0 2024-09-04T17:46:09.851 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-04T17:46:09.852 INFO:teuthology.orchestra.run.smithi092.stdout:729 read 0x3137ed thru 0x318fab (0x57bf bytes) 2024-09-04T17:46:09.852 INFO:teuthology.orchestra.run.smithi092.stdout:730 read 0x7e73e27 thru 0x7e74be9 (0xdc3 bytes) 2024-09-04T17:46:09.853 INFO:teuthology.orchestra.run.smithi092.stdout:731 flatten 2024-09-04T17:46:09.916 INFO:teuthology.orchestra.run.smithi092.stdout:732 punch from 0x6287837 to 0x62923aa, (0xab73 bytes) 2024-09-04T17:46:09.916 INFO:teuthology.orchestra.run.smithi092.stdout:733 write 0xecc5fa thru 0xed96f1 (0xd0f8 bytes) 2024-09-04T17:46:09.942 INFO:teuthology.orchestra.run.smithi092.stdout:734 write 0xb531a37 thru 0xb53d3bd (0xb987 bytes) 2024-09-04T17:46:09.944 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:09.944+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:09.952 INFO:teuthology.orchestra.run.smithi092.stdout:735 read 0xb119ffe thru 0xb123411 (0x9414 bytes) 2024-09-04T17:46:09.963 INFO:teuthology.orchestra.run.smithi092.stdout:736 writesame 0xbb59506 thru 0xbb5f29b data_size 0x21(0x5d96 bytes) 2024-09-04T17:46:09.975 INFO:teuthology.orchestra.run.smithi092.stdout:737 write 0xc22577f thru 0xc22d418 (0x7c9a bytes) 2024-09-04T17:46:09.980 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:09.980 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:09.986 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:09.986 INFO:teuthology.orchestra.run.smithi092.stdout:739 read 0x7f4b086 thru 0x7f50fcc (0x5f47 bytes) 2024-09-04T17:46:09.986 INFO:teuthology.orchestra.run.smithi092.stdout:741 write 0x9bc74f0 thru 0x9bceeac (0x79bd bytes) 2024-09-04T17:46:09.997 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-04T17:46:09.998 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:10.010 INFO:teuthology.orchestra.run.smithi092.stdout:743 trunc from 0xc22d419 to 0x3cca22e 2024-09-04T17:46:10.045 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:10.042+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:10.058 INFO:teuthology.orchestra.run.smithi092.stdout:744 write 0xa19082b thru 0xa191484 (0xc5a bytes) 2024-09-04T17:46:10.064 INFO:teuthology.orchestra.run.smithi092.stdout:746 read 0x5cc3a92 thru 0x5cc5b7f (0x20ee bytes) 2024-09-04T17:46:10.064 INFO:teuthology.orchestra.run.smithi092.stdout:747 write 0x708fc71 thru 0x7097e43 (0x81d3 bytes) 2024-09-04T17:46:10.073 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:10.073 INFO:teuthology.orchestra.run.smithi092.stdout:749 write 0x75f21 thru 0x7cf9e (0x707e bytes) 2024-09-04T17:46:10.076 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:10.076 INFO:teuthology.orchestra.run.smithi092.stdout:751 read 0xc1a26e thru 0xc283e8 (0xe17b bytes) 2024-09-04T17:46:10.096 INFO:teuthology.orchestra.run.smithi092.stdout:753 write 0xebdc008 thru 0xebe3969 (0x7962 bytes) 2024-09-04T17:46:10.104 INFO:teuthology.orchestra.run.smithi092.stdout:754 write 0x2fa7628 thru 0x2fb5408 (0xdde1 bytes) 2024-09-04T17:46:10.114 INFO:teuthology.orchestra.run.smithi092.stdout:755 write 0x7e1fb99 thru 0x7e1feb7 (0x31f bytes) 2024-09-04T17:46:10.118 INFO:teuthology.orchestra.run.smithi092.stdout:756 read 0x27e4721 thru 0x27ed261 (0x8b41 bytes) 2024-09-04T17:46:10.118 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:10.118 INFO:teuthology.orchestra.run.smithi092.stdout:760 punch from 0x597eecd to 0x5983c3b, (0x4d6e bytes) 2024-09-04T17:46:10.119 INFO:teuthology.orchestra.run.smithi092.stdout:761 write 0xd8abbcc thru 0xd8af128 (0x355d bytes) 2024-09-04T17:46:10.127 INFO:teuthology.orchestra.run.smithi092.stdout:763 read 0x6dda500 thru 0x6de0bfc (0x66fd bytes) 2024-09-04T17:46:10.128 INFO:teuthology.orchestra.run.smithi092.stdout:764 write 0x55212d8 thru 0x5524669 (0x3392 bytes) 2024-09-04T17:46:10.139 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:10.139 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:10.142 INFO:teuthology.orchestra.run.smithi092.stdout:766 trunc from 0xebe396a to 0xd47b661 2024-09-04T17:46:10.145 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:10.144+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:10.156 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-04T17:46:10.166 INFO:teuthology.orchestra.run.smithi092.stdout:767 write 0xa78ce2d thru 0xa78fd74 (0x2f48 bytes) 2024-09-04T17:46:10.170 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:10.171 INFO:teuthology.orchestra.run.smithi092.stdout:771 read 0x9d7dd48 thru 0x9d88dfe (0xb0b7 bytes) 2024-09-04T17:46:10.172 INFO:teuthology.orchestra.run.smithi092.stdout:772 writesame 0x365cbf8 thru 0x3660203 data_size 0x6(0x360c bytes) 2024-09-04T17:46:10.181 INFO:teuthology.orchestra.run.smithi092.stdout:774 read 0x33b50fb thru 0x33c2e59 (0xdd5f bytes) 2024-09-04T17:46:10.181 INFO:teuthology.orchestra.run.smithi092.stdout:775 read 0x3418f2e thru 0x341c9e9 (0x3abc bytes) 2024-09-04T17:46:10.181 INFO:teuthology.orchestra.run.smithi092.stdout:776 write 0xb9b950d thru 0xb9c03b5 (0x6ea9 bytes) 2024-09-04T17:46:10.189 INFO:teuthology.orchestra.run.smithi092.stdout:777 write 0xb880bb5 thru 0xb88758d (0x69d9 bytes) 2024-09-04T17:46:10.193 INFO:teuthology.orchestra.run.smithi092.stdout:778 read 0x6c30dfe thru 0x6c38f78 (0x817b bytes) 2024-09-04T17:46:10.193 INFO:teuthology.orchestra.run.smithi092.stdout:779 write 0x324201e thru 0x3244e59 (0x2e3c bytes) 2024-09-04T17:46:10.199 INFO:teuthology.orchestra.run.smithi092.stdout:781 punch from 0xbad49b to 0xbbbba2, (0xe707 bytes) 2024-09-04T17:46:10.199 INFO:teuthology.orchestra.run.smithi092.stdout:782 read 0x2c4393a thru 0x2c4f569 (0xbc30 bytes) 2024-09-04T17:46:10.200 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:10.200 INFO:teuthology.orchestra.run.smithi092.stdout:784 write 0xc4562f4 thru 0xc462a53 (0xc760 bytes) 2024-09-04T17:46:10.210 INFO:teuthology.orchestra.run.smithi092.stdout:785 read 0x4f06ae6 thru 0x4f07a57 (0xf72 bytes) 2024-09-04T17:46:10.210 INFO:teuthology.orchestra.run.smithi092.stdout:786 punch from 0xf9aa02 to 0xfa61d7, (0xb7d5 bytes) 2024-09-04T17:46:10.210 INFO:teuthology.orchestra.run.smithi092.stdout:787 punch from 0x3e539c1 to 0x3e62039, (0xe678 bytes) 2024-09-04T17:46:10.210 INFO:teuthology.orchestra.run.smithi092.stdout:789 write 0xa387ca5 thru 0xa390151 (0x84ad bytes) 2024-09-04T17:46:10.220 INFO:teuthology.orchestra.run.smithi092.stdout:790 read 0x27f7bee thru 0x2805b7e (0xdf91 bytes) 2024-09-04T17:46:10.220 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is set 2024-09-04T17:46:10.220 INFO:teuthology.orchestra.run.smithi092.stdout:791 writesame 0x42c589d thru 0x42cba1b data_size 0x1(0x617f bytes) 2024-09-04T17:46:10.247 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:10.246+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:10.257 INFO:teuthology.orchestra.run.smithi092.stdout:793 trunc from 0xd47b661 to 0x4f86c76 2024-09-04T17:46:10.287 INFO:teuthology.orchestra.run.smithi092.stdout:794 read 0x140c2ae thru 0x1412c50 (0x69a3 bytes) 2024-09-04T17:46:10.295 INFO:teuthology.orchestra.run.smithi092.stdout:795 trunc from 0x4f86c76 to 0xa0c6d8b 2024-09-04T17:46:10.303 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:10.303 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:10.307 INFO:teuthology.orchestra.run.smithi092.stdout:798 read 0x9d4abbe thru 0x9d59609 (0xea4c bytes) 2024-09-04T17:46:10.307 INFO:teuthology.orchestra.run.smithi092.stdout:799 writesame 0x4642f79 thru 0x4646e07 data_size 0x1(0x3e8f bytes) 2024-09-04T17:46:10.315 INFO:teuthology.orchestra.run.smithi092.stdout:800 write 0x7fbd8d4 thru 0x7fc121b (0x3948 bytes) 2024-09-04T17:46:10.319 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-04T17:46:10.322 INFO:teuthology.orchestra.run.smithi092.stdout:801 write 0x88e72cd thru 0x88eba19 (0x474d bytes) 2024-09-04T17:46:10.327 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:10.327 INFO:teuthology.orchestra.run.smithi092.stdout:805 write 0x103a87 thru 0x10e86c (0xade6 bytes) 2024-09-04T17:46:10.347 INFO:teuthology.orchestra.run.smithi092.stdout:806 write 0xe1eb53f thru 0xe1ecc0c (0x16ce bytes) 2024-09-04T17:46:10.347 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:46:10.348+0000 7f505d642640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26446) UID: 0 2024-09-04T17:46:10.353 INFO:teuthology.orchestra.run.smithi092.stdout:807 read 0x9c42969 thru 0x9c5049e (0xdb36 bytes) 2024-09-04T17:46:10.373 INFO:teuthology.orchestra.run.smithi092.stdout:809 trunc from 0xe1ecc0d to 0x4bceb8 2024-09-04T17:46:10.414 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:10.414 INFO:teuthology.orchestra.run.smithi092.stdout:812 read 0x2d8c9a thru 0x2d965a (0x9c1 bytes) 2024-09-04T17:46:10.424 INFO:teuthology.orchestra.run.smithi092.stdout:814 trunc from 0x4bceb8 to 0x6bf832a 2024-09-04T17:46:10.448 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:10.448+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:10.451 INFO:teuthology.orchestra.run.smithi092.stdout:815 write 0x8328ef6 thru 0x8331711 (0x881c bytes) 2024-09-04T17:46:10.452 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:10.453 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:10.471 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-04T17:46:10.473 INFO:teuthology.orchestra.run.smithi092.stdout:816 trunc from 0x8331712 to 0xe03e029 2024-09-04T17:46:10.484 INFO:teuthology.orchestra.run.smithi092.stdout:818 read 0xb537cf8 thru 0xb545cff (0xe008 bytes) 2024-09-04T17:46:10.494 INFO:teuthology.orchestra.run.smithi092.stdout:820 write 0xea1bae3 thru 0xea28365 (0xc883 bytes) 2024-09-04T17:46:10.505 INFO:teuthology.orchestra.run.smithi092.stdout:821 write 0xa6fc256 thru 0xa7000e8 (0x3e93 bytes) 2024-09-04T17:46:10.511 INFO:teuthology.orchestra.run.smithi092.stdout:823 read 0xb77e028 thru 0xb782877 (0x4850 bytes) 2024-09-04T17:46:10.511 INFO:teuthology.orchestra.run.smithi092.stdout:825 write 0xa817d47 thru 0xa824269 (0xc523 bytes) 2024-09-04T17:46:10.521 INFO:teuthology.orchestra.run.smithi092.stdout:826 read 0x3c7aa5f thru 0x3c8394c (0x8eee bytes) 2024-09-04T17:46:10.522 INFO:teuthology.orchestra.run.smithi092.stdout:828 writesame 0xcde43f1 thru 0xcde4dd1 data_size 0x3(0x9e1 bytes) 2024-09-04T17:46:10.525 INFO:teuthology.orchestra.run.smithi092.stdout:829 clone 7 order 23 su 262144 sc 6 2024-09-04T17:46:10.549 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:10.548+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:10.606 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:10.606 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:10.620 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-04T17:46:10.667 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:46:10.668+0000 7f505d642640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26446) UID: 0 2024-09-04T17:46:10.750 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:10.748+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:10.789 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:10.790 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:10.804 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-04T17:46:10.813 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-04T17:46:10.862 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:10.862+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:10.952 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:10.950+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:10.966 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:46:10.967 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:46:10.975 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:10.975 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:10.983 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:46:10.983 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:46:10.989 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-04T17:46:11.000 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-04T17:46:11.000 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-04T17:46:11.000 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-04T17:46:11.017 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:46:11.017 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:46:11.035 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:46:11.035 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:46:11.053 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:11.052+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:11.054 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-04T17:46:11.054 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-04T17:46:11.114 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:11.114 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:46:11.114 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:46:11.114 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:11.128 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-04T17:46:11.154 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:11.154+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:11.255 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:11.256+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:11.279 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:46:11.279 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:46:11.283 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-04T17:46:11.284 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-04T17:46:11.300 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:11.300 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:46:11.300 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:46:11.301 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:46:11.301 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:11.315 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-04T17:46:11.356 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:11.354+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:11.457 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:11.454+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:11.485 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:11.485 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:46:11.485 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:46:11.485 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:46:11.485 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.485 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4713 3.c 3:370edc5a:::rbd_data.113417f3df25.0000000000000000:5 [read 3047424~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:11.485 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.357178+0000", 2024-09-04T17:46:11.485 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.1281980090000001, 2024-09-04T17:46:11.485 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.028824949999999998, 2024-09-04T17:46:11.486 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:11.486 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:11.486 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:11.486 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:11.486 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:11.486 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:11.486 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4713 2024-09-04T17:46:11.486 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.486 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:11.486 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.486 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:11.486 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357178+0000", 2024-09-04T17:46:11.486 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:11.487 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.487 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.487 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:11.487 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357178+0000", 2024-09-04T17:46:11.487 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:11.487 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.487 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.487 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:11.487 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357179+0000", 2024-09-04T17:46:11.487 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.0299999999999998e-07 2024-09-04T17:46:11.487 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.487 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.487 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:11.488 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357182+0000", 2024-09-04T17:46:11.488 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.8140000000000001e-06 2024-09-04T17:46:11.488 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.488 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.488 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:11.488 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357192+0000", 2024-09-04T17:46:11.488 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5670000000000001e-06 2024-09-04T17:46:11.488 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.488 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.488 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:11.488 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357198+0000", 2024-09-04T17:46:11.488 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.9659999999999999e-06 2024-09-04T17:46:11.488 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.489 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.489 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:11.489 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.385796+0000", 2024-09-04T17:46:11.489 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.028597305999999999 2024-09-04T17:46:11.489 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.489 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.489 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:11.489 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.385871+0000", 2024-09-04T17:46:11.489 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.5469999999999994e-05 2024-09-04T17:46:11.489 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.489 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.489 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:11.489 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.386003+0000", 2024-09-04T17:46:11.490 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013202400000000001 2024-09-04T17:46:11.490 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:11.490 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:11.490 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:11.490 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.490 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.490 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4715 3.c 3:370edc5a:::rbd_data.113417f3df25.0000000000000000:5 [read 3473408~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:11.490 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.357229+0000", 2024-09-04T17:46:11.490 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.1281476079999999, 2024-09-04T17:46:11.490 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.029072164000000001, 2024-09-04T17:46:11.490 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:11.490 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:11.491 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:11.491 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:11.491 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:11.491 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:11.491 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4715 2024-09-04T17:46:11.491 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.491 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:11.491 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.491 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:11.491 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357229+0000", 2024-09-04T17:46:11.491 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:11.491 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.491 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.492 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:11.492 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357229+0000", 2024-09-04T17:46:11.492 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:11.492 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.492 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.492 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:11.492 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357229+0000", 2024-09-04T17:46:11.492 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1299999999999999e-07 2024-09-04T17:46:11.492 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.492 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.492 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:11.492 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357233+0000", 2024-09-04T17:46:11.493 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.754e-06 2024-09-04T17:46:11.493 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.493 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.493 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:11.493 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357245+0000", 2024-09-04T17:46:11.493 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1588e-05 2024-09-04T17:46:11.493 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.493 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.493 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:11.493 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357253+0000", 2024-09-04T17:46:11.493 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.0369999999999995e-06 2024-09-04T17:46:11.494 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.494 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.494 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:11.494 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.386036+0000", 2024-09-04T17:46:11.494 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.028783534 2024-09-04T17:46:11.494 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.494 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.494 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:11.494 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.386132+0000", 2024-09-04T17:46:11.494 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5561000000000002e-05 2024-09-04T17:46:11.495 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.495 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.495 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:11.495 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.386301+0000", 2024-09-04T17:46:11.495 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00016897700000000001 2024-09-04T17:46:11.495 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:11.495 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:11.495 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:11.495 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.495 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.495 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4716 3.c 3:370edc5a:::rbd_data.113417f3df25.0000000000000000:5 [read 3899392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:11.495 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.357265+0000", 2024-09-04T17:46:11.495 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.1281112240000004, 2024-09-04T17:46:11.495 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.029266948000000001, 2024-09-04T17:46:11.496 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:11.496 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:11.496 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:11.496 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:11.496 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:11.496 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:11.496 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4716 2024-09-04T17:46:11.496 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.496 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:11.496 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.496 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:11.496 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357265+0000", 2024-09-04T17:46:11.496 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:11.497 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.497 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.497 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:11.497 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357265+0000", 2024-09-04T17:46:11.497 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:11.497 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.497 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.497 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:11.497 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357266+0000", 2024-09-04T17:46:11.497 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1200000000000001e-07 2024-09-04T17:46:11.497 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.497 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.497 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:11.498 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357276+0000", 2024-09-04T17:46:11.498 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0208e-05 2024-09-04T17:46:11.498 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.498 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.498 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:11.498 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357285+0000", 2024-09-04T17:46:11.498 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9959999999999999e-06 2024-09-04T17:46:11.498 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.498 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.498 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:11.498 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357293+0000", 2024-09-04T17:46:11.498 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.311e-06 2024-09-04T17:46:11.498 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.500 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.500 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:11.500 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.386333+0000", 2024-09-04T17:46:11.500 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.029040442999999999 2024-09-04T17:46:11.500 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.501 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.501 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:11.501 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.386413+0000", 2024-09-04T17:46:11.501 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.9716999999999996e-05 2024-09-04T17:46:11.501 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.501 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.501 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:11.501 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.386532+0000", 2024-09-04T17:46:11.501 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000119361 2024-09-04T17:46:11.501 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:11.501 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:11.501 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:11.501 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.502 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.502 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4739 3.c 3:37f94306:::rbd_data.113417f3df25.0000000000000006:5 [read 229376~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:11.502 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.357303+0000", 2024-09-04T17:46:11.502 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.1280731169999996, 2024-09-04T17:46:11.502 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.029554939999999998, 2024-09-04T17:46:11.502 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:11.502 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:11.502 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:11.502 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:11.502 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:11.502 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:11.502 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4739 2024-09-04T17:46:11.503 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.503 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:11.503 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.503 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:11.503 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357303+0000", 2024-09-04T17:46:11.503 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:11.503 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.503 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.503 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:11.503 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357303+0000", 2024-09-04T17:46:11.503 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:11.503 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.503 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.504 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:11.504 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357304+0000", 2024-09-04T17:46:11.504 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.13e-07 2024-09-04T17:46:11.504 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.504 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.504 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:11.504 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357307+0000", 2024-09-04T17:46:11.504 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.8660000000000002e-06 2024-09-04T17:46:11.504 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.504 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.504 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:11.504 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357315+0000", 2024-09-04T17:46:11.505 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.4530000000000002e-06 2024-09-04T17:46:11.505 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.505 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.505 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:11.505 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357322+0000", 2024-09-04T17:46:11.505 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6470000000000001e-06 2024-09-04T17:46:11.505 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.505 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.505 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:11.505 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.386582+0000", 2024-09-04T17:46:11.505 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.029259721999999998 2024-09-04T17:46:11.505 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.505 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.505 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:11.506 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.386703+0000", 2024-09-04T17:46:11.506 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00012124400000000001 2024-09-04T17:46:11.506 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.506 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.506 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:11.506 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.386858+0000", 2024-09-04T17:46:11.506 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00015539500000000001 2024-09-04T17:46:11.506 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:11.506 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:11.506 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:11.506 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.506 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.506 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4742 3.c 3:37f94306:::rbd_data.113417f3df25.0000000000000006:5 [read 655360~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:11.507 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.357331+0000", 2024-09-04T17:46:11.507 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.1280451639999995, 2024-09-04T17:46:11.507 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.029685888000000001, 2024-09-04T17:46:11.507 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:11.507 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:11.507 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:11.507 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:11.507 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:11.509 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:11.509 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4742 2024-09-04T17:46:11.510 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.510 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:11.510 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.510 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:11.510 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357331+0000", 2024-09-04T17:46:11.510 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:11.510 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.510 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.510 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:11.510 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357331+0000", 2024-09-04T17:46:11.510 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:11.510 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.511 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.511 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:11.511 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357332+0000", 2024-09-04T17:46:11.511 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.0299999999999999e-07 2024-09-04T17:46:11.511 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.511 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.511 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:11.511 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357335+0000", 2024-09-04T17:46:11.511 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.791e-06 2024-09-04T17:46:11.511 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.511 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.511 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:11.512 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357343+0000", 2024-09-04T17:46:11.512 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.3610000000000001e-06 2024-09-04T17:46:11.512 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.512 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.512 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:11.512 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357349+0000", 2024-09-04T17:46:11.512 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.2879999999999998e-06 2024-09-04T17:46:11.512 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.512 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.512 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:11.513 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.386890+0000", 2024-09-04T17:46:11.513 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.029540789000000001 2024-09-04T17:46:11.513 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.513 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.513 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:11.514 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.386938+0000", 2024-09-04T17:46:11.514 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.8140000000000003e-05 2024-09-04T17:46:11.514 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.514 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.514 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:11.514 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.387017+0000", 2024-09-04T17:46:11.515 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.8916000000000001e-05 2024-09-04T17:46:11.515 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:11.515 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:11.515 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:11.515 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.516 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.516 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4747 3.c 3:37f94306:::rbd_data.113417f3df25.0000000000000006:5 [read 1081344~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:11.516 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.357360+0000", 2024-09-04T17:46:11.516 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.1280166109999996, 2024-09-04T17:46:11.516 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.029971241999999999, 2024-09-04T17:46:11.517 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:11.517 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:11.517 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:11.517 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:11.517 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:11.517 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:11.517 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4747 2024-09-04T17:46:11.517 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.517 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:11.517 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.518 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:11.518 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357360+0000", 2024-09-04T17:46:11.518 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:11.518 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.518 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.518 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:11.518 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357360+0000", 2024-09-04T17:46:11.518 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:11.518 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.518 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.518 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:11.518 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357360+0000", 2024-09-04T17:46:11.518 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.75e-07 2024-09-04T17:46:11.519 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.519 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.519 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:11.519 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357363+0000", 2024-09-04T17:46:11.519 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.683e-06 2024-09-04T17:46:11.519 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.519 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.519 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:11.519 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.357371+0000", 2024-09-04T17:46:11.519 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.9289999999999998e-06 2024-09-04T17:46:11.519 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.519 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.519 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:11.520 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.358831+0000", 2024-09-04T17:46:11.520 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0014602969999999999 2024-09-04T17:46:11.520 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.520 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.520 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:11.520 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.387067+0000", 2024-09-04T17:46:11.520 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.028235577000000001 2024-09-04T17:46:11.520 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.520 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.520 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:11.520 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.387167+0000", 2024-09-04T17:46:11.520 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.9773999999999997e-05 2024-09-04T17:46:11.520 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.521 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.521 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:11.521 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.387331+0000", 2024-09-04T17:46:11.521 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00016440700000000001 2024-09-04T17:46:11.521 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:11.521 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:11.521 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:11.521 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.521 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.521 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4786 3.c 3:370edc5a:::rbd_data.113417f3df25.0000000000000000:5 [read 983040~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:11.521 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.361528+0000", 2024-09-04T17:46:11.521 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.1238481269999996, 2024-09-04T17:46:11.522 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.028764359999999999, 2024-09-04T17:46:11.522 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:11.522 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:11.522 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:11.522 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:11.522 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:11.522 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:11.522 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4786 2024-09-04T17:46:11.522 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.522 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:11.522 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.522 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:11.522 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361528+0000", 2024-09-04T17:46:11.523 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:11.523 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.523 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.523 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:11.523 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361528+0000", 2024-09-04T17:46:11.523 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:11.523 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.523 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.523 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:11.523 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361529+0000", 2024-09-04T17:46:11.523 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9800000000000004e-07 2024-09-04T17:46:11.523 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.523 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.524 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:11.524 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361538+0000", 2024-09-04T17:46:11.524 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1789999999999997e-06 2024-09-04T17:46:11.524 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.524 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.524 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:11.524 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361547+0000", 2024-09-04T17:46:11.524 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9190000000000005e-06 2024-09-04T17:46:11.524 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.524 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.524 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:11.524 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361554+0000", 2024-09-04T17:46:11.524 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1269999999999997e-06 2024-09-04T17:46:11.525 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.525 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.525 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:11.525 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.390031+0000", 2024-09-04T17:46:11.525 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.028477308 2024-09-04T17:46:11.525 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.525 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.525 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:11.525 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.390131+0000", 2024-09-04T17:46:11.525 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.9776000000000002e-05 2024-09-04T17:46:11.525 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.525 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.525 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:11.526 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.390293+0000", 2024-09-04T17:46:11.526 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00016155299999999999 2024-09-04T17:46:11.526 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:11.526 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:11.526 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:11.526 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.526 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.526 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4787 3.c 3:370edc5a:::rbd_data.113417f3df25.0000000000000000:5 [read 1409024~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:11.526 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.361564+0000", 2024-09-04T17:46:11.526 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.123812311, 2024-09-04T17:46:11.526 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.028991560999999999, 2024-09-04T17:46:11.526 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:11.526 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:11.527 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:11.527 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:11.527 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:11.527 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:11.527 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4787 2024-09-04T17:46:11.527 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.527 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:11.527 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.527 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:11.527 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361564+0000", 2024-09-04T17:46:11.527 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:11.527 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.527 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.528 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:11.528 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361564+0000", 2024-09-04T17:46:11.528 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:11.528 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.528 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.528 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:11.528 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361565+0000", 2024-09-04T17:46:11.528 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.3e-07 2024-09-04T17:46:11.528 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.528 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.528 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:11.528 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361568+0000", 2024-09-04T17:46:11.529 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.8839999999999998e-06 2024-09-04T17:46:11.529 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.529 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.529 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:11.529 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361578+0000", 2024-09-04T17:46:11.529 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0406e-05 2024-09-04T17:46:11.529 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.529 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.529 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:11.529 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361584+0000", 2024-09-04T17:46:11.529 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.3940000000000001e-06 2024-09-04T17:46:11.529 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.529 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.530 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:11.530 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.390343+0000", 2024-09-04T17:46:11.530 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.028758224999999998 2024-09-04T17:46:11.530 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.530 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.530 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:11.530 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.390422+0000", 2024-09-04T17:46:11.530 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.9636999999999997e-05 2024-09-04T17:46:11.530 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.530 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.530 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:11.530 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.390556+0000", 2024-09-04T17:46:11.530 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000133385 2024-09-04T17:46:11.531 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:11.531 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:11.531 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:11.531 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.531 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.531 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4788 3.c 3:370edc5a:::rbd_data.113417f3df25.0000000000000000:5 [read 1835008~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:11.531 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.361593+0000", 2024-09-04T17:46:11.531 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.1237829699999997, 2024-09-04T17:46:11.531 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.029263407000000002, 2024-09-04T17:46:11.531 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:11.531 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:11.531 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:11.531 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:11.532 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:11.532 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:11.532 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4788 2024-09-04T17:46:11.532 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.532 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:11.532 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.532 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:11.532 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361593+0000", 2024-09-04T17:46:11.532 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:11.532 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.532 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.532 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:11.532 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361593+0000", 2024-09-04T17:46:11.532 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:11.533 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.533 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.533 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:11.533 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361594+0000", 2024-09-04T17:46:11.533 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7299999999999996e-07 2024-09-04T17:46:11.533 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.533 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.533 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:11.533 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361597+0000", 2024-09-04T17:46:11.533 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.8700000000000001e-06 2024-09-04T17:46:11.533 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.533 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.533 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:11.534 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361605+0000", 2024-09-04T17:46:11.534 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.3259999999999996e-06 2024-09-04T17:46:11.534 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.534 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.534 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:11.534 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361612+0000", 2024-09-04T17:46:11.534 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.3810000000000004e-06 2024-09-04T17:46:11.534 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.534 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.534 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:11.534 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.390608+0000", 2024-09-04T17:46:11.534 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.028996688999999999 2024-09-04T17:46:11.534 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.535 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.535 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:11.535 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.390703+0000", 2024-09-04T17:46:11.535 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4394999999999993e-05 2024-09-04T17:46:11.535 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.535 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.535 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:11.535 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.390857+0000", 2024-09-04T17:46:11.535 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00015417299999999999 2024-09-04T17:46:11.535 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:11.535 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:11.535 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:11.535 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.536 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.536 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4789 3.c 3:370edc5a:::rbd_data.113417f3df25.0000000000000000:5 [read 2260992~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:11.536 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.361620+0000", 2024-09-04T17:46:11.536 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.1237558520000004, 2024-09-04T17:46:11.536 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.029448236999999999, 2024-09-04T17:46:11.536 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:11.536 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:11.536 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:11.536 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:11.536 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:11.536 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:11.536 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4789 2024-09-04T17:46:11.536 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.537 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:11.537 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.537 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:11.537 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361620+0000", 2024-09-04T17:46:11.537 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:11.537 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.537 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.537 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:11.537 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361620+0000", 2024-09-04T17:46:11.537 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:11.537 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.537 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.537 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:11.538 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361621+0000", 2024-09-04T17:46:11.538 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7000000000000005e-07 2024-09-04T17:46:11.538 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.538 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.538 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:11.538 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361624+0000", 2024-09-04T17:46:11.538 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.8789999999999999e-06 2024-09-04T17:46:11.538 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.538 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.538 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:11.538 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361632+0000", 2024-09-04T17:46:11.538 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.3680000000000002e-06 2024-09-04T17:46:11.538 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.539 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.539 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:11.539 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361639+0000", 2024-09-04T17:46:11.539 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.3450000000000002e-06 2024-09-04T17:46:11.539 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.539 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.539 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:11.539 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.390900+0000", 2024-09-04T17:46:11.539 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.02926161 2024-09-04T17:46:11.539 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.539 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.539 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:11.540 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.390968+0000", 2024-09-04T17:46:11.540 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7941999999999994e-05 2024-09-04T17:46:11.540 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.540 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.540 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:11.540 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.391069+0000", 2024-09-04T17:46:11.540 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000100523 2024-09-04T17:46:11.540 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:11.540 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:11.540 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:11.540 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.540 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.541 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4790 3.c 3:370edc5a:::rbd_data.113417f3df25.0000000000000000:5 [read 2686976~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:11.541 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.361648+0000", 2024-09-04T17:46:11.541 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.1237285860000004, 2024-09-04T17:46:11.541 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.029697429000000001, 2024-09-04T17:46:11.541 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:11.541 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:11.541 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:11.541 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:11.541 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:11.541 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:11.541 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4790 2024-09-04T17:46:11.542 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.542 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:11.542 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.542 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:11.542 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361648+0000", 2024-09-04T17:46:11.542 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:11.542 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.542 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.542 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:11.542 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361648+0000", 2024-09-04T17:46:11.542 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:11.542 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.542 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.543 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:11.543 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361648+0000", 2024-09-04T17:46:11.543 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.1799999999999995e-07 2024-09-04T17:46:11.543 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.543 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.543 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:11.543 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361651+0000", 2024-09-04T17:46:11.543 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.706e-06 2024-09-04T17:46:11.543 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.543 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.543 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:11.543 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361659+0000", 2024-09-04T17:46:11.544 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.0439999999999996e-06 2024-09-04T17:46:11.544 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.544 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.544 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:11.544 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361666+0000", 2024-09-04T17:46:11.544 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.5069999999999997e-06 2024-09-04T17:46:11.544 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.544 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.544 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:11.544 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.391109+0000", 2024-09-04T17:46:11.544 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.029443039000000001 2024-09-04T17:46:11.544 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.544 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.545 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:11.545 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.391197+0000", 2024-09-04T17:46:11.545 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7991999999999994e-05 2024-09-04T17:46:11.545 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.545 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.545 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:11.545 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.391345+0000", 2024-09-04T17:46:11.545 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014852299999999999 2024-09-04T17:46:11.545 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:11.545 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:11.545 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:11.545 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.545 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.546 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4791 3.c 3:370edc5a:::rbd_data.113417f3df25.0000000000000000:5 [read 3112960~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:11.546 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.361675+0000", 2024-09-04T17:46:11.546 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.1237018020000002, 2024-09-04T17:46:11.546 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.029981876000000001, 2024-09-04T17:46:11.546 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:11.546 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:11.546 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:11.546 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:11.546 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:11.546 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:11.546 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4791 2024-09-04T17:46:11.546 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.547 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:11.547 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.547 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:11.547 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361675+0000", 2024-09-04T17:46:11.547 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:11.547 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.547 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.547 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:11.547 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361675+0000", 2024-09-04T17:46:11.547 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:11.547 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.547 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.548 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:11.548 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361675+0000", 2024-09-04T17:46:11.548 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6700000000000003e-07 2024-09-04T17:46:11.548 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.548 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.548 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:11.548 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361678+0000", 2024-09-04T17:46:11.548 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.954e-06 2024-09-04T17:46:11.548 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.548 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.548 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:11.548 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361686+0000", 2024-09-04T17:46:11.548 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.259e-06 2024-09-04T17:46:11.549 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.549 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.549 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:11.549 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361693+0000", 2024-09-04T17:46:11.549 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.5400000000000001e-06 2024-09-04T17:46:11.549 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.549 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.549 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:11.549 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.391401+0000", 2024-09-04T17:46:11.549 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.029708427999999999 2024-09-04T17:46:11.549 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.549 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.549 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:11.550 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.391500+0000", 2024-09-04T17:46:11.550 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.8329e-05 2024-09-04T17:46:11.550 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.550 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.550 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:11.550 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.391656+0000", 2024-09-04T17:46:11.550 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000156799 2024-09-04T17:46:11.550 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:11.550 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:11.550 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:11.550 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.550 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.550 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4792 3.c 3:370edc5a:::rbd_data.113417f3df25.0000000000000000:5 [read 3538944~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:11.551 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.361701+0000", 2024-09-04T17:46:11.551 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.1236748509999996, 2024-09-04T17:46:11.551 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.030264729000000001, 2024-09-04T17:46:11.551 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:11.551 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:11.551 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:11.551 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:11.551 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:11.551 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:11.551 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4792 2024-09-04T17:46:11.551 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.551 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:11.551 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.552 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:11.552 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361701+0000", 2024-09-04T17:46:11.552 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:11.552 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.552 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.552 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:11.552 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361701+0000", 2024-09-04T17:46:11.552 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:11.552 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.552 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.552 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:11.552 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361702+0000", 2024-09-04T17:46:11.552 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1200000000000003e-07 2024-09-04T17:46:11.553 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.553 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.553 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:11.553 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361705+0000", 2024-09-04T17:46:11.553 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.7870000000000002e-06 2024-09-04T17:46:11.553 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.553 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.553 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:11.553 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361713+0000", 2024-09-04T17:46:11.553 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.3829999999999992e-06 2024-09-04T17:46:11.553 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.553 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.554 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:11.554 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361720+0000", 2024-09-04T17:46:11.554 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7909999999999999e-06 2024-09-04T17:46:11.554 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.554 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.554 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:11.554 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.391734+0000", 2024-09-04T17:46:11.554 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.030014320000000001 2024-09-04T17:46:11.554 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.554 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.554 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:11.554 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.391821+0000", 2024-09-04T17:46:11.554 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.6546999999999997e-05 2024-09-04T17:46:11.554 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.555 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.555 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:11.555 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.391966+0000", 2024-09-04T17:46:11.555 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000145389 2024-09-04T17:46:11.555 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:11.555 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:11.555 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:11.555 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.555 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.555 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4793 3.c 3:370edc5a:::rbd_data.113417f3df25.0000000000000000:5 [read 3964928~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:11.555 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.361728+0000", 2024-09-04T17:46:11.555 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.1236479380000004, 2024-09-04T17:46:11.555 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.030481419999999999, 2024-09-04T17:46:11.556 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:11.556 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:11.556 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:11.556 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:11.556 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:11.556 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:11.556 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4793 2024-09-04T17:46:11.556 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.556 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:11.556 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.556 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:11.556 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361728+0000", 2024-09-04T17:46:11.556 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:11.557 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.557 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.557 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:11.557 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361728+0000", 2024-09-04T17:46:11.557 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:11.557 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.557 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.557 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:11.557 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361729+0000", 2024-09-04T17:46:11.557 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.1699999999999997e-07 2024-09-04T17:46:11.557 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.557 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.558 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:11.558 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361732+0000", 2024-09-04T17:46:11.558 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4159999999999998e-06 2024-09-04T17:46:11.558 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.558 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.558 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:11.558 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361739+0000", 2024-09-04T17:46:11.558 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.6149999999999999e-06 2024-09-04T17:46:11.558 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.558 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.558 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:11.558 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.361746+0000", 2024-09-04T17:46:11.558 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.3160000000000002e-06 2024-09-04T17:46:11.559 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.559 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.559 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:11.559 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.392002+0000", 2024-09-04T17:46:11.559 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.030256555000000001 2024-09-04T17:46:11.559 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.559 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.559 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:11.559 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.392076+0000", 2024-09-04T17:46:11.559 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3382999999999999e-05 2024-09-04T17:46:11.559 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.559 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.559 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:11.560 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.392210+0000", 2024-09-04T17:46:11.560 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000134318 2024-09-04T17:46:11.560 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:11.560 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:11.560 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:11.560 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.560 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.560 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4843 3.c 3:37f94306:::rbd_data.113417f3df25.0000000000000006:5 [read 98304~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:11.560 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.362055+0000", 2024-09-04T17:46:11.560 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.1233212979999996, 2024-09-04T17:46:11.560 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.030407970999999999, 2024-09-04T17:46:11.560 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:11.561 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:11.561 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:11.561 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:11.561 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:11.561 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:11.561 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4843 2024-09-04T17:46:11.561 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.561 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:11.561 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.561 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:11.561 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.362055+0000", 2024-09-04T17:46:11.561 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:11.561 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.562 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.562 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:11.562 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.362055+0000", 2024-09-04T17:46:11.562 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:11.562 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.562 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.562 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:11.562 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.362056+0000", 2024-09-04T17:46:11.562 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6400000000000002e-07 2024-09-04T17:46:11.562 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.562 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.562 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:11.562 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.362059+0000", 2024-09-04T17:46:11.563 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.8759999999999998e-06 2024-09-04T17:46:11.563 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.563 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.563 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:11.563 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.362067+0000", 2024-09-04T17:46:11.563 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.9969999999999998e-06 2024-09-04T17:46:11.563 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.563 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.563 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:11.563 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.362074+0000", 2024-09-04T17:46:11.563 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7120000000000003e-06 2024-09-04T17:46:11.563 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.563 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.564 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:11.564 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.392246+0000", 2024-09-04T17:46:11.564 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.030171678 2024-09-04T17:46:11.564 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.564 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.564 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:11.564 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.392334+0000", 2024-09-04T17:46:11.564 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7887999999999996e-05 2024-09-04T17:46:11.564 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.564 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.564 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:11.564 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.392463+0000", 2024-09-04T17:46:11.565 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00012925600000000001 2024-09-04T17:46:11.565 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:11.565 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:11.565 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:11.565 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.565 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.565 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4844 3.c 3:37f94306:::rbd_data.113417f3df25.0000000000000006:5 [read 524288~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:11.565 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.362083+0000", 2024-09-04T17:46:11.565 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.1232929379999996, 2024-09-04T17:46:11.565 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.030622511000000002, 2024-09-04T17:46:11.565 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:11.565 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:11.565 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:11.566 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:11.566 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:11.566 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:11.566 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4844 2024-09-04T17:46:11.566 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.566 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:11.566 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.566 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:11.566 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.362083+0000", 2024-09-04T17:46:11.566 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:11.566 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.566 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.567 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:11.567 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.362083+0000", 2024-09-04T17:46:11.567 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:11.567 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.567 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.567 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:11.567 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.362084+0000", 2024-09-04T17:46:11.567 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.8599999999999998e-07 2024-09-04T17:46:11.567 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.567 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.567 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:11.567 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.362087+0000", 2024-09-04T17:46:11.567 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.9040000000000001e-06 2024-09-04T17:46:11.568 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.568 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.568 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:11.568 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.362097+0000", 2024-09-04T17:46:11.568 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.6129999999999993e-06 2024-09-04T17:46:11.568 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.568 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.568 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:11.568 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.362103+0000", 2024-09-04T17:46:11.568 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6510000000000004e-06 2024-09-04T17:46:11.568 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.568 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.569 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:11.569 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.392496+0000", 2024-09-04T17:46:11.569 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.030393311999999999 2024-09-04T17:46:11.569 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.569 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.569 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:11.569 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.392581+0000", 2024-09-04T17:46:11.569 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.4668999999999999e-05 2024-09-04T17:46:11.569 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.569 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.569 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:11.569 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.392706+0000", 2024-09-04T17:46:11.570 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000124776 2024-09-04T17:46:11.570 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:11.570 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:11.570 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:11.570 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.570 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.570 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4845 3.c 3:37f94306:::rbd_data.113417f3df25.0000000000000006:5 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:11.570 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.362112+0000", 2024-09-04T17:46:11.570 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.1232642630000003, 2024-09-04T17:46:11.570 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.030831418999999999, 2024-09-04T17:46:11.570 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:11.570 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:11.570 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:11.571 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:11.571 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:11.571 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:11.571 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4845 2024-09-04T17:46:11.571 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.571 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:11.571 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.571 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:11.571 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.362112+0000", 2024-09-04T17:46:11.571 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:11.571 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.571 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.571 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:11.572 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.362112+0000", 2024-09-04T17:46:11.572 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:11.572 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.572 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.572 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:11.572 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.362113+0000", 2024-09-04T17:46:11.572 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.5199999999999996e-07 2024-09-04T17:46:11.572 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.572 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.572 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:11.572 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.362116+0000", 2024-09-04T17:46:11.572 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.8700000000000001e-06 2024-09-04T17:46:11.572 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.573 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.573 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:11.573 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.362124+0000", 2024-09-04T17:46:11.573 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.4780000000000008e-06 2024-09-04T17:46:11.573 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.573 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.573 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:11.573 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.362131+0000", 2024-09-04T17:46:11.573 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.489e-06 2024-09-04T17:46:11.573 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.573 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.573 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:11.574 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.392736+0000", 2024-09-04T17:46:11.574 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.030605842000000001 2024-09-04T17:46:11.574 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.574 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.574 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:11.574 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.392818+0000", 2024-09-04T17:46:11.574 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.1877000000000002e-05 2024-09-04T17:46:11.574 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.574 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.574 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:11.574 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.392944+0000", 2024-09-04T17:46:11.574 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00012521099999999999 2024-09-04T17:46:11.574 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:11.575 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:11.575 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:11.575 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.575 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.575 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4846 3.c 3:37f94306:::rbd_data.113417f3df25.0000000000000006:5 [read 1376256~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:11.575 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.362139+0000", 2024-09-04T17:46:11.575 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.1232369870000003, 2024-09-04T17:46:11.575 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.031054723999999999, 2024-09-04T17:46:11.575 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:11.575 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:11.575 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:11.575 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:11.575 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:11.576 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:11.576 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4846 2024-09-04T17:46:11.576 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.576 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:11.576 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.576 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:11.576 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.362139+0000", 2024-09-04T17:46:11.576 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:11.576 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.576 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.576 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:11.576 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.362139+0000", 2024-09-04T17:46:11.576 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:11.577 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.577 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.577 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:11.577 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.362140+0000", 2024-09-04T17:46:11.577 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.9100000000000004e-07 2024-09-04T17:46:11.577 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.577 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.577 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:11.577 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.362143+0000", 2024-09-04T17:46:11.577 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.615e-06 2024-09-04T17:46:11.577 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.577 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.577 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:11.578 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.362151+0000", 2024-09-04T17:46:11.578 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.9480000000000008e-06 2024-09-04T17:46:11.578 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.578 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.578 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:11.578 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.362157+0000", 2024-09-04T17:46:11.578 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.1489999999999999e-06 2024-09-04T17:46:11.578 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.578 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.578 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:11.578 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.392975+0000", 2024-09-04T17:46:11.578 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.030818808999999999 2024-09-04T17:46:11.578 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.579 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.579 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:11.579 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.393059+0000", 2024-09-04T17:46:11.579 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.3657000000000002e-05 2024-09-04T17:46:11.579 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.579 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.579 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:11.579 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.393194+0000", 2024-09-04T17:46:11.579 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000134955 2024-09-04T17:46:11.579 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:11.579 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:11.579 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:11.579 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.579 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.580 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4847 3.c 3:37f94306:::rbd_data.113417f3df25.0000000000000006:5 [read 1802240~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:11.580 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.362168+0000", 2024-09-04T17:46:11.580 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.1232082969999997, 2024-09-04T17:46:11.580 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.031266395000000002, 2024-09-04T17:46:11.580 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:11.580 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:11.580 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:11.580 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:11.580 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:11.580 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:11.580 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4847 2024-09-04T17:46:11.580 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.580 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:11.581 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.581 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:11.581 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.362168+0000", 2024-09-04T17:46:11.581 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:11.581 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.581 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.581 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:11.581 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.362168+0000", 2024-09-04T17:46:11.581 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:11.581 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.581 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.581 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:11.581 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.362169+0000", 2024-09-04T17:46:11.581 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.2799999999999996e-07 2024-09-04T17:46:11.582 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.582 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.582 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:11.582 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.362171+0000", 2024-09-04T17:46:11.582 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.8049999999999998e-06 2024-09-04T17:46:11.582 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.582 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.582 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:11.582 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.362180+0000", 2024-09-04T17:46:11.582 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.2379999999999997e-06 2024-09-04T17:46:11.582 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.582 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.582 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:11.583 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.362186+0000", 2024-09-04T17:46:11.583 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6109999999999999e-06 2024-09-04T17:46:11.583 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.583 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.583 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:11.583 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.393227+0000", 2024-09-04T17:46:11.583 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.031040221999999999 2024-09-04T17:46:11.583 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.583 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.583 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:11.583 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.393312+0000", 2024-09-04T17:46:11.583 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.5173999999999995e-05 2024-09-04T17:46:11.583 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.584 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.584 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:11.584 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.393434+0000", 2024-09-04T17:46:11.584 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000122717 2024-09-04T17:46:11.584 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:11.584 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:11.584 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:11.584 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.584 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.584 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:5506 3.5 3:a2d24818:::rbd_data.113431586065.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:11.584 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.765716+0000", 2024-09-04T17:46:11.584 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 3.7196605649999999, 2024-09-04T17:46:11.584 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.034908609, 2024-09-04T17:46:11.585 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:11.585 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:11.585 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:11.585 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:11.585 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:11.585 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:11.585 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 5506 2024-09-04T17:46:11.585 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.585 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:11.585 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.585 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:11.585 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.765716+0000", 2024-09-04T17:46:11.585 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:11.585 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.586 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.586 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:11.586 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.765716+0000", 2024-09-04T17:46:11.586 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:11.586 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.586 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.586 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:11.586 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.765718+0000", 2024-09-04T17:46:11.586 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.3860000000000001e-06 2024-09-04T17:46:11.586 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.586 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.586 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:11.586 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.770060+0000", 2024-09-04T17:46:11.587 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0043418170000000004 2024-09-04T17:46:11.587 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.587 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.587 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:11.587 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.770084+0000", 2024-09-04T17:46:11.587 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.3748999999999999e-05 2024-09-04T17:46:11.587 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.587 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.587 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:11.587 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.770099+0000", 2024-09-04T17:46:11.587 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4993e-05 2024-09-04T17:46:11.587 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.587 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.587 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:11.588 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.770236+0000", 2024-09-04T17:46:11.588 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013729700000000001 2024-09-04T17:46:11.588 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.588 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.588 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:11.588 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.770491+0000", 2024-09-04T17:46:11.588 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000255467 2024-09-04T17:46:11.588 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.588 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.588 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:11.588 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.800375+0000", 2024-09-04T17:46:11.588 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.029883542999999999 2024-09-04T17:46:11.588 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.589 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.589 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:11.589 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.800520+0000", 2024-09-04T17:46:11.589 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014463400000000001 2024-09-04T17:46:11.589 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:11.589 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:11.589 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:11.589 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.800624+0000", 2024-09-04T17:46:11.589 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00010472299999999999 2024-09-04T17:46:11.589 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:11.589 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:11.589 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:11.589 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:11.589 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:11.590 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:11.591 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-04T17:46:11.592 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:11.590+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:11.664 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #5, image image_client.0-clone5 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent6 2024-09-04T17:46:11.674 INFO:teuthology.orchestra.run.smithi092.stdout:831 read 0x6789fe thru 0x6797bd (0xdc0 bytes) 2024-09-04T17:46:11.676 INFO:teuthology.orchestra.run.smithi092.stdout:832 punch from 0x666b581 to 0x6670176, (0x4bf5 bytes) 2024-09-04T17:46:11.682 INFO:teuthology.orchestra.run.smithi092.stdout:833 read 0x29e3763 thru 0x29eabb2 (0x7450 bytes) 2024-09-04T17:46:11.682 INFO:teuthology.orchestra.run.smithi092.stdout:834 read 0xa7ae311 thru 0xa7b9c33 (0xb923 bytes) 2024-09-04T17:46:11.692 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:11.690+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:11.709 INFO:teuthology.orchestra.run.smithi092.stdout:835 trunc from 0xea28366 to 0x1669a09 2024-09-04T17:46:11.729 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:11.729 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:46:11.729 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:46:11.729 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:11.746 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-04T17:46:11.784 INFO:teuthology.orchestra.run.smithi092.stdout:836 writesame 0x98bb159 thru 0x98bd1cf data_size 0x1(0x2077 bytes) 2024-09-04T17:46:11.784 INFO:teuthology.orchestra.run.smithi092.stdout:838 writesame 0x49020e8 thru 0x490a222 data_size 0x1(0x813b bytes) 2024-09-04T17:46:11.795 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:11.796+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:11.830 INFO:teuthology.orchestra.run.smithi092.stdout:839 trunc from 0x98bd1d0 to 0x23d9292 2024-09-04T17:46:11.863 INFO:teuthology.orchestra.run.smithi092.stdout:840 trunc from 0x23d9292 to 0x35cc0a0 2024-09-04T17:46:11.873 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:11.873 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:46:11.873 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:46:11.873 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:46:11.873 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:11.888 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-04T17:46:11.893 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:11.890+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:11.897 INFO:teuthology.orchestra.run.smithi092.stdout:841 write 0x76a79fe thru 0x76ab4f7 (0x3afa bytes) 2024-09-04T17:46:11.907 INFO:teuthology.orchestra.run.smithi092.stdout:842 trunc from 0x76ab4f8 to 0x8b7ca09 2024-09-04T17:46:11.918 INFO:teuthology.orchestra.run.smithi092.stdout:843 read 0x54fc5ba thru 0x54fd131 (0xb78 bytes) 2024-09-04T17:46:11.918 INFO:teuthology.orchestra.run.smithi092.stdout:845 read 0x6e56dc8 thru 0x6e5e971 (0x7baa bytes) 2024-09-04T17:46:11.918 INFO:teuthology.orchestra.run.smithi092.stdout:846 write 0x6b74883 thru 0x6b75928 (0x10a6 bytes) 2024-09-04T17:46:11.938 INFO:teuthology.orchestra.run.smithi092.stdout:847 writesame 0xa84916f thru 0xa84f076 data_size 0x4(0x5f08 bytes) 2024-09-04T17:46:11.951 INFO:teuthology.orchestra.run.smithi092.stdout:848 trunc from 0xa84f077 to 0x5bd65fd 2024-09-04T17:46:11.955 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [3] live_osds: [0, 1, 2, 4, 5, 6, 7] 2024-09-04T17:46:11.955 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-04T17:46:11.956 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-04T17:46:11.956 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 1.0 2024-09-04T17:46:11.995 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:11.996+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:11.996 INFO:teuthology.orchestra.run.smithi092.stdout:849 write 0xd9aed2f thru 0xd9b592b (0x6bfd bytes) 2024-09-04T17:46:12.002 INFO:teuthology.orchestra.run.smithi092.stdout:850 trunc from 0xd9b592c to 0xe3ff222 2024-09-04T17:46:12.014 INFO:teuthology.orchestra.run.smithi092.stdout:853 writesame 0x3eb1024 thru 0x3eba292 data_size 0x1(0x926f bytes) 2024-09-04T17:46:12.062 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:12.062 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:46:12.062 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:46:12.062 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:46:12.062 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.062 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:1348 3.10 3:0afd7a80:::rbd_data.11346251f094.000000000000009b:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-04T17:46:12.062 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:02.609239+0000", 2024-09-04T17:46:12.063 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 9.4526479370000001, 2024-09-04T17:46:12.063 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.026961731999999999, 2024-09-04T17:46:12.063 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:12.063 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:12.063 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:12.063 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:12.063 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:12.063 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:12.063 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 1348 2024-09-04T17:46:12.063 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.063 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:12.063 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.063 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:12.063 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.609239+0000", 2024-09-04T17:46:12.064 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.064 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.064 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.064 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:12.064 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.609239+0000", 2024-09-04T17:46:12.064 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.064 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.064 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.064 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:12.064 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.609242+0000", 2024-09-04T17:46:12.064 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.3269999999999999e-06 2024-09-04T17:46:12.065 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.065 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.065 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:12.065 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.609252+0000", 2024-09-04T17:46:12.065 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0689e-05 2024-09-04T17:46:12.065 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.065 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.065 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:12.065 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.609282+0000", 2024-09-04T17:46:12.065 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.932e-05 2024-09-04T17:46:12.065 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.065 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.065 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:12.066 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.609296+0000", 2024-09-04T17:46:12.066 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4197e-05 2024-09-04T17:46:12.066 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.066 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.066 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:12.066 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.609408+0000", 2024-09-04T17:46:12.066 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000111962 2024-09-04T17:46:12.066 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.066 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.066 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:12.066 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.609544+0000", 2024-09-04T17:46:12.066 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000136706 2024-09-04T17:46:12.066 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.067 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.067 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 0", 2024-09-04T17:46:12.067 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.613984+0000", 2024-09-04T17:46:12.067 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0044397890000000004 2024-09-04T17:46:12.067 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.067 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.067 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:12.067 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.635928+0000", 2024-09-04T17:46:12.067 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.021943547000000001 2024-09-04T17:46:12.067 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.067 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.067 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:12.067 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.636030+0000", 2024-09-04T17:46:12.068 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00010199700000000001 2024-09-04T17:46:12.068 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.068 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.068 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:12.068 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.636123+0000", 2024-09-04T17:46:12.068 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.3628999999999994e-05 2024-09-04T17:46:12.068 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.068 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.068 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:12.068 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.636201+0000", 2024-09-04T17:46:12.068 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.7569000000000002e-05 2024-09-04T17:46:12.068 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.069 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:12.069 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.069 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.069 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.069 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4019 3.1e 3:7817bfb4:::rbd_data.113417f3df25.0000000000000001:5 [read 7634944~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-04T17:46:12.069 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:06.909047+0000", 2024-09-04T17:46:12.069 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 5.1528399819999997, 2024-09-04T17:46:12.069 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.026506477000000001, 2024-09-04T17:46:12.069 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:12.069 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:12.069 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:12.069 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:12.070 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:12.070 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:12.070 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4019 2024-09-04T17:46:12.070 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.070 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:12.070 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.070 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:12.070 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909047+0000", 2024-09-04T17:46:12.070 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.070 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.070 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.070 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:12.070 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909047+0000", 2024-09-04T17:46:12.071 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.071 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.071 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.071 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:12.071 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909048+0000", 2024-09-04T17:46:12.071 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0800000000000005e-07 2024-09-04T17:46:12.071 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.071 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.071 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:12.071 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909050+0000", 2024-09-04T17:46:12.071 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.587e-06 2024-09-04T17:46:12.071 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.071 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.072 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:12.072 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909057+0000", 2024-09-04T17:46:12.072 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.9269999999999999e-06 2024-09-04T17:46:12.072 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.072 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.072 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:12.072 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909062+0000", 2024-09-04T17:46:12.072 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2730000000000002e-06 2024-09-04T17:46:12.072 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.072 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.072 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:12.072 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.935341+0000", 2024-09-04T17:46:12.073 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.026278526 2024-09-04T17:46:12.073 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.073 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.073 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:12.073 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.935423+0000", 2024-09-04T17:46:12.073 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.2158999999999997e-05 2024-09-04T17:46:12.073 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.073 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.073 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:12.073 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.935554+0000", 2024-09-04T17:46:12.073 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013049700000000001 2024-09-04T17:46:12.073 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.073 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:12.074 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.074 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.074 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.074 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4020 3.1e 3:7817bfb4:::rbd_data.113417f3df25.0000000000000001:5 [read 8060928~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-04T17:46:12.074 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:06.909070+0000", 2024-09-04T17:46:12.074 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 5.1528167969999998, 2024-09-04T17:46:12.074 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.026770842999999999, 2024-09-04T17:46:12.074 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:12.074 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:12.074 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:12.074 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:12.074 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:12.075 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:12.075 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4020 2024-09-04T17:46:12.075 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.075 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:12.075 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.075 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:12.075 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909070+0000", 2024-09-04T17:46:12.075 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.075 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.075 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.075 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:12.075 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909070+0000", 2024-09-04T17:46:12.075 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.076 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.076 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.076 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:12.076 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909071+0000", 2024-09-04T17:46:12.076 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9599999999999999e-07 2024-09-04T17:46:12.076 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.076 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.076 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:12.076 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909073+0000", 2024-09-04T17:46:12.076 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2010000000000002e-06 2024-09-04T17:46:12.076 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.076 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.077 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:12.077 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909080+0000", 2024-09-04T17:46:12.077 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.391e-06 2024-09-04T17:46:12.077 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.077 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.077 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:12.077 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909086+0000", 2024-09-04T17:46:12.077 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1499999999999998e-06 2024-09-04T17:46:12.077 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.077 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.078 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:12.078 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.935601+0000", 2024-09-04T17:46:12.078 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.026515318999999999 2024-09-04T17:46:12.078 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.078 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.078 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:12.078 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.935699+0000", 2024-09-04T17:46:12.078 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.7956999999999997e-05 2024-09-04T17:46:12.079 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.079 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.079 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:12.079 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.935841+0000", 2024-09-04T17:46:12.079 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014232900000000001 2024-09-04T17:46:12.079 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.079 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:12.079 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.079 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.079 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.080 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4076 3.1e 3:7817bfb4:::rbd_data.113417f3df25.0000000000000001:5 [read 4259840~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-04T17:46:12.080 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:06.909093+0000", 2024-09-04T17:46:12.080 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 5.1527942879999999, 2024-09-04T17:46:12.080 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.026930059999999999, 2024-09-04T17:46:12.080 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:12.080 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:12.080 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:12.080 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:12.080 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:12.080 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:12.080 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4076 2024-09-04T17:46:12.081 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.081 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:12.081 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.081 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:12.081 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909093+0000", 2024-09-04T17:46:12.081 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.081 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.081 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.081 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:12.081 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909093+0000", 2024-09-04T17:46:12.081 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.082 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.082 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.082 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:12.082 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909094+0000", 2024-09-04T17:46:12.082 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7999999999999995e-07 2024-09-04T17:46:12.082 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.082 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.082 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:12.082 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909096+0000", 2024-09-04T17:46:12.082 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1959999999999998e-06 2024-09-04T17:46:12.082 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.082 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.082 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:12.083 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909103+0000", 2024-09-04T17:46:12.083 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.6499999999999996e-06 2024-09-04T17:46:12.083 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.083 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.083 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:12.083 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909108+0000", 2024-09-04T17:46:12.083 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0860000000000001e-06 2024-09-04T17:46:12.083 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.083 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.083 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:12.083 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.935885+0000", 2024-09-04T17:46:12.083 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.026776687 2024-09-04T17:46:12.083 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.083 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.083 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:12.084 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.935946+0000", 2024-09-04T17:46:12.084 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.0473000000000002e-05 2024-09-04T17:46:12.084 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.084 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.084 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:12.084 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.936023+0000", 2024-09-04T17:46:12.084 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.7287999999999996e-05 2024-09-04T17:46:12.084 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.084 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:12.084 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.084 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.084 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.084 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4077 3.1e 3:7817bfb4:::rbd_data.113417f3df25.0000000000000001:5 [read 4685824~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-04T17:46:12.084 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:06.909117+0000", 2024-09-04T17:46:12.085 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 5.1527705680000002, 2024-09-04T17:46:12.085 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.027219975, 2024-09-04T17:46:12.085 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:12.085 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:12.085 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:12.085 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:12.085 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:12.085 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:12.085 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4077 2024-09-04T17:46:12.085 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.085 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:12.085 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.086 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:12.086 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909117+0000", 2024-09-04T17:46:12.086 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.086 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.086 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.086 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:12.086 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909117+0000", 2024-09-04T17:46:12.086 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.086 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.086 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.086 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:12.086 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909117+0000", 2024-09-04T17:46:12.086 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.8299999999999997e-07 2024-09-04T17:46:12.086 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.087 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.087 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:12.087 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909119+0000", 2024-09-04T17:46:12.087 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2680000000000001e-06 2024-09-04T17:46:12.087 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.087 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.087 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:12.087 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909127+0000", 2024-09-04T17:46:12.087 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.9670000000000001e-06 2024-09-04T17:46:12.087 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.087 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.087 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:12.087 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909133+0000", 2024-09-04T17:46:12.087 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.215e-06 2024-09-04T17:46:12.087 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.088 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.088 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:12.088 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.936076+0000", 2024-09-04T17:46:12.088 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.026943292000000001 2024-09-04T17:46:12.088 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.088 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.088 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:12.088 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.936171+0000", 2024-09-04T17:46:12.088 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4933999999999996e-05 2024-09-04T17:46:12.088 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.088 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.088 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:12.088 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.936337+0000", 2024-09-04T17:46:12.089 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000165816 2024-09-04T17:46:12.089 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.089 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:12.089 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.089 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.089 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.089 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4078 3.1e 3:7817bfb4:::rbd_data.113417f3df25.0000000000000001:5 [read 5111808~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-04T17:46:12.089 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:06.909140+0000", 2024-09-04T17:46:12.089 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 5.1527469430000004, 2024-09-04T17:46:12.089 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.027371784, 2024-09-04T17:46:12.089 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:12.089 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:12.090 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:12.090 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:12.090 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:12.090 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:12.090 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4078 2024-09-04T17:46:12.090 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.090 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:12.090 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.090 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:12.090 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909140+0000", 2024-09-04T17:46:12.090 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.090 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.091 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.091 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:12.091 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909140+0000", 2024-09-04T17:46:12.091 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.091 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.091 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.091 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:12.091 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909141+0000", 2024-09-04T17:46:12.091 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6800000000000001e-07 2024-09-04T17:46:12.091 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.091 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.091 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:12.092 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909153+0000", 2024-09-04T17:46:12.092 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2542e-05 2024-09-04T17:46:12.092 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.092 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.092 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:12.092 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909162+0000", 2024-09-04T17:46:12.092 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.833e-06 2024-09-04T17:46:12.092 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.092 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.092 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:12.092 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909174+0000", 2024-09-04T17:46:12.092 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2429000000000001e-05 2024-09-04T17:46:12.092 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.093 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.093 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:12.093 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.936371+0000", 2024-09-04T17:46:12.093 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.027196708999999999 2024-09-04T17:46:12.093 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.093 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.093 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:12.093 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.936432+0000", 2024-09-04T17:46:12.093 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.0707e-05 2024-09-04T17:46:12.093 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.093 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.093 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:12.093 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.936512+0000", 2024-09-04T17:46:12.094 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.0096e-05 2024-09-04T17:46:12.094 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.094 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:12.094 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.094 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.094 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.094 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4079 3.1e 3:7817bfb4:::rbd_data.113417f3df25.0000000000000001:5 [read 5537792~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-04T17:46:12.094 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:06.909184+0000", 2024-09-04T17:46:12.094 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 5.1527036229999998, 2024-09-04T17:46:12.094 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.027638882, 2024-09-04T17:46:12.094 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:12.094 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:12.095 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:12.095 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:12.095 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:12.095 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:12.095 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4079 2024-09-04T17:46:12.095 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.095 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:12.095 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.095 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:12.095 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909184+0000", 2024-09-04T17:46:12.095 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.095 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.095 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.096 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:12.096 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909184+0000", 2024-09-04T17:46:12.096 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.096 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.096 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.096 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:12.096 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909184+0000", 2024-09-04T17:46:12.096 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2e-07 2024-09-04T17:46:12.096 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.097 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.097 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:12.097 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909187+0000", 2024-09-04T17:46:12.097 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.576e-06 2024-09-04T17:46:12.097 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.097 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.097 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:12.097 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909193+0000", 2024-09-04T17:46:12.097 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6449999999999999e-06 2024-09-04T17:46:12.097 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.097 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.097 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:12.097 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909199+0000", 2024-09-04T17:46:12.098 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2939999999999996e-06 2024-09-04T17:46:12.098 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.098 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.098 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:12.098 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.936548+0000", 2024-09-04T17:46:12.098 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.027349605999999999 2024-09-04T17:46:12.098 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.098 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.098 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:12.098 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.936640+0000", 2024-09-04T17:46:12.098 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.2194000000000005e-05 2024-09-04T17:46:12.098 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.098 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.099 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:12.099 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.936822+0000", 2024-09-04T17:46:12.099 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00018204699999999999 2024-09-04T17:46:12.099 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.099 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:12.099 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.099 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.099 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.099 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4080 3.1e 3:7817bfb4:::rbd_data.113417f3df25.0000000000000001:5 [read 5963776~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-04T17:46:12.099 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:06.909207+0000", 2024-09-04T17:46:12.099 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 5.1526806470000004, 2024-09-04T17:46:12.099 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.027874711, 2024-09-04T17:46:12.100 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:12.100 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:12.100 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:12.100 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:12.100 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:12.100 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:12.100 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4080 2024-09-04T17:46:12.100 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.100 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:12.100 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.100 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:12.100 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909207+0000", 2024-09-04T17:46:12.100 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.101 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.101 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.101 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:12.101 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909207+0000", 2024-09-04T17:46:12.101 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.101 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.101 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.101 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:12.101 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909207+0000", 2024-09-04T17:46:12.101 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9900000000000001e-07 2024-09-04T17:46:12.101 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.101 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.101 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:12.102 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909209+0000", 2024-09-04T17:46:12.102 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1639999999999999e-06 2024-09-04T17:46:12.102 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.102 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.102 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:12.102 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909217+0000", 2024-09-04T17:46:12.102 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4869999999999996e-06 2024-09-04T17:46:12.102 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.102 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.102 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:12.102 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909222+0000", 2024-09-04T17:46:12.102 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1780000000000002e-06 2024-09-04T17:46:12.103 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.103 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.103 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:12.103 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.936856+0000", 2024-09-04T17:46:12.103 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.027634394999999999 2024-09-04T17:46:12.103 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.103 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.103 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:12.103 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.936940+0000", 2024-09-04T17:46:12.103 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.4241999999999997e-05 2024-09-04T17:46:12.103 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.103 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.103 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:12.104 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.937081+0000", 2024-09-04T17:46:12.104 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000140746 2024-09-04T17:46:12.104 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.104 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:12.104 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.104 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.104 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.104 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4081 3.1e 3:7817bfb4:::rbd_data.113417f3df25.0000000000000001:5 [read 6389760~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-04T17:46:12.104 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:06.909230+0000", 2024-09-04T17:46:12.104 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 5.1526575709999998, 2024-09-04T17:46:12.104 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.028121515, 2024-09-04T17:46:12.104 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:12.104 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:12.104 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:12.105 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:12.105 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:12.105 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:12.105 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4081 2024-09-04T17:46:12.105 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.105 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:12.105 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.105 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:12.105 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909230+0000", 2024-09-04T17:46:12.105 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.105 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.105 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.106 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:12.106 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909230+0000", 2024-09-04T17:46:12.106 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.106 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.106 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.106 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:12.106 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909230+0000", 2024-09-04T17:46:12.106 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4700000000000002e-07 2024-09-04T17:46:12.106 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.106 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.106 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:12.106 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909232+0000", 2024-09-04T17:46:12.106 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2110000000000001e-06 2024-09-04T17:46:12.106 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.106 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.107 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:12.107 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909240+0000", 2024-09-04T17:46:12.107 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.5460000000000003e-06 2024-09-04T17:46:12.107 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.107 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.107 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:12.107 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909245+0000", 2024-09-04T17:46:12.107 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1900000000000003e-06 2024-09-04T17:46:12.107 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.107 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.107 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:12.107 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.937119+0000", 2024-09-04T17:46:12.107 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.027874448999999999 2024-09-04T17:46:12.107 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.108 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.108 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:12.108 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.937209+0000", 2024-09-04T17:46:12.108 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9423000000000001e-05 2024-09-04T17:46:12.108 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.108 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.108 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:12.108 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.937351+0000", 2024-09-04T17:46:12.108 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000142249 2024-09-04T17:46:12.108 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.108 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:12.108 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.108 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.109 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.109 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4082 3.1e 3:7817bfb4:::rbd_data.113417f3df25.0000000000000001:5 [read 6815744~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-04T17:46:12.109 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:06.909253+0000", 2024-09-04T17:46:12.109 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 5.1526345569999998, 2024-09-04T17:46:12.109 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.028352584, 2024-09-04T17:46:12.109 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:12.109 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:12.109 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:12.109 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:12.109 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:12.109 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:12.109 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4082 2024-09-04T17:46:12.110 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.110 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:12.110 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.110 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:12.110 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909253+0000", 2024-09-04T17:46:12.110 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.110 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.110 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.110 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:12.110 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909253+0000", 2024-09-04T17:46:12.110 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.110 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.110 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.111 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:12.111 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909253+0000", 2024-09-04T17:46:12.111 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0999999999999999e-07 2024-09-04T17:46:12.111 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.111 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.111 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:12.111 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909255+0000", 2024-09-04T17:46:12.111 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1840000000000002e-06 2024-09-04T17:46:12.111 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.111 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.111 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:12.111 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909264+0000", 2024-09-04T17:46:12.111 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.422e-06 2024-09-04T17:46:12.111 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.112 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.112 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:12.112 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909277+0000", 2024-09-04T17:46:12.112 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3723000000000001e-05 2024-09-04T17:46:12.112 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.112 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.112 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:12.112 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.937384+0000", 2024-09-04T17:46:12.112 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.028106973 2024-09-04T17:46:12.112 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.112 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.112 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:12.112 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.937471+0000", 2024-09-04T17:46:12.112 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.6366999999999995e-05 2024-09-04T17:46:12.113 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.113 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.113 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:12.113 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.937605+0000", 2024-09-04T17:46:12.113 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013440500000000001 2024-09-04T17:46:12.113 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.113 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:12.113 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.113 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.113 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.113 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4083 3.1e 3:7817bfb4:::rbd_data.113417f3df25.0000000000000001:5 [read 7241728~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-04T17:46:12.113 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:06.909286+0000", 2024-09-04T17:46:12.114 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 5.1526011110000001, 2024-09-04T17:46:12.114 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.028595905000000001, 2024-09-04T17:46:12.114 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:12.114 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:12.114 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:12.114 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:12.114 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:12.114 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:12.114 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4083 2024-09-04T17:46:12.114 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.114 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:12.114 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.114 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:12.114 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909286+0000", 2024-09-04T17:46:12.115 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.115 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.115 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.115 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:12.115 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909286+0000", 2024-09-04T17:46:12.115 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.115 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.115 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.115 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:12.115 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909287+0000", 2024-09-04T17:46:12.115 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2399999999999998e-07 2024-09-04T17:46:12.115 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.115 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.116 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:12.116 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909295+0000", 2024-09-04T17:46:12.116 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.2339999999999994e-06 2024-09-04T17:46:12.116 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.116 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.116 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:12.116 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909301+0000", 2024-09-04T17:46:12.116 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.8340000000000004e-06 2024-09-04T17:46:12.116 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.116 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.116 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:12.116 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909306+0000", 2024-09-04T17:46:12.117 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.3159999999999996e-06 2024-09-04T17:46:12.117 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.117 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.117 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:12.117 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.937638+0000", 2024-09-04T17:46:12.117 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.028332267000000001 2024-09-04T17:46:12.117 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.117 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.117 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:12.117 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.937741+0000", 2024-09-04T17:46:12.117 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000102756 2024-09-04T17:46:12.117 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.118 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.118 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:12.118 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.937882+0000", 2024-09-04T17:46:12.118 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014097400000000001 2024-09-04T17:46:12.118 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.118 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:12.118 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.118 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.118 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.118 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4084 3.1e 3:7817bfb4:::rbd_data.113417f3df25.0000000000000001:5 [read 7667712~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-04T17:46:12.118 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:06.909314+0000", 2024-09-04T17:46:12.118 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 5.1525729030000003, 2024-09-04T17:46:12.118 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.028838895, 2024-09-04T17:46:12.118 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:12.119 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:12.119 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:12.119 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:12.119 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:12.119 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:12.119 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4084 2024-09-04T17:46:12.119 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.119 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:12.119 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.119 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:12.119 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909314+0000", 2024-09-04T17:46:12.119 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.119 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.120 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.120 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:12.120 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909314+0000", 2024-09-04T17:46:12.120 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.120 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.120 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.120 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:12.120 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909315+0000", 2024-09-04T17:46:12.120 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7999999999999995e-07 2024-09-04T17:46:12.120 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.120 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.120 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:12.120 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909317+0000", 2024-09-04T17:46:12.120 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2529999999999999e-06 2024-09-04T17:46:12.121 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.121 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.121 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:12.121 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909325+0000", 2024-09-04T17:46:12.121 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.6279999999999996e-06 2024-09-04T17:46:12.121 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.121 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.121 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:12.121 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909330+0000", 2024-09-04T17:46:12.121 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.3040000000000004e-06 2024-09-04T17:46:12.121 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.121 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.121 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:12.122 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.937915+0000", 2024-09-04T17:46:12.122 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.028584766000000001 2024-09-04T17:46:12.122 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.122 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.122 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:12.122 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.938002+0000", 2024-09-04T17:46:12.122 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7590999999999995e-05 2024-09-04T17:46:12.122 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.122 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.122 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:12.122 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.938153+0000", 2024-09-04T17:46:12.122 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00015067300000000001 2024-09-04T17:46:12.122 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.122 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:12.123 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.123 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.123 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.123 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4085 3.1e 3:7817bfb4:::rbd_data.113417f3df25.0000000000000001:5 [read 8093696~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-04T17:46:12.123 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:06.909338+0000", 2024-09-04T17:46:12.123 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 5.152549069, 2024-09-04T17:46:12.123 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.029098151999999999, 2024-09-04T17:46:12.123 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:12.123 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:12.123 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:12.123 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:12.123 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:12.123 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:12.124 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4085 2024-09-04T17:46:12.124 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.124 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:12.124 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.124 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:12.124 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909338+0000", 2024-09-04T17:46:12.124 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.124 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.124 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.124 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:12.124 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909338+0000", 2024-09-04T17:46:12.124 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.124 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.124 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.125 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:12.125 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909339+0000", 2024-09-04T17:46:12.125 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6100000000000001e-07 2024-09-04T17:46:12.125 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.125 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.125 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:12.125 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909356+0000", 2024-09-04T17:46:12.125 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7147e-05 2024-09-04T17:46:12.125 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.125 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.125 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:12.125 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909362+0000", 2024-09-04T17:46:12.125 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.657e-06 2024-09-04T17:46:12.126 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.126 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.126 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:12.126 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909368+0000", 2024-09-04T17:46:12.126 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6459999999999998e-06 2024-09-04T17:46:12.126 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.126 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.126 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:12.126 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.938194+0000", 2024-09-04T17:46:12.126 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.028826424 2024-09-04T17:46:12.126 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.126 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.126 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:12.127 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.938296+0000", 2024-09-04T17:46:12.127 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000101087 2024-09-04T17:46:12.127 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.127 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.127 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:12.127 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.938436+0000", 2024-09-04T17:46:12.127 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014072999999999999 2024-09-04T17:46:12.127 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.127 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:12.127 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.127 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.127 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.128 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4797 3.1e 3:7817bfb4:::rbd_data.113417f3df25.0000000000000001:5 [read 1474560~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:12.128 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.350022+0000", 2024-09-04T17:46:12.128 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.7118655489999997, 2024-09-04T17:46:12.128 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.026399108000000001, 2024-09-04T17:46:12.128 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:12.128 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:12.128 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:12.128 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:12.128 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:12.128 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:12.128 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4797 2024-09-04T17:46:12.128 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.128 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:12.129 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.129 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:12.129 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350022+0000", 2024-09-04T17:46:12.129 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.129 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.129 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.129 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:12.129 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350022+0000", 2024-09-04T17:46:12.129 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.129 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.129 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.129 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:12.130 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350023+0000", 2024-09-04T17:46:12.130 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0160000000000001e-06 2024-09-04T17:46:12.130 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.130 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.130 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:12.130 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350038+0000", 2024-09-04T17:46:12.130 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5671999999999998e-05 2024-09-04T17:46:12.130 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.130 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.130 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:12.130 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350049+0000", 2024-09-04T17:46:12.130 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0266e-05 2024-09-04T17:46:12.130 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.131 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.131 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:12.131 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350056+0000", 2024-09-04T17:46:12.131 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4900000000000003e-06 2024-09-04T17:46:12.131 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.131 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.131 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:12.131 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.374015+0000", 2024-09-04T17:46:12.131 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.023959081 2024-09-04T17:46:12.131 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.131 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.131 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:12.131 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.374112+0000", 2024-09-04T17:46:12.131 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.7322000000000001e-05 2024-09-04T17:46:12.132 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.132 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.132 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:12.132 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.376421+0000", 2024-09-04T17:46:12.132 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0023082609999999998 2024-09-04T17:46:12.132 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.132 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:12.132 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.132 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.132 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.132 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4798 3.1e 3:7817bfb4:::rbd_data.113417f3df25.0000000000000001:5 [read 1900544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:12.132 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.350077+0000", 2024-09-04T17:46:12.133 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.7118098399999999, 2024-09-04T17:46:12.133 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.026604033999999999, 2024-09-04T17:46:12.133 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:12.133 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:12.133 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:12.133 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:12.133 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:12.133 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:12.133 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4798 2024-09-04T17:46:12.133 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.133 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:12.133 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.133 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:12.134 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350077+0000", 2024-09-04T17:46:12.134 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.134 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.134 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.134 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:12.134 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350077+0000", 2024-09-04T17:46:12.134 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.134 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.134 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.134 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:12.134 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350078+0000", 2024-09-04T17:46:12.134 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.4300000000000003e-07 2024-09-04T17:46:12.134 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.134 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.135 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:12.135 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350082+0000", 2024-09-04T17:46:12.135 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.9829999999999998e-06 2024-09-04T17:46:12.135 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.135 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.135 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:12.135 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350091+0000", 2024-09-04T17:46:12.135 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.3500000000000003e-06 2024-09-04T17:46:12.135 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.135 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.135 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:12.135 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350099+0000", 2024-09-04T17:46:12.135 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.7500000000000003e-06 2024-09-04T17:46:12.135 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.136 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.136 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:12.136 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.376457+0000", 2024-09-04T17:46:12.136 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.026358320000000001 2024-09-04T17:46:12.136 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.136 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.136 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:12.136 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.376552+0000", 2024-09-04T17:46:12.136 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5074000000000005e-05 2024-09-04T17:46:12.136 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.136 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.136 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:12.136 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.376681+0000", 2024-09-04T17:46:12.137 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00012891399999999999 2024-09-04T17:46:12.137 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.137 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:12.137 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.137 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.137 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.137 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4799 3.1e 3:7817bfb4:::rbd_data.113417f3df25.0000000000000001:5 [read 2326528~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:12.137 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.350111+0000", 2024-09-04T17:46:12.137 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.7117763960000003, 2024-09-04T17:46:12.137 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.029614627000000001, 2024-09-04T17:46:12.137 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:12.137 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:12.138 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:12.138 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:12.138 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:12.138 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:12.138 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4799 2024-09-04T17:46:12.138 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.138 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:12.138 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.138 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:12.139 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350111+0000", 2024-09-04T17:46:12.139 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.139 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.139 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.139 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:12.139 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350111+0000", 2024-09-04T17:46:12.139 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.139 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.139 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.139 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:12.139 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350111+0000", 2024-09-04T17:46:12.139 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.9599999999999999e-07 2024-09-04T17:46:12.139 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.140 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.140 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:12.140 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350115+0000", 2024-09-04T17:46:12.140 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.506e-06 2024-09-04T17:46:12.140 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.140 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.140 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:12.140 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350126+0000", 2024-09-04T17:46:12.140 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0658e-05 2024-09-04T17:46:12.140 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.140 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.140 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:12.141 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350133+0000", 2024-09-04T17:46:12.141 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2130000000000002e-06 2024-09-04T17:46:12.141 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.141 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.141 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:12.141 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.376712+0000", 2024-09-04T17:46:12.141 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.026579479 2024-09-04T17:46:12.141 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.141 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.141 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:12.141 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.379533+0000", 2024-09-04T17:46:12.142 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0028206780000000001 2024-09-04T17:46:12.142 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.142 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.142 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:12.142 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.379725+0000", 2024-09-04T17:46:12.142 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019239699999999999 2024-09-04T17:46:12.142 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.142 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:12.142 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.142 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.142 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.142 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4800 3.1e 3:7817bfb4:::rbd_data.113417f3df25.0000000000000001:5 [read 2752512~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:12.142 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.350151+0000", 2024-09-04T17:46:12.143 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.7117357320000002, 2024-09-04T17:46:12.143 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.029858974999999999, 2024-09-04T17:46:12.143 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:12.143 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:12.143 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:12.143 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:12.143 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:12.143 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:12.143 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4800 2024-09-04T17:46:12.143 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.143 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:12.143 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.144 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:12.144 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350151+0000", 2024-09-04T17:46:12.144 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.144 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.144 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.144 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:12.144 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350151+0000", 2024-09-04T17:46:12.144 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.144 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.144 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.144 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:12.144 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350152+0000", 2024-09-04T17:46:12.145 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.4899999999999995e-07 2024-09-04T17:46:12.145 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.145 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.145 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:12.145 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350156+0000", 2024-09-04T17:46:12.145 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.6160000000000002e-06 2024-09-04T17:46:12.145 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.145 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.145 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:12.146 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350166+0000", 2024-09-04T17:46:12.146 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0168e-05 2024-09-04T17:46:12.146 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.146 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.146 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:12.146 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350173+0000", 2024-09-04T17:46:12.146 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6830000000000002e-06 2024-09-04T17:46:12.146 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.146 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.146 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:12.146 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.379767+0000", 2024-09-04T17:46:12.146 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.029594915999999999 2024-09-04T17:46:12.147 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.147 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.147 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:12.147 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.379864+0000", 2024-09-04T17:46:12.147 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.6164999999999998e-05 2024-09-04T17:46:12.147 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.147 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.147 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:12.147 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.380010+0000", 2024-09-04T17:46:12.147 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000146778 2024-09-04T17:46:12.147 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.148 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:12.148 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.148 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.148 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.148 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4801 3.1e 3:7817bfb4:::rbd_data.113417f3df25.0000000000000001:5 [read 3178496~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:12.148 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.350183+0000", 2024-09-04T17:46:12.148 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.7117038830000002, 2024-09-04T17:46:12.148 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.030089563999999999, 2024-09-04T17:46:12.148 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:12.148 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:12.148 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:12.148 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:12.148 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:12.149 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:12.149 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4801 2024-09-04T17:46:12.149 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.149 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:12.149 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.149 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:12.149 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350183+0000", 2024-09-04T17:46:12.149 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.149 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.149 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.149 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:12.149 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350183+0000", 2024-09-04T17:46:12.149 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.150 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.150 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.150 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:12.150 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350184+0000", 2024-09-04T17:46:12.150 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3399999999999998e-07 2024-09-04T17:46:12.150 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.150 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.150 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:12.150 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350187+0000", 2024-09-04T17:46:12.150 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4470000000000002e-06 2024-09-04T17:46:12.150 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.151 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.151 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:12.151 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350196+0000", 2024-09-04T17:46:12.151 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9530000000000005e-06 2024-09-04T17:46:12.151 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.151 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.151 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:12.151 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350203+0000", 2024-09-04T17:46:12.151 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.4160000000000001e-06 2024-09-04T17:46:12.151 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.151 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.151 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:12.152 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.380042+0000", 2024-09-04T17:46:12.152 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.029839099000000001 2024-09-04T17:46:12.152 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.152 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.152 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:12.152 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.380131+0000", 2024-09-04T17:46:12.152 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8857999999999995e-05 2024-09-04T17:46:12.152 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.152 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.152 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:12.152 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.380273+0000", 2024-09-04T17:46:12.152 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014205700000000001 2024-09-04T17:46:12.153 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.153 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:12.153 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.153 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.153 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.153 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4802 3.1e 3:7817bfb4:::rbd_data.113417f3df25.0000000000000001:5 [read 3604480~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:12.153 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.350221+0000", 2024-09-04T17:46:12.153 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.7116657020000003, 2024-09-04T17:46:12.153 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.030306293000000002, 2024-09-04T17:46:12.153 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:12.153 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:12.153 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:12.153 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:12.154 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:12.154 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:12.154 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4802 2024-09-04T17:46:12.154 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.154 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:12.154 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.154 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:12.154 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350221+0000", 2024-09-04T17:46:12.154 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.154 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.154 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.154 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:12.155 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350221+0000", 2024-09-04T17:46:12.155 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.155 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.155 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.155 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:12.155 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350223+0000", 2024-09-04T17:46:12.155 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1009999999999999e-06 2024-09-04T17:46:12.155 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.155 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.155 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:12.155 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350226+0000", 2024-09-04T17:46:12.155 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7000000000000002e-06 2024-09-04T17:46:12.156 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.156 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.156 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:12.156 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350236+0000", 2024-09-04T17:46:12.156 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5899999999999997e-06 2024-09-04T17:46:12.156 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.156 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.156 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:12.156 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350243+0000", 2024-09-04T17:46:12.156 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.5299999999999999e-06 2024-09-04T17:46:12.156 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.157 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.157 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:12.157 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.380303+0000", 2024-09-04T17:46:12.157 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.030060104000000001 2024-09-04T17:46:12.157 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.157 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.157 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:12.157 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.380390+0000", 2024-09-04T17:46:12.157 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.6420000000000003e-05 2024-09-04T17:46:12.157 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.157 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.158 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:12.158 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.380528+0000", 2024-09-04T17:46:12.158 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013784800000000001 2024-09-04T17:46:12.158 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.158 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:12.158 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.158 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.158 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.158 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4803 3.1e 3:7817bfb4:::rbd_data.113417f3df25.0000000000000001:5 [read 4030464~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:12.158 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.350254+0000", 2024-09-04T17:46:12.158 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.71163285, 2024-09-04T17:46:12.158 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.030523386999999999, 2024-09-04T17:46:12.158 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:12.159 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:12.159 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:12.159 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:12.159 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:12.159 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:12.159 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4803 2024-09-04T17:46:12.159 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.159 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:12.159 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.159 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:12.159 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350254+0000", 2024-09-04T17:46:12.159 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.159 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.160 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.160 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:12.160 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350254+0000", 2024-09-04T17:46:12.160 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.160 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.160 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.160 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:12.160 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350255+0000", 2024-09-04T17:46:12.160 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.2799999999999996e-07 2024-09-04T17:46:12.160 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.160 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.160 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:12.160 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350258+0000", 2024-09-04T17:46:12.161 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.1300000000000001e-06 2024-09-04T17:46:12.161 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.161 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.161 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:12.161 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350278+0000", 2024-09-04T17:46:12.161 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9584000000000001e-05 2024-09-04T17:46:12.161 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.161 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.161 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:12.161 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350285+0000", 2024-09-04T17:46:12.161 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3270000000000003e-06 2024-09-04T17:46:12.161 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.161 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.162 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:12.162 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.380558+0000", 2024-09-04T17:46:12.162 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.030273220999999999 2024-09-04T17:46:12.162 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.162 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.162 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:12.162 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.380640+0000", 2024-09-04T17:46:12.162 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.1772000000000002e-05 2024-09-04T17:46:12.162 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.162 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.162 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:12.162 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.380778+0000", 2024-09-04T17:46:12.162 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000137725 2024-09-04T17:46:12.163 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.163 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:12.163 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.163 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.163 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:12.163 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:12.165 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-04T17:46:12.166 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:46:12.164+0000 7f505d642640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26446) UID: 0 2024-09-04T17:46:12.166 INFO:teuthology.orchestra.run.smithi092.stdout:856 writesame 0x6b153ef thru 0x6b198b9 data_size 0x1(0x44cb bytes) 2024-09-04T17:46:12.166 INFO:teuthology.orchestra.run.smithi092.stdout:857 write 0x1ad7f7 thru 0x1ba53a (0xcd44 bytes) 2024-09-04T17:46:12.166 INFO:teuthology.orchestra.run.smithi092.stdout:858 read 0x341f5d5 thru 0x3422bd3 (0x35ff bytes) 2024-09-04T17:46:12.166 INFO:teuthology.orchestra.run.smithi092.stdout:859 write 0xb8de28e thru 0xb8de938 (0x6ab bytes) 2024-09-04T17:46:12.166 INFO:teuthology.orchestra.run.smithi092.stdout:861 read 0xc971201 thru 0xc978883 (0x7683 bytes) 2024-09-04T17:46:12.167 INFO:teuthology.orchestra.run.smithi092.stdout:862 write 0x4f57e4e thru 0x4f5e066 (0x6219 bytes) 2024-09-04T17:46:12.167 INFO:teuthology.orchestra.run.smithi092.stdout:863 read 0xae86abc thru 0xae93c38 (0xd17d bytes) 2024-09-04T17:46:12.167 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:12.167 INFO:teuthology.orchestra.run.smithi092.stdout:866 write 0x42c40cd thru 0x42c4a7c (0x9b0 bytes) 2024-09-04T17:46:12.167 INFO:teuthology.orchestra.run.smithi092.stdout:868 writesame 0x7b4bee5 thru 0x7b59bbc data_size 0x94(0xdcd8 bytes) 2024-09-04T17:46:12.167 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:12.167 INFO:teuthology.orchestra.run.smithi092.stdout:871 writesame 0xc29db34 thru 0xc2a8e56 data_size 0xb(0xb323 bytes) 2024-09-04T17:46:12.167 INFO:teuthology.orchestra.run.smithi092.stdout:873 clone 8 order 20 su 4096 sc 9 2024-09-04T17:46:12.248 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-04T17:46:12.265 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:46:12.264+0000 7f505d642640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26446) UID: 0 2024-09-04T17:46:12.342 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:12.342 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:46:12.342 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:46:12.342 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:12.358 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-04T17:46:12.371 INFO:teuthology.orchestra.run.smithi092.stdout:flattening image image_client.0-clone7 2024-09-04T17:46:12.403 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:12.404+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:12.467 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:12.468+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:12.528 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:12.528 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:46:12.528 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:46:12.528 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:46:12.528 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:12.543 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-04T17:46:12.569 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #6, image image_client.0-clone6 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent7 2024-09-04T17:46:12.587 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:12.588+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:12.669 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:46:12.668+0000 7f505d642640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26446) UID: 0 2024-09-04T17:46:12.719 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:12.719 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:46:12.719 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:46:12.720 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:46:12.720 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.720 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4219.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-04T17:46:12.720 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:45:42.487196+0000", 2024-09-04T17:46:12.720 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 30.232098764, 2024-09-04T17:46:12.720 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.021893024000000001, 2024-09-04T17:46:12.720 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:12.720 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:12.720 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:12.720 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:12.720 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4219", 2024-09-04T17:46:12.721 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.190:0/3705085728", 2024-09-04T17:46:12.721 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 50 2024-09-04T17:46:12.721 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.721 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:12.721 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.721 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:12.721 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487196+0000", 2024-09-04T17:46:12.721 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.721 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.721 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.721 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:12.721 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487196+0000", 2024-09-04T17:46:12.721 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.722 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.722 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.722 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:12.722 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487197+0000", 2024-09-04T17:46:12.722 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.06e-07 2024-09-04T17:46:12.722 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.722 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.722 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:12.722 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487270+0000", 2024-09-04T17:46:12.722 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2992999999999997e-05 2024-09-04T17:46:12.722 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.722 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.722 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:12.723 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487281+0000", 2024-09-04T17:46:12.723 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.13e-05 2024-09-04T17:46:12.723 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.723 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.723 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:12.723 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487289+0000", 2024-09-04T17:46:12.723 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.8429999999999993e-06 2024-09-04T17:46:12.723 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.723 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.723 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:12.723 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.504017+0000", 2024-09-04T17:46:12.723 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.016728481 2024-09-04T17:46:12.724 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.724 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.724 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:12.724 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.504153+0000", 2024-09-04T17:46:12.724 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000135396 2024-09-04T17:46:12.724 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.724 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.724 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 0", 2024-09-04T17:46:12.724 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.504378+0000", 2024-09-04T17:46:12.724 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000225255 2024-09-04T17:46:12.725 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.725 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.725 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:12.725 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.508835+0000", 2024-09-04T17:46:12.725 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0044573759999999999 2024-09-04T17:46:12.725 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.725 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.725 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:12.725 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.508936+0000", 2024-09-04T17:46:12.725 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000100555 2024-09-04T17:46:12.725 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.725 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.725 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:12.726 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.509021+0000", 2024-09-04T17:46:12.726 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.4546e-05 2024-09-04T17:46:12.726 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.726 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.726 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:12.726 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.509089+0000", 2024-09-04T17:46:12.726 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.8572999999999996e-05 2024-09-04T17:46:12.726 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.726 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:12.726 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.726 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.726 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.726 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4219.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-04T17:46:12.727 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:45:42.487303+0000", 2024-09-04T17:46:12.727 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 30.231991935, 2024-09-04T17:46:12.727 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.022763057, 2024-09-04T17:46:12.727 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:12.727 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:12.727 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:12.727 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:12.727 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4219", 2024-09-04T17:46:12.727 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.190:0/3705085728", 2024-09-04T17:46:12.728 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 51 2024-09-04T17:46:12.728 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.728 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:12.728 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.728 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:12.728 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487303+0000", 2024-09-04T17:46:12.728 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.729 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.729 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.729 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:12.729 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487303+0000", 2024-09-04T17:46:12.729 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.729 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.729 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.729 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:12.729 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487304+0000", 2024-09-04T17:46:12.729 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.7000000000000004e-07 2024-09-04T17:46:12.729 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.729 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.729 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:12.730 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487313+0000", 2024-09-04T17:46:12.730 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8200000000000003e-06 2024-09-04T17:46:12.730 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.730 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.730 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:12.730 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487322+0000", 2024-09-04T17:46:12.730 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4199999999999996e-06 2024-09-04T17:46:12.730 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.730 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.730 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:12.730 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487329+0000", 2024-09-04T17:46:12.730 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2599999999999999e-06 2024-09-04T17:46:12.730 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.731 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.731 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:12.731 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.506432+0000", 2024-09-04T17:46:12.731 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.019102592000000002 2024-09-04T17:46:12.731 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.731 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.731 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:12.731 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.506570+0000", 2024-09-04T17:46:12.731 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013837300000000001 2024-09-04T17:46:12.731 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.731 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.731 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 0", 2024-09-04T17:46:12.731 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.506771+0000", 2024-09-04T17:46:12.732 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000200748 2024-09-04T17:46:12.732 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.732 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.732 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:12.732 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.508859+0000", 2024-09-04T17:46:12.732 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0020881010000000002 2024-09-04T17:46:12.732 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.732 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.732 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:12.732 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.509919+0000", 2024-09-04T17:46:12.732 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0010601580000000001 2024-09-04T17:46:12.732 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.732 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.732 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:12.733 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.510002+0000", 2024-09-04T17:46:12.733 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.2719999999999994e-05 2024-09-04T17:46:12.733 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.733 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.733 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:12.733 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.510066+0000", 2024-09-04T17:46:12.733 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.4095000000000002e-05 2024-09-04T17:46:12.733 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.733 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:12.733 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.733 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.733 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.733 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4219.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-04T17:46:12.734 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:45:42.487345+0000", 2024-09-04T17:46:12.734 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 30.231950002000001, 2024-09-04T17:46:12.734 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.025395999999999998, 2024-09-04T17:46:12.734 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:12.734 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:12.734 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:12.734 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:12.734 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4219", 2024-09-04T17:46:12.734 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.190:0/3705085728", 2024-09-04T17:46:12.735 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 52 2024-09-04T17:46:12.735 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.735 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:12.735 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.735 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:12.735 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487345+0000", 2024-09-04T17:46:12.735 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.735 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.735 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.735 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:12.735 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487345+0000", 2024-09-04T17:46:12.735 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.735 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.736 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.736 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:12.736 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487346+0000", 2024-09-04T17:46:12.736 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.4600000000000004e-07 2024-09-04T17:46:12.736 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.736 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.736 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:12.736 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487351+0000", 2024-09-04T17:46:12.736 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.9399999999999999e-06 2024-09-04T17:46:12.736 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.736 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.736 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:12.736 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487361+0000", 2024-09-04T17:46:12.737 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.3840000000000003e-06 2024-09-04T17:46:12.737 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.737 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.737 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:12.737 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487368+0000", 2024-09-04T17:46:12.737 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1729999999999998e-06 2024-09-04T17:46:12.737 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.737 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.737 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:12.737 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.507626+0000", 2024-09-04T17:46:12.737 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.020257547000000001 2024-09-04T17:46:12.737 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.738 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.738 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:12.738 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.507861+0000", 2024-09-04T17:46:12.738 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000235708 2024-09-04T17:46:12.738 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.738 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.738 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 0", 2024-09-04T17:46:12.738 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.508188+0000", 2024-09-04T17:46:12.738 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00032624800000000001 2024-09-04T17:46:12.738 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.738 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.738 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:12.738 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.509521+0000", 2024-09-04T17:46:12.738 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.001333433 2024-09-04T17:46:12.739 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.739 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.739 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:12.739 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.512613+0000", 2024-09-04T17:46:12.739 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0030923919999999998 2024-09-04T17:46:12.739 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.739 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.739 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:12.739 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.512686+0000", 2024-09-04T17:46:12.739 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3107000000000003e-05 2024-09-04T17:46:12.739 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.739 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.739 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:12.740 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.512741+0000", 2024-09-04T17:46:12.740 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4422000000000002e-05 2024-09-04T17:46:12.740 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.740 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:12.740 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.740 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.740 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.740 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:1430 3.f 3:f58475cb:::rbd_data.11346251f094.0000000000000074:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-04T17:46:12.740 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:02.778704+0000", 2024-09-04T17:46:12.740 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 9.9405909000000001, 2024-09-04T17:46:12.740 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.058567801000000003, 2024-09-04T17:46:12.740 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:12.741 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:12.741 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:12.741 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:12.741 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:12.741 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:12.741 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 1430 2024-09-04T17:46:12.741 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.741 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:12.741 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.741 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:12.741 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.778704+0000", 2024-09-04T17:46:12.741 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.742 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.742 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.742 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:12.742 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.778704+0000", 2024-09-04T17:46:12.742 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.742 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.742 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.742 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:12.742 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.778707+0000", 2024-09-04T17:46:12.742 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.8329999999999998e-06 2024-09-04T17:46:12.742 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.742 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.742 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:12.743 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.778718+0000", 2024-09-04T17:46:12.743 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.152e-05 2024-09-04T17:46:12.743 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.743 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.743 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:12.743 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.778750+0000", 2024-09-04T17:46:12.743 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.1696999999999998e-05 2024-09-04T17:46:12.743 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.743 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.743 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:12.743 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.778765+0000", 2024-09-04T17:46:12.743 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4800000000000001e-05 2024-09-04T17:46:12.743 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.744 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.744 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:12.744 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.778894+0000", 2024-09-04T17:46:12.744 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00012928500000000001 2024-09-04T17:46:12.744 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.744 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.744 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:12.744 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.779030+0000", 2024-09-04T17:46:12.744 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013566899999999999 2024-09-04T17:46:12.744 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.744 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.744 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 4", 2024-09-04T17:46:12.744 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.788794+0000", 2024-09-04T17:46:12.745 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.009764478 2024-09-04T17:46:12.745 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.745 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.745 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:12.745 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.836969+0000", 2024-09-04T17:46:12.745 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.048174626999999998 2024-09-04T17:46:12.745 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.745 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.745 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:12.745 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.837090+0000", 2024-09-04T17:46:12.745 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000120627 2024-09-04T17:46:12.745 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.745 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.746 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:12.746 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.837195+0000", 2024-09-04T17:46:12.746 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000105206 2024-09-04T17:46:12.746 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.746 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.746 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:12.746 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.837272+0000", 2024-09-04T17:46:12.746 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.7058999999999995e-05 2024-09-04T17:46:12.746 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.746 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:12.746 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.746 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.746 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.747 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4397 3.f 3:f7aed845:::rbd_data.1134cfbce83d.0000000000000007:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:12.747 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.341682+0000", 2024-09-04T17:46:12.747 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 5.3776130870000003, 2024-09-04T17:46:12.747 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.03043676, 2024-09-04T17:46:12.747 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:12.747 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:12.747 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:12.747 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:12.747 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:12.747 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:12.748 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4397 2024-09-04T17:46:12.748 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.748 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:12.748 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.748 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:12.748 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.341682+0000", 2024-09-04T17:46:12.748 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.748 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.748 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.748 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:12.748 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.341682+0000", 2024-09-04T17:46:12.749 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.749 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.749 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.749 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:12.749 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.341686+0000", 2024-09-04T17:46:12.749 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7799999999999998e-06 2024-09-04T17:46:12.749 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.749 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.749 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:12.749 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.341698+0000", 2024-09-04T17:46:12.749 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2478e-05 2024-09-04T17:46:12.749 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.750 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.750 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:12.750 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.341729+0000", 2024-09-04T17:46:12.750 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0666000000000001e-05 2024-09-04T17:46:12.750 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.750 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.750 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:12.750 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.341745+0000", 2024-09-04T17:46:12.750 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6490999999999998e-05 2024-09-04T17:46:12.750 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.750 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.750 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:12.750 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.342209+0000", 2024-09-04T17:46:12.750 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000463559 2024-09-04T17:46:12.751 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.751 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.751 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op must wait for map", 2024-09-04T17:46:12.751 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.342218+0000", 2024-09-04T17:46:12.751 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.3880000000000006e-06 2024-09-04T17:46:12.751 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.751 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.751 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:12.751 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.351602+0000", 2024-09-04T17:46:12.751 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0093833330000000006 2024-09-04T17:46:12.751 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.751 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.752 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:12.752 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.351774+0000", 2024-09-04T17:46:12.752 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00017225300000000001 2024-09-04T17:46:12.752 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.752 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.752 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:12.752 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.372119+0000", 2024-09-04T17:46:12.752 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.020344812 2024-09-04T17:46:12.752 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.752 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:12.752 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.752 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.752 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.753 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4848 3.15 3:ab4caf76:::rbd_data.1134cfbce83d.000000000000000a:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:12.753 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.365766+0000", 2024-09-04T17:46:12.753 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 5.3535288980000004, 2024-09-04T17:46:12.753 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.020025887999999999, 2024-09-04T17:46:12.753 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:12.753 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:12.753 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:12.753 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:12.753 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:12.753 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:12.753 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4848 2024-09-04T17:46:12.753 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.754 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:12.754 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.754 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:12.754 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.365766+0000", 2024-09-04T17:46:12.754 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.754 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.754 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.754 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:12.754 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.365766+0000", 2024-09-04T17:46:12.754 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.754 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.754 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.754 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:12.755 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.365767+0000", 2024-09-04T17:46:12.755 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.1099999999999995e-07 2024-09-04T17:46:12.755 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.755 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.755 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:12.755 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.365770+0000", 2024-09-04T17:46:12.755 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.9069999999999999e-06 2024-09-04T17:46:12.755 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.755 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.755 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:12.755 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.365779+0000", 2024-09-04T17:46:12.755 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4110000000000002e-06 2024-09-04T17:46:12.756 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.756 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.756 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:12.756 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.365786+0000", 2024-09-04T17:46:12.756 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2259999999999999e-06 2024-09-04T17:46:12.756 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.756 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.756 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:12.756 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.381324+0000", 2024-09-04T17:46:12.756 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.015537611999999999 2024-09-04T17:46:12.756 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.756 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.756 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:12.757 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.381459+0000", 2024-09-04T17:46:12.757 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013517100000000001 2024-09-04T17:46:12.757 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.757 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.757 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:12.757 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.385792+0000", 2024-09-04T17:46:12.757 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0043329500000000003 2024-09-04T17:46:12.757 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.757 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:12.758 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.758 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.758 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.758 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:5015 3.f 3:f7aed845:::rbd_data.1134cfbce83d.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:12.758 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.412372+0000", 2024-09-04T17:46:12.758 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 5.306922653, 2024-09-04T17:46:12.758 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.021470176000000001, 2024-09-04T17:46:12.758 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:12.758 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:12.758 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:12.758 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:12.758 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:12.759 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:12.759 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 5015 2024-09-04T17:46:12.759 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.759 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:12.759 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.759 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:12.759 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.412372+0000", 2024-09-04T17:46:12.759 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.759 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.759 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.759 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:12.759 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.412372+0000", 2024-09-04T17:46:12.759 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.760 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.760 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.760 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:12.760 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.412374+0000", 2024-09-04T17:46:12.760 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.815e-06 2024-09-04T17:46:12.760 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.760 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.760 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:12.760 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.419455+0000", 2024-09-04T17:46:12.760 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.007080607 2024-09-04T17:46:12.760 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.760 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.760 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:12.761 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.419476+0000", 2024-09-04T17:46:12.761 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1314e-05 2024-09-04T17:46:12.761 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.761 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.761 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:12.761 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.419488+0000", 2024-09-04T17:46:12.761 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2283000000000001e-05 2024-09-04T17:46:12.761 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.761 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.761 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:12.761 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.419579+0000", 2024-09-04T17:46:12.761 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.0433000000000006e-05 2024-09-04T17:46:12.762 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.762 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.762 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:12.762 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.419718+0000", 2024-09-04T17:46:12.762 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000139247 2024-09-04T17:46:12.762 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.762 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.762 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 4", 2024-09-04T17:46:12.762 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.420183+0000", 2024-09-04T17:46:12.762 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00046540299999999998 2024-09-04T17:46:12.762 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.762 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.762 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:12.763 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.431643+0000", 2024-09-04T17:46:12.763 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.011459741000000001 2024-09-04T17:46:12.763 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.763 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.763 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:12.763 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.433742+0000", 2024-09-04T17:46:12.763 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.002098964 2024-09-04T17:46:12.763 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.763 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.763 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:12.763 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.433813+0000", 2024-09-04T17:46:12.763 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1147000000000002e-05 2024-09-04T17:46:12.764 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.764 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.764 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:12.764 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.433842+0000", 2024-09-04T17:46:12.764 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.9221999999999999e-05 2024-09-04T17:46:12.764 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.764 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:12.764 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.764 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.764 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.764 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:5386 3.f 3:f4098008:::rbd_data.113431586065.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 8912896~5917] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:12.764 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.724609+0000", 2024-09-04T17:46:12.765 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.9946855039999996, 2024-09-04T17:46:12.765 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.020913503, 2024-09-04T17:46:12.765 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:12.765 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:12.765 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:12.765 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:12.765 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:12.765 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:12.765 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 5386 2024-09-04T17:46:12.765 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.765 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:12.765 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.765 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:12.766 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.724609+0000", 2024-09-04T17:46:12.766 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.766 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.766 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.766 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:12.766 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.724609+0000", 2024-09-04T17:46:12.766 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.766 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.766 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.766 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:12.766 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.724613+0000", 2024-09-04T17:46:12.766 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.2830000000000002e-06 2024-09-04T17:46:12.766 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.766 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.767 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:12.767 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.731404+0000", 2024-09-04T17:46:12.767 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.006791315 2024-09-04T17:46:12.767 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.767 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.767 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:12.767 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.731439+0000", 2024-09-04T17:46:12.767 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4932999999999999e-05 2024-09-04T17:46:12.767 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.767 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.767 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:12.767 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.731456+0000", 2024-09-04T17:46:12.767 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7535999999999999e-05 2024-09-04T17:46:12.768 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.768 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.768 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:12.768 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.731565+0000", 2024-09-04T17:46:12.768 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000108583 2024-09-04T17:46:12.768 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.768 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.768 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:12.768 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.731790+0000", 2024-09-04T17:46:12.768 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000225212 2024-09-04T17:46:12.768 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.768 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.768 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 4", 2024-09-04T17:46:12.769 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.732421+0000", 2024-09-04T17:46:12.769 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00063093399999999999 2024-09-04T17:46:12.769 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.769 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.769 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:12.769 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.745027+0000", 2024-09-04T17:46:12.769 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.012606009 2024-09-04T17:46:12.769 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.769 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.769 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:12.769 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.745421+0000", 2024-09-04T17:46:12.769 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00039343199999999998 2024-09-04T17:46:12.770 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.770 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.770 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:12.770 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.745474+0000", 2024-09-04T17:46:12.770 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.3536999999999999e-05 2024-09-04T17:46:12.770 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.770 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.770 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:12.770 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.745523+0000", 2024-09-04T17:46:12.770 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.8729000000000001e-05 2024-09-04T17:46:12.770 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.770 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:12.770 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.770 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.771 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.771 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:5751 3.d 3:b1ae20f2:::rbd_data.113431586065.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:12.771 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.947425+0000", 2024-09-04T17:46:12.771 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.7718695479999997, 2024-09-04T17:46:12.771 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.037571597999999998, 2024-09-04T17:46:12.771 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:12.771 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:12.771 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:12.771 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:12.771 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:12.771 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:12.771 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 5751 2024-09-04T17:46:12.772 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.772 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:12.772 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.772 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:12.772 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.947425+0000", 2024-09-04T17:46:12.772 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.772 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.772 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.772 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:12.772 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.947425+0000", 2024-09-04T17:46:12.772 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.772 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.772 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.773 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:12.773 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.947428+0000", 2024-09-04T17:46:12.773 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.74e-06 2024-09-04T17:46:12.773 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.773 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.773 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:12.773 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.951927+0000", 2024-09-04T17:46:12.773 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0044988930000000003 2024-09-04T17:46:12.773 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.773 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.773 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:12.773 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.951963+0000", 2024-09-04T17:46:12.773 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.5902999999999998e-05 2024-09-04T17:46:12.774 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.774 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.774 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:12.774 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.951986+0000", 2024-09-04T17:46:12.774 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.266e-05 2024-09-04T17:46:12.774 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.774 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.774 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:12.774 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.952065+0000", 2024-09-04T17:46:12.774 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.9668999999999999e-05 2024-09-04T17:46:12.774 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.774 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.774 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:12.775 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.952166+0000", 2024-09-04T17:46:12.775 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000101078 2024-09-04T17:46:12.775 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.775 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.775 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 5", 2024-09-04T17:46:12.775 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.955910+0000", 2024-09-04T17:46:12.775 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.003743732 2024-09-04T17:46:12.775 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.775 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.775 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:12.775 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.983683+0000", 2024-09-04T17:46:12.775 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.027773036000000001 2024-09-04T17:46:12.775 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.775 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.776 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:12.776 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.984827+0000", 2024-09-04T17:46:12.776 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0011435449999999999 2024-09-04T17:46:12.776 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.776 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.776 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:12.776 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.984888+0000", 2024-09-04T17:46:12.776 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.122e-05 2024-09-04T17:46:12.776 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.776 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.776 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:12.776 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.984997+0000", 2024-09-04T17:46:12.776 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000109122 2024-09-04T17:46:12.777 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.777 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:12.777 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.777 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.777 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.777 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:6147 3.d 3:b692b231:::rbd_data.1134cfbce83d.000000000000000c:6 [sparse-read 1343488~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:12.777 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:08.430599+0000", 2024-09-04T17:46:12.777 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.2886963429999998, 2024-09-04T17:46:12.777 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.019355033000000001, 2024-09-04T17:46:12.777 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:12.777 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:12.777 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:12.777 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:12.778 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:12.778 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:12.778 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 6147 2024-09-04T17:46:12.778 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.778 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:12.778 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.778 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:12.778 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.430599+0000", 2024-09-04T17:46:12.778 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.779 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.779 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.779 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:12.779 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.430599+0000", 2024-09-04T17:46:12.779 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.779 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.779 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.779 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:12.779 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.430600+0000", 2024-09-04T17:46:12.779 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1999999999999999e-06 2024-09-04T17:46:12.779 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.779 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.779 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:12.780 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.430614+0000", 2024-09-04T17:46:12.780 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4353e-05 2024-09-04T17:46:12.780 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.780 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.780 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:12.780 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.430634+0000", 2024-09-04T17:46:12.780 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.961e-05 2024-09-04T17:46:12.780 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.780 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.780 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:12.780 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.430648+0000", 2024-09-04T17:46:12.780 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4073000000000001e-05 2024-09-04T17:46:12.780 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.781 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.781 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:12.781 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.449011+0000", 2024-09-04T17:46:12.781 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.018362929 2024-09-04T17:46:12.781 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.781 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.781 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:12.781 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.449215+0000", 2024-09-04T17:46:12.781 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020451099999999999 2024-09-04T17:46:12.781 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.781 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.781 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:12.782 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.449954+0000", 2024-09-04T17:46:12.782 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00073835700000000003 2024-09-04T17:46:12.782 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.782 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:12.782 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.782 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.782 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.782 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:6148 3.d 3:b692b231:::rbd_data.1134cfbce83d.000000000000000c:6 [sparse-read 1703936~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:12.782 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:08.430668+0000", 2024-09-04T17:46:12.782 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.2886273309999998, 2024-09-04T17:46:12.782 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.020109115, 2024-09-04T17:46:12.782 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:12.782 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:12.783 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:12.783 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:12.783 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:12.783 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:12.783 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 6148 2024-09-04T17:46:12.783 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.783 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:12.783 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.783 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:12.783 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.430668+0000", 2024-09-04T17:46:12.783 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.783 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.783 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.784 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:12.784 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.430668+0000", 2024-09-04T17:46:12.784 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.784 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.784 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.784 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:12.784 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.430669+0000", 2024-09-04T17:46:12.784 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.234e-06 2024-09-04T17:46:12.784 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.784 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.784 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:12.784 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.430675+0000", 2024-09-04T17:46:12.784 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7359999999999998e-06 2024-09-04T17:46:12.785 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.785 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.785 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:12.785 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.430694+0000", 2024-09-04T17:46:12.785 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9283000000000002e-05 2024-09-04T17:46:12.785 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.785 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.785 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:12.785 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.430708+0000", 2024-09-04T17:46:12.785 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4324e-05 2024-09-04T17:46:12.785 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.785 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.785 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:12.786 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.450182+0000", 2024-09-04T17:46:12.786 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.019473796000000002 2024-09-04T17:46:12.786 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.786 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.786 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:12.786 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.450292+0000", 2024-09-04T17:46:12.786 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000110302 2024-09-04T17:46:12.786 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.786 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.786 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:12.786 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.450777+0000", 2024-09-04T17:46:12.786 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00048443999999999999 2024-09-04T17:46:12.786 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.786 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:12.787 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.787 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.787 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.787 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:6239 3.15 3:ab4caf76:::rbd_data.1134cfbce83d.000000000000000a:6 [read 688128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:12.787 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:08.431477+0000", 2024-09-04T17:46:12.787 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.2878181440000001, 2024-09-04T17:46:12.787 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.019689993999999999, 2024-09-04T17:46:12.787 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:12.787 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:12.787 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:12.787 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:12.787 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:12.787 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:12.787 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 6239 2024-09-04T17:46:12.788 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.788 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:12.788 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.788 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:12.788 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.431477+0000", 2024-09-04T17:46:12.788 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.788 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.788 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.788 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:12.788 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.431477+0000", 2024-09-04T17:46:12.788 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.788 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.788 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.789 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:12.789 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.431478+0000", 2024-09-04T17:46:12.789 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.207e-06 2024-09-04T17:46:12.789 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.789 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.789 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:12.789 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.431483+0000", 2024-09-04T17:46:12.789 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2900000000000002e-06 2024-09-04T17:46:12.789 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.789 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.789 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:12.789 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.431502+0000", 2024-09-04T17:46:12.790 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8853000000000001e-05 2024-09-04T17:46:12.790 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.790 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.790 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:12.790 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.431516+0000", 2024-09-04T17:46:12.790 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.341e-05 2024-09-04T17:46:12.790 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.790 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.790 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:12.790 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.450643+0000", 2024-09-04T17:46:12.790 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.019127613000000002 2024-09-04T17:46:12.790 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.791 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.791 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:12.791 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.450721+0000", 2024-09-04T17:46:12.791 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.7966999999999994e-05 2024-09-04T17:46:12.791 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.791 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.791 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:12.791 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.451167+0000", 2024-09-04T17:46:12.791 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00044565400000000003 2024-09-04T17:46:12.791 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.791 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:12.791 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.791 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.791 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.792 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:6240 3.15 3:ab4caf76:::rbd_data.1134cfbce83d.000000000000000a:6 [read 1048576~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:12.792 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:08.431534+0000", 2024-09-04T17:46:12.792 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.2877608340000002, 2024-09-04T17:46:12.792 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.020184402000000001, 2024-09-04T17:46:12.792 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:12.792 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:12.792 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:12.792 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:12.792 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:12.792 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:12.792 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 6240 2024-09-04T17:46:12.792 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.792 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:12.793 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.793 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:12.793 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.431534+0000", 2024-09-04T17:46:12.793 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.793 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.793 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.793 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:12.793 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.431534+0000", 2024-09-04T17:46:12.793 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.793 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.793 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.793 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:12.793 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.431535+0000", 2024-09-04T17:46:12.794 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.15e-06 2024-09-04T17:46:12.794 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.794 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.794 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:12.794 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.431540+0000", 2024-09-04T17:46:12.794 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1699999999999996e-06 2024-09-04T17:46:12.794 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.794 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.794 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:12.794 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.431559+0000", 2024-09-04T17:46:12.794 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.84e-05 2024-09-04T17:46:12.794 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.794 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.795 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:12.795 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.431572+0000", 2024-09-04T17:46:12.795 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.359e-05 2024-09-04T17:46:12.795 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.795 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.795 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:12.795 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.451221+0000", 2024-09-04T17:46:12.795 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.019648697999999999 2024-09-04T17:46:12.795 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.795 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.795 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:12.795 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.451317+0000", 2024-09-04T17:46:12.795 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5444000000000003e-05 2024-09-04T17:46:12.795 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.796 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.796 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:12.796 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.451718+0000", 2024-09-04T17:46:12.796 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00040194999999999999 2024-09-04T17:46:12.796 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.796 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:12.796 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.796 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.796 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.796 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:6241 3.15 3:ab4caf76:::rbd_data.1134cfbce83d.000000000000000a:6 [read 1409024~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:12.796 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:08.431591+0000", 2024-09-04T17:46:12.796 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.287704358, 2024-09-04T17:46:12.797 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.020692640000000002, 2024-09-04T17:46:12.797 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:12.797 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:12.797 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:12.797 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:12.797 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:12.797 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:12.797 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 6241 2024-09-04T17:46:12.797 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.797 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:12.797 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.797 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:12.797 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.431591+0000", 2024-09-04T17:46:12.798 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.798 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.798 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.798 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:12.798 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.431591+0000", 2024-09-04T17:46:12.798 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.798 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.798 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.798 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:12.798 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.431592+0000", 2024-09-04T17:46:12.798 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1570000000000001e-06 2024-09-04T17:46:12.798 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.798 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.799 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:12.799 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.431597+0000", 2024-09-04T17:46:12.799 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.3770000000000004e-06 2024-09-04T17:46:12.799 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.799 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.799 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:12.799 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.431618+0000", 2024-09-04T17:46:12.799 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0945999999999999e-05 2024-09-04T17:46:12.799 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.799 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.799 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:12.799 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.431632+0000", 2024-09-04T17:46:12.799 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3787e-05 2024-09-04T17:46:12.800 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.800 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.800 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:12.800 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.451782+0000", 2024-09-04T17:46:12.800 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.020150314999999999 2024-09-04T17:46:12.800 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.800 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.800 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:12.800 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.451884+0000", 2024-09-04T17:46:12.800 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000101821 2024-09-04T17:46:12.800 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.800 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.800 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:12.801 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.452283+0000", 2024-09-04T17:46:12.801 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00039923699999999999 2024-09-04T17:46:12.801 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.801 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:12.801 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.801 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.801 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.801 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:6242 3.15 3:ab4caf76:::rbd_data.1134cfbce83d.000000000000000a:6 [read 1769472~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:12.801 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:08.431650+0000", 2024-09-04T17:46:12.801 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.2876449909999996, 2024-09-04T17:46:12.801 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.021284912999999999, 2024-09-04T17:46:12.801 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:12.801 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:12.802 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:12.802 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:12.802 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:12.802 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:12.802 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 6242 2024-09-04T17:46:12.802 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.802 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:12.802 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.802 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:12.802 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.431650+0000", 2024-09-04T17:46:12.802 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.802 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.802 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.803 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:12.803 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.431650+0000", 2024-09-04T17:46:12.803 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.803 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.803 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.803 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:12.803 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.431651+0000", 2024-09-04T17:46:12.803 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.26e-06 2024-09-04T17:46:12.803 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.803 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.803 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:12.803 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.431657+0000", 2024-09-04T17:46:12.803 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.5999999999999997e-06 2024-09-04T17:46:12.804 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.804 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.804 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:12.804 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.434576+0000", 2024-09-04T17:46:12.804 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.002918861 2024-09-04T17:46:12.804 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.804 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.804 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:12.804 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.434588+0000", 2024-09-04T17:46:12.804 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2121e-05 2024-09-04T17:46:12.804 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.804 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.804 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:12.805 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.452339+0000", 2024-09-04T17:46:12.805 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.017751169000000001 2024-09-04T17:46:12.805 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.805 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.805 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:12.805 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.452476+0000", 2024-09-04T17:46:12.805 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000137436 2024-09-04T17:46:12.805 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.805 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.805 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:12.805 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.452935+0000", 2024-09-04T17:46:12.805 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00045846599999999998 2024-09-04T17:46:12.806 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.806 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:12.806 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.806 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.806 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.806 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:6250 3.d 3:b692b231:::rbd_data.1134cfbce83d.000000000000000c:6 [read 1048576~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:12.806 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:08.434667+0000", 2024-09-04T17:46:12.806 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.284628391, 2024-09-04T17:46:12.806 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.01935344, 2024-09-04T17:46:12.806 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:12.806 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:12.806 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:12.806 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:12.807 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:12.807 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:12.807 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 6250 2024-09-04T17:46:12.807 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.807 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:12.807 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.807 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:12.807 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.434667+0000", 2024-09-04T17:46:12.807 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.807 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.807 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.807 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:12.807 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.434667+0000", 2024-09-04T17:46:12.808 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.808 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.808 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.808 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:12.808 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.434667+0000", 2024-09-04T17:46:12.808 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2399999999999998e-07 2024-09-04T17:46:12.808 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.808 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.808 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:12.808 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.434669+0000", 2024-09-04T17:46:12.808 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.367e-06 2024-09-04T17:46:12.808 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.809 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.809 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:12.809 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.434676+0000", 2024-09-04T17:46:12.809 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7429999999999996e-06 2024-09-04T17:46:12.809 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.809 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.809 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:12.809 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.434682+0000", 2024-09-04T17:46:12.809 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4199999999999998e-06 2024-09-04T17:46:12.809 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.809 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.809 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:12.809 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.453639+0000", 2024-09-04T17:46:12.809 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.018957206000000001 2024-09-04T17:46:12.810 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.810 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.810 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:12.810 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.453709+0000", 2024-09-04T17:46:12.810 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.0576999999999999e-05 2024-09-04T17:46:12.810 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.810 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.810 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:12.810 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.454020+0000", 2024-09-04T17:46:12.810 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00031060300000000001 2024-09-04T17:46:12.810 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.810 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:12.810 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.811 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.811 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.811 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:6251 3.d 3:b692b231:::rbd_data.1134cfbce83d.000000000000000c:6 [read 1409024~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:12.811 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:08.434695+0000", 2024-09-04T17:46:12.811 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.284600202, 2024-09-04T17:46:12.811 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.019817794999999999, 2024-09-04T17:46:12.811 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:12.811 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:12.811 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:12.811 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:12.811 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:12.811 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:12.812 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 6251 2024-09-04T17:46:12.812 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.812 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:12.812 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.812 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:12.812 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.434695+0000", 2024-09-04T17:46:12.812 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.812 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.812 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.812 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:12.812 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.434695+0000", 2024-09-04T17:46:12.812 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.812 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.813 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.813 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:12.813 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.434695+0000", 2024-09-04T17:46:12.813 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2200000000000004e-07 2024-09-04T17:46:12.813 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.813 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.813 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:12.813 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.434698+0000", 2024-09-04T17:46:12.813 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.988e-06 2024-09-04T17:46:12.813 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.813 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.813 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:12.813 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.434705+0000", 2024-09-04T17:46:12.814 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.8510000000000001e-06 2024-09-04T17:46:12.814 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.814 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.814 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:12.814 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.434710+0000", 2024-09-04T17:46:12.814 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.3410000000000002e-06 2024-09-04T17:46:12.814 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.814 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.814 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:12.814 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.454071+0000", 2024-09-04T17:46:12.814 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.019360863999999998 2024-09-04T17:46:12.814 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.814 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.815 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:12.815 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.454161+0000", 2024-09-04T17:46:12.815 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9964999999999997e-05 2024-09-04T17:46:12.815 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.815 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.815 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:12.815 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.454512+0000", 2024-09-04T17:46:12.815 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000351264 2024-09-04T17:46:12.815 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.815 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:12.815 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.815 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.816 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.816 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:6252 3.d 3:b692b231:::rbd_data.1134cfbce83d.000000000000000c:6 [read 1769472~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:12.816 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:08.434718+0000", 2024-09-04T17:46:12.816 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.2845767050000001, 2024-09-04T17:46:12.816 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.020239798, 2024-09-04T17:46:12.816 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:12.816 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:12.816 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:12.816 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:12.816 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:12.816 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:12.816 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 6252 2024-09-04T17:46:12.816 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.817 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:12.817 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.817 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:12.817 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.434718+0000", 2024-09-04T17:46:12.817 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.817 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.817 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.817 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:12.817 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.434718+0000", 2024-09-04T17:46:12.817 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.817 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.817 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.817 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:12.818 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.434719+0000", 2024-09-04T17:46:12.818 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0100000000000005e-07 2024-09-04T17:46:12.818 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.818 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.818 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:12.818 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.434721+0000", 2024-09-04T17:46:12.818 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6240000000000002e-06 2024-09-04T17:46:12.818 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.818 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.818 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:12.818 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.434736+0000", 2024-09-04T17:46:12.818 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4302e-05 2024-09-04T17:46:12.818 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.819 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.819 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:12.819 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.434741+0000", 2024-09-04T17:46:12.819 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.3070000000000002e-06 2024-09-04T17:46:12.819 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.819 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.819 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:12.819 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.454554+0000", 2024-09-04T17:46:12.819 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.019813239 2024-09-04T17:46:12.819 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.819 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.819 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:12.820 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.454610+0000", 2024-09-04T17:46:12.820 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6289999999999998e-05 2024-09-04T17:46:12.820 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.820 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.820 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:12.820 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.454958+0000", 2024-09-04T17:46:12.820 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000347535 2024-09-04T17:46:12.820 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.820 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:12.820 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.820 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.820 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.820 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:6295 3.d 3:b1ae20f2:::rbd_data.113431586065.0000000000000003:head [stat,truncate 1572864] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:12.821 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:08.434914+0000", 2024-09-04T17:46:12.821 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.2843806249999998, 2024-09-04T17:46:12.821 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.025484347000000001, 2024-09-04T17:46:12.821 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:12.821 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:12.821 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:12.821 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:12.821 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:12.821 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:12.821 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 6295 2024-09-04T17:46:12.821 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.821 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:12.821 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.822 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:12.822 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.434914+0000", 2024-09-04T17:46:12.822 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.822 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.822 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.822 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:12.822 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.434914+0000", 2024-09-04T17:46:12.822 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.822 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.822 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.822 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:12.822 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.434915+0000", 2024-09-04T17:46:12.822 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.8799999999999998e-07 2024-09-04T17:46:12.823 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.823 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.823 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:12.823 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.434917+0000", 2024-09-04T17:46:12.823 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2340000000000001e-06 2024-09-04T17:46:12.823 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.823 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.823 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:12.823 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.434923+0000", 2024-09-04T17:46:12.823 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.285e-06 2024-09-04T17:46:12.823 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.823 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.824 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:12.824 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.434930+0000", 2024-09-04T17:46:12.824 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.5359999999999998e-06 2024-09-04T17:46:12.824 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.824 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.824 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:12.824 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.454993+0000", 2024-09-04T17:46:12.824 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.020062910999999999 2024-09-04T17:46:12.824 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.824 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.824 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:12.824 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.455079+0000", 2024-09-04T17:46:12.824 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.6300999999999999e-05 2024-09-04T17:46:12.825 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.825 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.825 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 5", 2024-09-04T17:46:12.825 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.455263+0000", 2024-09-04T17:46:12.825 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00018429300000000001 2024-09-04T17:46:12.825 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.825 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.825 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:12.825 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.459589+0000", 2024-09-04T17:46:12.825 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0043253520000000002 2024-09-04T17:46:12.825 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.825 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.825 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:12.826 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.460262+0000", 2024-09-04T17:46:12.826 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00067367100000000004 2024-09-04T17:46:12.826 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.826 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.826 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:12.826 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.460334+0000", 2024-09-04T17:46:12.826 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1413000000000002e-05 2024-09-04T17:46:12.826 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.826 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.826 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:12.826 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.460399+0000", 2024-09-04T17:46:12.826 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.4962999999999994e-05 2024-09-04T17:46:12.826 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.827 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:12.827 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.827 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.827 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.827 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:6301 3.f 3:f3ba70c3:::rbd_data.113431586065.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:12.827 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:08.450661+0000", 2024-09-04T17:46:12.827 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.2686338810000004, 2024-09-04T17:46:12.827 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.021587037999999999, 2024-09-04T17:46:12.827 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:12.827 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:12.827 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:12.827 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:12.827 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:12.828 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:12.828 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 6301 2024-09-04T17:46:12.828 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.828 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:12.828 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.828 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:12.828 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.450661+0000", 2024-09-04T17:46:12.828 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.828 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.828 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.828 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:12.828 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.450661+0000", 2024-09-04T17:46:12.828 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:12.829 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.829 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.829 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:12.829 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.450663+0000", 2024-09-04T17:46:12.829 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5430000000000001e-06 2024-09-04T17:46:12.829 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.829 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.829 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:12.829 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.455813+0000", 2024-09-04T17:46:12.829 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0051502010000000001 2024-09-04T17:46:12.829 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.829 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.829 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:12.829 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.455828+0000", 2024-09-04T17:46:12.830 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5400000000000002e-05 2024-09-04T17:46:12.830 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.830 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.830 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:12.830 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.455837+0000", 2024-09-04T17:46:12.830 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8100000000000004e-06 2024-09-04T17:46:12.830 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.830 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.830 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:12.830 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.455956+0000", 2024-09-04T17:46:12.830 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00011917 2024-09-04T17:46:12.830 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.830 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.831 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:12.831 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.456107+0000", 2024-09-04T17:46:12.831 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00015067699999999999 2024-09-04T17:46:12.831 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.831 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.831 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 4", 2024-09-04T17:46:12.831 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.456580+0000", 2024-09-04T17:46:12.831 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000473444 2024-09-04T17:46:12.831 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.831 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.831 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:12.831 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.469300+0000", 2024-09-04T17:46:12.831 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.012719729000000001 2024-09-04T17:46:12.832 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.832 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.832 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:12.832 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.472163+0000", 2024-09-04T17:46:12.832 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.002863258 2024-09-04T17:46:12.832 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.832 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.832 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:12.832 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.472203+0000", 2024-09-04T17:46:12.832 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.9434999999999998e-05 2024-09-04T17:46:12.832 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:12.832 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:12.832 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:12.832 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.472248+0000", 2024-09-04T17:46:12.833 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.5371000000000003e-05 2024-09-04T17:46:12.833 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.833 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:12.833 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.833 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:12.833 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:12.833 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:12.834 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-04T17:46:12.835 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:12.834+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:12.936 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:12.936+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:12.962 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:12.962 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:46:12.962 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:46:12.962 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:12.981 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-04T17:46:13.036 INFO:teuthology.orchestra.run.smithi092.stdout:875 trunc from 0xe3ff222 to 0x9ba81a1 2024-09-04T17:46:13.036 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:13.034+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:13.101 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:13.101 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:46:13.101 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:46:13.101 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:46:13.102 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:13.116 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-04T17:46:13.137 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:13.136+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:13.237 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:13.236+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:13.250 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is unset 2024-09-04T17:46:13.259 INFO:teuthology.orchestra.run.smithi092.stdout:moved 9 / 82 (10.9756%) 2024-09-04T17:46:13.259 INFO:teuthology.orchestra.run.smithi092.stdout:avg 11.7143 2024-09-04T17:46:13.259 INFO:teuthology.orchestra.run.smithi092.stdout:stddev 3.81993 -> 2.952 (expected baseline 3.16872) 2024-09-04T17:46:13.259 INFO:teuthology.orchestra.run.smithi092.stdout:min osd.2 with 5 -> 7 pgs (0.426829 -> 0.597561 * mean) 2024-09-04T17:46:13.259 INFO:teuthology.orchestra.run.smithi092.stdout:max osd.7 with 14 -> 12 pgs (1.19512 -> 1.02439 * mean) 2024-09-04T17:46:13.259 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:46:13.259 INFO:teuthology.orchestra.run.smithi092.stdout:oload 110 2024-09-04T17:46:13.259 INFO:teuthology.orchestra.run.smithi092.stdout:max_change 1 2024-09-04T17:46:13.260 INFO:teuthology.orchestra.run.smithi092.stdout:max_change_osds 4 2024-09-04T17:46:13.260 INFO:teuthology.orchestra.run.smithi092.stdout:average_utilization 0.0005 2024-09-04T17:46:13.260 INFO:teuthology.orchestra.run.smithi092.stdout:overload_utilization 0.0005 2024-09-04T17:46:13.260 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4 weight 1.0000 -> 0.7470 2024-09-04T17:46:13.260 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7 weight 1.0000 -> 0.7661 2024-09-04T17:46:13.260 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5 weight 1.0000 -> 0.8015 2024-09-04T17:46:13.260 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0 weight 1.0000 -> 0.9082 2024-09-04T17:46:13.282 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-09-04T17:46:13.283 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 1.0 2024-09-04T17:46:13.284 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-04T17:46:13.292 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:13.292 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:46:13.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:46:13.292 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:46:13.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.293 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:1095 3.1 3:86c08994:::rbd_data.11346251f094.000000000000006e:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-04T17:46:13.293 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:02.246406+0000", 2024-09-04T17:46:13.293 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 11.047978788, 2024-09-04T17:46:13.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.024871196000000002, 2024-09-04T17:46:13.293 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:13.293 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:13.293 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:13.293 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:13.293 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:13.293 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:13.293 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 1095 2024-09-04T17:46:13.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.294 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:13.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:13.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.246406+0000", 2024-09-04T17:46:13.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:13.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:13.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.246406+0000", 2024-09-04T17:46:13.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:13.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:13.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.246409+0000", 2024-09-04T17:46:13.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6599999999999999e-06 2024-09-04T17:46:13.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:13.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.246419+0000", 2024-09-04T17:46:13.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8840000000000006e-06 2024-09-04T17:46:13.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:13.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.246439+0000", 2024-09-04T17:46:13.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0088000000000001e-05 2024-09-04T17:46:13.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.296 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:13.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.246448+0000", 2024-09-04T17:46:13.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9339999999999995e-06 2024-09-04T17:46:13.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.296 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:13.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.246511+0000", 2024-09-04T17:46:13.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3727999999999997e-05 2024-09-04T17:46:13.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:13.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.246588+0000", 2024-09-04T17:46:13.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6143999999999995e-05 2024-09-04T17:46:13.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-04T17:46:13.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.250919+0000", 2024-09-04T17:46:13.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0043314529999999999 2024-09-04T17:46:13.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:13.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.271170+0000", 2024-09-04T17:46:13.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020251531999999999 2024-09-04T17:46:13.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:46:13.298 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.271186+0000", 2024-09-04T17:46:13.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5302e-05 2024-09-04T17:46:13.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:13.298 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.271230+0000", 2024-09-04T17:46:13.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4576000000000001e-05 2024-09-04T17:46:13.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:13.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.271277+0000", 2024-09-04T17:46:13.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6894999999999997e-05 2024-09-04T17:46:13.299 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:13.299 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:13.299 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:13.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.299 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6103 3.1f 3:fec7e474:::rbd_data.1134cfbce83d.0000000000000003:6 [read 1736704~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:13.299 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.421848+0000", 2024-09-04T17:46:13.299 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.8725366909999996, 2024-09-04T17:46:13.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027313884, 2024-09-04T17:46:13.299 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:13.299 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:13.300 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:13.300 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:13.300 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:13.300 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:13.300 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6103 2024-09-04T17:46:13.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.300 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:13.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.300 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:13.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.421848+0000", 2024-09-04T17:46:13.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:13.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:13.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.421848+0000", 2024-09-04T17:46:13.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:13.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:13.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.421849+0000", 2024-09-04T17:46:13.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.144e-06 2024-09-04T17:46:13.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:13.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.421855+0000", 2024-09-04T17:46:13.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.003e-06 2024-09-04T17:46:13.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:13.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.421872+0000", 2024-09-04T17:46:13.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7193e-05 2024-09-04T17:46:13.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:13.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.421886+0000", 2024-09-04T17:46:13.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.331e-05 2024-09-04T17:46:13.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:13.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.444849+0000", 2024-09-04T17:46:13.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.022962755000000001 2024-09-04T17:46:13.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.303 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:13.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.444990+0000", 2024-09-04T17:46:13.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014129999999999999 2024-09-04T17:46:13.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.303 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:13.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.449162+0000", 2024-09-04T17:46:13.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041721789999999998 2024-09-04T17:46:13.303 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:13.303 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:13.304 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:13.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.304 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6140 3.17 3:ed419000:::rbd_data.1134cfbce83d.000000000000000b:6 [sparse-read 622592~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:13.304 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.422576+0000", 2024-09-04T17:46:13.304 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.8718091030000004, 2024-09-04T17:46:13.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025138273999999999, 2024-09-04T17:46:13.304 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:13.304 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:13.304 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:13.304 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:13.304 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:13.304 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:13.305 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6140 2024-09-04T17:46:13.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.305 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:13.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:13.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422576+0000", 2024-09-04T17:46:13.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:13.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:13.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422576+0000", 2024-09-04T17:46:13.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:13.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:13.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422578+0000", 2024-09-04T17:46:13.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8870000000000001e-06 2024-09-04T17:46:13.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:13.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422587+0000", 2024-09-04T17:46:13.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5999999999999996e-06 2024-09-04T17:46:13.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:13.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422617+0000", 2024-09-04T17:46:13.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9339000000000001e-05 2024-09-04T17:46:13.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:13.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422636+0000", 2024-09-04T17:46:13.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9524000000000001e-05 2024-09-04T17:46:13.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:13.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.445586+0000", 2024-09-04T17:46:13.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.022950084999999999 2024-09-04T17:46:13.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.308 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:13.308 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.445788+0000", 2024-09-04T17:46:13.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000202029 2024-09-04T17:46:13.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.308 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:13.308 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.447714+0000", 2024-09-04T17:46:13.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00192581 2024-09-04T17:46:13.308 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:13.308 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:13.308 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:13.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.309 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6141 3.17 3:ed419000:::rbd_data.1134cfbce83d.000000000000000b:6 [sparse-read 983040~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:13.309 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.422660+0000", 2024-09-04T17:46:13.309 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.8717244400000004, 2024-09-04T17:46:13.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025710566000000001, 2024-09-04T17:46:13.309 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:13.309 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:13.309 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:13.309 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:13.309 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:13.309 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:13.309 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6141 2024-09-04T17:46:13.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.309 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:13.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:13.310 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422660+0000", 2024-09-04T17:46:13.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:13.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:13.310 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422660+0000", 2024-09-04T17:46:13.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:13.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:13.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422662+0000", 2024-09-04T17:46:13.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.313e-06 2024-09-04T17:46:13.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:13.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422691+0000", 2024-09-04T17:46:13.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9207e-05 2024-09-04T17:46:13.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:13.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422711+0000", 2024-09-04T17:46:13.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0225999999999999e-05 2024-09-04T17:46:13.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.312 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:13.312 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422727+0000", 2024-09-04T17:46:13.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5982999999999999e-05 2024-09-04T17:46:13.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.312 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:13.312 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.447767+0000", 2024-09-04T17:46:13.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025039558 2024-09-04T17:46:13.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.312 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:13.312 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.447871+0000", 2024-09-04T17:46:13.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104287 2024-09-04T17:46:13.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:13.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.448371+0000", 2024-09-04T17:46:13.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00049999199999999999 2024-09-04T17:46:13.313 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:13.313 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:13.313 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:13.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.313 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6142 3.17 3:ed419000:::rbd_data.1134cfbce83d.000000000000000b:6 [sparse-read 1343488~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:13.313 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.422750+0000", 2024-09-04T17:46:13.314 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.8716352079999998, 2024-09-04T17:46:13.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026566215000000001, 2024-09-04T17:46:13.314 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:13.314 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:13.314 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:13.314 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:13.314 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:13.314 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:13.314 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6142 2024-09-04T17:46:13.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.314 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:13.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.314 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:13.315 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422750+0000", 2024-09-04T17:46:13.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:13.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.315 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:13.315 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422750+0000", 2024-09-04T17:46:13.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:13.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.315 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:13.315 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422751+0000", 2024-09-04T17:46:13.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6700000000000001e-06 2024-09-04T17:46:13.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.315 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:13.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422757+0000", 2024-09-04T17:46:13.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9669999999999998e-06 2024-09-04T17:46:13.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:13.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422775+0000", 2024-09-04T17:46:13.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8073000000000001e-05 2024-09-04T17:46:13.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:13.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422789+0000", 2024-09-04T17:46:13.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3709999999999999e-05 2024-09-04T17:46:13.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:13.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.448420+0000", 2024-09-04T17:46:13.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025630627999999999 2024-09-04T17:46:13.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:13.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.448590+0000", 2024-09-04T17:46:13.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000170213 2024-09-04T17:46:13.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:13.318 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.449316+0000", 2024-09-04T17:46:13.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00072595400000000001 2024-09-04T17:46:13.318 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:13.318 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:13.318 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:13.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.318 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6143 3.17 3:ed419000:::rbd_data.1134cfbce83d.000000000000000b:6 [read 1736704~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:13.318 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.422807+0000", 2024-09-04T17:46:13.318 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.8715778780000001, 2024-09-04T17:46:13.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027271169000000001, 2024-09-04T17:46:13.318 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:13.318 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:13.319 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:13.319 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:13.319 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:13.319 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:13.319 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6143 2024-09-04T17:46:13.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.319 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:13.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:13.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422807+0000", 2024-09-04T17:46:13.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:13.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:13.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422807+0000", 2024-09-04T17:46:13.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:13.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:13.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422809+0000", 2024-09-04T17:46:13.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6139999999999999e-06 2024-09-04T17:46:13.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:13.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422814+0000", 2024-09-04T17:46:13.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1800000000000004e-06 2024-09-04T17:46:13.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:13.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422831+0000", 2024-09-04T17:46:13.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7433000000000001e-05 2024-09-04T17:46:13.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:13.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422844+0000", 2024-09-04T17:46:13.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3216e-05 2024-09-04T17:46:13.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:13.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.449397+0000", 2024-09-04T17:46:13.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026552572 2024-09-04T17:46:13.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:13.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.449526+0000", 2024-09-04T17:46:13.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000129207 2024-09-04T17:46:13.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:13.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.450078+0000", 2024-09-04T17:46:13.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00055194700000000001 2024-09-04T17:46:13.322 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:13.322 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:13.322 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:13.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.323 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6203 3.1f 3:fec7e474:::rbd_data.1134cfbce83d.0000000000000003:6 [read 327680~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:13.323 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423445+0000", 2024-09-04T17:46:13.323 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.8709394660000003, 2024-09-04T17:46:13.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027179207, 2024-09-04T17:46:13.323 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:13.323 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:13.323 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:13.323 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:13.323 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:13.323 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:13.323 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6203 2024-09-04T17:46:13.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.324 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:13.324 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.324 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:13.324 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423445+0000", 2024-09-04T17:46:13.324 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:13.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.324 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.324 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:13.324 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423445+0000", 2024-09-04T17:46:13.324 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:13.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.324 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.324 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:13.324 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423446+0000", 2024-09-04T17:46:13.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.077e-06 2024-09-04T17:46:13.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:13.325 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423452+0000", 2024-09-04T17:46:13.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.113e-06 2024-09-04T17:46:13.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:13.325 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423468+0000", 2024-09-04T17:46:13.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6685999999999999e-05 2024-09-04T17:46:13.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:13.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423481+0000", 2024-09-04T17:46:13.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2673999999999999e-05 2024-09-04T17:46:13.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:13.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.449779+0000", 2024-09-04T17:46:13.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026297838 2024-09-04T17:46:13.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:13.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.450020+0000", 2024-09-04T17:46:13.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024114399999999999 2024-09-04T17:46:13.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:13.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.450625+0000", 2024-09-04T17:46:13.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00060467500000000005 2024-09-04T17:46:13.327 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:13.327 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:13.327 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:13.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.327 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6204 3.1f 3:fec7e474:::rbd_data.1134cfbce83d.0000000000000003:6 [read 688128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:13.327 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423498+0000", 2024-09-04T17:46:13.327 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.8708865299999999, 2024-09-04T17:46:13.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029262579, 2024-09-04T17:46:13.327 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:13.328 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:13.328 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:13.328 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:13.328 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:13.328 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:13.328 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6204 2024-09-04T17:46:13.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.328 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:13.328 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.328 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:13.328 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423498+0000", 2024-09-04T17:46:13.328 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:13.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.329 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:13.329 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423498+0000", 2024-09-04T17:46:13.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:13.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.329 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:13.329 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423500+0000", 2024-09-04T17:46:13.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.28e-06 2024-09-04T17:46:13.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.329 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:13.330 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423506+0000", 2024-09-04T17:46:13.330 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0059999999999998e-06 2024-09-04T17:46:13.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.330 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:13.330 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423524+0000", 2024-09-04T17:46:13.330 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8683999999999999e-05 2024-09-04T17:46:13.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.330 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:13.330 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423537+0000", 2024-09-04T17:46:13.330 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3030000000000001e-05 2024-09-04T17:46:13.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.331 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:13.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.450697+0000", 2024-09-04T17:46:13.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027159793000000002 2024-09-04T17:46:13.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.331 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:13.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.450835+0000", 2024-09-04T17:46:13.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013773799999999999 2024-09-04T17:46:13.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.331 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:13.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.452761+0000", 2024-09-04T17:46:13.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019260480000000001 2024-09-04T17:46:13.331 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:13.331 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:13.332 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:13.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.332 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.332 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6205 3.1f 3:fec7e474:::rbd_data.1134cfbce83d.0000000000000003:6 [read 1048576~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:13.332 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423555+0000", 2024-09-04T17:46:13.332 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.8708295469999996, 2024-09-04T17:46:13.332 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029905979999999999, 2024-09-04T17:46:13.332 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:13.332 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:13.332 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:13.332 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:13.332 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:13.332 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:13.333 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6205 2024-09-04T17:46:13.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.333 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:13.333 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.333 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:13.333 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423555+0000", 2024-09-04T17:46:13.333 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:13.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.333 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.333 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:13.333 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423555+0000", 2024-09-04T17:46:13.333 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:13.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.334 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:13.334 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423556+0000", 2024-09-04T17:46:13.334 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1599999999999999e-06 2024-09-04T17:46:13.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.334 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:13.334 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423562+0000", 2024-09-04T17:46:13.334 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1270000000000002e-06 2024-09-04T17:46:13.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.334 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:13.334 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423579+0000", 2024-09-04T17:46:13.335 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6986e-05 2024-09-04T17:46:13.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.335 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:13.335 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423591+0000", 2024-09-04T17:46:13.335 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2544000000000001e-05 2024-09-04T17:46:13.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.335 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:13.335 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.452819+0000", 2024-09-04T17:46:13.335 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029227418000000002 2024-09-04T17:46:13.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.336 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:13.336 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.452985+0000", 2024-09-04T17:46:13.336 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000166675 2024-09-04T17:46:13.336 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.336 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.336 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:13.336 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.453461+0000", 2024-09-04T17:46:13.336 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00047606999999999999 2024-09-04T17:46:13.336 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:13.336 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:13.336 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:13.336 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.336 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.337 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6206 3.1f 3:fec7e474:::rbd_data.1134cfbce83d.0000000000000003:6 [sparse-read 1409024~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:13.337 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423608+0000", 2024-09-04T17:46:13.337 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.8707769140000003, 2024-09-04T17:46:13.337 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.031860753999999998, 2024-09-04T17:46:13.337 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:13.337 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:13.337 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:13.337 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:13.337 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:13.337 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:13.337 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6206 2024-09-04T17:46:13.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.337 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:13.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.338 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:13.338 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423608+0000", 2024-09-04T17:46:13.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:13.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.338 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:13.338 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423608+0000", 2024-09-04T17:46:13.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:13.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.339 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:13.339 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423609+0000", 2024-09-04T17:46:13.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0300000000000001e-06 2024-09-04T17:46:13.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.339 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:13.339 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423614+0000", 2024-09-04T17:46:13.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0200000000000002e-06 2024-09-04T17:46:13.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.339 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:13.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423630+0000", 2024-09-04T17:46:13.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6495999999999999e-05 2024-09-04T17:46:13.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.340 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:13.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423643+0000", 2024-09-04T17:46:13.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2053e-05 2024-09-04T17:46:13.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.340 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:13.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.453527+0000", 2024-09-04T17:46:13.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029884796000000002 2024-09-04T17:46:13.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.341 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:13.341 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.453632+0000", 2024-09-04T17:46:13.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104515 2024-09-04T17:46:13.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.341 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:13.341 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.455469+0000", 2024-09-04T17:46:13.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001836844 2024-09-04T17:46:13.341 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:13.341 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:13.341 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:13.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.342 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6207 3.1f 3:fec7e474:::rbd_data.1134cfbce83d.0000000000000003:6 [sparse-read 1769472~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:13.342 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423662+0000", 2024-09-04T17:46:13.342 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.8707224509999998, 2024-09-04T17:46:13.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.033074294999999997, 2024-09-04T17:46:13.342 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:13.342 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:13.342 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:13.342 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:13.342 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:13.342 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:13.342 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6207 2024-09-04T17:46:13.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.343 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:13.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.343 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:13.343 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423662+0000", 2024-09-04T17:46:13.343 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:13.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.343 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:13.343 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423662+0000", 2024-09-04T17:46:13.343 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:13.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.343 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:13.343 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423664+0000", 2024-09-04T17:46:13.344 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4929999999999999e-06 2024-09-04T17:46:13.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.344 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:13.344 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423669+0000", 2024-09-04T17:46:13.344 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3800000000000002e-06 2024-09-04T17:46:13.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.344 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:13.344 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423686+0000", 2024-09-04T17:46:13.344 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7010000000000001e-05 2024-09-04T17:46:13.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.345 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:13.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423704+0000", 2024-09-04T17:46:13.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7710000000000002e-05 2024-09-04T17:46:13.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.345 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:13.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.455985+0000", 2024-09-04T17:46:13.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032281272999999999 2024-09-04T17:46:13.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.345 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:13.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.456196+0000", 2024-09-04T17:46:13.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000210335 2024-09-04T17:46:13.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:13.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.456737+0000", 2024-09-04T17:46:13.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00054109399999999999 2024-09-04T17:46:13.346 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:13.346 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:13.346 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:13.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.346 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6243 3.17 3:ed419000:::rbd_data.1134cfbce83d.000000000000000b:6 [sparse-read 294912~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:13.346 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423721+0000", 2024-09-04T17:46:13.346 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.8706636650000004, 2024-09-04T17:46:13.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032666132000000001, 2024-09-04T17:46:13.347 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:13.347 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:13.347 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:13.347 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:13.347 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:13.347 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:13.347 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6243 2024-09-04T17:46:13.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.347 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:13.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:13.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423721+0000", 2024-09-04T17:46:13.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:13.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:13.348 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423721+0000", 2024-09-04T17:46:13.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:13.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:13.348 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423722+0000", 2024-09-04T17:46:13.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0470000000000001e-06 2024-09-04T17:46:13.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:13.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423727+0000", 2024-09-04T17:46:13.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0699999999999997e-06 2024-09-04T17:46:13.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:13.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423744+0000", 2024-09-04T17:46:13.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6659000000000001e-05 2024-09-04T17:46:13.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:13.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423756+0000", 2024-09-04T17:46:13.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2284e-05 2024-09-04T17:46:13.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:13.350 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.455553+0000", 2024-09-04T17:46:13.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.031797032000000003 2024-09-04T17:46:13.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:13.350 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.455716+0000", 2024-09-04T17:46:13.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016253700000000001 2024-09-04T17:46:13.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:13.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.456387+0000", 2024-09-04T17:46:13.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00067150300000000005 2024-09-04T17:46:13.351 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:13.351 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:13.351 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:13.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.351 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6244 3.17 3:ed419000:::rbd_data.1134cfbce83d.000000000000000b:6 [read 688128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:13.351 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423775+0000", 2024-09-04T17:46:13.351 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.8706098820000001, 2024-09-04T17:46:13.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.033129498, 2024-09-04T17:46:13.351 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:13.351 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:13.352 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:13.352 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:13.352 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:13.352 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:13.352 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6244 2024-09-04T17:46:13.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.352 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:13.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:13.352 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423775+0000", 2024-09-04T17:46:13.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:13.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:13.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423775+0000", 2024-09-04T17:46:13.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:13.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:13.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423776+0000", 2024-09-04T17:46:13.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1230000000000001e-06 2024-09-04T17:46:13.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:13.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423782+0000", 2024-09-04T17:46:13.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4929999999999998e-06 2024-09-04T17:46:13.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.354 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:13.354 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423799+0000", 2024-09-04T17:46:13.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7254000000000001e-05 2024-09-04T17:46:13.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.354 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:13.354 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423811+0000", 2024-09-04T17:46:13.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2553e-05 2024-09-04T17:46:13.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.354 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:13.355 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.456421+0000", 2024-09-04T17:46:13.355 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032609552 2024-09-04T17:46:13.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.355 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.355 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:13.355 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.456512+0000", 2024-09-04T17:46:13.355 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0593999999999994e-05 2024-09-04T17:46:13.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.355 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.355 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:13.355 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.456904+0000", 2024-09-04T17:46:13.355 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00039292899999999998 2024-09-04T17:46:13.355 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:13.356 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:13.356 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:13.356 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.356 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.356 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6245 3.17 3:ed419000:::rbd_data.1134cfbce83d.000000000000000b:6 [read 1048576~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:13.356 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423830+0000", 2024-09-04T17:46:13.356 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.8705548759999999, 2024-09-04T17:46:13.356 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.033577937000000002, 2024-09-04T17:46:13.356 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:13.356 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:13.356 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:13.356 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:13.356 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:13.357 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:13.357 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6245 2024-09-04T17:46:13.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.357 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:13.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.357 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:13.357 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423830+0000", 2024-09-04T17:46:13.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:13.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.357 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:13.357 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423830+0000", 2024-09-04T17:46:13.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:13.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:13.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423831+0000", 2024-09-04T17:46:13.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.164e-06 2024-09-04T17:46:13.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:13.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423847+0000", 2024-09-04T17:46:13.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5573000000000002e-05 2024-09-04T17:46:13.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:13.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423865+0000", 2024-09-04T17:46:13.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8715999999999998e-05 2024-09-04T17:46:13.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:13.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423880+0000", 2024-09-04T17:46:13.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4107000000000001e-05 2024-09-04T17:46:13.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:13.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.456951+0000", 2024-09-04T17:46:13.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.03307181 2024-09-04T17:46:13.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:13.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.457039+0000", 2024-09-04T17:46:13.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7795999999999998e-05 2024-09-04T17:46:13.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:13.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.457408+0000", 2024-09-04T17:46:13.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000368771 2024-09-04T17:46:13.360 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:13.360 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:13.360 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:13.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.360 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6246 3.17 3:ed419000:::rbd_data.1134cfbce83d.000000000000000b:6 [read 1409024~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:13.361 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423897+0000", 2024-09-04T17:46:13.361 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.8704875369999998, 2024-09-04T17:46:13.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034015888000000001, 2024-09-04T17:46:13.361 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:13.361 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:13.361 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:13.361 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:13.361 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:13.361 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:13.361 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6246 2024-09-04T17:46:13.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.361 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:13.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.362 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:13.362 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423897+0000", 2024-09-04T17:46:13.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:13.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.362 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:13.362 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423897+0000", 2024-09-04T17:46:13.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:13.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.362 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:13.362 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423899+0000", 2024-09-04T17:46:13.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3200000000000001e-06 2024-09-04T17:46:13.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:13.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423904+0000", 2024-09-04T17:46:13.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4369999999999998e-06 2024-09-04T17:46:13.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:13.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423922+0000", 2024-09-04T17:46:13.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8196999999999999e-05 2024-09-04T17:46:13.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:13.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423935+0000", 2024-09-04T17:46:13.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.314e-05 2024-09-04T17:46:13.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:13.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.457457+0000", 2024-09-04T17:46:13.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.033521663 2024-09-04T17:46:13.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:13.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.457536+0000", 2024-09-04T17:46:13.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8795999999999996e-05 2024-09-04T17:46:13.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:13.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.457913+0000", 2024-09-04T17:46:13.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00037733500000000002 2024-09-04T17:46:13.365 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:13.365 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:13.365 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:13.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.365 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6247 3.17 3:ed419000:::rbd_data.1134cfbce83d.000000000000000b:6 [read 1769472~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:13.365 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423953+0000", 2024-09-04T17:46:13.365 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.8704313529999999, 2024-09-04T17:46:13.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0344287, 2024-09-04T17:46:13.365 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:13.365 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:13.365 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:13.366 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:13.366 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:13.366 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:13.366 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6247 2024-09-04T17:46:13.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.366 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:13.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:13.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423953+0000", 2024-09-04T17:46:13.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:13.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:13.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423953+0000", 2024-09-04T17:46:13.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:13.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:13.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423955+0000", 2024-09-04T17:46:13.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0529999999999999e-06 2024-09-04T17:46:13.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:13.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423960+0000", 2024-09-04T17:46:13.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0470000000000001e-06 2024-09-04T17:46:13.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:13.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423978+0000", 2024-09-04T17:46:13.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8119999999999999e-05 2024-09-04T17:46:13.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:13.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423991+0000", 2024-09-04T17:46:13.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2813000000000001e-05 2024-09-04T17:46:13.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:13.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.457947+0000", 2024-09-04T17:46:13.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.033956865000000003 2024-09-04T17:46:13.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:13.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.458039+0000", 2024-09-04T17:46:13.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1834000000000002e-05 2024-09-04T17:46:13.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:13.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.458382+0000", 2024-09-04T17:46:13.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00034296799999999997 2024-09-04T17:46:13.369 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:13.369 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:13.370 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:13.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.370 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6305 3.17 3:ee7cd06a:::rbd_data.113431586065.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:13.370 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.455350+0000", 2024-09-04T17:46:13.370 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.8390348010000004, 2024-09-04T17:46:13.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029276888000000001, 2024-09-04T17:46:13.370 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:13.370 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:13.370 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:13.370 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:13.370 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:13.370 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:13.371 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6305 2024-09-04T17:46:13.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.371 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:13.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:13.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.455350+0000", 2024-09-04T17:46:13.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:13.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:13.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.455350+0000", 2024-09-04T17:46:13.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:13.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:13.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.455353+0000", 2024-09-04T17:46:13.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6400000000000001e-06 2024-09-04T17:46:13.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:13.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459745+0000", 2024-09-04T17:46:13.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0043925880000000002 2024-09-04T17:46:13.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:13.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459775+0000", 2024-09-04T17:46:13.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9254000000000001e-05 2024-09-04T17:46:13.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:13.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459789+0000", 2024-09-04T17:46:13.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4731e-05 2024-09-04T17:46:13.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:13.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459885+0000", 2024-09-04T17:46:13.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5502999999999997e-05 2024-09-04T17:46:13.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:13.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.460040+0000", 2024-09-04T17:46:13.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015491499999999999 2024-09-04T17:46:13.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5", 2024-09-04T17:46:13.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.460596+0000", 2024-09-04T17:46:13.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00055657500000000002 2024-09-04T17:46:13.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:13.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484484+0000", 2024-09-04T17:46:13.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.023887950000000002 2024-09-04T17:46:13.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:46:13.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484510+0000", 2024-09-04T17:46:13.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.561e-05 2024-09-04T17:46:13.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:13.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484569+0000", 2024-09-04T17:46:13.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9049999999999999e-05 2024-09-04T17:46:13.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:13.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484627+0000", 2024-09-04T17:46:13.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8072000000000002e-05 2024-09-04T17:46:13.376 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:13.376 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:13.376 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:13.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.376 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6306 3.17 3:ee7cd06a:::rbd_data.113431586065.0000000000000005:head [truncate 1572864] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:13.376 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.459834+0000", 2024-09-04T17:46:13.376 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.8345511229999998, 2024-09-04T17:46:13.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029166788999999999, 2024-09-04T17:46:13.376 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:13.376 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:13.376 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:13.377 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:13.377 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:13.377 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:13.377 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6306 2024-09-04T17:46:13.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.377 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:13.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:13.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459834+0000", 2024-09-04T17:46:13.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:13.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:13.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459834+0000", 2024-09-04T17:46:13.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:13.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.378 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:13.378 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459836+0000", 2024-09-04T17:46:13.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.835e-06 2024-09-04T17:46:13.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.378 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:13.378 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459841+0000", 2024-09-04T17:46:13.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.728e-06 2024-09-04T17:46:13.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:13.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459851+0000", 2024-09-04T17:46:13.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8570000000000007e-06 2024-09-04T17:46:13.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:13.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459859+0000", 2024-09-04T17:46:13.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4970000000000004e-06 2024-09-04T17:46:13.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:13.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.468789+0000", 2024-09-04T17:46:13.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0089301699999999994 2024-09-04T17:46:13.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:13.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.468890+0000", 2024-09-04T17:46:13.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000100963 2024-09-04T17:46:13.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5", 2024-09-04T17:46:13.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.469090+0000", 2024-09-04T17:46:13.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000199774 2024-09-04T17:46:13.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:13.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484704+0000", 2024-09-04T17:46:13.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015614836999999999 2024-09-04T17:46:13.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:46:13.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.488833+0000", 2024-09-04T17:46:13.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004128357 2024-09-04T17:46:13.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:13.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.488916+0000", 2024-09-04T17:46:13.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2928000000000003e-05 2024-09-04T17:46:13.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:13.382 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.489001+0000", 2024-09-04T17:46:13.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4843000000000001e-05 2024-09-04T17:46:13.382 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:13.382 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:13.382 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:13.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.383 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6307 3.17 3:eb6e082d:::rbd_data.113431586065.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:13.383 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.459871+0000", 2024-09-04T17:46:13.383 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.8345141629999997, 2024-09-04T17:46:13.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032107122000000002, 2024-09-04T17:46:13.383 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:13.383 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:13.383 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:13.383 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:13.383 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:13.383 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:13.383 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6307 2024-09-04T17:46:13.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.383 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:13.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:13.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459871+0000", 2024-09-04T17:46:13.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:13.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:13.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459871+0000", 2024-09-04T17:46:13.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:13.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:13.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459871+0000", 2024-09-04T17:46:13.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9699999999999995e-07 2024-09-04T17:46:13.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:13.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.463429+0000", 2024-09-04T17:46:13.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0035577930000000001 2024-09-04T17:46:13.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:13.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.463463+0000", 2024-09-04T17:46:13.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3460000000000002e-05 2024-09-04T17:46:13.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:13.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.463483+0000", 2024-09-04T17:46:13.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0327999999999999e-05 2024-09-04T17:46:13.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:13.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.477551+0000", 2024-09-04T17:46:13.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014068259 2024-09-04T17:46:13.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:13.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.477748+0000", 2024-09-04T17:46:13.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000197004 2024-09-04T17:46:13.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5", 2024-09-04T17:46:13.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.478346+0000", 2024-09-04T17:46:13.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00059796299999999997 2024-09-04T17:46:13.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:46:13.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.489016+0000", 2024-09-04T17:46:13.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010669781 2024-09-04T17:46:13.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:13.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.491871+0000", 2024-09-04T17:46:13.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028546219999999998 2024-09-04T17:46:13.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:13.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.491931+0000", 2024-09-04T17:46:13.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0736999999999998e-05 2024-09-04T17:46:13.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:13.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.491978+0000", 2024-09-04T17:46:13.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6378000000000002e-05 2024-09-04T17:46:13.388 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:13.388 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:13.389 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:13.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.389 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6308 3.17 3:eb6e082d:::rbd_data.113431586065.0000000000000007:head [truncate 1572864] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:13.389 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.463520+0000", 2024-09-04T17:46:13.389 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.830864654, 2024-09-04T17:46:13.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029348833000000001, 2024-09-04T17:46:13.389 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:13.389 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:13.389 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:13.389 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:13.389 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:13.389 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:13.389 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6308 2024-09-04T17:46:13.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.390 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:13.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:13.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.463520+0000", 2024-09-04T17:46:13.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:13.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:13.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.463520+0000", 2024-09-04T17:46:13.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:13.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:13.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.463522+0000", 2024-09-04T17:46:13.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1150000000000001e-06 2024-09-04T17:46:13.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:13.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.463527+0000", 2024-09-04T17:46:13.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5170000000000001e-06 2024-09-04T17:46:13.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:13.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.463555+0000", 2024-09-04T17:46:13.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8594000000000001e-05 2024-09-04T17:46:13.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:13.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.463563+0000", 2024-09-04T17:46:13.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4340000000000003e-06 2024-09-04T17:46:13.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:13.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484756+0000", 2024-09-04T17:46:13.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021193081999999999 2024-09-04T17:46:13.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:13.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484863+0000", 2024-09-04T17:46:13.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010721999999999999 2024-09-04T17:46:13.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5", 2024-09-04T17:46:13.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.485067+0000", 2024-09-04T17:46:13.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000204 2024-09-04T17:46:13.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:46:13.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.489593+0000", 2024-09-04T17:46:13.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045261529999999998 2024-09-04T17:46:13.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:13.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.492787+0000", 2024-09-04T17:46:13.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031940039999999999 2024-09-04T17:46:13.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:13.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.492834+0000", 2024-09-04T17:46:13.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6281e-05 2024-09-04T17:46:13.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:13.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:13.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:13.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.492869+0000", 2024-09-04T17:46:13.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5432999999999998e-05 2024-09-04T17:46:13.395 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:13.395 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:13.395 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:13.395 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:13.395 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:13.395 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:13.397 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-04T17:46:13.398 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:13.394+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:13.497 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:13.494+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:13.551 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:46:13.552 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:46:13.565 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:13.565 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:46:13.565 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:46:13.566 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:13.569 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:46:13.570 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:46:13.581 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-04T17:46:13.587 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-04T17:46:13.587 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-04T17:46:13.587 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-04T17:46:13.608 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:46:13.609 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:46:13.609 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:13.606+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:13.630 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:46:13.630 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:46:13.650 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-04T17:46:13.651 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-04T17:46:13.671 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:46:13.671 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:46:13.675 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-04T17:46:13.675 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-04T17:46:13.701 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:13.701 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:13.702 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:13.698+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:13.715 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-04T17:46:13.718 INFO:teuthology.orchestra.run.smithi092.stdout:moved 0 / 82 (0%) 2024-09-04T17:46:13.718 INFO:teuthology.orchestra.run.smithi092.stdout:avg 11.7143 2024-09-04T17:46:13.718 INFO:teuthology.orchestra.run.smithi092.stdout:stddev 2.952 -> 2.952 (expected baseline 3.16872) 2024-09-04T17:46:13.718 INFO:teuthology.orchestra.run.smithi092.stdout:min osd.6 with 6 -> 6 pgs (0.512195 -> 0.512195 * mean) 2024-09-04T17:46:13.718 INFO:teuthology.orchestra.run.smithi092.stdout:max osd.1 with 12 -> 12 pgs (1.02439 -> 1.02439 * mean) 2024-09-04T17:46:13.718 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:46:13.718 INFO:teuthology.orchestra.run.smithi092.stdout:oload 1000 2024-09-04T17:46:13.719 INFO:teuthology.orchestra.run.smithi092.stdout:max_change 1 2024-09-04T17:46:13.719 INFO:teuthology.orchestra.run.smithi092.stdout:max_change_osds 4 2024-09-04T17:46:13.719 INFO:teuthology.orchestra.run.smithi092.stdout:average_utilization 0.0005 2024-09-04T17:46:13.719 INFO:teuthology.orchestra.run.smithi092.stdout:overload_utilization 0.0048 2024-09-04T17:46:13.719 INFO:teuthology.orchestra.run.smithi092.stderr:no change 2024-09-04T17:46:13.739 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-04T17:46:13.739 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 1.0 2024-09-04T17:46:13.837 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:13.834+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:13.849 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:13.849 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:13.864 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-04T17:46:13.902 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:46:13.904+0000 7f505d642640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26446) UID: 0 2024-09-04T17:46:14.003 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:46:14.004+0000 7f505d642640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26446) UID: 0 2024-09-04T17:46:14.026 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:14.026 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:14.039 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-04T17:46:14.086 INFO:teuthology.orchestra.run.smithi092.stdout:moved 0 / 82 (0%) 2024-09-04T17:46:14.087 INFO:teuthology.orchestra.run.smithi092.stdout:avg 11.7143 2024-09-04T17:46:14.087 INFO:teuthology.orchestra.run.smithi092.stdout:stddev 2.952 -> 2.952 (expected baseline 3.16872) 2024-09-04T17:46:14.087 INFO:teuthology.orchestra.run.smithi092.stdout:min osd.6 with 6 -> 6 pgs (0.512195 -> 0.512195 * mean) 2024-09-04T17:46:14.087 INFO:teuthology.orchestra.run.smithi092.stdout:max osd.1 with 12 -> 12 pgs (1.02439 -> 1.02439 * mean) 2024-09-04T17:46:14.087 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:46:14.087 INFO:teuthology.orchestra.run.smithi092.stdout:oload 1000 2024-09-04T17:46:14.087 INFO:teuthology.orchestra.run.smithi092.stdout:max_change 1 2024-09-04T17:46:14.087 INFO:teuthology.orchestra.run.smithi092.stdout:max_change_osds 4 2024-09-04T17:46:14.087 INFO:teuthology.orchestra.run.smithi092.stdout:average_utilization 0.0005 2024-09-04T17:46:14.087 INFO:teuthology.orchestra.run.smithi092.stdout:overload_utilization 0.0048 2024-09-04T17:46:14.088 INFO:teuthology.orchestra.run.smithi092.stderr:no change 2024-09-04T17:46:14.103 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:14.102+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:14.105 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-04T17:46:14.106 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 0.05 2024-09-04T17:46:14.161 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:14.161 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:14.176 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-04T17:46:14.204 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:14.204+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:14.305 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:14.304+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:14.369 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:14.369 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:14.381 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-04T17:46:14.411 INFO:teuthology.orchestra.run.smithi092.stdout:876 write 0x7166a33 thru 0x7174298 (0xd866 bytes) 2024-09-04T17:46:14.427 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:14.428+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:14.506 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:14.506+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:14.522 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-09-04T17:46:14.520+0000 7fc85ed10640 -1 mgr.server reply reply (4) Interrupted system call no change 2024-09-04T17:46:14.522 INFO:teuthology.orchestra.run.smithi092.stdout:moved 3 / 82 (3.65854%) 2024-09-04T17:46:14.522 INFO:teuthology.orchestra.run.smithi092.stdout:avg 11.7143 2024-09-04T17:46:14.522 INFO:teuthology.orchestra.run.smithi092.stdout:stddev 2.952 -> 2.64575 (expected baseline 3.16872) 2024-09-04T17:46:14.522 INFO:teuthology.orchestra.run.smithi092.stdout:min osd.6 with 6 -> 7 pgs (0.512195 -> 0.597561 * mean) 2024-09-04T17:46:14.522 INFO:teuthology.orchestra.run.smithi092.stdout:max osd.1 with 12 -> 12 pgs (1.02439 -> 1.02439 * mean) 2024-09-04T17:46:14.523 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:46:14.523 INFO:teuthology.orchestra.run.smithi092.stdout:oload 110 2024-09-04T17:46:14.523 INFO:teuthology.orchestra.run.smithi092.stdout:max_change 0.05 2024-09-04T17:46:14.523 INFO:teuthology.orchestra.run.smithi092.stdout:max_change_osds 4 2024-09-04T17:46:14.523 INFO:teuthology.orchestra.run.smithi092.stdout:average_utilization 0.0005 2024-09-04T17:46:14.523 INFO:teuthology.orchestra.run.smithi092.stdout:overload_utilization 0.0005 2024-09-04T17:46:14.523 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4 weight 0.7470 -> 0.6970 2024-09-04T17:46:14.523 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7 weight 0.7661 -> 0.7161 2024-09-04T17:46:14.523 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5 weight 0.8015 -> 0.7515 2024-09-04T17:46:14.523 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0 weight 0.9082 -> 0.8583 2024-09-04T17:46:14.523 INFO:teuthology.orchestra.run.smithi092.stderr:no change 2024-09-04T17:46:14.527 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:14.527 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:14.540 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-04T17:46:14.540 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 1.0 2024-09-04T17:46:14.541 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-04T17:46:14.627 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:14.628+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:14.707 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:14.707 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:14.708 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:14.706+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:14.721 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-04T17:46:14.809 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:14.808+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:14.877 INFO:teuthology.orchestra.run.smithi092.stdout:moved 0 / 82 (0%) 2024-09-04T17:46:14.877 INFO:teuthology.orchestra.run.smithi092.stdout:avg 11.7143 2024-09-04T17:46:14.877 INFO:teuthology.orchestra.run.smithi092.stdout:stddev 2.952 -> 2.952 (expected baseline 3.16872) 2024-09-04T17:46:14.877 INFO:teuthology.orchestra.run.smithi092.stdout:min osd.6 with 6 -> 6 pgs (0.512195 -> 0.512195 * mean) 2024-09-04T17:46:14.878 INFO:teuthology.orchestra.run.smithi092.stdout:max osd.1 with 12 -> 12 pgs (1.02439 -> 1.02439 * mean) 2024-09-04T17:46:14.878 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:46:14.878 INFO:teuthology.orchestra.run.smithi092.stdout:oload 1000 2024-09-04T17:46:14.878 INFO:teuthology.orchestra.run.smithi092.stdout:max_change 1 2024-09-04T17:46:14.878 INFO:teuthology.orchestra.run.smithi092.stdout:max_change_osds 4 2024-09-04T17:46:14.878 INFO:teuthology.orchestra.run.smithi092.stdout:average_utilization 0.0005 2024-09-04T17:46:14.878 INFO:teuthology.orchestra.run.smithi092.stdout:overload_utilization 0.0048 2024-09-04T17:46:14.878 INFO:teuthology.orchestra.run.smithi092.stderr:no change 2024-09-04T17:46:14.885 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:14.885 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:14.900 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-04T17:46:14.939 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:14.938+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:15.011 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:46:15.012+0000 7f505d642640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26446) UID: 0 2024-09-04T17:46:15.068 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:15.068 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:15.082 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-04T17:46:15.127 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:15.128+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:15.212 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:15.212+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:15.252 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:15.252 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:15.272 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-04T17:46:15.274 INFO:teuthology.orchestra.run.smithi092.stdout:877 write 0x98db415 thru 0x98e2f63 (0x7b4f bytes) 2024-09-04T17:46:15.277 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-04T17:46:15.322 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:15.322+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:15.405 INFO:teuthology.orchestra.run.smithi092.stdout:878 trunc from 0x9ba81a1 to 0x10246f2 2024-09-04T17:46:15.413 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:15.410+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:15.447 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:15.447 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:15.465 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-04T17:46:15.514 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:15.512+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:15.603 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:15.603 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:15.616 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:15.616+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:15.617 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-04T17:46:15.677 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-04T17:46:15.722 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:15.722+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:15.783 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:15.783 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:15.799 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-04T17:46:15.817 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:15.814+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:15.817 INFO:teuthology.orchestra.run.smithi092.stdout:879 writesame 0xbd075fc thru 0xbd130dc data_size 0x3(0xbae1 bytes) 2024-09-04T17:46:15.819 INFO:teuthology.orchestra.run.smithi092.stdout:880 trunc from 0xbd130dd to 0xcc1cd48 2024-09-04T17:46:15.825 INFO:teuthology.orchestra.run.smithi092.stdout:881 read 0xa29adb6 thru 0xa2a8ebc (0xe107 bytes) 2024-09-04T17:46:15.825 INFO:teuthology.orchestra.run.smithi092.stdout:882 punch from 0x2f7711a to 0x2f7cc38, (0x5b1e bytes) 2024-09-04T17:46:15.830 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:46:15.831 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:46:15.834 INFO:teuthology.orchestra.run.smithi092.stdout:883 trunc from 0xcc1cd48 to 0x6941e24 2024-09-04T17:46:15.848 INFO:teuthology.orchestra.run.smithi092.stdout:884 trunc from 0x6941e24 to 0x1d7bc1f 2024-09-04T17:46:15.854 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:46:15.854 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:46:15.855 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:15.855 INFO:teuthology.orchestra.run.smithi092.stdout:886 read 0x350f5e thru 0x3584f8 (0x759b bytes) 2024-09-04T17:46:15.859 INFO:teuthology.orchestra.run.smithi092.stdout:887 read 0x12f27a8 thru 0x12f6a75 (0x42ce bytes) 2024-09-04T17:46:15.872 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-04T17:46:15.873 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-04T17:46:15.873 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-04T17:46:15.883 INFO:teuthology.orchestra.run.smithi092.stdout:888 write 0xc9ed2c4 thru 0xc9edfe0 (0xd1d bytes) 2024-09-04T17:46:15.889 INFO:teuthology.orchestra.run.smithi092.stdout:889 read 0x89321a thru 0x8a236a (0xf151 bytes) 2024-09-04T17:46:15.893 INFO:teuthology.orchestra.run.smithi092.stdout:890 write 0x8ad9ee7 thru 0x8ade702 (0x481c bytes) 2024-09-04T17:46:15.896 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:46:15.896 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:46:15.909 INFO:teuthology.orchestra.run.smithi092.stdout:893 write 0x65bbf1e thru 0x65c68f5 (0xa9d8 bytes) 2024-09-04T17:46:15.915 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:46:15.916 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:46:15.918 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:15.915+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:15.935 INFO:teuthology.orchestra.run.smithi092.stdout:894 read 0xbfebd0f thru 0xbfee083 (0x2375 bytes) 2024-09-04T17:46:15.935 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-04T17:46:15.935 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-04T17:46:15.936 INFO:teuthology.orchestra.run.smithi092.stdout:895 read 0x646f034 thru 0x647681c (0x77e9 bytes) 2024-09-04T17:46:15.936 INFO:teuthology.orchestra.run.smithi092.stdout:896 punch from 0xa9e1712 to 0xa9ec4ba, (0xada8 bytes) 2024-09-04T17:46:15.936 INFO:teuthology.orchestra.run.smithi092.stdout:897 write 0xbe964ad thru 0xbe9e7d2 (0x8326 bytes) 2024-09-04T17:46:15.937 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:15.937 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:46:15.938 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:46:15.938 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:46:15.938 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.938 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:1348 3.10 3:0afd7a80:::rbd_data.11346251f094.000000000000009b:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-04T17:46:15.938 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:02.609239+0000", 2024-09-04T17:46:15.938 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 13.329316941, 2024-09-04T17:46:15.938 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.026961731999999999, 2024-09-04T17:46:15.938 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:15.938 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:15.938 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:15.938 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:15.938 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:15.939 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:15.939 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 1348 2024-09-04T17:46:15.939 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.939 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:15.939 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.939 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:15.939 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.609239+0000", 2024-09-04T17:46:15.939 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:15.939 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.939 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.939 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:15.939 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.609239+0000", 2024-09-04T17:46:15.940 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:15.940 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.940 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.940 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:15.940 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.609242+0000", 2024-09-04T17:46:15.940 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.3269999999999999e-06 2024-09-04T17:46:15.940 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.940 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.940 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:15.940 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.609252+0000", 2024-09-04T17:46:15.940 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0689e-05 2024-09-04T17:46:15.940 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.940 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.940 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:15.941 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.609282+0000", 2024-09-04T17:46:15.941 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.932e-05 2024-09-04T17:46:15.941 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.941 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.941 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:15.941 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.609296+0000", 2024-09-04T17:46:15.941 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4197e-05 2024-09-04T17:46:15.941 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.941 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.941 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:15.941 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.609408+0000", 2024-09-04T17:46:15.941 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000111962 2024-09-04T17:46:15.941 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.942 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.942 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:15.942 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.609544+0000", 2024-09-04T17:46:15.942 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000136706 2024-09-04T17:46:15.942 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.942 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.942 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 0", 2024-09-04T17:46:15.942 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.613984+0000", 2024-09-04T17:46:15.942 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0044397890000000004 2024-09-04T17:46:15.942 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.942 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.942 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:15.942 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.635928+0000", 2024-09-04T17:46:15.942 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.021943547000000001 2024-09-04T17:46:15.943 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.943 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.943 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:15.943 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.636030+0000", 2024-09-04T17:46:15.943 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00010199700000000001 2024-09-04T17:46:15.943 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.943 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.943 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:15.943 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.636123+0000", 2024-09-04T17:46:15.943 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.3628999999999994e-05 2024-09-04T17:46:15.943 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.943 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.943 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:15.943 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.636201+0000", 2024-09-04T17:46:15.944 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.7569000000000002e-05 2024-09-04T17:46:15.944 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:15.944 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:15.944 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:15.944 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.944 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.944 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4019 3.1e 3:7817bfb4:::rbd_data.113417f3df25.0000000000000001:5 [read 7634944~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-04T17:46:15.944 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:06.909047+0000", 2024-09-04T17:46:15.944 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 9.0295089859999997, 2024-09-04T17:46:15.944 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.026506477000000001, 2024-09-04T17:46:15.944 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:15.944 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:15.944 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:15.945 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:15.945 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:15.945 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:15.945 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4019 2024-09-04T17:46:15.945 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.945 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:15.945 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.945 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:15.945 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909047+0000", 2024-09-04T17:46:15.945 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:15.945 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.945 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.946 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:15.946 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909047+0000", 2024-09-04T17:46:15.946 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:15.946 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.946 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.946 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:15.946 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909048+0000", 2024-09-04T17:46:15.946 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0800000000000005e-07 2024-09-04T17:46:15.946 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.946 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.946 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:15.946 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909050+0000", 2024-09-04T17:46:15.946 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.587e-06 2024-09-04T17:46:15.947 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.947 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.947 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:15.947 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909057+0000", 2024-09-04T17:46:15.947 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.9269999999999999e-06 2024-09-04T17:46:15.947 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.947 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.947 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:15.947 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909062+0000", 2024-09-04T17:46:15.947 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2730000000000002e-06 2024-09-04T17:46:15.947 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.947 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.947 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:15.948 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.935341+0000", 2024-09-04T17:46:15.948 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.026278526 2024-09-04T17:46:15.948 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.948 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.948 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:15.948 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.935423+0000", 2024-09-04T17:46:15.948 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.2158999999999997e-05 2024-09-04T17:46:15.948 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.948 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.948 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:15.948 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.935554+0000", 2024-09-04T17:46:15.948 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013049700000000001 2024-09-04T17:46:15.948 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:15.948 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:15.949 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:15.949 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.949 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.949 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4020 3.1e 3:7817bfb4:::rbd_data.113417f3df25.0000000000000001:5 [read 8060928~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-04T17:46:15.949 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:06.909070+0000", 2024-09-04T17:46:15.949 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 9.0294858009999999, 2024-09-04T17:46:15.949 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.026770842999999999, 2024-09-04T17:46:15.949 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:15.949 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:15.949 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:15.949 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:15.949 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:15.950 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:15.950 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4020 2024-09-04T17:46:15.950 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.950 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:15.950 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.950 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:15.950 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909070+0000", 2024-09-04T17:46:15.950 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:15.950 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.950 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.950 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:15.950 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909070+0000", 2024-09-04T17:46:15.951 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:15.951 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.951 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.951 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:15.951 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909071+0000", 2024-09-04T17:46:15.951 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9599999999999999e-07 2024-09-04T17:46:15.951 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.951 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.951 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:15.951 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909073+0000", 2024-09-04T17:46:15.951 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2010000000000002e-06 2024-09-04T17:46:15.951 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.951 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.951 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:15.951 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909080+0000", 2024-09-04T17:46:15.952 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.391e-06 2024-09-04T17:46:15.952 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.952 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.952 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:15.952 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909086+0000", 2024-09-04T17:46:15.952 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1499999999999998e-06 2024-09-04T17:46:15.952 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.952 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.952 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:15.952 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.935601+0000", 2024-09-04T17:46:15.952 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.026515318999999999 2024-09-04T17:46:15.952 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.953 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.953 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:15.953 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.935699+0000", 2024-09-04T17:46:15.953 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.7956999999999997e-05 2024-09-04T17:46:15.953 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.953 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.953 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:15.953 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.935841+0000", 2024-09-04T17:46:15.953 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014232900000000001 2024-09-04T17:46:15.953 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:15.953 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:15.953 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:15.953 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.954 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.954 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4076 3.1e 3:7817bfb4:::rbd_data.113417f3df25.0000000000000001:5 [read 4259840~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-04T17:46:15.954 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:06.909093+0000", 2024-09-04T17:46:15.954 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 9.0294632920000009, 2024-09-04T17:46:15.954 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.026930059999999999, 2024-09-04T17:46:15.954 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:15.954 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:15.954 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:15.954 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:15.954 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:15.954 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:15.954 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4076 2024-09-04T17:46:15.954 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.955 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:15.955 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.955 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:15.955 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909093+0000", 2024-09-04T17:46:15.955 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:15.955 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.955 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.955 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:15.955 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909093+0000", 2024-09-04T17:46:15.955 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:15.955 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.955 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.955 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:15.956 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909094+0000", 2024-09-04T17:46:15.956 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7999999999999995e-07 2024-09-04T17:46:15.956 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.956 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.956 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:15.956 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909096+0000", 2024-09-04T17:46:15.956 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1959999999999998e-06 2024-09-04T17:46:15.956 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.956 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.956 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:15.956 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909103+0000", 2024-09-04T17:46:15.956 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.6499999999999996e-06 2024-09-04T17:46:15.956 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.956 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.957 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:15.957 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909108+0000", 2024-09-04T17:46:15.957 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0860000000000001e-06 2024-09-04T17:46:15.957 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.957 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.957 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:15.957 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.935885+0000", 2024-09-04T17:46:15.957 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.026776687 2024-09-04T17:46:15.957 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.957 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.957 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:15.957 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.935946+0000", 2024-09-04T17:46:15.958 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.0473000000000002e-05 2024-09-04T17:46:15.958 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.958 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.958 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:15.958 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.936023+0000", 2024-09-04T17:46:15.958 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.7287999999999996e-05 2024-09-04T17:46:15.958 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:15.958 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:15.958 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:15.958 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.958 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.958 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4077 3.1e 3:7817bfb4:::rbd_data.113417f3df25.0000000000000001:5 [read 4685824~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-04T17:46:15.958 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:06.909117+0000", 2024-09-04T17:46:15.959 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 9.0294395719999994, 2024-09-04T17:46:15.959 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.027219975, 2024-09-04T17:46:15.959 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:15.959 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:15.959 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:15.959 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:15.959 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:15.959 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:15.959 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4077 2024-09-04T17:46:15.959 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.959 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:15.959 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.959 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:15.959 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909117+0000", 2024-09-04T17:46:15.960 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:15.960 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.960 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.960 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:15.960 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909117+0000", 2024-09-04T17:46:15.960 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:15.960 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.960 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.960 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:15.960 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909117+0000", 2024-09-04T17:46:15.960 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.8299999999999997e-07 2024-09-04T17:46:15.960 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.961 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.961 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:15.961 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909119+0000", 2024-09-04T17:46:15.961 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2680000000000001e-06 2024-09-04T17:46:15.961 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.961 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.961 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:15.961 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909127+0000", 2024-09-04T17:46:15.961 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.9670000000000001e-06 2024-09-04T17:46:15.961 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.961 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.961 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:15.961 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909133+0000", 2024-09-04T17:46:15.962 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.215e-06 2024-09-04T17:46:15.962 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.962 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.962 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:15.962 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.936076+0000", 2024-09-04T17:46:15.962 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.026943292000000001 2024-09-04T17:46:15.962 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.962 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.962 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:15.962 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.936171+0000", 2024-09-04T17:46:15.962 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4933999999999996e-05 2024-09-04T17:46:15.962 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.962 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.963 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:15.963 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.936337+0000", 2024-09-04T17:46:15.963 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000165816 2024-09-04T17:46:15.963 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:15.963 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:15.963 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:15.963 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.963 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.963 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4078 3.1e 3:7817bfb4:::rbd_data.113417f3df25.0000000000000001:5 [read 5111808~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-04T17:46:15.963 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:06.909140+0000", 2024-09-04T17:46:15.963 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 9.0294159470000004, 2024-09-04T17:46:15.963 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.027371784, 2024-09-04T17:46:15.963 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:15.964 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:15.964 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:15.964 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:15.964 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:15.964 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:15.964 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4078 2024-09-04T17:46:15.964 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.964 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:15.964 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.964 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:15.964 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909140+0000", 2024-09-04T17:46:15.964 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:15.964 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.965 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.965 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:15.965 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909140+0000", 2024-09-04T17:46:15.965 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:15.965 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.965 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.965 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:15.965 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909141+0000", 2024-09-04T17:46:15.965 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6800000000000001e-07 2024-09-04T17:46:15.965 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.965 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.966 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:15.966 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909153+0000", 2024-09-04T17:46:15.966 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2542e-05 2024-09-04T17:46:15.966 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.966 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.966 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:15.966 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909162+0000", 2024-09-04T17:46:15.966 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.833e-06 2024-09-04T17:46:15.966 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.966 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.966 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:15.966 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909174+0000", 2024-09-04T17:46:15.966 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2429000000000001e-05 2024-09-04T17:46:15.967 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.967 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.967 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:15.967 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.936371+0000", 2024-09-04T17:46:15.967 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.027196708999999999 2024-09-04T17:46:15.967 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.967 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.967 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:15.967 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.936432+0000", 2024-09-04T17:46:15.967 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.0707e-05 2024-09-04T17:46:15.967 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.967 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.967 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:15.968 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.936512+0000", 2024-09-04T17:46:15.968 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.0096e-05 2024-09-04T17:46:15.968 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:15.968 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:15.968 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:15.968 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.968 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.968 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4079 3.1e 3:7817bfb4:::rbd_data.113417f3df25.0000000000000001:5 [read 5537792~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-04T17:46:15.968 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:06.909184+0000", 2024-09-04T17:46:15.968 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 9.0293726270000008, 2024-09-04T17:46:15.968 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.027638882, 2024-09-04T17:46:15.968 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:15.968 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:15.969 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:15.969 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:15.969 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:15.969 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:15.969 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4079 2024-09-04T17:46:15.969 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.969 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:15.969 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.969 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:15.969 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909184+0000", 2024-09-04T17:46:15.969 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:15.969 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.969 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.970 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:15.970 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909184+0000", 2024-09-04T17:46:15.970 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:15.970 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.970 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.970 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:15.970 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909184+0000", 2024-09-04T17:46:15.970 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2e-07 2024-09-04T17:46:15.970 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.970 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.970 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:15.970 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909187+0000", 2024-09-04T17:46:15.970 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.576e-06 2024-09-04T17:46:15.971 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.971 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.971 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:15.971 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909193+0000", 2024-09-04T17:46:15.971 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6449999999999999e-06 2024-09-04T17:46:15.971 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.971 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.971 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:15.971 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909199+0000", 2024-09-04T17:46:15.971 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2939999999999996e-06 2024-09-04T17:46:15.971 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.971 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.971 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:15.972 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.936548+0000", 2024-09-04T17:46:15.972 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.027349605999999999 2024-09-04T17:46:15.972 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.972 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.972 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:15.972 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.936640+0000", 2024-09-04T17:46:15.972 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.2194000000000005e-05 2024-09-04T17:46:15.972 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.972 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.972 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:15.972 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.936822+0000", 2024-09-04T17:46:15.972 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00018204699999999999 2024-09-04T17:46:15.972 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:15.973 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:15.973 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:15.973 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.973 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.973 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4080 3.1e 3:7817bfb4:::rbd_data.113417f3df25.0000000000000001:5 [read 5963776~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-04T17:46:15.973 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:06.909207+0000", 2024-09-04T17:46:15.973 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 9.0293496510000004, 2024-09-04T17:46:15.973 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.027874711, 2024-09-04T17:46:15.973 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:15.973 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:15.973 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:15.973 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:15.973 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:15.973 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:15.974 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4080 2024-09-04T17:46:15.974 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.974 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:15.974 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.974 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:15.974 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909207+0000", 2024-09-04T17:46:15.974 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:15.974 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.974 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.974 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:15.974 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909207+0000", 2024-09-04T17:46:15.974 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:15.974 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.975 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.975 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:15.975 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909207+0000", 2024-09-04T17:46:15.975 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9900000000000001e-07 2024-09-04T17:46:15.975 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.975 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.975 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:15.975 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909209+0000", 2024-09-04T17:46:15.975 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1639999999999999e-06 2024-09-04T17:46:15.975 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.975 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.975 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:15.975 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909217+0000", 2024-09-04T17:46:15.975 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4869999999999996e-06 2024-09-04T17:46:15.976 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.976 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.976 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:15.976 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909222+0000", 2024-09-04T17:46:15.976 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1780000000000002e-06 2024-09-04T17:46:15.976 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.976 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.976 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:15.976 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.936856+0000", 2024-09-04T17:46:15.976 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.027634394999999999 2024-09-04T17:46:15.976 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.976 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.976 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:15.976 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.936940+0000", 2024-09-04T17:46:15.977 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.4241999999999997e-05 2024-09-04T17:46:15.977 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.977 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.977 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:15.977 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.937081+0000", 2024-09-04T17:46:15.977 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000140746 2024-09-04T17:46:15.977 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:15.977 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:15.977 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:15.977 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.977 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.977 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4081 3.1e 3:7817bfb4:::rbd_data.113417f3df25.0000000000000001:5 [read 6389760~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-04T17:46:15.977 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:06.909230+0000", 2024-09-04T17:46:15.977 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 9.0293265750000007, 2024-09-04T17:46:15.977 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.028121515, 2024-09-04T17:46:15.978 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:15.978 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:15.978 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:15.978 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:15.978 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:15.978 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:15.978 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4081 2024-09-04T17:46:15.978 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.978 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:15.978 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.978 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:15.978 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909230+0000", 2024-09-04T17:46:15.979 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:15.979 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.979 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.979 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:15.979 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909230+0000", 2024-09-04T17:46:15.979 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:15.979 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.979 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.979 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:15.979 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909230+0000", 2024-09-04T17:46:15.979 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4700000000000002e-07 2024-09-04T17:46:15.979 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.979 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.979 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:15.980 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909232+0000", 2024-09-04T17:46:15.980 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2110000000000001e-06 2024-09-04T17:46:15.980 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.980 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.980 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:15.980 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909240+0000", 2024-09-04T17:46:15.980 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.5460000000000003e-06 2024-09-04T17:46:15.980 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.980 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.980 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:15.980 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909245+0000", 2024-09-04T17:46:15.980 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1900000000000003e-06 2024-09-04T17:46:15.980 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.981 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.981 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:15.981 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.937119+0000", 2024-09-04T17:46:15.981 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.027874448999999999 2024-09-04T17:46:15.981 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.981 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.981 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:15.981 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.937209+0000", 2024-09-04T17:46:15.981 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9423000000000001e-05 2024-09-04T17:46:15.981 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.981 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.981 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:15.981 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.937351+0000", 2024-09-04T17:46:15.981 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000142249 2024-09-04T17:46:15.982 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:15.982 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:15.982 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:15.982 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.982 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.982 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4082 3.1e 3:7817bfb4:::rbd_data.113417f3df25.0000000000000001:5 [read 6815744~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-04T17:46:15.982 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:06.909253+0000", 2024-09-04T17:46:15.982 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 9.0293035610000008, 2024-09-04T17:46:15.982 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.028352584, 2024-09-04T17:46:15.982 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:15.982 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:15.982 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:15.982 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:15.982 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:15.983 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:15.983 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4082 2024-09-04T17:46:15.983 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.983 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:15.983 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.983 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:15.983 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909253+0000", 2024-09-04T17:46:15.983 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:15.983 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.983 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.983 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:15.983 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909253+0000", 2024-09-04T17:46:15.983 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:15.983 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.984 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.984 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:15.984 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909253+0000", 2024-09-04T17:46:15.984 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0999999999999999e-07 2024-09-04T17:46:15.984 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.984 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.984 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:15.984 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909255+0000", 2024-09-04T17:46:15.984 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1840000000000002e-06 2024-09-04T17:46:15.984 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.984 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.984 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:15.984 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909264+0000", 2024-09-04T17:46:15.984 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.422e-06 2024-09-04T17:46:15.985 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.985 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.985 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:15.985 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909277+0000", 2024-09-04T17:46:15.985 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3723000000000001e-05 2024-09-04T17:46:15.985 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.985 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.985 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:15.985 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.937384+0000", 2024-09-04T17:46:15.985 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.028106973 2024-09-04T17:46:15.985 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.985 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.985 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:15.986 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.937471+0000", 2024-09-04T17:46:15.986 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.6366999999999995e-05 2024-09-04T17:46:15.986 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.986 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.986 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:15.986 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.937605+0000", 2024-09-04T17:46:15.986 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013440500000000001 2024-09-04T17:46:15.986 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:15.986 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:15.986 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:15.986 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.986 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.986 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4083 3.1e 3:7817bfb4:::rbd_data.113417f3df25.0000000000000001:5 [read 7241728~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-04T17:46:15.987 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:06.909286+0000", 2024-09-04T17:46:15.987 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 9.0292701149999992, 2024-09-04T17:46:15.987 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.028595905000000001, 2024-09-04T17:46:15.987 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:15.987 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:15.987 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:15.987 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:15.987 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:15.987 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:15.987 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4083 2024-09-04T17:46:15.988 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.988 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:15.988 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.988 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:15.988 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909286+0000", 2024-09-04T17:46:15.988 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:15.988 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.988 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.988 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:15.988 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909286+0000", 2024-09-04T17:46:15.988 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:15.988 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.988 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.989 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:15.989 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909287+0000", 2024-09-04T17:46:15.989 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2399999999999998e-07 2024-09-04T17:46:15.989 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.989 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.989 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:15.989 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909295+0000", 2024-09-04T17:46:15.989 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.2339999999999994e-06 2024-09-04T17:46:15.989 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.989 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.989 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:15.989 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909301+0000", 2024-09-04T17:46:15.989 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.8340000000000004e-06 2024-09-04T17:46:15.990 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.990 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.990 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:15.990 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909306+0000", 2024-09-04T17:46:15.990 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.3159999999999996e-06 2024-09-04T17:46:15.990 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.990 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.990 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:15.990 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.937638+0000", 2024-09-04T17:46:15.990 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.028332267000000001 2024-09-04T17:46:15.990 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.990 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.990 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:15.990 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.937741+0000", 2024-09-04T17:46:15.991 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000102756 2024-09-04T17:46:15.991 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.991 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.991 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:15.991 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.937882+0000", 2024-09-04T17:46:15.991 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014097400000000001 2024-09-04T17:46:15.991 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:15.991 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:15.991 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:15.991 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.991 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.991 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4084 3.1e 3:7817bfb4:::rbd_data.113417f3df25.0000000000000001:5 [read 7667712~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-04T17:46:15.991 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:06.909314+0000", 2024-09-04T17:46:15.992 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 9.0292419069999994, 2024-09-04T17:46:15.992 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.028838895, 2024-09-04T17:46:15.992 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:15.992 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:15.992 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:15.992 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:15.992 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:15.992 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:15.992 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4084 2024-09-04T17:46:15.992 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.993 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:15.993 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.993 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:15.993 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909314+0000", 2024-09-04T17:46:15.993 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:15.993 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.993 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.993 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:15.993 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909314+0000", 2024-09-04T17:46:15.993 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:15.993 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.993 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.993 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:15.993 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909315+0000", 2024-09-04T17:46:15.994 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7999999999999995e-07 2024-09-04T17:46:15.994 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.994 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.994 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:15.994 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909317+0000", 2024-09-04T17:46:15.994 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2529999999999999e-06 2024-09-04T17:46:15.994 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.994 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.994 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:15.994 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909325+0000", 2024-09-04T17:46:15.994 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.6279999999999996e-06 2024-09-04T17:46:15.994 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.994 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.994 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:15.995 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909330+0000", 2024-09-04T17:46:15.995 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.3040000000000004e-06 2024-09-04T17:46:15.995 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.995 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.995 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:15.995 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.937915+0000", 2024-09-04T17:46:15.995 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.028584766000000001 2024-09-04T17:46:15.995 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.995 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.995 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:15.995 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.938002+0000", 2024-09-04T17:46:15.995 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7590999999999995e-05 2024-09-04T17:46:15.996 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.996 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.996 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:15.996 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.938153+0000", 2024-09-04T17:46:15.996 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00015067300000000001 2024-09-04T17:46:15.996 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:15.996 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:15.996 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:15.996 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.996 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.996 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4085 3.1e 3:7817bfb4:::rbd_data.113417f3df25.0000000000000001:5 [read 8093696~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-04T17:46:15.996 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:06.909338+0000", 2024-09-04T17:46:15.996 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 9.0292180729999991, 2024-09-04T17:46:15.997 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.029098151999999999, 2024-09-04T17:46:15.997 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:15.997 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:15.997 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:15.997 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:15.997 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:15.997 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:15.997 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4085 2024-09-04T17:46:15.997 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.997 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:15.997 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.998 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:15.998 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909338+0000", 2024-09-04T17:46:15.998 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:15.998 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.998 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.998 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:15.998 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909338+0000", 2024-09-04T17:46:15.998 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:15.998 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.998 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.998 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:15.998 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909339+0000", 2024-09-04T17:46:15.998 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6100000000000001e-07 2024-09-04T17:46:15.998 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.999 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.999 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:15.999 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909356+0000", 2024-09-04T17:46:15.999 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7147e-05 2024-09-04T17:46:15.999 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.999 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.999 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:15.999 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909362+0000", 2024-09-04T17:46:15.999 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.657e-06 2024-09-04T17:46:15.999 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:15.999 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:15.999 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:15.999 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.909368+0000", 2024-09-04T17:46:16.000 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6459999999999998e-06 2024-09-04T17:46:16.000 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.000 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.000 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:16.000 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.938194+0000", 2024-09-04T17:46:16.000 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.028826424 2024-09-04T17:46:16.000 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.000 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.000 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:16.000 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.938296+0000", 2024-09-04T17:46:16.000 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000101087 2024-09-04T17:46:16.000 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.000 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.001 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:16.001 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:06.938436+0000", 2024-09-04T17:46:16.001 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014072999999999999 2024-09-04T17:46:16.001 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:16.001 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:16.001 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:16.001 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.001 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.001 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4797 3.1e 3:7817bfb4:::rbd_data.113417f3df25.0000000000000001:5 [read 1474560~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:16.001 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.350022+0000", 2024-09-04T17:46:16.001 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 8.5885345530000006, 2024-09-04T17:46:16.002 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.026399108000000001, 2024-09-04T17:46:16.002 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:16.002 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:16.002 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:16.002 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:16.002 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:16.002 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:16.002 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4797 2024-09-04T17:46:16.003 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.003 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:16.003 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.003 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:16.003 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350022+0000", 2024-09-04T17:46:16.003 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:16.003 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.003 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.003 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:16.003 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350022+0000", 2024-09-04T17:46:16.004 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:16.004 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.004 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.004 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:16.004 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350023+0000", 2024-09-04T17:46:16.004 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0160000000000001e-06 2024-09-04T17:46:16.004 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.004 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.004 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:16.004 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350038+0000", 2024-09-04T17:46:16.004 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5671999999999998e-05 2024-09-04T17:46:16.004 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.004 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.005 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:16.005 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350049+0000", 2024-09-04T17:46:16.005 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0266e-05 2024-09-04T17:46:16.005 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.005 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.005 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:16.005 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350056+0000", 2024-09-04T17:46:16.005 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4900000000000003e-06 2024-09-04T17:46:16.005 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.005 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.005 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:16.005 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.374015+0000", 2024-09-04T17:46:16.005 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.023959081 2024-09-04T17:46:16.006 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.006 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.006 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:16.006 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.374112+0000", 2024-09-04T17:46:16.006 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.7322000000000001e-05 2024-09-04T17:46:16.006 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.006 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.006 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:16.006 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.376421+0000", 2024-09-04T17:46:16.006 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0023082609999999998 2024-09-04T17:46:16.006 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:16.006 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:16.006 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:16.006 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.007 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.007 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4798 3.1e 3:7817bfb4:::rbd_data.113417f3df25.0000000000000001:5 [read 1900544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:16.007 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.350077+0000", 2024-09-04T17:46:16.007 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 8.5884788440000008, 2024-09-04T17:46:16.007 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.026604033999999999, 2024-09-04T17:46:16.007 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:16.007 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:16.007 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:16.007 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:16.007 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:16.007 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:16.007 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4798 2024-09-04T17:46:16.007 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.007 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:16.008 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.008 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:16.008 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350077+0000", 2024-09-04T17:46:16.008 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:16.008 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.008 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.008 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:16.008 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350077+0000", 2024-09-04T17:46:16.008 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:16.008 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.008 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.008 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:16.008 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350078+0000", 2024-09-04T17:46:16.008 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.4300000000000003e-07 2024-09-04T17:46:16.009 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.009 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.009 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:16.009 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350082+0000", 2024-09-04T17:46:16.009 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.9829999999999998e-06 2024-09-04T17:46:16.009 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.009 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.009 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:16.009 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350091+0000", 2024-09-04T17:46:16.009 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.3500000000000003e-06 2024-09-04T17:46:16.009 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.009 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.009 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:16.009 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350099+0000", 2024-09-04T17:46:16.010 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.7500000000000003e-06 2024-09-04T17:46:16.010 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.010 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.010 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:16.010 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.376457+0000", 2024-09-04T17:46:16.010 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.026358320000000001 2024-09-04T17:46:16.010 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.010 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.010 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:16.010 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.376552+0000", 2024-09-04T17:46:16.010 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5074000000000005e-05 2024-09-04T17:46:16.010 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.010 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.010 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:16.011 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.376681+0000", 2024-09-04T17:46:16.013 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00012891399999999999 2024-09-04T17:46:16.013 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:16.013 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:16.013 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:16.013 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.013 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.013 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4799 3.1e 3:7817bfb4:::rbd_data.113417f3df25.0000000000000001:5 [read 2326528~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:16.013 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.350111+0000", 2024-09-04T17:46:16.013 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 8.5884453999999995, 2024-09-04T17:46:16.013 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.029614627000000001, 2024-09-04T17:46:16.013 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:16.014 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:16.014 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:16.014 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:16.014 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:16.014 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:16.014 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4799 2024-09-04T17:46:16.014 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.014 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:16.014 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.014 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:16.014 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350111+0000", 2024-09-04T17:46:16.014 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:16.015 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.015 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.015 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:16.015 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350111+0000", 2024-09-04T17:46:16.015 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:16.015 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.015 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.015 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:16.015 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350111+0000", 2024-09-04T17:46:16.015 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.9599999999999999e-07 2024-09-04T17:46:16.015 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.015 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.015 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:16.015 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350115+0000", 2024-09-04T17:46:16.016 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.506e-06 2024-09-04T17:46:16.016 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.016 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.016 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:16.016 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350126+0000", 2024-09-04T17:46:16.016 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0658e-05 2024-09-04T17:46:16.016 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.016 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.016 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:16.016 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350133+0000", 2024-09-04T17:46:16.016 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2130000000000002e-06 2024-09-04T17:46:16.016 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.017 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.017 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:16.017 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.376712+0000", 2024-09-04T17:46:16.017 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.026579479 2024-09-04T17:46:16.017 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.017 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.017 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:16.017 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.379533+0000", 2024-09-04T17:46:16.017 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0028206780000000001 2024-09-04T17:46:16.017 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.017 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.017 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:16.017 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.379725+0000", 2024-09-04T17:46:16.017 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019239699999999999 2024-09-04T17:46:16.018 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:16.018 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:16.018 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:16.018 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.018 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.018 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4800 3.1e 3:7817bfb4:::rbd_data.113417f3df25.0000000000000001:5 [read 2752512~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:16.018 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.350151+0000", 2024-09-04T17:46:16.018 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 8.5884047359999993, 2024-09-04T17:46:16.018 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.029858974999999999, 2024-09-04T17:46:16.018 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:16.018 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:16.018 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:16.018 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:16.019 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:16.019 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:16.019 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4800 2024-09-04T17:46:16.019 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.019 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:16.019 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.019 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:16.019 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350151+0000", 2024-09-04T17:46:16.019 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:16.019 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.019 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.019 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:16.019 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350151+0000", 2024-09-04T17:46:16.020 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:16.020 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.020 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.020 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:16.020 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350152+0000", 2024-09-04T17:46:16.020 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.4899999999999995e-07 2024-09-04T17:46:16.020 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.020 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.020 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:16.020 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350156+0000", 2024-09-04T17:46:16.020 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.6160000000000002e-06 2024-09-04T17:46:16.020 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.021 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.021 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:16.021 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350166+0000", 2024-09-04T17:46:16.021 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0168e-05 2024-09-04T17:46:16.021 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.021 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.021 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:16.021 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350173+0000", 2024-09-04T17:46:16.021 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6830000000000002e-06 2024-09-04T17:46:16.021 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.021 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.021 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:16.021 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.379767+0000", 2024-09-04T17:46:16.021 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.029594915999999999 2024-09-04T17:46:16.022 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.022 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.022 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:16.022 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.379864+0000", 2024-09-04T17:46:16.022 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.6164999999999998e-05 2024-09-04T17:46:16.022 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.022 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.022 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:16.022 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.380010+0000", 2024-09-04T17:46:16.022 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000146778 2024-09-04T17:46:16.022 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:16.022 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:16.022 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:16.023 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.023 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.023 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4801 3.1e 3:7817bfb4:::rbd_data.113417f3df25.0000000000000001:5 [read 3178496~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:16.023 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.350183+0000", 2024-09-04T17:46:16.023 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 8.5883728870000002, 2024-09-04T17:46:16.023 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.030089563999999999, 2024-09-04T17:46:16.023 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:16.023 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:16.023 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:16.023 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:16.023 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:16.023 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:16.023 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4801 2024-09-04T17:46:16.024 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.024 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:16.024 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.024 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:16.024 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350183+0000", 2024-09-04T17:46:16.024 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:16.024 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.024 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.024 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:16.024 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350183+0000", 2024-09-04T17:46:16.024 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:16.024 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.024 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.025 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:16.025 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350184+0000", 2024-09-04T17:46:16.025 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3399999999999998e-07 2024-09-04T17:46:16.025 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.025 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.025 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:16.025 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350187+0000", 2024-09-04T17:46:16.025 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4470000000000002e-06 2024-09-04T17:46:16.025 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.025 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.025 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:16.025 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350196+0000", 2024-09-04T17:46:16.025 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9530000000000005e-06 2024-09-04T17:46:16.026 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.026 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.026 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:16.026 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350203+0000", 2024-09-04T17:46:16.026 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.4160000000000001e-06 2024-09-04T17:46:16.026 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.026 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.026 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:16.026 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.380042+0000", 2024-09-04T17:46:16.026 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.029839099000000001 2024-09-04T17:46:16.026 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.026 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.026 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:16.026 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.380131+0000", 2024-09-04T17:46:16.027 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8857999999999995e-05 2024-09-04T17:46:16.027 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.027 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.027 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:16.027 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.380273+0000", 2024-09-04T17:46:16.027 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014205700000000001 2024-09-04T17:46:16.027 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:16.027 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:16.027 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:16.027 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.027 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.027 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4802 3.1e 3:7817bfb4:::rbd_data.113417f3df25.0000000000000001:5 [read 3604480~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:16.027 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.350221+0000", 2024-09-04T17:46:16.028 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 8.5883347059999995, 2024-09-04T17:46:16.028 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.030306293000000002, 2024-09-04T17:46:16.028 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:16.028 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:16.028 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:16.028 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:16.028 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:16.028 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:16.028 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4802 2024-09-04T17:46:16.028 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.028 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:16.028 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.028 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:16.028 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350221+0000", 2024-09-04T17:46:16.029 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:16.029 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.029 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.029 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:16.029 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350221+0000", 2024-09-04T17:46:16.029 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:16.029 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.029 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.029 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:16.029 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350223+0000", 2024-09-04T17:46:16.029 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1009999999999999e-06 2024-09-04T17:46:16.029 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.030 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.030 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:16.030 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350226+0000", 2024-09-04T17:46:16.030 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7000000000000002e-06 2024-09-04T17:46:16.030 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.030 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.030 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:16.030 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350236+0000", 2024-09-04T17:46:16.030 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5899999999999997e-06 2024-09-04T17:46:16.030 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.031 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.031 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:16.031 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350243+0000", 2024-09-04T17:46:16.031 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.5299999999999999e-06 2024-09-04T17:46:16.031 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.031 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.031 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:16.031 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.380303+0000", 2024-09-04T17:46:16.031 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.030060104000000001 2024-09-04T17:46:16.031 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.031 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.031 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:16.031 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.380390+0000", 2024-09-04T17:46:16.031 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.6420000000000003e-05 2024-09-04T17:46:16.032 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.032 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.032 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:16.032 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.380528+0000", 2024-09-04T17:46:16.032 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013784800000000001 2024-09-04T17:46:16.032 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:16.032 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:16.032 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:16.032 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.032 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.032 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4803 3.1e 3:7817bfb4:::rbd_data.113417f3df25.0000000000000001:5 [read 4030464~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:16.032 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.350254+0000", 2024-09-04T17:46:16.032 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 8.5883018540000009, 2024-09-04T17:46:16.033 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.030523386999999999, 2024-09-04T17:46:16.033 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:16.033 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:16.033 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:16.033 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:16.033 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:16.033 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:16.033 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4803 2024-09-04T17:46:16.033 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.033 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:16.033 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.033 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:16.034 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350254+0000", 2024-09-04T17:46:16.034 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:16.034 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.034 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.034 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:16.034 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350254+0000", 2024-09-04T17:46:16.034 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:16.034 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.034 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.034 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:16.034 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350255+0000", 2024-09-04T17:46:16.034 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.2799999999999996e-07 2024-09-04T17:46:16.034 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.034 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.035 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:16.035 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350258+0000", 2024-09-04T17:46:16.035 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.1300000000000001e-06 2024-09-04T17:46:16.035 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.035 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.035 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:16.035 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350278+0000", 2024-09-04T17:46:16.035 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9584000000000001e-05 2024-09-04T17:46:16.035 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.035 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.035 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:16.035 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.350285+0000", 2024-09-04T17:46:16.035 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3270000000000003e-06 2024-09-04T17:46:16.036 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.036 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.036 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:16.036 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.380558+0000", 2024-09-04T17:46:16.036 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.030273220999999999 2024-09-04T17:46:16.036 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.036 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.036 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:16.036 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.380640+0000", 2024-09-04T17:46:16.036 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.1772000000000002e-05 2024-09-04T17:46:16.036 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.036 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.036 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:16.037 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.380778+0000", 2024-09-04T17:46:16.037 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000137725 2024-09-04T17:46:16.037 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:16.037 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:16.037 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:16.037 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:16.037 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:16.037 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:16.038 INFO:teuthology.orchestra.run.smithi092.stdout:898 read 0x3090a44 thru 0x30a02db (0xf898 bytes) 2024-09-04T17:46:16.038 INFO:teuthology.orchestra.run.smithi092.stdout:899 read 0x437ff6f thru 0x438c5af (0xc641 bytes) 2024-09-04T17:46:16.039 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:46:16.040 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-04T17:46:16.040 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:46:16.035+0000 7f505d642640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26446) UID: 0 2024-09-04T17:46:16.040 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-04T17:46:16.041 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:46:16.041 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-04T17:46:16.041 INFO:teuthology.orchestra.run.smithi092.stdout:900 write 0xe301fdb thru 0xe311988 (0xf9ae bytes) 2024-09-04T17:46:16.041 INFO:teuthology.orchestra.run.smithi092.stdout:902 clone 9 order 24 su 262144 sc 12 2024-09-04T17:46:16.139 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:16.139+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:16.211 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:16.211 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:46:16.211 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:46:16.211 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:16.226 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-04T17:46:16.267 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:16.267+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:16.289 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is unset 2024-09-04T17:46:16.341 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:16.338+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:16.402 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:16.402 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:46:16.402 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:46:16.402 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:46:16.402 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:16.407 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone8 from 0xe311989 (overlap 0x10246f2) to 0xb4cb43 2024-09-04T17:46:16.421 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-04T17:46:16.442 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:16.442+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:16.543 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:16.543+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:16.547 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:16.547 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:46:16.547 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:46:16.547 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:46:16.548 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.548 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4219.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-04T17:46:16.548 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:45:42.487196+0000", 2024-09-04T17:46:16.548 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 34.060524047000001, 2024-09-04T17:46:16.548 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.021893024000000001, 2024-09-04T17:46:16.548 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:16.548 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:16.548 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:16.548 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:16.548 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4219", 2024-09-04T17:46:16.548 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.190:0/3705085728", 2024-09-04T17:46:16.549 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 50 2024-09-04T17:46:16.549 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.549 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:16.549 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.549 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:16.549 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487196+0000", 2024-09-04T17:46:16.549 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:16.549 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.549 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.550 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:16.550 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487196+0000", 2024-09-04T17:46:16.550 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:16.550 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.550 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.550 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:16.550 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487197+0000", 2024-09-04T17:46:16.550 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.06e-07 2024-09-04T17:46:16.550 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.550 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.551 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:16.551 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487270+0000", 2024-09-04T17:46:16.551 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2992999999999997e-05 2024-09-04T17:46:16.552 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.552 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.552 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:16.552 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487281+0000", 2024-09-04T17:46:16.552 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.13e-05 2024-09-04T17:46:16.552 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.552 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.552 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:16.553 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487289+0000", 2024-09-04T17:46:16.553 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.8429999999999993e-06 2024-09-04T17:46:16.553 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.553 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.553 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:16.553 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.504017+0000", 2024-09-04T17:46:16.553 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.016728481 2024-09-04T17:46:16.553 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.553 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.553 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:16.553 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.504153+0000", 2024-09-04T17:46:16.554 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000135396 2024-09-04T17:46:16.554 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.554 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.554 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 0", 2024-09-04T17:46:16.554 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.504378+0000", 2024-09-04T17:46:16.554 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000225255 2024-09-04T17:46:16.554 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.554 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.554 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:16.554 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.508835+0000", 2024-09-04T17:46:16.554 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0044573759999999999 2024-09-04T17:46:16.555 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.555 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.555 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:16.555 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.508936+0000", 2024-09-04T17:46:16.555 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000100555 2024-09-04T17:46:16.555 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.555 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.555 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:16.555 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.509021+0000", 2024-09-04T17:46:16.555 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.4546e-05 2024-09-04T17:46:16.555 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.556 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.556 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:16.556 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.509089+0000", 2024-09-04T17:46:16.556 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.8572999999999996e-05 2024-09-04T17:46:16.556 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:16.556 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:16.556 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:16.556 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.556 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.556 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4219.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-04T17:46:16.557 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:45:42.487303+0000", 2024-09-04T17:46:16.557 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 34.060417217999998, 2024-09-04T17:46:16.557 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.022763057, 2024-09-04T17:46:16.557 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:16.557 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:16.557 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:16.557 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:16.557 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4219", 2024-09-04T17:46:16.557 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.190:0/3705085728", 2024-09-04T17:46:16.557 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 51 2024-09-04T17:46:16.557 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.558 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:16.558 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.558 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:16.558 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487303+0000", 2024-09-04T17:46:16.558 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:16.558 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.558 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.558 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:16.558 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487303+0000", 2024-09-04T17:46:16.558 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:16.558 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.559 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.559 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:16.559 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487304+0000", 2024-09-04T17:46:16.559 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.7000000000000004e-07 2024-09-04T17:46:16.559 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.559 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.559 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:16.559 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487313+0000", 2024-09-04T17:46:16.559 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8200000000000003e-06 2024-09-04T17:46:16.559 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.559 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.559 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:16.560 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487322+0000", 2024-09-04T17:46:16.560 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4199999999999996e-06 2024-09-04T17:46:16.560 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.560 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.560 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:16.560 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487329+0000", 2024-09-04T17:46:16.560 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2599999999999999e-06 2024-09-04T17:46:16.560 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.560 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.560 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:16.560 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.506432+0000", 2024-09-04T17:46:16.561 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.019102592000000002 2024-09-04T17:46:16.561 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.561 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.561 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:16.561 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.506570+0000", 2024-09-04T17:46:16.561 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013837300000000001 2024-09-04T17:46:16.561 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.561 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.561 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 0", 2024-09-04T17:46:16.561 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.506771+0000", 2024-09-04T17:46:16.561 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000200748 2024-09-04T17:46:16.561 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.562 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.562 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:16.562 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.508859+0000", 2024-09-04T17:46:16.562 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0020881010000000002 2024-09-04T17:46:16.562 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.562 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.562 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:16.562 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.509919+0000", 2024-09-04T17:46:16.562 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0010601580000000001 2024-09-04T17:46:16.562 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.562 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.562 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:16.562 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.510002+0000", 2024-09-04T17:46:16.563 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.2719999999999994e-05 2024-09-04T17:46:16.563 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.563 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.563 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:16.563 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.510066+0000", 2024-09-04T17:46:16.563 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.4095000000000002e-05 2024-09-04T17:46:16.563 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:16.563 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:16.563 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:16.563 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.563 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.563 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4219.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-04T17:46:16.563 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:45:42.487345+0000", 2024-09-04T17:46:16.563 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 34.060375284999999, 2024-09-04T17:46:16.564 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.025395999999999998, 2024-09-04T17:46:16.564 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:16.564 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:16.564 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:16.564 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:16.564 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4219", 2024-09-04T17:46:16.564 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.190:0/3705085728", 2024-09-04T17:46:16.564 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 52 2024-09-04T17:46:16.564 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.564 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:16.564 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.564 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:16.564 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487345+0000", 2024-09-04T17:46:16.565 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:16.565 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.565 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.565 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:16.565 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487345+0000", 2024-09-04T17:46:16.565 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:16.565 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.565 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.565 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:16.565 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487346+0000", 2024-09-04T17:46:16.565 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.4600000000000004e-07 2024-09-04T17:46:16.565 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.566 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.566 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:16.566 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487351+0000", 2024-09-04T17:46:16.566 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.9399999999999999e-06 2024-09-04T17:46:16.566 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.566 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.566 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:16.566 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487361+0000", 2024-09-04T17:46:16.566 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.3840000000000003e-06 2024-09-04T17:46:16.566 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.566 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.566 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:16.566 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.487368+0000", 2024-09-04T17:46:16.566 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1729999999999998e-06 2024-09-04T17:46:16.567 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.567 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.567 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:16.567 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.507626+0000", 2024-09-04T17:46:16.567 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.020257547000000001 2024-09-04T17:46:16.567 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.567 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.567 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:16.567 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.507861+0000", 2024-09-04T17:46:16.567 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000235708 2024-09-04T17:46:16.567 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.567 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.567 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 0", 2024-09-04T17:46:16.568 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.508188+0000", 2024-09-04T17:46:16.568 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00032624800000000001 2024-09-04T17:46:16.568 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.568 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.568 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:16.568 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.509521+0000", 2024-09-04T17:46:16.568 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.001333433 2024-09-04T17:46:16.568 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.568 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.568 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:16.568 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.512613+0000", 2024-09-04T17:46:16.568 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0030923919999999998 2024-09-04T17:46:16.569 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.569 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.569 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:16.569 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.512686+0000", 2024-09-04T17:46:16.569 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3107000000000003e-05 2024-09-04T17:46:16.569 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.569 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.569 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:16.569 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:45:42.512741+0000", 2024-09-04T17:46:16.569 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4422000000000002e-05 2024-09-04T17:46:16.569 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:16.570 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:16.570 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:16.570 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.570 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.570 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:1430 3.f 3:f58475cb:::rbd_data.11346251f094.0000000000000074:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-04T17:46:16.570 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:02.778704+0000", 2024-09-04T17:46:16.570 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 13.769016183, 2024-09-04T17:46:16.570 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.058567801000000003, 2024-09-04T17:46:16.570 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:16.570 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:16.570 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:16.570 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:16.570 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:16.570 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:16.571 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 1430 2024-09-04T17:46:16.571 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.571 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:16.571 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.571 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:16.571 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.778704+0000", 2024-09-04T17:46:16.571 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:16.571 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.571 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.571 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:16.571 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.778704+0000", 2024-09-04T17:46:16.571 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:16.571 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.571 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.572 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:16.572 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.778707+0000", 2024-09-04T17:46:16.572 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.8329999999999998e-06 2024-09-04T17:46:16.572 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.572 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.572 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:16.572 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.778718+0000", 2024-09-04T17:46:16.572 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.152e-05 2024-09-04T17:46:16.572 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.572 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.572 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:16.572 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.778750+0000", 2024-09-04T17:46:16.573 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.1696999999999998e-05 2024-09-04T17:46:16.573 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.573 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.573 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:16.573 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.778765+0000", 2024-09-04T17:46:16.573 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4800000000000001e-05 2024-09-04T17:46:16.573 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.573 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.573 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:16.573 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.778894+0000", 2024-09-04T17:46:16.573 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00012928500000000001 2024-09-04T17:46:16.573 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.573 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.574 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:16.574 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.779030+0000", 2024-09-04T17:46:16.574 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013566899999999999 2024-09-04T17:46:16.574 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.574 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.574 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 4", 2024-09-04T17:46:16.574 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.788794+0000", 2024-09-04T17:46:16.574 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.009764478 2024-09-04T17:46:16.574 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.574 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.574 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:16.574 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.836969+0000", 2024-09-04T17:46:16.574 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.048174626999999998 2024-09-04T17:46:16.575 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.575 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.575 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:16.575 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.837090+0000", 2024-09-04T17:46:16.575 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000120627 2024-09-04T17:46:16.575 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.575 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.575 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:16.575 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.837195+0000", 2024-09-04T17:46:16.575 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000105206 2024-09-04T17:46:16.575 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.575 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.575 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:16.576 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.837272+0000", 2024-09-04T17:46:16.576 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.7058999999999995e-05 2024-09-04T17:46:16.576 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:16.576 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:16.576 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:16.576 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.576 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.576 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4397 3.f 3:f7aed845:::rbd_data.1134cfbce83d.0000000000000007:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:16.576 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.341682+0000", 2024-09-04T17:46:16.576 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 9.2060383699999999, 2024-09-04T17:46:16.576 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.03043676, 2024-09-04T17:46:16.577 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:16.577 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:16.577 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:16.577 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:16.577 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:16.577 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:16.577 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4397 2024-09-04T17:46:16.577 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.577 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:16.577 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.577 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:16.577 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.341682+0000", 2024-09-04T17:46:16.578 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:16.578 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.578 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.578 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:16.578 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.341682+0000", 2024-09-04T17:46:16.578 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:16.578 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.578 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.578 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:16.578 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.341686+0000", 2024-09-04T17:46:16.578 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7799999999999998e-06 2024-09-04T17:46:16.578 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.578 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.578 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:16.579 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.341698+0000", 2024-09-04T17:46:16.579 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2478e-05 2024-09-04T17:46:16.579 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.579 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.579 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:16.579 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.341729+0000", 2024-09-04T17:46:16.579 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0666000000000001e-05 2024-09-04T17:46:16.579 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.579 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.579 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:16.579 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.341745+0000", 2024-09-04T17:46:16.579 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6490999999999998e-05 2024-09-04T17:46:16.580 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.580 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.580 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:16.580 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.342209+0000", 2024-09-04T17:46:16.580 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000463559 2024-09-04T17:46:16.580 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.580 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.580 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op must wait for map", 2024-09-04T17:46:16.580 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.342218+0000", 2024-09-04T17:46:16.580 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.3880000000000006e-06 2024-09-04T17:46:16.580 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.580 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.581 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:16.581 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.351602+0000", 2024-09-04T17:46:16.581 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0093833330000000006 2024-09-04T17:46:16.581 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.581 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.581 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:16.581 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.351774+0000", 2024-09-04T17:46:16.581 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00017225300000000001 2024-09-04T17:46:16.581 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.581 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.581 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:16.581 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.372119+0000", 2024-09-04T17:46:16.581 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.020344812 2024-09-04T17:46:16.582 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:16.582 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:16.582 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:16.582 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.582 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.582 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4848 3.15 3:ab4caf76:::rbd_data.1134cfbce83d.000000000000000a:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:16.582 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.365766+0000", 2024-09-04T17:46:16.582 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 9.181954181, 2024-09-04T17:46:16.582 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.020025887999999999, 2024-09-04T17:46:16.582 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:16.582 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:16.582 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:16.583 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:16.583 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:16.583 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:16.583 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4848 2024-09-04T17:46:16.583 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.583 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:16.583 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.583 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:16.583 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.365766+0000", 2024-09-04T17:46:16.583 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:16.583 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.583 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.584 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:16.584 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.365766+0000", 2024-09-04T17:46:16.584 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:16.584 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.584 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.584 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:16.584 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.365767+0000", 2024-09-04T17:46:16.584 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.1099999999999995e-07 2024-09-04T17:46:16.584 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.584 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.584 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:16.584 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.365770+0000", 2024-09-04T17:46:16.584 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.9069999999999999e-06 2024-09-04T17:46:16.585 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.585 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.585 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:16.585 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.365779+0000", 2024-09-04T17:46:16.585 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4110000000000002e-06 2024-09-04T17:46:16.585 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.585 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.585 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:16.585 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.365786+0000", 2024-09-04T17:46:16.585 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2259999999999999e-06 2024-09-04T17:46:16.585 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.585 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.585 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:16.586 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.381324+0000", 2024-09-04T17:46:16.586 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.015537611999999999 2024-09-04T17:46:16.586 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.586 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.586 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:16.586 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.381459+0000", 2024-09-04T17:46:16.586 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013517100000000001 2024-09-04T17:46:16.586 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.586 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.586 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:16.586 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.385792+0000", 2024-09-04T17:46:16.586 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0043329500000000003 2024-09-04T17:46:16.586 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:16.587 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:16.587 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:16.587 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.587 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.587 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:5015 3.f 3:f7aed845:::rbd_data.1134cfbce83d.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:16.587 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.412372+0000", 2024-09-04T17:46:16.587 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 9.1353479360000005, 2024-09-04T17:46:16.587 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.021470176000000001, 2024-09-04T17:46:16.587 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:16.587 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:16.587 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:16.587 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:16.588 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:16.588 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:16.588 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 5015 2024-09-04T17:46:16.588 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.588 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:16.588 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.588 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:16.588 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.412372+0000", 2024-09-04T17:46:16.588 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:16.588 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.588 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.589 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:16.589 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.412372+0000", 2024-09-04T17:46:16.589 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:16.589 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.589 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.589 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:16.589 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.412374+0000", 2024-09-04T17:46:16.589 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.815e-06 2024-09-04T17:46:16.589 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.589 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.589 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:16.589 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.419455+0000", 2024-09-04T17:46:16.589 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.007080607 2024-09-04T17:46:16.590 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.590 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.590 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:16.590 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.419476+0000", 2024-09-04T17:46:16.590 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1314e-05 2024-09-04T17:46:16.590 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.590 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.590 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:16.590 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.419488+0000", 2024-09-04T17:46:16.590 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2283000000000001e-05 2024-09-04T17:46:16.590 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.590 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.591 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:16.591 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.419579+0000", 2024-09-04T17:46:16.591 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.0433000000000006e-05 2024-09-04T17:46:16.591 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.591 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.591 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:16.591 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.419718+0000", 2024-09-04T17:46:16.591 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000139247 2024-09-04T17:46:16.591 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.591 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.591 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 4", 2024-09-04T17:46:16.591 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.420183+0000", 2024-09-04T17:46:16.591 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00046540299999999998 2024-09-04T17:46:16.591 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.592 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.592 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:16.592 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.431643+0000", 2024-09-04T17:46:16.592 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.011459741000000001 2024-09-04T17:46:16.592 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.592 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.592 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:16.592 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.433742+0000", 2024-09-04T17:46:16.592 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.002098964 2024-09-04T17:46:16.592 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.592 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.592 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:16.593 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.433813+0000", 2024-09-04T17:46:16.593 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1147000000000002e-05 2024-09-04T17:46:16.593 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.593 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.593 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:16.593 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.433842+0000", 2024-09-04T17:46:16.593 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.9221999999999999e-05 2024-09-04T17:46:16.593 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:16.593 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:16.593 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:16.593 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.593 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.594 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:5386 3.f 3:f4098008:::rbd_data.113431586065.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 8912896~5917] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:16.594 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.724609+0000", 2024-09-04T17:46:16.594 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 8.8231107869999992, 2024-09-04T17:46:16.594 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.020913503, 2024-09-04T17:46:16.594 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:16.594 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:16.594 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:16.594 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:16.594 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:16.594 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:16.594 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 5386 2024-09-04T17:46:16.594 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.595 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:16.595 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.595 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:16.595 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.724609+0000", 2024-09-04T17:46:16.595 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:16.595 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.595 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.595 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:16.595 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.724609+0000", 2024-09-04T17:46:16.595 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:16.595 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.595 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.595 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:16.596 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.724613+0000", 2024-09-04T17:46:16.596 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.2830000000000002e-06 2024-09-04T17:46:16.596 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.596 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.596 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:16.596 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.731404+0000", 2024-09-04T17:46:16.596 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.006791315 2024-09-04T17:46:16.596 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.596 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.596 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:16.596 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.731439+0000", 2024-09-04T17:46:16.596 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4932999999999999e-05 2024-09-04T17:46:16.597 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.597 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.597 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:16.597 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.731456+0000", 2024-09-04T17:46:16.597 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7535999999999999e-05 2024-09-04T17:46:16.597 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.597 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.597 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:16.597 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.731565+0000", 2024-09-04T17:46:16.597 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000108583 2024-09-04T17:46:16.597 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.597 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.598 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:16.598 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.731790+0000", 2024-09-04T17:46:16.598 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000225212 2024-09-04T17:46:16.598 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.598 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.598 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 4", 2024-09-04T17:46:16.598 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.732421+0000", 2024-09-04T17:46:16.598 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00063093399999999999 2024-09-04T17:46:16.598 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.598 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.598 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:16.598 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.745027+0000", 2024-09-04T17:46:16.599 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.012606009 2024-09-04T17:46:16.599 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.599 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.599 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:16.599 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.745421+0000", 2024-09-04T17:46:16.599 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00039343199999999998 2024-09-04T17:46:16.599 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.599 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.599 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:16.599 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.745474+0000", 2024-09-04T17:46:16.599 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.3536999999999999e-05 2024-09-04T17:46:16.599 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.599 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.600 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:16.600 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.745523+0000", 2024-09-04T17:46:16.600 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.8729000000000001e-05 2024-09-04T17:46:16.600 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:16.600 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:16.600 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:16.600 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.600 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.600 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:5751 3.d 3:b1ae20f2:::rbd_data.113431586065.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:16.600 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.947425+0000", 2024-09-04T17:46:16.600 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 8.6002948309999994, 2024-09-04T17:46:16.601 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.037571597999999998, 2024-09-04T17:46:16.601 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:16.601 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:16.601 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:16.601 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:16.601 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:16.601 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:16.601 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 5751 2024-09-04T17:46:16.601 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.601 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:16.601 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.601 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:16.602 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.947425+0000", 2024-09-04T17:46:16.602 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:16.602 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.602 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.602 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:16.602 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.947425+0000", 2024-09-04T17:46:16.602 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:16.602 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.602 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.602 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:16.602 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.947428+0000", 2024-09-04T17:46:16.602 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.74e-06 2024-09-04T17:46:16.603 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.603 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.603 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:16.603 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.951927+0000", 2024-09-04T17:46:16.603 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0044988930000000003 2024-09-04T17:46:16.603 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.603 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.603 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:16.603 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.951963+0000", 2024-09-04T17:46:16.603 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.5902999999999998e-05 2024-09-04T17:46:16.603 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.603 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.604 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:16.604 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.951986+0000", 2024-09-04T17:46:16.604 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.266e-05 2024-09-04T17:46:16.604 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.604 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.604 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:16.604 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.952065+0000", 2024-09-04T17:46:16.604 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.9668999999999999e-05 2024-09-04T17:46:16.604 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.604 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.604 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:16.604 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.952166+0000", 2024-09-04T17:46:16.604 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000101078 2024-09-04T17:46:16.605 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.605 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.605 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 5", 2024-09-04T17:46:16.605 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.955910+0000", 2024-09-04T17:46:16.605 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.003743732 2024-09-04T17:46:16.605 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.605 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.605 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:16.605 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.983683+0000", 2024-09-04T17:46:16.605 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.027773036000000001 2024-09-04T17:46:16.605 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.605 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.605 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:16.606 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.984827+0000", 2024-09-04T17:46:16.606 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0011435449999999999 2024-09-04T17:46:16.606 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.606 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.606 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:16.606 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.984888+0000", 2024-09-04T17:46:16.606 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.122e-05 2024-09-04T17:46:16.606 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.606 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.606 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:16.606 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.984997+0000", 2024-09-04T17:46:16.606 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000109122 2024-09-04T17:46:16.607 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:16.607 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:16.607 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:16.607 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.607 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.607 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:6147 3.d 3:b692b231:::rbd_data.1134cfbce83d.000000000000000c:6 [sparse-read 1343488~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:16.607 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:08.430599+0000", 2024-09-04T17:46:16.607 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 8.1171216259999994, 2024-09-04T17:46:16.607 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.019355033000000001, 2024-09-04T17:46:16.607 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:16.607 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:16.607 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:16.608 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:16.608 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:16.608 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:16.608 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 6147 2024-09-04T17:46:16.608 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.608 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:16.608 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.608 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:16.608 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.430599+0000", 2024-09-04T17:46:16.608 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:16.608 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.608 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.608 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:16.609 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.430599+0000", 2024-09-04T17:46:16.609 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:16.609 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.609 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.609 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:16.609 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.430600+0000", 2024-09-04T17:46:16.609 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1999999999999999e-06 2024-09-04T17:46:16.609 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.609 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.609 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:16.609 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.430614+0000", 2024-09-04T17:46:16.609 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4353e-05 2024-09-04T17:46:16.610 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.610 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.610 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:16.610 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.430634+0000", 2024-09-04T17:46:16.610 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.961e-05 2024-09-04T17:46:16.610 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.610 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.610 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:16.610 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.430648+0000", 2024-09-04T17:46:16.610 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4073000000000001e-05 2024-09-04T17:46:16.610 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.610 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.610 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:16.611 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.449011+0000", 2024-09-04T17:46:16.611 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.018362929 2024-09-04T17:46:16.611 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.611 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.611 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:16.611 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.449215+0000", 2024-09-04T17:46:16.611 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020451099999999999 2024-09-04T17:46:16.611 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.611 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.611 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:16.611 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.449954+0000", 2024-09-04T17:46:16.611 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00073835700000000003 2024-09-04T17:46:16.612 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:16.612 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:16.612 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:16.612 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.612 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.612 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:6148 3.d 3:b692b231:::rbd_data.1134cfbce83d.000000000000000c:6 [sparse-read 1703936~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:16.612 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:08.430668+0000", 2024-09-04T17:46:16.612 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 8.1170526140000003, 2024-09-04T17:46:16.612 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.020109115, 2024-09-04T17:46:16.612 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:16.612 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:16.612 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:16.612 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:16.613 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:16.613 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:16.613 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 6148 2024-09-04T17:46:16.613 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.613 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:16.613 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.613 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:16.613 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.430668+0000", 2024-09-04T17:46:16.613 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:16.613 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.614 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.614 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:16.614 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.430668+0000", 2024-09-04T17:46:16.614 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:16.614 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.614 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.614 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:16.614 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.430669+0000", 2024-09-04T17:46:16.614 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.234e-06 2024-09-04T17:46:16.614 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.614 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.614 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:16.614 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.430675+0000", 2024-09-04T17:46:16.615 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7359999999999998e-06 2024-09-04T17:46:16.615 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.615 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.615 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:16.615 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.430694+0000", 2024-09-04T17:46:16.615 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9283000000000002e-05 2024-09-04T17:46:16.615 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.615 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.615 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:16.615 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.430708+0000", 2024-09-04T17:46:16.615 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4324e-05 2024-09-04T17:46:16.615 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.615 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.616 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:16.616 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.450182+0000", 2024-09-04T17:46:16.616 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.019473796000000002 2024-09-04T17:46:16.616 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.616 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.616 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:16.616 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.450292+0000", 2024-09-04T17:46:16.616 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000110302 2024-09-04T17:46:16.616 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.616 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.616 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:16.616 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.450777+0000", 2024-09-04T17:46:16.617 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00048443999999999999 2024-09-04T17:46:16.617 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:16.617 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:16.617 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:16.617 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.617 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.617 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:6239 3.15 3:ab4caf76:::rbd_data.1134cfbce83d.000000000000000a:6 [read 688128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:16.617 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:08.431477+0000", 2024-09-04T17:46:16.617 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 8.1162434270000006, 2024-09-04T17:46:16.617 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.019689993999999999, 2024-09-04T17:46:16.617 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:16.617 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:16.617 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:16.618 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:16.618 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:16.618 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:16.618 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 6239 2024-09-04T17:46:16.618 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.618 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:16.618 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.618 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:16.618 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.431477+0000", 2024-09-04T17:46:16.618 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:16.618 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.618 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.618 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:16.618 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.431477+0000", 2024-09-04T17:46:16.619 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:16.619 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.619 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.619 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:16.619 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.431478+0000", 2024-09-04T17:46:16.619 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.207e-06 2024-09-04T17:46:16.619 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.619 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.619 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:16.619 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.431483+0000", 2024-09-04T17:46:16.619 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2900000000000002e-06 2024-09-04T17:46:16.619 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.619 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.620 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:16.620 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.431502+0000", 2024-09-04T17:46:16.620 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8853000000000001e-05 2024-09-04T17:46:16.620 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.620 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.620 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:16.620 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.431516+0000", 2024-09-04T17:46:16.620 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.341e-05 2024-09-04T17:46:16.620 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.620 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.620 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:16.620 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.450643+0000", 2024-09-04T17:46:16.621 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.019127613000000002 2024-09-04T17:46:16.621 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.621 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.621 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:16.621 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.450721+0000", 2024-09-04T17:46:16.621 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.7966999999999994e-05 2024-09-04T17:46:16.621 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.621 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.621 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:16.621 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.451167+0000", 2024-09-04T17:46:16.621 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00044565400000000003 2024-09-04T17:46:16.621 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:16.621 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:16.622 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:16.622 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.622 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.622 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:6240 3.15 3:ab4caf76:::rbd_data.1134cfbce83d.000000000000000a:6 [read 1048576~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:16.622 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:08.431534+0000", 2024-09-04T17:46:16.622 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 8.1161861169999998, 2024-09-04T17:46:16.622 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.020184402000000001, 2024-09-04T17:46:16.622 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:16.622 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:16.622 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:16.622 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:16.622 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:16.622 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:16.623 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 6240 2024-09-04T17:46:16.623 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.623 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:16.623 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.623 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:16.623 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.431534+0000", 2024-09-04T17:46:16.623 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:16.623 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.623 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.623 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:16.623 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.431534+0000", 2024-09-04T17:46:16.624 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:16.624 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.624 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.624 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:16.624 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.431535+0000", 2024-09-04T17:46:16.624 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.15e-06 2024-09-04T17:46:16.624 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.624 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.624 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:16.624 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.431540+0000", 2024-09-04T17:46:16.624 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1699999999999996e-06 2024-09-04T17:46:16.624 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.624 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.625 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:16.625 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.431559+0000", 2024-09-04T17:46:16.625 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.84e-05 2024-09-04T17:46:16.625 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.625 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.625 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:16.625 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.431572+0000", 2024-09-04T17:46:16.625 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.359e-05 2024-09-04T17:46:16.625 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.625 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.625 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:16.625 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.451221+0000", 2024-09-04T17:46:16.625 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.019648697999999999 2024-09-04T17:46:16.625 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.626 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.626 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:16.626 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.451317+0000", 2024-09-04T17:46:16.626 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5444000000000003e-05 2024-09-04T17:46:16.626 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.626 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.626 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:16.626 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.451718+0000", 2024-09-04T17:46:16.626 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00040194999999999999 2024-09-04T17:46:16.626 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:16.626 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:16.626 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:16.626 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.627 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.627 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:6241 3.15 3:ab4caf76:::rbd_data.1134cfbce83d.000000000000000a:6 [read 1409024~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:16.627 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:08.431591+0000", 2024-09-04T17:46:16.627 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 8.1161296410000006, 2024-09-04T17:46:16.627 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.020692640000000002, 2024-09-04T17:46:16.627 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:16.627 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:16.627 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:16.627 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:16.627 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:16.627 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:16.627 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 6241 2024-09-04T17:46:16.627 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.627 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:16.628 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.628 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:16.628 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.431591+0000", 2024-09-04T17:46:16.628 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:16.628 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.628 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.628 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:16.628 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.431591+0000", 2024-09-04T17:46:16.628 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:16.628 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.628 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.628 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:16.629 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.431592+0000", 2024-09-04T17:46:16.629 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1570000000000001e-06 2024-09-04T17:46:16.629 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.629 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.629 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:16.629 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.431597+0000", 2024-09-04T17:46:16.629 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.3770000000000004e-06 2024-09-04T17:46:16.629 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.629 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.629 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:16.629 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.431618+0000", 2024-09-04T17:46:16.629 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0945999999999999e-05 2024-09-04T17:46:16.630 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.630 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.630 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:16.630 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.431632+0000", 2024-09-04T17:46:16.630 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3787e-05 2024-09-04T17:46:16.630 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.630 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.630 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:16.630 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.451782+0000", 2024-09-04T17:46:16.630 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.020150314999999999 2024-09-04T17:46:16.630 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.630 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.630 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:16.630 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.451884+0000", 2024-09-04T17:46:16.631 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000101821 2024-09-04T17:46:16.631 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.631 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.631 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:16.631 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.452283+0000", 2024-09-04T17:46:16.631 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00039923699999999999 2024-09-04T17:46:16.631 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:16.631 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:16.631 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:16.631 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.631 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.631 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:6242 3.15 3:ab4caf76:::rbd_data.1134cfbce83d.000000000000000a:6 [read 1769472~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:16.631 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:08.431650+0000", 2024-09-04T17:46:16.631 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 8.1160702740000001, 2024-09-04T17:46:16.632 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.021284912999999999, 2024-09-04T17:46:16.632 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:16.632 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:16.632 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:16.632 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:16.632 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:16.632 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:16.632 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 6242 2024-09-04T17:46:16.632 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.632 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:16.632 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.632 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:16.632 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.431650+0000", 2024-09-04T17:46:16.632 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:16.633 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.633 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.633 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:16.633 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.431650+0000", 2024-09-04T17:46:16.633 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:16.633 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.633 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.633 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:16.633 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.431651+0000", 2024-09-04T17:46:16.633 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.26e-06 2024-09-04T17:46:16.633 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.633 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.633 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:16.633 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.431657+0000", 2024-09-04T17:46:16.634 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.5999999999999997e-06 2024-09-04T17:46:16.634 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.634 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.634 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:16.634 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.434576+0000", 2024-09-04T17:46:16.634 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.002918861 2024-09-04T17:46:16.634 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.634 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.634 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:16.634 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.434588+0000", 2024-09-04T17:46:16.634 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2121e-05 2024-09-04T17:46:16.634 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.634 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.634 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:16.635 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.452339+0000", 2024-09-04T17:46:16.635 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.017751169000000001 2024-09-04T17:46:16.635 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.635 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.635 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:16.635 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.452476+0000", 2024-09-04T17:46:16.635 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000137436 2024-09-04T17:46:16.635 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.635 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.635 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:16.635 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.452935+0000", 2024-09-04T17:46:16.635 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00045846599999999998 2024-09-04T17:46:16.635 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:16.636 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:16.636 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:16.636 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.636 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.636 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:6250 3.d 3:b692b231:::rbd_data.1134cfbce83d.000000000000000c:6 [read 1048576~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:16.636 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:08.434667+0000", 2024-09-04T17:46:16.636 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 8.1130536739999997, 2024-09-04T17:46:16.636 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.01935344, 2024-09-04T17:46:16.636 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:16.636 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:16.636 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:16.636 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:16.636 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:16.637 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:16.637 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 6250 2024-09-04T17:46:16.637 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.637 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:16.637 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.637 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:16.637 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.434667+0000", 2024-09-04T17:46:16.637 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:16.637 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.637 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.637 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:16.637 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.434667+0000", 2024-09-04T17:46:16.637 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:16.637 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.638 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.638 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:16.638 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.434667+0000", 2024-09-04T17:46:16.638 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2399999999999998e-07 2024-09-04T17:46:16.638 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.638 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.638 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:16.638 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.434669+0000", 2024-09-04T17:46:16.638 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.367e-06 2024-09-04T17:46:16.638 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.638 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.638 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:16.638 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.434676+0000", 2024-09-04T17:46:16.638 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7429999999999996e-06 2024-09-04T17:46:16.639 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.639 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.639 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:16.639 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.434682+0000", 2024-09-04T17:46:16.639 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4199999999999998e-06 2024-09-04T17:46:16.639 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.639 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.639 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:16.639 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.453639+0000", 2024-09-04T17:46:16.639 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.018957206000000001 2024-09-04T17:46:16.639 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.639 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.640 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:16.640 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.453709+0000", 2024-09-04T17:46:16.640 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.0576999999999999e-05 2024-09-04T17:46:16.640 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.640 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.640 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:16.640 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.454020+0000", 2024-09-04T17:46:16.640 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00031060300000000001 2024-09-04T17:46:16.640 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:16.640 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:16.640 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:16.640 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.641 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.641 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:6251 3.d 3:b692b231:::rbd_data.1134cfbce83d.000000000000000c:6 [read 1409024~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:16.641 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:08.434695+0000", 2024-09-04T17:46:16.641 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 8.1130254849999996, 2024-09-04T17:46:16.641 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.019817794999999999, 2024-09-04T17:46:16.641 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:16.641 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:16.641 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:16.641 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:16.641 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:16.641 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:16.641 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 6251 2024-09-04T17:46:16.641 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.642 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:16.642 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.642 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:16.642 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.434695+0000", 2024-09-04T17:46:16.642 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:16.642 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.642 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.642 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:16.642 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.434695+0000", 2024-09-04T17:46:16.642 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:16.642 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.642 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.642 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:16.643 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.434695+0000", 2024-09-04T17:46:16.643 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2200000000000004e-07 2024-09-04T17:46:16.643 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.643 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.643 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:16.643 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.434698+0000", 2024-09-04T17:46:16.643 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.988e-06 2024-09-04T17:46:16.643 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.643 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.643 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:16.644 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.434705+0000", 2024-09-04T17:46:16.644 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.8510000000000001e-06 2024-09-04T17:46:16.644 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.644 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.644 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:16.644 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.434710+0000", 2024-09-04T17:46:16.644 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.3410000000000002e-06 2024-09-04T17:46:16.644 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.644 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.644 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:16.644 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.454071+0000", 2024-09-04T17:46:16.644 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.019360863999999998 2024-09-04T17:46:16.644 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.645 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.645 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:16.645 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.454161+0000", 2024-09-04T17:46:16.645 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9964999999999997e-05 2024-09-04T17:46:16.645 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.645 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.645 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:16.645 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.454512+0000", 2024-09-04T17:46:16.645 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000351264 2024-09-04T17:46:16.645 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:16.645 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:16.646 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:16.646 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.646 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.646 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:6252 3.d 3:b692b231:::rbd_data.1134cfbce83d.000000000000000c:6 [read 1769472~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:16.646 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:08.434718+0000", 2024-09-04T17:46:16.646 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 8.1130019880000006, 2024-09-04T17:46:16.646 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.020239798, 2024-09-04T17:46:16.646 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:16.646 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:16.646 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:16.646 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:16.646 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:16.646 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:16.647 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 6252 2024-09-04T17:46:16.647 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.647 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:16.647 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.647 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:16.647 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.434718+0000", 2024-09-04T17:46:16.647 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:16.647 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.647 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.647 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:16.647 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.434718+0000", 2024-09-04T17:46:16.647 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:16.647 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.647 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.648 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:16.648 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.434719+0000", 2024-09-04T17:46:16.648 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0100000000000005e-07 2024-09-04T17:46:16.648 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.648 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.648 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:16.648 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.434721+0000", 2024-09-04T17:46:16.648 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6240000000000002e-06 2024-09-04T17:46:16.648 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.648 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.648 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:16.648 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.434736+0000", 2024-09-04T17:46:16.648 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4302e-05 2024-09-04T17:46:16.648 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.648 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.649 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:16.649 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.434741+0000", 2024-09-04T17:46:16.649 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.3070000000000002e-06 2024-09-04T17:46:16.649 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.649 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.649 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:16.649 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.454554+0000", 2024-09-04T17:46:16.649 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.019813239 2024-09-04T17:46:16.649 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.649 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.649 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:16.649 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.454610+0000", 2024-09-04T17:46:16.649 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6289999999999998e-05 2024-09-04T17:46:16.650 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.650 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.650 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:16.650 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.454958+0000", 2024-09-04T17:46:16.650 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000347535 2024-09-04T17:46:16.650 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:16.650 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:16.650 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:16.650 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.650 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.650 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:6295 3.d 3:b1ae20f2:::rbd_data.113431586065.0000000000000003:head [stat,truncate 1572864] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:16.650 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:08.434914+0000", 2024-09-04T17:46:16.650 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 8.1128059080000003, 2024-09-04T17:46:16.650 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.025484347000000001, 2024-09-04T17:46:16.650 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:16.651 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:16.651 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:16.651 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:16.651 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:16.651 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:16.651 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 6295 2024-09-04T17:46:16.651 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.651 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:16.651 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.651 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:16.651 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.434914+0000", 2024-09-04T17:46:16.651 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:16.651 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.651 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.652 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:16.652 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.434914+0000", 2024-09-04T17:46:16.652 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:16.652 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.652 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.652 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:16.652 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.434915+0000", 2024-09-04T17:46:16.652 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.8799999999999998e-07 2024-09-04T17:46:16.652 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.652 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.652 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:16.652 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.434917+0000", 2024-09-04T17:46:16.652 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2340000000000001e-06 2024-09-04T17:46:16.653 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.653 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.653 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:16.653 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.434923+0000", 2024-09-04T17:46:16.653 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.285e-06 2024-09-04T17:46:16.653 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.653 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.653 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:16.653 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.434930+0000", 2024-09-04T17:46:16.653 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.5359999999999998e-06 2024-09-04T17:46:16.653 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.653 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.653 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:16.654 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.454993+0000", 2024-09-04T17:46:16.654 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.020062910999999999 2024-09-04T17:46:16.654 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.654 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.654 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:16.654 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.455079+0000", 2024-09-04T17:46:16.654 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.6300999999999999e-05 2024-09-04T17:46:16.654 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.654 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.654 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 5", 2024-09-04T17:46:16.654 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.455263+0000", 2024-09-04T17:46:16.654 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00018429300000000001 2024-09-04T17:46:16.654 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.655 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.655 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:16.655 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.459589+0000", 2024-09-04T17:46:16.655 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0043253520000000002 2024-09-04T17:46:16.655 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.655 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.655 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:16.655 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.460262+0000", 2024-09-04T17:46:16.655 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00067367100000000004 2024-09-04T17:46:16.655 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.655 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.655 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:16.655 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.460334+0000", 2024-09-04T17:46:16.656 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1413000000000002e-05 2024-09-04T17:46:16.656 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.656 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.656 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:16.656 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.460399+0000", 2024-09-04T17:46:16.656 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.4962999999999994e-05 2024-09-04T17:46:16.656 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:16.656 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:16.656 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:16.656 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.656 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.656 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:6301 3.f 3:f3ba70c3:::rbd_data.113431586065.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:16.656 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:08.450661+0000", 2024-09-04T17:46:16.657 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 8.0970591639999991, 2024-09-04T17:46:16.657 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.021587037999999999, 2024-09-04T17:46:16.657 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:16.657 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:16.657 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:16.657 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:16.657 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:16.657 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:16.657 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 6301 2024-09-04T17:46:16.657 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.657 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:16.657 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.658 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:16.658 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.450661+0000", 2024-09-04T17:46:16.658 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:16.658 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.658 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.658 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:16.658 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.450661+0000", 2024-09-04T17:46:16.658 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:16.658 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.658 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.658 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:16.658 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.450663+0000", 2024-09-04T17:46:16.658 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5430000000000001e-06 2024-09-04T17:46:16.659 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.659 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.659 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:16.659 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.455813+0000", 2024-09-04T17:46:16.659 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0051502010000000001 2024-09-04T17:46:16.659 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.659 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.659 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:16.659 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.455828+0000", 2024-09-04T17:46:16.659 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5400000000000002e-05 2024-09-04T17:46:16.660 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.660 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.660 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:16.660 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.455837+0000", 2024-09-04T17:46:16.660 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8100000000000004e-06 2024-09-04T17:46:16.660 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.660 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.660 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:16.660 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.455956+0000", 2024-09-04T17:46:16.660 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00011917 2024-09-04T17:46:16.660 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.660 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.661 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:16.661 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.456107+0000", 2024-09-04T17:46:16.661 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00015067699999999999 2024-09-04T17:46:16.661 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.661 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.661 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 4", 2024-09-04T17:46:16.661 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.456580+0000", 2024-09-04T17:46:16.661 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000473444 2024-09-04T17:46:16.661 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.661 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.661 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:16.662 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.469300+0000", 2024-09-04T17:46:16.662 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.012719729000000001 2024-09-04T17:46:16.662 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.662 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.662 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:16.662 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.472163+0000", 2024-09-04T17:46:16.662 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.002863258 2024-09-04T17:46:16.662 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.662 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.662 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:16.662 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.472203+0000", 2024-09-04T17:46:16.662 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.9434999999999998e-05 2024-09-04T17:46:16.663 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:16.663 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:16.663 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:16.663 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.472248+0000", 2024-09-04T17:46:16.663 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.5371000000000003e-05 2024-09-04T17:46:16.663 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:16.663 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:16.663 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:16.663 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:16.663 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:16.663 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:16.665 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:16.667+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:16.665 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-04T17:46:16.766 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:16.763+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:16.796 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:16.797 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:46:16.797 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:46:16.797 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:16.811 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-04T17:46:16.867 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:16.867+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:16.929 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:16.929 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:46:16.929 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:46:16.929 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:46:16.929 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:16.943 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-04T17:46:16.991 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:16.990+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:17.065 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:17.065 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:46:17.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:46:17.065 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:46:17.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.066 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:1095 3.1 3:86c08994:::rbd_data.11346251f094.000000000000006e:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-04T17:46:17.066 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:02.246406+0000", 2024-09-04T17:46:17.066 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.818095587, 2024-09-04T17:46:17.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.024871196000000002, 2024-09-04T17:46:17.066 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:17.067 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:17.067 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:17.067 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:17.067 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:17.067 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:17.067 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 1095 2024-09-04T17:46:17.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.068 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:17.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:17.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.246406+0000", 2024-09-04T17:46:17.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:17.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.246406+0000", 2024-09-04T17:46:17.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:17.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.246409+0000", 2024-09-04T17:46:17.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6599999999999999e-06 2024-09-04T17:46:17.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:17.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.246419+0000", 2024-09-04T17:46:17.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8840000000000006e-06 2024-09-04T17:46:17.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:17.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.246439+0000", 2024-09-04T17:46:17.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0088000000000001e-05 2024-09-04T17:46:17.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:17.072 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.246448+0000", 2024-09-04T17:46:17.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9339999999999995e-06 2024-09-04T17:46:17.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:17.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.246511+0000", 2024-09-04T17:46:17.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3727999999999997e-05 2024-09-04T17:46:17.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:17.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.246588+0000", 2024-09-04T17:46:17.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6143999999999995e-05 2024-09-04T17:46:17.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-04T17:46:17.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.250919+0000", 2024-09-04T17:46:17.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0043314529999999999 2024-09-04T17:46:17.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:17.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.271170+0000", 2024-09-04T17:46:17.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020251531999999999 2024-09-04T17:46:17.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:46:17.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.271186+0000", 2024-09-04T17:46:17.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5302e-05 2024-09-04T17:46:17.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:17.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.271230+0000", 2024-09-04T17:46:17.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4576000000000001e-05 2024-09-04T17:46:17.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:17.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.271277+0000", 2024-09-04T17:46:17.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6894999999999997e-05 2024-09-04T17:46:17.076 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.076 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:17.076 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.076 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6103 3.1f 3:fec7e474:::rbd_data.1134cfbce83d.0000000000000003:6 [read 1736704~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:17.076 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.421848+0000", 2024-09-04T17:46:17.076 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.6426534900000007, 2024-09-04T17:46:17.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027313884, 2024-09-04T17:46:17.076 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:17.076 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:17.076 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:17.076 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:17.077 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:17.077 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:17.077 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6103 2024-09-04T17:46:17.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.077 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:17.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.077 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:17.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.421848+0000", 2024-09-04T17:46:17.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.077 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:17.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.421848+0000", 2024-09-04T17:46:17.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:17.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.421849+0000", 2024-09-04T17:46:17.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.144e-06 2024-09-04T17:46:17.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:17.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.421855+0000", 2024-09-04T17:46:17.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.003e-06 2024-09-04T17:46:17.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:17.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.421872+0000", 2024-09-04T17:46:17.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7193e-05 2024-09-04T17:46:17.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:17.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.421886+0000", 2024-09-04T17:46:17.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.331e-05 2024-09-04T17:46:17.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:17.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.444849+0000", 2024-09-04T17:46:17.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.022962755000000001 2024-09-04T17:46:17.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:17.080 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.444990+0000", 2024-09-04T17:46:17.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014129999999999999 2024-09-04T17:46:17.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:17.080 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.449162+0000", 2024-09-04T17:46:17.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041721789999999998 2024-09-04T17:46:17.080 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.080 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:17.080 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.081 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6140 3.17 3:ed419000:::rbd_data.1134cfbce83d.000000000000000b:6 [sparse-read 622592~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:17.081 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.422576+0000", 2024-09-04T17:46:17.081 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.6419259020000005, 2024-09-04T17:46:17.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025138273999999999, 2024-09-04T17:46:17.081 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:17.081 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:17.081 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:17.081 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:17.081 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:17.081 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:17.081 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6140 2024-09-04T17:46:17.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.082 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:17.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:17.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422576+0000", 2024-09-04T17:46:17.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:17.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422576+0000", 2024-09-04T17:46:17.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:17.083 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422578+0000", 2024-09-04T17:46:17.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8870000000000001e-06 2024-09-04T17:46:17.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.083 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:17.083 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422587+0000", 2024-09-04T17:46:17.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5999999999999996e-06 2024-09-04T17:46:17.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.083 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:17.083 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422617+0000", 2024-09-04T17:46:17.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9339000000000001e-05 2024-09-04T17:46:17.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:17.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422636+0000", 2024-09-04T17:46:17.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9524000000000001e-05 2024-09-04T17:46:17.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:17.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.445586+0000", 2024-09-04T17:46:17.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.022950084999999999 2024-09-04T17:46:17.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:17.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.445788+0000", 2024-09-04T17:46:17.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000202029 2024-09-04T17:46:17.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:17.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.447714+0000", 2024-09-04T17:46:17.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00192581 2024-09-04T17:46:17.085 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.085 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:17.085 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.085 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6141 3.17 3:ed419000:::rbd_data.1134cfbce83d.000000000000000b:6 [sparse-read 983040~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:17.085 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.422660+0000", 2024-09-04T17:46:17.086 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.6418412389999997, 2024-09-04T17:46:17.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025710566000000001, 2024-09-04T17:46:17.086 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:17.086 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:17.086 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:17.086 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:17.086 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:17.086 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:17.086 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6141 2024-09-04T17:46:17.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.086 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:17.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:17.086 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422660+0000", 2024-09-04T17:46:17.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.087 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:17.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422660+0000", 2024-09-04T17:46:17.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.087 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:17.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422662+0000", 2024-09-04T17:46:17.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.313e-06 2024-09-04T17:46:17.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:17.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422691+0000", 2024-09-04T17:46:17.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9207e-05 2024-09-04T17:46:17.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:17.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422711+0000", 2024-09-04T17:46:17.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0225999999999999e-05 2024-09-04T17:46:17.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:17.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422727+0000", 2024-09-04T17:46:17.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5982999999999999e-05 2024-09-04T17:46:17.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:17.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.447767+0000", 2024-09-04T17:46:17.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025039558 2024-09-04T17:46:17.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:17.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.447871+0000", 2024-09-04T17:46:17.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104287 2024-09-04T17:46:17.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:17.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.448371+0000", 2024-09-04T17:46:17.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00049999199999999999 2024-09-04T17:46:17.090 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.090 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:17.090 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.090 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6142 3.17 3:ed419000:::rbd_data.1134cfbce83d.000000000000000b:6 [sparse-read 1343488~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:17.090 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.422750+0000", 2024-09-04T17:46:17.090 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.6417520070000009, 2024-09-04T17:46:17.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026566215000000001, 2024-09-04T17:46:17.090 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:17.090 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:17.091 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:17.091 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:17.091 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:17.091 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:17.091 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6142 2024-09-04T17:46:17.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.091 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:17.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:17.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422750+0000", 2024-09-04T17:46:17.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:17.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422750+0000", 2024-09-04T17:46:17.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:17.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422751+0000", 2024-09-04T17:46:17.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6700000000000001e-06 2024-09-04T17:46:17.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:17.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422757+0000", 2024-09-04T17:46:17.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9669999999999998e-06 2024-09-04T17:46:17.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:17.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422775+0000", 2024-09-04T17:46:17.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8073000000000001e-05 2024-09-04T17:46:17.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:17.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422789+0000", 2024-09-04T17:46:17.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3709999999999999e-05 2024-09-04T17:46:17.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:17.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.448420+0000", 2024-09-04T17:46:17.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025630627999999999 2024-09-04T17:46:17.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:17.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.448590+0000", 2024-09-04T17:46:17.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000170213 2024-09-04T17:46:17.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:17.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.449316+0000", 2024-09-04T17:46:17.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00072595400000000001 2024-09-04T17:46:17.094 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.094 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:17.094 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.095 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6143 3.17 3:ed419000:::rbd_data.1134cfbce83d.000000000000000b:6 [read 1736704~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:17.095 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.422807+0000", 2024-09-04T17:46:17.095 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.6416946770000003, 2024-09-04T17:46:17.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027271169000000001, 2024-09-04T17:46:17.095 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:17.095 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:17.095 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:17.095 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:17.095 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:17.095 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:17.095 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6143 2024-09-04T17:46:17.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.096 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:17.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:17.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422807+0000", 2024-09-04T17:46:17.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:17.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422807+0000", 2024-09-04T17:46:17.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:17.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422809+0000", 2024-09-04T17:46:17.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6139999999999999e-06 2024-09-04T17:46:17.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:17.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422814+0000", 2024-09-04T17:46:17.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1800000000000004e-06 2024-09-04T17:46:17.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:17.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422831+0000", 2024-09-04T17:46:17.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7433000000000001e-05 2024-09-04T17:46:17.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:17.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422844+0000", 2024-09-04T17:46:17.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3216e-05 2024-09-04T17:46:17.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:17.099 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.449397+0000", 2024-09-04T17:46:17.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026552572 2024-09-04T17:46:17.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:17.099 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.449526+0000", 2024-09-04T17:46:17.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000129207 2024-09-04T17:46:17.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:17.099 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.450078+0000", 2024-09-04T17:46:17.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00055194700000000001 2024-09-04T17:46:17.100 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.100 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:17.100 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.100 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6203 3.1f 3:fec7e474:::rbd_data.1134cfbce83d.0000000000000003:6 [read 327680~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:17.100 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423445+0000", 2024-09-04T17:46:17.100 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.6410562649999996, 2024-09-04T17:46:17.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027179207, 2024-09-04T17:46:17.101 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:17.101 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:17.101 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:17.101 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:17.101 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:17.101 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:17.101 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6203 2024-09-04T17:46:17.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.101 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:17.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:17.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423445+0000", 2024-09-04T17:46:17.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:17.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423445+0000", 2024-09-04T17:46:17.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:17.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423446+0000", 2024-09-04T17:46:17.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.077e-06 2024-09-04T17:46:17.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:17.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423452+0000", 2024-09-04T17:46:17.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.113e-06 2024-09-04T17:46:17.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.103 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:17.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423468+0000", 2024-09-04T17:46:17.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6685999999999999e-05 2024-09-04T17:46:17.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.103 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:17.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423481+0000", 2024-09-04T17:46:17.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2673999999999999e-05 2024-09-04T17:46:17.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.104 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:17.104 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.449779+0000", 2024-09-04T17:46:17.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026297838 2024-09-04T17:46:17.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.104 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:17.104 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.450020+0000", 2024-09-04T17:46:17.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024114399999999999 2024-09-04T17:46:17.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.104 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:17.104 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.450625+0000", 2024-09-04T17:46:17.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00060467500000000005 2024-09-04T17:46:17.105 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.105 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:17.105 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.105 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6204 3.1f 3:fec7e474:::rbd_data.1134cfbce83d.0000000000000003:6 [read 688128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:17.105 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423498+0000", 2024-09-04T17:46:17.105 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.6410033290000001, 2024-09-04T17:46:17.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029262579, 2024-09-04T17:46:17.105 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:17.106 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:17.106 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:17.106 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:17.106 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:17.106 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:17.106 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6204 2024-09-04T17:46:17.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.106 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:17.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:17.106 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423498+0000", 2024-09-04T17:46:17.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:17.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423498+0000", 2024-09-04T17:46:17.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:17.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423500+0000", 2024-09-04T17:46:17.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.28e-06 2024-09-04T17:46:17.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:17.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423506+0000", 2024-09-04T17:46:17.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0059999999999998e-06 2024-09-04T17:46:17.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:17.108 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423524+0000", 2024-09-04T17:46:17.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8683999999999999e-05 2024-09-04T17:46:17.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:17.108 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423537+0000", 2024-09-04T17:46:17.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3030000000000001e-05 2024-09-04T17:46:17.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.109 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:17.109 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.450697+0000", 2024-09-04T17:46:17.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027159793000000002 2024-09-04T17:46:17.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.109 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:17.109 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.450835+0000", 2024-09-04T17:46:17.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013773799999999999 2024-09-04T17:46:17.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.109 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:17.109 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.452761+0000", 2024-09-04T17:46:17.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019260480000000001 2024-09-04T17:46:17.109 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.110 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:17.110 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.110 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.110 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6205 3.1f 3:fec7e474:::rbd_data.1134cfbce83d.0000000000000003:6 [read 1048576~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:17.110 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423555+0000", 2024-09-04T17:46:17.110 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.6409463459999998, 2024-09-04T17:46:17.110 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029905979999999999, 2024-09-04T17:46:17.110 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:17.110 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:17.110 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:17.110 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:17.110 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:17.111 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:17.111 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6205 2024-09-04T17:46:17.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.111 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:17.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.111 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:17.111 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423555+0000", 2024-09-04T17:46:17.111 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.111 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:17.111 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423555+0000", 2024-09-04T17:46:17.111 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.112 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.112 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:17.112 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423556+0000", 2024-09-04T17:46:17.112 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1599999999999999e-06 2024-09-04T17:46:17.112 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.112 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.112 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:17.112 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423562+0000", 2024-09-04T17:46:17.112 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1270000000000002e-06 2024-09-04T17:46:17.112 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.112 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.112 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:17.112 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423579+0000", 2024-09-04T17:46:17.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6986e-05 2024-09-04T17:46:17.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:17.113 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423591+0000", 2024-09-04T17:46:17.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2544000000000001e-05 2024-09-04T17:46:17.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:17.113 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.452819+0000", 2024-09-04T17:46:17.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029227418000000002 2024-09-04T17:46:17.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.114 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:17.114 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.452985+0000", 2024-09-04T17:46:17.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000166675 2024-09-04T17:46:17.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.114 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.114 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:17.114 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.453461+0000", 2024-09-04T17:46:17.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00047606999999999999 2024-09-04T17:46:17.114 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.114 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:17.114 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.114 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.114 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6206 3.1f 3:fec7e474:::rbd_data.1134cfbce83d.0000000000000003:6 [sparse-read 1409024~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:17.115 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423608+0000", 2024-09-04T17:46:17.115 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.6408937130000005, 2024-09-04T17:46:17.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.031860753999999998, 2024-09-04T17:46:17.115 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:17.115 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:17.115 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:17.115 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:17.115 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:17.115 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:17.115 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6206 2024-09-04T17:46:17.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.115 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:17.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:17.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423608+0000", 2024-09-04T17:46:17.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:17.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423608+0000", 2024-09-04T17:46:17.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:17.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423609+0000", 2024-09-04T17:46:17.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0300000000000001e-06 2024-09-04T17:46:17.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:17.117 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423614+0000", 2024-09-04T17:46:17.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0200000000000002e-06 2024-09-04T17:46:17.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:17.117 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423630+0000", 2024-09-04T17:46:17.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6495999999999999e-05 2024-09-04T17:46:17.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:17.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423643+0000", 2024-09-04T17:46:17.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2053e-05 2024-09-04T17:46:17.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.118 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:17.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.453527+0000", 2024-09-04T17:46:17.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029884796000000002 2024-09-04T17:46:17.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.118 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:17.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.453632+0000", 2024-09-04T17:46:17.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104515 2024-09-04T17:46:17.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:17.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.455469+0000", 2024-09-04T17:46:17.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001836844 2024-09-04T17:46:17.119 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.119 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:17.119 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.119 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6207 3.1f 3:fec7e474:::rbd_data.1134cfbce83d.0000000000000003:6 [sparse-read 1769472~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:17.119 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423662+0000", 2024-09-04T17:46:17.119 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.6408392499999991, 2024-09-04T17:46:17.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.033074294999999997, 2024-09-04T17:46:17.119 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:17.120 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:17.120 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:17.120 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:17.120 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:17.120 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:17.120 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6207 2024-09-04T17:46:17.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.120 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:17.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.120 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:17.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423662+0000", 2024-09-04T17:46:17.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.121 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:17.121 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423662+0000", 2024-09-04T17:46:17.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.121 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:17.121 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423664+0000", 2024-09-04T17:46:17.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4929999999999999e-06 2024-09-04T17:46:17.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.121 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:17.121 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423669+0000", 2024-09-04T17:46:17.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3800000000000002e-06 2024-09-04T17:46:17.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:17.122 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423686+0000", 2024-09-04T17:46:17.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7010000000000001e-05 2024-09-04T17:46:17.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:17.122 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423704+0000", 2024-09-04T17:46:17.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7710000000000002e-05 2024-09-04T17:46:17.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:17.123 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.455985+0000", 2024-09-04T17:46:17.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032281272999999999 2024-09-04T17:46:17.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.123 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:17.123 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.456196+0000", 2024-09-04T17:46:17.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000210335 2024-09-04T17:46:17.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.123 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:17.123 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.456737+0000", 2024-09-04T17:46:17.124 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00054109399999999999 2024-09-04T17:46:17.124 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.124 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:17.124 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.124 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6243 3.17 3:ed419000:::rbd_data.1134cfbce83d.000000000000000b:6 [sparse-read 294912~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:17.124 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423721+0000", 2024-09-04T17:46:17.124 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.6407804640000005, 2024-09-04T17:46:17.124 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032666132000000001, 2024-09-04T17:46:17.124 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:17.124 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:17.125 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:17.125 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:17.125 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:17.125 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:17.125 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6243 2024-09-04T17:46:17.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.125 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:17.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.125 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:17.125 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423721+0000", 2024-09-04T17:46:17.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.126 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:17.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423721+0000", 2024-09-04T17:46:17.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.126 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:17.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423722+0000", 2024-09-04T17:46:17.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0470000000000001e-06 2024-09-04T17:46:17.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.126 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:17.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423727+0000", 2024-09-04T17:46:17.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0699999999999997e-06 2024-09-04T17:46:17.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:17.127 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423744+0000", 2024-09-04T17:46:17.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6659000000000001e-05 2024-09-04T17:46:17.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:17.127 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423756+0000", 2024-09-04T17:46:17.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2284e-05 2024-09-04T17:46:17.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:17.128 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.455553+0000", 2024-09-04T17:46:17.128 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.031797032000000003 2024-09-04T17:46:17.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.128 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:17.128 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.455716+0000", 2024-09-04T17:46:17.128 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016253700000000001 2024-09-04T17:46:17.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.128 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:17.128 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.456387+0000", 2024-09-04T17:46:17.128 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00067150300000000005 2024-09-04T17:46:17.128 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.128 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:17.129 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.129 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.129 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6244 3.17 3:ed419000:::rbd_data.1134cfbce83d.000000000000000b:6 [read 688128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:17.129 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423775+0000", 2024-09-04T17:46:17.129 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.6407266810000003, 2024-09-04T17:46:17.129 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.033129498, 2024-09-04T17:46:17.129 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:17.129 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:17.129 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:17.129 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:17.129 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:17.130 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:17.130 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6244 2024-09-04T17:46:17.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.130 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:17.130 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.130 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:17.130 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423775+0000", 2024-09-04T17:46:17.130 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.130 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.130 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:17.130 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423775+0000", 2024-09-04T17:46:17.130 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:17.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423776+0000", 2024-09-04T17:46:17.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1230000000000001e-06 2024-09-04T17:46:17.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:17.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423782+0000", 2024-09-04T17:46:17.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4929999999999998e-06 2024-09-04T17:46:17.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:17.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423799+0000", 2024-09-04T17:46:17.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7254000000000001e-05 2024-09-04T17:46:17.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.132 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:17.132 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423811+0000", 2024-09-04T17:46:17.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2553e-05 2024-09-04T17:46:17.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.132 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:17.132 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.456421+0000", 2024-09-04T17:46:17.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032609552 2024-09-04T17:46:17.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.132 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:17.133 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.456512+0000", 2024-09-04T17:46:17.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0593999999999994e-05 2024-09-04T17:46:17.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.133 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:17.133 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.456904+0000", 2024-09-04T17:46:17.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00039292899999999998 2024-09-04T17:46:17.133 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.133 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:17.133 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.134 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6245 3.17 3:ed419000:::rbd_data.1134cfbce83d.000000000000000b:6 [read 1048576~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:17.134 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423830+0000", 2024-09-04T17:46:17.134 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.6406716750000001, 2024-09-04T17:46:17.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.033577937000000002, 2024-09-04T17:46:17.134 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:17.134 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:17.134 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:17.134 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:17.134 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:17.134 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:17.134 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6245 2024-09-04T17:46:17.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.134 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:17.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.135 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:17.135 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423830+0000", 2024-09-04T17:46:17.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.135 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:17.135 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423830+0000", 2024-09-04T17:46:17.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.135 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:17.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423831+0000", 2024-09-04T17:46:17.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.164e-06 2024-09-04T17:46:17.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:17.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423847+0000", 2024-09-04T17:46:17.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5573000000000002e-05 2024-09-04T17:46:17.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:17.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423865+0000", 2024-09-04T17:46:17.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8715999999999998e-05 2024-09-04T17:46:17.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:17.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423880+0000", 2024-09-04T17:46:17.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4107000000000001e-05 2024-09-04T17:46:17.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:17.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.456951+0000", 2024-09-04T17:46:17.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.03307181 2024-09-04T17:46:17.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:17.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.457039+0000", 2024-09-04T17:46:17.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7795999999999998e-05 2024-09-04T17:46:17.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:17.138 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.457408+0000", 2024-09-04T17:46:17.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000368771 2024-09-04T17:46:17.138 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.138 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:17.138 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.139 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6246 3.17 3:ed419000:::rbd_data.1134cfbce83d.000000000000000b:6 [read 1409024~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:17.139 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423897+0000", 2024-09-04T17:46:17.139 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.6406043359999991, 2024-09-04T17:46:17.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034015888000000001, 2024-09-04T17:46:17.139 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:17.139 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:17.139 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:17.139 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:17.139 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:17.139 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:17.139 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6246 2024-09-04T17:46:17.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.140 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:17.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.140 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:17.140 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423897+0000", 2024-09-04T17:46:17.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.140 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:17.140 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423897+0000", 2024-09-04T17:46:17.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.140 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:17.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423899+0000", 2024-09-04T17:46:17.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3200000000000001e-06 2024-09-04T17:46:17.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:17.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423904+0000", 2024-09-04T17:46:17.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4369999999999998e-06 2024-09-04T17:46:17.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:17.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423922+0000", 2024-09-04T17:46:17.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8196999999999999e-05 2024-09-04T17:46:17.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.142 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:17.142 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423935+0000", 2024-09-04T17:46:17.142 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.314e-05 2024-09-04T17:46:17.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.142 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:17.142 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.457457+0000", 2024-09-04T17:46:17.142 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.033521663 2024-09-04T17:46:17.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.142 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:17.142 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.457536+0000", 2024-09-04T17:46:17.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8795999999999996e-05 2024-09-04T17:46:17.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:17.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.457913+0000", 2024-09-04T17:46:17.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00037733500000000002 2024-09-04T17:46:17.143 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.143 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:17.143 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.143 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6247 3.17 3:ed419000:::rbd_data.1134cfbce83d.000000000000000b:6 [read 1769472~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:17.143 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423953+0000", 2024-09-04T17:46:17.144 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.6405481520000009, 2024-09-04T17:46:17.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0344287, 2024-09-04T17:46:17.144 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:17.144 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:17.144 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:17.144 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:17.144 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:17.144 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:17.144 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6247 2024-09-04T17:46:17.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.144 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:17.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:17.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423953+0000", 2024-09-04T17:46:17.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:17.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423953+0000", 2024-09-04T17:46:17.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:17.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423955+0000", 2024-09-04T17:46:17.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0529999999999999e-06 2024-09-04T17:46:17.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.146 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:17.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423960+0000", 2024-09-04T17:46:17.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0470000000000001e-06 2024-09-04T17:46:17.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.146 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:17.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423978+0000", 2024-09-04T17:46:17.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8119999999999999e-05 2024-09-04T17:46:17.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.146 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:17.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423991+0000", 2024-09-04T17:46:17.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2813000000000001e-05 2024-09-04T17:46:17.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.147 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:17.147 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.457947+0000", 2024-09-04T17:46:17.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.033956865000000003 2024-09-04T17:46:17.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.147 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:17.147 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.458039+0000", 2024-09-04T17:46:17.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1834000000000002e-05 2024-09-04T17:46:17.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.147 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:17.148 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.458382+0000", 2024-09-04T17:46:17.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00034296799999999997 2024-09-04T17:46:17.148 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.148 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:17.148 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.148 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6305 3.17 3:ee7cd06a:::rbd_data.113431586065.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:17.148 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.455350+0000", 2024-09-04T17:46:17.148 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.6091516000000006, 2024-09-04T17:46:17.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029276888000000001, 2024-09-04T17:46:17.148 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:17.149 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:17.149 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:17.149 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:17.149 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:17.149 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:17.149 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6305 2024-09-04T17:46:17.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.149 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:17.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:17.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.455350+0000", 2024-09-04T17:46:17.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:17.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.455350+0000", 2024-09-04T17:46:17.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:17.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.455353+0000", 2024-09-04T17:46:17.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6400000000000001e-06 2024-09-04T17:46:17.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:17.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459745+0000", 2024-09-04T17:46:17.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0043925880000000002 2024-09-04T17:46:17.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.151 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:17.151 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459775+0000", 2024-09-04T17:46:17.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9254000000000001e-05 2024-09-04T17:46:17.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.151 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:17.151 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459789+0000", 2024-09-04T17:46:17.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4731e-05 2024-09-04T17:46:17.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.152 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:17.152 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459885+0000", 2024-09-04T17:46:17.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5502999999999997e-05 2024-09-04T17:46:17.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.152 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:17.152 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.460040+0000", 2024-09-04T17:46:17.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015491499999999999 2024-09-04T17:46:17.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.152 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5", 2024-09-04T17:46:17.152 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.460596+0000", 2024-09-04T17:46:17.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00055657500000000002 2024-09-04T17:46:17.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:17.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484484+0000", 2024-09-04T17:46:17.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.023887950000000002 2024-09-04T17:46:17.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:46:17.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484510+0000", 2024-09-04T17:46:17.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.561e-05 2024-09-04T17:46:17.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:17.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484569+0000", 2024-09-04T17:46:17.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9049999999999999e-05 2024-09-04T17:46:17.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:17.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484627+0000", 2024-09-04T17:46:17.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8072000000000002e-05 2024-09-04T17:46:17.154 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.154 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:17.154 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.154 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6306 3.17 3:ee7cd06a:::rbd_data.113431586065.0000000000000005:head [truncate 1572864] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:17.155 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.459834+0000", 2024-09-04T17:46:17.155 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.6046679220000009, 2024-09-04T17:46:17.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029166788999999999, 2024-09-04T17:46:17.155 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:17.155 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:17.155 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:17.155 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:17.155 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:17.155 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:17.155 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6306 2024-09-04T17:46:17.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.155 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:17.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:17.156 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459834+0000", 2024-09-04T17:46:17.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.156 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:17.156 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459834+0000", 2024-09-04T17:46:17.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:17.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459836+0000", 2024-09-04T17:46:17.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.835e-06 2024-09-04T17:46:17.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:17.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459841+0000", 2024-09-04T17:46:17.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.728e-06 2024-09-04T17:46:17.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:17.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459851+0000", 2024-09-04T17:46:17.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8570000000000007e-06 2024-09-04T17:46:17.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:17.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459859+0000", 2024-09-04T17:46:17.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4970000000000004e-06 2024-09-04T17:46:17.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:17.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.468789+0000", 2024-09-04T17:46:17.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0089301699999999994 2024-09-04T17:46:17.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:17.159 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.468890+0000", 2024-09-04T17:46:17.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000100963 2024-09-04T17:46:17.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.159 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5", 2024-09-04T17:46:17.159 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.469090+0000", 2024-09-04T17:46:17.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000199774 2024-09-04T17:46:17.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.159 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:17.159 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484704+0000", 2024-09-04T17:46:17.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015614836999999999 2024-09-04T17:46:17.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.160 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:46:17.160 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.488833+0000", 2024-09-04T17:46:17.160 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004128357 2024-09-04T17:46:17.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.160 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:17.160 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.488916+0000", 2024-09-04T17:46:17.160 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2928000000000003e-05 2024-09-04T17:46:17.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.160 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:17.160 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.489001+0000", 2024-09-04T17:46:17.161 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4843000000000001e-05 2024-09-04T17:46:17.161 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.161 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:17.161 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.161 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.161 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6307 3.17 3:eb6e082d:::rbd_data.113431586065.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:17.161 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.459871+0000", 2024-09-04T17:46:17.161 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.6046309619999999, 2024-09-04T17:46:17.161 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032107122000000002, 2024-09-04T17:46:17.161 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:17.161 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:17.161 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:17.162 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:17.162 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:17.162 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:17.162 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6307 2024-09-04T17:46:17.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.162 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:17.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:17.162 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459871+0000", 2024-09-04T17:46:17.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:17.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459871+0000", 2024-09-04T17:46:17.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.163 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.163 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:17.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459871+0000", 2024-09-04T17:46:17.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9699999999999995e-07 2024-09-04T17:46:17.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.163 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.163 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:17.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.463429+0000", 2024-09-04T17:46:17.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0035577930000000001 2024-09-04T17:46:17.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.164 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:17.164 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.463463+0000", 2024-09-04T17:46:17.164 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3460000000000002e-05 2024-09-04T17:46:17.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.164 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:17.164 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.463483+0000", 2024-09-04T17:46:17.164 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0327999999999999e-05 2024-09-04T17:46:17.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.164 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:17.164 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.477551+0000", 2024-09-04T17:46:17.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014068259 2024-09-04T17:46:17.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.165 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:17.165 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.477748+0000", 2024-09-04T17:46:17.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000197004 2024-09-04T17:46:17.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.165 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5", 2024-09-04T17:46:17.165 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.478346+0000", 2024-09-04T17:46:17.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00059796299999999997 2024-09-04T17:46:17.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.166 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:46:17.166 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.489016+0000", 2024-09-04T17:46:17.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010669781 2024-09-04T17:46:17.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.166 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:17.166 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.491871+0000", 2024-09-04T17:46:17.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028546219999999998 2024-09-04T17:46:17.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.166 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:17.166 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.491931+0000", 2024-09-04T17:46:17.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0736999999999998e-05 2024-09-04T17:46:17.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:17.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.491978+0000", 2024-09-04T17:46:17.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6378000000000002e-05 2024-09-04T17:46:17.167 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.167 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:17.167 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.167 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6308 3.17 3:eb6e082d:::rbd_data.113431586065.0000000000000007:head [truncate 1572864] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:17.167 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.463520+0000", 2024-09-04T17:46:17.168 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.6009814529999993, 2024-09-04T17:46:17.168 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029348833000000001, 2024-09-04T17:46:17.168 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:17.168 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:17.168 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:17.168 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:17.168 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:17.168 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:17.168 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6308 2024-09-04T17:46:17.168 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.168 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:17.168 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.169 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:17.169 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.463520+0000", 2024-09-04T17:46:17.169 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.169 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:17.169 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.463520+0000", 2024-09-04T17:46:17.169 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.169 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:17.169 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.463522+0000", 2024-09-04T17:46:17.169 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1150000000000001e-06 2024-09-04T17:46:17.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.170 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:17.170 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.463527+0000", 2024-09-04T17:46:17.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5170000000000001e-06 2024-09-04T17:46:17.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.170 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:17.170 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.463555+0000", 2024-09-04T17:46:17.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8594000000000001e-05 2024-09-04T17:46:17.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.170 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:17.171 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.463563+0000", 2024-09-04T17:46:17.171 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4340000000000003e-06 2024-09-04T17:46:17.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.171 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:17.171 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484756+0000", 2024-09-04T17:46:17.171 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021193081999999999 2024-09-04T17:46:17.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.171 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:17.171 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484863+0000", 2024-09-04T17:46:17.171 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010721999999999999 2024-09-04T17:46:17.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.172 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5", 2024-09-04T17:46:17.172 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.485067+0000", 2024-09-04T17:46:17.172 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000204 2024-09-04T17:46:17.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.172 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:46:17.172 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.489593+0000", 2024-09-04T17:46:17.172 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045261529999999998 2024-09-04T17:46:17.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.172 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:17.172 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.492787+0000", 2024-09-04T17:46:17.172 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031940039999999999 2024-09-04T17:46:17.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.173 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.173 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:17.173 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.492834+0000", 2024-09-04T17:46:17.173 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6281e-05 2024-09-04T17:46:17.173 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.173 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.173 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:17.173 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.492869+0000", 2024-09-04T17:46:17.173 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5432999999999998e-05 2024-09-04T17:46:17.173 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.173 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:17.173 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.173 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.174 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:17.174 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:17.175 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-04T17:46:17.175 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:17.174+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:17.276 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:17.275+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:17.353 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:17.353 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:46:17.353 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:46:17.353 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:17.368 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-04T17:46:17.377 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:17.375+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:17.457 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #7, image image_client.0-clone7 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent8 2024-09-04T17:46:17.478 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:17.477+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:17.510 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:17.510 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:46:17.510 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:46:17.510 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:46:17.510 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:17.529 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-04T17:46:17.615 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:17.613+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:17.680 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:17.681+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:17.709 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:17.709 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:46:17.710 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:46:17.710 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:46:17.710 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.710 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:1370 3.4 3:22db5e3d:::rbd_data.11346251f094.0000000000000174:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-04T17:46:17.710 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:02.677851+0000", 2024-09-04T17:46:17.710 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.031507571000001, 2024-09-04T17:46:17.710 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015980641, 2024-09-04T17:46:17.710 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:17.710 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:17.710 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:17.710 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:17.710 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:17.710 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:17.711 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 1370 2024-09-04T17:46:17.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.711 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:17.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.711 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:17.711 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.677851+0000", 2024-09-04T17:46:17.711 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.711 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:17.711 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.677851+0000", 2024-09-04T17:46:17.711 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.712 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:17.712 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.677853+0000", 2024-09-04T17:46:17.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.846e-06 2024-09-04T17:46:17.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.712 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:17.712 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.677860+0000", 2024-09-04T17:46:17.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6969999999999996e-06 2024-09-04T17:46:17.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.712 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:17.712 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.677873+0000", 2024-09-04T17:46:17.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3244000000000001e-05 2024-09-04T17:46:17.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.713 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:17.713 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.677879+0000", 2024-09-04T17:46:17.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2809999999999997e-06 2024-09-04T17:46:17.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.713 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:17.713 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.677928+0000", 2024-09-04T17:46:17.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8998000000000001e-05 2024-09-04T17:46:17.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.713 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:17.714 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.678026+0000", 2024-09-04T17:46:17.714 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7634999999999996e-05 2024-09-04T17:46:17.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.714 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.714 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2024-09-04T17:46:17.714 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.679911+0000", 2024-09-04T17:46:17.714 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001885227 2024-09-04T17:46:17.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.714 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.714 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:17.715 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.693674+0000", 2024-09-04T17:46:17.715 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013762968 2024-09-04T17:46:17.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.715 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:46:17.715 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.693700+0000", 2024-09-04T17:46:17.715 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6308999999999998e-05 2024-09-04T17:46:17.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.715 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:17.715 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.693769+0000", 2024-09-04T17:46:17.715 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8371e-05 2024-09-04T17:46:17.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.716 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:17.716 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.693832+0000", 2024-09-04T17:46:17.716 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3064999999999993e-05 2024-09-04T17:46:17.716 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.716 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:17.716 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.716 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.716 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:1493 3.8 3:17b4bedd:::rbd_data.11346251f094.000000000000010d:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-04T17:46:17.716 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:02.955681+0000", 2024-09-04T17:46:17.716 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.753677365, 2024-09-04T17:46:17.716 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014830360000000001, 2024-09-04T17:46:17.717 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:17.717 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:17.717 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:17.717 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:17.717 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:17.717 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:17.717 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 1493 2024-09-04T17:46:17.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.717 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:17.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.717 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:17.717 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.955681+0000", 2024-09-04T17:46:17.717 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.718 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:17.718 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.955681+0000", 2024-09-04T17:46:17.718 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.718 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:17.718 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.955683+0000", 2024-09-04T17:46:17.718 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6029999999999999e-06 2024-09-04T17:46:17.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.718 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:17.719 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.955689+0000", 2024-09-04T17:46:17.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8669999999999999e-06 2024-09-04T17:46:17.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.719 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.719 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:17.719 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.955701+0000", 2024-09-04T17:46:17.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2157000000000001e-05 2024-09-04T17:46:17.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.719 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.719 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:17.719 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.955707+0000", 2024-09-04T17:46:17.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2740000000000004e-06 2024-09-04T17:46:17.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.720 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:17.720 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.955756+0000", 2024-09-04T17:46:17.720 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8665999999999998e-05 2024-09-04T17:46:17.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.720 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:17.720 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.955850+0000", 2024-09-04T17:46:17.721 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4153999999999993e-05 2024-09-04T17:46:17.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.721 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.721 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:46:17.721 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.970361+0000", 2024-09-04T17:46:17.721 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01451098 2024-09-04T17:46:17.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.721 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.721 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:17.721 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.970408+0000", 2024-09-04T17:46:17.721 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7058999999999998e-05 2024-09-04T17:46:17.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.721 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.721 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:17.722 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.970512+0000", 2024-09-04T17:46:17.722 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0001036 2024-09-04T17:46:17.722 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.722 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:17.722 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.722 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:4881 3.4 3:25597cfa:::rbd_data.1134cfbce83d.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:17.722 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:07.376991+0000", 2024-09-04T17:46:17.722 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.332368304999999, 2024-09-04T17:46:17.722 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021363054999999999, 2024-09-04T17:46:17.722 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:17.722 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:17.722 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:17.723 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:17.723 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:17.723 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:17.723 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 4881 2024-09-04T17:46:17.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.723 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:17.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:17.723 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.376991+0000", 2024-09-04T17:46:17.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:17.724 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.376991+0000", 2024-09-04T17:46:17.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.724 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:17.724 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.376994+0000", 2024-09-04T17:46:17.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.433e-06 2024-09-04T17:46:17.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.724 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:17.724 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.380142+0000", 2024-09-04T17:46:17.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031476920000000001 2024-09-04T17:46:17.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:17.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.380186+0000", 2024-09-04T17:46:17.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4116000000000002e-05 2024-09-04T17:46:17.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:17.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.380373+0000", 2024-09-04T17:46:17.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00018714900000000001 2024-09-04T17:46:17.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:17.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.380666+0000", 2024-09-04T17:46:17.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00029291700000000001 2024-09-04T17:46:17.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.726 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:17.726 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.380767+0000", 2024-09-04T17:46:17.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000101345 2024-09-04T17:46:17.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.726 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2024-09-04T17:46:17.726 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.381169+0000", 2024-09-04T17:46:17.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00040205800000000001 2024-09-04T17:46:17.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.726 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:46:17.726 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.393205+0000", 2024-09-04T17:46:17.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012035445000000001 2024-09-04T17:46:17.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.727 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.727 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:17.727 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.398143+0000", 2024-09-04T17:46:17.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0049384210000000001 2024-09-04T17:46:17.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.727 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.727 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:17.727 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.398264+0000", 2024-09-04T17:46:17.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000120444 2024-09-04T17:46:17.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.727 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:17.728 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.398354+0000", 2024-09-04T17:46:17.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0035000000000001e-05 2024-09-04T17:46:17.728 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.728 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:17.728 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.728 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:4884 3.4 3:25597cfa:::rbd_data.1134cfbce83d.0000000000000006:head [delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:17.728 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:07.380619+0000", 2024-09-04T17:46:17.728 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.32873994, 2024-09-04T17:46:17.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018096067, 2024-09-04T17:46:17.728 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:17.729 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:17.729 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:17.729 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:17.729 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:17.729 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:17.729 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 4884 2024-09-04T17:46:17.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.729 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:17.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.729 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:17.729 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.380619+0000", 2024-09-04T17:46:17.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.730 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:17.730 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.380619+0000", 2024-09-04T17:46:17.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.730 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:17.730 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.380622+0000", 2024-09-04T17:46:17.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6400000000000001e-06 2024-09-04T17:46:17.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.730 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:17.730 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.380630+0000", 2024-09-04T17:46:17.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6829999999999997e-06 2024-09-04T17:46:17.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.731 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:17.731 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.380655+0000", 2024-09-04T17:46:17.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4386999999999999e-05 2024-09-04T17:46:17.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.731 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:17.731 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.380673+0000", 2024-09-04T17:46:17.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8012999999999998e-05 2024-09-04T17:46:17.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.731 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:17.731 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.385906+0000", 2024-09-04T17:46:17.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052335029999999996 2024-09-04T17:46:17.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:17.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.386205+0000", 2024-09-04T17:46:17.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00029915399999999999 2024-09-04T17:46:17.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2024-09-04T17:46:17.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.386749+0000", 2024-09-04T17:46:17.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00054368600000000004 2024-09-04T17:46:17.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:46:17.733 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.394514+0000", 2024-09-04T17:46:17.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0077648630000000003 2024-09-04T17:46:17.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:17.733 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.398515+0000", 2024-09-04T17:46:17.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040009290000000003 2024-09-04T17:46:17.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:17.733 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.398623+0000", 2024-09-04T17:46:17.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010852 2024-09-04T17:46:17.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.734 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:17.734 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.398715+0000", 2024-09-04T17:46:17.734 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1688999999999996e-05 2024-09-04T17:46:17.734 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.734 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:17.734 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.734 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:5306 3.4 3:25597cfa:::rbd_data.1134cfbce83d.0000000000000006:6 [read 1507328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:17.734 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:07.695965+0000", 2024-09-04T17:46:17.734 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.013393667000001, 2024-09-04T17:46:17.734 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014771266, 2024-09-04T17:46:17.735 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:17.735 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:17.735 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:17.735 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:17.735 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:17.735 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:17.735 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 5306 2024-09-04T17:46:17.735 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.735 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:17.735 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.735 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:17.735 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.695965+0000", 2024-09-04T17:46:17.735 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.736 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:17.736 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.695965+0000", 2024-09-04T17:46:17.736 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.736 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:17.736 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.695966+0000", 2024-09-04T17:46:17.736 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2300000000000001e-06 2024-09-04T17:46:17.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.736 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:17.736 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.695973+0000", 2024-09-04T17:46:17.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3029999999999996e-06 2024-09-04T17:46:17.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:17.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.695993+0000", 2024-09-04T17:46:17.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9962999999999998e-05 2024-09-04T17:46:17.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:17.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.696006+0000", 2024-09-04T17:46:17.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.348e-05 2024-09-04T17:46:17.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:17.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.709659+0000", 2024-09-04T17:46:17.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013652500999999999 2024-09-04T17:46:17.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:17.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.709915+0000", 2024-09-04T17:46:17.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000255942 2024-09-04T17:46:17.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:17.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.710736+0000", 2024-09-04T17:46:17.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00082184700000000001 2024-09-04T17:46:17.738 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.738 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:17.739 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.739 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.739 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:5307 3.4 3:25597cfa:::rbd_data.1134cfbce83d.0000000000000006:6 [read 1867776~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:17.739 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:07.696024+0000", 2024-09-04T17:46:17.739 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.013334874, 2024-09-04T17:46:17.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015970018999999998, 2024-09-04T17:46:17.739 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:17.739 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:17.739 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:17.739 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:17.739 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:17.739 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:17.740 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 5307 2024-09-04T17:46:17.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.740 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:17.740 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.740 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:17.740 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.696024+0000", 2024-09-04T17:46:17.740 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.740 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.740 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:17.740 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.696024+0000", 2024-09-04T17:46:17.740 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.740 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:17.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.696025+0000", 2024-09-04T17:46:17.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9599999999999998e-07 2024-09-04T17:46:17.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:17.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.696029+0000", 2024-09-04T17:46:17.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9539999999999998e-06 2024-09-04T17:46:17.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:17.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.696039+0000", 2024-09-04T17:46:17.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9930000000000007e-06 2024-09-04T17:46:17.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:17.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.696046+0000", 2024-09-04T17:46:17.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8029999999999999e-06 2024-09-04T17:46:17.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:17.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.710828+0000", 2024-09-04T17:46:17.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014782152 2024-09-04T17:46:17.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:17.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.711064+0000", 2024-09-04T17:46:17.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023633899999999999 2024-09-04T17:46:17.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.743 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:17.743 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.711994+0000", 2024-09-04T17:46:17.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00092998199999999999 2024-09-04T17:46:17.743 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.743 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:17.743 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.743 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:5356 3.4 3:207aa2a5:::rbd_data.1134cfbce83d.0000000000000011:6 [read 131072~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:17.743 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:07.696361+0000", 2024-09-04T17:46:17.743 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.012997857, 2024-09-04T17:46:17.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016279907, 2024-09-04T17:46:17.744 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:17.744 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:17.744 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:17.744 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:17.744 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:17.744 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:17.744 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 5356 2024-09-04T17:46:17.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.744 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:17.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.744 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:17.744 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.696361+0000", 2024-09-04T17:46:17.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:17.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.696361+0000", 2024-09-04T17:46:17.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:17.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.696362+0000", 2024-09-04T17:46:17.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9300000000000006e-07 2024-09-04T17:46:17.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:17.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.696365+0000", 2024-09-04T17:46:17.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.807e-06 2024-09-04T17:46:17.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:17.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.696374+0000", 2024-09-04T17:46:17.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1230000000000006e-06 2024-09-04T17:46:17.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:17.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.696381+0000", 2024-09-04T17:46:17.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7070000000000003e-06 2024-09-04T17:46:17.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.747 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:17.747 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.712092+0000", 2024-09-04T17:46:17.747 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015711447999999999 2024-09-04T17:46:17.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.747 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.747 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:17.747 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.712352+0000", 2024-09-04T17:46:17.747 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00025990200000000001 2024-09-04T17:46:17.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.747 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.747 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:17.747 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.712641+0000", 2024-09-04T17:46:17.747 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028892700000000002 2024-09-04T17:46:17.747 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.748 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:17.748 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.748 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.748 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:5357 3.4 3:207aa2a5:::rbd_data.1134cfbce83d.0000000000000011:6 [read 491520~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:17.748 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:07.696391+0000", 2024-09-04T17:46:17.748 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.012968153999999, 2024-09-04T17:46:17.748 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017049084999999999, 2024-09-04T17:46:17.748 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:17.748 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:17.748 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:17.748 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:17.748 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:17.749 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:17.749 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 5357 2024-09-04T17:46:17.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.749 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:17.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.749 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:17.749 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.696391+0000", 2024-09-04T17:46:17.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.749 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:17.749 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.696391+0000", 2024-09-04T17:46:17.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:17.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.696391+0000", 2024-09-04T17:46:17.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6000000000000003e-07 2024-09-04T17:46:17.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:17.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.696394+0000", 2024-09-04T17:46:17.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6869999999999999e-06 2024-09-04T17:46:17.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:17.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.696403+0000", 2024-09-04T17:46:17.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2529999999999993e-06 2024-09-04T17:46:17.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:17.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.696410+0000", 2024-09-04T17:46:17.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9399999999999996e-06 2024-09-04T17:46:17.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:17.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.712851+0000", 2024-09-04T17:46:17.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016440816 2024-09-04T17:46:17.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.752 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:17.752 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.713094+0000", 2024-09-04T17:46:17.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024340899999999999 2024-09-04T17:46:17.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.752 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:17.752 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.713440+0000", 2024-09-04T17:46:17.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00034531999999999998 2024-09-04T17:46:17.752 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.752 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:17.752 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.753 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:5358 3.4 3:207aa2a5:::rbd_data.1134cfbce83d.0000000000000011:6 [read 851968~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:17.753 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:07.696420+0000", 2024-09-04T17:46:17.753 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.012938936999999, 2024-09-04T17:46:17.753 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017981700999999999, 2024-09-04T17:46:17.753 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:17.753 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:17.753 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:17.753 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:17.753 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:17.753 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:17.753 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 5358 2024-09-04T17:46:17.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.754 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:17.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:17.754 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.696420+0000", 2024-09-04T17:46:17.754 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:17.754 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.696420+0000", 2024-09-04T17:46:17.754 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:17.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.696421+0000", 2024-09-04T17:46:17.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5300000000000004e-07 2024-09-04T17:46:17.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:17.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.696423+0000", 2024-09-04T17:46:17.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7930000000000002e-06 2024-09-04T17:46:17.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:17.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.696433+0000", 2024-09-04T17:46:17.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4129999999999995e-06 2024-09-04T17:46:17.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.756 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:17.756 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.696440+0000", 2024-09-04T17:46:17.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8299999999999998e-06 2024-09-04T17:46:17.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.756 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:17.756 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.713811+0000", 2024-09-04T17:46:17.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017370946000000002 2024-09-04T17:46:17.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.757 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:17.757 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.714072+0000", 2024-09-04T17:46:17.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026137799999999999 2024-09-04T17:46:17.757 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.757 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:17.757 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.714402+0000", 2024-09-04T17:46:17.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00032968800000000002 2024-09-04T17:46:17.757 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.757 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:17.757 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.757 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.758 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6016 3.4 3:25597cfa:::rbd_data.1134cfbce83d.0000000000000006:6 [read 557056~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:17.758 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.409642+0000", 2024-09-04T17:46:17.758 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.2997165220000007, 2024-09-04T17:46:17.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015587873, 2024-09-04T17:46:17.758 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:17.758 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:17.758 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:17.758 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:17.758 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:17.758 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:17.758 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6016 2024-09-04T17:46:17.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.758 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:17.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:17.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.409642+0000", 2024-09-04T17:46:17.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:17.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.409642+0000", 2024-09-04T17:46:17.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:17.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.409644+0000", 2024-09-04T17:46:17.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5099999999999999e-06 2024-09-04T17:46:17.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:17.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.409655+0000", 2024-09-04T17:46:17.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.079e-05 2024-09-04T17:46:17.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:17.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.409675+0000", 2024-09-04T17:46:17.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9967e-05 2024-09-04T17:46:17.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:17.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.409690+0000", 2024-09-04T17:46:17.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5800000000000001e-05 2024-09-04T17:46:17.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:17.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.421796+0000", 2024-09-04T17:46:17.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01210542 2024-09-04T17:46:17.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:17.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422216+0000", 2024-09-04T17:46:17.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00042046599999999998 2024-09-04T17:46:17.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.762 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:17.762 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.425230+0000", 2024-09-04T17:46:17.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0030139199999999998 2024-09-04T17:46:17.762 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.762 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:17.762 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.762 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6017 3.4 3:25597cfa:::rbd_data.1134cfbce83d.0000000000000006:6 [sparse-read 917504~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:17.762 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.409717+0000", 2024-09-04T17:46:17.762 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.2996422889999995, 2024-09-04T17:46:17.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016601551999999999, 2024-09-04T17:46:17.763 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:17.763 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:17.763 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:17.763 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:17.763 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:17.763 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:17.763 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6017 2024-09-04T17:46:17.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.763 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:17.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.763 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:17.763 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.409717+0000", 2024-09-04T17:46:17.763 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:17.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.409717+0000", 2024-09-04T17:46:17.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:17.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.409718+0000", 2024-09-04T17:46:17.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.294e-06 2024-09-04T17:46:17.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:17.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.409724+0000", 2024-09-04T17:46:17.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.443e-06 2024-09-04T17:46:17.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:17.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.409743+0000", 2024-09-04T17:46:17.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8865999999999999e-05 2024-09-04T17:46:17.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:17.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.409759+0000", 2024-09-04T17:46:17.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5656999999999999e-05 2024-09-04T17:46:17.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:17.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.425313+0000", 2024-09-04T17:46:17.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015554177000000001 2024-09-04T17:46:17.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.766 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:17.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.425523+0000", 2024-09-04T17:46:17.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000210235 2024-09-04T17:46:17.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.766 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:17.767 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.426318+0000", 2024-09-04T17:46:17.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00079487999999999998 2024-09-04T17:46:17.767 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.767 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:17.767 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.767 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6018 3.4 3.5f3e9aa4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:17.767 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.409784+0000", 2024-09-04T17:46:17.767 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.2995743429999997, 2024-09-04T17:46:17.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016628211, 2024-09-04T17:46:17.767 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:17.768 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:17.768 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-04T17:46:17.768 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:17.768 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:17.768 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:17.768 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6018 2024-09-04T17:46:17.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.768 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:17.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.768 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:17.768 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.409784+0000", 2024-09-04T17:46:17.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:17.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.409784+0000", 2024-09-04T17:46:17.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:17.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.409786+0000", 2024-09-04T17:46:17.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.207e-06 2024-09-04T17:46:17.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:17.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.409792+0000", 2024-09-04T17:46:17.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4400000000000002e-06 2024-09-04T17:46:17.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:17.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.409810+0000", 2024-09-04T17:46:17.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.774e-05 2024-09-04T17:46:17.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:17.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.409824+0000", 2024-09-04T17:46:17.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3866000000000001e-05 2024-09-04T17:46:17.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:17.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.426392+0000", 2024-09-04T17:46:17.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016567828999999999 2024-09-04T17:46:17.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:17.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.426413+0000", 2024-09-04T17:46:17.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1129e-05 2024-09-04T17:46:17.771 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.771 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:17.771 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.772 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6019 3.4 3.5f3e9aa4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:17.772 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.409848+0000", 2024-09-04T17:46:17.772 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.2995109399999993, 2024-09-04T17:46:17.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016648943999999999, 2024-09-04T17:46:17.772 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:17.772 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:17.772 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-04T17:46:17.772 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:17.772 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:17.772 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:17.772 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6019 2024-09-04T17:46:17.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.773 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:17.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.773 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:17.773 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.409848+0000", 2024-09-04T17:46:17.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.773 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:17.773 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.409848+0000", 2024-09-04T17:46:17.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:17.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.409849+0000", 2024-09-04T17:46:17.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.347e-06 2024-09-04T17:46:17.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:17.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.409855+0000", 2024-09-04T17:46:17.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.083e-06 2024-09-04T17:46:17.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:17.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.409874+0000", 2024-09-04T17:46:17.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8807000000000001e-05 2024-09-04T17:46:17.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:17.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.409887+0000", 2024-09-04T17:46:17.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3263e-05 2024-09-04T17:46:17.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:17.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.426477+0000", 2024-09-04T17:46:17.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016589881000000001 2024-09-04T17:46:17.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:17.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.426497+0000", 2024-09-04T17:46:17.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9562999999999999e-05 2024-09-04T17:46:17.776 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.776 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:17.776 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.776 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6020 3.4 3.5f3e9aa4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:17.776 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.409909+0000", 2024-09-04T17:46:17.776 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.2994501669999998, 2024-09-04T17:46:17.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016671056, 2024-09-04T17:46:17.776 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:17.776 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:17.776 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-04T17:46:17.777 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:17.777 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:17.777 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:17.777 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6020 2024-09-04T17:46:17.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.777 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:17.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.777 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:17.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.409909+0000", 2024-09-04T17:46:17.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:17.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.409909+0000", 2024-09-04T17:46:17.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:17.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.409910+0000", 2024-09-04T17:46:17.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2500000000000001e-06 2024-09-04T17:46:17.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:17.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.409917+0000", 2024-09-04T17:46:17.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9129999999999997e-06 2024-09-04T17:46:17.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:17.779 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.409936+0000", 2024-09-04T17:46:17.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9077e-05 2024-09-04T17:46:17.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:17.779 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.409950+0000", 2024-09-04T17:46:17.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4175999999999999e-05 2024-09-04T17:46:17.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:17.779 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.426561+0000", 2024-09-04T17:46:17.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016610789000000001 2024-09-04T17:46:17.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.780 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:17.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.426580+0000", 2024-09-04T17:46:17.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8851e-05 2024-09-04T17:46:17.780 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.780 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:17.780 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.780 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6031 3.8 3:146f7794:::rbd_data.1134cfbce83d.0000000000000008:6 [sparse-read 1638400~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:17.780 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.410347+0000", 2024-09-04T17:46:17.781 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.2990118499999994, 2024-09-04T17:46:17.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016074589, 2024-09-04T17:46:17.781 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:17.781 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:17.781 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:17.781 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:17.781 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:17.781 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:17.781 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6031 2024-09-04T17:46:17.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.781 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:17.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:17.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.410347+0000", 2024-09-04T17:46:17.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:17.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.410347+0000", 2024-09-04T17:46:17.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:17.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.410348+0000", 2024-09-04T17:46:17.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.26e-06 2024-09-04T17:46:17.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:17.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.410354+0000", 2024-09-04T17:46:17.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3399999999999997e-06 2024-09-04T17:46:17.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:17.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.410371+0000", 2024-09-04T17:46:17.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7433000000000001e-05 2024-09-04T17:46:17.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:17.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.410384+0000", 2024-09-04T17:46:17.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2982999999999999e-05 2024-09-04T17:46:17.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:17.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.424881+0000", 2024-09-04T17:46:17.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014497001000000001 2024-09-04T17:46:17.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:17.785 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.425114+0000", 2024-09-04T17:46:17.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023314200000000001 2024-09-04T17:46:17.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.785 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:17.785 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.426422+0000", 2024-09-04T17:46:17.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0013074300000000001 2024-09-04T17:46:17.785 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.785 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:17.785 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.785 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6032 3.8 3.29eef628 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:17.786 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.410402+0000", 2024-09-04T17:46:17.786 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.2989566739999994, 2024-09-04T17:46:17.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016123208999999999, 2024-09-04T17:46:17.786 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:17.786 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:17.786 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-04T17:46:17.786 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:17.786 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:17.786 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:17.786 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6032 2024-09-04T17:46:17.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.786 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:17.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:17.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.410402+0000", 2024-09-04T17:46:17.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:17.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.410402+0000", 2024-09-04T17:46:17.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:17.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.410403+0000", 2024-09-04T17:46:17.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.203e-06 2024-09-04T17:46:17.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:17.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.410409+0000", 2024-09-04T17:46:17.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2129999999999999e-06 2024-09-04T17:46:17.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:17.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.410425+0000", 2024-09-04T17:46:17.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6566999999999999e-05 2024-09-04T17:46:17.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:17.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.410438+0000", 2024-09-04T17:46:17.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2916e-05 2024-09-04T17:46:17.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:17.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.426503+0000", 2024-09-04T17:46:17.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016065313000000001 2024-09-04T17:46:17.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:17.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.426525+0000", 2024-09-04T17:46:17.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1997e-05 2024-09-04T17:46:17.789 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.790 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:17.790 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.790 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6065 3.4 3.a5455e04 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:17.790 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.410457+0000", 2024-09-04T17:46:17.790 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.2989016909999993, 2024-09-04T17:46:17.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016209151000000002, 2024-09-04T17:46:17.790 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:17.790 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:17.790 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-04T17:46:17.790 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:17.790 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:17.791 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:17.791 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6065 2024-09-04T17:46:17.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.791 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:17.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:17.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.410457+0000", 2024-09-04T17:46:17.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:17.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.410457+0000", 2024-09-04T17:46:17.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:17.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.410458+0000", 2024-09-04T17:46:17.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2300000000000001e-06 2024-09-04T17:46:17.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:17.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.410468+0000", 2024-09-04T17:46:17.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7929999999999992e-06 2024-09-04T17:46:17.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:17.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.410486+0000", 2024-09-04T17:46:17.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7727000000000002e-05 2024-09-04T17:46:17.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:17.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.410499+0000", 2024-09-04T17:46:17.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3566e-05 2024-09-04T17:46:17.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:17.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.426648+0000", 2024-09-04T17:46:17.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016148164 2024-09-04T17:46:17.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:17.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.426666+0000", 2024-09-04T17:46:17.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8671000000000001e-05 2024-09-04T17:46:17.794 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.794 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:17.794 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.794 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6066 3.4 3.a5455e04 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:17.794 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.410518+0000", 2024-09-04T17:46:17.794 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.2988405420000007, 2024-09-04T17:46:17.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016226063999999998, 2024-09-04T17:46:17.794 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:17.794 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:17.795 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-04T17:46:17.795 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:17.795 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:17.795 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:17.795 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6066 2024-09-04T17:46:17.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.795 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:17.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:17.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.410518+0000", 2024-09-04T17:46:17.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:17.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.410518+0000", 2024-09-04T17:46:17.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:17.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.410520+0000", 2024-09-04T17:46:17.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.24e-06 2024-09-04T17:46:17.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:17.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.410525+0000", 2024-09-04T17:46:17.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5729999999999998e-06 2024-09-04T17:46:17.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:17.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.410542+0000", 2024-09-04T17:46:17.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6566999999999999e-05 2024-09-04T17:46:17.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:17.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.410555+0000", 2024-09-04T17:46:17.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3162999999999999e-05 2024-09-04T17:46:17.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:17.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.426726+0000", 2024-09-04T17:46:17.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016170812999999999 2024-09-04T17:46:17.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:17.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.426744+0000", 2024-09-04T17:46:17.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8708000000000001e-05 2024-09-04T17:46:17.798 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.798 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:17.798 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.798 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6266 3.4 3:207aa2a5:::rbd_data.1134cfbce83d.0000000000000011:6 [sparse-read 393216~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:17.798 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.435295+0000", 2024-09-04T17:46:17.799 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.2740638549999996, 2024-09-04T17:46:17.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015174217, 2024-09-04T17:46:17.799 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:17.799 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:17.799 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:17.799 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:17.799 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:17.799 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:17.799 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6266 2024-09-04T17:46:17.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.799 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:17.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:17.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.435295+0000", 2024-09-04T17:46:17.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:17.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.435295+0000", 2024-09-04T17:46:17.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:17.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.435296+0000", 2024-09-04T17:46:17.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3399999999999998e-07 2024-09-04T17:46:17.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:17.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.435299+0000", 2024-09-04T17:46:17.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0860000000000002e-06 2024-09-04T17:46:17.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:17.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.435308+0000", 2024-09-04T17:46:17.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2110000000000004e-06 2024-09-04T17:46:17.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:17.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.435315+0000", 2024-09-04T17:46:17.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8970000000000002e-06 2024-09-04T17:46:17.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:17.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.449937+0000", 2024-09-04T17:46:17.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014622457 2024-09-04T17:46:17.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:17.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.450232+0000", 2024-09-04T17:46:17.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00029506500000000001 2024-09-04T17:46:17.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:17.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.450469+0000", 2024-09-04T17:46:17.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023676700000000001 2024-09-04T17:46:17.803 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.803 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:17.803 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.803 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6267 3.4 3:207aa2a5:::rbd_data.1134cfbce83d.0000000000000011:6 [read 786432~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:17.803 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.435324+0000", 2024-09-04T17:46:17.803 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.2740348210000008, 2024-09-04T17:46:17.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01547368, 2024-09-04T17:46:17.803 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:17.803 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:17.804 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:17.804 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:17.804 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:17.804 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:17.804 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6267 2024-09-04T17:46:17.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.804 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:17.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:17.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.435324+0000", 2024-09-04T17:46:17.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:17.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.435324+0000", 2024-09-04T17:46:17.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:17.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:17.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.435325+0000", 2024-09-04T17:46:17.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6599999999999996e-07 2024-09-04T17:46:17.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:17.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.435335+0000", 2024-09-04T17:46:17.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0091e-05 2024-09-04T17:46:17.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:17.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.435344+0000", 2024-09-04T17:46:17.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2520000000000005e-06 2024-09-04T17:46:17.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:17.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.435351+0000", 2024-09-04T17:46:17.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.922e-06 2024-09-04T17:46:17.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:17.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.450527+0000", 2024-09-04T17:46:17.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015175904 2024-09-04T17:46:17.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:17.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.450669+0000", 2024-09-04T17:46:17.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000142208 2024-09-04T17:46:17.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:17.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:17.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:17.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.450798+0000", 2024-09-04T17:46:17.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000128637 2024-09-04T17:46:17.807 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.807 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:17.807 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.807 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:17.808 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:17.808 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:17.809 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-04T17:46:17.810 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:17.809+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:17.868 INFO:teuthology.orchestra.run.smithi092.stdout:903 read 0x6a8fcc1 thru 0x6a9bce2 (0xc022 bytes) 2024-09-04T17:46:17.870 INFO:teuthology.orchestra.run.smithi092.stdout:904 write 0xb37620d thru 0xb380256 (0xa04a bytes) 2024-09-04T17:46:17.910 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:17.909+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:17.978 INFO:teuthology.orchestra.run.smithi092.stdout:905 read 0x3d334ba thru 0x3d3ce37 (0x997e bytes) 2024-09-04T17:46:17.979 INFO:teuthology.orchestra.run.smithi092.stdout:907 writesame 0x9a3d102 thru 0x9a3d466 data_size 0x1(0x365 bytes) 2024-09-04T17:46:18.005 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:18.005 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:46:18.006 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:46:18.006 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:18.012 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:46:18.011+0000 7f505d642640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26446) UID: 0 2024-09-04T17:46:18.022 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-04T17:46:18.042 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-04T17:46:18.072 INFO:teuthology.orchestra.run.smithi092.stdout:908 write 0x9f18b3d thru 0x9f1c87a (0x3d3e bytes) 2024-09-04T17:46:18.113 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:46:18.111+0000 7f505d642640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26446) UID: 0 2024-09-04T17:46:18.157 INFO:teuthology.orchestra.run.smithi092.stdout:910 trunc from 0xe311989 to 0x7b9c290 2024-09-04T17:46:18.215 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:46:18.211+0000 7f505d642640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26446) UID: 0 2024-09-04T17:46:18.239 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:18.239 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:46:18.240 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:46:18.240 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:46:18.240 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:18.258 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:46:18.258 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:46:18.259 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-04T17:46:18.290 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:46:18.291 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:46:18.314 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:18.311+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:18.316 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-04T17:46:18.319 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-04T17:46:18.319 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-04T17:46:18.319 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-04T17:46:18.351 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:46:18.351 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:46:18.373 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:46:18.373 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:46:18.402 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-04T17:46:18.402 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-04T17:46:18.415 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:18.417+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:18.421 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:18.421 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:18.424 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:46:18.424 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:46:18.428 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-04T17:46:18.429 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-04T17:46:18.438 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-04T17:46:18.446 INFO:teuthology.orchestra.run.smithi092.stdout:911 write 0x978046a thru 0x978ef07 (0xea9e bytes) 2024-09-04T17:46:18.458 INFO:teuthology.orchestra.run.smithi092.stdout:912 write 0x9dd55f5 thru 0x9de399d (0xe3a9 bytes) 2024-09-04T17:46:18.460 INFO:teuthology.orchestra.run.smithi092.stdout:913 read 0x907979e thru 0x90857d7 (0xc03a bytes) 2024-09-04T17:46:18.460 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:18.471 INFO:teuthology.orchestra.run.smithi092.stdout:916 trunc from 0x9de399e to 0x2fef309 2024-09-04T17:46:18.481 INFO:teuthology.orchestra.run.smithi092.stdout:918 write 0x1e270d3 thru 0x1e30dd1 (0x9cff bytes) 2024-09-04T17:46:18.483 INFO:teuthology.orchestra.run.smithi092.stdout:920 punch from 0x80ff71 to 0x81bfb5, (0xc044 bytes) 2024-09-04T17:46:18.483 INFO:teuthology.orchestra.run.smithi092.stdout:922 read 0x18447c1 thru 0x1852151 (0xd991 bytes) 2024-09-04T17:46:18.496 INFO:teuthology.orchestra.run.smithi092.stdout:923 write 0x617f833 thru 0x618dd70 (0xe53e bytes) 2024-09-04T17:46:18.513 INFO:teuthology.orchestra.run.smithi092.stdout:925 write 0xa9c4ff1 thru 0xa9c7684 (0x2694 bytes) 2024-09-04T17:46:18.515 INFO:teuthology.orchestra.run.smithi092.stdout:926 write 0x21008ae thru 0x21018e4 (0x1037 bytes) 2024-09-04T17:46:18.516 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:46:18.515+0000 7f505d642640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26446) UID: 0 2024-09-04T17:46:18.516 INFO:teuthology.orchestra.run.smithi092.stdout:932 write 0x44aaa5f thru 0x44b2909 (0x7eab bytes) 2024-09-04T17:46:18.536 INFO:teuthology.orchestra.run.smithi092.stdout:933 trunc from 0xa9c7685 to 0xc45264 2024-09-04T17:46:18.577 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:18.578 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:18.588 INFO:teuthology.orchestra.run.smithi092.stdout:934 writesame 0xcab7cf9 thru 0xcac3091 data_size 0x1(0xb399 bytes) 2024-09-04T17:46:18.588 INFO:teuthology.orchestra.run.smithi092.stdout:935 read 0x3a03422 thru 0x3a0543e (0x201d bytes) 2024-09-04T17:46:18.588 INFO:teuthology.orchestra.run.smithi092.stdout:936 read 0x435bba3 thru 0x4368e3f (0xd29d bytes) 2024-09-04T17:46:18.589 INFO:teuthology.orchestra.run.smithi092.stdout:937 read 0x9dee429 thru 0x9dee7f1 (0x3c9 bytes) 2024-09-04T17:46:18.590 INFO:teuthology.orchestra.run.smithi092.stdout:939 punch from 0xbaaa8e7 to 0xbaaf658, (0x4d71 bytes) 2024-09-04T17:46:18.592 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-04T17:46:18.603 INFO:teuthology.orchestra.run.smithi092.stdout:940 writesame 0xd5b3a05 thru 0xd5bba1f data_size 0x7(0x801b bytes) 2024-09-04T17:46:18.603 INFO:teuthology.orchestra.run.smithi092.stdout:941 write 0x6c1e936 thru 0x6c1fbc4 (0x128f bytes) 2024-09-04T17:46:18.605 INFO:teuthology.orchestra.run.smithi092.stdout:943 write 0x3c2dd4d thru 0x3c38170 (0xa424 bytes) 2024-09-04T17:46:18.607 INFO:teuthology.orchestra.run.smithi092.stdout:944 read 0x5fd3d0b thru 0x5fd4fdd (0x12d3 bytes) 2024-09-04T17:46:18.607 INFO:teuthology.orchestra.run.smithi092.stdout:945 read 0xd43fb69 thru 0xd4429c8 (0x2e60 bytes) 2024-09-04T17:46:18.616 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:18.617+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:18.616 INFO:teuthology.orchestra.run.smithi092.stdout:946 trunc from 0xd5bba20 to 0x8198b46 2024-09-04T17:46:18.633 INFO:teuthology.orchestra.run.smithi092.stdout:948 punch from 0x5de79a8 to 0x5de83fd, (0xa55 bytes) 2024-09-04T17:46:18.633 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:18.633 INFO:teuthology.orchestra.run.smithi092.stdout:952 read 0x5928fcf thru 0x592b4bb (0x24ed bytes) 2024-09-04T17:46:18.645 INFO:teuthology.orchestra.run.smithi092.stdout:953 writesame 0xa04a9a2 thru 0xa05720b data_size 0x33(0xc86a bytes) 2024-09-04T17:46:18.647 INFO:teuthology.orchestra.run.smithi092.stdout:954 read 0xb794e1 thru 0xb86b20 (0xd640 bytes) 2024-09-04T17:46:18.648 INFO:teuthology.orchestra.run.smithi092.stdout:955 read 0x2da622 thru 0x2dc943 (0x2322 bytes) 2024-09-04T17:46:18.649 INFO:teuthology.orchestra.run.smithi092.stdout:956 punch from 0x43ce1b to 0x446cf9, (0x9ede bytes) 2024-09-04T17:46:18.649 INFO:teuthology.orchestra.run.smithi092.stdout:957 punch from 0x2f2ea61 to 0x2f3b2ca, (0xc869 bytes) 2024-09-04T17:46:18.649 INFO:teuthology.orchestra.run.smithi092.stdout:958 read 0x22eb6c8 thru 0x22ed702 (0x203b bytes) 2024-09-04T17:46:18.650 INFO:teuthology.orchestra.run.smithi092.stdout:961 punch from 0x10f963b to 0x1101e8c, (0x8851 bytes) 2024-09-04T17:46:18.650 INFO:teuthology.orchestra.run.smithi092.stdout:962 read 0x7338519 thru 0x73400fb (0x7be3 bytes) 2024-09-04T17:46:18.651 INFO:teuthology.orchestra.run.smithi092.stdout:963 read 0x72f5311 thru 0x7302b92 (0xd882 bytes) 2024-09-04T17:46:18.652 INFO:teuthology.orchestra.run.smithi092.stdout:965 read 0x4a8b8f7 thru 0x4a94646 (0x8d50 bytes) 2024-09-04T17:46:18.666 INFO:teuthology.orchestra.run.smithi092.stdout:966 writesame 0xbe844a1 thru 0xbe925ac data_size 0x6(0xe10c bytes) 2024-09-04T17:46:18.668 INFO:teuthology.orchestra.run.smithi092.stdout:969 writesame 0x939511b thru 0x939d930 data_size 0x1(0x8816 bytes) 2024-09-04T17:46:18.692 INFO:teuthology.orchestra.run.smithi092.stdout:970 writesame 0x3861f03 thru 0x38691ac data_size 0x2(0x72aa bytes) 2024-09-04T17:46:18.707 INFO:teuthology.orchestra.run.smithi092.stdout:971 read 0x7aff3e9 thru 0x7b07a86 (0x869e bytes) 2024-09-04T17:46:18.708 INFO:teuthology.orchestra.run.smithi092.stdout:972 read 0x4dce34d thru 0x4dd7f45 (0x9bf9 bytes) 2024-09-04T17:46:18.709 INFO:teuthology.orchestra.run.smithi092.stdout:977 punch from 0x876ca8c to 0x876fa44, (0x2fb8 bytes) 2024-09-04T17:46:18.709 INFO:teuthology.orchestra.run.smithi092.stdout:978 write 0xa3dae79 thru 0xa3db74f (0x8d7 bytes) 2024-09-04T17:46:18.712 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:18.712 INFO:teuthology.orchestra.run.smithi092.stdout:980 punch from 0x9ce6975 to 0x9ceb4fb, (0x4b86 bytes) 2024-09-04T17:46:18.712 INFO:teuthology.orchestra.run.smithi092.stdout:981 clone 10 order 24 su 524288 sc 4 2024-09-04T17:46:18.717 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:18.717+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:18.764 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:18.764 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:18.778 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-04T17:46:18.818 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:18.817+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:18.900 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:18.900 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:18.914 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-04T17:46:18.920 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:18.921+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:19.021 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:19.021+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:19.032 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:19.032 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:19.047 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-04T17:46:19.122 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:19.121+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:19.164 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:19.165 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:19.180 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-04T17:46:19.223 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:19.225+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:19.286 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is set 2024-09-04T17:46:19.311 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:19.312 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:19.324 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:19.323+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:19.325 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-04T17:46:19.424 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:19.423+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:19.434 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone9 from 0xbe925ad (overlap 0xc45264) to 0xb0b08d 2024-09-04T17:46:19.507 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:19.507 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:19.522 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-04T17:46:19.563 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:46:19.563+0000 7f505d642640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26446) UID: 0 2024-09-04T17:46:19.609 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #8, image image_client.0-clone8 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent9 2024-09-04T17:46:19.627 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:19.623+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:19.710 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:19.710 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:19.727 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-04T17:46:19.771 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:19.771+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:19.829 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:19.827+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:19.896 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:19.897 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:19.898 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [3] live_osds: [0, 1, 2, 4, 5, 6, 7] 2024-09-04T17:46:19.898 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-04T17:46:19.898 INFO:tasks.thrashosds.thrasher:Reviving osd 3 2024-09-04T17:46:19.898 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-04T17:46:19.898 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-04T17:46:19.901 INFO:tasks.ceph.osd.3:Started 2024-09-04T17:46:19.901 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-04T17:46:19.910 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-04T17:46:19.947 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:19.949+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:20.031 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:20.033+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:20.047 INFO:teuthology.orchestra.run.smithi092.stderr:no valid command found; 10 closest matches: 2024-09-04T17:46:20.047 INFO:teuthology.orchestra.run.smithi092.stderr:0 2024-09-04T17:46:20.047 INFO:teuthology.orchestra.run.smithi092.stderr:1 2024-09-04T17:46:20.047 INFO:teuthology.orchestra.run.smithi092.stderr:2 2024-09-04T17:46:20.047 INFO:teuthology.orchestra.run.smithi092.stderr:abort 2024-09-04T17:46:20.047 INFO:teuthology.orchestra.run.smithi092.stderr:assert 2024-09-04T17:46:20.047 INFO:teuthology.orchestra.run.smithi092.stderr:config diff 2024-09-04T17:46:20.047 INFO:teuthology.orchestra.run.smithi092.stderr:config diff get 2024-09-04T17:46:20.047 INFO:teuthology.orchestra.run.smithi092.stderr:config get 2024-09-04T17:46:20.048 INFO:teuthology.orchestra.run.smithi092.stderr:config help [] 2024-09-04T17:46:20.048 INFO:teuthology.orchestra.run.smithi092.stderr:config set ... 2024-09-04T17:46:20.048 INFO:teuthology.orchestra.run.smithi092.stderr:admin_socket: invalid command 2024-09-04T17:46:20.048 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-04T17:46:20.049 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-3, ['dump_ops_in_flight'] 2024-09-04T17:46:20.074 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:20.074 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:20.089 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-04T17:46:20.132 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:20.133+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:20.209 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:20.209 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:20.223 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-04T17:46:20.267 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:20.267+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:20.334 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:46:20.331+0000 7f505d642640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26446) UID: 0 2024-09-04T17:46:20.340 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:20.340 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:20.354 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-04T17:46:20.430 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-04T17:46:20.435 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:46:20.435+0000 7f505d642640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26446) UID: 0 2024-09-04T17:46:20.505 INFO:teuthology.orchestra.run.smithi092.stdout:982 punch from 0x6573c50 to 0x6574d5d, (0x110d bytes) 2024-09-04T17:46:20.513 INFO:teuthology.orchestra.run.smithi092.stdout:983 write 0xb3d29c4 thru 0xb3d418a (0x17c7 bytes) 2024-09-04T17:46:20.517 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:20.517 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:20.529 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:20.529 INFO:teuthology.orchestra.run.smithi092.stdout:985 read 0x4e988a8 thru 0x4ea4d02 (0xc45b bytes) 2024-09-04T17:46:20.532 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-04T17:46:20.538 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:20.537+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:20.544 INFO:teuthology.orchestra.run.smithi092.stdout:987 trunc from 0xbe925ad to 0x22f36c6 2024-09-04T17:46:20.597 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:46:20.597 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:46:20.615 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:46:20.615 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:46:20.618 INFO:teuthology.orchestra.run.smithi092.stdout:989 trunc from 0x22f36c6 to 0x60c822b 2024-09-04T17:46:20.626 INFO:teuthology.orchestra.run.smithi092.stdout:990 punch from 0x60adddf to 0x60ba757, (0xc978 bytes) 2024-09-04T17:46:20.626 INFO:teuthology.orchestra.run.smithi092.stdout:992 read 0x12b1a83 thru 0x12bcc5c (0xb1da bytes) 2024-09-04T17:46:20.627 INFO:teuthology.orchestra.run.smithi092.stdout:995 punch from 0x300412c to 0x300cf39, (0x8e0d bytes) 2024-09-04T17:46:20.635 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-04T17:46:20.636 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-04T17:46:20.636 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-04T17:46:20.638 INFO:teuthology.orchestra.run.smithi092.stdout:998 trunc from 0x60c822b to 0xd3900d0 2024-09-04T17:46:20.638 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:20.635+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:20.645 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:20.653 INFO:teuthology.orchestra.run.smithi092.stdout:1000 write 0xd60ddd1 thru 0xd610bfa (0x2e2a bytes) 2024-09-04T17:46:20.658 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:46:20.658 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:46:20.659 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:20.659 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:20.659 INFO:teuthology.orchestra.run.smithi092.stdout:1005 write 0x215b5ac thru 0x216afd1 (0xfa26 bytes) 2024-09-04T17:46:20.662 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:20.662 INFO:teuthology.orchestra.run.smithi092.stdout:1007 punch from 0x5f7d4d0 to 0x5f86647, (0x9177 bytes) 2024-09-04T17:46:20.662 INFO:teuthology.orchestra.run.smithi092.stdout:1010 read 0xa67e730 thru 0xa68e5b9 (0xfe8a bytes) 2024-09-04T17:46:20.663 INFO:teuthology.orchestra.run.smithi092.stdout:1012 punch from 0xc8f73a2 to 0xc8ffc01, (0x885f bytes) 2024-09-04T17:46:20.663 INFO:teuthology.orchestra.run.smithi092.stdout:1013 read 0x288da7e thru 0x2898c8a (0xb20d bytes) 2024-09-04T17:46:20.663 INFO:teuthology.orchestra.run.smithi092.stdout:1014 write 0x5de0175 thru 0x5deaa1a (0xa8a6 bytes) 2024-09-04T17:46:20.668 INFO:teuthology.orchestra.run.smithi092.stdout:1015 write 0xce0adb6 thru 0xce0bf17 (0x1162 bytes) 2024-09-04T17:46:20.670 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:20.670 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:46:20.671 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:46:20.671 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:46:20.671 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:20.672 INFO:teuthology.orchestra.run.smithi092.stdout:1017 read 0x3fb9abb thru 0x3fc7fa7 (0xe4ed bytes) 2024-09-04T17:46:20.672 INFO:teuthology.orchestra.run.smithi092.stdout:1019 writesame 0x824404a thru 0x8244179 data_size 0x4(0x130 bytes) 2024-09-04T17:46:20.678 INFO:teuthology.orchestra.run.smithi092.stdout:1020 read 0x8fb333e thru 0x8fb7e9b (0x4b5e bytes) 2024-09-04T17:46:20.679 INFO:teuthology.orchestra.run.smithi092.stdout:1021 writesame 0x9c37324 thru 0x9c40b26 data_size 0x2b(0x9803 bytes) 2024-09-04T17:46:20.679 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:46:20.680 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:46:20.681 INFO:teuthology.orchestra.run.smithi092.stdout:1023 write 0x9e3235f thru 0x9e39ffb (0x7c9d bytes) 2024-09-04T17:46:20.684 INFO:teuthology.orchestra.run.smithi092.stdout:1024 read 0x57051c0 thru 0x5707480 (0x22c1 bytes) 2024-09-04T17:46:20.684 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:20.685 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-04T17:46:20.694 INFO:teuthology.orchestra.run.smithi092.stdout:1027 writesame 0xea50ae5 thru 0xea5972b data_size 0x1(0x8c47 bytes) 2024-09-04T17:46:20.695 INFO:teuthology.orchestra.run.smithi092.stdout:1028 write 0x9848c27 thru 0x9858068 (0xf442 bytes) 2024-09-04T17:46:20.698 INFO:teuthology.orchestra.run.smithi092.stdout:1029 read 0x5875887 thru 0x58812b2 (0xba2c bytes) 2024-09-04T17:46:20.698 INFO:teuthology.orchestra.run.smithi092.stdout:1030 write 0xbd8e88b thru 0xbd90d5b (0x24d1 bytes) 2024-09-04T17:46:20.702 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-04T17:46:20.703 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-04T17:46:20.705 INFO:teuthology.orchestra.run.smithi092.stdout:1031 writesame 0xaf74c54 thru 0xaf7b7bc data_size 0x1(0x6b69 bytes) 2024-09-04T17:46:20.724 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:20.724 INFO:teuthology.orchestra.run.smithi092.stdout:1033 read 0x141f7b8 thru 0x1422fdf (0x3828 bytes) 2024-09-04T17:46:20.725 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:20.725 INFO:teuthology.orchestra.run.smithi092.stdout:1036 write 0x70ad5b4 thru 0x70b40a6 (0x6af3 bytes) 2024-09-04T17:46:20.731 INFO:teuthology.orchestra.run.smithi092.stdout:1037 read 0xbbed88 thru 0xbc811c (0x9395 bytes) 2024-09-04T17:46:20.732 INFO:teuthology.orchestra.run.smithi092.stdout:1038 writesame 0x6ed55d1 thru 0x6ed966e data_size 0x9(0x409e bytes) 2024-09-04T17:46:20.735 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:46:20.735 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:46:20.737 INFO:teuthology.orchestra.run.smithi092.stdout:1040 punch from 0x583addc to 0x5847137, (0xc35b bytes) 2024-09-04T17:46:20.737 INFO:teuthology.orchestra.run.smithi092.stdout:1041 write 0x56471ae thru 0x5656470 (0xf2c3 bytes) 2024-09-04T17:46:20.741 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-04T17:46:20.742 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-04T17:46:20.742 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:20.739+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:20.755 INFO:teuthology.orchestra.run.smithi092.stdout:1042 trunc from 0xea5972c to 0x5ddadb3 2024-09-04T17:46:20.796 INFO:teuthology.orchestra.run.smithi092.stdout:1043 write 0xbe7872f thru 0xbe7b889 (0x315b bytes) 2024-09-04T17:46:20.814 INFO:teuthology.orchestra.run.smithi092.stdout:1044 writesame 0xccad65f thru 0xccbb5dc data_size 0x1(0xdf7e bytes) 2024-09-04T17:46:20.814 INFO:teuthology.orchestra.run.smithi092.stdout:1045 write 0xa960299 thru 0xa967ca9 (0x7a11 bytes) 2024-09-04T17:46:20.825 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:20.825 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:46:20.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:46:20.826 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:46:20.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.826 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:1095 3.1 3:86c08994:::rbd_data.11346251f094.000000000000006e:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-04T17:46:20.826 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:02.246406+0000", 2024-09-04T17:46:20.826 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.579167813000002, 2024-09-04T17:46:20.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.024871196000000002, 2024-09-04T17:46:20.826 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:20.826 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:20.826 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:20.826 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:20.826 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:20.826 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:20.826 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 1095 2024-09-04T17:46:20.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.827 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:20.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:20.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.246406+0000", 2024-09-04T17:46:20.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:20.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:20.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.246406+0000", 2024-09-04T17:46:20.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:20.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:20.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.246409+0000", 2024-09-04T17:46:20.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6599999999999999e-06 2024-09-04T17:46:20.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:20.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.246419+0000", 2024-09-04T17:46:20.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8840000000000006e-06 2024-09-04T17:46:20.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:20.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.246439+0000", 2024-09-04T17:46:20.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0088000000000001e-05 2024-09-04T17:46:20.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:20.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.246448+0000", 2024-09-04T17:46:20.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9339999999999995e-06 2024-09-04T17:46:20.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:20.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.246511+0000", 2024-09-04T17:46:20.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3727999999999997e-05 2024-09-04T17:46:20.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:20.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.246588+0000", 2024-09-04T17:46:20.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6143999999999995e-05 2024-09-04T17:46:20.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-04T17:46:20.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.250919+0000", 2024-09-04T17:46:20.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0043314529999999999 2024-09-04T17:46:20.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:20.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.271170+0000", 2024-09-04T17:46:20.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020251531999999999 2024-09-04T17:46:20.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:46:20.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.271186+0000", 2024-09-04T17:46:20.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5302e-05 2024-09-04T17:46:20.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:20.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.271230+0000", 2024-09-04T17:46:20.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4576000000000001e-05 2024-09-04T17:46:20.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:20.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.271277+0000", 2024-09-04T17:46:20.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6894999999999997e-05 2024-09-04T17:46:20.831 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:20.832 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:20.832 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:20.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.832 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6103 3.1f 3:fec7e474:::rbd_data.1134cfbce83d.0000000000000003:6 [read 1736704~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:20.832 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.421848+0000", 2024-09-04T17:46:20.832 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 12.403725716, 2024-09-04T17:46:20.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027313884, 2024-09-04T17:46:20.832 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:20.832 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:20.832 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:20.832 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:20.832 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:20.833 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:20.833 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6103 2024-09-04T17:46:20.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.833 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:20.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:20.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.421848+0000", 2024-09-04T17:46:20.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:20.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:20.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.421848+0000", 2024-09-04T17:46:20.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:20.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:20.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.421849+0000", 2024-09-04T17:46:20.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.144e-06 2024-09-04T17:46:20.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:20.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.421855+0000", 2024-09-04T17:46:20.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.003e-06 2024-09-04T17:46:20.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:20.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.421872+0000", 2024-09-04T17:46:20.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7193e-05 2024-09-04T17:46:20.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.835 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:20.835 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.421886+0000", 2024-09-04T17:46:20.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.331e-05 2024-09-04T17:46:20.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.835 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:20.835 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.444849+0000", 2024-09-04T17:46:20.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.022962755000000001 2024-09-04T17:46:20.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.835 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:20.835 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.444990+0000", 2024-09-04T17:46:20.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014129999999999999 2024-09-04T17:46:20.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:20.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.449162+0000", 2024-09-04T17:46:20.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041721789999999998 2024-09-04T17:46:20.836 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:20.836 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:20.836 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:20.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.836 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6140 3.17 3:ed419000:::rbd_data.1134cfbce83d.000000000000000b:6 [sparse-read 622592~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:20.836 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.422576+0000", 2024-09-04T17:46:20.836 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 12.402998128, 2024-09-04T17:46:20.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025138273999999999, 2024-09-04T17:46:20.837 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:20.837 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:20.837 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:20.837 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:20.837 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:20.837 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:20.837 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6140 2024-09-04T17:46:20.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.837 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:20.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.837 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:20.837 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422576+0000", 2024-09-04T17:46:20.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:20.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:20.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422576+0000", 2024-09-04T17:46:20.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:20.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:20.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422578+0000", 2024-09-04T17:46:20.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8870000000000001e-06 2024-09-04T17:46:20.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:20.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422587+0000", 2024-09-04T17:46:20.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5999999999999996e-06 2024-09-04T17:46:20.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:20.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422617+0000", 2024-09-04T17:46:20.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9339000000000001e-05 2024-09-04T17:46:20.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:20.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422636+0000", 2024-09-04T17:46:20.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9524000000000001e-05 2024-09-04T17:46:20.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:20.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.445586+0000", 2024-09-04T17:46:20.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.022950084999999999 2024-09-04T17:46:20.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:20.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.445788+0000", 2024-09-04T17:46:20.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000202029 2024-09-04T17:46:20.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:20.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.447714+0000", 2024-09-04T17:46:20.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00192581 2024-09-04T17:46:20.841 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:20.841 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:20.841 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:20.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.841 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6141 3.17 3:ed419000:::rbd_data.1134cfbce83d.000000000000000b:6 [sparse-read 983040~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:20.841 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.422660+0000", 2024-09-04T17:46:20.841 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 12.402913464999999, 2024-09-04T17:46:20.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025710566000000001, 2024-09-04T17:46:20.841 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:20.841 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:20.841 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:20.841 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:20.842 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:20.842 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:20.842 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6141 2024-09-04T17:46:20.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.842 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:20.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:20.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422660+0000", 2024-09-04T17:46:20.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:20.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:20.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422660+0000", 2024-09-04T17:46:20.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:20.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:20.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422662+0000", 2024-09-04T17:46:20.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.313e-06 2024-09-04T17:46:20.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:20.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422691+0000", 2024-09-04T17:46:20.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9207e-05 2024-09-04T17:46:20.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:20.844 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422711+0000", 2024-09-04T17:46:20.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0225999999999999e-05 2024-09-04T17:46:20.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:20.844 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422727+0000", 2024-09-04T17:46:20.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5982999999999999e-05 2024-09-04T17:46:20.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:20.844 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.447767+0000", 2024-09-04T17:46:20.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025039558 2024-09-04T17:46:20.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:20.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.447871+0000", 2024-09-04T17:46:20.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104287 2024-09-04T17:46:20.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:20.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.448371+0000", 2024-09-04T17:46:20.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00049999199999999999 2024-09-04T17:46:20.845 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:20.845 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:20.845 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:20.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.846 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6142 3.17 3:ed419000:::rbd_data.1134cfbce83d.000000000000000b:6 [sparse-read 1343488~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:20.846 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.422750+0000", 2024-09-04T17:46:20.846 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 12.402824233, 2024-09-04T17:46:20.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026566215000000001, 2024-09-04T17:46:20.846 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:20.846 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:20.846 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:20.846 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:20.846 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:20.846 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:20.846 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6142 2024-09-04T17:46:20.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.846 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:20.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:20.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422750+0000", 2024-09-04T17:46:20.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:20.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:20.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422750+0000", 2024-09-04T17:46:20.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:20.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:20.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422751+0000", 2024-09-04T17:46:20.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6700000000000001e-06 2024-09-04T17:46:20.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.848 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:20.848 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422757+0000", 2024-09-04T17:46:20.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9669999999999998e-06 2024-09-04T17:46:20.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.848 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:20.848 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422775+0000", 2024-09-04T17:46:20.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8073000000000001e-05 2024-09-04T17:46:20.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.848 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:20.848 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422789+0000", 2024-09-04T17:46:20.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3709999999999999e-05 2024-09-04T17:46:20.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:20.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.448420+0000", 2024-09-04T17:46:20.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025630627999999999 2024-09-04T17:46:20.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:20.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.448590+0000", 2024-09-04T17:46:20.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000170213 2024-09-04T17:46:20.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:20.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.449316+0000", 2024-09-04T17:46:20.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00072595400000000001 2024-09-04T17:46:20.850 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:20.850 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:20.850 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:20.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.851 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6143 3.17 3:ed419000:::rbd_data.1134cfbce83d.000000000000000b:6 [read 1736704~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:20.851 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.422807+0000", 2024-09-04T17:46:20.851 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 12.402766903, 2024-09-04T17:46:20.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027271169000000001, 2024-09-04T17:46:20.851 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:20.851 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:20.851 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:20.851 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:20.851 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:20.851 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:20.851 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6143 2024-09-04T17:46:20.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.851 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:20.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:20.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422807+0000", 2024-09-04T17:46:20.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:20.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:20.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422807+0000", 2024-09-04T17:46:20.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:20.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:20.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422809+0000", 2024-09-04T17:46:20.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6139999999999999e-06 2024-09-04T17:46:20.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:20.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422814+0000", 2024-09-04T17:46:20.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1800000000000004e-06 2024-09-04T17:46:20.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:20.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422831+0000", 2024-09-04T17:46:20.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7433000000000001e-05 2024-09-04T17:46:20.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.854 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:20.854 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422844+0000", 2024-09-04T17:46:20.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3216e-05 2024-09-04T17:46:20.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.854 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:20.854 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.449397+0000", 2024-09-04T17:46:20.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026552572 2024-09-04T17:46:20.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.854 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:20.854 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.449526+0000", 2024-09-04T17:46:20.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000129207 2024-09-04T17:46:20.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.855 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:20.855 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.450078+0000", 2024-09-04T17:46:20.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00055194700000000001 2024-09-04T17:46:20.855 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:20.855 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:20.855 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:20.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.855 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6203 3.1f 3:fec7e474:::rbd_data.1134cfbce83d.0000000000000003:6 [read 327680~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:20.855 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423445+0000", 2024-09-04T17:46:20.856 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 12.402128490999999, 2024-09-04T17:46:20.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027179207, 2024-09-04T17:46:20.856 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:20.856 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:20.856 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:20.856 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:20.856 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:20.856 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:20.856 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6203 2024-09-04T17:46:20.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.856 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:20.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:20.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423445+0000", 2024-09-04T17:46:20.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:20.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.857 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:20.857 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423445+0000", 2024-09-04T17:46:20.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:20.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.857 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:20.857 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423446+0000", 2024-09-04T17:46:20.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.077e-06 2024-09-04T17:46:20.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.858 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:20.858 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423452+0000", 2024-09-04T17:46:20.858 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.113e-06 2024-09-04T17:46:20.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.858 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.858 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:20.858 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423468+0000", 2024-09-04T17:46:20.858 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6685999999999999e-05 2024-09-04T17:46:20.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.858 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.858 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:20.858 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423481+0000", 2024-09-04T17:46:20.858 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2673999999999999e-05 2024-09-04T17:46:20.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.859 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.859 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:20.859 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.449779+0000", 2024-09-04T17:46:20.859 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026297838 2024-09-04T17:46:20.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.859 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.859 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:20.859 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.450020+0000", 2024-09-04T17:46:20.859 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024114399999999999 2024-09-04T17:46:20.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.859 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.859 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:20.859 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.450625+0000", 2024-09-04T17:46:20.860 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00060467500000000005 2024-09-04T17:46:20.860 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:20.860 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:20.860 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:20.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.860 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6204 3.1f 3:fec7e474:::rbd_data.1134cfbce83d.0000000000000003:6 [read 688128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:20.860 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423498+0000", 2024-09-04T17:46:20.860 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 12.402075555, 2024-09-04T17:46:20.860 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029262579, 2024-09-04T17:46:20.860 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:20.860 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:20.861 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:20.861 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:20.861 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:20.861 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:20.861 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6204 2024-09-04T17:46:20.861 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.861 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:20.861 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.861 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:20.861 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423498+0000", 2024-09-04T17:46:20.861 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:20.861 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.861 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.862 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:20.862 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423498+0000", 2024-09-04T17:46:20.862 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:20.862 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.862 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.862 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:20.862 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423500+0000", 2024-09-04T17:46:20.862 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.28e-06 2024-09-04T17:46:20.862 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.862 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.862 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:20.862 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423506+0000", 2024-09-04T17:46:20.862 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0059999999999998e-06 2024-09-04T17:46:20.862 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.863 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:20.863 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423524+0000", 2024-09-04T17:46:20.863 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8683999999999999e-05 2024-09-04T17:46:20.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.863 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:20.863 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423537+0000", 2024-09-04T17:46:20.863 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3030000000000001e-05 2024-09-04T17:46:20.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.863 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:20.863 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.450697+0000", 2024-09-04T17:46:20.863 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027159793000000002 2024-09-04T17:46:20.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.864 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:20.864 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.450835+0000", 2024-09-04T17:46:20.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013773799999999999 2024-09-04T17:46:20.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.864 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:20.864 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.452761+0000", 2024-09-04T17:46:20.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019260480000000001 2024-09-04T17:46:20.864 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:20.864 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:20.864 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:20.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.865 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.865 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6205 3.1f 3:fec7e474:::rbd_data.1134cfbce83d.0000000000000003:6 [read 1048576~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:20.865 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423555+0000", 2024-09-04T17:46:20.865 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 12.402018571999999, 2024-09-04T17:46:20.865 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029905979999999999, 2024-09-04T17:46:20.865 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:20.865 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:20.865 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:20.865 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:20.865 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:20.865 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:20.865 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6205 2024-09-04T17:46:20.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.866 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:20.866 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.866 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:20.866 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423555+0000", 2024-09-04T17:46:20.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:20.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.866 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.866 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:20.866 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423555+0000", 2024-09-04T17:46:20.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:20.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.866 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.866 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:20.866 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423556+0000", 2024-09-04T17:46:20.867 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1599999999999999e-06 2024-09-04T17:46:20.867 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.867 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.867 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:20.867 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423562+0000", 2024-09-04T17:46:20.867 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1270000000000002e-06 2024-09-04T17:46:20.867 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.867 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.867 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:20.867 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423579+0000", 2024-09-04T17:46:20.867 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6986e-05 2024-09-04T17:46:20.867 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.867 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.867 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:20.867 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423591+0000", 2024-09-04T17:46:20.868 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2544000000000001e-05 2024-09-04T17:46:20.868 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.868 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.868 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:20.868 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.452819+0000", 2024-09-04T17:46:20.868 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029227418000000002 2024-09-04T17:46:20.868 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.868 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.868 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:20.868 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.452985+0000", 2024-09-04T17:46:20.868 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000166675 2024-09-04T17:46:20.868 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.868 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.868 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:20.869 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.453461+0000", 2024-09-04T17:46:20.869 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00047606999999999999 2024-09-04T17:46:20.869 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:20.869 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:20.869 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:20.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.869 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.869 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6206 3.1f 3:fec7e474:::rbd_data.1134cfbce83d.0000000000000003:6 [sparse-read 1409024~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:20.869 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423608+0000", 2024-09-04T17:46:20.869 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 12.401965939, 2024-09-04T17:46:20.869 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.031860753999999998, 2024-09-04T17:46:20.869 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:20.869 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:20.870 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:20.870 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:20.870 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:20.870 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:20.870 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6206 2024-09-04T17:46:20.870 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.870 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:20.870 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.870 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:20.870 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423608+0000", 2024-09-04T17:46:20.870 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:20.870 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.870 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.870 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:20.871 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423608+0000", 2024-09-04T17:46:20.871 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:20.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.871 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.871 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:20.871 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423609+0000", 2024-09-04T17:46:20.871 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0300000000000001e-06 2024-09-04T17:46:20.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.871 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.871 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:20.871 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423614+0000", 2024-09-04T17:46:20.871 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0200000000000002e-06 2024-09-04T17:46:20.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.872 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.872 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:20.872 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423630+0000", 2024-09-04T17:46:20.872 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6495999999999999e-05 2024-09-04T17:46:20.872 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.872 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.872 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:20.872 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423643+0000", 2024-09-04T17:46:20.872 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2053e-05 2024-09-04T17:46:20.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.873 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.873 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:20.873 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.453527+0000", 2024-09-04T17:46:20.873 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029884796000000002 2024-09-04T17:46:20.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.873 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.873 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:20.873 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.453632+0000", 2024-09-04T17:46:20.873 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104515 2024-09-04T17:46:20.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.873 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.873 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:20.874 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.455469+0000", 2024-09-04T17:46:20.874 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001836844 2024-09-04T17:46:20.874 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:20.874 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:20.874 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:20.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.874 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6207 3.1f 3:fec7e474:::rbd_data.1134cfbce83d.0000000000000003:6 [sparse-read 1769472~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:20.874 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423662+0000", 2024-09-04T17:46:20.874 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 12.401911476, 2024-09-04T17:46:20.874 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.033074294999999997, 2024-09-04T17:46:20.874 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:20.875 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:20.875 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:20.875 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:20.875 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:20.875 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:20.875 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6207 2024-09-04T17:46:20.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.875 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:20.875 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.875 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:20.875 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423662+0000", 2024-09-04T17:46:20.875 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:20.876 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.876 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.876 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:20.876 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423662+0000", 2024-09-04T17:46:20.876 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:20.876 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.876 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.876 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:20.876 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423664+0000", 2024-09-04T17:46:20.876 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4929999999999999e-06 2024-09-04T17:46:20.876 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.876 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.876 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:20.877 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423669+0000", 2024-09-04T17:46:20.877 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3800000000000002e-06 2024-09-04T17:46:20.877 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.877 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.877 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:20.877 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423686+0000", 2024-09-04T17:46:20.877 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7010000000000001e-05 2024-09-04T17:46:20.877 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.877 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.877 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:20.877 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423704+0000", 2024-09-04T17:46:20.877 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7710000000000002e-05 2024-09-04T17:46:20.877 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.877 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.878 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:20.878 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.455985+0000", 2024-09-04T17:46:20.878 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032281272999999999 2024-09-04T17:46:20.878 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.878 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.878 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:20.878 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.456196+0000", 2024-09-04T17:46:20.878 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000210335 2024-09-04T17:46:20.878 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.878 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.878 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:20.878 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.456737+0000", 2024-09-04T17:46:20.878 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00054109399999999999 2024-09-04T17:46:20.878 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:20.879 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:20.879 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:20.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.879 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.879 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6243 3.17 3:ed419000:::rbd_data.1134cfbce83d.000000000000000b:6 [sparse-read 294912~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:20.879 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423721+0000", 2024-09-04T17:46:20.879 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 12.40185269, 2024-09-04T17:46:20.879 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032666132000000001, 2024-09-04T17:46:20.879 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:20.879 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:20.879 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:20.879 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:20.880 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:20.880 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:20.880 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6243 2024-09-04T17:46:20.880 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.880 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:20.880 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.880 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:20.880 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423721+0000", 2024-09-04T17:46:20.880 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:20.880 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.880 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.880 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:20.880 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423721+0000", 2024-09-04T17:46:20.880 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:20.881 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.881 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.881 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:20.881 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423722+0000", 2024-09-04T17:46:20.881 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0470000000000001e-06 2024-09-04T17:46:20.881 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.881 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.881 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:20.881 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423727+0000", 2024-09-04T17:46:20.881 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0699999999999997e-06 2024-09-04T17:46:20.881 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.881 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.881 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:20.881 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423744+0000", 2024-09-04T17:46:20.882 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6659000000000001e-05 2024-09-04T17:46:20.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.882 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:20.882 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423756+0000", 2024-09-04T17:46:20.882 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2284e-05 2024-09-04T17:46:20.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.882 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:20.882 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.455553+0000", 2024-09-04T17:46:20.882 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.031797032000000003 2024-09-04T17:46:20.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.882 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:20.883 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.455716+0000", 2024-09-04T17:46:20.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016253700000000001 2024-09-04T17:46:20.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.883 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.883 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:20.883 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.456387+0000", 2024-09-04T17:46:20.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00067150300000000005 2024-09-04T17:46:20.883 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:20.883 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:20.883 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:20.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.883 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.883 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6244 3.17 3:ed419000:::rbd_data.1134cfbce83d.000000000000000b:6 [read 688128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:20.883 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423775+0000", 2024-09-04T17:46:20.884 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 12.401798907, 2024-09-04T17:46:20.884 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.033129498, 2024-09-04T17:46:20.884 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:20.884 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:20.884 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:20.884 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:20.884 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:20.884 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:20.884 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6244 2024-09-04T17:46:20.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.884 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:20.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.885 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:20.885 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423775+0000", 2024-09-04T17:46:20.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:20.885 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.885 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.885 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:20.885 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423775+0000", 2024-09-04T17:46:20.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:20.885 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.885 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.885 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:20.885 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423776+0000", 2024-09-04T17:46:20.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1230000000000001e-06 2024-09-04T17:46:20.886 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.886 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.886 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:20.886 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423782+0000", 2024-09-04T17:46:20.886 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4929999999999998e-06 2024-09-04T17:46:20.886 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.886 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.886 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:20.886 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423799+0000", 2024-09-04T17:46:20.886 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7254000000000001e-05 2024-09-04T17:46:20.886 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.886 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.886 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:20.887 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423811+0000", 2024-09-04T17:46:20.887 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2553e-05 2024-09-04T17:46:20.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.887 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:20.887 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.456421+0000", 2024-09-04T17:46:20.887 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032609552 2024-09-04T17:46:20.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.887 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:20.887 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.456512+0000", 2024-09-04T17:46:20.887 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0593999999999994e-05 2024-09-04T17:46:20.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.888 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.888 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:20.888 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.456904+0000", 2024-09-04T17:46:20.888 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00039292899999999998 2024-09-04T17:46:20.888 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:20.888 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:20.888 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:20.888 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.888 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.888 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6245 3.17 3:ed419000:::rbd_data.1134cfbce83d.000000000000000b:6 [read 1048576~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:20.888 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423830+0000", 2024-09-04T17:46:20.888 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 12.401743901, 2024-09-04T17:46:20.888 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.033577937000000002, 2024-09-04T17:46:20.889 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:20.889 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:20.889 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:20.889 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:20.889 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:20.889 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:20.889 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6245 2024-09-04T17:46:20.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.889 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:20.889 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.889 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:20.889 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423830+0000", 2024-09-04T17:46:20.890 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:20.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.890 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.890 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:20.890 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423830+0000", 2024-09-04T17:46:20.890 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:20.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.890 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.890 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:20.890 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423831+0000", 2024-09-04T17:46:20.890 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.164e-06 2024-09-04T17:46:20.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.891 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.891 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:20.891 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423847+0000", 2024-09-04T17:46:20.891 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5573000000000002e-05 2024-09-04T17:46:20.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.891 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.891 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:20.891 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423865+0000", 2024-09-04T17:46:20.891 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8715999999999998e-05 2024-09-04T17:46:20.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.891 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.892 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:20.892 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423880+0000", 2024-09-04T17:46:20.892 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4107000000000001e-05 2024-09-04T17:46:20.892 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.892 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.892 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:20.892 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.456951+0000", 2024-09-04T17:46:20.892 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.03307181 2024-09-04T17:46:20.892 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.892 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.892 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:20.893 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.457039+0000", 2024-09-04T17:46:20.893 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7795999999999998e-05 2024-09-04T17:46:20.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.893 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.893 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:20.893 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.457408+0000", 2024-09-04T17:46:20.893 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000368771 2024-09-04T17:46:20.893 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:20.893 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:20.893 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:20.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.893 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.894 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6246 3.17 3:ed419000:::rbd_data.1134cfbce83d.000000000000000b:6 [read 1409024~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:20.894 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423897+0000", 2024-09-04T17:46:20.894 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 12.401676562, 2024-09-04T17:46:20.894 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034015888000000001, 2024-09-04T17:46:20.894 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:20.894 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:20.894 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:20.894 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:20.894 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:20.894 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:20.894 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6246 2024-09-04T17:46:20.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.894 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:20.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:20.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423897+0000", 2024-09-04T17:46:20.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:20.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:20.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423897+0000", 2024-09-04T17:46:20.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:20.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:20.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423899+0000", 2024-09-04T17:46:20.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3200000000000001e-06 2024-09-04T17:46:20.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.896 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:20.896 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423904+0000", 2024-09-04T17:46:20.896 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4369999999999998e-06 2024-09-04T17:46:20.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.896 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:20.896 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423922+0000", 2024-09-04T17:46:20.896 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8196999999999999e-05 2024-09-04T17:46:20.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.896 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:20.897 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423935+0000", 2024-09-04T17:46:20.897 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.314e-05 2024-09-04T17:46:20.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.897 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.897 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:20.897 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.457457+0000", 2024-09-04T17:46:20.897 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.033521663 2024-09-04T17:46:20.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.897 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.897 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:20.897 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.457536+0000", 2024-09-04T17:46:20.897 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8795999999999996e-05 2024-09-04T17:46:20.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.897 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.898 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:20.898 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.457913+0000", 2024-09-04T17:46:20.898 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00037733500000000002 2024-09-04T17:46:20.898 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:20.898 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:20.898 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:20.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.898 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6247 3.17 3:ed419000:::rbd_data.1134cfbce83d.000000000000000b:6 [read 1769472~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:20.898 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423953+0000", 2024-09-04T17:46:20.898 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 12.401620378, 2024-09-04T17:46:20.898 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0344287, 2024-09-04T17:46:20.898 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:20.898 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:20.899 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:20.899 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:20.899 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:20.899 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:20.899 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6247 2024-09-04T17:46:20.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.899 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:20.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.899 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:20.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423953+0000", 2024-09-04T17:46:20.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:20.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.900 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:20.900 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423953+0000", 2024-09-04T17:46:20.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:20.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.900 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:20.900 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423955+0000", 2024-09-04T17:46:20.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0529999999999999e-06 2024-09-04T17:46:20.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.900 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:20.900 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423960+0000", 2024-09-04T17:46:20.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0470000000000001e-06 2024-09-04T17:46:20.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:20.901 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423978+0000", 2024-09-04T17:46:20.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8119999999999999e-05 2024-09-04T17:46:20.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:20.901 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423991+0000", 2024-09-04T17:46:20.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2813000000000001e-05 2024-09-04T17:46:20.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:20.902 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.457947+0000", 2024-09-04T17:46:20.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.033956865000000003 2024-09-04T17:46:20.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.902 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:20.902 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.458039+0000", 2024-09-04T17:46:20.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1834000000000002e-05 2024-09-04T17:46:20.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.902 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:20.902 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.458382+0000", 2024-09-04T17:46:20.903 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00034296799999999997 2024-09-04T17:46:20.903 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:20.903 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:20.903 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:20.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.903 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6305 3.17 3:ee7cd06a:::rbd_data.113431586065.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:20.903 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.455350+0000", 2024-09-04T17:46:20.903 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 12.370223826, 2024-09-04T17:46:20.903 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029276888000000001, 2024-09-04T17:46:20.903 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:20.903 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:20.903 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:20.903 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:20.904 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:20.904 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:20.904 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6305 2024-09-04T17:46:20.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.904 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:20.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:20.904 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.455350+0000", 2024-09-04T17:46:20.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:20.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.905 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:20.905 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.455350+0000", 2024-09-04T17:46:20.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:20.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.905 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:20.905 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.455353+0000", 2024-09-04T17:46:20.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6400000000000001e-06 2024-09-04T17:46:20.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.906 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:20.906 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459745+0000", 2024-09-04T17:46:20.906 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0043925880000000002 2024-09-04T17:46:20.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.906 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.906 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:20.906 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459775+0000", 2024-09-04T17:46:20.906 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9254000000000001e-05 2024-09-04T17:46:20.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.906 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.906 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:20.906 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459789+0000", 2024-09-04T17:46:20.906 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4731e-05 2024-09-04T17:46:20.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.907 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.907 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:20.907 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459885+0000", 2024-09-04T17:46:20.907 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5502999999999997e-05 2024-09-04T17:46:20.907 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.907 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.907 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:20.907 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.460040+0000", 2024-09-04T17:46:20.907 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015491499999999999 2024-09-04T17:46:20.907 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.907 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.907 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5", 2024-09-04T17:46:20.907 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.460596+0000", 2024-09-04T17:46:20.907 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00055657500000000002 2024-09-04T17:46:20.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.908 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.908 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:20.908 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484484+0000", 2024-09-04T17:46:20.908 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.023887950000000002 2024-09-04T17:46:20.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.908 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.908 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:46:20.908 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484510+0000", 2024-09-04T17:46:20.908 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.561e-05 2024-09-04T17:46:20.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.908 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.908 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:20.908 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484569+0000", 2024-09-04T17:46:20.908 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9049999999999999e-05 2024-09-04T17:46:20.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.909 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.909 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:20.909 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484627+0000", 2024-09-04T17:46:20.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8072000000000002e-05 2024-09-04T17:46:20.909 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:20.909 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:20.909 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:20.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.910 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6306 3.17 3:ee7cd06a:::rbd_data.113431586065.0000000000000005:head [truncate 1572864] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:20.910 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.459834+0000", 2024-09-04T17:46:20.910 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 12.365740148, 2024-09-04T17:46:20.910 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029166788999999999, 2024-09-04T17:46:20.910 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:20.910 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:20.910 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:20.910 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:20.910 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:20.910 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:20.910 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6306 2024-09-04T17:46:20.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.910 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:20.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:20.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459834+0000", 2024-09-04T17:46:20.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:20.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:20.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459834+0000", 2024-09-04T17:46:20.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:20.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:20.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459836+0000", 2024-09-04T17:46:20.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.835e-06 2024-09-04T17:46:20.912 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.912 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:20.912 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459841+0000", 2024-09-04T17:46:20.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.728e-06 2024-09-04T17:46:20.912 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.912 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:20.912 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459851+0000", 2024-09-04T17:46:20.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8570000000000007e-06 2024-09-04T17:46:20.912 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.912 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:20.913 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459859+0000", 2024-09-04T17:46:20.913 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4970000000000004e-06 2024-09-04T17:46:20.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.913 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:20.913 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.468789+0000", 2024-09-04T17:46:20.913 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0089301699999999994 2024-09-04T17:46:20.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.913 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:20.913 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.468890+0000", 2024-09-04T17:46:20.913 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000100963 2024-09-04T17:46:20.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.914 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5", 2024-09-04T17:46:20.914 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.469090+0000", 2024-09-04T17:46:20.914 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000199774 2024-09-04T17:46:20.914 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.914 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:20.914 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484704+0000", 2024-09-04T17:46:20.914 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015614836999999999 2024-09-04T17:46:20.914 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.914 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:46:20.914 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.488833+0000", 2024-09-04T17:46:20.914 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004128357 2024-09-04T17:46:20.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.915 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:20.915 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.488916+0000", 2024-09-04T17:46:20.915 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2928000000000003e-05 2024-09-04T17:46:20.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.915 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:20.915 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.489001+0000", 2024-09-04T17:46:20.915 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4843000000000001e-05 2024-09-04T17:46:20.915 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:20.915 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:20.915 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:20.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.916 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6307 3.17 3:eb6e082d:::rbd_data.113431586065.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:20.916 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.459871+0000", 2024-09-04T17:46:20.916 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 12.365703187999999, 2024-09-04T17:46:20.916 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032107122000000002, 2024-09-04T17:46:20.916 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:20.916 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:20.916 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:20.916 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:20.916 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:20.916 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:20.917 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6307 2024-09-04T17:46:20.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.917 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:20.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.917 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:20.917 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459871+0000", 2024-09-04T17:46:20.917 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:20.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.917 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:20.917 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459871+0000", 2024-09-04T17:46:20.917 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:20.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.918 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:20.918 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459871+0000", 2024-09-04T17:46:20.918 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9699999999999995e-07 2024-09-04T17:46:20.918 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.918 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.918 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:20.918 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.463429+0000", 2024-09-04T17:46:20.918 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0035577930000000001 2024-09-04T17:46:20.918 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.918 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.918 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:20.918 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.463463+0000", 2024-09-04T17:46:20.919 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3460000000000002e-05 2024-09-04T17:46:20.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.919 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.919 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:20.919 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.463483+0000", 2024-09-04T17:46:20.919 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0327999999999999e-05 2024-09-04T17:46:20.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.919 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.919 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:20.919 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.477551+0000", 2024-09-04T17:46:20.919 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014068259 2024-09-04T17:46:20.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.920 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:20.920 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.477748+0000", 2024-09-04T17:46:20.920 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000197004 2024-09-04T17:46:20.920 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.920 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5", 2024-09-04T17:46:20.920 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.478346+0000", 2024-09-04T17:46:20.920 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00059796299999999997 2024-09-04T17:46:20.920 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.920 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:46:20.921 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.489016+0000", 2024-09-04T17:46:20.921 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010669781 2024-09-04T17:46:20.921 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.921 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.921 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:20.921 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.491871+0000", 2024-09-04T17:46:20.921 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028546219999999998 2024-09-04T17:46:20.921 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.921 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.921 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:20.921 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.491931+0000", 2024-09-04T17:46:20.921 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0736999999999998e-05 2024-09-04T17:46:20.921 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.922 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.922 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:20.922 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.491978+0000", 2024-09-04T17:46:20.922 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6378000000000002e-05 2024-09-04T17:46:20.922 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:20.922 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:20.922 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:20.922 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.922 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.922 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6308 3.17 3:eb6e082d:::rbd_data.113431586065.0000000000000007:head [truncate 1572864] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:20.922 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.463520+0000", 2024-09-04T17:46:20.922 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 12.362053679000001, 2024-09-04T17:46:20.922 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029348833000000001, 2024-09-04T17:46:20.923 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:20.923 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:20.923 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:20.923 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:20.923 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:20.923 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:20.923 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6308 2024-09-04T17:46:20.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.923 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:20.923 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.923 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:20.923 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.463520+0000", 2024-09-04T17:46:20.924 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:20.924 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.924 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:20.924 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.463520+0000", 2024-09-04T17:46:20.924 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:20.924 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.924 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:20.924 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.463522+0000", 2024-09-04T17:46:20.924 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1150000000000001e-06 2024-09-04T17:46:20.924 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.924 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:20.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.463527+0000", 2024-09-04T17:46:20.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5170000000000001e-06 2024-09-04T17:46:20.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:20.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.463555+0000", 2024-09-04T17:46:20.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8594000000000001e-05 2024-09-04T17:46:20.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:20.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.463563+0000", 2024-09-04T17:46:20.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4340000000000003e-06 2024-09-04T17:46:20.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.926 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:20.926 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484756+0000", 2024-09-04T17:46:20.926 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021193081999999999 2024-09-04T17:46:20.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.926 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.926 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:20.926 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484863+0000", 2024-09-04T17:46:20.926 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010721999999999999 2024-09-04T17:46:20.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.926 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.926 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5", 2024-09-04T17:46:20.926 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.485067+0000", 2024-09-04T17:46:20.926 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000204 2024-09-04T17:46:20.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.927 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.927 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:46:20.927 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.489593+0000", 2024-09-04T17:46:20.927 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045261529999999998 2024-09-04T17:46:20.927 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.927 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.927 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:20.927 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.492787+0000", 2024-09-04T17:46:20.927 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031940039999999999 2024-09-04T17:46:20.927 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.927 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.927 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:20.927 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.492834+0000", 2024-09-04T17:46:20.927 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6281e-05 2024-09-04T17:46:20.927 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:20.928 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:20.928 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:20.928 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.492869+0000", 2024-09-04T17:46:20.928 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5432999999999998e-05 2024-09-04T17:46:20.928 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:20.928 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:20.928 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:20.928 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:20.928 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:20.928 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:20.929 INFO:teuthology.orchestra.run.smithi092.stdout:1046 writesame 0xd4f4cd3 thru 0xd5025d0 data_size 0x6e(0xd8fe bytes) 2024-09-04T17:46:20.930 INFO:teuthology.orchestra.run.smithi092.stdout:1047 read 0x14cc8b8 thru 0x14dc295 (0xf9de bytes) 2024-09-04T17:46:20.930 INFO:teuthology.orchestra.run.smithi092.stdout:1049 writesame 0x99d638 thru 0x9a39aa data_size 0x1(0x6373 bytes) 2024-09-04T17:46:20.931 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-04T17:46:20.932 INFO:teuthology.orchestra.run.smithi092.stdout:1051 punch from 0x9b95368 to 0x9b95fc6, (0xc5e bytes) 2024-09-04T17:46:20.932 INFO:teuthology.orchestra.run.smithi092.stdout:1052 trunc from 0xd5025d1 to 0x873d0d3 2024-09-04T17:46:20.932 INFO:teuthology.orchestra.run.smithi092.stdout:1053 punch from 0x549a496 to 0x54a5111, (0xac7b bytes) 2024-09-04T17:46:20.932 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:20.932 INFO:teuthology.orchestra.run.smithi092.stdout:1057 read 0x63fc2 thru 0x65db7 (0x1df6 bytes) 2024-09-04T17:46:20.932 INFO:teuthology.orchestra.run.smithi092.stdout:1058 read 0x7785fff thru 0x779507d (0xf07f bytes) 2024-09-04T17:46:20.932 INFO:teuthology.orchestra.run.smithi092.stdout:1060 write 0x6174bc1 thru 0x617841f (0x385f bytes) 2024-09-04T17:46:20.932 INFO:teuthology.orchestra.run.smithi092.stdout:1061 writesame 0x9338b4e thru 0x93400fb data_size 0x3(0x75ae bytes) 2024-09-04T17:46:20.932 INFO:teuthology.orchestra.run.smithi092.stdout:1063 write 0x4bc4204 thru 0x4bd2934 (0xe731 bytes) 2024-09-04T17:46:20.933 INFO:teuthology.orchestra.run.smithi092.stdout:1064 read 0x5f50a68 thru 0x5f53047 (0x25e0 bytes) 2024-09-04T17:46:20.933 INFO:teuthology.orchestra.run.smithi092.stdout:1065 clone 11 order 18 su 16384 sc 14 2024-09-04T17:46:20.933 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:20.929+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:21.031 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:21.027+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:21.105 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:21.105 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:46:21.105 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:46:21.105 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:21.120 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-04T17:46:21.132 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:21.131+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:21.233 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:46:21.231+0000 7f505d642640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26446) UID: 0 2024-09-04T17:46:21.243 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:21.244 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:46:21.244 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:46:21.244 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:46:21.244 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:21.258 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-04T17:46:21.313 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-04T17:46:21.363 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:21.365+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:21.383 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:21.383 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:46:21.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:46:21.383 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:46:21.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.383 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15462 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 835584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:21.383 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.220338+0000", 2024-09-04T17:46:21.383 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.1632576119999998, 2024-09-04T17:46:21.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051751278999999997, 2024-09-04T17:46:21.383 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:21.384 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:21.384 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:21.384 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:21.384 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:21.384 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:21.384 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15462 2024-09-04T17:46:21.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.384 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:21.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:21.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.220338+0000", 2024-09-04T17:46:21.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:21.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:21.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.220338+0000", 2024-09-04T17:46:21.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:21.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:21.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.220600+0000", 2024-09-04T17:46:21.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026198900000000002 2024-09-04T17:46:21.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:21.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.220617+0000", 2024-09-04T17:46:21.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6956000000000002e-05 2024-09-04T17:46:21.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:21.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.220627+0000", 2024-09-04T17:46:21.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0492999999999999e-05 2024-09-04T17:46:21.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:21.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.220635+0000", 2024-09-04T17:46:21.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0059999999999992e-06 2024-09-04T17:46:21.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:21.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.271853+0000", 2024-09-04T17:46:21.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051218020000000003 2024-09-04T17:46:21.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:21.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.271941+0000", 2024-09-04T17:46:21.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7802999999999999e-05 2024-09-04T17:46:21.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:21.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.272089+0000", 2024-09-04T17:46:21.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014801199999999999 2024-09-04T17:46:21.388 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:21.388 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:21.388 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:21.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.388 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15500 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 294912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:21.388 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.228278+0000", 2024-09-04T17:46:21.388 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.155317615, 2024-09-04T17:46:21.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051770927000000001, 2024-09-04T17:46:21.389 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:21.389 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:21.389 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:21.389 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:21.389 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:21.389 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:21.389 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15500 2024-09-04T17:46:21.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.389 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:21.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:21.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.228278+0000", 2024-09-04T17:46:21.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:21.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:21.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.228278+0000", 2024-09-04T17:46:21.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:21.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:21.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234002+0000", 2024-09-04T17:46:21.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005724392 2024-09-04T17:46:21.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:21.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234009+0000", 2024-09-04T17:46:21.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1300000000000003e-06 2024-09-04T17:46:21.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:21.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234024+0000", 2024-09-04T17:46:21.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4063000000000001e-05 2024-09-04T17:46:21.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:21.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234035+0000", 2024-09-04T17:46:21.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1124000000000001e-05 2024-09-04T17:46:21.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:21.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.279808+0000", 2024-09-04T17:46:21.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045772992999999998 2024-09-04T17:46:21.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:21.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.279894+0000", 2024-09-04T17:46:21.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6174999999999994e-05 2024-09-04T17:46:21.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:21.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.280049+0000", 2024-09-04T17:46:21.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015505 2024-09-04T17:46:21.393 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:21.393 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:21.393 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:21.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.393 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15524 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 1540096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:21.393 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.234831+0000", 2024-09-04T17:46:21.393 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.1487647390000002, 2024-09-04T17:46:21.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051893631000000003, 2024-09-04T17:46:21.394 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:21.394 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:21.394 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:21.394 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:21.394 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:21.394 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:21.394 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15524 2024-09-04T17:46:21.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.394 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:21.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:21.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234831+0000", 2024-09-04T17:46:21.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:21.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:21.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234831+0000", 2024-09-04T17:46:21.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:21.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:21.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234831+0000", 2024-09-04T17:46:21.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2699999999999999e-07 2024-09-04T17:46:21.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:21.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234833+0000", 2024-09-04T17:46:21.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9810000000000002e-06 2024-09-04T17:46:21.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:21.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234839+0000", 2024-09-04T17:46:21.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1519999999999997e-06 2024-09-04T17:46:21.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:21.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234845+0000", 2024-09-04T17:46:21.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1109999999999999e-06 2024-09-04T17:46:21.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:21.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.286526+0000", 2024-09-04T17:46:21.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051681373000000003 2024-09-04T17:46:21.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:21.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.286594+0000", 2024-09-04T17:46:21.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8073999999999999e-05 2024-09-04T17:46:21.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:21.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.286724+0000", 2024-09-04T17:46:21.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013041300000000001 2024-09-04T17:46:21.398 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:21.398 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:21.398 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:21.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.398 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15525 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 335872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:21.398 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.234851+0000", 2024-09-04T17:46:21.399 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.1487442149999998, 2024-09-04T17:46:21.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052146248999999999, 2024-09-04T17:46:21.399 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:21.399 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:21.399 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:21.399 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:21.399 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:21.399 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:21.399 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15525 2024-09-04T17:46:21.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.399 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:21.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:21.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234851+0000", 2024-09-04T17:46:21.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:21.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:21.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234851+0000", 2024-09-04T17:46:21.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:21.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:21.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234852+0000", 2024-09-04T17:46:21.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0999999999999999e-07 2024-09-04T17:46:21.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:21.401 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234854+0000", 2024-09-04T17:46:21.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9929999999999998e-06 2024-09-04T17:46:21.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:21.401 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234860+0000", 2024-09-04T17:46:21.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1489999999999999e-06 2024-09-04T17:46:21.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:21.401 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234865+0000", 2024-09-04T17:46:21.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8600000000000001e-06 2024-09-04T17:46:21.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:21.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.286770+0000", 2024-09-04T17:46:21.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051904807999999997 2024-09-04T17:46:21.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:21.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.286852+0000", 2024-09-04T17:46:21.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2076000000000005e-05 2024-09-04T17:46:21.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:21.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.286998+0000", 2024-09-04T17:46:21.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014595299999999999 2024-09-04T17:46:21.403 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:21.403 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:21.403 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:21.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.403 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15526 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 811008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:21.403 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.234871+0000", 2024-09-04T17:46:21.403 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.1487244460000001, 2024-09-04T17:46:21.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052383817999999999, 2024-09-04T17:46:21.403 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:21.403 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:21.403 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:21.404 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:21.404 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:21.404 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:21.404 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15526 2024-09-04T17:46:21.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.404 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:21.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:21.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234871+0000", 2024-09-04T17:46:21.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:21.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:21.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234871+0000", 2024-09-04T17:46:21.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:21.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:21.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234872+0000", 2024-09-04T17:46:21.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4299999999999998e-07 2024-09-04T17:46:21.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:21.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234874+0000", 2024-09-04T17:46:21.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2220000000000001e-06 2024-09-04T17:46:21.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:21.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234880+0000", 2024-09-04T17:46:21.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4169999999999997e-06 2024-09-04T17:46:21.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:21.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234886+0000", 2024-09-04T17:46:21.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8150000000000002e-06 2024-09-04T17:46:21.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:21.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287040+0000", 2024-09-04T17:46:21.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052153928000000002 2024-09-04T17:46:21.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:21.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287118+0000", 2024-09-04T17:46:21.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8102999999999994e-05 2024-09-04T17:46:21.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:21.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287255+0000", 2024-09-04T17:46:21.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013689 2024-09-04T17:46:21.408 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:21.408 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:21.408 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:21.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.409 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15527 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 372736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:21.409 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.234906+0000", 2024-09-04T17:46:21.409 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.1486891309999998, 2024-09-04T17:46:21.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052620742999999998, 2024-09-04T17:46:21.409 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:21.409 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:21.409 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:21.409 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:21.409 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:21.409 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:21.409 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15527 2024-09-04T17:46:21.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.410 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:21.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:21.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234906+0000", 2024-09-04T17:46:21.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:21.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:21.411 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234906+0000", 2024-09-04T17:46:21.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:21.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.411 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:21.411 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234907+0000", 2024-09-04T17:46:21.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9100000000000004e-07 2024-09-04T17:46:21.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:21.412 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234910+0000", 2024-09-04T17:46:21.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7599999999999998e-06 2024-09-04T17:46:21.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:21.412 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234917+0000", 2024-09-04T17:46:21.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.861e-06 2024-09-04T17:46:21.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:21.412 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234923+0000", 2024-09-04T17:46:21.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4130000000000003e-06 2024-09-04T17:46:21.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:21.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287302+0000", 2024-09-04T17:46:21.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052379440999999999 2024-09-04T17:46:21.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:21.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287382+0000", 2024-09-04T17:46:21.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9406999999999994e-05 2024-09-04T17:46:21.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:21.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287527+0000", 2024-09-04T17:46:21.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014537 2024-09-04T17:46:21.414 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:21.414 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:21.414 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:21.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.414 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15528 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 847872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:21.414 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.234931+0000", 2024-09-04T17:46:21.415 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.148664213, 2024-09-04T17:46:21.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052862919000000001, 2024-09-04T17:46:21.415 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:21.415 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:21.415 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:21.415 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:21.415 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:21.415 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:21.415 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15528 2024-09-04T17:46:21.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.415 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:21.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:21.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234931+0000", 2024-09-04T17:46:21.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:21.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:21.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234931+0000", 2024-09-04T17:46:21.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:21.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:21.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234932+0000", 2024-09-04T17:46:21.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6100000000000001e-07 2024-09-04T17:46:21.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:21.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234934+0000", 2024-09-04T17:46:21.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3309999999999998e-06 2024-09-04T17:46:21.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:21.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234941+0000", 2024-09-04T17:46:21.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4869999999999999e-06 2024-09-04T17:46:21.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:21.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234946+0000", 2024-09-04T17:46:21.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5879999999999997e-06 2024-09-04T17:46:21.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:21.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287571+0000", 2024-09-04T17:46:21.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052624769000000002 2024-09-04T17:46:21.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:21.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287651+0000", 2024-09-04T17:46:21.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9975000000000006e-05 2024-09-04T17:46:21.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:21.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287794+0000", 2024-09-04T17:46:21.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000143308 2024-09-04T17:46:21.419 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:21.419 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:21.419 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:21.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.420 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15529 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 409600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:21.420 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.234954+0000", 2024-09-04T17:46:21.420 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.1486413419999999, 2024-09-04T17:46:21.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053149889999999998, 2024-09-04T17:46:21.420 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:21.420 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:21.420 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:21.420 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:21.420 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:21.421 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:21.421 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15529 2024-09-04T17:46:21.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.421 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:21.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:21.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234954+0000", 2024-09-04T17:46:21.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:21.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:21.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234954+0000", 2024-09-04T17:46:21.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:21.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:21.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234955+0000", 2024-09-04T17:46:21.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3799999999999997e-07 2024-09-04T17:46:21.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:21.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234957+0000", 2024-09-04T17:46:21.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3309999999999998e-06 2024-09-04T17:46:21.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:21.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234964+0000", 2024-09-04T17:46:21.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5780000000000004e-06 2024-09-04T17:46:21.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:21.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234969+0000", 2024-09-04T17:46:21.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0629999999999996e-06 2024-09-04T17:46:21.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:21.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287839+0000", 2024-09-04T17:46:21.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052870385999999998 2024-09-04T17:46:21.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:21.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287929+0000", 2024-09-04T17:46:21.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9514999999999999e-05 2024-09-04T17:46:21.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:21.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288104+0000", 2024-09-04T17:46:21.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000175379 2024-09-04T17:46:21.424 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:21.424 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:21.424 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:21.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.425 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15530 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 884736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:21.425 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.234976+0000", 2024-09-04T17:46:21.425 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.1486199240000001, 2024-09-04T17:46:21.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053414593000000003, 2024-09-04T17:46:21.425 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:21.425 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:21.425 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:21.425 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:21.425 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:21.425 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:21.425 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15530 2024-09-04T17:46:21.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.426 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:21.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:21.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234976+0000", 2024-09-04T17:46:21.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:21.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:21.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234976+0000", 2024-09-04T17:46:21.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:21.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:21.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234976+0000", 2024-09-04T17:46:21.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5299999999999999e-07 2024-09-04T17:46:21.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:21.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234978+0000", 2024-09-04T17:46:21.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0329999999999998e-06 2024-09-04T17:46:21.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:21.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234984+0000", 2024-09-04T17:46:21.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0249999999999999e-06 2024-09-04T17:46:21.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:21.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234990+0000", 2024-09-04T17:46:21.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7579999999999998e-06 2024-09-04T17:46:21.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:21.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288149+0000", 2024-09-04T17:46:21.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053158864 2024-09-04T17:46:21.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:21.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288236+0000", 2024-09-04T17:46:21.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7306999999999996e-05 2024-09-04T17:46:21.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:21.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288390+0000", 2024-09-04T17:46:21.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000154153 2024-09-04T17:46:21.429 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:21.429 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:21.429 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:21.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.429 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15531 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 446464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:21.429 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.234997+0000", 2024-09-04T17:46:21.429 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.1485989409999999, 2024-09-04T17:46:21.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053721086000000001, 2024-09-04T17:46:21.430 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:21.430 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:21.430 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:21.430 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:21.430 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:21.430 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:21.430 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15531 2024-09-04T17:46:21.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.430 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:21.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.430 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:21.430 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234997+0000", 2024-09-04T17:46:21.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:21.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:21.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234997+0000", 2024-09-04T17:46:21.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:21.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:21.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234997+0000", 2024-09-04T17:46:21.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8200000000000001e-07 2024-09-04T17:46:21.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:21.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234999+0000", 2024-09-04T17:46:21.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.058e-06 2024-09-04T17:46:21.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:21.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235005+0000", 2024-09-04T17:46:21.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0700000000000003e-06 2024-09-04T17:46:21.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:21.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235012+0000", 2024-09-04T17:46:21.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7020000000000004e-06 2024-09-04T17:46:21.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:21.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288441+0000", 2024-09-04T17:46:21.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053429343999999997 2024-09-04T17:46:21.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:21.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288535+0000", 2024-09-04T17:46:21.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3406000000000007e-05 2024-09-04T17:46:21.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:21.434 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288718+0000", 2024-09-04T17:46:21.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000183124 2024-09-04T17:46:21.434 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:21.434 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:21.434 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:21.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.434 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15532 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 921600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:21.434 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235020+0000", 2024-09-04T17:46:21.434 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.1485753139999999, 2024-09-04T17:46:21.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053966157000000001, 2024-09-04T17:46:21.434 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:21.434 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:21.435 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:21.435 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:21.435 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:21.435 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:21.435 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15532 2024-09-04T17:46:21.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.435 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:21.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:21.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235020+0000", 2024-09-04T17:46:21.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:21.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:21.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235020+0000", 2024-09-04T17:46:21.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:21.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:21.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235021+0000", 2024-09-04T17:46:21.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9799999999999999e-07 2024-09-04T17:46:21.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:21.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235023+0000", 2024-09-04T17:46:21.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1560000000000002e-06 2024-09-04T17:46:21.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:21.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235029+0000", 2024-09-04T17:46:21.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0270000000000001e-06 2024-09-04T17:46:21.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:21.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235034+0000", 2024-09-04T17:46:21.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0000000000000004e-06 2024-09-04T17:46:21.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:21.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288760+0000", 2024-09-04T17:46:21.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053726077999999997 2024-09-04T17:46:21.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:21.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288845+0000", 2024-09-04T17:46:21.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5242999999999997e-05 2024-09-04T17:46:21.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:21.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288986+0000", 2024-09-04T17:46:21.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014125499999999999 2024-09-04T17:46:21.438 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:21.438 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:21.439 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:21.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.439 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15533 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 483328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:21.439 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235041+0000", 2024-09-04T17:46:21.439 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.1485542390000001, 2024-09-04T17:46:21.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054242994000000003, 2024-09-04T17:46:21.439 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:21.439 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:21.439 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:21.439 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:21.439 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:21.439 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:21.440 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15533 2024-09-04T17:46:21.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.440 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:21.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:21.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235041+0000", 2024-09-04T17:46:21.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:21.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:21.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235041+0000", 2024-09-04T17:46:21.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:21.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:21.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235042+0000", 2024-09-04T17:46:21.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5499999999999999e-07 2024-09-04T17:46:21.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:21.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235044+0000", 2024-09-04T17:46:21.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2259999999999999e-06 2024-09-04T17:46:21.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:21.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235051+0000", 2024-09-04T17:46:21.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4440000000000002e-06 2024-09-04T17:46:21.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:21.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235056+0000", 2024-09-04T17:46:21.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8740000000000003e-06 2024-09-04T17:46:21.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:21.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289028+0000", 2024-09-04T17:46:21.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053971742000000003 2024-09-04T17:46:21.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:21.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289108+0000", 2024-09-04T17:46:21.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0502999999999998e-05 2024-09-04T17:46:21.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:21.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289284+0000", 2024-09-04T17:46:21.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017584999999999999 2024-09-04T17:46:21.443 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:21.443 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:21.443 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:21.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.444 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15534 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 958464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:21.444 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235068+0000", 2024-09-04T17:46:21.444 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.1485275389999998, 2024-09-04T17:46:21.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054505452000000003, 2024-09-04T17:46:21.444 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:21.444 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:21.444 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:21.444 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:21.444 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:21.444 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:21.444 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15534 2024-09-04T17:46:21.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.445 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:21.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:21.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235068+0000", 2024-09-04T17:46:21.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:21.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:21.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235068+0000", 2024-09-04T17:46:21.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:21.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:21.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235068+0000", 2024-09-04T17:46:21.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7899999999999999e-07 2024-09-04T17:46:21.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:21.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235071+0000", 2024-09-04T17:46:21.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4109999999999999e-06 2024-09-04T17:46:21.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:21.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235078+0000", 2024-09-04T17:46:21.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0489999999999998e-06 2024-09-04T17:46:21.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:21.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235083+0000", 2024-09-04T17:46:21.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2249999999999999e-06 2024-09-04T17:46:21.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:21.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289328+0000", 2024-09-04T17:46:21.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054244894000000002 2024-09-04T17:46:21.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:21.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289417+0000", 2024-09-04T17:46:21.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8988999999999998e-05 2024-09-04T17:46:21.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:21.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289573+0000", 2024-09-04T17:46:21.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000156405 2024-09-04T17:46:21.448 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:21.448 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:21.448 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:21.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.448 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15535 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 520192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:21.449 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235173+0000", 2024-09-04T17:46:21.449 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.1484230100000001, 2024-09-04T17:46:21.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054718671000000003, 2024-09-04T17:46:21.449 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:21.449 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:21.449 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:21.449 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:21.449 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:21.449 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:21.449 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15535 2024-09-04T17:46:21.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.449 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:21.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.450 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:21.450 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235173+0000", 2024-09-04T17:46:21.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:21.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.450 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:21.450 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235173+0000", 2024-09-04T17:46:21.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:21.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.450 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:21.450 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235173+0000", 2024-09-04T17:46:21.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2600000000000002e-07 2024-09-04T17:46:21.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.451 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:21.451 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235177+0000", 2024-09-04T17:46:21.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.625e-06 2024-09-04T17:46:21.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.451 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:21.451 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235188+0000", 2024-09-04T17:46:21.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1464000000000001e-05 2024-09-04T17:46:21.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.451 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:21.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235195+0000", 2024-09-04T17:46:21.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5910000000000001e-06 2024-09-04T17:46:21.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:21.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289623+0000", 2024-09-04T17:46:21.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054427595000000002 2024-09-04T17:46:21.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:21.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289718+0000", 2024-09-04T17:46:21.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5958999999999994e-05 2024-09-04T17:46:21.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.453 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:21.453 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289891+0000", 2024-09-04T17:46:21.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017271100000000001 2024-09-04T17:46:21.453 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:21.453 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:21.453 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:21.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.453 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15536 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 995328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:21.453 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235206+0000", 2024-09-04T17:46:21.453 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.1483898689999998, 2024-09-04T17:46:21.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054984065999999998, 2024-09-04T17:46:21.453 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:21.453 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:21.454 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:21.454 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:21.454 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:21.454 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:21.454 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15536 2024-09-04T17:46:21.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.454 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:21.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.454 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:21.454 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235206+0000", 2024-09-04T17:46:21.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:21.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:21.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235206+0000", 2024-09-04T17:46:21.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:21.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:21.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235206+0000", 2024-09-04T17:46:21.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5300000000000004e-07 2024-09-04T17:46:21.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:21.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235209+0000", 2024-09-04T17:46:21.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6089999999999999e-06 2024-09-04T17:46:21.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:21.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235216+0000", 2024-09-04T17:46:21.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.277e-06 2024-09-04T17:46:21.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:21.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235221+0000", 2024-09-04T17:46:21.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2530000000000004e-06 2024-09-04T17:46:21.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:21.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289936+0000", 2024-09-04T17:46:21.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054714888000000003 2024-09-04T17:46:21.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:21.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290035+0000", 2024-09-04T17:46:21.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8652000000000003e-05 2024-09-04T17:46:21.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:21.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290190+0000", 2024-09-04T17:46:21.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015483399999999999 2024-09-04T17:46:21.458 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:21.458 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:21.458 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:21.458 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.458 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.458 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15537 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 1323008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:21.458 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235264+0000", 2024-09-04T17:46:21.458 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.1483318009999999, 2024-09-04T17:46:21.458 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055255669, 2024-09-04T17:46:21.458 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:21.458 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:21.458 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:21.458 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:21.459 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:21.459 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:21.459 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15537 2024-09-04T17:46:21.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.459 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:21.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:21.459 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235264+0000", 2024-09-04T17:46:21.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:21.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:21.459 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235264+0000", 2024-09-04T17:46:21.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:21.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:21.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235264+0000", 2024-09-04T17:46:21.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4499999999999997e-07 2024-09-04T17:46:21.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:21.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235267+0000", 2024-09-04T17:46:21.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9210000000000001e-06 2024-09-04T17:46:21.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:21.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235276+0000", 2024-09-04T17:46:21.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2309999999999996e-06 2024-09-04T17:46:21.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:21.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235281+0000", 2024-09-04T17:46:21.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8560000000000003e-06 2024-09-04T17:46:21.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:21.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290246+0000", 2024-09-04T17:46:21.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054964528999999998 2024-09-04T17:46:21.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.462 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:21.462 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290344+0000", 2024-09-04T17:46:21.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8073999999999996e-05 2024-09-04T17:46:21.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.462 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:21.462 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290519+0000", 2024-09-04T17:46:21.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000175413 2024-09-04T17:46:21.462 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:21.462 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:21.463 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:21.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.463 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15538 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 1032192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:21.463 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235290+0000", 2024-09-04T17:46:21.463 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.1483058919999998, 2024-09-04T17:46:21.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055518932, 2024-09-04T17:46:21.463 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:21.463 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:21.463 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:21.463 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:21.463 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:21.463 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:21.464 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15538 2024-09-04T17:46:21.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.464 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:21.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.464 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:21.464 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235290+0000", 2024-09-04T17:46:21.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:21.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.464 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:21.464 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235290+0000", 2024-09-04T17:46:21.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:21.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:21.465 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235290+0000", 2024-09-04T17:46:21.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3399999999999999e-07 2024-09-04T17:46:21.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:21.465 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235292+0000", 2024-09-04T17:46:21.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2639999999999998e-06 2024-09-04T17:46:21.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:21.465 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235299+0000", 2024-09-04T17:46:21.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2029999999999997e-06 2024-09-04T17:46:21.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:21.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235305+0000", 2024-09-04T17:46:21.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7469999999999999e-06 2024-09-04T17:46:21.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:21.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290562+0000", 2024-09-04T17:46:21.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055256580999999999 2024-09-04T17:46:21.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.467 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:21.467 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290651+0000", 2024-09-04T17:46:21.467 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8953e-05 2024-09-04T17:46:21.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.467 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.467 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:21.467 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290809+0000", 2024-09-04T17:46:21.467 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015765000000000001 2024-09-04T17:46:21.467 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:21.467 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:21.467 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:21.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.467 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.467 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15539 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 1359872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:21.468 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235312+0000", 2024-09-04T17:46:21.468 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.1482831519999999, 2024-09-04T17:46:21.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055809865, 2024-09-04T17:46:21.468 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:21.468 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:21.468 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:21.468 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:21.468 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:21.468 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:21.468 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15539 2024-09-04T17:46:21.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.468 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:21.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.469 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:21.469 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235312+0000", 2024-09-04T17:46:21.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:21.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.469 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:21.469 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235312+0000", 2024-09-04T17:46:21.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:21.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.469 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:21.470 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235313+0000", 2024-09-04T17:46:21.470 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9000000000000002e-07 2024-09-04T17:46:21.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.470 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:21.470 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235315+0000", 2024-09-04T17:46:21.470 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1270000000000001e-06 2024-09-04T17:46:21.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.470 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:21.470 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235320+0000", 2024-09-04T17:46:21.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5430000000000001e-06 2024-09-04T17:46:21.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:21.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235326+0000", 2024-09-04T17:46:21.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2039999999999996e-06 2024-09-04T17:46:21.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:21.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290858+0000", 2024-09-04T17:46:21.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055532440000000002 2024-09-04T17:46:21.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.472 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:21.472 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290953+0000", 2024-09-04T17:46:21.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5031000000000005e-05 2024-09-04T17:46:21.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.472 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:21.472 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.291122+0000", 2024-09-04T17:46:21.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016913 2024-09-04T17:46:21.472 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:21.472 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:21.472 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:21.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.473 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15540 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 1396736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:21.473 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235333+0000", 2024-09-04T17:46:21.473 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.148262645, 2024-09-04T17:46:21.473 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056080528999999997, 2024-09-04T17:46:21.473 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:21.473 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:21.473 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:21.473 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:21.473 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:21.473 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:21.473 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15540 2024-09-04T17:46:21.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.474 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:21.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.474 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:21.474 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235333+0000", 2024-09-04T17:46:21.474 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:21.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.474 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:21.474 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235333+0000", 2024-09-04T17:46:21.474 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:21.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.474 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:21.475 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235333+0000", 2024-09-04T17:46:21.475 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0999999999999999e-07 2024-09-04T17:46:21.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.475 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:21.475 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235336+0000", 2024-09-04T17:46:21.475 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.278e-06 2024-09-04T17:46:21.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.475 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:21.475 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235341+0000", 2024-09-04T17:46:21.475 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8359999999999997e-06 2024-09-04T17:46:21.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:21.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235346+0000", 2024-09-04T17:46:21.476 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4329999999999997e-06 2024-09-04T17:46:21.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:21.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.291167+0000", 2024-09-04T17:46:21.476 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055821603999999997 2024-09-04T17:46:21.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:21.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.291257+0000", 2024-09-04T17:46:21.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9349000000000001e-05 2024-09-04T17:46:21.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.477 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:21.477 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.291413+0000", 2024-09-04T17:46:21.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015661900000000001 2024-09-04T17:46:21.477 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:21.477 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:21.477 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:21.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.477 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15541 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 1433600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:21.477 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235355+0000", 2024-09-04T17:46:21.478 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.148240457, 2024-09-04T17:46:21.478 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056371783000000002, 2024-09-04T17:46:21.478 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:21.478 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:21.478 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:21.478 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:21.478 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:21.478 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:21.478 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15541 2024-09-04T17:46:21.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.478 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:21.478 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.478 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:21.478 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235355+0000", 2024-09-04T17:46:21.479 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:21.479 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.479 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:21.479 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235355+0000", 2024-09-04T17:46:21.479 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:21.479 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.479 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:21.479 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235356+0000", 2024-09-04T17:46:21.479 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3099999999999998e-07 2024-09-04T17:46:21.479 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.480 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:21.480 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235358+0000", 2024-09-04T17:46:21.480 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3599999999999999e-06 2024-09-04T17:46:21.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.480 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.480 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:21.480 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235364+0000", 2024-09-04T17:46:21.480 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5760000000000002e-06 2024-09-04T17:46:21.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.480 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.480 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:21.480 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235370+0000", 2024-09-04T17:46:21.481 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0819999999999998e-06 2024-09-04T17:46:21.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.481 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:21.481 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.291462+0000", 2024-09-04T17:46:21.481 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056092301999999997 2024-09-04T17:46:21.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.481 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:21.481 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.291559+0000", 2024-09-04T17:46:21.481 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6988e-05 2024-09-04T17:46:21.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.482 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:21.482 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.291727+0000", 2024-09-04T17:46:21.482 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016804400000000001 2024-09-04T17:46:21.482 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:21.482 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:21.482 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:21.482 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:21.482 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:21.482 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:21.484 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-04T17:46:21.484 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone10 from 0x93400fc (overlap 0x22f36c6) to 0xa7c3a1 2024-09-04T17:46:21.484 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:21.485+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:21.584 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:21.585+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:21.632 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #9, image image_client.0-clone9 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent10 2024-09-04T17:46:21.649 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:21.649 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:46:21.649 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:46:21.649 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:21.660 INFO:teuthology.orchestra.run.smithi092.stdout:1066 punch from 0x56a9388 to 0x56aad15, (0x198d bytes) 2024-09-04T17:46:21.664 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-04T17:46:21.668 INFO:teuthology.orchestra.run.smithi092.stdout:1067 write 0x36e79e4 thru 0x36e89df (0xffc bytes) 2024-09-04T17:46:21.680 INFO:teuthology.orchestra.run.smithi092.stdout:1068 read 0x2d92a65 thru 0x2d9d10c (0xa6a8 bytes) 2024-09-04T17:46:21.682 INFO:teuthology.orchestra.run.smithi092.stdout:1069 read 0x7e0842 thru 0x7e64ff (0x5cbe bytes) 2024-09-04T17:46:21.685 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:21.683+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:21.700 INFO:teuthology.orchestra.run.smithi092.stdout:1070 writesame 0xc144e67 thru 0xc1523d3 data_size 0x1(0xd56d bytes) 2024-09-04T17:46:21.700 INFO:teuthology.orchestra.run.smithi092.stdout:1071 writesame 0xfb9cc thru 0x102d8a data_size 0x53(0x73bf bytes) 2024-09-04T17:46:21.720 INFO:teuthology.orchestra.run.smithi092.stdout:1072 read 0xb2062c2 thru 0xb214096 (0xddd5 bytes) 2024-09-04T17:46:21.721 INFO:teuthology.orchestra.run.smithi092.stdout:1073 writesame 0xaec78d9 thru 0xaec7a23 data_size 0x1(0x14b bytes) 2024-09-04T17:46:21.739 INFO:teuthology.orchestra.run.smithi092.stdout:1075 trunc from 0xc1523d4 to 0x41a4d33 2024-09-04T17:46:21.786 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:21.785+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:21.791 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:21.791 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:46:21.792 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:46:21.792 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:46:21.792 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:21.803 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:46:21.805+0000 7f4ae73767c0 -1 Falling back to public interface 2024-09-04T17:46:21.807 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-04T17:46:21.808 INFO:teuthology.orchestra.run.smithi092.stdout:1077 read 0x2b4e673 thru 0x2b50d51 (0x26df bytes) 2024-09-04T17:46:21.809 INFO:teuthology.orchestra.run.smithi092.stdout:1078 read 0x1fa6ad thru 0x205bdb (0xb52f bytes) 2024-09-04T17:46:21.811 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:21.811 INFO:teuthology.orchestra.run.smithi092.stdout:1080 read 0x3e796d9 thru 0x3e7db15 (0x443d bytes) 2024-09-04T17:46:21.812 INFO:teuthology.orchestra.run.smithi092.stdout:1081 write 0x1b9df18 thru 0x1ba6d8d (0x8e76 bytes) 2024-09-04T17:46:21.826 INFO:teuthology.orchestra.run.smithi092.stdout:1083 read 0x21ead3 thru 0x22ab68 (0xc096 bytes) 2024-09-04T17:46:21.828 INFO:teuthology.orchestra.run.smithi092.stdout:1085 writesame 0x1bb4470 thru 0x1bbabc8 data_size 0x1(0x6759 bytes) 2024-09-04T17:46:21.846 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:21.846 INFO:teuthology.orchestra.run.smithi092.stdout:1087 read 0x2e7d963 thru 0x2e865a1 (0x8c3f bytes) 2024-09-04T17:46:21.861 INFO:teuthology.orchestra.run.smithi092.stdout:1088 write 0x693c54d thru 0x694ad74 (0xe828 bytes) 2024-09-04T17:46:21.873 INFO:teuthology.orchestra.run.smithi092.stdout:1089 read 0x385d4c7 thru 0x386bfe5 (0xeb1f bytes) 2024-09-04T17:46:21.874 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:21.874 INFO:teuthology.orchestra.run.smithi092.stdout:1091 writesame 0xf524ad thru 0xf5c2bf data_size 0x7b(0x9e13 bytes) 2024-09-04T17:46:21.886 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:21.885+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:21.892 INFO:teuthology.orchestra.run.smithi092.stdout:1093 read 0x7c3a32 thru 0x7c5233 (0x1802 bytes) 2024-09-04T17:46:21.916 INFO:teuthology.orchestra.run.smithi092.stdout:1094 write 0xe661b17 thru 0xe6660f3 (0x45dd bytes) 2024-09-04T17:46:21.924 INFO:teuthology.orchestra.run.smithi092.stdout:1095 read 0xa099639 thru 0xa0a22d6 (0x8c9e bytes) 2024-09-04T17:46:21.924 INFO:teuthology.orchestra.run.smithi092.stdout:1096 writesame 0xdb51155 thru 0xdb5bc9e data_size 0x19(0xab4a bytes) 2024-09-04T17:46:21.934 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:21.935 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:46:21.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:46:21.935 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:46:21.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.935 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:4264 3.9 e30/24 3:970e51a0:::rbd_data.1134cfbce83d.0000000000000009:head v 30'83, mlcod=30'81)", 2024-09-04T17:46:21.935 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:07.027227+0000", 2024-09-04T17:46:21.935 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.90876001, 2024-09-04T17:46:21.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0096959590000000005, 2024-09-04T17:46:21.935 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:21.935 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:21.935 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:21.936 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:21.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.936 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:21.936 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.027227+0000", 2024-09-04T17:46:21.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:21.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.936 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:21.936 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.027227+0000", 2024-09-04T17:46:21.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:21.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.937 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:21.937 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.027229+0000", 2024-09-04T17:46:21.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4839999999999998e-06 2024-09-04T17:46:21.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.937 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:21.937 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.031984+0000", 2024-09-04T17:46:21.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047544789999999998 2024-09-04T17:46:21.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.937 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:21.938 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.032019+0000", 2024-09-04T17:46:21.938 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5253999999999998e-05 2024-09-04T17:46:21.938 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.938 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.938 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:21.938 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.032036+0000", 2024-09-04T17:46:21.938 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6713000000000001e-05 2024-09-04T17:46:21.938 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.938 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.938 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:21.938 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.032129+0000", 2024-09-04T17:46:21.938 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2961000000000007e-05 2024-09-04T17:46:21.938 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.939 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:21.939 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.032154+0000", 2024-09-04T17:46:21.939 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5148999999999999e-05 2024-09-04T17:46:21.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.939 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:21.939 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.036778+0000", 2024-09-04T17:46:21.939 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046239159999999996 2024-09-04T17:46:21.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.939 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:21.939 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.036923+0000", 2024-09-04T17:46:21.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014500299999999999 2024-09-04T17:46:21.940 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:21.940 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:21.940 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:21.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.940 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.941 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:4281 3.4 e30/24 3:215ebead:::rbd_header.1134cfbce83d:head v 30'24, mlcod=30'22)", 2024-09-04T17:46:21.941 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:07.073105+0000", 2024-09-04T17:46:21.941 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.862881246000001, 2024-09-04T17:46:21.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0032176499999999999, 2024-09-04T17:46:21.941 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:21.941 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:21.941 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:21.941 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:21.942 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.942 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:21.942 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.073105+0000", 2024-09-04T17:46:21.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:21.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.942 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.942 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:21.942 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.073105+0000", 2024-09-04T17:46:21.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:21.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.943 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:21.943 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.073107+0000", 2024-09-04T17:46:21.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1260000000000001e-06 2024-09-04T17:46:21.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.943 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:21.943 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.073115+0000", 2024-09-04T17:46:21.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0430000000000008e-06 2024-09-04T17:46:21.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:21.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.073133+0000", 2024-09-04T17:46:21.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8308000000000002e-05 2024-09-04T17:46:21.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:21.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.073144+0000", 2024-09-04T17:46:21.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1219999999999999e-05 2024-09-04T17:46:21.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:21.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.073218+0000", 2024-09-04T17:46:21.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3464e-05 2024-09-04T17:46:21.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:21.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.073244+0000", 2024-09-04T17:46:21.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6308e-05 2024-09-04T17:46:21.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:21.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.076286+0000", 2024-09-04T17:46:21.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00304163 2024-09-04T17:46:21.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:21.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.076323+0000", 2024-09-04T17:46:21.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7551e-05 2024-09-04T17:46:21.946 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:21.946 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:21.946 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:21.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.946 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:4399 3.9 e31/24 3:970e51a0:::rbd_data.1134cfbce83d.0000000000000009:head v 31'87, mlcod=30'84)", 2024-09-04T17:46:21.946 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:07.343236+0000", 2024-09-04T17:46:21.946 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.592750326000001, 2024-09-04T17:46:21.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0036134539999999999, 2024-09-04T17:46:21.946 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:21.947 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:21.947 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:21.947 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:21.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:21.947 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.343236+0000", 2024-09-04T17:46:21.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:21.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:21.947 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.343236+0000", 2024-09-04T17:46:21.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:21.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:21.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.343237+0000", 2024-09-04T17:46:21.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4000000000000001e-07 2024-09-04T17:46:21.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:21.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.343250+0000", 2024-09-04T17:46:21.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2573e-05 2024-09-04T17:46:21.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:21.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.343279+0000", 2024-09-04T17:46:21.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9263000000000001e-05 2024-09-04T17:46:21.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:21.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.343299+0000", 2024-09-04T17:46:21.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9720000000000001e-05 2024-09-04T17:46:21.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:21.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.343389+0000", 2024-09-04T17:46:21.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9924000000000002e-05 2024-09-04T17:46:21.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:21.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.343413+0000", 2024-09-04T17:46:21.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4227999999999999e-05 2024-09-04T17:46:21.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:21.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.346796+0000", 2024-09-04T17:46:21.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0033831310000000002 2024-09-04T17:46:21.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:21.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.346850+0000", 2024-09-04T17:46:21.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3875000000000002e-05 2024-09-04T17:46:21.952 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:21.952 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:21.952 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:21.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.953 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:4881 3.4 e31/24 3:25597cfa:::rbd_data.1134cfbce83d.0000000000000006:head v 31'28, mlcod=31'26)", 2024-09-04T17:46:21.953 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:07.381336+0000", 2024-09-04T17:46:21.953 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.554650697, 2024-09-04T17:46:21.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013330164, 2024-09-04T17:46:21.953 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:21.954 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:21.954 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:21.954 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:21.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.954 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:21.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.381336+0000", 2024-09-04T17:46:21.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:21.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:21.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.381336+0000", 2024-09-04T17:46:21.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:21.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:21.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.381339+0000", 2024-09-04T17:46:21.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.287e-06 2024-09-04T17:46:21.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.956 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:21.956 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.384162+0000", 2024-09-04T17:46:21.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028228649999999999 2024-09-04T17:46:21.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.957 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:21.957 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.384209+0000", 2024-09-04T17:46:21.957 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6768999999999999e-05 2024-09-04T17:46:21.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.957 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:21.957 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.384225+0000", 2024-09-04T17:46:21.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5987000000000001e-05 2024-09-04T17:46:21.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:21.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.384372+0000", 2024-09-04T17:46:21.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000146809 2024-09-04T17:46:21.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:21.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.384409+0000", 2024-09-04T17:46:21.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6909999999999997e-05 2024-09-04T17:46:21.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.959 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:21.959 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.394535+0000", 2024-09-04T17:46:21.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010126575000000001 2024-09-04T17:46:21.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.959 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:21.959 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.394666+0000", 2024-09-04T17:46:21.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013096200000000001 2024-09-04T17:46:21.959 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:21.959 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:21.959 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:21.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.960 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.960 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:4884 3.4 e31/24 3:25597cfa:::rbd_data.1134cfbce83d.0000000000000006:head v 31'30, mlcod=31'26)", 2024-09-04T17:46:21.960 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:07.386970+0000", 2024-09-04T17:46:21.960 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.549016763000001, 2024-09-04T17:46:21.960 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0088306779999999998, 2024-09-04T17:46:21.960 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:21.960 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:21.960 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:21.960 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:21.960 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.960 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:21.960 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.386970+0000", 2024-09-04T17:46:21.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:21.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:21.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.386970+0000", 2024-09-04T17:46:21.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:21.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:21.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.386973+0000", 2024-09-04T17:46:21.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9229999999999998e-06 2024-09-04T17:46:21.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:21.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.386993+0000", 2024-09-04T17:46:21.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.054e-05 2024-09-04T17:46:21.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:21.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.387029+0000", 2024-09-04T17:46:21.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5753000000000001e-05 2024-09-04T17:46:21.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:21.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.387051+0000", 2024-09-04T17:46:21.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1552999999999998e-05 2024-09-04T17:46:21.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:21.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.388844+0000", 2024-09-04T17:46:21.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001793712 2024-09-04T17:46:21.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:21.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.388866+0000", 2024-09-04T17:46:21.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1123000000000001e-05 2024-09-04T17:46:21.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.964 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.964 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:21.964 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.395711+0000", 2024-09-04T17:46:21.964 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0068458859999999998 2024-09-04T17:46:21.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.964 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.964 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:21.964 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.395801+0000", 2024-09-04T17:46:21.964 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9188000000000001e-05 2024-09-04T17:46:21.964 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:21.964 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:21.964 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:21.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.965 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:4997 3.9 e31/24 3:940f1d52:::rbd_data.1134cfbce83d.0000000000000010:head v 31'89, mlcod=30'85)", 2024-09-04T17:46:21.965 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:07.400336+0000", 2024-09-04T17:46:21.965 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.535651111, 2024-09-04T17:46:21.965 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003411533, 2024-09-04T17:46:21.965 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:21.965 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:21.965 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:21.965 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:21.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:21.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.400336+0000", 2024-09-04T17:46:21.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:21.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:21.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.400336+0000", 2024-09-04T17:46:21.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:21.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:21.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.400338+0000", 2024-09-04T17:46:21.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3489999999999999e-06 2024-09-04T17:46:21.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:21.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.400353+0000", 2024-09-04T17:46:21.968 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5091e-05 2024-09-04T17:46:21.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.968 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:21.968 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.400379+0000", 2024-09-04T17:46:21.968 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5528999999999999e-05 2024-09-04T17:46:21.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.968 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:21.968 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.400407+0000", 2024-09-04T17:46:21.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8104000000000001e-05 2024-09-04T17:46:21.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.969 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:21.969 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.400502+0000", 2024-09-04T17:46:21.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5402999999999994e-05 2024-09-04T17:46:21.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.969 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:21.969 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.400519+0000", 2024-09-04T17:46:21.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7329e-05 2024-09-04T17:46:21.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.970 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:21.970 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.403698+0000", 2024-09-04T17:46:21.970 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031787270000000001 2024-09-04T17:46:21.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.970 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:21.970 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.403747+0000", 2024-09-04T17:46:21.970 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9001000000000001e-05 2024-09-04T17:46:21.970 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:21.970 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:21.971 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:21.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.971 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:4999 3.4 e31/24 3:207aa2a5:::rbd_data.1134cfbce83d.0000000000000011:head v 31'32, mlcod=31'28)", 2024-09-04T17:46:21.971 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:07.401780+0000", 2024-09-04T17:46:21.971 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.53420715, 2024-09-04T17:46:21.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020640929999999999, 2024-09-04T17:46:21.971 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:21.971 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:21.971 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:21.971 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:21.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:21.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.401780+0000", 2024-09-04T17:46:21.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:21.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:21.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.401780+0000", 2024-09-04T17:46:21.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:21.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:21.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.401781+0000", 2024-09-04T17:46:21.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4950000000000001e-06 2024-09-04T17:46:21.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.973 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.973 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:21.973 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.401791+0000", 2024-09-04T17:46:21.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0284e-05 2024-09-04T17:46:21.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.973 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.973 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:21.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.401811+0000", 2024-09-04T17:46:21.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9171e-05 2024-09-04T17:46:21.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.974 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:21.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.401824+0000", 2024-09-04T17:46:21.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3611999999999999e-05 2024-09-04T17:46:21.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.974 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:21.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.401897+0000", 2024-09-04T17:46:21.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2707999999999996e-05 2024-09-04T17:46:21.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:21.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.401920+0000", 2024-09-04T17:46:21.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3496999999999999e-05 2024-09-04T17:46:21.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:21.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.403808+0000", 2024-09-04T17:46:21.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0018875179999999999 2024-09-04T17:46:21.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:21.976 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.403844+0000", 2024-09-04T17:46:21.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5808e-05 2024-09-04T17:46:21.976 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:21.976 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:21.976 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:21.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.977 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:5000 3.9 e31/24 3:94906c79:::rbd_data.1134cfbce83d.0000000000000012:head v 31'91, mlcod=30'85)", 2024-09-04T17:46:21.977 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:07.402507+0000", 2024-09-04T17:46:21.977 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.533479383, 2024-09-04T17:46:21.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004039735, 2024-09-04T17:46:21.977 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:21.977 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:21.977 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:21.977 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:21.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.978 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:21.978 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.402507+0000", 2024-09-04T17:46:21.978 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:21.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.978 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.978 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:21.978 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.402507+0000", 2024-09-04T17:46:21.978 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:21.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.978 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.978 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:21.978 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.402509+0000", 2024-09-04T17:46:21.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5939999999999999e-06 2024-09-04T17:46:21.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:21.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.402521+0000", 2024-09-04T17:46:21.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1786e-05 2024-09-04T17:46:21.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:21.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.402543+0000", 2024-09-04T17:46:21.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2158e-05 2024-09-04T17:46:21.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:21.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.402555+0000", 2024-09-04T17:46:21.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2034e-05 2024-09-04T17:46:21.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:21.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.402660+0000", 2024-09-04T17:46:21.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010535900000000001 2024-09-04T17:46:21.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:21.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.402696+0000", 2024-09-04T17:46:21.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5488000000000003e-05 2024-09-04T17:46:21.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:21.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.406510+0000", 2024-09-04T17:46:21.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0038142390000000001 2024-09-04T17:46:21.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:21.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.406547+0000", 2024-09-04T17:46:21.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7076999999999998e-05 2024-09-04T17:46:21.982 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:21.982 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:21.982 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:21.982 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.982 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.982 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:5834 3.9 e32/24 3:952cd39c:::rbd_header.11343d768295:head v 32'92, mlcod=31'89)", 2024-09-04T17:46:21.982 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.339000+0000", 2024-09-04T17:46:21.983 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.59698717, 2024-09-04T17:46:21.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020947890000000001, 2024-09-04T17:46:21.983 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:21.983 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:21.983 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:21.983 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:21.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.983 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:21.983 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.339000+0000", 2024-09-04T17:46:21.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:21.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:21.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.339000+0000", 2024-09-04T17:46:21.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:21.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:21.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.339000+0000", 2024-09-04T17:46:21.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8299999999999997e-07 2024-09-04T17:46:21.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:21.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.339009+0000", 2024-09-04T17:46:21.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6300000000000004e-06 2024-09-04T17:46:21.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:21.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.339038+0000", 2024-09-04T17:46:21.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9442e-05 2024-09-04T17:46:21.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:21.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.339053+0000", 2024-09-04T17:46:21.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.503e-05 2024-09-04T17:46:21.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.986 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.986 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:21.986 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.339212+0000", 2024-09-04T17:46:21.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015852600000000001 2024-09-04T17:46:21.986 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.986 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.986 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:21.986 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.339277+0000", 2024-09-04T17:46:21.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5025999999999997e-05 2024-09-04T17:46:21.986 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.986 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.986 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:21.986 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.340983+0000", 2024-09-04T17:46:21.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0017062170000000001 2024-09-04T17:46:21.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.987 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.987 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:21.987 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.341094+0000", 2024-09-04T17:46:21.987 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011143500000000001 2024-09-04T17:46:21.987 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:21.987 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:21.987 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:21.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.987 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.987 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:6311 3.9 e32/24 3:941d6897:::rbd_data.113431586065.0000000000000008:head v 32'97, mlcod=32'95)", 2024-09-04T17:46:21.987 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.479986+0000", 2024-09-04T17:46:21.988 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.456000486000001, 2024-09-04T17:46:21.988 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018493539, 2024-09-04T17:46:21.988 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:21.988 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:21.988 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:21.988 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:21.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.988 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:21.988 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.479986+0000", 2024-09-04T17:46:21.988 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:21.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.988 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:21.989 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.479986+0000", 2024-09-04T17:46:21.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:21.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.989 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.989 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:21.989 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.479988+0000", 2024-09-04T17:46:21.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.193e-06 2024-09-04T17:46:21.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.989 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.989 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:21.989 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484109+0000", 2024-09-04T17:46:21.990 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041201650000000003 2024-09-04T17:46:21.990 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.990 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.990 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:21.990 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484139+0000", 2024-09-04T17:46:21.990 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.082e-05 2024-09-04T17:46:21.990 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.990 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.990 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:21.990 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484153+0000", 2024-09-04T17:46:21.990 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3456e-05 2024-09-04T17:46:21.990 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.991 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.991 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:21.991 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484250+0000", 2024-09-04T17:46:21.991 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7638000000000003e-05 2024-09-04T17:46:21.991 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.991 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.991 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:21.992 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484282+0000", 2024-09-04T17:46:21.992 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1511999999999999e-05 2024-09-04T17:46:21.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.992 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:21.992 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.498411+0000", 2024-09-04T17:46:21.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014128870999999999 2024-09-04T17:46:21.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:21.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.498480+0000", 2024-09-04T17:46:21.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8884e-05 2024-09-04T17:46:21.994 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:21.994 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:21.994 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:21.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.994 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:6312 3.9 e32/24 3:941d6897:::rbd_data.113431586065.0000000000000008:head v 32'99, mlcod=32'95)", 2024-09-04T17:46:21.995 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.486036+0000", 2024-09-04T17:46:21.995 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.449950407999999, 2024-09-04T17:46:21.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013432773, 2024-09-04T17:46:21.995 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:21.995 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:21.995 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:21.995 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:21.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.996 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:21.996 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.486036+0000", 2024-09-04T17:46:21.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:21.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.996 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:21.996 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.486036+0000", 2024-09-04T17:46:21.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:21.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.996 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:21.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.486038+0000", 2024-09-04T17:46:21.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.536e-06 2024-09-04T17:46:21.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:21.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.486062+0000", 2024-09-04T17:46:21.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3672e-05 2024-09-04T17:46:21.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:21.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.486078+0000", 2024-09-04T17:46:21.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.613e-05 2024-09-04T17:46:21.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:21.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.486086+0000", 2024-09-04T17:46:21.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8849999999999999e-06 2024-09-04T17:46:21.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:21.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.491843+0000", 2024-09-04T17:46:21.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057572040000000001 2024-09-04T17:46:21.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:21.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.491876+0000", 2024-09-04T17:46:21.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3515e-05 2024-09-04T17:46:21.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:21.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:21.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:21.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.499410+0000", 2024-09-04T17:46:21.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0075333780000000003 2024-09-04T17:46:21.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:22.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.499469+0000", 2024-09-04T17:46:22.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9453000000000002e-05 2024-09-04T17:46:22.000 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:22.000 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:22.000 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:22.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.000 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:8161 3.4 e35/24 3:250dcf35:::rbd_data.11345bd5898b.000000000000000d:head v 35'46, mlcod=34'44)", 2024-09-04T17:46:22.000 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:12.105762+0000", 2024-09-04T17:46:22.000 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.8302249889999995, 2024-09-04T17:46:22.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002035111, 2024-09-04T17:46:22.001 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:22.001 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:22.001 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:22.001 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:22.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:22.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:12.105762+0000", 2024-09-04T17:46:22.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:22.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:22.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:12.105762+0000", 2024-09-04T17:46:22.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:22.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:22.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:12.105764+0000", 2024-09-04T17:46:22.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9889999999999999e-06 2024-09-04T17:46:22.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:22.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:12.105806+0000", 2024-09-04T17:46:22.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2315999999999999e-05 2024-09-04T17:46:22.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:22.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:12.105824+0000", 2024-09-04T17:46:22.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7839e-05 2024-09-04T17:46:22.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:22.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:12.105834+0000", 2024-09-04T17:46:22.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6420000000000002e-06 2024-09-04T17:46:22.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:22.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:12.105896+0000", 2024-09-04T17:46:22.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2541999999999999e-05 2024-09-04T17:46:22.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:22.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:12.105917+0000", 2024-09-04T17:46:22.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1097000000000001e-05 2024-09-04T17:46:22.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:22.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:12.107755+0000", 2024-09-04T17:46:22.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0018381820000000001 2024-09-04T17:46:22.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.006 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:22.006 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:12.107797+0000", 2024-09-04T17:46:22.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1504000000000001e-05 2024-09-04T17:46:22.006 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:22.006 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:22.007 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:22.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.007 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:12583 3.1f e39/37 3:fb91fe3f:::rbd_object_map.1134ef3a5a7d:head v 39'106, mlcod=39'102)", 2024-09-04T17:46:22.007 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:15.954581+0000", 2024-09-04T17:46:22.008 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.9814052870000003, 2024-09-04T17:46:22.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002257897, 2024-09-04T17:46:22.008 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:22.008 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:22.008 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:22.008 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:22.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:22.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:15.954581+0000", 2024-09-04T17:46:22.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:22.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:22.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:15.954581+0000", 2024-09-04T17:46:22.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:22.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:22.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:15.954583+0000", 2024-09-04T17:46:22.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.939e-06 2024-09-04T17:46:22.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:22.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:15.954596+0000", 2024-09-04T17:46:22.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2576e-05 2024-09-04T17:46:22.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:22.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:15.954632+0000", 2024-09-04T17:46:22.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6044999999999998e-05 2024-09-04T17:46:22.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:22.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:15.954655+0000", 2024-09-04T17:46:22.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2753999999999999e-05 2024-09-04T17:46:22.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:22.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:15.954729+0000", 2024-09-04T17:46:22.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4575999999999999e-05 2024-09-04T17:46:22.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:22.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:15.954745+0000", 2024-09-04T17:46:22.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5221e-05 2024-09-04T17:46:22.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.016 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:22.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:15.956760+0000", 2024-09-04T17:46:22.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002015209 2024-09-04T17:46:22.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:22.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:15.956839+0000", 2024-09-04T17:46:22.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9577000000000001e-05 2024-09-04T17:46:22.017 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:22.017 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:22.017 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:22.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.018 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:12734 3.0 e40/24 3:042a5eed:::rbd_data.1134ef3a5a7d.00000000000000e3:head v 40'22, mlcod=39'19)", 2024-09-04T17:46:22.018 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:16.432897+0000", 2024-09-04T17:46:22.018 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.5030894369999999, 2024-09-04T17:46:22.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002600261, 2024-09-04T17:46:22.018 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:22.019 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:22.019 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:22.019 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:22.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:22.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:16.432897+0000", 2024-09-04T17:46:22.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:22.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:22.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:16.432897+0000", 2024-09-04T17:46:22.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:22.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.021 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:22.021 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:16.432901+0000", 2024-09-04T17:46:22.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3239999999999999e-06 2024-09-04T17:46:22.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:22.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:16.432917+0000", 2024-09-04T17:46:22.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6801000000000001e-05 2024-09-04T17:46:22.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:22.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:16.432952+0000", 2024-09-04T17:46:22.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4348999999999999e-05 2024-09-04T17:46:22.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:22.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:16.432972+0000", 2024-09-04T17:46:22.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9845000000000001e-05 2024-09-04T17:46:22.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:22.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:16.433130+0000", 2024-09-04T17:46:22.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000158414 2024-09-04T17:46:22.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:22.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:16.433182+0000", 2024-09-04T17:46:22.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1891000000000002e-05 2024-09-04T17:46:22.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:22.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:16.435370+0000", 2024-09-04T17:46:22.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0021883850000000002 2024-09-04T17:46:22.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:22.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:16.435498+0000", 2024-09-04T17:46:22.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012725200000000001 2024-09-04T17:46:22.028 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:22.028 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:22.028 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:22.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.028 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:12752 3.9 e41/24 3:9714f662:::rbd_data.1134ef3a5a7d.000000000000000a:head v 41'132, mlcod=35'129)", 2024-09-04T17:46:22.029 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:17.329909+0000", 2024-09-04T17:46:22.029 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.6060772349999999, 2024-09-04T17:46:22.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002007609, 2024-09-04T17:46:22.029 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:22.029 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:22.029 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:22.029 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:22.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:22.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.329909+0000", 2024-09-04T17:46:22.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:22.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:22.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.329909+0000", 2024-09-04T17:46:22.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:22.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:22.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.329910+0000", 2024-09-04T17:46:22.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5900000000000002e-07 2024-09-04T17:46:22.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:22.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.329916+0000", 2024-09-04T17:46:22.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9780000000000002e-06 2024-09-04T17:46:22.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:22.033 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.329933+0000", 2024-09-04T17:46:22.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7541e-05 2024-09-04T17:46:22.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:22.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.329944+0000", 2024-09-04T17:46:22.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0856000000000001e-05 2024-09-04T17:46:22.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:22.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.330096+0000", 2024-09-04T17:46:22.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015196900000000001 2024-09-04T17:46:22.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:22.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.330162+0000", 2024-09-04T17:46:22.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5449000000000003e-05 2024-09-04T17:46:22.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:22.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.331823+0000", 2024-09-04T17:46:22.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001661748 2024-09-04T17:46:22.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:22.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.331917+0000", 2024-09-04T17:46:22.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3609000000000004e-05 2024-09-04T17:46:22.036 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:22.036 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:22.036 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:22.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.036 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:12747 3.1f e41/37 3:fb91fe3f:::rbd_object_map.1134ef3a5a7d:head v 41'125, mlcod=41'122)", 2024-09-04T17:46:22.037 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:17.330322+0000", 2024-09-04T17:46:22.037 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.6056649570000001, 2024-09-04T17:46:22.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004205762, 2024-09-04T17:46:22.037 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:22.037 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:22.037 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:22.037 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:22.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.037 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:22.037 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.330322+0000", 2024-09-04T17:46:22.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:22.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:22.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.330322+0000", 2024-09-04T17:46:22.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:22.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:22.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.330323+0000", 2024-09-04T17:46:22.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9500000000000001e-07 2024-09-04T17:46:22.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:22.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.330329+0000", 2024-09-04T17:46:22.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2040000000000002e-06 2024-09-04T17:46:22.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:22.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.330341+0000", 2024-09-04T17:46:22.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1727e-05 2024-09-04T17:46:22.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:22.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.330348+0000", 2024-09-04T17:46:22.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.187e-06 2024-09-04T17:46:22.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:22.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.330455+0000", 2024-09-04T17:46:22.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107311 2024-09-04T17:46:22.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:22.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.330495+0000", 2024-09-04T17:46:22.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0191999999999997e-05 2024-09-04T17:46:22.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:22.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.334480+0000", 2024-09-04T17:46:22.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039848410000000003 2024-09-04T17:46:22.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:22.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.334528+0000", 2024-09-04T17:46:22.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7404999999999998e-05 2024-09-04T17:46:22.041 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:22.041 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:22.041 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:22.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.042 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:12748 3.1f e41/37 3:fb91fe3f:::rbd_object_map.1134ef3a5a7d:head v 41'126, mlcod=41'122)", 2024-09-04T17:46:22.042 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:17.332613+0000", 2024-09-04T17:46:22.042 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.6033739780000005, 2024-09-04T17:46:22.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0044275870000000002, 2024-09-04T17:46:22.042 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:22.042 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:22.042 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:22.042 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:22.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:22.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.332613+0000", 2024-09-04T17:46:22.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:22.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:22.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.332613+0000", 2024-09-04T17:46:22.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:22.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:22.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.332614+0000", 2024-09-04T17:46:22.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.051e-06 2024-09-04T17:46:22.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:22.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.332620+0000", 2024-09-04T17:46:22.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1680000000000001e-06 2024-09-04T17:46:22.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:22.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.332635+0000", 2024-09-04T17:46:22.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5543e-05 2024-09-04T17:46:22.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:22.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.332643+0000", 2024-09-04T17:46:22.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.537e-06 2024-09-04T17:46:22.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:22.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.332717+0000", 2024-09-04T17:46:22.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3701999999999993e-05 2024-09-04T17:46:22.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:22.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.332786+0000", 2024-09-04T17:46:22.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9669e-05 2024-09-04T17:46:22.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:22.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.336923+0000", 2024-09-04T17:46:22.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041365259999999997 2024-09-04T17:46:22.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:22.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.337040+0000", 2024-09-04T17:46:22.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000117391 2024-09-04T17:46:22.047 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:22.047 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:22.047 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:22.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.047 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:12749 3.1f e41/37 3:fb91fe3f:::rbd_object_map.1134ef3a5a7d:head v 41'127, mlcod=41'122)", 2024-09-04T17:46:22.047 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:17.335529+0000", 2024-09-04T17:46:22.048 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.6004579369999998, 2024-09-04T17:46:22.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052795899999999998, 2024-09-04T17:46:22.048 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:22.048 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:22.048 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:22.048 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:22.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:22.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.335529+0000", 2024-09-04T17:46:22.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:22.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:22.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.335529+0000", 2024-09-04T17:46:22.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:22.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:22.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.335535+0000", 2024-09-04T17:46:22.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4060000000000002e-06 2024-09-04T17:46:22.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:22.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.335546+0000", 2024-09-04T17:46:22.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0652000000000001e-05 2024-09-04T17:46:22.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:22.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.335571+0000", 2024-09-04T17:46:22.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5148e-05 2024-09-04T17:46:22.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:22.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.335583+0000", 2024-09-04T17:46:22.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2315e-05 2024-09-04T17:46:22.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:22.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.335699+0000", 2024-09-04T17:46:22.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000115443 2024-09-04T17:46:22.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:22.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.335736+0000", 2024-09-04T17:46:22.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6779999999999997e-05 2024-09-04T17:46:22.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:22.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.340722+0000", 2024-09-04T17:46:22.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0049860390000000003 2024-09-04T17:46:22.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:22.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.340808+0000", 2024-09-04T17:46:22.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6806999999999997e-05 2024-09-04T17:46:22.052 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:22.052 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:22.052 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:22.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.052 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:12750 3.1f e41/37 3:fb91fe3f:::rbd_object_map.1134ef3a5a7d:head v 41'128, mlcod=41'123)", 2024-09-04T17:46:22.053 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:17.337581+0000", 2024-09-04T17:46:22.053 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.5984053930000002, 2024-09-04T17:46:22.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045169579999999997, 2024-09-04T17:46:22.053 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:22.053 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:22.053 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:22.053 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:22.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:22.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.337581+0000", 2024-09-04T17:46:22.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:22.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:22.054 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.337581+0000", 2024-09-04T17:46:22.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:22.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:22.054 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.337583+0000", 2024-09-04T17:46:22.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5200000000000001e-06 2024-09-04T17:46:22.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:22.054 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.337595+0000", 2024-09-04T17:46:22.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2395000000000001e-05 2024-09-04T17:46:22.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:22.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.337620+0000", 2024-09-04T17:46:22.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4794e-05 2024-09-04T17:46:22.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:22.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.337634+0000", 2024-09-04T17:46:22.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4241e-05 2024-09-04T17:46:22.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:22.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.337731+0000", 2024-09-04T17:46:22.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6246e-05 2024-09-04T17:46:22.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:22.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.337762+0000", 2024-09-04T17:46:22.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1279999999999999e-05 2024-09-04T17:46:22.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:22.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.342050+0000", 2024-09-04T17:46:22.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0042879729999999996 2024-09-04T17:46:22.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:22.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.342098+0000", 2024-09-04T17:46:22.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8508999999999999e-05 2024-09-04T17:46:22.059 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:22.059 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:22.059 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:22.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.059 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:12756 3.0 e41/24 3:0074e6f8:::rbd_data.1134ef3a5a7d.000000000000000e:head v 41'24, mlcod=40'20)", 2024-09-04T17:46:22.059 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:17.339316+0000", 2024-09-04T17:46:22.059 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.5966711700000005, 2024-09-04T17:46:22.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052778790000000001, 2024-09-04T17:46:22.059 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:22.059 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:22.059 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:22.059 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:22.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.060 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:22.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.339316+0000", 2024-09-04T17:46:22.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:22.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.060 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:22.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.339316+0000", 2024-09-04T17:46:22.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:22.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:22.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.339317+0000", 2024-09-04T17:46:22.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.13e-06 2024-09-04T17:46:22.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:22.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.339328+0000", 2024-09-04T17:46:22.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1742e-05 2024-09-04T17:46:22.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:22.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.339358+0000", 2024-09-04T17:46:22.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9149000000000002e-05 2024-09-04T17:46:22.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:22.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.339379+0000", 2024-09-04T17:46:22.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1288e-05 2024-09-04T17:46:22.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:22.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.339454+0000", 2024-09-04T17:46:22.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4703000000000006e-05 2024-09-04T17:46:22.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:22.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.339506+0000", 2024-09-04T17:46:22.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2188999999999998e-05 2024-09-04T17:46:22.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:22.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.344475+0000", 2024-09-04T17:46:22.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0049691919999999999 2024-09-04T17:46:22.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:22.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:22.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:22.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.344593+0000", 2024-09-04T17:46:22.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000118486 2024-09-04T17:46:22.064 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:22.065 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:22.065 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:22.065 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:22.065 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:22.065 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:22.066 INFO:teuthology.orchestra.run.smithi092.stdout:1099 writesame 0x699eed8 thru 0x69ad7a3 data_size 0x5e(0xe8cc bytes) 2024-09-04T17:46:22.067 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-04T17:46:22.070 INFO:teuthology.orchestra.run.smithi092.stdout:1100 read 0xc482a13 thru 0xc487c4e (0x523c bytes) 2024-09-04T17:46:22.070 INFO:teuthology.orchestra.run.smithi092.stdout:1101 read 0x7949e6c thru 0x7955fd1 (0xc166 bytes) 2024-09-04T17:46:22.070 INFO:teuthology.orchestra.run.smithi092.stdout:1102 write 0x4c5b3da thru 0x4c5f502 (0x4129 bytes) 2024-09-04T17:46:22.070 INFO:teuthology.orchestra.run.smithi092.stdout:1104 writesame 0x7a04b35 thru 0x7a0be30 data_size 0x16(0x72fc bytes) 2024-09-04T17:46:22.070 INFO:teuthology.orchestra.run.smithi092.stdout:1105 write 0x9be2f27 thru 0x9be5b40 (0x2c1a bytes) 2024-09-04T17:46:22.070 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:22.070 INFO:teuthology.orchestra.run.smithi092.stdout:1107 trunc from 0xe6660f4 to 0xd2602cd 2024-09-04T17:46:22.070 INFO:teuthology.orchestra.run.smithi092.stdout:1110 write 0x6b722a thru 0x6c2822 (0xb5f9 bytes) 2024-09-04T17:46:22.070 INFO:teuthology.orchestra.run.smithi092.stdout:1111 read 0x9bcd160 thru 0x9bd3569 (0x640a bytes) 2024-09-04T17:46:22.070 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:22.071 INFO:teuthology.orchestra.run.smithi092.stdout:1113 read 0xbfc310b thru 0xbfcd247 (0xa13d bytes) 2024-09-04T17:46:22.071 INFO:teuthology.orchestra.run.smithi092.stdout:1114 read 0xa119d24 thru 0xa11e892 (0x4b6f bytes) 2024-09-04T17:46:22.071 INFO:teuthology.orchestra.run.smithi092.stdout:1116 read 0x5f42fc thru 0x5f46fd (0x402 bytes) 2024-09-04T17:46:22.071 INFO:teuthology.orchestra.run.smithi092.stdout:1117 writesame 0x539726c thru 0x53a3a40 data_size 0x1(0xc7d5 bytes) 2024-09-04T17:46:22.071 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:22.071 INFO:teuthology.orchestra.run.smithi092.stdout:1120 writesame 0x5e40cfe thru 0x5e445be data_size 0x1d(0x38c1 bytes) 2024-09-04T17:46:22.071 INFO:teuthology.orchestra.run.smithi092.stdout:1121 punch from 0x8bc0a5d to 0x8bc12a6, (0x849 bytes) 2024-09-04T17:46:22.071 INFO:teuthology.orchestra.run.smithi092.stdout:1122 read 0x681bde1 thru 0x6827ae2 (0xbd02 bytes) 2024-09-04T17:46:22.071 INFO:teuthology.orchestra.run.smithi092.stdout:1124 read 0xb144f4 thru 0xb21da4 (0xd8b1 bytes) 2024-09-04T17:46:22.071 INFO:teuthology.orchestra.run.smithi092.stdout:1126 read 0xa5d1188 thru 0xa5d3a1f (0x2898 bytes) 2024-09-04T17:46:22.071 INFO:teuthology.orchestra.run.smithi092.stdout:1127 read 0x5afb6a5 thru 0x5b07d1c (0xc678 bytes) 2024-09-04T17:46:22.071 INFO:teuthology.orchestra.run.smithi092.stdout:1128 flatten 2024-09-04T17:46:22.072 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:22.065+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:22.167 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:22.167+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:22.262 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:22.262 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:46:22.262 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:46:22.262 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:22.267 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:22.269+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:22.282 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-04T17:46:22.303 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is set 2024-09-04T17:46:22.342 INFO:teuthology.orchestra.run.smithi092.stdout:1129 punch from 0xa097bb3 to 0xa09fb54, (0x7fa1 bytes) 2024-09-04T17:46:22.342 INFO:teuthology.orchestra.run.smithi092.stdout:1130 read 0x786b6ec thru 0x787b6c8 (0xffdd bytes) 2024-09-04T17:46:22.343 INFO:teuthology.orchestra.run.smithi092.stdout:1131 writesame 0x2925058 thru 0x2932c09 data_size 0x3d(0xdbb2 bytes) 2024-09-04T17:46:22.351 INFO:teuthology.orchestra.run.smithi092.stdout:1133 writesame 0x6faccda thru 0x6fb4e95 data_size 0x4c(0x81bc bytes) 2024-09-04T17:46:22.361 INFO:teuthology.orchestra.run.smithi092.stdout:1134 read 0x3a5f17a thru 0x3a67bdd (0x8a64 bytes) 2024-09-04T17:46:22.361 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:22.361 INFO:teuthology.orchestra.run.smithi092.stdout:1137 writesame 0x254141d thru 0x254d543 data_size 0x1(0xc127 bytes) 2024-09-04T17:46:22.369 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:22.369+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:22.380 INFO:teuthology.orchestra.run.smithi092.stdout:1138 punch from 0x84a723f to 0x84acb6f, (0x5930 bytes) 2024-09-04T17:46:22.380 INFO:teuthology.orchestra.run.smithi092.stdout:1139 write 0x3ed652 thru 0x3f87a1 (0xb150 bytes) 2024-09-04T17:46:22.389 INFO:teuthology.orchestra.run.smithi092.stdout:1141 write 0x1eb5dde thru 0x1ebc0d0 (0x62f3 bytes) 2024-09-04T17:46:22.395 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:22.395 INFO:teuthology.orchestra.run.smithi092.stdout:1147 write 0x13e4259 thru 0x13f1714 (0xd4bc bytes) 2024-09-04T17:46:22.401 INFO:teuthology.orchestra.run.smithi092.stdout:1148 clone 12 order 24 su 524288 sc 6 2024-09-04T17:46:22.452 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:22.452 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:46:22.452 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:46:22.452 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:46:22.452 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:22.466 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-04T17:46:22.470 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:22.467+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:22.570 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:22.567+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:22.592 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:22.592 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:46:22.592 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:46:22.592 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:46:22.592 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.592 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4264 3.9 3:970e51a0:::rbd_data.1134cfbce83d.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 1597545~615] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-04T17:46:22.592 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.021843+0000", 2024-09-04T17:46:22.592 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 15.570351218999999, 2024-09-04T17:46:22.593 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.015077662, 2024-09-04T17:46:22.593 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:22.593 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:22.593 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:22.593 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:22.593 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:22.593 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:22.593 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4264 2024-09-04T17:46:22.593 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.593 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:22.594 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.594 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:22.594 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.021843+0000", 2024-09-04T17:46:22.594 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:22.594 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.594 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.594 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:22.594 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.021843+0000", 2024-09-04T17:46:22.594 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:22.594 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.594 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.594 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:22.595 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.021845+0000", 2024-09-04T17:46:22.595 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.328e-06 2024-09-04T17:46:22.595 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.595 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.595 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:22.595 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.026201+0000", 2024-09-04T17:46:22.595 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.004355935 2024-09-04T17:46:22.595 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.595 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.595 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:22.595 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.026214+0000", 2024-09-04T17:46:22.596 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3315999999999999e-05 2024-09-04T17:46:22.596 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.596 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.596 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:22.596 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.026221+0000", 2024-09-04T17:46:22.596 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7070000000000003e-06 2024-09-04T17:46:22.596 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.596 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.596 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:22.596 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.026266+0000", 2024-09-04T17:46:22.597 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.5120999999999997e-05 2024-09-04T17:46:22.597 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.597 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.597 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:22.597 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.026375+0000", 2024-09-04T17:46:22.597 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00010935 2024-09-04T17:46:22.597 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.597 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.597 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 2", 2024-09-04T17:46:22.597 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.026610+0000", 2024-09-04T17:46:22.598 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00023530400000000001 2024-09-04T17:46:22.598 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.598 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.598 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:22.598 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.032096+0000", 2024-09-04T17:46:22.598 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0054859050000000001 2024-09-04T17:46:22.598 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.598 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.598 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:22.598 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.036851+0000", 2024-09-04T17:46:22.598 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0047543409999999996 2024-09-04T17:46:22.598 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.599 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.599 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:22.599 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.036890+0000", 2024-09-04T17:46:22.599 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.9128999999999998e-05 2024-09-04T17:46:22.599 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.599 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.599 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:22.599 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.036921+0000", 2024-09-04T17:46:22.599 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.1226000000000002e-05 2024-09-04T17:46:22.599 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:22.599 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:22.600 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:22.600 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.600 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.600 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4651 3.3 3:c43d9109:::rbd_data.113417f3df25.0000000000000002:5 [read 4063232~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:22.600 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.349383+0000", 2024-09-04T17:46:22.600 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 15.242811213, 2024-09-04T17:46:22.600 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.014661281, 2024-09-04T17:46:22.600 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:22.600 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:22.600 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:22.600 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:22.600 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:22.601 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:22.601 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4651 2024-09-04T17:46:22.601 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.601 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:22.601 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.601 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:22.601 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.349383+0000", 2024-09-04T17:46:22.601 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:22.601 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.601 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.601 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:22.601 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.349383+0000", 2024-09-04T17:46:22.602 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:22.602 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.602 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.602 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:22.602 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.349384+0000", 2024-09-04T17:46:22.602 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.4499999999999996e-07 2024-09-04T17:46:22.602 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.602 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.602 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:22.602 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.349401+0000", 2024-09-04T17:46:22.602 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6339000000000001e-05 2024-09-04T17:46:22.603 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.603 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.603 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:22.603 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.349411+0000", 2024-09-04T17:46:22.603 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0013000000000001e-05 2024-09-04T17:46:22.603 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.603 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.603 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:22.603 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.349433+0000", 2024-09-04T17:46:22.603 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1926999999999999e-05 2024-09-04T17:46:22.603 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.603 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.604 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:22.604 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.363824+0000", 2024-09-04T17:46:22.604 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.014391736 2024-09-04T17:46:22.604 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.604 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.604 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:22.604 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.363920+0000", 2024-09-04T17:46:22.604 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.6142000000000002e-05 2024-09-04T17:46:22.604 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.604 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.604 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:22.604 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.364045+0000", 2024-09-04T17:46:22.605 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00012427900000000001 2024-09-04T17:46:22.605 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:22.605 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:22.605 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:22.605 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.605 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.605 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4734 3.3 3:c43d9109:::rbd_data.113417f3df25.0000000000000002:5 [read 196608~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:22.605 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.349456+0000", 2024-09-04T17:46:22.605 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 15.242738147000001, 2024-09-04T17:46:22.605 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.014853676999999999, 2024-09-04T17:46:22.605 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:22.605 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:22.606 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:22.606 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:22.606 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:22.606 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:22.606 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4734 2024-09-04T17:46:22.606 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.606 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:22.606 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.606 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:22.606 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.349456+0000", 2024-09-04T17:46:22.606 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:22.606 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.606 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.607 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:22.607 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.349456+0000", 2024-09-04T17:46:22.607 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:22.607 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.607 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.607 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:22.607 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.349457+0000", 2024-09-04T17:46:22.607 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.9999999999999996e-07 2024-09-04T17:46:22.607 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.607 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.607 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:22.607 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.349461+0000", 2024-09-04T17:46:22.607 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.9099999999999998e-06 2024-09-04T17:46:22.608 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.608 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.608 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:22.608 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.349471+0000", 2024-09-04T17:46:22.608 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.8069999999999994e-06 2024-09-04T17:46:22.608 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.608 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.608 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:22.608 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.349479+0000", 2024-09-04T17:46:22.608 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.8769999999999993e-06 2024-09-04T17:46:22.608 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.609 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.609 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:22.609 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.364085+0000", 2024-09-04T17:46:22.609 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.014606022 2024-09-04T17:46:22.609 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.609 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.609 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:22.609 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.364184+0000", 2024-09-04T17:46:22.609 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.8948000000000002e-05 2024-09-04T17:46:22.609 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.609 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.610 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:22.610 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.364310+0000", 2024-09-04T17:46:22.610 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00012631299999999999 2024-09-04T17:46:22.610 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:22.610 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:22.610 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:22.610 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.610 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.610 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4735 3.3 3:c43d9109:::rbd_data.113417f3df25.0000000000000002:5 [read 622592~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:22.610 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.349490+0000", 2024-09-04T17:46:22.610 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 15.242704764999999, 2024-09-04T17:46:22.610 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.015069948999999999, 2024-09-04T17:46:22.611 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:22.611 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:22.611 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:22.611 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:22.611 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:22.611 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:22.611 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4735 2024-09-04T17:46:22.611 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.611 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:22.611 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.611 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:22.612 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.349490+0000", 2024-09-04T17:46:22.612 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:22.612 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.612 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.612 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:22.612 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.349490+0000", 2024-09-04T17:46:22.612 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:22.612 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.612 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.612 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:22.613 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.349491+0000", 2024-09-04T17:46:22.613 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6000000000000003e-07 2024-09-04T17:46:22.613 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.613 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.613 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:22.613 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.349493+0000", 2024-09-04T17:46:22.613 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.9349999999999999e-06 2024-09-04T17:46:22.613 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.613 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.613 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:22.613 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.349503+0000", 2024-09-04T17:46:22.613 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4460000000000007e-06 2024-09-04T17:46:22.614 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.614 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.614 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:22.614 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.349510+0000", 2024-09-04T17:46:22.614 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7109999999999998e-06 2024-09-04T17:46:22.614 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.614 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.614 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:22.614 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.364347+0000", 2024-09-04T17:46:22.614 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.014837250999999999 2024-09-04T17:46:22.614 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.615 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.615 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:22.615 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.364440+0000", 2024-09-04T17:46:22.615 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.3121000000000005e-05 2024-09-04T17:46:22.615 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.615 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.615 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:22.615 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.364560+0000", 2024-09-04T17:46:22.615 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000119825 2024-09-04T17:46:22.615 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:22.615 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:22.616 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:22.616 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.616 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.616 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4811 3.3 3:c43d9109:::rbd_data.113417f3df25.0000000000000002:5 [read 3244032~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:22.616 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.355350+0000", 2024-09-04T17:46:22.616 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 15.236844915000001, 2024-09-04T17:46:22.616 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.015364724, 2024-09-04T17:46:22.616 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:22.616 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:22.616 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:22.616 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:22.617 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:22.617 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:22.617 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4811 2024-09-04T17:46:22.617 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.617 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:22.617 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.617 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:22.617 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.355350+0000", 2024-09-04T17:46:22.617 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:22.617 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.617 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.617 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:22.618 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.355350+0000", 2024-09-04T17:46:22.618 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:22.618 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.618 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.618 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:22.618 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.355350+0000", 2024-09-04T17:46:22.618 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.9400000000000005e-07 2024-09-04T17:46:22.618 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.618 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.618 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:22.618 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.355354+0000", 2024-09-04T17:46:22.619 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.8600000000000003e-06 2024-09-04T17:46:22.619 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.619 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.619 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:22.619 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.355363+0000", 2024-09-04T17:46:22.619 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.0739999999999999e-06 2024-09-04T17:46:22.619 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.619 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.619 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:22.619 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.355370+0000", 2024-09-04T17:46:22.619 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2370000000000003e-06 2024-09-04T17:46:22.620 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.620 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.620 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:22.620 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.370484+0000", 2024-09-04T17:46:22.620 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.015113909 2024-09-04T17:46:22.620 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.620 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.620 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:22.620 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.370584+0000", 2024-09-04T17:46:22.620 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000100042 2024-09-04T17:46:22.620 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.621 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.621 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:22.621 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.370714+0000", 2024-09-04T17:46:22.621 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013000800000000001 2024-09-04T17:46:22.621 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:22.621 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:22.621 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:22.621 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.621 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.621 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4812 3.3 3:c43d9109:::rbd_data.113417f3df25.0000000000000002:5 [read 3670016~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:22.621 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.355380+0000", 2024-09-04T17:46:22.622 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 15.236814768, 2024-09-04T17:46:22.622 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.015590292, 2024-09-04T17:46:22.622 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:22.622 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:22.622 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:22.622 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:22.622 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:22.622 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:22.622 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4812 2024-09-04T17:46:22.622 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.622 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:22.622 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.622 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:22.623 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.355380+0000", 2024-09-04T17:46:22.623 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:22.623 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.623 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.623 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:22.623 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.355380+0000", 2024-09-04T17:46:22.623 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:22.623 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.623 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.623 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:22.623 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.355381+0000", 2024-09-04T17:46:22.624 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.3799999999999997e-07 2024-09-04T17:46:22.624 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.624 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.624 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:22.624 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.355384+0000", 2024-09-04T17:46:22.624 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.027e-06 2024-09-04T17:46:22.624 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.624 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.624 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:22.624 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.355393+0000", 2024-09-04T17:46:22.624 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.0049999999999993e-06 2024-09-04T17:46:22.625 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.625 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.625 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:22.625 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.355404+0000", 2024-09-04T17:46:22.625 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1154e-05 2024-09-04T17:46:22.625 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.625 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.625 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:22.625 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.370754+0000", 2024-09-04T17:46:22.625 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.015350205 2024-09-04T17:46:22.625 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.626 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.626 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:22.626 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.370847+0000", 2024-09-04T17:46:22.626 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.3441000000000002e-05 2024-09-04T17:46:22.626 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.626 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.626 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:22.626 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.370970+0000", 2024-09-04T17:46:22.626 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000122822 2024-09-04T17:46:22.626 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:22.626 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:22.626 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:22.627 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.627 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.627 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4813 3.3 3:c43d9109:::rbd_data.113417f3df25.0000000000000002:5 [read 4096000~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:22.627 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.355699+0000", 2024-09-04T17:46:22.627 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 15.236495138, 2024-09-04T17:46:22.627 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.01551072, 2024-09-04T17:46:22.627 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:22.627 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:22.627 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:22.627 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:22.627 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:22.628 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:22.628 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4813 2024-09-04T17:46:22.628 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.628 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:22.628 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.628 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:22.628 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.355699+0000", 2024-09-04T17:46:22.628 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:22.628 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.628 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.629 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:22.629 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.355699+0000", 2024-09-04T17:46:22.629 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:22.629 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.629 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.629 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:22.629 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.355701+0000", 2024-09-04T17:46:22.629 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1000000000000001e-06 2024-09-04T17:46:22.629 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.629 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.629 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:22.629 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.355705+0000", 2024-09-04T17:46:22.630 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7720000000000002e-06 2024-09-04T17:46:22.630 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.630 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.630 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:22.630 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.355720+0000", 2024-09-04T17:46:22.630 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4548e-05 2024-09-04T17:46:22.630 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.630 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.630 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:22.630 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.355730+0000", 2024-09-04T17:46:22.630 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0113e-05 2024-09-04T17:46:22.630 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.631 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.631 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:22.631 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.371006+0000", 2024-09-04T17:46:22.631 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.015276035 2024-09-04T17:46:22.631 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.631 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.631 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:22.631 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.371097+0000", 2024-09-04T17:46:22.631 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.0789000000000001e-05 2024-09-04T17:46:22.631 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.631 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.632 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:22.632 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.371210+0000", 2024-09-04T17:46:22.632 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00011336300000000001 2024-09-04T17:46:22.632 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:22.632 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:22.632 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:22.632 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.632 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.632 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:5009 3.14 3:2de14b79:::rbd_data.1134cfbce83d.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:22.632 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.410784+0000", 2024-09-04T17:46:22.632 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 15.181410905, 2024-09-04T17:46:22.633 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.016695742, 2024-09-04T17:46:22.633 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:22.633 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:22.633 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:22.633 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:22.633 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:22.633 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:22.633 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 5009 2024-09-04T17:46:22.633 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.633 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:22.633 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.633 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:22.634 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.410784+0000", 2024-09-04T17:46:22.634 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:22.634 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.634 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.634 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:22.634 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.410784+0000", 2024-09-04T17:46:22.634 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:22.634 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.634 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.634 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:22.634 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.410786+0000", 2024-09-04T17:46:22.634 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0490000000000002e-06 2024-09-04T17:46:22.635 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.635 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.635 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:22.635 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.415833+0000", 2024-09-04T17:46:22.635 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0050473430000000001 2024-09-04T17:46:22.635 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.635 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.635 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:22.635 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.415853+0000", 2024-09-04T17:46:22.635 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9748000000000001e-05 2024-09-04T17:46:22.635 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.635 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.636 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:22.636 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.415861+0000", 2024-09-04T17:46:22.636 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.0160000000000008e-06 2024-09-04T17:46:22.636 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.636 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.636 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:22.636 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.415929+0000", 2024-09-04T17:46:22.636 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.8510999999999995e-05 2024-09-04T17:46:22.636 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.636 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.636 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:22.637 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.416039+0000", 2024-09-04T17:46:22.637 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00010938300000000001 2024-09-04T17:46:22.637 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.637 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.637 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 7", 2024-09-04T17:46:22.637 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.416401+0000", 2024-09-04T17:46:22.637 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000361858 2024-09-04T17:46:22.637 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.637 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.637 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:22.637 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.426442+0000", 2024-09-04T17:46:22.638 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.010041599999999999 2024-09-04T17:46:22.638 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.638 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.638 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:22.638 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.427370+0000", 2024-09-04T17:46:22.638 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00092744300000000002 2024-09-04T17:46:22.638 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.638 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.638 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:22.638 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.427439+0000", 2024-09-04T17:46:22.638 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.9239000000000006e-05 2024-09-04T17:46:22.638 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.639 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.639 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:22.639 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.427479+0000", 2024-09-04T17:46:22.639 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0552000000000001e-05 2024-09-04T17:46:22.639 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:22.639 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:22.639 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:22.639 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.639 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.639 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:5010 3.14 3:2de14b79:::rbd_data.1134cfbce83d.0000000000000002:head [delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:22.639 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.415884+0000", 2024-09-04T17:46:22.639 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 15.176310902999999, 2024-09-04T17:46:22.640 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.014765199999999999, 2024-09-04T17:46:22.640 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:22.640 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:22.640 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:22.640 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:22.640 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:22.640 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:22.640 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 5010 2024-09-04T17:46:22.640 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.640 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:22.640 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.641 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:22.641 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.415884+0000", 2024-09-04T17:46:22.641 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:22.641 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.641 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.641 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:22.641 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.415884+0000", 2024-09-04T17:46:22.641 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:22.641 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.641 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.641 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:22.642 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.415886+0000", 2024-09-04T17:46:22.642 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2900000000000001e-06 2024-09-04T17:46:22.642 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.642 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.642 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:22.642 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.415890+0000", 2024-09-04T17:46:22.642 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.8179999999999997e-06 2024-09-04T17:46:22.642 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.642 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.642 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:22.642 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.415897+0000", 2024-09-04T17:46:22.642 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7630000000000003e-06 2024-09-04T17:46:22.643 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.643 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.643 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:22.643 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.415900+0000", 2024-09-04T17:46:22.643 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.8500000000000004e-06 2024-09-04T17:46:22.643 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.643 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.643 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:22.643 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.420517+0000", 2024-09-04T17:46:22.643 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0046166219999999999 2024-09-04T17:46:22.643 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.644 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.644 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:22.644 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.420775+0000", 2024-09-04T17:46:22.644 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00025835399999999997 2024-09-04T17:46:22.644 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.644 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.644 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 7", 2024-09-04T17:46:22.644 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.423010+0000", 2024-09-04T17:46:22.644 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.002234691 2024-09-04T17:46:22.644 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.644 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.645 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:22.645 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.429961+0000", 2024-09-04T17:46:22.645 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0069508649999999996 2024-09-04T17:46:22.645 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.645 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.645 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:22.645 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.430511+0000", 2024-09-04T17:46:22.645 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00054961799999999998 2024-09-04T17:46:22.645 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.645 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.645 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:22.646 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.430580+0000", 2024-09-04T17:46:22.646 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.9139000000000003e-05 2024-09-04T17:46:22.646 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.646 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.646 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:22.646 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.430649+0000", 2024-09-04T17:46:22.646 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.9190000000000007e-05 2024-09-04T17:46:22.646 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:22.646 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:22.646 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:22.646 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.646 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.647 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:5384 3.14 3:283b8c0b:::rbd_data.113431586065.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 9392185~44999] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:22.647 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.714129+0000", 2024-09-04T17:46:22.647 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 14.878065594000001, 2024-09-04T17:46:22.647 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.021221803000000001, 2024-09-04T17:46:22.647 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:22.647 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:22.647 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:22.647 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:22.647 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:22.647 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:22.647 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 5384 2024-09-04T17:46:22.648 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.648 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:22.648 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.648 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:22.648 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.714129+0000", 2024-09-04T17:46:22.648 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:22.648 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.648 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.648 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:22.648 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.714129+0000", 2024-09-04T17:46:22.648 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:22.648 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.649 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.649 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:22.649 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.714131+0000", 2024-09-04T17:46:22.649 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1270000000000001e-06 2024-09-04T17:46:22.649 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.649 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.649 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:22.649 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.719337+0000", 2024-09-04T17:46:22.649 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0052062059999999997 2024-09-04T17:46:22.649 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.649 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.650 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:22.650 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.719361+0000", 2024-09-04T17:46:22.650 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4051999999999999e-05 2024-09-04T17:46:22.650 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.650 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.650 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:22.650 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.719373+0000", 2024-09-04T17:46:22.650 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1270000000000001e-05 2024-09-04T17:46:22.650 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.650 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.650 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:22.650 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.719449+0000", 2024-09-04T17:46:22.651 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.6785000000000004e-05 2024-09-04T17:46:22.651 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.651 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.651 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:22.651 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.719566+0000", 2024-09-04T17:46:22.651 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000116537 2024-09-04T17:46:22.651 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.651 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.651 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 7", 2024-09-04T17:46:22.651 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.719930+0000", 2024-09-04T17:46:22.651 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000363624 2024-09-04T17:46:22.651 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.652 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.652 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:22.652 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.730485+0000", 2024-09-04T17:46:22.652 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.010555266000000001 2024-09-04T17:46:22.652 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.652 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.652 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:22.652 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.735229+0000", 2024-09-04T17:46:22.652 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0047438369999999999 2024-09-04T17:46:22.652 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.652 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.653 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:22.653 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.735300+0000", 2024-09-04T17:46:22.653 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1408999999999993e-05 2024-09-04T17:46:22.653 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.653 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.653 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:22.653 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.735351+0000", 2024-09-04T17:46:22.653 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0689999999999997e-05 2024-09-04T17:46:22.653 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:22.653 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:22.653 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:22.654 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.654 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.654 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:5645 3.1d e31/24 3:bd9dd029:::rbd_data.113431586065.0000000000000006:head v 31'23, mlcod=30'20)", 2024-09-04T17:46:22.654 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.908329+0000", 2024-09-04T17:46:22.654 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 14.683865899000001, 2024-09-04T17:46:22.654 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.019759324000000002, 2024-09-04T17:46:22.654 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:22.654 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:22.654 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:22.654 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:22.654 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.654 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:22.655 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.908329+0000", 2024-09-04T17:46:22.655 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:22.655 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.655 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.655 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:22.655 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.908329+0000", 2024-09-04T17:46:22.655 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:22.655 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.655 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.655 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:22.655 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.908331+0000", 2024-09-04T17:46:22.655 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.3e-06 2024-09-04T17:46:22.656 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.656 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.656 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:22.656 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.912239+0000", 2024-09-04T17:46:22.656 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0039075730000000001 2024-09-04T17:46:22.656 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.656 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.656 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:22.656 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.912280+0000", 2024-09-04T17:46:22.656 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.1457e-05 2024-09-04T17:46:22.656 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.657 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.657 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:22.657 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.912298+0000", 2024-09-04T17:46:22.657 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7980000000000001e-05 2024-09-04T17:46:22.657 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.657 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.657 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:22.657 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.912432+0000", 2024-09-04T17:46:22.657 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000133516 2024-09-04T17:46:22.657 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.657 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.658 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:22.658 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.912452+0000", 2024-09-04T17:46:22.658 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0679999999999999e-05 2024-09-04T17:46:22.658 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.658 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.658 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:22.658 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.928026+0000", 2024-09-04T17:46:22.658 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.015574202000000001 2024-09-04T17:46:22.658 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.658 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.658 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:22.659 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.928088+0000", 2024-09-04T17:46:22.659 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.1616000000000001e-05 2024-09-04T17:46:22.659 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:22.659 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:22.659 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:22.659 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.659 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.659 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:6292 3.14 3:283b8c0b:::rbd_data.113431586065.0000000000000000:head [stat,truncate 2097152] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:22.659 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:08.428248+0000", 2024-09-04T17:46:22.659 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 14.163946660000001, 2024-09-04T17:46:22.659 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.017598231999999998, 2024-09-04T17:46:22.660 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:22.660 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:22.660 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:22.660 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:22.660 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:22.660 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:22.660 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 6292 2024-09-04T17:46:22.660 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.660 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:22.660 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.660 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:22.661 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.428248+0000", 2024-09-04T17:46:22.661 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:22.661 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.661 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.661 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:22.661 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.428248+0000", 2024-09-04T17:46:22.661 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:22.661 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.661 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.661 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:22.661 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.428249+0000", 2024-09-04T17:46:22.662 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.116e-06 2024-09-04T17:46:22.662 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.662 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.662 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:22.662 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.428254+0000", 2024-09-04T17:46:22.662 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0830000000000003e-06 2024-09-04T17:46:22.662 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.662 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.662 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:22.662 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.428271+0000", 2024-09-04T17:46:22.662 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6714e-05 2024-09-04T17:46:22.663 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.663 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.663 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:22.663 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.428279+0000", 2024-09-04T17:46:22.663 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.0690000000000002e-06 2024-09-04T17:46:22.663 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.663 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.663 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:22.663 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.428336+0000", 2024-09-04T17:46:22.663 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7336000000000001e-05 2024-09-04T17:46:22.663 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.664 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.664 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:22.664 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.428423+0000", 2024-09-04T17:46:22.664 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.6720999999999999e-05 2024-09-04T17:46:22.664 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.664 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.664 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 7", 2024-09-04T17:46:22.664 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.428609+0000", 2024-09-04T17:46:22.664 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00018606199999999999 2024-09-04T17:46:22.664 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.664 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.665 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:22.665 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.433297+0000", 2024-09-04T17:46:22.665 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0046874580000000002 2024-09-04T17:46:22.665 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.665 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.665 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:22.665 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.445776+0000", 2024-09-04T17:46:22.665 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.012479762 2024-09-04T17:46:22.665 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.665 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.665 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:22.666 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.445809+0000", 2024-09-04T17:46:22.666 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.2212000000000003e-05 2024-09-04T17:46:22.666 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.666 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.666 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:22.666 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.445846+0000", 2024-09-04T17:46:22.666 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7698999999999999e-05 2024-09-04T17:46:22.666 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:22.666 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:22.666 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:22.666 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.666 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.667 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:6301 3.f e32/24 3:f3ba70c3:::rbd_data.113431586065.0000000000000004:head v 32'17, mlcod=31'14)", 2024-09-04T17:46:22.667 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:08.456787+0000", 2024-09-04T17:46:22.667 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 14.135407335, 2024-09-04T17:46:22.667 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.015206687999999999, 2024-09-04T17:46:22.667 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:22.667 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:22.667 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:22.667 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:22.667 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.667 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:22.667 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.456787+0000", 2024-09-04T17:46:22.668 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:22.668 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.668 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.668 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:22.668 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.456787+0000", 2024-09-04T17:46:22.668 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:22.668 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.668 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.668 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:22.668 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.456789+0000", 2024-09-04T17:46:22.668 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.432e-06 2024-09-04T17:46:22.668 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.669 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.669 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:22.669 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.458945+0000", 2024-09-04T17:46:22.669 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0021565709999999999 2024-09-04T17:46:22.669 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.669 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.669 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:22.669 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.458976+0000", 2024-09-04T17:46:22.669 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0704000000000003e-05 2024-09-04T17:46:22.669 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.669 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.670 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:22.670 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.458995+0000", 2024-09-04T17:46:22.670 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8649999999999999e-05 2024-09-04T17:46:22.670 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.670 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.670 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:22.670 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.459094+0000", 2024-09-04T17:46:22.670 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.9302999999999994e-05 2024-09-04T17:46:22.670 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.670 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.670 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:22.670 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.459124+0000", 2024-09-04T17:46:22.671 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.9869000000000002e-05 2024-09-04T17:46:22.671 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.671 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.671 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:22.671 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.471957+0000", 2024-09-04T17:46:22.671 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.012833456 2024-09-04T17:46:22.671 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.671 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.671 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:22.671 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.471994+0000", 2024-09-04T17:46:22.671 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.6702999999999997e-05 2024-09-04T17:46:22.672 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:22.672 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:22.672 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:22.672 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.672 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.672 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:6311 3.9 3:941d6897:::rbd_data.113431586065.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:22.672 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:08.474866+0000", 2024-09-04T17:46:22.672 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 14.117328297, 2024-09-04T17:46:22.672 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.023556524999999998, 2024-09-04T17:46:22.672 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:22.672 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:22.673 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:22.673 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:22.673 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:22.673 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:22.673 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 6311 2024-09-04T17:46:22.673 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.673 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:22.673 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.673 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:22.673 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.474866+0000", 2024-09-04T17:46:22.673 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:22.673 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.674 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.674 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:22.674 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.474866+0000", 2024-09-04T17:46:22.674 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:22.674 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.674 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.674 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:22.674 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.474868+0000", 2024-09-04T17:46:22.674 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.607e-06 2024-09-04T17:46:22.674 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.674 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.675 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:22.675 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.478761+0000", 2024-09-04T17:46:22.675 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0038930269999999999 2024-09-04T17:46:22.675 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.675 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.675 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:22.675 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.478781+0000", 2024-09-04T17:46:22.675 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0372000000000001e-05 2024-09-04T17:46:22.675 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.675 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.675 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:22.675 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.478790+0000", 2024-09-04T17:46:22.676 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.6130000000000004e-06 2024-09-04T17:46:22.676 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.676 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.676 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:22.676 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.478868+0000", 2024-09-04T17:46:22.676 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.7589000000000005e-05 2024-09-04T17:46:22.676 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.676 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.676 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:22.676 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.479006+0000", 2024-09-04T17:46:22.677 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013884600000000001 2024-09-04T17:46:22.677 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.677 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.677 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 2", 2024-09-04T17:46:22.677 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.479366+0000", 2024-09-04T17:46:22.677 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00035964299999999999 2024-09-04T17:46:22.677 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.677 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.677 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:22.677 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.497050+0000", 2024-09-04T17:46:22.677 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.017683770000000001 2024-09-04T17:46:22.678 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.678 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.678 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:22.678 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.498311+0000", 2024-09-04T17:46:22.678 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.001261278 2024-09-04T17:46:22.678 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.678 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.678 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:22.678 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.498359+0000", 2024-09-04T17:46:22.678 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7675e-05 2024-09-04T17:46:22.678 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.678 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.679 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:22.679 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.498423+0000", 2024-09-04T17:46:22.679 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.4104999999999997e-05 2024-09-04T17:46:22.679 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:22.679 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:22.679 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:22.679 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.679 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.679 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:6312 3.9 3:941d6897:::rbd_data.113431586065.0000000000000008:head [truncate 1572864] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:22.679 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:08.478816+0000", 2024-09-04T17:46:22.679 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 14.113378636, 2024-09-04T17:46:22.680 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.020677345999999999, 2024-09-04T17:46:22.680 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:22.680 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:22.680 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:22.680 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:22.680 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:22.680 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:22.680 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 6312 2024-09-04T17:46:22.680 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.680 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:22.680 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.681 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:22.681 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.478816+0000", 2024-09-04T17:46:22.681 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:22.681 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.681 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.681 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:22.681 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.478816+0000", 2024-09-04T17:46:22.681 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:22.681 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.681 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.681 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:22.682 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.478818+0000", 2024-09-04T17:46:22.682 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6500000000000001e-06 2024-09-04T17:46:22.682 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.682 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.682 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:22.682 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.478822+0000", 2024-09-04T17:46:22.682 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0289999999999999e-06 2024-09-04T17:46:22.682 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.682 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.682 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:22.682 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.478831+0000", 2024-09-04T17:46:22.682 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.3850000000000008e-06 2024-09-04T17:46:22.683 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.683 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.683 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:22.683 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.478839+0000", 2024-09-04T17:46:22.683 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4749999999999996e-06 2024-09-04T17:46:22.683 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.683 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.683 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:22.683 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.485267+0000", 2024-09-04T17:46:22.683 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0064281199999999998 2024-09-04T17:46:22.683 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.683 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.684 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:22.684 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.485357+0000", 2024-09-04T17:46:22.684 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.0129000000000004e-05 2024-09-04T17:46:22.684 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.684 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.684 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 2", 2024-09-04T17:46:22.684 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.485514+0000", 2024-09-04T17:46:22.684 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00015751599999999999 2024-09-04T17:46:22.684 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.684 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.684 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:22.685 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.497433+0000", 2024-09-04T17:46:22.685 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.011918307 2024-09-04T17:46:22.685 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.685 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.685 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:22.685 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.499312+0000", 2024-09-04T17:46:22.685 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0018790040000000001 2024-09-04T17:46:22.685 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.685 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.685 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:22.685 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.499413+0000", 2024-09-04T17:46:22.686 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000101179 2024-09-04T17:46:22.686 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.686 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.686 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:22.686 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.499493+0000", 2024-09-04T17:46:22.686 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.0551999999999997e-05 2024-09-04T17:46:22.686 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:22.686 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:22.686 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:22.686 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.686 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.686 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:12722 3.1b 3:d8152af1:::rbd_data.1134ef3a5a7d.00000000000000bd:head [delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-04T17:46:22.687 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:16.421965+0000", 2024-09-04T17:46:22.687 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 6.1702297269999997, 2024-09-04T17:46:22.687 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.890002811, 2024-09-04T17:46:22.687 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:22.687 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:22.687 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:22.687 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:22.687 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:22.687 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:22.687 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 12722 2024-09-04T17:46:22.687 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.688 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:22.688 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.688 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:22.688 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421965+0000", 2024-09-04T17:46:22.688 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:22.688 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.688 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.688 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:22.688 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421965+0000", 2024-09-04T17:46:22.688 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:22.688 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.689 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.689 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:22.689 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421966+0000", 2024-09-04T17:46:22.689 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.102e-06 2024-09-04T17:46:22.689 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.689 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.689 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:22.689 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421971+0000", 2024-09-04T17:46:22.689 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9989999999999999e-06 2024-09-04T17:46:22.689 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.689 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.690 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:22.690 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421981+0000", 2024-09-04T17:46:22.690 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.7380000000000006e-06 2024-09-04T17:46:22.690 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.690 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.690 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:22.690 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421989+0000", 2024-09-04T17:46:22.690 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.1890000000000007e-06 2024-09-04T17:46:22.690 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.690 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.690 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:22.690 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.422050+0000", 2024-09-04T17:46:22.691 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.0702000000000003e-05 2024-09-04T17:46:22.691 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.691 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.691 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for peered", 2024-09-04T17:46:22.691 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.422059+0000", 2024-09-04T17:46:22.691 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5440000000000004e-06 2024-09-04T17:46:22.691 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.691 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.691 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:22.691 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.308156+0000", 2024-09-04T17:46:22.691 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.88609676199999998 2024-09-04T17:46:22.692 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.692 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.692 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:22.692 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.308398+0000", 2024-09-04T17:46:22.692 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024215400000000001 2024-09-04T17:46:22.692 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.692 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.692 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 7", 2024-09-04T17:46:22.692 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.308795+0000", 2024-09-04T17:46:22.692 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000396704 2024-09-04T17:46:22.692 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.693 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.693 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:22.693 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.310831+0000", 2024-09-04T17:46:22.693 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0020364200000000002 2024-09-04T17:46:22.693 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.693 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.693 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:22.693 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.311818+0000", 2024-09-04T17:46:22.693 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00098649799999999993 2024-09-04T17:46:22.693 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.693 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.694 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:22.694 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.311905+0000", 2024-09-04T17:46:22.694 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7070999999999994e-05 2024-09-04T17:46:22.694 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.694 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.694 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:22.694 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.311968+0000", 2024-09-04T17:46:22.694 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.2928000000000005e-05 2024-09-04T17:46:22.694 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:22.694 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:22.694 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:22.695 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.695 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.695 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:19998 3.3 3:c7add6f4:::rbd_data.113418a7bbc1.0000000000000007:d [read 344064~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-04T17:46:22.695 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:21.756774+0000", 2024-09-04T17:46:22.695 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.83542097999999998, 2024-09-04T17:46:22.695 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.015397421999999999, 2024-09-04T17:46:22.695 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:22.695 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:22.695 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:22.695 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:22.695 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:22.696 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:22.696 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 19998 2024-09-04T17:46:22.696 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.696 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:22.696 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.696 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:22.696 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:21.756774+0000", 2024-09-04T17:46:22.696 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:22.696 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.696 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.696 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:22.697 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:21.756774+0000", 2024-09-04T17:46:22.697 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:22.697 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.697 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.697 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:22.697 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:21.756775+0000", 2024-09-04T17:46:22.697 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1429999999999999e-06 2024-09-04T17:46:22.697 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.697 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.697 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:22.697 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:21.756780+0000", 2024-09-04T17:46:22.698 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.8930000000000004e-06 2024-09-04T17:46:22.698 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.698 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.698 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:22.698 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:21.756797+0000", 2024-09-04T17:46:22.698 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7268000000000001e-05 2024-09-04T17:46:22.698 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.698 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.698 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:22.698 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:21.756813+0000", 2024-09-04T17:46:22.698 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6209e-05 2024-09-04T17:46:22.698 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.699 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.699 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:22.699 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:21.771691+0000", 2024-09-04T17:46:22.699 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.014877727 2024-09-04T17:46:22.699 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.699 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.699 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:22.699 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:21.771926+0000", 2024-09-04T17:46:22.699 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000234949 2024-09-04T17:46:22.699 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.699 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.700 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:22.700 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:21.772171+0000", 2024-09-04T17:46:22.700 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024523299999999997 2024-09-04T17:46:22.700 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:22.700 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:22.700 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:22.700 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.700 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.700 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:20370 3.15 3:ab7a1a4b:::rbd_data.1134fdde05d4.0000000000000144:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-04T17:46:22.700 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:22.031420+0000", 2024-09-04T17:46:22.701 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.56077505299999997, 2024-09-04T17:46:22.701 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.018949611000000002, 2024-09-04T17:46:22.701 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:22.701 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:22.701 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:22.701 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:22.701 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:22.701 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:22.701 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 20370 2024-09-04T17:46:22.701 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.701 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:22.701 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.702 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:22.702 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.031420+0000", 2024-09-04T17:46:22.702 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:22.702 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.702 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.702 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:22.702 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.031420+0000", 2024-09-04T17:46:22.702 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:22.702 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.702 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.702 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:22.703 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.031421+0000", 2024-09-04T17:46:22.703 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5409999999999999e-06 2024-09-04T17:46:22.703 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.703 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.703 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:22.703 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.031436+0000", 2024-09-04T17:46:22.703 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4851000000000001e-05 2024-09-04T17:46:22.703 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.703 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.703 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:22.703 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.031453+0000", 2024-09-04T17:46:22.703 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6841e-05 2024-09-04T17:46:22.704 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.704 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.704 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:22.704 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.031466+0000", 2024-09-04T17:46:22.704 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2899e-05 2024-09-04T17:46:22.704 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.704 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.704 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:22.704 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.031544+0000", 2024-09-04T17:46:22.704 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.7811000000000004e-05 2024-09-04T17:46:22.704 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.705 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.705 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:22.705 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.031736+0000", 2024-09-04T17:46:22.705 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019233100000000001 2024-09-04T17:46:22.705 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.705 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.705 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 7", 2024-09-04T17:46:22.705 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.035047+0000", 2024-09-04T17:46:22.705 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0033107980000000002 2024-09-04T17:46:22.705 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.705 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.705 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:22.706 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.050001+0000", 2024-09-04T17:46:22.706 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.014953855 2024-09-04T17:46:22.706 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.706 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.706 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:22.706 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.050075+0000", 2024-09-04T17:46:22.706 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4808999999999995e-05 2024-09-04T17:46:22.706 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.706 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.706 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:22.706 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.050144+0000", 2024-09-04T17:46:22.706 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.8658000000000006e-05 2024-09-04T17:46:22.707 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.707 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.707 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:22.707 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.050369+0000", 2024-09-04T17:46:22.707 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000225217 2024-09-04T17:46:22.707 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:22.707 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:22.707 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:22.707 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.707 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.707 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:20372 3.9 3:928af84f:::rbd_data.1134fdde05d4.0000000000000146:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-04T17:46:22.708 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:22.033751+0000", 2024-09-04T17:46:22.708 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.55844406499999999, 2024-09-04T17:46:22.708 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.020054303999999998, 2024-09-04T17:46:22.708 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:22.708 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:22.708 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:22.708 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:22.708 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:22.708 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:22.708 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 20372 2024-09-04T17:46:22.708 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.709 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:22.709 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.709 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:22.709 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.033751+0000", 2024-09-04T17:46:22.709 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:22.709 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.709 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.709 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:22.709 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.033751+0000", 2024-09-04T17:46:22.709 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:22.709 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.710 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.710 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:22.710 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.033753+0000", 2024-09-04T17:46:22.710 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.898e-06 2024-09-04T17:46:22.710 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.710 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.710 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:22.710 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.033767+0000", 2024-09-04T17:46:22.710 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3909e-05 2024-09-04T17:46:22.710 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.710 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.711 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:22.711 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.033793+0000", 2024-09-04T17:46:22.711 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5210000000000001e-05 2024-09-04T17:46:22.711 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.711 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.711 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:22.711 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.033807+0000", 2024-09-04T17:46:22.711 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3995000000000001e-05 2024-09-04T17:46:22.711 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.711 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.711 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:22.711 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.033926+0000", 2024-09-04T17:46:22.712 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000119096 2024-09-04T17:46:22.712 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.712 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.712 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:22.712 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.034212+0000", 2024-09-04T17:46:22.712 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00028630000000000002 2024-09-04T17:46:22.712 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.712 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.712 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 2", 2024-09-04T17:46:22.712 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.038305+0000", 2024-09-04T17:46:22.712 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0040928910000000004 2024-09-04T17:46:22.713 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.713 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.713 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:22.713 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.053691+0000", 2024-09-04T17:46:22.713 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.015385853 2024-09-04T17:46:22.713 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.713 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.713 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:22.713 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.053708+0000", 2024-09-04T17:46:22.713 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6937999999999999e-05 2024-09-04T17:46:22.713 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.714 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.714 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:22.714 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.053758+0000", 2024-09-04T17:46:22.714 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0003999999999998e-05 2024-09-04T17:46:22.714 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.714 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.714 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:22.714 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.053805+0000", 2024-09-04T17:46:22.714 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7209999999999997e-05 2024-09-04T17:46:22.714 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:22.714 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:22.714 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:22.715 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.715 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.715 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:24189 3.1a 3:58ba337c:::rbd_data.1134fdde05d4.0000000000000090:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-04T17:46:22.715 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:22.366956+0000", 2024-09-04T17:46:22.715 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.225238204, 2024-09-04T17:46:22.715 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.015895158999999999, 2024-09-04T17:46:22.715 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:22.715 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:22.715 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:22.715 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:22.715 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:22.716 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:22.716 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 24189 2024-09-04T17:46:22.716 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.716 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:22.716 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.716 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:22.716 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.366956+0000", 2024-09-04T17:46:22.716 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:22.716 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.716 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.716 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:22.717 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.366956+0000", 2024-09-04T17:46:22.717 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:22.717 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.717 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.717 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:22.717 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.366959+0000", 2024-09-04T17:46:22.717 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2919999999999998e-06 2024-09-04T17:46:22.717 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.717 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.717 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:22.717 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.366967+0000", 2024-09-04T17:46:22.717 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.2369999999999992e-06 2024-09-04T17:46:22.718 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.718 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.718 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:22.718 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.366984+0000", 2024-09-04T17:46:22.718 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6793999999999999e-05 2024-09-04T17:46:22.718 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.718 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.718 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:22.718 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.366992+0000", 2024-09-04T17:46:22.718 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.5469999999999997e-06 2024-09-04T17:46:22.718 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.719 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.719 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:22.719 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.367068+0000", 2024-09-04T17:46:22.719 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.5319000000000002e-05 2024-09-04T17:46:22.719 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.719 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.719 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:22.719 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.367229+0000", 2024-09-04T17:46:22.719 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000160903 2024-09-04T17:46:22.719 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.719 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.719 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 1", 2024-09-04T17:46:22.720 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.368287+0000", 2024-09-04T17:46:22.720 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0010584100000000001 2024-09-04T17:46:22.720 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.720 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.720 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:22.720 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.382769+0000", 2024-09-04T17:46:22.720 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.014481677 2024-09-04T17:46:22.720 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.720 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.720 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:22.720 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.382780+0000", 2024-09-04T17:46:22.721 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.113e-05 2024-09-04T17:46:22.721 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.721 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.721 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:22.721 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.382815+0000", 2024-09-04T17:46:22.721 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.5397e-05 2024-09-04T17:46:22.721 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:22.721 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:22.721 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:22.721 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.382852+0000", 2024-09-04T17:46:22.721 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.6452999999999998e-05 2024-09-04T17:46:22.722 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:22.722 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:22.722 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:22.722 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:22.722 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:22.722 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:22.724 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-04T17:46:22.724 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:22.723+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:22.742 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-04T17:46:22.825 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:22.825+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:22.895 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:22.895 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:46:22.895 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:46:22.895 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:22.895 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:46:22.896 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:46:22.908 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-04T17:46:22.912 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:46:22.912 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:46:22.928 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-04T17:46:22.929 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-04T17:46:22.929 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-04T17:46:22.947 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:46:22.947 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:46:22.955 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:22.955+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:22.965 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:46:22.965 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:46:22.983 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-04T17:46:22.983 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-04T17:46:23.001 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:46:23.002 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:46:23.005 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-04T17:46:23.006 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-04T17:46:23.027 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:23.023+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:23.028 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:23.028 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:23.043 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-04T17:46:23.132 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:23.131+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:23.205 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:23.205 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:23.219 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-04T17:46:23.263 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:23.263+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:23.328 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:23.327+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:23.389 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:23.389 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:23.404 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-04T17:46:23.448 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:23.447+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:23.531 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:23.533+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:23.572 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:23.572 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:23.586 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-04T17:46:23.632 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:23.633+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:23.690 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #10, image image_client.0-clone10 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent11 2024-09-04T17:46:23.716 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:23.716 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:23.718 INFO:teuthology.orchestra.run.smithi092.stdout:1149 write 0x8d0f370 thru 0x8d17143 (0x7dd4 bytes) 2024-09-04T17:46:23.731 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-04T17:46:23.732 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:23.731+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:23.737 INFO:teuthology.orchestra.run.smithi092.stdout:1150 write 0x45f7d85 thru 0x4606984 (0xec00 bytes) 2024-09-04T17:46:23.786 INFO:teuthology.orchestra.run.smithi092.stdout:1151 trunc from 0xd2602cd to 0xc172c68 2024-09-04T17:46:23.804 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:23.806 INFO:teuthology.orchestra.run.smithi092.stdout:1153 trunc from 0xc172c68 to 0xd4cd72a 2024-09-04T17:46:23.831 INFO:teuthology.orchestra.run.smithi092.stdout:1154 writesame 0xd918f08 thru 0xd926962 data_size 0x3(0xda5b bytes) 2024-09-04T17:46:23.833 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:46:23.831+0000 7f505d642640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26446) UID: 0 2024-09-04T17:46:23.833 INFO:teuthology.orchestra.run.smithi092.stdout:1156 write 0x857e3a2 thru 0x85884e1 (0xa140 bytes) 2024-09-04T17:46:23.846 INFO:teuthology.orchestra.run.smithi092.stdout:1158 write 0xa16a287 thru 0xa170e6b (0x6be5 bytes) 2024-09-04T17:46:23.859 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:23.860 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:23.860 INFO:teuthology.orchestra.run.smithi092.stdout:1161 read 0xd584f52 thru 0xd58611a (0x11c9 bytes) 2024-09-04T17:46:23.860 INFO:teuthology.orchestra.run.smithi092.stdout:1164 write 0x75d9631 thru 0x75dbfc9 (0x2999 bytes) 2024-09-04T17:46:23.861 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:23.862 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:23.873 INFO:teuthology.orchestra.run.smithi092.stdout:1165 write 0x4a25a8d thru 0x4a304b2 (0xaa26 bytes) 2024-09-04T17:46:23.880 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-04T17:46:23.907 INFO:teuthology.orchestra.run.smithi092.stdout:1166 writesame 0x3fe794b thru 0x3ff2538 data_size 0x3b(0xabee bytes) 2024-09-04T17:46:23.913 INFO:teuthology.orchestra.run.smithi092.stdout:1167 trunc from 0xd926963 to 0xdc851a7 2024-09-04T17:46:23.925 INFO:teuthology.orchestra.run.smithi092.stdout:1168 read 0xca46b2c thru 0xca4ee1c (0x82f1 bytes) 2024-09-04T17:46:23.926 INFO:teuthology.orchestra.run.smithi092.stdout:1169 read 0xc90916 thru 0xc91651 (0xd3c bytes) 2024-09-04T17:46:23.928 INFO:teuthology.orchestra.run.smithi092.stdout:1170 read 0x5f3667d thru 0x5f3ed39 (0x86bd bytes) 2024-09-04T17:46:23.928 INFO:teuthology.orchestra.run.smithi092.stdout:1171 write 0x9eacc09 thru 0x9eb44d1 (0x78c9 bytes) 2024-09-04T17:46:23.934 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:23.933+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:23.941 INFO:teuthology.orchestra.run.smithi092.stdout:1172 read 0x3f57227 thru 0x3f61a94 (0xa86e bytes) 2024-09-04T17:46:23.942 INFO:teuthology.orchestra.run.smithi092.stdout:1174 write 0x56fbf92 thru 0x57017bf (0x582e bytes) 2024-09-04T17:46:23.979 INFO:teuthology.orchestra.run.smithi092.stdout:1175 read 0x69f5fa0 thru 0x69f9478 (0x34d9 bytes) 2024-09-04T17:46:23.980 INFO:teuthology.orchestra.run.smithi092.stdout:1177 write 0x41416ae thru 0x414df99 (0xc8ec bytes) 2024-09-04T17:46:23.983 INFO:teuthology.orchestra.run.smithi092.stdout:1179 writesame 0x77c5b60 thru 0x77cf59d data_size 0x3(0x9a3e bytes) 2024-09-04T17:46:23.989 INFO:teuthology.orchestra.run.smithi092.stdout:1180 read 0xd3ccf76 thru 0xd3d9a9d (0xcb28 bytes) 2024-09-04T17:46:23.989 INFO:teuthology.orchestra.run.smithi092.stdout:1181 writesame 0x83af29a thru 0x83bbba8 data_size 0x15(0xc90f bytes) 2024-09-04T17:46:23.992 INFO:teuthology.orchestra.run.smithi092.stdout:1182 write 0xaff775f thru 0xaff96a1 (0x1f43 bytes) 2024-09-04T17:46:23.994 INFO:teuthology.orchestra.run.smithi092.stdout:1183 write 0x7304fa2 thru 0x73064b1 (0x1510 bytes) 2024-09-04T17:46:23.997 INFO:teuthology.orchestra.run.smithi092.stdout:1184 write 0x5042f8d thru 0x504b509 (0x857d bytes) 2024-09-04T17:46:24.008 INFO:teuthology.orchestra.run.smithi092.stdout:1185 write 0xe496a6b thru 0xe49b550 (0x4ae6 bytes) 2024-09-04T17:46:24.010 INFO:teuthology.orchestra.run.smithi092.stdout:1186 punch from 0x92fed5b to 0x92ff2c1, (0x566 bytes) 2024-09-04T17:46:24.010 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:24.010 INFO:teuthology.orchestra.run.smithi092.stdout:1188 read 0x9c903f0 thru 0x9c95778 (0x5389 bytes) 2024-09-04T17:46:24.011 INFO:teuthology.orchestra.run.smithi092.stdout:1189 read 0xe0bcfd3 thru 0xe0c4096 (0x70c4 bytes) 2024-09-04T17:46:24.011 INFO:teuthology.orchestra.run.smithi092.stdout:1190 write 0x6936039 thru 0x693aa37 (0x49ff bytes) 2024-09-04T17:46:24.015 INFO:teuthology.orchestra.run.smithi092.stdout:1191 punch from 0x604d787 to 0x605b566, (0xdddf bytes) 2024-09-04T17:46:24.015 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:24.015 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:24.015 INFO:teuthology.orchestra.run.smithi092.stdout:1194 write 0x6c83965 thru 0x6c8b778 (0x7e14 bytes) 2024-09-04T17:46:24.016 INFO:teuthology.orchestra.run.smithi092.stdout:1195 write 0x5cf2d8e thru 0x5cf5b04 (0x2d77 bytes) 2024-09-04T17:46:24.020 INFO:teuthology.orchestra.run.smithi092.stdout:1196 read 0x49717b9 thru 0x49756e2 (0x3f2a bytes) 2024-09-04T17:46:24.020 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:24.021 INFO:teuthology.orchestra.run.smithi092.stdout:1198 write 0x2c7ec9e thru 0x2c8d924 (0xec87 bytes) 2024-09-04T17:46:24.024 INFO:teuthology.orchestra.run.smithi092.stdout:1199 punch from 0x356f688 to 0x3576511, (0x6e89 bytes) 2024-09-04T17:46:24.024 INFO:teuthology.orchestra.run.smithi092.stdout:1200 writesame 0x3998ed5 thru 0x39a28de data_size 0x1(0x9a0a bytes) 2024-09-04T17:46:24.035 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:24.031+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:24.056 INFO:teuthology.orchestra.run.smithi092.stdout:1201 read 0x71c636e thru 0x71c88d3 (0x2566 bytes) 2024-09-04T17:46:24.057 INFO:teuthology.orchestra.run.smithi092.stdout:1202 punch from 0x6a7f5ed to 0x6a886e1, (0x90f4 bytes) 2024-09-04T17:46:24.058 INFO:teuthology.orchestra.run.smithi092.stdout:1203 flatten 2024-09-04T17:46:24.068 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:24.068 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:24.085 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-04T17:46:24.094 INFO:teuthology.orchestra.run.smithi092.stdout:1204 write 0x87c683c thru 0x87cb8a5 (0x506a bytes) 2024-09-04T17:46:24.097 INFO:teuthology.orchestra.run.smithi092.stdout:1205 writesame 0x26c8199 thru 0x26cacc3 data_size 0x1(0x2b2b bytes) 2024-09-04T17:46:24.113 INFO:teuthology.orchestra.run.smithi092.stdout:1207 write 0xd3d91f7 thru 0xd3e5952 (0xc75c bytes) 2024-09-04T17:46:24.119 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:24.119 INFO:teuthology.orchestra.run.smithi092.stdout:1209 read 0xc7255d6 thru 0xc72ec69 (0x9694 bytes) 2024-09-04T17:46:24.119 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:24.120 INFO:teuthology.orchestra.run.smithi092.stdout:1213 writesame 0xe1c45b5 thru 0xe1cc262 data_size 0x1(0x7cae bytes) 2024-09-04T17:46:24.165 INFO:teuthology.orchestra.run.smithi092.stdout:1214 punch from 0x8b38b29 to 0x8b3cee1, (0x43b8 bytes) 2024-09-04T17:46:24.171 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:24.171+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:24.178 INFO:teuthology.orchestra.run.smithi092.stdout:1215 trunc from 0xe49b551 to 0x9aefe2f 2024-09-04T17:46:24.201 INFO:teuthology.orchestra.run.smithi092.stdout:1217 punch from 0x1daee67 to 0x1db8697, (0x9830 bytes) 2024-09-04T17:46:24.201 INFO:teuthology.orchestra.run.smithi092.stdout:1218 write 0x1c72f95 thru 0x1c7e012 (0xb07e bytes) 2024-09-04T17:46:24.204 INFO:teuthology.orchestra.run.smithi092.stdout:1219 writesame 0x4782962 thru 0x4789a4a data_size 0x7b(0x70e9 bytes) 2024-09-04T17:46:24.208 INFO:teuthology.orchestra.run.smithi092.stdout:1220 read 0x57b054f thru 0x57ba919 (0xa3cb bytes) 2024-09-04T17:46:24.229 INFO:teuthology.orchestra.run.smithi092.stdout:1221 write 0xc776ae4 thru 0xc780382 (0x989f bytes) 2024-09-04T17:46:24.237 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:24.235+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:24.246 INFO:teuthology.orchestra.run.smithi092.stdout:1222 trunc from 0xc780383 to 0x80715dc 2024-09-04T17:46:24.253 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:24.253 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:24.269 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-04T17:46:24.273 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:24.273 INFO:teuthology.orchestra.run.smithi092.stdout:1224 write 0x12d4015 thru 0x12e1e32 (0xde1e bytes) 2024-09-04T17:46:24.285 INFO:teuthology.orchestra.run.smithi092.stdout:1227 trunc from 0x80715dc to 0xbd29009 2024-09-04T17:46:24.295 INFO:teuthology.orchestra.run.smithi092.stdout:1228 writesame 0xcdee94 thru 0xce8a99 data_size 0x3f(0x9c06 bytes) 2024-09-04T17:46:24.297 INFO:teuthology.orchestra.run.smithi092.stdout:1230 write 0x1abc69a thru 0x1acb16b (0xead2 bytes) 2024-09-04T17:46:24.300 INFO:teuthology.orchestra.run.smithi092.stdout:1231 punch from 0x5c8fc23 to 0x5c9af52, (0xb32f bytes) 2024-09-04T17:46:24.302 INFO:teuthology.orchestra.run.smithi092.stdout:1232 trunc from 0xbd29009 to 0xcc57531 2024-09-04T17:46:24.312 INFO:teuthology.orchestra.run.smithi092.stdout:1233 write 0xca70036 thru 0xca7d1f1 (0xd1bc bytes) 2024-09-04T17:46:24.317 INFO:teuthology.orchestra.run.smithi092.stdout:1235 trunc from 0xcc57531 to 0xd185ca7 2024-09-04T17:46:24.325 INFO:teuthology.orchestra.run.smithi092.stdout:1236 read 0x553b060 thru 0x5548830 (0xd7d1 bytes) 2024-09-04T17:46:24.329 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-04T17:46:24.341 INFO:teuthology.orchestra.run.smithi092.stdout:1237 trunc from 0xd185ca7 to 0x67ef034 2024-09-04T17:46:24.341 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:24.341+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:24.368 INFO:teuthology.orchestra.run.smithi092.stdout:1238 trunc from 0x67ef034 to 0x321000b 2024-09-04T17:46:24.400 INFO:teuthology.orchestra.run.smithi092.stdout:1239 punch from 0x16ce093 to 0x16cffbd, (0x1f2a bytes) 2024-09-04T17:46:24.405 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:24.405 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:24.422 INFO:teuthology.orchestra.run.smithi092.stdout:1240 trunc from 0x321000b to 0xd1f2230 2024-09-04T17:46:24.427 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-04T17:46:24.433 INFO:teuthology.orchestra.run.smithi092.stdout:1243 read 0x4e8cdd8 thru 0x4e93edf (0x7108 bytes) 2024-09-04T17:46:24.434 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:24.438 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:46:24.435+0000 7f505d642640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26446) UID: 0 2024-09-04T17:46:24.447 INFO:teuthology.orchestra.run.smithi092.stdout:1245 writesame 0xe3602e0 thru 0xe36df9e data_size 0xbd(0xdcbf bytes) 2024-09-04T17:46:24.447 INFO:teuthology.orchestra.run.smithi092.stdout:1246 punch from 0xd7486aa to 0xd751e71, (0x97c7 bytes) 2024-09-04T17:46:24.447 INFO:teuthology.orchestra.run.smithi092.stdout:1247 writesame 0xbfd3554 thru 0xbfe151b data_size 0x2c(0xdfc8 bytes) 2024-09-04T17:46:24.451 INFO:teuthology.orchestra.run.smithi092.stdout:1248 punch from 0x741154d to 0x7418365, (0x6e18 bytes) 2024-09-04T17:46:24.464 INFO:teuthology.orchestra.run.smithi092.stdout:1249 trunc from 0xe36df9f to 0x5fcafe0 2024-09-04T17:46:24.485 INFO:teuthology.orchestra.run.smithi092.stdout:1250 read 0x3877f18 thru 0x387a30e (0x23f7 bytes) 2024-09-04T17:46:24.485 INFO:teuthology.orchestra.run.smithi092.stdout:1251 punch from 0x2b2e9a4 to 0x2b38b56, (0xa1b2 bytes) 2024-09-04T17:46:24.509 INFO:teuthology.orchestra.run.smithi092.stdout:1253 writesame 0xdc87116 thru 0xdc909a1 data_size 0x1a(0x988c bytes) 2024-09-04T17:46:24.509 INFO:teuthology.orchestra.run.smithi092.stdout:1257 writesame 0x9ed8abd thru 0x9ee69d3 data_size 0x1(0xdf17 bytes) 2024-09-04T17:46:24.539 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:46:24.535+0000 7f505d642640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26446) UID: 0 2024-09-04T17:46:24.548 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:24.548 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:24.549 INFO:teuthology.orchestra.run.smithi092.stdout:1262 write 0x974594e thru 0x97460d5 (0x788 bytes) 2024-09-04T17:46:24.553 INFO:teuthology.orchestra.run.smithi092.stdout:1263 read 0x4821fab thru 0x4823c81 (0x1cd7 bytes) 2024-09-04T17:46:24.558 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:24.558 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:24.565 INFO:teuthology.orchestra.run.smithi092.stdout:1264 write 0xe10cb66 thru 0xe10fc07 (0x30a2 bytes) 2024-09-04T17:46:24.572 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:24.573 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-04T17:46:24.591 INFO:teuthology.orchestra.run.smithi092.stdout:1267 trunc from 0xe10fc08 to 0xee7c84 2024-09-04T17:46:24.612 INFO:teuthology.orchestra.run.smithi092.stdout:1268 read 0x3e9436 thru 0x3f3b97 (0xa762 bytes) 2024-09-04T17:46:24.632 INFO:teuthology.orchestra.run.smithi092.stdout:1270 trunc from 0xee7c84 to 0xe4fe6eb 2024-09-04T17:46:24.640 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:24.639+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:24.653 INFO:teuthology.orchestra.run.smithi092.stdout:1271 writesame 0xe57914a thru 0xe5883d7 data_size 0x83(0xf28e bytes) 2024-09-04T17:46:24.653 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:24.653 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:24.653 INFO:teuthology.orchestra.run.smithi092.stdout:1274 read 0x61ec978 thru 0x61f6be1 (0xa26a bytes) 2024-09-04T17:46:24.653 INFO:teuthology.orchestra.run.smithi092.stdout:1275 punch from 0x4a195c3 to 0x4a28972, (0xf3af bytes) 2024-09-04T17:46:24.653 INFO:teuthology.orchestra.run.smithi092.stdout:1276 punch from 0x2ae752f to 0x2af2c96, (0xb767 bytes) 2024-09-04T17:46:24.653 INFO:teuthology.orchestra.run.smithi092.stdout:1277 punch from 0x88207f0 to 0x8827c5d, (0x746d bytes) 2024-09-04T17:46:24.654 INFO:teuthology.orchestra.run.smithi092.stdout:1278 read 0x5f16ef thru 0x5fe5dc (0xceee bytes) 2024-09-04T17:46:24.654 INFO:teuthology.orchestra.run.smithi092.stdout:1279 write 0x82d030b thru 0x82db443 (0xb139 bytes) 2024-09-04T17:46:24.657 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:24.667 INFO:teuthology.orchestra.run.smithi092.stdout:1282 write 0xe5d7ea3 thru 0xe5e4869 (0xc9c7 bytes) 2024-09-04T17:46:24.672 INFO:teuthology.orchestra.run.smithi092.stdout:1283 read 0x4618dc2 thru 0x4625ff0 (0xd22f bytes) 2024-09-04T17:46:24.673 INFO:teuthology.orchestra.run.smithi092.stdout:1285 write 0x5c170f2 thru 0x5c1882c (0x173b bytes) 2024-09-04T17:46:24.676 INFO:teuthology.orchestra.run.smithi092.stdout:1286 writesame 0x8b03ccc thru 0x8b08134 data_size 0x1(0x4469 bytes) 2024-09-04T17:46:24.703 INFO:teuthology.orchestra.run.smithi092.stdout:1287 writesame 0x34021f4 thru 0x340d2f2 data_size 0x1(0xb0ff bytes) 2024-09-04T17:46:24.712 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:24.712 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:24.725 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-04T17:46:24.741 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:46:24.739+0000 7f505d642640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26446) UID: 0 2024-09-04T17:46:24.760 INFO:teuthology.orchestra.run.smithi092.stdout:1288 read 0xd573d12 thru 0xd579f3c (0x622b bytes) 2024-09-04T17:46:24.760 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:24.760 INFO:teuthology.orchestra.run.smithi092.stdout:1291 read 0x821ff09 thru 0x82279fa (0x7af2 bytes) 2024-09-04T17:46:24.761 INFO:teuthology.orchestra.run.smithi092.stdout:1292 write 0x2ac03da thru 0x2ac0ee9 (0xb10 bytes) 2024-09-04T17:46:24.764 INFO:teuthology.orchestra.run.smithi092.stdout:1293 write 0x16d4fbc thru 0x16e4dec (0xfe31 bytes) 2024-09-04T17:46:24.767 INFO:teuthology.orchestra.run.smithi092.stdout:1295 writesame 0x8f04cb1 thru 0x8f11417 data_size 0x1(0xc767 bytes) 2024-09-04T17:46:24.789 INFO:teuthology.orchestra.run.smithi092.stdout:1298 write 0x2cb43f thru 0x2cd377 (0x1f39 bytes) 2024-09-04T17:46:24.791 INFO:teuthology.orchestra.run.smithi092.stdout:1300 punch from 0x2aa3ce8 to 0x2ab3a3e, (0xfd56 bytes) 2024-09-04T17:46:24.791 INFO:teuthology.orchestra.run.smithi092.stdout:1301 read 0xe3993ff thru 0xe3a046f (0x7071 bytes) 2024-09-04T17:46:24.791 INFO:teuthology.orchestra.run.smithi092.stdout:1303 read 0x7ac5159 thru 0x7ace941 (0x97e9 bytes) 2024-09-04T17:46:24.791 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:24.791 INFO:teuthology.orchestra.run.smithi092.stdout:1305 writesame 0x18a0fab thru 0x18af4c2 data_size 0x4(0xe518 bytes) 2024-09-04T17:46:24.811 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:24.812 INFO:teuthology.orchestra.run.smithi092.stdout:1307 write 0xe33ee1f thru 0xe34df46 (0xf128 bytes) 2024-09-04T17:46:24.816 INFO:teuthology.orchestra.run.smithi092.stdout:1310 write 0x62649ce thru 0x626b305 (0x6938 bytes) 2024-09-04T17:46:24.818 INFO:teuthology.orchestra.run.smithi092.stdout:1311 writesame 0xde4972c thru 0xde519e1 data_size 0x8f(0x82b6 bytes) 2024-09-04T17:46:24.823 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:24.823 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:24.824 INFO:teuthology.orchestra.run.smithi092.stdout:1314 write 0x8d7a826 thru 0x8d7f1bc (0x4997 bytes) 2024-09-04T17:46:24.829 INFO:teuthology.orchestra.run.smithi092.stdout:1316 writesame 0x6015470 thru 0x6024bec data_size 0x31(0xf77d bytes) 2024-09-04T17:46:24.830 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:24.830 INFO:teuthology.orchestra.run.smithi092.stdout:1318 writesame 0x963b0f3 thru 0x964571b data_size 0xb(0xa629 bytes) 2024-09-04T17:46:24.833 INFO:teuthology.orchestra.run.smithi092.stdout:1320 punch from 0x60597dd to 0x605cf4a, (0x376d bytes) 2024-09-04T17:46:24.834 INFO:teuthology.orchestra.run.smithi092.stdout:1321 write 0x3fb94dc thru 0x3fbcd7a (0x389f bytes) 2024-09-04T17:46:24.836 INFO:teuthology.orchestra.run.smithi092.stdout:1322 read 0x97ced6d thru 0x97d9e72 (0xb106 bytes) 2024-09-04T17:46:24.837 INFO:teuthology.orchestra.run.smithi092.stdout:1323 write 0x8be781e thru 0x8beb89e (0x4081 bytes) 2024-09-04T17:46:24.842 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:24.841+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:24.858 INFO:teuthology.orchestra.run.smithi092.stdout:1324 trunc from 0xe5e486a to 0x1d53530 2024-09-04T17:46:24.891 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:24.892 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:24.894 INFO:teuthology.orchestra.run.smithi092.stdout:1325 writesame 0x4ef20cc thru 0x4efaa60 data_size 0x1(0x8995 bytes) 2024-09-04T17:46:24.896 INFO:teuthology.orchestra.run.smithi092.stdout:1326 read 0x79ce1c thru 0x79f38e (0x2573 bytes) 2024-09-04T17:46:24.909 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-04T17:46:24.916 INFO:teuthology.orchestra.run.smithi092.stdout:1331 write 0x8f343f5 thru 0x8f4072b (0xc337 bytes) 2024-09-04T17:46:24.967 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:24.969+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:25.007 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-04T17:46:25.040 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:25.040 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:25.043 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:25.045+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:25.050 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-04T17:46:25.054 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-04T17:46:25.144 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:25.145+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:25.148 INFO:teuthology.orchestra.run.smithi092.stdout:1333 writesame 0xccec44d thru 0xccfaab7 data_size 0x1(0xe66b bytes) 2024-09-04T17:46:25.148 INFO:teuthology.orchestra.run.smithi092.stdout:1334 read 0x5b49d0d thru 0x5b57776 (0xda6a bytes) 2024-09-04T17:46:25.148 INFO:teuthology.orchestra.run.smithi092.stdout:1335 punch from 0x51b7a76 to 0x51bdede, (0x6468 bytes) 2024-09-04T17:46:25.148 INFO:teuthology.orchestra.run.smithi092.stdout:1336 punch from 0x4090764 to 0x409427c, (0x3b18 bytes) 2024-09-04T17:46:25.149 INFO:teuthology.orchestra.run.smithi092.stdout:1337 punch from 0x15d57d2 to 0x15df0e1, (0x990f bytes) 2024-09-04T17:46:25.149 INFO:teuthology.orchestra.run.smithi092.stdout:1338 read 0xc559143 thru 0xc56189f (0x875d bytes) 2024-09-04T17:46:25.149 INFO:teuthology.orchestra.run.smithi092.stdout:1340 trunc from 0xccfaab8 to 0xce15688 2024-09-04T17:46:25.153 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:46:25.153 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:46:25.166 INFO:teuthology.orchestra.run.smithi092.stderr:no valid command found; 10 closest matches: 2024-09-04T17:46:25.166 INFO:teuthology.orchestra.run.smithi092.stderr:0 2024-09-04T17:46:25.166 INFO:teuthology.orchestra.run.smithi092.stderr:1 2024-09-04T17:46:25.166 INFO:teuthology.orchestra.run.smithi092.stderr:2 2024-09-04T17:46:25.166 INFO:teuthology.orchestra.run.smithi092.stderr:abort 2024-09-04T17:46:25.166 INFO:teuthology.orchestra.run.smithi092.stderr:assert 2024-09-04T17:46:25.166 INFO:teuthology.orchestra.run.smithi092.stderr:config diff 2024-09-04T17:46:25.166 INFO:teuthology.orchestra.run.smithi092.stderr:config diff get 2024-09-04T17:46:25.166 INFO:teuthology.orchestra.run.smithi092.stderr:config get 2024-09-04T17:46:25.167 INFO:teuthology.orchestra.run.smithi092.stderr:config help [] 2024-09-04T17:46:25.167 INFO:teuthology.orchestra.run.smithi092.stderr:config set ... 2024-09-04T17:46:25.167 INFO:teuthology.orchestra.run.smithi092.stderr:admin_socket: invalid command 2024-09-04T17:46:25.168 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-04T17:46:25.168 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-3, ['dump_ops_in_flight'] 2024-09-04T17:46:25.169 INFO:teuthology.orchestra.run.smithi092.stdout:1342 write 0xdad6ab4 thru 0xdadef74 (0x84c1 bytes) 2024-09-04T17:46:25.172 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:46:25.173 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:46:25.175 INFO:teuthology.orchestra.run.smithi092.stdout:1344 read 0x949ada4 thru 0x94a728a (0xc4e7 bytes) 2024-09-04T17:46:25.175 INFO:teuthology.orchestra.run.smithi092.stdout:1347 read 0x1c2e150 thru 0x1c30bdd (0x2a8e bytes) 2024-09-04T17:46:25.176 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:25.176 INFO:teuthology.orchestra.run.smithi092.stdout:1350 write 0x5b40d5 thru 0x5be9b1 (0xa8dd bytes) 2024-09-04T17:46:25.177 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:25.177 INFO:teuthology.orchestra.run.smithi092.stdout:1352 read 0xb6ce80c thru 0xb6d3c38 (0x542d bytes) 2024-09-04T17:46:25.192 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-04T17:46:25.192 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-04T17:46:25.192 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-04T17:46:25.195 INFO:teuthology.orchestra.run.smithi092.stdout:1353 trunc from 0xdadef75 to 0xcc65f2 2024-09-04T17:46:25.216 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:46:25.216 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:46:25.232 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:25.233 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:46:25.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:46:25.233 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:46:25.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.233 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15462 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 835584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:25.233 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.220338+0000", 2024-09-04T17:46:25.233 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 7.0133474659999999, 2024-09-04T17:46:25.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051751278999999997, 2024-09-04T17:46:25.233 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:25.233 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:25.233 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:25.233 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:25.234 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:25.234 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:25.234 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15462 2024-09-04T17:46:25.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.234 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:25.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:25.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.220338+0000", 2024-09-04T17:46:25.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:25.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.220338+0000", 2024-09-04T17:46:25.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:25.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.220600+0000", 2024-09-04T17:46:25.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026198900000000002 2024-09-04T17:46:25.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:25.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.220617+0000", 2024-09-04T17:46:25.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6956000000000002e-05 2024-09-04T17:46:25.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:25.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.220627+0000", 2024-09-04T17:46:25.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0492999999999999e-05 2024-09-04T17:46:25.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:25.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.220635+0000", 2024-09-04T17:46:25.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0059999999999992e-06 2024-09-04T17:46:25.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:25.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.271853+0000", 2024-09-04T17:46:25.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051218020000000003 2024-09-04T17:46:25.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:25.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.271941+0000", 2024-09-04T17:46:25.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7802999999999999e-05 2024-09-04T17:46:25.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:25.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.272089+0000", 2024-09-04T17:46:25.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014801199999999999 2024-09-04T17:46:25.238 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.238 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:25.238 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.238 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15500 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 294912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:25.238 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.228278+0000", 2024-09-04T17:46:25.238 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 7.0054074689999997, 2024-09-04T17:46:25.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051770927000000001, 2024-09-04T17:46:25.238 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:25.238 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:25.238 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:25.238 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:25.239 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:25.239 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:25.239 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15500 2024-09-04T17:46:25.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.239 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:25.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.239 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:25.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.228278+0000", 2024-09-04T17:46:25.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.239 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:25.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.228278+0000", 2024-09-04T17:46:25.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:25.240 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234002+0000", 2024-09-04T17:46:25.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005724392 2024-09-04T17:46:25.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:25.240 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234009+0000", 2024-09-04T17:46:25.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1300000000000003e-06 2024-09-04T17:46:25.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:25.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234024+0000", 2024-09-04T17:46:25.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4063000000000001e-05 2024-09-04T17:46:25.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:25.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234035+0000", 2024-09-04T17:46:25.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1124000000000001e-05 2024-09-04T17:46:25.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:25.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.279808+0000", 2024-09-04T17:46:25.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045772992999999998 2024-09-04T17:46:25.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:25.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.279894+0000", 2024-09-04T17:46:25.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6174999999999994e-05 2024-09-04T17:46:25.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:25.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.280049+0000", 2024-09-04T17:46:25.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015505 2024-09-04T17:46:25.242 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.242 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:25.242 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.243 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15524 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 1540096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:25.243 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.234831+0000", 2024-09-04T17:46:25.243 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.9988545929999999, 2024-09-04T17:46:25.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051893631000000003, 2024-09-04T17:46:25.243 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:25.243 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:25.243 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:25.243 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:25.243 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:25.243 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:25.243 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15524 2024-09-04T17:46:25.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.243 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:25.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.244 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:25.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234831+0000", 2024-09-04T17:46:25.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.244 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:25.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234831+0000", 2024-09-04T17:46:25.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.244 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:25.245 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234831+0000", 2024-09-04T17:46:25.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2699999999999999e-07 2024-09-04T17:46:25.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.245 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:25.245 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234833+0000", 2024-09-04T17:46:25.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9810000000000002e-06 2024-09-04T17:46:25.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.245 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:25.245 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234839+0000", 2024-09-04T17:46:25.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1519999999999997e-06 2024-09-04T17:46:25.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:25.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234845+0000", 2024-09-04T17:46:25.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1109999999999999e-06 2024-09-04T17:46:25.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:25.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.286526+0000", 2024-09-04T17:46:25.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051681373000000003 2024-09-04T17:46:25.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:25.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.286594+0000", 2024-09-04T17:46:25.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8073999999999999e-05 2024-09-04T17:46:25.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:25.247 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.286724+0000", 2024-09-04T17:46:25.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013041300000000001 2024-09-04T17:46:25.247 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.247 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:25.247 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.247 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15525 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 335872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:25.247 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.234851+0000", 2024-09-04T17:46:25.248 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.9988340689999999, 2024-09-04T17:46:25.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052146248999999999, 2024-09-04T17:46:25.248 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:25.248 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:25.248 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:25.248 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:25.248 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:25.248 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:25.248 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15525 2024-09-04T17:46:25.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.249 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:25.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:25.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234851+0000", 2024-09-04T17:46:25.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:25.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234851+0000", 2024-09-04T17:46:25.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:25.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234852+0000", 2024-09-04T17:46:25.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0999999999999999e-07 2024-09-04T17:46:25.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:25.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234854+0000", 2024-09-04T17:46:25.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9929999999999998e-06 2024-09-04T17:46:25.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:25.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234860+0000", 2024-09-04T17:46:25.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1489999999999999e-06 2024-09-04T17:46:25.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:25.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234865+0000", 2024-09-04T17:46:25.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8600000000000001e-06 2024-09-04T17:46:25.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:25.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.286770+0000", 2024-09-04T17:46:25.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051904807999999997 2024-09-04T17:46:25.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:25.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.286852+0000", 2024-09-04T17:46:25.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2076000000000005e-05 2024-09-04T17:46:25.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:25.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.286998+0000", 2024-09-04T17:46:25.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014595299999999999 2024-09-04T17:46:25.252 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.252 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:25.252 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.252 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15526 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 811008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:25.252 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.234871+0000", 2024-09-04T17:46:25.253 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.9988143000000003, 2024-09-04T17:46:25.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052383817999999999, 2024-09-04T17:46:25.253 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:25.253 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:25.253 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:25.253 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:25.253 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:25.253 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:25.253 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15526 2024-09-04T17:46:25.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.253 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:25.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:25.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234871+0000", 2024-09-04T17:46:25.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:25.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234871+0000", 2024-09-04T17:46:25.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:25.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234872+0000", 2024-09-04T17:46:25.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4299999999999998e-07 2024-09-04T17:46:25.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:25.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234874+0000", 2024-09-04T17:46:25.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2220000000000001e-06 2024-09-04T17:46:25.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:25.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234880+0000", 2024-09-04T17:46:25.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4169999999999997e-06 2024-09-04T17:46:25.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:25.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234886+0000", 2024-09-04T17:46:25.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8150000000000002e-06 2024-09-04T17:46:25.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:25.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287040+0000", 2024-09-04T17:46:25.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052153928000000002 2024-09-04T17:46:25.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:25.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287118+0000", 2024-09-04T17:46:25.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8102999999999994e-05 2024-09-04T17:46:25.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:25.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287255+0000", 2024-09-04T17:46:25.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013689 2024-09-04T17:46:25.257 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.257 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:25.257 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.258 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15527 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 372736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:25.258 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.234906+0000", 2024-09-04T17:46:25.258 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.9987789849999995, 2024-09-04T17:46:25.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052620742999999998, 2024-09-04T17:46:25.258 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:25.258 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:25.258 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:25.258 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:25.258 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:25.258 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:25.258 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15527 2024-09-04T17:46:25.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.259 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:25.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:25.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234906+0000", 2024-09-04T17:46:25.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:25.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234906+0000", 2024-09-04T17:46:25.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:25.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234907+0000", 2024-09-04T17:46:25.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9100000000000004e-07 2024-09-04T17:46:25.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:25.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234910+0000", 2024-09-04T17:46:25.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7599999999999998e-06 2024-09-04T17:46:25.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:25.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234917+0000", 2024-09-04T17:46:25.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.861e-06 2024-09-04T17:46:25.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:25.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234923+0000", 2024-09-04T17:46:25.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4130000000000003e-06 2024-09-04T17:46:25.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:25.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287302+0000", 2024-09-04T17:46:25.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052379440999999999 2024-09-04T17:46:25.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:25.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287382+0000", 2024-09-04T17:46:25.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9406999999999994e-05 2024-09-04T17:46:25.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:25.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287527+0000", 2024-09-04T17:46:25.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014537 2024-09-04T17:46:25.262 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.262 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:25.262 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.262 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15528 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 847872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:25.262 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.234931+0000", 2024-09-04T17:46:25.262 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.9987540670000001, 2024-09-04T17:46:25.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052862919000000001, 2024-09-04T17:46:25.263 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:25.263 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:25.263 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:25.263 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:25.263 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:25.263 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:25.263 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15528 2024-09-04T17:46:25.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.263 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:25.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:25.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234931+0000", 2024-09-04T17:46:25.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:25.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234931+0000", 2024-09-04T17:46:25.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:25.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234932+0000", 2024-09-04T17:46:25.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6100000000000001e-07 2024-09-04T17:46:25.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:25.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234934+0000", 2024-09-04T17:46:25.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3309999999999998e-06 2024-09-04T17:46:25.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:25.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234941+0000", 2024-09-04T17:46:25.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4869999999999999e-06 2024-09-04T17:46:25.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:25.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234946+0000", 2024-09-04T17:46:25.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5879999999999997e-06 2024-09-04T17:46:25.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:25.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287571+0000", 2024-09-04T17:46:25.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052624769000000002 2024-09-04T17:46:25.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:25.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287651+0000", 2024-09-04T17:46:25.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9975000000000006e-05 2024-09-04T17:46:25.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:25.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287794+0000", 2024-09-04T17:46:25.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000143308 2024-09-04T17:46:25.267 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.267 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:25.267 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.267 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15529 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 409600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:25.267 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.234954+0000", 2024-09-04T17:46:25.267 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.9987311959999996, 2024-09-04T17:46:25.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053149889999999998, 2024-09-04T17:46:25.267 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:25.267 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:25.268 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:25.268 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:25.268 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:25.268 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:25.268 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15529 2024-09-04T17:46:25.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.268 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:25.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:25.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234954+0000", 2024-09-04T17:46:25.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:25.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234954+0000", 2024-09-04T17:46:25.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:25.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234955+0000", 2024-09-04T17:46:25.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3799999999999997e-07 2024-09-04T17:46:25.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:25.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234957+0000", 2024-09-04T17:46:25.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3309999999999998e-06 2024-09-04T17:46:25.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:25.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234964+0000", 2024-09-04T17:46:25.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5780000000000004e-06 2024-09-04T17:46:25.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:25.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234969+0000", 2024-09-04T17:46:25.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0629999999999996e-06 2024-09-04T17:46:25.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:25.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287839+0000", 2024-09-04T17:46:25.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052870385999999998 2024-09-04T17:46:25.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:25.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287929+0000", 2024-09-04T17:46:25.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9514999999999999e-05 2024-09-04T17:46:25.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:25.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288104+0000", 2024-09-04T17:46:25.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000175379 2024-09-04T17:46:25.271 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.271 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:25.272 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.272 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15530 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 884736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:25.272 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.234976+0000", 2024-09-04T17:46:25.272 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.9987097780000003, 2024-09-04T17:46:25.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053414593000000003, 2024-09-04T17:46:25.272 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:25.272 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:25.272 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:25.272 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:25.272 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:25.272 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:25.273 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15530 2024-09-04T17:46:25.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.273 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:25.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:25.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234976+0000", 2024-09-04T17:46:25.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:25.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234976+0000", 2024-09-04T17:46:25.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:25.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234976+0000", 2024-09-04T17:46:25.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5299999999999999e-07 2024-09-04T17:46:25.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:25.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234978+0000", 2024-09-04T17:46:25.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0329999999999998e-06 2024-09-04T17:46:25.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:25.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234984+0000", 2024-09-04T17:46:25.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0249999999999999e-06 2024-09-04T17:46:25.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:25.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234990+0000", 2024-09-04T17:46:25.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7579999999999998e-06 2024-09-04T17:46:25.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:25.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288149+0000", 2024-09-04T17:46:25.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053158864 2024-09-04T17:46:25.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:25.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288236+0000", 2024-09-04T17:46:25.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7306999999999996e-05 2024-09-04T17:46:25.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:25.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288390+0000", 2024-09-04T17:46:25.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000154153 2024-09-04T17:46:25.276 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.276 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:25.276 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.276 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15531 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 446464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:25.276 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.234997+0000", 2024-09-04T17:46:25.277 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.9986887949999996, 2024-09-04T17:46:25.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053721086000000001, 2024-09-04T17:46:25.277 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:25.277 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:25.277 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:25.277 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:25.277 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:25.277 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:25.277 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15531 2024-09-04T17:46:25.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.277 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:25.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:25.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234997+0000", 2024-09-04T17:46:25.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:25.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234997+0000", 2024-09-04T17:46:25.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:25.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234997+0000", 2024-09-04T17:46:25.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8200000000000001e-07 2024-09-04T17:46:25.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:25.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234999+0000", 2024-09-04T17:46:25.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.058e-06 2024-09-04T17:46:25.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:25.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235005+0000", 2024-09-04T17:46:25.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0700000000000003e-06 2024-09-04T17:46:25.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:25.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235012+0000", 2024-09-04T17:46:25.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7020000000000004e-06 2024-09-04T17:46:25.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:25.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288441+0000", 2024-09-04T17:46:25.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053429343999999997 2024-09-04T17:46:25.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:25.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288535+0000", 2024-09-04T17:46:25.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3406000000000007e-05 2024-09-04T17:46:25.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:25.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288718+0000", 2024-09-04T17:46:25.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000183124 2024-09-04T17:46:25.281 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.281 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:25.281 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.281 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15532 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 921600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:25.281 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235020+0000", 2024-09-04T17:46:25.281 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.9986651680000005, 2024-09-04T17:46:25.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053966157000000001, 2024-09-04T17:46:25.281 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:25.282 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:25.282 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:25.282 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:25.282 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:25.282 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:25.282 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15532 2024-09-04T17:46:25.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.282 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:25.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:25.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235020+0000", 2024-09-04T17:46:25.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:25.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235020+0000", 2024-09-04T17:46:25.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:25.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235021+0000", 2024-09-04T17:46:25.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9799999999999999e-07 2024-09-04T17:46:25.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:25.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235023+0000", 2024-09-04T17:46:25.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1560000000000002e-06 2024-09-04T17:46:25.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:25.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235029+0000", 2024-09-04T17:46:25.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0270000000000001e-06 2024-09-04T17:46:25.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:25.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235034+0000", 2024-09-04T17:46:25.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0000000000000004e-06 2024-09-04T17:46:25.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:25.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288760+0000", 2024-09-04T17:46:25.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053726077999999997 2024-09-04T17:46:25.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:25.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288845+0000", 2024-09-04T17:46:25.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5242999999999997e-05 2024-09-04T17:46:25.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:25.286 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288986+0000", 2024-09-04T17:46:25.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014125499999999999 2024-09-04T17:46:25.286 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.286 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:25.286 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.286 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15533 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 483328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:25.286 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235041+0000", 2024-09-04T17:46:25.286 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.9986440930000002, 2024-09-04T17:46:25.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054242994000000003, 2024-09-04T17:46:25.287 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:25.287 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:25.287 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:25.287 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:25.287 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:25.287 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:25.287 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15533 2024-09-04T17:46:25.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.287 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:25.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:25.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235041+0000", 2024-09-04T17:46:25.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:25.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235041+0000", 2024-09-04T17:46:25.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:25.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235042+0000", 2024-09-04T17:46:25.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5499999999999999e-07 2024-09-04T17:46:25.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:25.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235044+0000", 2024-09-04T17:46:25.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2259999999999999e-06 2024-09-04T17:46:25.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:25.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235051+0000", 2024-09-04T17:46:25.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4440000000000002e-06 2024-09-04T17:46:25.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:25.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235056+0000", 2024-09-04T17:46:25.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8740000000000003e-06 2024-09-04T17:46:25.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:25.290 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289028+0000", 2024-09-04T17:46:25.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053971742000000003 2024-09-04T17:46:25.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:25.290 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289108+0000", 2024-09-04T17:46:25.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0502999999999998e-05 2024-09-04T17:46:25.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:25.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289284+0000", 2024-09-04T17:46:25.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017584999999999999 2024-09-04T17:46:25.291 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.291 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:25.291 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.291 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15534 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 958464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:25.291 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235068+0000", 2024-09-04T17:46:25.291 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.998617393, 2024-09-04T17:46:25.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054505452000000003, 2024-09-04T17:46:25.291 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:25.291 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:25.292 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:25.292 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:25.292 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:25.292 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:25.292 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15534 2024-09-04T17:46:25.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.292 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:25.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:25.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235068+0000", 2024-09-04T17:46:25.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:25.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235068+0000", 2024-09-04T17:46:25.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:25.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235068+0000", 2024-09-04T17:46:25.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7899999999999999e-07 2024-09-04T17:46:25.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:25.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235071+0000", 2024-09-04T17:46:25.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4109999999999999e-06 2024-09-04T17:46:25.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:25.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235078+0000", 2024-09-04T17:46:25.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0489999999999998e-06 2024-09-04T17:46:25.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:25.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235083+0000", 2024-09-04T17:46:25.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2249999999999999e-06 2024-09-04T17:46:25.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:25.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289328+0000", 2024-09-04T17:46:25.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054244894000000002 2024-09-04T17:46:25.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:25.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289417+0000", 2024-09-04T17:46:25.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8988999999999998e-05 2024-09-04T17:46:25.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:25.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289573+0000", 2024-09-04T17:46:25.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000156405 2024-09-04T17:46:25.296 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.296 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:25.296 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.296 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15535 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 520192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:25.296 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235173+0000", 2024-09-04T17:46:25.296 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.9985128640000003, 2024-09-04T17:46:25.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054718671000000003, 2024-09-04T17:46:25.296 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:25.296 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:25.296 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:25.296 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:25.296 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:25.297 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:25.297 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15535 2024-09-04T17:46:25.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.297 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:25.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:25.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235173+0000", 2024-09-04T17:46:25.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:25.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235173+0000", 2024-09-04T17:46:25.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:25.298 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235173+0000", 2024-09-04T17:46:25.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2600000000000002e-07 2024-09-04T17:46:25.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:25.298 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235177+0000", 2024-09-04T17:46:25.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.625e-06 2024-09-04T17:46:25.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:25.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235188+0000", 2024-09-04T17:46:25.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1464000000000001e-05 2024-09-04T17:46:25.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:25.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235195+0000", 2024-09-04T17:46:25.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5910000000000001e-06 2024-09-04T17:46:25.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:25.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289623+0000", 2024-09-04T17:46:25.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054427595000000002 2024-09-04T17:46:25.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.300 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:25.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289718+0000", 2024-09-04T17:46:25.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5958999999999994e-05 2024-09-04T17:46:25.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.300 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:25.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289891+0000", 2024-09-04T17:46:25.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017271100000000001 2024-09-04T17:46:25.300 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.300 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:25.300 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.301 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15536 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 995328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:25.301 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235206+0000", 2024-09-04T17:46:25.301 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.998479723, 2024-09-04T17:46:25.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054984065999999998, 2024-09-04T17:46:25.301 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:25.301 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:25.301 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:25.301 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:25.301 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:25.301 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:25.301 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15536 2024-09-04T17:46:25.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.302 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:25.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:25.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235206+0000", 2024-09-04T17:46:25.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:25.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235206+0000", 2024-09-04T17:46:25.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:25.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235206+0000", 2024-09-04T17:46:25.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5300000000000004e-07 2024-09-04T17:46:25.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.303 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:25.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235209+0000", 2024-09-04T17:46:25.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6089999999999999e-06 2024-09-04T17:46:25.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.303 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:25.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235216+0000", 2024-09-04T17:46:25.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.277e-06 2024-09-04T17:46:25.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:25.304 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235221+0000", 2024-09-04T17:46:25.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2530000000000004e-06 2024-09-04T17:46:25.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:25.304 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289936+0000", 2024-09-04T17:46:25.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054714888000000003 2024-09-04T17:46:25.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:25.304 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290035+0000", 2024-09-04T17:46:25.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8652000000000003e-05 2024-09-04T17:46:25.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:25.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290190+0000", 2024-09-04T17:46:25.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015483399999999999 2024-09-04T17:46:25.305 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.305 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:25.305 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.305 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15537 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 1323008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:25.305 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235264+0000", 2024-09-04T17:46:25.305 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.9984216549999996, 2024-09-04T17:46:25.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055255669, 2024-09-04T17:46:25.306 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:25.306 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:25.306 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:25.306 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:25.306 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:25.306 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:25.306 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15537 2024-09-04T17:46:25.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.306 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:25.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:25.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235264+0000", 2024-09-04T17:46:25.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:25.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235264+0000", 2024-09-04T17:46:25.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:25.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235264+0000", 2024-09-04T17:46:25.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4499999999999997e-07 2024-09-04T17:46:25.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:25.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235267+0000", 2024-09-04T17:46:25.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9210000000000001e-06 2024-09-04T17:46:25.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.308 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:25.308 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235276+0000", 2024-09-04T17:46:25.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2309999999999996e-06 2024-09-04T17:46:25.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.308 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:25.308 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235281+0000", 2024-09-04T17:46:25.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8560000000000003e-06 2024-09-04T17:46:25.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.309 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:25.309 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290246+0000", 2024-09-04T17:46:25.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054964528999999998 2024-09-04T17:46:25.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.309 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:25.309 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290344+0000", 2024-09-04T17:46:25.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8073999999999996e-05 2024-09-04T17:46:25.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.309 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:25.309 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290519+0000", 2024-09-04T17:46:25.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000175413 2024-09-04T17:46:25.310 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.310 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:25.310 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.310 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15538 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 1032192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:25.310 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235290+0000", 2024-09-04T17:46:25.310 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.9983957459999999, 2024-09-04T17:46:25.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055518932, 2024-09-04T17:46:25.310 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:25.310 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:25.310 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:25.310 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:25.310 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:25.311 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:25.311 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15538 2024-09-04T17:46:25.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.311 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:25.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:25.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235290+0000", 2024-09-04T17:46:25.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:25.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235290+0000", 2024-09-04T17:46:25.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.312 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:25.312 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235290+0000", 2024-09-04T17:46:25.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3399999999999999e-07 2024-09-04T17:46:25.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.312 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:25.312 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235292+0000", 2024-09-04T17:46:25.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2639999999999998e-06 2024-09-04T17:46:25.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.312 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:25.312 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235299+0000", 2024-09-04T17:46:25.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2029999999999997e-06 2024-09-04T17:46:25.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:25.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235305+0000", 2024-09-04T17:46:25.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7469999999999999e-06 2024-09-04T17:46:25.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:25.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290562+0000", 2024-09-04T17:46:25.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055256580999999999 2024-09-04T17:46:25.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.314 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:25.314 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290651+0000", 2024-09-04T17:46:25.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8953e-05 2024-09-04T17:46:25.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.314 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:25.314 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290809+0000", 2024-09-04T17:46:25.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015765000000000001 2024-09-04T17:46:25.314 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.314 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:25.314 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.314 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15539 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 1359872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:25.315 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235312+0000", 2024-09-04T17:46:25.315 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.9983730059999996, 2024-09-04T17:46:25.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055809865, 2024-09-04T17:46:25.315 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:25.315 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:25.315 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:25.315 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:25.315 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:25.315 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:25.315 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15539 2024-09-04T17:46:25.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.315 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:25.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:25.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235312+0000", 2024-09-04T17:46:25.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:25.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235312+0000", 2024-09-04T17:46:25.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:25.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235313+0000", 2024-09-04T17:46:25.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9000000000000002e-07 2024-09-04T17:46:25.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:25.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235315+0000", 2024-09-04T17:46:25.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1270000000000001e-06 2024-09-04T17:46:25.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:25.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235320+0000", 2024-09-04T17:46:25.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5430000000000001e-06 2024-09-04T17:46:25.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:25.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235326+0000", 2024-09-04T17:46:25.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2039999999999996e-06 2024-09-04T17:46:25.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.318 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:25.318 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290858+0000", 2024-09-04T17:46:25.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055532440000000002 2024-09-04T17:46:25.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.318 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:25.318 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290953+0000", 2024-09-04T17:46:25.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5031000000000005e-05 2024-09-04T17:46:25.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.318 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:25.318 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.291122+0000", 2024-09-04T17:46:25.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016913 2024-09-04T17:46:25.319 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.319 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:25.319 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.319 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15540 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 1396736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:25.319 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235333+0000", 2024-09-04T17:46:25.319 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.9983524990000001, 2024-09-04T17:46:25.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056080528999999997, 2024-09-04T17:46:25.319 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:25.319 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:25.319 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:25.320 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:25.320 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:25.320 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:25.320 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15540 2024-09-04T17:46:25.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.320 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:25.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:25.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235333+0000", 2024-09-04T17:46:25.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:25.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235333+0000", 2024-09-04T17:46:25.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:25.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235333+0000", 2024-09-04T17:46:25.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0999999999999999e-07 2024-09-04T17:46:25.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:25.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235336+0000", 2024-09-04T17:46:25.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.278e-06 2024-09-04T17:46:25.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:25.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235341+0000", 2024-09-04T17:46:25.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8359999999999997e-06 2024-09-04T17:46:25.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:25.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235346+0000", 2024-09-04T17:46:25.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4329999999999997e-06 2024-09-04T17:46:25.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:25.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.291167+0000", 2024-09-04T17:46:25.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055821603999999997 2024-09-04T17:46:25.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.323 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:25.323 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.291257+0000", 2024-09-04T17:46:25.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9349000000000001e-05 2024-09-04T17:46:25.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.323 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:25.323 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.291413+0000", 2024-09-04T17:46:25.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015661900000000001 2024-09-04T17:46:25.323 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.323 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:25.323 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.324 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.324 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15541 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 1433600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:25.324 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235355+0000", 2024-09-04T17:46:25.324 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.9983303110000001, 2024-09-04T17:46:25.324 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056371783000000002, 2024-09-04T17:46:25.324 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:25.324 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:25.324 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:25.324 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:25.324 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:25.324 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:25.324 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15541 2024-09-04T17:46:25.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.324 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:25.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:25.325 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235355+0000", 2024-09-04T17:46:25.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:25.325 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235355+0000", 2024-09-04T17:46:25.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:25.325 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235356+0000", 2024-09-04T17:46:25.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3099999999999998e-07 2024-09-04T17:46:25.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:25.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235358+0000", 2024-09-04T17:46:25.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3599999999999999e-06 2024-09-04T17:46:25.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:25.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235364+0000", 2024-09-04T17:46:25.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5760000000000002e-06 2024-09-04T17:46:25.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:25.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235370+0000", 2024-09-04T17:46:25.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0819999999999998e-06 2024-09-04T17:46:25.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:25.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.291462+0000", 2024-09-04T17:46:25.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056092301999999997 2024-09-04T17:46:25.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:25.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.291559+0000", 2024-09-04T17:46:25.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6988e-05 2024-09-04T17:46:25.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.328 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:25.328 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.291727+0000", 2024-09-04T17:46:25.328 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016804400000000001 2024-09-04T17:46:25.328 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.328 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:25.328 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.328 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.328 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:25.328 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:25.329 INFO:teuthology.orchestra.run.smithi092.stdout:1354 write 0x71419a8 thru 0x714d8e0 (0xbf39 bytes) 2024-09-04T17:46:25.329 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:46:25.329 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:46:25.330 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-04T17:46:25.331 INFO:teuthology.orchestra.run.smithi092.stdout:1355 write 0xe4d6475 thru 0xe4db409 (0x4f95 bytes) 2024-09-04T17:46:25.331 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-04T17:46:25.331 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-04T17:46:25.332 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:25.332 INFO:teuthology.orchestra.run.smithi092.stdout:1357 write 0x885b411 thru 0x885c276 (0xe66 bytes) 2024-09-04T17:46:25.332 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:25.332 INFO:teuthology.orchestra.run.smithi092.stdout:1359 read 0x4968838 thru 0x496b943 (0x310c bytes) 2024-09-04T17:46:25.332 INFO:teuthology.orchestra.run.smithi092.stdout:1360 trunc from 0xe4db40a to 0x9df414f 2024-09-04T17:46:25.333 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:46:25.333 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-04T17:46:25.333 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:46:25.334 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-04T17:46:25.334 INFO:teuthology.orchestra.run.smithi092.stdout:1361 punch from 0xf60cb0 to 0xf6485f, (0x3baf bytes) 2024-09-04T17:46:25.334 INFO:teuthology.orchestra.run.smithi092.stdout:1362 read 0x3e5ea6e thru 0x3e69fd9 (0xb56c bytes) 2024-09-04T17:46:25.335 INFO:teuthology.orchestra.run.smithi092.stdout:1363 write 0x618fc72 thru 0x619505b (0x53ea bytes) 2024-09-04T17:46:25.335 INFO:teuthology.orchestra.run.smithi092.stdout:1364 write 0xca3ae08 thru 0xca3c775 (0x196e bytes) 2024-09-04T17:46:25.335 INFO:teuthology.orchestra.run.smithi092.stdout:1365 read 0x5ebd45a thru 0x5ecc9ea (0xf591 bytes) 2024-09-04T17:46:25.335 INFO:teuthology.orchestra.run.smithi092.stdout:1366 write 0x63bda2b thru 0x63bf4f9 (0x1acf bytes) 2024-09-04T17:46:25.335 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is unset 2024-09-04T17:46:25.335 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:25.329+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:25.336 INFO:teuthology.orchestra.run.smithi092.stdout:1367 trunc from 0xca3c776 to 0x78c9932 2024-09-04T17:46:25.381 INFO:teuthology.orchestra.run.smithi092.stdout:1368 writesame 0xec79a0c thru 0xec82e9b data_size 0x8(0x9490 bytes) 2024-09-04T17:46:25.381 INFO:teuthology.orchestra.run.smithi092.stdout:1370 punch from 0xd1b8356 to 0xd1bb1aa, (0x2e54 bytes) 2024-09-04T17:46:25.381 INFO:teuthology.orchestra.run.smithi092.stdout:1372 punch from 0x8a383d9 to 0x8a3ff0d, (0x7b34 bytes) 2024-09-04T17:46:25.381 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:25.381 INFO:teuthology.orchestra.run.smithi092.stdout:1375 write 0x36aaf0c thru 0x36b272d (0x7822 bytes) 2024-09-04T17:46:25.391 INFO:teuthology.orchestra.run.smithi092.stdout:1376 punch from 0x5193fd5 to 0x51a072e, (0xc759 bytes) 2024-09-04T17:46:25.391 INFO:teuthology.orchestra.run.smithi092.stdout:1377 punch from 0xcdcfd91 to 0xcdd7740, (0x79af bytes) 2024-09-04T17:46:25.391 INFO:teuthology.orchestra.run.smithi092.stdout:1380 read 0xb1b8870 thru 0xb1c017e (0x790f bytes) 2024-09-04T17:46:25.392 INFO:teuthology.orchestra.run.smithi092.stdout:1381 writesame 0x9b9e7d2 thru 0x9babce4 data_size 0x1(0xd513 bytes) 2024-09-04T17:46:25.422 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:25.422 INFO:teuthology.orchestra.run.smithi092.stdout:1384 punch from 0x1ef43cd to 0x1efb084, (0x6cb7 bytes) 2024-09-04T17:46:25.423 INFO:teuthology.orchestra.run.smithi092.stdout:1385 write 0x53dc234 thru 0x53e1b2e (0x58fb bytes) 2024-09-04T17:46:25.424 INFO:teuthology.orchestra.run.smithi092.stdout:1386 read 0x18613bd thru 0x1866a1d (0x5661 bytes) 2024-09-04T17:46:25.424 INFO:teuthology.orchestra.run.smithi092.stdout:1387 write 0x51bcd40 thru 0x51cb205 (0xe4c6 bytes) 2024-09-04T17:46:25.428 INFO:teuthology.orchestra.run.smithi092.stdout:1388 write 0xc0c27c6 thru 0xc0c4ac9 (0x2304 bytes) 2024-09-04T17:46:25.429 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:25.429+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:25.430 INFO:teuthology.orchestra.run.smithi092.stdout:1389 writesame 0x2e10ba3 thru 0x2e1ffa3 data_size 0x41(0xf401 bytes) 2024-09-04T17:46:25.442 INFO:teuthology.orchestra.run.smithi092.stdout:1390 trunc from 0xec82e9c to 0xa732c65 2024-09-04T17:46:25.460 INFO:teuthology.orchestra.run.smithi092.stdout:1391 punch from 0x3a2a648 to 0x3a33749, (0x9101 bytes) 2024-09-04T17:46:25.460 INFO:teuthology.orchestra.run.smithi092.stdout:1392 read 0x5828c79 thru 0x58324f3 (0x987b bytes) 2024-09-04T17:46:25.461 INFO:teuthology.orchestra.run.smithi092.stdout:1393 read 0x935f37e thru 0x936bda5 (0xca28 bytes) 2024-09-04T17:46:25.461 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:25.462 INFO:teuthology.orchestra.run.smithi092.stdout:1396 write 0x416b318 thru 0x416cc2b (0x1914 bytes) 2024-09-04T17:46:25.467 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:25.467 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:46:25.467 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:46:25.467 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:25.477 INFO:teuthology.orchestra.run.smithi092.stdout:1397 write 0xbc5d06d thru 0xbc68141 (0xb0d5 bytes) 2024-09-04T17:46:25.478 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:25.478 INFO:teuthology.orchestra.run.smithi092.stdout:1400 write 0x2a80ab1 thru 0x2a9065e (0xfbae bytes) 2024-09-04T17:46:25.482 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-04T17:46:25.482 INFO:teuthology.orchestra.run.smithi092.stdout:1402 writesame 0x8498725 thru 0x849e68b data_size 0x7(0x5f67 bytes) 2024-09-04T17:46:25.489 INFO:teuthology.orchestra.run.smithi092.stdout:1405 punch from 0x7d53183 to 0x7d5fd6d, (0xcbea bytes) 2024-09-04T17:46:25.490 INFO:teuthology.orchestra.run.smithi092.stdout:1408 punch from 0x96f71fb to 0x9704958, (0xd75d bytes) 2024-09-04T17:46:25.490 INFO:teuthology.orchestra.run.smithi092.stdout:1409 read 0x23aa1ae thru 0x23b8466 (0xe2b9 bytes) 2024-09-04T17:46:25.490 INFO:teuthology.orchestra.run.smithi092.stdout:1410 punch from 0x5516c97 to 0x551ed26, (0x808f bytes) 2024-09-04T17:46:25.490 INFO:teuthology.orchestra.run.smithi092.stdout:1413 writesame 0x8e410ec thru 0x8e44965 data_size 0x1(0x387a bytes) 2024-09-04T17:46:25.507 INFO:teuthology.orchestra.run.smithi092.stdout:1414 writesame 0x85ab10a thru 0x85b641d data_size 0x49(0xb314 bytes) 2024-09-04T17:46:25.511 INFO:teuthology.orchestra.run.smithi092.stdout:1415 write 0x27afc86 thru 0x27b33e3 (0x375e bytes) 2024-09-04T17:46:25.514 INFO:teuthology.orchestra.run.smithi092.stdout:1416 writesame 0x36421e6 thru 0x36492f8 data_size 0x1(0x7113 bytes) 2024-09-04T17:46:25.528 INFO:teuthology.orchestra.run.smithi092.stdout:1418 write 0x526b182 thru 0x5272cc9 (0x7b48 bytes) 2024-09-04T17:46:25.531 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:25.531 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:25.533+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:25.535 INFO:teuthology.orchestra.run.smithi092.stdout:1420 trunc from 0xbc68142 to 0x8f0ceef 2024-09-04T17:46:25.547 INFO:teuthology.orchestra.run.smithi092.stdout:1421 read 0x428e5b1 thru 0x4299a8d (0xb4dd bytes) 2024-09-04T17:46:25.548 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:25.559 INFO:teuthology.orchestra.run.smithi092.stdout:1425 trunc from 0x8f0ceef to 0x2d3342a 2024-09-04T17:46:25.579 INFO:teuthology.orchestra.run.smithi092.stdout:1426 write 0x1deaba0 thru 0x1deea85 (0x3ee6 bytes) 2024-09-04T17:46:25.581 INFO:teuthology.orchestra.run.smithi092.stdout:1427 punch from 0x9a1a32 to 0x9a5abc, (0x408a bytes) 2024-09-04T17:46:25.592 INFO:teuthology.orchestra.run.smithi092.stdout:1428 trunc from 0x2d3342a to 0xad64d32 2024-09-04T17:46:25.603 INFO:teuthology.orchestra.run.smithi092.stdout:1429 write 0x36e1fd0 thru 0x36e5b21 (0x3b52 bytes) 2024-09-04T17:46:25.621 INFO:teuthology.orchestra.run.smithi092.stdout:1432 write 0xe341c7a thru 0xe34761c (0x59a3 bytes) 2024-09-04T17:46:25.625 INFO:teuthology.orchestra.run.smithi092.stdout:1434 read 0x9df8fcd thru 0x9df92ac (0x2e0 bytes) 2024-09-04T17:46:25.632 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:25.633+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:25.633 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:25.633 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:46:25.633 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:46:25.634 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:46:25.634 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:25.636 INFO:teuthology.orchestra.run.smithi092.stdout:1436 trunc from 0xe34761d to 0x6b9576d 2024-09-04T17:46:25.654 INFO:teuthology.orchestra.run.smithi092.stdout:1437 punch from 0x36c86d0 to 0x36cdf73, (0x58a3 bytes) 2024-09-04T17:46:25.658 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-04T17:46:25.674 INFO:teuthology.orchestra.run.smithi092.stdout:1438 write 0xd2e2721 thru 0xd2ed87c (0xb15c bytes) 2024-09-04T17:46:25.677 INFO:teuthology.orchestra.run.smithi092.stdout:1439 punch from 0x4d2ae9d to 0x4d37fa9, (0xd10c bytes) 2024-09-04T17:46:25.677 INFO:teuthology.orchestra.run.smithi092.stdout:1443 punch from 0xb289610 to 0xb28fd99, (0x6789 bytes) 2024-09-04T17:46:25.677 INFO:teuthology.orchestra.run.smithi092.stdout:1444 write 0x31acae3 thru 0x31bc699 (0xfbb7 bytes) 2024-09-04T17:46:25.679 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:25.679 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:25.680 INFO:teuthology.orchestra.run.smithi092.stdout:1447 read 0x39c5033 thru 0x39c943f (0x440d bytes) 2024-09-04T17:46:25.681 INFO:teuthology.orchestra.run.smithi092.stdout:1450 writesame 0x36af198 thru 0x36b6e99 data_size 0x1(0x7d02 bytes) 2024-09-04T17:46:25.719 INFO:teuthology.orchestra.run.smithi092.stdout:1452 write 0xd248a69 thru 0xd2538c3 (0xae5b bytes) 2024-09-04T17:46:25.725 INFO:teuthology.orchestra.run.smithi092.stdout:1453 punch from 0x8753f43 to 0x875db55, (0x9c12 bytes) 2024-09-04T17:46:25.725 INFO:teuthology.orchestra.run.smithi092.stdout:1454 read 0xbcc9b7d thru 0xbcd717d (0xd601 bytes) 2024-09-04T17:46:25.725 INFO:teuthology.orchestra.run.smithi092.stdout:1455 write 0xc6c0a54 thru 0xc6cc484 (0xba31 bytes) 2024-09-04T17:46:25.727 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:25.733 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:46:25.731+0000 7f505d642640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26446) UID: 0 2024-09-04T17:46:25.745 INFO:teuthology.orchestra.run.smithi092.stdout:1457 trunc from 0xd2ed87d to 0x891f2f 2024-09-04T17:46:25.785 INFO:teuthology.orchestra.run.smithi092.stdout:1458 trunc from 0x891f2f to 0xd17bd4e 2024-09-04T17:46:25.798 INFO:teuthology.orchestra.run.smithi092.stdout:1459 read 0x9f936cf thru 0x9f9fca2 (0xc5d4 bytes) 2024-09-04T17:46:25.798 INFO:teuthology.orchestra.run.smithi092.stdout:1460 read 0x25a9faa thru 0x25b78f4 (0xd94b bytes) 2024-09-04T17:46:25.808 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:25.808 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:46:25.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:46:25.809 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:46:25.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.809 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:4264 3.9 e30/24 3:970e51a0:::rbd_data.1134cfbce83d.0000000000000009:head v 30'83, mlcod=30'81)", 2024-09-04T17:46:25.809 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:07.027227+0000", 2024-09-04T17:46:25.809 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.782437556000001, 2024-09-04T17:46:25.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0096959590000000005, 2024-09-04T17:46:25.809 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:25.809 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:25.809 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:25.810 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:25.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:25.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.027227+0000", 2024-09-04T17:46:25.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:25.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.027227+0000", 2024-09-04T17:46:25.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:25.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.027229+0000", 2024-09-04T17:46:25.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4839999999999998e-06 2024-09-04T17:46:25.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:25.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.031984+0000", 2024-09-04T17:46:25.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047544789999999998 2024-09-04T17:46:25.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:25.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.032019+0000", 2024-09-04T17:46:25.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5253999999999998e-05 2024-09-04T17:46:25.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:25.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.032036+0000", 2024-09-04T17:46:25.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6713000000000001e-05 2024-09-04T17:46:25.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:25.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.032129+0000", 2024-09-04T17:46:25.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2961000000000007e-05 2024-09-04T17:46:25.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:25.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.032154+0000", 2024-09-04T17:46:25.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5148999999999999e-05 2024-09-04T17:46:25.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:25.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.036778+0000", 2024-09-04T17:46:25.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046239159999999996 2024-09-04T17:46:25.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:25.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.036923+0000", 2024-09-04T17:46:25.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014500299999999999 2024-09-04T17:46:25.813 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.813 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:25.813 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.814 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:4281 3.4 e30/24 3:215ebead:::rbd_header.1134cfbce83d:head v 30'24, mlcod=30'22)", 2024-09-04T17:46:25.814 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:07.073105+0000", 2024-09-04T17:46:25.814 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.736558792, 2024-09-04T17:46:25.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0032176499999999999, 2024-09-04T17:46:25.814 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:25.814 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:25.814 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:25.814 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:25.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:25.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.073105+0000", 2024-09-04T17:46:25.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:25.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.073105+0000", 2024-09-04T17:46:25.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:25.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.073107+0000", 2024-09-04T17:46:25.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1260000000000001e-06 2024-09-04T17:46:25.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:25.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.073115+0000", 2024-09-04T17:46:25.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0430000000000008e-06 2024-09-04T17:46:25.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:25.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.073133+0000", 2024-09-04T17:46:25.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8308000000000002e-05 2024-09-04T17:46:25.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:25.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.073144+0000", 2024-09-04T17:46:25.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1219999999999999e-05 2024-09-04T17:46:25.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:25.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.073218+0000", 2024-09-04T17:46:25.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3464e-05 2024-09-04T17:46:25.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:25.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.073244+0000", 2024-09-04T17:46:25.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6308e-05 2024-09-04T17:46:25.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:25.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.076286+0000", 2024-09-04T17:46:25.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00304163 2024-09-04T17:46:25.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:25.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.076323+0000", 2024-09-04T17:46:25.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7551e-05 2024-09-04T17:46:25.818 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.818 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:25.818 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.818 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:4399 3.9 e31/24 3:970e51a0:::rbd_data.1134cfbce83d.0000000000000009:head v 31'87, mlcod=30'84)", 2024-09-04T17:46:25.818 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:07.343236+0000", 2024-09-04T17:46:25.818 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.466427872000001, 2024-09-04T17:46:25.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0036134539999999999, 2024-09-04T17:46:25.819 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:25.819 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:25.819 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:25.819 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:25.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:25.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.343236+0000", 2024-09-04T17:46:25.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:25.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.343236+0000", 2024-09-04T17:46:25.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:25.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.343237+0000", 2024-09-04T17:46:25.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4000000000000001e-07 2024-09-04T17:46:25.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:25.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.343250+0000", 2024-09-04T17:46:25.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2573e-05 2024-09-04T17:46:25.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:25.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.343279+0000", 2024-09-04T17:46:25.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9263000000000001e-05 2024-09-04T17:46:25.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:25.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.343299+0000", 2024-09-04T17:46:25.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9720000000000001e-05 2024-09-04T17:46:25.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:25.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.343389+0000", 2024-09-04T17:46:25.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9924000000000002e-05 2024-09-04T17:46:25.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:25.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.343413+0000", 2024-09-04T17:46:25.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4227999999999999e-05 2024-09-04T17:46:25.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:25.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.346796+0000", 2024-09-04T17:46:25.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0033831310000000002 2024-09-04T17:46:25.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:25.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.346850+0000", 2024-09-04T17:46:25.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3875000000000002e-05 2024-09-04T17:46:25.823 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.823 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:25.823 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.823 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:4881 3.4 e31/24 3:25597cfa:::rbd_data.1134cfbce83d.0000000000000006:head v 31'28, mlcod=31'26)", 2024-09-04T17:46:25.823 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:07.381336+0000", 2024-09-04T17:46:25.823 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.428328242999999, 2024-09-04T17:46:25.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013330164, 2024-09-04T17:46:25.823 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:25.823 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:25.823 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:25.824 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:25.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:25.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.381336+0000", 2024-09-04T17:46:25.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:25.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.381336+0000", 2024-09-04T17:46:25.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:25.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.381339+0000", 2024-09-04T17:46:25.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.287e-06 2024-09-04T17:46:25.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:25.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.384162+0000", 2024-09-04T17:46:25.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028228649999999999 2024-09-04T17:46:25.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:25.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.384209+0000", 2024-09-04T17:46:25.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6768999999999999e-05 2024-09-04T17:46:25.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:25.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.384225+0000", 2024-09-04T17:46:25.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5987000000000001e-05 2024-09-04T17:46:25.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:25.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.384372+0000", 2024-09-04T17:46:25.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000146809 2024-09-04T17:46:25.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:25.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.384409+0000", 2024-09-04T17:46:25.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6909999999999997e-05 2024-09-04T17:46:25.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:25.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.394535+0000", 2024-09-04T17:46:25.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010126575000000001 2024-09-04T17:46:25.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:25.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.394666+0000", 2024-09-04T17:46:25.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013096200000000001 2024-09-04T17:46:25.827 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.827 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:25.827 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.828 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:4884 3.4 e31/24 3:25597cfa:::rbd_data.1134cfbce83d.0000000000000006:head v 31'30, mlcod=31'26)", 2024-09-04T17:46:25.828 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:07.386970+0000", 2024-09-04T17:46:25.828 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.422694309000001, 2024-09-04T17:46:25.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0088306779999999998, 2024-09-04T17:46:25.828 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:25.828 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:25.828 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:25.828 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:25.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:25.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.386970+0000", 2024-09-04T17:46:25.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:25.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.386970+0000", 2024-09-04T17:46:25.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:25.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.386973+0000", 2024-09-04T17:46:25.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9229999999999998e-06 2024-09-04T17:46:25.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:25.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.386993+0000", 2024-09-04T17:46:25.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.054e-05 2024-09-04T17:46:25.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:25.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.387029+0000", 2024-09-04T17:46:25.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5753000000000001e-05 2024-09-04T17:46:25.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:25.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.387051+0000", 2024-09-04T17:46:25.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1552999999999998e-05 2024-09-04T17:46:25.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:25.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.388844+0000", 2024-09-04T17:46:25.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001793712 2024-09-04T17:46:25.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:25.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.388866+0000", 2024-09-04T17:46:25.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1123000000000001e-05 2024-09-04T17:46:25.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:25.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.395711+0000", 2024-09-04T17:46:25.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0068458859999999998 2024-09-04T17:46:25.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:25.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.395801+0000", 2024-09-04T17:46:25.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9188000000000001e-05 2024-09-04T17:46:25.832 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.832 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:25.832 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.832 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:4997 3.9 e31/24 3:940f1d52:::rbd_data.1134cfbce83d.0000000000000010:head v 31'89, mlcod=30'85)", 2024-09-04T17:46:25.832 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:07.400336+0000", 2024-09-04T17:46:25.832 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.409328657, 2024-09-04T17:46:25.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003411533, 2024-09-04T17:46:25.833 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:25.833 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:25.833 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:25.833 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:25.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:25.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.400336+0000", 2024-09-04T17:46:25.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:25.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.400336+0000", 2024-09-04T17:46:25.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:25.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.400338+0000", 2024-09-04T17:46:25.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3489999999999999e-06 2024-09-04T17:46:25.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:25.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.400353+0000", 2024-09-04T17:46:25.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5091e-05 2024-09-04T17:46:25.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.835 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:25.835 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.400379+0000", 2024-09-04T17:46:25.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5528999999999999e-05 2024-09-04T17:46:25.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.835 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:25.835 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.400407+0000", 2024-09-04T17:46:25.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8104000000000001e-05 2024-09-04T17:46:25.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.835 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:25.835 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.400502+0000", 2024-09-04T17:46:25.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5402999999999994e-05 2024-09-04T17:46:25.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:25.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.400519+0000", 2024-09-04T17:46:25.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7329e-05 2024-09-04T17:46:25.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:25.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.403698+0000", 2024-09-04T17:46:25.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031787270000000001 2024-09-04T17:46:25.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:25.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.403747+0000", 2024-09-04T17:46:25.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9001000000000001e-05 2024-09-04T17:46:25.837 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.837 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:25.837 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.837 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:4999 3.4 e31/24 3:207aa2a5:::rbd_data.1134cfbce83d.0000000000000011:head v 31'32, mlcod=31'28)", 2024-09-04T17:46:25.837 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:07.401780+0000", 2024-09-04T17:46:25.837 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.407884696, 2024-09-04T17:46:25.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020640929999999999, 2024-09-04T17:46:25.837 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:25.837 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:25.837 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:25.837 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:25.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:25.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.401780+0000", 2024-09-04T17:46:25.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:25.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.401780+0000", 2024-09-04T17:46:25.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:25.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.401781+0000", 2024-09-04T17:46:25.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4950000000000001e-06 2024-09-04T17:46:25.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:25.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.401791+0000", 2024-09-04T17:46:25.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0284e-05 2024-09-04T17:46:25.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:25.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.401811+0000", 2024-09-04T17:46:25.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9171e-05 2024-09-04T17:46:25.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:25.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.401824+0000", 2024-09-04T17:46:25.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3611999999999999e-05 2024-09-04T17:46:25.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:25.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.401897+0000", 2024-09-04T17:46:25.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2707999999999996e-05 2024-09-04T17:46:25.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:25.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.401920+0000", 2024-09-04T17:46:25.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3496999999999999e-05 2024-09-04T17:46:25.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:25.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.403808+0000", 2024-09-04T17:46:25.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0018875179999999999 2024-09-04T17:46:25.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:25.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.403844+0000", 2024-09-04T17:46:25.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5808e-05 2024-09-04T17:46:25.841 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.841 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:25.841 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.842 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:5000 3.9 e31/24 3:94906c79:::rbd_data.1134cfbce83d.0000000000000012:head v 31'91, mlcod=30'85)", 2024-09-04T17:46:25.842 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:07.402507+0000", 2024-09-04T17:46:25.842 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.407156928999999, 2024-09-04T17:46:25.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004039735, 2024-09-04T17:46:25.842 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:25.842 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:25.842 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:25.842 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:25.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:25.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.402507+0000", 2024-09-04T17:46:25.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:25.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.402507+0000", 2024-09-04T17:46:25.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:25.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.402509+0000", 2024-09-04T17:46:25.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5939999999999999e-06 2024-09-04T17:46:25.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:25.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.402521+0000", 2024-09-04T17:46:25.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1786e-05 2024-09-04T17:46:25.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:25.844 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.402543+0000", 2024-09-04T17:46:25.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2158e-05 2024-09-04T17:46:25.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:25.844 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.402555+0000", 2024-09-04T17:46:25.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2034e-05 2024-09-04T17:46:25.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:25.844 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.402660+0000", 2024-09-04T17:46:25.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010535900000000001 2024-09-04T17:46:25.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:25.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.402696+0000", 2024-09-04T17:46:25.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5488000000000003e-05 2024-09-04T17:46:25.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:25.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.406510+0000", 2024-09-04T17:46:25.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0038142390000000001 2024-09-04T17:46:25.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:25.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.406547+0000", 2024-09-04T17:46:25.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7076999999999998e-05 2024-09-04T17:46:25.846 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.846 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:25.846 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.846 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:5834 3.9 e32/24 3:952cd39c:::rbd_header.11343d768295:head v 32'92, mlcod=31'89)", 2024-09-04T17:46:25.846 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.339000+0000", 2024-09-04T17:46:25.846 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.470664716000002, 2024-09-04T17:46:25.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020947890000000001, 2024-09-04T17:46:25.846 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:25.847 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:25.847 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:25.847 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:25.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:25.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.339000+0000", 2024-09-04T17:46:25.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:25.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.339000+0000", 2024-09-04T17:46:25.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.848 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:25.848 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.339000+0000", 2024-09-04T17:46:25.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8299999999999997e-07 2024-09-04T17:46:25.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.848 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:25.848 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.339009+0000", 2024-09-04T17:46:25.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6300000000000004e-06 2024-09-04T17:46:25.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.848 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:25.848 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.339038+0000", 2024-09-04T17:46:25.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9442e-05 2024-09-04T17:46:25.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:25.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.339053+0000", 2024-09-04T17:46:25.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.503e-05 2024-09-04T17:46:25.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:25.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.339212+0000", 2024-09-04T17:46:25.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015852600000000001 2024-09-04T17:46:25.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:25.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.339277+0000", 2024-09-04T17:46:25.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5025999999999997e-05 2024-09-04T17:46:25.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:25.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.340983+0000", 2024-09-04T17:46:25.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0017062170000000001 2024-09-04T17:46:25.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:25.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.341094+0000", 2024-09-04T17:46:25.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011143500000000001 2024-09-04T17:46:25.853 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.853 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:25.853 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.854 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:6311 3.9 e32/24 3:941d6897:::rbd_data.113431586065.0000000000000008:head v 32'97, mlcod=32'95)", 2024-09-04T17:46:25.854 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.479986+0000", 2024-09-04T17:46:25.854 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.329678032, 2024-09-04T17:46:25.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018493539, 2024-09-04T17:46:25.854 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:25.854 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:25.854 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:25.854 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:25.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.854 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:25.854 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.479986+0000", 2024-09-04T17:46:25.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.855 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:25.855 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.479986+0000", 2024-09-04T17:46:25.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.855 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:25.855 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.479988+0000", 2024-09-04T17:46:25.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.193e-06 2024-09-04T17:46:25.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.855 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:25.855 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484109+0000", 2024-09-04T17:46:25.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041201650000000003 2024-09-04T17:46:25.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:25.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484139+0000", 2024-09-04T17:46:25.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.082e-05 2024-09-04T17:46:25.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:25.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484153+0000", 2024-09-04T17:46:25.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3456e-05 2024-09-04T17:46:25.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:25.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484250+0000", 2024-09-04T17:46:25.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7638000000000003e-05 2024-09-04T17:46:25.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.857 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:25.857 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484282+0000", 2024-09-04T17:46:25.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1511999999999999e-05 2024-09-04T17:46:25.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.857 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:25.857 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.498411+0000", 2024-09-04T17:46:25.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014128870999999999 2024-09-04T17:46:25.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.857 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:25.858 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.498480+0000", 2024-09-04T17:46:25.858 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8884e-05 2024-09-04T17:46:25.858 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.858 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:25.858 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.858 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.858 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:6312 3.9 e32/24 3:941d6897:::rbd_data.113431586065.0000000000000008:head v 32'99, mlcod=32'95)", 2024-09-04T17:46:25.858 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.486036+0000", 2024-09-04T17:46:25.858 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.323627953999999, 2024-09-04T17:46:25.858 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013432773, 2024-09-04T17:46:25.858 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:25.858 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:25.858 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:25.859 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:25.859 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.859 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:25.859 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.486036+0000", 2024-09-04T17:46:25.859 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.859 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.859 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:25.859 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.486036+0000", 2024-09-04T17:46:25.859 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.859 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.859 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:25.859 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.486038+0000", 2024-09-04T17:46:25.860 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.536e-06 2024-09-04T17:46:25.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.860 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:25.860 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.486062+0000", 2024-09-04T17:46:25.860 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3672e-05 2024-09-04T17:46:25.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.860 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:25.860 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.486078+0000", 2024-09-04T17:46:25.860 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.613e-05 2024-09-04T17:46:25.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.860 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:25.861 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.486086+0000", 2024-09-04T17:46:25.861 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8849999999999999e-06 2024-09-04T17:46:25.861 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.861 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.861 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:25.861 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.491843+0000", 2024-09-04T17:46:25.861 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057572040000000001 2024-09-04T17:46:25.861 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.861 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.861 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:25.861 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.491876+0000", 2024-09-04T17:46:25.861 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3515e-05 2024-09-04T17:46:25.861 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.861 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.862 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:25.862 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.499410+0000", 2024-09-04T17:46:25.862 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0075333780000000003 2024-09-04T17:46:25.862 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.862 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.862 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:25.862 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.499469+0000", 2024-09-04T17:46:25.862 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9453000000000002e-05 2024-09-04T17:46:25.862 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.862 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:25.862 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.862 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.862 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.863 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:12583 3.1f e39/37 3:fb91fe3f:::rbd_object_map.1134ef3a5a7d:head v 39'106, mlcod=39'102)", 2024-09-04T17:46:25.863 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:15.954581+0000", 2024-09-04T17:46:25.863 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.8550828330000009, 2024-09-04T17:46:25.863 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002257897, 2024-09-04T17:46:25.863 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:25.863 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:25.863 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:25.863 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:25.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.863 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:25.863 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:15.954581+0000", 2024-09-04T17:46:25.863 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.864 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:25.864 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:15.954581+0000", 2024-09-04T17:46:25.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.864 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:25.864 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:15.954583+0000", 2024-09-04T17:46:25.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.939e-06 2024-09-04T17:46:25.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.864 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:25.864 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:15.954596+0000", 2024-09-04T17:46:25.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2576e-05 2024-09-04T17:46:25.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.865 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.865 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:25.865 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:15.954632+0000", 2024-09-04T17:46:25.865 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6044999999999998e-05 2024-09-04T17:46:25.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.865 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.865 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:25.865 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:15.954655+0000", 2024-09-04T17:46:25.865 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2753999999999999e-05 2024-09-04T17:46:25.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.865 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.865 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:25.865 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:15.954729+0000", 2024-09-04T17:46:25.865 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4575999999999999e-05 2024-09-04T17:46:25.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.866 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.866 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:25.866 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:15.954745+0000", 2024-09-04T17:46:25.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5221e-05 2024-09-04T17:46:25.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.866 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.866 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:25.866 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:15.956760+0000", 2024-09-04T17:46:25.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002015209 2024-09-04T17:46:25.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.866 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.866 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:25.866 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:15.956839+0000", 2024-09-04T17:46:25.867 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9577000000000001e-05 2024-09-04T17:46:25.867 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.867 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:25.867 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.867 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.867 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.867 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:12734 3.0 e40/24 3:042a5eed:::rbd_data.1134ef3a5a7d.00000000000000e3:head v 40'22, mlcod=39'19)", 2024-09-04T17:46:25.867 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:16.432897+0000", 2024-09-04T17:46:25.867 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.3767669829999996, 2024-09-04T17:46:25.867 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002600261, 2024-09-04T17:46:25.867 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:25.867 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:25.867 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:25.867 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:25.868 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.868 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:25.868 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:16.432897+0000", 2024-09-04T17:46:25.868 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.868 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.868 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.868 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:25.868 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:16.432897+0000", 2024-09-04T17:46:25.868 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.868 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.868 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.868 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:25.868 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:16.432901+0000", 2024-09-04T17:46:25.868 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3239999999999999e-06 2024-09-04T17:46:25.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.869 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.869 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:25.869 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:16.432917+0000", 2024-09-04T17:46:25.869 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6801000000000001e-05 2024-09-04T17:46:25.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.869 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.869 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:25.869 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:16.432952+0000", 2024-09-04T17:46:25.869 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4348999999999999e-05 2024-09-04T17:46:25.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.869 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.869 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:25.869 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:16.432972+0000", 2024-09-04T17:46:25.870 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9845000000000001e-05 2024-09-04T17:46:25.870 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.870 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.870 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:25.870 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:16.433130+0000", 2024-09-04T17:46:25.870 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000158414 2024-09-04T17:46:25.870 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.870 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.870 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:25.870 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:16.433182+0000", 2024-09-04T17:46:25.870 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1891000000000002e-05 2024-09-04T17:46:25.870 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.870 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.870 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:25.871 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:16.435370+0000", 2024-09-04T17:46:25.871 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0021883850000000002 2024-09-04T17:46:25.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.871 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.871 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:25.871 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:16.435498+0000", 2024-09-04T17:46:25.871 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012725200000000001 2024-09-04T17:46:25.871 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.871 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:25.871 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.871 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.871 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:12747 3.1f e41/37 3:fb91fe3f:::rbd_object_map.1134ef3a5a7d:head v 41'125, mlcod=41'122)", 2024-09-04T17:46:25.871 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:17.330322+0000", 2024-09-04T17:46:25.872 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.4793425029999998, 2024-09-04T17:46:25.872 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004205762, 2024-09-04T17:46:25.872 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:25.872 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:25.872 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:25.872 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:25.872 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.872 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:25.872 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.330322+0000", 2024-09-04T17:46:25.872 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.872 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.872 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.872 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:25.872 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.330322+0000", 2024-09-04T17:46:25.873 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.873 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.873 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:25.873 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.330323+0000", 2024-09-04T17:46:25.873 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9500000000000001e-07 2024-09-04T17:46:25.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.873 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.873 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:25.873 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.330329+0000", 2024-09-04T17:46:25.873 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2040000000000002e-06 2024-09-04T17:46:25.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.873 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.873 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:25.874 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.330341+0000", 2024-09-04T17:46:25.874 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1727e-05 2024-09-04T17:46:25.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.874 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:25.874 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.330348+0000", 2024-09-04T17:46:25.874 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.187e-06 2024-09-04T17:46:25.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.874 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:25.874 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.330455+0000", 2024-09-04T17:46:25.874 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107311 2024-09-04T17:46:25.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.875 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:25.875 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.330495+0000", 2024-09-04T17:46:25.875 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0191999999999997e-05 2024-09-04T17:46:25.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.875 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.875 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:25.875 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.334480+0000", 2024-09-04T17:46:25.875 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039848410000000003 2024-09-04T17:46:25.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.875 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.875 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:25.875 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.334528+0000", 2024-09-04T17:46:25.875 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7404999999999998e-05 2024-09-04T17:46:25.875 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.876 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:25.876 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.876 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.876 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.876 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:12748 3.1f e41/37 3:fb91fe3f:::rbd_object_map.1134ef3a5a7d:head v 41'126, mlcod=41'122)", 2024-09-04T17:46:25.876 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:17.332613+0000", 2024-09-04T17:46:25.876 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.4770515240000002, 2024-09-04T17:46:25.876 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0044275870000000002, 2024-09-04T17:46:25.876 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:25.876 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:25.876 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:25.876 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:25.876 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.876 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:25.877 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.332613+0000", 2024-09-04T17:46:25.877 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.877 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.877 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.877 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:25.877 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.332613+0000", 2024-09-04T17:46:25.877 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.877 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.877 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.877 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:25.877 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.332614+0000", 2024-09-04T17:46:25.877 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.051e-06 2024-09-04T17:46:25.877 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.877 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.878 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:25.878 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.332620+0000", 2024-09-04T17:46:25.878 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1680000000000001e-06 2024-09-04T17:46:25.878 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.878 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.878 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:25.878 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.332635+0000", 2024-09-04T17:46:25.878 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5543e-05 2024-09-04T17:46:25.878 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.878 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.878 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:25.878 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.332643+0000", 2024-09-04T17:46:25.879 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.537e-06 2024-09-04T17:46:25.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.879 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.879 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:25.879 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.332717+0000", 2024-09-04T17:46:25.879 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3701999999999993e-05 2024-09-04T17:46:25.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.879 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.879 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:25.879 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.332786+0000", 2024-09-04T17:46:25.879 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9669e-05 2024-09-04T17:46:25.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.879 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.879 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:25.880 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.336923+0000", 2024-09-04T17:46:25.880 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041365259999999997 2024-09-04T17:46:25.880 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.880 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.880 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:25.880 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.337040+0000", 2024-09-04T17:46:25.880 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000117391 2024-09-04T17:46:25.880 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.880 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:25.880 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.880 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.880 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.880 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:12749 3.1f e41/37 3:fb91fe3f:::rbd_object_map.1134ef3a5a7d:head v 41'127, mlcod=41'122)", 2024-09-04T17:46:25.881 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:17.335529+0000", 2024-09-04T17:46:25.881 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.4741354829999995, 2024-09-04T17:46:25.881 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052795899999999998, 2024-09-04T17:46:25.881 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:25.881 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:25.881 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:25.881 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:25.881 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.881 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:25.881 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.335529+0000", 2024-09-04T17:46:25.881 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.881 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.881 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.881 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:25.882 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.335529+0000", 2024-09-04T17:46:25.882 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.882 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:25.882 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.335535+0000", 2024-09-04T17:46:25.882 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4060000000000002e-06 2024-09-04T17:46:25.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.882 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:25.882 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.335546+0000", 2024-09-04T17:46:25.882 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0652000000000001e-05 2024-09-04T17:46:25.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.883 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:25.883 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.335571+0000", 2024-09-04T17:46:25.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5148e-05 2024-09-04T17:46:25.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.883 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.883 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:25.883 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.335583+0000", 2024-09-04T17:46:25.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2315e-05 2024-09-04T17:46:25.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.883 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.883 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:25.883 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.335699+0000", 2024-09-04T17:46:25.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000115443 2024-09-04T17:46:25.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.884 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:25.884 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.335736+0000", 2024-09-04T17:46:25.884 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6779999999999997e-05 2024-09-04T17:46:25.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.884 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:25.884 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.340722+0000", 2024-09-04T17:46:25.884 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0049860390000000003 2024-09-04T17:46:25.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.884 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:25.884 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.340808+0000", 2024-09-04T17:46:25.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6806999999999997e-05 2024-09-04T17:46:25.885 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.885 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:25.885 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.885 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.885 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.885 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:12750 3.1f e41/37 3:fb91fe3f:::rbd_object_map.1134ef3a5a7d:head v 41'128, mlcod=41'123)", 2024-09-04T17:46:25.885 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:17.337581+0000", 2024-09-04T17:46:25.885 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.4720829389999999, 2024-09-04T17:46:25.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045169579999999997, 2024-09-04T17:46:25.885 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:25.885 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:25.886 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:25.886 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:25.886 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.886 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:25.886 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.337581+0000", 2024-09-04T17:46:25.886 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.886 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.886 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.886 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:25.886 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.337581+0000", 2024-09-04T17:46:25.886 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.886 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.886 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.886 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:25.887 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.337583+0000", 2024-09-04T17:46:25.887 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5200000000000001e-06 2024-09-04T17:46:25.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.887 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:25.887 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.337595+0000", 2024-09-04T17:46:25.887 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2395000000000001e-05 2024-09-04T17:46:25.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.887 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:25.887 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.337620+0000", 2024-09-04T17:46:25.887 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4794e-05 2024-09-04T17:46:25.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.888 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:25.888 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.337634+0000", 2024-09-04T17:46:25.888 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4241e-05 2024-09-04T17:46:25.888 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.888 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.888 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:25.888 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.337731+0000", 2024-09-04T17:46:25.888 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6246e-05 2024-09-04T17:46:25.888 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.888 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.888 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:25.888 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.337762+0000", 2024-09-04T17:46:25.888 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1279999999999999e-05 2024-09-04T17:46:25.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.889 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.889 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:25.889 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.342050+0000", 2024-09-04T17:46:25.889 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0042879729999999996 2024-09-04T17:46:25.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.889 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.889 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:25.889 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.342098+0000", 2024-09-04T17:46:25.889 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8508999999999999e-05 2024-09-04T17:46:25.889 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.889 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:25.889 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.889 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.890 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:12756 3.0 e41/24 3:0074e6f8:::rbd_data.1134ef3a5a7d.000000000000000e:head v 41'24, mlcod=40'20)", 2024-09-04T17:46:25.890 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:17.339316+0000", 2024-09-04T17:46:25.890 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.4703487160000002, 2024-09-04T17:46:25.890 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052778790000000001, 2024-09-04T17:46:25.890 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:25.890 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:25.890 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:25.890 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:25.890 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.890 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:25.890 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.339316+0000", 2024-09-04T17:46:25.890 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.890 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.891 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:25.891 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.339316+0000", 2024-09-04T17:46:25.891 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.891 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.891 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:25.891 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.339317+0000", 2024-09-04T17:46:25.891 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.13e-06 2024-09-04T17:46:25.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.891 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.891 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:25.891 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.339328+0000", 2024-09-04T17:46:25.891 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1742e-05 2024-09-04T17:46:25.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.892 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.892 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:25.892 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.339358+0000", 2024-09-04T17:46:25.892 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9149000000000002e-05 2024-09-04T17:46:25.892 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.892 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.892 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:25.892 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.339379+0000", 2024-09-04T17:46:25.892 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1288e-05 2024-09-04T17:46:25.892 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.892 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.892 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:25.892 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.339454+0000", 2024-09-04T17:46:25.892 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4703000000000006e-05 2024-09-04T17:46:25.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.893 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.893 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:25.893 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.339506+0000", 2024-09-04T17:46:25.893 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2188999999999998e-05 2024-09-04T17:46:25.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.893 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.893 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:25.893 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.344475+0000", 2024-09-04T17:46:25.893 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0049691919999999999 2024-09-04T17:46:25.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.893 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.893 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:25.893 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.344593+0000", 2024-09-04T17:46:25.894 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000118486 2024-09-04T17:46:25.894 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.894 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:25.894 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.894 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:20372 3.9 e43/24 3:928af84f:::rbd_data.1134fdde05d4.0000000000000146:head v 43'161, mlcod=43'159)", 2024-09-04T17:46:25.894 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:22.045430+0000", 2024-09-04T17:46:25.894 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.7642344039999998, 2024-09-04T17:46:25.894 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025192280000000001, 2024-09-04T17:46:25.894 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:25.894 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:25.895 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:25.895 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:25.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:25.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:22.045430+0000", 2024-09-04T17:46:25.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:25.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:22.045430+0000", 2024-09-04T17:46:25.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:25.896 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:22.045432+0000", 2024-09-04T17:46:25.896 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.21e-06 2024-09-04T17:46:25.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.896 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:25.896 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:22.046092+0000", 2024-09-04T17:46:25.896 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00065987100000000003 2024-09-04T17:46:25.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.896 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:25.896 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:22.046109+0000", 2024-09-04T17:46:25.896 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6948000000000001e-05 2024-09-04T17:46:25.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.897 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:25.897 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:22.046116+0000", 2024-09-04T17:46:25.897 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5680000000000002e-06 2024-09-04T17:46:25.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.897 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.897 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:25.897 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:22.046249+0000", 2024-09-04T17:46:25.897 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013281999999999999 2024-09-04T17:46:25.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.897 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.897 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:25.897 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:22.046314+0000", 2024-09-04T17:46:25.897 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4757000000000004e-05 2024-09-04T17:46:25.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.898 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:25.898 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:22.047866+0000", 2024-09-04T17:46:25.898 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0015522050000000001 2024-09-04T17:46:25.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.898 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:25.898 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:22.047949+0000", 2024-09-04T17:46:25.898 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2849000000000006e-05 2024-09-04T17:46:25.898 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.898 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:25.898 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.899 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:20639 3.4 e43/24 3:2371624a:::rbd_data.1134fdde05d4.000000000000000a:head v 43'69, mlcod=43'67)", 2024-09-04T17:46:25.899 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:22.104708+0000", 2024-09-04T17:46:25.899 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.704955854, 2024-09-04T17:46:25.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0030909819999999999, 2024-09-04T17:46:25.899 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:25.899 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:25.899 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:25.899 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:25.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.899 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:25.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:22.104708+0000", 2024-09-04T17:46:25.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.900 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:25.900 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:22.104708+0000", 2024-09-04T17:46:25.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:25.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.900 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:25.900 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:22.104712+0000", 2024-09-04T17:46:25.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0099999999999997e-06 2024-09-04T17:46:25.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:25.901 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:22.105167+0000", 2024-09-04T17:46:25.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00045474900000000001 2024-09-04T17:46:25.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:25.901 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:22.105215+0000", 2024-09-04T17:46:25.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7447999999999997e-05 2024-09-04T17:46:25.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:25.901 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:22.105238+0000", 2024-09-04T17:46:25.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3408e-05 2024-09-04T17:46:25.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.902 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:25.902 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:22.105432+0000", 2024-09-04T17:46:25.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019435500000000001 2024-09-04T17:46:25.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.902 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:25.902 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:22.105474+0000", 2024-09-04T17:46:25.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1295999999999999e-05 2024-09-04T17:46:25.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.902 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:25.902 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:22.107451+0000", 2024-09-04T17:46:25.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001977548 2024-09-04T17:46:25.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:25.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:25.903 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:25.903 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:22.107799+0000", 2024-09-04T17:46:25.903 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00034816799999999999 2024-09-04T17:46:25.903 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.903 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:25.903 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.903 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:25.903 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:25.903 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:25.904 INFO:teuthology.orchestra.run.smithi092.stdout:1461 write 0xd5b4ea9 thru 0xd5b7243 (0x239b bytes) 2024-09-04T17:46:25.904 INFO:teuthology.orchestra.run.smithi092.stdout:1462 write 0x929879f thru 0x92a0322 (0x7b84 bytes) 2024-09-04T17:46:25.905 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-04T17:46:25.906 INFO:teuthology.orchestra.run.smithi092.stdout:1463 trunc from 0xd5b7244 to 0x65b8681 2024-09-04T17:46:25.906 INFO:teuthology.orchestra.run.smithi092.stdout:1464 write 0xa891c4e thru 0xa89ee3c (0xd1ef bytes) 2024-09-04T17:46:25.906 INFO:teuthology.orchestra.run.smithi092.stdout:1465 write 0x963084f thru 0x963c64c (0xbdfe bytes) 2024-09-04T17:46:25.906 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:25.907 INFO:teuthology.orchestra.run.smithi092.stdout:1467 write 0xdc36d5a thru 0xdc45bd4 (0xee7b bytes) 2024-09-04T17:46:25.907 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:25.907 INFO:teuthology.orchestra.run.smithi092.stdout:1471 read 0x5d4c110 thru 0x5d520dc (0x5fcd bytes) 2024-09-04T17:46:25.907 INFO:teuthology.orchestra.run.smithi092.stdout:1472 write 0xd3ec07c thru 0xd3fba1c (0xf9a1 bytes) 2024-09-04T17:46:25.907 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:25.903+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:25.908 INFO:teuthology.orchestra.run.smithi092.stdout:1474 punch from 0x2ce506e to 0x2cf0dff, (0xbd91 bytes) 2024-09-04T17:46:25.908 INFO:teuthology.orchestra.run.smithi092.stdout:1475 write 0x9d9f6f1 thru 0x9da4c0a (0x551a bytes) 2024-09-04T17:46:25.914 INFO:teuthology.orchestra.run.smithi092.stdout:1476 read 0x534d41a thru 0x535bd8e (0xe975 bytes) 2024-09-04T17:46:25.915 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:25.915 INFO:teuthology.orchestra.run.smithi092.stdout:1480 read 0xb1ec9fc thru 0xb1fa81d (0xde22 bytes) 2024-09-04T17:46:25.915 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:25.915 INFO:teuthology.orchestra.run.smithi092.stdout:1483 writesame 0x5349b63 thru 0x5353708 data_size 0x57(0x9ba6 bytes) 2024-09-04T17:46:25.919 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:25.920 INFO:teuthology.orchestra.run.smithi092.stdout:1486 read 0xc54b1d4 thru 0xc552935 (0x7762 bytes) 2024-09-04T17:46:25.920 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:25.920 INFO:teuthology.orchestra.run.smithi092.stdout:1491 punch from 0xa98c404 to 0xa990848, (0x4444 bytes) 2024-09-04T17:46:25.920 INFO:teuthology.orchestra.run.smithi092.stdout:1492 write 0x56becbc thru 0x56cc7a4 (0xdae9 bytes) 2024-09-04T17:46:25.922 INFO:teuthology.orchestra.run.smithi092.stdout:1493 writesame 0x1244688 thru 0x1247448 data_size 0x11(0x2dc1 bytes) 2024-09-04T17:46:25.924 INFO:teuthology.orchestra.run.smithi092.stdout:1495 read 0xdab1c5 thru 0xdb08e7 (0x5723 bytes) 2024-09-04T17:46:25.925 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:25.925 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:25.925 INFO:teuthology.orchestra.run.smithi092.stdout:1499 write 0xbdd02c2 thru 0xbdd3713 (0x3452 bytes) 2024-09-04T17:46:25.926 INFO:teuthology.orchestra.run.smithi092.stdout:1500 write 0x81e4a5f thru 0x81f0163 (0xb705 bytes) 2024-09-04T17:46:25.929 INFO:teuthology.orchestra.run.smithi092.stdout:1502 write 0x6ccabe5 thru 0x6ccff4e (0x536a bytes) 2024-09-04T17:46:25.931 INFO:teuthology.orchestra.run.smithi092.stdout:1503 write 0x6fdb97f thru 0x6fde4c2 (0x2b44 bytes) 2024-09-04T17:46:25.933 INFO:teuthology.orchestra.run.smithi092.stdout:1504 read 0xa028a9f thru 0xa0373d0 (0xe932 bytes) 2024-09-04T17:46:25.933 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:25.933 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:25.933 INFO:teuthology.orchestra.run.smithi092.stdout:1509 writesame 0x9430e79 thru 0x9435353 data_size 0x1(0x44db bytes) 2024-09-04T17:46:25.952 INFO:teuthology.orchestra.run.smithi092.stdout:1512 read 0x238c608 thru 0x2398513 (0xbf0c bytes) 2024-09-04T17:46:25.953 INFO:teuthology.orchestra.run.smithi092.stdout:1513 write 0x16e745b thru 0x16f429a (0xce40 bytes) 2024-09-04T17:46:25.955 INFO:teuthology.orchestra.run.smithi092.stdout:1515 writesame 0x5d730d thru 0x5e6401 data_size 0xd(0xf0f5 bytes) 2024-09-04T17:46:25.962 INFO:teuthology.orchestra.run.smithi092.stdout:1516 read 0x563dcf8 thru 0x564aa0d (0xcd16 bytes) 2024-09-04T17:46:25.963 INFO:teuthology.orchestra.run.smithi092.stdout:1518 read 0x6806ec7 thru 0x6808b03 (0x1c3d bytes) 2024-09-04T17:46:25.963 INFO:teuthology.orchestra.run.smithi092.stdout:1519 write 0x1809919 thru 0x1809fe3 (0x6cb bytes) 2024-09-04T17:46:25.967 INFO:teuthology.orchestra.run.smithi092.stdout:1520 punch from 0x2ba0b9b to 0x2ba17bb, (0xc20 bytes) 2024-09-04T17:46:25.967 INFO:teuthology.orchestra.run.smithi092.stdout:1521 read 0x30ac78b thru 0x30b63d1 (0x9c47 bytes) 2024-09-04T17:46:25.968 INFO:teuthology.orchestra.run.smithi092.stdout:1524 write 0x4cc1639 thru 0x4cccd5d (0xb725 bytes) 2024-09-04T17:46:25.970 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:25.988 INFO:teuthology.orchestra.run.smithi092.stdout:1528 trunc from 0xdc45bd5 to 0x1f81e40 2024-09-04T17:46:26.006 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:26.003+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:26.012 INFO:teuthology.orchestra.run.smithi092.stdout:1529 punch from 0xeba819 to 0xebb042, (0x829 bytes) 2024-09-04T17:46:26.040 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:26.040 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:46:26.040 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:46:26.040 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:26.044 INFO:teuthology.orchestra.run.smithi092.stdout:1532 write 0xdd66026 thru 0xdd693fd (0x33d8 bytes) 2024-09-04T17:46:26.050 INFO:teuthology.orchestra.run.smithi092.stdout:1533 punch from 0x559e23a to 0x55a2eaa, (0x4c70 bytes) 2024-09-04T17:46:26.050 INFO:teuthology.orchestra.run.smithi092.stdout:1534 read 0x5b978ab thru 0x5ba3533 (0xbc89 bytes) 2024-09-04T17:46:26.051 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:26.056 INFO:teuthology.orchestra.run.smithi092.stdout:1538 trunc from 0xdd693fe to 0xc12ec23 2024-09-04T17:46:26.058 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-04T17:46:26.073 INFO:teuthology.orchestra.run.smithi092.stdout:1539 writesame 0x601378 thru 0x608421 data_size 0x72(0x70aa bytes) 2024-09-04T17:46:26.076 INFO:teuthology.orchestra.run.smithi092.stdout:1540 read 0xc08354a thru 0xc086468 (0x2f1f bytes) 2024-09-04T17:46:26.076 INFO:teuthology.orchestra.run.smithi092.stdout:1541 read 0xaae96d7 thru 0xaaf0eea (0x7814 bytes) 2024-09-04T17:46:26.098 INFO:teuthology.orchestra.run.smithi092.stdout:1544 trunc from 0xc12ec23 to 0x29094 2024-09-04T17:46:26.106 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:26.105+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:26.115 INFO:teuthology.orchestra.run.smithi092.stdout:1545 read 0x1c4d7 thru 0x1fc62 (0x378c bytes) 2024-09-04T17:46:26.122 INFO:teuthology.orchestra.run.smithi092.stdout:1546 trunc from 0x29094 to 0x31d72ed 2024-09-04T17:46:26.130 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:26.130 INFO:teuthology.orchestra.run.smithi092.stdout:1548 punch from 0x14f8a8e to 0x1500069, (0x75db bytes) 2024-09-04T17:46:26.130 INFO:teuthology.orchestra.run.smithi092.stdout:1549 punch from 0x224133a to 0x2243634, (0x22fa bytes) 2024-09-04T17:46:26.139 INFO:teuthology.orchestra.run.smithi092.stdout:1551 trunc from 0x31d72ed to 0x8048fcd 2024-09-04T17:46:26.146 INFO:teuthology.orchestra.run.smithi092.stdout:1552 writesame 0x32fb482 thru 0x33075f9 data_size 0x29(0xc178 bytes) 2024-09-04T17:46:26.167 INFO:teuthology.orchestra.run.smithi092.stdout:1553 writesame 0xc051639 thru 0xc06135d data_size 0xd(0xfd25 bytes) 2024-09-04T17:46:26.169 INFO:teuthology.orchestra.run.smithi092.stdout:1556 trunc from 0xc06135e to 0xcf1c655 2024-09-04T17:46:26.176 INFO:teuthology.orchestra.run.smithi092.stdout:1558 read 0x6d1110c thru 0x6d12af1 (0x19e6 bytes) 2024-09-04T17:46:26.177 INFO:teuthology.orchestra.run.smithi092.stdout:1561 read 0x3775711 thru 0x37834da (0xddca bytes) 2024-09-04T17:46:26.177 INFO:teuthology.orchestra.run.smithi092.stdout:1562 writesame 0xc982f8e thru 0xc98c08d data_size 0x74(0x9100 bytes) 2024-09-04T17:46:26.180 INFO:teuthology.orchestra.run.smithi092.stdout:1563 writesame 0x9af9bae thru 0x9b081d5 data_size 0x3c(0xe628 bytes) 2024-09-04T17:46:26.188 INFO:teuthology.orchestra.run.smithi092.stdout:1565 writesame 0x77cebd6 thru 0x77d486b data_size 0x7(0x5c96 bytes) 2024-09-04T17:46:26.193 INFO:teuthology.orchestra.run.smithi092.stdout:1566 writesame 0x60fc8dd thru 0x6102c99 data_size 0x3(0x63bd bytes) 2024-09-04T17:46:26.205 INFO:teuthology.orchestra.run.smithi092.stdout:1567 write 0x8ad87b3 thru 0x8adec60 (0x64ae bytes) 2024-09-04T17:46:26.206 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:26.205+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:26.209 INFO:teuthology.orchestra.run.smithi092.stdout:1568 write 0xbb78b5d thru 0xbb85db2 (0xd256 bytes) 2024-09-04T17:46:26.213 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:26.213 INFO:teuthology.orchestra.run.smithi092.stdout:1570 read 0xae2bc63 thru 0xae2d2af (0x164d bytes) 2024-09-04T17:46:26.213 INFO:teuthology.orchestra.run.smithi092.stdout:1571 read 0x3e9e994 thru 0x3ea0f91 (0x25fe bytes) 2024-09-04T17:46:26.213 INFO:teuthology.orchestra.run.smithi092.stdout:1572 write 0x6f117bd thru 0x6f1fd2b (0xe56f bytes) 2024-09-04T17:46:26.234 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:26.234 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:46:26.234 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:46:26.234 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:46:26.234 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:26.235 INFO:teuthology.orchestra.run.smithi092.stdout:1573 trunc from 0xcf1c655 to 0x6ccadf5 2024-09-04T17:46:26.249 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-04T17:46:26.256 INFO:teuthology.orchestra.run.smithi092.stdout:1574 punch from 0x5180eb1 to 0x5185052, (0x41a1 bytes) 2024-09-04T17:46:26.256 INFO:teuthology.orchestra.run.smithi092.stdout:1575 write 0x367ad8e thru 0x368805a (0xd2cd bytes) 2024-09-04T17:46:26.262 INFO:teuthology.orchestra.run.smithi092.stdout:1576 punch from 0x55b07da to 0x55b3f35, (0x375b bytes) 2024-09-04T17:46:26.262 INFO:teuthology.orchestra.run.smithi092.stdout:1577 read 0x3e18be9 thru 0x3e20000 (0x7418 bytes) 2024-09-04T17:46:26.263 INFO:teuthology.orchestra.run.smithi092.stdout:1578 read 0x26e826a thru 0x26ea3f4 (0x218b bytes) 2024-09-04T17:46:26.279 INFO:teuthology.orchestra.run.smithi092.stdout:1581 write 0x8e499d8 thru 0x8e4a44b (0xa74 bytes) 2024-09-04T17:46:26.302 INFO:teuthology.orchestra.run.smithi092.stdout:1583 trunc from 0x8e4a44c to 0x6810f8 2024-09-04T17:46:26.307 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:26.303+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:26.350 INFO:teuthology.orchestra.run.smithi092.stdout:1584 trunc from 0x6810f8 to 0xb3681ea 2024-09-04T17:46:26.363 INFO:teuthology.orchestra.run.smithi092.stdout:1585 writesame 0x56c7776 thru 0x56cfe03 data_size 0x3(0x868e bytes) 2024-09-04T17:46:26.378 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:26.379 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:46:26.379 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:46:26.379 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:46:26.379 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.379 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4264 3.9 3:970e51a0:::rbd_data.1134cfbce83d.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 1597545~615] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-04T17:46:26.379 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.021843+0000", 2024-09-04T17:46:26.379 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 19.355405816000001, 2024-09-04T17:46:26.379 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.015077662, 2024-09-04T17:46:26.379 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:26.379 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:26.379 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:26.379 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:26.380 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:26.380 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:26.380 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4264 2024-09-04T17:46:26.380 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.380 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:26.380 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.380 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:26.380 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.021843+0000", 2024-09-04T17:46:26.380 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:26.380 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.380 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.380 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:26.380 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.021843+0000", 2024-09-04T17:46:26.381 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:26.381 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.381 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.381 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:26.381 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.021845+0000", 2024-09-04T17:46:26.381 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.328e-06 2024-09-04T17:46:26.381 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.381 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.381 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:26.381 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.026201+0000", 2024-09-04T17:46:26.381 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.004355935 2024-09-04T17:46:26.381 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.382 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.382 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:26.382 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.026214+0000", 2024-09-04T17:46:26.382 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3315999999999999e-05 2024-09-04T17:46:26.382 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.382 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.382 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:26.382 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.026221+0000", 2024-09-04T17:46:26.383 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7070000000000003e-06 2024-09-04T17:46:26.383 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.383 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.383 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:26.383 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.026266+0000", 2024-09-04T17:46:26.383 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.5120999999999997e-05 2024-09-04T17:46:26.383 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.383 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.383 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:26.383 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.026375+0000", 2024-09-04T17:46:26.383 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00010935 2024-09-04T17:46:26.383 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.384 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.384 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 2", 2024-09-04T17:46:26.384 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.026610+0000", 2024-09-04T17:46:26.384 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00023530400000000001 2024-09-04T17:46:26.384 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.384 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.384 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:26.384 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.032096+0000", 2024-09-04T17:46:26.384 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0054859050000000001 2024-09-04T17:46:26.384 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.384 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.384 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:26.384 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.036851+0000", 2024-09-04T17:46:26.385 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0047543409999999996 2024-09-04T17:46:26.385 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.385 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.385 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:26.385 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.036890+0000", 2024-09-04T17:46:26.385 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.9128999999999998e-05 2024-09-04T17:46:26.385 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.385 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.385 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:26.385 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.036921+0000", 2024-09-04T17:46:26.385 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.1226000000000002e-05 2024-09-04T17:46:26.385 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:26.386 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:26.386 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:26.386 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.386 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.386 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4651 3.3 3:c43d9109:::rbd_data.113417f3df25.0000000000000002:5 [read 4063232~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:26.386 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.349383+0000", 2024-09-04T17:46:26.386 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 19.027865810000002, 2024-09-04T17:46:26.386 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.014661281, 2024-09-04T17:46:26.387 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:26.387 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:26.387 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:26.387 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:26.387 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:26.387 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:26.387 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4651 2024-09-04T17:46:26.387 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.387 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:26.387 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.387 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:26.387 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.349383+0000", 2024-09-04T17:46:26.387 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:26.388 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.388 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.388 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:26.388 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.349383+0000", 2024-09-04T17:46:26.388 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:26.388 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.388 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.388 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:26.388 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.349384+0000", 2024-09-04T17:46:26.388 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.4499999999999996e-07 2024-09-04T17:46:26.388 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.388 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.389 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:26.389 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.349401+0000", 2024-09-04T17:46:26.389 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6339000000000001e-05 2024-09-04T17:46:26.389 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.389 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.389 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:26.389 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.349411+0000", 2024-09-04T17:46:26.389 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0013000000000001e-05 2024-09-04T17:46:26.389 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.389 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.389 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:26.389 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.349433+0000", 2024-09-04T17:46:26.389 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1926999999999999e-05 2024-09-04T17:46:26.390 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.390 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.390 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:26.390 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.363824+0000", 2024-09-04T17:46:26.390 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.014391736 2024-09-04T17:46:26.390 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.390 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.390 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:26.390 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.363920+0000", 2024-09-04T17:46:26.390 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.6142000000000002e-05 2024-09-04T17:46:26.390 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.390 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.390 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:26.390 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.364045+0000", 2024-09-04T17:46:26.391 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00012427900000000001 2024-09-04T17:46:26.391 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:26.391 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:26.391 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:26.391 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.391 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.391 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4734 3.3 3:c43d9109:::rbd_data.113417f3df25.0000000000000002:5 [read 196608~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:26.391 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.349456+0000", 2024-09-04T17:46:26.391 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 19.027792743999999, 2024-09-04T17:46:26.391 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.014853676999999999, 2024-09-04T17:46:26.391 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:26.391 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:26.391 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:26.391 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:26.392 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:26.392 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:26.392 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4734 2024-09-04T17:46:26.392 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.392 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:26.392 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.392 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:26.392 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.349456+0000", 2024-09-04T17:46:26.392 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:26.392 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.392 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.392 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:26.392 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.349456+0000", 2024-09-04T17:46:26.393 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:26.393 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.393 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.393 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:26.393 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.349457+0000", 2024-09-04T17:46:26.393 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.9999999999999996e-07 2024-09-04T17:46:26.393 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.393 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.393 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:26.393 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.349461+0000", 2024-09-04T17:46:26.393 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.9099999999999998e-06 2024-09-04T17:46:26.393 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.393 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.394 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:26.394 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.349471+0000", 2024-09-04T17:46:26.394 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.8069999999999994e-06 2024-09-04T17:46:26.394 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.394 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.394 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:26.394 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.349479+0000", 2024-09-04T17:46:26.394 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.8769999999999993e-06 2024-09-04T17:46:26.394 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.395 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.395 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:26.395 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.364085+0000", 2024-09-04T17:46:26.395 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.014606022 2024-09-04T17:46:26.395 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.395 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.395 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:26.395 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.364184+0000", 2024-09-04T17:46:26.395 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.8948000000000002e-05 2024-09-04T17:46:26.395 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.395 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.395 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:26.395 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.364310+0000", 2024-09-04T17:46:26.395 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00012631299999999999 2024-09-04T17:46:26.396 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:26.396 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:26.396 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:26.396 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.396 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.396 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4735 3.3 3:c43d9109:::rbd_data.113417f3df25.0000000000000002:5 [read 622592~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:26.396 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.349490+0000", 2024-09-04T17:46:26.396 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 19.027759362000001, 2024-09-04T17:46:26.396 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.015069948999999999, 2024-09-04T17:46:26.396 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:26.396 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:26.396 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:26.397 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:26.397 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:26.397 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:26.397 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4735 2024-09-04T17:46:26.397 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.397 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:26.397 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.397 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:26.397 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.349490+0000", 2024-09-04T17:46:26.397 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:26.397 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.397 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.397 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:26.398 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.349490+0000", 2024-09-04T17:46:26.398 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:26.398 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.398 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.398 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:26.398 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.349491+0000", 2024-09-04T17:46:26.398 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6000000000000003e-07 2024-09-04T17:46:26.398 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.398 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.398 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:26.398 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.349493+0000", 2024-09-04T17:46:26.398 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.9349999999999999e-06 2024-09-04T17:46:26.398 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.399 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.399 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:26.399 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.349503+0000", 2024-09-04T17:46:26.399 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4460000000000007e-06 2024-09-04T17:46:26.399 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.399 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.399 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:26.399 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.349510+0000", 2024-09-04T17:46:26.399 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7109999999999998e-06 2024-09-04T17:46:26.399 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.399 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.399 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:26.400 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.364347+0000", 2024-09-04T17:46:26.400 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.014837250999999999 2024-09-04T17:46:26.400 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.400 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.400 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:26.400 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.364440+0000", 2024-09-04T17:46:26.400 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.3121000000000005e-05 2024-09-04T17:46:26.400 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.400 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.400 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:26.400 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.364560+0000", 2024-09-04T17:46:26.400 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000119825 2024-09-04T17:46:26.400 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:26.401 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:26.401 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:26.401 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.401 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.401 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4811 3.3 3:c43d9109:::rbd_data.113417f3df25.0000000000000002:5 [read 3244032~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:26.401 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.355350+0000", 2024-09-04T17:46:26.401 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 19.021899512000001, 2024-09-04T17:46:26.401 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.015364724, 2024-09-04T17:46:26.401 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:26.401 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:26.401 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:26.401 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:26.401 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:26.401 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:26.402 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4811 2024-09-04T17:46:26.402 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.402 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:26.402 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.402 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:26.402 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.355350+0000", 2024-09-04T17:46:26.402 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:26.402 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.402 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.402 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:26.402 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.355350+0000", 2024-09-04T17:46:26.402 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:26.403 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.403 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.403 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:26.403 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.355350+0000", 2024-09-04T17:46:26.403 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.9400000000000005e-07 2024-09-04T17:46:26.403 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.403 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.403 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:26.403 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.355354+0000", 2024-09-04T17:46:26.403 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.8600000000000003e-06 2024-09-04T17:46:26.403 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.403 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.403 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:26.403 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.355363+0000", 2024-09-04T17:46:26.404 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.0739999999999999e-06 2024-09-04T17:46:26.404 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.404 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.404 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:26.404 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.355370+0000", 2024-09-04T17:46:26.404 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2370000000000003e-06 2024-09-04T17:46:26.404 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.404 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.404 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:26.404 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.370484+0000", 2024-09-04T17:46:26.404 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.015113909 2024-09-04T17:46:26.404 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.405 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.405 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:26.405 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.370584+0000", 2024-09-04T17:46:26.405 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000100042 2024-09-04T17:46:26.405 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.405 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.405 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:26.405 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.370714+0000", 2024-09-04T17:46:26.405 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013000800000000001 2024-09-04T17:46:26.405 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:26.405 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:26.405 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:26.405 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.406 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.406 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4812 3.3 3:c43d9109:::rbd_data.113417f3df25.0000000000000002:5 [read 3670016~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:26.406 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.355380+0000", 2024-09-04T17:46:26.406 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 19.021869365000001, 2024-09-04T17:46:26.406 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.015590292, 2024-09-04T17:46:26.406 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:26.406 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:26.406 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:26.406 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:26.406 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:26.406 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:26.406 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4812 2024-09-04T17:46:26.406 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.406 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:26.407 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.407 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:26.407 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.355380+0000", 2024-09-04T17:46:26.407 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:26.407 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.407 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.407 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:26.407 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.355380+0000", 2024-09-04T17:46:26.407 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:26.407 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.407 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.407 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:26.407 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.355381+0000", 2024-09-04T17:46:26.408 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.3799999999999997e-07 2024-09-04T17:46:26.408 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.408 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.408 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:26.408 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.355384+0000", 2024-09-04T17:46:26.408 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.027e-06 2024-09-04T17:46:26.408 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.408 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.408 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:26.408 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.355393+0000", 2024-09-04T17:46:26.408 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.0049999999999993e-06 2024-09-04T17:46:26.408 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.408 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.408 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:26.409 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.355404+0000", 2024-09-04T17:46:26.409 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1154e-05 2024-09-04T17:46:26.409 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.409 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.409 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:26.409 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.370754+0000", 2024-09-04T17:46:26.409 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.015350205 2024-09-04T17:46:26.409 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.409 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.409 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:26.409 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.370847+0000", 2024-09-04T17:46:26.409 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.3441000000000002e-05 2024-09-04T17:46:26.410 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.410 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.410 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:26.410 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.370970+0000", 2024-09-04T17:46:26.410 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000122822 2024-09-04T17:46:26.410 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:26.410 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:26.410 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:26.410 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.410 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.410 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4813 3.3 3:c43d9109:::rbd_data.113417f3df25.0000000000000002:5 [read 4096000~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:26.410 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.355699+0000", 2024-09-04T17:46:26.410 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 19.021549735000001, 2024-09-04T17:46:26.410 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.01551072, 2024-09-04T17:46:26.411 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:26.411 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:26.411 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:26.411 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:26.411 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:26.411 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:26.411 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4813 2024-09-04T17:46:26.411 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.411 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:26.411 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.411 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:26.411 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.355699+0000", 2024-09-04T17:46:26.412 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:26.412 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.412 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.412 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:26.412 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.355699+0000", 2024-09-04T17:46:26.412 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:26.412 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.412 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.412 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:26.412 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.355701+0000", 2024-09-04T17:46:26.412 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1000000000000001e-06 2024-09-04T17:46:26.412 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.412 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.413 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:26.413 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.355705+0000", 2024-09-04T17:46:26.413 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7720000000000002e-06 2024-09-04T17:46:26.413 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.413 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.413 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:26.413 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.355720+0000", 2024-09-04T17:46:26.413 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4548e-05 2024-09-04T17:46:26.413 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.413 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.413 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:26.413 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.355730+0000", 2024-09-04T17:46:26.413 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0113e-05 2024-09-04T17:46:26.414 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.414 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.414 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:26.414 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.371006+0000", 2024-09-04T17:46:26.414 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.015276035 2024-09-04T17:46:26.414 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.414 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.414 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:26.414 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.371097+0000", 2024-09-04T17:46:26.414 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.0789000000000001e-05 2024-09-04T17:46:26.414 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.414 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.414 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:26.414 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.371210+0000", 2024-09-04T17:46:26.415 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00011336300000000001 2024-09-04T17:46:26.415 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:26.415 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:26.415 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:26.415 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.415 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.415 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:5009 3.14 3:2de14b79:::rbd_data.1134cfbce83d.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:26.415 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.410784+0000", 2024-09-04T17:46:26.415 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 18.966465501999998, 2024-09-04T17:46:26.415 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.016695742, 2024-09-04T17:46:26.415 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:26.415 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:26.415 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:26.415 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:26.416 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:26.416 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:26.416 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 5009 2024-09-04T17:46:26.416 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.416 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:26.416 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.416 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:26.416 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.410784+0000", 2024-09-04T17:46:26.416 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:26.416 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.416 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.416 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:26.416 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.410784+0000", 2024-09-04T17:46:26.417 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:26.417 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.417 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.417 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:26.417 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.410786+0000", 2024-09-04T17:46:26.417 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0490000000000002e-06 2024-09-04T17:46:26.417 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.417 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.417 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:26.417 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.415833+0000", 2024-09-04T17:46:26.417 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0050473430000000001 2024-09-04T17:46:26.417 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.418 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.418 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:26.418 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.415853+0000", 2024-09-04T17:46:26.418 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9748000000000001e-05 2024-09-04T17:46:26.418 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.418 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.418 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:26.418 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.415861+0000", 2024-09-04T17:46:26.418 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.0160000000000008e-06 2024-09-04T17:46:26.418 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.418 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.418 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:26.419 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.415929+0000", 2024-09-04T17:46:26.419 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.8510999999999995e-05 2024-09-04T17:46:26.419 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.419 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.419 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:26.419 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.416039+0000", 2024-09-04T17:46:26.419 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00010938300000000001 2024-09-04T17:46:26.419 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.419 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.419 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 7", 2024-09-04T17:46:26.419 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.416401+0000", 2024-09-04T17:46:26.419 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000361858 2024-09-04T17:46:26.420 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.420 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.420 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:26.420 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.426442+0000", 2024-09-04T17:46:26.420 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.010041599999999999 2024-09-04T17:46:26.420 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.420 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.420 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:26.420 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.427370+0000", 2024-09-04T17:46:26.420 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00092744300000000002 2024-09-04T17:46:26.420 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.420 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.421 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:26.421 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.427439+0000", 2024-09-04T17:46:26.421 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.9239000000000006e-05 2024-09-04T17:46:26.421 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.421 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.421 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:26.421 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.427479+0000", 2024-09-04T17:46:26.421 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0552000000000001e-05 2024-09-04T17:46:26.421 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:26.421 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:26.422 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:26.422 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.422 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.422 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:5010 3.14 3:2de14b79:::rbd_data.1134cfbce83d.0000000000000002:head [delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:26.422 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.415884+0000", 2024-09-04T17:46:26.422 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 18.961365499999999, 2024-09-04T17:46:26.422 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.014765199999999999, 2024-09-04T17:46:26.422 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:26.422 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:26.422 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:26.422 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:26.423 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:26.423 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:26.423 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 5010 2024-09-04T17:46:26.423 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.423 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:26.423 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.423 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:26.423 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.415884+0000", 2024-09-04T17:46:26.423 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:26.423 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.423 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.423 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:26.423 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.415884+0000", 2024-09-04T17:46:26.424 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:26.424 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.424 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.424 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:26.424 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.415886+0000", 2024-09-04T17:46:26.424 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2900000000000001e-06 2024-09-04T17:46:26.424 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.424 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.424 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:26.424 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.415890+0000", 2024-09-04T17:46:26.424 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.8179999999999997e-06 2024-09-04T17:46:26.424 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.425 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.425 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:26.425 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.415897+0000", 2024-09-04T17:46:26.425 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7630000000000003e-06 2024-09-04T17:46:26.425 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.425 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.425 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:26.425 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.415900+0000", 2024-09-04T17:46:26.425 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.8500000000000004e-06 2024-09-04T17:46:26.425 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.426 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.426 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:26.426 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.420517+0000", 2024-09-04T17:46:26.426 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0046166219999999999 2024-09-04T17:46:26.426 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.426 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.426 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:26.426 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.420775+0000", 2024-09-04T17:46:26.426 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00025835399999999997 2024-09-04T17:46:26.426 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.426 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.426 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 7", 2024-09-04T17:46:26.427 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.423010+0000", 2024-09-04T17:46:26.427 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.002234691 2024-09-04T17:46:26.427 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.427 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.427 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:26.427 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.429961+0000", 2024-09-04T17:46:26.427 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0069508649999999996 2024-09-04T17:46:26.427 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.427 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.427 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:26.427 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.430511+0000", 2024-09-04T17:46:26.427 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00054961799999999998 2024-09-04T17:46:26.427 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.428 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.428 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:26.428 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.430580+0000", 2024-09-04T17:46:26.428 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.9139000000000003e-05 2024-09-04T17:46:26.428 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.428 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.428 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:26.428 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.430649+0000", 2024-09-04T17:46:26.428 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.9190000000000007e-05 2024-09-04T17:46:26.428 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:26.428 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:26.428 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:26.429 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.429 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.429 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:5384 3.14 3:283b8c0b:::rbd_data.113431586065.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 9392185~44999] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:26.429 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.714129+0000", 2024-09-04T17:46:26.429 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 18.663120191000001, 2024-09-04T17:46:26.429 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.021221803000000001, 2024-09-04T17:46:26.429 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:26.429 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:26.429 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:26.429 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:26.429 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:26.429 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:26.430 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 5384 2024-09-04T17:46:26.430 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.430 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:26.430 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.430 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:26.430 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.714129+0000", 2024-09-04T17:46:26.430 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:26.430 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.430 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.430 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:26.430 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.714129+0000", 2024-09-04T17:46:26.430 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:26.431 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.431 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.431 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:26.431 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.714131+0000", 2024-09-04T17:46:26.431 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1270000000000001e-06 2024-09-04T17:46:26.431 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.431 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.431 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:26.431 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.719337+0000", 2024-09-04T17:46:26.431 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0052062059999999997 2024-09-04T17:46:26.431 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.431 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.432 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:26.432 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.719361+0000", 2024-09-04T17:46:26.432 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4051999999999999e-05 2024-09-04T17:46:26.432 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.432 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.432 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:26.432 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.719373+0000", 2024-09-04T17:46:26.432 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1270000000000001e-05 2024-09-04T17:46:26.432 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.432 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.432 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:26.432 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.719449+0000", 2024-09-04T17:46:26.433 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.6785000000000004e-05 2024-09-04T17:46:26.433 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.433 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.433 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:26.433 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.719566+0000", 2024-09-04T17:46:26.433 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000116537 2024-09-04T17:46:26.433 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.433 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.433 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 7", 2024-09-04T17:46:26.433 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.719930+0000", 2024-09-04T17:46:26.433 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000363624 2024-09-04T17:46:26.433 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.434 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.434 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:26.434 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.730485+0000", 2024-09-04T17:46:26.434 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.010555266000000001 2024-09-04T17:46:26.434 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.434 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.434 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:26.434 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.735229+0000", 2024-09-04T17:46:26.434 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0047438369999999999 2024-09-04T17:46:26.434 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.434 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.434 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:26.435 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.735300+0000", 2024-09-04T17:46:26.435 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1408999999999993e-05 2024-09-04T17:46:26.435 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.435 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.435 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:26.435 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.735351+0000", 2024-09-04T17:46:26.435 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0689999999999997e-05 2024-09-04T17:46:26.435 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:26.435 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:26.436 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:26.436 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.436 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.436 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:5645 3.1d e31/24 3:bd9dd029:::rbd_data.113431586065.0000000000000006:head v 31'23, mlcod=30'20)", 2024-09-04T17:46:26.436 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.908329+0000", 2024-09-04T17:46:26.436 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 18.468920495999999, 2024-09-04T17:46:26.436 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.019759324000000002, 2024-09-04T17:46:26.436 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:26.436 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:26.436 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:26.436 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:26.437 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.437 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:26.437 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.908329+0000", 2024-09-04T17:46:26.437 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:26.437 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.437 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.437 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:26.437 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.908329+0000", 2024-09-04T17:46:26.437 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:26.437 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.438 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.438 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:26.438 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.908331+0000", 2024-09-04T17:46:26.438 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.3e-06 2024-09-04T17:46:26.438 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.438 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.438 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:26.438 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.912239+0000", 2024-09-04T17:46:26.438 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0039075730000000001 2024-09-04T17:46:26.438 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.438 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.438 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:26.439 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.912280+0000", 2024-09-04T17:46:26.439 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.1457e-05 2024-09-04T17:46:26.439 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.439 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.439 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:26.439 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.912298+0000", 2024-09-04T17:46:26.439 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7980000000000001e-05 2024-09-04T17:46:26.439 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.439 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.439 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:26.439 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.912432+0000", 2024-09-04T17:46:26.439 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000133516 2024-09-04T17:46:26.439 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.440 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.440 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:26.440 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.912452+0000", 2024-09-04T17:46:26.440 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0679999999999999e-05 2024-09-04T17:46:26.440 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.440 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.440 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:26.440 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.928026+0000", 2024-09-04T17:46:26.440 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.015574202000000001 2024-09-04T17:46:26.440 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.440 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.441 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:26.441 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.928088+0000", 2024-09-04T17:46:26.441 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.1616000000000001e-05 2024-09-04T17:46:26.441 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:26.441 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:26.441 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:26.441 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.441 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.441 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:6292 3.14 3:283b8c0b:::rbd_data.113431586065.0000000000000000:head [stat,truncate 2097152] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:26.441 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:08.428248+0000", 2024-09-04T17:46:26.441 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.949001256999999, 2024-09-04T17:46:26.441 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.017598231999999998, 2024-09-04T17:46:26.442 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:26.442 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:26.442 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:26.442 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:26.442 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:26.442 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:26.442 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 6292 2024-09-04T17:46:26.442 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.442 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:26.442 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.442 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:26.443 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.428248+0000", 2024-09-04T17:46:26.443 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:26.443 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.443 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.443 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:26.443 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.428248+0000", 2024-09-04T17:46:26.443 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:26.443 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.443 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.443 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:26.443 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.428249+0000", 2024-09-04T17:46:26.444 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.116e-06 2024-09-04T17:46:26.444 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.444 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.444 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:26.444 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.428254+0000", 2024-09-04T17:46:26.444 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0830000000000003e-06 2024-09-04T17:46:26.444 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.444 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.444 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:26.444 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.428271+0000", 2024-09-04T17:46:26.444 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6714e-05 2024-09-04T17:46:26.444 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.445 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.445 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:26.445 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.428279+0000", 2024-09-04T17:46:26.445 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.0690000000000002e-06 2024-09-04T17:46:26.445 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.445 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.445 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:26.445 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.428336+0000", 2024-09-04T17:46:26.445 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7336000000000001e-05 2024-09-04T17:46:26.445 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.445 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.445 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:26.445 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.428423+0000", 2024-09-04T17:46:26.446 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.6720999999999999e-05 2024-09-04T17:46:26.446 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.446 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.446 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 7", 2024-09-04T17:46:26.446 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.428609+0000", 2024-09-04T17:46:26.446 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00018606199999999999 2024-09-04T17:46:26.446 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.446 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.446 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:26.446 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.433297+0000", 2024-09-04T17:46:26.446 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0046874580000000002 2024-09-04T17:46:26.446 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.446 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.447 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:26.447 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.445776+0000", 2024-09-04T17:46:26.447 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.012479762 2024-09-04T17:46:26.447 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.447 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.447 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:26.447 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.445809+0000", 2024-09-04T17:46:26.447 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.2212000000000003e-05 2024-09-04T17:46:26.448 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.448 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.448 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:26.448 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.445846+0000", 2024-09-04T17:46:26.448 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7698999999999999e-05 2024-09-04T17:46:26.448 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:26.448 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:26.448 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:26.448 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.448 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.448 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:6301 3.f e32/24 3:f3ba70c3:::rbd_data.113431586065.0000000000000004:head v 32'17, mlcod=31'14)", 2024-09-04T17:46:26.448 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:08.456787+0000", 2024-09-04T17:46:26.449 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.920461931999998, 2024-09-04T17:46:26.449 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.015206687999999999, 2024-09-04T17:46:26.449 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:26.449 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:26.449 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:26.449 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:26.449 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.449 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:26.449 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.456787+0000", 2024-09-04T17:46:26.449 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:26.449 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.449 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.449 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:26.449 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.456787+0000", 2024-09-04T17:46:26.450 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:26.450 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.450 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.450 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:26.450 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.456789+0000", 2024-09-04T17:46:26.450 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.432e-06 2024-09-04T17:46:26.450 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.450 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.450 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:26.450 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.458945+0000", 2024-09-04T17:46:26.450 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0021565709999999999 2024-09-04T17:46:26.450 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.450 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.451 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:26.451 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.458976+0000", 2024-09-04T17:46:26.451 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0704000000000003e-05 2024-09-04T17:46:26.451 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.451 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.451 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:26.451 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.458995+0000", 2024-09-04T17:46:26.451 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8649999999999999e-05 2024-09-04T17:46:26.451 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.451 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.451 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:26.451 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.459094+0000", 2024-09-04T17:46:26.451 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.9302999999999994e-05 2024-09-04T17:46:26.452 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.452 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.452 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:26.452 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.459124+0000", 2024-09-04T17:46:26.452 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.9869000000000002e-05 2024-09-04T17:46:26.452 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.452 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.452 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:26.452 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.471957+0000", 2024-09-04T17:46:26.452 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.012833456 2024-09-04T17:46:26.452 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.452 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.452 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:26.452 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.471994+0000", 2024-09-04T17:46:26.453 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.6702999999999997e-05 2024-09-04T17:46:26.453 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:26.453 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:26.453 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:26.453 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.453 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.453 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:6311 3.9 3:941d6897:::rbd_data.113431586065.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:26.453 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:08.474866+0000", 2024-09-04T17:46:26.453 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.902382893999999, 2024-09-04T17:46:26.453 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.023556524999999998, 2024-09-04T17:46:26.453 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:26.453 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:26.453 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:26.453 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:26.454 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:26.454 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:26.454 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 6311 2024-09-04T17:46:26.454 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.454 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:26.454 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.454 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:26.454 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.474866+0000", 2024-09-04T17:46:26.454 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:26.454 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.454 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.454 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:26.454 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.474866+0000", 2024-09-04T17:46:26.454 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:26.455 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.455 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.455 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:26.455 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.474868+0000", 2024-09-04T17:46:26.455 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.607e-06 2024-09-04T17:46:26.455 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.455 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.455 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:26.455 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.478761+0000", 2024-09-04T17:46:26.455 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0038930269999999999 2024-09-04T17:46:26.455 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.455 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.455 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:26.456 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.478781+0000", 2024-09-04T17:46:26.456 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0372000000000001e-05 2024-09-04T17:46:26.456 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.456 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.456 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:26.456 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.478790+0000", 2024-09-04T17:46:26.456 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.6130000000000004e-06 2024-09-04T17:46:26.456 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.456 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.456 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:26.456 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.478868+0000", 2024-09-04T17:46:26.456 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.7589000000000005e-05 2024-09-04T17:46:26.456 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.457 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.457 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:26.457 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.479006+0000", 2024-09-04T17:46:26.457 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013884600000000001 2024-09-04T17:46:26.457 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.457 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.457 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 2", 2024-09-04T17:46:26.457 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.479366+0000", 2024-09-04T17:46:26.457 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00035964299999999999 2024-09-04T17:46:26.457 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.457 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.457 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:26.457 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.497050+0000", 2024-09-04T17:46:26.458 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.017683770000000001 2024-09-04T17:46:26.458 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.458 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.458 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:26.458 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.498311+0000", 2024-09-04T17:46:26.458 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.001261278 2024-09-04T17:46:26.458 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.458 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.458 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:26.458 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.498359+0000", 2024-09-04T17:46:26.458 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7675e-05 2024-09-04T17:46:26.458 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.458 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.459 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:26.459 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.498423+0000", 2024-09-04T17:46:26.459 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.4104999999999997e-05 2024-09-04T17:46:26.459 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:26.459 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:26.459 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:26.459 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.459 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.459 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:6312 3.9 3:941d6897:::rbd_data.113431586065.0000000000000008:head [truncate 1572864] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:26.459 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:08.478816+0000", 2024-09-04T17:46:26.459 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.898433232999999, 2024-09-04T17:46:26.459 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.020677345999999999, 2024-09-04T17:46:26.459 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:26.460 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:26.460 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:26.460 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:26.460 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:26.460 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:26.460 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 6312 2024-09-04T17:46:26.460 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.460 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:26.460 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.460 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:26.460 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.478816+0000", 2024-09-04T17:46:26.460 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:26.460 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.461 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.461 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:26.461 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.478816+0000", 2024-09-04T17:46:26.461 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:26.461 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.461 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.461 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:26.461 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.478818+0000", 2024-09-04T17:46:26.461 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6500000000000001e-06 2024-09-04T17:46:26.461 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.461 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.461 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:26.461 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.478822+0000", 2024-09-04T17:46:26.461 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0289999999999999e-06 2024-09-04T17:46:26.462 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.462 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.462 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:26.462 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.478831+0000", 2024-09-04T17:46:26.462 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.3850000000000008e-06 2024-09-04T17:46:26.462 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.462 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.462 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:26.462 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.478839+0000", 2024-09-04T17:46:26.462 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4749999999999996e-06 2024-09-04T17:46:26.462 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.462 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.462 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:26.462 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.485267+0000", 2024-09-04T17:46:26.463 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0064281199999999998 2024-09-04T17:46:26.463 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.463 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.463 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:26.463 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.485357+0000", 2024-09-04T17:46:26.463 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.0129000000000004e-05 2024-09-04T17:46:26.463 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.463 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.463 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 2", 2024-09-04T17:46:26.463 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.485514+0000", 2024-09-04T17:46:26.463 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00015751599999999999 2024-09-04T17:46:26.463 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.463 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.464 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:26.464 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.497433+0000", 2024-09-04T17:46:26.464 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.011918307 2024-09-04T17:46:26.464 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.464 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.464 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:26.464 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.499312+0000", 2024-09-04T17:46:26.464 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0018790040000000001 2024-09-04T17:46:26.464 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.464 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.464 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:26.464 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.499413+0000", 2024-09-04T17:46:26.464 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000101179 2024-09-04T17:46:26.465 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.465 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.465 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:26.465 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.499493+0000", 2024-09-04T17:46:26.465 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.0551999999999997e-05 2024-09-04T17:46:26.465 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:26.465 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:26.465 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:26.468 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.468 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.468 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:12722 3.1b 3:d8152af1:::rbd_data.1134ef3a5a7d.00000000000000bd:head [delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-04T17:46:26.468 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:16.421965+0000", 2024-09-04T17:46:26.468 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 9.9552843240000009, 2024-09-04T17:46:26.468 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.890002811, 2024-09-04T17:46:26.468 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:26.468 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:26.469 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:26.469 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:26.469 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:26.469 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:26.469 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 12722 2024-09-04T17:46:26.469 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.469 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:26.469 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.469 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:26.469 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421965+0000", 2024-09-04T17:46:26.469 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:26.469 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.469 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.470 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:26.470 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421965+0000", 2024-09-04T17:46:26.470 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:26.470 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.470 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.470 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:26.470 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421966+0000", 2024-09-04T17:46:26.470 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.102e-06 2024-09-04T17:46:26.470 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.470 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.470 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:26.471 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421971+0000", 2024-09-04T17:46:26.471 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9989999999999999e-06 2024-09-04T17:46:26.471 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.471 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.471 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:26.471 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421981+0000", 2024-09-04T17:46:26.471 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.7380000000000006e-06 2024-09-04T17:46:26.471 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.471 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.471 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:26.471 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421989+0000", 2024-09-04T17:46:26.471 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.1890000000000007e-06 2024-09-04T17:46:26.472 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.472 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.472 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:26.472 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.422050+0000", 2024-09-04T17:46:26.472 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.0702000000000003e-05 2024-09-04T17:46:26.472 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.472 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.472 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for peered", 2024-09-04T17:46:26.472 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.422059+0000", 2024-09-04T17:46:26.472 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5440000000000004e-06 2024-09-04T17:46:26.472 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.472 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.473 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:26.473 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.308156+0000", 2024-09-04T17:46:26.473 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.88609676199999998 2024-09-04T17:46:26.473 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.473 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.473 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:26.473 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.308398+0000", 2024-09-04T17:46:26.473 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024215400000000001 2024-09-04T17:46:26.473 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.473 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.473 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 7", 2024-09-04T17:46:26.473 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.308795+0000", 2024-09-04T17:46:26.473 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000396704 2024-09-04T17:46:26.474 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.474 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.474 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:26.474 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.310831+0000", 2024-09-04T17:46:26.474 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0020364200000000002 2024-09-04T17:46:26.474 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.474 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.474 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:26.474 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.311818+0000", 2024-09-04T17:46:26.474 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00098649799999999993 2024-09-04T17:46:26.474 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.474 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.474 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:26.475 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.311905+0000", 2024-09-04T17:46:26.475 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7070999999999994e-05 2024-09-04T17:46:26.475 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.475 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.475 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:26.475 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.311968+0000", 2024-09-04T17:46:26.475 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.2928000000000005e-05 2024-09-04T17:46:26.475 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:26.475 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:26.475 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:26.475 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.475 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.475 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:19998 3.3 3:c7add6f4:::rbd_data.113418a7bbc1.0000000000000007:d [read 344064~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-04T17:46:26.476 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:21.756774+0000", 2024-09-04T17:46:26.476 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.6204755769999997, 2024-09-04T17:46:26.476 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.015397421999999999, 2024-09-04T17:46:26.476 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:26.476 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:26.476 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:26.476 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:26.476 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:26.476 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:26.476 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 19998 2024-09-04T17:46:26.476 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.476 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:26.477 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.477 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:26.477 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:21.756774+0000", 2024-09-04T17:46:26.477 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:26.477 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.477 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.477 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:26.477 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:21.756774+0000", 2024-09-04T17:46:26.477 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:26.477 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.477 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.477 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:26.477 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:21.756775+0000", 2024-09-04T17:46:26.478 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1429999999999999e-06 2024-09-04T17:46:26.478 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.478 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.478 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:26.478 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:21.756780+0000", 2024-09-04T17:46:26.478 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.8930000000000004e-06 2024-09-04T17:46:26.478 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.478 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.478 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:26.478 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:21.756797+0000", 2024-09-04T17:46:26.478 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7268000000000001e-05 2024-09-04T17:46:26.478 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.478 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.479 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:26.479 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:21.756813+0000", 2024-09-04T17:46:26.479 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6209e-05 2024-09-04T17:46:26.479 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.479 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.479 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:26.479 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:21.771691+0000", 2024-09-04T17:46:26.479 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.014877727 2024-09-04T17:46:26.479 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.479 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.479 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:26.479 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:21.771926+0000", 2024-09-04T17:46:26.480 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000234949 2024-09-04T17:46:26.480 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.480 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.480 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:26.480 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:21.772171+0000", 2024-09-04T17:46:26.480 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024523299999999997 2024-09-04T17:46:26.480 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:26.480 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:26.480 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:26.480 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.480 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.480 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:20370 3.15 3:ab7a1a4b:::rbd_data.1134fdde05d4.0000000000000144:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-04T17:46:26.480 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:22.031420+0000", 2024-09-04T17:46:26.481 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.3458296499999998, 2024-09-04T17:46:26.481 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.018949611000000002, 2024-09-04T17:46:26.481 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:26.481 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:26.481 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:26.481 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:26.481 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:26.481 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:26.481 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 20370 2024-09-04T17:46:26.481 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.481 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:26.481 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.481 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:26.482 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.031420+0000", 2024-09-04T17:46:26.482 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:26.482 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.482 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.482 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:26.482 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.031420+0000", 2024-09-04T17:46:26.482 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:26.482 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.482 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.482 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:26.482 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.031421+0000", 2024-09-04T17:46:26.483 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5409999999999999e-06 2024-09-04T17:46:26.483 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.483 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.483 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:26.483 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.031436+0000", 2024-09-04T17:46:26.483 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4851000000000001e-05 2024-09-04T17:46:26.483 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.483 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.483 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:26.483 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.031453+0000", 2024-09-04T17:46:26.484 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6841e-05 2024-09-04T17:46:26.484 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.484 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.484 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:26.484 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.031466+0000", 2024-09-04T17:46:26.484 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2899e-05 2024-09-04T17:46:26.484 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.484 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.484 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:26.484 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.031544+0000", 2024-09-04T17:46:26.484 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.7811000000000004e-05 2024-09-04T17:46:26.484 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.484 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.485 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:26.485 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.031736+0000", 2024-09-04T17:46:26.485 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019233100000000001 2024-09-04T17:46:26.485 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.485 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.485 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 7", 2024-09-04T17:46:26.485 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.035047+0000", 2024-09-04T17:46:26.485 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0033107980000000002 2024-09-04T17:46:26.485 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.485 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.485 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:26.485 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.050001+0000", 2024-09-04T17:46:26.485 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.014953855 2024-09-04T17:46:26.486 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.486 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.486 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:26.486 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.050075+0000", 2024-09-04T17:46:26.486 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4808999999999995e-05 2024-09-04T17:46:26.486 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.486 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.486 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:26.486 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.050144+0000", 2024-09-04T17:46:26.486 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.8658000000000006e-05 2024-09-04T17:46:26.486 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.486 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.487 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:26.487 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.050369+0000", 2024-09-04T17:46:26.487 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000225217 2024-09-04T17:46:26.487 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:26.487 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:26.487 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:26.487 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.487 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.487 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:20372 3.9 3:928af84f:::rbd_data.1134fdde05d4.0000000000000146:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-04T17:46:26.487 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:22.033751+0000", 2024-09-04T17:46:26.487 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.343498662, 2024-09-04T17:46:26.487 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.020054303999999998, 2024-09-04T17:46:26.487 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:26.488 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:26.488 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:26.488 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:26.488 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:26.488 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:26.488 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 20372 2024-09-04T17:46:26.488 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.488 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:26.488 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.488 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:26.488 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.033751+0000", 2024-09-04T17:46:26.488 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:26.488 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.489 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.489 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:26.489 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.033751+0000", 2024-09-04T17:46:26.489 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:26.489 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.489 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.489 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:26.489 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.033753+0000", 2024-09-04T17:46:26.489 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.898e-06 2024-09-04T17:46:26.489 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.489 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.490 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:26.490 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.033767+0000", 2024-09-04T17:46:26.490 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3909e-05 2024-09-04T17:46:26.490 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.490 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.490 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:26.490 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.033793+0000", 2024-09-04T17:46:26.490 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5210000000000001e-05 2024-09-04T17:46:26.490 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.490 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.490 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:26.490 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.033807+0000", 2024-09-04T17:46:26.490 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3995000000000001e-05 2024-09-04T17:46:26.491 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.491 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.491 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:26.491 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.033926+0000", 2024-09-04T17:46:26.491 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000119096 2024-09-04T17:46:26.491 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.491 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.491 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:26.491 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.034212+0000", 2024-09-04T17:46:26.491 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00028630000000000002 2024-09-04T17:46:26.491 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.492 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.492 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 2", 2024-09-04T17:46:26.492 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.038305+0000", 2024-09-04T17:46:26.492 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0040928910000000004 2024-09-04T17:46:26.492 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.492 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.492 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:26.492 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.053691+0000", 2024-09-04T17:46:26.492 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.015385853 2024-09-04T17:46:26.493 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.493 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.493 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:26.493 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.053708+0000", 2024-09-04T17:46:26.493 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6937999999999999e-05 2024-09-04T17:46:26.493 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.493 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.493 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:26.493 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.053758+0000", 2024-09-04T17:46:26.493 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0003999999999998e-05 2024-09-04T17:46:26.493 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.494 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.494 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:26.494 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.053805+0000", 2024-09-04T17:46:26.494 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7209999999999997e-05 2024-09-04T17:46:26.494 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:26.494 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:26.494 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:26.494 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.494 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.495 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:24189 3.1a 3:58ba337c:::rbd_data.1134fdde05d4.0000000000000090:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-04T17:46:26.495 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:22.366956+0000", 2024-09-04T17:46:26.495 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.0102928010000003, 2024-09-04T17:46:26.495 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.015895158999999999, 2024-09-04T17:46:26.495 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:26.495 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:26.495 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:26.495 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:26.495 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:26.495 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:26.495 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 24189 2024-09-04T17:46:26.495 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.496 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:26.496 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.496 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:26.496 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.366956+0000", 2024-09-04T17:46:26.496 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:26.496 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.496 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.496 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:26.496 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.366956+0000", 2024-09-04T17:46:26.496 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:26.496 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.496 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.497 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:26.497 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.366959+0000", 2024-09-04T17:46:26.497 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2919999999999998e-06 2024-09-04T17:46:26.497 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.497 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.497 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:26.497 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.366967+0000", 2024-09-04T17:46:26.497 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.2369999999999992e-06 2024-09-04T17:46:26.497 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.497 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.497 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:26.497 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.366984+0000", 2024-09-04T17:46:26.497 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6793999999999999e-05 2024-09-04T17:46:26.498 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.498 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.498 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:26.498 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.366992+0000", 2024-09-04T17:46:26.498 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.5469999999999997e-06 2024-09-04T17:46:26.498 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.498 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.498 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:26.498 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.367068+0000", 2024-09-04T17:46:26.498 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.5319000000000002e-05 2024-09-04T17:46:26.498 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.498 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.499 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:26.499 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.367229+0000", 2024-09-04T17:46:26.499 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000160903 2024-09-04T17:46:26.499 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.499 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.499 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 1", 2024-09-04T17:46:26.499 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.368287+0000", 2024-09-04T17:46:26.499 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0010584100000000001 2024-09-04T17:46:26.499 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.499 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.499 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:26.499 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.382769+0000", 2024-09-04T17:46:26.499 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.014481677 2024-09-04T17:46:26.500 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.500 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.500 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:26.500 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.382780+0000", 2024-09-04T17:46:26.500 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.113e-05 2024-09-04T17:46:26.500 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.500 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.500 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:26.500 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.382815+0000", 2024-09-04T17:46:26.500 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.5397e-05 2024-09-04T17:46:26.500 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:26.500 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:26.500 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:26.501 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.382852+0000", 2024-09-04T17:46:26.501 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.6452999999999998e-05 2024-09-04T17:46:26.501 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:26.501 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:26.501 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:26.501 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:26.501 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:26.501 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:26.503 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-04T17:46:26.504 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:26.505 INFO:teuthology.orchestra.run.smithi092.stdout:1587 write 0x6830932 thru 0x68358ba (0x4f89 bytes) 2024-09-04T17:46:26.505 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:26.505 INFO:teuthology.orchestra.run.smithi092.stdout:1589 write 0x4e91ed2 thru 0x4e9fa26 (0xdb55 bytes) 2024-09-04T17:46:26.505 INFO:teuthology.orchestra.run.smithi092.stdout:1591 punch from 0x8e71c23 to 0x8e7da16, (0xbdf3 bytes) 2024-09-04T17:46:26.505 INFO:teuthology.orchestra.run.smithi092.stdout:1592 read 0x6b73801 thru 0x6b78a48 (0x5248 bytes) 2024-09-04T17:46:26.505 INFO:teuthology.orchestra.run.smithi092.stdout:1594 punch from 0x29fde13 to 0x29fe66e, (0x85b bytes) 2024-09-04T17:46:26.505 INFO:teuthology.orchestra.run.smithi092.stdout:1595 read 0x668eb53 thru 0x66992e4 (0xa792 bytes) 2024-09-04T17:46:26.505 INFO:teuthology.orchestra.run.smithi092.stdout:1596 read 0x494f7ac thru 0x495a897 (0xb0ec bytes) 2024-09-04T17:46:26.505 INFO:teuthology.orchestra.run.smithi092.stdout:1597 read 0x8e5942d thru 0x8e63c27 (0xa7fb bytes) 2024-09-04T17:46:26.505 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:26.505 INFO:teuthology.orchestra.run.smithi092.stdout:1599 punch from 0x3c2d76d to 0x3c33fea, (0x687d bytes) 2024-09-04T17:46:26.505 INFO:teuthology.orchestra.run.smithi092.stdout:1600 write 0x5abbf07 thru 0x5ac652c (0xa626 bytes) 2024-09-04T17:46:26.506 INFO:teuthology.orchestra.run.smithi092.stdout:1601 writesame 0xeb263a9 thru 0xeb2907c data_size 0x13(0x2cd4 bytes) 2024-09-04T17:46:26.506 INFO:teuthology.orchestra.run.smithi092.stdout:1602 read 0x1ddb5b7 thru 0x1ddcb1f (0x1569 bytes) 2024-09-04T17:46:26.506 INFO:teuthology.orchestra.run.smithi092.stdout:1603 punch from 0x888554c to 0x8886131, (0xbe5 bytes) 2024-09-04T17:46:26.506 INFO:teuthology.orchestra.run.smithi092.stdout:1604 punch from 0x6d8197b to 0x6d8693d, (0x4fc2 bytes) 2024-09-04T17:46:26.506 INFO:teuthology.orchestra.run.smithi092.stdout:1605 write 0xea65b21 thru 0xea75150 (0xf630 bytes) 2024-09-04T17:46:26.506 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:46:26.405+0000 7f4ae73767c0 -1 osd.3 23 log_to_monitors true 2024-09-04T17:46:26.507 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:26.507 INFO:teuthology.orchestra.run.smithi092.stdout:1609 write 0x78cd899 thru 0x78d9ef6 (0xc65e bytes) 2024-09-04T17:46:26.508 INFO:teuthology.orchestra.run.smithi092.stdout:1611 read 0x26f17d4 thru 0x26fe53c (0xcd69 bytes) 2024-09-04T17:46:26.508 INFO:teuthology.orchestra.run.smithi092.stdout:1612 read 0x6572fe9 thru 0x657e1ff (0xb217 bytes) 2024-09-04T17:46:26.508 INFO:teuthology.orchestra.run.smithi092.stdout:1613 punch from 0x8873da2 to 0x8875e19, (0x2077 bytes) 2024-09-04T17:46:26.508 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:26.508 INFO:teuthology.orchestra.run.smithi092.stdout:1615 writesame 0x3743641 thru 0x374375c data_size 0x2(0x11c bytes) 2024-09-04T17:46:26.508 INFO:teuthology.orchestra.run.smithi092.stdout:1616 read 0x274733b thru 0x274cd97 (0x5a5d bytes) 2024-09-04T17:46:26.508 INFO:teuthology.orchestra.run.smithi092.stdout:1618 trunc from 0xeb2907d to 0x1f7e36b 2024-09-04T17:46:26.508 INFO:teuthology.orchestra.run.smithi092.stdout:1621 read 0x1b183cd thru 0x1b22024 (0x9c58 bytes) 2024-09-04T17:46:26.508 INFO:teuthology.orchestra.run.smithi092.stdout:1622 writesame 0x4f2d63b thru 0x4f39895 data_size 0x1f(0xc25b bytes) 2024-09-04T17:46:26.508 INFO:teuthology.orchestra.run.smithi092.stdout:1624 write 0x536c438 thru 0x5373d08 (0x78d1 bytes) 2024-09-04T17:46:26.508 INFO:teuthology.orchestra.run.smithi092.stdout:1626 writesame 0x5c77ef1 thru 0x5c7bcd6 data_size 0x26(0x3de6 bytes) 2024-09-04T17:46:26.508 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:26.501+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:26.509 INFO:teuthology.orchestra.run.smithi092.stdout:1627 read 0x13f4d78 thru 0x13f5bce (0xe57 bytes) 2024-09-04T17:46:26.509 INFO:teuthology.orchestra.run.smithi092.stdout:1628 read 0x4e9ee6f thru 0x4eae610 (0xf7a2 bytes) 2024-09-04T17:46:26.528 INFO:teuthology.orchestra.run.smithi092.stdout:1629 write 0x9a61c38 thru 0x9a6b004 (0x93cd bytes) 2024-09-04T17:46:26.535 INFO:teuthology.orchestra.run.smithi092.stdout:1630 write 0x5e6c496 thru 0x5e7ba16 (0xf581 bytes) 2024-09-04T17:46:26.538 INFO:teuthology.orchestra.run.smithi092.stdout:1631 punch from 0x5816b60 to 0x5820cef, (0xa18f bytes) 2024-09-04T17:46:26.539 INFO:teuthology.orchestra.run.smithi092.stdout:1634 read 0x69cfe9c thru 0x69d25da (0x273f bytes) 2024-09-04T17:46:26.539 INFO:teuthology.orchestra.run.smithi092.stdout:1638 punch from 0x463eba0 to 0x464152f, (0x298f bytes) 2024-09-04T17:46:26.549 INFO:teuthology.orchestra.run.smithi092.stdout:1639 trunc from 0x9a6b005 to 0xd53b033 2024-09-04T17:46:26.576 INFO:teuthology.orchestra.run.smithi092.stdout:1640 write 0xdb90994 thru 0xdb99f3a (0x95a7 bytes) 2024-09-04T17:46:26.582 INFO:teuthology.orchestra.run.smithi092.stdout:1643 read 0xcf70bc6 thru 0xcf784bd (0x78f8 bytes) 2024-09-04T17:46:26.582 INFO:teuthology.orchestra.run.smithi092.stdout:1645 read 0xbb4ff71 thru 0xbb52494 (0x2524 bytes) 2024-09-04T17:46:26.583 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:26.583 INFO:teuthology.orchestra.run.smithi092.stdout:1649 read 0xa791ea1 thru 0xa79b407 (0x9567 bytes) 2024-09-04T17:46:26.583 INFO:teuthology.orchestra.run.smithi092.stdout:1650 write 0x185f76e thru 0x186852c (0x8dbf bytes) 2024-09-04T17:46:26.586 INFO:teuthology.orchestra.run.smithi092.stdout:1651 punch from 0xa189a85 to 0xa195ca4, (0xc21f bytes) 2024-09-04T17:46:26.598 INFO:teuthology.orchestra.run.smithi092.stdout:1652 write 0xde116d2 thru 0xde189fc (0x732b bytes) 2024-09-04T17:46:26.603 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:26.605+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:26.604 INFO:teuthology.orchestra.run.smithi092.stdout:1653 read 0xa0896c thru 0xa112e8 (0x897d bytes) 2024-09-04T17:46:26.605 INFO:teuthology.orchestra.run.smithi092.stdout:1654 write 0x578e8d8 thru 0x5795360 (0x6a89 bytes) 2024-09-04T17:46:26.608 INFO:teuthology.orchestra.run.smithi092.stdout:1655 punch from 0x2978040 to 0x297b294, (0x3254 bytes) 2024-09-04T17:46:26.608 INFO:teuthology.orchestra.run.smithi092.stdout:1656 write 0xd2b4624 thru 0xd2b4a9a (0x477 bytes) 2024-09-04T17:46:26.611 INFO:teuthology.orchestra.run.smithi092.stdout:1657 punch from 0x4269f30 to 0x426e1cb, (0x429b bytes) 2024-09-04T17:46:26.611 INFO:teuthology.orchestra.run.smithi092.stdout:1659 write 0x48c8e95 thru 0x48cc2e1 (0x344d bytes) 2024-09-04T17:46:26.634 INFO:teuthology.orchestra.run.smithi092.stdout:1660 trunc from 0xde189fd to 0x6bcdb68 2024-09-04T17:46:26.682 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:26.682 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:46:26.682 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:46:26.682 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:26.684 INFO:teuthology.orchestra.run.smithi092.stdout:1661 writesame 0xce58889 thru 0xce5e8e3 data_size 0x1(0x605b bytes) 2024-09-04T17:46:26.697 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-04T17:46:26.700 INFO:teuthology.orchestra.run.smithi092.stdout:1662 write 0xe1863ce thru 0xe186826 (0x459 bytes) 2024-09-04T17:46:26.706 INFO:teuthology.orchestra.run.smithi092.stdout:1664 clone 13 order 18 su 65536 sc 15 2024-09-04T17:46:26.744 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:26.743+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:26.806 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:26.805+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:26.819 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:26.819 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:46:26.819 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:46:26.820 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:46:26.820 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:26.834 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-04T17:46:26.907 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:26.905+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:27.002 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:27.002 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:46:27.002 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:46:27.003 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:46:27.003 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.003 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8831 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2453504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:27.003 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:46:27.003 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 13.944938864999999, 2024-09-04T17:46:27.003 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14731111799999999, 2024-09-04T17:46:27.003 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:27.003 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:27.003 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:27.003 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:27.003 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:27.004 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:27.004 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8831 2024-09-04T17:46:27.004 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.004 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:27.004 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.004 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:27.004 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:46:27.004 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:27.004 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.004 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.005 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:27.005 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:46:27.005 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:27.005 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.005 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.005 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:27.005 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057811+0000", 2024-09-04T17:46:27.006 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.083e-06 2024-09-04T17:46:27.006 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.006 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.006 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:27.006 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057814+0000", 2024-09-04T17:46:27.006 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.407e-06 2024-09-04T17:46:27.006 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.006 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.006 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:27.006 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057825+0000", 2024-09-04T17:46:27.006 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.17e-05 2024-09-04T17:46:27.006 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.006 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.007 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:27.007 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057836+0000", 2024-09-04T17:46:27.007 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0283e-05 2024-09-04T17:46:27.007 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.007 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.007 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:27.007 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.204762+0000", 2024-09-04T17:46:27.007 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.146926205 2024-09-04T17:46:27.007 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.007 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.007 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:27.007 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.204895+0000", 2024-09-04T17:46:27.007 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013328999999999999 2024-09-04T17:46:27.008 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.008 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.008 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:27.008 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205121+0000", 2024-09-04T17:46:27.008 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00022614999999999999 2024-09-04T17:46:27.008 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:27.008 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:27.008 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:27.008 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.008 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.008 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8832 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2490368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:27.008 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:46:27.008 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 13.944900832, 2024-09-04T17:46:27.008 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14770012299999999, 2024-09-04T17:46:27.009 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:27.009 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:27.009 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:27.009 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:27.009 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:27.009 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:27.009 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8832 2024-09-04T17:46:27.009 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.009 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:27.009 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.009 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:27.009 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:46:27.009 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:27.009 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.010 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.010 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:27.010 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:46:27.010 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:27.010 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.010 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.010 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:27.010 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057849+0000", 2024-09-04T17:46:27.010 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6700000000000003e-07 2024-09-04T17:46:27.010 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.010 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.010 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:27.011 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057852+0000", 2024-09-04T17:46:27.011 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6359999999999998e-06 2024-09-04T17:46:27.011 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.011 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.011 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:27.011 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057860+0000", 2024-09-04T17:46:27.011 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8140000000000007e-06 2024-09-04T17:46:27.011 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.011 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.011 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:27.011 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057868+0000", 2024-09-04T17:46:27.011 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1999999999999997e-06 2024-09-04T17:46:27.011 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.011 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.012 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:27.012 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205208+0000", 2024-09-04T17:46:27.012 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14734067300000001 2024-09-04T17:46:27.012 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.012 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.012 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:27.012 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205347+0000", 2024-09-04T17:46:27.012 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013914900000000001 2024-09-04T17:46:27.012 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.012 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.012 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:27.012 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205548+0000", 2024-09-04T17:46:27.012 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000200984 2024-09-04T17:46:27.013 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:27.013 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:27.013 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:27.013 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.013 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.013 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8833 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2527232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:27.013 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:46:27.013 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 13.944868961999999, 2024-09-04T17:46:27.013 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148062572, 2024-09-04T17:46:27.014 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:27.014 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:27.014 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:27.014 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:27.014 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:27.014 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:27.014 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8833 2024-09-04T17:46:27.014 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.014 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:27.014 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.014 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:27.014 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:46:27.014 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:27.014 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.015 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.015 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:27.015 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:46:27.015 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:27.015 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.015 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.015 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:27.015 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057881+0000", 2024-09-04T17:46:27.015 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4300000000000002e-07 2024-09-04T17:46:27.015 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.015 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.015 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:27.015 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057893+0000", 2024-09-04T17:46:27.015 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1973999999999999e-05 2024-09-04T17:46:27.016 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.016 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.016 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:27.016 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057906+0000", 2024-09-04T17:46:27.016 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2716e-05 2024-09-04T17:46:27.016 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.016 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.016 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:27.016 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057913+0000", 2024-09-04T17:46:27.016 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3930000000000002e-06 2024-09-04T17:46:27.016 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.016 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.016 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:27.017 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205621+0000", 2024-09-04T17:46:27.017 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14770788800000001 2024-09-04T17:46:27.017 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.017 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.017 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:27.017 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205740+0000", 2024-09-04T17:46:27.017 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000118936 2024-09-04T17:46:27.017 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.017 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.017 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:27.017 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205943+0000", 2024-09-04T17:46:27.017 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020292199999999999 2024-09-04T17:46:27.017 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:27.018 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:27.018 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:27.018 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.018 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.018 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8834 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2564096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:27.018 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:46:27.018 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 13.944825682999999, 2024-09-04T17:46:27.018 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14846753200000001, 2024-09-04T17:46:27.018 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:27.018 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:27.018 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:27.018 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:27.018 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:27.019 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:27.019 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8834 2024-09-04T17:46:27.019 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.019 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:27.019 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.019 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:27.019 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:46:27.019 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:27.019 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.019 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.019 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:27.019 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:46:27.019 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:27.019 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.020 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.020 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:27.020 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057925+0000", 2024-09-04T17:46:27.020 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0270000000000001e-06 2024-09-04T17:46:27.020 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.020 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.020 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:27.020 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057936+0000", 2024-09-04T17:46:27.020 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.112e-05 2024-09-04T17:46:27.020 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.020 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.020 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:27.020 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057946+0000", 2024-09-04T17:46:27.021 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.009e-05 2024-09-04T17:46:27.021 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.021 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.021 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:27.021 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057953+0000", 2024-09-04T17:46:27.021 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.6229999999999997e-06 2024-09-04T17:46:27.021 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.021 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.021 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:27.021 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206027+0000", 2024-09-04T17:46:27.021 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148073591 2024-09-04T17:46:27.021 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.021 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.021 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:27.022 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206161+0000", 2024-09-04T17:46:27.022 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013411599999999999 2024-09-04T17:46:27.022 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.022 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.022 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:27.022 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206391+0000", 2024-09-04T17:46:27.022 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000229965 2024-09-04T17:46:27.022 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:27.022 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:27.022 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:27.022 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.022 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.022 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8835 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2600960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:27.023 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:46:27.023 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 13.944786033, 2024-09-04T17:46:27.023 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14882941599999999, 2024-09-04T17:46:27.023 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:27.023 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:27.023 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:27.023 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:27.023 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:27.023 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:27.023 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8835 2024-09-04T17:46:27.023 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.023 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:27.023 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.023 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:27.024 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:46:27.024 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:27.024 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.024 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.024 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:27.024 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:46:27.024 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:27.024 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.024 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.024 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:27.024 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058544+0000", 2024-09-04T17:46:27.024 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00058061899999999999 2024-09-04T17:46:27.024 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.025 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.025 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:27.025 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058550+0000", 2024-09-04T17:46:27.025 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.1129999999999997e-06 2024-09-04T17:46:27.025 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.025 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.025 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:27.025 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058566+0000", 2024-09-04T17:46:27.025 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5820000000000001e-05 2024-09-04T17:46:27.025 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.025 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.025 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:27.026 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058578+0000", 2024-09-04T17:46:27.026 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2537e-05 2024-09-04T17:46:27.026 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.026 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.026 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:27.026 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206470+0000", 2024-09-04T17:46:27.026 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14789153799999999 2024-09-04T17:46:27.026 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.026 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.026 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:27.026 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206589+0000", 2024-09-04T17:46:27.026 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000118837 2024-09-04T17:46:27.026 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.026 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.027 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:27.027 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206793+0000", 2024-09-04T17:46:27.027 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000203952 2024-09-04T17:46:27.027 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:27.027 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:27.027 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:27.027 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.027 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.027 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8836 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2654208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:27.027 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:46:27.027 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 13.944153345, 2024-09-04T17:46:27.027 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14862433999999999, 2024-09-04T17:46:27.027 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:27.028 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:27.028 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:27.028 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:27.028 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:27.028 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:27.028 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8836 2024-09-04T17:46:27.028 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.028 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:27.028 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.028 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:27.028 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:46:27.028 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:27.028 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.028 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.029 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:27.029 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:46:27.029 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:27.029 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.029 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.029 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:27.029 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062663+0000", 2024-09-04T17:46:27.029 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0040670619999999998 2024-09-04T17:46:27.029 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.029 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.029 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:27.029 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062668+0000", 2024-09-04T17:46:27.029 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4199999999999998e-06 2024-09-04T17:46:27.029 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.030 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.030 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:27.030 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062689+0000", 2024-09-04T17:46:27.030 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0356999999999998e-05 2024-09-04T17:46:27.030 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.030 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.030 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:27.030 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062726+0000", 2024-09-04T17:46:27.030 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7302999999999998e-05 2024-09-04T17:46:27.031 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.031 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.031 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:27.031 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206875+0000", 2024-09-04T17:46:27.031 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14414883100000001 2024-09-04T17:46:27.031 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.031 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.031 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:27.031 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.207016+0000", 2024-09-04T17:46:27.031 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000140822 2024-09-04T17:46:27.031 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.031 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.031 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:27.031 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.207220+0000", 2024-09-04T17:46:27.032 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020454499999999999 2024-09-04T17:46:27.032 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:27.032 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:27.032 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:27.032 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.032 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.032 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8844 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2928640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:27.032 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:46:27.032 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 13.939789262, 2024-09-04T17:46:27.032 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14752153100000001, 2024-09-04T17:46:27.032 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:27.032 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:27.032 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:27.032 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:27.033 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:27.033 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:27.033 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8844 2024-09-04T17:46:27.033 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.033 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:27.033 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.033 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:27.033 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:46:27.033 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:27.033 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.033 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.033 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:27.033 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:46:27.034 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:27.034 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.034 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.034 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:27.034 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062961+0000", 2024-09-04T17:46:27.034 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6700000000000003e-07 2024-09-04T17:46:27.034 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.034 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.034 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:27.034 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062973+0000", 2024-09-04T17:46:27.034 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2109999999999999e-05 2024-09-04T17:46:27.034 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.034 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.034 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:27.035 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062982+0000", 2024-09-04T17:46:27.035 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.6900000000000004e-06 2024-09-04T17:46:27.035 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.035 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.035 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:27.035 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062990+0000", 2024-09-04T17:46:27.035 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1899999999999998e-06 2024-09-04T17:46:27.035 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.035 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.035 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:27.035 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210090+0000", 2024-09-04T17:46:27.035 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14710026300000001 2024-09-04T17:46:27.035 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.036 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.036 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:27.036 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210225+0000", 2024-09-04T17:46:27.036 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013532400000000001 2024-09-04T17:46:27.036 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.036 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.036 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:27.036 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210481+0000", 2024-09-04T17:46:27.036 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00025628700000000001 2024-09-04T17:46:27.036 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:27.036 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:27.036 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:27.036 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.037 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.037 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8845 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2965504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:27.037 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:46:27.037 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 13.939749538999999, 2024-09-04T17:46:27.037 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147916819, 2024-09-04T17:46:27.037 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:27.037 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:27.037 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:27.037 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:27.037 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:27.037 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:27.037 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8845 2024-09-04T17:46:27.037 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.037 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:27.038 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.038 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:27.038 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:46:27.038 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:27.038 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.038 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.038 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:27.038 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:46:27.038 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:27.038 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.038 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.038 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:27.038 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:46:27.038 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.0399999999999995e-07 2024-09-04T17:46:27.039 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.039 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.039 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:27.039 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063003+0000", 2024-09-04T17:46:27.039 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6529999999999998e-06 2024-09-04T17:46:27.039 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.039 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.039 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:27.039 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063012+0000", 2024-09-04T17:46:27.039 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9500000000000007e-06 2024-09-04T17:46:27.039 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.039 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.039 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:27.040 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063027+0000", 2024-09-04T17:46:27.040 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5443000000000001e-05 2024-09-04T17:46:27.040 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.040 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.040 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:27.040 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210571+0000", 2024-09-04T17:46:27.040 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14754353100000001 2024-09-04T17:46:27.040 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.040 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.040 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:27.040 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210714+0000", 2024-09-04T17:46:27.040 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014295599999999999 2024-09-04T17:46:27.040 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.041 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.041 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:27.041 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210916+0000", 2024-09-04T17:46:27.041 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020258200000000001 2024-09-04T17:46:27.041 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:27.041 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:27.041 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:27.041 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.041 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.041 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8846 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3002368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:27.041 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:46:27.041 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 13.939712002, 2024-09-04T17:46:27.041 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14828011599999999, 2024-09-04T17:46:27.041 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:27.042 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:27.042 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:27.042 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:27.042 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:27.042 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:27.042 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8846 2024-09-04T17:46:27.042 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.042 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:27.042 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.042 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:27.042 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:46:27.042 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:27.042 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.042 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.043 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:27.043 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:46:27.043 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:27.043 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.043 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.043 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:27.043 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063038+0000", 2024-09-04T17:46:27.043 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.23e-07 2024-09-04T17:46:27.043 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.043 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.043 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:27.043 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063051+0000", 2024-09-04T17:46:27.043 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2757e-05 2024-09-04T17:46:27.044 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.044 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.044 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:27.044 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063070+0000", 2024-09-04T17:46:27.044 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9663000000000001e-05 2024-09-04T17:46:27.044 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.044 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.044 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:27.044 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063079+0000", 2024-09-04T17:46:27.044 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.0929999999999992e-06 2024-09-04T17:46:27.044 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.044 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.044 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:27.044 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210989+0000", 2024-09-04T17:46:27.045 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14790984500000001 2024-09-04T17:46:27.045 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.045 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.045 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:27.045 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211109+0000", 2024-09-04T17:46:27.045 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000120056 2024-09-04T17:46:27.045 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.045 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.045 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:27.045 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211317+0000", 2024-09-04T17:46:27.045 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020797900000000001 2024-09-04T17:46:27.046 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:27.046 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:27.046 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:27.046 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.046 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.046 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8847 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3039232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:27.046 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:46:27.046 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 13.93963757, 2024-09-04T17:46:27.047 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14859408900000001, 2024-09-04T17:46:27.047 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:27.047 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:27.047 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:27.047 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:27.047 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:27.048 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:27.048 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8847 2024-09-04T17:46:27.048 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.048 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:27.048 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.049 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:27.049 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:46:27.049 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:27.049 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.049 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.049 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:27.049 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:46:27.049 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:27.049 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.049 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.050 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:27.050 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063113+0000", 2024-09-04T17:46:27.050 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.204e-06 2024-09-04T17:46:27.050 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.050 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.050 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:27.050 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063116+0000", 2024-09-04T17:46:27.050 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.3330000000000001e-06 2024-09-04T17:46:27.050 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.050 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.051 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:27.051 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063126+0000", 2024-09-04T17:46:27.051 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0166999999999999e-05 2024-09-04T17:46:27.051 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.051 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.051 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:27.051 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063134+0000", 2024-09-04T17:46:27.051 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4329999999999998e-06 2024-09-04T17:46:27.051 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.051 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.051 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:27.051 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211392+0000", 2024-09-04T17:46:27.051 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148258109 2024-09-04T17:46:27.052 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.052 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.052 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:27.052 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211506+0000", 2024-09-04T17:46:27.052 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000113813 2024-09-04T17:46:27.052 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.052 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.052 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:27.052 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211706+0000", 2024-09-04T17:46:27.053 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020002999999999999 2024-09-04T17:46:27.053 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:27.053 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:27.053 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:27.053 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.053 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.053 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8848 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:27.053 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:46:27.054 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 13.939605233, 2024-09-04T17:46:27.054 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14907967599999999, 2024-09-04T17:46:27.054 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:27.054 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:27.054 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:27.054 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:27.054 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:27.054 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:27.054 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8848 2024-09-04T17:46:27.054 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.055 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:27.055 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.055 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:27.055 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:46:27.055 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:27.055 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.055 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.055 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:27.055 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:46:27.055 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:27.055 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.056 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.056 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:27.056 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070589+0000", 2024-09-04T17:46:27.056 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.007445386 2024-09-04T17:46:27.056 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.056 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.056 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:27.056 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070600+0000", 2024-09-04T17:46:27.057 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0346999999999999e-05 2024-09-04T17:46:27.057 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.057 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.057 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:27.057 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070622+0000", 2024-09-04T17:46:27.057 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2289999999999998e-05 2024-09-04T17:46:27.057 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.057 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.057 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:27.058 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070636+0000", 2024-09-04T17:46:27.058 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4093e-05 2024-09-04T17:46:27.058 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.058 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.058 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:27.058 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211815+0000", 2024-09-04T17:46:27.058 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14117874699999999 2024-09-04T17:46:27.059 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.059 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.059 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:27.059 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211983+0000", 2024-09-04T17:46:27.059 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00016852800000000001 2024-09-04T17:46:27.059 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.059 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.059 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:27.060 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.212224+0000", 2024-09-04T17:46:27.060 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024028499999999999 2024-09-04T17:46:27.060 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:27.060 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:27.060 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:27.060 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.060 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.060 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8862 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2019328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:27.060 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:46:27.060 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 13.931555765000001, 2024-09-04T17:46:27.060 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14755236399999999, 2024-09-04T17:46:27.060 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:27.061 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:27.061 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:27.061 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:27.061 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:27.061 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:27.061 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8862 2024-09-04T17:46:27.061 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.061 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:27.061 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.061 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:27.062 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:46:27.062 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:27.062 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.062 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.062 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:27.062 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:46:27.062 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:27.062 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.062 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.062 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:27.062 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071194+0000", 2024-09-04T17:46:27.062 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.5000000000000002e-07 2024-09-04T17:46:27.062 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.063 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.063 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:27.063 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071197+0000", 2024-09-04T17:46:27.063 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5900000000000002e-06 2024-09-04T17:46:27.063 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.063 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.063 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:27.063 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071206+0000", 2024-09-04T17:46:27.063 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9700000000000005e-06 2024-09-04T17:46:27.063 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.063 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.064 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:27.064 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071213+0000", 2024-09-04T17:46:27.064 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.9070000000000001e-06 2024-09-04T17:46:27.064 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.064 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.064 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:27.064 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218396+0000", 2024-09-04T17:46:27.064 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147183273 2024-09-04T17:46:27.064 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.065 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.065 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:27.065 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218538+0000", 2024-09-04T17:46:27.065 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014194199999999999 2024-09-04T17:46:27.065 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.065 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.065 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:27.065 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218746+0000", 2024-09-04T17:46:27.065 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000208032 2024-09-04T17:46:27.065 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:27.066 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:27.066 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:27.066 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.066 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.066 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8863 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2056192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:27.067 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:46:27.067 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 13.931527454999999, 2024-09-04T17:46:27.067 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14792281099999999, 2024-09-04T17:46:27.067 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:27.067 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:27.067 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:27.067 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:27.067 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:27.067 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:27.068 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8863 2024-09-04T17:46:27.068 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.068 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:27.068 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.068 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:27.068 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:46:27.068 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:27.069 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.069 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.069 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:27.069 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:46:27.069 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:27.069 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.070 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.070 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:27.070 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:46:27.070 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1299999999999999e-07 2024-09-04T17:46:27.070 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.070 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.070 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:27.070 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071225+0000", 2024-09-04T17:46:27.070 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6199999999999999e-06 2024-09-04T17:46:27.070 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.070 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.071 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:27.071 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071234+0000", 2024-09-04T17:46:27.071 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8769999999999999e-06 2024-09-04T17:46:27.071 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.071 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.071 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:27.071 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071251+0000", 2024-09-04T17:46:27.072 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.738e-05 2024-09-04T17:46:27.072 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.072 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.072 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:27.072 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218798+0000", 2024-09-04T17:46:27.072 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14754645 2024-09-04T17:46:27.072 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.072 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.072 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:27.072 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218926+0000", 2024-09-04T17:46:27.072 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000128446 2024-09-04T17:46:27.073 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.073 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.073 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:27.073 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219145+0000", 2024-09-04T17:46:27.073 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021832500000000001 2024-09-04T17:46:27.073 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:27.073 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:27.073 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:27.074 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.074 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.074 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8864 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2093056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:27.074 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:46:27.074 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 13.931485879, 2024-09-04T17:46:27.074 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148277309, 2024-09-04T17:46:27.074 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:27.074 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:27.074 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:27.074 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:27.074 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:27.074 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:27.074 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8864 2024-09-04T17:46:27.074 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.075 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:27.075 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.075 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:27.075 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:46:27.075 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:27.075 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.075 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.075 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:27.075 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:46:27.075 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:27.075 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.075 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.075 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:27.076 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071264+0000", 2024-09-04T17:46:27.076 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3e-07 2024-09-04T17:46:27.076 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.076 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.076 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:27.076 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071267+0000", 2024-09-04T17:46:27.076 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0139999999999999e-06 2024-09-04T17:46:27.076 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.076 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.076 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:27.076 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071276+0000", 2024-09-04T17:46:27.076 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1689999999999998e-06 2024-09-04T17:46:27.076 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.077 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.077 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:27.077 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071283+0000", 2024-09-04T17:46:27.077 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2269999999999996e-06 2024-09-04T17:46:27.077 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.077 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.077 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:27.077 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219195+0000", 2024-09-04T17:46:27.077 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147911868 2024-09-04T17:46:27.077 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.077 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.077 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:27.077 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219324+0000", 2024-09-04T17:46:27.077 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00012835900000000001 2024-09-04T17:46:27.078 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.078 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.078 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:27.078 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219541+0000", 2024-09-04T17:46:27.078 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000216942 2024-09-04T17:46:27.078 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:27.078 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:27.078 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:27.078 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.078 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.078 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8865 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2109440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:27.078 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:46:27.078 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 13.931456419, 2024-09-04T17:46:27.079 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14865747300000001, 2024-09-04T17:46:27.079 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:27.079 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:27.079 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:27.079 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:27.079 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:27.079 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:27.079 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8865 2024-09-04T17:46:27.079 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.079 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:27.079 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.079 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:27.079 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:46:27.080 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:27.080 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.080 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.080 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:27.080 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:46:27.080 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:27.080 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.080 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.080 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:27.080 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:46:27.080 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7999999999999995e-07 2024-09-04T17:46:27.080 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.080 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.081 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:27.081 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071296+0000", 2024-09-04T17:46:27.081 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6900000000000001e-06 2024-09-04T17:46:27.081 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.081 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.081 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:27.081 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071305+0000", 2024-09-04T17:46:27.081 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.5499999999999995e-06 2024-09-04T17:46:27.081 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.081 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.081 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:27.081 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071312+0000", 2024-09-04T17:46:27.081 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1559999999999998e-06 2024-09-04T17:46:27.082 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.082 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.082 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:27.082 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219591+0000", 2024-09-04T17:46:27.082 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14827944900000001 2024-09-04T17:46:27.082 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.082 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.082 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:27.082 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219731+0000", 2024-09-04T17:46:27.082 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013931600000000001 2024-09-04T17:46:27.082 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.082 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.082 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:27.083 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219950+0000", 2024-09-04T17:46:27.083 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021963199999999999 2024-09-04T17:46:27.083 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:27.083 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:27.083 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:27.083 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.083 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.083 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8866 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2146304~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:27.083 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:46:27.083 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 13.931428186, 2024-09-04T17:46:27.083 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14906117899999999, 2024-09-04T17:46:27.083 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:27.083 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:27.084 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:27.084 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:27.084 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:27.084 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:27.084 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8866 2024-09-04T17:46:27.084 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.084 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:27.084 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.084 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:27.084 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:46:27.084 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:27.084 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.085 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.085 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:27.085 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:46:27.085 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:27.085 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.085 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.085 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:27.085 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071322+0000", 2024-09-04T17:46:27.085 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7999999999999995e-07 2024-09-04T17:46:27.085 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.085 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.085 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:27.085 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071338+0000", 2024-09-04T17:46:27.085 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5872999999999999e-05 2024-09-04T17:46:27.086 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.086 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.086 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:27.086 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071347+0000", 2024-09-04T17:46:27.086 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4669999999999993e-06 2024-09-04T17:46:27.086 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.086 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.086 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:27.086 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071354+0000", 2024-09-04T17:46:27.086 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2099999999999996e-06 2024-09-04T17:46:27.087 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.087 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.087 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:27.087 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220008+0000", 2024-09-04T17:46:27.087 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148653642 2024-09-04T17:46:27.087 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.087 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.087 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:27.087 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220155+0000", 2024-09-04T17:46:27.087 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014726600000000001 2024-09-04T17:46:27.087 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.087 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.088 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:27.088 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220382+0000", 2024-09-04T17:46:27.088 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00022704100000000001 2024-09-04T17:46:27.088 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:27.088 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:27.088 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:27.088 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.088 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.088 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8867 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2183168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:27.088 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:46:27.088 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 13.931384832999999, 2024-09-04T17:46:27.088 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.149445831, 2024-09-04T17:46:27.088 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:27.088 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:27.088 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:27.089 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:27.089 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:27.089 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:27.089 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8867 2024-09-04T17:46:27.089 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.089 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:27.089 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.089 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:27.089 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:46:27.089 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:27.089 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.089 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.089 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:27.090 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:46:27.090 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:27.090 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.090 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.090 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:27.090 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078653+0000", 2024-09-04T17:46:27.090 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0072890430000000003 2024-09-04T17:46:27.090 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.090 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.090 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:27.090 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078664+0000", 2024-09-04T17:46:27.090 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0307e-05 2024-09-04T17:46:27.090 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.091 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.091 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:27.091 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078690+0000", 2024-09-04T17:46:27.091 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6780000000000001e-05 2024-09-04T17:46:27.091 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.091 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.091 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:27.091 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078706+0000", 2024-09-04T17:46:27.091 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5676999999999999e-05 2024-09-04T17:46:27.091 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.091 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.091 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:27.092 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220467+0000", 2024-09-04T17:46:27.092 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14176103500000001 2024-09-04T17:46:27.092 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.092 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.092 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:27.092 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220595+0000", 2024-09-04T17:46:27.092 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000127427 2024-09-04T17:46:27.092 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.092 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.092 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:27.092 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220810+0000", 2024-09-04T17:46:27.092 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021556200000000001 2024-09-04T17:46:27.093 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:27.093 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:27.093 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:27.093 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.093 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.093 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8880 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2658304~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:27.093 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.079158+0000", 2024-09-04T17:46:27.093 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 13.923591554, 2024-09-04T17:46:27.093 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14718968499999999, 2024-09-04T17:46:27.093 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:27.093 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:27.093 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:27.094 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:27.094 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:27.094 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:27.094 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8880 2024-09-04T17:46:27.094 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.094 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:27.094 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.094 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:27.094 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079158+0000", 2024-09-04T17:46:27.094 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:27.094 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.094 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.094 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:27.094 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079158+0000", 2024-09-04T17:46:27.095 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:27.095 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.095 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.095 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:27.095 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079158+0000", 2024-09-04T17:46:27.095 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3e-07 2024-09-04T17:46:27.095 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.095 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.095 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:27.095 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079161+0000", 2024-09-04T17:46:27.095 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.57e-06 2024-09-04T17:46:27.095 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.095 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.095 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:27.096 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079170+0000", 2024-09-04T17:46:27.096 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.0170000000000002e-06 2024-09-04T17:46:27.096 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.096 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.096 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:27.096 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079177+0000", 2024-09-04T17:46:27.096 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.0530000000000001e-06 2024-09-04T17:46:27.096 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.096 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.096 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:27.096 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.225946+0000", 2024-09-04T17:46:27.096 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14676936500000001 2024-09-04T17:46:27.096 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.097 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.097 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:27.097 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226098+0000", 2024-09-04T17:46:27.097 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000151805 2024-09-04T17:46:27.097 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.097 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.097 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:27.097 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226347+0000", 2024-09-04T17:46:27.097 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024914499999999999 2024-09-04T17:46:27.097 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:27.097 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:27.097 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:27.097 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.097 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.098 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8881 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2695168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:27.098 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:46:27.098 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 13.923562764, 2024-09-04T17:46:27.098 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147543547, 2024-09-04T17:46:27.098 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:27.098 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:27.098 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:27.098 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:27.098 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:27.098 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:27.098 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8881 2024-09-04T17:46:27.098 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.098 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:27.099 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.099 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:27.099 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:46:27.099 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:27.099 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.099 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.099 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:27.099 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:46:27.099 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:27.099 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.099 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.099 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:27.099 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079187+0000", 2024-09-04T17:46:27.100 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7700000000000004e-07 2024-09-04T17:46:27.100 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.100 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.100 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:27.100 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079198+0000", 2024-09-04T17:46:27.100 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0849999999999999e-05 2024-09-04T17:46:27.100 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.100 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.100 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:27.100 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079207+0000", 2024-09-04T17:46:27.100 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5599999999999999e-06 2024-09-04T17:46:27.100 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.100 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.101 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:27.101 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079224+0000", 2024-09-04T17:46:27.101 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6387000000000001e-05 2024-09-04T17:46:27.101 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.101 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.101 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:27.101 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226414+0000", 2024-09-04T17:46:27.101 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14718975000000001 2024-09-04T17:46:27.101 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.101 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.101 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:27.101 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226534+0000", 2024-09-04T17:46:27.101 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000120288 2024-09-04T17:46:27.101 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.102 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.102 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:27.102 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226730+0000", 2024-09-04T17:46:27.102 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000196035 2024-09-04T17:46:27.102 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:27.102 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:27.102 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:27.102 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.102 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.102 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8882 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2732032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:27.102 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:46:27.102 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 13.92351455, 2024-09-04T17:46:27.102 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14795022399999999, 2024-09-04T17:46:27.102 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:27.103 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:27.103 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:27.103 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:27.103 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:27.103 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:27.103 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8882 2024-09-04T17:46:27.103 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.103 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:27.103 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.103 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:27.103 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:46:27.103 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:27.103 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.104 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.104 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:27.104 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:46:27.104 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:27.104 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.104 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.104 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:27.104 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086588+0000", 2024-09-04T17:46:27.104 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0073530959999999999 2024-09-04T17:46:27.104 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.104 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.104 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:27.104 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086598+0000", 2024-09-04T17:46:27.104 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0122999999999999e-05 2024-09-04T17:46:27.105 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.105 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.105 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:27.105 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086626+0000", 2024-09-04T17:46:27.105 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.7914000000000001e-05 2024-09-04T17:46:27.105 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.105 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.105 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:27.105 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086641+0000", 2024-09-04T17:46:27.105 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5270000000000001e-05 2024-09-04T17:46:27.105 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.105 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.105 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:27.106 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226801+0000", 2024-09-04T17:46:27.106 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.140160321 2024-09-04T17:46:27.106 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.106 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.106 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:27.106 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226951+0000", 2024-09-04T17:46:27.106 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014965200000000001 2024-09-04T17:46:27.106 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:27.106 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:27.106 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:27.106 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.227185+0000", 2024-09-04T17:46:27.106 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00023384799999999999 2024-09-04T17:46:27.106 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:27.107 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:27.107 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:27.107 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:27.107 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:27.107 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:27.108 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-04T17:46:27.109 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:27.107+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:27.210 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:27.209+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:27.281 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:27.281 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:46:27.281 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:46:27.281 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:27.295 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-04T17:46:27.334 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-04T17:46:27.339 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:27.338+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:27.346 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-04T17:46:27.355 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:46:27.357+0000 7f4adb151640 -1 osd.3 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-04T17:46:27.411 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:27.410+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:27.431 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:27.431 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:46:27.432 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:46:27.432 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:46:27.432 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:27.446 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-04T17:46:27.496 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:46:27.497 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:46:27.515 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:46:27.516 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:46:27.534 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-04T17:46:27.535 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-04T17:46:27.535 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-04T17:46:27.539 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:46:27.538+0000 7f505d642640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26446) UID: 0 2024-09-04T17:46:27.556 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:46:27.556 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:46:27.577 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:46:27.577 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:46:27.598 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-04T17:46:27.598 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-04T17:46:27.612 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #11, image image_client.0-clone11 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent12 2024-09-04T17:46:27.613 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:27.612+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:27.618 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:27.618 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:27.619 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:46:27.619 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:46:27.622 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-04T17:46:27.623 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-04T17:46:27.632 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-04T17:46:27.725 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:27.718+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:27.804 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:27.804 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:27.816 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:27.816+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:27.818 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-04T17:46:27.917 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:46:27.914+0000 7f505d642640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26446) UID: 0 2024-09-04T17:46:27.981 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:27.982 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:27.998 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-04T17:46:28.011 INFO:teuthology.orchestra.run.smithi092.stdout:1665 read 0x78b1b35 thru 0x78c05cf (0xea9b bytes) 2024-09-04T17:46:28.012 INFO:teuthology.orchestra.run.smithi092.stdout:1667 write 0xdaf963d thru 0xdb08eb5 (0xf879 bytes) 2024-09-04T17:46:28.017 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:28.016+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:28.031 INFO:teuthology.orchestra.run.smithi092.stdout:1668 write 0x1e93d76 thru 0x1e97dc5 (0x4050 bytes) 2024-09-04T17:46:28.038 INFO:teuthology.orchestra.run.smithi092.stdout:1669 punch from 0xaf4f87f to 0xaf4fcd2, (0x453 bytes) 2024-09-04T17:46:28.038 INFO:teuthology.orchestra.run.smithi092.stdout:1670 write 0xbbbc8e thru 0xbc94c4 (0xd837 bytes) 2024-09-04T17:46:28.047 INFO:teuthology.orchestra.run.smithi092.stdout:1671 punch from 0x4c1cc72 to 0x4c27e73, (0xb201 bytes) 2024-09-04T17:46:28.047 INFO:teuthology.orchestra.run.smithi092.stdout:1672 writesame 0x1b6a360 thru 0x1b6cd26 data_size 0x45(0x29c7 bytes) 2024-09-04T17:46:28.054 INFO:teuthology.orchestra.run.smithi092.stdout:1673 read 0x6ecc9a1 thru 0x6edb432 (0xea92 bytes) 2024-09-04T17:46:28.054 INFO:teuthology.orchestra.run.smithi092.stdout:1674 clone 14 order 18 su 131072 sc 8 2024-09-04T17:46:28.119 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:28.114+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:28.161 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:28.162 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:28.175 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-04T17:46:28.220 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:28.218+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:28.297 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:28.298 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:28.311 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-04T17:46:28.320 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:28.318+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:28.329 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is unset 2024-09-04T17:46:28.421 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:28.418+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:28.440 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:28.440 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:28.453 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-04T17:46:28.522 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:46:28.518+0000 7f505d642640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26446) UID: 0 2024-09-04T17:46:28.568 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:28.568 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:28.581 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-04T17:46:28.627 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:28.628+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:28.699 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:28.699 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:28.713 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-04T17:46:28.755 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:28.756+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:28.824 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:28.824+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:28.877 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:28.877 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:28.891 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-04T17:46:28.925 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:28.926+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:29.026 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:29.026+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:29.056 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:29.057 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:29.070 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-04T17:46:29.129 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:29.130+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:29.187 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:29.187 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:29.202 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-04T17:46:29.230 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:29.226+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:29.324 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:29.324 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:29.333 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:29.332+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:29.346 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-04T17:46:29.432 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:29.430+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:29.515 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:29.515 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:29.528 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-04T17:46:29.533 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:29.532+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:29.624 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-04T17:46:29.662 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:29.662 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:29.671 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:29.672+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:29.677 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-04T17:46:29.734 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:29.730+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:29.772 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:46:29.772 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:46:29.789 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:46:29.789 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:46:29.794 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:29.794 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:29.805 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-04T17:46:29.805 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-04T17:46:29.808 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-04T17:46:29.821 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-04T17:46:29.822 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_enable_op_tracker = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' 2024-09-04T17:46:29.836 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:29.836+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:29.838 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:46:29.839 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:46:29.858 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:46:29.858 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:46:29.879 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-04T17:46:29.879 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-04T17:46:29.897 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:46:29.897 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:46:29.936 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:29.934+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:29.979 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:29.979 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:46:29.979 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:46:29.979 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:46:29.979 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:29.979 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4264 3.9 3:970e51a0:::rbd_data.1134cfbce83d.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 1597545~615] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-04T17:46:29.979 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.021843+0000", 2024-09-04T17:46:29.979 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.957640868999999, 2024-09-04T17:46:29.979 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.015077662, 2024-09-04T17:46:29.979 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:29.980 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:29.980 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:29.980 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:29.980 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:29.980 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:29.980 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4264 2024-09-04T17:46:29.980 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:29.980 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:29.980 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:29.980 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:29.980 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.021843+0000", 2024-09-04T17:46:29.980 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:29.981 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:29.981 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:29.981 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:29.981 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.021843+0000", 2024-09-04T17:46:29.981 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:29.981 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:29.981 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:29.981 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:29.981 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.021845+0000", 2024-09-04T17:46:29.981 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.328e-06 2024-09-04T17:46:29.981 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:29.982 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:29.982 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:29.982 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.026201+0000", 2024-09-04T17:46:29.982 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.004355935 2024-09-04T17:46:29.982 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:29.982 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:29.982 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:29.982 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.026214+0000", 2024-09-04T17:46:29.982 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3315999999999999e-05 2024-09-04T17:46:29.982 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:29.982 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:29.983 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:29.983 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.026221+0000", 2024-09-04T17:46:29.983 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7070000000000003e-06 2024-09-04T17:46:29.983 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:29.983 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:29.983 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:29.983 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.026266+0000", 2024-09-04T17:46:29.983 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.5120999999999997e-05 2024-09-04T17:46:29.983 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:29.983 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:29.983 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:29.983 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.026375+0000", 2024-09-04T17:46:29.983 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00010935 2024-09-04T17:46:29.983 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:29.984 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:29.984 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 2", 2024-09-04T17:46:29.984 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.026610+0000", 2024-09-04T17:46:29.984 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00023530400000000001 2024-09-04T17:46:29.984 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:29.984 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:29.984 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:29.984 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.032096+0000", 2024-09-04T17:46:29.984 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0054859050000000001 2024-09-04T17:46:29.984 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:29.984 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:29.984 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:29.984 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.036851+0000", 2024-09-04T17:46:29.985 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0047543409999999996 2024-09-04T17:46:29.985 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:29.985 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:29.985 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:29.985 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.036890+0000", 2024-09-04T17:46:29.985 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.9128999999999998e-05 2024-09-04T17:46:29.985 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:29.985 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:29.985 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:29.985 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.036921+0000", 2024-09-04T17:46:29.985 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.1226000000000002e-05 2024-09-04T17:46:29.985 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:29.985 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:29.986 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:29.986 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:29.986 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:29.986 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4651 3.3 3:c43d9109:::rbd_data.113417f3df25.0000000000000002:5 [read 4063232~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:29.986 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.349383+0000", 2024-09-04T17:46:29.986 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.630100862999999, 2024-09-04T17:46:29.986 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.014661281, 2024-09-04T17:46:29.986 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:29.986 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:29.986 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:29.986 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:29.986 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:29.986 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:29.987 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4651 2024-09-04T17:46:29.987 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:29.987 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:29.987 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:29.987 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:29.987 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.349383+0000", 2024-09-04T17:46:29.987 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:29.987 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:29.987 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:29.987 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:29.987 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.349383+0000", 2024-09-04T17:46:29.987 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:29.987 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:29.988 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:29.988 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:29.988 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.349384+0000", 2024-09-04T17:46:29.988 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.4499999999999996e-07 2024-09-04T17:46:29.988 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:29.988 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:29.988 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:29.988 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.349401+0000", 2024-09-04T17:46:29.988 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6339000000000001e-05 2024-09-04T17:46:29.988 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:29.988 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:29.988 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:29.988 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.349411+0000", 2024-09-04T17:46:29.989 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0013000000000001e-05 2024-09-04T17:46:29.989 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:29.989 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:29.989 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:29.989 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.349433+0000", 2024-09-04T17:46:29.989 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1926999999999999e-05 2024-09-04T17:46:29.989 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:29.989 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:29.989 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:29.989 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.363824+0000", 2024-09-04T17:46:29.989 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.014391736 2024-09-04T17:46:29.989 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:29.989 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:29.989 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:29.990 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.363920+0000", 2024-09-04T17:46:29.990 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.6142000000000002e-05 2024-09-04T17:46:29.990 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:29.990 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:29.990 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:29.990 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.364045+0000", 2024-09-04T17:46:29.990 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00012427900000000001 2024-09-04T17:46:29.990 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:29.990 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:29.990 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:29.990 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:29.990 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:29.990 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4734 3.3 3:c43d9109:::rbd_data.113417f3df25.0000000000000002:5 [read 196608~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:29.991 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.349456+0000", 2024-09-04T17:46:29.991 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.630027797, 2024-09-04T17:46:29.991 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.014853676999999999, 2024-09-04T17:46:29.991 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:29.991 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:29.991 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:29.991 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:29.991 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:29.991 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:29.991 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4734 2024-09-04T17:46:29.991 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:29.991 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:29.992 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:29.992 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:29.992 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.349456+0000", 2024-09-04T17:46:29.992 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:29.992 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:29.992 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:29.992 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:29.992 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.349456+0000", 2024-09-04T17:46:29.992 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:29.992 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:29.992 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:29.992 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:29.992 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.349457+0000", 2024-09-04T17:46:29.992 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.9999999999999996e-07 2024-09-04T17:46:29.993 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:29.993 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:29.993 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:29.993 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.349461+0000", 2024-09-04T17:46:29.993 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.9099999999999998e-06 2024-09-04T17:46:29.993 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:29.993 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:29.993 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:29.993 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.349471+0000", 2024-09-04T17:46:29.993 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.8069999999999994e-06 2024-09-04T17:46:29.993 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:29.993 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:29.993 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:29.993 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.349479+0000", 2024-09-04T17:46:29.994 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.8769999999999993e-06 2024-09-04T17:46:29.994 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:29.994 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:29.994 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:29.994 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.364085+0000", 2024-09-04T17:46:29.994 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.014606022 2024-09-04T17:46:29.994 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:29.994 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:29.994 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:29.994 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.364184+0000", 2024-09-04T17:46:29.994 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.8948000000000002e-05 2024-09-04T17:46:29.994 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:29.994 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:29.994 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:29.995 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.364310+0000", 2024-09-04T17:46:29.995 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00012631299999999999 2024-09-04T17:46:29.995 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:29.995 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:29.995 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:29.995 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:29.995 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:29.995 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4735 3.3 3:c43d9109:::rbd_data.113417f3df25.0000000000000002:5 [read 622592~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:29.995 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.349490+0000", 2024-09-04T17:46:29.995 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.629994414999999, 2024-09-04T17:46:29.995 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.015069948999999999, 2024-09-04T17:46:29.995 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:29.995 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:29.995 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:29.995 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:29.996 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:29.996 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:29.996 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4735 2024-09-04T17:46:29.996 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:29.996 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:29.996 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:29.996 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:29.996 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.349490+0000", 2024-09-04T17:46:29.996 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:29.996 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:29.996 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:29.996 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:29.996 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.349490+0000", 2024-09-04T17:46:29.996 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:29.997 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:29.997 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:29.997 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:29.997 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.349491+0000", 2024-09-04T17:46:29.997 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6000000000000003e-07 2024-09-04T17:46:29.997 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:29.997 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:29.997 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:29.997 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.349493+0000", 2024-09-04T17:46:29.997 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.9349999999999999e-06 2024-09-04T17:46:29.997 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:29.997 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:29.998 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:29.998 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.349503+0000", 2024-09-04T17:46:29.998 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4460000000000007e-06 2024-09-04T17:46:29.998 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:29.998 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:29.998 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:29.998 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.349510+0000", 2024-09-04T17:46:29.998 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7109999999999998e-06 2024-09-04T17:46:29.998 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:29.998 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:29.998 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:29.998 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.364347+0000", 2024-09-04T17:46:29.999 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.014837250999999999 2024-09-04T17:46:29.999 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:29.999 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:29.999 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:29.999 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.364440+0000", 2024-09-04T17:46:29.999 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.3121000000000005e-05 2024-09-04T17:46:29.999 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:29.999 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:29.999 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:29.999 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.364560+0000", 2024-09-04T17:46:29.999 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000119825 2024-09-04T17:46:29.999 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:29.999 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:29.999 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.000 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.000 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.000 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4811 3.3 3:c43d9109:::rbd_data.113417f3df25.0000000000000002:5 [read 3244032~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:30.000 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.355350+0000", 2024-09-04T17:46:30.000 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.624134564999999, 2024-09-04T17:46:30.000 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.015364724, 2024-09-04T17:46:30.000 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:30.000 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:30.000 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:30.000 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:30.000 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:30.000 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:30.000 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4811 2024-09-04T17:46:30.000 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.001 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:30.001 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.001 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:30.001 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.355350+0000", 2024-09-04T17:46:30.001 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.001 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.001 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.001 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:30.001 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.355350+0000", 2024-09-04T17:46:30.001 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.001 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.001 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.001 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:30.001 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.355350+0000", 2024-09-04T17:46:30.001 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.9400000000000005e-07 2024-09-04T17:46:30.002 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.002 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.002 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:30.002 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.355354+0000", 2024-09-04T17:46:30.002 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.8600000000000003e-06 2024-09-04T17:46:30.002 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.002 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.002 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:30.002 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.355363+0000", 2024-09-04T17:46:30.002 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.0739999999999999e-06 2024-09-04T17:46:30.002 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.002 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.002 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:30.002 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.355370+0000", 2024-09-04T17:46:30.002 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2370000000000003e-06 2024-09-04T17:46:30.003 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.003 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.003 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:30.003 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.370484+0000", 2024-09-04T17:46:30.003 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.015113909 2024-09-04T17:46:30.003 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.003 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.003 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:30.003 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.370584+0000", 2024-09-04T17:46:30.003 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000100042 2024-09-04T17:46:30.003 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.003 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.003 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:30.003 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.370714+0000", 2024-09-04T17:46:30.004 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013000800000000001 2024-09-04T17:46:30.004 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.004 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:30.004 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.004 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.004 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.004 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4812 3.3 3:c43d9109:::rbd_data.113417f3df25.0000000000000002:5 [read 3670016~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:30.004 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.355380+0000", 2024-09-04T17:46:30.004 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.624104418000002, 2024-09-04T17:46:30.004 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.015590292, 2024-09-04T17:46:30.004 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:30.004 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:30.005 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:30.005 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:30.005 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:30.005 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:30.005 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4812 2024-09-04T17:46:30.005 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.005 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:30.005 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.005 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:30.005 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.355380+0000", 2024-09-04T17:46:30.005 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.005 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.005 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.006 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:30.006 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.355380+0000", 2024-09-04T17:46:30.006 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.006 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.006 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.006 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:30.006 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.355381+0000", 2024-09-04T17:46:30.006 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.3799999999999997e-07 2024-09-04T17:46:30.006 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.006 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.006 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:30.006 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.355384+0000", 2024-09-04T17:46:30.006 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.027e-06 2024-09-04T17:46:30.006 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.006 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.007 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:30.007 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.355393+0000", 2024-09-04T17:46:30.007 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.0049999999999993e-06 2024-09-04T17:46:30.007 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.007 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.007 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:30.007 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.355404+0000", 2024-09-04T17:46:30.007 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1154e-05 2024-09-04T17:46:30.007 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.007 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.007 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:30.007 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.370754+0000", 2024-09-04T17:46:30.007 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.015350205 2024-09-04T17:46:30.008 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.008 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.008 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:30.008 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.370847+0000", 2024-09-04T17:46:30.008 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.3441000000000002e-05 2024-09-04T17:46:30.008 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.008 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.008 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:30.008 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.370970+0000", 2024-09-04T17:46:30.008 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000122822 2024-09-04T17:46:30.009 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.009 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:30.009 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.009 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.009 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.009 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:4813 3.3 3:c43d9109:::rbd_data.113417f3df25.0000000000000002:5 [read 4096000~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:30.009 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.355699+0000", 2024-09-04T17:46:30.009 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.623784788000002, 2024-09-04T17:46:30.009 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.01551072, 2024-09-04T17:46:30.009 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:30.009 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:30.009 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:30.009 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:30.010 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:30.010 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:30.010 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 4813 2024-09-04T17:46:30.010 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.010 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:30.010 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.010 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:30.010 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.355699+0000", 2024-09-04T17:46:30.010 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.010 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.010 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.010 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:30.010 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.355699+0000", 2024-09-04T17:46:30.010 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.011 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.011 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.011 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:30.011 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.355701+0000", 2024-09-04T17:46:30.011 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1000000000000001e-06 2024-09-04T17:46:30.011 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.011 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.011 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:30.011 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.355705+0000", 2024-09-04T17:46:30.011 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7720000000000002e-06 2024-09-04T17:46:30.011 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.011 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.011 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:30.011 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.355720+0000", 2024-09-04T17:46:30.011 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4548e-05 2024-09-04T17:46:30.012 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.012 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.012 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:30.012 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.355730+0000", 2024-09-04T17:46:30.012 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0113e-05 2024-09-04T17:46:30.012 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.012 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.012 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:30.012 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.371006+0000", 2024-09-04T17:46:30.012 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.015276035 2024-09-04T17:46:30.012 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.012 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.012 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:30.013 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.371097+0000", 2024-09-04T17:46:30.013 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.0789000000000001e-05 2024-09-04T17:46:30.013 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.013 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.013 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:30.013 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.371210+0000", 2024-09-04T17:46:30.013 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00011336300000000001 2024-09-04T17:46:30.013 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.013 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:30.013 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.013 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.013 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.014 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:5009 3.14 3:2de14b79:::rbd_data.1134cfbce83d.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:30.014 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.410784+0000", 2024-09-04T17:46:30.014 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.568700554999999, 2024-09-04T17:46:30.014 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.016695742, 2024-09-04T17:46:30.014 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:30.014 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:30.014 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:30.014 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:30.014 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:30.014 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:30.014 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 5009 2024-09-04T17:46:30.014 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.014 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:30.015 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.015 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:30.015 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.410784+0000", 2024-09-04T17:46:30.015 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.015 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.015 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.015 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:30.015 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.410784+0000", 2024-09-04T17:46:30.015 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.015 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.015 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.015 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:30.015 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.410786+0000", 2024-09-04T17:46:30.016 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0490000000000002e-06 2024-09-04T17:46:30.016 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.016 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.016 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:30.016 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.415833+0000", 2024-09-04T17:46:30.016 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0050473430000000001 2024-09-04T17:46:30.016 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.016 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.017 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:30.017 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.415853+0000", 2024-09-04T17:46:30.017 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9748000000000001e-05 2024-09-04T17:46:30.017 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.017 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.017 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:30.017 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.415861+0000", 2024-09-04T17:46:30.018 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.0160000000000008e-06 2024-09-04T17:46:30.018 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.018 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.018 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:30.018 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.415929+0000", 2024-09-04T17:46:30.018 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.8510999999999995e-05 2024-09-04T17:46:30.018 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.018 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.018 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:30.018 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.416039+0000", 2024-09-04T17:46:30.018 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00010938300000000001 2024-09-04T17:46:30.018 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.018 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.018 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 7", 2024-09-04T17:46:30.019 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.416401+0000", 2024-09-04T17:46:30.019 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000361858 2024-09-04T17:46:30.019 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.019 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.019 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:30.019 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.426442+0000", 2024-09-04T17:46:30.019 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.010041599999999999 2024-09-04T17:46:30.019 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.019 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.019 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:30.019 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.427370+0000", 2024-09-04T17:46:30.019 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00092744300000000002 2024-09-04T17:46:30.019 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.019 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.020 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:30.020 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.427439+0000", 2024-09-04T17:46:30.020 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.9239000000000006e-05 2024-09-04T17:46:30.020 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.020 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.020 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:30.020 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.427479+0000", 2024-09-04T17:46:30.020 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0552000000000001e-05 2024-09-04T17:46:30.020 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.020 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:30.020 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.020 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.021 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.021 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:5010 3.14 3:2de14b79:::rbd_data.1134cfbce83d.0000000000000002:head [delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:30.021 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.415884+0000", 2024-09-04T17:46:30.021 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.563600553000001, 2024-09-04T17:46:30.021 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.014765199999999999, 2024-09-04T17:46:30.021 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:30.021 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:30.021 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:30.021 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:30.022 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:30.022 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:30.022 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 5010 2024-09-04T17:46:30.022 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.022 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:30.022 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.022 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:30.022 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.415884+0000", 2024-09-04T17:46:30.022 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.022 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.022 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.022 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:30.022 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.415884+0000", 2024-09-04T17:46:30.022 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.022 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.023 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.023 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:30.023 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.415886+0000", 2024-09-04T17:46:30.023 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2900000000000001e-06 2024-09-04T17:46:30.023 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.023 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.023 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:30.023 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.415890+0000", 2024-09-04T17:46:30.023 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.8179999999999997e-06 2024-09-04T17:46:30.023 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.023 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.023 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:30.023 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.415897+0000", 2024-09-04T17:46:30.023 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7630000000000003e-06 2024-09-04T17:46:30.024 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.024 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.024 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:30.024 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.415900+0000", 2024-09-04T17:46:30.024 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.8500000000000004e-06 2024-09-04T17:46:30.024 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.024 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.024 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:30.024 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.420517+0000", 2024-09-04T17:46:30.024 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0046166219999999999 2024-09-04T17:46:30.024 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.024 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.024 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:30.024 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.420775+0000", 2024-09-04T17:46:30.025 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00025835399999999997 2024-09-04T17:46:30.025 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.025 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.025 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 7", 2024-09-04T17:46:30.025 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.423010+0000", 2024-09-04T17:46:30.025 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.002234691 2024-09-04T17:46:30.025 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.025 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.025 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:30.025 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.429961+0000", 2024-09-04T17:46:30.025 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0069508649999999996 2024-09-04T17:46:30.025 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.025 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.026 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:30.026 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.430511+0000", 2024-09-04T17:46:30.026 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00054961799999999998 2024-09-04T17:46:30.026 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.026 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.026 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:30.026 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.430580+0000", 2024-09-04T17:46:30.026 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.9139000000000003e-05 2024-09-04T17:46:30.026 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.026 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.026 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:30.026 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.430649+0000", 2024-09-04T17:46:30.026 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.9190000000000007e-05 2024-09-04T17:46:30.026 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.027 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:30.027 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.027 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.027 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.027 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:5384 3.14 3:283b8c0b:::rbd_data.113431586065.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 9392185~44999] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-04T17:46:30.027 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.714129+0000", 2024-09-04T17:46:30.027 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.265355243999998, 2024-09-04T17:46:30.027 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.021221803000000001, 2024-09-04T17:46:30.027 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:30.027 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:30.027 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:30.027 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:30.027 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:30.027 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:30.027 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 5384 2024-09-04T17:46:30.027 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.028 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:30.028 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.028 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:30.028 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.714129+0000", 2024-09-04T17:46:30.028 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.028 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.028 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.028 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:30.028 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.714129+0000", 2024-09-04T17:46:30.028 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.028 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.028 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.028 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:30.028 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.714131+0000", 2024-09-04T17:46:30.029 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1270000000000001e-06 2024-09-04T17:46:30.029 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.029 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.029 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:30.029 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.719337+0000", 2024-09-04T17:46:30.029 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0052062059999999997 2024-09-04T17:46:30.029 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.029 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.029 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:30.029 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.719361+0000", 2024-09-04T17:46:30.029 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4051999999999999e-05 2024-09-04T17:46:30.029 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.029 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.029 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:30.029 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.719373+0000", 2024-09-04T17:46:30.030 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1270000000000001e-05 2024-09-04T17:46:30.030 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.030 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.030 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:30.030 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.719449+0000", 2024-09-04T17:46:30.030 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.6785000000000004e-05 2024-09-04T17:46:30.030 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.030 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.030 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:30.030 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.719566+0000", 2024-09-04T17:46:30.030 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000116537 2024-09-04T17:46:30.030 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.030 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.030 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 7", 2024-09-04T17:46:30.031 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.719930+0000", 2024-09-04T17:46:30.031 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000363624 2024-09-04T17:46:30.031 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.031 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.031 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:30.031 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.730485+0000", 2024-09-04T17:46:30.031 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.010555266000000001 2024-09-04T17:46:30.031 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.031 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.031 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:30.031 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.735229+0000", 2024-09-04T17:46:30.031 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0047438369999999999 2024-09-04T17:46:30.031 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.031 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.031 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:30.032 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.735300+0000", 2024-09-04T17:46:30.032 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1408999999999993e-05 2024-09-04T17:46:30.032 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.032 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.032 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:30.032 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.735351+0000", 2024-09-04T17:46:30.032 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0689999999999997e-05 2024-09-04T17:46:30.032 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.032 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:30.032 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.032 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.032 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.032 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:5645 3.1d e31/24 3:bd9dd029:::rbd_data.113431586065.0000000000000006:head v 31'23, mlcod=30'20)", 2024-09-04T17:46:30.033 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:07.908329+0000", 2024-09-04T17:46:30.033 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.071155549, 2024-09-04T17:46:30.033 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.019759324000000002, 2024-09-04T17:46:30.033 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:30.033 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:30.033 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:30.033 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:30.033 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.033 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:30.033 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.908329+0000", 2024-09-04T17:46:30.033 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.033 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.033 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.034 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:30.034 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.908329+0000", 2024-09-04T17:46:30.034 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.034 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.034 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.034 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:30.034 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.908331+0000", 2024-09-04T17:46:30.034 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.3e-06 2024-09-04T17:46:30.034 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.034 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.034 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:30.034 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.912239+0000", 2024-09-04T17:46:30.035 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0039075730000000001 2024-09-04T17:46:30.035 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.035 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.035 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:30.035 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.912280+0000", 2024-09-04T17:46:30.035 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.1457e-05 2024-09-04T17:46:30.035 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.035 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.035 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:30.035 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.912298+0000", 2024-09-04T17:46:30.035 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7980000000000001e-05 2024-09-04T17:46:30.035 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.035 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.035 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:30.036 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.912432+0000", 2024-09-04T17:46:30.036 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000133516 2024-09-04T17:46:30.036 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.036 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.036 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:30.036 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.912452+0000", 2024-09-04T17:46:30.036 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0679999999999999e-05 2024-09-04T17:46:30.036 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.036 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.036 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:30.036 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.928026+0000", 2024-09-04T17:46:30.036 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.015574202000000001 2024-09-04T17:46:30.036 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.037 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.037 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:30.037 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:07.928088+0000", 2024-09-04T17:46:30.037 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.1616000000000001e-05 2024-09-04T17:46:30.037 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.037 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:30.037 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.037 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.037 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.037 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:6292 3.14 3:283b8c0b:::rbd_data.113431586065.0000000000000000:head [stat,truncate 2097152] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:30.037 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:08.428248+0000", 2024-09-04T17:46:30.037 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 21.55123631, 2024-09-04T17:46:30.038 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.017598231999999998, 2024-09-04T17:46:30.038 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:30.038 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:30.038 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:30.038 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:30.038 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:30.038 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:30.038 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 6292 2024-09-04T17:46:30.038 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.038 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:30.038 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.038 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:30.039 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.428248+0000", 2024-09-04T17:46:30.039 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.039 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.039 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.039 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:30.039 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.428248+0000", 2024-09-04T17:46:30.039 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.039 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.039 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.039 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:30.039 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.428249+0000", 2024-09-04T17:46:30.039 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.116e-06 2024-09-04T17:46:30.039 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.039 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.040 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:30.040 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.428254+0000", 2024-09-04T17:46:30.040 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0830000000000003e-06 2024-09-04T17:46:30.040 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.040 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.040 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:30.040 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.428271+0000", 2024-09-04T17:46:30.040 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6714e-05 2024-09-04T17:46:30.040 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.040 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.040 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:30.040 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.428279+0000", 2024-09-04T17:46:30.040 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.0690000000000002e-06 2024-09-04T17:46:30.040 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.041 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.041 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:30.041 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.428336+0000", 2024-09-04T17:46:30.041 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7336000000000001e-05 2024-09-04T17:46:30.041 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.041 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.041 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:30.041 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.428423+0000", 2024-09-04T17:46:30.041 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.6720999999999999e-05 2024-09-04T17:46:30.041 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.041 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.041 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 7", 2024-09-04T17:46:30.041 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.428609+0000", 2024-09-04T17:46:30.041 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00018606199999999999 2024-09-04T17:46:30.042 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.042 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.042 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:30.042 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.433297+0000", 2024-09-04T17:46:30.042 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0046874580000000002 2024-09-04T17:46:30.042 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.042 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.042 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:30.042 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.445776+0000", 2024-09-04T17:46:30.042 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.012479762 2024-09-04T17:46:30.042 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.042 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.042 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:30.042 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.445809+0000", 2024-09-04T17:46:30.043 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.2212000000000003e-05 2024-09-04T17:46:30.043 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.043 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.043 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:30.043 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.445846+0000", 2024-09-04T17:46:30.043 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7698999999999999e-05 2024-09-04T17:46:30.043 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.043 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:30.043 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.043 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.043 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.043 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:6301 3.f e32/24 3:f3ba70c3:::rbd_data.113431586065.0000000000000004:head v 32'17, mlcod=31'14)", 2024-09-04T17:46:30.043 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:08.456787+0000", 2024-09-04T17:46:30.044 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 21.522696985, 2024-09-04T17:46:30.044 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.015206687999999999, 2024-09-04T17:46:30.044 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:30.044 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:30.044 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:30.044 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:30.044 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.044 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:30.044 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.456787+0000", 2024-09-04T17:46:30.044 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.044 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.044 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.044 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:30.044 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.456787+0000", 2024-09-04T17:46:30.044 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.045 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.045 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.045 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:30.045 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.456789+0000", 2024-09-04T17:46:30.045 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.432e-06 2024-09-04T17:46:30.045 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.045 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.045 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:30.045 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.458945+0000", 2024-09-04T17:46:30.045 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0021565709999999999 2024-09-04T17:46:30.045 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.045 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.045 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:30.045 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.458976+0000", 2024-09-04T17:46:30.046 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0704000000000003e-05 2024-09-04T17:46:30.046 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.046 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.046 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:30.046 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.458995+0000", 2024-09-04T17:46:30.046 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8649999999999999e-05 2024-09-04T17:46:30.046 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.046 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.046 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:30.046 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.459094+0000", 2024-09-04T17:46:30.046 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.9302999999999994e-05 2024-09-04T17:46:30.046 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.046 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.046 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:30.047 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.459124+0000", 2024-09-04T17:46:30.047 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.9869000000000002e-05 2024-09-04T17:46:30.047 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.047 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.047 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:30.047 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.471957+0000", 2024-09-04T17:46:30.047 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.012833456 2024-09-04T17:46:30.047 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.047 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.047 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:30.047 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.471994+0000", 2024-09-04T17:46:30.047 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.6702999999999997e-05 2024-09-04T17:46:30.047 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.047 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:30.047 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.047 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.048 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.048 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:6311 3.9 3:941d6897:::rbd_data.113431586065.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:30.048 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:08.474866+0000", 2024-09-04T17:46:30.048 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 21.504617947, 2024-09-04T17:46:30.048 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.023556524999999998, 2024-09-04T17:46:30.048 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:30.048 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:30.048 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:30.048 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:30.048 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:30.048 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:30.048 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 6311 2024-09-04T17:46:30.048 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.049 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:30.049 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.049 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:30.049 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.474866+0000", 2024-09-04T17:46:30.049 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.049 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.049 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.049 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:30.049 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.474866+0000", 2024-09-04T17:46:30.049 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.049 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.049 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.049 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:30.050 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.474868+0000", 2024-09-04T17:46:30.050 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.607e-06 2024-09-04T17:46:30.050 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.050 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.050 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:30.050 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.478761+0000", 2024-09-04T17:46:30.050 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0038930269999999999 2024-09-04T17:46:30.050 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.050 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.050 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:30.050 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.478781+0000", 2024-09-04T17:46:30.050 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0372000000000001e-05 2024-09-04T17:46:30.050 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.050 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.051 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:30.051 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.478790+0000", 2024-09-04T17:46:30.051 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.6130000000000004e-06 2024-09-04T17:46:30.051 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.051 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.051 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:30.051 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.478868+0000", 2024-09-04T17:46:30.051 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.7589000000000005e-05 2024-09-04T17:46:30.051 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.051 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.051 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:30.051 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.479006+0000", 2024-09-04T17:46:30.051 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013884600000000001 2024-09-04T17:46:30.051 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.051 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.052 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 2", 2024-09-04T17:46:30.052 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.479366+0000", 2024-09-04T17:46:30.052 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00035964299999999999 2024-09-04T17:46:30.052 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.052 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.052 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:30.052 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.497050+0000", 2024-09-04T17:46:30.052 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.017683770000000001 2024-09-04T17:46:30.052 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.052 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.052 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:30.052 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.498311+0000", 2024-09-04T17:46:30.052 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.001261278 2024-09-04T17:46:30.052 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.053 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.053 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:30.053 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.498359+0000", 2024-09-04T17:46:30.053 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7675e-05 2024-09-04T17:46:30.053 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.053 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.053 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:30.053 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.498423+0000", 2024-09-04T17:46:30.053 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.4104999999999997e-05 2024-09-04T17:46:30.053 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.053 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:30.053 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.053 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.053 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.054 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:6312 3.9 3:941d6897:::rbd_data.113431586065.0000000000000008:head [truncate 1572864] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:30.054 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:08.478816+0000", 2024-09-04T17:46:30.054 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 21.500668286, 2024-09-04T17:46:30.054 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.020677345999999999, 2024-09-04T17:46:30.054 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:30.054 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:30.054 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:30.054 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:30.054 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:30.054 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:30.054 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 6312 2024-09-04T17:46:30.054 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.054 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:30.054 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.054 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:30.055 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.478816+0000", 2024-09-04T17:46:30.055 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.055 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.055 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.055 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:30.055 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.478816+0000", 2024-09-04T17:46:30.055 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.055 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.055 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.055 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:30.055 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.478818+0000", 2024-09-04T17:46:30.055 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6500000000000001e-06 2024-09-04T17:46:30.055 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.055 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.056 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:30.056 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.478822+0000", 2024-09-04T17:46:30.056 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0289999999999999e-06 2024-09-04T17:46:30.056 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.056 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.056 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:30.056 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.478831+0000", 2024-09-04T17:46:30.056 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.3850000000000008e-06 2024-09-04T17:46:30.056 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.056 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.056 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:30.056 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.478839+0000", 2024-09-04T17:46:30.056 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4749999999999996e-06 2024-09-04T17:46:30.057 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.057 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.057 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:30.057 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.485267+0000", 2024-09-04T17:46:30.057 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0064281199999999998 2024-09-04T17:46:30.057 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.057 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.057 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:30.057 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.485357+0000", 2024-09-04T17:46:30.057 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.0129000000000004e-05 2024-09-04T17:46:30.057 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.057 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.057 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 2", 2024-09-04T17:46:30.058 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.485514+0000", 2024-09-04T17:46:30.058 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00015751599999999999 2024-09-04T17:46:30.058 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.058 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.058 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:30.058 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.497433+0000", 2024-09-04T17:46:30.058 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.011918307 2024-09-04T17:46:30.058 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.058 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.058 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:30.058 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.499312+0000", 2024-09-04T17:46:30.058 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0018790040000000001 2024-09-04T17:46:30.058 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.058 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.059 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:30.059 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.499413+0000", 2024-09-04T17:46:30.059 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000101179 2024-09-04T17:46:30.059 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.059 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.059 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:30.059 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:08.499493+0000", 2024-09-04T17:46:30.059 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.0551999999999997e-05 2024-09-04T17:46:30.059 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.059 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:30.059 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.059 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.059 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.059 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:12722 3.1b 3:d8152af1:::rbd_data.1134ef3a5a7d.00000000000000bd:head [delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-04T17:46:30.059 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:16.421965+0000", 2024-09-04T17:46:30.060 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 13.557519377, 2024-09-04T17:46:30.060 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.890002811, 2024-09-04T17:46:30.060 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:30.060 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:30.060 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:30.060 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:30.060 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:30.060 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:30.060 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 12722 2024-09-04T17:46:30.060 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.060 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:30.060 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.060 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:30.061 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421965+0000", 2024-09-04T17:46:30.061 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.061 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.061 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.061 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:30.061 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421965+0000", 2024-09-04T17:46:30.061 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.061 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.061 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.061 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:30.061 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421966+0000", 2024-09-04T17:46:30.061 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.102e-06 2024-09-04T17:46:30.061 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.061 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.062 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:30.062 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421971+0000", 2024-09-04T17:46:30.062 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9989999999999999e-06 2024-09-04T17:46:30.062 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.062 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.062 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:30.062 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421981+0000", 2024-09-04T17:46:30.062 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.7380000000000006e-06 2024-09-04T17:46:30.062 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.062 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.062 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:30.062 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421989+0000", 2024-09-04T17:46:30.062 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.1890000000000007e-06 2024-09-04T17:46:30.062 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.062 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.063 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:30.063 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.422050+0000", 2024-09-04T17:46:30.063 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.0702000000000003e-05 2024-09-04T17:46:30.063 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.063 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.063 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for peered", 2024-09-04T17:46:30.063 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.422059+0000", 2024-09-04T17:46:30.063 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5440000000000004e-06 2024-09-04T17:46:30.063 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.063 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.063 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:30.063 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.308156+0000", 2024-09-04T17:46:30.063 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.88609676199999998 2024-09-04T17:46:30.063 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.064 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.064 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:30.064 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.308398+0000", 2024-09-04T17:46:30.064 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024215400000000001 2024-09-04T17:46:30.064 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.064 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.064 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 7", 2024-09-04T17:46:30.064 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.308795+0000", 2024-09-04T17:46:30.064 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000396704 2024-09-04T17:46:30.064 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.064 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.064 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:30.064 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.310831+0000", 2024-09-04T17:46:30.065 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0020364200000000002 2024-09-04T17:46:30.065 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.065 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.065 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:30.065 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.311818+0000", 2024-09-04T17:46:30.065 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00098649799999999993 2024-09-04T17:46:30.065 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.065 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.065 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:30.065 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.311905+0000", 2024-09-04T17:46:30.065 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7070999999999994e-05 2024-09-04T17:46:30.065 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.065 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.065 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:30.066 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.311968+0000", 2024-09-04T17:46:30.066 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.2928000000000005e-05 2024-09-04T17:46:30.066 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.066 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:30.066 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.066 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.066 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.066 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:19998 3.3 3:c7add6f4:::rbd_data.113418a7bbc1.0000000000000007:d [read 344064~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-04T17:46:30.066 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:21.756774+0000", 2024-09-04T17:46:30.066 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 8.2227106299999999, 2024-09-04T17:46:30.066 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.015397421999999999, 2024-09-04T17:46:30.066 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:30.066 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:30.067 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:30.067 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:30.067 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:30.067 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:30.067 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 19998 2024-09-04T17:46:30.067 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.067 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:30.067 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.067 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:30.067 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:21.756774+0000", 2024-09-04T17:46:30.067 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.067 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.067 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.067 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:30.068 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:21.756774+0000", 2024-09-04T17:46:30.068 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.068 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.068 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.068 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:30.068 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:21.756775+0000", 2024-09-04T17:46:30.068 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1429999999999999e-06 2024-09-04T17:46:30.068 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.068 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.068 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:30.068 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:21.756780+0000", 2024-09-04T17:46:30.068 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.8930000000000004e-06 2024-09-04T17:46:30.068 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.068 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.069 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:30.069 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:21.756797+0000", 2024-09-04T17:46:30.069 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7268000000000001e-05 2024-09-04T17:46:30.069 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.069 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.069 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:30.069 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:21.756813+0000", 2024-09-04T17:46:30.069 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6209e-05 2024-09-04T17:46:30.069 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.069 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.069 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:30.069 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:21.771691+0000", 2024-09-04T17:46:30.069 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.014877727 2024-09-04T17:46:30.070 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.070 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.070 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:30.070 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:21.771926+0000", 2024-09-04T17:46:30.070 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000234949 2024-09-04T17:46:30.070 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.070 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.070 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:30.070 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:21.772171+0000", 2024-09-04T17:46:30.070 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024523299999999997 2024-09-04T17:46:30.070 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.070 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:30.070 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.070 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.071 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.071 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:20370 3.15 3:ab7a1a4b:::rbd_data.1134fdde05d4.0000000000000144:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-04T17:46:30.071 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:22.031420+0000", 2024-09-04T17:46:30.071 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 7.948064703, 2024-09-04T17:46:30.071 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.018949611000000002, 2024-09-04T17:46:30.071 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:30.071 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:30.071 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:30.071 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:30.071 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:30.071 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:30.071 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 20370 2024-09-04T17:46:30.071 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.072 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:30.072 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.072 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:30.072 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.031420+0000", 2024-09-04T17:46:30.072 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.072 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.072 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.072 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:30.072 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.031420+0000", 2024-09-04T17:46:30.072 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.072 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.072 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.072 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:30.072 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.031421+0000", 2024-09-04T17:46:30.073 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5409999999999999e-06 2024-09-04T17:46:30.073 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.073 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.073 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:30.073 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.031436+0000", 2024-09-04T17:46:30.073 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4851000000000001e-05 2024-09-04T17:46:30.073 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.073 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.073 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:30.073 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.031453+0000", 2024-09-04T17:46:30.073 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6841e-05 2024-09-04T17:46:30.073 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.073 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.074 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:30.074 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.031466+0000", 2024-09-04T17:46:30.074 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2899e-05 2024-09-04T17:46:30.074 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.074 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.074 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:30.074 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.031544+0000", 2024-09-04T17:46:30.074 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.7811000000000004e-05 2024-09-04T17:46:30.074 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.074 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.074 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:30.074 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.031736+0000", 2024-09-04T17:46:30.074 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019233100000000001 2024-09-04T17:46:30.075 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.075 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.075 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 7", 2024-09-04T17:46:30.075 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.035047+0000", 2024-09-04T17:46:30.075 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0033107980000000002 2024-09-04T17:46:30.075 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.075 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.075 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:30.075 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.050001+0000", 2024-09-04T17:46:30.075 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.014953855 2024-09-04T17:46:30.075 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.075 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.075 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:30.075 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.050075+0000", 2024-09-04T17:46:30.076 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4808999999999995e-05 2024-09-04T17:46:30.076 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.076 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.076 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:30.076 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.050144+0000", 2024-09-04T17:46:30.076 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.8658000000000006e-05 2024-09-04T17:46:30.076 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.076 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.076 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:30.076 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.050369+0000", 2024-09-04T17:46:30.076 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000225217 2024-09-04T17:46:30.076 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.076 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:30.076 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.077 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.077 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.077 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:20372 3.9 3:928af84f:::rbd_data.1134fdde05d4.0000000000000146:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-04T17:46:30.077 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:22.033751+0000", 2024-09-04T17:46:30.077 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 7.9457337150000003, 2024-09-04T17:46:30.077 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.020054303999999998, 2024-09-04T17:46:30.077 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:30.077 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:30.077 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:30.077 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:30.077 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:30.077 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:30.077 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 20372 2024-09-04T17:46:30.077 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.078 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:30.078 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.078 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:30.078 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.033751+0000", 2024-09-04T17:46:30.078 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.078 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.078 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.078 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:30.078 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.033751+0000", 2024-09-04T17:46:30.078 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.078 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.078 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.078 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:30.078 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.033753+0000", 2024-09-04T17:46:30.079 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.898e-06 2024-09-04T17:46:30.079 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.079 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.079 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:30.079 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.033767+0000", 2024-09-04T17:46:30.079 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3909e-05 2024-09-04T17:46:30.079 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.079 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.079 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:30.079 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.033793+0000", 2024-09-04T17:46:30.079 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5210000000000001e-05 2024-09-04T17:46:30.079 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.079 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.080 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:30.080 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.033807+0000", 2024-09-04T17:46:30.080 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3995000000000001e-05 2024-09-04T17:46:30.080 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.080 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.080 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:30.080 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.033926+0000", 2024-09-04T17:46:30.080 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000119096 2024-09-04T17:46:30.080 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.080 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.080 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:30.080 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.034212+0000", 2024-09-04T17:46:30.080 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00028630000000000002 2024-09-04T17:46:30.081 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.081 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.081 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 2", 2024-09-04T17:46:30.081 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.038305+0000", 2024-09-04T17:46:30.081 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0040928910000000004 2024-09-04T17:46:30.081 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.081 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.081 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:30.081 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.053691+0000", 2024-09-04T17:46:30.081 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.015385853 2024-09-04T17:46:30.081 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.081 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.081 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:30.081 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.053708+0000", 2024-09-04T17:46:30.082 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6937999999999999e-05 2024-09-04T17:46:30.082 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.082 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.082 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:30.082 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.053758+0000", 2024-09-04T17:46:30.082 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0003999999999998e-05 2024-09-04T17:46:30.082 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.082 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.082 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:30.082 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.053805+0000", 2024-09-04T17:46:30.082 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7209999999999997e-05 2024-09-04T17:46:30.082 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.082 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:30.082 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.083 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.083 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.083 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:24189 3.1a 3:58ba337c:::rbd_data.1134fdde05d4.0000000000000090:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-04T17:46:30.083 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:22.366956+0000", 2024-09-04T17:46:30.083 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 7.6125278539999996, 2024-09-04T17:46:30.083 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.015895158999999999, 2024-09-04T17:46:30.083 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:30.083 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:30.083 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:30.084 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:30.084 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:30.084 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:30.084 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 24189 2024-09-04T17:46:30.084 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.084 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:30.084 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.084 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:30.084 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.366956+0000", 2024-09-04T17:46:30.084 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.084 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.084 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.084 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:30.084 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.366956+0000", 2024-09-04T17:46:30.085 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.085 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.085 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.085 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:30.085 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.366959+0000", 2024-09-04T17:46:30.085 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2919999999999998e-06 2024-09-04T17:46:30.085 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.085 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.085 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:30.085 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.366967+0000", 2024-09-04T17:46:30.085 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.2369999999999992e-06 2024-09-04T17:46:30.085 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.085 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.086 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:30.086 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.366984+0000", 2024-09-04T17:46:30.086 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6793999999999999e-05 2024-09-04T17:46:30.086 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.086 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.086 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:30.086 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.366992+0000", 2024-09-04T17:46:30.086 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.5469999999999997e-06 2024-09-04T17:46:30.086 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.086 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.086 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:30.086 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.367068+0000", 2024-09-04T17:46:30.086 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.5319000000000002e-05 2024-09-04T17:46:30.086 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.087 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.087 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:30.087 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.367229+0000", 2024-09-04T17:46:30.087 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000160903 2024-09-04T17:46:30.087 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.087 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.087 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 1", 2024-09-04T17:46:30.087 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.368287+0000", 2024-09-04T17:46:30.087 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0010584100000000001 2024-09-04T17:46:30.087 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.087 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.087 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:30.087 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.382769+0000", 2024-09-04T17:46:30.087 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.014481677 2024-09-04T17:46:30.088 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.088 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.088 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:30.088 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.382780+0000", 2024-09-04T17:46:30.088 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.113e-05 2024-09-04T17:46:30.088 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.088 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.088 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:30.088 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.382815+0000", 2024-09-04T17:46:30.088 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.5397e-05 2024-09-04T17:46:30.088 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.088 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.088 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:30.089 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:22.382852+0000", 2024-09-04T17:46:30.089 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.6452999999999998e-05 2024-09-04T17:46:30.089 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.089 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:30.089 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.089 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.089 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:30.089 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:30.090 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-04T17:46:30.091 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:30.088+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:30.170 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-04T17:46:30.216 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:30.216+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:30.258 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:30.259 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:46:30.259 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:46:30.259 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:30.273 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-04T17:46:30.304 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:30.305 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:46:30.305 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:46:30.305 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:30.320 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:30.318+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:30.363 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-04T17:46:30.387 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone13 from 0xe186827 (overlap 0xe186827) to 0x9dde281 2024-09-04T17:46:30.394 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:30.390+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:30.415 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:30.415 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:46:30.415 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:46:30.415 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:46:30.415 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:30.436 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-04T17:46:30.494 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:30.492+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:30.595 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:30.596+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:30.637 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:30.637 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:46:30.638 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:46:30.638 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:46:30.638 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.638 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8831 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2453504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:30.638 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:46:30.638 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.580841801999998, 2024-09-04T17:46:30.638 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14731111799999999, 2024-09-04T17:46:30.638 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:30.638 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:30.638 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:30.638 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:30.638 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:30.638 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:30.639 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8831 2024-09-04T17:46:30.639 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.639 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:30.639 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.639 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:30.639 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:46:30.639 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.639 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.639 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.639 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:30.639 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:46:30.639 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.639 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.639 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.640 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:30.640 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057811+0000", 2024-09-04T17:46:30.640 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.083e-06 2024-09-04T17:46:30.640 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.640 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.640 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:30.640 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057814+0000", 2024-09-04T17:46:30.640 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.407e-06 2024-09-04T17:46:30.640 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.640 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.640 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:30.640 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057825+0000", 2024-09-04T17:46:30.641 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.17e-05 2024-09-04T17:46:30.641 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.641 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.641 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:30.641 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057836+0000", 2024-09-04T17:46:30.641 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0283e-05 2024-09-04T17:46:30.641 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.641 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.641 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:30.641 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.204762+0000", 2024-09-04T17:46:30.641 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.146926205 2024-09-04T17:46:30.641 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.641 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.641 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:30.642 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.204895+0000", 2024-09-04T17:46:30.642 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013328999999999999 2024-09-04T17:46:30.642 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.642 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.642 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:30.642 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205121+0000", 2024-09-04T17:46:30.642 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00022614999999999999 2024-09-04T17:46:30.642 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.642 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:30.642 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.642 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.642 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.642 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8832 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2490368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:30.642 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:46:30.643 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.580803768999999, 2024-09-04T17:46:30.643 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14770012299999999, 2024-09-04T17:46:30.643 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:30.643 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:30.643 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:30.643 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:30.643 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:30.643 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:30.643 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8832 2024-09-04T17:46:30.643 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.643 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:30.643 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.643 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:30.644 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:46:30.644 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.644 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.644 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.644 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:30.644 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:46:30.644 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.644 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.644 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.644 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:30.644 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057849+0000", 2024-09-04T17:46:30.644 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6700000000000003e-07 2024-09-04T17:46:30.644 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.644 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.645 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:30.645 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057852+0000", 2024-09-04T17:46:30.645 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6359999999999998e-06 2024-09-04T17:46:30.645 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.645 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.645 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:30.645 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057860+0000", 2024-09-04T17:46:30.645 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8140000000000007e-06 2024-09-04T17:46:30.645 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.645 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.645 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:30.645 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057868+0000", 2024-09-04T17:46:30.645 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1999999999999997e-06 2024-09-04T17:46:30.646 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.646 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.646 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:30.646 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205208+0000", 2024-09-04T17:46:30.646 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14734067300000001 2024-09-04T17:46:30.646 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.646 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.646 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:30.646 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205347+0000", 2024-09-04T17:46:30.646 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013914900000000001 2024-09-04T17:46:30.646 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.646 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.647 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:30.647 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205548+0000", 2024-09-04T17:46:30.647 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000200984 2024-09-04T17:46:30.647 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.647 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:30.647 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.647 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.647 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.647 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8833 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2527232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:30.647 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:46:30.647 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.580771898999998, 2024-09-04T17:46:30.647 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148062572, 2024-09-04T17:46:30.647 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:30.647 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:30.648 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:30.648 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:30.648 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:30.648 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:30.648 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8833 2024-09-04T17:46:30.648 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.648 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:30.648 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.648 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:30.648 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:46:30.648 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.648 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.648 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.649 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:30.649 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:46:30.649 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.649 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.649 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.649 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:30.649 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057881+0000", 2024-09-04T17:46:30.649 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4300000000000002e-07 2024-09-04T17:46:30.649 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.649 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.649 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:30.649 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057893+0000", 2024-09-04T17:46:30.649 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1973999999999999e-05 2024-09-04T17:46:30.649 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.649 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.650 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:30.650 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057906+0000", 2024-09-04T17:46:30.650 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2716e-05 2024-09-04T17:46:30.650 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.650 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.650 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:30.650 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057913+0000", 2024-09-04T17:46:30.650 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3930000000000002e-06 2024-09-04T17:46:30.650 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.650 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.650 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:30.650 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205621+0000", 2024-09-04T17:46:30.650 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14770788800000001 2024-09-04T17:46:30.650 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.651 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.651 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:30.651 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205740+0000", 2024-09-04T17:46:30.651 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000118936 2024-09-04T17:46:30.651 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.651 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.651 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:30.651 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205943+0000", 2024-09-04T17:46:30.651 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020292199999999999 2024-09-04T17:46:30.651 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.651 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:30.651 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.651 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.651 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.652 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8834 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2564096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:30.652 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:46:30.652 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.580728619999999, 2024-09-04T17:46:30.652 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14846753200000001, 2024-09-04T17:46:30.652 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:30.652 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:30.652 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:30.652 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:30.652 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:30.652 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:30.652 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8834 2024-09-04T17:46:30.652 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.653 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:30.653 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.653 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:30.653 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:46:30.653 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.653 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.653 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.653 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:30.653 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:46:30.653 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.653 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.653 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.653 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:30.654 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057925+0000", 2024-09-04T17:46:30.654 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0270000000000001e-06 2024-09-04T17:46:30.654 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.654 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.654 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:30.654 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057936+0000", 2024-09-04T17:46:30.654 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.112e-05 2024-09-04T17:46:30.654 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.654 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.654 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:30.654 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057946+0000", 2024-09-04T17:46:30.654 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.009e-05 2024-09-04T17:46:30.654 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.654 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.655 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:30.655 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057953+0000", 2024-09-04T17:46:30.655 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.6229999999999997e-06 2024-09-04T17:46:30.655 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.655 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.655 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:30.655 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206027+0000", 2024-09-04T17:46:30.655 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148073591 2024-09-04T17:46:30.655 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.655 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.655 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:30.655 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206161+0000", 2024-09-04T17:46:30.655 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013411599999999999 2024-09-04T17:46:30.656 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.656 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.656 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:30.656 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206391+0000", 2024-09-04T17:46:30.656 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000229965 2024-09-04T17:46:30.656 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.656 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:30.656 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.656 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.656 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.656 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8835 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2600960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:30.656 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:46:30.656 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.580688970000001, 2024-09-04T17:46:30.657 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14882941599999999, 2024-09-04T17:46:30.657 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:30.657 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:30.657 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:30.657 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:30.657 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:30.657 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:30.657 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8835 2024-09-04T17:46:30.657 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.657 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:30.658 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.658 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:30.658 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:46:30.658 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.658 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.658 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.658 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:30.658 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:46:30.658 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.658 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.658 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.658 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:30.658 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058544+0000", 2024-09-04T17:46:30.659 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00058061899999999999 2024-09-04T17:46:30.659 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.659 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.659 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:30.659 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058550+0000", 2024-09-04T17:46:30.659 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.1129999999999997e-06 2024-09-04T17:46:30.659 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.659 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.659 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:30.659 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058566+0000", 2024-09-04T17:46:30.659 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5820000000000001e-05 2024-09-04T17:46:30.659 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.659 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.660 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:30.660 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058578+0000", 2024-09-04T17:46:30.660 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2537e-05 2024-09-04T17:46:30.660 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.660 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.660 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:30.660 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206470+0000", 2024-09-04T17:46:30.660 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14789153799999999 2024-09-04T17:46:30.660 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.660 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.660 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:30.660 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206589+0000", 2024-09-04T17:46:30.660 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000118837 2024-09-04T17:46:30.661 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.661 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.661 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:30.661 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206793+0000", 2024-09-04T17:46:30.661 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000203952 2024-09-04T17:46:30.661 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.661 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:30.661 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.661 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.661 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.661 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8836 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2654208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:30.661 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:46:30.661 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.580056282000001, 2024-09-04T17:46:30.662 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14862433999999999, 2024-09-04T17:46:30.662 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:30.662 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:30.662 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:30.662 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:30.662 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:30.662 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:30.662 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8836 2024-09-04T17:46:30.662 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.662 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:30.662 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.662 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:30.662 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:46:30.663 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.663 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.663 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.663 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:30.663 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:46:30.663 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.663 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.663 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.663 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:30.663 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062663+0000", 2024-09-04T17:46:30.663 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0040670619999999998 2024-09-04T17:46:30.663 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.663 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.663 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:30.664 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062668+0000", 2024-09-04T17:46:30.664 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4199999999999998e-06 2024-09-04T17:46:30.664 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.664 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.664 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:30.664 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062689+0000", 2024-09-04T17:46:30.664 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0356999999999998e-05 2024-09-04T17:46:30.664 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.664 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.664 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:30.664 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062726+0000", 2024-09-04T17:46:30.664 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7302999999999998e-05 2024-09-04T17:46:30.664 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.664 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.665 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:30.665 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206875+0000", 2024-09-04T17:46:30.665 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14414883100000001 2024-09-04T17:46:30.665 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.665 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.665 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:30.665 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.207016+0000", 2024-09-04T17:46:30.665 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000140822 2024-09-04T17:46:30.665 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.665 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.665 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:30.665 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.207220+0000", 2024-09-04T17:46:30.665 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020454499999999999 2024-09-04T17:46:30.666 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.666 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:30.666 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.666 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.666 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.666 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8844 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2928640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:30.666 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:46:30.666 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.575692198999999, 2024-09-04T17:46:30.666 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14752153100000001, 2024-09-04T17:46:30.666 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:30.666 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:30.666 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:30.666 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:30.666 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:30.666 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:30.667 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8844 2024-09-04T17:46:30.667 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.667 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:30.667 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.667 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:30.667 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:46:30.667 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.667 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.667 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.667 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:30.667 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:46:30.668 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.668 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.668 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.668 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:30.668 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062961+0000", 2024-09-04T17:46:30.668 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6700000000000003e-07 2024-09-04T17:46:30.668 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.668 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.668 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:30.668 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062973+0000", 2024-09-04T17:46:30.668 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2109999999999999e-05 2024-09-04T17:46:30.668 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.668 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.669 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:30.669 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062982+0000", 2024-09-04T17:46:30.669 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.6900000000000004e-06 2024-09-04T17:46:30.669 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.669 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.669 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:30.669 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062990+0000", 2024-09-04T17:46:30.669 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1899999999999998e-06 2024-09-04T17:46:30.669 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.669 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.669 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:30.669 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210090+0000", 2024-09-04T17:46:30.669 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14710026300000001 2024-09-04T17:46:30.670 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.670 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.670 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:30.670 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210225+0000", 2024-09-04T17:46:30.670 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013532400000000001 2024-09-04T17:46:30.670 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.670 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.670 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:30.670 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210481+0000", 2024-09-04T17:46:30.670 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00025628700000000001 2024-09-04T17:46:30.670 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.670 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:30.670 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.671 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.671 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.671 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8845 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2965504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:30.671 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:46:30.671 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.575652475999998, 2024-09-04T17:46:30.671 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147916819, 2024-09-04T17:46:30.671 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:30.671 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:30.671 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:30.671 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:30.671 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:30.671 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:30.671 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8845 2024-09-04T17:46:30.671 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.671 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:30.672 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.672 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:30.672 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:46:30.672 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.672 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.672 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.672 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:30.672 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:46:30.672 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.672 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.672 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.672 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:30.672 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:46:30.672 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.0399999999999995e-07 2024-09-04T17:46:30.673 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.673 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.673 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:30.673 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063003+0000", 2024-09-04T17:46:30.673 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6529999999999998e-06 2024-09-04T17:46:30.673 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.673 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.673 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:30.673 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063012+0000", 2024-09-04T17:46:30.673 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9500000000000007e-06 2024-09-04T17:46:30.673 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.673 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.673 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:30.674 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063027+0000", 2024-09-04T17:46:30.674 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5443000000000001e-05 2024-09-04T17:46:30.674 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.674 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.674 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:30.674 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210571+0000", 2024-09-04T17:46:30.674 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14754353100000001 2024-09-04T17:46:30.674 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.674 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.674 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:30.674 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210714+0000", 2024-09-04T17:46:30.674 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014295599999999999 2024-09-04T17:46:30.674 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.675 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.675 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:30.675 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210916+0000", 2024-09-04T17:46:30.675 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020258200000000001 2024-09-04T17:46:30.675 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.675 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:30.675 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.675 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.675 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.675 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8846 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3002368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:30.675 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:46:30.675 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.575614939000001, 2024-09-04T17:46:30.675 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14828011599999999, 2024-09-04T17:46:30.676 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:30.676 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:30.676 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:30.676 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:30.676 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:30.676 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:30.676 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8846 2024-09-04T17:46:30.676 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.676 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:30.676 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.676 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:30.676 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:46:30.676 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.677 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.677 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.677 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:30.677 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:46:30.677 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.677 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.677 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.677 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:30.677 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063038+0000", 2024-09-04T17:46:30.677 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.23e-07 2024-09-04T17:46:30.677 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.677 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.677 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:30.678 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063051+0000", 2024-09-04T17:46:30.678 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2757e-05 2024-09-04T17:46:30.678 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.678 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.678 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:30.678 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063070+0000", 2024-09-04T17:46:30.678 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9663000000000001e-05 2024-09-04T17:46:30.678 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.678 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.678 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:30.678 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063079+0000", 2024-09-04T17:46:30.678 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.0929999999999992e-06 2024-09-04T17:46:30.678 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.678 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.679 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:30.679 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210989+0000", 2024-09-04T17:46:30.679 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14790984500000001 2024-09-04T17:46:30.679 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.679 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.679 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:30.679 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211109+0000", 2024-09-04T17:46:30.679 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000120056 2024-09-04T17:46:30.679 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.679 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.679 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:30.679 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211317+0000", 2024-09-04T17:46:30.679 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020797900000000001 2024-09-04T17:46:30.679 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.680 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:30.680 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.680 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.680 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.680 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8847 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3039232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:30.680 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:46:30.680 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.575540506999999, 2024-09-04T17:46:30.680 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14859408900000001, 2024-09-04T17:46:30.680 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:30.680 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:30.680 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:30.680 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:30.680 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:30.680 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:30.681 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8847 2024-09-04T17:46:30.681 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.681 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:30.681 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.681 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:30.681 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:46:30.681 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.681 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.681 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.681 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:30.681 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:46:30.681 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.681 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.681 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.682 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:30.682 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063113+0000", 2024-09-04T17:46:30.682 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.204e-06 2024-09-04T17:46:30.682 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.682 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.682 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:30.682 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063116+0000", 2024-09-04T17:46:30.682 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.3330000000000001e-06 2024-09-04T17:46:30.682 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.682 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.682 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:30.682 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063126+0000", 2024-09-04T17:46:30.682 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0166999999999999e-05 2024-09-04T17:46:30.682 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.682 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.683 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:30.683 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063134+0000", 2024-09-04T17:46:30.683 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4329999999999998e-06 2024-09-04T17:46:30.683 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.683 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.683 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:30.683 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211392+0000", 2024-09-04T17:46:30.683 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148258109 2024-09-04T17:46:30.683 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.683 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.683 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:30.683 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211506+0000", 2024-09-04T17:46:30.683 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000113813 2024-09-04T17:46:30.683 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.684 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.684 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:30.684 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211706+0000", 2024-09-04T17:46:30.684 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020002999999999999 2024-09-04T17:46:30.684 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.684 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:30.684 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.684 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.684 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.684 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8848 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:30.684 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:46:30.684 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.575508169999999, 2024-09-04T17:46:30.684 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14907967599999999, 2024-09-04T17:46:30.684 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:30.685 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:30.685 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:30.685 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:30.685 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:30.685 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:30.685 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8848 2024-09-04T17:46:30.685 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.685 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:30.685 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.685 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:30.685 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:46:30.685 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.685 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.685 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.686 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:30.686 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:46:30.686 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.686 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.686 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.686 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:30.686 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070589+0000", 2024-09-04T17:46:30.686 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.007445386 2024-09-04T17:46:30.686 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.686 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.686 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:30.686 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070600+0000", 2024-09-04T17:46:30.686 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0346999999999999e-05 2024-09-04T17:46:30.687 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.687 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.687 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:30.687 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070622+0000", 2024-09-04T17:46:30.687 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2289999999999998e-05 2024-09-04T17:46:30.687 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.687 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.687 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:30.687 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070636+0000", 2024-09-04T17:46:30.687 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4093e-05 2024-09-04T17:46:30.687 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.687 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.687 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:30.687 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211815+0000", 2024-09-04T17:46:30.687 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14117874699999999 2024-09-04T17:46:30.688 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.688 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.688 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:30.688 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211983+0000", 2024-09-04T17:46:30.688 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00016852800000000001 2024-09-04T17:46:30.688 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.688 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.688 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:30.688 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.212224+0000", 2024-09-04T17:46:30.688 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024028499999999999 2024-09-04T17:46:30.688 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.688 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:30.688 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.688 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.688 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.689 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8862 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2019328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:30.689 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:46:30.689 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.567458702, 2024-09-04T17:46:30.689 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14755236399999999, 2024-09-04T17:46:30.689 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:30.689 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:30.689 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:30.689 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:30.689 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:30.689 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:30.689 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8862 2024-09-04T17:46:30.689 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.689 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:30.689 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.690 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:30.690 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:46:30.690 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.690 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.690 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.690 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:30.690 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:46:30.690 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.690 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.690 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.690 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:30.690 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071194+0000", 2024-09-04T17:46:30.690 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.5000000000000002e-07 2024-09-04T17:46:30.691 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.691 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.691 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:30.691 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071197+0000", 2024-09-04T17:46:30.691 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5900000000000002e-06 2024-09-04T17:46:30.691 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.691 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.691 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:30.691 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071206+0000", 2024-09-04T17:46:30.691 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9700000000000005e-06 2024-09-04T17:46:30.691 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.691 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.691 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:30.691 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071213+0000", 2024-09-04T17:46:30.691 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.9070000000000001e-06 2024-09-04T17:46:30.692 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.692 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.692 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:30.692 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218396+0000", 2024-09-04T17:46:30.692 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147183273 2024-09-04T17:46:30.692 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.692 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.692 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:30.692 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218538+0000", 2024-09-04T17:46:30.692 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014194199999999999 2024-09-04T17:46:30.692 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.692 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.692 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:30.692 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218746+0000", 2024-09-04T17:46:30.693 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000208032 2024-09-04T17:46:30.693 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.693 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:30.693 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.693 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.693 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.693 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8863 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2056192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:30.693 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:46:30.693 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.567430391999999, 2024-09-04T17:46:30.693 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14792281099999999, 2024-09-04T17:46:30.693 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:30.693 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:30.694 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:30.694 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:30.694 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:30.694 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:30.694 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8863 2024-09-04T17:46:30.694 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.694 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:30.694 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.694 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:30.694 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:46:30.694 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.694 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.694 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.694 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:30.694 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:46:30.695 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.695 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.695 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.695 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:30.695 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:46:30.695 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1299999999999999e-07 2024-09-04T17:46:30.695 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.695 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.695 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:30.695 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071225+0000", 2024-09-04T17:46:30.695 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6199999999999999e-06 2024-09-04T17:46:30.695 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.695 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.696 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:30.696 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071234+0000", 2024-09-04T17:46:30.696 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8769999999999999e-06 2024-09-04T17:46:30.696 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.696 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.696 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:30.696 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071251+0000", 2024-09-04T17:46:30.696 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.738e-05 2024-09-04T17:46:30.696 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.696 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.696 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:30.696 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218798+0000", 2024-09-04T17:46:30.696 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14754645 2024-09-04T17:46:30.696 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.696 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.697 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:30.697 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218926+0000", 2024-09-04T17:46:30.697 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000128446 2024-09-04T17:46:30.697 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.697 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.697 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:30.697 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219145+0000", 2024-09-04T17:46:30.697 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021832500000000001 2024-09-04T17:46:30.697 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.697 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:30.697 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.697 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.697 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.698 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8864 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2093056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:30.698 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:46:30.698 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.567388816000001, 2024-09-04T17:46:30.698 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148277309, 2024-09-04T17:46:30.698 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:30.698 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:30.698 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:30.698 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:30.698 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:30.698 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:30.698 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8864 2024-09-04T17:46:30.698 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.698 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:30.698 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.698 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:30.699 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:46:30.699 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.699 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.699 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.699 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:30.699 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:46:30.699 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.699 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.699 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.699 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:30.699 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071264+0000", 2024-09-04T17:46:30.699 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3e-07 2024-09-04T17:46:30.699 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.699 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.699 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:30.700 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071267+0000", 2024-09-04T17:46:30.700 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0139999999999999e-06 2024-09-04T17:46:30.700 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.700 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.700 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:30.700 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071276+0000", 2024-09-04T17:46:30.700 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1689999999999998e-06 2024-09-04T17:46:30.700 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.700 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.700 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:30.700 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071283+0000", 2024-09-04T17:46:30.700 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2269999999999996e-06 2024-09-04T17:46:30.700 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.701 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.701 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:30.701 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219195+0000", 2024-09-04T17:46:30.701 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147911868 2024-09-04T17:46:30.701 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.701 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.701 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:30.701 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219324+0000", 2024-09-04T17:46:30.701 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00012835900000000001 2024-09-04T17:46:30.701 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.701 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.701 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:30.701 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219541+0000", 2024-09-04T17:46:30.701 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000216942 2024-09-04T17:46:30.701 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.702 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:30.702 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.702 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.702 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.702 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8865 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2109440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:30.702 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:46:30.702 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.567359356000001, 2024-09-04T17:46:30.702 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14865747300000001, 2024-09-04T17:46:30.702 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:30.702 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:30.702 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:30.702 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:30.702 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:30.702 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:30.703 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8865 2024-09-04T17:46:30.703 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.703 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:30.703 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.703 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:30.703 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:46:30.703 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.703 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.703 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.703 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:30.703 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:46:30.703 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.703 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.703 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.703 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:30.704 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:46:30.704 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7999999999999995e-07 2024-09-04T17:46:30.704 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.704 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.704 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:30.704 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071296+0000", 2024-09-04T17:46:30.704 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6900000000000001e-06 2024-09-04T17:46:30.704 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.704 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.704 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:30.704 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071305+0000", 2024-09-04T17:46:30.704 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.5499999999999995e-06 2024-09-04T17:46:30.704 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.704 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.705 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:30.705 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071312+0000", 2024-09-04T17:46:30.705 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1559999999999998e-06 2024-09-04T17:46:30.705 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.705 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.705 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:30.705 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219591+0000", 2024-09-04T17:46:30.705 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14827944900000001 2024-09-04T17:46:30.705 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.705 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.705 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:30.705 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219731+0000", 2024-09-04T17:46:30.706 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013931600000000001 2024-09-04T17:46:30.706 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.706 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.706 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:30.706 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219950+0000", 2024-09-04T17:46:30.706 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021963199999999999 2024-09-04T17:46:30.706 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.706 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:30.706 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.706 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.706 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.706 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8866 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2146304~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:30.706 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:46:30.707 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.567331122999999, 2024-09-04T17:46:30.707 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14906117899999999, 2024-09-04T17:46:30.707 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:30.707 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:30.707 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:30.707 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:30.707 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:30.707 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:30.707 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8866 2024-09-04T17:46:30.707 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.707 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:30.707 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.707 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:30.707 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:46:30.707 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.708 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.708 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.708 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:30.708 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:46:30.708 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.708 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.708 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.708 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:30.708 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071322+0000", 2024-09-04T17:46:30.708 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7999999999999995e-07 2024-09-04T17:46:30.708 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.708 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.708 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:30.708 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071338+0000", 2024-09-04T17:46:30.708 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5872999999999999e-05 2024-09-04T17:46:30.709 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.709 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.709 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:30.709 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071347+0000", 2024-09-04T17:46:30.709 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4669999999999993e-06 2024-09-04T17:46:30.709 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.709 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.709 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:30.709 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071354+0000", 2024-09-04T17:46:30.709 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2099999999999996e-06 2024-09-04T17:46:30.709 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.709 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.709 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:30.710 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220008+0000", 2024-09-04T17:46:30.710 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148653642 2024-09-04T17:46:30.710 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.710 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.710 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:30.710 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220155+0000", 2024-09-04T17:46:30.710 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014726600000000001 2024-09-04T17:46:30.710 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.710 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.710 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:30.710 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220382+0000", 2024-09-04T17:46:30.710 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00022704100000000001 2024-09-04T17:46:30.710 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.710 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:30.710 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.711 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.711 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.711 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8867 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2183168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:30.711 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:46:30.711 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.56728777, 2024-09-04T17:46:30.711 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.149445831, 2024-09-04T17:46:30.711 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:30.711 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:30.711 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:30.711 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:30.711 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:30.711 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:30.711 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8867 2024-09-04T17:46:30.711 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.712 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:30.712 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.712 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:30.712 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:46:30.712 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.712 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.712 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.712 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:30.712 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:46:30.712 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.712 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.712 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.712 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:30.712 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078653+0000", 2024-09-04T17:46:30.712 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0072890430000000003 2024-09-04T17:46:30.713 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.713 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.713 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:30.713 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078664+0000", 2024-09-04T17:46:30.713 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0307e-05 2024-09-04T17:46:30.713 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.713 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.713 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:30.713 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078690+0000", 2024-09-04T17:46:30.713 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6780000000000001e-05 2024-09-04T17:46:30.713 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.713 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.713 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:30.713 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078706+0000", 2024-09-04T17:46:30.714 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5676999999999999e-05 2024-09-04T17:46:30.714 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.714 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.714 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:30.714 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220467+0000", 2024-09-04T17:46:30.714 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14176103500000001 2024-09-04T17:46:30.714 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.714 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.714 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:30.714 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220595+0000", 2024-09-04T17:46:30.714 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000127427 2024-09-04T17:46:30.714 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.714 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.714 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:30.715 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220810+0000", 2024-09-04T17:46:30.715 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021556200000000001 2024-09-04T17:46:30.715 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.715 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:30.715 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.715 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.715 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.715 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8880 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2658304~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:30.715 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.079158+0000", 2024-09-04T17:46:30.715 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.559494490999999, 2024-09-04T17:46:30.715 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14718968499999999, 2024-09-04T17:46:30.715 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:30.715 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:30.715 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:30.716 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:30.716 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:30.716 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:30.716 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8880 2024-09-04T17:46:30.716 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.716 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:30.716 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.716 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:30.716 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079158+0000", 2024-09-04T17:46:30.716 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.716 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.716 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.716 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:30.717 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079158+0000", 2024-09-04T17:46:30.717 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.717 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.717 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.717 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:30.717 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079158+0000", 2024-09-04T17:46:30.717 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3e-07 2024-09-04T17:46:30.717 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.717 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.717 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:30.717 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079161+0000", 2024-09-04T17:46:30.717 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.57e-06 2024-09-04T17:46:30.718 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.718 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.718 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:30.718 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079170+0000", 2024-09-04T17:46:30.718 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.0170000000000002e-06 2024-09-04T17:46:30.718 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.718 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.718 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:30.718 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079177+0000", 2024-09-04T17:46:30.718 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.0530000000000001e-06 2024-09-04T17:46:30.718 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.718 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.718 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:30.718 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.225946+0000", 2024-09-04T17:46:30.719 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14676936500000001 2024-09-04T17:46:30.719 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.719 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.719 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:30.719 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226098+0000", 2024-09-04T17:46:30.719 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000151805 2024-09-04T17:46:30.719 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.719 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.719 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:30.719 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226347+0000", 2024-09-04T17:46:30.719 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024914499999999999 2024-09-04T17:46:30.719 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.719 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:30.719 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.720 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.720 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.720 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8881 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2695168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:30.720 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:46:30.720 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.559465701000001, 2024-09-04T17:46:30.720 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147543547, 2024-09-04T17:46:30.720 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:30.720 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:30.720 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:30.720 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:30.720 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:30.720 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:30.721 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8881 2024-09-04T17:46:30.721 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.721 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:30.721 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.721 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:30.721 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:46:30.721 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.721 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.721 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.721 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:30.721 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:46:30.721 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.721 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.721 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.722 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:30.722 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079187+0000", 2024-09-04T17:46:30.722 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7700000000000004e-07 2024-09-04T17:46:30.722 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.722 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.722 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:30.722 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079198+0000", 2024-09-04T17:46:30.722 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0849999999999999e-05 2024-09-04T17:46:30.722 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.722 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.722 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:30.722 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079207+0000", 2024-09-04T17:46:30.722 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5599999999999999e-06 2024-09-04T17:46:30.722 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.723 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.723 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:30.723 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079224+0000", 2024-09-04T17:46:30.723 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6387000000000001e-05 2024-09-04T17:46:30.723 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.723 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.723 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:30.723 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226414+0000", 2024-09-04T17:46:30.723 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14718975000000001 2024-09-04T17:46:30.723 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.723 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.723 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:30.724 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226534+0000", 2024-09-04T17:46:30.724 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000120288 2024-09-04T17:46:30.724 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.724 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.724 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:30.724 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226730+0000", 2024-09-04T17:46:30.724 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000196035 2024-09-04T17:46:30.724 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.724 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:30.724 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.724 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.724 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.724 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8882 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2732032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:30.724 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:46:30.725 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.559417487000001, 2024-09-04T17:46:30.725 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14795022399999999, 2024-09-04T17:46:30.725 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:30.725 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:30.725 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:30.725 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:30.725 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:30.725 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:30.725 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8882 2024-09-04T17:46:30.725 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.725 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:30.725 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.725 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:30.725 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:46:30.726 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.726 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.726 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.726 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:30.726 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:46:30.726 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:30.726 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.726 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.726 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:30.726 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086588+0000", 2024-09-04T17:46:30.726 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0073530959999999999 2024-09-04T17:46:30.726 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.726 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.727 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:30.727 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086598+0000", 2024-09-04T17:46:30.727 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0122999999999999e-05 2024-09-04T17:46:30.727 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.727 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.727 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:30.727 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086626+0000", 2024-09-04T17:46:30.727 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.7914000000000001e-05 2024-09-04T17:46:30.727 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.727 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.727 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:30.727 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086641+0000", 2024-09-04T17:46:30.727 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5270000000000001e-05 2024-09-04T17:46:30.727 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.727 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.728 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:30.728 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226801+0000", 2024-09-04T17:46:30.728 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.140160321 2024-09-04T17:46:30.728 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.728 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.728 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:30.728 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226951+0000", 2024-09-04T17:46:30.728 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014965200000000001 2024-09-04T17:46:30.728 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:30.728 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:30.728 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:30.728 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.227185+0000", 2024-09-04T17:46:30.728 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00023384799999999999 2024-09-04T17:46:30.729 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.729 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:30.729 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.729 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:30.729 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:30.729 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:30.730 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-04T17:46:30.731 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:30.726+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:30.804 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #12, image image_client.0-clone12 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent13 2024-09-04T17:46:30.831 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:30.830+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:30.901 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:30.901 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:46:30.901 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:46:30.901 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:30.915 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-04T17:46:30.932 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:46:30.932+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:46:31.033 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:46:31.032+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:46:31.082 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:31.082 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:46:31.082 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:46:31.082 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:46:31.082 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:31.096 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-04T17:46:31.134 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:31.132+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:31.199 INFO:teuthology.orchestra.run.smithi092.stdout:1676 writesame 0x2f544db thru 0x2f5f022 data_size 0xe(0xab48 bytes) 2024-09-04T17:46:31.219 INFO:teuthology.orchestra.run.smithi092.stdout:1677 writesame 0xbfc7bb7 thru 0xbfd5232 data_size 0x6a(0xd67c bytes) 2024-09-04T17:46:31.222 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:31.223 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:46:31.223 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:46:31.223 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:46:31.223 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.223 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:13127 3.1e 3:7bbed1a3:::rbd_data.11345bd5898b.0000000000000003:a [read 331776~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:31.223 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:17.905384+0000", 2024-09-04T17:46:31.223 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 13.317857657999999, 2024-09-04T17:46:31.223 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.059561699000000003, 2024-09-04T17:46:31.223 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:31.223 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:31.223 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:31.224 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:31.224 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:31.224 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:31.224 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 13127 2024-09-04T17:46:31.224 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.224 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:31.224 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.224 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:31.224 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905384+0000", 2024-09-04T17:46:31.224 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.224 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.224 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.224 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:31.225 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905384+0000", 2024-09-04T17:46:31.225 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.225 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.225 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.225 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:31.225 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905385+0000", 2024-09-04T17:46:31.225 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.144e-06 2024-09-04T17:46:31.225 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.225 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.226 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:31.226 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905390+0000", 2024-09-04T17:46:31.226 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0289999999999996e-06 2024-09-04T17:46:31.226 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.226 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.226 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:31.226 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905407+0000", 2024-09-04T17:46:31.226 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7322000000000001e-05 2024-09-04T17:46:31.226 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.226 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.226 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:31.226 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905421+0000", 2024-09-04T17:46:31.226 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3309000000000001e-05 2024-09-04T17:46:31.226 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.227 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.227 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:31.227 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.964510+0000", 2024-09-04T17:46:31.227 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.059089147000000002 2024-09-04T17:46:31.227 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.227 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.227 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:31.227 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.964684+0000", 2024-09-04T17:46:31.227 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000174463 2024-09-04T17:46:31.227 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.227 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.227 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:31.227 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.964946+0000", 2024-09-04T17:46:31.227 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00026128499999999999 2024-09-04T17:46:31.228 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.228 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:31.228 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.228 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.228 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.228 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:13128 3.1e 3:7bbed1a3:::rbd_data.11345bd5898b.0000000000000003:a [read 368640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:31.228 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:17.905438+0000", 2024-09-04T17:46:31.228 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 13.317803313000001, 2024-09-04T17:46:31.228 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.060341850000000002, 2024-09-04T17:46:31.228 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:31.228 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:31.228 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:31.228 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:31.228 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:31.229 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:31.229 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 13128 2024-09-04T17:46:31.229 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.229 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:31.229 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.229 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:31.229 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905438+0000", 2024-09-04T17:46:31.229 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.229 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.229 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.229 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:31.229 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905438+0000", 2024-09-04T17:46:31.229 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.229 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.230 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.230 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:31.230 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905439+0000", 2024-09-04T17:46:31.230 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1599999999999999e-06 2024-09-04T17:46:31.230 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.230 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.230 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:31.230 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905458+0000", 2024-09-04T17:46:31.230 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8961e-05 2024-09-04T17:46:31.230 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.230 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.230 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:31.230 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905476+0000", 2024-09-04T17:46:31.230 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7861999999999999e-05 2024-09-04T17:46:31.231 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.231 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.231 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:31.231 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905489+0000", 2024-09-04T17:46:31.231 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3232e-05 2024-09-04T17:46:31.231 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.231 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.231 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:31.231 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.965026+0000", 2024-09-04T17:46:31.231 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.059536981000000003 2024-09-04T17:46:31.231 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.231 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.231 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:31.232 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.965252+0000", 2024-09-04T17:46:31.232 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000225462 2024-09-04T17:46:31.232 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.232 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.232 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:31.232 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.965780+0000", 2024-09-04T17:46:31.232 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00052819200000000003 2024-09-04T17:46:31.232 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.232 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:31.232 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.232 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.232 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.232 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:13129 3.1e 3:7bbed1a3:::rbd_data.11345bd5898b.0000000000000003:a [read 405504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:31.233 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:17.905510+0000", 2024-09-04T17:46:31.233 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 13.317731923, 2024-09-04T17:46:31.233 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.060788440999999999, 2024-09-04T17:46:31.233 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:31.233 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:31.233 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:31.233 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:31.233 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:31.233 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:31.233 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 13129 2024-09-04T17:46:31.233 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.233 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:31.233 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.233 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:31.234 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905510+0000", 2024-09-04T17:46:31.234 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.234 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.234 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.234 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:31.234 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905510+0000", 2024-09-04T17:46:31.234 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.234 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.234 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.234 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:31.234 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905511+0000", 2024-09-04T17:46:31.234 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1659999999999999e-06 2024-09-04T17:46:31.234 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.234 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.234 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:31.235 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905516+0000", 2024-09-04T17:46:31.235 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9899999999999997e-06 2024-09-04T17:46:31.235 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.235 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.235 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:31.235 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905534+0000", 2024-09-04T17:46:31.235 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7808e-05 2024-09-04T17:46:31.235 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.235 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.235 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:31.235 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905547+0000", 2024-09-04T17:46:31.235 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3135999999999999e-05 2024-09-04T17:46:31.235 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.235 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.235 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:31.236 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.965898+0000", 2024-09-04T17:46:31.236 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.060351162 2024-09-04T17:46:31.236 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.236 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.236 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:31.236 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.966051+0000", 2024-09-04T17:46:31.236 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00015318299999999999 2024-09-04T17:46:31.236 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.236 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.236 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:31.236 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.966298+0000", 2024-09-04T17:46:31.236 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024699599999999998 2024-09-04T17:46:31.236 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.236 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:31.236 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.237 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.237 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.237 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:13130 3.1e 3:7bbed1a3:::rbd_data.11345bd5898b.0000000000000003:a [read 442368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:31.237 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:17.905565+0000", 2024-09-04T17:46:31.237 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 13.317676347999999, 2024-09-04T17:46:31.237 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.061174443000000002, 2024-09-04T17:46:31.237 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:31.237 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:31.237 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:31.237 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:31.237 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:31.237 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:31.238 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 13130 2024-09-04T17:46:31.238 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.238 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:31.238 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.238 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:31.238 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905565+0000", 2024-09-04T17:46:31.238 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.238 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.238 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.238 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:31.238 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905565+0000", 2024-09-04T17:46:31.238 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.238 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.238 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.238 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:31.239 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905567+0000", 2024-09-04T17:46:31.239 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6300000000000001e-06 2024-09-04T17:46:31.239 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.239 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.239 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:31.239 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905572+0000", 2024-09-04T17:46:31.239 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9790000000000001e-06 2024-09-04T17:46:31.239 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.239 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.239 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:31.239 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905590+0000", 2024-09-04T17:46:31.239 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8031999999999999e-05 2024-09-04T17:46:31.239 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.239 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.239 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:31.240 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905602+0000", 2024-09-04T17:46:31.240 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2651e-05 2024-09-04T17:46:31.240 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.240 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.240 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:31.240 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.966373+0000", 2024-09-04T17:46:31.240 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.060770532000000002 2024-09-04T17:46:31.240 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.240 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.240 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:31.240 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.966514+0000", 2024-09-04T17:46:31.240 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014103 2024-09-04T17:46:31.240 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.240 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.241 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:31.241 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.966740+0000", 2024-09-04T17:46:31.241 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000225589 2024-09-04T17:46:31.241 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.241 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:31.241 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.241 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.241 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.241 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:13131 3.1e 3:7bbed1a3:::rbd_data.11345bd5898b.0000000000000003:a [read 479232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:31.241 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:17.905621+0000", 2024-09-04T17:46:31.241 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 13.317620723999999, 2024-09-04T17:46:31.241 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.061498005000000001, 2024-09-04T17:46:31.241 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:31.241 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:31.242 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:31.242 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:31.242 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:31.242 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:31.242 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 13131 2024-09-04T17:46:31.242 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.242 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:31.242 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.242 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:31.242 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905621+0000", 2024-09-04T17:46:31.242 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.242 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.242 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.243 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:31.243 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905621+0000", 2024-09-04T17:46:31.243 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.243 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.243 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.243 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:31.243 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905622+0000", 2024-09-04T17:46:31.243 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.313e-06 2024-09-04T17:46:31.243 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.243 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.243 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:31.243 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905627+0000", 2024-09-04T17:46:31.243 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0100000000000003e-06 2024-09-04T17:46:31.243 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.244 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.244 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:31.244 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905644+0000", 2024-09-04T17:46:31.244 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6918e-05 2024-09-04T17:46:31.244 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.244 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.244 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:31.244 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905657+0000", 2024-09-04T17:46:31.244 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2496e-05 2024-09-04T17:46:31.244 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.244 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.244 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:31.244 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.966805+0000", 2024-09-04T17:46:31.244 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.061148518999999998 2024-09-04T17:46:31.245 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.245 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.245 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:31.245 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.966920+0000", 2024-09-04T17:46:31.245 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00011513700000000001 2024-09-04T17:46:31.245 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.245 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.245 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:31.245 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.967119+0000", 2024-09-04T17:46:31.245 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000198612 2024-09-04T17:46:31.245 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.245 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:31.245 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.245 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.246 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.246 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:13132 3.1e 3:7bbed1a3:::rbd_data.11345bd5898b.0000000000000003:a [read 516096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:31.246 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:17.905674+0000", 2024-09-04T17:46:31.246 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 13.317567158999999, 2024-09-04T17:46:31.246 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.061794957999999997, 2024-09-04T17:46:31.246 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:31.246 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:31.246 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:31.246 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:31.246 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:31.246 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:31.246 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 13132 2024-09-04T17:46:31.246 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.246 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:31.246 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.247 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:31.247 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905674+0000", 2024-09-04T17:46:31.247 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.247 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.247 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.247 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:31.247 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905674+0000", 2024-09-04T17:46:31.247 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.247 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.247 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.247 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:31.247 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905676+0000", 2024-09-04T17:46:31.247 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1230000000000001e-06 2024-09-04T17:46:31.247 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.247 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.248 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:31.248 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905680+0000", 2024-09-04T17:46:31.248 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9169999999999997e-06 2024-09-04T17:46:31.248 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.248 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.248 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:31.248 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905699+0000", 2024-09-04T17:46:31.248 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9023999999999999e-05 2024-09-04T17:46:31.248 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.248 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.248 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:31.248 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905713+0000", 2024-09-04T17:46:31.248 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3115999999999999e-05 2024-09-04T17:46:31.248 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.249 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.249 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:31.249 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.967167+0000", 2024-09-04T17:46:31.249 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.061454323999999998 2024-09-04T17:46:31.249 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.249 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.249 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:31.249 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.967280+0000", 2024-09-04T17:46:31.249 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000113269 2024-09-04T17:46:31.249 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.249 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.249 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:31.249 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.967469+0000", 2024-09-04T17:46:31.249 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000189185 2024-09-04T17:46:31.250 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.250 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:31.250 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.250 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.250 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.250 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:13133 3.1e 3:7bbed1a3:::rbd_data.11345bd5898b.0000000000000003:a [read 806912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:31.250 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:17.905745+0000", 2024-09-04T17:46:31.250 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 13.317496886000001, 2024-09-04T17:46:31.250 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.062050424999999999, 2024-09-04T17:46:31.250 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:31.250 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:31.250 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:31.250 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:31.250 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:31.250 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:31.251 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 13133 2024-09-04T17:46:31.251 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.251 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:31.251 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.251 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:31.251 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905745+0000", 2024-09-04T17:46:31.251 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.251 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.251 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.251 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:31.251 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905745+0000", 2024-09-04T17:46:31.251 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.251 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.252 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.252 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:31.252 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905746+0000", 2024-09-04T17:46:31.252 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0839999999999999e-06 2024-09-04T17:46:31.252 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.252 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.252 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:31.252 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905762+0000", 2024-09-04T17:46:31.252 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5917999999999999e-05 2024-09-04T17:46:31.252 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.252 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.252 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:31.252 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905780+0000", 2024-09-04T17:46:31.252 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7830999999999999e-05 2024-09-04T17:46:31.253 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.253 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.253 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:31.253 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905804+0000", 2024-09-04T17:46:31.253 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4912000000000001e-05 2024-09-04T17:46:31.253 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.253 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.253 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:31.253 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.967515+0000", 2024-09-04T17:46:31.253 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.061710458000000003 2024-09-04T17:46:31.254 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.254 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.254 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:31.254 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.967619+0000", 2024-09-04T17:46:31.254 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00010409899999999999 2024-09-04T17:46:31.254 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.254 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.254 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:31.254 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.967795+0000", 2024-09-04T17:46:31.254 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00017612300000000001 2024-09-04T17:46:31.255 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.255 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:31.255 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.255 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.255 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.255 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:13134 3.1e 3:7bbed1a3:::rbd_data.11345bd5898b.0000000000000003:a [read 843776~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:31.255 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:17.905824+0000", 2024-09-04T17:46:31.255 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 13.317418050000001, 2024-09-04T17:46:31.255 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.062450675999999997, 2024-09-04T17:46:31.255 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:31.255 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:31.255 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:31.255 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:31.255 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:31.256 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:31.256 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 13134 2024-09-04T17:46:31.256 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.256 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:31.256 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.256 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:31.256 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905824+0000", 2024-09-04T17:46:31.256 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.256 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.256 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.256 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:31.256 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905824+0000", 2024-09-04T17:46:31.256 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.256 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.256 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.257 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:31.257 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905825+0000", 2024-09-04T17:46:31.257 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.717e-06 2024-09-04T17:46:31.257 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.257 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.257 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:31.257 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905830+0000", 2024-09-04T17:46:31.257 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2220000000000001e-06 2024-09-04T17:46:31.257 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.257 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.257 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:31.257 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905848+0000", 2024-09-04T17:46:31.257 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7635e-05 2024-09-04T17:46:31.257 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.258 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.258 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:31.258 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905873+0000", 2024-09-04T17:46:31.258 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5160999999999998e-05 2024-09-04T17:46:31.258 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.258 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.258 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:31.258 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.967894+0000", 2024-09-04T17:46:31.258 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.062020947999999999 2024-09-04T17:46:31.258 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.258 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.258 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:31.258 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.968038+0000", 2024-09-04T17:46:31.258 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000143442 2024-09-04T17:46:31.258 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.258 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.259 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:31.259 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.968274+0000", 2024-09-04T17:46:31.259 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00023655099999999999 2024-09-04T17:46:31.259 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.259 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:31.259 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.259 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.259 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.259 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:13135 3.1e 3:7bbed1a3:::rbd_data.11345bd5898b.0000000000000003:a [read 880640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:31.259 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:17.905893+0000", 2024-09-04T17:46:31.259 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 13.317348634, 2024-09-04T17:46:31.259 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.062733087000000007, 2024-09-04T17:46:31.259 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:31.260 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:31.260 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:31.260 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:31.260 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:31.260 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:31.260 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 13135 2024-09-04T17:46:31.260 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.260 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:31.260 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.260 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:31.260 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905893+0000", 2024-09-04T17:46:31.260 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.260 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.260 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.260 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:31.261 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905893+0000", 2024-09-04T17:46:31.261 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.261 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.261 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.261 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:31.261 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905895+0000", 2024-09-04T17:46:31.261 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6139999999999999e-06 2024-09-04T17:46:31.261 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.261 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.261 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:31.261 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905900+0000", 2024-09-04T17:46:31.261 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2360000000000004e-06 2024-09-04T17:46:31.261 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.261 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.262 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:31.262 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905917+0000", 2024-09-04T17:46:31.262 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7642000000000002e-05 2024-09-04T17:46:31.262 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.262 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.262 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:31.262 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905930+0000", 2024-09-04T17:46:31.262 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2914999999999999e-05 2024-09-04T17:46:31.262 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.262 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.262 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:31.262 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.968339+0000", 2024-09-04T17:46:31.262 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.062408533000000002 2024-09-04T17:46:31.262 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.262 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.263 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:31.263 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.968446+0000", 2024-09-04T17:46:31.263 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000107481 2024-09-04T17:46:31.263 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.263 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.263 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:31.263 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.968626+0000", 2024-09-04T17:46:31.263 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00017966600000000001 2024-09-04T17:46:31.263 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.263 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:31.263 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.263 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.263 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.263 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:13136 3.1e 3:7bbed1a3:::rbd_data.11345bd5898b.0000000000000003:a [read 917504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:31.263 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:17.905948+0000", 2024-09-04T17:46:31.264 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 13.317293999, 2024-09-04T17:46:31.264 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.063001171999999994, 2024-09-04T17:46:31.264 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:31.264 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:31.264 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:31.264 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:31.264 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:31.264 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:31.264 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 13136 2024-09-04T17:46:31.264 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.264 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:31.264 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.264 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:31.264 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905948+0000", 2024-09-04T17:46:31.265 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.265 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.265 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.265 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:31.265 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905948+0000", 2024-09-04T17:46:31.265 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.265 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.265 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.265 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:31.265 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905949+0000", 2024-09-04T17:46:31.265 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.364e-06 2024-09-04T17:46:31.265 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.265 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.265 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:31.266 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905954+0000", 2024-09-04T17:46:31.266 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.8720000000000001e-06 2024-09-04T17:46:31.266 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.266 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.266 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:31.266 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905972+0000", 2024-09-04T17:46:31.266 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7756000000000001e-05 2024-09-04T17:46:31.266 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.266 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.266 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:31.266 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905984+0000", 2024-09-04T17:46:31.266 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2581e-05 2024-09-04T17:46:31.266 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.267 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.267 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:31.267 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.968668+0000", 2024-09-04T17:46:31.267 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.062683991999999994 2024-09-04T17:46:31.267 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.267 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.267 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:31.267 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.968773+0000", 2024-09-04T17:46:31.267 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000105027 2024-09-04T17:46:31.267 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.267 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.267 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:31.267 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.968949+0000", 2024-09-04T17:46:31.267 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00017558 2024-09-04T17:46:31.268 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.268 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:31.268 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.268 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.268 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.268 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:13137 3.1e 3:7bbed1a3:::rbd_data.11345bd5898b.0000000000000003:a [read 954368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:31.268 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:17.906005+0000", 2024-09-04T17:46:31.268 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 13.317236734, 2024-09-04T17:46:31.268 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.063260424999999995, 2024-09-04T17:46:31.268 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:31.268 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:31.268 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:31.268 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:31.269 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:31.269 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:31.269 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 13137 2024-09-04T17:46:31.269 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.269 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:31.269 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.269 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:31.269 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906005+0000", 2024-09-04T17:46:31.269 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.269 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.269 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.269 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:31.269 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906005+0000", 2024-09-04T17:46:31.270 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.270 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.270 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.270 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:31.270 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906006+0000", 2024-09-04T17:46:31.270 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.116e-06 2024-09-04T17:46:31.270 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.270 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.270 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:31.270 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906011+0000", 2024-09-04T17:46:31.270 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9429999999999999e-06 2024-09-04T17:46:31.270 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.271 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.271 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:31.271 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906028+0000", 2024-09-04T17:46:31.271 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6968000000000001e-05 2024-09-04T17:46:31.271 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.271 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.271 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:31.271 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906041+0000", 2024-09-04T17:46:31.271 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3269e-05 2024-09-04T17:46:31.271 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.271 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.272 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:31.272 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.968991+0000", 2024-09-04T17:46:31.272 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.062949450000000004 2024-09-04T17:46:31.272 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.272 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.272 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:31.272 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.969093+0000", 2024-09-04T17:46:31.272 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000102521 2024-09-04T17:46:31.272 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.272 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.272 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:31.272 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.969265+0000", 2024-09-04T17:46:31.272 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000172158 2024-09-04T17:46:31.273 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.273 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:31.273 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.273 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.273 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.273 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:13138 3.1e 3:7bbed1a3:::rbd_data.11345bd5898b.0000000000000003:a [read 991232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:31.273 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:17.906058+0000", 2024-09-04T17:46:31.273 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 13.317183063, 2024-09-04T17:46:31.273 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.063688720000000004, 2024-09-04T17:46:31.273 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:31.273 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:31.273 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:31.274 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:31.274 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:31.274 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:31.274 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 13138 2024-09-04T17:46:31.274 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.274 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:31.274 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.274 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:31.274 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906058+0000", 2024-09-04T17:46:31.274 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.274 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.274 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.274 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:31.275 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906058+0000", 2024-09-04T17:46:31.275 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.275 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.275 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.275 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:31.275 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906060+0000", 2024-09-04T17:46:31.275 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2100000000000001e-06 2024-09-04T17:46:31.275 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.275 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.275 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:31.275 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906065+0000", 2024-09-04T17:46:31.275 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.8670000000000002e-06 2024-09-04T17:46:31.275 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.276 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.276 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:31.276 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906082+0000", 2024-09-04T17:46:31.276 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7283999999999999e-05 2024-09-04T17:46:31.276 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.276 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.276 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:31.276 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906095+0000", 2024-09-04T17:46:31.276 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2763e-05 2024-09-04T17:46:31.276 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.276 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.276 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:31.276 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.969365+0000", 2024-09-04T17:46:31.276 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.063269912999999997 2024-09-04T17:46:31.277 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.277 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.277 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:31.277 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.969514+0000", 2024-09-04T17:46:31.277 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014945599999999999 2024-09-04T17:46:31.277 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.277 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.277 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:31.277 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.969747+0000", 2024-09-04T17:46:31.277 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00023322699999999999 2024-09-04T17:46:31.277 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.277 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:31.277 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.277 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.278 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.278 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:13139 3.1e 3:7bbed1a3:::rbd_data.11345bd5898b.0000000000000003:a [read 1028096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:31.278 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:17.906111+0000", 2024-09-04T17:46:31.278 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 13.317130378, 2024-09-04T17:46:31.278 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.063986543000000007, 2024-09-04T17:46:31.278 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:31.278 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:31.278 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:31.278 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:31.278 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:31.278 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:31.278 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 13139 2024-09-04T17:46:31.278 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.278 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:31.279 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.279 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:31.279 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906111+0000", 2024-09-04T17:46:31.279 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.279 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.279 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.279 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:31.279 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906111+0000", 2024-09-04T17:46:31.279 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.279 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.279 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.279 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:31.279 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906112+0000", 2024-09-04T17:46:31.279 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1370000000000001e-06 2024-09-04T17:46:31.279 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.280 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.280 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:31.280 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906118+0000", 2024-09-04T17:46:31.280 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.3460000000000001e-06 2024-09-04T17:46:31.280 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.280 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.280 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:31.280 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906135+0000", 2024-09-04T17:46:31.280 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6895e-05 2024-09-04T17:46:31.280 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.280 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.280 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:31.280 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906147+0000", 2024-09-04T17:46:31.281 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2578999999999999e-05 2024-09-04T17:46:31.281 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.281 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.281 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:31.281 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.969812+0000", 2024-09-04T17:46:31.281 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.063664539000000006 2024-09-04T17:46:31.281 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.281 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.281 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:31.281 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.969918+0000", 2024-09-04T17:46:31.281 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000106827 2024-09-04T17:46:31.281 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.282 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.282 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:31.282 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.970098+0000", 2024-09-04T17:46:31.282 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00017922000000000001 2024-09-04T17:46:31.282 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.282 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:31.282 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.282 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.282 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.282 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:13140 3.1e 3:7bbed1a3:::rbd_data.11345bd5898b.0000000000000003:a [read 1318912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:31.282 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:17.906165+0000", 2024-09-04T17:46:31.282 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 13.317076675999999, 2024-09-04T17:46:31.282 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.064276715999999998, 2024-09-04T17:46:31.282 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:31.283 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:31.283 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:31.283 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:31.283 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:31.283 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:31.283 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 13140 2024-09-04T17:46:31.283 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.283 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:31.283 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.283 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:31.283 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906165+0000", 2024-09-04T17:46:31.283 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.283 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.284 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.284 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:31.284 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906165+0000", 2024-09-04T17:46:31.284 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.284 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.284 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.284 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:31.284 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906166+0000", 2024-09-04T17:46:31.284 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.11e-06 2024-09-04T17:46:31.284 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.284 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.284 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:31.285 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906181+0000", 2024-09-04T17:46:31.285 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4918e-05 2024-09-04T17:46:31.285 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.285 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.285 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:31.285 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906198+0000", 2024-09-04T17:46:31.285 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7125e-05 2024-09-04T17:46:31.285 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.285 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.285 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:31.285 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906211+0000", 2024-09-04T17:46:31.285 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3169e-05 2024-09-04T17:46:31.285 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.286 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.286 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:31.286 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.970142+0000", 2024-09-04T17:46:31.286 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.063930511999999995 2024-09-04T17:46:31.286 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.286 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.286 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:31.286 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.970245+0000", 2024-09-04T17:46:31.286 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00010333800000000001 2024-09-04T17:46:31.286 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.286 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.286 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:31.286 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.970442+0000", 2024-09-04T17:46:31.286 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019654399999999999 2024-09-04T17:46:31.287 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.287 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:31.287 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.287 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.287 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.287 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:13141 3.1e 3:7bbed1a3:::rbd_data.11345bd5898b.0000000000000003:a [read 1355776~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:31.287 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:17.906252+0000", 2024-09-04T17:46:31.287 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 13.316989897999999, 2024-09-04T17:46:31.287 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.064674651, 2024-09-04T17:46:31.287 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:31.287 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:31.287 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:31.287 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:31.288 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:31.288 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:31.288 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 13141 2024-09-04T17:46:31.288 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.288 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:31.288 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.288 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:31.288 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906252+0000", 2024-09-04T17:46:31.288 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.288 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.288 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.288 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:31.288 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906252+0000", 2024-09-04T17:46:31.289 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.289 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.289 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.289 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:31.289 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906253+0000", 2024-09-04T17:46:31.289 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5600000000000001e-06 2024-09-04T17:46:31.289 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.289 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.289 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:31.289 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906259+0000", 2024-09-04T17:46:31.289 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7490000000000004e-06 2024-09-04T17:46:31.289 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.289 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.289 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:31.290 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906293+0000", 2024-09-04T17:46:31.290 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4204e-05 2024-09-04T17:46:31.290 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.290 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.290 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:31.290 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906307+0000", 2024-09-04T17:46:31.290 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3801000000000001e-05 2024-09-04T17:46:31.290 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.290 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.290 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:31.290 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.970535+0000", 2024-09-04T17:46:31.290 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.064228104999999994 2024-09-04T17:46:31.290 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.290 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.290 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:31.291 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.970692+0000", 2024-09-04T17:46:31.291 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00015668499999999999 2024-09-04T17:46:31.291 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.291 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.291 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:31.291 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.970926+0000", 2024-09-04T17:46:31.291 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00023454699999999999 2024-09-04T17:46:31.291 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.291 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:31.291 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.291 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.291 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.291 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:13142 3.1e 3:7bbed1a3:::rbd_data.11345bd5898b.0000000000000003:a [read 1392640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:31.291 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:17.906327+0000", 2024-09-04T17:46:31.292 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 13.316914142, 2024-09-04T17:46:31.292 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.064950401000000005, 2024-09-04T17:46:31.292 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:31.292 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:31.292 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:31.292 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:31.292 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:31.292 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:31.292 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 13142 2024-09-04T17:46:31.292 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.292 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:31.292 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.292 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:31.292 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906327+0000", 2024-09-04T17:46:31.293 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.293 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.293 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.293 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:31.293 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906327+0000", 2024-09-04T17:46:31.293 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.293 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.293 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.293 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:31.293 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906329+0000", 2024-09-04T17:46:31.293 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.33e-06 2024-09-04T17:46:31.293 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.293 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.294 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:31.294 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906334+0000", 2024-09-04T17:46:31.294 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4160000000000003e-06 2024-09-04T17:46:31.294 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.294 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.294 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:31.294 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906351+0000", 2024-09-04T17:46:31.294 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7328000000000001e-05 2024-09-04T17:46:31.294 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.294 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.294 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:31.294 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906379+0000", 2024-09-04T17:46:31.294 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.7923999999999999e-05 2024-09-04T17:46:31.295 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.295 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.295 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:31.295 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.970992+0000", 2024-09-04T17:46:31.295 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.064612554000000003 2024-09-04T17:46:31.295 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.295 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.295 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:31.295 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.971097+0000", 2024-09-04T17:46:31.295 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000105438 2024-09-04T17:46:31.295 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.295 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.295 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:31.296 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.971278+0000", 2024-09-04T17:46:31.296 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000180411 2024-09-04T17:46:31.296 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.296 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:31.296 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.296 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.296 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.296 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:13143 3.1e 3:7bbed1a3:::rbd_data.11345bd5898b.0000000000000003:a [read 1429504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:31.296 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:17.906399+0000", 2024-09-04T17:46:31.296 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 13.316843022, 2024-09-04T17:46:31.296 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.065281594999999998, 2024-09-04T17:46:31.296 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:31.296 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:31.296 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:31.297 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:31.297 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:31.297 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:31.297 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 13143 2024-09-04T17:46:31.297 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.297 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:31.297 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.297 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:31.297 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906399+0000", 2024-09-04T17:46:31.297 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.297 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.297 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.297 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:31.297 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906399+0000", 2024-09-04T17:46:31.298 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.298 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.298 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.298 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:31.298 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906400+0000", 2024-09-04T17:46:31.298 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.139e-06 2024-09-04T17:46:31.298 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.298 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.298 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:31.298 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906405+0000", 2024-09-04T17:46:31.298 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2299999999999999e-06 2024-09-04T17:46:31.298 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.298 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.298 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:31.298 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906422+0000", 2024-09-04T17:46:31.299 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6884999999999998e-05 2024-09-04T17:46:31.299 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.299 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.299 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:31.299 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906435+0000", 2024-09-04T17:46:31.299 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3339000000000001e-05 2024-09-04T17:46:31.299 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.299 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.299 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:31.299 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.971343+0000", 2024-09-04T17:46:31.299 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.064907555000000006 2024-09-04T17:46:31.299 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.299 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.300 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:31.300 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.971474+0000", 2024-09-04T17:46:31.300 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013157699999999999 2024-09-04T17:46:31.300 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.300 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.300 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:31.300 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.971680+0000", 2024-09-04T17:46:31.300 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020587000000000001 2024-09-04T17:46:31.300 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.300 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:31.300 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.300 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.300 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.300 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:13144 3.1e 3:7bbed1a3:::rbd_data.11345bd5898b.0000000000000003:a [read 1466368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:31.301 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:17.906452+0000", 2024-09-04T17:46:31.301 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 13.316789270999999, 2024-09-04T17:46:31.301 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.065571372000000003, 2024-09-04T17:46:31.301 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:31.301 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:31.301 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:31.301 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:31.301 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:31.301 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:31.301 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 13144 2024-09-04T17:46:31.301 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.301 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:31.302 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.302 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:31.302 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906452+0000", 2024-09-04T17:46:31.302 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.302 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.302 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.302 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:31.302 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906452+0000", 2024-09-04T17:46:31.302 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.302 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.302 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.302 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:31.303 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906453+0000", 2024-09-04T17:46:31.303 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.083e-06 2024-09-04T17:46:31.303 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.303 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.303 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:31.303 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906487+0000", 2024-09-04T17:46:31.303 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4084000000000001e-05 2024-09-04T17:46:31.303 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.303 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.303 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:31.303 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906505+0000", 2024-09-04T17:46:31.303 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7971000000000001e-05 2024-09-04T17:46:31.303 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.303 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.304 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:31.304 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906519+0000", 2024-09-04T17:46:31.304 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3526000000000001e-05 2024-09-04T17:46:31.304 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.304 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.304 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:31.304 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.971741+0000", 2024-09-04T17:46:31.304 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.065221836000000005 2024-09-04T17:46:31.304 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.304 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.304 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:31.304 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.971846+0000", 2024-09-04T17:46:31.304 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000105278 2024-09-04T17:46:31.305 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.305 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.305 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:31.305 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.972024+0000", 2024-09-04T17:46:31.305 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00017759399999999999 2024-09-04T17:46:31.305 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.305 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:31.305 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.305 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.305 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.305 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:13145 3.1e 3:7bbed1a3:::rbd_data.11345bd5898b.0000000000000003:a [read 1503232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:31.305 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:17.906538+0000", 2024-09-04T17:46:31.305 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 13.316703928999999, 2024-09-04T17:46:31.306 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.065800844999999997, 2024-09-04T17:46:31.306 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:31.306 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:31.306 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:31.306 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:31.306 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:31.306 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:31.306 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 13145 2024-09-04T17:46:31.306 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.306 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:31.306 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.306 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:31.306 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906538+0000", 2024-09-04T17:46:31.306 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.307 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.307 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.307 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:31.307 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906538+0000", 2024-09-04T17:46:31.307 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.307 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.307 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.307 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:31.307 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906539+0000", 2024-09-04T17:46:31.308 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6470000000000001e-06 2024-09-04T17:46:31.308 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.308 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.308 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:31.308 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906545+0000", 2024-09-04T17:46:31.308 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2390000000000002e-06 2024-09-04T17:46:31.308 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.308 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.308 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:31.308 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906562+0000", 2024-09-04T17:46:31.308 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7102e-05 2024-09-04T17:46:31.309 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.309 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.309 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:31.309 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906576+0000", 2024-09-04T17:46:31.309 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4758e-05 2024-09-04T17:46:31.309 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.309 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.309 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:31.309 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.972066+0000", 2024-09-04T17:46:31.309 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.065489503000000004 2024-09-04T17:46:31.309 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.309 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.309 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:31.310 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.972167+0000", 2024-09-04T17:46:31.310 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00010100200000000001 2024-09-04T17:46:31.310 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.310 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.310 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:31.310 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.972338+0000", 2024-09-04T17:46:31.310 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00017159400000000001 2024-09-04T17:46:31.310 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.310 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:31.310 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.310 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.310 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.310 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:13146 3.1e 3:7bbed1a3:::rbd_data.11345bd5898b.0000000000000003:a [read 1540096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:31.310 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:17.906595+0000", 2024-09-04T17:46:31.311 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 13.316646378, 2024-09-04T17:46:31.311 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.066220854999999995, 2024-09-04T17:46:31.311 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:31.311 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:31.311 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:31.311 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:31.311 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:31.311 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:31.311 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 13146 2024-09-04T17:46:31.311 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.311 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:31.311 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.311 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:31.311 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906595+0000", 2024-09-04T17:46:31.312 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.312 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.312 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.312 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:31.312 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906595+0000", 2024-09-04T17:46:31.312 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.312 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.312 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.312 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:31.312 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906597+0000", 2024-09-04T17:46:31.312 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6330000000000001e-06 2024-09-04T17:46:31.312 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.312 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.312 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:31.313 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906602+0000", 2024-09-04T17:46:31.313 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9529999999999998e-06 2024-09-04T17:46:31.313 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.313 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.313 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:31.313 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906619+0000", 2024-09-04T17:46:31.313 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7561999999999999e-05 2024-09-04T17:46:31.313 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.313 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.313 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:31.313 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906632+0000", 2024-09-04T17:46:31.313 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2901999999999999e-05 2024-09-04T17:46:31.313 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.313 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.314 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:31.314 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.972443+0000", 2024-09-04T17:46:31.314 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.065810653999999996 2024-09-04T17:46:31.314 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.314 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.314 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:31.314 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.972586+0000", 2024-09-04T17:46:31.314 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014348599999999999 2024-09-04T17:46:31.314 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.314 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.314 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:31.314 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.972816+0000", 2024-09-04T17:46:31.314 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00022966499999999999 2024-09-04T17:46:31.315 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.315 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:31.315 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.315 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.315 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:31.315 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:31.317 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-04T17:46:31.318 INFO:teuthology.orchestra.run.smithi092.stdout:1678 punch from 0x6461e20 to 0x6462de7, (0xfc7 bytes) 2024-09-04T17:46:31.318 INFO:teuthology.orchestra.run.smithi092.stdout:1679 read 0x7476ab3 thru 0x7486299 (0xf7e7 bytes) 2024-09-04T17:46:31.318 INFO:teuthology.orchestra.run.smithi092.stdout:1681 read 0x9d3b132 thru 0x9d47776 (0xc645 bytes) 2024-09-04T17:46:31.318 INFO:teuthology.orchestra.run.smithi092.stdout:1682 read 0x3c83eeb thru 0x3c91a2a (0xdb40 bytes) 2024-09-04T17:46:31.318 INFO:teuthology.orchestra.run.smithi092.stdout:1683 writesame 0x9391f3 thru 0x93d551 data_size 0x1(0x435f bytes) 2024-09-04T17:46:31.318 INFO:teuthology.orchestra.run.smithi092.stdout:1684 read 0x193e77f thru 0x1941e43 (0x36c5 bytes) 2024-09-04T17:46:31.318 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:31.318 INFO:teuthology.orchestra.run.smithi092.stdout:1688 punch from 0xcedbccf to 0xcedffb8, (0x42e9 bytes) 2024-09-04T17:46:31.318 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:31.319 INFO:teuthology.orchestra.run.smithi092.stdout:1690 trunc from 0xe186827 to 0xb7dbd91 2024-09-04T17:46:31.319 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:31.319 INFO:teuthology.orchestra.run.smithi092.stdout:1693 read 0x513391e thru 0x513f4ca (0xbbad bytes) 2024-09-04T17:46:31.319 INFO:teuthology.orchestra.run.smithi092.stdout:1694 write 0xd41cab8 thru 0xd41d111 (0x65a bytes) 2024-09-04T17:46:31.319 INFO:teuthology.orchestra.run.smithi092.stdout:1695 writesame 0x8a74ab4 thru 0x8a79eb0 data_size 0x3(0x53fd bytes) 2024-09-04T17:46:31.319 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:31.316+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:31.325 INFO:teuthology.orchestra.run.smithi092.stdout:1696 write 0xed39f79 thru 0xed3bab8 (0x1b40 bytes) 2024-09-04T17:46:31.332 INFO:teuthology.orchestra.run.smithi092.stdout:1698 punch from 0x1893156 to 0x1894822, (0x16cc bytes) 2024-09-04T17:46:31.332 INFO:teuthology.orchestra.run.smithi092.stdout:1699 writesame 0x307d444 thru 0x307f7e0 data_size 0x3(0x239d bytes) 2024-09-04T17:46:31.341 INFO:teuthology.orchestra.run.smithi092.stdout:1701 punch from 0xe136051 to 0xe143468, (0xd417 bytes) 2024-09-04T17:46:31.341 INFO:teuthology.orchestra.run.smithi092.stdout:1702 writesame 0x55bf524 thru 0x55c11f4 data_size 0x1(0x1cd1 bytes) 2024-09-04T17:46:31.354 INFO:teuthology.orchestra.run.smithi092.stdout:1704 writesame 0x624e640 thru 0x625116d data_size 0x1(0x2b2e bytes) 2024-09-04T17:46:31.364 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is set 2024-09-04T17:46:31.404 INFO:teuthology.orchestra.run.smithi092.stdout:1705 trunc from 0xed3bab9 to 0x342425d 2024-09-04T17:46:31.418 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:31.416+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:31.451 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:31.451 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:46:31.451 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:46:31.452 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:31.466 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-04T17:46:31.518 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:31.514+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:31.590 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:31.590 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:46:31.590 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:46:31.590 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:46:31.590 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:31.604 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-04T17:46:31.648 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:31.646+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:31.721 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:31.720+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:31.771 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:31.771 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:46:31.771 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:46:31.771 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:46:31.771 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.771 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:1430 3.f 3:f58475cb:::rbd_data.11346251f094.0000000000000074:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-04T17:46:31.771 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:02.778704+0000", 2024-09-04T17:46:31.771 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 28.991783726000001, 2024-09-04T17:46:31.771 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.058567801000000003, 2024-09-04T17:46:31.771 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:31.771 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:31.772 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:31.772 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:31.772 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:31.772 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:31.772 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 1430 2024-09-04T17:46:31.772 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.772 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:31.772 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.772 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:31.772 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.778704+0000", 2024-09-04T17:46:31.772 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.772 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.772 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.773 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:31.773 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.778704+0000", 2024-09-04T17:46:31.773 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.773 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.773 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.773 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:31.773 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.778707+0000", 2024-09-04T17:46:31.773 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.8329999999999998e-06 2024-09-04T17:46:31.773 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.773 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.773 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:31.773 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.778718+0000", 2024-09-04T17:46:31.773 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.152e-05 2024-09-04T17:46:31.774 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.774 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.774 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:31.774 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.778750+0000", 2024-09-04T17:46:31.774 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.1696999999999998e-05 2024-09-04T17:46:31.774 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.774 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.774 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:31.774 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.778765+0000", 2024-09-04T17:46:31.774 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4800000000000001e-05 2024-09-04T17:46:31.774 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.774 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.775 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:31.775 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.778894+0000", 2024-09-04T17:46:31.775 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00012928500000000001 2024-09-04T17:46:31.775 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.775 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.775 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:31.775 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.779030+0000", 2024-09-04T17:46:31.775 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013566899999999999 2024-09-04T17:46:31.775 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.775 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.775 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 4", 2024-09-04T17:46:31.775 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.788794+0000", 2024-09-04T17:46:31.775 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.009764478 2024-09-04T17:46:31.776 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.776 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.776 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:31.776 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.836969+0000", 2024-09-04T17:46:31.776 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.048174626999999998 2024-09-04T17:46:31.776 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.776 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.776 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:31.776 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.837090+0000", 2024-09-04T17:46:31.776 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000120627 2024-09-04T17:46:31.776 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.776 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.776 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:31.777 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.837195+0000", 2024-09-04T17:46:31.777 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000105206 2024-09-04T17:46:31.777 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.777 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.777 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:31.777 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.837272+0000", 2024-09-04T17:46:31.777 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.7058999999999995e-05 2024-09-04T17:46:31.777 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.777 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:31.777 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.777 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.777 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.777 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14369 3.d 3:b763de10:::rbd_data.11345bd5898b.0000000000000002:a [read 77824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:31.778 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.182808+0000", 2024-09-04T17:46:31.778 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 13.587679758, 2024-09-04T17:46:31.778 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042057997999999999, 2024-09-04T17:46:31.778 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:31.778 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:31.778 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:31.778 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:31.778 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:31.778 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:31.778 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14369 2024-09-04T17:46:31.778 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.778 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:31.778 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.778 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:31.779 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182808+0000", 2024-09-04T17:46:31.779 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.779 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.779 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.779 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:31.779 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182808+0000", 2024-09-04T17:46:31.779 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.779 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.779 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.779 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:31.779 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182809+0000", 2024-09-04T17:46:31.779 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.063e-06 2024-09-04T17:46:31.779 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.780 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.780 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:31.780 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182814+0000", 2024-09-04T17:46:31.780 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1900000000000003e-06 2024-09-04T17:46:31.780 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.780 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.780 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:31.780 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182832+0000", 2024-09-04T17:46:31.780 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7791999999999998e-05 2024-09-04T17:46:31.780 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.780 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.780 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:31.780 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182849+0000", 2024-09-04T17:46:31.781 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7265000000000002e-05 2024-09-04T17:46:31.781 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.781 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.781 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:31.781 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.224431+0000", 2024-09-04T17:46:31.781 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.041581448 2024-09-04T17:46:31.781 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.781 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.781 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:31.781 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.224615+0000", 2024-09-04T17:46:31.781 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00018389200000000001 2024-09-04T17:46:31.781 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.781 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.782 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:31.782 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.224866+0000", 2024-09-04T17:46:31.782 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00025134799999999998 2024-09-04T17:46:31.782 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.782 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:31.782 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.782 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.782 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.782 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14370 3.d 3:b763de10:::rbd_data.11345bd5898b.0000000000000002:a [read 114688~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:31.782 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.182867+0000", 2024-09-04T17:46:31.782 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 13.587620456, 2024-09-04T17:46:31.782 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042368301999999997, 2024-09-04T17:46:31.782 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:31.783 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:31.783 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:31.783 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:31.783 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:31.783 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:31.783 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14370 2024-09-04T17:46:31.783 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.783 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:31.783 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.783 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:31.783 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182867+0000", 2024-09-04T17:46:31.783 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.784 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.784 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.784 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:31.784 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182867+0000", 2024-09-04T17:46:31.784 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.784 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.784 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.784 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:31.784 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182868+0000", 2024-09-04T17:46:31.784 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.006e-06 2024-09-04T17:46:31.784 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.784 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.784 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:31.784 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182875+0000", 2024-09-04T17:46:31.785 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.3990000000000001e-06 2024-09-04T17:46:31.785 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.785 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.785 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:31.785 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182892+0000", 2024-09-04T17:46:31.785 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7096e-05 2024-09-04T17:46:31.785 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.785 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.785 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:31.785 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182905+0000", 2024-09-04T17:46:31.785 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3258000000000001e-05 2024-09-04T17:46:31.785 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.785 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.786 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:31.786 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.224926+0000", 2024-09-04T17:46:31.786 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042020891999999997 2024-09-04T17:46:31.786 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.786 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.786 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:31.786 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.225064+0000", 2024-09-04T17:46:31.786 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000138307 2024-09-04T17:46:31.786 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.786 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.786 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:31.786 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.225236+0000", 2024-09-04T17:46:31.787 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000171344 2024-09-04T17:46:31.787 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.787 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:31.787 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.787 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.787 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.787 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14375 3.d 3:b763de10:::rbd_data.11345bd5898b.0000000000000002:a [read 188416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:31.787 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.183941+0000", 2024-09-04T17:46:31.787 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 13.586546395999999, 2024-09-04T17:46:31.787 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042015839999999999, 2024-09-04T17:46:31.787 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:31.787 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:31.787 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:31.787 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:31.788 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:31.788 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:31.788 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14375 2024-09-04T17:46:31.788 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.788 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:31.788 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.788 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:31.788 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.183941+0000", 2024-09-04T17:46:31.788 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.788 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.788 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.788 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:31.789 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.183941+0000", 2024-09-04T17:46:31.789 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.789 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.789 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.789 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:31.789 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.183943+0000", 2024-09-04T17:46:31.789 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7099999999999999e-06 2024-09-04T17:46:31.789 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.789 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.789 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:31.789 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.183964+0000", 2024-09-04T17:46:31.789 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0483999999999999e-05 2024-09-04T17:46:31.789 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.790 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.790 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:31.790 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.183983+0000", 2024-09-04T17:46:31.790 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9918000000000001e-05 2024-09-04T17:46:31.790 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.790 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.790 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:31.790 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.183998+0000", 2024-09-04T17:46:31.790 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4683e-05 2024-09-04T17:46:31.790 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.790 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.790 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:31.790 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.225640+0000", 2024-09-04T17:46:31.791 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.041641425000000003 2024-09-04T17:46:31.791 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.791 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.791 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:31.791 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.225777+0000", 2024-09-04T17:46:31.791 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013778999999999999 2024-09-04T17:46:31.791 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.791 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.791 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:31.791 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.225957+0000", 2024-09-04T17:46:31.791 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00017982999999999999 2024-09-04T17:46:31.791 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.791 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:31.791 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.792 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.792 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.792 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14377 3.d 3:b763de10:::rbd_data.11345bd5898b.0000000000000002:a [read 225280~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:31.792 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.184021+0000", 2024-09-04T17:46:31.792 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 13.586466850000001, 2024-09-04T17:46:31.792 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042298810999999999, 2024-09-04T17:46:31.792 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:31.792 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:31.792 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:31.792 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:31.792 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:31.792 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:31.793 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14377 2024-09-04T17:46:31.793 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.793 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:31.793 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.793 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:31.793 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184021+0000", 2024-09-04T17:46:31.793 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.793 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.793 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.793 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:31.793 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184021+0000", 2024-09-04T17:46:31.793 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.793 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.794 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.794 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:31.794 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184022+0000", 2024-09-04T17:46:31.794 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5099999999999999e-06 2024-09-04T17:46:31.794 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.794 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.794 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:31.794 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184028+0000", 2024-09-04T17:46:31.794 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.852e-06 2024-09-04T17:46:31.794 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.794 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.794 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:31.794 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184046+0000", 2024-09-04T17:46:31.795 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7822e-05 2024-09-04T17:46:31.795 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.795 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.795 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:31.795 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184059+0000", 2024-09-04T17:46:31.795 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3372e-05 2024-09-04T17:46:31.795 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.795 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.795 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:31.795 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.226009+0000", 2024-09-04T17:46:31.795 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.041949187999999998 2024-09-04T17:46:31.795 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.796 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.796 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:31.796 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.226142+0000", 2024-09-04T17:46:31.796 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013381299999999999 2024-09-04T17:46:31.796 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.796 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.796 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:31.796 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.226320+0000", 2024-09-04T17:46:31.796 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00017725400000000001 2024-09-04T17:46:31.796 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.796 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:31.796 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.796 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.796 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.797 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14381 3.d 3:b763de10:::rbd_data.11345bd5898b.0000000000000002:a [read 552960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:31.797 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.184135+0000", 2024-09-04T17:46:31.797 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 13.586352798, 2024-09-04T17:46:31.797 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042553818, 2024-09-04T17:46:31.797 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:31.797 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:31.797 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:31.797 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:31.797 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:31.797 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:31.797 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14381 2024-09-04T17:46:31.797 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.798 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:31.798 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.798 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:31.798 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184135+0000", 2024-09-04T17:46:31.798 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.798 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.798 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.798 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:31.798 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184135+0000", 2024-09-04T17:46:31.798 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.798 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.798 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.798 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:31.798 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184136+0000", 2024-09-04T17:46:31.799 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1200000000000001e-06 2024-09-04T17:46:31.799 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.799 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.799 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:31.799 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184142+0000", 2024-09-04T17:46:31.799 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4840000000000003e-06 2024-09-04T17:46:31.799 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.799 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.799 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:31.799 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184157+0000", 2024-09-04T17:46:31.799 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5727999999999999e-05 2024-09-04T17:46:31.799 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.799 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.800 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:31.800 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184194+0000", 2024-09-04T17:46:31.800 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7049999999999999e-05 2024-09-04T17:46:31.800 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.800 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.800 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:31.800 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.226376+0000", 2024-09-04T17:46:31.800 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042181933999999997 2024-09-04T17:46:31.800 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.800 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.800 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:31.800 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.226522+0000", 2024-09-04T17:46:31.800 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014602899999999999 2024-09-04T17:46:31.801 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.801 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.801 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:31.801 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.226689+0000", 2024-09-04T17:46:31.801 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00016647299999999999 2024-09-04T17:46:31.801 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.801 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:31.801 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.801 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.801 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.801 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14383 3.d 3:b763de10:::rbd_data.11345bd5898b.0000000000000002:a [read 589824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:31.801 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.184302+0000", 2024-09-04T17:46:31.801 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 13.586185799000001, 2024-09-04T17:46:31.801 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042760362000000003, 2024-09-04T17:46:31.802 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:31.802 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:31.802 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:31.802 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:31.802 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:31.802 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:31.802 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14383 2024-09-04T17:46:31.802 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.802 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:31.802 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.802 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:31.802 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184302+0000", 2024-09-04T17:46:31.802 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.803 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.803 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.803 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:31.803 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184302+0000", 2024-09-04T17:46:31.803 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.803 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.803 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.803 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:31.803 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184303+0000", 2024-09-04T17:46:31.803 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.206e-06 2024-09-04T17:46:31.803 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.803 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.803 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:31.804 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184340+0000", 2024-09-04T17:46:31.804 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7005999999999998e-05 2024-09-04T17:46:31.804 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.804 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.804 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:31.804 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184382+0000", 2024-09-04T17:46:31.804 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2345999999999997e-05 2024-09-04T17:46:31.804 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.804 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.804 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:31.804 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184397+0000", 2024-09-04T17:46:31.804 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4729000000000001e-05 2024-09-04T17:46:31.804 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.805 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.805 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:31.805 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.226749+0000", 2024-09-04T17:46:31.805 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042352278 2024-09-04T17:46:31.805 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.805 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.805 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:31.805 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.226889+0000", 2024-09-04T17:46:31.805 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000139376 2024-09-04T17:46:31.805 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.805 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.805 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:31.805 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.227062+0000", 2024-09-04T17:46:31.806 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00017342099999999999 2024-09-04T17:46:31.806 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.806 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:31.806 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.806 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.806 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.806 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14385 3.d 3:b763de10:::rbd_data.11345bd5898b.0000000000000002:a [read 626688~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:31.806 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.184481+0000", 2024-09-04T17:46:31.806 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 13.586007135999999, 2024-09-04T17:46:31.806 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042873850999999998, 2024-09-04T17:46:31.806 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:31.806 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:31.806 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:31.807 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:31.807 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:31.807 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:31.807 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14385 2024-09-04T17:46:31.807 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.807 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:31.807 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.807 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:31.807 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184481+0000", 2024-09-04T17:46:31.807 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.807 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.807 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.807 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:31.808 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184481+0000", 2024-09-04T17:46:31.808 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.808 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.808 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.808 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:31.808 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184482+0000", 2024-09-04T17:46:31.808 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2130000000000001e-06 2024-09-04T17:46:31.808 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.808 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.808 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:31.808 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184487+0000", 2024-09-04T17:46:31.808 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4260000000000002e-06 2024-09-04T17:46:31.808 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.809 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.809 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:31.809 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184505+0000", 2024-09-04T17:46:31.809 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7659000000000002e-05 2024-09-04T17:46:31.809 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.809 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.809 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:31.809 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184518+0000", 2024-09-04T17:46:31.809 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2982e-05 2024-09-04T17:46:31.809 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.809 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.809 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:31.809 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.227120+0000", 2024-09-04T17:46:31.810 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042601858999999999 2024-09-04T17:46:31.810 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.810 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.810 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:31.810 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.227196+0000", 2024-09-04T17:46:31.810 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.6540999999999998e-05 2024-09-04T17:46:31.810 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.810 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.810 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:31.810 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.227354+0000", 2024-09-04T17:46:31.810 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000158171 2024-09-04T17:46:31.810 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.811 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:31.811 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.811 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.811 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.811 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14389 3.d 3:b763de10:::rbd_data.11345bd5898b.0000000000000002:a [read 663552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:31.811 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.184555+0000", 2024-09-04T17:46:31.811 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 13.585933000000001, 2024-09-04T17:46:31.811 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043553640999999997, 2024-09-04T17:46:31.811 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:31.811 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:31.811 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:31.811 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:31.811 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:31.812 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:31.812 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14389 2024-09-04T17:46:31.812 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.812 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:31.812 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.812 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:31.812 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184555+0000", 2024-09-04T17:46:31.812 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.812 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.812 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.812 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:31.812 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184555+0000", 2024-09-04T17:46:31.812 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.813 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.813 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.813 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:31.813 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184556+0000", 2024-09-04T17:46:31.813 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.353e-06 2024-09-04T17:46:31.813 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.813 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.813 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:31.813 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184562+0000", 2024-09-04T17:46:31.813 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7899999999999996e-06 2024-09-04T17:46:31.813 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.813 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.813 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:31.814 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184580+0000", 2024-09-04T17:46:31.814 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7682000000000001e-05 2024-09-04T17:46:31.814 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.814 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.814 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:31.814 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184593+0000", 2024-09-04T17:46:31.814 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3625000000000001e-05 2024-09-04T17:46:31.814 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.814 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.814 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:31.814 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.227413+0000", 2024-09-04T17:46:31.814 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042819572 2024-09-04T17:46:31.814 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.814 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.815 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:31.815 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.227552+0000", 2024-09-04T17:46:31.815 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000138842 2024-09-04T17:46:31.815 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.815 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.815 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:31.815 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228108+0000", 2024-09-04T17:46:31.815 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00055677700000000005 2024-09-04T17:46:31.815 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.815 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:31.815 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.815 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.815 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.816 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14392 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:31.816 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.184613+0000", 2024-09-04T17:46:31.816 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 13.585874737999999, 2024-09-04T17:46:31.816 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043677522000000003, 2024-09-04T17:46:31.816 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:31.816 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:31.816 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:31.816 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:31.816 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:31.816 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:31.816 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14392 2024-09-04T17:46:31.816 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.816 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:31.817 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.817 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:31.817 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184613+0000", 2024-09-04T17:46:31.817 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.817 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.817 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.817 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:31.817 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184613+0000", 2024-09-04T17:46:31.817 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.817 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.817 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.817 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:31.817 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184614+0000", 2024-09-04T17:46:31.818 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.35e-06 2024-09-04T17:46:31.818 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.818 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.818 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:31.818 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184620+0000", 2024-09-04T17:46:31.818 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.5589999999999996e-06 2024-09-04T17:46:31.818 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.818 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.818 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:31.818 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184671+0000", 2024-09-04T17:46:31.818 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0615000000000002e-05 2024-09-04T17:46:31.818 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.818 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.819 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:31.819 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184684+0000", 2024-09-04T17:46:31.819 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3596e-05 2024-09-04T17:46:31.819 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.819 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.819 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:31.819 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228277+0000", 2024-09-04T17:46:31.819 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043592509000000002 2024-09-04T17:46:31.819 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.819 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.819 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:31.819 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228291+0000", 2024-09-04T17:46:31.820 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3893000000000001e-05 2024-09-04T17:46:31.820 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.820 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:31.820 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.820 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.820 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.820 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14395 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:31.820 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.184708+0000", 2024-09-04T17:46:31.820 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 13.585779547, 2024-09-04T17:46:31.820 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043643612999999998, 2024-09-04T17:46:31.820 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:31.820 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:31.820 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:31.820 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:31.821 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:31.821 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:31.821 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14395 2024-09-04T17:46:31.821 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.821 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:31.821 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.821 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:31.821 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184708+0000", 2024-09-04T17:46:31.821 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.821 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.821 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.821 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:31.821 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184708+0000", 2024-09-04T17:46:31.822 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.822 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.822 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.822 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:31.822 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184709+0000", 2024-09-04T17:46:31.822 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2359999999999999e-06 2024-09-04T17:46:31.822 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.822 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.822 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:31.822 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184716+0000", 2024-09-04T17:46:31.822 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.3500000000000002e-06 2024-09-04T17:46:31.822 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.822 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.823 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:31.823 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184733+0000", 2024-09-04T17:46:31.823 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6775000000000001e-05 2024-09-04T17:46:31.823 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.823 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.823 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:31.823 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184777+0000", 2024-09-04T17:46:31.823 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4183e-05 2024-09-04T17:46:31.823 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.823 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.823 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:31.823 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228339+0000", 2024-09-04T17:46:31.823 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043562030000000002 2024-09-04T17:46:31.824 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.824 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.824 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:31.824 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228352+0000", 2024-09-04T17:46:31.824 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3039e-05 2024-09-04T17:46:31.824 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.824 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:31.824 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.824 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.824 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.824 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14398 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:31.824 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.184847+0000", 2024-09-04T17:46:31.824 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 13.5856409, 2024-09-04T17:46:31.825 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043564139000000002, 2024-09-04T17:46:31.825 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:31.825 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:31.825 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:31.825 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:31.825 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:31.825 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:31.825 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14398 2024-09-04T17:46:31.825 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.825 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:31.825 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.825 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:31.825 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184847+0000", 2024-09-04T17:46:31.826 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.826 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.826 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.826 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:31.826 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184847+0000", 2024-09-04T17:46:31.826 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.826 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.826 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.826 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:31.826 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184848+0000", 2024-09-04T17:46:31.826 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.28e-06 2024-09-04T17:46:31.826 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.826 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.826 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:31.827 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184861+0000", 2024-09-04T17:46:31.827 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3125000000000001e-05 2024-09-04T17:46:31.827 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.827 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.827 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:31.827 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184884+0000", 2024-09-04T17:46:31.827 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2357999999999998e-05 2024-09-04T17:46:31.827 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.827 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.827 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:31.827 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184900+0000", 2024-09-04T17:46:31.827 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6872e-05 2024-09-04T17:46:31.827 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.828 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.828 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:31.828 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228398+0000", 2024-09-04T17:46:31.828 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043497313000000003 2024-09-04T17:46:31.828 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.828 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.828 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:31.828 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228411+0000", 2024-09-04T17:46:31.828 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3191e-05 2024-09-04T17:46:31.828 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.828 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:31.828 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.828 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.829 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.829 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14399 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:31.829 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.184925+0000", 2024-09-04T17:46:31.829 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 13.585562584, 2024-09-04T17:46:31.829 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043544799000000002, 2024-09-04T17:46:31.829 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:31.829 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:31.829 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:31.829 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:31.829 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:31.829 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:31.829 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14399 2024-09-04T17:46:31.829 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.830 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:31.830 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.830 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:31.830 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184925+0000", 2024-09-04T17:46:31.830 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.830 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.830 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.830 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:31.830 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184925+0000", 2024-09-04T17:46:31.830 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.830 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.830 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.830 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:31.830 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184927+0000", 2024-09-04T17:46:31.831 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7e-06 2024-09-04T17:46:31.831 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.831 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.831 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:31.831 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184949+0000", 2024-09-04T17:46:31.831 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1880999999999999e-05 2024-09-04T17:46:31.831 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.831 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.831 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:31.831 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184969+0000", 2024-09-04T17:46:31.831 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0758e-05 2024-09-04T17:46:31.831 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.832 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.832 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:31.832 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184983+0000", 2024-09-04T17:46:31.832 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3336000000000001e-05 2024-09-04T17:46:31.832 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.832 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.832 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:31.832 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228456+0000", 2024-09-04T17:46:31.832 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043472960999999997 2024-09-04T17:46:31.832 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.832 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.832 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:31.832 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228470+0000", 2024-09-04T17:46:31.833 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4163e-05 2024-09-04T17:46:31.833 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.833 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:31.833 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.833 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.833 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.833 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14400 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:31.833 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.185003+0000", 2024-09-04T17:46:31.833 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 13.585485148, 2024-09-04T17:46:31.833 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043532659000000001, 2024-09-04T17:46:31.833 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:31.833 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:31.834 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:31.834 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:31.834 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:31.834 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:31.834 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14400 2024-09-04T17:46:31.834 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.834 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:31.834 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.834 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:31.834 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185003+0000", 2024-09-04T17:46:31.834 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.834 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.834 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.834 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:31.835 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185003+0000", 2024-09-04T17:46:31.835 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.835 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.835 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.835 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:31.835 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185004+0000", 2024-09-04T17:46:31.835 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.294e-06 2024-09-04T17:46:31.835 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.835 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.835 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:31.835 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185010+0000", 2024-09-04T17:46:31.835 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7690000000000002e-06 2024-09-04T17:46:31.835 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.835 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.836 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:31.836 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185029+0000", 2024-09-04T17:46:31.836 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9100999999999999e-05 2024-09-04T17:46:31.836 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.836 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.836 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:31.836 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185043+0000", 2024-09-04T17:46:31.836 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3862e-05 2024-09-04T17:46:31.836 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.836 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.836 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:31.836 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228510+0000", 2024-09-04T17:46:31.836 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043467766999999997 2024-09-04T17:46:31.837 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.837 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.837 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:31.837 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228535+0000", 2024-09-04T17:46:31.837 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4865999999999999e-05 2024-09-04T17:46:31.837 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.837 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:31.837 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.837 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.837 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.837 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14401 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:31.837 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.185062+0000", 2024-09-04T17:46:31.837 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 13.585425882999999, 2024-09-04T17:46:31.837 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043556096000000002, 2024-09-04T17:46:31.838 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:31.838 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:31.838 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:31.838 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:31.838 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:31.838 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:31.838 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14401 2024-09-04T17:46:31.838 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.838 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:31.838 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.838 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:31.838 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185062+0000", 2024-09-04T17:46:31.838 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.839 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.839 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.839 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:31.839 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185062+0000", 2024-09-04T17:46:31.839 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.839 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.839 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.839 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:31.839 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185063+0000", 2024-09-04T17:46:31.839 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.409e-06 2024-09-04T17:46:31.839 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.839 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.839 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:31.840 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185069+0000", 2024-09-04T17:46:31.840 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.5400000000000003e-06 2024-09-04T17:46:31.840 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.840 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.840 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:31.840 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185087+0000", 2024-09-04T17:46:31.840 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8195000000000001e-05 2024-09-04T17:46:31.840 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.840 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.840 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:31.840 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185101+0000", 2024-09-04T17:46:31.840 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3769e-05 2024-09-04T17:46:31.840 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.840 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.840 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:31.841 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228604+0000", 2024-09-04T17:46:31.841 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043503537000000002 2024-09-04T17:46:31.841 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.841 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.841 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:31.841 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228618+0000", 2024-09-04T17:46:31.841 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3645999999999999e-05 2024-09-04T17:46:31.841 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.841 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:31.841 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.841 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.841 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.841 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14402 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:31.842 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.185121+0000", 2024-09-04T17:46:31.842 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 13.585366588999999, 2024-09-04T17:46:31.842 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043551542999999998, 2024-09-04T17:46:31.842 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:31.842 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:31.842 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:31.842 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:31.842 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:31.842 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:31.842 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14402 2024-09-04T17:46:31.842 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.842 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:31.842 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.843 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:31.843 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185121+0000", 2024-09-04T17:46:31.843 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.843 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.843 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.843 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:31.843 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185121+0000", 2024-09-04T17:46:31.843 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.843 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.843 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.843 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:31.843 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185122+0000", 2024-09-04T17:46:31.843 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1769999999999999e-06 2024-09-04T17:46:31.843 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.844 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.844 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:31.844 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185128+0000", 2024-09-04T17:46:31.844 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7790000000000001e-06 2024-09-04T17:46:31.844 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.844 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.844 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:31.844 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185146+0000", 2024-09-04T17:46:31.844 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8114999999999999e-05 2024-09-04T17:46:31.844 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.844 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.844 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:31.844 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185160+0000", 2024-09-04T17:46:31.844 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4158999999999999e-05 2024-09-04T17:46:31.845 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.845 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.845 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:31.845 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228659+0000", 2024-09-04T17:46:31.845 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043498715 2024-09-04T17:46:31.845 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.845 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.845 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:31.845 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228673+0000", 2024-09-04T17:46:31.845 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3597999999999999e-05 2024-09-04T17:46:31.845 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.845 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:31.845 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.846 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.846 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.846 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14403 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:31.846 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.185183+0000", 2024-09-04T17:46:31.846 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 13.585304540999999, 2024-09-04T17:46:31.846 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043541467, 2024-09-04T17:46:31.846 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:31.846 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:31.846 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:31.846 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:31.846 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:31.846 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:31.846 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14403 2024-09-04T17:46:31.846 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.847 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:31.847 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.847 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:31.847 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185183+0000", 2024-09-04T17:46:31.847 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.847 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.847 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.847 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:31.847 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185183+0000", 2024-09-04T17:46:31.847 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.847 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.847 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.847 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:31.847 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185184+0000", 2024-09-04T17:46:31.848 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.156e-06 2024-09-04T17:46:31.848 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.848 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.848 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:31.848 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185190+0000", 2024-09-04T17:46:31.848 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1800000000000004e-06 2024-09-04T17:46:31.848 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.848 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.848 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:31.848 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185207+0000", 2024-09-04T17:46:31.848 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7288000000000001e-05 2024-09-04T17:46:31.848 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.848 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.849 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:31.849 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185221+0000", 2024-09-04T17:46:31.849 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3729e-05 2024-09-04T17:46:31.849 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.849 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.849 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:31.849 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228711+0000", 2024-09-04T17:46:31.849 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043490595 2024-09-04T17:46:31.849 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.849 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.849 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:31.849 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228725+0000", 2024-09-04T17:46:31.849 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3519e-05 2024-09-04T17:46:31.850 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.850 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:31.850 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.850 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.850 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.850 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14404 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:31.850 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.185239+0000", 2024-09-04T17:46:31.850 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 13.585248403, 2024-09-04T17:46:31.850 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043542099000000001, 2024-09-04T17:46:31.850 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:31.850 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:31.850 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:31.850 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:31.850 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:31.851 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:31.851 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14404 2024-09-04T17:46:31.851 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.851 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:31.851 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.851 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:31.851 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185239+0000", 2024-09-04T17:46:31.851 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.851 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.851 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.851 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:31.851 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185239+0000", 2024-09-04T17:46:31.851 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.851 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.852 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.852 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:31.852 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185240+0000", 2024-09-04T17:46:31.852 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.173e-06 2024-09-04T17:46:31.852 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.852 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.852 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:31.852 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185246+0000", 2024-09-04T17:46:31.852 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7529999999999998e-06 2024-09-04T17:46:31.852 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.852 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.852 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:31.852 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185264+0000", 2024-09-04T17:46:31.853 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7872000000000001e-05 2024-09-04T17:46:31.853 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.853 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.853 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:31.853 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185278+0000", 2024-09-04T17:46:31.853 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3532e-05 2024-09-04T17:46:31.853 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.853 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.853 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:31.853 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228768+0000", 2024-09-04T17:46:31.853 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043490586999999997 2024-09-04T17:46:31.853 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.853 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.853 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:31.854 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228781+0000", 2024-09-04T17:46:31.854 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3182e-05 2024-09-04T17:46:31.854 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.854 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:31.854 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.854 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.854 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.854 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14405 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:31.854 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.185297+0000", 2024-09-04T17:46:31.854 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 13.585191212, 2024-09-04T17:46:31.854 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043535049999999999, 2024-09-04T17:46:31.854 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:31.854 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:31.855 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:31.855 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:31.855 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:31.855 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:31.855 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14405 2024-09-04T17:46:31.855 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.855 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:31.855 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.855 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:31.855 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185297+0000", 2024-09-04T17:46:31.855 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.855 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.855 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.856 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:31.856 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185297+0000", 2024-09-04T17:46:31.856 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.856 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.856 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.856 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:31.856 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185298+0000", 2024-09-04T17:46:31.856 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.22e-06 2024-09-04T17:46:31.856 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.856 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.856 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:31.856 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185308+0000", 2024-09-04T17:46:31.857 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0173000000000001e-05 2024-09-04T17:46:31.857 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.857 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.857 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:31.857 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185329+0000", 2024-09-04T17:46:31.857 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1087999999999999e-05 2024-09-04T17:46:31.857 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.857 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.857 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:31.857 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185345+0000", 2024-09-04T17:46:31.857 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6035000000000001e-05 2024-09-04T17:46:31.857 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.857 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.858 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:31.858 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228822+0000", 2024-09-04T17:46:31.858 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043477427999999999 2024-09-04T17:46:31.858 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.858 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.858 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:31.858 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228832+0000", 2024-09-04T17:46:31.858 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1060000000000006e-06 2024-09-04T17:46:31.858 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.858 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:31.858 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.858 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.858 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.858 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14411 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:31.858 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.186171+0000", 2024-09-04T17:46:31.859 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 13.584316389, 2024-09-04T17:46:31.859 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.04271581, 2024-09-04T17:46:31.859 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:31.859 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:31.859 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:31.859 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:31.859 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:31.859 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:31.859 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14411 2024-09-04T17:46:31.859 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.859 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:31.859 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.859 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:31.859 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.186171+0000", 2024-09-04T17:46:31.860 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.860 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.860 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.860 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:31.860 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.186171+0000", 2024-09-04T17:46:31.860 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:31.860 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.860 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.860 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:31.860 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.186173+0000", 2024-09-04T17:46:31.860 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.012e-06 2024-09-04T17:46:31.860 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.860 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.861 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:31.861 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.186183+0000", 2024-09-04T17:46:31.861 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.7729999999999994e-06 2024-09-04T17:46:31.861 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.861 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.861 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:31.861 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.186205+0000", 2024-09-04T17:46:31.861 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2280999999999999e-05 2024-09-04T17:46:31.861 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.861 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.861 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:31.861 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.186224+0000", 2024-09-04T17:46:31.862 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8695e-05 2024-09-04T17:46:31.862 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.862 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.862 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:31.862 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228873+0000", 2024-09-04T17:46:31.862 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042648798000000002 2024-09-04T17:46:31.862 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:31.862 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:31.862 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:31.862 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228887+0000", 2024-09-04T17:46:31.862 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4251e-05 2024-09-04T17:46:31.862 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.862 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:31.862 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.862 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:31.863 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:31.863 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:31.864 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-04T17:46:31.864 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:31.862+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:31.915 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-04T17:46:31.965 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:31.962+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:31.995 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:31.996 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:46:31.996 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:46:31.996 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:32.011 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-04T17:46:32.066 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:32.062+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:32.068 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:46:32.068 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:46:32.085 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:46:32.086 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:46:32.103 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-04T17:46:32.103 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-04T17:46:32.120 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-04T17:46:32.121 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-04T17:46:32.129 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:32.129 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:32.141 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:46:32.141 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:46:32.143 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-04T17:46:32.162 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:46:32.163 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:46:32.167 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:32.164+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:32.183 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-04T17:46:32.184 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-04T17:46:32.205 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:46:32.205 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:46:32.259 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:32.259 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:32.268 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:32.270+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:32.272 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-04T17:46:32.369 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:46:32.366+0000 7f505d642640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26446) UID: 0 2024-09-04T17:46:32.415 INFO:teuthology.orchestra.run.smithi092.stdout:1707 clone 15 order 18 su 4096 sc 5 2024-09-04T17:46:32.440 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:32.441 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:32.454 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-04T17:46:32.500 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:32.500+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:32.572 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:32.570+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:32.575 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:32.576 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:32.590 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-04T17:46:32.673 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:32.674+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:32.756 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:32.756 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:32.770 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-04T17:46:32.773 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:32.774+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:32.874 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:32.872+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:32.893 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:32.893 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:32.907 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-04T17:46:32.995 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:32.996+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:33.076 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:33.076 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:33.076 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:33.074+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:33.090 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-04T17:46:33.180 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:33.180+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:33.252 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:33.252 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:33.266 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-04T17:46:33.278 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:33.278+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:33.378 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:33.376+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:33.398 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-04T17:46:33.435 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:33.435 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:33.451 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-04T17:46:33.500 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:46:33.500+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:46:33.528 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone14 from 0x342425d (overlap 0x342425d) to 0x1a1212e 2024-09-04T17:46:33.580 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:33.578+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:33.651 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:33.651 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:33.669 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-04T17:46:33.716 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:33.716+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:33.764 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #13, image image_client.0-clone13 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent14 2024-09-04T17:46:33.782 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:46:33.780+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:46:33.837 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:33.837 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:33.859 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-04T17:46:33.904 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:46:33.904+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:46:33.984 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:33.984+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:34.015 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:34.015 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:34.029 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-04T17:46:34.084 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:34.084+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:34.150 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:34.150 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:34.165 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-04T17:46:34.174 INFO:teuthology.orchestra.run.smithi092.stdout:1708 punch from 0x2805626 to 0x28062e7, (0xcc1 bytes) 2024-09-04T17:46:34.180 INFO:teuthology.orchestra.run.smithi092.stdout:1709 read 0x10d0cf8 thru 0x10d811d (0x7426 bytes) 2024-09-04T17:46:34.180 INFO:teuthology.orchestra.run.smithi092.stdout:1710 punch from 0x288d906 to 0x2893c0d, (0x6307 bytes) 2024-09-04T17:46:34.180 INFO:teuthology.orchestra.run.smithi092.stdout:1711 punch from 0x2fe39e1 to 0x2fe6e7e, (0x349d bytes) 2024-09-04T17:46:34.180 INFO:teuthology.orchestra.run.smithi092.stdout:truncating to largest ever: 0xe96f527 2024-09-04T17:46:34.185 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:34.184+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:34.194 INFO:teuthology.orchestra.run.smithi092.stdout:1713 trunc from 0x342425d to 0xe96f527 2024-09-04T17:46:34.203 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:34.203 INFO:teuthology.orchestra.run.smithi092.stdout:1716 punch from 0xd545064 to 0xd5451f7, (0x193 bytes) 2024-09-04T17:46:34.203 INFO:teuthology.orchestra.run.smithi092.stdout:1717 write 0x6f92672 thru 0x6f9e38e (0xbd1d bytes) 2024-09-04T17:46:34.210 INFO:teuthology.orchestra.run.smithi092.stdout:1718 write 0x69290db thru 0x693827b (0xf1a1 bytes) 2024-09-04T17:46:34.218 INFO:teuthology.orchestra.run.smithi092.stdout:1719 read 0xe36c965 thru 0xe36e28a (0x1926 bytes) 2024-09-04T17:46:34.219 INFO:teuthology.orchestra.run.smithi092.stdout:1720 read 0x8de441e thru 0x8de5cf9 (0x18dc bytes) 2024-09-04T17:46:34.219 INFO:teuthology.orchestra.run.smithi092.stdout:1722 write 0x8454104 thru 0x8454631 (0x52e bytes) 2024-09-04T17:46:34.222 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-04T17:46:34.223 INFO:teuthology.orchestra.run.smithi092.stdout:1725 write 0x93fa0a6 thru 0x9401d59 (0x7cb4 bytes) 2024-09-04T17:46:34.240 INFO:teuthology.orchestra.run.smithi092.stdout:1726 trunc from 0xe96f527 to 0xac99f9f 2024-09-04T17:46:34.248 INFO:teuthology.orchestra.run.smithi092.stdout:1727 read 0x8b7aeaa thru 0x8b88e6a (0xdfc1 bytes) 2024-09-04T17:46:34.254 INFO:teuthology.orchestra.run.smithi092.stdout:1728 trunc from 0xac99f9f to 0x55c1bee 2024-09-04T17:46:34.273 INFO:teuthology.orchestra.run.smithi092.stdout:1729 write 0x536f5a6 thru 0x537b75a (0xc1b5 bytes) 2024-09-04T17:46:34.286 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:34.282+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:34.286 INFO:teuthology.orchestra.run.smithi092.stdout:1730 trunc from 0x55c1bee to 0x610ae2a 2024-09-04T17:46:34.292 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:34.292 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:34.297 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:34.308 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-04T17:46:34.311 INFO:teuthology.orchestra.run.smithi092.stdout:1732 write 0x9b49047 thru 0x9b4a82a (0x17e4 bytes) 2024-09-04T17:46:34.315 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:34.326 INFO:teuthology.orchestra.run.smithi092.stdout:1734 writesame 0xac13aba thru 0xac1a1e3 data_size 0x5f(0x672a bytes) 2024-09-04T17:46:34.326 INFO:teuthology.orchestra.run.smithi092.stdout:1737 read 0x2f6269b thru 0x2f6da26 (0xb38c bytes) 2024-09-04T17:46:34.328 INFO:teuthology.orchestra.run.smithi092.stdout:1738 writesame 0x96219d7 thru 0x962a356 data_size 0xa0(0x8980 bytes) 2024-09-04T17:46:34.343 INFO:teuthology.orchestra.run.smithi092.stdout:1739 write 0xabad2e6 thru 0xabb5e2b (0x8b46 bytes) 2024-09-04T17:46:34.353 INFO:teuthology.orchestra.run.smithi092.stdout:1740 punch from 0x1e37937 to 0x1e3f7ed, (0x7eb6 bytes) 2024-09-04T17:46:34.354 INFO:teuthology.orchestra.run.smithi092.stdout:1741 write 0x7e49522 thru 0x7e564bd (0xcf9c bytes) 2024-09-04T17:46:34.358 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:34.358 INFO:teuthology.orchestra.run.smithi092.stdout:1743 punch from 0x89c5c79 to 0x89ce3a7, (0x872e bytes) 2024-09-04T17:46:34.358 INFO:teuthology.orchestra.run.smithi092.stdout:1744 writesame 0x96b44ec thru 0x96b609d data_size 0x5(0x1bb2 bytes) 2024-09-04T17:46:34.360 INFO:teuthology.orchestra.run.smithi092.stdout:1745 read 0x67931ce thru 0x679f135 (0xbf68 bytes) 2024-09-04T17:46:34.360 INFO:teuthology.orchestra.run.smithi092.stdout:1746 read 0xa7bb0bb thru 0xa7bf482 (0x43c8 bytes) 2024-09-04T17:46:34.360 INFO:teuthology.orchestra.run.smithi092.stdout:1747 writesame 0x29bee70 thru 0x29c4210 data_size 0x1(0x53a1 bytes) 2024-09-04T17:46:34.377 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:46:34.377 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:46:34.380 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is set 2024-09-04T17:46:34.386 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:34.384+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:34.393 INFO:teuthology.orchestra.run.smithi092.stdout:1748 write 0x49bc578 thru 0x49befc9 (0x2a52 bytes) 2024-09-04T17:46:34.402 INFO:teuthology.orchestra.run.smithi092.stdout:1749 read 0x6c8aa2d thru 0x6c8d98b (0x2f5f bytes) 2024-09-04T17:46:34.402 INFO:teuthology.orchestra.run.smithi092.stdout:1750 write 0x91018 thru 0x9a203 (0x91ec bytes) 2024-09-04T17:46:34.404 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:46:34.405 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:46:34.428 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-04T17:46:34.428 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-04T17:46:34.433 INFO:teuthology.orchestra.run.smithi092.stdout:1751 write 0x187c3d5 thru 0x187c760 (0x38c bytes) 2024-09-04T17:46:34.442 INFO:teuthology.orchestra.run.smithi092.stdout:1752 read 0x6b80d96 thru 0x6b87eb0 (0x711b bytes) 2024-09-04T17:46:34.445 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-04T17:46:34.445 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-04T17:46:34.453 INFO:teuthology.orchestra.run.smithi092.stdout:1754 trunc from 0xac1a1e4 to 0x37308be 2024-09-04T17:46:34.467 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:46:34.467 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:46:34.470 INFO:teuthology.orchestra.run.smithi092.stdout:1755 punch from 0xe5207 to 0xed340, (0x8139 bytes) 2024-09-04T17:46:34.470 INFO:teuthology.orchestra.run.smithi092.stdout:1756 read 0xf5105d thru 0xf5aa4d (0x99f1 bytes) 2024-09-04T17:46:34.481 INFO:teuthology.orchestra.run.smithi092.stdout:1757 write 0x3c6c433 thru 0x3c74ae2 (0x86b0 bytes) 2024-09-04T17:46:34.488 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:46:34.488 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:46:34.488 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:34.486+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:34.493 INFO:teuthology.orchestra.run.smithi092.stdout:1758 trunc from 0x3c74ae3 to 0x5835a4 2024-09-04T17:46:34.504 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:34.504 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:46:34.504 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.504 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:32021 3.1c 3.967f817c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-04T17:46:34.504 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:34.502866+0000", 2024-09-04T17:46:34.504 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.00013453299999999999, 2024-09-04T17:46:34.504 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000157882, 2024-09-04T17:46:34.504 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:34.504 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:34.504 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "queued for pg", 2024-09-04T17:46:34.504 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:34.505 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:34.505 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:34.505 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 32021 2024-09-04T17:46:34.505 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.505 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:34.505 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.505 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:34.505 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.502866+0000", 2024-09-04T17:46:34.505 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.505 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.505 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.505 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:34.505 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.502866+0000", 2024-09-04T17:46:34.506 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.506 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.506 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.506 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:34.506 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.502867+0000", 2024-09-04T17:46:34.506 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6100000000000001e-07 2024-09-04T17:46:34.506 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.506 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.506 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:34.506 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.502869+0000", 2024-09-04T17:46:34.506 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.3039999999999999e-06 2024-09-04T17:46:34.506 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.506 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.506 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:34.507 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.502874+0000", 2024-09-04T17:46:34.507 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.378e-06 2024-09-04T17:46:34.507 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.507 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.507 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:34.507 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.502878+0000", 2024-09-04T17:46:34.507 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.963e-06 2024-09-04T17:46:34.507 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.508 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:34.508 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.508 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.508 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.508 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:32022 3.1c 3.967f817c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-04T17:46:34.508 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:34.502884+0000", 2024-09-04T17:46:34.508 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.000116563, 2024-09-04T17:46:34.508 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021719299999999999, 2024-09-04T17:46:34.508 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:34.508 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:34.508 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "queued for pg", 2024-09-04T17:46:34.508 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:34.508 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:34.509 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:34.509 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 32022 2024-09-04T17:46:34.509 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.509 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:34.509 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.509 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:34.509 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.502884+0000", 2024-09-04T17:46:34.509 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.509 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.509 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.509 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:34.509 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.502884+0000", 2024-09-04T17:46:34.509 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.510 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.510 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.510 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:34.510 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.502884+0000", 2024-09-04T17:46:34.510 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.1100000000000002e-07 2024-09-04T17:46:34.510 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.510 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.510 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:34.510 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.502886+0000", 2024-09-04T17:46:34.510 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.463e-06 2024-09-04T17:46:34.510 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.510 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.510 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:34.510 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.502890+0000", 2024-09-04T17:46:34.511 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6979999999999997e-06 2024-09-04T17:46:34.511 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.511 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.511 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:34.511 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.502896+0000", 2024-09-04T17:46:34.511 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7400000000000001e-06 2024-09-04T17:46:34.511 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.511 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:34.511 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.511 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.511 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.511 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:32023 3.1c 3.967f817c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-04T17:46:34.511 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:34.502904+0000", 2024-09-04T17:46:34.512 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 9.6521000000000006e-05, 2024-09-04T17:46:34.512 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00026222300000000001, 2024-09-04T17:46:34.512 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:34.512 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:34.512 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "queued for pg", 2024-09-04T17:46:34.512 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:34.512 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:34.512 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:34.512 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 32023 2024-09-04T17:46:34.512 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.512 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:34.512 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.512 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:34.513 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.502904+0000", 2024-09-04T17:46:34.513 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.513 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.513 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.513 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:34.513 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.502904+0000", 2024-09-04T17:46:34.513 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.513 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.513 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.513 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:34.513 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.502905+0000", 2024-09-04T17:46:34.513 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6700000000000003e-07 2024-09-04T17:46:34.513 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.514 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.514 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:34.514 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.502907+0000", 2024-09-04T17:46:34.514 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.402e-06 2024-09-04T17:46:34.514 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.514 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.514 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:34.514 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.502915+0000", 2024-09-04T17:46:34.514 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.6059999999999997e-06 2024-09-04T17:46:34.514 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.514 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.514 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:34.514 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.502922+0000", 2024-09-04T17:46:34.515 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2620000000000001e-06 2024-09-04T17:46:34.515 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.515 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:34.515 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.515 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.515 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.515 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:32024 3.1c 3.967f817c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-04T17:46:34.515 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:34.502948+0000", 2024-09-04T17:46:34.515 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 5.3078000000000002e-05, 2024-09-04T17:46:34.515 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00028453899999999999, 2024-09-04T17:46:34.515 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:34.515 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:34.515 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "queued for pg", 2024-09-04T17:46:34.515 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:34.516 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:34.516 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:34.516 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 32024 2024-09-04T17:46:34.516 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.516 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:34.516 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.516 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:34.516 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.502948+0000", 2024-09-04T17:46:34.516 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.516 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.516 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.516 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:34.516 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.502948+0000", 2024-09-04T17:46:34.517 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.517 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.517 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.517 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:34.517 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.502948+0000", 2024-09-04T17:46:34.517 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.8400000000000003e-07 2024-09-04T17:46:34.517 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.517 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.517 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:34.518 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.502952+0000", 2024-09-04T17:46:34.518 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4740000000000001e-06 2024-09-04T17:46:34.518 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.518 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.518 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:34.518 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.502962+0000", 2024-09-04T17:46:34.518 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0549e-05 2024-09-04T17:46:34.518 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.518 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.518 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:34.518 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.502968+0000", 2024-09-04T17:46:34.518 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.8669999999999999e-06 2024-09-04T17:46:34.518 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.519 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:34.519 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.519 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.519 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.519 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:32025 3.1c 3.967f817c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-04T17:46:34.519 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:34.502976+0000", 2024-09-04T17:46:34.519 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 2.4119000000000001e-05, 2024-09-04T17:46:34.519 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00031966900000000002, 2024-09-04T17:46:34.519 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:34.519 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:34.519 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "queued for pg", 2024-09-04T17:46:34.519 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:34.519 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:34.519 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:34.520 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 32025 2024-09-04T17:46:34.520 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.520 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:34.520 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.520 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:34.520 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.502976+0000", 2024-09-04T17:46:34.520 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.520 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.520 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.520 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:34.520 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.502976+0000", 2024-09-04T17:46:34.520 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.521 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.521 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.521 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:34.521 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.502977+0000", 2024-09-04T17:46:34.521 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7800000000000002e-07 2024-09-04T17:46:34.521 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.521 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.521 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:34.521 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.502980+0000", 2024-09-04T17:46:34.521 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.7089999999999998e-06 2024-09-04T17:46:34.521 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.521 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.521 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:34.521 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.502987+0000", 2024-09-04T17:46:34.522 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.6229999999999997e-06 2024-09-04T17:46:34.522 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.522 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.522 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:34.522 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.502993+0000", 2024-09-04T17:46:34.522 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.0120000000000002e-06 2024-09-04T17:46:34.522 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.522 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:34.522 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.522 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.522 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.522 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:32026 3.1c 3.967f817c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-04T17:46:34.522 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:34.503001+0000", 2024-09-04T17:46:34.523 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4294967296, 2024-09-04T17:46:34.523 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00036386200000000001, 2024-09-04T17:46:34.523 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:34.523 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:34.523 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "queued for pg", 2024-09-04T17:46:34.523 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:34.523 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:34.523 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:34.523 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 32026 2024-09-04T17:46:34.523 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.523 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:34.523 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.523 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:34.524 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503001+0000", 2024-09-04T17:46:34.524 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.524 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.524 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.524 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:34.524 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503001+0000", 2024-09-04T17:46:34.524 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.524 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.524 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.524 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:34.524 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503001+0000", 2024-09-04T17:46:34.524 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.3799999999999998e-07 2024-09-04T17:46:34.524 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.524 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.525 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:34.525 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503003+0000", 2024-09-04T17:46:34.525 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.277e-06 2024-09-04T17:46:34.525 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.525 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.525 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:34.525 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503023+0000", 2024-09-04T17:46:34.525 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9570000000000001e-05 2024-09-04T17:46:34.525 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.525 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.525 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:34.525 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503030+0000", 2024-09-04T17:46:34.525 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4070000000000004e-06 2024-09-04T17:46:34.526 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.526 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:34.526 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.526 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.526 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.526 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:32027 3.1c 3.967f817c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-04T17:46:34.526 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:34.503040+0000", 2024-09-04T17:46:34.526 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4294967295.9999609, 2024-09-04T17:46:34.526 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00038992999999999997, 2024-09-04T17:46:34.526 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:34.526 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:34.526 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "queued for pg", 2024-09-04T17:46:34.526 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:34.527 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:34.527 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:34.527 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 32027 2024-09-04T17:46:34.527 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.527 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:34.527 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.527 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:34.527 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503040+0000", 2024-09-04T17:46:34.527 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.527 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.527 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.527 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:34.527 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503040+0000", 2024-09-04T17:46:34.527 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.528 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.528 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.528 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:34.528 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503040+0000", 2024-09-04T17:46:34.528 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.51e-07 2024-09-04T17:46:34.528 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.528 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.528 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:34.528 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503043+0000", 2024-09-04T17:46:34.528 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.7269999999999999e-06 2024-09-04T17:46:34.528 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.528 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.528 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:34.529 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503053+0000", 2024-09-04T17:46:34.529 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0046e-05 2024-09-04T17:46:34.529 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.529 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.529 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:34.529 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503061+0000", 2024-09-04T17:46:34.529 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.2460000000000003e-06 2024-09-04T17:46:34.529 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.529 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:34.529 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.529 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.529 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.529 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:32028 3.1c 3.967f817c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-04T17:46:34.530 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:34.503071+0000", 2024-09-04T17:46:34.530 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4294967295.999929, 2024-09-04T17:46:34.530 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00042570200000000002, 2024-09-04T17:46:34.530 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:34.530 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:34.530 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "queued for pg", 2024-09-04T17:46:34.530 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:34.530 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:34.530 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:34.530 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 32028 2024-09-04T17:46:34.530 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.530 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:34.530 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.530 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:34.531 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503071+0000", 2024-09-04T17:46:34.531 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.531 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.531 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.531 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:34.531 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503071+0000", 2024-09-04T17:46:34.531 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.531 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.531 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.531 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:34.531 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503072+0000", 2024-09-04T17:46:34.531 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4300000000000003e-07 2024-09-04T17:46:34.531 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.531 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.532 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:34.532 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503075+0000", 2024-09-04T17:46:34.532 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.9399999999999998e-06 2024-09-04T17:46:34.532 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.532 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.532 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:34.532 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503084+0000", 2024-09-04T17:46:34.532 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.3850000000000008e-06 2024-09-04T17:46:34.532 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.532 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.532 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:34.532 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503094+0000", 2024-09-04T17:46:34.532 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0027000000000001e-05 2024-09-04T17:46:34.532 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.533 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:34.533 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.533 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.533 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.533 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:32029 3.1c 3.967f817c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-04T17:46:34.533 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:34.503105+0000", 2024-09-04T17:46:34.533 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4294967295.9998956, 2024-09-04T17:46:34.533 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00045186599999999999, 2024-09-04T17:46:34.533 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:34.533 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:34.533 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "queued for pg", 2024-09-04T17:46:34.533 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:34.533 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:34.533 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:34.534 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 32029 2024-09-04T17:46:34.534 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.534 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:34.534 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.534 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:34.534 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503105+0000", 2024-09-04T17:46:34.534 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.534 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.534 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.534 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:34.534 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503105+0000", 2024-09-04T17:46:34.534 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.534 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.535 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.535 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:34.535 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503106+0000", 2024-09-04T17:46:34.535 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4099999999999998e-07 2024-09-04T17:46:34.535 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.535 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.535 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:34.535 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503109+0000", 2024-09-04T17:46:34.535 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.9969999999999999e-06 2024-09-04T17:46:34.535 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.535 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.535 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:34.535 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503117+0000", 2024-09-04T17:46:34.536 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.3080000000000008e-06 2024-09-04T17:46:34.536 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.536 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.536 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:34.536 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503124+0000", 2024-09-04T17:46:34.536 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7469999999999999e-06 2024-09-04T17:46:34.536 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.536 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:34.536 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.536 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.536 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.536 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:32030 3.1c 3.967f817c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-04T17:46:34.536 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:34.503132+0000", 2024-09-04T17:46:34.537 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4294967295.9998689, 2024-09-04T17:46:34.537 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000485463, 2024-09-04T17:46:34.537 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:34.537 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:34.537 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "queued for pg", 2024-09-04T17:46:34.537 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:34.537 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:34.537 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:34.537 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 32030 2024-09-04T17:46:34.537 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.537 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:34.537 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.537 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:34.537 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503132+0000", 2024-09-04T17:46:34.538 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.538 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.538 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.538 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:34.538 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503132+0000", 2024-09-04T17:46:34.538 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.538 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.538 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.538 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:34.538 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503133+0000", 2024-09-04T17:46:34.538 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.0999999999999998e-07 2024-09-04T17:46:34.538 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.538 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.539 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:34.539 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503135+0000", 2024-09-04T17:46:34.539 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4990000000000001e-06 2024-09-04T17:46:34.539 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.539 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.539 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:34.539 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503143+0000", 2024-09-04T17:46:34.539 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.0509999999999997e-06 2024-09-04T17:46:34.539 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.539 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.539 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:34.539 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503150+0000", 2024-09-04T17:46:34.539 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.4790000000000001e-06 2024-09-04T17:46:34.539 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.540 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:34.540 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.540 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.540 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.540 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:32031 3.1c 3.967f817c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-04T17:46:34.540 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:34.503175+0000", 2024-09-04T17:46:34.540 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4294967295.999825, 2024-09-04T17:46:34.540 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00050883999999999999, 2024-09-04T17:46:34.540 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:34.540 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:34.540 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "queued for pg", 2024-09-04T17:46:34.540 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:34.541 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:34.541 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:34.541 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 32031 2024-09-04T17:46:34.541 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.541 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:34.541 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.541 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:34.541 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503175+0000", 2024-09-04T17:46:34.541 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.541 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.541 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.541 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:34.541 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503175+0000", 2024-09-04T17:46:34.541 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.542 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.542 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.542 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:34.542 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503176+0000", 2024-09-04T17:46:34.542 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.8999999999999996e-07 2024-09-04T17:46:34.542 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.542 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.542 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:34.542 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503179+0000", 2024-09-04T17:46:34.542 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.1980000000000001e-06 2024-09-04T17:46:34.542 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.542 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.542 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:34.542 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503188+0000", 2024-09-04T17:46:34.542 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8810000000000002e-06 2024-09-04T17:46:34.543 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.543 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.543 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:34.543 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503195+0000", 2024-09-04T17:46:34.543 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.0509999999999999e-06 2024-09-04T17:46:34.543 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.543 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:34.543 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.543 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.543 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.543 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:32032 3.1c 3.967f817c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-04T17:46:34.543 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:34.503205+0000", 2024-09-04T17:46:34.543 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4294967295.9997954, 2024-09-04T17:46:34.543 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00052590000000000004, 2024-09-04T17:46:34.544 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:34.544 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:34.544 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "queued for pg", 2024-09-04T17:46:34.544 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:34.544 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:34.544 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:34.544 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 32032 2024-09-04T17:46:34.544 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.544 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:34.544 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.544 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:34.544 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503205+0000", 2024-09-04T17:46:34.544 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.545 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.545 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.545 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:34.545 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503205+0000", 2024-09-04T17:46:34.545 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.545 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.545 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.545 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:34.545 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503206+0000", 2024-09-04T17:46:34.545 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.3399999999999999e-07 2024-09-04T17:46:34.545 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.545 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.545 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:34.546 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503208+0000", 2024-09-04T17:46:34.546 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6299999999999998e-06 2024-09-04T17:46:34.546 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.546 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.546 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:34.546 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503217+0000", 2024-09-04T17:46:34.546 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.1650000000000006e-06 2024-09-04T17:46:34.546 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.546 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.546 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:34.546 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503232+0000", 2024-09-04T17:46:34.546 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4972e-05 2024-09-04T17:46:34.546 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.546 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:34.546 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.547 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.547 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.547 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:32033 3.1c 3.967f817c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-04T17:46:34.547 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:34.503249+0000", 2024-09-04T17:46:34.547 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4294967295.9997516, 2024-09-04T17:46:34.547 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00054588499999999997, 2024-09-04T17:46:34.547 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:34.547 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:34.547 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "queued for pg", 2024-09-04T17:46:34.547 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:34.547 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:34.547 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:34.547 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 32033 2024-09-04T17:46:34.547 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.548 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:34.548 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.548 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:34.548 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503249+0000", 2024-09-04T17:46:34.548 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.548 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.548 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.548 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:34.548 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503249+0000", 2024-09-04T17:46:34.548 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.548 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.548 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.548 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:34.549 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503250+0000", 2024-09-04T17:46:34.549 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.54e-07 2024-09-04T17:46:34.549 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.549 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.549 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:34.549 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503252+0000", 2024-09-04T17:46:34.549 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.7709999999999998e-06 2024-09-04T17:46:34.549 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.549 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.549 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:34.549 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503261+0000", 2024-09-04T17:46:34.549 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7239999999999998e-06 2024-09-04T17:46:34.549 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.549 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.550 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:34.550 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503268+0000", 2024-09-04T17:46:34.550 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.8549999999999996e-06 2024-09-04T17:46:34.550 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.550 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:34.550 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.550 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.550 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.550 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:32034 3.1c 3.967f817c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-04T17:46:34.550 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:34.503276+0000", 2024-09-04T17:46:34.550 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4294967295.9997244, 2024-09-04T17:46:34.550 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00058336, 2024-09-04T17:46:34.550 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:34.550 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:34.551 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "queued for pg", 2024-09-04T17:46:34.551 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:34.551 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:34.551 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:34.551 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 32034 2024-09-04T17:46:34.551 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.551 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:34.551 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.551 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:34.551 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503276+0000", 2024-09-04T17:46:34.551 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.551 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.551 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.551 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:34.552 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503276+0000", 2024-09-04T17:46:34.552 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.552 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.552 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.552 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:34.552 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503277+0000", 2024-09-04T17:46:34.552 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.9999999999999997e-07 2024-09-04T17:46:34.552 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.552 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.552 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:34.552 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503280+0000", 2024-09-04T17:46:34.552 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.717e-06 2024-09-04T17:46:34.552 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.553 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.553 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:34.553 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503288+0000", 2024-09-04T17:46:34.553 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.0139999999999998e-06 2024-09-04T17:46:34.553 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.553 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.553 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:34.553 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503295+0000", 2024-09-04T17:46:34.553 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.5549999999999997e-06 2024-09-04T17:46:34.553 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.553 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:34.553 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.553 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.553 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.554 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:32035 3.1c 3.967f817c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-04T17:46:34.554 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:34.503303+0000", 2024-09-04T17:46:34.554 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4294967295.9996972, 2024-09-04T17:46:34.554 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000621556, 2024-09-04T17:46:34.554 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:34.554 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:34.554 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "queued for pg", 2024-09-04T17:46:34.554 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:34.554 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:34.554 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:34.554 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 32035 2024-09-04T17:46:34.554 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.554 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:34.555 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.555 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:34.555 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503303+0000", 2024-09-04T17:46:34.555 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.555 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.555 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.555 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:34.555 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503303+0000", 2024-09-04T17:46:34.555 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.555 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.555 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.555 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:34.555 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503304+0000", 2024-09-04T17:46:34.556 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.89e-07 2024-09-04T17:46:34.556 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.556 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.556 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:34.556 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503306+0000", 2024-09-04T17:46:34.556 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.367e-06 2024-09-04T17:46:34.556 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.556 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.557 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:34.557 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503315+0000", 2024-09-04T17:46:34.557 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.3180000000000007e-06 2024-09-04T17:46:34.557 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.557 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.557 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:34.557 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503321+0000", 2024-09-04T17:46:34.557 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.5919999999999997e-06 2024-09-04T17:46:34.557 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.557 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:34.557 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.557 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.557 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.558 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:32036 3.1c 3.967f817c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-04T17:46:34.558 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:34.503330+0000", 2024-09-04T17:46:34.558 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4294967295.9996705, 2024-09-04T17:46:34.558 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00066056199999999995, 2024-09-04T17:46:34.558 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:34.558 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:34.558 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "queued for pg", 2024-09-04T17:46:34.558 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:34.558 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:34.558 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:34.558 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 32036 2024-09-04T17:46:34.558 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.558 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:34.558 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.559 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:34.559 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503330+0000", 2024-09-04T17:46:34.559 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.559 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.559 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.559 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:34.559 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503330+0000", 2024-09-04T17:46:34.559 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.559 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.559 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.559 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:34.559 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503331+0000", 2024-09-04T17:46:34.559 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4700000000000001e-07 2024-09-04T17:46:34.559 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.559 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.560 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:34.560 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503333+0000", 2024-09-04T17:46:34.560 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.8480000000000001e-06 2024-09-04T17:46:34.560 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.560 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.560 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:34.560 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503341+0000", 2024-09-04T17:46:34.560 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.0339999999999997e-06 2024-09-04T17:46:34.560 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.560 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.560 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:34.560 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503349+0000", 2024-09-04T17:46:34.560 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.6499999999999996e-06 2024-09-04T17:46:34.560 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.560 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:34.561 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.561 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.561 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.561 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:32037 3.1c 3.967f817c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-04T17:46:34.561 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:34.503358+0000", 2024-09-04T17:46:34.561 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4294967295.9996428, 2024-09-04T17:46:34.561 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00070010600000000004, 2024-09-04T17:46:34.561 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:34.561 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:34.561 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "queued for pg", 2024-09-04T17:46:34.561 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:34.561 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:34.561 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:34.562 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 32037 2024-09-04T17:46:34.562 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.562 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:34.562 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.562 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:34.562 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503358+0000", 2024-09-04T17:46:34.562 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.562 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.562 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.562 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:34.562 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503358+0000", 2024-09-04T17:46:34.562 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.562 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.562 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.563 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:34.563 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503358+0000", 2024-09-04T17:46:34.563 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0399999999999996e-07 2024-09-04T17:46:34.563 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.563 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.563 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:34.563 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503361+0000", 2024-09-04T17:46:34.563 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.446e-06 2024-09-04T17:46:34.563 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.563 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.563 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:34.563 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503368+0000", 2024-09-04T17:46:34.564 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.9340000000000006e-06 2024-09-04T17:46:34.564 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.564 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.564 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:34.564 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503375+0000", 2024-09-04T17:46:34.564 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.4489999999999996e-06 2024-09-04T17:46:34.564 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.564 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:34.564 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.564 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.564 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.564 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:32038 3.1c 3.967f817c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-04T17:46:34.564 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:34.503383+0000", 2024-09-04T17:46:34.564 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4294967295.9996181, 2024-09-04T17:46:34.565 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00074382500000000004, 2024-09-04T17:46:34.565 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:34.565 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:34.565 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "queued for pg", 2024-09-04T17:46:34.565 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:34.565 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:34.565 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:34.565 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 32038 2024-09-04T17:46:34.565 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.565 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:34.565 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.565 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:34.566 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503383+0000", 2024-09-04T17:46:34.566 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.566 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.566 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.566 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:34.566 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503383+0000", 2024-09-04T17:46:34.566 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.566 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.566 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.566 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:34.566 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503383+0000", 2024-09-04T17:46:34.566 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1399999999999997e-07 2024-09-04T17:46:34.566 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.567 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.567 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:34.567 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503386+0000", 2024-09-04T17:46:34.567 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6620000000000001e-06 2024-09-04T17:46:34.567 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.567 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.567 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:34.567 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503394+0000", 2024-09-04T17:46:34.567 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.9810000000000003e-06 2024-09-04T17:46:34.567 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.567 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.567 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:34.567 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503400+0000", 2024-09-04T17:46:34.567 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.5010000000000001e-06 2024-09-04T17:46:34.568 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.568 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:34.568 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.568 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.568 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.568 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:32039 3.1c 3.967f817c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-04T17:46:34.568 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:34.503409+0000", 2024-09-04T17:46:34.568 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4294967295.9995918, 2024-09-04T17:46:34.568 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00078188800000000001, 2024-09-04T17:46:34.568 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:34.568 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:34.568 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "queued for pg", 2024-09-04T17:46:34.568 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:34.569 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:34.569 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:34.569 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 32039 2024-09-04T17:46:34.569 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.569 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:34.569 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.569 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:34.569 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503409+0000", 2024-09-04T17:46:34.569 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.569 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.569 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.569 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:34.569 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503409+0000", 2024-09-04T17:46:34.569 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.570 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.570 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.570 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:34.570 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503410+0000", 2024-09-04T17:46:34.570 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1200000000000002e-07 2024-09-04T17:46:34.570 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.570 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.570 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:34.570 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503412+0000", 2024-09-04T17:46:34.570 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6259999999999999e-06 2024-09-04T17:46:34.570 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.570 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.570 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:34.571 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503420+0000", 2024-09-04T17:46:34.571 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.157e-06 2024-09-04T17:46:34.571 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.571 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.571 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:34.571 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503427+0000", 2024-09-04T17:46:34.571 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.3690000000000003e-06 2024-09-04T17:46:34.571 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.571 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:34.571 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.571 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.571 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.571 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:32040 3.1c 3.967f817c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-04T17:46:34.571 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:34.503435+0000", 2024-09-04T17:46:34.572 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4294967295.9995661, 2024-09-04T17:46:34.572 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00082140900000000005, 2024-09-04T17:46:34.572 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:34.572 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:34.572 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "queued for pg", 2024-09-04T17:46:34.572 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:34.572 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:34.572 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:34.572 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 32040 2024-09-04T17:46:34.572 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.572 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:34.572 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.572 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:34.573 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503435+0000", 2024-09-04T17:46:34.573 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.573 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.573 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.573 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:34.573 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503435+0000", 2024-09-04T17:46:34.573 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.573 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.573 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.573 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:34.573 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503435+0000", 2024-09-04T17:46:34.573 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9100000000000004e-07 2024-09-04T17:46:34.573 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.573 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.574 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:34.574 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503437+0000", 2024-09-04T17:46:34.574 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2560000000000001e-06 2024-09-04T17:46:34.574 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.574 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.574 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:34.574 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503445+0000", 2024-09-04T17:46:34.574 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.8159999999999994e-06 2024-09-04T17:46:34.574 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.574 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.574 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:34.574 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503451+0000", 2024-09-04T17:46:34.574 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.2110000000000003e-06 2024-09-04T17:46:34.575 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.575 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:34.575 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.575 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.575 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.575 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:32041 3.1c 3.967f817c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-04T17:46:34.575 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:34.503459+0000", 2024-09-04T17:46:34.575 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4294967295.9995413, 2024-09-04T17:46:34.575 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000861041, 2024-09-04T17:46:34.575 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:34.575 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:34.575 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "queued for pg", 2024-09-04T17:46:34.575 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:34.575 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:34.576 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:34.576 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 32041 2024-09-04T17:46:34.576 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.576 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:34.576 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.576 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:34.576 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503459+0000", 2024-09-04T17:46:34.576 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.576 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.576 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.576 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:34.576 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503459+0000", 2024-09-04T17:46:34.576 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.577 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.577 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.577 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:34.577 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503460+0000", 2024-09-04T17:46:34.577 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.8800000000000002e-07 2024-09-04T17:46:34.577 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.577 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.577 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:34.577 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503462+0000", 2024-09-04T17:46:34.577 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.3329999999999999e-06 2024-09-04T17:46:34.577 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.577 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.577 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:34.578 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503470+0000", 2024-09-04T17:46:34.578 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.8830000000000006e-06 2024-09-04T17:46:34.578 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.578 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.578 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:34.578 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503476+0000", 2024-09-04T17:46:34.578 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.2550000000000003e-06 2024-09-04T17:46:34.578 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.578 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:34.578 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.578 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.578 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.578 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:32042 3.1c 3.967f817c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-04T17:46:34.578 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:34.503484+0000", 2024-09-04T17:46:34.579 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4294967295.9995165, 2024-09-04T17:46:34.579 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00090129399999999997, 2024-09-04T17:46:34.579 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:34.579 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:34.579 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "queued for pg", 2024-09-04T17:46:34.579 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:34.579 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:34.579 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:34.579 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 32042 2024-09-04T17:46:34.579 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.579 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:34.579 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.579 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:34.579 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503484+0000", 2024-09-04T17:46:34.580 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.580 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.580 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.580 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:34.580 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503484+0000", 2024-09-04T17:46:34.580 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.580 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.580 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.580 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:34.580 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503485+0000", 2024-09-04T17:46:34.580 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9800000000000004e-07 2024-09-04T17:46:34.580 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.580 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.581 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:34.581 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503492+0000", 2024-09-04T17:46:34.581 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2740000000000002e-06 2024-09-04T17:46:34.581 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.581 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.581 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:34.581 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503500+0000", 2024-09-04T17:46:34.581 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.977e-06 2024-09-04T17:46:34.581 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.581 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.581 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:34.581 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503506+0000", 2024-09-04T17:46:34.581 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.4080000000000003e-06 2024-09-04T17:46:34.582 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.582 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:34.582 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.582 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.582 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.582 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:32043 3.1c 3.967f817c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-04T17:46:34.582 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:34.503514+0000", 2024-09-04T17:46:34.582 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4294967295.999486, 2024-09-04T17:46:34.582 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00093572399999999997, 2024-09-04T17:46:34.582 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:34.582 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:34.582 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "queued for pg", 2024-09-04T17:46:34.582 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:34.583 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:34.583 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:34.583 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 32043 2024-09-04T17:46:34.583 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.583 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:34.583 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.583 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:34.583 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503514+0000", 2024-09-04T17:46:34.583 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.583 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.583 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.583 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:34.583 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503514+0000", 2024-09-04T17:46:34.583 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.584 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.584 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.584 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:34.584 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503515+0000", 2024-09-04T17:46:34.584 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2699999999999999e-07 2024-09-04T17:46:34.584 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.584 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.584 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:34.584 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503518+0000", 2024-09-04T17:46:34.584 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6029999999999999e-06 2024-09-04T17:46:34.584 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.584 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.584 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:34.585 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503526+0000", 2024-09-04T17:46:34.585 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.0600000000000008e-06 2024-09-04T17:46:34.585 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.585 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.585 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:34.585 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503532+0000", 2024-09-04T17:46:34.585 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.2310000000000001e-06 2024-09-04T17:46:34.585 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.585 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:34.585 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.585 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.585 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.585 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:32044 3.1c 3.967f817c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-04T17:46:34.586 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:34.503574+0000", 2024-09-04T17:46:34.586 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4294967295.9994268, 2024-09-04T17:46:34.586 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00094115500000000005, 2024-09-04T17:46:34.586 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:34.586 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:34.586 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "queued for pg", 2024-09-04T17:46:34.586 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:34.586 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:34.586 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:34.586 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 32044 2024-09-04T17:46:34.586 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.586 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:34.586 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.587 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:34.587 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503574+0000", 2024-09-04T17:46:34.587 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.587 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.587 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.587 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:34.587 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503574+0000", 2024-09-04T17:46:34.587 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.587 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.587 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.587 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:34.587 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503575+0000", 2024-09-04T17:46:34.587 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.8400000000000003e-07 2024-09-04T17:46:34.588 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.588 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.588 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:34.588 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503578+0000", 2024-09-04T17:46:34.588 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.241e-06 2024-09-04T17:46:34.588 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.588 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.588 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:34.588 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503588+0000", 2024-09-04T17:46:34.588 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.9699999999999994e-06 2024-09-04T17:46:34.588 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.588 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.588 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:34.589 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503595+0000", 2024-09-04T17:46:34.589 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.861e-06 2024-09-04T17:46:34.589 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.589 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:34.589 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.589 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.589 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.589 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:32083 3.1c 3.967f817c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-04T17:46:34.589 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:34.503716+0000", 2024-09-04T17:46:34.589 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4294967295.9992847, 2024-09-04T17:46:34.589 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00086489399999999995, 2024-09-04T17:46:34.589 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:34.589 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:34.589 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "queued for pg", 2024-09-04T17:46:34.590 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:34.590 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:34.590 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:34.590 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 32083 2024-09-04T17:46:34.590 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.590 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:34.590 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.590 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:34.590 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503716+0000", 2024-09-04T17:46:34.590 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.590 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.590 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.590 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:34.591 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503716+0000", 2024-09-04T17:46:34.591 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.591 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.591 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.591 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:34.591 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503717+0000", 2024-09-04T17:46:34.591 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.065e-06 2024-09-04T17:46:34.591 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.591 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.591 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:34.591 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503721+0000", 2024-09-04T17:46:34.591 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.9450000000000003e-06 2024-09-04T17:46:34.591 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.591 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.592 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:34.592 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503745+0000", 2024-09-04T17:46:34.592 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.3955999999999999e-05 2024-09-04T17:46:34.592 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.592 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.592 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:34.592 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503755+0000", 2024-09-04T17:46:34.592 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0101e-05 2024-09-04T17:46:34.592 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.592 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:34.592 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.592 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.592 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.593 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:32084 3.1c 3.967f817c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-04T17:46:34.593 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:34.503767+0000", 2024-09-04T17:46:34.593 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4294967295.9992342, 2024-09-04T17:46:34.593 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00087827999999999995, 2024-09-04T17:46:34.593 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:34.593 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:34.593 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "queued for pg", 2024-09-04T17:46:34.593 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:34.593 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:34.593 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:34.593 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 32084 2024-09-04T17:46:34.593 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.593 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:34.593 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.594 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:34.594 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503767+0000", 2024-09-04T17:46:34.594 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.594 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.594 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.594 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:34.594 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503767+0000", 2024-09-04T17:46:34.594 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.594 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.594 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.594 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:34.594 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503767+0000", 2024-09-04T17:46:34.594 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.9699999999999995e-07 2024-09-04T17:46:34.595 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.595 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.595 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:34.595 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503770+0000", 2024-09-04T17:46:34.595 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.8200000000000001e-06 2024-09-04T17:46:34.595 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.595 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.595 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:34.595 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503779+0000", 2024-09-04T17:46:34.595 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.6090000000000001e-06 2024-09-04T17:46:34.595 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.595 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.595 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:34.595 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503786+0000", 2024-09-04T17:46:34.596 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.43e-06 2024-09-04T17:46:34.596 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.596 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:34.596 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.596 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.596 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.596 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:32085 3.1c 3.967f817c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-04T17:46:34.596 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:34.503805+0000", 2024-09-04T17:46:34.596 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4294967295.9991956, 2024-09-04T17:46:34.596 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00090345300000000003, 2024-09-04T17:46:34.596 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:34.596 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:34.596 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "queued for pg", 2024-09-04T17:46:34.597 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:34.597 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:34.597 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:34.597 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 32085 2024-09-04T17:46:34.597 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.597 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:34.597 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.597 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:34.597 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503805+0000", 2024-09-04T17:46:34.597 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.597 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.597 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.597 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:34.598 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503805+0000", 2024-09-04T17:46:34.598 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.598 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.598 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.598 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:34.598 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503806+0000", 2024-09-04T17:46:34.598 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.6300000000000004e-07 2024-09-04T17:46:34.598 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.598 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.598 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:34.598 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503809+0000", 2024-09-04T17:46:34.598 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.2880000000000001e-06 2024-09-04T17:46:34.598 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.599 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.599 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:34.599 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503818+0000", 2024-09-04T17:46:34.599 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7579999999999998e-06 2024-09-04T17:46:34.599 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.599 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.599 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:34.599 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503825+0000", 2024-09-04T17:46:34.599 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.058e-06 2024-09-04T17:46:34.599 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.599 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:34.599 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.599 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.599 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.600 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:32086 3.1c 3.967f817c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-04T17:46:34.600 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:34.503833+0000", 2024-09-04T17:46:34.600 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4294967295.999167, 2024-09-04T17:46:34.600 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00093882999999999998, 2024-09-04T17:46:34.600 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:34.600 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:34.600 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "queued for pg", 2024-09-04T17:46:34.600 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:34.600 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:34.600 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:34.600 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 32086 2024-09-04T17:46:34.600 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.600 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:34.601 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.601 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:34.601 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503833+0000", 2024-09-04T17:46:34.601 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.601 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.601 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.601 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:34.601 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503833+0000", 2024-09-04T17:46:34.601 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.601 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.601 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.601 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:34.601 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503834+0000", 2024-09-04T17:46:34.601 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4499999999999997e-07 2024-09-04T17:46:34.601 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.602 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.602 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:34.602 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503837+0000", 2024-09-04T17:46:34.602 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.621e-06 2024-09-04T17:46:34.602 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.602 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.602 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:34.602 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503845+0000", 2024-09-04T17:46:34.602 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.1149999999999994e-06 2024-09-04T17:46:34.602 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.602 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.602 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:34.602 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503851+0000", 2024-09-04T17:46:34.603 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.1929999999999998e-06 2024-09-04T17:46:34.603 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.603 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:34.603 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.603 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.603 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.603 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:32087 3.1c 3.967f817c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-04T17:46:34.603 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:34.503859+0000", 2024-09-04T17:46:34.603 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4294967295.9991417, 2024-09-04T17:46:34.603 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00097130299999999997, 2024-09-04T17:46:34.603 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:34.603 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:34.603 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "queued for pg", 2024-09-04T17:46:34.603 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:34.604 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:34.604 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:34.604 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 32087 2024-09-04T17:46:34.604 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.604 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:34.604 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.604 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:34.604 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503859+0000", 2024-09-04T17:46:34.604 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.604 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.604 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.604 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:34.604 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503859+0000", 2024-09-04T17:46:34.605 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.605 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.605 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.605 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:34.605 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503859+0000", 2024-09-04T17:46:34.605 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2099999999999996e-07 2024-09-04T17:46:34.605 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.605 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.605 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:34.605 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503862+0000", 2024-09-04T17:46:34.605 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.277e-06 2024-09-04T17:46:34.605 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.605 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.606 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:34.606 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503870+0000", 2024-09-04T17:46:34.606 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.9400000000000002e-06 2024-09-04T17:46:34.606 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.606 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.606 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:34.606 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503878+0000", 2024-09-04T17:46:34.606 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.3019999999999995e-06 2024-09-04T17:46:34.606 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.606 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:34.606 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.606 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.606 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.606 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:32088 3.1c 3.967f817c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-04T17:46:34.607 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:34.503886+0000", 2024-09-04T17:46:34.607 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4294967295.9991145, 2024-09-04T17:46:34.607 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000998243, 2024-09-04T17:46:34.607 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:34.607 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:34.607 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "queued for pg", 2024-09-04T17:46:34.607 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:34.607 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:34.607 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:34.607 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 32088 2024-09-04T17:46:34.607 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.607 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:34.607 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.608 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:34.608 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503886+0000", 2024-09-04T17:46:34.608 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.608 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.608 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.608 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:34.608 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503886+0000", 2024-09-04T17:46:34.608 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.608 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.608 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.608 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:34.608 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503886+0000", 2024-09-04T17:46:34.609 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.13e-07 2024-09-04T17:46:34.609 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.609 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.609 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:34.609 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503889+0000", 2024-09-04T17:46:34.609 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.8949999999999998e-06 2024-09-04T17:46:34.609 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.609 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.609 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:34.609 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503897+0000", 2024-09-04T17:46:34.609 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.1140000000000006e-06 2024-09-04T17:46:34.609 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.609 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.610 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:34.610 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503904+0000", 2024-09-04T17:46:34.610 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.5239999999999997e-06 2024-09-04T17:46:34.610 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.610 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:34.610 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.610 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.610 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.610 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:32089 3.1c 3.967f817c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-04T17:46:34.610 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:34.503912+0000", 2024-09-04T17:46:34.610 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4294967295.9990892, 2024-09-04T17:46:34.610 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0010315190000000001, 2024-09-04T17:46:34.610 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:34.610 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:34.611 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "queued for pg", 2024-09-04T17:46:34.611 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:34.611 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:34.611 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:34.611 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 32089 2024-09-04T17:46:34.611 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.611 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:34.611 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.611 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:34.611 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503912+0000", 2024-09-04T17:46:34.611 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.611 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.611 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.611 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:34.612 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503912+0000", 2024-09-04T17:46:34.612 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.612 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.612 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.612 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:34.612 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503912+0000", 2024-09-04T17:46:34.612 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.1399999999999997e-07 2024-09-04T17:46:34.612 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.612 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.612 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:34.612 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503915+0000", 2024-09-04T17:46:34.612 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.3700000000000002e-06 2024-09-04T17:46:34.612 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.613 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.613 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:34.613 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503922+0000", 2024-09-04T17:46:34.613 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.6979999999999998e-06 2024-09-04T17:46:34.613 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.613 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.613 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:34.613 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503928+0000", 2024-09-04T17:46:34.613 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.1070000000000001e-06 2024-09-04T17:46:34.613 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.613 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:34.613 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.613 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.613 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.614 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:32090 3.1c 3.967f817c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-04T17:46:34.614 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:34.503946+0000", 2024-09-04T17:46:34.614 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4294967295.9990549, 2024-09-04T17:46:34.614 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0010549999999999999, 2024-09-04T17:46:34.614 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:34.614 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:34.614 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "queued for pg", 2024-09-04T17:46:34.614 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:34.614 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:34.614 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:34.614 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 32090 2024-09-04T17:46:34.614 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.614 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:34.615 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.615 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:34.615 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503946+0000", 2024-09-04T17:46:34.615 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.615 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.615 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.615 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:34.615 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503946+0000", 2024-09-04T17:46:34.615 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.615 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.615 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.615 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:34.615 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503946+0000", 2024-09-04T17:46:34.615 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.8899999999999999e-07 2024-09-04T17:46:34.615 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.616 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.616 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:34.616 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503949+0000", 2024-09-04T17:46:34.616 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.8770000000000002e-06 2024-09-04T17:46:34.616 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.616 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.616 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:34.616 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503958+0000", 2024-09-04T17:46:34.616 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9660000000000002e-06 2024-09-04T17:46:34.616 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.616 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.616 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:34.616 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:34.503966+0000", 2024-09-04T17:46:34.617 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.5650000000000004e-06 2024-09-04T17:46:34.617 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.617 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:34.617 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.617 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.617 INFO:teuthology.orchestra.run.smithi190.stdout: ], 2024-09-04T17:46:34.617 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 32 2024-09-04T17:46:34.617 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:34.618 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-04T17:46:34.618 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-04T17:46:34.619 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:46:34.619 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:46:34.620 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-04T17:46:34.620 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:34.614+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:34.620 INFO:teuthology.orchestra.run.smithi092.stdout:1759 writesame 0x7d542f1 thru 0x7d601b4 data_size 0x3a(0xbec4 bytes) 2024-09-04T17:46:34.620 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:34.621 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:34.621 INFO:teuthology.orchestra.run.smithi092.stdout:1764 write 0x3b08058 thru 0x3b0c6db (0x4684 bytes) 2024-09-04T17:46:34.622 INFO:teuthology.orchestra.run.smithi092.stdout:1767 writesame 0x854b771 thru 0x855444f data_size 0x3(0x8cdf bytes) 2024-09-04T17:46:34.622 INFO:teuthology.orchestra.run.smithi092.stdout:1768 read 0x3896f75 thru 0x38a213f (0xb1cb bytes) 2024-09-04T17:46:34.622 INFO:teuthology.orchestra.run.smithi092.stdout:1769 write 0xc09f6ec thru 0xc0aced0 (0xd7e5 bytes) 2024-09-04T17:46:34.622 INFO:teuthology.orchestra.run.smithi092.stdout:1770 trunc from 0xc0aced1 to 0x3c935ce 2024-09-04T17:46:34.622 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:34.622 INFO:teuthology.orchestra.run.smithi092.stdout:1773 read 0x5f8e79 thru 0x6001a7 (0x732f bytes) 2024-09-04T17:46:34.622 INFO:teuthology.orchestra.run.smithi092.stdout:1774 punch from 0x33a3a30 to 0x33af6fe, (0xbcce bytes) 2024-09-04T17:46:34.623 INFO:teuthology.orchestra.run.smithi092.stdout:1775 write 0x4f3fd59 thru 0x4f4c207 (0xc4af bytes) 2024-09-04T17:46:34.631 INFO:teuthology.orchestra.run.smithi092.stdout:1776 trunc from 0x4f4c208 to 0xe3ed2f7 2024-09-04T17:46:34.643 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:34.655 INFO:teuthology.orchestra.run.smithi092.stdout:1779 trunc from 0xe3ed2f7 to 0x4458714 2024-09-04T17:46:34.676 INFO:teuthology.orchestra.run.smithi092.stdout:1780 punch from 0x32b1171 to 0x32b183a, (0x6c9 bytes) 2024-09-04T17:46:34.688 INFO:teuthology.orchestra.run.smithi092.stdout:1781 write 0x52effa5 thru 0x52fd449 (0xd4a5 bytes) 2024-09-04T17:46:34.699 INFO:teuthology.orchestra.run.smithi092.stdout:1782 read 0x3e784b3 thru 0x3e7d23a (0x4d88 bytes) 2024-09-04T17:46:34.713 INFO:teuthology.orchestra.run.smithi092.stdout:1783 write 0x83a1aa0 thru 0x83aa441 (0x89a2 bytes) 2024-09-04T17:46:34.719 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:34.716+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:34.725 INFO:teuthology.orchestra.run.smithi092.stdout:1784 punch from 0x7e93374 to 0x7e97459, (0x40e5 bytes) 2024-09-04T17:46:34.725 INFO:teuthology.orchestra.run.smithi092.stdout:1785 punch from 0x6e79728 to 0x6e80f6b, (0x7843 bytes) 2024-09-04T17:46:34.725 INFO:teuthology.orchestra.run.smithi092.stdout:1786 read 0x78c947c thru 0x78cc514 (0x3099 bytes) 2024-09-04T17:46:34.738 INFO:teuthology.orchestra.run.smithi092.stdout:1787 writesame 0x9f21fce thru 0x9f2809b data_size 0x1(0x60ce bytes) 2024-09-04T17:46:34.738 INFO:teuthology.orchestra.run.smithi092.stdout:1788 write 0x141088d thru 0x1413326 (0x2a9a bytes) 2024-09-04T17:46:34.749 INFO:teuthology.orchestra.run.smithi092.stdout:1789 read 0x446a9f7 thru 0x447a9d2 (0xffdc bytes) 2024-09-04T17:46:34.762 INFO:teuthology.orchestra.run.smithi092.stdout:1790 trunc from 0x9f2809c to 0x445d996 2024-09-04T17:46:34.799 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:34.800 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:46:34.800 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:46:34.800 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:46:34.800 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:34.814 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-04T17:46:34.818 INFO:teuthology.orchestra.run.smithi092.stdout:1791 writesame 0xd4ff896 thru 0xd507208 data_size 0x1(0x7973 bytes) 2024-09-04T17:46:34.818 INFO:teuthology.orchestra.run.smithi092.stdout:1792 read 0x7aa39b5 thru 0x7aaf9e1 (0xc02d bytes) 2024-09-04T17:46:34.818 INFO:teuthology.orchestra.run.smithi092.stdout:1793 write 0x7a85353 thru 0x7a8ae7b (0x5b29 bytes) 2024-09-04T17:46:34.837 INFO:teuthology.orchestra.run.smithi092.stdout:1796 write 0x777468a thru 0x777cbb5 (0x852c bytes) 2024-09-04T17:46:34.852 INFO:teuthology.orchestra.run.smithi092.stdout:1798 write 0x39653b8 thru 0x39740ad (0xecf6 bytes) 2024-09-04T17:46:34.860 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:34.858+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:34.869 INFO:teuthology.orchestra.run.smithi092.stdout:1799 read 0x87fffb thru 0x88e55c (0xe562 bytes) 2024-09-04T17:46:34.869 INFO:teuthology.orchestra.run.smithi092.stdout:1800 write 0x81e92b7 thru 0x81ed8d2 (0x461c bytes) 2024-09-04T17:46:34.882 INFO:teuthology.orchestra.run.smithi092.stdout:1801 punch from 0xc465e5f to 0xc46a945, (0x4ae6 bytes) 2024-09-04T17:46:34.882 INFO:teuthology.orchestra.run.smithi092.stdout:1803 read 0x4dbd93a thru 0x4dc0205 (0x28cc bytes) 2024-09-04T17:46:34.882 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:34.882 INFO:teuthology.orchestra.run.smithi092.stdout:1806 read 0xb29e26 thru 0xb2cf35 (0x3110 bytes) 2024-09-04T17:46:34.890 INFO:teuthology.orchestra.run.smithi092.stdout:1807 write 0xd71f58b thru 0xd722ee0 (0x3956 bytes) 2024-09-04T17:46:34.902 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:34.902 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:34.902 INFO:teuthology.orchestra.run.smithi092.stdout:1810 writesame 0x451dc64 thru 0x4529dd8 data_size 0x23(0xc175 bytes) 2024-09-04T17:46:34.921 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:46:34.918+0000 7f505d642640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26446) UID: 0 2024-09-04T17:46:34.942 INFO:teuthology.orchestra.run.smithi092.stdout:1811 trunc from 0xd722ee1 to 0x3a55f08 2024-09-04T17:46:34.948 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:34.948 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:46:34.948 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:46:34.948 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:46:34.949 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.949 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8831 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2453504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:34.949 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:46:34.949 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 21.889141405, 2024-09-04T17:46:34.949 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14731111799999999, 2024-09-04T17:46:34.949 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:34.949 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:34.949 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:34.949 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:34.949 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:34.949 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:34.949 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8831 2024-09-04T17:46:34.949 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.950 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:34.950 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.950 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:34.950 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:46:34.950 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.950 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.950 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.950 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:34.950 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:46:34.950 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.950 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.950 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.950 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:34.950 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057811+0000", 2024-09-04T17:46:34.951 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.083e-06 2024-09-04T17:46:34.951 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.951 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.951 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:34.951 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057814+0000", 2024-09-04T17:46:34.951 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.407e-06 2024-09-04T17:46:34.951 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.951 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.951 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:34.951 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057825+0000", 2024-09-04T17:46:34.951 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.17e-05 2024-09-04T17:46:34.951 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.951 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.952 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:34.952 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057836+0000", 2024-09-04T17:46:34.952 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0283e-05 2024-09-04T17:46:34.952 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.952 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.952 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:34.952 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.204762+0000", 2024-09-04T17:46:34.952 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.146926205 2024-09-04T17:46:34.952 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.952 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.952 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:34.952 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.204895+0000", 2024-09-04T17:46:34.952 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013328999999999999 2024-09-04T17:46:34.952 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.953 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.953 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:34.953 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205121+0000", 2024-09-04T17:46:34.953 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00022614999999999999 2024-09-04T17:46:34.953 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.953 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:34.953 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.953 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.953 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.953 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8832 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2490368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:34.953 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:46:34.953 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 21.889103372000001, 2024-09-04T17:46:34.953 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14770012299999999, 2024-09-04T17:46:34.954 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:34.954 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:34.954 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:34.954 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:34.955 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:34.955 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:34.955 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8832 2024-09-04T17:46:34.955 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.955 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:34.955 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.955 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:34.955 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:46:34.955 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.956 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.956 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.956 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:34.956 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:46:34.956 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.956 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.956 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.956 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:34.956 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057849+0000", 2024-09-04T17:46:34.956 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6700000000000003e-07 2024-09-04T17:46:34.956 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.957 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.957 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:34.957 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057852+0000", 2024-09-04T17:46:34.957 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6359999999999998e-06 2024-09-04T17:46:34.957 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.957 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.957 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:34.957 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057860+0000", 2024-09-04T17:46:34.957 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8140000000000007e-06 2024-09-04T17:46:34.957 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.957 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.957 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:34.958 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057868+0000", 2024-09-04T17:46:34.958 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1999999999999997e-06 2024-09-04T17:46:34.958 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.958 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.958 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:34.958 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205208+0000", 2024-09-04T17:46:34.958 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14734067300000001 2024-09-04T17:46:34.958 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.958 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.958 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:34.958 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205347+0000", 2024-09-04T17:46:34.958 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013914900000000001 2024-09-04T17:46:34.958 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.958 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.959 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:34.959 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205548+0000", 2024-09-04T17:46:34.959 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000200984 2024-09-04T17:46:34.959 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.959 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:34.959 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.959 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.959 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.959 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8833 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2527232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:34.959 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:46:34.959 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 21.889071502, 2024-09-04T17:46:34.959 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148062572, 2024-09-04T17:46:34.959 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:34.959 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:34.960 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:34.960 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:34.960 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:34.960 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:34.960 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8833 2024-09-04T17:46:34.960 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.960 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:34.960 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.960 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:34.960 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:46:34.960 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.960 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.960 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.961 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:34.961 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:46:34.961 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.961 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.961 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.961 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:34.961 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057881+0000", 2024-09-04T17:46:34.961 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4300000000000002e-07 2024-09-04T17:46:34.961 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.961 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.961 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:34.961 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057893+0000", 2024-09-04T17:46:34.961 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1973999999999999e-05 2024-09-04T17:46:34.961 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.962 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.962 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:34.962 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057906+0000", 2024-09-04T17:46:34.962 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2716e-05 2024-09-04T17:46:34.962 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.962 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.962 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:34.962 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057913+0000", 2024-09-04T17:46:34.962 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3930000000000002e-06 2024-09-04T17:46:34.962 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.962 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.962 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:34.962 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205621+0000", 2024-09-04T17:46:34.962 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14770788800000001 2024-09-04T17:46:34.963 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.963 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.963 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:34.963 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205740+0000", 2024-09-04T17:46:34.963 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000118936 2024-09-04T17:46:34.963 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.963 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.963 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:34.963 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205943+0000", 2024-09-04T17:46:34.963 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020292199999999999 2024-09-04T17:46:34.963 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.963 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:34.963 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.963 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.963 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.964 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8834 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2564096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:34.964 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:46:34.964 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 21.889028223, 2024-09-04T17:46:34.964 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14846753200000001, 2024-09-04T17:46:34.964 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:34.964 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:34.964 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:34.964 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:34.964 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:34.964 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:34.964 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8834 2024-09-04T17:46:34.964 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.964 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:34.965 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.965 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:34.965 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:46:34.965 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.965 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.965 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.965 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:34.965 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:46:34.965 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.965 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.965 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.965 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:34.965 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057925+0000", 2024-09-04T17:46:34.966 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0270000000000001e-06 2024-09-04T17:46:34.966 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.966 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.966 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:34.966 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057936+0000", 2024-09-04T17:46:34.966 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.112e-05 2024-09-04T17:46:34.966 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.967 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.967 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:34.967 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057946+0000", 2024-09-04T17:46:34.967 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.009e-05 2024-09-04T17:46:34.967 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.967 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.967 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:34.967 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057953+0000", 2024-09-04T17:46:34.967 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.6229999999999997e-06 2024-09-04T17:46:34.967 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.968 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.968 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:34.968 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206027+0000", 2024-09-04T17:46:34.968 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148073591 2024-09-04T17:46:34.968 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.968 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.969 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:34.969 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206161+0000", 2024-09-04T17:46:34.969 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013411599999999999 2024-09-04T17:46:34.969 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.969 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.969 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:34.969 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206391+0000", 2024-09-04T17:46:34.969 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000229965 2024-09-04T17:46:34.969 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.969 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:34.969 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.970 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.970 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.970 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8835 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2600960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:34.970 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:46:34.970 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 21.888988572999999, 2024-09-04T17:46:34.970 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14882941599999999, 2024-09-04T17:46:34.970 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:34.970 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:34.971 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:34.971 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:34.971 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:34.971 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:34.971 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8835 2024-09-04T17:46:34.971 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.971 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:34.971 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.971 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:34.971 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:46:34.971 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.971 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.971 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.972 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:34.972 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:46:34.972 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.972 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.972 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.972 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:34.972 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058544+0000", 2024-09-04T17:46:34.972 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00058061899999999999 2024-09-04T17:46:34.972 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.972 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.972 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:34.972 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058550+0000", 2024-09-04T17:46:34.972 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.1129999999999997e-06 2024-09-04T17:46:34.973 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.973 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.973 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:34.973 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058566+0000", 2024-09-04T17:46:34.973 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5820000000000001e-05 2024-09-04T17:46:34.973 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.973 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.973 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:34.973 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058578+0000", 2024-09-04T17:46:34.973 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2537e-05 2024-09-04T17:46:34.973 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.973 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.973 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:34.974 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206470+0000", 2024-09-04T17:46:34.974 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14789153799999999 2024-09-04T17:46:34.974 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.974 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.974 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:34.974 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206589+0000", 2024-09-04T17:46:34.974 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000118837 2024-09-04T17:46:34.974 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.974 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.974 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:34.974 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206793+0000", 2024-09-04T17:46:34.974 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000203952 2024-09-04T17:46:34.974 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.974 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:34.975 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.975 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.975 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.975 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8836 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2654208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:34.975 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:46:34.975 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 21.888355884999999, 2024-09-04T17:46:34.975 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14862433999999999, 2024-09-04T17:46:34.975 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:34.975 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:34.975 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:34.975 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:34.975 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:34.975 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:34.976 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8836 2024-09-04T17:46:34.976 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.976 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:34.976 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.976 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:34.976 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:46:34.976 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.976 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.976 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.976 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:34.976 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:46:34.976 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.976 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.977 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.977 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:34.977 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062663+0000", 2024-09-04T17:46:34.977 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0040670619999999998 2024-09-04T17:46:34.977 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.977 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.977 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:34.977 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062668+0000", 2024-09-04T17:46:34.977 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4199999999999998e-06 2024-09-04T17:46:34.977 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.978 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.978 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:34.978 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062689+0000", 2024-09-04T17:46:34.978 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0356999999999998e-05 2024-09-04T17:46:34.978 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.978 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.978 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:34.978 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062726+0000", 2024-09-04T17:46:34.978 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7302999999999998e-05 2024-09-04T17:46:34.978 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.978 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.978 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:34.978 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206875+0000", 2024-09-04T17:46:34.978 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14414883100000001 2024-09-04T17:46:34.979 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.979 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.979 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:34.979 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.207016+0000", 2024-09-04T17:46:34.979 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000140822 2024-09-04T17:46:34.979 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.979 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.979 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:34.979 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.207220+0000", 2024-09-04T17:46:34.979 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020454499999999999 2024-09-04T17:46:34.979 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.979 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:34.979 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.980 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.980 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.980 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8844 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2928640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:34.980 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:46:34.980 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 21.883991802000001, 2024-09-04T17:46:34.980 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14752153100000001, 2024-09-04T17:46:34.980 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:34.980 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:34.980 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:34.980 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:34.980 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:34.980 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:34.980 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8844 2024-09-04T17:46:34.981 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.981 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:34.981 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.981 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:34.981 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:46:34.981 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.981 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.981 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.981 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:34.981 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:46:34.981 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.981 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.982 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.982 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:34.982 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062961+0000", 2024-09-04T17:46:34.982 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6700000000000003e-07 2024-09-04T17:46:34.982 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.982 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.982 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:34.982 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062973+0000", 2024-09-04T17:46:34.982 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2109999999999999e-05 2024-09-04T17:46:34.982 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.982 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.982 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:34.982 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062982+0000", 2024-09-04T17:46:34.982 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.6900000000000004e-06 2024-09-04T17:46:34.983 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.983 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.983 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:34.983 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062990+0000", 2024-09-04T17:46:34.983 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1899999999999998e-06 2024-09-04T17:46:34.983 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.983 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.983 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:34.983 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210090+0000", 2024-09-04T17:46:34.984 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14710026300000001 2024-09-04T17:46:34.984 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.984 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.984 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:34.984 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210225+0000", 2024-09-04T17:46:34.984 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013532400000000001 2024-09-04T17:46:34.984 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.984 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.984 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:34.984 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210481+0000", 2024-09-04T17:46:34.984 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00025628700000000001 2024-09-04T17:46:34.984 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.984 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:34.985 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.985 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.985 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.985 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8845 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2965504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:34.985 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:46:34.985 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 21.883952079, 2024-09-04T17:46:34.985 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147916819, 2024-09-04T17:46:34.985 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:34.985 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:34.985 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:34.985 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:34.985 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:34.985 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:34.986 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8845 2024-09-04T17:46:34.986 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.986 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:34.986 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.986 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:34.986 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:46:34.986 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.986 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.986 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.986 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:34.986 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:46:34.986 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.986 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.987 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.987 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:34.987 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:46:34.987 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.0399999999999995e-07 2024-09-04T17:46:34.987 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.987 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.987 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:34.987 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063003+0000", 2024-09-04T17:46:34.987 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6529999999999998e-06 2024-09-04T17:46:34.987 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.987 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.987 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:34.988 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063012+0000", 2024-09-04T17:46:34.988 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9500000000000007e-06 2024-09-04T17:46:34.988 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.988 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.988 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:34.988 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063027+0000", 2024-09-04T17:46:34.988 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5443000000000001e-05 2024-09-04T17:46:34.988 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.988 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.988 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:34.988 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210571+0000", 2024-09-04T17:46:34.988 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14754353100000001 2024-09-04T17:46:34.988 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.988 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.989 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:34.989 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210714+0000", 2024-09-04T17:46:34.989 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014295599999999999 2024-09-04T17:46:34.989 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.989 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.989 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:34.989 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210916+0000", 2024-09-04T17:46:34.989 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020258200000000001 2024-09-04T17:46:34.989 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.989 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:34.989 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.989 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.989 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.990 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8846 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3002368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:34.990 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:46:34.990 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 21.883914541999999, 2024-09-04T17:46:34.990 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14828011599999999, 2024-09-04T17:46:34.990 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:34.990 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:34.990 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:34.990 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:34.990 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:34.990 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:34.990 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8846 2024-09-04T17:46:34.990 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.990 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:34.991 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.991 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:34.991 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:46:34.991 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.991 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.991 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.991 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:34.991 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:46:34.991 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.991 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.991 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.991 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:34.991 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063038+0000", 2024-09-04T17:46:34.991 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.23e-07 2024-09-04T17:46:34.992 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.992 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.992 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:34.992 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063051+0000", 2024-09-04T17:46:34.992 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2757e-05 2024-09-04T17:46:34.992 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.992 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.992 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:34.992 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063070+0000", 2024-09-04T17:46:34.992 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9663000000000001e-05 2024-09-04T17:46:34.992 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.992 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.993 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:34.993 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063079+0000", 2024-09-04T17:46:34.993 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.0929999999999992e-06 2024-09-04T17:46:34.993 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.993 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.993 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:34.993 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210989+0000", 2024-09-04T17:46:34.993 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14790984500000001 2024-09-04T17:46:34.993 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.993 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.993 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:34.993 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211109+0000", 2024-09-04T17:46:34.993 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000120056 2024-09-04T17:46:34.994 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.994 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.994 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:34.994 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211317+0000", 2024-09-04T17:46:34.994 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020797900000000001 2024-09-04T17:46:34.994 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.994 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:34.994 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.994 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.994 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.994 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8847 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3039232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:34.994 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:46:34.994 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 21.883840110000001, 2024-09-04T17:46:34.995 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14859408900000001, 2024-09-04T17:46:34.995 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:34.995 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:34.995 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:34.995 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:34.995 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:34.995 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:34.995 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8847 2024-09-04T17:46:34.995 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.995 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:34.995 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.995 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:34.995 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:46:34.995 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.996 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.996 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.996 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:34.996 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:46:34.996 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:34.996 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.996 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.996 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:34.996 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063113+0000", 2024-09-04T17:46:34.996 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.204e-06 2024-09-04T17:46:34.996 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.996 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.997 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:34.997 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063116+0000", 2024-09-04T17:46:34.997 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.3330000000000001e-06 2024-09-04T17:46:34.997 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.997 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.997 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:34.997 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063126+0000", 2024-09-04T17:46:34.997 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0166999999999999e-05 2024-09-04T17:46:34.997 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.997 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.997 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:34.997 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063134+0000", 2024-09-04T17:46:34.997 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4329999999999998e-06 2024-09-04T17:46:34.997 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.998 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.998 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:34.998 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211392+0000", 2024-09-04T17:46:34.998 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148258109 2024-09-04T17:46:34.998 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.998 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.998 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:34.998 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211506+0000", 2024-09-04T17:46:34.998 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000113813 2024-09-04T17:46:34.998 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.998 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.998 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:34.998 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211706+0000", 2024-09-04T17:46:34.999 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020002999999999999 2024-09-04T17:46:34.999 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.999 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:34.999 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:34.999 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:34.999 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:34.999 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8848 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:34.999 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:46:34.999 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 21.883807773000001, 2024-09-04T17:46:34.999 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14907967599999999, 2024-09-04T17:46:34.999 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:34.999 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:35.000 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:35.000 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:35.000 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:35.000 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:35.000 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8848 2024-09-04T17:46:35.000 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.000 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:35.000 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.000 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:35.000 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:46:35.000 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:35.000 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.000 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.001 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:35.001 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:46:35.001 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:35.001 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.001 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.001 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:35.001 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070589+0000", 2024-09-04T17:46:35.001 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.007445386 2024-09-04T17:46:35.001 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.001 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.001 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:35.001 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070600+0000", 2024-09-04T17:46:35.001 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0346999999999999e-05 2024-09-04T17:46:35.002 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.002 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.002 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:35.002 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070622+0000", 2024-09-04T17:46:35.002 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2289999999999998e-05 2024-09-04T17:46:35.002 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.002 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.002 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:35.002 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070636+0000", 2024-09-04T17:46:35.002 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4093e-05 2024-09-04T17:46:35.002 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.002 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.002 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:35.003 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211815+0000", 2024-09-04T17:46:35.003 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14117874699999999 2024-09-04T17:46:35.003 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.003 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.003 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:35.003 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211983+0000", 2024-09-04T17:46:35.003 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00016852800000000001 2024-09-04T17:46:35.003 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.003 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.003 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:35.003 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.212224+0000", 2024-09-04T17:46:35.003 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024028499999999999 2024-09-04T17:46:35.003 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:35.004 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:35.004 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:35.004 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.004 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.004 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8862 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2019328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:35.004 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:46:35.004 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 21.875758305000002, 2024-09-04T17:46:35.004 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14755236399999999, 2024-09-04T17:46:35.004 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:35.004 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:35.004 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:35.004 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:35.005 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:35.005 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:35.005 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8862 2024-09-04T17:46:35.005 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.005 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:35.005 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.005 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:35.005 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:46:35.005 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:35.005 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.005 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.005 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:35.005 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:46:35.006 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:35.006 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.006 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.006 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:35.006 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071194+0000", 2024-09-04T17:46:35.006 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.5000000000000002e-07 2024-09-04T17:46:35.006 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.006 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.006 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:35.006 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071197+0000", 2024-09-04T17:46:35.006 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5900000000000002e-06 2024-09-04T17:46:35.006 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.006 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.007 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:35.007 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071206+0000", 2024-09-04T17:46:35.007 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9700000000000005e-06 2024-09-04T17:46:35.007 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.007 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.007 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:35.007 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071213+0000", 2024-09-04T17:46:35.007 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.9070000000000001e-06 2024-09-04T17:46:35.007 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.007 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.007 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:35.007 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218396+0000", 2024-09-04T17:46:35.007 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147183273 2024-09-04T17:46:35.007 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.008 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.008 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:35.008 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218538+0000", 2024-09-04T17:46:35.008 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014194199999999999 2024-09-04T17:46:35.008 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.008 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.008 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:35.008 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218746+0000", 2024-09-04T17:46:35.008 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000208032 2024-09-04T17:46:35.008 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:35.008 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:35.008 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:35.008 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.009 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.009 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8863 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2056192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:35.009 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:46:35.009 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 21.875729995, 2024-09-04T17:46:35.009 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14792281099999999, 2024-09-04T17:46:35.009 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:35.009 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:35.009 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:35.009 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:35.009 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:35.009 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:35.009 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8863 2024-09-04T17:46:35.009 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.009 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:35.010 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.010 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:35.010 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:46:35.010 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:35.010 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.010 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.010 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:35.010 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:46:35.010 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:35.010 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.010 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.010 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:35.010 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:46:35.011 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1299999999999999e-07 2024-09-04T17:46:35.011 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.011 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.011 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:35.011 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071225+0000", 2024-09-04T17:46:35.011 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6199999999999999e-06 2024-09-04T17:46:35.011 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.011 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.011 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:35.011 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071234+0000", 2024-09-04T17:46:35.011 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8769999999999999e-06 2024-09-04T17:46:35.011 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.011 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.012 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:35.012 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071251+0000", 2024-09-04T17:46:35.012 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.738e-05 2024-09-04T17:46:35.012 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.012 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.012 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:35.012 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218798+0000", 2024-09-04T17:46:35.012 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14754645 2024-09-04T17:46:35.012 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.012 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.012 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:35.012 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218926+0000", 2024-09-04T17:46:35.012 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000128446 2024-09-04T17:46:35.013 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.013 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.013 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:35.013 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219145+0000", 2024-09-04T17:46:35.013 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021832500000000001 2024-09-04T17:46:35.013 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:35.013 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:35.013 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:35.013 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.013 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.013 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8864 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2093056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:35.013 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:46:35.013 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 21.875688418999999, 2024-09-04T17:46:35.013 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148277309, 2024-09-04T17:46:35.014 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:35.014 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:35.014 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:35.014 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:35.014 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:35.014 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:35.014 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8864 2024-09-04T17:46:35.014 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.014 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:35.014 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.014 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:35.014 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:46:35.015 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:35.015 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.015 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.015 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:35.015 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:46:35.015 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:35.015 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.015 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.015 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:35.015 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071264+0000", 2024-09-04T17:46:35.015 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3e-07 2024-09-04T17:46:35.015 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.015 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.016 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:35.016 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071267+0000", 2024-09-04T17:46:35.016 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0139999999999999e-06 2024-09-04T17:46:35.016 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.016 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.016 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:35.016 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071276+0000", 2024-09-04T17:46:35.016 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1689999999999998e-06 2024-09-04T17:46:35.016 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.016 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.016 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:35.016 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071283+0000", 2024-09-04T17:46:35.016 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2269999999999996e-06 2024-09-04T17:46:35.017 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.017 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.017 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:35.017 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219195+0000", 2024-09-04T17:46:35.017 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147911868 2024-09-04T17:46:35.017 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.017 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.017 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:35.017 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219324+0000", 2024-09-04T17:46:35.018 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00012835900000000001 2024-09-04T17:46:35.018 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.018 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.018 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:35.018 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219541+0000", 2024-09-04T17:46:35.018 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000216942 2024-09-04T17:46:35.018 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:35.018 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:35.018 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:35.018 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.018 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.018 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8865 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2109440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:35.018 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:46:35.018 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 21.875658958999999, 2024-09-04T17:46:35.019 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14865747300000001, 2024-09-04T17:46:35.019 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:35.019 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:35.019 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:35.019 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:35.019 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:35.019 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:35.019 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8865 2024-09-04T17:46:35.019 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.019 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:35.019 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.019 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:35.020 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:46:35.020 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:35.020 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.020 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.020 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:35.020 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:46:35.020 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:35.020 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.020 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.021 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:35.021 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:46:35.021 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7999999999999995e-07 2024-09-04T17:46:35.021 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.021 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.021 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:35.021 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071296+0000", 2024-09-04T17:46:35.021 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6900000000000001e-06 2024-09-04T17:46:35.021 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.021 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.021 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:35.021 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071305+0000", 2024-09-04T17:46:35.021 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.5499999999999995e-06 2024-09-04T17:46:35.022 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.022 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.022 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:35.022 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071312+0000", 2024-09-04T17:46:35.022 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1559999999999998e-06 2024-09-04T17:46:35.022 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.022 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.022 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:35.022 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219591+0000", 2024-09-04T17:46:35.022 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14827944900000001 2024-09-04T17:46:35.022 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.022 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.022 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:35.023 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219731+0000", 2024-09-04T17:46:35.023 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013931600000000001 2024-09-04T17:46:35.023 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.023 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.023 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:35.023 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219950+0000", 2024-09-04T17:46:35.023 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021963199999999999 2024-09-04T17:46:35.023 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:35.023 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:35.023 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:35.023 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.023 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.023 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8866 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2146304~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:35.024 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:46:35.024 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 21.875630726000001, 2024-09-04T17:46:35.024 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14906117899999999, 2024-09-04T17:46:35.024 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:35.024 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:35.024 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:35.024 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:35.024 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:35.024 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:35.024 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8866 2024-09-04T17:46:35.024 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.024 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:35.024 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.025 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:35.025 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:46:35.025 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:35.025 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.025 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.025 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:35.025 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:46:35.025 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:35.025 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.025 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.025 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:35.025 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071322+0000", 2024-09-04T17:46:35.025 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7999999999999995e-07 2024-09-04T17:46:35.025 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.026 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.026 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:35.026 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071338+0000", 2024-09-04T17:46:35.026 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5872999999999999e-05 2024-09-04T17:46:35.026 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.026 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.026 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:35.026 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071347+0000", 2024-09-04T17:46:35.026 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4669999999999993e-06 2024-09-04T17:46:35.026 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.026 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.026 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:35.026 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071354+0000", 2024-09-04T17:46:35.027 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2099999999999996e-06 2024-09-04T17:46:35.027 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.027 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.027 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:35.027 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220008+0000", 2024-09-04T17:46:35.027 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148653642 2024-09-04T17:46:35.027 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.027 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.027 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:35.027 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220155+0000", 2024-09-04T17:46:35.027 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014726600000000001 2024-09-04T17:46:35.027 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.027 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.028 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:35.028 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220382+0000", 2024-09-04T17:46:35.028 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00022704100000000001 2024-09-04T17:46:35.028 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:35.028 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:35.028 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:35.028 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.028 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.028 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8867 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2183168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:35.028 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:46:35.028 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 21.875587372999998, 2024-09-04T17:46:35.028 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.149445831, 2024-09-04T17:46:35.029 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:35.029 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:35.029 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:35.029 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:35.029 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:35.029 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:35.029 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8867 2024-09-04T17:46:35.029 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.029 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:35.029 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.029 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:35.029 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:46:35.029 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:35.029 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.030 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.030 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:35.030 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:46:35.030 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:35.030 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.030 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.030 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:35.030 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078653+0000", 2024-09-04T17:46:35.030 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0072890430000000003 2024-09-04T17:46:35.030 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.030 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.030 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:35.031 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078664+0000", 2024-09-04T17:46:35.031 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0307e-05 2024-09-04T17:46:35.031 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.031 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.031 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:35.031 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078690+0000", 2024-09-04T17:46:35.031 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6780000000000001e-05 2024-09-04T17:46:35.031 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.031 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.031 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:35.031 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078706+0000", 2024-09-04T17:46:35.031 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5676999999999999e-05 2024-09-04T17:46:35.032 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.032 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.032 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:35.032 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220467+0000", 2024-09-04T17:46:35.032 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14176103500000001 2024-09-04T17:46:35.032 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.032 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.032 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:35.032 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220595+0000", 2024-09-04T17:46:35.032 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000127427 2024-09-04T17:46:35.032 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.032 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.032 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:35.032 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220810+0000", 2024-09-04T17:46:35.033 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021556200000000001 2024-09-04T17:46:35.033 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:35.033 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:35.033 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:35.033 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.033 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.033 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8880 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2658304~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:35.033 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.079158+0000", 2024-09-04T17:46:35.033 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 21.867794094000001, 2024-09-04T17:46:35.033 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14718968499999999, 2024-09-04T17:46:35.033 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:35.033 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:35.033 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:35.034 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:35.034 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:35.034 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:35.034 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8880 2024-09-04T17:46:35.034 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.034 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:35.034 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.034 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:35.034 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079158+0000", 2024-09-04T17:46:35.034 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:35.034 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.034 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.034 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:35.035 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079158+0000", 2024-09-04T17:46:35.035 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:35.035 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.035 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.035 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:35.035 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079158+0000", 2024-09-04T17:46:35.035 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3e-07 2024-09-04T17:46:35.035 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.035 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.035 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:35.035 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079161+0000", 2024-09-04T17:46:35.035 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.57e-06 2024-09-04T17:46:35.035 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.035 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.036 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:35.036 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079170+0000", 2024-09-04T17:46:35.036 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.0170000000000002e-06 2024-09-04T17:46:35.036 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.036 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.036 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:35.036 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079177+0000", 2024-09-04T17:46:35.036 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.0530000000000001e-06 2024-09-04T17:46:35.036 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.036 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.036 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:35.036 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.225946+0000", 2024-09-04T17:46:35.036 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14676936500000001 2024-09-04T17:46:35.037 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.037 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.037 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:35.037 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226098+0000", 2024-09-04T17:46:35.037 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000151805 2024-09-04T17:46:35.037 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.037 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.037 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:35.037 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226347+0000", 2024-09-04T17:46:35.037 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024914499999999999 2024-09-04T17:46:35.037 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:35.037 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:35.037 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:35.038 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.038 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.038 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8881 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2695168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:35.038 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:46:35.038 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 21.867765303999999, 2024-09-04T17:46:35.038 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147543547, 2024-09-04T17:46:35.038 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:35.038 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:35.038 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:35.038 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:35.038 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:35.038 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:35.038 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8881 2024-09-04T17:46:35.039 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.039 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:35.039 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.039 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:35.039 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:46:35.039 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:35.039 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.039 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.039 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:35.039 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:46:35.039 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:35.039 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.040 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.040 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:35.040 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079187+0000", 2024-09-04T17:46:35.040 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7700000000000004e-07 2024-09-04T17:46:35.040 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.040 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.040 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:35.040 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079198+0000", 2024-09-04T17:46:35.040 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0849999999999999e-05 2024-09-04T17:46:35.040 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.040 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.040 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:35.040 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079207+0000", 2024-09-04T17:46:35.041 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5599999999999999e-06 2024-09-04T17:46:35.041 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.041 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.041 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:35.041 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079224+0000", 2024-09-04T17:46:35.041 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6387000000000001e-05 2024-09-04T17:46:35.041 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.041 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.041 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:35.041 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226414+0000", 2024-09-04T17:46:35.041 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14718975000000001 2024-09-04T17:46:35.041 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.042 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.042 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:35.042 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226534+0000", 2024-09-04T17:46:35.042 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000120288 2024-09-04T17:46:35.042 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.042 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.043 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:35.043 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226730+0000", 2024-09-04T17:46:35.043 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000196035 2024-09-04T17:46:35.043 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:35.043 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:35.043 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:35.043 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.043 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.043 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8882 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2732032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:35.043 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:46:35.043 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 21.867717089999999, 2024-09-04T17:46:35.043 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14795022399999999, 2024-09-04T17:46:35.043 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:35.044 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:35.044 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:35.044 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:35.044 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:35.044 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:35.044 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8882 2024-09-04T17:46:35.044 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.044 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:35.044 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.044 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:35.044 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:46:35.044 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:35.045 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.045 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.045 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:35.045 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:46:35.045 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:35.045 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.045 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.045 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:35.045 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086588+0000", 2024-09-04T17:46:35.045 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0073530959999999999 2024-09-04T17:46:35.045 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.045 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.045 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:35.046 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086598+0000", 2024-09-04T17:46:35.046 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0122999999999999e-05 2024-09-04T17:46:35.046 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.046 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.046 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:35.046 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086626+0000", 2024-09-04T17:46:35.046 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.7914000000000001e-05 2024-09-04T17:46:35.046 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.046 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.046 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:35.046 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086641+0000", 2024-09-04T17:46:35.046 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5270000000000001e-05 2024-09-04T17:46:35.047 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.047 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.047 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:35.047 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226801+0000", 2024-09-04T17:46:35.047 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.140160321 2024-09-04T17:46:35.047 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.047 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.047 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:35.047 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226951+0000", 2024-09-04T17:46:35.047 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014965200000000001 2024-09-04T17:46:35.047 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.047 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.047 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:35.048 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.227185+0000", 2024-09-04T17:46:35.048 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00023384799999999999 2024-09-04T17:46:35.048 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:35.048 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:35.048 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:35.048 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:35.048 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:35.048 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:35.050 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-04T17:46:35.051 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:35.046+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:35.051 INFO:teuthology.orchestra.run.smithi092.stdout:1812 read 0x2ab158a thru 0x2abd4a9 (0xbf20 bytes) 2024-09-04T17:46:35.051 INFO:teuthology.orchestra.run.smithi092.stdout:1814 read 0x11de440 thru 0x11e6261 (0x7e22 bytes) 2024-09-04T17:46:35.051 INFO:teuthology.orchestra.run.smithi092.stdout:1816 trunc from 0x3a55f08 to 0xdd0ec83 2024-09-04T17:46:35.051 INFO:teuthology.orchestra.run.smithi092.stdout:1817 punch from 0x5732f43 to 0x573f998, (0xca55 bytes) 2024-09-04T17:46:35.051 INFO:teuthology.orchestra.run.smithi092.stdout:1818 write 0x4e2e6d7 thru 0x4e31d07 (0x3631 bytes) 2024-09-04T17:46:35.051 INFO:teuthology.orchestra.run.smithi092.stdout:1819 read 0x767d0ce thru 0x76801f8 (0x312b bytes) 2024-09-04T17:46:35.052 INFO:teuthology.orchestra.run.smithi092.stdout:1821 trunc from 0xdd0ec83 to 0xd9260e2 2024-09-04T17:46:35.052 INFO:teuthology.orchestra.run.smithi092.stdout:1822 read 0x439faee thru 0x43a8b02 (0x9015 bytes) 2024-09-04T17:46:35.052 INFO:teuthology.orchestra.run.smithi092.stdout:1823 writesame 0x1375923 thru 0x13838d4 data_size 0x26(0xdfb2 bytes) 2024-09-04T17:46:35.052 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:35.052 INFO:teuthology.orchestra.run.smithi092.stdout:1825 read 0xb2b2df9 thru 0xb2b4f5b (0x2163 bytes) 2024-09-04T17:46:35.052 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:35.052 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:35.059 INFO:teuthology.orchestra.run.smithi092.stdout:1829 trunc from 0xd9260e2 to 0x960938e 2024-09-04T17:46:35.074 INFO:teuthology.orchestra.run.smithi092.stdout:1830 read 0x184fe06 thru 0x18572e2 (0x74dd bytes) 2024-09-04T17:46:35.074 INFO:teuthology.orchestra.run.smithi092.stdout:1831 punch from 0x53d0519 to 0x53d4bd7, (0x46be bytes) 2024-09-04T17:46:35.074 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:35.074 INFO:teuthology.orchestra.run.smithi092.stdout:1833 clone 16 order 23 su 262144 sc 9 2024-09-04T17:46:35.150 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:46:35.146+0000 7f505d642640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26446) UID: 0 2024-09-04T17:46:35.220 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:35.221 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:46:35.221 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:46:35.221 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:35.234 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-04T17:46:35.277 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:35.274+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:35.319 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 4, 5, 6, 7, 3] 2024-09-04T17:46:35.319 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-04T17:46:35.320 INFO:tasks.thrashosds.thrasher:inject_pause on osd.7 2024-09-04T17:46:35.320 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-04T17:46:35.320 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-04T17:46:35.320 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok config set filestore_inject_stall 3 2024-09-04T17:46:35.351 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:35.352+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:35.397 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:35.397 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:46:35.397 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:46:35.397 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:46:35.397 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:35.413 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-04T17:46:35.449 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:35.449 INFO:teuthology.orchestra.run.smithi190.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-04T17:46:35.449 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:35.452 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:35.452+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:35.540 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:35.540 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:46:35.540 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:46:35.540 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:46:35.540 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.540 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:13127 3.1e 3:7bbed1a3:::rbd_data.11345bd5898b.0000000000000003:a [read 331776~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:35.540 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:17.905384+0000", 2024-09-04T17:46:35.540 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.634673538000001, 2024-09-04T17:46:35.540 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.059561699000000003, 2024-09-04T17:46:35.540 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:35.541 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:35.541 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:35.541 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:35.541 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:35.541 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:35.541 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 13127 2024-09-04T17:46:35.541 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.541 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:35.541 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.541 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:35.541 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905384+0000", 2024-09-04T17:46:35.541 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:35.541 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.542 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.542 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:35.542 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905384+0000", 2024-09-04T17:46:35.542 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:35.542 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.542 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.542 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:35.542 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905385+0000", 2024-09-04T17:46:35.542 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.144e-06 2024-09-04T17:46:35.542 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.542 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.542 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:35.542 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905390+0000", 2024-09-04T17:46:35.543 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0289999999999996e-06 2024-09-04T17:46:35.543 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.543 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.543 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:35.543 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905407+0000", 2024-09-04T17:46:35.543 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7322000000000001e-05 2024-09-04T17:46:35.543 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.543 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.543 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:35.543 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905421+0000", 2024-09-04T17:46:35.543 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3309000000000001e-05 2024-09-04T17:46:35.543 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.544 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.544 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:35.544 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.964510+0000", 2024-09-04T17:46:35.544 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.059089147000000002 2024-09-04T17:46:35.544 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.544 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.544 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:35.544 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.964684+0000", 2024-09-04T17:46:35.544 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000174463 2024-09-04T17:46:35.544 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.544 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.544 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:35.544 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.964946+0000", 2024-09-04T17:46:35.545 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00026128499999999999 2024-09-04T17:46:35.545 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:35.545 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:35.545 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:35.545 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.545 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.545 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:13128 3.1e 3:7bbed1a3:::rbd_data.11345bd5898b.0000000000000003:a [read 368640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:35.545 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:17.905438+0000", 2024-09-04T17:46:35.545 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.634619192999999, 2024-09-04T17:46:35.545 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.060341850000000002, 2024-09-04T17:46:35.545 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:35.545 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:35.546 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:35.546 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:35.546 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:35.546 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:35.546 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 13128 2024-09-04T17:46:35.546 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.546 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:35.546 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.546 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:35.546 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905438+0000", 2024-09-04T17:46:35.546 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:35.546 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.546 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.547 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:35.547 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905438+0000", 2024-09-04T17:46:35.547 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:35.547 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.547 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.547 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:35.547 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905439+0000", 2024-09-04T17:46:35.547 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1599999999999999e-06 2024-09-04T17:46:35.547 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.547 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.547 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:35.547 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905458+0000", 2024-09-04T17:46:35.547 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8961e-05 2024-09-04T17:46:35.548 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.548 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.548 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:35.548 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905476+0000", 2024-09-04T17:46:35.548 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7861999999999999e-05 2024-09-04T17:46:35.548 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.548 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.548 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:35.548 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905489+0000", 2024-09-04T17:46:35.548 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3232e-05 2024-09-04T17:46:35.548 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.548 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.548 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:35.549 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.965026+0000", 2024-09-04T17:46:35.549 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.059536981000000003 2024-09-04T17:46:35.549 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.549 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.549 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:35.549 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.965252+0000", 2024-09-04T17:46:35.549 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000225462 2024-09-04T17:46:35.549 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.549 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.549 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:35.549 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.965780+0000", 2024-09-04T17:46:35.549 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00052819200000000003 2024-09-04T17:46:35.550 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:35.550 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:35.550 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:35.550 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.550 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.550 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:13129 3.1e 3:7bbed1a3:::rbd_data.11345bd5898b.0000000000000003:a [read 405504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:35.550 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:17.905510+0000", 2024-09-04T17:46:35.550 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.634547803, 2024-09-04T17:46:35.550 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.060788440999999999, 2024-09-04T17:46:35.550 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:35.550 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:35.550 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:35.550 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:35.551 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:35.551 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:35.551 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 13129 2024-09-04T17:46:35.551 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.551 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:35.551 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.551 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:35.551 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905510+0000", 2024-09-04T17:46:35.551 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:35.551 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.551 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.551 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:35.552 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905510+0000", 2024-09-04T17:46:35.552 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:35.552 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.552 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.552 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:35.552 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905511+0000", 2024-09-04T17:46:35.552 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1659999999999999e-06 2024-09-04T17:46:35.552 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.552 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.552 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:35.552 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905516+0000", 2024-09-04T17:46:35.552 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9899999999999997e-06 2024-09-04T17:46:35.552 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.553 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.553 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:35.553 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905534+0000", 2024-09-04T17:46:35.553 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7808e-05 2024-09-04T17:46:35.553 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.553 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.553 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:35.553 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905547+0000", 2024-09-04T17:46:35.553 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3135999999999999e-05 2024-09-04T17:46:35.553 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.553 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.553 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:35.554 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.965898+0000", 2024-09-04T17:46:35.554 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.060351162 2024-09-04T17:46:35.554 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.554 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.554 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:35.554 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.966051+0000", 2024-09-04T17:46:35.554 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00015318299999999999 2024-09-04T17:46:35.554 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.554 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.554 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:35.554 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.966298+0000", 2024-09-04T17:46:35.554 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024699599999999998 2024-09-04T17:46:35.554 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:35.555 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:35.555 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:35.555 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.555 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.555 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:13130 3.1e 3:7bbed1a3:::rbd_data.11345bd5898b.0000000000000003:a [read 442368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:35.555 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:17.905565+0000", 2024-09-04T17:46:35.555 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.634492227999999, 2024-09-04T17:46:35.555 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.061174443000000002, 2024-09-04T17:46:35.555 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:35.555 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:35.555 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:35.555 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:35.555 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:35.555 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:35.556 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 13130 2024-09-04T17:46:35.556 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.556 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:35.556 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.556 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:35.556 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905565+0000", 2024-09-04T17:46:35.556 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:35.556 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.556 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.556 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:35.556 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905565+0000", 2024-09-04T17:46:35.556 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:35.556 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.556 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.556 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:35.557 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905567+0000", 2024-09-04T17:46:35.557 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6300000000000001e-06 2024-09-04T17:46:35.557 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.557 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.557 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:35.557 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905572+0000", 2024-09-04T17:46:35.557 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9790000000000001e-06 2024-09-04T17:46:35.557 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.557 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.557 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:35.557 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905590+0000", 2024-09-04T17:46:35.557 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8031999999999999e-05 2024-09-04T17:46:35.557 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.558 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.558 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:35.558 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905602+0000", 2024-09-04T17:46:35.558 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2651e-05 2024-09-04T17:46:35.558 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.558 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.558 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:35.558 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.966373+0000", 2024-09-04T17:46:35.558 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.060770532000000002 2024-09-04T17:46:35.558 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.558 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.558 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:35.558 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.966514+0000", 2024-09-04T17:46:35.558 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014103 2024-09-04T17:46:35.559 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.559 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.559 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:35.559 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.966740+0000", 2024-09-04T17:46:35.559 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000225589 2024-09-04T17:46:35.559 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:35.559 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:35.559 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:35.559 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.559 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.559 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:13131 3.1e 3:7bbed1a3:::rbd_data.11345bd5898b.0000000000000003:a [read 479232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:35.559 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:17.905621+0000", 2024-09-04T17:46:35.559 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.634436604000001, 2024-09-04T17:46:35.559 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.061498005000000001, 2024-09-04T17:46:35.560 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:35.560 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:35.560 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:35.560 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:35.560 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:35.560 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:35.560 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 13131 2024-09-04T17:46:35.560 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.560 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:35.560 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.560 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:35.560 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905621+0000", 2024-09-04T17:46:35.560 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:35.561 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.561 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.561 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:35.561 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905621+0000", 2024-09-04T17:46:35.561 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:35.561 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.561 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.561 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:35.561 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905622+0000", 2024-09-04T17:46:35.561 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.313e-06 2024-09-04T17:46:35.561 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.561 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.561 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:35.561 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905627+0000", 2024-09-04T17:46:35.562 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0100000000000003e-06 2024-09-04T17:46:35.562 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.562 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.562 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:35.562 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905644+0000", 2024-09-04T17:46:35.562 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6918e-05 2024-09-04T17:46:35.562 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.562 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.562 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:35.562 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905657+0000", 2024-09-04T17:46:35.562 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2496e-05 2024-09-04T17:46:35.562 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.562 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.562 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:35.563 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.966805+0000", 2024-09-04T17:46:35.563 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.061148518999999998 2024-09-04T17:46:35.563 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.563 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.563 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:35.563 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.966920+0000", 2024-09-04T17:46:35.563 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00011513700000000001 2024-09-04T17:46:35.563 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.563 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.563 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:35.563 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.967119+0000", 2024-09-04T17:46:35.563 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000198612 2024-09-04T17:46:35.563 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:35.563 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:35.564 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:35.564 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.564 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.564 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:13132 3.1e 3:7bbed1a3:::rbd_data.11345bd5898b.0000000000000003:a [read 516096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:35.564 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:17.905674+0000", 2024-09-04T17:46:35.564 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.634383038999999, 2024-09-04T17:46:35.564 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.061794957999999997, 2024-09-04T17:46:35.564 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:35.564 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:35.564 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:35.564 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:35.564 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:35.564 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:35.565 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 13132 2024-09-04T17:46:35.565 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.565 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:35.565 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.565 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:35.565 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905674+0000", 2024-09-04T17:46:35.565 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:35.565 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.565 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.565 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:35.565 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905674+0000", 2024-09-04T17:46:35.565 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:35.565 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.566 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.566 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:35.566 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905676+0000", 2024-09-04T17:46:35.566 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1230000000000001e-06 2024-09-04T17:46:35.566 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.566 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.566 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:35.566 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905680+0000", 2024-09-04T17:46:35.566 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9169999999999997e-06 2024-09-04T17:46:35.566 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.566 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.566 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:35.566 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905699+0000", 2024-09-04T17:46:35.566 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9023999999999999e-05 2024-09-04T17:46:35.567 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.567 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.567 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:35.567 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905713+0000", 2024-09-04T17:46:35.567 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3115999999999999e-05 2024-09-04T17:46:35.567 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.567 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.567 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:35.567 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.967167+0000", 2024-09-04T17:46:35.567 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.061454323999999998 2024-09-04T17:46:35.567 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.567 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.567 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:35.568 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.967280+0000", 2024-09-04T17:46:35.568 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000113269 2024-09-04T17:46:35.568 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.568 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.568 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:35.568 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.967469+0000", 2024-09-04T17:46:35.568 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000189185 2024-09-04T17:46:35.568 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:35.568 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:35.568 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:35.568 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.568 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.569 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:13133 3.1e 3:7bbed1a3:::rbd_data.11345bd5898b.0000000000000003:a [read 806912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:35.569 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:17.905745+0000", 2024-09-04T17:46:35.569 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.634312766000001, 2024-09-04T17:46:35.569 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.062050424999999999, 2024-09-04T17:46:35.569 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:35.569 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:35.569 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:35.569 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:35.570 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:35.570 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:35.570 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 13133 2024-09-04T17:46:35.571 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.571 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:35.571 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.571 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:35.571 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905745+0000", 2024-09-04T17:46:35.571 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:35.571 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.571 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.571 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:35.571 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905745+0000", 2024-09-04T17:46:35.571 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:35.571 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.571 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.571 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:35.572 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905746+0000", 2024-09-04T17:46:35.572 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0839999999999999e-06 2024-09-04T17:46:35.572 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.572 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.572 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:35.572 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905762+0000", 2024-09-04T17:46:35.572 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5917999999999999e-05 2024-09-04T17:46:35.572 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.572 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.572 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:35.572 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905780+0000", 2024-09-04T17:46:35.572 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7830999999999999e-05 2024-09-04T17:46:35.573 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.573 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.573 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:35.573 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905804+0000", 2024-09-04T17:46:35.573 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4912000000000001e-05 2024-09-04T17:46:35.573 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.573 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.573 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:35.573 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.967515+0000", 2024-09-04T17:46:35.573 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.061710458000000003 2024-09-04T17:46:35.573 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.573 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.573 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:35.574 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.967619+0000", 2024-09-04T17:46:35.574 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00010409899999999999 2024-09-04T17:46:35.574 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.574 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.574 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:35.574 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.967795+0000", 2024-09-04T17:46:35.574 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00017612300000000001 2024-09-04T17:46:35.574 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:35.574 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:35.574 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:35.574 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.574 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.574 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:13134 3.1e 3:7bbed1a3:::rbd_data.11345bd5898b.0000000000000003:a [read 843776~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:35.575 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:17.905824+0000", 2024-09-04T17:46:35.575 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.634233930000001, 2024-09-04T17:46:35.575 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.062450675999999997, 2024-09-04T17:46:35.575 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:35.575 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:35.575 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:35.575 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:35.575 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:35.575 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:35.575 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 13134 2024-09-04T17:46:35.575 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.575 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:35.575 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.575 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:35.575 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905824+0000", 2024-09-04T17:46:35.576 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:35.576 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.576 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.576 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:35.576 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905824+0000", 2024-09-04T17:46:35.576 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:35.576 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.576 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.576 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:35.576 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905825+0000", 2024-09-04T17:46:35.576 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.717e-06 2024-09-04T17:46:35.576 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.576 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.577 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:35.577 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905830+0000", 2024-09-04T17:46:35.577 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2220000000000001e-06 2024-09-04T17:46:35.577 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.577 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.577 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:35.577 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905848+0000", 2024-09-04T17:46:35.577 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7635e-05 2024-09-04T17:46:35.577 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.577 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.577 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:35.577 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905873+0000", 2024-09-04T17:46:35.577 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5160999999999998e-05 2024-09-04T17:46:35.578 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.578 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.578 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:35.578 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.967894+0000", 2024-09-04T17:46:35.578 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.062020947999999999 2024-09-04T17:46:35.578 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.578 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.578 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:35.578 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.968038+0000", 2024-09-04T17:46:35.578 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000143442 2024-09-04T17:46:35.578 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.578 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.578 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:35.578 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.968274+0000", 2024-09-04T17:46:35.579 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00023655099999999999 2024-09-04T17:46:35.579 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:35.579 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:35.579 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:35.579 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.579 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.579 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:13135 3.1e 3:7bbed1a3:::rbd_data.11345bd5898b.0000000000000003:a [read 880640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:35.579 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:17.905893+0000", 2024-09-04T17:46:35.579 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.634164513999998, 2024-09-04T17:46:35.579 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.062733087000000007, 2024-09-04T17:46:35.579 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:35.579 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:35.579 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:35.579 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:35.580 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:35.580 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:35.580 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 13135 2024-09-04T17:46:35.580 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.580 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:35.580 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.580 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:35.580 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905893+0000", 2024-09-04T17:46:35.580 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:35.580 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.580 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.580 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:35.580 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905893+0000", 2024-09-04T17:46:35.581 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:35.581 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.581 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.581 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:35.581 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905895+0000", 2024-09-04T17:46:35.581 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6139999999999999e-06 2024-09-04T17:46:35.581 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.581 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.581 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:35.581 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905900+0000", 2024-09-04T17:46:35.581 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2360000000000004e-06 2024-09-04T17:46:35.581 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.582 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.582 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:35.582 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905917+0000", 2024-09-04T17:46:35.582 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7642000000000002e-05 2024-09-04T17:46:35.582 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.582 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.582 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:35.582 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905930+0000", 2024-09-04T17:46:35.582 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2914999999999999e-05 2024-09-04T17:46:35.582 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.582 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.582 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:35.582 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.968339+0000", 2024-09-04T17:46:35.583 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.062408533000000002 2024-09-04T17:46:35.583 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.583 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.583 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:35.583 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.968446+0000", 2024-09-04T17:46:35.583 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000107481 2024-09-04T17:46:35.583 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.583 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.583 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:35.583 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.968626+0000", 2024-09-04T17:46:35.583 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00017966600000000001 2024-09-04T17:46:35.583 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:35.583 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:35.583 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:35.584 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.584 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.584 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:13136 3.1e 3:7bbed1a3:::rbd_data.11345bd5898b.0000000000000003:a [read 917504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:35.584 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:17.905948+0000", 2024-09-04T17:46:35.584 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.634109879, 2024-09-04T17:46:35.584 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.063001171999999994, 2024-09-04T17:46:35.584 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:35.584 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:35.584 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:35.584 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:35.584 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:35.584 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:35.585 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 13136 2024-09-04T17:46:35.585 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.585 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:35.585 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.585 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:35.585 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905948+0000", 2024-09-04T17:46:35.585 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:35.585 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.585 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.585 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:35.585 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905948+0000", 2024-09-04T17:46:35.585 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:35.585 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.586 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.586 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:35.586 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905949+0000", 2024-09-04T17:46:35.586 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.364e-06 2024-09-04T17:46:35.586 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.586 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.586 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:35.586 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905954+0000", 2024-09-04T17:46:35.586 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.8720000000000001e-06 2024-09-04T17:46:35.586 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.586 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.586 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:35.586 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905972+0000", 2024-09-04T17:46:35.586 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7756000000000001e-05 2024-09-04T17:46:35.587 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.587 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.587 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:35.587 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905984+0000", 2024-09-04T17:46:35.587 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2581e-05 2024-09-04T17:46:35.587 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.587 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.587 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:35.587 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.968668+0000", 2024-09-04T17:46:35.587 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.062683991999999994 2024-09-04T17:46:35.587 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.587 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.587 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:35.588 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.968773+0000", 2024-09-04T17:46:35.588 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000105027 2024-09-04T17:46:35.588 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.588 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.588 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:35.588 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.968949+0000", 2024-09-04T17:46:35.588 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00017558 2024-09-04T17:46:35.588 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:35.588 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:35.588 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:35.588 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.588 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.588 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:13137 3.1e 3:7bbed1a3:::rbd_data.11345bd5898b.0000000000000003:a [read 954368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:35.589 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:17.906005+0000", 2024-09-04T17:46:35.589 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.634052614000002, 2024-09-04T17:46:35.589 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.063260424999999995, 2024-09-04T17:46:35.589 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:35.589 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:35.589 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:35.589 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:35.589 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:35.589 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:35.589 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 13137 2024-09-04T17:46:35.589 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.589 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:35.589 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.590 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:35.590 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906005+0000", 2024-09-04T17:46:35.590 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:35.590 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.590 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.590 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:35.590 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906005+0000", 2024-09-04T17:46:35.590 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:35.590 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.590 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.590 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:35.590 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906006+0000", 2024-09-04T17:46:35.591 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.116e-06 2024-09-04T17:46:35.591 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.591 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.591 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:35.591 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906011+0000", 2024-09-04T17:46:35.591 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9429999999999999e-06 2024-09-04T17:46:35.591 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.591 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.591 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:35.591 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906028+0000", 2024-09-04T17:46:35.591 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6968000000000001e-05 2024-09-04T17:46:35.591 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.591 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.591 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:35.592 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906041+0000", 2024-09-04T17:46:35.592 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3269e-05 2024-09-04T17:46:35.592 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.592 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.592 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:35.592 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.968991+0000", 2024-09-04T17:46:35.592 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.062949450000000004 2024-09-04T17:46:35.592 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.592 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.592 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:35.592 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.969093+0000", 2024-09-04T17:46:35.592 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000102521 2024-09-04T17:46:35.592 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.593 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.593 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:35.593 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.969265+0000", 2024-09-04T17:46:35.593 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000172158 2024-09-04T17:46:35.593 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:35.593 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:35.593 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:35.593 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.593 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.593 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:13138 3.1e 3:7bbed1a3:::rbd_data.11345bd5898b.0000000000000003:a [read 991232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:35.593 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:17.906058+0000", 2024-09-04T17:46:35.593 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.633998943000002, 2024-09-04T17:46:35.593 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.063688720000000004, 2024-09-04T17:46:35.593 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:35.594 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:35.594 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:35.594 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:35.594 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:35.594 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:35.594 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 13138 2024-09-04T17:46:35.594 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.594 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:35.594 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.594 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:35.594 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906058+0000", 2024-09-04T17:46:35.594 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:35.594 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.594 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.595 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:35.595 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906058+0000", 2024-09-04T17:46:35.595 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:35.595 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.595 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.595 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:35.595 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906060+0000", 2024-09-04T17:46:35.595 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2100000000000001e-06 2024-09-04T17:46:35.595 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.595 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.595 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:35.595 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906065+0000", 2024-09-04T17:46:35.595 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.8670000000000002e-06 2024-09-04T17:46:35.595 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.596 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.596 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:35.596 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906082+0000", 2024-09-04T17:46:35.596 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7283999999999999e-05 2024-09-04T17:46:35.596 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.596 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.596 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:35.596 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906095+0000", 2024-09-04T17:46:35.596 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2763e-05 2024-09-04T17:46:35.596 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.596 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.596 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:35.596 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.969365+0000", 2024-09-04T17:46:35.596 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.063269912999999997 2024-09-04T17:46:35.597 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.597 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.597 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:35.597 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.969514+0000", 2024-09-04T17:46:35.597 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014945599999999999 2024-09-04T17:46:35.597 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.597 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.597 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:35.597 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.969747+0000", 2024-09-04T17:46:35.597 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00023322699999999999 2024-09-04T17:46:35.597 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:35.597 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:35.597 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:35.598 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.598 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.598 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:13139 3.1e 3:7bbed1a3:::rbd_data.11345bd5898b.0000000000000003:a [read 1028096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:35.598 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:17.906111+0000", 2024-09-04T17:46:35.598 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.633946258000002, 2024-09-04T17:46:35.598 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.063986543000000007, 2024-09-04T17:46:35.598 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:35.598 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:35.598 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:35.598 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:35.598 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:35.598 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:35.598 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 13139 2024-09-04T17:46:35.598 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.599 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:35.599 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.599 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:35.599 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906111+0000", 2024-09-04T17:46:35.599 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:35.599 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.599 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.599 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:35.599 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906111+0000", 2024-09-04T17:46:35.599 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:35.599 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.599 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.599 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:35.599 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906112+0000", 2024-09-04T17:46:35.600 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1370000000000001e-06 2024-09-04T17:46:35.600 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.600 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.600 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:35.600 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906118+0000", 2024-09-04T17:46:35.600 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.3460000000000001e-06 2024-09-04T17:46:35.600 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.600 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.600 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:35.600 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906135+0000", 2024-09-04T17:46:35.600 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6895e-05 2024-09-04T17:46:35.600 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.600 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.601 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:35.601 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906147+0000", 2024-09-04T17:46:35.601 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2578999999999999e-05 2024-09-04T17:46:35.601 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.601 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.601 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:35.601 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.969812+0000", 2024-09-04T17:46:35.601 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.063664539000000006 2024-09-04T17:46:35.601 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.601 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.601 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:35.601 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.969918+0000", 2024-09-04T17:46:35.601 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000106827 2024-09-04T17:46:35.602 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.602 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.602 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:35.602 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.970098+0000", 2024-09-04T17:46:35.602 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00017922000000000001 2024-09-04T17:46:35.602 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:35.602 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:35.602 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:35.602 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.602 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.602 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:13140 3.1e 3:7bbed1a3:::rbd_data.11345bd5898b.0000000000000003:a [read 1318912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:35.602 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:17.906165+0000", 2024-09-04T17:46:35.602 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.633892555999999, 2024-09-04T17:46:35.603 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.064276715999999998, 2024-09-04T17:46:35.603 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:35.603 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:35.603 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:35.603 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:35.603 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:35.603 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:35.603 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 13140 2024-09-04T17:46:35.603 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.603 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:35.603 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.603 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:35.603 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906165+0000", 2024-09-04T17:46:35.604 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:35.604 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.604 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.604 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:35.604 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906165+0000", 2024-09-04T17:46:35.604 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:35.604 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.604 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.604 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:35.604 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906166+0000", 2024-09-04T17:46:35.604 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.11e-06 2024-09-04T17:46:35.604 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.605 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.605 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:35.605 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906181+0000", 2024-09-04T17:46:35.605 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4918e-05 2024-09-04T17:46:35.605 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.605 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.605 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:35.605 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906198+0000", 2024-09-04T17:46:35.605 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7125e-05 2024-09-04T17:46:35.605 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.605 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.605 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:35.605 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906211+0000", 2024-09-04T17:46:35.606 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3169e-05 2024-09-04T17:46:35.606 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.606 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.606 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:35.606 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.970142+0000", 2024-09-04T17:46:35.606 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.063930511999999995 2024-09-04T17:46:35.606 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.606 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.606 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:35.606 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.970245+0000", 2024-09-04T17:46:35.606 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00010333800000000001 2024-09-04T17:46:35.606 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.606 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.606 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:35.607 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.970442+0000", 2024-09-04T17:46:35.607 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019654399999999999 2024-09-04T17:46:35.607 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:35.607 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:35.607 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:35.607 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.607 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.607 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:13141 3.1e 3:7bbed1a3:::rbd_data.11345bd5898b.0000000000000003:a [read 1355776~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:35.607 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:17.906252+0000", 2024-09-04T17:46:35.607 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.633805777999999, 2024-09-04T17:46:35.607 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.064674651, 2024-09-04T17:46:35.607 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:35.607 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:35.608 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:35.608 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:35.608 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:35.608 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:35.608 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 13141 2024-09-04T17:46:35.608 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.608 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:35.608 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.608 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:35.608 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906252+0000", 2024-09-04T17:46:35.608 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:35.608 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.608 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.608 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:35.609 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906252+0000", 2024-09-04T17:46:35.609 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:35.609 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.609 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.609 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:35.609 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906253+0000", 2024-09-04T17:46:35.609 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5600000000000001e-06 2024-09-04T17:46:35.609 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.609 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.609 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:35.609 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906259+0000", 2024-09-04T17:46:35.609 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7490000000000004e-06 2024-09-04T17:46:35.610 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.610 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.610 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:35.610 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906293+0000", 2024-09-04T17:46:35.610 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4204e-05 2024-09-04T17:46:35.610 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.610 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.610 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:35.610 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906307+0000", 2024-09-04T17:46:35.610 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3801000000000001e-05 2024-09-04T17:46:35.610 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.610 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.610 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:35.611 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.970535+0000", 2024-09-04T17:46:35.611 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.064228104999999994 2024-09-04T17:46:35.611 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.611 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.611 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:35.611 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.970692+0000", 2024-09-04T17:46:35.611 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00015668499999999999 2024-09-04T17:46:35.611 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.611 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.611 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:35.611 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.970926+0000", 2024-09-04T17:46:35.611 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00023454699999999999 2024-09-04T17:46:35.611 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:35.611 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:35.612 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:35.612 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.612 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.612 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:13142 3.1e 3:7bbed1a3:::rbd_data.11345bd5898b.0000000000000003:a [read 1392640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:35.612 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:17.906327+0000", 2024-09-04T17:46:35.612 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.633730022000002, 2024-09-04T17:46:35.612 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.064950401000000005, 2024-09-04T17:46:35.612 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:35.612 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:35.612 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:35.612 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:35.612 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:35.612 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:35.613 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 13142 2024-09-04T17:46:35.613 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.613 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:35.613 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.613 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:35.613 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906327+0000", 2024-09-04T17:46:35.613 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:35.613 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.613 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.613 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:35.613 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906327+0000", 2024-09-04T17:46:35.613 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:35.613 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.614 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.614 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:35.614 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906329+0000", 2024-09-04T17:46:35.614 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.33e-06 2024-09-04T17:46:35.614 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.614 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.614 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:35.614 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906334+0000", 2024-09-04T17:46:35.614 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4160000000000003e-06 2024-09-04T17:46:35.614 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.614 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.614 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:35.614 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906351+0000", 2024-09-04T17:46:35.614 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7328000000000001e-05 2024-09-04T17:46:35.615 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.615 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.615 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:35.615 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906379+0000", 2024-09-04T17:46:35.615 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.7923999999999999e-05 2024-09-04T17:46:35.615 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.615 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.615 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:35.615 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.970992+0000", 2024-09-04T17:46:35.615 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.064612554000000003 2024-09-04T17:46:35.615 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.615 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.615 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:35.615 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.971097+0000", 2024-09-04T17:46:35.616 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000105438 2024-09-04T17:46:35.616 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.616 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.616 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:35.616 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.971278+0000", 2024-09-04T17:46:35.616 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000180411 2024-09-04T17:46:35.616 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:35.616 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:35.616 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:35.616 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.616 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.616 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:13143 3.1e 3:7bbed1a3:::rbd_data.11345bd5898b.0000000000000003:a [read 1429504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:35.616 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:17.906399+0000", 2024-09-04T17:46:35.617 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.633658902000001, 2024-09-04T17:46:35.617 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.065281594999999998, 2024-09-04T17:46:35.617 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:35.617 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:35.617 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:35.617 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:35.617 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:35.617 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:35.617 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 13143 2024-09-04T17:46:35.617 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.617 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:35.617 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.617 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:35.618 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906399+0000", 2024-09-04T17:46:35.618 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:35.618 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.618 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.618 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:35.618 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906399+0000", 2024-09-04T17:46:35.618 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:35.618 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.618 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.618 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:35.618 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906400+0000", 2024-09-04T17:46:35.618 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.139e-06 2024-09-04T17:46:35.618 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.618 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.618 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:35.619 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906405+0000", 2024-09-04T17:46:35.619 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2299999999999999e-06 2024-09-04T17:46:35.619 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.619 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.619 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:35.619 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906422+0000", 2024-09-04T17:46:35.619 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6884999999999998e-05 2024-09-04T17:46:35.619 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.619 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.619 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:35.619 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906435+0000", 2024-09-04T17:46:35.619 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3339000000000001e-05 2024-09-04T17:46:35.619 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.619 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.620 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:35.620 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.971343+0000", 2024-09-04T17:46:35.620 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.064907555000000006 2024-09-04T17:46:35.620 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.620 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.620 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:35.620 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.971474+0000", 2024-09-04T17:46:35.620 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013157699999999999 2024-09-04T17:46:35.620 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.620 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.620 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:35.620 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.971680+0000", 2024-09-04T17:46:35.620 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020587000000000001 2024-09-04T17:46:35.621 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:35.621 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:35.621 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:35.621 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.621 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.621 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:13144 3.1e 3:7bbed1a3:::rbd_data.11345bd5898b.0000000000000003:a [read 1466368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:35.621 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:17.906452+0000", 2024-09-04T17:46:35.621 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.633605151000001, 2024-09-04T17:46:35.621 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.065571372000000003, 2024-09-04T17:46:35.621 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:35.621 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:35.621 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:35.621 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:35.622 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:35.622 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:35.622 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 13144 2024-09-04T17:46:35.622 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.622 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:35.622 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.622 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:35.622 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906452+0000", 2024-09-04T17:46:35.622 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:35.622 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.622 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.622 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:35.622 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906452+0000", 2024-09-04T17:46:35.622 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:35.623 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.623 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.623 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:35.623 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906453+0000", 2024-09-04T17:46:35.623 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.083e-06 2024-09-04T17:46:35.623 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.623 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.623 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:35.623 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906487+0000", 2024-09-04T17:46:35.623 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4084000000000001e-05 2024-09-04T17:46:35.623 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.623 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.623 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:35.624 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906505+0000", 2024-09-04T17:46:35.624 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7971000000000001e-05 2024-09-04T17:46:35.624 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.624 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.624 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:35.624 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906519+0000", 2024-09-04T17:46:35.624 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3526000000000001e-05 2024-09-04T17:46:35.624 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.624 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.624 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:35.624 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.971741+0000", 2024-09-04T17:46:35.624 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.065221836000000005 2024-09-04T17:46:35.624 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.625 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.625 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:35.625 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.971846+0000", 2024-09-04T17:46:35.625 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000105278 2024-09-04T17:46:35.625 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.625 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.625 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:35.625 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.972024+0000", 2024-09-04T17:46:35.625 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00017759399999999999 2024-09-04T17:46:35.625 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:35.625 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:35.625 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:35.625 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.625 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.626 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:13145 3.1e 3:7bbed1a3:::rbd_data.11345bd5898b.0000000000000003:a [read 1503232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:35.626 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:17.906538+0000", 2024-09-04T17:46:35.626 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.633519808999999, 2024-09-04T17:46:35.626 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.065800844999999997, 2024-09-04T17:46:35.626 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:35.626 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:35.626 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:35.626 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:35.626 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:35.626 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:35.626 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 13145 2024-09-04T17:46:35.626 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.626 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:35.627 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.627 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:35.627 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906538+0000", 2024-09-04T17:46:35.627 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:35.627 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.627 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.627 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:35.627 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906538+0000", 2024-09-04T17:46:35.627 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:35.627 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.627 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.627 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:35.627 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906539+0000", 2024-09-04T17:46:35.628 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6470000000000001e-06 2024-09-04T17:46:35.628 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.628 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.628 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:35.628 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906545+0000", 2024-09-04T17:46:35.628 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2390000000000002e-06 2024-09-04T17:46:35.628 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.628 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.628 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:35.628 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906562+0000", 2024-09-04T17:46:35.628 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7102e-05 2024-09-04T17:46:35.628 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.628 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.628 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:35.629 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906576+0000", 2024-09-04T17:46:35.629 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4758e-05 2024-09-04T17:46:35.629 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.629 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.629 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:35.629 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.972066+0000", 2024-09-04T17:46:35.629 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.065489503000000004 2024-09-04T17:46:35.629 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.629 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.629 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:35.629 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.972167+0000", 2024-09-04T17:46:35.629 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00010100200000000001 2024-09-04T17:46:35.629 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.629 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.630 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:35.630 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.972338+0000", 2024-09-04T17:46:35.630 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00017159400000000001 2024-09-04T17:46:35.630 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:35.630 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:35.630 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:35.630 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.630 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.630 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:13146 3.1e 3:7bbed1a3:::rbd_data.11345bd5898b.0000000000000003:a [read 1540096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:35.630 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:17.906595+0000", 2024-09-04T17:46:35.630 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.633462258000002, 2024-09-04T17:46:35.630 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.066220854999999995, 2024-09-04T17:46:35.630 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:35.631 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:35.631 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:35.631 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:35.631 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:35.631 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:35.631 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 13146 2024-09-04T17:46:35.631 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.631 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:35.631 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.631 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:35.631 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906595+0000", 2024-09-04T17:46:35.631 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:35.631 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.631 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.632 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:35.632 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906595+0000", 2024-09-04T17:46:35.632 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:35.632 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.632 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.632 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:35.632 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906597+0000", 2024-09-04T17:46:35.632 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6330000000000001e-06 2024-09-04T17:46:35.632 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.632 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.632 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:35.632 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906602+0000", 2024-09-04T17:46:35.632 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9529999999999998e-06 2024-09-04T17:46:35.633 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.633 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.633 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:35.633 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906619+0000", 2024-09-04T17:46:35.633 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7561999999999999e-05 2024-09-04T17:46:35.633 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.633 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.633 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:35.633 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906632+0000", 2024-09-04T17:46:35.633 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2901999999999999e-05 2024-09-04T17:46:35.633 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.633 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.633 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:35.634 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.972443+0000", 2024-09-04T17:46:35.634 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.065810653999999996 2024-09-04T17:46:35.634 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.634 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.634 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:35.634 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.972586+0000", 2024-09-04T17:46:35.634 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014348599999999999 2024-09-04T17:46:35.634 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:35.634 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:35.634 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:35.634 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.972816+0000", 2024-09-04T17:46:35.634 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00022966499999999999 2024-09-04T17:46:35.634 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:35.634 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:35.634 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:35.635 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:35.635 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:35.635 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:35.635 INFO:teuthology.orchestra.run.smithi092.stdout:leaving image image_client.0-clone15 intact 2024-09-04T17:46:35.636 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-04T17:46:35.637 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:35.636+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:35.737 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:46:35.736+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:46:35.767 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:35.768 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:46:35.768 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:46:35.768 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:35.783 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-04T17:46:35.839 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:35.834+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:35.844 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #14, image image_client.0-clone14 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent15 2024-09-04T17:46:35.920 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:35.920 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:46:35.920 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:46:35.920 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:46:35.920 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:35.934 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-04T17:46:35.935 INFO:teuthology.orchestra.run.smithi092.stdout:1834 writesame 0xf36b32 thru 0xf45514 data_size 0x19(0xe9e3 bytes) 2024-09-04T17:46:35.975 INFO:teuthology.orchestra.run.smithi092.stdout:1836 write 0x9d9617b thru 0x9d9a483 (0x4309 bytes) 2024-09-04T17:46:35.976 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:46:35.974+0000 7f505d642640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26446) UID: 0 2024-09-04T17:46:35.981 INFO:teuthology.orchestra.run.smithi092.stdout:1837 punch from 0x83c4ba6 to 0x83c5aff, (0xf59 bytes) 2024-09-04T17:46:35.981 INFO:teuthology.orchestra.run.smithi092.stdout:1838 punch from 0x71bd45 to 0x7291cb, (0xd486 bytes) 2024-09-04T17:46:35.985 INFO:teuthology.orchestra.run.smithi092.stdout:1839 trunc from 0x9d9a484 to 0xbf09441 2024-09-04T17:46:35.993 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:35.993 INFO:teuthology.orchestra.run.smithi092.stdout:1841 writesame 0x755fe23 thru 0x756ebff data_size 0x1(0xeddd bytes) 2024-09-04T17:46:36.040 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:36.038+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:36.043 INFO:teuthology.orchestra.run.smithi092.stdout:1842 read 0xb76fb40 thru 0xb773ce8 (0x41a9 bytes) 2024-09-04T17:46:36.043 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:36.043 INFO:teuthology.orchestra.run.smithi092.stdout:1844 read 0x27a8d3b thru 0x27abaa2 (0x2d68 bytes) 2024-09-04T17:46:36.057 INFO:teuthology.orchestra.run.smithi092.stdout:1845 write 0xe9f6d69 thru 0xea0536e (0xe606 bytes) 2024-09-04T17:46:36.066 INFO:teuthology.orchestra.run.smithi092.stdout:1846 write 0xc6eb2b2 thru 0xc6f60a4 (0xadf3 bytes) 2024-09-04T17:46:36.073 INFO:teuthology.orchestra.run.smithi092.stdout:1847 write 0xb02df8f thru 0xb032522 (0x4594 bytes) 2024-09-04T17:46:36.081 INFO:teuthology.orchestra.run.smithi092.stdout:1850 write 0x97dab3a thru 0x97e1748 (0x6c0f bytes) 2024-09-04T17:46:36.088 INFO:teuthology.orchestra.run.smithi092.stdout:1851 read 0x7afeb9f thru 0x7b080c3 (0x9525 bytes) 2024-09-04T17:46:36.088 INFO:teuthology.orchestra.run.smithi092.stdout:1852 write 0xd33700 thru 0xd41885 (0xe186 bytes) 2024-09-04T17:46:36.110 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:36.110 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:46:36.110 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:46:36.110 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:46:36.110 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.110 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:1430 3.f 3:f58475cb:::rbd_data.11346251f094.0000000000000074:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-04T17:46:36.110 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:02.778704+0000", 2024-09-04T17:46:36.110 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 33.329085685000003, 2024-09-04T17:46:36.110 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.058567801000000003, 2024-09-04T17:46:36.110 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:36.110 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:36.110 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:36.111 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:36.111 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:36.111 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:36.111 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 1430 2024-09-04T17:46:36.111 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.111 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:36.111 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.111 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:36.111 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.778704+0000", 2024-09-04T17:46:36.111 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:36.111 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.111 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.111 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:36.111 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.778704+0000", 2024-09-04T17:46:36.112 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:36.112 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.112 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.112 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:36.112 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.778707+0000", 2024-09-04T17:46:36.112 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.8329999999999998e-06 2024-09-04T17:46:36.112 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.112 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.112 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:36.112 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.778718+0000", 2024-09-04T17:46:36.112 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.152e-05 2024-09-04T17:46:36.112 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.112 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.113 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:36.113 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.778750+0000", 2024-09-04T17:46:36.113 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.1696999999999998e-05 2024-09-04T17:46:36.113 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.113 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.113 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:36.113 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.778765+0000", 2024-09-04T17:46:36.113 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4800000000000001e-05 2024-09-04T17:46:36.113 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.113 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.113 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:36.113 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.778894+0000", 2024-09-04T17:46:36.114 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00012928500000000001 2024-09-04T17:46:36.114 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.114 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.114 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:36.114 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.779030+0000", 2024-09-04T17:46:36.114 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013566899999999999 2024-09-04T17:46:36.114 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.114 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.114 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 4", 2024-09-04T17:46:36.114 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.788794+0000", 2024-09-04T17:46:36.114 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.009764478 2024-09-04T17:46:36.114 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.114 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.114 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:36.115 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.836969+0000", 2024-09-04T17:46:36.115 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.048174626999999998 2024-09-04T17:46:36.115 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.115 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.115 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:36.115 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.837090+0000", 2024-09-04T17:46:36.115 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000120627 2024-09-04T17:46:36.115 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.115 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.115 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:36.115 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.837195+0000", 2024-09-04T17:46:36.115 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000105206 2024-09-04T17:46:36.116 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.116 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.116 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:36.116 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.837272+0000", 2024-09-04T17:46:36.116 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.7058999999999995e-05 2024-09-04T17:46:36.116 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:36.116 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:36.116 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:36.116 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.116 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.117 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14369 3.d 3:b763de10:::rbd_data.11345bd5898b.0000000000000002:a [read 77824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:36.117 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.182808+0000", 2024-09-04T17:46:36.117 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.924981717000001, 2024-09-04T17:46:36.117 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042057997999999999, 2024-09-04T17:46:36.117 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:36.117 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:36.117 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:36.117 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:36.117 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:36.117 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:36.117 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14369 2024-09-04T17:46:36.117 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.117 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:36.118 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.118 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:36.118 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182808+0000", 2024-09-04T17:46:36.118 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:36.118 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.118 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.118 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:36.118 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182808+0000", 2024-09-04T17:46:36.118 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:36.118 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.118 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.118 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:36.118 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182809+0000", 2024-09-04T17:46:36.119 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.063e-06 2024-09-04T17:46:36.119 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.119 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.119 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:36.119 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182814+0000", 2024-09-04T17:46:36.119 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1900000000000003e-06 2024-09-04T17:46:36.119 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.119 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.119 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:36.119 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182832+0000", 2024-09-04T17:46:36.119 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7791999999999998e-05 2024-09-04T17:46:36.119 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.119 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.120 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:36.120 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182849+0000", 2024-09-04T17:46:36.120 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7265000000000002e-05 2024-09-04T17:46:36.120 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.120 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.120 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:36.120 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.224431+0000", 2024-09-04T17:46:36.120 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.041581448 2024-09-04T17:46:36.120 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.120 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.120 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:36.120 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.224615+0000", 2024-09-04T17:46:36.120 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00018389200000000001 2024-09-04T17:46:36.120 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.121 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.121 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:36.121 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.224866+0000", 2024-09-04T17:46:36.121 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00025134799999999998 2024-09-04T17:46:36.121 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:36.121 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:36.121 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:36.121 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.121 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.121 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14370 3.d 3:b763de10:::rbd_data.11345bd5898b.0000000000000002:a [read 114688~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:36.121 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.182867+0000", 2024-09-04T17:46:36.121 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.924922415000001, 2024-09-04T17:46:36.121 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042368301999999997, 2024-09-04T17:46:36.122 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:36.122 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:36.122 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:36.122 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:36.122 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:36.122 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:36.122 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14370 2024-09-04T17:46:36.122 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.122 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:36.122 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.122 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:36.122 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182867+0000", 2024-09-04T17:46:36.122 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:36.122 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.123 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.123 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:36.123 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182867+0000", 2024-09-04T17:46:36.123 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:36.123 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.123 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.123 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:36.123 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182868+0000", 2024-09-04T17:46:36.123 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.006e-06 2024-09-04T17:46:36.123 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.123 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.123 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:36.123 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182875+0000", 2024-09-04T17:46:36.124 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.3990000000000001e-06 2024-09-04T17:46:36.124 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.124 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.124 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:36.124 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182892+0000", 2024-09-04T17:46:36.124 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7096e-05 2024-09-04T17:46:36.124 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.124 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.124 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:36.124 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182905+0000", 2024-09-04T17:46:36.124 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3258000000000001e-05 2024-09-04T17:46:36.124 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.124 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.124 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:36.125 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.224926+0000", 2024-09-04T17:46:36.125 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042020891999999997 2024-09-04T17:46:36.125 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.125 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.125 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:36.125 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.225064+0000", 2024-09-04T17:46:36.125 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000138307 2024-09-04T17:46:36.125 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.125 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.125 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:36.125 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.225236+0000", 2024-09-04T17:46:36.125 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000171344 2024-09-04T17:46:36.126 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:36.126 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:36.126 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:36.126 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.126 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.126 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14375 3.d 3:b763de10:::rbd_data.11345bd5898b.0000000000000002:a [read 188416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:36.126 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.183941+0000", 2024-09-04T17:46:36.126 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.923848355000001, 2024-09-04T17:46:36.126 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042015839999999999, 2024-09-04T17:46:36.126 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:36.126 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:36.126 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:36.126 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:36.127 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:36.127 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:36.127 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14375 2024-09-04T17:46:36.127 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.127 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:36.127 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.127 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:36.127 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.183941+0000", 2024-09-04T17:46:36.127 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:36.127 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.127 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.127 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:36.127 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.183941+0000", 2024-09-04T17:46:36.128 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:36.128 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.128 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.128 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:36.128 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.183943+0000", 2024-09-04T17:46:36.128 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7099999999999999e-06 2024-09-04T17:46:36.128 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.128 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.128 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:36.128 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.183964+0000", 2024-09-04T17:46:36.128 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0483999999999999e-05 2024-09-04T17:46:36.128 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.128 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.129 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:36.129 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.183983+0000", 2024-09-04T17:46:36.129 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9918000000000001e-05 2024-09-04T17:46:36.129 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.129 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.129 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:36.129 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.183998+0000", 2024-09-04T17:46:36.129 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4683e-05 2024-09-04T17:46:36.129 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.129 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.129 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:36.129 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.225640+0000", 2024-09-04T17:46:36.129 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.041641425000000003 2024-09-04T17:46:36.130 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.130 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.130 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:36.130 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.225777+0000", 2024-09-04T17:46:36.130 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013778999999999999 2024-09-04T17:46:36.130 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.130 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.130 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:36.130 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.225957+0000", 2024-09-04T17:46:36.130 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00017982999999999999 2024-09-04T17:46:36.130 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:36.130 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:36.130 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:36.130 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.131 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.131 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14377 3.d 3:b763de10:::rbd_data.11345bd5898b.0000000000000002:a [read 225280~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:36.131 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.184021+0000", 2024-09-04T17:46:36.131 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.923768808999998, 2024-09-04T17:46:36.131 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042298810999999999, 2024-09-04T17:46:36.131 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:36.131 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:36.131 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:36.131 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:36.131 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:36.131 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:36.131 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14377 2024-09-04T17:46:36.131 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.132 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:36.132 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.132 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:36.132 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184021+0000", 2024-09-04T17:46:36.132 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:36.132 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.132 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.132 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:36.132 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184021+0000", 2024-09-04T17:46:36.132 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:36.132 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.132 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.132 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:36.133 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184022+0000", 2024-09-04T17:46:36.133 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5099999999999999e-06 2024-09-04T17:46:36.133 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.133 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.133 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:36.133 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184028+0000", 2024-09-04T17:46:36.133 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.852e-06 2024-09-04T17:46:36.133 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.133 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.133 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:36.133 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184046+0000", 2024-09-04T17:46:36.133 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7822e-05 2024-09-04T17:46:36.133 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.134 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.134 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:36.134 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184059+0000", 2024-09-04T17:46:36.134 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3372e-05 2024-09-04T17:46:36.134 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.134 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.134 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:36.134 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.226009+0000", 2024-09-04T17:46:36.134 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.041949187999999998 2024-09-04T17:46:36.134 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.134 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.134 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:36.134 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.226142+0000", 2024-09-04T17:46:36.134 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013381299999999999 2024-09-04T17:46:36.135 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.135 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.135 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:36.135 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.226320+0000", 2024-09-04T17:46:36.135 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00017725400000000001 2024-09-04T17:46:36.135 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:36.135 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:36.135 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:36.135 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.135 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.135 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14381 3.d 3:b763de10:::rbd_data.11345bd5898b.0000000000000002:a [read 552960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:36.135 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.184135+0000", 2024-09-04T17:46:36.135 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.923654757000001, 2024-09-04T17:46:36.136 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042553818, 2024-09-04T17:46:36.136 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:36.136 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:36.136 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:36.136 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:36.136 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:36.136 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:36.136 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14381 2024-09-04T17:46:36.136 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.136 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:36.136 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.136 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:36.136 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184135+0000", 2024-09-04T17:46:36.137 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:36.137 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.137 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.137 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:36.137 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184135+0000", 2024-09-04T17:46:36.137 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:36.137 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.137 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.137 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:36.138 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184136+0000", 2024-09-04T17:46:36.138 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1200000000000001e-06 2024-09-04T17:46:36.138 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.138 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.138 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:36.138 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184142+0000", 2024-09-04T17:46:36.138 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4840000000000003e-06 2024-09-04T17:46:36.138 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.138 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.138 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:36.138 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184157+0000", 2024-09-04T17:46:36.138 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5727999999999999e-05 2024-09-04T17:46:36.138 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.138 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.139 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:36.139 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184194+0000", 2024-09-04T17:46:36.139 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7049999999999999e-05 2024-09-04T17:46:36.139 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.139 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.139 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:36.139 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.226376+0000", 2024-09-04T17:46:36.139 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042181933999999997 2024-09-04T17:46:36.139 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.139 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.139 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:36.139 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.226522+0000", 2024-09-04T17:46:36.139 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014602899999999999 2024-09-04T17:46:36.139 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.140 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.140 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:36.140 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.226689+0000", 2024-09-04T17:46:36.140 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00016647299999999999 2024-09-04T17:46:36.140 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:36.140 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:36.140 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:36.140 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.140 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.140 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14383 3.d 3:b763de10:::rbd_data.11345bd5898b.0000000000000002:a [read 589824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:36.140 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.184302+0000", 2024-09-04T17:46:36.140 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.923487758, 2024-09-04T17:46:36.141 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042760362000000003, 2024-09-04T17:46:36.141 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:36.141 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:36.141 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:36.141 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:36.141 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:36.141 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:36.141 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14383 2024-09-04T17:46:36.141 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.141 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:36.141 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.141 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:36.141 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184302+0000", 2024-09-04T17:46:36.142 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:36.142 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.142 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.142 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:36.142 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184302+0000", 2024-09-04T17:46:36.142 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:36.142 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.142 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.142 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:36.142 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184303+0000", 2024-09-04T17:46:36.142 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.206e-06 2024-09-04T17:46:36.142 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.142 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.143 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:36.143 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184340+0000", 2024-09-04T17:46:36.143 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7005999999999998e-05 2024-09-04T17:46:36.143 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.143 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.143 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:36.143 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184382+0000", 2024-09-04T17:46:36.143 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2345999999999997e-05 2024-09-04T17:46:36.143 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.143 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.143 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:36.143 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184397+0000", 2024-09-04T17:46:36.143 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4729000000000001e-05 2024-09-04T17:46:36.144 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.144 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.144 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:36.144 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.226749+0000", 2024-09-04T17:46:36.144 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042352278 2024-09-04T17:46:36.144 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.144 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.144 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:36.144 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.226889+0000", 2024-09-04T17:46:36.144 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000139376 2024-09-04T17:46:36.144 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.144 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.144 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:36.145 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.227062+0000", 2024-09-04T17:46:36.145 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00017342099999999999 2024-09-04T17:46:36.145 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:36.145 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:36.145 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:36.145 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.145 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.145 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14385 3.d 3:b763de10:::rbd_data.11345bd5898b.0000000000000002:a [read 626688~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:36.145 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.184481+0000", 2024-09-04T17:46:36.145 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.923309095, 2024-09-04T17:46:36.145 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042873850999999998, 2024-09-04T17:46:36.145 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:36.145 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:36.145 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:36.146 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:36.146 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:36.146 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:36.146 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14385 2024-09-04T17:46:36.146 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.146 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:36.146 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.146 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:36.146 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184481+0000", 2024-09-04T17:46:36.146 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:36.146 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.146 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.146 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:36.147 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184481+0000", 2024-09-04T17:46:36.147 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:36.147 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.147 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.147 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:36.147 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184482+0000", 2024-09-04T17:46:36.147 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2130000000000001e-06 2024-09-04T17:46:36.147 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.147 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.147 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:36.147 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184487+0000", 2024-09-04T17:46:36.147 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4260000000000002e-06 2024-09-04T17:46:36.147 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.147 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.148 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:36.148 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184505+0000", 2024-09-04T17:46:36.148 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7659000000000002e-05 2024-09-04T17:46:36.148 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.148 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.148 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:36.148 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184518+0000", 2024-09-04T17:46:36.148 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2982e-05 2024-09-04T17:46:36.148 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.148 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.148 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:36.148 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.227120+0000", 2024-09-04T17:46:36.148 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042601858999999999 2024-09-04T17:46:36.149 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.149 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.149 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:36.149 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.227196+0000", 2024-09-04T17:46:36.149 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.6540999999999998e-05 2024-09-04T17:46:36.149 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.149 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.149 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:36.149 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.227354+0000", 2024-09-04T17:46:36.149 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000158171 2024-09-04T17:46:36.149 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:36.149 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:36.150 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:36.150 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.150 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.150 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14389 3.d 3:b763de10:::rbd_data.11345bd5898b.0000000000000002:a [read 663552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:36.150 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.184555+0000", 2024-09-04T17:46:36.150 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.923234958999998, 2024-09-04T17:46:36.150 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043553640999999997, 2024-09-04T17:46:36.150 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:36.150 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:36.150 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:36.150 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:36.150 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:36.150 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:36.150 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14389 2024-09-04T17:46:36.151 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.151 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:36.151 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.151 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:36.151 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184555+0000", 2024-09-04T17:46:36.151 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:36.151 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.151 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.151 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:36.151 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184555+0000", 2024-09-04T17:46:36.151 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:36.151 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.151 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.151 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:36.152 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184556+0000", 2024-09-04T17:46:36.152 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.353e-06 2024-09-04T17:46:36.152 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.152 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.152 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:36.152 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184562+0000", 2024-09-04T17:46:36.152 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7899999999999996e-06 2024-09-04T17:46:36.152 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.152 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.152 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:36.152 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184580+0000", 2024-09-04T17:46:36.152 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7682000000000001e-05 2024-09-04T17:46:36.152 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.153 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.153 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:36.153 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184593+0000", 2024-09-04T17:46:36.153 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3625000000000001e-05 2024-09-04T17:46:36.153 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.153 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.153 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:36.153 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.227413+0000", 2024-09-04T17:46:36.153 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042819572 2024-09-04T17:46:36.153 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.153 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.153 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:36.153 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.227552+0000", 2024-09-04T17:46:36.154 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000138842 2024-09-04T17:46:36.154 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.154 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.154 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:36.154 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228108+0000", 2024-09-04T17:46:36.154 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00055677700000000005 2024-09-04T17:46:36.154 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:36.154 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:36.154 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:36.154 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.154 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.154 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14392 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:36.154 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.184613+0000", 2024-09-04T17:46:36.154 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.923176696999999, 2024-09-04T17:46:36.155 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043677522000000003, 2024-09-04T17:46:36.155 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:36.155 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:36.155 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:36.155 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:36.155 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:36.155 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:36.155 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14392 2024-09-04T17:46:36.155 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.155 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:36.155 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.155 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:36.155 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184613+0000", 2024-09-04T17:46:36.156 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:36.156 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.156 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.156 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:36.156 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184613+0000", 2024-09-04T17:46:36.156 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:36.156 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.156 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.156 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:36.156 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184614+0000", 2024-09-04T17:46:36.156 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.35e-06 2024-09-04T17:46:36.156 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.156 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.157 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:36.157 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184620+0000", 2024-09-04T17:46:36.157 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.5589999999999996e-06 2024-09-04T17:46:36.157 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.157 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.157 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:36.157 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184671+0000", 2024-09-04T17:46:36.157 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0615000000000002e-05 2024-09-04T17:46:36.157 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.157 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.157 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:36.157 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184684+0000", 2024-09-04T17:46:36.157 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3596e-05 2024-09-04T17:46:36.157 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.158 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.158 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:36.158 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228277+0000", 2024-09-04T17:46:36.158 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043592509000000002 2024-09-04T17:46:36.158 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.158 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.158 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:36.158 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228291+0000", 2024-09-04T17:46:36.158 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3893000000000001e-05 2024-09-04T17:46:36.158 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:36.158 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:36.158 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:36.158 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.159 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.159 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14395 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:36.159 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.184708+0000", 2024-09-04T17:46:36.159 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.923081505999999, 2024-09-04T17:46:36.159 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043643612999999998, 2024-09-04T17:46:36.159 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:36.159 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:36.159 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:36.159 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:36.159 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:36.159 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:36.159 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14395 2024-09-04T17:46:36.159 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.159 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:36.160 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.160 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:36.160 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184708+0000", 2024-09-04T17:46:36.160 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:36.160 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.160 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.160 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:36.160 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184708+0000", 2024-09-04T17:46:36.160 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:36.160 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.160 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.160 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:36.160 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184709+0000", 2024-09-04T17:46:36.161 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2359999999999999e-06 2024-09-04T17:46:36.161 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.161 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.161 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:36.161 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184716+0000", 2024-09-04T17:46:36.161 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.3500000000000002e-06 2024-09-04T17:46:36.161 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.161 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.161 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:36.161 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184733+0000", 2024-09-04T17:46:36.161 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6775000000000001e-05 2024-09-04T17:46:36.161 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.161 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.162 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:36.162 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184777+0000", 2024-09-04T17:46:36.162 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4183e-05 2024-09-04T17:46:36.162 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.162 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.162 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:36.162 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228339+0000", 2024-09-04T17:46:36.162 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043562030000000002 2024-09-04T17:46:36.162 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.162 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.162 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:36.162 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228352+0000", 2024-09-04T17:46:36.162 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3039e-05 2024-09-04T17:46:36.162 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:36.163 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:36.163 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:36.163 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.163 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.163 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14398 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:36.163 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.184847+0000", 2024-09-04T17:46:36.163 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.922942858999999, 2024-09-04T17:46:36.163 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043564139000000002, 2024-09-04T17:46:36.163 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:36.163 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:36.163 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:36.163 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:36.163 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:36.163 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:36.164 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14398 2024-09-04T17:46:36.164 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.164 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:36.164 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.164 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:36.164 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184847+0000", 2024-09-04T17:46:36.164 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:36.164 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.164 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.164 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:36.164 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184847+0000", 2024-09-04T17:46:36.164 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:36.164 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.165 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.165 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:36.165 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184848+0000", 2024-09-04T17:46:36.165 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.28e-06 2024-09-04T17:46:36.165 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.165 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.165 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:36.165 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184861+0000", 2024-09-04T17:46:36.165 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3125000000000001e-05 2024-09-04T17:46:36.165 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.165 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.165 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:36.165 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184884+0000", 2024-09-04T17:46:36.165 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2357999999999998e-05 2024-09-04T17:46:36.166 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.166 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.166 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:36.166 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184900+0000", 2024-09-04T17:46:36.166 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6872e-05 2024-09-04T17:46:36.166 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.166 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.166 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:36.166 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228398+0000", 2024-09-04T17:46:36.166 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043497313000000003 2024-09-04T17:46:36.166 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.166 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.166 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:36.167 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228411+0000", 2024-09-04T17:46:36.167 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3191e-05 2024-09-04T17:46:36.167 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:36.167 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:36.167 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:36.167 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.167 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.167 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14399 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:36.167 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.184925+0000", 2024-09-04T17:46:36.167 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.922864542999999, 2024-09-04T17:46:36.167 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043544799000000002, 2024-09-04T17:46:36.167 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:36.167 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:36.168 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:36.168 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:36.168 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:36.168 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:36.168 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14399 2024-09-04T17:46:36.168 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.168 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:36.168 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.168 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:36.168 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184925+0000", 2024-09-04T17:46:36.168 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:36.168 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.168 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.169 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:36.169 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184925+0000", 2024-09-04T17:46:36.169 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:36.169 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.169 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.169 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:36.169 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184927+0000", 2024-09-04T17:46:36.169 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7e-06 2024-09-04T17:46:36.169 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.169 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.169 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:36.169 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184949+0000", 2024-09-04T17:46:36.169 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1880999999999999e-05 2024-09-04T17:46:36.170 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.170 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.170 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:36.170 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184969+0000", 2024-09-04T17:46:36.170 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0758e-05 2024-09-04T17:46:36.170 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.170 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.170 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:36.170 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184983+0000", 2024-09-04T17:46:36.170 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3336000000000001e-05 2024-09-04T17:46:36.170 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.170 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.170 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:36.170 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228456+0000", 2024-09-04T17:46:36.171 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043472960999999997 2024-09-04T17:46:36.171 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.171 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.171 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:36.171 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228470+0000", 2024-09-04T17:46:36.171 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4163e-05 2024-09-04T17:46:36.171 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:36.171 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:36.171 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:36.171 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.171 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.171 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14400 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:36.171 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.185003+0000", 2024-09-04T17:46:36.172 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.922787107000001, 2024-09-04T17:46:36.172 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043532659000000001, 2024-09-04T17:46:36.172 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:36.172 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:36.172 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:36.172 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:36.172 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:36.172 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:36.172 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14400 2024-09-04T17:46:36.172 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.172 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:36.172 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.172 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:36.173 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185003+0000", 2024-09-04T17:46:36.173 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:36.173 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.173 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.173 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:36.173 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185003+0000", 2024-09-04T17:46:36.173 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:36.173 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.173 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.173 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:36.173 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185004+0000", 2024-09-04T17:46:36.173 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.294e-06 2024-09-04T17:46:36.174 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.174 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.174 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:36.174 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185010+0000", 2024-09-04T17:46:36.174 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7690000000000002e-06 2024-09-04T17:46:36.174 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.174 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.174 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:36.174 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185029+0000", 2024-09-04T17:46:36.174 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9100999999999999e-05 2024-09-04T17:46:36.174 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.174 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.174 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:36.175 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185043+0000", 2024-09-04T17:46:36.175 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3862e-05 2024-09-04T17:46:36.175 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.175 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.175 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:36.175 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228510+0000", 2024-09-04T17:46:36.175 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043467766999999997 2024-09-04T17:46:36.175 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.175 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.175 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:36.175 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228535+0000", 2024-09-04T17:46:36.175 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4865999999999999e-05 2024-09-04T17:46:36.175 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:36.176 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:36.176 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:36.176 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.176 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.176 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14401 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:36.176 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.185062+0000", 2024-09-04T17:46:36.176 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.922727842, 2024-09-04T17:46:36.176 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043556096000000002, 2024-09-04T17:46:36.176 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:36.176 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:36.176 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:36.176 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:36.176 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:36.177 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:36.177 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14401 2024-09-04T17:46:36.177 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.177 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:36.177 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.177 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:36.177 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185062+0000", 2024-09-04T17:46:36.177 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:36.177 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.177 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.177 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:36.177 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185062+0000", 2024-09-04T17:46:36.177 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:36.177 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.178 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.178 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:36.178 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185063+0000", 2024-09-04T17:46:36.178 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.409e-06 2024-09-04T17:46:36.178 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.178 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.178 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:36.178 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185069+0000", 2024-09-04T17:46:36.178 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.5400000000000003e-06 2024-09-04T17:46:36.178 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.178 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.178 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:36.178 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185087+0000", 2024-09-04T17:46:36.179 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8195000000000001e-05 2024-09-04T17:46:36.179 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.179 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.179 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:36.179 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185101+0000", 2024-09-04T17:46:36.179 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3769e-05 2024-09-04T17:46:36.179 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.179 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.179 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:36.179 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228604+0000", 2024-09-04T17:46:36.179 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043503537000000002 2024-09-04T17:46:36.179 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.179 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.180 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:36.180 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228618+0000", 2024-09-04T17:46:36.180 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3645999999999999e-05 2024-09-04T17:46:36.180 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:36.180 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:36.180 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:36.180 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.180 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.180 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14402 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:36.180 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.185121+0000", 2024-09-04T17:46:36.180 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.922668548000001, 2024-09-04T17:46:36.180 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043551542999999998, 2024-09-04T17:46:36.181 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:36.181 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:36.181 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:36.181 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:36.181 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:36.181 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:36.181 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14402 2024-09-04T17:46:36.181 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.181 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:36.181 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.181 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:36.181 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185121+0000", 2024-09-04T17:46:36.181 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:36.182 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.182 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.182 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:36.182 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185121+0000", 2024-09-04T17:46:36.182 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:36.182 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.182 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.182 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:36.182 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185122+0000", 2024-09-04T17:46:36.182 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1769999999999999e-06 2024-09-04T17:46:36.182 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.182 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.182 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:36.183 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185128+0000", 2024-09-04T17:46:36.183 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7790000000000001e-06 2024-09-04T17:46:36.183 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.183 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.183 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:36.183 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185146+0000", 2024-09-04T17:46:36.183 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8114999999999999e-05 2024-09-04T17:46:36.183 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.183 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.183 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:36.183 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185160+0000", 2024-09-04T17:46:36.183 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4158999999999999e-05 2024-09-04T17:46:36.183 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.183 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.184 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:36.184 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228659+0000", 2024-09-04T17:46:36.184 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043498715 2024-09-04T17:46:36.184 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.184 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.184 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:36.184 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228673+0000", 2024-09-04T17:46:36.184 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3597999999999999e-05 2024-09-04T17:46:36.184 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:36.184 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:36.184 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:36.184 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.184 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.185 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14403 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:36.185 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.185183+0000", 2024-09-04T17:46:36.185 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.922606500000001, 2024-09-04T17:46:36.185 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043541467, 2024-09-04T17:46:36.185 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:36.185 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:36.185 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:36.185 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:36.185 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:36.185 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:36.185 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14403 2024-09-04T17:46:36.185 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.186 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:36.186 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.186 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:36.186 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185183+0000", 2024-09-04T17:46:36.186 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:36.186 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.186 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.186 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:36.186 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185183+0000", 2024-09-04T17:46:36.186 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:36.186 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.186 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.186 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:36.187 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185184+0000", 2024-09-04T17:46:36.187 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.156e-06 2024-09-04T17:46:36.187 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.187 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.187 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:36.187 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185190+0000", 2024-09-04T17:46:36.187 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1800000000000004e-06 2024-09-04T17:46:36.187 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.187 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.187 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:36.187 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185207+0000", 2024-09-04T17:46:36.187 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7288000000000001e-05 2024-09-04T17:46:36.187 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.188 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.188 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:36.188 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185221+0000", 2024-09-04T17:46:36.188 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3729e-05 2024-09-04T17:46:36.188 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.188 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.188 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:36.188 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228711+0000", 2024-09-04T17:46:36.188 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043490595 2024-09-04T17:46:36.188 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.188 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.188 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:36.189 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228725+0000", 2024-09-04T17:46:36.189 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3519e-05 2024-09-04T17:46:36.189 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:36.189 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:36.189 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:36.189 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.189 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.189 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14404 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:36.189 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.185239+0000", 2024-09-04T17:46:36.189 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.922550361999999, 2024-09-04T17:46:36.189 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043542099000000001, 2024-09-04T17:46:36.189 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:36.189 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:36.189 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:36.190 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:36.190 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:36.190 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:36.190 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14404 2024-09-04T17:46:36.190 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.190 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:36.190 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.190 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:36.190 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185239+0000", 2024-09-04T17:46:36.190 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:36.190 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.190 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.191 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:36.191 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185239+0000", 2024-09-04T17:46:36.191 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:36.191 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.191 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.191 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:36.191 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185240+0000", 2024-09-04T17:46:36.191 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.173e-06 2024-09-04T17:46:36.191 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.191 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.191 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:36.191 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185246+0000", 2024-09-04T17:46:36.191 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7529999999999998e-06 2024-09-04T17:46:36.191 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.192 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.192 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:36.192 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185264+0000", 2024-09-04T17:46:36.192 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7872000000000001e-05 2024-09-04T17:46:36.192 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.192 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.192 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:36.192 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185278+0000", 2024-09-04T17:46:36.192 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3532e-05 2024-09-04T17:46:36.192 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.192 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.192 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:36.192 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228768+0000", 2024-09-04T17:46:36.193 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043490586999999997 2024-09-04T17:46:36.193 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.193 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.193 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:36.193 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228781+0000", 2024-09-04T17:46:36.193 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3182e-05 2024-09-04T17:46:36.193 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:36.193 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:36.193 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:36.194 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.194 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.194 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14405 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:36.194 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.185297+0000", 2024-09-04T17:46:36.194 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.922493170999999, 2024-09-04T17:46:36.194 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043535049999999999, 2024-09-04T17:46:36.194 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:36.194 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:36.194 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:36.195 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:36.195 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:36.195 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:36.195 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14405 2024-09-04T17:46:36.195 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.195 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:36.195 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.195 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:36.195 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185297+0000", 2024-09-04T17:46:36.195 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:36.195 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.196 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.196 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:36.196 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185297+0000", 2024-09-04T17:46:36.196 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:36.196 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.196 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.196 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:36.196 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185298+0000", 2024-09-04T17:46:36.196 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.22e-06 2024-09-04T17:46:36.196 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.196 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.196 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:36.197 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185308+0000", 2024-09-04T17:46:36.197 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0173000000000001e-05 2024-09-04T17:46:36.197 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.197 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.197 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:36.197 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185329+0000", 2024-09-04T17:46:36.197 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1087999999999999e-05 2024-09-04T17:46:36.197 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.197 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.197 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:36.197 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185345+0000", 2024-09-04T17:46:36.197 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6035000000000001e-05 2024-09-04T17:46:36.197 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.198 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.198 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:36.198 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228822+0000", 2024-09-04T17:46:36.198 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043477427999999999 2024-09-04T17:46:36.198 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.198 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.198 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:36.198 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228832+0000", 2024-09-04T17:46:36.199 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1060000000000006e-06 2024-09-04T17:46:36.199 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:36.199 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:36.199 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:36.199 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.199 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.199 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14411 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:36.199 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.186171+0000", 2024-09-04T17:46:36.199 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.921618347999999, 2024-09-04T17:46:36.199 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.04271581, 2024-09-04T17:46:36.199 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:36.199 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:36.200 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:36.200 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:36.200 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:36.200 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:36.200 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14411 2024-09-04T17:46:36.200 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.200 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:36.200 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.200 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:36.200 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.186171+0000", 2024-09-04T17:46:36.200 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:36.200 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.200 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.201 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:36.201 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.186171+0000", 2024-09-04T17:46:36.201 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:36.201 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.201 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.201 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:36.201 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.186173+0000", 2024-09-04T17:46:36.201 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.012e-06 2024-09-04T17:46:36.201 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.201 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.201 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:36.201 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.186183+0000", 2024-09-04T17:46:36.201 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.7729999999999994e-06 2024-09-04T17:46:36.201 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.202 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.202 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:36.202 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.186205+0000", 2024-09-04T17:46:36.202 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2280999999999999e-05 2024-09-04T17:46:36.202 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.202 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.202 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:36.202 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.186224+0000", 2024-09-04T17:46:36.202 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8695e-05 2024-09-04T17:46:36.202 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.202 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.202 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:36.203 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228873+0000", 2024-09-04T17:46:36.203 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042648798000000002 2024-09-04T17:46:36.203 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:36.203 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:36.203 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:36.203 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228887+0000", 2024-09-04T17:46:36.203 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4251e-05 2024-09-04T17:46:36.203 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:36.203 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:36.203 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:36.203 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:36.203 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:36.204 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:36.205 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:36.205 INFO:teuthology.orchestra.run.smithi092.stdout:1854 write 0xb79b376 thru 0xb79efba (0x3c45 bytes) 2024-09-04T17:46:36.205 INFO:teuthology.orchestra.run.smithi092.stdout:1856 punch from 0xc60443d to 0xc609df0, (0x59b3 bytes) 2024-09-04T17:46:36.206 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:36.206 INFO:teuthology.orchestra.run.smithi092.stdout:1861 read 0xbe69181 thru 0xbe6ebbb (0x5a3b bytes) 2024-09-04T17:46:36.206 INFO:teuthology.orchestra.run.smithi092.stdout:1862 writesame 0xb4eb8ee thru 0xb4ebd24 data_size 0x1(0x437 bytes) 2024-09-04T17:46:36.206 INFO:teuthology.orchestra.run.smithi092.stdout:1863 clone 17 order 19 su 262144 sc 2 2024-09-04T17:46:36.206 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-04T17:46:36.207 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:46:36.202+0000 7f505d642640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26446) UID: 0 2024-09-04T17:46:36.307 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:36.304+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:36.338 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:36.338 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:46:36.338 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:46:36.338 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:36.351 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-04T17:46:36.406 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-04T17:46:36.407 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:36.402+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:36.508 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:36.508+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:36.529 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:36.529 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:46:36.529 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:46:36.529 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:46:36.529 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:36.547 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-04T17:46:36.559 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone16 from 0xea0536f (overlap 0x960938e) to 0x0 2024-09-04T17:46:36.609 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:46:36.608+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:46:36.622 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-04T17:46:36.709 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:36.709 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:46:36.709 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:46:36.709 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:46:36.709 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.709 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:27242 3.b 3.34414deb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-04T17:46:36.709 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:28.402347+0000", 2024-09-04T17:46:36.709 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.3055622410000005, 2024-09-04T17:46:36.709 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.93265329299999999, 2024-09-04T17:46:36.710 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:36.710 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:36.710 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-04T17:46:36.710 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:36.710 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:36.710 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:36.710 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 27242 2024-09-04T17:46:36.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.710 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:36.710 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.710 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:36.710 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402347+0000", 2024-09-04T17:46:36.710 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:36.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.711 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:36.711 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402347+0000", 2024-09-04T17:46:36.711 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:36.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.711 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:36.711 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402349+0000", 2024-09-04T17:46:36.711 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.471e-06 2024-09-04T17:46:36.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.711 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:36.711 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402355+0000", 2024-09-04T17:46:36.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8399999999999997e-06 2024-09-04T17:46:36.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.712 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:36.712 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402364+0000", 2024-09-04T17:46:36.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1269999999999992e-06 2024-09-04T17:46:36.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.712 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:36.712 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402373+0000", 2024-09-04T17:46:36.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7909999999999993e-06 2024-09-04T17:46:36.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.713 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:36.713 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402460+0000", 2024-09-04T17:46:36.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6686000000000003e-05 2024-09-04T17:46:36.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.713 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-04T17:46:36.713 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402473+0000", 2024-09-04T17:46:36.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3281e-05 2024-09-04T17:46:36.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.713 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:36.713 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:29.334989+0000", 2024-09-04T17:46:36.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.93251549099999997 2024-09-04T17:46:36.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.714 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.714 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:36.714 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:29.335000+0000", 2024-09-04T17:46:36.714 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1606e-05 2024-09-04T17:46:36.714 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:36.714 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:36.714 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:36.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.714 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.714 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:27430 3.15 3:af3f346f:::rbd_data.1134eedad849.00000000000002ba:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 10=[10] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-04T17:46:36.714 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:30.441488+0000", 2024-09-04T17:46:36.714 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.266421029, 2024-09-04T17:46:36.715 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011517892, 2024-09-04T17:46:36.715 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:36.715 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:36.715 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:36.715 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:36.715 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:36.715 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:36.715 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 27430 2024-09-04T17:46:36.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.715 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:36.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.715 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:36.715 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.441488+0000", 2024-09-04T17:46:36.715 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:36.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.716 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.716 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:36.716 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.441488+0000", 2024-09-04T17:46:36.716 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:36.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.716 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.716 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:36.716 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.441490+0000", 2024-09-04T17:46:36.716 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.903e-06 2024-09-04T17:46:36.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.717 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:36.717 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.441497+0000", 2024-09-04T17:46:36.717 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.742e-06 2024-09-04T17:46:36.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.717 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:36.717 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.441510+0000", 2024-09-04T17:46:36.717 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3455000000000001e-05 2024-09-04T17:46:36.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.717 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:36.717 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.441519+0000", 2024-09-04T17:46:36.718 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8629999999999997e-06 2024-09-04T17:46:36.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.718 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:36.718 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.441575+0000", 2024-09-04T17:46:36.718 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6270999999999997e-05 2024-09-04T17:46:36.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.718 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:36.718 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.441651+0000", 2024-09-04T17:46:36.718 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5094999999999998e-05 2024-09-04T17:46:36.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.719 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-04T17:46:36.719 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.441931+0000", 2024-09-04T17:46:36.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028048999999999999 2024-09-04T17:46:36.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.719 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.719 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:46:36.719 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.442795+0000", 2024-09-04T17:46:36.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00086421299999999996 2024-09-04T17:46:36.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.719 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.719 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:36.719 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.452846+0000", 2024-09-04T17:46:36.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010050777 2024-09-04T17:46:36.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.720 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:36.720 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.452931+0000", 2024-09-04T17:46:36.720 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5165000000000002e-05 2024-09-04T17:46:36.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.720 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:36.720 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.453006+0000", 2024-09-04T17:46:36.720 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4918000000000003e-05 2024-09-04T17:46:36.720 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:36.720 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:36.720 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:36.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.721 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.721 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:28118 3.15 3:ae11ba99:::rbd_data.1134b705d747.0000000000000021:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-04T17:46:36.721 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:31.234554+0000", 2024-09-04T17:46:36.721 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.4733551330000001, 2024-09-04T17:46:36.721 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012096567000000001, 2024-09-04T17:46:36.721 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:36.721 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:36.721 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:36.721 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:36.721 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:36.721 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:36.721 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 28118 2024-09-04T17:46:36.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.722 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:36.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.722 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:36.722 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:31.234554+0000", 2024-09-04T17:46:36.722 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:36.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.722 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:36.722 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:31.234554+0000", 2024-09-04T17:46:36.722 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:36.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:36.723 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:31.234555+0000", 2024-09-04T17:46:36.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1030000000000001e-06 2024-09-04T17:46:36.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:36.723 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:31.234560+0000", 2024-09-04T17:46:36.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6940000000000003e-06 2024-09-04T17:46:36.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:36.723 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:31.234570+0000", 2024-09-04T17:46:36.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0324e-05 2024-09-04T17:46:36.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.724 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:36.724 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:31.234578+0000", 2024-09-04T17:46:36.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9510000000000006e-06 2024-09-04T17:46:36.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.724 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:36.724 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:31.234621+0000", 2024-09-04T17:46:36.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2997000000000002e-05 2024-09-04T17:46:36.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:36.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:31.234703+0000", 2024-09-04T17:46:36.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1581000000000004e-05 2024-09-04T17:46:36.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-04T17:46:36.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:31.235920+0000", 2024-09-04T17:46:36.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0012176380000000001 2024-09-04T17:46:36.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:46:36.726 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:31.245608+0000", 2024-09-04T17:46:36.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0096877809999999995 2024-09-04T17:46:36.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.726 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:36.726 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:31.246574+0000", 2024-09-04T17:46:36.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00096599200000000004 2024-09-04T17:46:36.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.726 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:36.726 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:31.246607+0000", 2024-09-04T17:46:36.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2546999999999999e-05 2024-09-04T17:46:36.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.727 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:36.727 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:31.246651+0000", 2024-09-04T17:46:36.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3958999999999997e-05 2024-09-04T17:46:36.727 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:36.727 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:36.727 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:36.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.727 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.727 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:32555 3.2 3:408b02f8:::rbd_object_map.1134b8120a27:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-04T17:46:36.727 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:34.821003+0000", 2024-09-04T17:46:36.727 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.8869064469999999, 2024-09-04T17:46:36.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012356361, 2024-09-04T17:46:36.727 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:36.728 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:36.728 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:36.728 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:36.728 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:36.728 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:36.728 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 32555 2024-09-04T17:46:36.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.728 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:36.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:36.728 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.821003+0000", 2024-09-04T17:46:36.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:36.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.729 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:36.729 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.821003+0000", 2024-09-04T17:46:36.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:36.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.729 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:36.729 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.821005+0000", 2024-09-04T17:46:36.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.26e-06 2024-09-04T17:46:36.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.730 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:36.730 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.821016+0000", 2024-09-04T17:46:36.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1212e-05 2024-09-04T17:46:36.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.730 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:36.730 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.821045+0000", 2024-09-04T17:46:36.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9014999999999999e-05 2024-09-04T17:46:36.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.730 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:36.731 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.821067+0000", 2024-09-04T17:46:36.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1654999999999998e-05 2024-09-04T17:46:36.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.731 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:36.731 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.821188+0000", 2024-09-04T17:46:36.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012147200000000001 2024-09-04T17:46:36.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.731 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:36.731 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.821323+0000", 2024-09-04T17:46:36.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013457399999999999 2024-09-04T17:46:36.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5", 2024-09-04T17:46:36.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.821893+0000", 2024-09-04T17:46:36.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00057031400000000004 2024-09-04T17:46:36.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:36.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.827686+0000", 2024-09-04T17:46:36.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057927559999999996 2024-09-04T17:46:36.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:46:36.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.833177+0000", 2024-09-04T17:46:36.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054906360000000001 2024-09-04T17:46:36.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:36.733 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.833266+0000", 2024-09-04T17:46:36.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9536999999999993e-05 2024-09-04T17:46:36.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:36.733 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.833359+0000", 2024-09-04T17:46:36.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2930000000000006e-05 2024-09-04T17:46:36.733 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:36.733 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:36.734 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:36.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.734 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:32556 3.2 3:408b02f8:::rbd_object_map.1134b8120a27:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-04T17:46:36.734 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:34.821154+0000", 2024-09-04T17:46:36.734 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.886754968, 2024-09-04T17:46:36.734 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012347499, 2024-09-04T17:46:36.734 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:36.734 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:36.734 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:36.734 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:36.734 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:36.734 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:36.735 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 32556 2024-09-04T17:46:36.735 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.735 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:36.735 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.735 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:36.735 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.821154+0000", 2024-09-04T17:46:36.735 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:36.735 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.735 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.735 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:36.735 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.821154+0000", 2024-09-04T17:46:36.735 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:36.735 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.736 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:36.736 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.821156+0000", 2024-09-04T17:46:36.736 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7829999999999999e-06 2024-09-04T17:46:36.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.736 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:36.736 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.821166+0000", 2024-09-04T17:46:36.736 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8130000000000007e-06 2024-09-04T17:46:36.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.736 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:36.736 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.821190+0000", 2024-09-04T17:46:36.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3859e-05 2024-09-04T17:46:36.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:36.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.821207+0000", 2024-09-04T17:46:36.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7391999999999999e-05 2024-09-04T17:46:36.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:36.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.822754+0000", 2024-09-04T17:46:36.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001547297 2024-09-04T17:46:36.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:36.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.822898+0000", 2024-09-04T17:46:36.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014305399999999999 2024-09-04T17:46:36.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5", 2024-09-04T17:46:36.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.823857+0000", 2024-09-04T17:46:36.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00095949999999999996 2024-09-04T17:46:36.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:36.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.827805+0000", 2024-09-04T17:46:36.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003948399 2024-09-04T17:46:36.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.739 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.739 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:46:36.739 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.833377+0000", 2024-09-04T17:46:36.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055716539999999997 2024-09-04T17:46:36.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.739 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.739 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:36.739 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.833450+0000", 2024-09-04T17:46:36.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2597999999999998e-05 2024-09-04T17:46:36.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.739 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.740 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:36.740 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.833502+0000", 2024-09-04T17:46:36.740 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.215e-05 2024-09-04T17:46:36.740 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:36.740 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:36.740 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:36.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.740 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.740 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:32557 3.2 3:408b02f8:::rbd_object_map.1134b8120a27:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-04T17:46:36.740 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:34.821237+0000", 2024-09-04T17:46:36.740 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.88667191, 2024-09-04T17:46:36.740 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012496975, 2024-09-04T17:46:36.740 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:36.741 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:36.741 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:36.741 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:36.741 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:36.741 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:36.741 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 32557 2024-09-04T17:46:36.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.741 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:36.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:36.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.821237+0000", 2024-09-04T17:46:36.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:36.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:36.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.821237+0000", 2024-09-04T17:46:36.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:36.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:36.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.821239+0000", 2024-09-04T17:46:36.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3570000000000001e-06 2024-09-04T17:46:36.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:36.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.821246+0000", 2024-09-04T17:46:36.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7360000000000001e-06 2024-09-04T17:46:36.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.743 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:36.743 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.821265+0000", 2024-09-04T17:46:36.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8782000000000001e-05 2024-09-04T17:46:36.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.743 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:36.743 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.821280+0000", 2024-09-04T17:46:36.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5019e-05 2024-09-04T17:46:36.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.744 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:36.744 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.825032+0000", 2024-09-04T17:46:36.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003751295 2024-09-04T17:46:36.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.744 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:36.744 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.825285+0000", 2024-09-04T17:46:36.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000253677 2024-09-04T17:46:36.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.744 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5", 2024-09-04T17:46:36.744 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.826475+0000", 2024-09-04T17:46:36.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001189641 2024-09-04T17:46:36.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:36.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.833031+0000", 2024-09-04T17:46:36.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0065556959999999997 2024-09-04T17:46:36.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:46:36.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.833529+0000", 2024-09-04T17:46:36.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00049881199999999995 2024-09-04T17:46:36.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:36.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.833668+0000", 2024-09-04T17:46:36.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013811499999999999 2024-09-04T17:46:36.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:36.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.833734+0000", 2024-09-04T17:46:36.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6845000000000001e-05 2024-09-04T17:46:36.746 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:36.746 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:36.746 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:36.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.746 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:32558 3.2 3:408b02f8:::rbd_object_map.1134b8120a27:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-04T17:46:36.747 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:34.821300+0000", 2024-09-04T17:46:36.747 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.886608828, 2024-09-04T17:46:36.747 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012648744999999999, 2024-09-04T17:46:36.747 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:36.747 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:36.747 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:36.747 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:36.747 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:36.747 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:36.747 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 32558 2024-09-04T17:46:36.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.747 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:36.747 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.747 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:36.748 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.821300+0000", 2024-09-04T17:46:36.748 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:36.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.748 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.748 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:36.748 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.821300+0000", 2024-09-04T17:46:36.748 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:36.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.748 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.748 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:36.748 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.821302+0000", 2024-09-04T17:46:36.748 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.08e-06 2024-09-04T17:46:36.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.749 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:36.749 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.821309+0000", 2024-09-04T17:46:36.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4970000000000004e-06 2024-09-04T17:46:36.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.749 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:36.749 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.821326+0000", 2024-09-04T17:46:36.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7201999999999999e-05 2024-09-04T17:46:36.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.749 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:36.749 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.821340+0000", 2024-09-04T17:46:36.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3439e-05 2024-09-04T17:46:36.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:36.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.827925+0000", 2024-09-04T17:46:36.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0065850099999999996 2024-09-04T17:46:36.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:36.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.828172+0000", 2024-09-04T17:46:36.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024722299999999997 2024-09-04T17:46:36.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5", 2024-09-04T17:46:36.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.829284+0000", 2024-09-04T17:46:36.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001111981 2024-09-04T17:46:36.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:36.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.833157+0000", 2024-09-04T17:46:36.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0038731310000000001 2024-09-04T17:46:36.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:46:36.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.833753+0000", 2024-09-04T17:46:36.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00059566899999999995 2024-09-04T17:46:36.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.752 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:36.752 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.833843+0000", 2024-09-04T17:46:36.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0247000000000005e-05 2024-09-04T17:46:36.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.752 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:36.752 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.833949+0000", 2024-09-04T17:46:36.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106266 2024-09-04T17:46:36.752 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:36.752 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:36.752 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:36.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.753 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:32559 3.2 3:408b02f8:::rbd_object_map.1134b8120a27:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-04T17:46:36.753 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:34.821358+0000", 2024-09-04T17:46:36.753 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.8865508520000001, 2024-09-04T17:46:36.753 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01291278, 2024-09-04T17:46:36.753 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:36.753 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:36.753 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:36.753 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:36.753 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:36.753 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:36.753 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 32559 2024-09-04T17:46:36.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.754 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:36.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:36.754 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.821358+0000", 2024-09-04T17:46:36.754 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:36.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:36.754 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.821358+0000", 2024-09-04T17:46:36.754 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:36.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:36.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.821360+0000", 2024-09-04T17:46:36.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5770000000000001e-06 2024-09-04T17:46:36.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:36.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.821367+0000", 2024-09-04T17:46:36.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9530000000000002e-06 2024-09-04T17:46:36.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:36.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.821384+0000", 2024-09-04T17:46:36.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6832000000000001e-05 2024-09-04T17:46:36.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.756 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:36.756 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.821397+0000", 2024-09-04T17:46:36.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3256e-05 2024-09-04T17:46:36.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.756 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:36.756 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.830417+0000", 2024-09-04T17:46:36.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0090203869999999995 2024-09-04T17:46:36.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.757 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:36.757 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.830686+0000", 2024-09-04T17:46:36.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026871799999999998 2024-09-04T17:46:36.757 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.757 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5", 2024-09-04T17:46:36.757 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.831863+0000", 2024-09-04T17:46:36.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011768659999999999 2024-09-04T17:46:36.757 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.757 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:46:36.757 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.833968+0000", 2024-09-04T17:46:36.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0021046950000000002 2024-09-04T17:46:36.757 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.758 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:36.758 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.834096+0000", 2024-09-04T17:46:36.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012856799999999999 2024-09-04T17:46:36.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.758 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:36.758 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.834182+0000", 2024-09-04T17:46:36.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5357000000000003e-05 2024-09-04T17:46:36.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.758 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:36.758 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.834271+0000", 2024-09-04T17:46:36.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9571e-05 2024-09-04T17:46:36.759 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:36.759 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:36.759 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:36.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.759 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:32567 3.2 3:408b02f8:::rbd_object_map.1134b8120a27:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-04T17:46:36.759 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:34.840058+0000", 2024-09-04T17:46:36.759 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.8678516329999999, 2024-09-04T17:46:36.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010756577, 2024-09-04T17:46:36.759 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:36.759 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:36.759 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:36.759 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:36.760 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:36.760 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:36.760 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 32567 2024-09-04T17:46:36.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.760 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:36.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:36.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.840058+0000", 2024-09-04T17:46:36.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:36.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:36.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.840058+0000", 2024-09-04T17:46:36.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:36.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:36.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.840059+0000", 2024-09-04T17:46:36.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4589999999999999e-06 2024-09-04T17:46:36.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:36.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.840067+0000", 2024-09-04T17:46:36.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5569999999999998e-06 2024-09-04T17:46:36.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.762 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:36.762 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.840086+0000", 2024-09-04T17:46:36.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9765000000000001e-05 2024-09-04T17:46:36.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.762 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:36.762 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.840104+0000", 2024-09-04T17:46:36.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7685000000000001e-05 2024-09-04T17:46:36.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.762 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:36.762 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.843578+0000", 2024-09-04T17:46:36.763 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003473631 2024-09-04T17:46:36.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.763 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:36.763 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.843849+0000", 2024-09-04T17:46:36.763 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027083899999999999 2024-09-04T17:46:36.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.763 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5", 2024-09-04T17:46:36.763 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.844697+0000", 2024-09-04T17:46:36.763 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00084863400000000002 2024-09-04T17:46:36.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:36.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.850531+0000", 2024-09-04T17:46:36.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058337459999999999 2024-09-04T17:46:36.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:46:36.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.850650+0000", 2024-09-04T17:46:36.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000119355 2024-09-04T17:46:36.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:36.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.850731+0000", 2024-09-04T17:46:36.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0748999999999995e-05 2024-09-04T17:46:36.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:36.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.850814+0000", 2024-09-04T17:46:36.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3157000000000004e-05 2024-09-04T17:46:36.765 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:36.765 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:36.765 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:36.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.765 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:32568 3.2 3:408b02f8:::rbd_object_map.1134b8120a27:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-04T17:46:36.765 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:34.840121+0000", 2024-09-04T17:46:36.765 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.8677882480000001, 2024-09-04T17:46:36.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010857660999999999, 2024-09-04T17:46:36.766 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:36.766 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:36.766 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:36.766 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:36.766 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:36.766 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:36.766 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 32568 2024-09-04T17:46:36.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.766 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:36.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.766 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:36.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.840121+0000", 2024-09-04T17:46:36.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:36.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.767 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:36.767 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.840121+0000", 2024-09-04T17:46:36.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:36.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.767 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:36.767 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.840122+0000", 2024-09-04T17:46:36.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7400000000000002e-07 2024-09-04T17:46:36.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.768 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:36.768 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.840126+0000", 2024-09-04T17:46:36.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3329999999999998e-06 2024-09-04T17:46:36.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.768 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:36.768 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.840138+0000", 2024-09-04T17:46:36.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1999e-05 2024-09-04T17:46:36.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.768 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:36.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.840145+0000", 2024-09-04T17:46:36.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3390000000000004e-06 2024-09-04T17:46:36.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:36.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.846264+0000", 2024-09-04T17:46:36.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061184029999999997 2024-09-04T17:46:36.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:36.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.846450+0000", 2024-09-04T17:46:36.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00018564100000000001 2024-09-04T17:46:36.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5", 2024-09-04T17:46:36.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.847244+0000", 2024-09-04T17:46:36.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000794668 2024-09-04T17:46:36.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:36.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.850630+0000", 2024-09-04T17:46:36.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0033853350000000002 2024-09-04T17:46:36.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:46:36.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.850830+0000", 2024-09-04T17:46:36.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020032200000000001 2024-09-04T17:46:36.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:36.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.850899+0000", 2024-09-04T17:46:36.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9295000000000007e-05 2024-09-04T17:46:36.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:36.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.850979+0000", 2024-09-04T17:46:36.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9552e-05 2024-09-04T17:46:36.771 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:36.771 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:36.772 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:36.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.772 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:32569 3.2 3:408b02f8:::rbd_object_map.1134b8120a27:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-04T17:46:36.772 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:34.840156+0000", 2024-09-04T17:46:36.772 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.8677531699999999, 2024-09-04T17:46:36.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012642151000000001, 2024-09-04T17:46:36.772 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:36.772 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:36.772 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:36.772 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:36.772 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:36.772 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:36.773 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 32569 2024-09-04T17:46:36.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.773 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:36.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.773 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:36.773 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.840156+0000", 2024-09-04T17:46:36.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:36.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.773 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:36.773 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.840156+0000", 2024-09-04T17:46:36.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:36.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:36.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.840157+0000", 2024-09-04T17:46:36.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7000000000000004e-07 2024-09-04T17:46:36.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:36.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.840161+0000", 2024-09-04T17:46:36.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9029999999999997e-06 2024-09-04T17:46:36.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:36.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.840184+0000", 2024-09-04T17:46:36.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2809000000000001e-05 2024-09-04T17:46:36.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:36.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.840191+0000", 2024-09-04T17:46:36.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8990000000000001e-06 2024-09-04T17:46:36.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:36.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.848589+0000", 2024-09-04T17:46:36.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0083977730000000007 2024-09-04T17:46:36.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:36.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.848755+0000", 2024-09-04T17:46:36.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016585499999999999 2024-09-04T17:46:36.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5", 2024-09-04T17:46:36.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.849657+0000", 2024-09-04T17:46:36.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00090234799999999997 2024-09-04T17:46:36.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:46:36.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.851199+0000", 2024-09-04T17:46:36.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0015417009999999999 2024-09-04T17:46:36.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.777 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:36.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.852630+0000", 2024-09-04T17:46:36.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014305940000000001 2024-09-04T17:46:36.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.777 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:36.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.852712+0000", 2024-09-04T17:46:36.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1885000000000006e-05 2024-09-04T17:46:36.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:36.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.852798+0000", 2024-09-04T17:46:36.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6713999999999997e-05 2024-09-04T17:46:36.778 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:36.778 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:36.778 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:36.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.778 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:32578 3.2 3:408b02f8:::rbd_object_map.1134b8120a27:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-04T17:46:36.778 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:34.855657+0000", 2024-09-04T17:46:36.778 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.8522521059999999, 2024-09-04T17:46:36.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011763456, 2024-09-04T17:46:36.779 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:36.779 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:36.779 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:36.779 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:36.779 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:36.779 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:36.779 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 32578 2024-09-04T17:46:36.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.779 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:36.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:36.779 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.855657+0000", 2024-09-04T17:46:36.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:36.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.780 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:36.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.855657+0000", 2024-09-04T17:46:36.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:36.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.780 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:36.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.855658+0000", 2024-09-04T17:46:36.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8000000000000004e-07 2024-09-04T17:46:36.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.780 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:36.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.855662+0000", 2024-09-04T17:46:36.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0570000000000003e-06 2024-09-04T17:46:36.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:36.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.855674+0000", 2024-09-04T17:46:36.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2069000000000001e-05 2024-09-04T17:46:36.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:36.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.855682+0000", 2024-09-04T17:46:36.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4200000000000001e-06 2024-09-04T17:46:36.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:36.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.862129+0000", 2024-09-04T17:46:36.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006447744 2024-09-04T17:46:36.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:36.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.862300+0000", 2024-09-04T17:46:36.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017106099999999999 2024-09-04T17:46:36.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5", 2024-09-04T17:46:36.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.862909+0000", 2024-09-04T17:46:36.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00060858799999999997 2024-09-04T17:46:36.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:36.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.866984+0000", 2024-09-04T17:46:36.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040750769999999999 2024-09-04T17:46:36.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:46:36.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.867161+0000", 2024-09-04T17:46:36.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017652799999999999 2024-09-04T17:46:36.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:36.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.867296+0000", 2024-09-04T17:46:36.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000134921 2024-09-04T17:46:36.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:36.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.867421+0000", 2024-09-04T17:46:36.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012511099999999999 2024-09-04T17:46:36.784 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:36.784 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:36.784 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:36.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.785 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:32579 3.2 3:408b02f8:::rbd_object_map.1134b8120a27:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-04T17:46:36.785 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:34.855692+0000", 2024-09-04T17:46:36.785 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.8522169609999999, 2024-09-04T17:46:36.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012140372999999999, 2024-09-04T17:46:36.785 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:36.785 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:36.785 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:36.785 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:36.785 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:36.785 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:36.785 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 32579 2024-09-04T17:46:36.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.786 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:36.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:36.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.855692+0000", 2024-09-04T17:46:36.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:36.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:36.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.855692+0000", 2024-09-04T17:46:36.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:36.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:36.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.855693+0000", 2024-09-04T17:46:36.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9999999999999997e-07 2024-09-04T17:46:36.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:36.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.855697+0000", 2024-09-04T17:46:36.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8859999999999997e-06 2024-09-04T17:46:36.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:36.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.855711+0000", 2024-09-04T17:46:36.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4276e-05 2024-09-04T17:46:36.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:36.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.855719+0000", 2024-09-04T17:46:36.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4059999999999999e-06 2024-09-04T17:46:36.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:36.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.864379+0000", 2024-09-04T17:46:36.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0086602650000000003 2024-09-04T17:46:36.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:36.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.864552+0000", 2024-09-04T17:46:36.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017304500000000001 2024-09-04T17:46:36.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5", 2024-09-04T17:46:36.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.865680+0000", 2024-09-04T17:46:36.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011276190000000001 2024-09-04T17:46:36.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:36.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.867127+0000", 2024-09-04T17:46:36.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014476370000000001 2024-09-04T17:46:36.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:46:36.790 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.867558+0000", 2024-09-04T17:46:36.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00043088399999999998 2024-09-04T17:46:36.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.790 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:36.790 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.867694+0000", 2024-09-04T17:46:36.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013608399999999999 2024-09-04T17:46:36.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.790 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:36.790 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.867833+0000", 2024-09-04T17:46:36.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013857100000000001 2024-09-04T17:46:36.790 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:36.791 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:36.791 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:36.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.791 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:32615 3.2 3:408b02f8:::rbd_object_map.1134b8120a27:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-04T17:46:36.791 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:34.905431+0000", 2024-09-04T17:46:36.791 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.802478389, 2024-09-04T17:46:36.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0095260680000000004, 2024-09-04T17:46:36.791 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:36.791 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:36.791 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:36.791 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:36.792 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:36.792 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:36.792 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 32615 2024-09-04T17:46:36.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.792 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:36.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:36.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.905431+0000", 2024-09-04T17:46:36.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:36.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:36.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.905431+0000", 2024-09-04T17:46:36.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:36.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:36.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.905432+0000", 2024-09-04T17:46:36.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4670000000000001e-06 2024-09-04T17:46:36.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:36.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.905442+0000", 2024-09-04T17:46:36.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4150000000000005e-06 2024-09-04T17:46:36.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:36.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.905459+0000", 2024-09-04T17:46:36.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7676000000000002e-05 2024-09-04T17:46:36.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:36.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.905469+0000", 2024-09-04T17:46:36.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2429999999999994e-06 2024-09-04T17:46:36.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:36.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.909685+0000", 2024-09-04T17:46:36.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004215824 2024-09-04T17:46:36.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:36.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.909834+0000", 2024-09-04T17:46:36.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014938899999999999 2024-09-04T17:46:36.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5", 2024-09-04T17:46:36.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.910508+0000", 2024-09-04T17:46:36.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00067406400000000004 2024-09-04T17:46:36.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:36.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.914511+0000", 2024-09-04T17:46:36.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040029870000000004 2024-09-04T17:46:36.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:46:36.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.914690+0000", 2024-09-04T17:46:36.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017892799999999999 2024-09-04T17:46:36.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:36.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.914843+0000", 2024-09-04T17:46:36.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015279599999999999 2024-09-04T17:46:36.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:36.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.914957+0000", 2024-09-04T17:46:36.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011427899999999999 2024-09-04T17:46:36.797 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:36.797 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:36.797 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:36.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.797 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:32616 3.2 3:408b02f8:::rbd_object_map.1134b8120a27:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-04T17:46:36.797 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:34.905493+0000", 2024-09-04T17:46:36.797 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.802415876, 2024-09-04T17:46:36.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097090519999999993, 2024-09-04T17:46:36.798 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:36.798 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:36.798 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:36.798 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:36.798 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:36.798 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:36.798 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 32616 2024-09-04T17:46:36.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.798 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:36.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:36.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.905493+0000", 2024-09-04T17:46:36.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:36.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:36.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.905493+0000", 2024-09-04T17:46:36.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:36.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:36.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.905494+0000", 2024-09-04T17:46:36.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1299999999999999e-07 2024-09-04T17:46:36.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:36.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.905499+0000", 2024-09-04T17:46:36.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.459e-06 2024-09-04T17:46:36.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:36.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.905509+0000", 2024-09-04T17:46:36.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8500000000000006e-06 2024-09-04T17:46:36.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:36.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.905516+0000", 2024-09-04T17:46:36.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2089999999999999e-06 2024-09-04T17:46:36.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:36.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.911732+0000", 2024-09-04T17:46:36.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0062166000000000001 2024-09-04T17:46:36.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:36.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.911933+0000", 2024-09-04T17:46:36.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020084700000000001 2024-09-04T17:46:36.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5", 2024-09-04T17:46:36.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.912632+0000", 2024-09-04T17:46:36.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000698899 2024-09-04T17:46:36.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:36.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.914653+0000", 2024-09-04T17:46:36.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020205790000000002 2024-09-04T17:46:36.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:46:36.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.914978+0000", 2024-09-04T17:46:36.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00032491399999999999 2024-09-04T17:46:36.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:36.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.915107+0000", 2024-09-04T17:46:36.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012920499999999999 2024-09-04T17:46:36.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:36.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:34.915202+0000", 2024-09-04T17:46:36.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5676999999999999e-05 2024-09-04T17:46:36.803 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:36.803 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:36.803 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:36.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.803 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:32692 3.2 3:408b02f8:::rbd_object_map.1134b8120a27:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-04T17:46:36.804 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:35.034469+0000", 2024-09-04T17:46:36.804 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.673440321, 2024-09-04T17:46:36.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010215584999999999, 2024-09-04T17:46:36.804 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:36.804 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:36.804 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:36.804 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:36.804 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:36.804 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:36.804 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 32692 2024-09-04T17:46:36.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.804 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:36.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:36.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.034469+0000", 2024-09-04T17:46:36.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:36.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:36.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.034469+0000", 2024-09-04T17:46:36.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:36.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:36.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.034471+0000", 2024-09-04T17:46:36.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.787e-06 2024-09-04T17:46:36.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:36.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.034480+0000", 2024-09-04T17:46:36.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8759999999999994e-06 2024-09-04T17:46:36.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:36.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.034501+0000", 2024-09-04T17:46:36.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1809000000000001e-05 2024-09-04T17:46:36.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:36.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.034517+0000", 2024-09-04T17:46:36.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5602000000000001e-05 2024-09-04T17:46:36.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:36.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.037633+0000", 2024-09-04T17:46:36.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031161280000000001 2024-09-04T17:46:36.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:36.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.037796+0000", 2024-09-04T17:46:36.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016321200000000001 2024-09-04T17:46:36.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5", 2024-09-04T17:46:36.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.038390+0000", 2024-09-04T17:46:36.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00059376399999999999 2024-09-04T17:46:36.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:36.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.044388+0000", 2024-09-04T17:46:36.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059981870000000003 2024-09-04T17:46:36.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:46:36.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.044532+0000", 2024-09-04T17:46:36.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014401 2024-09-04T17:46:36.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:36.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.044616+0000", 2024-09-04T17:46:36.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3885000000000001e-05 2024-09-04T17:46:36.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:36.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.044685+0000", 2024-09-04T17:46:36.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8324999999999994e-05 2024-09-04T17:46:36.809 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:36.810 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:36.810 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:36.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.810 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:32693 3.2 3:408b02f8:::rbd_object_map.1134b8120a27:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-04T17:46:36.810 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:35.034542+0000", 2024-09-04T17:46:36.810 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.6733676420000001, 2024-09-04T17:46:36.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010268161, 2024-09-04T17:46:36.810 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:36.810 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:36.810 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:36.810 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:36.810 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:36.810 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:36.811 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 32693 2024-09-04T17:46:36.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.811 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:36.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:36.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.034542+0000", 2024-09-04T17:46:36.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:36.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:36.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.034542+0000", 2024-09-04T17:46:36.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:36.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:36.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.034543+0000", 2024-09-04T17:46:36.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.113e-06 2024-09-04T17:46:36.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:36.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.034550+0000", 2024-09-04T17:46:36.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8730000000000001e-06 2024-09-04T17:46:36.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:36.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.034568+0000", 2024-09-04T17:46:36.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8661999999999999e-05 2024-09-04T17:46:36.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:36.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.034582+0000", 2024-09-04T17:46:36.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3515e-05 2024-09-04T17:46:36.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:36.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.039678+0000", 2024-09-04T17:46:36.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050965250000000002 2024-09-04T17:46:36.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:36.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.039816+0000", 2024-09-04T17:46:36.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013797399999999999 2024-09-04T17:46:36.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5", 2024-09-04T17:46:36.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.040376+0000", 2024-09-04T17:46:36.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00056009199999999999 2024-09-04T17:46:36.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:36.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.044504+0000", 2024-09-04T17:46:36.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041272510000000002 2024-09-04T17:46:36.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:46:36.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.044698+0000", 2024-09-04T17:46:36.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019385299999999999 2024-09-04T17:46:36.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:36.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.044761+0000", 2024-09-04T17:46:36.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3860000000000002e-05 2024-09-04T17:46:36.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:36.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.044810+0000", 2024-09-04T17:46:36.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8442999999999997e-05 2024-09-04T17:46:36.816 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:36.816 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:36.816 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:36.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.816 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:32694 3.2 3:408b02f8:::rbd_object_map.1134b8120a27:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-04T17:46:36.816 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:35.034608+0000", 2024-09-04T17:46:36.816 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.6733010340000001, 2024-09-04T17:46:36.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011227350000000001, 2024-09-04T17:46:36.817 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:36.817 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:36.817 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:36.817 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:36.817 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:36.817 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:36.817 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 32694 2024-09-04T17:46:36.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.817 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:36.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:36.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.034608+0000", 2024-09-04T17:46:36.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:36.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:36.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.034608+0000", 2024-09-04T17:46:36.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:36.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:36.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.034610+0000", 2024-09-04T17:46:36.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4130000000000001e-06 2024-09-04T17:46:36.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:36.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.034621+0000", 2024-09-04T17:46:36.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1226999999999999e-05 2024-09-04T17:46:36.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:36.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.034640+0000", 2024-09-04T17:46:36.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8615e-05 2024-09-04T17:46:36.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:36.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.034654+0000", 2024-09-04T17:46:36.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4269e-05 2024-09-04T17:46:36.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:36.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.041563+0000", 2024-09-04T17:46:36.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0069096820000000003 2024-09-04T17:46:36.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:36.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.041861+0000", 2024-09-04T17:46:36.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00029740300000000001 2024-09-04T17:46:36.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5", 2024-09-04T17:46:36.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.043012+0000", 2024-09-04T17:46:36.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001150669 2024-09-04T17:46:36.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:46:36.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.045308+0000", 2024-09-04T17:46:36.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002296111 2024-09-04T17:46:36.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:36.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.045640+0000", 2024-09-04T17:46:36.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00033241100000000001 2024-09-04T17:46:36.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:36.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.045768+0000", 2024-09-04T17:46:36.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012837800000000001 2024-09-04T17:46:36.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:36.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.045836+0000", 2024-09-04T17:46:36.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7172e-05 2024-09-04T17:46:36.822 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:36.822 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:36.822 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:36.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.823 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:33552 3.7 3:e79ad1db:::rbd_data.11342e8f3384.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-04T17:46:36.823 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:35.999748+0000", 2024-09-04T17:46:36.823 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.70816119099999997, 2024-09-04T17:46:36.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045683823999999998, 2024-09-04T17:46:36.823 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:36.823 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:36.823 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:36.823 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:36.823 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:36.823 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:36.823 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 33552 2024-09-04T17:46:36.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.824 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:36.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:36.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999748+0000", 2024-09-04T17:46:36.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:36.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:36.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999748+0000", 2024-09-04T17:46:36.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:36.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:36.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999750+0000", 2024-09-04T17:46:36.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1299999999999999e-06 2024-09-04T17:46:36.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:36.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999759+0000", 2024-09-04T17:46:36.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4649999999999994e-06 2024-09-04T17:46:36.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:36.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999778+0000", 2024-09-04T17:46:36.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8833000000000001e-05 2024-09-04T17:46:36.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:36.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999786+0000", 2024-09-04T17:46:36.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7369999999999995e-06 2024-09-04T17:46:36.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:36.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999856+0000", 2024-09-04T17:46:36.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9247999999999998e-05 2024-09-04T17:46:36.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:36.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999973+0000", 2024-09-04T17:46:36.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000117921 2024-09-04T17:46:36.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 7", 2024-09-04T17:46:36.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.005955+0000", 2024-09-04T17:46:36.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059820560000000004 2024-09-04T17:46:36.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:46:36.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.045297+0000", 2024-09-04T17:46:36.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.039341650999999998 2024-09-04T17:46:36.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:36.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.045343+0000", 2024-09-04T17:46:36.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5899000000000002e-05 2024-09-04T17:46:36.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:36.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.045388+0000", 2024-09-04T17:46:36.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4601999999999997e-05 2024-09-04T17:46:36.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:36.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.045432+0000", 2024-09-04T17:46:36.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4282000000000001e-05 2024-09-04T17:46:36.829 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:36.829 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:36.829 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:36.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.829 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop_reply(client.4404.0:33552 3.7 e55/48 ondisk, result = 0)", 2024-09-04T17:46:36.829 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:36.033716+0000", 2024-09-04T17:46:36.829 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.67419293199999997, 2024-09-04T17:46:36.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011723193, 2024-09-04T17:46:36.829 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:36.829 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:36.829 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:36.830 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:36.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:36.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.033716+0000", 2024-09-04T17:46:36.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:36.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:36.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.033716+0000", 2024-09-04T17:46:36.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:36.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:36.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.033718+0000", 2024-09-04T17:46:36.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.818e-06 2024-09-04T17:46:36.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:36.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.033725+0000", 2024-09-04T17:46:36.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2339999999999997e-06 2024-09-04T17:46:36.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:36.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.033750+0000", 2024-09-04T17:46:36.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4358e-05 2024-09-04T17:46:36.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:36.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.033763+0000", 2024-09-04T17:46:36.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2923e-05 2024-09-04T17:46:36.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:36.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.045331+0000", 2024-09-04T17:46:36.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011568294 2024-09-04T17:46:36.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:36.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.045338+0000", 2024-09-04T17:46:36.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4549999999999998e-06 2024-09-04T17:46:36.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:36.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:36.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:36.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.045440+0000", 2024-09-04T17:46:36.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000101111 2024-09-04T17:46:36.833 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:36.833 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:36.833 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:36.833 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:36.833 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:36.833 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:36.835 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:46:36.835 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-04T17:46:36.836 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:46:36.836 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:46:36.836 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:46:36.836 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-04T17:46:36.837 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-04T17:46:36.837 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #15, image image_client.0-clone15 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent16 2024-09-04T17:46:36.837 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-04T17:46:36.838 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-04T17:46:36.838 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:36.832+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:36.841 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:46:36.842 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:46:36.863 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:46:36.863 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:46:36.908 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-04T17:46:36.908 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-04T17:46:36.935 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:46:36.936+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:46:36.954 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:46:36.954 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:46:36.978 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:36.979 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:36.995 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-04T17:46:37.040 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:37.040+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:37.138 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:37.134+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:37.162 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:37.162 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:37.169 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:37.169 INFO:teuthology.orchestra.run.smithi092.stdout:1866 write 0x9d80486 thru 0x9d84723 (0x429e bytes) 2024-09-04T17:46:37.177 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-04T17:46:37.182 INFO:teuthology.orchestra.run.smithi092.stdout:1869 read 0x5946711 thru 0x5954b40 (0xe430 bytes) 2024-09-04T17:46:37.182 INFO:teuthology.orchestra.run.smithi092.stdout:1870 read 0x90c288d thru 0x90cbc1e (0x9392 bytes) 2024-09-04T17:46:37.183 INFO:teuthology.orchestra.run.smithi092.stdout:1873 writesame 0xb46c78f thru 0xb473c68 data_size 0x1(0x74da bytes) 2024-09-04T17:46:37.227 INFO:teuthology.orchestra.run.smithi092.stdout:1875 read 0x12c0f42 thru 0x12c53df (0x449e bytes) 2024-09-04T17:46:37.229 INFO:teuthology.orchestra.run.smithi092.stdout:1876 writesame 0x588cd43 thru 0x589c37f data_size 0xd(0xf63d bytes) 2024-09-04T17:46:37.236 INFO:teuthology.orchestra.run.smithi092.stdout:1877 read 0x4c92ed4 thru 0x4c9a0a6 (0x71d3 bytes) 2024-09-04T17:46:37.239 INFO:teuthology.orchestra.run.smithi092.stdout:1878 trunc from 0xea0536f to 0xd18d491 2024-09-04T17:46:37.239 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:46:37.234+0000 7f505d642640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26446) UID: 0 2024-09-04T17:46:37.253 INFO:teuthology.orchestra.run.smithi092.stdout:1880 read 0x903aa5a thru 0x90417ba (0x6d61 bytes) 2024-09-04T17:46:37.254 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:37.254 INFO:teuthology.orchestra.run.smithi092.stdout:1882 read 0xa8c29fe thru 0xa8c546d (0x2a70 bytes) 2024-09-04T17:46:37.261 INFO:teuthology.orchestra.run.smithi092.stdout:1884 trunc from 0xd18d491 to 0x862849c 2024-09-04T17:46:37.287 INFO:teuthology.orchestra.run.smithi092.stdout:1885 read 0x6e9eedb thru 0x6ea95bf (0xa6e5 bytes) 2024-09-04T17:46:37.288 INFO:teuthology.orchestra.run.smithi092.stdout:1886 punch from 0x2cf338 to 0x2cf5a6, (0x26e bytes) 2024-09-04T17:46:37.301 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:37.301 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:37.302 INFO:teuthology.orchestra.run.smithi092.stdout:1887 trunc from 0x862849c to 0x7f031f 2024-09-04T17:46:37.316 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-04T17:46:37.326 INFO:teuthology.orchestra.run.smithi092.stdout:1888 read 0x3c6c6f thru 0x3cb746 (0x4ad8 bytes) 2024-09-04T17:46:37.328 INFO:teuthology.orchestra.run.smithi092.stdout:1889 read 0x7207a1 thru 0x722787 (0x1fe7 bytes) 2024-09-04T17:46:37.328 INFO:teuthology.orchestra.run.smithi092.stdout:1890 read 0x4f38f4 thru 0x4f996c (0x6079 bytes) 2024-09-04T17:46:37.328 INFO:teuthology.orchestra.run.smithi092.stdout:1891 read 0x2acd2b thru 0x2b0a4b (0x3d21 bytes) 2024-09-04T17:46:37.330 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:37.330 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:37.339 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:46:37.340+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:46:37.349 INFO:teuthology.orchestra.run.smithi092.stdout:1895 write 0x55ebfef thru 0x55f29d7 (0x69e9 bytes) 2024-09-04T17:46:37.357 INFO:teuthology.orchestra.run.smithi092.stdout:1897 writesame 0x40e4d2f thru 0x40e8990 data_size 0x3b(0x3c62 bytes) 2024-09-04T17:46:37.367 INFO:teuthology.orchestra.run.smithi092.stdout:1898 trunc from 0x55f29d8 to 0x1ddec07 2024-09-04T17:46:37.401 INFO:teuthology.orchestra.run.smithi092.stdout:1901 write 0x7875931 thru 0x7880703 (0xadd3 bytes) 2024-09-04T17:46:37.419 INFO:teuthology.orchestra.run.smithi092.stdout:1903 trunc from 0x7880704 to 0x2b9fe50 2024-09-04T17:46:37.425 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is unset 2024-09-04T17:46:37.440 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:37.440+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:37.442 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:37.442 INFO:teuthology.orchestra.run.smithi092.stdout:1905 punch from 0x20da9cb to 0x20e65ae, (0xbbe3 bytes) 2024-09-04T17:46:37.442 INFO:teuthology.orchestra.run.smithi092.stdout:1906 read 0x28507df thru 0x2855016 (0x4838 bytes) 2024-09-04T17:46:37.442 INFO:teuthology.orchestra.run.smithi092.stdout:1908 punch from 0x19d8d5e to 0x19deb20, (0x5dc2 bytes) 2024-09-04T17:46:37.454 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:37.454 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:37.455 INFO:teuthology.orchestra.run.smithi092.stdout:1910 trunc from 0x2b9fe50 to 0xd8e1859 2024-09-04T17:46:37.463 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:37.464 INFO:teuthology.orchestra.run.smithi092.stdout:1912 clone 18 order 18 su 1024 sc 6 2024-09-04T17:46:37.469 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-04T17:46:37.541 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:46:37.538+0000 7f505d642640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26446) UID: 0 2024-09-04T17:46:37.630 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:37.630 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:37.643 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:37.642+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:37.644 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-04T17:46:37.744 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:37.744+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:37.766 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:37.766 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:37.780 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-04T17:46:37.844 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:37.846+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:37.945 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:37.945 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:37.945 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:46:37.946+0000 7f505d642640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26446) UID: 0 2024-09-04T17:46:37.959 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-04T17:46:38.046 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:38.044+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:38.077 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:38.077 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:38.092 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-04T17:46:38.147 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:46:38.146+0000 7f505d642640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26446) UID: 0 2024-09-04T17:46:38.248 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:46:38.248+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:46:38.265 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:38.265 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:38.279 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-04T17:46:38.349 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:46:38.346+0000 7f505d642640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26446) UID: 0 2024-09-04T17:46:38.411 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:38.411 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:38.426 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-04T17:46:38.468 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:46:38.466+0000 7f505d642640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26446) UID: 0 2024-09-04T17:46:38.551 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:38.546+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:38.597 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:38.597 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:38.612 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-04T17:46:38.645 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone17 from 0xd8e1859 (overlap 0x7f031f) to 0xcb383 2024-09-04T17:46:38.656 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:38.654+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:38.753 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:38.752+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:38.785 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:38.785 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:38.799 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-04T17:46:38.854 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:46:38.854+0000 7f505d642640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26446) UID: 0 2024-09-04T17:46:38.867 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #16, image image_client.0-clone16 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent17 2024-09-04T17:46:38.870 INFO:teuthology.orchestra.run.smithi092.stdout:1913 read 0x8abbf20 thru 0x8ac6d25 (0xae06 bytes) 2024-09-04T17:46:38.871 INFO:teuthology.orchestra.run.smithi092.stdout:1915 punch from 0x374c54f to 0x3754c3e, (0x86ef bytes) 2024-09-04T17:46:38.878 INFO:teuthology.orchestra.run.smithi092.stdout:1916 read 0x4b2a604 thru 0x4b3011a (0x5b17 bytes) 2024-09-04T17:46:38.878 INFO:teuthology.orchestra.run.smithi092.stdout:1917 writesame 0x89c463f thru 0x89c6594 data_size 0x2a(0x1f56 bytes) 2024-09-04T17:46:38.899 INFO:teuthology.orchestra.run.smithi092.stdout:1918 read 0xa739199 thru 0xa73992e (0x796 bytes) 2024-09-04T17:46:38.922 INFO:teuthology.orchestra.run.smithi092.stdout:1921 trunc from 0xd8e1859 to 0x585840a 2024-09-04T17:46:38.931 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:38.931 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:38.949 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-04T17:46:38.955 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:38.952+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:38.972 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-04T17:46:39.001 INFO:teuthology.orchestra.run.smithi092.stdout:1922 write 0xe798dfd thru 0xe7a52b8 (0xc4bc bytes) 2024-09-04T17:46:39.015 INFO:teuthology.orchestra.run.smithi092.stdout:truncating to largest ever: 0xe9adc77 2024-09-04T17:46:39.016 INFO:teuthology.orchestra.run.smithi092.stdout:1923 trunc from 0xe7a52b9 to 0xe9adc77 2024-09-04T17:46:39.023 INFO:teuthology.orchestra.run.smithi092.stdout:1925 write 0x910cb2c thru 0x9110d55 (0x422a bytes) 2024-09-04T17:46:39.036 INFO:teuthology.orchestra.run.smithi092.stdout:1926 writesame 0x72960cf thru 0x729badb data_size 0x1(0x5a0d bytes) 2024-09-04T17:46:39.055 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:39.054+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:39.083 INFO:teuthology.orchestra.run.smithi092.stdout:1928 trunc from 0xe9adc77 to 0x142cc17 2024-09-04T17:46:39.136 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:39.136 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:39.138 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:46:39.138 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:46:39.150 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-04T17:46:39.151 INFO:teuthology.orchestra.run.smithi092.stdout:1929 writesame 0xd35ab74 thru 0xd364bd4 data_size 0x1(0xa061 bytes) 2024-09-04T17:46:39.156 INFO:teuthology.orchestra.run.smithi092.stdout:1930 trunc from 0xd364bd5 to 0x8e2034d 2024-09-04T17:46:39.157 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:46:39.157 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:46:39.165 INFO:teuthology.orchestra.run.smithi092.stdout:1931 punch from 0x8a684d1 to 0x8a74ee7, (0xca16 bytes) 2024-09-04T17:46:39.166 INFO:teuthology.orchestra.run.smithi092.stdout:1932 write 0x398c060 thru 0x399be4c (0xfded bytes) 2024-09-04T17:46:39.174 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-04T17:46:39.174 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-04T17:46:39.181 INFO:teuthology.orchestra.run.smithi092.stdout:1934 write 0x336eef8 thru 0x3370363 (0x146c bytes) 2024-09-04T17:46:39.191 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-04T17:46:39.192 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-04T17:46:39.192 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:39.194+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:39.195 INFO:teuthology.orchestra.run.smithi092.stdout:1935 punch from 0x131c38d to 0x132c349, (0xffbc bytes) 2024-09-04T17:46:39.204 INFO:teuthology.orchestra.run.smithi092.stdout:1936 writesame 0xa0f378e thru 0xa0fb427 data_size 0x29(0x7c9a bytes) 2024-09-04T17:46:39.206 INFO:teuthology.orchestra.run.smithi092.stdout:1937 trunc from 0xa0fb428 to 0x9052880 2024-09-04T17:46:39.211 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:46:39.212 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:46:39.226 INFO:teuthology.orchestra.run.smithi092.stdout:1938 write 0xd47d7d7 thru 0xd484eb9 (0x76e3 bytes) 2024-09-04T17:46:39.230 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:46:39.231 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:46:39.241 INFO:teuthology.orchestra.run.smithi092.stdout:1940 write 0xd69a661 thru 0xd6a21ff (0x7b9f bytes) 2024-09-04T17:46:39.250 INFO:teuthology.orchestra.run.smithi092.stdout:1941 write 0x9060b05 thru 0x906de92 (0xd38e bytes) 2024-09-04T17:46:39.250 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-04T17:46:39.250 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-04T17:46:39.257 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:39.256+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:39.260 INFO:teuthology.orchestra.run.smithi092.stdout:1942 read 0xd54fc37 thru 0xd55c12f (0xc4f9 bytes) 2024-09-04T17:46:39.261 INFO:teuthology.orchestra.run.smithi092.stdout:1944 writesame 0x181c35f thru 0x18205eb data_size 0x3(0x428d bytes) 2024-09-04T17:46:39.269 INFO:teuthology.orchestra.run.smithi092.stdout:1945 read 0x3fc671e thru 0x3fc8884 (0x2167 bytes) 2024-09-04T17:46:39.269 INFO:teuthology.orchestra.run.smithi092.stdout:1947 write 0x5bda682 thru 0x5bdfe59 (0x57d8 bytes) 2024-09-04T17:46:39.270 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:46:39.270 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:46:39.283 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:39.283 INFO:teuthology.orchestra.run.smithi092.stdout:1949 read 0x6d31e17 thru 0x6d3c54f (0xa739 bytes) 2024-09-04T17:46:39.283 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:39.283 INFO:teuthology.orchestra.run.smithi092.stdout:1951 punch from 0x9b1fb92 to 0x9b221df, (0x264d bytes) 2024-09-04T17:46:39.283 INFO:teuthology.orchestra.run.smithi092.stdout:1953 punch from 0x29eea58 to 0x29f6828, (0x7dd0 bytes) 2024-09-04T17:46:39.283 INFO:teuthology.orchestra.run.smithi092.stdout:1954 read 0x98beb95 thru 0x98ccfa5 (0xe411 bytes) 2024-09-04T17:46:39.283 INFO:teuthology.orchestra.run.smithi092.stdout:1955 read 0x4b4309f thru 0x4b4fff6 (0xcf58 bytes) 2024-09-04T17:46:39.283 INFO:teuthology.orchestra.run.smithi092.stdout:1956 read 0xf26117 thru 0xf2a015 (0x3eff bytes) 2024-09-04T17:46:39.284 INFO:teuthology.orchestra.run.smithi092.stdout:1957 punch from 0x98ae950 to 0x98aedfd, (0x4ad bytes) 2024-09-04T17:46:39.284 INFO:teuthology.orchestra.run.smithi092.stdout:1958 write 0x7cb38a thru 0x7d25fa (0x7271 bytes) 2024-09-04T17:46:39.330 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:39.330 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:46:39.330 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:46:39.330 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:46:39.330 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.330 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8831 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2453504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:39.330 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:46:39.330 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 26.27054287, 2024-09-04T17:46:39.330 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14731111799999999, 2024-09-04T17:46:39.330 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:39.331 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:39.331 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:39.331 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:39.331 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:39.331 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:39.331 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8831 2024-09-04T17:46:39.331 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.331 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:39.331 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.331 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:39.331 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:46:39.331 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:39.331 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.332 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.332 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:39.332 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:46:39.332 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:39.332 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.332 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.332 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:39.332 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057811+0000", 2024-09-04T17:46:39.332 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.083e-06 2024-09-04T17:46:39.332 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.332 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.332 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:39.332 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057814+0000", 2024-09-04T17:46:39.332 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.407e-06 2024-09-04T17:46:39.333 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.333 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.333 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:39.333 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057825+0000", 2024-09-04T17:46:39.333 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.17e-05 2024-09-04T17:46:39.333 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.333 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.333 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:39.333 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057836+0000", 2024-09-04T17:46:39.333 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0283e-05 2024-09-04T17:46:39.333 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.333 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.333 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:39.334 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.204762+0000", 2024-09-04T17:46:39.334 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.146926205 2024-09-04T17:46:39.334 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.334 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.334 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:39.334 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.204895+0000", 2024-09-04T17:46:39.334 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013328999999999999 2024-09-04T17:46:39.334 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.334 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.334 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:39.334 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205121+0000", 2024-09-04T17:46:39.334 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00022614999999999999 2024-09-04T17:46:39.334 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:39.334 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:39.335 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:39.335 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.335 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.335 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8832 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2490368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:39.335 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:46:39.335 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 26.270504837000001, 2024-09-04T17:46:39.335 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14770012299999999, 2024-09-04T17:46:39.335 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:39.335 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:39.335 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:39.335 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:39.335 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:39.335 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:39.335 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8832 2024-09-04T17:46:39.336 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.336 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:39.336 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.336 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:39.336 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:46:39.336 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:39.336 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.336 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.336 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:39.336 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:46:39.336 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:39.336 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.336 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.337 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:39.337 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057849+0000", 2024-09-04T17:46:39.337 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6700000000000003e-07 2024-09-04T17:46:39.337 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.337 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.337 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:39.337 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057852+0000", 2024-09-04T17:46:39.337 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6359999999999998e-06 2024-09-04T17:46:39.338 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.338 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.338 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:39.338 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057860+0000", 2024-09-04T17:46:39.338 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8140000000000007e-06 2024-09-04T17:46:39.338 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.338 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.338 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:39.338 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057868+0000", 2024-09-04T17:46:39.338 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1999999999999997e-06 2024-09-04T17:46:39.338 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.338 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.338 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:39.339 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205208+0000", 2024-09-04T17:46:39.339 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14734067300000001 2024-09-04T17:46:39.339 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.339 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.339 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:39.339 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205347+0000", 2024-09-04T17:46:39.339 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013914900000000001 2024-09-04T17:46:39.339 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.339 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.339 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:39.339 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205548+0000", 2024-09-04T17:46:39.339 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000200984 2024-09-04T17:46:39.339 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:39.340 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:39.340 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:39.340 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.340 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.340 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8833 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2527232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:39.340 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:46:39.340 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 26.270472967, 2024-09-04T17:46:39.340 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148062572, 2024-09-04T17:46:39.340 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:39.340 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:39.340 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:39.340 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:39.340 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:39.340 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:39.341 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8833 2024-09-04T17:46:39.341 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.341 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:39.341 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.341 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:39.341 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:46:39.341 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:39.341 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.341 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.341 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:39.341 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:46:39.341 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:39.341 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.342 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.342 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:39.342 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057881+0000", 2024-09-04T17:46:39.342 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4300000000000002e-07 2024-09-04T17:46:39.342 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.342 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.342 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:39.342 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057893+0000", 2024-09-04T17:46:39.342 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1973999999999999e-05 2024-09-04T17:46:39.342 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.342 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.342 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:39.342 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057906+0000", 2024-09-04T17:46:39.343 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2716e-05 2024-09-04T17:46:39.343 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.343 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.343 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:39.343 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057913+0000", 2024-09-04T17:46:39.343 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3930000000000002e-06 2024-09-04T17:46:39.343 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.343 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.343 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:39.343 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205621+0000", 2024-09-04T17:46:39.343 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14770788800000001 2024-09-04T17:46:39.343 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.343 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.344 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:39.344 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205740+0000", 2024-09-04T17:46:39.344 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000118936 2024-09-04T17:46:39.344 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.344 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.344 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:39.344 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205943+0000", 2024-09-04T17:46:39.344 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020292199999999999 2024-09-04T17:46:39.344 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:39.344 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:39.344 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:39.344 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.344 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.345 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8834 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2564096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:39.345 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:46:39.345 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 26.270429688, 2024-09-04T17:46:39.345 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14846753200000001, 2024-09-04T17:46:39.345 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:39.345 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:39.345 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:39.345 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:39.345 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:39.345 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:39.345 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8834 2024-09-04T17:46:39.345 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.345 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:39.346 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.346 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:39.346 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:46:39.346 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:39.346 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.346 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.346 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:39.346 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:46:39.346 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:39.346 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.346 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.346 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:39.346 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057925+0000", 2024-09-04T17:46:39.347 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0270000000000001e-06 2024-09-04T17:46:39.347 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.347 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.347 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:39.347 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057936+0000", 2024-09-04T17:46:39.347 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.112e-05 2024-09-04T17:46:39.347 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.347 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.347 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:39.347 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057946+0000", 2024-09-04T17:46:39.347 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.009e-05 2024-09-04T17:46:39.347 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.348 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.348 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:39.348 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057953+0000", 2024-09-04T17:46:39.348 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.6229999999999997e-06 2024-09-04T17:46:39.348 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.348 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.348 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:39.348 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206027+0000", 2024-09-04T17:46:39.348 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148073591 2024-09-04T17:46:39.348 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.348 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.348 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:39.348 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206161+0000", 2024-09-04T17:46:39.349 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013411599999999999 2024-09-04T17:46:39.349 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.349 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.349 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:39.349 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206391+0000", 2024-09-04T17:46:39.349 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000229965 2024-09-04T17:46:39.349 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:39.349 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:39.349 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:39.349 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.349 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.349 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8835 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2600960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:39.349 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:46:39.350 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 26.270390037999999, 2024-09-04T17:46:39.350 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14882941599999999, 2024-09-04T17:46:39.350 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:39.350 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:39.350 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:39.350 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:39.350 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:39.350 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:39.350 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8835 2024-09-04T17:46:39.350 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.350 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:39.350 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.351 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:39.351 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:46:39.351 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:39.351 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.351 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.351 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:39.351 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:46:39.351 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:39.351 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.351 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.351 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:39.351 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058544+0000", 2024-09-04T17:46:39.351 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00058061899999999999 2024-09-04T17:46:39.351 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.352 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.352 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:39.352 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058550+0000", 2024-09-04T17:46:39.352 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.1129999999999997e-06 2024-09-04T17:46:39.352 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.352 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.352 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:39.352 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058566+0000", 2024-09-04T17:46:39.352 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5820000000000001e-05 2024-09-04T17:46:39.352 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.352 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.352 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:39.352 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058578+0000", 2024-09-04T17:46:39.353 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2537e-05 2024-09-04T17:46:39.353 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.353 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.353 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:39.353 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206470+0000", 2024-09-04T17:46:39.353 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14789153799999999 2024-09-04T17:46:39.353 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.353 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.353 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:39.353 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206589+0000", 2024-09-04T17:46:39.353 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000118837 2024-09-04T17:46:39.353 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.353 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.354 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:39.354 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206793+0000", 2024-09-04T17:46:39.354 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000203952 2024-09-04T17:46:39.354 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:39.354 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:39.354 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:39.354 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.354 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.354 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8836 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2654208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:39.354 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:46:39.354 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 26.269757349999999, 2024-09-04T17:46:39.354 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14862433999999999, 2024-09-04T17:46:39.354 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:39.354 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:39.355 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:39.355 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:39.355 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:39.355 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:39.355 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8836 2024-09-04T17:46:39.355 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.355 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:39.355 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.355 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:39.355 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:46:39.355 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:39.355 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.355 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.356 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:39.356 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:46:39.356 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:39.356 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.356 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.356 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:39.356 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062663+0000", 2024-09-04T17:46:39.356 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0040670619999999998 2024-09-04T17:46:39.356 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.356 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.356 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:39.356 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062668+0000", 2024-09-04T17:46:39.356 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4199999999999998e-06 2024-09-04T17:46:39.357 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.357 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.357 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:39.357 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062689+0000", 2024-09-04T17:46:39.357 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0356999999999998e-05 2024-09-04T17:46:39.357 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.357 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.357 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:39.357 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062726+0000", 2024-09-04T17:46:39.357 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7302999999999998e-05 2024-09-04T17:46:39.357 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.357 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.357 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:39.357 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206875+0000", 2024-09-04T17:46:39.358 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14414883100000001 2024-09-04T17:46:39.358 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.358 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.358 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:39.358 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.207016+0000", 2024-09-04T17:46:39.358 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000140822 2024-09-04T17:46:39.358 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.358 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.358 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:39.358 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.207220+0000", 2024-09-04T17:46:39.358 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020454499999999999 2024-09-04T17:46:39.358 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:39.358 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:39.358 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:39.359 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.359 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.359 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8844 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2928640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:39.359 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:46:39.359 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 26.265393267, 2024-09-04T17:46:39.359 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14752153100000001, 2024-09-04T17:46:39.359 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:39.359 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:39.359 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:39.359 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:39.359 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:39.359 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:39.359 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8844 2024-09-04T17:46:39.359 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.360 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:39.360 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.360 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:39.360 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:46:39.360 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:39.360 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.360 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.360 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:39.360 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:46:39.360 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:39.360 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.360 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.361 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:39.361 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062961+0000", 2024-09-04T17:46:39.361 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6700000000000003e-07 2024-09-04T17:46:39.361 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.361 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.361 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:39.361 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062973+0000", 2024-09-04T17:46:39.361 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2109999999999999e-05 2024-09-04T17:46:39.361 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.361 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.361 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:39.361 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062982+0000", 2024-09-04T17:46:39.361 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.6900000000000004e-06 2024-09-04T17:46:39.362 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.362 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.362 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:39.362 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062990+0000", 2024-09-04T17:46:39.362 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1899999999999998e-06 2024-09-04T17:46:39.362 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.362 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.362 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:39.362 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210090+0000", 2024-09-04T17:46:39.362 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14710026300000001 2024-09-04T17:46:39.362 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.362 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.362 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:39.362 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210225+0000", 2024-09-04T17:46:39.363 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013532400000000001 2024-09-04T17:46:39.363 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.363 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.363 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:39.363 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210481+0000", 2024-09-04T17:46:39.363 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00025628700000000001 2024-09-04T17:46:39.363 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:39.363 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:39.363 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:39.363 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.363 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.363 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8845 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2965504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:39.363 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:46:39.363 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 26.265353544, 2024-09-04T17:46:39.364 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147916819, 2024-09-04T17:46:39.364 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:39.364 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:39.364 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:39.364 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:39.364 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:39.364 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:39.364 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8845 2024-09-04T17:46:39.364 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.364 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:39.364 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.364 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:39.364 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:46:39.365 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:39.365 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.365 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.365 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:39.365 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:46:39.365 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:39.365 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.365 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.365 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:39.365 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:46:39.365 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.0399999999999995e-07 2024-09-04T17:46:39.365 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.365 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.366 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:39.366 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063003+0000", 2024-09-04T17:46:39.366 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6529999999999998e-06 2024-09-04T17:46:39.366 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.366 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.366 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:39.366 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063012+0000", 2024-09-04T17:46:39.366 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9500000000000007e-06 2024-09-04T17:46:39.366 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.366 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.366 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:39.366 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063027+0000", 2024-09-04T17:46:39.366 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5443000000000001e-05 2024-09-04T17:46:39.366 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.367 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.367 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:39.367 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210571+0000", 2024-09-04T17:46:39.367 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14754353100000001 2024-09-04T17:46:39.367 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.367 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.367 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:39.367 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210714+0000", 2024-09-04T17:46:39.367 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014295599999999999 2024-09-04T17:46:39.367 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.367 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.367 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:39.367 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210916+0000", 2024-09-04T17:46:39.367 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020258200000000001 2024-09-04T17:46:39.368 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:39.368 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:39.368 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:39.368 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.368 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.368 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8846 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3002368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:39.368 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:46:39.368 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 26.265316006999999, 2024-09-04T17:46:39.368 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14828011599999999, 2024-09-04T17:46:39.368 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:39.368 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:39.368 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:39.369 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:39.369 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:39.369 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:39.369 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8846 2024-09-04T17:46:39.369 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.369 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:39.369 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.369 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:39.369 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:46:39.369 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:39.369 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.369 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.369 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:39.370 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:46:39.370 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:39.370 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.370 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.370 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:39.370 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063038+0000", 2024-09-04T17:46:39.370 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.23e-07 2024-09-04T17:46:39.370 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.370 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.370 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:39.370 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063051+0000", 2024-09-04T17:46:39.370 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2757e-05 2024-09-04T17:46:39.371 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.371 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.371 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:39.371 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063070+0000", 2024-09-04T17:46:39.371 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9663000000000001e-05 2024-09-04T17:46:39.371 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.371 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.371 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:39.371 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063079+0000", 2024-09-04T17:46:39.371 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.0929999999999992e-06 2024-09-04T17:46:39.371 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.371 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.371 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:39.371 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210989+0000", 2024-09-04T17:46:39.372 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14790984500000001 2024-09-04T17:46:39.372 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.372 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.372 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:39.372 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211109+0000", 2024-09-04T17:46:39.372 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000120056 2024-09-04T17:46:39.372 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.372 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.372 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:39.373 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211317+0000", 2024-09-04T17:46:39.373 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020797900000000001 2024-09-04T17:46:39.373 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:39.373 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:39.373 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:39.373 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.373 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.373 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8847 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3039232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:39.373 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:46:39.373 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 26.265241575000001, 2024-09-04T17:46:39.373 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14859408900000001, 2024-09-04T17:46:39.373 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:39.374 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:39.374 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:39.374 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:39.374 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:39.374 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:39.374 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8847 2024-09-04T17:46:39.374 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.374 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:39.374 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.374 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:39.374 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:46:39.374 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:39.374 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.374 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.375 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:39.375 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:46:39.375 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:39.375 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.375 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.375 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:39.375 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063113+0000", 2024-09-04T17:46:39.375 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.204e-06 2024-09-04T17:46:39.375 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.375 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.375 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:39.375 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063116+0000", 2024-09-04T17:46:39.375 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.3330000000000001e-06 2024-09-04T17:46:39.375 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.376 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.376 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:39.376 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063126+0000", 2024-09-04T17:46:39.376 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0166999999999999e-05 2024-09-04T17:46:39.376 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.376 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.376 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:39.376 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063134+0000", 2024-09-04T17:46:39.376 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4329999999999998e-06 2024-09-04T17:46:39.376 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.376 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.376 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:39.376 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211392+0000", 2024-09-04T17:46:39.377 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148258109 2024-09-04T17:46:39.377 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.377 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.377 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:39.377 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211506+0000", 2024-09-04T17:46:39.377 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000113813 2024-09-04T17:46:39.377 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.377 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.377 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:39.377 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211706+0000", 2024-09-04T17:46:39.377 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020002999999999999 2024-09-04T17:46:39.377 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:39.377 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:39.377 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:39.378 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.378 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.378 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8848 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:39.378 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:46:39.378 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 26.265209238000001, 2024-09-04T17:46:39.378 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14907967599999999, 2024-09-04T17:46:39.378 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:39.378 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:39.378 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:39.378 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:39.378 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:39.378 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:39.378 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8848 2024-09-04T17:46:39.379 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.379 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:39.379 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.379 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:39.379 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:46:39.379 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:39.379 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.379 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.379 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:39.379 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:46:39.379 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:39.379 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.379 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.379 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:39.380 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070589+0000", 2024-09-04T17:46:39.380 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.007445386 2024-09-04T17:46:39.380 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.380 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.380 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:39.380 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070600+0000", 2024-09-04T17:46:39.380 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0346999999999999e-05 2024-09-04T17:46:39.380 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.380 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.380 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:39.380 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070622+0000", 2024-09-04T17:46:39.380 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2289999999999998e-05 2024-09-04T17:46:39.381 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.381 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.381 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:39.381 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070636+0000", 2024-09-04T17:46:39.381 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4093e-05 2024-09-04T17:46:39.381 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.381 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.381 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:39.381 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211815+0000", 2024-09-04T17:46:39.381 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14117874699999999 2024-09-04T17:46:39.381 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.381 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.381 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:39.382 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211983+0000", 2024-09-04T17:46:39.382 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00016852800000000001 2024-09-04T17:46:39.382 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.382 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.382 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:39.382 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.212224+0000", 2024-09-04T17:46:39.382 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024028499999999999 2024-09-04T17:46:39.382 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:39.382 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:39.382 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:39.383 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.383 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.383 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8862 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2019328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:39.384 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:46:39.384 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 26.257159770000001, 2024-09-04T17:46:39.384 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14755236399999999, 2024-09-04T17:46:39.384 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:39.384 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:39.384 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:39.384 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:39.384 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:39.384 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:39.384 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8862 2024-09-04T17:46:39.384 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.384 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:39.384 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.385 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:39.385 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:46:39.385 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:39.385 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.385 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.385 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:39.385 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:46:39.385 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:39.385 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.385 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.385 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:39.385 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071194+0000", 2024-09-04T17:46:39.385 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.5000000000000002e-07 2024-09-04T17:46:39.386 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.386 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.386 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:39.386 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071197+0000", 2024-09-04T17:46:39.386 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5900000000000002e-06 2024-09-04T17:46:39.386 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.386 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.386 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:39.386 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071206+0000", 2024-09-04T17:46:39.386 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9700000000000005e-06 2024-09-04T17:46:39.386 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.386 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.386 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:39.387 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071213+0000", 2024-09-04T17:46:39.387 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.9070000000000001e-06 2024-09-04T17:46:39.387 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.387 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.387 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:39.387 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218396+0000", 2024-09-04T17:46:39.387 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147183273 2024-09-04T17:46:39.387 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.387 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.387 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:39.387 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218538+0000", 2024-09-04T17:46:39.387 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014194199999999999 2024-09-04T17:46:39.387 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.387 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.388 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:39.388 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218746+0000", 2024-09-04T17:46:39.388 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000208032 2024-09-04T17:46:39.388 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:39.388 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:39.388 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:39.388 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.388 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.388 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8863 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2056192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:39.388 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:46:39.388 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 26.25713146, 2024-09-04T17:46:39.388 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14792281099999999, 2024-09-04T17:46:39.388 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:39.389 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:39.389 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:39.389 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:39.389 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:39.389 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:39.389 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8863 2024-09-04T17:46:39.389 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.389 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:39.389 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.389 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:39.389 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:46:39.389 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:39.389 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.390 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.390 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:39.390 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:46:39.390 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:39.390 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.390 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.390 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:39.390 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:46:39.390 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1299999999999999e-07 2024-09-04T17:46:39.390 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.390 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.390 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:39.390 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071225+0000", 2024-09-04T17:46:39.391 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6199999999999999e-06 2024-09-04T17:46:39.391 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.391 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.391 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:39.391 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071234+0000", 2024-09-04T17:46:39.391 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8769999999999999e-06 2024-09-04T17:46:39.391 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.391 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.391 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:39.391 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071251+0000", 2024-09-04T17:46:39.391 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.738e-05 2024-09-04T17:46:39.391 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.391 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.391 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:39.392 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218798+0000", 2024-09-04T17:46:39.392 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14754645 2024-09-04T17:46:39.392 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.392 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.392 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:39.392 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218926+0000", 2024-09-04T17:46:39.392 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000128446 2024-09-04T17:46:39.392 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.392 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.392 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:39.392 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219145+0000", 2024-09-04T17:46:39.392 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021832500000000001 2024-09-04T17:46:39.392 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:39.392 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:39.393 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:39.393 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.393 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.393 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8864 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2093056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:39.393 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:46:39.393 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 26.257089883999999, 2024-09-04T17:46:39.393 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148277309, 2024-09-04T17:46:39.393 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:39.393 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:39.393 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:39.393 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:39.393 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:39.394 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:39.394 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8864 2024-09-04T17:46:39.394 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.394 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:39.394 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.394 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:39.394 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:46:39.394 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:39.394 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.394 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.394 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:39.395 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:46:39.395 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:39.395 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.395 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.395 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:39.395 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071264+0000", 2024-09-04T17:46:39.395 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3e-07 2024-09-04T17:46:39.395 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.395 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.395 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:39.395 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071267+0000", 2024-09-04T17:46:39.395 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0139999999999999e-06 2024-09-04T17:46:39.395 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.396 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.396 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:39.396 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071276+0000", 2024-09-04T17:46:39.396 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1689999999999998e-06 2024-09-04T17:46:39.396 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.396 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.396 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:39.396 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071283+0000", 2024-09-04T17:46:39.396 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2269999999999996e-06 2024-09-04T17:46:39.396 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.396 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.396 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:39.396 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219195+0000", 2024-09-04T17:46:39.397 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147911868 2024-09-04T17:46:39.397 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.397 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.397 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:39.397 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219324+0000", 2024-09-04T17:46:39.397 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00012835900000000001 2024-09-04T17:46:39.397 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.397 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.397 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:39.397 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219541+0000", 2024-09-04T17:46:39.397 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000216942 2024-09-04T17:46:39.397 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:39.397 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:39.398 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:39.398 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.398 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.398 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8865 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2109440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:39.398 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:46:39.398 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 26.257060423999999, 2024-09-04T17:46:39.398 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14865747300000001, 2024-09-04T17:46:39.398 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:39.398 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:39.398 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:39.398 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:39.398 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:39.398 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:39.399 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8865 2024-09-04T17:46:39.399 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.399 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:39.399 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.399 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:39.399 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:46:39.399 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:39.399 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.399 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.399 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:39.399 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:46:39.399 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:39.399 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.399 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.400 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:39.400 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:46:39.400 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7999999999999995e-07 2024-09-04T17:46:39.400 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.400 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.400 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:39.400 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071296+0000", 2024-09-04T17:46:39.400 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6900000000000001e-06 2024-09-04T17:46:39.400 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.400 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.400 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:39.400 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071305+0000", 2024-09-04T17:46:39.400 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.5499999999999995e-06 2024-09-04T17:46:39.401 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.401 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.401 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:39.401 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071312+0000", 2024-09-04T17:46:39.401 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1559999999999998e-06 2024-09-04T17:46:39.401 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.401 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.401 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:39.401 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219591+0000", 2024-09-04T17:46:39.401 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14827944900000001 2024-09-04T17:46:39.401 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.401 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.401 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:39.402 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219731+0000", 2024-09-04T17:46:39.402 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013931600000000001 2024-09-04T17:46:39.402 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.402 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.402 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:39.402 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219950+0000", 2024-09-04T17:46:39.402 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021963199999999999 2024-09-04T17:46:39.402 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:39.402 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:39.402 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:39.402 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.402 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.402 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8866 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2146304~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:39.403 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:46:39.403 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 26.257032191, 2024-09-04T17:46:39.403 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14906117899999999, 2024-09-04T17:46:39.403 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:39.403 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:39.403 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:39.403 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:39.403 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:39.403 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:39.403 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8866 2024-09-04T17:46:39.403 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.403 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:39.403 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.404 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:39.404 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:46:39.404 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:39.404 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.404 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.404 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:39.404 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:46:39.404 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:39.404 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.404 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.404 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:39.404 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071322+0000", 2024-09-04T17:46:39.404 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7999999999999995e-07 2024-09-04T17:46:39.405 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.405 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.405 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:39.405 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071338+0000", 2024-09-04T17:46:39.405 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5872999999999999e-05 2024-09-04T17:46:39.405 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.405 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.405 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:39.405 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071347+0000", 2024-09-04T17:46:39.405 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4669999999999993e-06 2024-09-04T17:46:39.405 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.405 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.405 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:39.405 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071354+0000", 2024-09-04T17:46:39.406 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2099999999999996e-06 2024-09-04T17:46:39.406 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.406 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.406 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:39.406 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220008+0000", 2024-09-04T17:46:39.406 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148653642 2024-09-04T17:46:39.406 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.406 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.406 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:39.406 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220155+0000", 2024-09-04T17:46:39.406 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014726600000000001 2024-09-04T17:46:39.406 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.406 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.407 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:39.407 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220382+0000", 2024-09-04T17:46:39.407 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00022704100000000001 2024-09-04T17:46:39.407 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:39.407 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:39.407 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:39.407 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.407 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.407 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8867 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2183168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:39.407 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:46:39.407 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 26.256988838000002, 2024-09-04T17:46:39.407 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.149445831, 2024-09-04T17:46:39.407 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:39.408 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:39.408 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:39.408 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:39.408 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:39.408 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:39.408 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8867 2024-09-04T17:46:39.408 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.408 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:39.408 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.408 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:39.408 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:46:39.408 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:39.408 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.409 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.409 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:39.409 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:46:39.409 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:39.409 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.409 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.409 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:39.409 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078653+0000", 2024-09-04T17:46:39.409 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0072890430000000003 2024-09-04T17:46:39.409 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.409 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.409 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:39.409 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078664+0000", 2024-09-04T17:46:39.410 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0307e-05 2024-09-04T17:46:39.410 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.410 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.410 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:39.410 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078690+0000", 2024-09-04T17:46:39.410 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6780000000000001e-05 2024-09-04T17:46:39.410 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.410 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.410 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:39.410 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078706+0000", 2024-09-04T17:46:39.410 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5676999999999999e-05 2024-09-04T17:46:39.410 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.410 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.411 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:39.411 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220467+0000", 2024-09-04T17:46:39.411 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14176103500000001 2024-09-04T17:46:39.411 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.411 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.411 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:39.411 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220595+0000", 2024-09-04T17:46:39.411 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000127427 2024-09-04T17:46:39.411 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.411 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.411 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:39.411 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220810+0000", 2024-09-04T17:46:39.411 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021556200000000001 2024-09-04T17:46:39.411 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:39.412 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:39.412 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:39.412 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.412 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.412 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8880 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2658304~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:39.412 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.079158+0000", 2024-09-04T17:46:39.412 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 26.249195559, 2024-09-04T17:46:39.412 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14718968499999999, 2024-09-04T17:46:39.412 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:39.412 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:39.412 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:39.412 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:39.412 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:39.413 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:39.413 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8880 2024-09-04T17:46:39.413 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.413 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:39.413 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.413 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:39.413 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079158+0000", 2024-09-04T17:46:39.413 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:39.413 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.413 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.413 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:39.413 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079158+0000", 2024-09-04T17:46:39.413 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:39.414 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.414 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.414 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:39.414 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079158+0000", 2024-09-04T17:46:39.414 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3e-07 2024-09-04T17:46:39.414 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.414 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.414 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:39.414 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079161+0000", 2024-09-04T17:46:39.415 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.57e-06 2024-09-04T17:46:39.415 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.415 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.415 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:39.415 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079170+0000", 2024-09-04T17:46:39.415 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.0170000000000002e-06 2024-09-04T17:46:39.415 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.415 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.415 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:39.415 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079177+0000", 2024-09-04T17:46:39.415 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.0530000000000001e-06 2024-09-04T17:46:39.415 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.415 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.416 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:39.416 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.225946+0000", 2024-09-04T17:46:39.416 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14676936500000001 2024-09-04T17:46:39.416 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.416 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.416 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:39.416 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226098+0000", 2024-09-04T17:46:39.416 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000151805 2024-09-04T17:46:39.416 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.416 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.416 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:39.416 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226347+0000", 2024-09-04T17:46:39.416 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024914499999999999 2024-09-04T17:46:39.417 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:39.417 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:39.417 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:39.417 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.417 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.417 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8881 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2695168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:39.417 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:46:39.417 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 26.249166768999999, 2024-09-04T17:46:39.417 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147543547, 2024-09-04T17:46:39.417 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:39.417 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:39.417 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:39.417 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:39.418 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:39.418 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:39.418 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8881 2024-09-04T17:46:39.418 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.418 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:39.418 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.418 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:39.418 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:46:39.418 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:39.418 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.418 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.418 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:39.418 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:46:39.419 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:39.419 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.419 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.419 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:39.419 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079187+0000", 2024-09-04T17:46:39.419 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7700000000000004e-07 2024-09-04T17:46:39.419 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.419 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.419 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:39.419 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079198+0000", 2024-09-04T17:46:39.419 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0849999999999999e-05 2024-09-04T17:46:39.419 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.419 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.420 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:39.420 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079207+0000", 2024-09-04T17:46:39.420 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5599999999999999e-06 2024-09-04T17:46:39.420 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.420 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.420 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:39.420 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079224+0000", 2024-09-04T17:46:39.420 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6387000000000001e-05 2024-09-04T17:46:39.420 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.420 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.420 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:39.420 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226414+0000", 2024-09-04T17:46:39.420 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14718975000000001 2024-09-04T17:46:39.420 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.421 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.421 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:39.421 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226534+0000", 2024-09-04T17:46:39.421 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000120288 2024-09-04T17:46:39.421 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.421 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.421 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:39.421 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226730+0000", 2024-09-04T17:46:39.421 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000196035 2024-09-04T17:46:39.421 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:39.421 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:39.421 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:39.422 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.422 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.422 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8882 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2732032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:39.422 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:46:39.422 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 26.249118554999999, 2024-09-04T17:46:39.422 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14795022399999999, 2024-09-04T17:46:39.422 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:39.422 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:39.422 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:39.422 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:39.422 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:39.422 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:39.422 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8882 2024-09-04T17:46:39.423 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.423 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:39.423 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.423 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:39.423 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:46:39.423 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:39.423 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.423 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.423 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:39.423 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:46:39.423 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:39.423 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.423 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.423 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:39.424 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086588+0000", 2024-09-04T17:46:39.424 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0073530959999999999 2024-09-04T17:46:39.424 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.424 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.424 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:39.424 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086598+0000", 2024-09-04T17:46:39.424 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0122999999999999e-05 2024-09-04T17:46:39.424 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.424 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.424 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:39.424 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086626+0000", 2024-09-04T17:46:39.424 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.7914000000000001e-05 2024-09-04T17:46:39.424 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.424 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.425 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:39.425 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086641+0000", 2024-09-04T17:46:39.425 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5270000000000001e-05 2024-09-04T17:46:39.425 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.425 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.425 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:39.425 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226801+0000", 2024-09-04T17:46:39.425 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.140160321 2024-09-04T17:46:39.425 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.425 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.425 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:39.425 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226951+0000", 2024-09-04T17:46:39.425 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014965200000000001 2024-09-04T17:46:39.426 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:39.426 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:39.426 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:39.426 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.227185+0000", 2024-09-04T17:46:39.426 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00023384799999999999 2024-09-04T17:46:39.426 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:39.426 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:39.426 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:39.426 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:39.426 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:39.426 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:39.428 INFO:teuthology.orchestra.run.smithi092.stdout:1959 write 0x71d6729 thru 0x71da6ed (0x3fc5 bytes) 2024-09-04T17:46:39.428 INFO:teuthology.orchestra.run.smithi092.stdout:1960 trunc from 0xd6a2200 to 0x3b31e8c 2024-09-04T17:46:39.428 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:39.428 INFO:teuthology.orchestra.run.smithi092.stdout:1963 writesame 0x4da4053 thru 0x4da9526 data_size 0x59(0x54d4 bytes) 2024-09-04T17:46:39.429 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-04T17:46:39.429 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:46:39.426+0000 7f505d642640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26446) UID: 0 2024-09-04T17:46:39.431 INFO:teuthology.orchestra.run.smithi092.stdout:1965 writesame 0x907a895 thru 0x9080252 data_size 0x15(0x59be bytes) 2024-09-04T17:46:39.431 INFO:teuthology.orchestra.run.smithi092.stdout:1966 punch from 0x275079e to 0x2754309, (0x3b6b bytes) 2024-09-04T17:46:39.438 INFO:teuthology.orchestra.run.smithi092.stdout:1967 trunc from 0x9080253 to 0x4c64d93 2024-09-04T17:46:39.447 INFO:teuthology.orchestra.run.smithi092.stdout:1968 writesame 0x261d965 thru 0x262c736 data_size 0x6(0xedd2 bytes) 2024-09-04T17:46:39.448 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-04T17:46:39.465 INFO:teuthology.orchestra.run.smithi092.stdout:1969 read 0x1cb33d0 thru 0x1cb3809 (0x43a bytes) 2024-09-04T17:46:39.465 INFO:teuthology.orchestra.run.smithi092.stdout:1970 read 0x43f673f thru 0x43f96a5 (0x2f67 bytes) 2024-09-04T17:46:39.466 INFO:teuthology.orchestra.run.smithi092.stdout:1971 punch from 0x1d97dba to 0x1d9c4d1, (0x4717 bytes) 2024-09-04T17:46:39.466 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:39.484 INFO:teuthology.orchestra.run.smithi092.stdout:1973 write 0x9a4be6c thru 0x9a59671 (0xd806 bytes) 2024-09-04T17:46:39.506 INFO:teuthology.orchestra.run.smithi092.stdout:1974 write 0xa87de6d thru 0xa88cd79 (0xef0d bytes) 2024-09-04T17:46:39.518 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:39.518 INFO:teuthology.orchestra.run.smithi092.stdout:1977 write 0x482f458 thru 0x483b273 (0xbe1c bytes) 2024-09-04T17:46:39.528 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:46:39.528+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:46:39.530 INFO:teuthology.orchestra.run.smithi092.stdout:1978 writesame 0x43af4ca thru 0x43be21c data_size 0x1d(0xed53 bytes) 2024-09-04T17:46:39.543 INFO:teuthology.orchestra.run.smithi092.stdout:1979 punch from 0x10f662d to 0x10fdbd9, (0x75ac bytes) 2024-09-04T17:46:39.544 INFO:teuthology.orchestra.run.smithi092.stdout:1981 writesame 0x36e5767 thru 0x36ed53a data_size 0x2(0x7dd4 bytes) 2024-09-04T17:46:39.561 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:39.561 INFO:teuthology.orchestra.run.smithi092.stdout:1983 read 0x270b43e thru 0x2713806 (0x83c9 bytes) 2024-09-04T17:46:39.576 INFO:teuthology.orchestra.run.smithi092.stdout:1984 writesame 0xe1631eb thru 0xe171bf2 data_size 0x4(0xea08 bytes) 2024-09-04T17:46:39.577 INFO:teuthology.orchestra.run.smithi092.stdout:1985 writesame 0x9899b98 thru 0x98a332b data_size 0x59(0x9794 bytes) 2024-09-04T17:46:39.589 INFO:teuthology.orchestra.run.smithi092.stdout:1986 write 0xa2fc87d thru 0xa30b302 (0xea86 bytes) 2024-09-04T17:46:39.601 INFO:teuthology.orchestra.run.smithi092.stdout:1987 read 0x9d990ee thru 0x9d9e2c1 (0x51d4 bytes) 2024-09-04T17:46:39.602 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:39.602 INFO:teuthology.orchestra.run.smithi092.stdout:1989 punch from 0xdbbf8b0 to 0xdbce7e0, (0xef30 bytes) 2024-09-04T17:46:39.602 INFO:teuthology.orchestra.run.smithi092.stdout:1991 write 0x62db6af thru 0x62e9968 (0xe2ba bytes) 2024-09-04T17:46:39.614 INFO:teuthology.orchestra.run.smithi092.stdout:1992 read 0xaaeec50 thru 0xaaf4ead (0x625e bytes) 2024-09-04T17:46:39.614 INFO:teuthology.orchestra.run.smithi092.stdout:1993 write 0x7db9d7 thru 0x7e76d9 (0xbd03 bytes) 2024-09-04T17:46:39.622 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:39.622 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:46:39.622 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:46:39.622 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:39.629 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:46:39.630+0000 7f505d642640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26446) UID: 0 2024-09-04T17:46:39.636 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-04T17:46:39.679 INFO:teuthology.orchestra.run.smithi092.stdout:1995 trunc from 0xe171bf3 to 0x8d90411 2024-09-04T17:46:39.702 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:39.702 INFO:teuthology.orchestra.run.smithi092.stdout:1997 read 0x624e413 thru 0x625123b (0x2e29 bytes) 2024-09-04T17:46:39.703 INFO:teuthology.orchestra.run.smithi092.stdout:1998 punch from 0x3524d85 to 0x352ea23, (0x9c9e bytes) 2024-09-04T17:46:39.718 INFO:teuthology.orchestra.run.smithi092.stdout:1999 trunc from 0x8d90411 to 0x2297961 2024-09-04T17:46:39.734 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:39.728+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:39.763 INFO:teuthology.orchestra.run.smithi092.stdout:2000 write 0x417e769 thru 0x418ad13 (0xc5ab bytes) 2024-09-04T17:46:39.777 INFO:teuthology.orchestra.run.smithi092.stdout:2001 read 0x6e5f9c thru 0x6eaaac (0x4b11 bytes) 2024-09-04T17:46:39.798 INFO:teuthology.orchestra.run.smithi092.stdout:2002 writesame 0xad4c7eb thru 0xad59ce9 data_size 0x1(0xd4ff bytes) 2024-09-04T17:46:39.798 INFO:teuthology.orchestra.run.smithi092.stdout:2003 write 0x1e14343 thru 0x1e1b1af (0x6e6d bytes) 2024-09-04T17:46:39.814 INFO:teuthology.orchestra.run.smithi092.stdout:2004 read 0x199c094 thru 0x19ab2f0 (0xf25d bytes) 2024-09-04T17:46:39.816 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:39.816 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:46:39.817 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:46:39.817 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:46:39.817 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:39.824 INFO:teuthology.orchestra.run.smithi092.stdout:2005 write 0xb4ff18b thru 0xb50ad97 (0xbc0d bytes) 2024-09-04T17:46:39.832 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-04T17:46:39.832 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:39.830+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:39.853 INFO:teuthology.orchestra.run.smithi092.stdout:2006 writesame 0xe99a9c5 thru 0xe9a48e7 data_size 0x1(0x9f23 bytes) 2024-09-04T17:46:39.853 INFO:teuthology.orchestra.run.smithi092.stdout:2008 writesame 0xe0ef50e thru 0xe0f59c9 data_size 0xe(0x64bc bytes) 2024-09-04T17:46:39.867 INFO:teuthology.orchestra.run.smithi092.stdout:2009 punch from 0xb70db66 to 0xb71b21b, (0xd6b5 bytes) 2024-09-04T17:46:39.868 INFO:teuthology.orchestra.run.smithi092.stdout:2010 punch from 0x6d0df7e to 0x6d1dd2c, (0xfdae bytes) 2024-09-04T17:46:39.878 INFO:teuthology.orchestra.run.smithi092.stdout:2012 trunc from 0xe9a48e8 to 0x891118b 2024-09-04T17:46:39.898 INFO:teuthology.orchestra.run.smithi092.stdout:2013 read 0x4efc354 thru 0x4f0669b (0xa348 bytes) 2024-09-04T17:46:39.913 INFO:teuthology.orchestra.run.smithi092.stdout:2014 trunc from 0x891118b to 0x803734 2024-09-04T17:46:39.932 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:39.932+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:39.969 INFO:teuthology.orchestra.run.smithi092.stdout:2015 write 0xac2f5b4 thru 0xac345fd (0x504a bytes) 2024-09-04T17:46:39.988 INFO:teuthology.orchestra.run.smithi092.stdout:2016 read 0x836a829 thru 0x83709c0 (0x6198 bytes) 2024-09-04T17:46:39.988 INFO:teuthology.orchestra.run.smithi092.stdout:2017 punch from 0x6f25917 to 0x6f34447, (0xeb30 bytes) 2024-09-04T17:46:39.988 INFO:teuthology.orchestra.run.smithi092.stdout:2018 write 0x64e77e3 thru 0x64ee138 (0x6956 bytes) 2024-09-04T17:46:40.011 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:40.011 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:46:40.011 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:46:40.011 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:46:40.012 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.012 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:13127 3.1e 3:7bbed1a3:::rbd_data.11345bd5898b.0000000000000003:a [read 331776~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:40.012 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:17.905384+0000", 2024-09-04T17:46:40.012 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.106499612, 2024-09-04T17:46:40.012 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.059561699000000003, 2024-09-04T17:46:40.012 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:40.012 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:40.012 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:40.012 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:40.012 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:40.012 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:40.012 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 13127 2024-09-04T17:46:40.012 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.013 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:40.013 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.013 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:40.013 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905384+0000", 2024-09-04T17:46:40.013 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.013 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.013 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.013 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:40.013 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905384+0000", 2024-09-04T17:46:40.013 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.013 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.013 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.013 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:40.014 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905385+0000", 2024-09-04T17:46:40.014 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.144e-06 2024-09-04T17:46:40.014 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.014 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.014 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:40.014 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905390+0000", 2024-09-04T17:46:40.014 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0289999999999996e-06 2024-09-04T17:46:40.014 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.014 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.014 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:40.014 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905407+0000", 2024-09-04T17:46:40.014 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7322000000000001e-05 2024-09-04T17:46:40.014 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.015 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.015 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:40.015 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905421+0000", 2024-09-04T17:46:40.015 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3309000000000001e-05 2024-09-04T17:46:40.015 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.015 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.015 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:40.015 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.964510+0000", 2024-09-04T17:46:40.015 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.059089147000000002 2024-09-04T17:46:40.015 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.016 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.016 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:40.016 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.964684+0000", 2024-09-04T17:46:40.016 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000174463 2024-09-04T17:46:40.016 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.016 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.016 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:40.017 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.964946+0000", 2024-09-04T17:46:40.017 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00026128499999999999 2024-09-04T17:46:40.017 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.017 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:40.017 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.017 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.017 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.017 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:13128 3.1e 3:7bbed1a3:::rbd_data.11345bd5898b.0000000000000003:a [read 368640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:40.018 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:17.905438+0000", 2024-09-04T17:46:40.018 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.106445267000002, 2024-09-04T17:46:40.018 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.060341850000000002, 2024-09-04T17:46:40.018 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:40.018 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:40.018 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:40.018 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:40.019 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:40.019 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:40.019 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 13128 2024-09-04T17:46:40.019 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.019 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:40.019 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.019 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:40.019 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905438+0000", 2024-09-04T17:46:40.019 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.020 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.020 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.020 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:40.020 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905438+0000", 2024-09-04T17:46:40.020 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.020 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.020 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.020 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:40.021 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905439+0000", 2024-09-04T17:46:40.021 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1599999999999999e-06 2024-09-04T17:46:40.021 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.021 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.021 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:40.021 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905458+0000", 2024-09-04T17:46:40.022 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8961e-05 2024-09-04T17:46:40.022 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.022 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.022 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:40.022 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905476+0000", 2024-09-04T17:46:40.022 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7861999999999999e-05 2024-09-04T17:46:40.022 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.023 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.023 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:40.023 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905489+0000", 2024-09-04T17:46:40.023 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3232e-05 2024-09-04T17:46:40.023 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.023 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.023 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:40.024 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.965026+0000", 2024-09-04T17:46:40.024 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.059536981000000003 2024-09-04T17:46:40.024 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.024 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.024 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:40.024 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.965252+0000", 2024-09-04T17:46:40.025 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000225462 2024-09-04T17:46:40.025 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.025 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.025 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:40.025 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.965780+0000", 2024-09-04T17:46:40.025 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00052819200000000003 2024-09-04T17:46:40.026 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.026 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:40.026 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.026 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.026 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.026 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:13129 3.1e 3:7bbed1a3:::rbd_data.11345bd5898b.0000000000000003:a [read 405504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:40.026 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:17.905510+0000", 2024-09-04T17:46:40.026 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.106373876999999, 2024-09-04T17:46:40.026 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.060788440999999999, 2024-09-04T17:46:40.026 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:40.026 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:40.027 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:40.027 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:40.027 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:40.027 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:40.027 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 13129 2024-09-04T17:46:40.027 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.027 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:40.027 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.027 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:40.027 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905510+0000", 2024-09-04T17:46:40.027 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.027 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.027 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.028 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:40.028 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905510+0000", 2024-09-04T17:46:40.028 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.028 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.028 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.028 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:40.028 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905511+0000", 2024-09-04T17:46:40.028 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1659999999999999e-06 2024-09-04T17:46:40.028 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.028 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.028 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:40.028 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905516+0000", 2024-09-04T17:46:40.029 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9899999999999997e-06 2024-09-04T17:46:40.029 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.029 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.029 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:40.029 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905534+0000", 2024-09-04T17:46:40.029 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7808e-05 2024-09-04T17:46:40.029 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.029 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.029 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:40.029 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905547+0000", 2024-09-04T17:46:40.029 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3135999999999999e-05 2024-09-04T17:46:40.029 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.029 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.030 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:40.030 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.965898+0000", 2024-09-04T17:46:40.030 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.060351162 2024-09-04T17:46:40.030 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.030 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.030 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:40.030 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.966051+0000", 2024-09-04T17:46:40.030 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00015318299999999999 2024-09-04T17:46:40.030 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.030 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.030 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:40.030 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.966298+0000", 2024-09-04T17:46:40.031 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024699599999999998 2024-09-04T17:46:40.031 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.031 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:40.031 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.031 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.031 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.031 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:13130 3.1e 3:7bbed1a3:::rbd_data.11345bd5898b.0000000000000003:a [read 442368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:40.031 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:17.905565+0000", 2024-09-04T17:46:40.031 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.106318301999998, 2024-09-04T17:46:40.031 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.061174443000000002, 2024-09-04T17:46:40.031 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:40.031 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:40.031 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:40.032 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:40.032 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:40.032 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:40.032 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 13130 2024-09-04T17:46:40.032 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.032 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:40.033 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.033 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:40.033 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905565+0000", 2024-09-04T17:46:40.033 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.033 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.033 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.033 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:40.033 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905565+0000", 2024-09-04T17:46:40.033 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.033 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.033 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.033 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:40.034 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905567+0000", 2024-09-04T17:46:40.034 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6300000000000001e-06 2024-09-04T17:46:40.034 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.034 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.034 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:40.034 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905572+0000", 2024-09-04T17:46:40.034 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9790000000000001e-06 2024-09-04T17:46:40.034 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.034 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.034 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:40.034 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905590+0000", 2024-09-04T17:46:40.034 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8031999999999999e-05 2024-09-04T17:46:40.034 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.035 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.035 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:40.035 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905602+0000", 2024-09-04T17:46:40.035 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2651e-05 2024-09-04T17:46:40.035 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.035 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.035 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:40.035 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.966373+0000", 2024-09-04T17:46:40.035 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.060770532000000002 2024-09-04T17:46:40.035 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.035 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.035 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:40.035 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.966514+0000", 2024-09-04T17:46:40.036 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014103 2024-09-04T17:46:40.036 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.036 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.036 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:40.036 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.966740+0000", 2024-09-04T17:46:40.036 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000225589 2024-09-04T17:46:40.036 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.036 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:40.036 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.036 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.036 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.036 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:13131 3.1e 3:7bbed1a3:::rbd_data.11345bd5898b.0000000000000003:a [read 479232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:40.037 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:17.905621+0000", 2024-09-04T17:46:40.037 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.106262678, 2024-09-04T17:46:40.037 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.061498005000000001, 2024-09-04T17:46:40.037 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:40.037 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:40.037 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:40.037 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:40.037 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:40.037 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:40.037 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 13131 2024-09-04T17:46:40.037 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.037 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:40.037 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.037 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:40.038 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905621+0000", 2024-09-04T17:46:40.038 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.038 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.038 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.038 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:40.038 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905621+0000", 2024-09-04T17:46:40.038 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.038 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.038 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.038 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:40.038 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905622+0000", 2024-09-04T17:46:40.038 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.313e-06 2024-09-04T17:46:40.038 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.039 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.039 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:40.039 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905627+0000", 2024-09-04T17:46:40.039 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0100000000000003e-06 2024-09-04T17:46:40.039 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.039 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.039 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:40.039 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905644+0000", 2024-09-04T17:46:40.039 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6918e-05 2024-09-04T17:46:40.039 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.039 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.039 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:40.039 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905657+0000", 2024-09-04T17:46:40.040 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2496e-05 2024-09-04T17:46:40.040 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.040 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.040 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:40.040 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.966805+0000", 2024-09-04T17:46:40.040 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.061148518999999998 2024-09-04T17:46:40.040 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.040 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.040 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:40.040 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.966920+0000", 2024-09-04T17:46:40.040 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00011513700000000001 2024-09-04T17:46:40.040 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.040 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.041 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:40.041 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.967119+0000", 2024-09-04T17:46:40.041 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000198612 2024-09-04T17:46:40.041 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.041 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:40.041 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.041 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.041 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.041 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:13132 3.1e 3:7bbed1a3:::rbd_data.11345bd5898b.0000000000000003:a [read 516096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:40.041 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:17.905674+0000", 2024-09-04T17:46:40.041 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.106209112999998, 2024-09-04T17:46:40.041 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.061794957999999997, 2024-09-04T17:46:40.041 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:40.042 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:40.042 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:40.042 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:40.042 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:40.042 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:40.042 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 13132 2024-09-04T17:46:40.042 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.042 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:40.042 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.042 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:40.042 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905674+0000", 2024-09-04T17:46:40.042 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.042 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.042 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.043 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:40.043 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905674+0000", 2024-09-04T17:46:40.043 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.043 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.043 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.043 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:40.043 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905676+0000", 2024-09-04T17:46:40.043 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1230000000000001e-06 2024-09-04T17:46:40.043 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.043 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.043 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:40.043 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905680+0000", 2024-09-04T17:46:40.044 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9169999999999997e-06 2024-09-04T17:46:40.044 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.044 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.044 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:40.044 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905699+0000", 2024-09-04T17:46:40.044 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9023999999999999e-05 2024-09-04T17:46:40.044 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.044 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.044 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:40.044 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905713+0000", 2024-09-04T17:46:40.044 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3115999999999999e-05 2024-09-04T17:46:40.044 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.044 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.044 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:40.045 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.967167+0000", 2024-09-04T17:46:40.045 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.061454323999999998 2024-09-04T17:46:40.045 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.045 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.045 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:40.045 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.967280+0000", 2024-09-04T17:46:40.045 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000113269 2024-09-04T17:46:40.045 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.045 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.045 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:40.045 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.967469+0000", 2024-09-04T17:46:40.045 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000189185 2024-09-04T17:46:40.045 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.046 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:40.046 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.046 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.046 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.046 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:13133 3.1e 3:7bbed1a3:::rbd_data.11345bd5898b.0000000000000003:a [read 806912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:40.046 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:17.905745+0000", 2024-09-04T17:46:40.046 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.10613884, 2024-09-04T17:46:40.046 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.062050424999999999, 2024-09-04T17:46:40.046 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:40.046 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:40.046 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:40.046 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:40.046 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:40.047 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:40.047 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 13133 2024-09-04T17:46:40.047 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.047 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:40.047 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.047 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:40.047 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905745+0000", 2024-09-04T17:46:40.047 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.047 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.047 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.047 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:40.048 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905745+0000", 2024-09-04T17:46:40.048 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.048 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.048 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.048 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:40.048 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905746+0000", 2024-09-04T17:46:40.048 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0839999999999999e-06 2024-09-04T17:46:40.048 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.048 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.048 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:40.048 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905762+0000", 2024-09-04T17:46:40.048 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5917999999999999e-05 2024-09-04T17:46:40.048 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.048 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.049 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:40.049 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905780+0000", 2024-09-04T17:46:40.049 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7830999999999999e-05 2024-09-04T17:46:40.049 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.049 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.049 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:40.049 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905804+0000", 2024-09-04T17:46:40.049 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4912000000000001e-05 2024-09-04T17:46:40.049 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.049 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.049 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:40.049 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.967515+0000", 2024-09-04T17:46:40.050 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.061710458000000003 2024-09-04T17:46:40.050 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.050 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.050 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:40.050 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.967619+0000", 2024-09-04T17:46:40.050 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00010409899999999999 2024-09-04T17:46:40.050 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.050 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.050 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:40.050 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.967795+0000", 2024-09-04T17:46:40.050 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00017612300000000001 2024-09-04T17:46:40.050 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.050 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:40.051 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.051 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.051 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.051 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:13134 3.1e 3:7bbed1a3:::rbd_data.11345bd5898b.0000000000000003:a [read 843776~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:40.051 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:17.905824+0000", 2024-09-04T17:46:40.051 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.106060004, 2024-09-04T17:46:40.051 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.062450675999999997, 2024-09-04T17:46:40.051 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:40.051 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:40.051 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:40.051 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:40.051 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:40.052 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:40.052 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 13134 2024-09-04T17:46:40.052 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.052 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:40.052 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.052 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:40.052 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905824+0000", 2024-09-04T17:46:40.052 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.052 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.052 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.052 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:40.052 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905824+0000", 2024-09-04T17:46:40.052 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.053 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.053 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.053 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:40.053 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905825+0000", 2024-09-04T17:46:40.053 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.717e-06 2024-09-04T17:46:40.053 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.053 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.053 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:40.053 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905830+0000", 2024-09-04T17:46:40.053 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2220000000000001e-06 2024-09-04T17:46:40.053 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.053 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.053 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:40.054 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905848+0000", 2024-09-04T17:46:40.054 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7635e-05 2024-09-04T17:46:40.054 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.054 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.054 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:40.054 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905873+0000", 2024-09-04T17:46:40.054 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5160999999999998e-05 2024-09-04T17:46:40.054 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.054 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.054 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:40.054 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.967894+0000", 2024-09-04T17:46:40.054 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.062020947999999999 2024-09-04T17:46:40.054 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.055 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.055 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:40.055 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.968038+0000", 2024-09-04T17:46:40.055 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000143442 2024-09-04T17:46:40.055 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.055 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.055 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:40.055 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.968274+0000", 2024-09-04T17:46:40.055 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00023655099999999999 2024-09-04T17:46:40.055 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.055 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:40.055 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.055 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.055 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.055 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:13135 3.1e 3:7bbed1a3:::rbd_data.11345bd5898b.0000000000000003:a [read 880640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:40.056 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:17.905893+0000", 2024-09-04T17:46:40.056 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.105990588000001, 2024-09-04T17:46:40.056 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.062733087000000007, 2024-09-04T17:46:40.056 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:40.056 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:40.056 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:40.056 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:40.056 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:40.056 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:40.056 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 13135 2024-09-04T17:46:40.056 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.056 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:40.056 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.057 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:40.057 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905893+0000", 2024-09-04T17:46:40.057 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.057 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.057 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.057 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:40.057 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905893+0000", 2024-09-04T17:46:40.057 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.057 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.057 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.057 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:40.057 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905895+0000", 2024-09-04T17:46:40.057 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6139999999999999e-06 2024-09-04T17:46:40.058 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.058 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.058 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:40.058 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905900+0000", 2024-09-04T17:46:40.058 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2360000000000004e-06 2024-09-04T17:46:40.058 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.058 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.058 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:40.058 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905917+0000", 2024-09-04T17:46:40.058 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7642000000000002e-05 2024-09-04T17:46:40.059 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.059 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.059 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:40.059 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905930+0000", 2024-09-04T17:46:40.059 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2914999999999999e-05 2024-09-04T17:46:40.059 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.059 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.059 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:40.059 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.968339+0000", 2024-09-04T17:46:40.059 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.062408533000000002 2024-09-04T17:46:40.059 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.059 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.059 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:40.060 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.968446+0000", 2024-09-04T17:46:40.060 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000107481 2024-09-04T17:46:40.060 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.060 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.060 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:40.060 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.968626+0000", 2024-09-04T17:46:40.060 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00017966600000000001 2024-09-04T17:46:40.060 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.060 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:40.061 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.061 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.061 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.061 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:13136 3.1e 3:7bbed1a3:::rbd_data.11345bd5898b.0000000000000003:a [read 917504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:40.061 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:17.905948+0000", 2024-09-04T17:46:40.061 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.105935952999999, 2024-09-04T17:46:40.061 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.063001171999999994, 2024-09-04T17:46:40.061 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:40.061 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:40.061 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:40.061 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:40.061 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:40.061 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:40.062 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 13136 2024-09-04T17:46:40.062 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.062 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:40.062 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.062 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:40.062 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905948+0000", 2024-09-04T17:46:40.062 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.062 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.062 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.062 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:40.062 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905948+0000", 2024-09-04T17:46:40.062 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.062 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.063 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.063 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:40.063 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905949+0000", 2024-09-04T17:46:40.063 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.364e-06 2024-09-04T17:46:40.063 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.063 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.063 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:40.063 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905954+0000", 2024-09-04T17:46:40.063 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.8720000000000001e-06 2024-09-04T17:46:40.063 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.063 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.063 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:40.064 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905972+0000", 2024-09-04T17:46:40.064 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7756000000000001e-05 2024-09-04T17:46:40.064 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.064 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.064 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:40.064 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.905984+0000", 2024-09-04T17:46:40.064 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2581e-05 2024-09-04T17:46:40.064 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.064 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.064 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:40.064 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.968668+0000", 2024-09-04T17:46:40.064 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.062683991999999994 2024-09-04T17:46:40.065 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.065 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.065 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:40.065 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.968773+0000", 2024-09-04T17:46:40.065 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000105027 2024-09-04T17:46:40.065 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.065 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.065 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:40.065 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.968949+0000", 2024-09-04T17:46:40.065 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00017558 2024-09-04T17:46:40.065 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.065 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:40.066 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.066 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.066 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.066 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:13137 3.1e 3:7bbed1a3:::rbd_data.11345bd5898b.0000000000000003:a [read 954368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:40.066 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:17.906005+0000", 2024-09-04T17:46:40.066 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.105878688000001, 2024-09-04T17:46:40.066 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.063260424999999995, 2024-09-04T17:46:40.066 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:40.066 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:40.066 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:40.066 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:40.066 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:40.066 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:40.067 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 13137 2024-09-04T17:46:40.067 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.067 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:40.067 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.067 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:40.067 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906005+0000", 2024-09-04T17:46:40.067 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.067 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.067 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.067 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:40.067 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906005+0000", 2024-09-04T17:46:40.067 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.068 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.068 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.068 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:40.068 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906006+0000", 2024-09-04T17:46:40.068 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.116e-06 2024-09-04T17:46:40.068 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.068 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.068 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:40.068 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906011+0000", 2024-09-04T17:46:40.068 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9429999999999999e-06 2024-09-04T17:46:40.068 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.068 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.068 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:40.069 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906028+0000", 2024-09-04T17:46:40.069 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6968000000000001e-05 2024-09-04T17:46:40.069 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.069 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.069 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:40.069 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906041+0000", 2024-09-04T17:46:40.069 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3269e-05 2024-09-04T17:46:40.069 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.069 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.069 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:40.069 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.968991+0000", 2024-09-04T17:46:40.069 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.062949450000000004 2024-09-04T17:46:40.069 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.070 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.070 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:40.070 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.969093+0000", 2024-09-04T17:46:40.070 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000102521 2024-09-04T17:46:40.070 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.070 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.070 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:40.070 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.969265+0000", 2024-09-04T17:46:40.070 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000172158 2024-09-04T17:46:40.070 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.070 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:40.070 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.070 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.071 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.071 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:13138 3.1e 3:7bbed1a3:::rbd_data.11345bd5898b.0000000000000003:a [read 991232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:40.071 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:17.906058+0000", 2024-09-04T17:46:40.071 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.105825017000001, 2024-09-04T17:46:40.071 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.063688720000000004, 2024-09-04T17:46:40.071 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:40.071 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:40.071 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:40.071 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:40.071 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:40.071 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:40.071 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 13138 2024-09-04T17:46:40.071 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.071 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:40.072 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.072 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:40.072 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906058+0000", 2024-09-04T17:46:40.072 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.072 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.072 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.072 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:40.072 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906058+0000", 2024-09-04T17:46:40.072 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.072 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.072 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.072 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:40.072 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906060+0000", 2024-09-04T17:46:40.072 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2100000000000001e-06 2024-09-04T17:46:40.073 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.073 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.073 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:40.073 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906065+0000", 2024-09-04T17:46:40.073 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.8670000000000002e-06 2024-09-04T17:46:40.073 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.073 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.073 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:40.073 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906082+0000", 2024-09-04T17:46:40.073 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7283999999999999e-05 2024-09-04T17:46:40.073 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.073 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.073 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:40.073 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906095+0000", 2024-09-04T17:46:40.074 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2763e-05 2024-09-04T17:46:40.074 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.074 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.074 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:40.074 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.969365+0000", 2024-09-04T17:46:40.074 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.063269912999999997 2024-09-04T17:46:40.074 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.074 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.074 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:40.074 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.969514+0000", 2024-09-04T17:46:40.074 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014945599999999999 2024-09-04T17:46:40.074 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.074 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.075 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:40.075 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.969747+0000", 2024-09-04T17:46:40.075 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00023322699999999999 2024-09-04T17:46:40.075 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.075 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:40.075 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.075 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.075 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.075 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:13139 3.1e 3:7bbed1a3:::rbd_data.11345bd5898b.0000000000000003:a [read 1028096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:40.075 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:17.906111+0000", 2024-09-04T17:46:40.075 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.105772332000001, 2024-09-04T17:46:40.075 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.063986543000000007, 2024-09-04T17:46:40.075 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:40.075 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:40.075 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:40.076 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:40.076 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:40.076 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:40.076 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 13139 2024-09-04T17:46:40.076 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.076 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:40.076 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.076 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:40.076 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906111+0000", 2024-09-04T17:46:40.076 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.076 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.076 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.076 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:40.076 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906111+0000", 2024-09-04T17:46:40.076 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.077 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.077 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.077 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:40.077 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906112+0000", 2024-09-04T17:46:40.077 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1370000000000001e-06 2024-09-04T17:46:40.077 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.077 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.077 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:40.077 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906118+0000", 2024-09-04T17:46:40.077 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.3460000000000001e-06 2024-09-04T17:46:40.077 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.077 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.078 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:40.078 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906135+0000", 2024-09-04T17:46:40.078 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6895e-05 2024-09-04T17:46:40.078 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.078 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.078 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:40.078 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906147+0000", 2024-09-04T17:46:40.078 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2578999999999999e-05 2024-09-04T17:46:40.078 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.078 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.078 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:40.078 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.969812+0000", 2024-09-04T17:46:40.078 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.063664539000000006 2024-09-04T17:46:40.078 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.079 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.079 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:40.079 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.969918+0000", 2024-09-04T17:46:40.079 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000106827 2024-09-04T17:46:40.079 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.079 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.079 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:40.079 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.970098+0000", 2024-09-04T17:46:40.079 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00017922000000000001 2024-09-04T17:46:40.079 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.079 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:40.079 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.079 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.079 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.079 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:13140 3.1e 3:7bbed1a3:::rbd_data.11345bd5898b.0000000000000003:a [read 1318912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:40.080 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:17.906165+0000", 2024-09-04T17:46:40.080 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.105718629999998, 2024-09-04T17:46:40.080 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.064276715999999998, 2024-09-04T17:46:40.080 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:40.080 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:40.080 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:40.080 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:40.080 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:40.080 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:40.080 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 13140 2024-09-04T17:46:40.080 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.080 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:40.080 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.081 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:40.081 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906165+0000", 2024-09-04T17:46:40.081 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.081 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.081 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.081 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:40.081 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906165+0000", 2024-09-04T17:46:40.081 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.081 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.081 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.081 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:40.081 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906166+0000", 2024-09-04T17:46:40.081 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.11e-06 2024-09-04T17:46:40.082 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.082 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.082 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:40.082 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906181+0000", 2024-09-04T17:46:40.082 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4918e-05 2024-09-04T17:46:40.082 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.082 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.082 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:40.082 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906198+0000", 2024-09-04T17:46:40.082 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7125e-05 2024-09-04T17:46:40.082 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.082 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.082 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:40.082 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906211+0000", 2024-09-04T17:46:40.083 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3169e-05 2024-09-04T17:46:40.083 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.083 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.083 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:40.083 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.970142+0000", 2024-09-04T17:46:40.083 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.063930511999999995 2024-09-04T17:46:40.083 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.083 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.083 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:40.083 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.970245+0000", 2024-09-04T17:46:40.083 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00010333800000000001 2024-09-04T17:46:40.083 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.084 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.084 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:40.084 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.970442+0000", 2024-09-04T17:46:40.084 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019654399999999999 2024-09-04T17:46:40.084 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.084 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:40.084 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.084 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.084 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.084 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:13141 3.1e 3:7bbed1a3:::rbd_data.11345bd5898b.0000000000000003:a [read 1355776~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:40.084 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:17.906252+0000", 2024-09-04T17:46:40.084 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.105631851999998, 2024-09-04T17:46:40.085 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.064674651, 2024-09-04T17:46:40.085 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:40.085 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:40.085 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:40.085 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:40.085 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:40.085 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:40.085 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 13141 2024-09-04T17:46:40.085 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.085 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:40.085 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.085 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:40.085 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906252+0000", 2024-09-04T17:46:40.086 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.086 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.086 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.086 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:40.086 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906252+0000", 2024-09-04T17:46:40.086 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.086 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.086 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.086 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:40.086 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906253+0000", 2024-09-04T17:46:40.086 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5600000000000001e-06 2024-09-04T17:46:40.086 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.086 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.087 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:40.087 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906259+0000", 2024-09-04T17:46:40.087 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7490000000000004e-06 2024-09-04T17:46:40.087 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.087 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.087 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:40.087 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906293+0000", 2024-09-04T17:46:40.087 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4204e-05 2024-09-04T17:46:40.087 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.087 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.087 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:40.087 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906307+0000", 2024-09-04T17:46:40.087 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3801000000000001e-05 2024-09-04T17:46:40.087 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.088 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.088 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:40.088 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.970535+0000", 2024-09-04T17:46:40.088 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.064228104999999994 2024-09-04T17:46:40.088 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.088 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.088 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:40.088 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.970692+0000", 2024-09-04T17:46:40.088 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00015668499999999999 2024-09-04T17:46:40.088 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.088 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.088 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:40.088 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.970926+0000", 2024-09-04T17:46:40.089 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00023454699999999999 2024-09-04T17:46:40.089 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.089 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:40.089 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.089 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.089 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.089 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:13142 3.1e 3:7bbed1a3:::rbd_data.11345bd5898b.0000000000000003:a [read 1392640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:40.089 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:17.906327+0000", 2024-09-04T17:46:40.089 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.105556096000001, 2024-09-04T17:46:40.089 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.064950401000000005, 2024-09-04T17:46:40.089 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:40.089 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:40.089 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:40.089 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:40.090 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:40.090 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:40.090 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 13142 2024-09-04T17:46:40.090 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.090 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:40.090 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.090 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:40.090 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906327+0000", 2024-09-04T17:46:40.090 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.090 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.090 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.090 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:40.090 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906327+0000", 2024-09-04T17:46:40.090 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.091 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.091 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.091 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:40.091 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906329+0000", 2024-09-04T17:46:40.091 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.33e-06 2024-09-04T17:46:40.091 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.091 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.091 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:40.091 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906334+0000", 2024-09-04T17:46:40.091 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4160000000000003e-06 2024-09-04T17:46:40.091 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.091 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.091 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:40.091 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906351+0000", 2024-09-04T17:46:40.092 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7328000000000001e-05 2024-09-04T17:46:40.092 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.092 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.092 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:40.092 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906379+0000", 2024-09-04T17:46:40.092 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.7923999999999999e-05 2024-09-04T17:46:40.092 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.092 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.092 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:40.092 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.970992+0000", 2024-09-04T17:46:40.092 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.064612554000000003 2024-09-04T17:46:40.092 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.092 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.093 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:40.093 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.971097+0000", 2024-09-04T17:46:40.093 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000105438 2024-09-04T17:46:40.093 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.093 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.093 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:40.093 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.971278+0000", 2024-09-04T17:46:40.093 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000180411 2024-09-04T17:46:40.093 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.093 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:40.093 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.093 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.093 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.094 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:13143 3.1e 3:7bbed1a3:::rbd_data.11345bd5898b.0000000000000003:a [read 1429504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:40.094 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:17.906399+0000", 2024-09-04T17:46:40.094 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.105484976, 2024-09-04T17:46:40.094 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.065281594999999998, 2024-09-04T17:46:40.094 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:40.094 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:40.094 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:40.094 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:40.094 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:40.094 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:40.094 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 13143 2024-09-04T17:46:40.094 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.094 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:40.095 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.095 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:40.095 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906399+0000", 2024-09-04T17:46:40.095 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.095 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.095 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.095 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:40.095 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906399+0000", 2024-09-04T17:46:40.095 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.095 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.095 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.095 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:40.095 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906400+0000", 2024-09-04T17:46:40.096 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.139e-06 2024-09-04T17:46:40.096 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.096 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.096 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:40.096 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906405+0000", 2024-09-04T17:46:40.096 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2299999999999999e-06 2024-09-04T17:46:40.096 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.096 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.096 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:40.096 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906422+0000", 2024-09-04T17:46:40.096 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6884999999999998e-05 2024-09-04T17:46:40.096 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.096 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.097 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:40.097 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906435+0000", 2024-09-04T17:46:40.097 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3339000000000001e-05 2024-09-04T17:46:40.097 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.097 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.097 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:40.097 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.971343+0000", 2024-09-04T17:46:40.097 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.064907555000000006 2024-09-04T17:46:40.097 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.097 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.097 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:40.097 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.971474+0000", 2024-09-04T17:46:40.097 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013157699999999999 2024-09-04T17:46:40.098 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.098 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.098 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:40.098 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.971680+0000", 2024-09-04T17:46:40.098 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020587000000000001 2024-09-04T17:46:40.098 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.098 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:40.098 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.098 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.098 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.098 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:13144 3.1e 3:7bbed1a3:::rbd_data.11345bd5898b.0000000000000003:a [read 1466368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:40.098 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:17.906452+0000", 2024-09-04T17:46:40.098 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.105431225, 2024-09-04T17:46:40.099 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.065571372000000003, 2024-09-04T17:46:40.099 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:40.099 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:40.099 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:40.099 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:40.099 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:40.099 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:40.099 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 13144 2024-09-04T17:46:40.099 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.099 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:40.099 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.099 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:40.099 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906452+0000", 2024-09-04T17:46:40.100 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.100 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.100 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.100 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:40.100 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906452+0000", 2024-09-04T17:46:40.100 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.100 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.100 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.100 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:40.100 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906453+0000", 2024-09-04T17:46:40.100 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.083e-06 2024-09-04T17:46:40.100 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.100 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.101 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:40.101 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906487+0000", 2024-09-04T17:46:40.101 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4084000000000001e-05 2024-09-04T17:46:40.101 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.101 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.101 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:40.101 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906505+0000", 2024-09-04T17:46:40.101 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7971000000000001e-05 2024-09-04T17:46:40.101 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.101 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.101 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:40.101 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906519+0000", 2024-09-04T17:46:40.101 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3526000000000001e-05 2024-09-04T17:46:40.102 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.102 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.102 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:40.102 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.971741+0000", 2024-09-04T17:46:40.102 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.065221836000000005 2024-09-04T17:46:40.102 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.102 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.102 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:40.102 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.971846+0000", 2024-09-04T17:46:40.102 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000105278 2024-09-04T17:46:40.102 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.102 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.102 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:40.102 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.972024+0000", 2024-09-04T17:46:40.102 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00017759399999999999 2024-09-04T17:46:40.103 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.103 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:40.103 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.103 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.103 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.103 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:13145 3.1e 3:7bbed1a3:::rbd_data.11345bd5898b.0000000000000003:a [read 1503232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:40.103 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:17.906538+0000", 2024-09-04T17:46:40.103 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.105345882999998, 2024-09-04T17:46:40.103 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.065800844999999997, 2024-09-04T17:46:40.103 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:40.103 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:40.103 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:40.103 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:40.103 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:40.104 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:40.104 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 13145 2024-09-04T17:46:40.104 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.104 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:40.104 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.104 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:40.104 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906538+0000", 2024-09-04T17:46:40.104 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.104 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.104 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.104 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:40.104 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906538+0000", 2024-09-04T17:46:40.104 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.104 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.105 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.105 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:40.105 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906539+0000", 2024-09-04T17:46:40.105 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6470000000000001e-06 2024-09-04T17:46:40.105 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.105 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.105 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:40.105 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906545+0000", 2024-09-04T17:46:40.105 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2390000000000002e-06 2024-09-04T17:46:40.105 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.105 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.105 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:40.105 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906562+0000", 2024-09-04T17:46:40.105 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7102e-05 2024-09-04T17:46:40.105 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.106 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.106 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:40.106 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906576+0000", 2024-09-04T17:46:40.106 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4758e-05 2024-09-04T17:46:40.106 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.106 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.106 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:40.106 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.972066+0000", 2024-09-04T17:46:40.106 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.065489503000000004 2024-09-04T17:46:40.106 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.106 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.106 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:40.106 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.972167+0000", 2024-09-04T17:46:40.107 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00010100200000000001 2024-09-04T17:46:40.107 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.107 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.107 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:40.107 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.972338+0000", 2024-09-04T17:46:40.107 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00017159400000000001 2024-09-04T17:46:40.107 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.107 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:40.107 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.107 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.107 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.107 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:13146 3.1e 3:7bbed1a3:::rbd_data.11345bd5898b.0000000000000003:a [read 1540096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:40.107 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:17.906595+0000", 2024-09-04T17:46:40.107 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.105288332000001, 2024-09-04T17:46:40.107 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.066220854999999995, 2024-09-04T17:46:40.108 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:40.108 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:40.108 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:40.108 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:40.108 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:40.108 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:40.108 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 13146 2024-09-04T17:46:40.108 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.108 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:40.108 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.108 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:40.108 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906595+0000", 2024-09-04T17:46:40.108 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.108 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.109 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.109 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:40.109 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906595+0000", 2024-09-04T17:46:40.109 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.109 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.109 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.109 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:40.109 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906597+0000", 2024-09-04T17:46:40.109 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6330000000000001e-06 2024-09-04T17:46:40.109 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.109 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.109 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:40.109 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906602+0000", 2024-09-04T17:46:40.110 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9529999999999998e-06 2024-09-04T17:46:40.110 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.110 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.110 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:40.110 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906619+0000", 2024-09-04T17:46:40.110 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7561999999999999e-05 2024-09-04T17:46:40.110 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.110 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.110 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:40.110 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.906632+0000", 2024-09-04T17:46:40.110 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2901999999999999e-05 2024-09-04T17:46:40.110 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.110 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.111 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:40.111 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.972443+0000", 2024-09-04T17:46:40.111 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.065810653999999996 2024-09-04T17:46:40.111 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.111 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.111 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:40.111 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.972586+0000", 2024-09-04T17:46:40.111 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014348599999999999 2024-09-04T17:46:40.111 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.111 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.111 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:40.111 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.972816+0000", 2024-09-04T17:46:40.112 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00022966499999999999 2024-09-04T17:46:40.112 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.112 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:40.112 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.112 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.112 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:40.112 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:40.114 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-04T17:46:40.115 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:46:40.114+0000 7f505d642640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26446) UID: 0 2024-09-04T17:46:40.115 INFO:teuthology.orchestra.run.smithi092.stdout:2019 trunc from 0xac345fe to 0x210babf 2024-09-04T17:46:40.115 INFO:teuthology.orchestra.run.smithi092.stdout:2021 trunc from 0x210babf to 0xacf121f 2024-09-04T17:46:40.115 INFO:teuthology.orchestra.run.smithi092.stdout:2022 read 0x663ef41 thru 0x664bd1e (0xcdde bytes) 2024-09-04T17:46:40.115 INFO:teuthology.orchestra.run.smithi092.stdout:2023 punch from 0x14508c2 to 0x145b7e4, (0xaf22 bytes) 2024-09-04T17:46:40.115 INFO:teuthology.orchestra.run.smithi092.stdout:truncating to largest ever: 0xee69976 2024-09-04T17:46:40.115 INFO:teuthology.orchestra.run.smithi092.stdout:2025 trunc from 0xacf121f to 0xee69976 2024-09-04T17:46:40.115 INFO:teuthology.orchestra.run.smithi092.stdout:2026 punch from 0x87cea40 to 0x87d5b11, (0x70d1 bytes) 2024-09-04T17:46:40.137 INFO:teuthology.orchestra.run.smithi092.stdout:2027 trunc from 0xee69976 to 0x4708455 2024-09-04T17:46:40.151 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:40.151 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:40.178 INFO:teuthology.orchestra.run.smithi092.stdout:2030 writesame 0xb9be623 thru 0xb9c593b data_size 0x47(0x7319 bytes) 2024-09-04T17:46:40.179 INFO:teuthology.orchestra.run.smithi092.stdout:2031 writesame 0xb96f325 thru 0xb97c4b9 data_size 0x1(0xd195 bytes) 2024-09-04T17:46:40.208 INFO:teuthology.orchestra.run.smithi092.stdout:2032 write 0x46ade86 thru 0x46b8b03 (0xac7e bytes) 2024-09-04T17:46:40.214 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:40.212+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:40.225 INFO:teuthology.orchestra.run.smithi092.stdout:2033 writesame 0x5afa88f thru 0x5b00102 data_size 0x6f(0x5874 bytes) 2024-09-04T17:46:40.267 INFO:teuthology.orchestra.run.smithi092.stdout:2034 trunc from 0xb9c593c to 0xff46be 2024-09-04T17:46:40.300 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:40.300 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:46:40.300 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:46:40.300 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:40.314 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-04T17:46:40.315 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:40.314+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:40.321 INFO:teuthology.orchestra.run.smithi092.stdout:2035 trunc from 0xff46be to 0xe563eec 2024-09-04T17:46:40.332 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:40.333 INFO:teuthology.orchestra.run.smithi092.stdout:2037 writesame 0x2bf0b40 thru 0x2bf57fc data_size 0x1(0x4cbd bytes) 2024-09-04T17:46:40.348 INFO:teuthology.orchestra.run.smithi092.stdout:2039 writesame 0x44776f6 thru 0x4484f10 data_size 0x1b(0xd81b bytes) 2024-09-04T17:46:40.359 INFO:teuthology.orchestra.run.smithi092.stdout:2040 write 0xcaa06f8 thru 0xcaa27ab (0x20b4 bytes) 2024-09-04T17:46:40.370 INFO:teuthology.orchestra.run.smithi092.stdout:2041 read 0x4a8ca6a thru 0x4a91a4f (0x4fe6 bytes) 2024-09-04T17:46:40.370 INFO:teuthology.orchestra.run.smithi092.stdout:2042 read 0x71b009e thru 0x71be51a (0xe47d bytes) 2024-09-04T17:46:40.370 INFO:teuthology.orchestra.run.smithi092.stdout:2047 write 0x813336d thru 0x8133f81 (0xc15 bytes) 2024-09-04T17:46:40.379 INFO:teuthology.orchestra.run.smithi092.stdout:2048 writesame 0x92f28fe thru 0x92f693f data_size 0x46(0x4042 bytes) 2024-09-04T17:46:40.415 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:46:40.414+0000 7f505d642640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26446) UID: 0 2024-09-04T17:46:40.428 INFO:teuthology.orchestra.run.smithi092.stdout:2050 trunc from 0xe563eec to 0xb2f95 2024-09-04T17:46:40.452 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:40.452 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:46:40.452 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:46:40.452 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:46:40.453 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:40.459 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is unset 2024-09-04T17:46:40.466 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 4, 5, 6, 7, 3] 2024-09-04T17:46:40.466 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-04T17:46:40.466 INFO:tasks.thrashosds.thrasher:Killing osd 6, live_osds are [0, 1, 2, 4, 5, 6, 7, 3] 2024-09-04T17:46:40.466 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-04T17:46:40.466 INFO:teuthology.orchestra.run:waiting for 300 2024-09-04T17:46:40.474 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-04T17:46:40.516 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:40.518+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:40.557 INFO:tasks.ceph.osd.6:Stopped 2024-09-04T17:46:40.558 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.6 2024-09-04T17:46:40.558 INFO:tasks.thrashosds.thrasher:remote for osd 6 is ubuntu@smithi190.front.sepia.ceph.com 2024-09-04T17:46:40.558 DEBUG:teuthology.orchestra.run.smithi190:> type ceph-objectstore-tool 2024-09-04T17:46:40.607 INFO:teuthology.orchestra.run.smithi190.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-04T17:46:40.608 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-6 --journal-path /var/lib/ceph/osd/ceph-6/journal --op list-pgs 2024-09-04T17:46:40.639 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:40.640 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:46:40.640 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:46:40.640 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:46:40.640 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.640 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:1430 3.f 3:f58475cb:::rbd_data.11346251f094.0000000000000074:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-04T17:46:40.640 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:02.778704+0000", 2024-09-04T17:46:40.640 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 37.860498634000002, 2024-09-04T17:46:40.640 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.058567801000000003, 2024-09-04T17:46:40.640 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:40.640 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:40.640 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:40.640 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:40.640 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:40.641 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:40.641 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 1430 2024-09-04T17:46:40.641 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.641 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:40.641 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.641 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:40.641 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.778704+0000", 2024-09-04T17:46:40.641 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.641 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.641 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.641 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:40.641 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.778704+0000", 2024-09-04T17:46:40.642 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.642 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.642 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.642 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:40.642 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.778707+0000", 2024-09-04T17:46:40.642 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.8329999999999998e-06 2024-09-04T17:46:40.642 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.642 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.642 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:40.642 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.778718+0000", 2024-09-04T17:46:40.642 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.152e-05 2024-09-04T17:46:40.643 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.643 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.643 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:40.643 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.778750+0000", 2024-09-04T17:46:40.643 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.1696999999999998e-05 2024-09-04T17:46:40.643 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.643 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.643 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:40.643 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.778765+0000", 2024-09-04T17:46:40.643 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4800000000000001e-05 2024-09-04T17:46:40.643 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.643 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.643 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:40.643 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.778894+0000", 2024-09-04T17:46:40.643 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00012928500000000001 2024-09-04T17:46:40.644 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.644 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.644 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:40.644 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.779030+0000", 2024-09-04T17:46:40.644 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013566899999999999 2024-09-04T17:46:40.644 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.644 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.644 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 4", 2024-09-04T17:46:40.644 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.788794+0000", 2024-09-04T17:46:40.644 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.009764478 2024-09-04T17:46:40.644 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.645 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.645 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:40.645 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.836969+0000", 2024-09-04T17:46:40.645 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.048174626999999998 2024-09-04T17:46:40.645 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.645 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.645 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:40.645 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.837090+0000", 2024-09-04T17:46:40.645 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000120627 2024-09-04T17:46:40.645 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.645 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.645 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:40.645 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.837195+0000", 2024-09-04T17:46:40.646 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000105206 2024-09-04T17:46:40.646 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.646 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.646 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:40.646 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.837272+0000", 2024-09-04T17:46:40.646 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.7058999999999995e-05 2024-09-04T17:46:40.646 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.646 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:40.647 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.647 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.647 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.647 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14369 3.d 3:b763de10:::rbd_data.11345bd5898b.0000000000000002:a [read 77824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:40.647 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.182808+0000", 2024-09-04T17:46:40.647 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.456394666000001, 2024-09-04T17:46:40.648 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042057997999999999, 2024-09-04T17:46:40.648 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:40.648 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:40.648 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:40.648 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:40.648 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:40.649 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:40.649 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14369 2024-09-04T17:46:40.649 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.649 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:40.649 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.649 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:40.649 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182808+0000", 2024-09-04T17:46:40.649 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.649 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.649 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.649 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:40.650 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182808+0000", 2024-09-04T17:46:40.650 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.650 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.650 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.650 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:40.650 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182809+0000", 2024-09-04T17:46:40.650 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.063e-06 2024-09-04T17:46:40.651 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.651 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.651 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:40.651 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182814+0000", 2024-09-04T17:46:40.651 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1900000000000003e-06 2024-09-04T17:46:40.651 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.651 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.651 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:40.652 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182832+0000", 2024-09-04T17:46:40.652 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7791999999999998e-05 2024-09-04T17:46:40.652 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.652 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.652 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:40.652 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182849+0000", 2024-09-04T17:46:40.652 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7265000000000002e-05 2024-09-04T17:46:40.652 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.652 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.652 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:40.653 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.224431+0000", 2024-09-04T17:46:40.653 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.041581448 2024-09-04T17:46:40.653 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.653 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.653 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:40.653 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.224615+0000", 2024-09-04T17:46:40.653 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00018389200000000001 2024-09-04T17:46:40.653 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.653 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.653 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:40.653 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.224866+0000", 2024-09-04T17:46:40.654 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00025134799999999998 2024-09-04T17:46:40.654 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.654 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:40.654 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.654 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.654 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.654 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14370 3.d 3:b763de10:::rbd_data.11345bd5898b.0000000000000002:a [read 114688~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:40.654 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.182867+0000", 2024-09-04T17:46:40.654 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.456335364000001, 2024-09-04T17:46:40.654 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042368301999999997, 2024-09-04T17:46:40.654 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:40.654 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:40.654 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:40.655 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:40.655 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:40.655 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:40.655 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14370 2024-09-04T17:46:40.655 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.655 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:40.655 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.655 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:40.655 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182867+0000", 2024-09-04T17:46:40.655 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.655 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.655 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.655 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:40.655 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182867+0000", 2024-09-04T17:46:40.656 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.656 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.656 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.656 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:40.656 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182868+0000", 2024-09-04T17:46:40.656 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.006e-06 2024-09-04T17:46:40.656 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.656 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.656 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:40.656 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182875+0000", 2024-09-04T17:46:40.656 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.3990000000000001e-06 2024-09-04T17:46:40.656 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.656 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.656 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:40.657 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182892+0000", 2024-09-04T17:46:40.657 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7096e-05 2024-09-04T17:46:40.657 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.657 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.657 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:40.657 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182905+0000", 2024-09-04T17:46:40.657 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3258000000000001e-05 2024-09-04T17:46:40.657 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.657 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.657 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:40.657 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.224926+0000", 2024-09-04T17:46:40.657 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042020891999999997 2024-09-04T17:46:40.657 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.658 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.658 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:40.658 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.225064+0000", 2024-09-04T17:46:40.658 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000138307 2024-09-04T17:46:40.658 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.658 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.658 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:40.658 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.225236+0000", 2024-09-04T17:46:40.658 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000171344 2024-09-04T17:46:40.658 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.658 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:40.658 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.658 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.659 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.659 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14377 3.d 3:b763de10:::rbd_data.11345bd5898b.0000000000000002:a [read 225280~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:40.659 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.184021+0000", 2024-09-04T17:46:40.659 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.455181757999998, 2024-09-04T17:46:40.659 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042298810999999999, 2024-09-04T17:46:40.659 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:40.659 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:40.659 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:40.659 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:40.659 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:40.660 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:40.660 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14377 2024-09-04T17:46:40.660 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.660 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:40.660 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.660 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:40.661 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184021+0000", 2024-09-04T17:46:40.661 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.661 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.661 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.661 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:40.661 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184021+0000", 2024-09-04T17:46:40.661 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.662 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.662 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.662 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:40.662 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184022+0000", 2024-09-04T17:46:40.662 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5099999999999999e-06 2024-09-04T17:46:40.662 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.662 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.662 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:40.662 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184028+0000", 2024-09-04T17:46:40.663 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.852e-06 2024-09-04T17:46:40.663 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.663 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.663 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:40.663 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184046+0000", 2024-09-04T17:46:40.663 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7822e-05 2024-09-04T17:46:40.663 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.663 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.663 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:40.663 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184059+0000", 2024-09-04T17:46:40.664 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3372e-05 2024-09-04T17:46:40.664 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.664 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.664 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:40.664 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.226009+0000", 2024-09-04T17:46:40.664 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.041949187999999998 2024-09-04T17:46:40.664 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.664 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.664 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:40.665 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.226142+0000", 2024-09-04T17:46:40.665 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013381299999999999 2024-09-04T17:46:40.665 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.665 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.665 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:40.665 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.226320+0000", 2024-09-04T17:46:40.665 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00017725400000000001 2024-09-04T17:46:40.665 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.665 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:40.665 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.665 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.665 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.665 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14381 3.d 3:b763de10:::rbd_data.11345bd5898b.0000000000000002:a [read 552960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:40.666 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.184135+0000", 2024-09-04T17:46:40.666 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.455067706000001, 2024-09-04T17:46:40.666 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042553818, 2024-09-04T17:46:40.666 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:40.666 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:40.666 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:40.666 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:40.666 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:40.666 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:40.666 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14381 2024-09-04T17:46:40.666 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.666 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:40.666 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.666 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:40.666 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184135+0000", 2024-09-04T17:46:40.667 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.667 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.667 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.667 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:40.667 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184135+0000", 2024-09-04T17:46:40.667 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.667 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.667 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.667 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:40.667 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184136+0000", 2024-09-04T17:46:40.667 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1200000000000001e-06 2024-09-04T17:46:40.667 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.667 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.668 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:40.668 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184142+0000", 2024-09-04T17:46:40.668 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4840000000000003e-06 2024-09-04T17:46:40.668 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.668 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.668 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:40.668 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184157+0000", 2024-09-04T17:46:40.668 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5727999999999999e-05 2024-09-04T17:46:40.668 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.668 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.668 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:40.668 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184194+0000", 2024-09-04T17:46:40.669 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7049999999999999e-05 2024-09-04T17:46:40.669 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.669 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.669 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:40.669 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.226376+0000", 2024-09-04T17:46:40.669 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042181933999999997 2024-09-04T17:46:40.669 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.669 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.669 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:40.669 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.226522+0000", 2024-09-04T17:46:40.669 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014602899999999999 2024-09-04T17:46:40.669 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.670 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.670 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:40.670 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.226689+0000", 2024-09-04T17:46:40.670 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00016647299999999999 2024-09-04T17:46:40.670 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.670 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:40.670 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.670 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.670 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.670 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14383 3.d 3:b763de10:::rbd_data.11345bd5898b.0000000000000002:a [read 589824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:40.671 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.184302+0000", 2024-09-04T17:46:40.671 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.454900707, 2024-09-04T17:46:40.671 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042760362000000003, 2024-09-04T17:46:40.671 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:40.671 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:40.671 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:40.671 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:40.671 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:40.671 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:40.671 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14383 2024-09-04T17:46:40.671 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.671 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:40.671 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.672 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:40.672 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184302+0000", 2024-09-04T17:46:40.672 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.672 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.672 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.672 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:40.672 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184302+0000", 2024-09-04T17:46:40.673 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.673 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.673 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.673 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:40.673 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184303+0000", 2024-09-04T17:46:40.673 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.206e-06 2024-09-04T17:46:40.674 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.674 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.674 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:40.674 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184340+0000", 2024-09-04T17:46:40.674 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7005999999999998e-05 2024-09-04T17:46:40.674 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.675 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.675 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:40.675 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184382+0000", 2024-09-04T17:46:40.675 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2345999999999997e-05 2024-09-04T17:46:40.675 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.675 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.675 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:40.675 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184397+0000", 2024-09-04T17:46:40.676 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4729000000000001e-05 2024-09-04T17:46:40.676 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.676 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.676 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:40.676 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.226749+0000", 2024-09-04T17:46:40.676 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042352278 2024-09-04T17:46:40.676 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.676 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.676 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:40.676 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.226889+0000", 2024-09-04T17:46:40.676 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000139376 2024-09-04T17:46:40.677 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.677 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.677 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:40.677 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.227062+0000", 2024-09-04T17:46:40.677 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00017342099999999999 2024-09-04T17:46:40.677 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.677 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:40.677 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.677 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.677 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.677 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14385 3.d 3:b763de10:::rbd_data.11345bd5898b.0000000000000002:a [read 626688~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:40.677 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.184481+0000", 2024-09-04T17:46:40.678 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.454722044, 2024-09-04T17:46:40.678 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042873850999999998, 2024-09-04T17:46:40.678 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:40.678 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:40.678 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:40.678 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:40.678 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:40.678 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:40.678 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14385 2024-09-04T17:46:40.678 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.678 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:40.679 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.679 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:40.679 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184481+0000", 2024-09-04T17:46:40.679 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.679 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.679 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.679 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:40.679 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184481+0000", 2024-09-04T17:46:40.679 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.679 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.679 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.679 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:40.680 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184482+0000", 2024-09-04T17:46:40.680 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2130000000000001e-06 2024-09-04T17:46:40.680 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.680 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.680 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:40.680 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184487+0000", 2024-09-04T17:46:40.680 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4260000000000002e-06 2024-09-04T17:46:40.680 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.680 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.680 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:40.681 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184505+0000", 2024-09-04T17:46:40.681 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7659000000000002e-05 2024-09-04T17:46:40.681 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.681 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.681 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:40.681 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184518+0000", 2024-09-04T17:46:40.681 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2982e-05 2024-09-04T17:46:40.681 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.681 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.681 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:40.681 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.227120+0000", 2024-09-04T17:46:40.681 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042601858999999999 2024-09-04T17:46:40.682 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.682 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.682 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:40.682 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.227196+0000", 2024-09-04T17:46:40.682 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.6540999999999998e-05 2024-09-04T17:46:40.682 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.682 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.682 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:40.682 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.227354+0000", 2024-09-04T17:46:40.682 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000158171 2024-09-04T17:46:40.682 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.682 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:40.682 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.683 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.683 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.683 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14389 3.d 3:b763de10:::rbd_data.11345bd5898b.0000000000000002:a [read 663552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:40.683 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.184555+0000", 2024-09-04T17:46:40.683 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.454647907999998, 2024-09-04T17:46:40.683 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043553640999999997, 2024-09-04T17:46:40.683 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:40.683 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:40.683 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:40.683 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:40.683 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:40.683 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:40.683 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14389 2024-09-04T17:46:40.684 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.684 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:40.684 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.684 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:40.684 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184555+0000", 2024-09-04T17:46:40.684 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.684 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.684 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.684 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:40.684 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184555+0000", 2024-09-04T17:46:40.684 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.684 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.684 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.685 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:40.685 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184556+0000", 2024-09-04T17:46:40.685 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.353e-06 2024-09-04T17:46:40.685 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.685 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.685 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:40.685 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184562+0000", 2024-09-04T17:46:40.685 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7899999999999996e-06 2024-09-04T17:46:40.685 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.685 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.686 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:40.686 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184580+0000", 2024-09-04T17:46:40.686 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7682000000000001e-05 2024-09-04T17:46:40.686 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.686 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.686 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:40.686 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184593+0000", 2024-09-04T17:46:40.686 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3625000000000001e-05 2024-09-04T17:46:40.686 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.686 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.686 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:40.686 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.227413+0000", 2024-09-04T17:46:40.687 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042819572 2024-09-04T17:46:40.687 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.687 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.687 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:40.687 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.227552+0000", 2024-09-04T17:46:40.687 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000138842 2024-09-04T17:46:40.687 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.687 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.687 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:40.687 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228108+0000", 2024-09-04T17:46:40.687 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00055677700000000005 2024-09-04T17:46:40.687 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.687 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:40.688 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.688 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.688 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.688 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14392 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:40.688 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.184613+0000", 2024-09-04T17:46:40.688 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.454589645999999, 2024-09-04T17:46:40.688 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043677522000000003, 2024-09-04T17:46:40.688 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:40.688 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:40.688 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:40.688 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:40.688 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:40.688 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:40.689 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14392 2024-09-04T17:46:40.689 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.689 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:40.689 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.689 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:40.689 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184613+0000", 2024-09-04T17:46:40.689 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.689 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.689 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.689 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:40.689 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184613+0000", 2024-09-04T17:46:40.690 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.690 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.690 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.690 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:40.690 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184614+0000", 2024-09-04T17:46:40.690 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.35e-06 2024-09-04T17:46:40.690 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.690 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.690 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:40.691 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184620+0000", 2024-09-04T17:46:40.691 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.5589999999999996e-06 2024-09-04T17:46:40.691 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.691 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.691 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:40.691 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184671+0000", 2024-09-04T17:46:40.691 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0615000000000002e-05 2024-09-04T17:46:40.691 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.691 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.691 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:40.691 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184684+0000", 2024-09-04T17:46:40.692 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3596e-05 2024-09-04T17:46:40.692 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.692 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.692 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:40.692 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228277+0000", 2024-09-04T17:46:40.692 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043592509000000002 2024-09-04T17:46:40.692 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.692 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.692 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:40.692 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228291+0000", 2024-09-04T17:46:40.692 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3893000000000001e-05 2024-09-04T17:46:40.693 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.693 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:40.693 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.693 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.693 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.693 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14395 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:40.693 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.184708+0000", 2024-09-04T17:46:40.693 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.454494454999999, 2024-09-04T17:46:40.693 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043643612999999998, 2024-09-04T17:46:40.693 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:40.693 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:40.693 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:40.693 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:40.693 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:40.694 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:40.694 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14395 2024-09-04T17:46:40.694 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.694 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:40.694 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.694 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:40.694 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184708+0000", 2024-09-04T17:46:40.694 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.694 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.695 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.695 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:40.695 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184708+0000", 2024-09-04T17:46:40.695 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.695 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.695 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.695 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:40.695 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184709+0000", 2024-09-04T17:46:40.695 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2359999999999999e-06 2024-09-04T17:46:40.695 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.695 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.695 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:40.695 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184716+0000", 2024-09-04T17:46:40.696 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.3500000000000002e-06 2024-09-04T17:46:40.696 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.696 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.696 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:40.696 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184733+0000", 2024-09-04T17:46:40.696 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6775000000000001e-05 2024-09-04T17:46:40.696 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.696 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.696 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:40.696 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184777+0000", 2024-09-04T17:46:40.696 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4183e-05 2024-09-04T17:46:40.696 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.696 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.696 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:40.697 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228339+0000", 2024-09-04T17:46:40.697 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043562030000000002 2024-09-04T17:46:40.697 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.697 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.697 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:40.697 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228352+0000", 2024-09-04T17:46:40.697 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3039e-05 2024-09-04T17:46:40.697 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.697 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:40.697 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.697 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.697 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.697 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14398 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:40.698 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.184847+0000", 2024-09-04T17:46:40.698 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.454355807999999, 2024-09-04T17:46:40.698 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043564139000000002, 2024-09-04T17:46:40.698 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:40.698 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:40.698 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:40.698 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:40.698 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:40.698 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:40.698 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14398 2024-09-04T17:46:40.698 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.698 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:40.698 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.699 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:40.699 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184847+0000", 2024-09-04T17:46:40.699 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.699 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.699 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.699 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:40.699 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184847+0000", 2024-09-04T17:46:40.699 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.699 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.699 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.699 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:40.699 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184848+0000", 2024-09-04T17:46:40.699 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.28e-06 2024-09-04T17:46:40.699 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.700 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.700 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:40.700 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184861+0000", 2024-09-04T17:46:40.700 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3125000000000001e-05 2024-09-04T17:46:40.700 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.700 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.700 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:40.700 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184884+0000", 2024-09-04T17:46:40.700 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2357999999999998e-05 2024-09-04T17:46:40.700 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.700 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.700 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:40.700 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184900+0000", 2024-09-04T17:46:40.700 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6872e-05 2024-09-04T17:46:40.701 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.701 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.701 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:40.701 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228398+0000", 2024-09-04T17:46:40.701 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043497313000000003 2024-09-04T17:46:40.701 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.701 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.701 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:40.701 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228411+0000", 2024-09-04T17:46:40.701 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3191e-05 2024-09-04T17:46:40.702 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.702 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:40.702 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.702 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.702 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.702 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14399 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:40.702 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.184925+0000", 2024-09-04T17:46:40.702 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.454277491999999, 2024-09-04T17:46:40.702 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043544799000000002, 2024-09-04T17:46:40.702 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:40.703 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:40.703 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:40.703 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:40.703 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:40.703 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:40.703 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14399 2024-09-04T17:46:40.703 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.703 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:40.704 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.704 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:40.704 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184925+0000", 2024-09-04T17:46:40.704 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.704 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.704 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.704 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:40.704 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184925+0000", 2024-09-04T17:46:40.704 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.705 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.705 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.705 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:40.705 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184927+0000", 2024-09-04T17:46:40.705 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7e-06 2024-09-04T17:46:40.705 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.705 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.705 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:40.705 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184949+0000", 2024-09-04T17:46:40.705 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1880999999999999e-05 2024-09-04T17:46:40.705 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.706 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.706 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:40.706 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184969+0000", 2024-09-04T17:46:40.706 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0758e-05 2024-09-04T17:46:40.706 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.706 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.706 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:40.706 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184983+0000", 2024-09-04T17:46:40.706 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3336000000000001e-05 2024-09-04T17:46:40.706 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.706 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.706 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:40.706 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228456+0000", 2024-09-04T17:46:40.706 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043472960999999997 2024-09-04T17:46:40.707 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.707 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.707 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:40.707 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228470+0000", 2024-09-04T17:46:40.707 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4163e-05 2024-09-04T17:46:40.707 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.707 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:40.707 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.707 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.707 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.707 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14400 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:40.707 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.185003+0000", 2024-09-04T17:46:40.707 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.454200056000001, 2024-09-04T17:46:40.707 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043532659000000001, 2024-09-04T17:46:40.707 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:40.707 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:40.708 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:40.708 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:40.708 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:40.708 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:40.708 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14400 2024-09-04T17:46:40.708 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.708 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:40.708 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.708 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:40.708 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185003+0000", 2024-09-04T17:46:40.708 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.709 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.709 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.709 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:40.709 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185003+0000", 2024-09-04T17:46:40.709 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.709 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.709 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.709 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:40.709 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185004+0000", 2024-09-04T17:46:40.709 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.294e-06 2024-09-04T17:46:40.709 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.709 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.710 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:40.710 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185010+0000", 2024-09-04T17:46:40.710 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7690000000000002e-06 2024-09-04T17:46:40.710 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.710 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.710 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:40.710 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185029+0000", 2024-09-04T17:46:40.710 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9100999999999999e-05 2024-09-04T17:46:40.710 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.710 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.710 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:40.710 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185043+0000", 2024-09-04T17:46:40.710 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3862e-05 2024-09-04T17:46:40.710 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.711 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.711 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:40.711 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228510+0000", 2024-09-04T17:46:40.711 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043467766999999997 2024-09-04T17:46:40.711 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.711 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.711 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:40.711 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228535+0000", 2024-09-04T17:46:40.711 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4865999999999999e-05 2024-09-04T17:46:40.711 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.711 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:40.711 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.711 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.711 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.712 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14401 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:40.712 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.185062+0000", 2024-09-04T17:46:40.712 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.454140791, 2024-09-04T17:46:40.712 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043556096000000002, 2024-09-04T17:46:40.712 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:40.712 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:40.712 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:40.713 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:40.713 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:40.713 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:40.713 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14401 2024-09-04T17:46:40.713 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.713 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:40.713 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.713 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:40.714 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185062+0000", 2024-09-04T17:46:40.714 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.714 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.714 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.714 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:40.714 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185062+0000", 2024-09-04T17:46:40.714 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.714 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.714 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.715 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:40.715 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185063+0000", 2024-09-04T17:46:40.715 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.409e-06 2024-09-04T17:46:40.715 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.715 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.715 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:40.716 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185069+0000", 2024-09-04T17:46:40.716 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.5400000000000003e-06 2024-09-04T17:46:40.716 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.716 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.716 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:40.716 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185087+0000", 2024-09-04T17:46:40.716 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8195000000000001e-05 2024-09-04T17:46:40.716 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.716 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.716 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:40.716 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185101+0000", 2024-09-04T17:46:40.716 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3769e-05 2024-09-04T17:46:40.717 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.717 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.717 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:40.717 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228604+0000", 2024-09-04T17:46:40.717 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043503537000000002 2024-09-04T17:46:40.717 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.718 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.718 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:40.718 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228618+0000", 2024-09-04T17:46:40.718 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3645999999999999e-05 2024-09-04T17:46:40.718 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.718 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:40.719 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.719 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.719 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.719 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14402 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:40.719 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.185121+0000", 2024-09-04T17:46:40.719 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.454081497000001, 2024-09-04T17:46:40.719 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043551542999999998, 2024-09-04T17:46:40.719 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:40.719 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:40.719 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:40.719 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:40.719 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:40.720 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:40.720 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14402 2024-09-04T17:46:40.720 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.720 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:40.720 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.720 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:40.720 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185121+0000", 2024-09-04T17:46:40.720 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.721 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.721 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.721 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:40.721 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185121+0000", 2024-09-04T17:46:40.721 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.721 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.721 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.721 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:40.721 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185122+0000", 2024-09-04T17:46:40.721 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1769999999999999e-06 2024-09-04T17:46:40.721 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.721 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.721 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:40.721 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185128+0000", 2024-09-04T17:46:40.722 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7790000000000001e-06 2024-09-04T17:46:40.722 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.722 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.722 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:40.722 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185146+0000", 2024-09-04T17:46:40.722 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8114999999999999e-05 2024-09-04T17:46:40.722 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.722 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.722 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:40.722 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185160+0000", 2024-09-04T17:46:40.722 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4158999999999999e-05 2024-09-04T17:46:40.723 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.723 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.723 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:40.723 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228659+0000", 2024-09-04T17:46:40.723 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043498715 2024-09-04T17:46:40.723 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.723 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.723 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:40.723 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228673+0000", 2024-09-04T17:46:40.723 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3597999999999999e-05 2024-09-04T17:46:40.723 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.724 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:40.724 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.724 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.724 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.724 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14403 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:40.724 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.185183+0000", 2024-09-04T17:46:40.724 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.454019449, 2024-09-04T17:46:40.724 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043541467, 2024-09-04T17:46:40.724 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:40.724 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:40.724 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:40.724 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:40.724 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:40.725 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:40.725 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14403 2024-09-04T17:46:40.725 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.725 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:40.725 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.725 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:40.725 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185183+0000", 2024-09-04T17:46:40.725 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.725 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.725 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.725 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:40.725 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185183+0000", 2024-09-04T17:46:40.725 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.725 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.726 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.726 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:40.726 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185184+0000", 2024-09-04T17:46:40.726 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.156e-06 2024-09-04T17:46:40.726 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.726 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.726 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:40.726 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185190+0000", 2024-09-04T17:46:40.726 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1800000000000004e-06 2024-09-04T17:46:40.726 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.726 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.726 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:40.726 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185207+0000", 2024-09-04T17:46:40.726 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7288000000000001e-05 2024-09-04T17:46:40.726 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.727 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.727 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:40.727 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185221+0000", 2024-09-04T17:46:40.727 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3729e-05 2024-09-04T17:46:40.727 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.727 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.727 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:40.727 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228711+0000", 2024-09-04T17:46:40.727 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043490595 2024-09-04T17:46:40.727 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.727 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.727 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:40.727 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228725+0000", 2024-09-04T17:46:40.727 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3519e-05 2024-09-04T17:46:40.727 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.728 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:40.728 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.728 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.728 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.728 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14404 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:40.728 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.185239+0000", 2024-09-04T17:46:40.728 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.453963310999999, 2024-09-04T17:46:40.728 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043542099000000001, 2024-09-04T17:46:40.728 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:40.728 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:40.728 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:40.728 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:40.728 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:40.728 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:40.729 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14404 2024-09-04T17:46:40.729 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.729 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:40.729 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.729 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:40.729 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185239+0000", 2024-09-04T17:46:40.729 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.729 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.729 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.729 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:40.729 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185239+0000", 2024-09-04T17:46:40.729 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.730 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.730 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.730 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:40.730 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185240+0000", 2024-09-04T17:46:40.730 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.173e-06 2024-09-04T17:46:40.730 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.730 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.730 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:40.730 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185246+0000", 2024-09-04T17:46:40.730 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7529999999999998e-06 2024-09-04T17:46:40.730 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.730 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.731 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:40.731 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185264+0000", 2024-09-04T17:46:40.731 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7872000000000001e-05 2024-09-04T17:46:40.731 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.731 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.731 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:40.731 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185278+0000", 2024-09-04T17:46:40.731 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3532e-05 2024-09-04T17:46:40.731 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.731 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.731 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:40.731 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228768+0000", 2024-09-04T17:46:40.731 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043490586999999997 2024-09-04T17:46:40.731 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.732 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.732 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:40.732 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228781+0000", 2024-09-04T17:46:40.732 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3182e-05 2024-09-04T17:46:40.732 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.732 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:40.732 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.732 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.732 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.732 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14405 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:40.732 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.185297+0000", 2024-09-04T17:46:40.732 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.453906119999999, 2024-09-04T17:46:40.732 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043535049999999999, 2024-09-04T17:46:40.733 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:40.733 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:40.733 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:40.733 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:40.733 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:40.733 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:40.733 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14405 2024-09-04T17:46:40.733 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.733 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:40.734 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.734 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:40.734 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185297+0000", 2024-09-04T17:46:40.734 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.734 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.734 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.734 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:40.734 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185297+0000", 2024-09-04T17:46:40.734 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.734 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.734 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.734 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:40.734 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185298+0000", 2024-09-04T17:46:40.735 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.22e-06 2024-09-04T17:46:40.735 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.735 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.735 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:40.735 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185308+0000", 2024-09-04T17:46:40.735 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0173000000000001e-05 2024-09-04T17:46:40.735 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.735 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.735 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:40.735 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185329+0000", 2024-09-04T17:46:40.735 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1087999999999999e-05 2024-09-04T17:46:40.735 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.735 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.736 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:40.736 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185345+0000", 2024-09-04T17:46:40.736 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6035000000000001e-05 2024-09-04T17:46:40.736 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.736 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.736 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:40.736 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228822+0000", 2024-09-04T17:46:40.736 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043477427999999999 2024-09-04T17:46:40.736 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.736 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.736 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:40.736 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228832+0000", 2024-09-04T17:46:40.736 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1060000000000006e-06 2024-09-04T17:46:40.737 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.737 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:40.737 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.737 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.737 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.737 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14411 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:40.737 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.186171+0000", 2024-09-04T17:46:40.737 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.453031296999999, 2024-09-04T17:46:40.737 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.04271581, 2024-09-04T17:46:40.737 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:40.737 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:40.738 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:40.738 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:40.738 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:40.738 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:40.738 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14411 2024-09-04T17:46:40.738 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.738 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:40.738 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.738 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:40.738 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.186171+0000", 2024-09-04T17:46:40.738 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.738 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.738 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.739 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:40.739 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.186171+0000", 2024-09-04T17:46:40.739 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.739 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.739 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.739 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:40.739 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.186173+0000", 2024-09-04T17:46:40.739 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.012e-06 2024-09-04T17:46:40.739 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.739 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.739 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:40.739 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.186183+0000", 2024-09-04T17:46:40.739 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.7729999999999994e-06 2024-09-04T17:46:40.739 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.740 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.740 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:40.740 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.186205+0000", 2024-09-04T17:46:40.740 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2280999999999999e-05 2024-09-04T17:46:40.740 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.740 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.740 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:40.740 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.186224+0000", 2024-09-04T17:46:40.740 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8695e-05 2024-09-04T17:46:40.740 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.740 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.740 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:40.740 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228873+0000", 2024-09-04T17:46:40.741 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042648798000000002 2024-09-04T17:46:40.741 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.741 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.741 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:40.741 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228887+0000", 2024-09-04T17:46:40.741 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4251e-05 2024-09-04T17:46:40.741 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.741 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:40.741 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.741 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.741 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.741 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:37359 3.f 3:f29e50a5:::rbd_data.1134c5a43465.0000000000000021:head [stat,write 62464~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-04T17:46:40.741 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.616887+0000", 2024-09-04T17:46:40.742 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 1.0223153810000001, 2024-09-04T17:46:40.742 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.054183104000000003, 2024-09-04T17:46:40.742 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:40.742 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:40.742 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:40.742 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:40.742 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:40.742 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:40.742 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 37359 2024-09-04T17:46:40.742 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.742 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:40.743 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.743 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:40.743 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.616887+0000", 2024-09-04T17:46:40.743 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.743 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.743 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.743 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:40.743 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.616887+0000", 2024-09-04T17:46:40.743 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:40.743 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.743 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.744 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:40.744 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.616889+0000", 2024-09-04T17:46:40.744 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.362e-06 2024-09-04T17:46:40.744 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.744 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.744 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:40.744 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.616904+0000", 2024-09-04T17:46:40.744 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.507e-05 2024-09-04T17:46:40.744 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.744 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.744 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:40.745 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.616920+0000", 2024-09-04T17:46:40.745 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6385999999999999e-05 2024-09-04T17:46:40.745 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.745 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.745 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:40.745 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.616929+0000", 2024-09-04T17:46:40.745 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8540000000000003e-06 2024-09-04T17:46:40.745 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.745 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.745 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:40.746 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.616983+0000", 2024-09-04T17:46:40.746 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4218999999999997e-05 2024-09-04T17:46:40.746 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.746 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.746 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:40.746 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617065+0000", 2024-09-04T17:46:40.746 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.1864000000000001e-05 2024-09-04T17:46:40.746 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.746 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.746 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 4", 2024-09-04T17:46:40.746 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617205+0000", 2024-09-04T17:46:40.747 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013962600000000001 2024-09-04T17:46:40.747 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.747 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.747 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:40.747 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.618670+0000", 2024-09-04T17:46:40.747 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.001465647 2024-09-04T17:46:40.747 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.747 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.747 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:40.747 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670962+0000", 2024-09-04T17:46:40.747 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.052292000999999998 2024-09-04T17:46:40.747 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.747 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.747 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:40.748 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671011+0000", 2024-09-04T17:46:40.748 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9036000000000003e-05 2024-09-04T17:46:40.748 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:40.748 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:40.748 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:40.748 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671070+0000", 2024-09-04T17:46:40.748 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.9039000000000002e-05 2024-09-04T17:46:40.748 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.748 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:40.748 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.748 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:40.748 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:40.748 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:40.749 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:40.638+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:40.750 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-04T17:46:40.792 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:40.794+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:40.852 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:40.852+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:40.883 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:40.884 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:46:40.884 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:46:40.884 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:40.897 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-04T17:46:40.988 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:40.988+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:41.054 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:41.054+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:41.062 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:41.062 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:46:41.062 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:46:41.063 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:46:41.063 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:41.077 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-04T17:46:41.168 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:41.168+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:41.182 INFO:teuthology.orchestra.run.smithi190.stdout:3.d 2024-09-04T17:46:41.182 INFO:teuthology.orchestra.run.smithi190.stdout:3.a 2024-09-04T17:46:41.182 INFO:teuthology.orchestra.run.smithi190.stdout:3.1e 2024-09-04T17:46:41.183 INFO:teuthology.orchestra.run.smithi190.stdout:3.10 2024-09-04T17:46:41.183 INFO:teuthology.orchestra.run.smithi190.stdout:2.7 2024-09-04T17:46:41.183 INFO:teuthology.orchestra.run.smithi190.stdout:2.6 2024-09-04T17:46:41.249 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:41.249 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:46:41.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:46:41.249 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:46:41.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.249 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:1095 3.1 3:86c08994:::rbd_data.11346251f094.000000000000006e:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-04T17:46:41.249 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:02.246406+0000", 2024-09-04T17:46:41.250 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 39.001182438000001, 2024-09-04T17:46:41.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.024871196000000002, 2024-09-04T17:46:41.250 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:41.250 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:41.250 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:41.250 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:41.250 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:41.250 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:41.251 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 1095 2024-09-04T17:46:41.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.251 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:41.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:41.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.246406+0000", 2024-09-04T17:46:41.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:41.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:41.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.246406+0000", 2024-09-04T17:46:41.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:41.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:41.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.246409+0000", 2024-09-04T17:46:41.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6599999999999999e-06 2024-09-04T17:46:41.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:41.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.246419+0000", 2024-09-04T17:46:41.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8840000000000006e-06 2024-09-04T17:46:41.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:41.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.246439+0000", 2024-09-04T17:46:41.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0088000000000001e-05 2024-09-04T17:46:41.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:41.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.246448+0000", 2024-09-04T17:46:41.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9339999999999995e-06 2024-09-04T17:46:41.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:41.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.246511+0000", 2024-09-04T17:46:41.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3727999999999997e-05 2024-09-04T17:46:41.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:41.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.246588+0000", 2024-09-04T17:46:41.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6143999999999995e-05 2024-09-04T17:46:41.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-04T17:46:41.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.250919+0000", 2024-09-04T17:46:41.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0043314529999999999 2024-09-04T17:46:41.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:41.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.271170+0000", 2024-09-04T17:46:41.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020251531999999999 2024-09-04T17:46:41.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:46:41.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.271186+0000", 2024-09-04T17:46:41.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5302e-05 2024-09-04T17:46:41.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:41.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.271230+0000", 2024-09-04T17:46:41.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4576000000000001e-05 2024-09-04T17:46:41.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:41.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.271277+0000", 2024-09-04T17:46:41.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6894999999999997e-05 2024-09-04T17:46:41.256 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:41.256 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:41.256 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:41.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.256 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6103 3.1f 3:fec7e474:::rbd_data.1134cfbce83d.0000000000000003:6 [read 1736704~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:41.256 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.421848+0000", 2024-09-04T17:46:41.257 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.825740340999999, 2024-09-04T17:46:41.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027313884, 2024-09-04T17:46:41.257 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:41.257 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:41.257 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:41.257 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:41.257 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:41.257 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:41.257 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6103 2024-09-04T17:46:41.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.257 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:41.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:41.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.421848+0000", 2024-09-04T17:46:41.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:41.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:41.258 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.421848+0000", 2024-09-04T17:46:41.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:41.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:41.258 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.421849+0000", 2024-09-04T17:46:41.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.144e-06 2024-09-04T17:46:41.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:41.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.421855+0000", 2024-09-04T17:46:41.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.003e-06 2024-09-04T17:46:41.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:41.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.421872+0000", 2024-09-04T17:46:41.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7193e-05 2024-09-04T17:46:41.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:41.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.421886+0000", 2024-09-04T17:46:41.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.331e-05 2024-09-04T17:46:41.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:41.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.444849+0000", 2024-09-04T17:46:41.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.022962755000000001 2024-09-04T17:46:41.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:41.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.444990+0000", 2024-09-04T17:46:41.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014129999999999999 2024-09-04T17:46:41.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:41.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.449162+0000", 2024-09-04T17:46:41.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041721789999999998 2024-09-04T17:46:41.262 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:41.262 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:41.263 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:41.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.263 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6140 3.17 3:ed419000:::rbd_data.1134cfbce83d.000000000000000b:6 [sparse-read 622592~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:41.263 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.422576+0000", 2024-09-04T17:46:41.264 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.825012753000003, 2024-09-04T17:46:41.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025138273999999999, 2024-09-04T17:46:41.264 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:41.264 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:41.264 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:41.265 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:41.265 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:41.265 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:41.265 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6140 2024-09-04T17:46:41.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.266 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:41.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:41.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422576+0000", 2024-09-04T17:46:41.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:41.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.267 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:41.267 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422576+0000", 2024-09-04T17:46:41.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:41.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:41.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422578+0000", 2024-09-04T17:46:41.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8870000000000001e-06 2024-09-04T17:46:41.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:41.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422587+0000", 2024-09-04T17:46:41.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5999999999999996e-06 2024-09-04T17:46:41.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:41.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422617+0000", 2024-09-04T17:46:41.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9339000000000001e-05 2024-09-04T17:46:41.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:41.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422636+0000", 2024-09-04T17:46:41.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9524000000000001e-05 2024-09-04T17:46:41.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:41.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.445586+0000", 2024-09-04T17:46:41.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.022950084999999999 2024-09-04T17:46:41.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:41.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.445788+0000", 2024-09-04T17:46:41.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000202029 2024-09-04T17:46:41.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:41.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.447714+0000", 2024-09-04T17:46:41.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00192581 2024-09-04T17:46:41.271 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:41.271 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:41.271 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:41.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.271 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6141 3.17 3:ed419000:::rbd_data.1134cfbce83d.000000000000000b:6 [sparse-read 983040~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:41.271 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.422660+0000", 2024-09-04T17:46:41.271 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.82492809, 2024-09-04T17:46:41.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025710566000000001, 2024-09-04T17:46:41.271 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:41.271 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:41.271 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:41.271 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:41.272 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:41.272 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:41.272 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6141 2024-09-04T17:46:41.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.272 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:41.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:41.272 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422660+0000", 2024-09-04T17:46:41.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:41.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:41.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422660+0000", 2024-09-04T17:46:41.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:41.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:41.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422662+0000", 2024-09-04T17:46:41.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.313e-06 2024-09-04T17:46:41.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:41.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422691+0000", 2024-09-04T17:46:41.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9207e-05 2024-09-04T17:46:41.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:41.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422711+0000", 2024-09-04T17:46:41.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0225999999999999e-05 2024-09-04T17:46:41.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:41.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422727+0000", 2024-09-04T17:46:41.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5982999999999999e-05 2024-09-04T17:46:41.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:41.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.447767+0000", 2024-09-04T17:46:41.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025039558 2024-09-04T17:46:41.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:41.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.447871+0000", 2024-09-04T17:46:41.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104287 2024-09-04T17:46:41.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:41.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.448371+0000", 2024-09-04T17:46:41.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00049999199999999999 2024-09-04T17:46:41.275 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:41.276 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:41.276 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:41.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.276 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6142 3.17 3:ed419000:::rbd_data.1134cfbce83d.000000000000000b:6 [sparse-read 1343488~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:41.276 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.422750+0000", 2024-09-04T17:46:41.276 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.824838858, 2024-09-04T17:46:41.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026566215000000001, 2024-09-04T17:46:41.276 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:41.277 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:41.277 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:41.277 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:41.277 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:41.277 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:41.277 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6142 2024-09-04T17:46:41.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.277 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:41.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:41.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422750+0000", 2024-09-04T17:46:41.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:41.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:41.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422750+0000", 2024-09-04T17:46:41.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:41.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:41.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422751+0000", 2024-09-04T17:46:41.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6700000000000001e-06 2024-09-04T17:46:41.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:41.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422757+0000", 2024-09-04T17:46:41.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9669999999999998e-06 2024-09-04T17:46:41.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:41.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422775+0000", 2024-09-04T17:46:41.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8073000000000001e-05 2024-09-04T17:46:41.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:41.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422789+0000", 2024-09-04T17:46:41.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3709999999999999e-05 2024-09-04T17:46:41.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:41.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.448420+0000", 2024-09-04T17:46:41.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025630627999999999 2024-09-04T17:46:41.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:41.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.448590+0000", 2024-09-04T17:46:41.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000170213 2024-09-04T17:46:41.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:41.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.449316+0000", 2024-09-04T17:46:41.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00072595400000000001 2024-09-04T17:46:41.282 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:41.282 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:41.282 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:41.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.282 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6143 3.17 3:ed419000:::rbd_data.1134cfbce83d.000000000000000b:6 [read 1736704~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:41.282 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.422807+0000", 2024-09-04T17:46:41.282 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.824781528000003, 2024-09-04T17:46:41.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027271169000000001, 2024-09-04T17:46:41.282 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:41.282 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:41.282 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:41.283 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:41.283 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:41.283 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:41.283 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6143 2024-09-04T17:46:41.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.283 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:41.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:41.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422807+0000", 2024-09-04T17:46:41.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:41.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:41.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422807+0000", 2024-09-04T17:46:41.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:41.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:41.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422809+0000", 2024-09-04T17:46:41.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6139999999999999e-06 2024-09-04T17:46:41.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:41.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422814+0000", 2024-09-04T17:46:41.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1800000000000004e-06 2024-09-04T17:46:41.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:41.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422831+0000", 2024-09-04T17:46:41.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7433000000000001e-05 2024-09-04T17:46:41.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:41.286 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422844+0000", 2024-09-04T17:46:41.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3216e-05 2024-09-04T17:46:41.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:41.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.449397+0000", 2024-09-04T17:46:41.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026552572 2024-09-04T17:46:41.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:41.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.449526+0000", 2024-09-04T17:46:41.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000129207 2024-09-04T17:46:41.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:41.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.450078+0000", 2024-09-04T17:46:41.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00055194700000000001 2024-09-04T17:46:41.289 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:41.289 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:41.290 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:41.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.290 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6203 3.1f 3:fec7e474:::rbd_data.1134cfbce83d.0000000000000003:6 [read 327680~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:41.290 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423445+0000", 2024-09-04T17:46:41.290 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.824143116000002, 2024-09-04T17:46:41.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027179207, 2024-09-04T17:46:41.291 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:41.291 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:41.291 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:41.291 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:41.291 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:41.291 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:41.291 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6203 2024-09-04T17:46:41.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.291 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:41.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:41.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423445+0000", 2024-09-04T17:46:41.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:41.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:41.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423445+0000", 2024-09-04T17:46:41.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:41.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:41.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423446+0000", 2024-09-04T17:46:41.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.077e-06 2024-09-04T17:46:41.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:41.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423452+0000", 2024-09-04T17:46:41.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.113e-06 2024-09-04T17:46:41.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:41.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423468+0000", 2024-09-04T17:46:41.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6685999999999999e-05 2024-09-04T17:46:41.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:41.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423481+0000", 2024-09-04T17:46:41.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2673999999999999e-05 2024-09-04T17:46:41.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:41.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.449779+0000", 2024-09-04T17:46:41.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026297838 2024-09-04T17:46:41.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:41.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.450020+0000", 2024-09-04T17:46:41.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024114399999999999 2024-09-04T17:46:41.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.296 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:41.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.450625+0000", 2024-09-04T17:46:41.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00060467500000000005 2024-09-04T17:46:41.296 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:41.296 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:41.296 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:41.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.297 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6204 3.1f 3:fec7e474:::rbd_data.1134cfbce83d.0000000000000003:6 [read 688128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:41.297 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423498+0000", 2024-09-04T17:46:41.297 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.824090179999999, 2024-09-04T17:46:41.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029262579, 2024-09-04T17:46:41.297 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:41.297 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:41.297 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:41.297 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:41.297 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:41.297 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:41.297 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6204 2024-09-04T17:46:41.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.297 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:41.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:41.298 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423498+0000", 2024-09-04T17:46:41.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:41.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:41.298 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423498+0000", 2024-09-04T17:46:41.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:41.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:41.298 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423500+0000", 2024-09-04T17:46:41.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.28e-06 2024-09-04T17:46:41.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:41.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423506+0000", 2024-09-04T17:46:41.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0059999999999998e-06 2024-09-04T17:46:41.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:41.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423524+0000", 2024-09-04T17:46:41.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8683999999999999e-05 2024-09-04T17:46:41.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:41.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423537+0000", 2024-09-04T17:46:41.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3030000000000001e-05 2024-09-04T17:46:41.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.300 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:41.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.450697+0000", 2024-09-04T17:46:41.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027159793000000002 2024-09-04T17:46:41.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.300 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:41.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.450835+0000", 2024-09-04T17:46:41.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013773799999999999 2024-09-04T17:46:41.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:41.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.452761+0000", 2024-09-04T17:46:41.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019260480000000001 2024-09-04T17:46:41.301 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:41.301 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:41.301 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:41.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.301 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6205 3.1f 3:fec7e474:::rbd_data.1134cfbce83d.0000000000000003:6 [read 1048576~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:41.301 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423555+0000", 2024-09-04T17:46:41.301 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.824033196999999, 2024-09-04T17:46:41.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029905979999999999, 2024-09-04T17:46:41.302 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:41.302 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:41.302 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:41.302 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:41.302 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:41.302 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:41.302 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6205 2024-09-04T17:46:41.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.302 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:41.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:41.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423555+0000", 2024-09-04T17:46:41.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:41.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.303 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:41.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423555+0000", 2024-09-04T17:46:41.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:41.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.303 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:41.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423556+0000", 2024-09-04T17:46:41.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1599999999999999e-06 2024-09-04T17:46:41.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.303 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:41.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423562+0000", 2024-09-04T17:46:41.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1270000000000002e-06 2024-09-04T17:46:41.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:41.304 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423579+0000", 2024-09-04T17:46:41.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6986e-05 2024-09-04T17:46:41.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:41.304 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423591+0000", 2024-09-04T17:46:41.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2544000000000001e-05 2024-09-04T17:46:41.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:41.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.452819+0000", 2024-09-04T17:46:41.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029227418000000002 2024-09-04T17:46:41.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:41.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.452985+0000", 2024-09-04T17:46:41.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000166675 2024-09-04T17:46:41.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:41.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.453461+0000", 2024-09-04T17:46:41.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00047606999999999999 2024-09-04T17:46:41.305 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:41.305 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:41.306 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:41.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.306 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6206 3.1f 3:fec7e474:::rbd_data.1134cfbce83d.0000000000000003:6 [sparse-read 1409024~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:41.306 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423608+0000", 2024-09-04T17:46:41.306 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.823980564000003, 2024-09-04T17:46:41.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.031860753999999998, 2024-09-04T17:46:41.306 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:41.306 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:41.306 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:41.307 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:41.307 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:41.307 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:41.307 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6206 2024-09-04T17:46:41.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.307 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:41.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:41.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423608+0000", 2024-09-04T17:46:41.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:41.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:41.308 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423608+0000", 2024-09-04T17:46:41.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:41.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.308 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:41.308 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423609+0000", 2024-09-04T17:46:41.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0300000000000001e-06 2024-09-04T17:46:41.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.308 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:41.308 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423614+0000", 2024-09-04T17:46:41.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0200000000000002e-06 2024-09-04T17:46:41.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.309 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:41.309 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423630+0000", 2024-09-04T17:46:41.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6495999999999999e-05 2024-09-04T17:46:41.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.309 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:41.309 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423643+0000", 2024-09-04T17:46:41.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2053e-05 2024-09-04T17:46:41.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:41.310 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.453527+0000", 2024-09-04T17:46:41.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029884796000000002 2024-09-04T17:46:41.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:41.310 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.453632+0000", 2024-09-04T17:46:41.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104515 2024-09-04T17:46:41.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:41.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.455469+0000", 2024-09-04T17:46:41.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001836844 2024-09-04T17:46:41.311 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:41.311 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:41.311 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:41.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.311 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6207 3.1f 3:fec7e474:::rbd_data.1134cfbce83d.0000000000000003:6 [sparse-read 1769472~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:41.311 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423662+0000", 2024-09-04T17:46:41.311 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.823926100999998, 2024-09-04T17:46:41.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.033074294999999997, 2024-09-04T17:46:41.312 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:41.312 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:41.312 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:41.312 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:41.312 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:41.312 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:41.312 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6207 2024-09-04T17:46:41.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.312 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:41.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.312 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:41.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423662+0000", 2024-09-04T17:46:41.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:41.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:41.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423662+0000", 2024-09-04T17:46:41.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:41.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:41.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423664+0000", 2024-09-04T17:46:41.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4929999999999999e-06 2024-09-04T17:46:41.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.314 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:41.314 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423669+0000", 2024-09-04T17:46:41.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3800000000000002e-06 2024-09-04T17:46:41.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.314 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:41.314 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423686+0000", 2024-09-04T17:46:41.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7010000000000001e-05 2024-09-04T17:46:41.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.314 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:41.314 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423704+0000", 2024-09-04T17:46:41.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7710000000000002e-05 2024-09-04T17:46:41.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.315 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:41.315 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.455985+0000", 2024-09-04T17:46:41.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032281272999999999 2024-09-04T17:46:41.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.315 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:41.315 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.456196+0000", 2024-09-04T17:46:41.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000210335 2024-09-04T17:46:41.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:41.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.456737+0000", 2024-09-04T17:46:41.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00054109399999999999 2024-09-04T17:46:41.316 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:41.316 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:41.316 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:41.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.316 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6243 3.17 3:ed419000:::rbd_data.1134cfbce83d.000000000000000b:6 [sparse-read 294912~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:41.316 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423721+0000", 2024-09-04T17:46:41.316 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.823867315000001, 2024-09-04T17:46:41.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032666132000000001, 2024-09-04T17:46:41.316 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:41.317 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:41.317 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:41.317 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:41.317 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:41.317 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:41.317 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6243 2024-09-04T17:46:41.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.317 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:41.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.318 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:41.318 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423721+0000", 2024-09-04T17:46:41.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:41.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.318 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:41.318 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423721+0000", 2024-09-04T17:46:41.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:41.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.318 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:41.318 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423722+0000", 2024-09-04T17:46:41.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0470000000000001e-06 2024-09-04T17:46:41.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:41.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423727+0000", 2024-09-04T17:46:41.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0699999999999997e-06 2024-09-04T17:46:41.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:41.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423744+0000", 2024-09-04T17:46:41.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6659000000000001e-05 2024-09-04T17:46:41.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:41.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423756+0000", 2024-09-04T17:46:41.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2284e-05 2024-09-04T17:46:41.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:41.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.455553+0000", 2024-09-04T17:46:41.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.031797032000000003 2024-09-04T17:46:41.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:41.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.455716+0000", 2024-09-04T17:46:41.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016253700000000001 2024-09-04T17:46:41.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:41.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.456387+0000", 2024-09-04T17:46:41.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00067150300000000005 2024-09-04T17:46:41.321 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:41.321 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:41.321 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:41.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.321 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6244 3.17 3:ed419000:::rbd_data.1134cfbce83d.000000000000000b:6 [read 688128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:41.321 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423775+0000", 2024-09-04T17:46:41.321 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.823813532000003, 2024-09-04T17:46:41.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.033129498, 2024-09-04T17:46:41.322 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:41.322 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:41.322 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:41.322 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:41.322 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:41.322 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:41.322 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6244 2024-09-04T17:46:41.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.322 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:41.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.323 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:41.323 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423775+0000", 2024-09-04T17:46:41.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:41.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.323 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:41.323 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423775+0000", 2024-09-04T17:46:41.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:41.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.324 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.324 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:41.324 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423776+0000", 2024-09-04T17:46:41.324 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1230000000000001e-06 2024-09-04T17:46:41.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.324 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.324 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:41.324 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423782+0000", 2024-09-04T17:46:41.324 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4929999999999998e-06 2024-09-04T17:46:41.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:41.325 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423799+0000", 2024-09-04T17:46:41.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7254000000000001e-05 2024-09-04T17:46:41.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:41.325 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423811+0000", 2024-09-04T17:46:41.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2553e-05 2024-09-04T17:46:41.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:41.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.456421+0000", 2024-09-04T17:46:41.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032609552 2024-09-04T17:46:41.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:41.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.456512+0000", 2024-09-04T17:46:41.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0593999999999994e-05 2024-09-04T17:46:41.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:41.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.456904+0000", 2024-09-04T17:46:41.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00039292899999999998 2024-09-04T17:46:41.327 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:41.327 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:41.327 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:41.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.327 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6245 3.17 3:ed419000:::rbd_data.1134cfbce83d.000000000000000b:6 [read 1048576~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:41.327 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423830+0000", 2024-09-04T17:46:41.327 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.823758525999999, 2024-09-04T17:46:41.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.033577937000000002, 2024-09-04T17:46:41.327 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:41.327 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:41.327 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:41.327 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:41.327 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:41.328 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:41.328 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6245 2024-09-04T17:46:41.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.328 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:41.328 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.328 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:41.328 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423830+0000", 2024-09-04T17:46:41.328 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:41.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.328 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.328 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:41.328 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423830+0000", 2024-09-04T17:46:41.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:41.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.329 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:41.329 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423831+0000", 2024-09-04T17:46:41.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.164e-06 2024-09-04T17:46:41.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.329 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:41.329 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423847+0000", 2024-09-04T17:46:41.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5573000000000002e-05 2024-09-04T17:46:41.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.330 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:41.330 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423865+0000", 2024-09-04T17:46:41.330 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8715999999999998e-05 2024-09-04T17:46:41.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.330 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:41.330 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423880+0000", 2024-09-04T17:46:41.330 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4107000000000001e-05 2024-09-04T17:46:41.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.330 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:41.330 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.456951+0000", 2024-09-04T17:46:41.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.03307181 2024-09-04T17:46:41.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.331 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:41.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.457039+0000", 2024-09-04T17:46:41.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7795999999999998e-05 2024-09-04T17:46:41.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.331 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:41.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.457408+0000", 2024-09-04T17:46:41.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000368771 2024-09-04T17:46:41.331 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:41.331 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:41.331 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:41.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.332 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.332 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6246 3.17 3:ed419000:::rbd_data.1134cfbce83d.000000000000000b:6 [read 1409024~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:41.332 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423897+0000", 2024-09-04T17:46:41.332 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.823691187000001, 2024-09-04T17:46:41.332 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034015888000000001, 2024-09-04T17:46:41.332 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:41.332 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:41.332 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:41.332 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:41.332 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:41.332 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:41.333 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6246 2024-09-04T17:46:41.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.333 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:41.333 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.333 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:41.333 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423897+0000", 2024-09-04T17:46:41.333 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:41.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.333 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.334 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:41.334 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423897+0000", 2024-09-04T17:46:41.334 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:41.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.334 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:41.334 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423899+0000", 2024-09-04T17:46:41.334 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3200000000000001e-06 2024-09-04T17:46:41.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.334 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:41.334 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423904+0000", 2024-09-04T17:46:41.334 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4369999999999998e-06 2024-09-04T17:46:41.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.335 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:41.335 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423922+0000", 2024-09-04T17:46:41.335 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8196999999999999e-05 2024-09-04T17:46:41.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.335 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:41.335 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423935+0000", 2024-09-04T17:46:41.335 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.314e-05 2024-09-04T17:46:41.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.335 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:41.335 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.457457+0000", 2024-09-04T17:46:41.335 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.033521663 2024-09-04T17:46:41.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.336 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.336 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:41.336 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.457536+0000", 2024-09-04T17:46:41.336 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8795999999999996e-05 2024-09-04T17:46:41.336 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.336 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.336 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:41.336 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.457913+0000", 2024-09-04T17:46:41.336 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00037733500000000002 2024-09-04T17:46:41.336 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:41.336 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:41.336 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:41.336 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.337 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6247 3.17 3:ed419000:::rbd_data.1134cfbce83d.000000000000000b:6 [read 1769472~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:41.337 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423953+0000", 2024-09-04T17:46:41.337 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.823635003, 2024-09-04T17:46:41.337 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0344287, 2024-09-04T17:46:41.337 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:41.337 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:41.337 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:41.337 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:41.337 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:41.337 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:41.337 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6247 2024-09-04T17:46:41.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.338 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:41.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.338 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:41.338 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423953+0000", 2024-09-04T17:46:41.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:41.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.338 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:41.338 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423953+0000", 2024-09-04T17:46:41.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:41.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.338 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:41.339 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423955+0000", 2024-09-04T17:46:41.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0529999999999999e-06 2024-09-04T17:46:41.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.339 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:41.339 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423960+0000", 2024-09-04T17:46:41.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0470000000000001e-06 2024-09-04T17:46:41.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.339 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:41.339 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423978+0000", 2024-09-04T17:46:41.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8119999999999999e-05 2024-09-04T17:46:41.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.339 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:41.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423991+0000", 2024-09-04T17:46:41.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2813000000000001e-05 2024-09-04T17:46:41.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.340 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:41.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.457947+0000", 2024-09-04T17:46:41.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.033956865000000003 2024-09-04T17:46:41.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.340 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:41.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.458039+0000", 2024-09-04T17:46:41.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1834000000000002e-05 2024-09-04T17:46:41.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.341 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:41.341 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.458382+0000", 2024-09-04T17:46:41.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00034296799999999997 2024-09-04T17:46:41.341 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:41.341 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:41.341 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:41.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.341 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6305 3.17 3:ee7cd06a:::rbd_data.113431586065.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:41.341 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.455350+0000", 2024-09-04T17:46:41.341 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.792238451000003, 2024-09-04T17:46:41.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029276888000000001, 2024-09-04T17:46:41.342 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:41.342 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:41.342 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:41.342 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:41.342 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:41.342 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:41.342 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6305 2024-09-04T17:46:41.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.342 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:41.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.342 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:41.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.455350+0000", 2024-09-04T17:46:41.343 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:41.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.343 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:41.343 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.455350+0000", 2024-09-04T17:46:41.343 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:41.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.343 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:41.343 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.455353+0000", 2024-09-04T17:46:41.343 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6400000000000001e-06 2024-09-04T17:46:41.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.344 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:41.344 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459745+0000", 2024-09-04T17:46:41.344 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0043925880000000002 2024-09-04T17:46:41.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.344 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:41.344 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459775+0000", 2024-09-04T17:46:41.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9254000000000001e-05 2024-09-04T17:46:41.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.345 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:41.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459789+0000", 2024-09-04T17:46:41.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4731e-05 2024-09-04T17:46:41.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.345 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:41.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459885+0000", 2024-09-04T17:46:41.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5502999999999997e-05 2024-09-04T17:46:41.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:41.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.460040+0000", 2024-09-04T17:46:41.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015491499999999999 2024-09-04T17:46:41.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5", 2024-09-04T17:46:41.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.460596+0000", 2024-09-04T17:46:41.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00055657500000000002 2024-09-04T17:46:41.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:41.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484484+0000", 2024-09-04T17:46:41.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.023887950000000002 2024-09-04T17:46:41.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:46:41.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484510+0000", 2024-09-04T17:46:41.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.561e-05 2024-09-04T17:46:41.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:41.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484569+0000", 2024-09-04T17:46:41.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9049999999999999e-05 2024-09-04T17:46:41.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:41.348 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484627+0000", 2024-09-04T17:46:41.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8072000000000002e-05 2024-09-04T17:46:41.348 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:41.348 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:41.348 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:41.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.348 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6306 3.17 3:ee7cd06a:::rbd_data.113431586065.0000000000000005:head [truncate 1572864] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:41.348 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.459834+0000", 2024-09-04T17:46:41.348 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.787754773000003, 2024-09-04T17:46:41.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029166788999999999, 2024-09-04T17:46:41.349 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:41.349 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:41.349 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:41.349 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:41.349 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:41.349 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:41.349 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6306 2024-09-04T17:46:41.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.350 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:41.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:41.350 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459834+0000", 2024-09-04T17:46:41.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:41.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:41.350 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459834+0000", 2024-09-04T17:46:41.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:41.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:41.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459836+0000", 2024-09-04T17:46:41.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.835e-06 2024-09-04T17:46:41.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:41.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459841+0000", 2024-09-04T17:46:41.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.728e-06 2024-09-04T17:46:41.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:41.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459851+0000", 2024-09-04T17:46:41.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8570000000000007e-06 2024-09-04T17:46:41.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:41.352 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459859+0000", 2024-09-04T17:46:41.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4970000000000004e-06 2024-09-04T17:46:41.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:41.352 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.468789+0000", 2024-09-04T17:46:41.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0089301699999999994 2024-09-04T17:46:41.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:41.352 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.468890+0000", 2024-09-04T17:46:41.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000100963 2024-09-04T17:46:41.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5", 2024-09-04T17:46:41.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.469090+0000", 2024-09-04T17:46:41.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000199774 2024-09-04T17:46:41.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:41.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484704+0000", 2024-09-04T17:46:41.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015614836999999999 2024-09-04T17:46:41.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.354 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:46:41.354 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.488833+0000", 2024-09-04T17:46:41.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004128357 2024-09-04T17:46:41.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.354 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:41.354 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.488916+0000", 2024-09-04T17:46:41.355 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2928000000000003e-05 2024-09-04T17:46:41.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.355 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.355 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:41.355 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.489001+0000", 2024-09-04T17:46:41.355 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4843000000000001e-05 2024-09-04T17:46:41.355 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:41.355 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:41.355 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:41.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.355 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.356 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6307 3.17 3:eb6e082d:::rbd_data.113431586065.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:41.356 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.459871+0000", 2024-09-04T17:46:41.356 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.787717813, 2024-09-04T17:46:41.356 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032107122000000002, 2024-09-04T17:46:41.356 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:41.356 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:41.356 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:41.356 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:41.356 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:41.356 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:41.356 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6307 2024-09-04T17:46:41.356 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.356 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:41.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.357 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:41.357 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459871+0000", 2024-09-04T17:46:41.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:41.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.357 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:41.357 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459871+0000", 2024-09-04T17:46:41.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:41.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.357 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:41.357 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459871+0000", 2024-09-04T17:46:41.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9699999999999995e-07 2024-09-04T17:46:41.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:41.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.463429+0000", 2024-09-04T17:46:41.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0035577930000000001 2024-09-04T17:46:41.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:41.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.463463+0000", 2024-09-04T17:46:41.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3460000000000002e-05 2024-09-04T17:46:41.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:41.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.463483+0000", 2024-09-04T17:46:41.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0327999999999999e-05 2024-09-04T17:46:41.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:41.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.477551+0000", 2024-09-04T17:46:41.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014068259 2024-09-04T17:46:41.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:41.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.477748+0000", 2024-09-04T17:46:41.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000197004 2024-09-04T17:46:41.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5", 2024-09-04T17:46:41.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.478346+0000", 2024-09-04T17:46:41.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00059796299999999997 2024-09-04T17:46:41.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:46:41.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.489016+0000", 2024-09-04T17:46:41.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010669781 2024-09-04T17:46:41.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:41.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.491871+0000", 2024-09-04T17:46:41.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028546219999999998 2024-09-04T17:46:41.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:41.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.491931+0000", 2024-09-04T17:46:41.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0736999999999998e-05 2024-09-04T17:46:41.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:41.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.491978+0000", 2024-09-04T17:46:41.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6378000000000002e-05 2024-09-04T17:46:41.361 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:41.362 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:41.362 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:41.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.362 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6308 3.17 3:eb6e082d:::rbd_data.113431586065.0000000000000007:head [truncate 1572864] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:41.362 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.463520+0000", 2024-09-04T17:46:41.362 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.784068304000002, 2024-09-04T17:46:41.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029348833000000001, 2024-09-04T17:46:41.362 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:41.362 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:41.362 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:41.362 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:41.362 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:41.363 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:41.363 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6308 2024-09-04T17:46:41.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.363 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:41.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:41.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.463520+0000", 2024-09-04T17:46:41.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:41.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:41.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.463520+0000", 2024-09-04T17:46:41.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:41.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:41.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.463522+0000", 2024-09-04T17:46:41.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1150000000000001e-06 2024-09-04T17:46:41.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:41.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.463527+0000", 2024-09-04T17:46:41.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5170000000000001e-06 2024-09-04T17:46:41.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:41.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.463555+0000", 2024-09-04T17:46:41.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8594000000000001e-05 2024-09-04T17:46:41.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:41.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.463563+0000", 2024-09-04T17:46:41.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4340000000000003e-06 2024-09-04T17:46:41.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:41.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484756+0000", 2024-09-04T17:46:41.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021193081999999999 2024-09-04T17:46:41.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:41.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484863+0000", 2024-09-04T17:46:41.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010721999999999999 2024-09-04T17:46:41.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5", 2024-09-04T17:46:41.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.485067+0000", 2024-09-04T17:46:41.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000204 2024-09-04T17:46:41.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:46:41.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.489593+0000", 2024-09-04T17:46:41.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045261529999999998 2024-09-04T17:46:41.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:41.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.492787+0000", 2024-09-04T17:46:41.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031940039999999999 2024-09-04T17:46:41.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:41.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.492834+0000", 2024-09-04T17:46:41.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6281e-05 2024-09-04T17:46:41.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:41.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:41.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:41.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.492869+0000", 2024-09-04T17:46:41.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5432999999999998e-05 2024-09-04T17:46:41.368 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:41.369 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:41.369 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:41.369 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:41.369 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:41.369 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:41.369 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-04T17:46:41.371 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-04T17:46:41.371 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:41.370+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:41.471 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:41.468+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:41.487 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-6 --journal-path /var/lib/ceph/osd/ceph-6/journal --op export --pgid 3.d --file /var/log/ceph/exp.3.d.6 2024-09-04T17:46:41.538 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:41.538 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:46:41.539 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:46:41.539 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:41.555 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-04T17:46:41.573 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:46:41.573 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:46:41.574 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:41.572+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:41.591 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:46:41.591 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:46:41.608 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-04T17:46:41.608 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-04T17:46:41.625 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-04T17:46:41.625 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-04T17:46:41.647 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:46:41.647 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:46:41.668 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:46:41.668 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:46:41.668 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-04T17:46:41.673 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:41.671+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:41.688 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:41.688 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:41.690 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:46:41.690 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:46:41.695 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-04T17:46:41.696 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-04T17:46:41.703 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-04T17:46:41.788 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:46:41.787+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:46:41.863 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:41.863 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:41.875 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:41.874+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:41.877 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-04T17:46:41.963 INFO:teuthology.orchestra.run.smithi190.stderr:Exporting 3.d info 3.d( v 58'499 (0'0,58'499] local-lis/les=37/38 n=31 ec=24/24 lis/c=37/37 les/c/f=38/38/0 sis=37) 2024-09-04T17:46:41.967 INFO:teuthology.orchestra.run.smithi190.stderr:Read #3:b049e35c:::rbd_data.11346251f094.0000000000000001:head# 2024-09-04T17:46:41.968 INFO:teuthology.orchestra.run.smithi190.stderr:Read #3:b0775251:::rbd_data.1134eedad849.0000000000000320:head# 2024-09-04T17:46:41.968 INFO:teuthology.orchestra.run.smithi190.stderr:Read #3:b0a816c9:::rbd_data.11345f88eb9a.0000000000000001:head# 2024-09-04T17:46:41.970 INFO:teuthology.orchestra.run.smithi190.stderr:Read #3:b0b7c425:::rbd_header.1134eedad849:head# 2024-09-04T17:46:41.970 INFO:teuthology.orchestra.run.smithi190.stderr:Read #3:b0c50386:::rbd_data.1134b705d747.00000000000000ae:head# 2024-09-04T17:46:41.970 INFO:teuthology.orchestra.run.smithi190.stderr:Read #3:b0f77b97:::rbd_data.1134eedad849.000000000000028b:head# 2024-09-04T17:46:41.971 INFO:teuthology.orchestra.run.smithi190.stderr:Read #3:b1423a69:::rbd_data.11348d83498b.000000000000000a:c# 2024-09-04T17:46:41.976 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:41.975+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:41.978 INFO:teuthology.orchestra.run.smithi190.stderr:Read #3:b1423a69:::rbd_data.11348d83498b.000000000000000a:head# 2024-09-04T17:46:41.980 INFO:teuthology.orchestra.run.smithi190.stderr:Read #3:b1784c2c:::rbd_data.1134eedad849.00000000000002b6:head# 2024-09-04T17:46:41.980 INFO:teuthology.orchestra.run.smithi190.stderr:Read #3:b18d5173:::rbd_data.1134ef3a5a7d.0000000000000009:b# 2024-09-04T17:46:41.980 INFO:teuthology.orchestra.run.smithi190.stderr:Read #3:b18d5173:::rbd_data.1134ef3a5a7d.0000000000000009:head# 2024-09-04T17:46:41.981 INFO:teuthology.orchestra.run.smithi190.stderr:Read #3:b1ae20f2:::rbd_data.113431586065.0000000000000003:7# 2024-09-04T17:46:41.991 INFO:teuthology.orchestra.run.smithi190.stderr:Read #3:b1ae20f2:::rbd_data.113431586065.0000000000000003:head# 2024-09-04T17:46:41.994 INFO:teuthology.orchestra.run.smithi190.stderr:Read #3:b2224173:::rbd_object_map.1134fdde05d4:head# 2024-09-04T17:46:41.995 INFO:teuthology.orchestra.run.smithi190.stderr:Read #3:b25c3c94:::rbd_data.1134b705d747.00000000000000bf:head# 2024-09-04T17:46:41.995 INFO:teuthology.orchestra.run.smithi190.stderr:Read #3:b29f91c0:::rbd_data.1134eedad849.00000000000002bb:head# 2024-09-04T17:46:41.995 INFO:teuthology.orchestra.run.smithi190.stderr:Read #3:b30cd052:::rbd_data.1134eedad849.00000000000002c0:head# 2024-09-04T17:46:41.995 INFO:teuthology.orchestra.run.smithi190.stderr:Read #3:b324edf9:::rbd_data.1134eedad849.000000000000027c:head# 2024-09-04T17:46:41.996 INFO:teuthology.orchestra.run.smithi190.stderr:Read #3:b34757d9:::rbd_data.1134b705d747.0000000000000089:head# 2024-09-04T17:46:41.996 INFO:teuthology.orchestra.run.smithi190.stderr:Read #3:b35da36f:::rbd_data.1134eedad849.00000000000002d8:head# 2024-09-04T17:46:41.996 INFO:teuthology.orchestra.run.smithi190.stderr:Read #3:b41c7f2e:::rbd_object_map.1134c5a43465:head# 2024-09-04T17:46:41.996 INFO:teuthology.orchestra.run.smithi190.stderr:Read #3:b56cc28c:::rbd_data.1134b705d747.00000000000000b7:head# 2024-09-04T17:46:41.996 INFO:teuthology.orchestra.run.smithi190.stderr:Read #3:b5c98cda:::rbd_object_map.113418a7bbc1.000000000000000d:head# 2024-09-04T17:46:41.996 INFO:teuthology.orchestra.run.smithi190.stderr:Read #3:b637205b:::rbd_data.1134eedad849.0000000000000351:head# 2024-09-04T17:46:41.997 INFO:teuthology.orchestra.run.smithi190.stderr:Read #3:b692b231:::rbd_data.1134cfbce83d.000000000000000c:6# 2024-09-04T17:46:41.999 INFO:teuthology.orchestra.run.smithi190.stderr:Read #3:b692b231:::rbd_data.1134cfbce83d.000000000000000c:head# 2024-09-04T17:46:41.999 INFO:teuthology.orchestra.run.smithi190.stderr:Read #3:b7304e79:::rbd_data.1134eedad849.00000000000002d4:head# 2024-09-04T17:46:41.999 INFO:teuthology.orchestra.run.smithi190.stderr:Read #3:b763de10:::rbd_data.11345bd5898b.0000000000000002:head# 2024-09-04T17:46:42.003 INFO:teuthology.orchestra.run.smithi190.stderr:Read #3:b76ef7b6:::rbd_data.1134eedad849.0000000000000331:head# 2024-09-04T17:46:42.004 INFO:teuthology.orchestra.run.smithi190.stderr:Read #3:b77d7938:::rbd_data.1134eedad849.0000000000000279:head# 2024-09-04T17:46:42.004 INFO:teuthology.orchestra.run.smithi190.stderr:Read #3:b7c36f16:::rbd_data.1134eedad849.0000000000000381:head# 2024-09-04T17:46:42.004 INFO:teuthology.orchestra.run.smithi190.stderr:Export successful 2024-09-04T17:46:42.046 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:42.046 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:42.061 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-04T17:46:42.104 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:42.103+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:42.178 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:42.178+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:42.224 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:42.224 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:42.238 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-04T17:46:42.284 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:46:42.283+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:46:42.287 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-6 --journal-path /var/lib/ceph/osd/ceph-6/journal --force --op remove --pgid 3.d 2024-09-04T17:46:42.354 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:42.354 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:42.368 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-04T17:46:42.416 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:42.418+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:42.480 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:42.478+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:42.489 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-04T17:46:42.492 INFO:teuthology.orchestra.run.smithi092.stdout:2051 read 0x5c656 thru 0x61692 (0x503d bytes) 2024-09-04T17:46:42.494 INFO:teuthology.orchestra.run.smithi092.stdout:2052 read 0x763cd thru 0x773e6 (0x101a bytes) 2024-09-04T17:46:42.495 INFO:teuthology.orchestra.run.smithi092.stdout:2053 read 0x64817 thru 0x6ada6 (0x6590 bytes) 2024-09-04T17:46:42.497 INFO:teuthology.orchestra.run.smithi092.stdout:2054 punch from 0x7e550 to 0x81fe4, (0x3a94 bytes) 2024-09-04T17:46:42.498 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:42.499 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:42.513 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-04T17:46:42.531 INFO:teuthology.orchestra.run.smithi092.stdout:2055 write 0x980307b thru 0x980788d (0x4813 bytes) 2024-09-04T17:46:42.566 INFO:teuthology.orchestra.run.smithi092.stdout:2056 write 0xdde70e0 thru 0xddeb7ff (0x4720 bytes) 2024-09-04T17:46:42.576 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:42.576 INFO:teuthology.orchestra.run.smithi092.stdout:2058 write 0x68ea680 thru 0x68f70d1 (0xca52 bytes) 2024-09-04T17:46:42.585 INFO:teuthology.orchestra.run.smithi092.stdout:2059 punch from 0xbe542e2 to 0xbe589ec, (0x470a bytes) 2024-09-04T17:46:42.585 INFO:teuthology.orchestra.run.smithi092.stdout:2060 punch from 0xa412d0d to 0xa414264, (0x1557 bytes) 2024-09-04T17:46:42.585 INFO:teuthology.orchestra.run.smithi092.stdout:2063 read 0x6f775db thru 0x6f83a61 (0xc487 bytes) 2024-09-04T17:46:42.592 INFO:teuthology.orchestra.run.smithi092.stdout:2064 trunc from 0xddeb800 to 0xac394cb 2024-09-04T17:46:42.600 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:42.602+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:42.609 INFO:teuthology.orchestra.run.smithi092.stdout:2065 trunc from 0xac394cb to 0xc6fd503 2024-09-04T17:46:42.616 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:42.616 INFO:teuthology.orchestra.run.smithi092.stdout:2069 writesame 0xbca0d8b thru 0xbca332f data_size 0x1(0x25a5 bytes) 2024-09-04T17:46:42.626 INFO:teuthology.orchestra.run.smithi092.stdout:2070 read 0x98017fa thru 0x980e78c (0xcf93 bytes) 2024-09-04T17:46:42.627 INFO:teuthology.orchestra.run.smithi092.stdout:2071 punch from 0x41a108d to 0x41a6f11, (0x5e84 bytes) 2024-09-04T17:46:42.627 INFO:teuthology.orchestra.run.smithi092.stdout:2073 punch from 0x69e5059 to 0x69eb4ba, (0x6461 bytes) 2024-09-04T17:46:42.627 INFO:teuthology.orchestra.run.smithi092.stdout:2074 write 0x38a3af4 thru 0x38a435d (0x86a bytes) 2024-09-04T17:46:42.651 INFO:teuthology.orchestra.run.smithi092.stdout:2079 trunc from 0xc6fd503 to 0xe57ba8 2024-09-04T17:46:42.665 INFO:teuthology.orchestra.run.smithi092.stdout:2080 read 0x9d5d92 thru 0x9e13b6 (0xb625 bytes) 2024-09-04T17:46:42.665 INFO:teuthology.orchestra.run.smithi092.stdout:2083 read 0x2cc405 thru 0x2dc181 (0xfd7d bytes) 2024-09-04T17:46:42.665 INFO:teuthology.orchestra.run.smithi092.stdout:2084 read 0x4aa564 thru 0x4b6a85 (0xc522 bytes) 2024-09-04T17:46:42.681 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:42.682+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:42.686 INFO:teuthology.orchestra.run.smithi092.stdout:2085 write 0xafd3791 thru 0xafdd20f (0x9a7f bytes) 2024-09-04T17:46:42.687 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:42.687 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:42.700 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:42.700 INFO:teuthology.orchestra.run.smithi092.stdout:2088 punch from 0xaa38f78 to 0xaa41b5b, (0x8be3 bytes) 2024-09-04T17:46:42.700 INFO:teuthology.orchestra.run.smithi092.stdout:2089 write 0xac3d40f thru 0xac46b17 (0x9709 bytes) 2024-09-04T17:46:42.703 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-04T17:46:42.721 INFO:teuthology.orchestra.run.smithi092.stdout:2090 write 0xe034e43 thru 0xe0386ee (0x38ac bytes) 2024-09-04T17:46:42.731 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:42.731 INFO:teuthology.orchestra.run.smithi092.stdout:2092 read 0x2fb9a84 thru 0x2fba2d5 (0x852 bytes) 2024-09-04T17:46:42.731 INFO:teuthology.orchestra.run.smithi092.stdout:2093 read 0xd8e6d55 thru 0xd8f2225 (0xb4d1 bytes) 2024-09-04T17:46:42.731 INFO:teuthology.orchestra.run.smithi092.stdout:2094 write 0xc3dd86b thru 0xc3e1798 (0x3f2e bytes) 2024-09-04T17:46:42.739 INFO:teuthology.orchestra.run.smithi092.stdout:2097 write 0x9a946d2 thru 0x9aa1a60 (0xd38f bytes) 2024-09-04T17:46:42.750 INFO:teuthology.orchestra.run.smithi092.stdout:2098 read 0x39b4762 thru 0x39b8e36 (0x46d5 bytes) 2024-09-04T17:46:42.750 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:42.750 INFO:teuthology.orchestra.run.smithi092.stdout:2100 read 0x2101e76 thru 0x2109eb5 (0x8040 bytes) 2024-09-04T17:46:42.751 INFO:teuthology.orchestra.run.smithi092.stdout:2103 writesame 0x9c969dd thru 0x9ca058d data_size 0x1(0x9bb1 bytes) 2024-09-04T17:46:42.766 INFO:teuthology.orchestra.run.smithi092.stdout:2106 writesame 0xd21df01 thru 0xd225b4e data_size 0x7(0x7c4e bytes) 2024-09-04T17:46:42.778 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:42.779 INFO:teuthology.orchestra.run.smithi092.stdout:2109 trunc from 0xe0386ef to 0xdc141de 2024-09-04T17:46:42.788 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:42.790+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:42.795 INFO:teuthology.orchestra.run.smithi092.stdout:2110 read 0x63c7d8d thru 0x63cd4fe (0x5772 bytes) 2024-09-04T17:46:42.795 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:42.795 INFO:teuthology.orchestra.run.smithi092.stdout:2112 read 0x2451d32 thru 0x2453674 (0x1943 bytes) 2024-09-04T17:46:42.795 INFO:teuthology.orchestra.run.smithi092.stdout:2113 read 0xf449c6 thru 0xf50788 (0xbdc3 bytes) 2024-09-04T17:46:42.795 INFO:teuthology.orchestra.run.smithi092.stdout:2114 write 0x3d79fd5 thru 0x3d7e487 (0x44b3 bytes) 2024-09-04T17:46:42.804 INFO:teuthology.orchestra.run.smithi092.stdout:2115 write 0xa6caa27 thru 0xa6d7340 (0xc91a bytes) 2024-09-04T17:46:42.817 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:42.817 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:42.839 INFO:teuthology.orchestra.run.smithi092.stdout:2120 trunc from 0xdc141de to 0x4259db8 2024-09-04T17:46:42.872 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:42.872 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:42.872 INFO:teuthology.orchestra.run.smithi092.stdout:2124 punch from 0x3b7b136 to 0x3b89766, (0xe630 bytes) 2024-09-04T17:46:42.882 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:42.882+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:42.883 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:42.883 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:42.887 INFO:teuthology.orchestra.run.smithi092.stdout:2125 write 0x70fa850 thru 0x70fbee6 (0x1697 bytes) 2024-09-04T17:46:42.898 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-04T17:46:42.918 INFO:teuthology.orchestra.run.smithi092.stdout:2126 write 0xce8b538 thru 0xce8e3bd (0x2e86 bytes) 2024-09-04T17:46:42.928 INFO:teuthology.orchestra.run.smithi092.stdout:2127 write 0xaeb8288 thru 0xaebb531 (0x32aa bytes) 2024-09-04T17:46:42.938 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:42.938 INFO:teuthology.orchestra.run.smithi092.stdout:2129 writesame 0x153c3f8 thru 0x153c4ff data_size 0xb(0x108 bytes) 2024-09-04T17:46:42.944 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:42.944 INFO:teuthology.orchestra.run.smithi092.stdout:2131 write 0xa85484d thru 0xa860b87 (0xc33b bytes) 2024-09-04T17:46:42.955 INFO:teuthology.orchestra.run.smithi092.stdout:2133 read 0x96d3d34 thru 0x96d8608 (0x48d5 bytes) 2024-09-04T17:46:42.956 INFO:teuthology.orchestra.run.smithi092.stdout:2134 punch from 0x9bd3f7e to 0x9bd75eb, (0x366d bytes) 2024-09-04T17:46:42.956 INFO:teuthology.orchestra.run.smithi092.stdout:2135 write 0x4c8899d thru 0x4c91e06 (0x946a bytes) 2024-09-04T17:46:42.965 INFO:teuthology.orchestra.run.smithi092.stdout:2136 writesame 0x7934fa0 thru 0x7935c45 data_size 0x1(0xca6 bytes) 2024-09-04T17:46:42.974 INFO:teuthology.orchestra.run.smithi092.stdout:2137 punch from 0xb5c9216 to 0xb5d531a, (0xc104 bytes) 2024-09-04T17:46:42.992 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:42.994+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:42.993 INFO:teuthology.orchestra.run.smithi092.stdout:2138 trunc from 0xce8e3be to 0x5e80df6 2024-09-04T17:46:43.021 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:43.029 INFO:teuthology.orchestra.run.smithi092.stdout:2140 trunc from 0x5e80df6 to 0x8ff7ed5 2024-09-04T17:46:43.040 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:43.040 INFO:teuthology.orchestra.run.smithi092.stdout:2144 punch from 0x6123363 to 0x6124b29, (0x17c6 bytes) 2024-09-04T17:46:43.040 INFO:teuthology.orchestra.run.smithi092.stdout:2145 read 0x7b4d341 thru 0x7b531b8 (0x5e78 bytes) 2024-09-04T17:46:43.040 INFO:teuthology.orchestra.run.smithi092.stdout:2146 punch from 0x4f50f19 to 0x4f5f63d, (0xe724 bytes) 2024-09-04T17:46:43.040 INFO:teuthology.orchestra.run.smithi092.stdout:2148 writesame 0x3ed3ca6 thru 0x3ede30c data_size 0x1(0xa667 bytes) 2024-09-04T17:46:43.062 INFO:teuthology.orchestra.run.smithi092.stdout:2149 read 0x1633094 thru 0x163c815 (0x9782 bytes) 2024-09-04T17:46:43.063 INFO:teuthology.orchestra.run.smithi092.stdout:2150 write 0x4dbaa5c thru 0x4dc5df8 (0xb39d bytes) 2024-09-04T17:46:43.068 INFO:teuthology.orchestra.run.smithi092.stdout:2152 punch from 0x8dc4548 to 0x8dcc48c, (0x7f44 bytes) 2024-09-04T17:46:43.071 INFO:teuthology.orchestra.run.smithi092.stdout:2154 trunc from 0x8ff7ed5 to 0x9eaf573 2024-09-04T17:46:43.084 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:43.082+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:43.084 INFO:teuthology.orchestra.run.smithi092.stdout:2155 write 0x6a6b0ff thru 0x6a76994 (0xb896 bytes) 2024-09-04T17:46:43.086 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:43.086 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:43.095 INFO:teuthology.orchestra.run.smithi092.stdout:2156 read 0x930e396 thru 0x931d95c (0xf5c7 bytes) 2024-09-04T17:46:43.095 INFO:teuthology.orchestra.run.smithi092.stdout:2157 read 0x8fa1c3f thru 0x8faa4d4 (0x8896 bytes) 2024-09-04T17:46:43.096 INFO:teuthology.orchestra.run.smithi092.stdout:2161 punch from 0x5e0dbce to 0x5e12183, (0x45b5 bytes) 2024-09-04T17:46:43.096 INFO:teuthology.orchestra.run.smithi092.stdout:2163 punch from 0x4388714 to 0x4388c37, (0x523 bytes) 2024-09-04T17:46:43.106 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-04T17:46:43.107 INFO:teuthology.orchestra.run.smithi092.stdout:2166 write 0xa2df07e thru 0xa2ea543 (0xb4c6 bytes) 2024-09-04T17:46:43.120 INFO:teuthology.orchestra.run.smithi092.stdout:2167 flatten 2024-09-04T17:46:43.132 INFO:teuthology.orchestra.run.smithi092.stdout:2168 read 0x68689f6 thru 0x6870ab2 (0x80bd bytes) 2024-09-04T17:46:43.135 INFO:teuthology.orchestra.run.smithi092.stdout:2169 trunc from 0xa2ea544 to 0xb2e711d 2024-09-04T17:46:43.160 INFO:teuthology.orchestra.run.smithi092.stdout:2171 write 0xc4733bb thru 0xc479825 (0x646b bytes) 2024-09-04T17:46:43.170 INFO:teuthology.orchestra.run.smithi092.stdout:2172 write 0x7289e5c thru 0x728db98 (0x3d3d bytes) 2024-09-04T17:46:43.182 INFO:teuthology.orchestra.run.smithi092.stdout:2173 punch from 0x4fb6bfc to 0x4fc46a6, (0xdaaa bytes) 2024-09-04T17:46:43.185 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:46:43.183+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:46:43.200 INFO:teuthology.orchestra.run.smithi092.stdout:2174 trunc from 0xc479826 to 0x594efcd 2024-09-04T17:46:43.226 INFO:teuthology.orchestra.run.smithi092.stdout:2175 punch from 0x16cbe8a to 0x16d87e7, (0xc95d bytes) 2024-09-04T17:46:43.226 INFO:teuthology.orchestra.run.smithi092.stdout:2177 write 0x284e912 thru 0x2853175 (0x4864 bytes) 2024-09-04T17:46:43.239 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:43.239 INFO:teuthology.orchestra.run.smithi092.stdout:2180 write 0x5241d7e thru 0x524b65d (0x98e0 bytes) 2024-09-04T17:46:43.250 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:43.275 INFO:teuthology.orchestra.run.smithi092.stdout:2186 write 0xa6fda89 thru 0xa70a836 (0xcdae bytes) 2024-09-04T17:46:43.286 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:43.286+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:43.289 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:43.289 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:43.289 INFO:teuthology.orchestra.run.smithi092.stdout:2189 writesame 0x1db5cea thru 0x1dc45a0 data_size 0x5(0xe8b7 bytes) 2024-09-04T17:46:43.305 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-04T17:46:43.314 INFO:teuthology.orchestra.run.smithi092.stdout:2190 write 0x89acf37 thru 0x89b15de (0x46a8 bytes) 2024-09-04T17:46:43.328 INFO:teuthology.orchestra.run.smithi092.stdout:2191 read 0x800811a thru 0x8013b3d (0xba24 bytes) 2024-09-04T17:46:43.328 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:43.328 INFO:teuthology.orchestra.run.smithi092.stdout:2194 read 0x8922d0f thru 0x8923052 (0x344 bytes) 2024-09-04T17:46:43.329 INFO:teuthology.orchestra.run.smithi092.stdout:2195 writesame 0x57fc2a0 thru 0x58054fd data_size 0xf(0x925e bytes) 2024-09-04T17:46:43.365 INFO:teuthology.orchestra.run.smithi092.stdout:2198 write 0xe6c0e5c thru 0xe6c4797 (0x393c bytes) 2024-09-04T17:46:43.375 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:43.375 INFO:teuthology.orchestra.run.smithi092.stdout:2202 write 0xc8123 thru 0xd1cc3 (0x9ba1 bytes) 2024-09-04T17:46:43.379 INFO:teuthology.orchestra.run.smithi092.stdout:2203 read 0xb4ea2bb thru 0xb4efad7 (0x581d bytes) 2024-09-04T17:46:43.379 INFO:teuthology.orchestra.run.smithi092.stdout:2204 write 0xa7339a7 thru 0xa73c7ab (0x8e05 bytes) 2024-09-04T17:46:43.384 INFO:teuthology.orchestra.run.smithi092.stdout:2205 write 0x39d0e thru 0x3a307 (0x5fa bytes) 2024-09-04T17:46:43.387 INFO:teuthology.orchestra.run.smithi092.stdout:2206 write 0x3dfcc68 thru 0x3dfcf02 (0x29b bytes) 2024-09-04T17:46:43.390 INFO:teuthology.orchestra.run.smithi092.stdout:2207 write 0x1cf75f thru 0x1d200e (0x28b0 bytes) 2024-09-04T17:46:43.392 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:43.394+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:43.400 INFO:teuthology.orchestra.run.smithi092.stdout:2208 write 0xba5af87 thru 0xba5b80d (0x887 bytes) 2024-09-04T17:46:43.408 INFO:teuthology.orchestra.run.smithi092.stdout:2210 read 0xc4c2b1c thru 0xc4d10ce (0xe5b3 bytes) 2024-09-04T17:46:43.409 INFO:teuthology.orchestra.run.smithi092.stdout:2211 write 0x7971ae thru 0x7a6851 (0xf6a4 bytes) 2024-09-04T17:46:43.437 INFO:teuthology.orchestra.run.smithi092.stdout:2212 trunc from 0xe6c4798 to 0x7b92572 2024-09-04T17:46:43.464 INFO:teuthology.orchestra.run.smithi092.stdout:2213 write 0xc550fd thru 0xc58ed2 (0x3dd6 bytes) 2024-09-04T17:46:43.482 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:43.482 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:43.487 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:43.486+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:43.496 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-04T17:46:43.498 INFO:teuthology.orchestra.run.smithi092.stdout:2214 write 0xb226d3f thru 0xb2306ac (0x996e bytes) 2024-09-04T17:46:43.507 INFO:teuthology.orchestra.run.smithi092.stdout:2215 writesame 0x97cc360 thru 0x97d6b10 data_size 0x1(0xa7b1 bytes) 2024-09-04T17:46:43.528 INFO:teuthology.orchestra.run.smithi092.stdout:2218 read 0x272404d thru 0x272aa65 (0x6a19 bytes) 2024-09-04T17:46:43.529 INFO:teuthology.orchestra.run.smithi092.stdout:2219 punch from 0x195993f to 0x195c57c, (0x2c3d bytes) 2024-09-04T17:46:43.529 INFO:teuthology.orchestra.run.smithi092.stdout:2220 write 0xac00514 thru 0xac10296 (0xfd83 bytes) 2024-09-04T17:46:43.558 INFO:teuthology.orchestra.run.smithi092.stdout:2221 writesame 0xd04ec82 thru 0xd051628 data_size 0x1(0x29a7 bytes) 2024-09-04T17:46:43.558 INFO:teuthology.orchestra.run.smithi092.stdout:2222 write 0x679deb0 thru 0x67a4ef1 (0x7042 bytes) 2024-09-04T17:46:43.569 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:43.569 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:43.569 INFO:teuthology.orchestra.run.smithi092.stdout:2225 read 0x8996bda thru 0x899cf77 (0x639e bytes) 2024-09-04T17:46:43.569 INFO:teuthology.orchestra.run.smithi092.stdout:2226 write 0x959b20a thru 0x959e64b (0x3442 bytes) 2024-09-04T17:46:43.589 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:43.587+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:43.596 INFO:teuthology.orchestra.run.smithi092.stdout:2227 write 0xe57276f thru 0xe577dbe (0x5650 bytes) 2024-09-04T17:46:43.612 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:43.612 INFO:teuthology.orchestra.run.smithi092.stdout:2230 read 0x52425c4 thru 0x524ae7d (0x88ba bytes) 2024-09-04T17:46:43.614 INFO:teuthology.orchestra.run.smithi092.stdout:2233 write 0xe0ca493 thru 0xe0da265 (0xfdd3 bytes) 2024-09-04T17:46:43.628 INFO:teuthology.orchestra.run.smithi092.stdout:2234 punch from 0xa5818c to 0xa5a233, (0x20a7 bytes) 2024-09-04T17:46:43.628 INFO:teuthology.orchestra.run.smithi092.stdout:2235 read 0xa43b34e thru 0xa44833a (0xcfed bytes) 2024-09-04T17:46:43.628 INFO:teuthology.orchestra.run.smithi092.stdout:2236 punch from 0x7385df1 to 0x738634f, (0x55e bytes) 2024-09-04T17:46:43.628 INFO:teuthology.orchestra.run.smithi092.stdout:2237 read 0x8f5cbb8 thru 0x8f60562 (0x39ab bytes) 2024-09-04T17:46:43.628 INFO:teuthology.orchestra.run.smithi092.stdout:2238 punch from 0x953a24d to 0x953ee0b, (0x4bbe bytes) 2024-09-04T17:46:43.628 INFO:teuthology.orchestra.run.smithi092.stdout:2239 write 0xdd55196 thru 0xdd600fb (0xaf66 bytes) 2024-09-04T17:46:43.639 INFO:teuthology.orchestra.run.smithi092.stdout:2243 punch from 0x757f962 to 0x7584495, (0x4b33 bytes) 2024-09-04T17:46:43.639 INFO:teuthology.orchestra.run.smithi092.stdout:2245 write 0x61f893e thru 0x61fa956 (0x2019 bytes) 2024-09-04T17:46:43.673 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:43.674 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:43.685 INFO:teuthology.orchestra.run.smithi092.stdout:2246 trunc from 0xe577dbf to 0x21d979f 2024-09-04T17:46:43.687 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-04T17:46:43.689 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:46:43.687+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:46:43.696 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-04T17:46:43.733 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:43.733 INFO:teuthology.orchestra.run.smithi092.stdout:2248 read 0x4b4208 thru 0x4c1941 (0xd73a bytes) 2024-09-04T17:46:43.734 INFO:teuthology.orchestra.run.smithi092.stdout:2249 punch from 0xbd4b47 to 0xbe3036, (0xe4ef bytes) 2024-09-04T17:46:43.734 INFO:teuthology.orchestra.run.smithi092.stdout:2250 read 0x20606de thru 0x2062776 (0x2099 bytes) 2024-09-04T17:46:43.753 INFO:teuthology.orchestra.run.smithi092.stdout:2252 write 0x82f79c0 thru 0x82fe21a (0x685b bytes) 2024-09-04T17:46:43.781 INFO:teuthology.orchestra.run.smithi092.stdout:2253 write 0xc3fddf9 thru 0xc40da20 (0xfc28 bytes) 2024-09-04T17:46:43.789 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:43.790+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:43.801 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:43.801 INFO:teuthology.orchestra.run.smithi092.stdout:2256 write 0x344ac0c thru 0x34585eb (0xd9e0 bytes) 2024-09-04T17:46:43.817 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is set 2024-09-04T17:46:43.820 INFO:teuthology.orchestra.run.smithi092.stdout:2258 punch from 0x463c1c9 to 0x4641e5d, (0x5c94 bytes) 2024-09-04T17:46:43.824 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:43.825 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:43.832 INFO:teuthology.orchestra.run.smithi092.stdout:2259 write 0xd1ede9b thru 0xd1f9e78 (0xbfde bytes) 2024-09-04T17:46:43.844 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-04T17:46:43.844 INFO:teuthology.orchestra.run.smithi092.stdout:2260 writesame 0x5d581d1 thru 0x5d5abf5 data_size 0x1(0x2a25 bytes) 2024-09-04T17:46:43.852 INFO:teuthology.orchestra.run.smithi092.stdout:2262 punch from 0x5383dfc to 0x539023b, (0xc43f bytes) 2024-09-04T17:46:43.852 INFO:teuthology.orchestra.run.smithi092.stdout:2263 writesame 0xb8e3026 thru 0xb8ef8a0 data_size 0x1(0xc87b bytes) 2024-09-04T17:46:43.869 INFO:teuthology.orchestra.run.smithi092.stdout:2264 write 0x68fe4e4 thru 0x6909c0a (0xb727 bytes) 2024-09-04T17:46:43.878 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:46:43.878 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:46:43.886 INFO:teuthology.orchestra.run.smithi092.stdout:2265 writesame 0xa6ebee9 thru 0xa6ee012 data_size 0xf(0x212a bytes) 2024-09-04T17:46:43.892 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:43.894+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:43.895 INFO:teuthology.orchestra.run.smithi092.stdout:2266 write 0x433c041 thru 0x4344680 (0x8640 bytes) 2024-09-04T17:46:43.897 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:46:43.897 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:46:43.907 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:43.907 INFO:teuthology.orchestra.run.smithi092.stdout:2268 write 0x8096b05 thru 0x809efa0 (0x849c bytes) 2024-09-04T17:46:43.915 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-04T17:46:43.915 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-04T17:46:43.916 INFO:teuthology.orchestra.run.smithi092.stdout:2271 write 0xa70d4fc thru 0xa718adf (0xb5e4 bytes) 2024-09-04T17:46:43.919 INFO:teuthology.orchestra.run.smithi092.stdout:2273 punch from 0x5b89102 to 0x5b8d437, (0x4335 bytes) 2024-09-04T17:46:43.919 INFO:teuthology.orchestra.run.smithi092.stdout:2275 writesame 0x67320a4 thru 0x673e48f data_size 0x2(0xc3ec bytes) 2024-09-04T17:46:43.929 INFO:teuthology.orchestra.run.smithi092.stdout:2276 write 0x3d0101d thru 0x3d0f65a (0xe63e bytes) 2024-09-04T17:46:43.934 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-04T17:46:43.934 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-04T17:46:43.945 INFO:teuthology.orchestra.run.smithi092.stdout:2278 writesame 0xed44885 thru 0xed4c066 data_size 0x5d(0x77e2 bytes) 2024-09-04T17:46:43.946 INFO:teuthology.orchestra.run.smithi092.stdout:2279 read 0xe580f93 thru 0xe59069b (0xf709 bytes) 2024-09-04T17:46:43.946 INFO:teuthology.orchestra.run.smithi092.stdout:2280 punch from 0x7c620a2 to 0x7c64094, (0x1ff2 bytes) 2024-09-04T17:46:43.946 INFO:teuthology.orchestra.run.smithi092.stdout:2281 write 0xa2c00f3 thru 0xa2c84f2 (0x8400 bytes) 2024-09-04T17:46:43.953 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:46:43.953 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:46:43.956 INFO:teuthology.orchestra.run.smithi092.stdout:2283 read 0x650f182 thru 0x650f986 (0x805 bytes) 2024-09-04T17:46:43.956 INFO:teuthology.orchestra.run.smithi092.stdout:2284 read 0x23ee99d thru 0x23fe873 (0xfed7 bytes) 2024-09-04T17:46:43.957 INFO:teuthology.orchestra.run.smithi092.stdout:2285 punch from 0xd42c600 to 0xd433fc6, (0x79c6 bytes) 2024-09-04T17:46:43.957 INFO:teuthology.orchestra.run.smithi092.stdout:2288 writesame 0x1fcd848 thru 0x1fd9dad data_size 0xb(0xc566 bytes) 2024-09-04T17:46:43.965 INFO:teuthology.orchestra.run.smithi092.stdout:2290 write 0xc2320dc thru 0xc23e9ce (0xc8f3 bytes) 2024-09-04T17:46:43.974 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:46:43.975 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:46:43.975 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-04T17:46:43.976 INFO:teuthology.orchestra.run.smithi092.stdout:2291 punch from 0xb4fdbf6 to 0xb501015, (0x341f bytes) 2024-09-04T17:46:43.976 INFO:teuthology.orchestra.run.smithi092.stdout:2293 write 0xce7993d thru 0xce7e5d4 (0x4c98 bytes) 2024-09-04T17:46:43.987 INFO:teuthology.orchestra.run.smithi092.stdout:2295 writesame 0x551fa3a thru 0x55281d1 data_size 0x4(0x8798 bytes) 2024-09-04T17:46:43.991 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:43.990+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:43.993 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:46:43.993 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:46:43.997 INFO:teuthology.orchestra.run.smithi092.stdout:2296 write 0xd3c4254 thru 0xd3c7376 (0x3123 bytes) 2024-09-04T17:46:43.998 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-04T17:46:43.999 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-04T17:46:44.007 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:44.007 INFO:teuthology.orchestra.run.smithi092.stdout:2299 write 0x91e79ed thru 0x91f63ea (0xe9fe bytes) 2024-09-04T17:46:44.015 INFO:teuthology.orchestra.run.smithi092.stdout:2300 write 0x1530a1e thru 0x153eb7e (0xe161 bytes) 2024-09-04T17:46:44.022 INFO:teuthology.orchestra.run.smithi092.stdout:2301 punch from 0x1519ec3 to 0x1520992, (0x6acf bytes) 2024-09-04T17:46:44.034 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:44.035 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:46:44.035 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:46:44.035 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:46:44.035 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.035 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:1430 3.f 3:f58475cb:::rbd_data.11346251f094.0000000000000074:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-04T17:46:44.035 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:02.778704+0000", 2024-09-04T17:46:44.035 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 41.256458514000002, 2024-09-04T17:46:44.035 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.058567801000000003, 2024-09-04T17:46:44.035 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:44.035 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:44.035 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:44.036 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:44.036 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:44.036 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:44.036 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 1430 2024-09-04T17:46:44.036 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.036 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:44.036 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.036 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:44.036 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.778704+0000", 2024-09-04T17:46:44.036 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:44.036 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.036 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.036 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:44.037 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.778704+0000", 2024-09-04T17:46:44.037 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:44.037 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.037 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.037 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:44.037 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.778707+0000", 2024-09-04T17:46:44.037 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.8329999999999998e-06 2024-09-04T17:46:44.037 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.037 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.037 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:44.038 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.778718+0000", 2024-09-04T17:46:44.038 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.152e-05 2024-09-04T17:46:44.038 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.038 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.038 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:44.038 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.778750+0000", 2024-09-04T17:46:44.038 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.1696999999999998e-05 2024-09-04T17:46:44.038 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.038 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.038 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:44.038 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.778765+0000", 2024-09-04T17:46:44.038 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4800000000000001e-05 2024-09-04T17:46:44.039 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.039 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.039 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:44.039 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.778894+0000", 2024-09-04T17:46:44.039 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00012928500000000001 2024-09-04T17:46:44.039 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.039 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.039 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:44.039 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.779030+0000", 2024-09-04T17:46:44.039 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013566899999999999 2024-09-04T17:46:44.039 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.039 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.039 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 4", 2024-09-04T17:46:44.040 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.788794+0000", 2024-09-04T17:46:44.040 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.009764478 2024-09-04T17:46:44.040 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.040 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.040 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:44.040 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.836969+0000", 2024-09-04T17:46:44.040 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.048174626999999998 2024-09-04T17:46:44.040 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.040 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.040 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:44.040 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.837090+0000", 2024-09-04T17:46:44.040 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000120627 2024-09-04T17:46:44.040 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.041 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.041 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:44.041 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.837195+0000", 2024-09-04T17:46:44.041 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000105206 2024-09-04T17:46:44.041 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.041 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.041 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:44.041 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.837272+0000", 2024-09-04T17:46:44.041 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.7058999999999995e-05 2024-09-04T17:46:44.041 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:44.041 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:44.041 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:44.041 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.042 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.042 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14369 3.d 3:b763de10:::rbd_data.11345bd5898b.0000000000000002:a [read 77824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:44.042 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.182808+0000", 2024-09-04T17:46:44.042 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 25.852354546000001, 2024-09-04T17:46:44.042 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042057997999999999, 2024-09-04T17:46:44.042 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:44.042 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:44.042 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:44.042 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:44.042 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:44.042 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:44.042 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14369 2024-09-04T17:46:44.042 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.043 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:44.043 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.043 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:44.043 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182808+0000", 2024-09-04T17:46:44.043 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:44.043 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.043 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.043 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:44.043 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182808+0000", 2024-09-04T17:46:44.043 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:44.043 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.043 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.043 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:44.043 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182809+0000", 2024-09-04T17:46:44.044 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.063e-06 2024-09-04T17:46:44.044 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.044 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.044 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:44.044 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182814+0000", 2024-09-04T17:46:44.044 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1900000000000003e-06 2024-09-04T17:46:44.044 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.044 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.044 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:44.044 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182832+0000", 2024-09-04T17:46:44.044 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7791999999999998e-05 2024-09-04T17:46:44.044 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.044 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.044 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:44.045 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182849+0000", 2024-09-04T17:46:44.045 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7265000000000002e-05 2024-09-04T17:46:44.045 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.045 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.045 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:44.045 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.224431+0000", 2024-09-04T17:46:44.045 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.041581448 2024-09-04T17:46:44.045 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.045 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.045 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:44.045 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.224615+0000", 2024-09-04T17:46:44.045 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00018389200000000001 2024-09-04T17:46:44.045 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.046 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.046 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:44.046 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.224866+0000", 2024-09-04T17:46:44.046 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00025134799999999998 2024-09-04T17:46:44.046 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:44.046 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:44.046 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:44.046 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.046 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.046 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14370 3.d 3:b763de10:::rbd_data.11345bd5898b.0000000000000002:a [read 114688~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:44.046 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.182867+0000", 2024-09-04T17:46:44.046 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 25.852295244, 2024-09-04T17:46:44.047 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042368301999999997, 2024-09-04T17:46:44.047 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:44.047 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:44.047 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:44.047 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:44.047 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:44.047 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:44.047 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14370 2024-09-04T17:46:44.047 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.047 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:44.047 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.047 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:44.047 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182867+0000", 2024-09-04T17:46:44.047 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:44.048 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.048 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.048 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:44.048 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182867+0000", 2024-09-04T17:46:44.048 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:44.048 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.048 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.048 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:44.048 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182868+0000", 2024-09-04T17:46:44.048 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.006e-06 2024-09-04T17:46:44.048 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.048 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.048 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:44.049 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182875+0000", 2024-09-04T17:46:44.049 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.3990000000000001e-06 2024-09-04T17:46:44.049 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.049 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.049 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:44.049 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182892+0000", 2024-09-04T17:46:44.049 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7096e-05 2024-09-04T17:46:44.049 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.049 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.049 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:44.049 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182905+0000", 2024-09-04T17:46:44.049 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3258000000000001e-05 2024-09-04T17:46:44.049 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.050 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.050 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:44.050 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.224926+0000", 2024-09-04T17:46:44.050 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042020891999999997 2024-09-04T17:46:44.050 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.050 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.050 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:44.050 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.225064+0000", 2024-09-04T17:46:44.050 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000138307 2024-09-04T17:46:44.050 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.050 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.050 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:44.051 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.225236+0000", 2024-09-04T17:46:44.051 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000171344 2024-09-04T17:46:44.051 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:44.051 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:44.051 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:44.051 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.051 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.051 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14377 3.d 3:b763de10:::rbd_data.11345bd5898b.0000000000000002:a [read 225280~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:44.051 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.184021+0000", 2024-09-04T17:46:44.051 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 25.851141638000001, 2024-09-04T17:46:44.051 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042298810999999999, 2024-09-04T17:46:44.051 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:44.052 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:44.052 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:44.052 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:44.052 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:44.052 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:44.052 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14377 2024-09-04T17:46:44.052 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.052 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:44.052 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.052 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:44.052 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184021+0000", 2024-09-04T17:46:44.052 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:44.053 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.053 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.053 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:44.053 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184021+0000", 2024-09-04T17:46:44.053 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:44.053 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.053 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.053 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:44.053 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184022+0000", 2024-09-04T17:46:44.053 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5099999999999999e-06 2024-09-04T17:46:44.053 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.053 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.053 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:44.054 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184028+0000", 2024-09-04T17:46:44.054 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.852e-06 2024-09-04T17:46:44.054 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.054 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.054 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:44.054 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184046+0000", 2024-09-04T17:46:44.054 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7822e-05 2024-09-04T17:46:44.054 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.054 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.054 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:44.054 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184059+0000", 2024-09-04T17:46:44.054 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3372e-05 2024-09-04T17:46:44.055 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.055 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.055 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:44.055 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.226009+0000", 2024-09-04T17:46:44.055 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.041949187999999998 2024-09-04T17:46:44.055 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.055 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.055 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:44.055 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.226142+0000", 2024-09-04T17:46:44.055 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013381299999999999 2024-09-04T17:46:44.055 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.055 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.055 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:44.056 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.226320+0000", 2024-09-04T17:46:44.056 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00017725400000000001 2024-09-04T17:46:44.056 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:44.056 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:44.056 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:44.056 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.056 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.056 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14381 3.d 3:b763de10:::rbd_data.11345bd5898b.0000000000000002:a [read 552960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:44.056 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.184135+0000", 2024-09-04T17:46:44.056 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 25.851027586000001, 2024-09-04T17:46:44.056 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042553818, 2024-09-04T17:46:44.056 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:44.056 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:44.057 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:44.057 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:44.057 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:44.057 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:44.057 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14381 2024-09-04T17:46:44.057 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.057 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:44.057 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.057 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:44.057 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184135+0000", 2024-09-04T17:46:44.057 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:44.057 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.057 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.058 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:44.058 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184135+0000", 2024-09-04T17:46:44.058 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:44.058 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.058 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.058 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:44.058 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184136+0000", 2024-09-04T17:46:44.058 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1200000000000001e-06 2024-09-04T17:46:44.058 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.058 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.058 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:44.058 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184142+0000", 2024-09-04T17:46:44.058 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4840000000000003e-06 2024-09-04T17:46:44.058 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.059 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.059 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:44.059 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184157+0000", 2024-09-04T17:46:44.059 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5727999999999999e-05 2024-09-04T17:46:44.059 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.059 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.059 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:44.059 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184194+0000", 2024-09-04T17:46:44.059 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7049999999999999e-05 2024-09-04T17:46:44.059 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.059 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.060 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:44.060 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.226376+0000", 2024-09-04T17:46:44.060 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042181933999999997 2024-09-04T17:46:44.060 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.060 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.060 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:44.060 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.226522+0000", 2024-09-04T17:46:44.060 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014602899999999999 2024-09-04T17:46:44.060 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.060 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.060 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:44.060 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.226689+0000", 2024-09-04T17:46:44.061 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00016647299999999999 2024-09-04T17:46:44.061 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:44.061 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:44.061 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:44.061 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.061 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.061 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14383 3.d 3:b763de10:::rbd_data.11345bd5898b.0000000000000002:a [read 589824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:44.061 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.184302+0000", 2024-09-04T17:46:44.061 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 25.850860587, 2024-09-04T17:46:44.061 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042760362000000003, 2024-09-04T17:46:44.061 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:44.062 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:44.062 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:44.062 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:44.062 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:44.062 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:44.062 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14383 2024-09-04T17:46:44.062 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.062 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:44.062 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.062 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:44.062 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184302+0000", 2024-09-04T17:46:44.062 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:44.063 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.063 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.063 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:44.063 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184302+0000", 2024-09-04T17:46:44.063 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:44.063 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.063 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.063 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:44.063 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184303+0000", 2024-09-04T17:46:44.063 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.206e-06 2024-09-04T17:46:44.064 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.064 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.064 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:44.064 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184340+0000", 2024-09-04T17:46:44.064 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7005999999999998e-05 2024-09-04T17:46:44.064 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.064 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.064 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:44.064 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184382+0000", 2024-09-04T17:46:44.065 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2345999999999997e-05 2024-09-04T17:46:44.065 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.065 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.065 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:44.065 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184397+0000", 2024-09-04T17:46:44.065 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4729000000000001e-05 2024-09-04T17:46:44.065 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.065 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.065 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:44.065 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.226749+0000", 2024-09-04T17:46:44.066 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042352278 2024-09-04T17:46:44.066 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.066 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.066 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:44.066 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.226889+0000", 2024-09-04T17:46:44.066 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000139376 2024-09-04T17:46:44.066 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.066 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.066 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:44.066 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.227062+0000", 2024-09-04T17:46:44.066 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00017342099999999999 2024-09-04T17:46:44.067 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:44.067 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:44.067 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:44.067 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.067 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.067 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14385 3.d 3:b763de10:::rbd_data.11345bd5898b.0000000000000002:a [read 626688~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:44.067 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.184481+0000", 2024-09-04T17:46:44.067 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 25.850681924, 2024-09-04T17:46:44.067 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042873850999999998, 2024-09-04T17:46:44.067 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:44.067 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:44.067 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:44.068 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:44.068 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:44.068 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:44.068 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14385 2024-09-04T17:46:44.068 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.068 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:44.068 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.068 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:44.068 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184481+0000", 2024-09-04T17:46:44.068 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:44.069 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.069 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.069 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:44.069 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184481+0000", 2024-09-04T17:46:44.069 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:44.069 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.069 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.069 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:44.069 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184482+0000", 2024-09-04T17:46:44.069 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2130000000000001e-06 2024-09-04T17:46:44.069 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.069 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.070 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:44.070 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184487+0000", 2024-09-04T17:46:44.070 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4260000000000002e-06 2024-09-04T17:46:44.070 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.070 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.070 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:44.070 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184505+0000", 2024-09-04T17:46:44.070 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7659000000000002e-05 2024-09-04T17:46:44.070 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.070 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.070 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:44.070 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184518+0000", 2024-09-04T17:46:44.070 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2982e-05 2024-09-04T17:46:44.071 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.071 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.071 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:44.071 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.227120+0000", 2024-09-04T17:46:44.071 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042601858999999999 2024-09-04T17:46:44.071 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.071 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.071 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:44.071 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.227196+0000", 2024-09-04T17:46:44.071 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.6540999999999998e-05 2024-09-04T17:46:44.071 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.071 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.072 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:44.072 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.227354+0000", 2024-09-04T17:46:44.072 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000158171 2024-09-04T17:46:44.072 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:44.072 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:44.072 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:44.072 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.072 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.072 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14389 3.d 3:b763de10:::rbd_data.11345bd5898b.0000000000000002:a [read 663552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:44.072 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.184555+0000", 2024-09-04T17:46:44.073 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 25.850607788000001, 2024-09-04T17:46:44.073 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043553640999999997, 2024-09-04T17:46:44.073 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:44.073 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:44.073 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:44.073 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:44.073 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:44.073 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:44.073 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14389 2024-09-04T17:46:44.073 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.074 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:44.074 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.074 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:44.074 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184555+0000", 2024-09-04T17:46:44.074 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:44.074 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.074 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.075 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:44.075 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184555+0000", 2024-09-04T17:46:44.075 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:44.075 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.075 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.075 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:44.075 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184556+0000", 2024-09-04T17:46:44.075 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.353e-06 2024-09-04T17:46:44.075 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.075 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.075 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:44.075 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184562+0000", 2024-09-04T17:46:44.076 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7899999999999996e-06 2024-09-04T17:46:44.076 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.076 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.076 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:44.076 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184580+0000", 2024-09-04T17:46:44.076 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7682000000000001e-05 2024-09-04T17:46:44.076 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.076 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.076 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:44.076 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184593+0000", 2024-09-04T17:46:44.077 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3625000000000001e-05 2024-09-04T17:46:44.077 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.077 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.077 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:44.077 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.227413+0000", 2024-09-04T17:46:44.077 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042819572 2024-09-04T17:46:44.077 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.077 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.077 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:44.077 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.227552+0000", 2024-09-04T17:46:44.077 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000138842 2024-09-04T17:46:44.078 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.078 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.078 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:44.078 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228108+0000", 2024-09-04T17:46:44.078 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00055677700000000005 2024-09-04T17:46:44.078 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:44.078 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:44.078 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:44.078 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.078 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.078 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14392 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:44.078 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.184613+0000", 2024-09-04T17:46:44.078 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 25.850549526000002, 2024-09-04T17:46:44.079 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043677522000000003, 2024-09-04T17:46:44.079 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:44.079 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:44.079 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:44.079 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:44.079 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:44.079 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:44.079 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14392 2024-09-04T17:46:44.079 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.080 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:44.080 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.080 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:44.080 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184613+0000", 2024-09-04T17:46:44.080 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:44.080 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.080 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.080 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:44.080 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184613+0000", 2024-09-04T17:46:44.080 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:44.080 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.080 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.081 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:44.081 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184614+0000", 2024-09-04T17:46:44.081 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.35e-06 2024-09-04T17:46:44.081 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.081 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.081 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:44.081 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184620+0000", 2024-09-04T17:46:44.081 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.5589999999999996e-06 2024-09-04T17:46:44.081 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.081 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.081 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:44.081 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184671+0000", 2024-09-04T17:46:44.082 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0615000000000002e-05 2024-09-04T17:46:44.082 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.082 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.082 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:44.082 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184684+0000", 2024-09-04T17:46:44.082 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3596e-05 2024-09-04T17:46:44.082 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.082 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.082 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:44.082 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228277+0000", 2024-09-04T17:46:44.082 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043592509000000002 2024-09-04T17:46:44.082 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.082 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.082 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:44.082 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228291+0000", 2024-09-04T17:46:44.083 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3893000000000001e-05 2024-09-04T17:46:44.083 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:44.083 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:44.083 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:44.083 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.083 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.083 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14395 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:44.083 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.184708+0000", 2024-09-04T17:46:44.083 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 25.850454334999998, 2024-09-04T17:46:44.083 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043643612999999998, 2024-09-04T17:46:44.083 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:44.083 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:44.083 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:44.083 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:44.083 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:44.084 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:44.084 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14395 2024-09-04T17:46:44.084 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.084 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:44.084 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.084 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:44.084 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184708+0000", 2024-09-04T17:46:44.084 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:44.084 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.084 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.084 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:44.084 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184708+0000", 2024-09-04T17:46:44.084 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:44.085 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.085 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.085 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:44.085 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184709+0000", 2024-09-04T17:46:44.085 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2359999999999999e-06 2024-09-04T17:46:44.085 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.085 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.085 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:44.085 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184716+0000", 2024-09-04T17:46:44.085 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.3500000000000002e-06 2024-09-04T17:46:44.085 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.086 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.086 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:44.086 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184733+0000", 2024-09-04T17:46:44.086 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6775000000000001e-05 2024-09-04T17:46:44.086 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.086 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.086 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:44.086 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184777+0000", 2024-09-04T17:46:44.086 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4183e-05 2024-09-04T17:46:44.086 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.086 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.086 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:44.086 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228339+0000", 2024-09-04T17:46:44.087 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043562030000000002 2024-09-04T17:46:44.087 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.087 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.087 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:44.087 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228352+0000", 2024-09-04T17:46:44.087 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3039e-05 2024-09-04T17:46:44.087 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:44.087 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:44.087 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:44.087 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.087 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.087 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14398 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:44.087 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.184847+0000", 2024-09-04T17:46:44.087 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 25.850315687999998, 2024-09-04T17:46:44.088 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043564139000000002, 2024-09-04T17:46:44.088 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:44.088 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:44.088 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:44.088 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:44.088 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:44.088 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:44.088 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14398 2024-09-04T17:46:44.088 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.088 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:44.088 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.088 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:44.088 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184847+0000", 2024-09-04T17:46:44.089 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:44.089 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.089 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.089 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:44.089 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184847+0000", 2024-09-04T17:46:44.089 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:44.089 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.089 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.089 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:44.089 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184848+0000", 2024-09-04T17:46:44.089 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.28e-06 2024-09-04T17:46:44.089 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.089 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.090 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:44.090 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184861+0000", 2024-09-04T17:46:44.090 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3125000000000001e-05 2024-09-04T17:46:44.090 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.090 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.090 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:44.090 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184884+0000", 2024-09-04T17:46:44.090 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2357999999999998e-05 2024-09-04T17:46:44.090 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.090 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.090 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:44.090 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184900+0000", 2024-09-04T17:46:44.091 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6872e-05 2024-09-04T17:46:44.091 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.091 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.091 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:44.091 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228398+0000", 2024-09-04T17:46:44.091 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043497313000000003 2024-09-04T17:46:44.091 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.091 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.091 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:44.091 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228411+0000", 2024-09-04T17:46:44.091 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3191e-05 2024-09-04T17:46:44.091 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:44.091 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:44.091 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:44.092 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.092 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.092 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14399 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:44.092 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.184925+0000", 2024-09-04T17:46:44.092 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 25.850237371999999, 2024-09-04T17:46:44.092 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043544799000000002, 2024-09-04T17:46:44.092 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:44.092 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:44.092 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:44.092 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:44.092 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:44.092 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:44.092 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14399 2024-09-04T17:46:44.092 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.093 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:44.093 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.093 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:44.093 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184925+0000", 2024-09-04T17:46:44.093 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:44.093 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.093 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.093 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:44.093 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184925+0000", 2024-09-04T17:46:44.093 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:44.093 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.093 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.093 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:44.094 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184927+0000", 2024-09-04T17:46:44.094 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7e-06 2024-09-04T17:46:44.094 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.094 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.094 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:44.094 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184949+0000", 2024-09-04T17:46:44.094 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1880999999999999e-05 2024-09-04T17:46:44.094 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.094 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.094 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:44.094 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184969+0000", 2024-09-04T17:46:44.094 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0758e-05 2024-09-04T17:46:44.095 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.095 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.095 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:44.095 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184983+0000", 2024-09-04T17:46:44.095 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3336000000000001e-05 2024-09-04T17:46:44.095 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.095 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.095 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:44.095 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228456+0000", 2024-09-04T17:46:44.095 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043472960999999997 2024-09-04T17:46:44.095 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.095 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.095 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:44.096 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228470+0000", 2024-09-04T17:46:44.096 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4163e-05 2024-09-04T17:46:44.096 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:44.096 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:44.096 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:44.096 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.096 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.096 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14400 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:44.096 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.185003+0000", 2024-09-04T17:46:44.096 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 25.850159936000001, 2024-09-04T17:46:44.096 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043532659000000001, 2024-09-04T17:46:44.096 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:44.096 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:44.096 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:44.097 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:44.097 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:44.097 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:44.097 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14400 2024-09-04T17:46:44.097 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.097 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:44.097 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.097 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:44.097 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185003+0000", 2024-09-04T17:46:44.097 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:44.097 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.097 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.097 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:44.098 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185003+0000", 2024-09-04T17:46:44.098 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:44.098 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.098 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.098 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:44.098 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185004+0000", 2024-09-04T17:46:44.098 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.294e-06 2024-09-04T17:46:44.098 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.098 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.098 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:44.098 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185010+0000", 2024-09-04T17:46:44.098 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7690000000000002e-06 2024-09-04T17:46:44.098 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.098 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.099 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:44.099 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185029+0000", 2024-09-04T17:46:44.099 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9100999999999999e-05 2024-09-04T17:46:44.099 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.099 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.099 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:44.099 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185043+0000", 2024-09-04T17:46:44.099 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3862e-05 2024-09-04T17:46:44.099 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.099 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.099 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:44.099 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228510+0000", 2024-09-04T17:46:44.100 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043467766999999997 2024-09-04T17:46:44.100 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.100 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.100 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:44.100 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228535+0000", 2024-09-04T17:46:44.100 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4865999999999999e-05 2024-09-04T17:46:44.100 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:44.100 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:44.100 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:44.100 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.100 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.100 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14401 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:44.100 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.185062+0000", 2024-09-04T17:46:44.101 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 25.850100671, 2024-09-04T17:46:44.101 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043556096000000002, 2024-09-04T17:46:44.101 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:44.101 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:44.101 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:44.101 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:44.101 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:44.101 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:44.101 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14401 2024-09-04T17:46:44.101 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.101 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:44.101 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.101 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:44.101 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185062+0000", 2024-09-04T17:46:44.102 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:44.102 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.102 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.102 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:44.102 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185062+0000", 2024-09-04T17:46:44.102 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:44.102 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.102 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.102 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:44.102 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185063+0000", 2024-09-04T17:46:44.102 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.409e-06 2024-09-04T17:46:44.102 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.102 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.102 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:44.103 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185069+0000", 2024-09-04T17:46:44.103 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.5400000000000003e-06 2024-09-04T17:46:44.103 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.103 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.103 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:44.103 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185087+0000", 2024-09-04T17:46:44.103 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8195000000000001e-05 2024-09-04T17:46:44.103 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.103 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.103 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:44.103 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185101+0000", 2024-09-04T17:46:44.103 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3769e-05 2024-09-04T17:46:44.103 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.104 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.104 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:44.104 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228604+0000", 2024-09-04T17:46:44.104 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043503537000000002 2024-09-04T17:46:44.104 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.104 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.104 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:44.104 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228618+0000", 2024-09-04T17:46:44.104 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3645999999999999e-05 2024-09-04T17:46:44.104 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:44.104 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:44.104 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:44.104 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.104 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.105 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14402 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:44.105 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.185121+0000", 2024-09-04T17:46:44.105 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 25.850041377, 2024-09-04T17:46:44.105 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043551542999999998, 2024-09-04T17:46:44.105 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:44.105 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:44.105 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:44.105 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:44.105 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:44.105 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:44.105 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14402 2024-09-04T17:46:44.105 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.105 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:44.106 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.106 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:44.106 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185121+0000", 2024-09-04T17:46:44.106 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:44.106 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.106 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.106 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:44.106 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185121+0000", 2024-09-04T17:46:44.106 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:44.106 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.106 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.106 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:44.106 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185122+0000", 2024-09-04T17:46:44.106 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1769999999999999e-06 2024-09-04T17:46:44.106 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.107 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.107 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:44.107 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185128+0000", 2024-09-04T17:46:44.107 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7790000000000001e-06 2024-09-04T17:46:44.107 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.107 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.107 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:44.107 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185146+0000", 2024-09-04T17:46:44.107 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8114999999999999e-05 2024-09-04T17:46:44.107 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.107 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.107 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:44.107 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185160+0000", 2024-09-04T17:46:44.107 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4158999999999999e-05 2024-09-04T17:46:44.107 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.108 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.108 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:44.108 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228659+0000", 2024-09-04T17:46:44.108 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043498715 2024-09-04T17:46:44.108 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.108 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.108 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:44.108 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228673+0000", 2024-09-04T17:46:44.108 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3597999999999999e-05 2024-09-04T17:46:44.108 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:44.108 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:44.108 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:44.108 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.109 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.109 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14403 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:44.109 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.185183+0000", 2024-09-04T17:46:44.109 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 25.849979329, 2024-09-04T17:46:44.109 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043541467, 2024-09-04T17:46:44.109 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:44.109 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:44.109 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:44.109 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:44.109 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:44.109 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:44.109 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14403 2024-09-04T17:46:44.109 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.109 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:44.110 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.110 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:44.110 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185183+0000", 2024-09-04T17:46:44.110 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:44.110 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.110 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.110 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:44.110 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185183+0000", 2024-09-04T17:46:44.110 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:44.110 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.110 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.110 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:44.111 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185184+0000", 2024-09-04T17:46:44.111 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.156e-06 2024-09-04T17:46:44.111 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.111 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.111 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:44.111 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185190+0000", 2024-09-04T17:46:44.111 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1800000000000004e-06 2024-09-04T17:46:44.111 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.111 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.111 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:44.111 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185207+0000", 2024-09-04T17:46:44.111 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7288000000000001e-05 2024-09-04T17:46:44.112 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.112 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.112 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:44.112 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185221+0000", 2024-09-04T17:46:44.112 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3729e-05 2024-09-04T17:46:44.112 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.112 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.112 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:44.112 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228711+0000", 2024-09-04T17:46:44.112 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043490595 2024-09-04T17:46:44.112 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.112 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.112 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:44.113 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228725+0000", 2024-09-04T17:46:44.113 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3519e-05 2024-09-04T17:46:44.113 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:44.113 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:44.113 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:44.113 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.113 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.113 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14404 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:44.113 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.185239+0000", 2024-09-04T17:46:44.113 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 25.849923190999998, 2024-09-04T17:46:44.113 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043542099000000001, 2024-09-04T17:46:44.113 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:44.114 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:44.114 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:44.114 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:44.114 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:44.114 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:44.114 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14404 2024-09-04T17:46:44.114 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.114 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:44.114 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.114 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:44.114 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185239+0000", 2024-09-04T17:46:44.114 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:44.114 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.114 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.115 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:44.115 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185239+0000", 2024-09-04T17:46:44.115 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:44.115 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.115 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.115 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:44.115 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185240+0000", 2024-09-04T17:46:44.115 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.173e-06 2024-09-04T17:46:44.115 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.115 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.115 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:44.115 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185246+0000", 2024-09-04T17:46:44.115 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7529999999999998e-06 2024-09-04T17:46:44.115 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.116 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.116 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:44.116 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185264+0000", 2024-09-04T17:46:44.116 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7872000000000001e-05 2024-09-04T17:46:44.116 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.116 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.116 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:44.116 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185278+0000", 2024-09-04T17:46:44.116 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3532e-05 2024-09-04T17:46:44.116 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.116 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.116 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:44.116 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228768+0000", 2024-09-04T17:46:44.117 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043490586999999997 2024-09-04T17:46:44.117 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.117 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.117 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:44.117 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228781+0000", 2024-09-04T17:46:44.117 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3182e-05 2024-09-04T17:46:44.117 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:44.117 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:44.117 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:44.117 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.117 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.117 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14405 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:44.117 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.185297+0000", 2024-09-04T17:46:44.117 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 25.849865999999999, 2024-09-04T17:46:44.118 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043535049999999999, 2024-09-04T17:46:44.118 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:44.118 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:44.118 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:44.118 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:44.118 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:44.118 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:44.118 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14405 2024-09-04T17:46:44.118 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.118 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:44.118 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.118 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:44.118 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185297+0000", 2024-09-04T17:46:44.118 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:44.119 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.119 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.119 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:44.119 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185297+0000", 2024-09-04T17:46:44.119 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:44.119 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.119 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.119 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:44.119 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185298+0000", 2024-09-04T17:46:44.119 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.22e-06 2024-09-04T17:46:44.119 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.119 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.119 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:44.120 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185308+0000", 2024-09-04T17:46:44.120 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0173000000000001e-05 2024-09-04T17:46:44.120 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.120 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.120 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:44.120 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185329+0000", 2024-09-04T17:46:44.120 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1087999999999999e-05 2024-09-04T17:46:44.120 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.120 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.120 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:44.120 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185345+0000", 2024-09-04T17:46:44.120 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6035000000000001e-05 2024-09-04T17:46:44.120 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.120 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.121 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:44.121 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228822+0000", 2024-09-04T17:46:44.121 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043477427999999999 2024-09-04T17:46:44.121 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.121 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.121 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:44.121 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228832+0000", 2024-09-04T17:46:44.121 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1060000000000006e-06 2024-09-04T17:46:44.121 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:44.121 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:44.121 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:44.121 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.121 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.122 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14411 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:44.122 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.186171+0000", 2024-09-04T17:46:44.122 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 25.848991176999998, 2024-09-04T17:46:44.122 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.04271581, 2024-09-04T17:46:44.122 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:44.122 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:44.122 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:44.122 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:44.122 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:44.122 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:44.122 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14411 2024-09-04T17:46:44.122 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.122 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:44.123 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.123 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:44.123 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.186171+0000", 2024-09-04T17:46:44.123 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:44.123 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.123 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.123 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:44.123 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.186171+0000", 2024-09-04T17:46:44.123 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:44.123 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.123 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.123 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:44.123 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.186173+0000", 2024-09-04T17:46:44.123 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.012e-06 2024-09-04T17:46:44.123 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.124 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.124 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:44.124 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.186183+0000", 2024-09-04T17:46:44.124 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.7729999999999994e-06 2024-09-04T17:46:44.124 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.124 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.124 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:44.124 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.186205+0000", 2024-09-04T17:46:44.124 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2280999999999999e-05 2024-09-04T17:46:44.124 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.124 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.124 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:44.125 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.186224+0000", 2024-09-04T17:46:44.125 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8695e-05 2024-09-04T17:46:44.125 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.125 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.125 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:44.125 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228873+0000", 2024-09-04T17:46:44.125 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042648798000000002 2024-09-04T17:46:44.125 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.125 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.125 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:44.125 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228887+0000", 2024-09-04T17:46:44.125 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4251e-05 2024-09-04T17:46:44.125 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:44.125 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:44.126 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:44.126 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.126 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.126 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:37359 3.f 3:f29e50a5:::rbd_data.1134c5a43465.0000000000000021:head [stat,write 62464~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-04T17:46:44.126 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.616887+0000", 2024-09-04T17:46:44.126 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.4182752609999998, 2024-09-04T17:46:44.126 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.054183104000000003, 2024-09-04T17:46:44.126 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:44.126 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:44.126 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:44.126 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:44.126 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:44.126 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:44.126 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 37359 2024-09-04T17:46:44.127 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.127 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:44.127 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.127 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:44.127 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.616887+0000", 2024-09-04T17:46:44.127 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:44.127 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.127 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.127 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:44.127 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.616887+0000", 2024-09-04T17:46:44.127 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:44.127 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.127 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.127 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:44.128 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.616889+0000", 2024-09-04T17:46:44.128 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.362e-06 2024-09-04T17:46:44.128 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.128 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.128 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:44.128 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.616904+0000", 2024-09-04T17:46:44.128 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.507e-05 2024-09-04T17:46:44.128 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.128 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.128 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:44.128 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.616920+0000", 2024-09-04T17:46:44.128 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6385999999999999e-05 2024-09-04T17:46:44.128 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.128 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.129 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:44.129 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.616929+0000", 2024-09-04T17:46:44.129 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8540000000000003e-06 2024-09-04T17:46:44.129 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.129 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.129 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:44.129 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.616983+0000", 2024-09-04T17:46:44.129 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4218999999999997e-05 2024-09-04T17:46:44.129 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.129 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.129 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:44.129 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617065+0000", 2024-09-04T17:46:44.129 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.1864000000000001e-05 2024-09-04T17:46:44.130 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.130 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.130 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 4", 2024-09-04T17:46:44.130 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617205+0000", 2024-09-04T17:46:44.130 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013962600000000001 2024-09-04T17:46:44.130 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.130 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.130 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:44.130 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.618670+0000", 2024-09-04T17:46:44.130 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.001465647 2024-09-04T17:46:44.130 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.130 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.130 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:44.130 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670962+0000", 2024-09-04T17:46:44.130 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.052292000999999998 2024-09-04T17:46:44.131 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.131 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.131 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:44.131 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671011+0000", 2024-09-04T17:46:44.131 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9036000000000003e-05 2024-09-04T17:46:44.131 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:44.131 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:44.131 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:44.131 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671070+0000", 2024-09-04T17:46:44.131 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.9039000000000002e-05 2024-09-04T17:46:44.131 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:44.131 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:44.131 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:44.131 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:44.131 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:44.132 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:44.133 INFO:teuthology.orchestra.run.smithi092.stdout:2302 trunc from 0xed4c067 to 0x278a56b 2024-09-04T17:46:44.133 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:44.133 INFO:teuthology.orchestra.run.smithi092.stdout:2304 punch from 0x1e90803 to 0x1e93bce, (0x33cb bytes) 2024-09-04T17:46:44.134 INFO:teuthology.orchestra.run.smithi092.stdout:2305 trunc from 0x278a56b to 0x6b3313b 2024-09-04T17:46:44.134 INFO:teuthology.orchestra.run.smithi092.stdout:2307 writesame 0x33b02b0 thru 0x33b4cfb data_size 0x1e(0x4a4c bytes) 2024-09-04T17:46:44.134 INFO:teuthology.orchestra.run.smithi092.stdout:2308 write 0x9e1e588 thru 0x9e205de (0x2057 bytes) 2024-09-04T17:46:44.134 INFO:teuthology.orchestra.run.smithi092.stdout:2310 trunc from 0x9e205df to 0x5909501 2024-09-04T17:46:44.134 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-04T17:46:44.134 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:44.131+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:44.144 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:44.144 INFO:teuthology.orchestra.run.smithi092.stdout:2313 punch from 0x4478c34 to 0x4480c7f, (0x804b bytes) 2024-09-04T17:46:44.144 INFO:teuthology.orchestra.run.smithi092.stdout:2314 punch from 0x1da4f5 to 0x1e2ab0, (0x85bb bytes) 2024-09-04T17:46:44.145 INFO:teuthology.orchestra.run.smithi092.stdout:2315 read 0x49ad3c9 thru 0x49b20bf (0x4cf7 bytes) 2024-09-04T17:46:44.145 INFO:teuthology.orchestra.run.smithi092.stdout:2316 punch from 0x2269309 to 0x2278df7, (0xfaee bytes) 2024-09-04T17:46:44.157 INFO:teuthology.orchestra.run.smithi092.stdout:2317 write 0x92bf0eb thru 0x92bf8d6 (0x7ec bytes) 2024-09-04T17:46:44.165 INFO:teuthology.orchestra.run.smithi092.stdout:2318 punch from 0x520d506 to 0x52185a1, (0xb09b bytes) 2024-09-04T17:46:44.165 INFO:teuthology.orchestra.run.smithi092.stdout:2319 punch from 0x3fe48c5 to 0x3ff2297, (0xd9d2 bytes) 2024-09-04T17:46:44.165 INFO:teuthology.orchestra.run.smithi092.stdout:2320 punch from 0x3ce8e6 to 0x3cf5ed, (0xd07 bytes) 2024-09-04T17:46:44.165 INFO:teuthology.orchestra.run.smithi092.stdout:2321 read 0x4f0c321 thru 0x4f0e594 (0x2274 bytes) 2024-09-04T17:46:44.175 INFO:teuthology.orchestra.run.smithi092.stdout:2322 write 0xaed4043 thru 0xaed751b (0x34d9 bytes) 2024-09-04T17:46:44.193 INFO:teuthology.orchestra.run.smithi092.stdout:2324 write 0x20535ef thru 0x205c2ca (0x8cdc bytes) 2024-09-04T17:46:44.196 INFO:teuthology.orchestra.run.smithi092.stdout:2325 write 0x2416c80 thru 0x2425163 (0xe4e4 bytes) 2024-09-04T17:46:44.209 INFO:teuthology.orchestra.run.smithi092.stdout:2326 read 0x4cdbd54 thru 0x4cdc511 (0x7be bytes) 2024-09-04T17:46:44.209 INFO:teuthology.orchestra.run.smithi092.stdout:2328 write 0x7fd6ddb thru 0x7fe27d0 (0xb9f6 bytes) 2024-09-04T17:46:44.220 INFO:teuthology.orchestra.run.smithi092.stdout:2329 punch from 0x7fb6faf to 0x7fbe476, (0x74c7 bytes) 2024-09-04T17:46:44.220 INFO:teuthology.orchestra.run.smithi092.stdout:2330 write 0xa9cf063 thru 0xa9d7507 (0x84a5 bytes) 2024-09-04T17:46:44.234 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:44.233+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:44.238 INFO:teuthology.orchestra.run.smithi092.stdout:2333 writesame 0xc2ededd thru 0xc2f9275 data_size 0x1(0xb399 bytes) 2024-09-04T17:46:44.239 INFO:teuthology.orchestra.run.smithi092.stdout:2335 read 0x2195d0e thru 0x21a2d33 (0xd026 bytes) 2024-09-04T17:46:44.239 INFO:teuthology.orchestra.run.smithi092.stdout:2337 write 0x5b3a1be thru 0x5b42442 (0x8285 bytes) 2024-09-04T17:46:44.251 INFO:teuthology.orchestra.run.smithi092.stdout:2338 punch from 0x8b55173 to 0x8b612de, (0xc16b bytes) 2024-09-04T17:46:44.251 INFO:teuthology.orchestra.run.smithi092.stdout:2339 writesame 0x655f405 thru 0x6564f26 data_size 0x5(0x5b22 bytes) 2024-09-04T17:46:44.262 INFO:teuthology.orchestra.run.smithi092.stdout:2340 read 0x8bc9683 thru 0x8bd8a0d (0xf38b bytes) 2024-09-04T17:46:44.263 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:44.264 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:46:44.264 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:46:44.264 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:44.268 INFO:teuthology.orchestra.run.smithi092.stdout:2342 write 0xc484c25 thru 0xc48ce6a (0x8246 bytes) 2024-09-04T17:46:44.271 INFO:teuthology.orchestra.run.smithi190.stdout: marking collection for removal 2024-09-04T17:46:44.271 INFO:teuthology.orchestra.run.smithi190.stdout:setting '_remove' omap key 2024-09-04T17:46:44.271 INFO:teuthology.orchestra.run.smithi190.stdout:finish_remove_pgs 3.d_head removing 3.d 2024-09-04T17:46:44.277 INFO:teuthology.orchestra.run.smithi092.stdout:2343 read 0xb410be9 thru 0xb41db3b (0xcf53 bytes) 2024-09-04T17:46:44.277 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:44.277 INFO:teuthology.orchestra.run.smithi092.stdout:2345 read 0x3d5a670 thru 0x3d664fb (0xbe8c bytes) 2024-09-04T17:46:44.280 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-04T17:46:44.284 INFO:teuthology.orchestra.run.smithi190.stdout:Remove successful 2024-09-04T17:46:44.295 INFO:teuthology.orchestra.run.smithi092.stdout:2346 trunc from 0xc48ce6b to 0xe2082e 2024-09-04T17:46:44.333 INFO:teuthology.orchestra.run.smithi092.stdout:2347 write 0x4558f1c thru 0x455d9ef (0x4ad4 bytes) 2024-09-04T17:46:44.334 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:44.333+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:44.343 INFO:teuthology.orchestra.run.smithi092.stdout:2348 trunc from 0x455d9f0 to 0x3372bb4 2024-09-04T17:46:44.356 INFO:teuthology.orchestra.run.smithi092.stdout:2349 punch from 0x81b8fe to 0x8234b6, (0x7bb8 bytes) 2024-09-04T17:46:44.356 INFO:teuthology.orchestra.run.smithi092.stdout:2350 punch from 0x10c77d3 to 0x10d2139, (0xa966 bytes) 2024-09-04T17:46:44.371 INFO:teuthology.orchestra.run.smithi092.stdout:2351 write 0x8771646 thru 0x8771f6d (0x928 bytes) 2024-09-04T17:46:44.378 INFO:teuthology.orchestra.run.smithi092.stdout:2352 read 0x241b513 thru 0x2420159 (0x4c47 bytes) 2024-09-04T17:46:44.378 INFO:teuthology.orchestra.run.smithi092.stdout:2354 read 0x5c436df thru 0x5c45413 (0x1d35 bytes) 2024-09-04T17:46:44.378 INFO:teuthology.orchestra.run.smithi092.stdout:2357 read 0x4167135 thru 0x4174316 (0xd1e2 bytes) 2024-09-04T17:46:44.379 INFO:teuthology.orchestra.run.smithi092.stdout:2358 read 0x75b64d0 thru 0x75c1ff9 (0xbb2a bytes) 2024-09-04T17:46:44.379 INFO:teuthology.orchestra.run.smithi092.stdout:2359 read 0x103642f thru 0x103d137 (0x6d09 bytes) 2024-09-04T17:46:44.379 INFO:teuthology.orchestra.run.smithi092.stdout:2360 punch from 0x2f2171b to 0x2f27146, (0x5a2b bytes) 2024-09-04T17:46:44.379 INFO:teuthology.orchestra.run.smithi092.stdout:2361 read 0x65b908f thru 0x65bc4b8 (0x342a bytes) 2024-09-04T17:46:44.379 INFO:teuthology.orchestra.run.smithi092.stdout:2362 writesame 0x788894d thru 0x7894314 data_size 0x52(0xb9c8 bytes) 2024-09-04T17:46:44.389 INFO:teuthology.orchestra.run.smithi092.stdout:2364 write 0x3968985 thru 0x39746b0 (0xbd2c bytes) 2024-09-04T17:46:44.405 INFO:teuthology.orchestra.run.smithi092.stdout:2365 punch from 0x7cd027a to 0x7cdcc78, (0xc9fe bytes) 2024-09-04T17:46:44.420 INFO:teuthology.orchestra.run.smithi092.stdout:2368 writesame 0xb9d3f54 thru 0xb9d8408 data_size 0x29(0x44b5 bytes) 2024-09-04T17:46:44.421 INFO:teuthology.orchestra.run.smithi092.stdout:2369 write 0x36926a3 thru 0x369450c (0x1e6a bytes) 2024-09-04T17:46:44.431 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:44.431 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:46:44.431 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:46:44.431 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:46:44.431 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:44.436 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:44.433+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:44.449 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-04T17:46:44.455 INFO:teuthology.orchestra.run.smithi092.stdout:2370 writesame 0xda4d28a thru 0xda50b08 data_size 0x3(0x387f bytes) 2024-09-04T17:46:44.455 INFO:teuthology.orchestra.run.smithi092.stdout:2371 read 0x6fbfa0d thru 0x6fc13d1 (0x19c5 bytes) 2024-09-04T17:46:44.455 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:44.455 INFO:teuthology.orchestra.run.smithi092.stdout:2375 write 0xcb03bab thru 0xcb129f0 (0xee46 bytes) 2024-09-04T17:46:44.475 INFO:teuthology.orchestra.run.smithi092.stdout:2376 punch from 0x891481f to 0x891b13d, (0x691e bytes) 2024-09-04T17:46:44.475 INFO:teuthology.orchestra.run.smithi092.stdout:2378 writesame 0x2973987 thru 0x2974d46 data_size 0x4(0x13c0 bytes) 2024-09-04T17:46:44.491 INFO:teuthology.orchestra.run.smithi092.stdout:2379 read 0xbf08b72 thru 0xbf10198 (0x7627 bytes) 2024-09-04T17:46:44.491 INFO:teuthology.orchestra.run.smithi092.stdout:2381 read 0x934affc thru 0x934f38f (0x4394 bytes) 2024-09-04T17:46:44.505 INFO:teuthology.orchestra.run.smithi092.stdout:2382 trunc from 0xda50b09 to 0x45e87e4 2024-09-04T17:46:44.532 INFO:teuthology.orchestra.run.smithi092.stdout:2384 punch from 0x1ca4c97 to 0x1ca5b81, (0xeea bytes) 2024-09-04T17:46:44.532 INFO:teuthology.orchestra.run.smithi092.stdout:2385 read 0x2816f31 thru 0x281ed20 (0x7df0 bytes) 2024-09-04T17:46:44.536 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:46:44.535+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:46:44.550 INFO:teuthology.orchestra.run.smithi092.stdout:2386 write 0x8a0e3c4 thru 0x8a1c480 (0xe0bd bytes) 2024-09-04T17:46:44.567 INFO:teuthology.orchestra.run.smithi092.stdout:2387 read 0x4018ebb thru 0x4028544 (0xf68a bytes) 2024-09-04T17:46:44.568 INFO:teuthology.orchestra.run.smithi092.stdout:2388 punch from 0x6cb1a01 to 0x6cb9fa5, (0x85a4 bytes) 2024-09-04T17:46:44.568 INFO:teuthology.orchestra.run.smithi092.stdout:2390 write 0x64a2189 thru 0x64a8359 (0x61d1 bytes) 2024-09-04T17:46:44.582 INFO:teuthology.orchestra.run.smithi092.stdout:2392 writesame 0x5187dca thru 0x5189b27 data_size 0x6(0x1d5e bytes) 2024-09-04T17:46:44.596 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:44.596 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:46:44.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:46:44.597 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:46:44.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.597 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:27242 3.b 3.34414deb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-04T17:46:44.597 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:28.402347+0000", 2024-09-04T17:46:44.597 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 16.192366699000001, 2024-09-04T17:46:44.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.93265329299999999, 2024-09-04T17:46:44.597 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:44.597 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:44.597 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-04T17:46:44.597 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:44.597 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:44.598 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:44.598 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 27242 2024-09-04T17:46:44.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.598 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:44.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:44.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402347+0000", 2024-09-04T17:46:44.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:44.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:44.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402347+0000", 2024-09-04T17:46:44.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:44.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:44.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402349+0000", 2024-09-04T17:46:44.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.471e-06 2024-09-04T17:46:44.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:44.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402355+0000", 2024-09-04T17:46:44.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8399999999999997e-06 2024-09-04T17:46:44.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:44.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402364+0000", 2024-09-04T17:46:44.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1269999999999992e-06 2024-09-04T17:46:44.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:44.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402373+0000", 2024-09-04T17:46:44.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7909999999999993e-06 2024-09-04T17:46:44.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:44.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402460+0000", 2024-09-04T17:46:44.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6686000000000003e-05 2024-09-04T17:46:44.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-04T17:46:44.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402473+0000", 2024-09-04T17:46:44.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3281e-05 2024-09-04T17:46:44.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:44.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:29.334989+0000", 2024-09-04T17:46:44.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.93251549099999997 2024-09-04T17:46:44.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:44.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:29.335000+0000", 2024-09-04T17:46:44.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1606e-05 2024-09-04T17:46:44.602 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:44.602 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:44.602 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:44.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.602 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:33552 3.7 3:e79ad1db:::rbd_data.11342e8f3384.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-04T17:46:44.602 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:35.999748+0000", 2024-09-04T17:46:44.602 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.5949656490000006, 2024-09-04T17:46:44.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045683823999999998, 2024-09-04T17:46:44.602 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:44.602 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:44.603 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:44.603 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:44.603 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:44.603 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:44.603 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 33552 2024-09-04T17:46:44.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.603 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:44.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:44.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999748+0000", 2024-09-04T17:46:44.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:44.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:44.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999748+0000", 2024-09-04T17:46:44.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:44.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:44.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999750+0000", 2024-09-04T17:46:44.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1299999999999999e-06 2024-09-04T17:46:44.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:44.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999759+0000", 2024-09-04T17:46:44.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4649999999999994e-06 2024-09-04T17:46:44.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:44.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999778+0000", 2024-09-04T17:46:44.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8833000000000001e-05 2024-09-04T17:46:44.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:44.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999786+0000", 2024-09-04T17:46:44.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7369999999999995e-06 2024-09-04T17:46:44.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:44.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999856+0000", 2024-09-04T17:46:44.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9247999999999998e-05 2024-09-04T17:46:44.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:44.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999973+0000", 2024-09-04T17:46:44.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000117921 2024-09-04T17:46:44.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 7", 2024-09-04T17:46:44.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.005955+0000", 2024-09-04T17:46:44.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059820560000000004 2024-09-04T17:46:44.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:46:44.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.045297+0000", 2024-09-04T17:46:44.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.039341650999999998 2024-09-04T17:46:44.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:44.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.045343+0000", 2024-09-04T17:46:44.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5899000000000002e-05 2024-09-04T17:46:44.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:44.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.045388+0000", 2024-09-04T17:46:44.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4601999999999997e-05 2024-09-04T17:46:44.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:44.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.045432+0000", 2024-09-04T17:46:44.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4282000000000001e-05 2024-09-04T17:46:44.608 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:44.608 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:44.608 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:44.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.608 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:36981 3.8 3:1148ac0b:::rbd_data.1134bad081d3.000000000000000f:14 [read 432128~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-04T17:46:44.608 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:39.310030+0000", 2024-09-04T17:46:44.608 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.2846834989999998, 2024-09-04T17:46:44.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016093505000000001, 2024-09-04T17:46:44.609 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:44.609 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:44.609 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:44.609 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:44.609 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:44.609 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:44.609 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 36981 2024-09-04T17:46:44.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.609 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:44.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:44.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310030+0000", 2024-09-04T17:46:44.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:44.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:44.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310030+0000", 2024-09-04T17:46:44.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:44.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:44.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310031+0000", 2024-09-04T17:46:44.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3900000000000005e-07 2024-09-04T17:46:44.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:44.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310033+0000", 2024-09-04T17:46:44.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.261e-06 2024-09-04T17:46:44.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:44.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310039+0000", 2024-09-04T17:46:44.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7459999999999997e-06 2024-09-04T17:46:44.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:44.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310043+0000", 2024-09-04T17:46:44.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6970000000000001e-06 2024-09-04T17:46:44.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:44.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.325981+0000", 2024-09-04T17:46:44.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015937647999999999 2024-09-04T17:46:44.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:44.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.326061+0000", 2024-09-04T17:46:44.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9513999999999998e-05 2024-09-04T17:46:44.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:44.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.326124+0000", 2024-09-04T17:46:44.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3100000000000002e-05 2024-09-04T17:46:44.612 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:44.612 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:44.613 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:44.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.613 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:36982 3.8 3:1148ac0b:::rbd_data.1134bad081d3.000000000000000f:14 [read 438272~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-04T17:46:44.613 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:39.310072+0000", 2024-09-04T17:46:44.613 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.2846421460000004, 2024-09-04T17:46:44.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016199199000000001, 2024-09-04T17:46:44.613 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:44.613 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:44.613 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:44.613 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:44.613 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:44.614 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:44.614 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 36982 2024-09-04T17:46:44.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.614 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:44.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:44.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310072+0000", 2024-09-04T17:46:44.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:44.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:44.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310072+0000", 2024-09-04T17:46:44.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:44.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:44.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310072+0000", 2024-09-04T17:46:44.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.72e-07 2024-09-04T17:46:44.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:44.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310073+0000", 2024-09-04T17:46:44.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5650000000000001e-06 2024-09-04T17:46:44.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:44.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310077+0000", 2024-09-04T17:46:44.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7610000000000001e-06 2024-09-04T17:46:44.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:44.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310080+0000", 2024-09-04T17:46:44.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9610000000000001e-06 2024-09-04T17:46:44.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:44.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.326154+0000", 2024-09-04T17:46:44.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016073519000000001 2024-09-04T17:46:44.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:44.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.326200+0000", 2024-09-04T17:46:44.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6575e-05 2024-09-04T17:46:44.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.617 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:44.617 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.326271+0000", 2024-09-04T17:46:44.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0545999999999998e-05 2024-09-04T17:46:44.617 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:44.617 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:44.617 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:44.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.617 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:36983 3.8 3:1148ac0b:::rbd_data.1134bad081d3.000000000000000f:14 [read 444416~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-04T17:46:44.617 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:39.310084+0000", 2024-09-04T17:46:44.617 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.2846297069999997, 2024-09-04T17:46:44.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016332103000000001, 2024-09-04T17:46:44.618 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:44.618 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:44.618 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:44.618 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:44.618 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:44.618 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:44.618 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 36983 2024-09-04T17:46:44.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.618 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:44.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.618 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:44.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310084+0000", 2024-09-04T17:46:44.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:44.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:44.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310084+0000", 2024-09-04T17:46:44.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:44.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:44.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310084+0000", 2024-09-04T17:46:44.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4600000000000001e-07 2024-09-04T17:46:44.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:44.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310086+0000", 2024-09-04T17:46:44.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3540000000000001e-06 2024-09-04T17:46:44.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:44.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310092+0000", 2024-09-04T17:46:44.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9070000000000004e-06 2024-09-04T17:46:44.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:44.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310096+0000", 2024-09-04T17:46:44.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3850000000000004e-06 2024-09-04T17:46:44.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:44.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.326306+0000", 2024-09-04T17:46:44.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016209931 2024-09-04T17:46:44.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:44.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.326356+0000", 2024-09-04T17:46:44.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9854000000000001e-05 2024-09-04T17:46:44.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.622 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:44.622 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.326416+0000", 2024-09-04T17:46:44.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0426e-05 2024-09-04T17:46:44.622 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:44.622 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:44.622 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:44.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.622 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:36984 3.8 3:1148ac0b:::rbd_data.1134bad081d3.000000000000000f:14 [read 450560~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-04T17:46:44.622 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:39.310111+0000", 2024-09-04T17:46:44.622 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.2846028690000004, 2024-09-04T17:46:44.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016436365000000001, 2024-09-04T17:46:44.623 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:44.623 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:44.623 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:44.623 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:44.623 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:44.623 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:44.623 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 36984 2024-09-04T17:46:44.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.623 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:44.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.623 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:44.623 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310111+0000", 2024-09-04T17:46:44.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:44.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:44.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310111+0000", 2024-09-04T17:46:44.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:44.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:44.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310111+0000", 2024-09-04T17:46:44.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1200000000000003e-07 2024-09-04T17:46:44.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:44.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310114+0000", 2024-09-04T17:46:44.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1380000000000001e-06 2024-09-04T17:46:44.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:44.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310119+0000", 2024-09-04T17:46:44.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.857e-06 2024-09-04T17:46:44.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:44.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310124+0000", 2024-09-04T17:46:44.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6829999999999999e-06 2024-09-04T17:46:44.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:44.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.326444+0000", 2024-09-04T17:46:44.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016319796000000001 2024-09-04T17:46:44.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:44.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.326491+0000", 2024-09-04T17:46:44.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6910999999999999e-05 2024-09-04T17:46:44.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:44.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.326547+0000", 2024-09-04T17:46:44.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6468000000000002e-05 2024-09-04T17:46:44.627 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:44.627 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:44.627 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:44.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.627 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:36985 3.8 3:1148ac0b:::rbd_data.1134bad081d3.000000000000000f:14 [read 456704~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-04T17:46:44.627 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:39.310151+0000", 2024-09-04T17:46:44.627 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.2845629560000003, 2024-09-04T17:46:44.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016537973000000001, 2024-09-04T17:46:44.628 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:44.628 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:44.628 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:44.628 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:44.628 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:44.628 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:44.628 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 36985 2024-09-04T17:46:44.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.628 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:44.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:44.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310151+0000", 2024-09-04T17:46:44.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:44.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:44.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310151+0000", 2024-09-04T17:46:44.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:44.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:44.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310151+0000", 2024-09-04T17:46:44.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0800000000000001e-07 2024-09-04T17:46:44.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:44.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310153+0000", 2024-09-04T17:46:44.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.714e-06 2024-09-04T17:46:44.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:44.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310156+0000", 2024-09-04T17:46:44.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.636e-06 2024-09-04T17:46:44.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:44.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310159+0000", 2024-09-04T17:46:44.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.835e-06 2024-09-04T17:46:44.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:44.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.326582+0000", 2024-09-04T17:46:44.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016422929999999999 2024-09-04T17:46:44.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:44.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.326630+0000", 2024-09-04T17:46:44.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8087999999999998e-05 2024-09-04T17:46:44.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:44.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.326689+0000", 2024-09-04T17:46:44.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8461999999999997e-05 2024-09-04T17:46:44.632 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:44.632 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:44.632 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:44.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.632 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:36986 3.8 3:1148ac0b:::rbd_data.1134bad081d3.000000000000000f:14 [read 462848~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-04T17:46:44.632 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:39.310163+0000", 2024-09-04T17:46:44.632 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.2845505790000002, 2024-09-04T17:46:44.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016643022, 2024-09-04T17:46:44.633 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:44.633 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:44.633 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:44.633 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:44.633 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:44.633 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:44.633 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 36986 2024-09-04T17:46:44.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.633 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:44.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:44.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310163+0000", 2024-09-04T17:46:44.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:44.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:44.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310163+0000", 2024-09-04T17:46:44.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:44.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:44.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310163+0000", 2024-09-04T17:46:44.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.22e-07 2024-09-04T17:46:44.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:44.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310165+0000", 2024-09-04T17:46:44.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.542e-06 2024-09-04T17:46:44.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:44.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310171+0000", 2024-09-04T17:46:44.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9100000000000002e-06 2024-09-04T17:46:44.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:44.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310174+0000", 2024-09-04T17:46:44.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.942e-06 2024-09-04T17:46:44.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:44.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.326717+0000", 2024-09-04T17:46:44.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016542787999999999 2024-09-04T17:46:44.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:44.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.326764+0000", 2024-09-04T17:46:44.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7502e-05 2024-09-04T17:46:44.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:44.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.326806+0000", 2024-09-04T17:46:44.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2116000000000001e-05 2024-09-04T17:46:44.637 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:44.637 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:44.637 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:44.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.637 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:36987 3.8 3:1148ac0b:::rbd_data.1134bad081d3.000000000000000f:14 [read 468992~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-04T17:46:44.637 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:39.310189+0000", 2024-09-04T17:46:44.637 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.2845248189999996, 2024-09-04T17:46:44.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016746924999999999, 2024-09-04T17:46:44.637 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:44.637 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:44.637 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:44.638 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:44.638 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:44.638 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:44.638 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 36987 2024-09-04T17:46:44.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.638 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:44.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:44.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310189+0000", 2024-09-04T17:46:44.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:44.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:44.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310189+0000", 2024-09-04T17:46:44.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:44.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:44.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310189+0000", 2024-09-04T17:46:44.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.37e-07 2024-09-04T17:46:44.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:44.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310192+0000", 2024-09-04T17:46:44.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1550000000000001e-06 2024-09-04T17:46:44.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:44.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310199+0000", 2024-09-04T17:46:44.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3209999999999999e-06 2024-09-04T17:46:44.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:44.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310205+0000", 2024-09-04T17:46:44.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9139999999999996e-06 2024-09-04T17:46:44.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:44.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.326841+0000", 2024-09-04T17:46:44.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016635666 2024-09-04T17:46:44.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:44.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.326888+0000", 2024-09-04T17:46:44.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7456000000000001e-05 2024-09-04T17:46:44.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:44.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.326936+0000", 2024-09-04T17:46:44.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7876e-05 2024-09-04T17:46:44.642 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:44.642 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:44.642 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:44.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.642 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:36988 3.8 3:1148ac0b:::rbd_data.1134bad081d3.000000000000000f:14 [read 475136~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-04T17:46:44.642 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:39.310215+0000", 2024-09-04T17:46:44.642 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.2844985940000004, 2024-09-04T17:46:44.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016834186000000001, 2024-09-04T17:46:44.642 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:44.642 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:44.642 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:44.642 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:44.643 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:44.643 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:44.643 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 36988 2024-09-04T17:46:44.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.643 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:44.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:44.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310215+0000", 2024-09-04T17:46:44.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:44.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:44.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310215+0000", 2024-09-04T17:46:44.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:44.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.644 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:44.644 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310216+0000", 2024-09-04T17:46:44.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5499999999999998e-07 2024-09-04T17:46:44.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.644 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:44.644 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310238+0000", 2024-09-04T17:46:44.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2347000000000001e-05 2024-09-04T17:46:44.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:44.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310244+0000", 2024-09-04T17:46:44.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7440000000000004e-06 2024-09-04T17:46:44.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:44.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310247+0000", 2024-09-04T17:46:44.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4319999999999999e-06 2024-09-04T17:46:44.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:44.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.326963+0000", 2024-09-04T17:46:44.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016716101000000001 2024-09-04T17:46:44.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:44.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.327008+0000", 2024-09-04T17:46:44.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4926000000000003e-05 2024-09-04T17:46:44.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:44.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.327049+0000", 2024-09-04T17:46:44.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1081000000000002e-05 2024-09-04T17:46:44.646 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:44.647 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:44.647 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:44.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.647 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:36989 3.8 3:1148ac0b:::rbd_data.1134bad081d3.000000000000000f:14 [read 481280~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-04T17:46:44.647 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:39.310253+0000", 2024-09-04T17:46:44.647 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.2844609179999997, 2024-09-04T17:46:44.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016925381999999999, 2024-09-04T17:46:44.647 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:44.647 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:44.647 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:44.647 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:44.648 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:44.648 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:44.648 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 36989 2024-09-04T17:46:44.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.648 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:44.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.648 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:44.648 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310253+0000", 2024-09-04T17:46:44.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:44.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.648 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:44.648 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310253+0000", 2024-09-04T17:46:44.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:44.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.649 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:44.649 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310253+0000", 2024-09-04T17:46:44.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1899999999999998e-07 2024-09-04T17:46:44.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.649 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:44.649 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310254+0000", 2024-09-04T17:46:44.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.302e-06 2024-09-04T17:46:44.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:44.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310258+0000", 2024-09-04T17:46:44.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5990000000000002e-06 2024-09-04T17:46:44.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:44.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310263+0000", 2024-09-04T17:46:44.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5940000000000004e-06 2024-09-04T17:46:44.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:44.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.327083+0000", 2024-09-04T17:46:44.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016820845000000001 2024-09-04T17:46:44.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:44.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.327131+0000", 2024-09-04T17:46:44.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7302000000000002e-05 2024-09-04T17:46:44.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:44.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.327178+0000", 2024-09-04T17:46:44.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7420999999999999e-05 2024-09-04T17:46:44.651 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:44.652 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:44.652 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:44.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.652 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:36990 3.8 3:1148ac0b:::rbd_data.1134bad081d3.000000000000000f:14 [read 487424~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-04T17:46:44.652 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:39.310268+0000", 2024-09-04T17:46:44.652 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.2844457130000002, 2024-09-04T17:46:44.652 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017021723999999998, 2024-09-04T17:46:44.652 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:44.652 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:44.652 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:44.652 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:44.652 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:44.652 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:44.653 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 36990 2024-09-04T17:46:44.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.653 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:44.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:44.653 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310268+0000", 2024-09-04T17:46:44.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:44.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:44.653 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310268+0000", 2024-09-04T17:46:44.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:44.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:44.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310269+0000", 2024-09-04T17:46:44.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7999999999999995e-07 2024-09-04T17:46:44.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:44.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310271+0000", 2024-09-04T17:46:44.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9920000000000002e-06 2024-09-04T17:46:44.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:44.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310282+0000", 2024-09-04T17:46:44.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.183e-05 2024-09-04T17:46:44.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:44.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310287+0000", 2024-09-04T17:46:44.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7960000000000003e-06 2024-09-04T17:46:44.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:44.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.327206+0000", 2024-09-04T17:46:44.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016918636000000001 2024-09-04T17:46:44.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:44.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.327248+0000", 2024-09-04T17:46:44.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2061999999999998e-05 2024-09-04T17:46:44.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:44.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.327290+0000", 2024-09-04T17:46:44.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1828e-05 2024-09-04T17:46:44.656 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:44.656 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:44.656 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:44.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.656 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:36991 3.8 3:1148ac0b:::rbd_data.1134bad081d3.000000000000000f:14 [read 493568~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-04T17:46:44.656 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:39.310294+0000", 2024-09-04T17:46:44.656 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.284420044, 2024-09-04T17:46:44.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017125095, 2024-09-04T17:46:44.657 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:44.657 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:44.657 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:44.657 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:44.657 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:44.657 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:44.657 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 36991 2024-09-04T17:46:44.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.657 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:44.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.657 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:44.657 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310294+0000", 2024-09-04T17:46:44.657 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:44.658 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.658 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.658 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:44.658 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310294+0000", 2024-09-04T17:46:44.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:44.658 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.658 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.658 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:44.658 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310294+0000", 2024-09-04T17:46:44.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.15e-07 2024-09-04T17:46:44.658 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.658 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.658 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:44.658 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310296+0000", 2024-09-04T17:46:44.659 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1619999999999998e-06 2024-09-04T17:46:44.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.659 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.659 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:44.659 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310302+0000", 2024-09-04T17:46:44.659 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1630000000000001e-06 2024-09-04T17:46:44.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.659 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.659 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:44.659 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310307+0000", 2024-09-04T17:46:44.659 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6099999999999999e-06 2024-09-04T17:46:44.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.660 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:44.660 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.327325+0000", 2024-09-04T17:46:44.660 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017017478999999999 2024-09-04T17:46:44.660 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.660 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:44.660 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.327372+0000", 2024-09-04T17:46:44.660 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7382000000000001e-05 2024-09-04T17:46:44.660 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.660 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:44.660 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.327419+0000", 2024-09-04T17:46:44.661 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6884e-05 2024-09-04T17:46:44.661 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:44.661 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:44.661 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:44.661 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.661 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.661 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:36992 3.8 3:1148ac0b:::rbd_data.1134bad081d3.000000000000000f:14 [read 499712~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-04T17:46:44.661 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:39.310326+0000", 2024-09-04T17:46:44.661 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.2843878990000004, 2024-09-04T17:46:44.661 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017208496, 2024-09-04T17:46:44.661 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:44.661 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:44.661 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:44.661 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:44.662 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:44.662 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:44.662 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 36992 2024-09-04T17:46:44.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.662 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:44.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.662 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:44.662 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310326+0000", 2024-09-04T17:46:44.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:44.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.662 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:44.662 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310326+0000", 2024-09-04T17:46:44.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:44.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:44.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310326+0000", 2024-09-04T17:46:44.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4900000000000002e-07 2024-09-04T17:46:44.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:44.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310328+0000", 2024-09-04T17:46:44.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5e-06 2024-09-04T17:46:44.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:44.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310336+0000", 2024-09-04T17:46:44.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2789999999999998e-06 2024-09-04T17:46:44.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:44.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310339+0000", 2024-09-04T17:46:44.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.909e-06 2024-09-04T17:46:44.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:44.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.327448+0000", 2024-09-04T17:46:44.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017108838000000001 2024-09-04T17:46:44.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:44.665 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.327492+0000", 2024-09-04T17:46:44.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4079000000000003e-05 2024-09-04T17:46:44.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.665 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:44.665 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.327534+0000", 2024-09-04T17:46:44.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2642000000000003e-05 2024-09-04T17:46:44.665 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:44.665 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:44.665 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:44.666 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.666 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:36993 3.8 3:1148ac0b:::rbd_data.1134bad081d3.000000000000000f:14 [read 505856~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-04T17:46:44.666 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:39.310343+0000", 2024-09-04T17:46:44.666 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.2843710789999996, 2024-09-04T17:46:44.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017319060000000001, 2024-09-04T17:46:44.666 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:44.666 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:44.666 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:44.666 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:44.666 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:44.666 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:44.666 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 36993 2024-09-04T17:46:44.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.667 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:44.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.667 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:44.667 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310343+0000", 2024-09-04T17:46:44.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:44.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.667 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:44.667 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310343+0000", 2024-09-04T17:46:44.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:44.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.667 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:44.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310353+0000", 2024-09-04T17:46:44.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0237e-05 2024-09-04T17:46:44.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.668 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:44.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310355+0000", 2024-09-04T17:46:44.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0370000000000001e-06 2024-09-04T17:46:44.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.668 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:44.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310364+0000", 2024-09-04T17:46:44.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7169999999999997e-06 2024-09-04T17:46:44.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.669 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:44.669 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310374+0000", 2024-09-04T17:46:44.669 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0407999999999999e-05 2024-09-04T17:46:44.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.669 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:44.669 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.327568+0000", 2024-09-04T17:46:44.669 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017194313999999999 2024-09-04T17:46:44.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.670 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:44.670 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.327615+0000", 2024-09-04T17:46:44.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6239999999999998e-05 2024-09-04T17:46:44.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.670 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:44.670 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.327662+0000", 2024-09-04T17:46:44.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7107000000000001e-05 2024-09-04T17:46:44.670 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:44.670 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:44.670 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:44.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.670 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:36994 3.8 3:1148ac0b:::rbd_data.1134bad081d3.000000000000000f:14 [read 512000~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-04T17:46:44.671 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:39.310382+0000", 2024-09-04T17:46:44.671 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.2843316490000003, 2024-09-04T17:46:44.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017390433, 2024-09-04T17:46:44.671 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:44.671 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:44.671 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:44.671 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:44.671 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:44.671 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:44.671 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 36994 2024-09-04T17:46:44.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.671 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:44.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.672 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:44.672 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310382+0000", 2024-09-04T17:46:44.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:44.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.672 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:44.672 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.310382+0000", 2024-09-04T17:46:44.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:44.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.672 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:44.672 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.318000+0000", 2024-09-04T17:46:44.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0076178469999999996 2024-09-04T17:46:44.673 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.673 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.673 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:44.673 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.318005+0000", 2024-09-04T17:46:44.673 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.622e-06 2024-09-04T17:46:44.673 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.673 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.673 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:44.673 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.318012+0000", 2024-09-04T17:46:44.673 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2289999999999997e-06 2024-09-04T17:46:44.673 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.673 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.674 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:44.674 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.318017+0000", 2024-09-04T17:46:44.674 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8110000000000002e-06 2024-09-04T17:46:44.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.674 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.674 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:44.674 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.327689+0000", 2024-09-04T17:46:44.674 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0096724270000000008 2024-09-04T17:46:44.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.674 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.674 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:44.674 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.327732+0000", 2024-09-04T17:46:44.674 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2722000000000002e-05 2024-09-04T17:46:44.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.675 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.675 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:44.675 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.327773+0000", 2024-09-04T17:46:44.675 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0775000000000002e-05 2024-09-04T17:46:44.675 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:44.675 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:44.675 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:44.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.675 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.675 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:37121 3.8 3.d0351288 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-04T17:46:44.675 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:39.319726+0000", 2024-09-04T17:46:44.675 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.2749877109999996, 2024-09-04T17:46:44.675 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016063583999999999, 2024-09-04T17:46:44.675 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:44.676 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:44.676 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-04T17:46:44.676 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:44.676 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:44.676 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:44.676 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 37121 2024-09-04T17:46:44.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.676 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:44.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.676 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:44.676 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.319726+0000", 2024-09-04T17:46:44.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:44.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:44.677 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.319726+0000", 2024-09-04T17:46:44.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:44.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:44.677 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.319726+0000", 2024-09-04T17:46:44.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1299999999999999e-07 2024-09-04T17:46:44.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.678 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:44.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.319728+0000", 2024-09-04T17:46:44.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.257e-06 2024-09-04T17:46:44.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.678 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:44.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.319730+0000", 2024-09-04T17:46:44.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8899999999999999e-06 2024-09-04T17:46:44.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.678 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:44.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.319733+0000", 2024-09-04T17:46:44.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5519999999999999e-06 2024-09-04T17:46:44.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.679 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:44.679 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.335785+0000", 2024-09-04T17:46:44.679 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016052055999999999 2024-09-04T17:46:44.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.679 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:44.679 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.335790+0000", 2024-09-04T17:46:44.679 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6160000000000004e-06 2024-09-04T17:46:44.679 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:44.679 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:44.679 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:44.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.680 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:37122 3.8 3.d0351288 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-04T17:46:44.680 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:39.319736+0000", 2024-09-04T17:46:44.680 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.2749775909999999, 2024-09-04T17:46:44.680 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016099481999999998, 2024-09-04T17:46:44.680 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:44.680 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:44.680 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-04T17:46:44.680 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:44.680 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:44.680 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:44.680 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 37122 2024-09-04T17:46:44.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.681 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:44.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:44.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.319736+0000", 2024-09-04T17:46:44.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:44.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:44.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.319736+0000", 2024-09-04T17:46:44.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:44.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:44.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.319736+0000", 2024-09-04T17:46:44.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1e-07 2024-09-04T17:46:44.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:44.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.319737+0000", 2024-09-04T17:46:44.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1000000000000001e-06 2024-09-04T17:46:44.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:44.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.319740+0000", 2024-09-04T17:46:44.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.942e-06 2024-09-04T17:46:44.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.683 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:44.683 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.319743+0000", 2024-09-04T17:46:44.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5270000000000001e-06 2024-09-04T17:46:44.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.683 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:44.683 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.335820+0000", 2024-09-04T17:46:44.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016077504999999999 2024-09-04T17:46:44.684 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.684 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.684 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:44.684 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.335836+0000", 2024-09-04T17:46:44.684 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5197999999999999e-05 2024-09-04T17:46:44.684 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:44.684 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:44.684 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:44.684 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.684 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.684 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:37123 3.8 3.d0351288 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-04T17:46:44.685 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:39.319747+0000", 2024-09-04T17:46:44.685 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.2749670289999999, 2024-09-04T17:46:44.685 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016114317, 2024-09-04T17:46:44.685 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:44.685 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:44.685 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-04T17:46:44.685 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:44.685 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:44.685 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:44.685 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 37123 2024-09-04T17:46:44.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.685 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:44.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:44.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.319747+0000", 2024-09-04T17:46:44.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:44.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:44.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.319747+0000", 2024-09-04T17:46:44.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:44.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:44.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.319747+0000", 2024-09-04T17:46:44.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3300000000000003e-07 2024-09-04T17:46:44.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:44.687 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.319748+0000", 2024-09-04T17:46:44.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.097e-06 2024-09-04T17:46:44.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:44.687 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.319752+0000", 2024-09-04T17:46:44.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1529999999999999e-06 2024-09-04T17:46:44.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.688 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:44.688 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.319755+0000", 2024-09-04T17:46:44.688 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7030000000000002e-06 2024-09-04T17:46:44.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.688 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.688 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:44.688 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.335856+0000", 2024-09-04T17:46:44.688 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016100593 2024-09-04T17:46:44.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.688 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.688 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:44.688 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.335861+0000", 2024-09-04T17:46:44.688 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3380000000000004e-06 2024-09-04T17:46:44.689 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:44.689 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:44.689 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:44.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.689 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:37357 3.15 3:a897312a:::rbd_data.1134c5a43465.000000000000001f:head [stat,write 62464~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-04T17:46:44.689 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:39.616724+0000", 2024-09-04T17:46:44.689 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.9779902409999996, 2024-09-04T17:46:44.689 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054965854000000001, 2024-09-04T17:46:44.689 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:44.689 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:44.689 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:44.689 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:44.690 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:44.690 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:44.690 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 37357 2024-09-04T17:46:44.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.690 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:44.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.690 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:44.690 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616724+0000", 2024-09-04T17:46:44.690 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:44.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.690 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:44.691 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616724+0000", 2024-09-04T17:46:44.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:44.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.691 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:44.691 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616725+0000", 2024-09-04T17:46:44.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.815e-06 2024-09-04T17:46:44.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.691 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:44.691 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616739+0000", 2024-09-04T17:46:44.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3723000000000001e-05 2024-09-04T17:46:44.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.692 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.692 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:44.692 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616755+0000", 2024-09-04T17:46:44.692 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5964000000000002e-05 2024-09-04T17:46:44.692 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.692 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.692 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:44.692 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616763+0000", 2024-09-04T17:46:44.692 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3869999999999995e-06 2024-09-04T17:46:44.692 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.692 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.692 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:44.693 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616825+0000", 2024-09-04T17:46:44.693 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1854999999999996e-05 2024-09-04T17:46:44.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.693 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:44.696 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616895+0000", 2024-09-04T17:46:44.696 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9597999999999993e-05 2024-09-04T17:46:44.696 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.696 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.696 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-04T17:46:44.696 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616991+0000", 2024-09-04T17:46:44.697 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5828000000000005e-05 2024-09-04T17:46:44.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.697 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:46:44.697 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.617659+0000", 2024-09-04T17:46:44.697 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00066821600000000001 2024-09-04T17:46:44.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.697 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:44.697 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.671482+0000", 2024-09-04T17:46:44.697 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053823533999999999 2024-09-04T17:46:44.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.698 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.698 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:44.698 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.671579+0000", 2024-09-04T17:46:44.698 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6657999999999995e-05 2024-09-04T17:46:44.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:44.698 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:44.698 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:44.698 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.671689+0000", 2024-09-04T17:46:44.698 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000110276 2024-09-04T17:46:44.698 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:44.698 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:44.699 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:44.699 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:44.699 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:44.699 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:44.700 INFO:teuthology.orchestra.run.smithi092.stdout:2396 writesame 0x3166e74 thru 0x316e199 data_size 0x33(0x7326 bytes) 2024-09-04T17:46:44.700 INFO:teuthology.orchestra.run.smithi092.stdout:2397 write 0x7d06970 thru 0x7d13cb2 (0xd343 bytes) 2024-09-04T17:46:44.701 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-04T17:46:44.702 INFO:teuthology.orchestra.run.smithi092.stdout:2398 punch from 0x5d37e2b to 0x5d3acef, (0x2ec4 bytes) 2024-09-04T17:46:44.702 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:44.702 INFO:teuthology.orchestra.run.smithi092.stdout:2404 write 0xde5aa86 thru 0xde5fec7 (0x5442 bytes) 2024-09-04T17:46:44.702 INFO:teuthology.orchestra.run.smithi092.stdout:2408 read 0x3ccdc41 thru 0x3cdd1ca (0xf58a bytes) 2024-09-04T17:46:44.702 INFO:teuthology.orchestra.run.smithi092.stdout:2409 trunc from 0xde5fec8 to 0xe223f16 2024-09-04T17:46:44.703 INFO:teuthology.orchestra.run.smithi092.stdout:2411 punch from 0x46d1e3d to 0x46e0686, (0xe849 bytes) 2024-09-04T17:46:44.703 INFO:teuthology.orchestra.run.smithi092.stdout:2412 trunc from 0xe223f16 to 0x22e7de2 2024-09-04T17:46:44.703 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:44.699+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:44.755 INFO:teuthology.orchestra.run.smithi092.stdout:2414 writesame 0xd4ef723 thru 0xd4fb610 data_size 0x1(0xbeee bytes) 2024-09-04T17:46:44.755 INFO:teuthology.orchestra.run.smithi092.stdout:2415 read 0x26649c1 thru 0x266c498 (0x7ad8 bytes) 2024-09-04T17:46:44.755 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:44.756 INFO:teuthology.orchestra.run.smithi092.stdout:2417 punch from 0xaea8424 to 0xaeb02ab, (0x7e87 bytes) 2024-09-04T17:46:44.756 INFO:teuthology.orchestra.run.smithi092.stdout:2418 read 0x1588af4 thru 0x159385e (0xad6b bytes) 2024-09-04T17:46:44.756 INFO:teuthology.orchestra.run.smithi092.stdout:2421 write 0x1322671 thru 0x132bf35 (0x98c5 bytes) 2024-09-04T17:46:44.773 INFO:teuthology.orchestra.run.smithi092.stdout:2425 punch from 0x8a0307 to 0x8a57cd, (0x54c6 bytes) 2024-09-04T17:46:44.784 INFO:teuthology.orchestra.run.smithi092.stdout:2426 trunc from 0xd4fb611 to 0x6976fe6 2024-09-04T17:46:44.795 INFO:teuthology.orchestra.run.smithi092.stdout:2428 read 0x4d794eb thru 0x4d8041f (0x6f35 bytes) 2024-09-04T17:46:44.795 INFO:teuthology.orchestra.run.smithi092.stdout:2429 read 0xa2db95 thru 0xa3a341 (0xc7ad bytes) 2024-09-04T17:46:44.795 INFO:teuthology.orchestra.run.smithi092.stdout:2430 read 0x692712e thru 0x692ff4f (0x8e22 bytes) 2024-09-04T17:46:44.801 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:44.799+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:44.802 INFO:teuthology.orchestra.run.smithi092.stdout:2432 trunc from 0x6976fe6 to 0xb496f42 2024-09-04T17:46:44.809 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-6 --journal-path /var/lib/ceph/osd/ceph-6/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.3.d.6 2024-09-04T17:46:44.811 INFO:teuthology.orchestra.run.smithi092.stdout:2433 punch from 0x218035f to 0x218e295, (0xdf36 bytes) 2024-09-04T17:46:44.811 INFO:teuthology.orchestra.run.smithi092.stdout:2434 write 0x2263987 thru 0x227020e (0xc888 bytes) 2024-09-04T17:46:44.825 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:44.825 INFO:teuthology.orchestra.run.smithi092.stdout:2436 writesame 0x66a52ba thru 0x66b2675 data_size 0x6(0xd3bc bytes) 2024-09-04T17:46:44.844 INFO:teuthology.orchestra.run.smithi092.stdout:2437 write 0x7d9f0de thru 0x7dae3be (0xf2e1 bytes) 2024-09-04T17:46:44.862 INFO:teuthology.orchestra.run.smithi092.stdout:2438 read 0x5078641 thru 0x50797f5 (0x11b5 bytes) 2024-09-04T17:46:44.877 INFO:teuthology.orchestra.run.smithi092.stdout:2441 trunc from 0xb496f42 to 0x216fe07 2024-09-04T17:46:44.886 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:44.887 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:46:44.887 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:46:44.887 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:44.902 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:46:44.899+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:46:44.903 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-04T17:46:44.922 INFO:teuthology.orchestra.run.smithi092.stdout:2442 trunc from 0x216fe07 to 0xc4a5a9f 2024-09-04T17:46:44.931 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:44.931 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:44.931 INFO:teuthology.orchestra.run.smithi092.stdout:2446 write 0x21fd4 thru 0x2d293 (0xb2c0 bytes) 2024-09-04T17:46:44.937 INFO:teuthology.orchestra.run.smithi092.stdout:2447 punch from 0xaf6c595 to 0xaf72daf, (0x681a bytes) 2024-09-04T17:46:44.937 INFO:teuthology.orchestra.run.smithi092.stdout:2449 read 0x3dd4ddf thru 0x3dd684e (0x1a70 bytes) 2024-09-04T17:46:44.951 INFO:teuthology.orchestra.run.smithi092.stdout:2450 trunc from 0xc4a5a9f to 0x3d89495 2024-09-04T17:46:44.963 INFO:teuthology.orchestra.run.smithi092.stdout:2451 read 0x9e86fb thru 0x9f13d3 (0x8cd9 bytes) 2024-09-04T17:46:44.963 INFO:teuthology.orchestra.run.smithi092.stdout:2452 clone 19 order 22 su 2097152 sc 9 2024-09-04T17:46:45.003 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:44.999+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:45.047 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:45.047 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:46:45.047 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:46:45.048 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:46:45.048 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:45.061 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-04T17:46:45.104 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:45.105+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:45.205 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:45.203+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:45.228 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:45.229 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:46:45.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:46:45.229 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:46:45.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.229 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:1095 3.1 3:86c08994:::rbd_data.11346251f094.000000000000006e:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-04T17:46:45.229 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:02.246406+0000", 2024-09-04T17:46:45.229 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.981388744999997, 2024-09-04T17:46:45.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.024871196000000002, 2024-09-04T17:46:45.229 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:45.229 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:45.229 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:45.229 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:45.229 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:45.230 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:45.230 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 1095 2024-09-04T17:46:45.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.230 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:45.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:45.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.246406+0000", 2024-09-04T17:46:45.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:45.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.246406+0000", 2024-09-04T17:46:45.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:45.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.246409+0000", 2024-09-04T17:46:45.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6599999999999999e-06 2024-09-04T17:46:45.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:45.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.246419+0000", 2024-09-04T17:46:45.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8840000000000006e-06 2024-09-04T17:46:45.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:45.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.246439+0000", 2024-09-04T17:46:45.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0088000000000001e-05 2024-09-04T17:46:45.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:45.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.246448+0000", 2024-09-04T17:46:45.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9339999999999995e-06 2024-09-04T17:46:45.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:45.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.246511+0000", 2024-09-04T17:46:45.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3727999999999997e-05 2024-09-04T17:46:45.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:45.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.246588+0000", 2024-09-04T17:46:45.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6143999999999995e-05 2024-09-04T17:46:45.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-04T17:46:45.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.250919+0000", 2024-09-04T17:46:45.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0043314529999999999 2024-09-04T17:46:45.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:45.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.271170+0000", 2024-09-04T17:46:45.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020251531999999999 2024-09-04T17:46:45.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:46:45.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.271186+0000", 2024-09-04T17:46:45.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5302e-05 2024-09-04T17:46:45.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:45.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.271230+0000", 2024-09-04T17:46:45.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4576000000000001e-05 2024-09-04T17:46:45.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:45.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:02.271277+0000", 2024-09-04T17:46:45.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6894999999999997e-05 2024-09-04T17:46:45.234 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.235 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:45.235 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.235 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6103 3.1f 3:fec7e474:::rbd_data.1134cfbce83d.0000000000000003:6 [read 1736704~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:45.235 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.421848+0000", 2024-09-04T17:46:45.235 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.805946648000003, 2024-09-04T17:46:45.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027313884, 2024-09-04T17:46:45.235 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:45.235 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:45.235 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:45.235 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:45.235 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:45.236 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:45.236 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6103 2024-09-04T17:46:45.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.236 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:45.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:45.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.421848+0000", 2024-09-04T17:46:45.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:45.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.421848+0000", 2024-09-04T17:46:45.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:45.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.421849+0000", 2024-09-04T17:46:45.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.144e-06 2024-09-04T17:46:45.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:45.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.421855+0000", 2024-09-04T17:46:45.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.003e-06 2024-09-04T17:46:45.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:45.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.421872+0000", 2024-09-04T17:46:45.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7193e-05 2024-09-04T17:46:45.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:45.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.421886+0000", 2024-09-04T17:46:45.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.331e-05 2024-09-04T17:46:45.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:45.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.444849+0000", 2024-09-04T17:46:45.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.022962755000000001 2024-09-04T17:46:45.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:45.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.444990+0000", 2024-09-04T17:46:45.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014129999999999999 2024-09-04T17:46:45.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.239 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:45.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.449162+0000", 2024-09-04T17:46:45.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041721789999999998 2024-09-04T17:46:45.239 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.239 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:45.239 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.239 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6140 3.17 3:ed419000:::rbd_data.1134cfbce83d.000000000000000b:6 [sparse-read 622592~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:45.239 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.422576+0000", 2024-09-04T17:46:45.239 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.805219059999999, 2024-09-04T17:46:45.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025138273999999999, 2024-09-04T17:46:45.240 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:45.240 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:45.240 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:45.240 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:45.240 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:45.240 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:45.240 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6140 2024-09-04T17:46:45.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.240 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:45.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:45.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422576+0000", 2024-09-04T17:46:45.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:45.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422576+0000", 2024-09-04T17:46:45.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:45.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422578+0000", 2024-09-04T17:46:45.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8870000000000001e-06 2024-09-04T17:46:45.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:45.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422587+0000", 2024-09-04T17:46:45.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5999999999999996e-06 2024-09-04T17:46:45.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:45.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422617+0000", 2024-09-04T17:46:45.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9339000000000001e-05 2024-09-04T17:46:45.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:45.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422636+0000", 2024-09-04T17:46:45.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9524000000000001e-05 2024-09-04T17:46:45.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:45.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.445586+0000", 2024-09-04T17:46:45.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.022950084999999999 2024-09-04T17:46:45.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.244 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:45.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.445788+0000", 2024-09-04T17:46:45.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000202029 2024-09-04T17:46:45.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.244 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:45.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.447714+0000", 2024-09-04T17:46:45.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00192581 2024-09-04T17:46:45.244 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.245 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:45.245 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.245 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6141 3.17 3:ed419000:::rbd_data.1134cfbce83d.000000000000000b:6 [sparse-read 983040~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:45.245 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.422660+0000", 2024-09-04T17:46:45.245 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.805134397000003, 2024-09-04T17:46:45.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025710566000000001, 2024-09-04T17:46:45.245 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:45.245 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:45.245 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:45.245 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:45.246 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:45.246 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:45.246 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6141 2024-09-04T17:46:45.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.246 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:45.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:45.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422660+0000", 2024-09-04T17:46:45.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:45.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422660+0000", 2024-09-04T17:46:45.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:45.247 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422662+0000", 2024-09-04T17:46:45.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.313e-06 2024-09-04T17:46:45.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:45.247 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422691+0000", 2024-09-04T17:46:45.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9207e-05 2024-09-04T17:46:45.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:45.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422711+0000", 2024-09-04T17:46:45.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0225999999999999e-05 2024-09-04T17:46:45.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:45.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422727+0000", 2024-09-04T17:46:45.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5982999999999999e-05 2024-09-04T17:46:45.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:45.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.447767+0000", 2024-09-04T17:46:45.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025039558 2024-09-04T17:46:45.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:45.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.447871+0000", 2024-09-04T17:46:45.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104287 2024-09-04T17:46:45.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:45.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.448371+0000", 2024-09-04T17:46:45.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00049999199999999999 2024-09-04T17:46:45.249 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.249 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:45.249 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.250 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6142 3.17 3:ed419000:::rbd_data.1134cfbce83d.000000000000000b:6 [sparse-read 1343488~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:45.250 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.422750+0000", 2024-09-04T17:46:45.250 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.805045165000003, 2024-09-04T17:46:45.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026566215000000001, 2024-09-04T17:46:45.250 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:45.250 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:45.250 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:45.250 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:45.250 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:45.250 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:45.250 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6142 2024-09-04T17:46:45.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.250 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:45.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:45.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422750+0000", 2024-09-04T17:46:45.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:45.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422750+0000", 2024-09-04T17:46:45.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:45.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422751+0000", 2024-09-04T17:46:45.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6700000000000001e-06 2024-09-04T17:46:45.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:45.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422757+0000", 2024-09-04T17:46:45.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9669999999999998e-06 2024-09-04T17:46:45.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:45.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422775+0000", 2024-09-04T17:46:45.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8073000000000001e-05 2024-09-04T17:46:45.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:45.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422789+0000", 2024-09-04T17:46:45.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3709999999999999e-05 2024-09-04T17:46:45.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:45.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.448420+0000", 2024-09-04T17:46:45.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025630627999999999 2024-09-04T17:46:45.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:45.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.448590+0000", 2024-09-04T17:46:45.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000170213 2024-09-04T17:46:45.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:45.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.449316+0000", 2024-09-04T17:46:45.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00072595400000000001 2024-09-04T17:46:45.254 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.254 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:45.254 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.254 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6143 3.17 3:ed419000:::rbd_data.1134cfbce83d.000000000000000b:6 [read 1736704~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:45.254 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.422807+0000", 2024-09-04T17:46:45.254 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.804987834999999, 2024-09-04T17:46:45.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027271169000000001, 2024-09-04T17:46:45.255 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:45.255 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:45.255 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:45.255 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:45.255 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:45.255 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:45.255 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6143 2024-09-04T17:46:45.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.255 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:45.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:45.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422807+0000", 2024-09-04T17:46:45.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:45.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422807+0000", 2024-09-04T17:46:45.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:45.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422809+0000", 2024-09-04T17:46:45.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6139999999999999e-06 2024-09-04T17:46:45.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:45.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422814+0000", 2024-09-04T17:46:45.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1800000000000004e-06 2024-09-04T17:46:45.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:45.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422831+0000", 2024-09-04T17:46:45.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7433000000000001e-05 2024-09-04T17:46:45.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:45.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422844+0000", 2024-09-04T17:46:45.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3216e-05 2024-09-04T17:46:45.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:45.258 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.449397+0000", 2024-09-04T17:46:45.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026552572 2024-09-04T17:46:45.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:45.258 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.449526+0000", 2024-09-04T17:46:45.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000129207 2024-09-04T17:46:45.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:45.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.450078+0000", 2024-09-04T17:46:45.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00055194700000000001 2024-09-04T17:46:45.259 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.259 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:45.259 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.259 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6203 3.1f 3:fec7e474:::rbd_data.1134cfbce83d.0000000000000003:6 [read 327680~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:45.259 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423445+0000", 2024-09-04T17:46:45.259 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.804349422999998, 2024-09-04T17:46:45.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027179207, 2024-09-04T17:46:45.260 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:45.260 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:45.260 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:45.260 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:45.260 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:45.260 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:45.260 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6203 2024-09-04T17:46:45.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.261 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:45.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:45.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423445+0000", 2024-09-04T17:46:45.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:45.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423445+0000", 2024-09-04T17:46:45.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:45.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423446+0000", 2024-09-04T17:46:45.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.077e-06 2024-09-04T17:46:45.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:45.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423452+0000", 2024-09-04T17:46:45.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.113e-06 2024-09-04T17:46:45.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:45.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423468+0000", 2024-09-04T17:46:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6685999999999999e-05 2024-09-04T17:46:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423481+0000", 2024-09-04T17:46:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2673999999999999e-05 2024-09-04T17:46:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.449779+0000", 2024-09-04T17:46:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026297838 2024-09-04T17:46:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.450020+0000", 2024-09-04T17:46:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024114399999999999 2024-09-04T17:46:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.450625+0000", 2024-09-04T17:46:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00060467500000000005 2024-09-04T17:46:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6204 3.1f 3:fec7e474:::rbd_data.1134cfbce83d.0000000000000003:6 [read 688128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423498+0000", 2024-09-04T17:46:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.804296487000002, 2024-09-04T17:46:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029262579, 2024-09-04T17:46:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6204 2024-09-04T17:46:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423498+0000", 2024-09-04T17:46:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423498+0000", 2024-09-04T17:46:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423500+0000", 2024-09-04T17:46:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.28e-06 2024-09-04T17:46:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423506+0000", 2024-09-04T17:46:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0059999999999998e-06 2024-09-04T17:46:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423524+0000", 2024-09-04T17:46:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8683999999999999e-05 2024-09-04T17:46:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423537+0000", 2024-09-04T17:46:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3030000000000001e-05 2024-09-04T17:46:45.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:45.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.450697+0000", 2024-09-04T17:46:45.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027159793000000002 2024-09-04T17:46:45.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:45.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.450835+0000", 2024-09-04T17:46:45.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013773799999999999 2024-09-04T17:46:45.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:45.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.452761+0000", 2024-09-04T17:46:45.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019260480000000001 2024-09-04T17:46:45.269 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.269 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:45.269 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.269 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6205 3.1f 3:fec7e474:::rbd_data.1134cfbce83d.0000000000000003:6 [read 1048576~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:45.269 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423555+0000", 2024-09-04T17:46:45.269 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.804239504000002, 2024-09-04T17:46:45.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029905979999999999, 2024-09-04T17:46:45.269 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:45.269 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:45.270 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:45.270 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:45.270 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:45.270 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:45.270 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6205 2024-09-04T17:46:45.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.270 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:45.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:45.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423555+0000", 2024-09-04T17:46:45.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:45.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423555+0000", 2024-09-04T17:46:45.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:45.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423556+0000", 2024-09-04T17:46:45.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1599999999999999e-06 2024-09-04T17:46:45.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:45.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423562+0000", 2024-09-04T17:46:45.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1270000000000002e-06 2024-09-04T17:46:45.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:45.272 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423579+0000", 2024-09-04T17:46:45.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6986e-05 2024-09-04T17:46:45.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:45.272 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423591+0000", 2024-09-04T17:46:45.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2544000000000001e-05 2024-09-04T17:46:45.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:45.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.452819+0000", 2024-09-04T17:46:45.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029227418000000002 2024-09-04T17:46:45.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:45.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.452985+0000", 2024-09-04T17:46:45.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000166675 2024-09-04T17:46:45.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:45.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.453461+0000", 2024-09-04T17:46:45.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00047606999999999999 2024-09-04T17:46:45.273 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.273 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:45.273 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.274 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6206 3.1f 3:fec7e474:::rbd_data.1134cfbce83d.0000000000000003:6 [sparse-read 1409024~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:45.274 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423608+0000", 2024-09-04T17:46:45.274 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.804186870999999, 2024-09-04T17:46:45.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.031860753999999998, 2024-09-04T17:46:45.274 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:45.274 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:45.274 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:45.274 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:45.274 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:45.274 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:45.274 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6206 2024-09-04T17:46:45.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.275 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:45.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:45.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423608+0000", 2024-09-04T17:46:45.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:45.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423608+0000", 2024-09-04T17:46:45.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:45.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423609+0000", 2024-09-04T17:46:45.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0300000000000001e-06 2024-09-04T17:46:45.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:45.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423614+0000", 2024-09-04T17:46:45.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0200000000000002e-06 2024-09-04T17:46:45.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:45.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423630+0000", 2024-09-04T17:46:45.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6495999999999999e-05 2024-09-04T17:46:45.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:45.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423643+0000", 2024-09-04T17:46:45.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2053e-05 2024-09-04T17:46:45.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:45.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.453527+0000", 2024-09-04T17:46:45.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029884796000000002 2024-09-04T17:46:45.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:45.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.453632+0000", 2024-09-04T17:46:45.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104515 2024-09-04T17:46:45.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:45.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.455469+0000", 2024-09-04T17:46:45.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001836844 2024-09-04T17:46:45.278 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.278 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:45.278 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.278 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6207 3.1f 3:fec7e474:::rbd_data.1134cfbce83d.0000000000000003:6 [sparse-read 1769472~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:45.278 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423662+0000", 2024-09-04T17:46:45.278 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.804132408000001, 2024-09-04T17:46:45.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.033074294999999997, 2024-09-04T17:46:45.278 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:45.279 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:45.279 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:45.279 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:45.279 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:45.279 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:45.279 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6207 2024-09-04T17:46:45.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.279 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:45.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:45.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423662+0000", 2024-09-04T17:46:45.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:45.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423662+0000", 2024-09-04T17:46:45.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:45.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423664+0000", 2024-09-04T17:46:45.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4929999999999999e-06 2024-09-04T17:46:45.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:45.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423669+0000", 2024-09-04T17:46:45.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3800000000000002e-06 2024-09-04T17:46:45.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:45.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423686+0000", 2024-09-04T17:46:45.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7010000000000001e-05 2024-09-04T17:46:45.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:45.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423704+0000", 2024-09-04T17:46:45.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7710000000000002e-05 2024-09-04T17:46:45.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:45.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.455985+0000", 2024-09-04T17:46:45.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032281272999999999 2024-09-04T17:46:45.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:45.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.456196+0000", 2024-09-04T17:46:45.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000210335 2024-09-04T17:46:45.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:45.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.456737+0000", 2024-09-04T17:46:45.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00054109399999999999 2024-09-04T17:46:45.282 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.282 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:45.282 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.283 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6243 3.17 3:ed419000:::rbd_data.1134cfbce83d.000000000000000b:6 [sparse-read 294912~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:45.283 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423721+0000", 2024-09-04T17:46:45.283 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.804073621999997, 2024-09-04T17:46:45.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032666132000000001, 2024-09-04T17:46:45.283 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:45.283 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:45.283 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:45.283 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:45.283 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:45.283 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:45.283 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6243 2024-09-04T17:46:45.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.283 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:45.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:45.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423721+0000", 2024-09-04T17:46:45.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:45.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423721+0000", 2024-09-04T17:46:45.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:45.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423722+0000", 2024-09-04T17:46:45.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0470000000000001e-06 2024-09-04T17:46:45.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:45.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423727+0000", 2024-09-04T17:46:45.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0699999999999997e-06 2024-09-04T17:46:45.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:45.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423744+0000", 2024-09-04T17:46:45.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6659000000000001e-05 2024-09-04T17:46:45.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:45.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423756+0000", 2024-09-04T17:46:45.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2284e-05 2024-09-04T17:46:45.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:45.286 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.455553+0000", 2024-09-04T17:46:45.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.031797032000000003 2024-09-04T17:46:45.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:45.286 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.455716+0000", 2024-09-04T17:46:45.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016253700000000001 2024-09-04T17:46:45.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:45.286 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.456387+0000", 2024-09-04T17:46:45.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00067150300000000005 2024-09-04T17:46:45.287 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.287 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:45.287 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.287 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6244 3.17 3:ed419000:::rbd_data.1134cfbce83d.000000000000000b:6 [read 688128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:45.287 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423775+0000", 2024-09-04T17:46:45.287 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.804019838999999, 2024-09-04T17:46:45.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.033129498, 2024-09-04T17:46:45.287 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:45.287 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:45.287 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:45.287 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:45.287 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:45.287 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:45.288 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6244 2024-09-04T17:46:45.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.288 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:45.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:45.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423775+0000", 2024-09-04T17:46:45.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:45.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423775+0000", 2024-09-04T17:46:45.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:45.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423776+0000", 2024-09-04T17:46:45.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1230000000000001e-06 2024-09-04T17:46:45.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:45.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423782+0000", 2024-09-04T17:46:45.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4929999999999998e-06 2024-09-04T17:46:45.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:45.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423799+0000", 2024-09-04T17:46:45.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7254000000000001e-05 2024-09-04T17:46:45.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:45.290 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423811+0000", 2024-09-04T17:46:45.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2553e-05 2024-09-04T17:46:45.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:45.290 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.456421+0000", 2024-09-04T17:46:45.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032609552 2024-09-04T17:46:45.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:45.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.456512+0000", 2024-09-04T17:46:45.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0593999999999994e-05 2024-09-04T17:46:45.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:45.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.456904+0000", 2024-09-04T17:46:45.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00039292899999999998 2024-09-04T17:46:45.291 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.291 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:45.291 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.291 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6245 3.17 3:ed419000:::rbd_data.1134cfbce83d.000000000000000b:6 [read 1048576~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:45.291 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423830+0000", 2024-09-04T17:46:45.292 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.803964833000002, 2024-09-04T17:46:45.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.033577937000000002, 2024-09-04T17:46:45.292 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:45.292 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:45.292 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:45.292 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:45.292 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:45.292 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:45.292 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6245 2024-09-04T17:46:45.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.292 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:45.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:45.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423830+0000", 2024-09-04T17:46:45.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:45.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423830+0000", 2024-09-04T17:46:45.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:45.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423831+0000", 2024-09-04T17:46:45.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.164e-06 2024-09-04T17:46:45.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:45.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423847+0000", 2024-09-04T17:46:45.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5573000000000002e-05 2024-09-04T17:46:45.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:45.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423865+0000", 2024-09-04T17:46:45.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8715999999999998e-05 2024-09-04T17:46:45.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:45.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423880+0000", 2024-09-04T17:46:45.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4107000000000001e-05 2024-09-04T17:46:45.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:45.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.456951+0000", 2024-09-04T17:46:45.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.03307181 2024-09-04T17:46:45.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:45.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.457039+0000", 2024-09-04T17:46:45.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7795999999999998e-05 2024-09-04T17:46:45.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.296 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:45.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.457408+0000", 2024-09-04T17:46:45.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000368771 2024-09-04T17:46:45.296 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.296 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:45.296 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.296 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6246 3.17 3:ed419000:::rbd_data.1134cfbce83d.000000000000000b:6 [read 1409024~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:45.296 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423897+0000", 2024-09-04T17:46:45.296 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.803897493999997, 2024-09-04T17:46:45.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034015888000000001, 2024-09-04T17:46:45.296 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:45.297 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:45.297 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:45.297 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:45.297 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:45.297 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:45.297 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6246 2024-09-04T17:46:45.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.297 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:45.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:45.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423897+0000", 2024-09-04T17:46:45.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:45.298 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423897+0000", 2024-09-04T17:46:45.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:45.298 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423899+0000", 2024-09-04T17:46:45.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3200000000000001e-06 2024-09-04T17:46:45.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:45.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423904+0000", 2024-09-04T17:46:45.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4369999999999998e-06 2024-09-04T17:46:45.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:45.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423922+0000", 2024-09-04T17:46:45.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8196999999999999e-05 2024-09-04T17:46:45.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:45.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423935+0000", 2024-09-04T17:46:45.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.314e-05 2024-09-04T17:46:45.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.300 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:45.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.457457+0000", 2024-09-04T17:46:45.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.033521663 2024-09-04T17:46:45.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.300 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:45.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.457536+0000", 2024-09-04T17:46:45.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8795999999999996e-05 2024-09-04T17:46:45.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.300 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:45.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.457913+0000", 2024-09-04T17:46:45.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00037733500000000002 2024-09-04T17:46:45.300 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.300 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:45.301 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.301 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6247 3.17 3:ed419000:::rbd_data.1134cfbce83d.000000000000000b:6 [read 1769472~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:45.301 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423953+0000", 2024-09-04T17:46:45.301 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.803841310000003, 2024-09-04T17:46:45.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0344287, 2024-09-04T17:46:45.301 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:45.301 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:45.301 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:45.301 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:45.301 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:45.301 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:45.302 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6247 2024-09-04T17:46:45.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.302 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:45.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:45.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423953+0000", 2024-09-04T17:46:45.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:45.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423953+0000", 2024-09-04T17:46:45.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:45.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423955+0000", 2024-09-04T17:46:45.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0529999999999999e-06 2024-09-04T17:46:45.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.303 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:45.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423960+0000", 2024-09-04T17:46:45.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0470000000000001e-06 2024-09-04T17:46:45.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.303 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:45.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423978+0000", 2024-09-04T17:46:45.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8119999999999999e-05 2024-09-04T17:46:45.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:45.304 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423991+0000", 2024-09-04T17:46:45.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2813000000000001e-05 2024-09-04T17:46:45.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:45.304 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.457947+0000", 2024-09-04T17:46:45.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.033956865000000003 2024-09-04T17:46:45.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:45.304 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.458039+0000", 2024-09-04T17:46:45.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1834000000000002e-05 2024-09-04T17:46:45.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:45.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.458382+0000", 2024-09-04T17:46:45.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00034296799999999997 2024-09-04T17:46:45.305 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.305 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:45.305 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.305 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6305 3.17 3:ee7cd06a:::rbd_data.113431586065.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:45.305 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.455350+0000", 2024-09-04T17:46:45.305 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.772444757999999, 2024-09-04T17:46:45.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029276888000000001, 2024-09-04T17:46:45.305 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:45.306 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:45.306 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:45.306 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:45.306 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:45.306 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:45.306 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6305 2024-09-04T17:46:45.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.306 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:45.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:45.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.455350+0000", 2024-09-04T17:46:45.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:45.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.455350+0000", 2024-09-04T17:46:45.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:45.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.455353+0000", 2024-09-04T17:46:45.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6400000000000001e-06 2024-09-04T17:46:45.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:45.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459745+0000", 2024-09-04T17:46:45.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0043925880000000002 2024-09-04T17:46:45.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.308 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:45.308 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459775+0000", 2024-09-04T17:46:45.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9254000000000001e-05 2024-09-04T17:46:45.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.308 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:45.308 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459789+0000", 2024-09-04T17:46:45.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4731e-05 2024-09-04T17:46:45.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.308 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:45.308 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459885+0000", 2024-09-04T17:46:45.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5502999999999997e-05 2024-09-04T17:46:45.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.309 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:45.309 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.460040+0000", 2024-09-04T17:46:45.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015491499999999999 2024-09-04T17:46:45.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.309 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5", 2024-09-04T17:46:45.309 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.460596+0000", 2024-09-04T17:46:45.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00055657500000000002 2024-09-04T17:46:45.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.309 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:45.309 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484484+0000", 2024-09-04T17:46:45.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.023887950000000002 2024-09-04T17:46:45.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:46:45.310 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484510+0000", 2024-09-04T17:46:45.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.561e-05 2024-09-04T17:46:45.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:45.310 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484569+0000", 2024-09-04T17:46:45.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9049999999999999e-05 2024-09-04T17:46:45.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:45.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484627+0000", 2024-09-04T17:46:45.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8072000000000002e-05 2024-09-04T17:46:45.311 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.311 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:45.311 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.311 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6306 3.17 3:ee7cd06a:::rbd_data.113431586065.0000000000000005:head [truncate 1572864] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:45.311 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.459834+0000", 2024-09-04T17:46:45.311 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.767961079999999, 2024-09-04T17:46:45.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029166788999999999, 2024-09-04T17:46:45.312 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:45.312 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:45.312 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:45.312 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:45.312 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:45.312 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:45.312 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6306 2024-09-04T17:46:45.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.312 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:45.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.312 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:45.312 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459834+0000", 2024-09-04T17:46:45.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:45.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459834+0000", 2024-09-04T17:46:45.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:45.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459836+0000", 2024-09-04T17:46:45.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.835e-06 2024-09-04T17:46:45.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:45.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459841+0000", 2024-09-04T17:46:45.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.728e-06 2024-09-04T17:46:45.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.314 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:45.314 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459851+0000", 2024-09-04T17:46:45.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8570000000000007e-06 2024-09-04T17:46:45.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.314 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:45.314 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459859+0000", 2024-09-04T17:46:45.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4970000000000004e-06 2024-09-04T17:46:45.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.314 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:45.315 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.468789+0000", 2024-09-04T17:46:45.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0089301699999999994 2024-09-04T17:46:45.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.315 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:45.315 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.468890+0000", 2024-09-04T17:46:45.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000100963 2024-09-04T17:46:45.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.315 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5", 2024-09-04T17:46:45.315 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.469090+0000", 2024-09-04T17:46:45.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000199774 2024-09-04T17:46:45.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:45.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484704+0000", 2024-09-04T17:46:45.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015614836999999999 2024-09-04T17:46:45.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:46:45.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.488833+0000", 2024-09-04T17:46:45.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004128357 2024-09-04T17:46:45.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:45.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.488916+0000", 2024-09-04T17:46:45.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2928000000000003e-05 2024-09-04T17:46:45.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:45.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.489001+0000", 2024-09-04T17:46:45.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4843000000000001e-05 2024-09-04T17:46:45.317 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.317 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:45.317 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.317 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6307 3.17 3:eb6e082d:::rbd_data.113431586065.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:45.317 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.459871+0000", 2024-09-04T17:46:45.317 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.767924120000004, 2024-09-04T17:46:45.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032107122000000002, 2024-09-04T17:46:45.318 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:45.318 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:45.318 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:45.318 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:45.318 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:45.318 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:45.318 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6307 2024-09-04T17:46:45.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.318 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:45.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.318 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:45.318 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459871+0000", 2024-09-04T17:46:45.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:45.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459871+0000", 2024-09-04T17:46:45.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:45.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459871+0000", 2024-09-04T17:46:45.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9699999999999995e-07 2024-09-04T17:46:45.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:45.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.463429+0000", 2024-09-04T17:46:45.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0035577930000000001 2024-09-04T17:46:45.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:45.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.463463+0000", 2024-09-04T17:46:45.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3460000000000002e-05 2024-09-04T17:46:45.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:45.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.463483+0000", 2024-09-04T17:46:45.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0327999999999999e-05 2024-09-04T17:46:45.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:45.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.477551+0000", 2024-09-04T17:46:45.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014068259 2024-09-04T17:46:45.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:45.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.477748+0000", 2024-09-04T17:46:45.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000197004 2024-09-04T17:46:45.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5", 2024-09-04T17:46:45.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.478346+0000", 2024-09-04T17:46:45.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00059796299999999997 2024-09-04T17:46:45.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:46:45.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.489016+0000", 2024-09-04T17:46:45.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010669781 2024-09-04T17:46:45.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:45.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.491871+0000", 2024-09-04T17:46:45.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028546219999999998 2024-09-04T17:46:45.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:45.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.491931+0000", 2024-09-04T17:46:45.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0736999999999998e-05 2024-09-04T17:46:45.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.323 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:45.323 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.491978+0000", 2024-09-04T17:46:45.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6378000000000002e-05 2024-09-04T17:46:45.323 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.323 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:45.323 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.323 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6308 3.17 3:eb6e082d:::rbd_data.113431586065.0000000000000007:head [truncate 1572864] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:45.323 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.463520+0000", 2024-09-04T17:46:45.323 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.764274610999998, 2024-09-04T17:46:45.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029348833000000001, 2024-09-04T17:46:45.324 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:45.324 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:45.324 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:45.324 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:45.324 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:45.324 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:45.324 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6308 2024-09-04T17:46:45.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.324 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:45.324 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.324 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:45.324 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.463520+0000", 2024-09-04T17:46:45.324 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:45.325 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.463520+0000", 2024-09-04T17:46:45.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:45.325 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.463522+0000", 2024-09-04T17:46:45.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1150000000000001e-06 2024-09-04T17:46:45.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:45.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.463527+0000", 2024-09-04T17:46:45.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5170000000000001e-06 2024-09-04T17:46:45.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:45.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.463555+0000", 2024-09-04T17:46:45.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8594000000000001e-05 2024-09-04T17:46:45.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:45.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.463563+0000", 2024-09-04T17:46:45.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4340000000000003e-06 2024-09-04T17:46:45.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:45.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484756+0000", 2024-09-04T17:46:45.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021193081999999999 2024-09-04T17:46:45.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:45.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484863+0000", 2024-09-04T17:46:45.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010721999999999999 2024-09-04T17:46:45.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5", 2024-09-04T17:46:45.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.485067+0000", 2024-09-04T17:46:45.328 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000204 2024-09-04T17:46:45.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.328 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.328 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:46:45.328 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.489593+0000", 2024-09-04T17:46:45.328 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045261529999999998 2024-09-04T17:46:45.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.328 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.328 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:45.328 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.492787+0000", 2024-09-04T17:46:45.328 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031940039999999999 2024-09-04T17:46:45.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.328 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.329 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:45.329 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.492834+0000", 2024-09-04T17:46:45.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6281e-05 2024-09-04T17:46:45.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.329 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:45.329 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.492869+0000", 2024-09-04T17:46:45.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5432999999999998e-05 2024-09-04T17:46:45.329 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.329 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:45.329 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.329 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.330 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:45.330 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:45.331 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-04T17:46:45.331 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:45.327+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:45.432 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:45.433+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:45.503 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:45.504 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:46:45.504 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:46:45.504 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:45.518 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-04T17:46:45.564 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:46:45.563+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:46:45.634 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:45.633+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:45.647 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:45.647 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:46:45.647 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:46:45.647 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:46:45.647 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:45.662 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-04T17:46:45.730 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #17, image image_client.0-clone17 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent18 2024-09-04T17:46:45.735 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:45.731+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:45.762 INFO:teuthology.orchestra.run.smithi092.stdout:2453 writesame 0x68fa57e thru 0x6904aa3 data_size 0x1(0xa526 bytes) 2024-09-04T17:46:45.762 INFO:teuthology.orchestra.run.smithi092.stdout:2456 read 0x1cf709c thru 0x1d02689 (0xb5ee bytes) 2024-09-04T17:46:45.794 INFO:teuthology.orchestra.run.smithi092.stdout:2457 write 0xebd36d9 thru 0xebe11f6 (0xdb1e bytes) 2024-09-04T17:46:45.799 INFO:teuthology.orchestra.run.smithi092.stdout:2459 read 0x3d3805c thru 0x3d460d5 (0xe07a bytes) 2024-09-04T17:46:45.799 INFO:teuthology.orchestra.run.smithi092.stdout:2460 punch from 0xbc0a4c8 to 0xbc0c811, (0x2349 bytes) 2024-09-04T17:46:45.799 INFO:teuthology.orchestra.run.smithi092.stdout:2462 writesame 0x2da7dd thru 0x2e9f26 data_size 0x9(0xf74a bytes) 2024-09-04T17:46:45.814 INFO:teuthology.orchestra.run.smithi092.stdout:2464 write 0xb5a4ef4 thru 0xb5ac0b5 (0x71c2 bytes) 2024-09-04T17:46:45.836 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:45.833+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:45.838 INFO:teuthology.orchestra.run.smithi092.stdout:2465 trunc from 0xebe11f7 to 0x429fce8 2024-09-04T17:46:45.842 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-04T17:46:45.852 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:45.852 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:46:45.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:46:45.852 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:46:45.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.852 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15462 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 835584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:45.853 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.220338+0000", 2024-09-04T17:46:45.853 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.629471834, 2024-09-04T17:46:45.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051751278999999997, 2024-09-04T17:46:45.853 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:45.853 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:45.853 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:45.853 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:45.853 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:45.853 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:45.853 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15462 2024-09-04T17:46:45.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.853 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:45.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.854 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:45.854 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.220338+0000", 2024-09-04T17:46:45.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.854 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:45.854 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.220338+0000", 2024-09-04T17:46:45.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.854 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:45.854 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.220600+0000", 2024-09-04T17:46:45.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026198900000000002 2024-09-04T17:46:45.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.855 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:45.855 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.220617+0000", 2024-09-04T17:46:45.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6956000000000002e-05 2024-09-04T17:46:45.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.855 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:45.855 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.220627+0000", 2024-09-04T17:46:45.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0492999999999999e-05 2024-09-04T17:46:45.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:45.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.220635+0000", 2024-09-04T17:46:45.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0059999999999992e-06 2024-09-04T17:46:45.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:45.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.271853+0000", 2024-09-04T17:46:45.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051218020000000003 2024-09-04T17:46:45.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:45.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.271941+0000", 2024-09-04T17:46:45.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7802999999999999e-05 2024-09-04T17:46:45.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.857 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:45.857 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.272089+0000", 2024-09-04T17:46:45.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014801199999999999 2024-09-04T17:46:45.857 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.857 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:45.857 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.857 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15500 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 294912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:45.857 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.228278+0000", 2024-09-04T17:46:45.858 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.621531836999999, 2024-09-04T17:46:45.858 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051770927000000001, 2024-09-04T17:46:45.858 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:45.858 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:45.858 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:45.858 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:45.858 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:45.858 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:45.858 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15500 2024-09-04T17:46:45.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.858 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:45.858 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.858 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:45.859 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.228278+0000", 2024-09-04T17:46:45.859 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.859 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.859 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:45.859 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.228278+0000", 2024-09-04T17:46:45.859 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.859 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.859 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:45.859 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234002+0000", 2024-09-04T17:46:45.859 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005724392 2024-09-04T17:46:45.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.859 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.860 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:45.860 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234009+0000", 2024-09-04T17:46:45.860 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1300000000000003e-06 2024-09-04T17:46:45.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.860 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:45.860 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234024+0000", 2024-09-04T17:46:45.860 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4063000000000001e-05 2024-09-04T17:46:45.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.860 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:45.861 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234035+0000", 2024-09-04T17:46:45.861 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1124000000000001e-05 2024-09-04T17:46:45.861 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.861 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.861 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:45.861 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.279808+0000", 2024-09-04T17:46:45.861 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045772992999999998 2024-09-04T17:46:45.861 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.861 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.861 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:45.861 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.279894+0000", 2024-09-04T17:46:45.861 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6174999999999994e-05 2024-09-04T17:46:45.861 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.862 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.862 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:45.862 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.280049+0000", 2024-09-04T17:46:45.862 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015505 2024-09-04T17:46:45.862 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.862 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:45.862 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.862 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.862 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.862 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15524 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 1540096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:45.862 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.234831+0000", 2024-09-04T17:46:45.862 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.614978960999998, 2024-09-04T17:46:45.862 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051893631000000003, 2024-09-04T17:46:45.862 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:45.863 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:45.863 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:45.863 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:45.863 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:45.863 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:45.863 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15524 2024-09-04T17:46:45.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.863 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:45.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.863 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:45.863 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234831+0000", 2024-09-04T17:46:45.863 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.864 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:45.864 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234831+0000", 2024-09-04T17:46:45.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.864 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:45.864 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234831+0000", 2024-09-04T17:46:45.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2699999999999999e-07 2024-09-04T17:46:45.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.864 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:45.864 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234833+0000", 2024-09-04T17:46:45.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9810000000000002e-06 2024-09-04T17:46:45.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.865 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.865 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:45.865 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234839+0000", 2024-09-04T17:46:45.865 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1519999999999997e-06 2024-09-04T17:46:45.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.865 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.865 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:45.865 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234845+0000", 2024-09-04T17:46:45.865 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1109999999999999e-06 2024-09-04T17:46:45.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.866 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.866 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:45.866 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.286526+0000", 2024-09-04T17:46:45.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051681373000000003 2024-09-04T17:46:45.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.866 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.866 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:45.866 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.286594+0000", 2024-09-04T17:46:45.867 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8073999999999999e-05 2024-09-04T17:46:45.867 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.867 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.867 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:45.867 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.286724+0000", 2024-09-04T17:46:45.867 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013041300000000001 2024-09-04T17:46:45.867 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.867 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:45.867 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.867 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.867 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.867 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15525 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 335872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:45.867 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.234851+0000", 2024-09-04T17:46:45.868 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.614958436999999, 2024-09-04T17:46:45.868 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052146248999999999, 2024-09-04T17:46:45.868 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:45.868 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:45.868 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:45.868 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:45.868 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:45.868 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:45.868 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15525 2024-09-04T17:46:45.868 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.868 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:45.868 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.868 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:45.869 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234851+0000", 2024-09-04T17:46:45.869 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.869 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.869 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:45.869 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234851+0000", 2024-09-04T17:46:45.869 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.869 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.869 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:45.869 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234852+0000", 2024-09-04T17:46:45.869 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0999999999999999e-07 2024-09-04T17:46:45.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.869 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.870 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:45.870 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234854+0000", 2024-09-04T17:46:45.870 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9929999999999998e-06 2024-09-04T17:46:45.870 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.870 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.870 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:45.870 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234860+0000", 2024-09-04T17:46:45.870 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1489999999999999e-06 2024-09-04T17:46:45.870 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.870 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.870 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:45.870 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234865+0000", 2024-09-04T17:46:45.870 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8600000000000001e-06 2024-09-04T17:46:45.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.871 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.871 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:45.871 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.286770+0000", 2024-09-04T17:46:45.871 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051904807999999997 2024-09-04T17:46:45.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.871 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.871 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:45.871 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.286852+0000", 2024-09-04T17:46:45.871 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2076000000000005e-05 2024-09-04T17:46:45.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.871 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.871 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:45.871 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.286998+0000", 2024-09-04T17:46:45.872 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014595299999999999 2024-09-04T17:46:45.872 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.872 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:45.872 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.872 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.872 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.872 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15526 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 811008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:45.872 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.234871+0000", 2024-09-04T17:46:45.872 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.614938668000001, 2024-09-04T17:46:45.873 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052383817999999999, 2024-09-04T17:46:45.873 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:45.873 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:45.873 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:45.873 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:45.873 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:45.873 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:45.873 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15526 2024-09-04T17:46:45.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.873 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:45.873 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.873 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:45.873 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234871+0000", 2024-09-04T17:46:45.874 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.874 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:45.874 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234871+0000", 2024-09-04T17:46:45.874 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.874 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:45.874 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234872+0000", 2024-09-04T17:46:45.874 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4299999999999998e-07 2024-09-04T17:46:45.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.875 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:45.875 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234874+0000", 2024-09-04T17:46:45.875 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2220000000000001e-06 2024-09-04T17:46:45.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.875 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.875 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:45.875 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234880+0000", 2024-09-04T17:46:45.875 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4169999999999997e-06 2024-09-04T17:46:45.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.875 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.875 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:45.875 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234886+0000", 2024-09-04T17:46:45.875 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8150000000000002e-06 2024-09-04T17:46:45.876 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.876 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.876 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:45.876 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287040+0000", 2024-09-04T17:46:45.876 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052153928000000002 2024-09-04T17:46:45.876 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.876 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.876 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:45.876 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287118+0000", 2024-09-04T17:46:45.876 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8102999999999994e-05 2024-09-04T17:46:45.876 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.876 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.877 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:45.877 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287255+0000", 2024-09-04T17:46:45.877 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013689 2024-09-04T17:46:45.877 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.877 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:45.877 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.877 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.877 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.877 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15527 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 372736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:45.877 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.234906+0000", 2024-09-04T17:46:45.877 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.614903352999999, 2024-09-04T17:46:45.878 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052620742999999998, 2024-09-04T17:46:45.878 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:45.878 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:45.878 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:45.878 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:45.878 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:45.878 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:45.878 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15527 2024-09-04T17:46:45.878 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.878 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:45.878 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.878 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:45.878 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234906+0000", 2024-09-04T17:46:45.879 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.879 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.879 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:45.879 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234906+0000", 2024-09-04T17:46:45.879 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.879 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.879 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:45.879 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234907+0000", 2024-09-04T17:46:45.879 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9100000000000004e-07 2024-09-04T17:46:45.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.879 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.880 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:45.880 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234910+0000", 2024-09-04T17:46:45.880 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7599999999999998e-06 2024-09-04T17:46:45.880 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.880 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.880 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:45.881 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234917+0000", 2024-09-04T17:46:45.881 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.861e-06 2024-09-04T17:46:45.881 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.881 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.881 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:45.881 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234923+0000", 2024-09-04T17:46:45.881 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4130000000000003e-06 2024-09-04T17:46:45.881 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.881 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.881 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:45.881 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287302+0000", 2024-09-04T17:46:45.881 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052379440999999999 2024-09-04T17:46:45.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.882 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:45.882 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287382+0000", 2024-09-04T17:46:45.882 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9406999999999994e-05 2024-09-04T17:46:45.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.882 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:45.882 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287527+0000", 2024-09-04T17:46:45.882 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014537 2024-09-04T17:46:45.883 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.883 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:45.883 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.883 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.883 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15528 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 847872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:45.883 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.234931+0000", 2024-09-04T17:46:45.883 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.614878435000001, 2024-09-04T17:46:45.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052862919000000001, 2024-09-04T17:46:45.883 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:45.883 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:45.883 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:45.883 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:45.884 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:45.884 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:45.884 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15528 2024-09-04T17:46:45.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.884 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:45.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.884 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:45.884 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234931+0000", 2024-09-04T17:46:45.884 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.884 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:45.885 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234931+0000", 2024-09-04T17:46:45.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.885 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.885 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.885 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:45.885 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234932+0000", 2024-09-04T17:46:45.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6100000000000001e-07 2024-09-04T17:46:45.885 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.885 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.885 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:45.885 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234934+0000", 2024-09-04T17:46:45.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3309999999999998e-06 2024-09-04T17:46:45.885 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.886 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.886 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:45.886 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234941+0000", 2024-09-04T17:46:45.886 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4869999999999999e-06 2024-09-04T17:46:45.886 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.886 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.886 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:45.886 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234946+0000", 2024-09-04T17:46:45.886 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5879999999999997e-06 2024-09-04T17:46:45.886 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.886 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.886 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:45.886 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287571+0000", 2024-09-04T17:46:45.887 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052624769000000002 2024-09-04T17:46:45.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.887 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:45.887 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287651+0000", 2024-09-04T17:46:45.887 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9975000000000006e-05 2024-09-04T17:46:45.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.887 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:45.887 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287794+0000", 2024-09-04T17:46:45.887 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000143308 2024-09-04T17:46:45.887 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.887 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:45.888 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.888 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.888 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.888 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15529 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 409600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:45.888 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.234954+0000", 2024-09-04T17:46:45.888 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.614855563999999, 2024-09-04T17:46:45.888 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053149889999999998, 2024-09-04T17:46:45.888 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:45.888 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:45.888 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:45.888 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:45.888 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:45.888 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:45.889 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15529 2024-09-04T17:46:45.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.889 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:45.889 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.889 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:45.889 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234954+0000", 2024-09-04T17:46:45.889 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.889 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.889 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:45.889 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234954+0000", 2024-09-04T17:46:45.889 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.890 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.890 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:45.890 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234955+0000", 2024-09-04T17:46:45.890 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3799999999999997e-07 2024-09-04T17:46:45.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.890 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.890 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:45.890 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234957+0000", 2024-09-04T17:46:45.890 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3309999999999998e-06 2024-09-04T17:46:45.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.890 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.890 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:45.890 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234964+0000", 2024-09-04T17:46:45.891 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5780000000000004e-06 2024-09-04T17:46:45.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.891 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.891 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:45.891 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234969+0000", 2024-09-04T17:46:45.891 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0629999999999996e-06 2024-09-04T17:46:45.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.891 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.891 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:45.891 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287839+0000", 2024-09-04T17:46:45.891 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052870385999999998 2024-09-04T17:46:45.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.891 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.892 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:45.892 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287929+0000", 2024-09-04T17:46:45.892 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9514999999999999e-05 2024-09-04T17:46:45.892 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.892 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.892 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:45.892 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288104+0000", 2024-09-04T17:46:45.892 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000175379 2024-09-04T17:46:45.892 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.892 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:45.892 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.892 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.892 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.893 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15530 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 884736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:45.893 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.234976+0000", 2024-09-04T17:46:45.893 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.614834146, 2024-09-04T17:46:45.893 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053414593000000003, 2024-09-04T17:46:45.893 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:45.893 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:45.893 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:45.893 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:45.893 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:45.893 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:45.893 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15530 2024-09-04T17:46:45.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.894 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:45.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.894 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:45.894 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234976+0000", 2024-09-04T17:46:45.894 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.894 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:45.894 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234976+0000", 2024-09-04T17:46:45.894 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:45.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234976+0000", 2024-09-04T17:46:45.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5299999999999999e-07 2024-09-04T17:46:45.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:45.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234978+0000", 2024-09-04T17:46:45.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0329999999999998e-06 2024-09-04T17:46:45.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:45.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234984+0000", 2024-09-04T17:46:45.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0249999999999999e-06 2024-09-04T17:46:45.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.896 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:45.896 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234990+0000", 2024-09-04T17:46:45.896 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7579999999999998e-06 2024-09-04T17:46:45.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.896 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:45.896 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288149+0000", 2024-09-04T17:46:45.896 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053158864 2024-09-04T17:46:45.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.896 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:45.897 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288236+0000", 2024-09-04T17:46:45.897 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7306999999999996e-05 2024-09-04T17:46:45.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.897 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.897 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:45.897 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288390+0000", 2024-09-04T17:46:45.897 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000154153 2024-09-04T17:46:45.897 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.897 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:45.897 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.897 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.897 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15531 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 446464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:45.898 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.234997+0000", 2024-09-04T17:46:45.898 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.614813163000001, 2024-09-04T17:46:45.898 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053721086000000001, 2024-09-04T17:46:45.898 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:45.898 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:45.898 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:45.898 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:45.898 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:45.898 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:45.898 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15531 2024-09-04T17:46:45.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.898 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:45.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.898 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:45.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234997+0000", 2024-09-04T17:46:45.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.899 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:45.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234997+0000", 2024-09-04T17:46:45.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.899 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:45.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234997+0000", 2024-09-04T17:46:45.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8200000000000001e-07 2024-09-04T17:46:45.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.900 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:45.900 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234999+0000", 2024-09-04T17:46:45.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.058e-06 2024-09-04T17:46:45.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.900 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:45.900 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235005+0000", 2024-09-04T17:46:45.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0700000000000003e-06 2024-09-04T17:46:45.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.900 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:45.900 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235012+0000", 2024-09-04T17:46:45.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7020000000000004e-06 2024-09-04T17:46:45.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:45.901 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288441+0000", 2024-09-04T17:46:45.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053429343999999997 2024-09-04T17:46:45.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:45.901 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288535+0000", 2024-09-04T17:46:45.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3406000000000007e-05 2024-09-04T17:46:45.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:45.902 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288718+0000", 2024-09-04T17:46:45.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000183124 2024-09-04T17:46:45.902 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.902 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:45.902 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.902 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15532 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 921600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:45.902 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235020+0000", 2024-09-04T17:46:45.902 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.614789536, 2024-09-04T17:46:45.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053966157000000001, 2024-09-04T17:46:45.902 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:45.902 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:45.902 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:45.903 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:45.903 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:45.903 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:45.903 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15532 2024-09-04T17:46:45.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.903 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:45.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.903 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:45.903 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235020+0000", 2024-09-04T17:46:45.903 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.903 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:45.903 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235020+0000", 2024-09-04T17:46:45.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:45.904 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235021+0000", 2024-09-04T17:46:45.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9799999999999999e-07 2024-09-04T17:46:45.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:45.904 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235023+0000", 2024-09-04T17:46:45.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1560000000000002e-06 2024-09-04T17:46:45.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.905 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:45.905 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235029+0000", 2024-09-04T17:46:45.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0270000000000001e-06 2024-09-04T17:46:45.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.905 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:45.905 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235034+0000", 2024-09-04T17:46:45.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0000000000000004e-06 2024-09-04T17:46:45.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.905 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:45.905 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288760+0000", 2024-09-04T17:46:45.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053726077999999997 2024-09-04T17:46:45.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.906 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.906 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:45.906 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288845+0000", 2024-09-04T17:46:45.906 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5242999999999997e-05 2024-09-04T17:46:45.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.906 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.906 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:45.906 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288986+0000", 2024-09-04T17:46:45.906 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014125499999999999 2024-09-04T17:46:45.906 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.906 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:45.906 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.907 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.907 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.907 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15533 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 483328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:45.907 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235041+0000", 2024-09-04T17:46:45.907 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.614768461000001, 2024-09-04T17:46:45.907 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054242994000000003, 2024-09-04T17:46:45.907 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:45.907 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:45.907 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:45.907 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:45.907 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:45.907 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:45.907 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15533 2024-09-04T17:46:45.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.908 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:45.908 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.908 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:45.908 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235041+0000", 2024-09-04T17:46:45.908 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.908 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.908 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:45.908 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235041+0000", 2024-09-04T17:46:45.908 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.908 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.909 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:45.909 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235042+0000", 2024-09-04T17:46:45.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5499999999999999e-07 2024-09-04T17:46:45.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.909 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.909 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:45.909 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235044+0000", 2024-09-04T17:46:45.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2259999999999999e-06 2024-09-04T17:46:45.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.909 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.909 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:45.909 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235051+0000", 2024-09-04T17:46:45.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4440000000000002e-06 2024-09-04T17:46:45.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.910 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:45.910 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235056+0000", 2024-09-04T17:46:45.910 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8740000000000003e-06 2024-09-04T17:46:45.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.910 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:45.910 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289028+0000", 2024-09-04T17:46:45.910 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053971742000000003 2024-09-04T17:46:45.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:45.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289108+0000", 2024-09-04T17:46:45.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0502999999999998e-05 2024-09-04T17:46:45.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:45.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289284+0000", 2024-09-04T17:46:45.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017584999999999999 2024-09-04T17:46:45.911 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.911 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:45.911 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.912 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15534 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 958464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:45.912 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235068+0000", 2024-09-04T17:46:45.912 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.614741761000001, 2024-09-04T17:46:45.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054505452000000003, 2024-09-04T17:46:45.912 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:45.912 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:45.912 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:45.912 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:45.912 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:45.912 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:45.913 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15534 2024-09-04T17:46:45.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.913 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:45.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.913 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:45.913 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235068+0000", 2024-09-04T17:46:45.913 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.913 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:45.913 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235068+0000", 2024-09-04T17:46:45.913 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.914 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:45.914 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235068+0000", 2024-09-04T17:46:45.914 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7899999999999999e-07 2024-09-04T17:46:45.914 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.914 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:45.914 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235071+0000", 2024-09-04T17:46:45.914 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4109999999999999e-06 2024-09-04T17:46:45.914 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.914 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:45.914 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235078+0000", 2024-09-04T17:46:45.914 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0489999999999998e-06 2024-09-04T17:46:45.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.915 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:45.915 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235083+0000", 2024-09-04T17:46:45.915 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2249999999999999e-06 2024-09-04T17:46:45.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.915 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:45.915 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289328+0000", 2024-09-04T17:46:45.915 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054244894000000002 2024-09-04T17:46:45.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.915 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:45.915 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289417+0000", 2024-09-04T17:46:45.916 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8988999999999998e-05 2024-09-04T17:46:45.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.916 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:45.916 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289573+0000", 2024-09-04T17:46:45.916 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000156405 2024-09-04T17:46:45.916 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.916 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:45.916 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.916 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15535 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 520192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:45.916 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235173+0000", 2024-09-04T17:46:45.916 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.614637232, 2024-09-04T17:46:45.917 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054718671000000003, 2024-09-04T17:46:45.917 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:45.917 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:45.917 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:45.917 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:45.917 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:45.917 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:45.917 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15535 2024-09-04T17:46:45.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.917 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:45.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.917 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:45.917 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235173+0000", 2024-09-04T17:46:45.917 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.918 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.918 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.918 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:45.918 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235173+0000", 2024-09-04T17:46:45.918 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.918 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.918 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.918 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:45.918 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235173+0000", 2024-09-04T17:46:45.918 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2600000000000002e-07 2024-09-04T17:46:45.918 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.918 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.918 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:45.919 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235177+0000", 2024-09-04T17:46:45.919 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.625e-06 2024-09-04T17:46:45.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.919 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.919 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:45.919 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235188+0000", 2024-09-04T17:46:45.919 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1464000000000001e-05 2024-09-04T17:46:45.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.919 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.919 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:45.919 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235195+0000", 2024-09-04T17:46:45.919 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5910000000000001e-06 2024-09-04T17:46:45.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.920 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:45.920 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289623+0000", 2024-09-04T17:46:45.920 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054427595000000002 2024-09-04T17:46:45.920 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.920 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:45.920 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289718+0000", 2024-09-04T17:46:45.920 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5958999999999994e-05 2024-09-04T17:46:45.920 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.921 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:45.921 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289891+0000", 2024-09-04T17:46:45.921 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017271100000000001 2024-09-04T17:46:45.921 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.921 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:45.921 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.921 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.921 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.921 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15536 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 995328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:45.921 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235206+0000", 2024-09-04T17:46:45.921 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.614604091, 2024-09-04T17:46:45.921 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054984065999999998, 2024-09-04T17:46:45.921 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:45.921 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:45.922 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:45.922 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:45.922 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:45.922 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:45.922 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15536 2024-09-04T17:46:45.922 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.922 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:45.922 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.922 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:45.922 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235206+0000", 2024-09-04T17:46:45.922 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.922 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.922 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.922 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:45.922 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235206+0000", 2024-09-04T17:46:45.923 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.923 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.923 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:45.923 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235206+0000", 2024-09-04T17:46:45.923 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5300000000000004e-07 2024-09-04T17:46:45.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.923 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.923 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:45.923 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235209+0000", 2024-09-04T17:46:45.923 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6089999999999999e-06 2024-09-04T17:46:45.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.923 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.923 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:45.923 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235216+0000", 2024-09-04T17:46:45.924 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.277e-06 2024-09-04T17:46:45.924 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.924 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:45.924 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235221+0000", 2024-09-04T17:46:45.924 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2530000000000004e-06 2024-09-04T17:46:45.924 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.924 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:45.924 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289936+0000", 2024-09-04T17:46:45.924 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054714888000000003 2024-09-04T17:46:45.924 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.924 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:45.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290035+0000", 2024-09-04T17:46:45.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8652000000000003e-05 2024-09-04T17:46:45.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:45.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290190+0000", 2024-09-04T17:46:45.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015483399999999999 2024-09-04T17:46:45.925 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.925 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:45.925 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.925 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15537 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 1323008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:45.925 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235264+0000", 2024-09-04T17:46:45.926 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.614546022999999, 2024-09-04T17:46:45.926 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055255669, 2024-09-04T17:46:45.926 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:45.926 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:45.926 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:45.926 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:45.926 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:45.926 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:45.926 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15537 2024-09-04T17:46:45.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.926 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:45.926 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.926 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:45.926 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235264+0000", 2024-09-04T17:46:45.927 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.927 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.927 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.927 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:45.927 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235264+0000", 2024-09-04T17:46:45.927 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.927 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.927 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.927 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:45.927 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235264+0000", 2024-09-04T17:46:45.927 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4499999999999997e-07 2024-09-04T17:46:45.927 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.927 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.927 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:45.927 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235267+0000", 2024-09-04T17:46:45.928 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9210000000000001e-06 2024-09-04T17:46:45.928 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.928 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.928 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:45.928 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235276+0000", 2024-09-04T17:46:45.928 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2309999999999996e-06 2024-09-04T17:46:45.928 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.928 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.928 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:45.929 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235281+0000", 2024-09-04T17:46:45.929 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8560000000000003e-06 2024-09-04T17:46:45.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.929 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:45.929 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290246+0000", 2024-09-04T17:46:45.929 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054964528999999998 2024-09-04T17:46:45.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.930 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:45.930 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290344+0000", 2024-09-04T17:46:45.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8073999999999996e-05 2024-09-04T17:46:45.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.930 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:45.930 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290519+0000", 2024-09-04T17:46:45.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000175413 2024-09-04T17:46:45.930 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.930 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:45.930 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.930 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15538 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 1032192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:45.931 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235290+0000", 2024-09-04T17:46:45.931 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.614520114000001, 2024-09-04T17:46:45.931 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055518932, 2024-09-04T17:46:45.931 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:45.931 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:45.931 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:45.931 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:45.931 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:45.931 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:45.931 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15538 2024-09-04T17:46:45.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.931 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:45.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.932 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:45.932 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235290+0000", 2024-09-04T17:46:45.932 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.932 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.932 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:45.932 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235290+0000", 2024-09-04T17:46:45.932 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.932 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.932 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:45.933 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235290+0000", 2024-09-04T17:46:45.933 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3399999999999999e-07 2024-09-04T17:46:45.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.936 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:45.936 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235292+0000", 2024-09-04T17:46:45.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2639999999999998e-06 2024-09-04T17:46:45.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.936 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:45.936 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235299+0000", 2024-09-04T17:46:45.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2029999999999997e-06 2024-09-04T17:46:45.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.936 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:45.937 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235305+0000", 2024-09-04T17:46:45.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7469999999999999e-06 2024-09-04T17:46:45.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.937 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:45.937 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290562+0000", 2024-09-04T17:46:45.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055256580999999999 2024-09-04T17:46:45.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.937 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:45.937 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290651+0000", 2024-09-04T17:46:45.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8953e-05 2024-09-04T17:46:45.938 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.938 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.938 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:45.938 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290809+0000", 2024-09-04T17:46:45.938 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015765000000000001 2024-09-04T17:46:45.938 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.938 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:45.938 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.938 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.938 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.938 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15539 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 1359872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:45.938 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235312+0000", 2024-09-04T17:46:45.939 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.614497373999999, 2024-09-04T17:46:45.939 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055809865, 2024-09-04T17:46:45.939 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:45.939 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:45.939 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:45.939 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:45.939 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:45.939 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:45.939 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15539 2024-09-04T17:46:45.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.939 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:45.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.939 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:45.940 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235312+0000", 2024-09-04T17:46:45.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.940 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.940 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:45.940 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235312+0000", 2024-09-04T17:46:45.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.940 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.940 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:45.940 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235313+0000", 2024-09-04T17:46:45.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9000000000000002e-07 2024-09-04T17:46:45.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:45.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235315+0000", 2024-09-04T17:46:45.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1270000000000001e-06 2024-09-04T17:46:45.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:45.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235320+0000", 2024-09-04T17:46:45.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5430000000000001e-06 2024-09-04T17:46:45.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:45.942 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235326+0000", 2024-09-04T17:46:45.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2039999999999996e-06 2024-09-04T17:46:45.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.942 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.942 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:45.942 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290858+0000", 2024-09-04T17:46:45.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055532440000000002 2024-09-04T17:46:45.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.942 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.942 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:45.942 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290953+0000", 2024-09-04T17:46:45.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5031000000000005e-05 2024-09-04T17:46:45.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.942 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.943 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:45.943 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.291122+0000", 2024-09-04T17:46:45.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016913 2024-09-04T17:46:45.943 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.943 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:45.943 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.943 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15540 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 1396736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:45.944 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235333+0000", 2024-09-04T17:46:45.944 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.614476867, 2024-09-04T17:46:45.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056080528999999997, 2024-09-04T17:46:45.944 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:45.944 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:45.944 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:45.944 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:45.944 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:45.944 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:45.944 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15540 2024-09-04T17:46:45.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.944 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:45.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:45.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235333+0000", 2024-09-04T17:46:45.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:45.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235333+0000", 2024-09-04T17:46:45.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:45.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235333+0000", 2024-09-04T17:46:45.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0999999999999999e-07 2024-09-04T17:46:45.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:45.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235336+0000", 2024-09-04T17:46:45.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.278e-06 2024-09-04T17:46:45.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:45.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235341+0000", 2024-09-04T17:46:45.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8359999999999997e-06 2024-09-04T17:46:45.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:45.947 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235346+0000", 2024-09-04T17:46:45.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4329999999999997e-06 2024-09-04T17:46:45.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:45.947 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.291167+0000", 2024-09-04T17:46:45.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055821603999999997 2024-09-04T17:46:45.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:45.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.291257+0000", 2024-09-04T17:46:45.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9349000000000001e-05 2024-09-04T17:46:45.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:45.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.291413+0000", 2024-09-04T17:46:45.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015661900000000001 2024-09-04T17:46:45.948 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.948 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:45.948 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.949 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15541 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 1433600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:45.949 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235355+0000", 2024-09-04T17:46:45.949 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.614454679000001, 2024-09-04T17:46:45.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056371783000000002, 2024-09-04T17:46:45.949 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:45.949 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:45.949 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:45.949 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:45.949 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:45.949 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:45.949 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15541 2024-09-04T17:46:45.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.949 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:45.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:45.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235355+0000", 2024-09-04T17:46:45.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:45.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235355+0000", 2024-09-04T17:46:45.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:45.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:45.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235356+0000", 2024-09-04T17:46:45.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3099999999999998e-07 2024-09-04T17:46:45.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:45.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235358+0000", 2024-09-04T17:46:45.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3599999999999999e-06 2024-09-04T17:46:45.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:45.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235364+0000", 2024-09-04T17:46:45.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5760000000000002e-06 2024-09-04T17:46:45.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:45.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235370+0000", 2024-09-04T17:46:45.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0819999999999998e-06 2024-09-04T17:46:45.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:45.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.291462+0000", 2024-09-04T17:46:45.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056092301999999997 2024-09-04T17:46:45.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:45.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.291559+0000", 2024-09-04T17:46:45.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6988e-05 2024-09-04T17:46:45.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:45.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:45.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:45.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.291727+0000", 2024-09-04T17:46:45.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016804400000000001 2024-09-04T17:46:45.953 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.953 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:45.953 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.953 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:45.953 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:45.953 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:45.954 INFO:teuthology.orchestra.run.smithi092.stdout:2466 writesame 0x248b644 thru 0x248c58c data_size 0xd(0xf49 bytes) 2024-09-04T17:46:45.955 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-04T17:46:45.956 INFO:teuthology.orchestra.run.smithi092.stdout:2467 read 0xa77fdf thru 0xa7fa2f (0x7a51 bytes) 2024-09-04T17:46:45.957 INFO:teuthology.orchestra.run.smithi092.stdout:2468 read 0xbcdf8d thru 0xbd3d81 (0x5df5 bytes) 2024-09-04T17:46:45.957 INFO:teuthology.orchestra.run.smithi092.stdout:2469 write 0x75e722a thru 0x75f39cb (0xc7a2 bytes) 2024-09-04T17:46:45.957 INFO:teuthology.orchestra.run.smithi092.stdout:2470 punch from 0x69df139 to 0x69e65de, (0x74a5 bytes) 2024-09-04T17:46:45.957 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:45.957 INFO:teuthology.orchestra.run.smithi092.stdout:2472 write 0xae3be96 thru 0xae45335 (0x94a0 bytes) 2024-09-04T17:46:45.957 INFO:teuthology.orchestra.run.smithi092.stdout:2473 writesame 0x3d4394 thru 0x3da907 data_size 0x2b(0x6574 bytes) 2024-09-04T17:46:45.957 INFO:teuthology.orchestra.run.smithi092.stdout:2474 read 0x2abd0cf thru 0x2ac093b (0x386d bytes) 2024-09-04T17:46:45.957 INFO:teuthology.orchestra.run.smithi092.stdout:2476 punch from 0x3ce7f61 to 0x3cea2b6, (0x2355 bytes) 2024-09-04T17:46:45.957 INFO:teuthology.orchestra.run.smithi092.stdout:2477 trunc from 0xae45336 to 0x7360372 2024-09-04T17:46:45.957 INFO:teuthology.orchestra.run.smithi092.stdout:2478 write 0x6aa78e9 thru 0x6ab1131 (0x9849 bytes) 2024-09-04T17:46:45.957 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:45.951+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:45.967 INFO:teuthology.orchestra.run.smithi092.stdout:2479 trunc from 0x7360372 to 0xbbd1107 2024-09-04T17:46:45.979 INFO:teuthology.orchestra.run.smithi092.stdout:2480 writesame 0x239c15d thru 0x239fa92 data_size 0x3(0x3936 bytes) 2024-09-04T17:46:45.997 INFO:teuthology.orchestra.run.smithi092.stdout:2482 writesame 0xc333ee1 thru 0xc33bdad data_size 0xd(0x7ecd bytes) 2024-09-04T17:46:45.998 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:45.998 INFO:teuthology.orchestra.run.smithi092.stdout:2484 write 0x6f1cf00 thru 0x6f1f95a (0x2a5b bytes) 2024-09-04T17:46:45.999 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-04T17:46:46.003 INFO:teuthology.orchestra.run.smithi092.stdout:2485 writesame 0x7aad3db thru 0x7ab768d data_size 0x1(0xa2b3 bytes) 2024-09-04T17:46:46.056 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:46.053+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:46.058 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:46.058 INFO:teuthology.orchestra.run.smithi092.stdout:2487 writesame 0x434446 thru 0x437e74 data_size 0xf(0x3a2f bytes) 2024-09-04T17:46:46.064 INFO:teuthology.orchestra.run.smithi092.stdout:2488 writesame 0x861094e thru 0x8612217 data_size 0x13(0x18ca bytes) 2024-09-04T17:46:46.069 INFO:teuthology.orchestra.run.smithi092.stdout:2489 read 0x8bb42ed thru 0x8bb5bb6 (0x18ca bytes) 2024-09-04T17:46:46.079 INFO:teuthology.orchestra.run.smithi092.stdout:2491 write 0xce08e5e thru 0xce14ea2 (0xc045 bytes) 2024-09-04T17:46:46.084 INFO:teuthology.orchestra.run.smithi092.stdout:2492 punch from 0x80636bc to 0x806f545, (0xbe89 bytes) 2024-09-04T17:46:46.084 INFO:teuthology.orchestra.run.smithi092.stdout:2493 writesame 0xc47ae68 thru 0xc4876ad data_size 0xf(0xc846 bytes) 2024-09-04T17:46:46.089 INFO:teuthology.orchestra.run.smithi092.stdout:2494 punch from 0x2825ffa to 0x28322d4, (0xc2da bytes) 2024-09-04T17:46:46.089 INFO:teuthology.orchestra.run.smithi092.stdout:2495 write 0x5493d97 thru 0x549f062 (0xb2cc bytes) 2024-09-04T17:46:46.093 INFO:teuthology.orchestra.run.smithi092.stdout:2496 punch from 0x747ffc0 to 0x748d8bd, (0xd8fd bytes) 2024-09-04T17:46:46.093 INFO:teuthology.orchestra.run.smithi092.stdout:2497 punch from 0xb645af1 to 0xb6525b6, (0xcac5 bytes) 2024-09-04T17:46:46.103 INFO:teuthology.orchestra.run.smithi092.stdout:2498 write 0xd70697c thru 0xd711aff (0xb184 bytes) 2024-09-04T17:46:46.105 INFO:teuthology.orchestra.run.smithi092.stdout:2499 write 0x3408064 thru 0x3410f93 (0x8f30 bytes) 2024-09-04T17:46:46.107 INFO:teuthology.orchestra.run.smithi092.stdout:2500 read 0x486d08a thru 0x4875378 (0x82ef bytes) 2024-09-04T17:46:46.107 INFO:teuthology.orchestra.run.smithi092.stdout:2501 write 0xb90450a thru 0xb90ad4b (0x6842 bytes) 2024-09-04T17:46:46.111 INFO:teuthology.orchestra.run.smithi092.stdout:2502 writesame 0x868b320 thru 0x869a869 data_size 0x1(0xf54a bytes) 2024-09-04T17:46:46.139 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:46.140 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:46:46.140 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:46:46.140 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:46.141 INFO:teuthology.orchestra.run.smithi092.stdout:2503 read 0x2aefe3c thru 0x2af26f9 (0x28be bytes) 2024-09-04T17:46:46.142 INFO:teuthology.orchestra.run.smithi092.stdout:2504 writesame 0x4df1def thru 0x4df1fa2 data_size 0x2(0x1b4 bytes) 2024-09-04T17:46:46.146 INFO:teuthology.orchestra.run.smithi092.stdout:2505 write 0xe225d0 thru 0xe2d49d (0xaece bytes) 2024-09-04T17:46:46.151 INFO:teuthology.orchestra.run.smithi092.stdout:2506 punch from 0x90911dd to 0x909b104, (0x9f27 bytes) 2024-09-04T17:46:46.151 INFO:teuthology.orchestra.run.smithi092.stdout:2507 write 0xa7a92f5 thru 0xa7b621c (0xcf28 bytes) 2024-09-04T17:46:46.154 INFO:teuthology.orchestra.run.smithi092.stdout:2508 write 0x9754c20 thru 0x975d6b0 (0x8a91 bytes) 2024-09-04T17:46:46.157 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-04T17:46:46.157 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:46.157+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:46.161 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:46:46.161 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:46:46.166 INFO:teuthology.orchestra.run.smithi092.stdout:2509 trunc from 0xd711b00 to 0x7f12553 2024-09-04T17:46:46.183 INFO:teuthology.orchestra.run.smithi092.stdout:2510 read 0x3fe3116 thru 0x3fe5f3e (0x2e29 bytes) 2024-09-04T17:46:46.183 INFO:teuthology.orchestra.run.smithi092.stdout:2513 write 0x7bcf5c thru 0x7c9a36 (0xcadb bytes) 2024-09-04T17:46:46.188 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:46.193 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:46:46.194 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:46:46.198 INFO:teuthology.orchestra.run.smithi092.stdout:2516 write 0x99ff66a thru 0x9a0c01c (0xc9b3 bytes) 2024-09-04T17:46:46.204 INFO:teuthology.orchestra.run.smithi092.stdout:2517 punch from 0x48050e1 to 0x480af8a, (0x5ea9 bytes) 2024-09-04T17:46:46.204 INFO:teuthology.orchestra.run.smithi092.stdout:2518 read 0x382c3cf thru 0x3830da8 (0x49da bytes) 2024-09-04T17:46:46.204 INFO:teuthology.orchestra.run.smithi092.stdout:2519 writesame 0x4aa2d52 thru 0x4aadfcc data_size 0x1(0xb27b bytes) 2024-09-04T17:46:46.219 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-04T17:46:46.220 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-04T17:46:46.238 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-04T17:46:46.238 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-04T17:46:46.246 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:46.246 INFO:teuthology.orchestra.run.smithi092.stdout:2522 read 0x6747491 thru 0x674f768 (0x82d8 bytes) 2024-09-04T17:46:46.247 INFO:teuthology.orchestra.run.smithi092.stdout:2523 writesame 0x7c2a237 thru 0x7c2df06 data_size 0x1(0x3cd0 bytes) 2024-09-04T17:46:46.258 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:46.257+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:46.263 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:46:46.263 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:46:46.267 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:46.267 INFO:teuthology.orchestra.run.smithi092.stdout:2525 writesame 0x8f8d5f8 thru 0x8f9d4e6 data_size 0x11(0xfeef bytes) 2024-09-04T17:46:46.273 INFO:teuthology.orchestra.run.smithi092.stdout:2526 write 0x77a0846 thru 0x77aee5c (0xe617 bytes) 2024-09-04T17:46:46.279 INFO:teuthology.orchestra.run.smithi092.stdout:2529 write 0x8ba146b thru 0x8ba1714 (0x2aa bytes) 2024-09-04T17:46:46.282 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:46.282 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:46.284 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:46.284 INFO:teuthology.orchestra.run.smithi092.stdout:2532 punch from 0x216c15d to 0x2178f49, (0xcdec bytes) 2024-09-04T17:46:46.284 INFO:teuthology.orchestra.run.smithi092.stdout:2533 trunc from 0x9a0c01d to 0x9c097b5 2024-09-04T17:46:46.285 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:46:46.285 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:46:46.285 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-04T17:46:46.294 INFO:teuthology.orchestra.run.smithi092.stdout:2534 read 0x93aacae thru 0x93b1fd3 (0x7326 bytes) 2024-09-04T17:46:46.294 INFO:teuthology.orchestra.run.smithi092.stdout:2535 punch from 0xd3fc2 to 0xda365, (0x63a3 bytes) 2024-09-04T17:46:46.294 INFO:teuthology.orchestra.run.smithi092.stdout:2536 read 0x79cc115 thru 0x79d49da (0x88c6 bytes) 2024-09-04T17:46:46.295 INFO:teuthology.orchestra.run.smithi092.stdout:2537 write 0x32c397f thru 0x32c7809 (0x3e8b bytes) 2024-09-04T17:46:46.297 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:46.297 INFO:teuthology.orchestra.run.smithi092.stdout:2540 read 0x6324be4 thru 0x6327b3e (0x2f5b bytes) 2024-09-04T17:46:46.297 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:46.297 INFO:teuthology.orchestra.run.smithi092.stdout:2543 read 0x57c5a66 thru 0x57cb88b (0x5e26 bytes) 2024-09-04T17:46:46.299 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-04T17:46:46.308 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:46:46.308 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:46:46.311 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-04T17:46:46.311 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-04T17:46:46.312 INFO:teuthology.orchestra.run.smithi092.stdout:2545 writesame 0xc415b0d thru 0xc421b60 data_size 0x42(0xc054 bytes) 2024-09-04T17:46:46.312 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:46.318 INFO:teuthology.orchestra.run.smithi092.stdout:2548 trunc from 0xc421b61 to 0x80faaab 2024-09-04T17:46:46.342 INFO:teuthology.orchestra.run.smithi092.stdout:2552 write 0x71d6fa8 thru 0x71d8609 (0x1662 bytes) 2024-09-04T17:46:46.348 INFO:teuthology.orchestra.run.smithi092.stdout:2553 read 0x68d1735 thru 0x68e0799 (0xf065 bytes) 2024-09-04T17:46:46.348 INFO:teuthology.orchestra.run.smithi092.stdout:2554 write 0x13dbaa6 thru 0x13e3e08 (0x8363 bytes) 2024-09-04T17:46:46.358 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:46:46.355+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:46:46.362 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:46.372 INFO:teuthology.orchestra.run.smithi092.stdout:2557 write 0x8e77636 thru 0x8e7b10a (0x3ad5 bytes) 2024-09-04T17:46:46.375 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:46.383 INFO:teuthology.orchestra.run.smithi092.stdout:2560 trunc from 0x8e7b10b to 0xea3c87b 2024-09-04T17:46:46.392 INFO:teuthology.orchestra.run.smithi092.stdout:2563 write 0x6d1f4d5 thru 0x6d25aaa (0x65d6 bytes) 2024-09-04T17:46:46.397 INFO:teuthology.orchestra.run.smithi092.stdout:2564 punch from 0x9b6f988 to 0x9b76db5, (0x742d bytes) 2024-09-04T17:46:46.397 INFO:teuthology.orchestra.run.smithi092.stdout:2565 punch from 0x8fdac7b to 0x8fe7f7b, (0xd300 bytes) 2024-09-04T17:46:46.398 INFO:teuthology.orchestra.run.smithi092.stdout:2566 punch from 0x8c99cf1 to 0x8ca511a, (0xb429 bytes) 2024-09-04T17:46:46.398 INFO:teuthology.orchestra.run.smithi092.stdout:2567 write 0xccc593 thru 0xcd2df0 (0x685e bytes) 2024-09-04T17:46:46.418 INFO:teuthology.orchestra.run.smithi092.stdout:2568 trunc from 0xea3c87b to 0x619eba6 2024-09-04T17:46:46.433 INFO:teuthology.orchestra.run.smithi190.stdout:get_pg_num_history pg_num_history pg_num_history(e58 pg_nums {1={16=1},2={19=8},3={24=32}} deleted_pools ) 2024-09-04T17:46:46.433 INFO:teuthology.orchestra.run.smithi190.stdout:Importing pgid 3.d 2024-09-04T17:46:46.435 INFO:teuthology.orchestra.run.smithi092.stdout:2569 write 0x5762a55 thru 0x576aeea (0x8496 bytes) 2024-09-04T17:46:46.435 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:46.436 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:46.439 INFO:teuthology.orchestra.run.smithi190.stdout:Write #3:b049e35c:::rbd_data.11346251f094.0000000000000001:head# 2024-09-04T17:46:46.439 INFO:teuthology.orchestra.run.smithi190.stdout:snapset 0=[]:{} 2024-09-04T17:46:46.440 INFO:teuthology.orchestra.run.smithi092.stdout:2571 writesame 0x550a7b thru 0x551c06 data_size 0x2(0x118c bytes) 2024-09-04T17:46:46.440 INFO:teuthology.orchestra.run.smithi190.stdout:Write #3:b0775251:::rbd_data.1134eedad849.0000000000000320:head# 2024-09-04T17:46:46.441 INFO:teuthology.orchestra.run.smithi190.stdout:snapset 10=[]:{} 2024-09-04T17:46:46.442 INFO:teuthology.orchestra.run.smithi092.stdout:2572 read 0x1e07f4a thru 0x1e0be70 (0x3f27 bytes) 2024-09-04T17:46:46.443 INFO:teuthology.orchestra.run.smithi092.stdout:2574 read 0x1dab7aa thru 0x1db4c90 (0x94e7 bytes) 2024-09-04T17:46:46.443 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:46.443 INFO:teuthology.orchestra.run.smithi092.stdout:2576 read 0x3e24b49 thru 0x3e2ad01 (0x61b9 bytes) 2024-09-04T17:46:46.443 INFO:teuthology.orchestra.run.smithi092.stdout:2577 read 0x3299aae thru 0x32a5067 (0xb5ba bytes) 2024-09-04T17:46:46.443 INFO:teuthology.orchestra.run.smithi190.stdout:Write #3:b0a816c9:::rbd_data.11345f88eb9a.0000000000000001:head# 2024-09-04T17:46:46.443 INFO:teuthology.orchestra.run.smithi190.stdout:snapset 0=[]:{} 2024-09-04T17:46:46.444 INFO:teuthology.orchestra.run.smithi092.stdout:2578 read 0x59e8044 thru 0x59ec736 (0x46f3 bytes) 2024-09-04T17:46:46.444 INFO:teuthology.orchestra.run.smithi092.stdout:2579 write 0x345d5bb thru 0x345e231 (0xc77 bytes) 2024-09-04T17:46:46.446 INFO:teuthology.orchestra.run.smithi190.stdout:Write #3:b0b7c425:::rbd_header.1134eedad849:head# 2024-09-04T17:46:46.446 INFO:teuthology.orchestra.run.smithi190.stdout:snapset 0=[]:{} 2024-09-04T17:46:46.446 INFO:teuthology.orchestra.run.smithi092.stdout:2580 write 0x99b0a4 thru 0x9a2b5e (0x7abb bytes) 2024-09-04T17:46:46.448 INFO:teuthology.orchestra.run.smithi190.stdout:Write #3:b0c50386:::rbd_data.1134b705d747.00000000000000ae:head# 2024-09-04T17:46:46.448 INFO:teuthology.orchestra.run.smithi190.stdout:snapset 11=[]:{} 2024-09-04T17:46:46.449 INFO:teuthology.orchestra.run.smithi190.stdout:Write #3:b0f77b97:::rbd_data.1134eedad849.000000000000028b:head# 2024-09-04T17:46:46.449 INFO:teuthology.orchestra.run.smithi190.stdout:snapset 10=[]:{} 2024-09-04T17:46:46.450 INFO:teuthology.orchestra.run.smithi190.stdout:Write #3:b1423a69:::rbd_data.11348d83498b.000000000000000a:c# 2024-09-04T17:46:46.453 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-04T17:46:46.454 INFO:teuthology.orchestra.run.smithi092.stdout:2581 read 0xf6d8d3 thru 0xf7a7c6 (0xcef4 bytes) 2024-09-04T17:46:46.454 INFO:teuthology.orchestra.run.smithi092.stdout:2583 writesame 0x199608d thru 0x19962bf data_size 0x1(0x233 bytes) 2024-09-04T17:46:46.456 INFO:teuthology.orchestra.run.smithi092.stdout:2584 read 0x1513225 thru 0x1517bbc (0x4998 bytes) 2024-09-04T17:46:46.457 INFO:teuthology.orchestra.run.smithi092.stdout:2585 punch from 0x4d49da9 to 0x4d49e19, (0x70 bytes) 2024-09-04T17:46:46.459 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:46.455+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:46.476 INFO:teuthology.orchestra.run.smithi190.stdout:Write #3:b1423a69:::rbd_data.11348d83498b.000000000000000a:head# 2024-09-04T17:46:46.476 INFO:teuthology.orchestra.run.smithi190.stdout:snapset c=[]:{c=[c]} 2024-09-04T17:46:46.478 INFO:teuthology.orchestra.run.smithi190.stdout:Write #3:b1784c2c:::rbd_data.1134eedad849.00000000000002b6:head# 2024-09-04T17:46:46.478 INFO:teuthology.orchestra.run.smithi190.stdout:snapset 10=[]:{} 2024-09-04T17:46:46.480 INFO:teuthology.orchestra.run.smithi190.stdout:Write #3:b18d5173:::rbd_data.1134ef3a5a7d.0000000000000009:b# 2024-09-04T17:46:46.482 INFO:teuthology.orchestra.run.smithi190.stdout:Write #3:b18d5173:::rbd_data.1134ef3a5a7d.0000000000000009:head# 2024-09-04T17:46:46.482 INFO:teuthology.orchestra.run.smithi190.stdout:snapset b=[]:{b=[b]} 2024-09-04T17:46:46.483 INFO:teuthology.orchestra.run.smithi092.stdout:2586 write 0xb6e8876 thru 0xb6f7dbd (0xf548 bytes) 2024-09-04T17:46:46.483 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is set 2024-09-04T17:46:46.483 INFO:teuthology.orchestra.run.smithi190.stdout:Write #3:b1ae20f2:::rbd_data.113431586065.0000000000000003:7# 2024-09-04T17:46:46.493 INFO:teuthology.orchestra.run.smithi092.stdout:2587 writesame 0x3354875 thru 0x335bea5 data_size 0x1(0x7631 bytes) 2024-09-04T17:46:46.501 INFO:teuthology.orchestra.run.smithi190.stdout:Write #3:b1ae20f2:::rbd_data.113431586065.0000000000000003:head# 2024-09-04T17:46:46.502 INFO:teuthology.orchestra.run.smithi190.stdout:snapset 7=[]:{7=[7]} 2024-09-04T17:46:46.505 INFO:teuthology.orchestra.run.smithi190.stdout:Write #3:b2224173:::rbd_object_map.1134fdde05d4:head# 2024-09-04T17:46:46.505 INFO:teuthology.orchestra.run.smithi190.stdout:snapset 0=[]:{} 2024-09-04T17:46:46.507 INFO:teuthology.orchestra.run.smithi190.stdout:Write #3:b25c3c94:::rbd_data.1134b705d747.00000000000000bf:head# 2024-09-04T17:46:46.507 INFO:teuthology.orchestra.run.smithi190.stdout:snapset 11=[]:{} 2024-09-04T17:46:46.508 INFO:teuthology.orchestra.run.smithi092.stdout:2588 write 0x4df913f thru 0x4e014d8 (0x839a bytes) 2024-09-04T17:46:46.509 INFO:teuthology.orchestra.run.smithi190.stdout:Write #3:b29f91c0:::rbd_data.1134eedad849.00000000000002bb:head# 2024-09-04T17:46:46.509 INFO:teuthology.orchestra.run.smithi190.stdout:snapset 10=[]:{} 2024-09-04T17:46:46.512 INFO:teuthology.orchestra.run.smithi190.stdout:Write #3:b30cd052:::rbd_data.1134eedad849.00000000000002c0:head# 2024-09-04T17:46:46.512 INFO:teuthology.orchestra.run.smithi190.stdout:snapset 10=[]:{} 2024-09-04T17:46:46.513 INFO:teuthology.orchestra.run.smithi190.stdout:Write #3:b324edf9:::rbd_data.1134eedad849.000000000000027c:head# 2024-09-04T17:46:46.513 INFO:teuthology.orchestra.run.smithi190.stdout:snapset 10=[]:{} 2024-09-04T17:46:46.514 INFO:teuthology.orchestra.run.smithi190.stdout:Write #3:b34757d9:::rbd_data.1134b705d747.0000000000000089:head# 2024-09-04T17:46:46.514 INFO:teuthology.orchestra.run.smithi190.stdout:snapset 11=[]:{} 2024-09-04T17:46:46.514 INFO:teuthology.orchestra.run.smithi190.stdout:Write #3:b35da36f:::rbd_data.1134eedad849.00000000000002d8:head# 2024-09-04T17:46:46.514 INFO:teuthology.orchestra.run.smithi190.stdout:snapset 10=[]:{} 2024-09-04T17:46:46.514 INFO:teuthology.orchestra.run.smithi092.stdout:2589 write 0x4093cef thru 0x409c5cd (0x88df bytes) 2024-09-04T17:46:46.515 INFO:teuthology.orchestra.run.smithi190.stdout:Write #3:b41c7f2e:::rbd_object_map.1134c5a43465:head# 2024-09-04T17:46:46.515 INFO:teuthology.orchestra.run.smithi190.stdout:snapset 0=[]:{} 2024-09-04T17:46:46.516 INFO:teuthology.orchestra.run.smithi190.stdout:Write #3:b56cc28c:::rbd_data.1134b705d747.00000000000000b7:head# 2024-09-04T17:46:46.516 INFO:teuthology.orchestra.run.smithi190.stdout:snapset 11=[]:{} 2024-09-04T17:46:46.516 INFO:teuthology.orchestra.run.smithi190.stdout:Write #3:b5c98cda:::rbd_object_map.113418a7bbc1.000000000000000d:head# 2024-09-04T17:46:46.516 INFO:teuthology.orchestra.run.smithi190.stdout:snapset 0=[]:{} 2024-09-04T17:46:46.517 INFO:teuthology.orchestra.run.smithi190.stdout:Write #3:b637205b:::rbd_data.1134eedad849.0000000000000351:head# 2024-09-04T17:46:46.517 INFO:teuthology.orchestra.run.smithi190.stdout:snapset 10=[]:{} 2024-09-04T17:46:46.517 INFO:teuthology.orchestra.run.smithi190.stdout:Write #3:b692b231:::rbd_data.1134cfbce83d.000000000000000c:6# 2024-09-04T17:46:46.519 INFO:teuthology.orchestra.run.smithi092.stdout:2590 clone 20 order 21 su 32768 sc 5 2024-09-04T17:46:46.520 INFO:teuthology.orchestra.run.smithi190.stdout:Write #3:b692b231:::rbd_data.1134cfbce83d.000000000000000c:head# 2024-09-04T17:46:46.520 INFO:teuthology.orchestra.run.smithi190.stdout:snapset 6=[]:{6=[6]} 2024-09-04T17:46:46.521 INFO:teuthology.orchestra.run.smithi190.stdout:Write #3:b7304e79:::rbd_data.1134eedad849.00000000000002d4:head# 2024-09-04T17:46:46.521 INFO:teuthology.orchestra.run.smithi190.stdout:snapset 10=[]:{} 2024-09-04T17:46:46.521 INFO:teuthology.orchestra.run.smithi190.stdout:Write #3:b763de10:::rbd_data.11345bd5898b.0000000000000002:head# 2024-09-04T17:46:46.522 INFO:teuthology.orchestra.run.smithi190.stdout:snapset 0=[]:{} 2024-09-04T17:46:46.532 INFO:teuthology.orchestra.run.smithi190.stdout:Write #3:b76ef7b6:::rbd_data.1134eedad849.0000000000000331:head# 2024-09-04T17:46:46.533 INFO:teuthology.orchestra.run.smithi190.stdout:snapset 10=[]:{} 2024-09-04T17:46:46.533 INFO:teuthology.orchestra.run.smithi190.stdout:Write #3:b77d7938:::rbd_data.1134eedad849.0000000000000279:head# 2024-09-04T17:46:46.533 INFO:teuthology.orchestra.run.smithi190.stdout:snapset 10=[]:{} 2024-09-04T17:46:46.534 INFO:teuthology.orchestra.run.smithi190.stdout:Write #3:b7c36f16:::rbd_data.1134eedad849.0000000000000381:head# 2024-09-04T17:46:46.534 INFO:teuthology.orchestra.run.smithi190.stdout:snapset 10=[]:{} 2024-09-04T17:46:46.534 INFO:teuthology.orchestra.run.smithi190.stdout:write_pg epoch 58 info 3.d( v 58'499 (0'0,58'499] local-lis/les=37/38 n=31 ec=24/24 lis/c=37/37 les/c/f=38/38/0 sis=37) 2024-09-04T17:46:46.536 INFO:teuthology.orchestra.run.smithi190.stdout:Import successful 2024-09-04T17:46:46.560 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:46:46.559+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:46:46.585 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:46.586 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:46.599 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-04T17:46:46.684 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:46.685+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:46.762 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:46:46.759+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:46:46.763 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:46.763 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:46.777 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-04T17:46:46.863 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:46.859+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:46.948 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:46.948 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:46.962 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-04T17:46:46.965 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:46:46.963+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:46:47.066 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:47.063+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:47.085 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f /var/log/ceph/exp.3.d.6 2024-09-04T17:46:47.086 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:47.086 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:47.101 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-04T17:46:47.169 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:47.167+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:47.265 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:47.265 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:47.269 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:47.267+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:47.279 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-04T17:46:47.372 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:47.369+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:47.400 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:47.400 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:47.414 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-04T17:46:47.500 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:47.501+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:47.572 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:47.569+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:47.589 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:47.589 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:47.603 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-04T17:46:47.635 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone19 from 0xb6f7dbe (overlap 0x3d89495) to 0x24ebf8c 2024-09-04T17:46:47.692 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:47.693+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:47.770 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:47.771 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:47.774 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:47.771+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:47.787 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-04T17:46:47.826 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #18, image image_client.0-clone18 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent19 2024-09-04T17:46:47.875 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:47.871+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:47.952 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:47.953 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:47.967 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-04T17:46:48.012 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:48.011+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:48.048 INFO:teuthology.orchestra.run.smithi092.stdout:2591 read 0x18c46ba thru 0x18d3ce8 (0xf62f bytes) 2024-09-04T17:46:48.071 INFO:teuthology.orchestra.run.smithi092.stdout:2592 write 0xda14d82 thru 0xda1ec7a (0x9ef9 bytes) 2024-09-04T17:46:48.076 INFO:teuthology.orchestra.run.smithi092.stdout:2593 write 0x5b50478 thru 0x5b5c960 (0xc4e9 bytes) 2024-09-04T17:46:48.076 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:48.077+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:48.108 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:48.109 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:48.116 INFO:teuthology.orchestra.run.smithi092.stdout:2594 trunc from 0xda1ec7b to 0x59ac2a 2024-09-04T17:46:48.131 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-04T17:46:48.195 INFO:teuthology.orchestra.run.smithi092.stdout:2595 read 0x4002eb thru 0x40c868 (0xc57e bytes) 2024-09-04T17:46:48.195 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:46:48.191+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:46:48.235 INFO:teuthology.orchestra.run.smithi092.stdout:2597 write 0xdfe7c0d thru 0xdfe825e (0x652 bytes) 2024-09-04T17:46:48.242 INFO:teuthology.orchestra.run.smithi092.stdout:2598 punch from 0xd9a1f71 to 0xd9ae3a7, (0xc436 bytes) 2024-09-04T17:46:48.242 INFO:teuthology.orchestra.run.smithi092.stdout:2599 write 0xdda3fc4 thru 0xddadea6 (0x9ee3 bytes) 2024-09-04T17:46:48.248 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:48.248 INFO:teuthology.orchestra.run.smithi092.stdout:2601 read 0x52077de thru 0x520a96d (0x3190 bytes) 2024-09-04T17:46:48.248 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:48.248 INFO:teuthology.orchestra.run.smithi092.stdout:2603 write 0x1cbff04 thru 0x1ccc688 (0xc785 bytes) 2024-09-04T17:46:48.278 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:48.275+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:48.280 INFO:teuthology.orchestra.run.smithi092.stdout:2604 trunc from 0xdfe825f to 0x194b0b 2024-09-04T17:46:48.308 INFO:teuthology.orchestra.run.smithi092.stdout:2606 punch from 0x9ccae to 0xa10f8, (0x444a bytes) 2024-09-04T17:46:48.312 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-04T17:46:48.334 INFO:teuthology.orchestra.run.smithi092.stdout:2607 writesame 0x7b2087d thru 0x7b2f94b data_size 0x1(0xf0cf bytes) 2024-09-04T17:46:48.335 INFO:teuthology.orchestra.run.smithi092.stdout:2608 read 0x274d55d thru 0x275cc34 (0xf6d8 bytes) 2024-09-04T17:46:48.335 INFO:teuthology.orchestra.run.smithi092.stdout:2609 writesame 0x4a49c25 thru 0x4a4a94b data_size 0xd(0xd27 bytes) 2024-09-04T17:46:48.335 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:48.335 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:48.340 INFO:teuthology.orchestra.run.smithi092.stdout:2610 read 0x6e37201 thru 0x6e3da67 (0x6867 bytes) 2024-09-04T17:46:48.340 INFO:teuthology.orchestra.run.smithi092.stdout:2611 punch from 0x2c71f10 to 0x2c77e51, (0x5f41 bytes) 2024-09-04T17:46:48.353 INFO:teuthology.orchestra.run.smithi092.stdout:2612 write 0x8843a81 thru 0x884efcc (0xb54c bytes) 2024-09-04T17:46:48.353 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-04T17:46:48.374 INFO:teuthology.orchestra.run.smithi092.stdout:2613 writesame 0xae1cad7 thru 0xae1fb10 data_size 0x1(0x303a bytes) 2024-09-04T17:46:48.374 INFO:teuthology.orchestra.run.smithi092.stdout:2614 read 0x133dd9b thru 0x133e54a (0x7b0 bytes) 2024-09-04T17:46:48.379 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:48.377+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:48.388 INFO:teuthology.orchestra.run.smithi092.stdout:2615 trunc from 0xae1fb11 to 0x187fc13 2024-09-04T17:46:48.411 INFO:teuthology.orchestra.run.smithi092.stdout:2617 trunc from 0x187fc13 to 0x5294014 2024-09-04T17:46:48.422 INFO:teuthology.orchestra.run.smithi092.stdout:2618 read 0x1f777bd thru 0x1f78339 (0xb7d bytes) 2024-09-04T17:46:48.422 INFO:teuthology.orchestra.run.smithi092.stdout:2621 punch from 0xd2ca29 to 0xd32dc9, (0x63a0 bytes) 2024-09-04T17:46:48.423 INFO:teuthology.orchestra.run.smithi092.stdout:2625 writesame 0x3e2ec18 thru 0x3e3dc74 data_size 0x35(0xf05d bytes) 2024-09-04T17:46:48.444 INFO:teuthology.orchestra.run.smithi092.stdout:2626 writesame 0x594ac7a thru 0x5953001 data_size 0x8a(0x8388 bytes) 2024-09-04T17:46:48.444 INFO:teuthology.orchestra.run.smithi092.stdout:2627 writesame 0x1fd36ff thru 0x1fddca3 data_size 0x37(0xa5a5 bytes) 2024-09-04T17:46:48.468 INFO:teuthology.orchestra.run.smithi092.stdout:2630 write 0xc0f3b41 thru 0xc0fcda9 (0x9269 bytes) 2024-09-04T17:46:48.475 INFO:teuthology.orchestra.run.smithi092.stdout:2631 trunc from 0xc0fcdaa to 0xdeecaed 2024-09-04T17:46:48.481 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:48.481+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:48.490 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:46:48.491 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:46:48.498 INFO:teuthology.orchestra.run.smithi092.stdout:2632 trunc from 0xdeecaed to 0x589ec65 2024-09-04T17:46:48.509 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-04T17:46:48.510 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:46:48.511 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:46:48.514 INFO:teuthology.orchestra.run.smithi092.stdout:2635 trunc from 0x589ec65 to 0x46a5902 2024-09-04T17:46:48.524 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:48.524 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:48.525 INFO:teuthology.orchestra.run.smithi092.stdout:2636 punch from 0x16ea8de to 0x16f0eeb, (0x660d bytes) 2024-09-04T17:46:48.529 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-04T17:46:48.530 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-04T17:46:48.531 INFO:teuthology.orchestra.run.smithi092.stdout:2637 trunc from 0x46a5902 to 0x1275ba0 2024-09-04T17:46:48.547 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-04T17:46:48.549 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-04T17:46:48.549 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-04T17:46:48.567 INFO:teuthology.orchestra.run.smithi092.stdout:2639 writesame 0x7d47250 thru 0x7d4ff42 data_size 0x1(0x8cf3 bytes) 2024-09-04T17:46:48.567 INFO:teuthology.orchestra.run.smithi092.stdout:2641 write 0x59e0425 thru 0x59ef484 (0xf060 bytes) 2024-09-04T17:46:48.571 INFO:teuthology.orchestra.run.smithi092.stdout:2642 punch from 0x3c4cb5c to 0x3c4da52, (0xef6 bytes) 2024-09-04T17:46:48.571 INFO:teuthology.orchestra.run.smithi092.stdout:2643 write 0x27ca123 thru 0x27cb704 (0x15e2 bytes) 2024-09-04T17:46:48.574 INFO:teuthology.orchestra.run.smithi092.stdout:2644 read 0x971d4b thru 0x974672 (0x2928 bytes) 2024-09-04T17:46:48.575 INFO:teuthology.orchestra.run.smithi092.stdout:2647 writesame 0x1c0d77b thru 0x1c1657a data_size 0x80(0x8e00 bytes) 2024-09-04T17:46:48.581 INFO:teuthology.orchestra.run.smithi092.stdout:2648 read 0x53721b thru 0x544411 (0xd1f7 bytes) 2024-09-04T17:46:48.581 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:48.581+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:48.581 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:48.582 INFO:teuthology.orchestra.run.smithi092.stdout:2651 read 0xc2c3aa thru 0xc3366d (0x72c4 bytes) 2024-09-04T17:46:48.582 INFO:teuthology.orchestra.run.smithi092.stdout:2653 writesame 0x26bbf48 thru 0x26c91c1 data_size 0x1d(0xd27a bytes) 2024-09-04T17:46:48.585 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:46:48.585 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:46:48.588 INFO:teuthology.orchestra.run.smithi092.stdout:2657 read 0x448af78 thru 0x448e736 (0x37bf bytes) 2024-09-04T17:46:48.588 INFO:teuthology.orchestra.run.smithi092.stdout:2660 read 0x7ba149e thru 0x7bafb97 (0xe6fa bytes) 2024-09-04T17:46:48.588 INFO:teuthology.orchestra.run.smithi092.stdout:2661 read 0x6fbb3e9 thru 0x6fc9215 (0xde2d bytes) 2024-09-04T17:46:48.588 INFO:teuthology.orchestra.run.smithi092.stdout:2663 read 0x573eb5c thru 0x574c28b (0xd730 bytes) 2024-09-04T17:46:48.600 INFO:teuthology.orchestra.run.smithi092.stdout:2664 writesame 0x82cea90 thru 0x82d4e03 data_size 0x5f(0x6374 bytes) 2024-09-04T17:46:48.601 INFO:teuthology.orchestra.run.smithi092.stdout:2665 read 0x7e04a16 thru 0x7e14275 (0xf860 bytes) 2024-09-04T17:46:48.601 INFO:teuthology.orchestra.run.smithi092.stdout:2667 punch from 0x8072703 to 0x8082399, (0xfc96 bytes) 2024-09-04T17:46:48.609 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:46:48.610 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:46:48.610 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-04T17:46:48.615 INFO:teuthology.orchestra.run.smithi092.stdout:2668 write 0x9c9e530 thru 0x9ca3f02 (0x59d3 bytes) 2024-09-04T17:46:48.622 INFO:teuthology.orchestra.run.smithi092.stdout:2670 writesame 0x799aa14 thru 0x799f796 data_size 0x1(0x4d83 bytes) 2024-09-04T17:46:48.643 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:46:48.643 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:46:48.646 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-04T17:46:48.647 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-04T17:46:48.653 INFO:teuthology.orchestra.run.smithi092.stdout:2671 punch from 0x1756874 to 0x1760580, (0x9d0c bytes) 2024-09-04T17:46:48.653 INFO:teuthology.orchestra.run.smithi092.stdout:2673 read 0x10e89e5 thru 0x10ebb3a (0x3156 bytes) 2024-09-04T17:46:48.654 INFO:teuthology.orchestra.run.smithi092.stdout:2674 trunc from 0x9ca3f03 to 0xafc7af5 2024-09-04T17:46:48.664 INFO:teuthology.orchestra.run.smithi092.stdout:2675 writesame 0x22ed729 thru 0x22f5dfc data_size 0x2(0x86d4 bytes) 2024-09-04T17:46:48.674 INFO:teuthology.orchestra.run.smithi092.stdout:2676 punch from 0xa2a1511 to 0xa2af222, (0xdd11 bytes) 2024-09-04T17:46:48.674 INFO:teuthology.orchestra.run.smithi092.stdout:2677 punch from 0x4710ea5 to 0x471619c, (0x52f7 bytes) 2024-09-04T17:46:48.674 INFO:teuthology.orchestra.run.smithi092.stdout:2678 write 0x6021e5 thru 0x610530 (0xe34c bytes) 2024-09-04T17:46:48.678 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:48.678 INFO:teuthology.orchestra.run.smithi092.stdout:2680 write 0x8334d71 thru 0x8338577 (0x3807 bytes) 2024-09-04T17:46:48.681 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:48.679+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:48.685 INFO:teuthology.orchestra.run.smithi092.stdout:2681 punch from 0x26dad94 to 0x26e36c9, (0x8935 bytes) 2024-09-04T17:46:48.687 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:48.687 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:46:48.687 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:46:48.687 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:48.688 INFO:teuthology.orchestra.run.smithi092.stdout:2682 trunc from 0xafc7af5 to 0x95e9884 2024-09-04T17:46:48.705 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-04T17:46:48.706 INFO:teuthology.orchestra.run.smithi092.stdout:2685 write 0x804cb88 thru 0x804d0e6 (0x55f bytes) 2024-09-04T17:46:48.711 INFO:teuthology.orchestra.run.smithi092.stdout:2687 write 0x730f482 thru 0x731a618 (0xb197 bytes) 2024-09-04T17:46:48.717 INFO:teuthology.orchestra.run.smithi092.stdout:2688 punch from 0x54ef4d to 0x554bb4, (0x5c67 bytes) 2024-09-04T17:46:48.717 INFO:teuthology.orchestra.run.smithi092.stdout:2689 write 0x6680fe3 thru 0x66821df (0x11fd bytes) 2024-09-04T17:46:48.722 INFO:teuthology.orchestra.run.smithi092.stdout:2690 read 0x4f77ddb thru 0x4f828b3 (0xaad9 bytes) 2024-09-04T17:46:48.722 INFO:teuthology.orchestra.run.smithi092.stdout:2691 punch from 0x225d316 to 0x22612c2, (0x3fac bytes) 2024-09-04T17:46:48.727 INFO:teuthology.orchestra.run.smithi092.stdout:2692 trunc from 0x95e9884 to 0xd015a6b 2024-09-04T17:46:48.738 INFO:teuthology.orchestra.run.smithi092.stdout:2693 punch from 0xbf557f6 to 0xbf5f8cc, (0xa0d6 bytes) 2024-09-04T17:46:48.743 INFO:teuthology.orchestra.run.smithi092.stdout:2694 trunc from 0xd015a6b to 0x92e42e7 2024-09-04T17:46:48.754 INFO:teuthology.orchestra.run.smithi092.stdout:2695 read 0x8d817b7 thru 0x8d887c6 (0x7010 bytes) 2024-09-04T17:46:48.754 INFO:teuthology.orchestra.run.smithi092.stdout:2697 punch from 0x3c0cdce to 0x3c1ba63, (0xec95 bytes) 2024-09-04T17:46:48.754 INFO:teuthology.orchestra.run.smithi092.stdout:2698 punch from 0x70fe8fe to 0x7102179, (0x387b bytes) 2024-09-04T17:46:48.754 INFO:teuthology.orchestra.run.smithi092.stdout:2699 punch from 0x773770d to 0x7738c9e, (0x1591 bytes) 2024-09-04T17:46:48.754 INFO:teuthology.orchestra.run.smithi092.stdout:2700 read 0x6c0ef9b thru 0x6c18264 (0x92ca bytes) 2024-09-04T17:46:48.754 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:48.754 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:48.754 INFO:teuthology.orchestra.run.smithi092.stdout:2703 read 0x78d0fd4 thru 0x78e0f6d (0xff9a bytes) 2024-09-04T17:46:48.755 INFO:teuthology.orchestra.run.smithi092.stdout:2704 punch from 0x52fab55 to 0x530227f, (0x772a bytes) 2024-09-04T17:46:48.771 INFO:teuthology.orchestra.run.smithi092.stdout:2705 write 0xe0b3808 thru 0xe0c26dd (0xeed6 bytes) 2024-09-04T17:46:48.781 INFO:teuthology.orchestra.run.smithi092.stdout:2706 read 0x4fc4a2f thru 0x4fce31b (0x98ed bytes) 2024-09-04T17:46:48.781 INFO:teuthology.orchestra.run.smithi092.stdout:2708 punch from 0x4fc09f0 to 0x4fc4d0a, (0x431a bytes) 2024-09-04T17:46:48.781 INFO:teuthology.orchestra.run.smithi092.stdout:2709 write 0xde4b494 thru 0xde51837 (0x63a4 bytes) 2024-09-04T17:46:48.782 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:48.781+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:48.797 INFO:teuthology.orchestra.run.smithi092.stdout:2710 trunc from 0xe0c26de to 0x6d5a83b 2024-09-04T17:46:48.822 INFO:teuthology.orchestra.run.smithi092.stdout:2712 read 0x2c437ad thru 0x2c474cc (0x3d20 bytes) 2024-09-04T17:46:48.833 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:48.834 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:46:48.834 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:46:48.834 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:46:48.834 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:48.844 INFO:teuthology.orchestra.run.smithi092.stdout:2713 writesame 0xc1ac73e thru 0xc1b6db4 data_size 0xf(0xa677 bytes) 2024-09-04T17:46:48.844 INFO:teuthology.orchestra.run.smithi092.stdout:2714 write 0xb8e4c82 thru 0xb8e6924 (0x1ca3 bytes) 2024-09-04T17:46:48.848 INFO:teuthology.orchestra.run.smithi092.stdout:2715 writesame 0x7cf2e3b thru 0x7d01b0a data_size 0x24(0xecd0 bytes) 2024-09-04T17:46:48.850 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-04T17:46:48.857 INFO:teuthology.orchestra.run.smithi092.stdout:2716 read 0x6ae6cc7 thru 0x6aeb442 (0x477c bytes) 2024-09-04T17:46:48.860 INFO:teuthology.orchestra.run.smithi092.stdout:2717 trunc from 0xc1b6db5 to 0xd989f16 2024-09-04T17:46:48.872 INFO:teuthology.orchestra.run.smithi092.stdout:2718 read 0x6172e9d thru 0x6173e67 (0xfcb bytes) 2024-09-04T17:46:48.872 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:48.872 INFO:teuthology.orchestra.run.smithi092.stdout:2721 punch from 0xb8c057 to 0xb9b921, (0xf8ca bytes) 2024-09-04T17:46:48.872 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:48.872 INFO:teuthology.orchestra.run.smithi092.stdout:2724 write 0xd312aa3 thru 0xd31ce38 (0xa396 bytes) 2024-09-04T17:46:48.883 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:48.881+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:48.885 INFO:teuthology.orchestra.run.smithi092.stdout:2725 trunc from 0xd989f16 to 0x7deb72d 2024-09-04T17:46:48.906 INFO:teuthology.orchestra.run.smithi092.stdout:2726 write 0x6dee37 thru 0x6eb1e8 (0xc3b2 bytes) 2024-09-04T17:46:48.926 INFO:teuthology.orchestra.run.smithi092.stdout:2727 write 0xb7fc712 thru 0xb80c68c (0xff7b bytes) 2024-09-04T17:46:48.938 INFO:teuthology.orchestra.run.smithi092.stdout:2729 trunc from 0xb80c68d to 0xdeaaf06 2024-09-04T17:46:48.949 INFO:teuthology.orchestra.run.smithi092.stdout:2730 writesame 0x118cf5e thru 0x119a212 data_size 0x13(0xd2b5 bytes) 2024-09-04T17:46:48.958 INFO:teuthology.orchestra.run.smithi092.stdout:2731 write 0x329a47b thru 0x32a420f (0x9d95 bytes) 2024-09-04T17:46:48.964 INFO:teuthology.orchestra.run.smithi092.stdout:2732 write 0x79b423 thru 0x79b439 (0x17 bytes) 2024-09-04T17:46:48.967 INFO:teuthology.orchestra.run.smithi092.stdout:2733 read 0xcc55ac8 thru 0xcc638e2 (0xde1b bytes) 2024-09-04T17:46:48.967 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:48.967 INFO:teuthology.orchestra.run.smithi092.stdout:2736 read 0x17abafe thru 0x17ac77d (0xc80 bytes) 2024-09-04T17:46:48.968 INFO:teuthology.orchestra.run.smithi092.stdout:2737 read 0x107b2d4 thru 0x1085d4f (0xaa7c bytes) 2024-09-04T17:46:48.969 INFO:teuthology.orchestra.run.smithi092.stdout:2739 writesame 0x1cc9f1b thru 0x1ccc7c6 data_size 0x13(0x28ac bytes) 2024-09-04T17:46:48.974 INFO:teuthology.orchestra.run.smithi092.stdout:2741 writesame 0x3266636 thru 0x326b722 data_size 0x1(0x50ed bytes) 2024-09-04T17:46:48.984 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:48.985+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:48.990 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:48.990 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:46:48.990 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:46:48.990 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:46:48.990 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:48.990 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6103 3.1f 3:fec7e474:::rbd_data.1134cfbce83d.0000000000000003:6 [read 1736704~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:48.990 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.421848+0000", 2024-09-04T17:46:48.991 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.567826721000003, 2024-09-04T17:46:48.991 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027313884, 2024-09-04T17:46:48.991 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:48.991 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:48.991 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:48.991 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:48.991 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:48.991 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:48.991 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6103 2024-09-04T17:46:48.991 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:48.991 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:48.991 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:48.992 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:48.992 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.421848+0000", 2024-09-04T17:46:48.992 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:48.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:48.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:48.992 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:48.992 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.421848+0000", 2024-09-04T17:46:48.992 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:48.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:48.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:48.992 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:48.992 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.421849+0000", 2024-09-04T17:46:48.992 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.144e-06 2024-09-04T17:46:48.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:48.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:48.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:48.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.421855+0000", 2024-09-04T17:46:48.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.003e-06 2024-09-04T17:46:48.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:48.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:48.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:48.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.421872+0000", 2024-09-04T17:46:48.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7193e-05 2024-09-04T17:46:48.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:48.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:48.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:48.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.421886+0000", 2024-09-04T17:46:48.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.331e-05 2024-09-04T17:46:48.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:48.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:48.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:48.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.444849+0000", 2024-09-04T17:46:48.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.022962755000000001 2024-09-04T17:46:48.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:48.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:48.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:48.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.444990+0000", 2024-09-04T17:46:48.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014129999999999999 2024-09-04T17:46:48.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:48.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:48.995 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:48.995 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.449162+0000", 2024-09-04T17:46:48.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041721789999999998 2024-09-04T17:46:48.995 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:48.995 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:48.995 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:48.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:48.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:48.995 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6140 3.17 3:ed419000:::rbd_data.1134cfbce83d.000000000000000b:6 [sparse-read 622592~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:48.995 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.422576+0000", 2024-09-04T17:46:48.995 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.567099132999999, 2024-09-04T17:46:48.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025138273999999999, 2024-09-04T17:46:48.996 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:48.996 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:48.996 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:48.996 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:48.996 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:48.996 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:48.996 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6140 2024-09-04T17:46:48.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:48.996 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:48.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:48.996 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:48.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422576+0000", 2024-09-04T17:46:48.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:48.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:48.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:48.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:48.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422576+0000", 2024-09-04T17:46:48.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:48.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:48.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:48.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:48.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422578+0000", 2024-09-04T17:46:48.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8870000000000001e-06 2024-09-04T17:46:48.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:48.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:48.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:48.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422587+0000", 2024-09-04T17:46:48.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5999999999999996e-06 2024-09-04T17:46:48.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:48.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:48.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:48.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422617+0000", 2024-09-04T17:46:48.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9339000000000001e-05 2024-09-04T17:46:48.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:48.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:48.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:48.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422636+0000", 2024-09-04T17:46:48.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9524000000000001e-05 2024-09-04T17:46:48.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:48.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:48.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:48.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.445586+0000", 2024-09-04T17:46:48.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.022950084999999999 2024-09-04T17:46:48.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:48.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:48.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:48.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.445788+0000", 2024-09-04T17:46:48.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000202029 2024-09-04T17:46:48.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:48.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:48.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:48.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.447714+0000", 2024-09-04T17:46:49.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00192581 2024-09-04T17:46:49.000 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.000 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:49.000 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.000 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6141 3.17 3:ed419000:::rbd_data.1134cfbce83d.000000000000000b:6 [sparse-read 983040~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:49.000 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.422660+0000", 2024-09-04T17:46:49.000 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.567014469999997, 2024-09-04T17:46:49.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025710566000000001, 2024-09-04T17:46:49.000 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:49.000 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:49.000 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:49.001 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:49.001 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:49.001 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:49.001 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6141 2024-09-04T17:46:49.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.001 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:49.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:49.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422660+0000", 2024-09-04T17:46:49.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:49.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422660+0000", 2024-09-04T17:46:49.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:49.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422662+0000", 2024-09-04T17:46:49.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.313e-06 2024-09-04T17:46:49.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:49.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422691+0000", 2024-09-04T17:46:49.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9207e-05 2024-09-04T17:46:49.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:49.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422711+0000", 2024-09-04T17:46:49.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0225999999999999e-05 2024-09-04T17:46:49.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:49.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422727+0000", 2024-09-04T17:46:49.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5982999999999999e-05 2024-09-04T17:46:49.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:49.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.447767+0000", 2024-09-04T17:46:49.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025039558 2024-09-04T17:46:49.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:49.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.447871+0000", 2024-09-04T17:46:49.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104287 2024-09-04T17:46:49.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:49.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.448371+0000", 2024-09-04T17:46:49.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00049999199999999999 2024-09-04T17:46:49.004 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.004 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:49.005 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.005 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6142 3.17 3:ed419000:::rbd_data.1134cfbce83d.000000000000000b:6 [sparse-read 1343488~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:49.005 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.422750+0000", 2024-09-04T17:46:49.005 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.566925238000003, 2024-09-04T17:46:49.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026566215000000001, 2024-09-04T17:46:49.005 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:49.005 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:49.005 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:49.005 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:49.005 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:49.006 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:49.006 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6142 2024-09-04T17:46:49.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.006 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:49.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.006 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:49.006 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422750+0000", 2024-09-04T17:46:49.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.006 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:49.006 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422750+0000", 2024-09-04T17:46:49.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:49.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422751+0000", 2024-09-04T17:46:49.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6700000000000001e-06 2024-09-04T17:46:49.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:49.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422757+0000", 2024-09-04T17:46:49.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9669999999999998e-06 2024-09-04T17:46:49.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:49.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422775+0000", 2024-09-04T17:46:49.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8073000000000001e-05 2024-09-04T17:46:49.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:49.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422789+0000", 2024-09-04T17:46:49.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3709999999999999e-05 2024-09-04T17:46:49.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:49.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.448420+0000", 2024-09-04T17:46:49.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025630627999999999 2024-09-04T17:46:49.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:49.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.448590+0000", 2024-09-04T17:46:49.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000170213 2024-09-04T17:46:49.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:49.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.449316+0000", 2024-09-04T17:46:49.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00072595400000000001 2024-09-04T17:46:49.009 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.009 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:49.009 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.010 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6143 3.17 3:ed419000:::rbd_data.1134cfbce83d.000000000000000b:6 [read 1736704~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:49.010 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.422807+0000", 2024-09-04T17:46:49.010 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.566867907999999, 2024-09-04T17:46:49.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027271169000000001, 2024-09-04T17:46:49.010 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:49.010 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:49.010 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:49.010 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:49.010 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:49.011 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:49.011 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6143 2024-09-04T17:46:49.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.011 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:49.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:49.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422807+0000", 2024-09-04T17:46:49.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:49.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422807+0000", 2024-09-04T17:46:49.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:49.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422809+0000", 2024-09-04T17:46:49.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6139999999999999e-06 2024-09-04T17:46:49.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:49.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422814+0000", 2024-09-04T17:46:49.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1800000000000004e-06 2024-09-04T17:46:49.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:49.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422831+0000", 2024-09-04T17:46:49.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7433000000000001e-05 2024-09-04T17:46:49.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:49.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422844+0000", 2024-09-04T17:46:49.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3216e-05 2024-09-04T17:46:49.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:49.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.449397+0000", 2024-09-04T17:46:49.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026552572 2024-09-04T17:46:49.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:49.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.449526+0000", 2024-09-04T17:46:49.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000129207 2024-09-04T17:46:49.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:49.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.450078+0000", 2024-09-04T17:46:49.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00055194700000000001 2024-09-04T17:46:49.014 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.014 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:49.014 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.015 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6203 3.1f 3:fec7e474:::rbd_data.1134cfbce83d.0000000000000003:6 [read 327680~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:49.015 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423445+0000", 2024-09-04T17:46:49.015 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.566229495999998, 2024-09-04T17:46:49.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027179207, 2024-09-04T17:46:49.015 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:49.015 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:49.015 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:49.015 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:49.015 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:49.015 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:49.015 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6203 2024-09-04T17:46:49.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.016 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:49.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.016 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:49.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423445+0000", 2024-09-04T17:46:49.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.016 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:49.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423445+0000", 2024-09-04T17:46:49.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:49.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423446+0000", 2024-09-04T17:46:49.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.077e-06 2024-09-04T17:46:49.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:49.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423452+0000", 2024-09-04T17:46:49.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.113e-06 2024-09-04T17:46:49.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:49.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423468+0000", 2024-09-04T17:46:49.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6685999999999999e-05 2024-09-04T17:46:49.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:49.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423481+0000", 2024-09-04T17:46:49.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2673999999999999e-05 2024-09-04T17:46:49.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:49.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.449779+0000", 2024-09-04T17:46:49.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026297838 2024-09-04T17:46:49.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:49.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.450020+0000", 2024-09-04T17:46:49.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024114399999999999 2024-09-04T17:46:49.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:49.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.450625+0000", 2024-09-04T17:46:49.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00060467500000000005 2024-09-04T17:46:49.020 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.020 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:49.020 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.021 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6204 3.1f 3:fec7e474:::rbd_data.1134cfbce83d.0000000000000003:6 [read 688128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:49.021 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423498+0000", 2024-09-04T17:46:49.021 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.566176560000002, 2024-09-04T17:46:49.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029262579, 2024-09-04T17:46:49.021 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:49.021 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:49.021 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:49.021 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:49.021 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:49.021 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:49.021 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6204 2024-09-04T17:46:49.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.022 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:49.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:49.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423498+0000", 2024-09-04T17:46:49.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:49.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423498+0000", 2024-09-04T17:46:49.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:49.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423500+0000", 2024-09-04T17:46:49.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.28e-06 2024-09-04T17:46:49.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:49.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423506+0000", 2024-09-04T17:46:49.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0059999999999998e-06 2024-09-04T17:46:49.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:49.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423524+0000", 2024-09-04T17:46:49.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8683999999999999e-05 2024-09-04T17:46:49.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:49.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423537+0000", 2024-09-04T17:46:49.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3030000000000001e-05 2024-09-04T17:46:49.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:49.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.450697+0000", 2024-09-04T17:46:49.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027159793000000002 2024-09-04T17:46:49.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:49.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.450835+0000", 2024-09-04T17:46:49.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013773799999999999 2024-09-04T17:46:49.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:49.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.452761+0000", 2024-09-04T17:46:49.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019260480000000001 2024-09-04T17:46:49.025 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.025 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:49.025 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.026 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6205 3.1f 3:fec7e474:::rbd_data.1134cfbce83d.0000000000000003:6 [read 1048576~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:49.026 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423555+0000", 2024-09-04T17:46:49.026 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.566119577000002, 2024-09-04T17:46:49.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029905979999999999, 2024-09-04T17:46:49.026 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:49.026 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:49.026 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:49.026 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:49.026 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:49.026 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:49.026 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6205 2024-09-04T17:46:49.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.027 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:49.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:49.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423555+0000", 2024-09-04T17:46:49.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:49.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423555+0000", 2024-09-04T17:46:49.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:49.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423556+0000", 2024-09-04T17:46:49.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1599999999999999e-06 2024-09-04T17:46:49.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:49.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423562+0000", 2024-09-04T17:46:49.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1270000000000002e-06 2024-09-04T17:46:49.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:49.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423579+0000", 2024-09-04T17:46:49.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6986e-05 2024-09-04T17:46:49.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:49.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423591+0000", 2024-09-04T17:46:49.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2544000000000001e-05 2024-09-04T17:46:49.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:49.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.452819+0000", 2024-09-04T17:46:49.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029227418000000002 2024-09-04T17:46:49.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:49.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.452985+0000", 2024-09-04T17:46:49.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000166675 2024-09-04T17:46:49.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:49.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.453461+0000", 2024-09-04T17:46:49.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00047606999999999999 2024-09-04T17:46:49.030 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.030 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:49.030 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.031 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6206 3.1f 3:fec7e474:::rbd_data.1134cfbce83d.0000000000000003:6 [sparse-read 1409024~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:49.031 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423608+0000", 2024-09-04T17:46:49.031 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.566066943999999, 2024-09-04T17:46:49.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.031860753999999998, 2024-09-04T17:46:49.031 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:49.031 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:49.031 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:49.031 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:49.031 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:49.031 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:49.031 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6206 2024-09-04T17:46:49.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.031 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:49.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:49.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423608+0000", 2024-09-04T17:46:49.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:49.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423608+0000", 2024-09-04T17:46:49.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:49.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423609+0000", 2024-09-04T17:46:49.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0300000000000001e-06 2024-09-04T17:46:49.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:49.033 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423614+0000", 2024-09-04T17:46:49.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0200000000000002e-06 2024-09-04T17:46:49.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:49.033 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423630+0000", 2024-09-04T17:46:49.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6495999999999999e-05 2024-09-04T17:46:49.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:49.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423643+0000", 2024-09-04T17:46:49.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2053e-05 2024-09-04T17:46:49.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:49.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.453527+0000", 2024-09-04T17:46:49.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029884796000000002 2024-09-04T17:46:49.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:49.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.453632+0000", 2024-09-04T17:46:49.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104515 2024-09-04T17:46:49.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:49.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.455469+0000", 2024-09-04T17:46:49.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001836844 2024-09-04T17:46:49.035 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.035 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:49.035 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.035 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6207 3.1f 3:fec7e474:::rbd_data.1134cfbce83d.0000000000000003:6 [sparse-read 1769472~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:49.035 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423662+0000", 2024-09-04T17:46:49.035 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.566012481000001, 2024-09-04T17:46:49.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.033074294999999997, 2024-09-04T17:46:49.036 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:49.036 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:49.036 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:49.036 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:49.036 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:49.036 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:49.036 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6207 2024-09-04T17:46:49.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.036 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:49.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:49.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423662+0000", 2024-09-04T17:46:49.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.037 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:49.037 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423662+0000", 2024-09-04T17:46:49.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.037 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:49.037 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423664+0000", 2024-09-04T17:46:49.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4929999999999999e-06 2024-09-04T17:46:49.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:49.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423669+0000", 2024-09-04T17:46:49.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3800000000000002e-06 2024-09-04T17:46:49.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:49.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423686+0000", 2024-09-04T17:46:49.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7010000000000001e-05 2024-09-04T17:46:49.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:49.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423704+0000", 2024-09-04T17:46:49.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7710000000000002e-05 2024-09-04T17:46:49.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:49.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.455985+0000", 2024-09-04T17:46:49.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032281272999999999 2024-09-04T17:46:49.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:49.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.456196+0000", 2024-09-04T17:46:49.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000210335 2024-09-04T17:46:49.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:49.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.456737+0000", 2024-09-04T17:46:49.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00054109399999999999 2024-09-04T17:46:49.040 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.040 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:49.040 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.040 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6243 3.17 3:ed419000:::rbd_data.1134cfbce83d.000000000000000b:6 [sparse-read 294912~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:49.040 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423721+0000", 2024-09-04T17:46:49.040 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.565953694999997, 2024-09-04T17:46:49.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032666132000000001, 2024-09-04T17:46:49.040 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:49.041 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:49.041 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:49.041 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:49.041 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:49.041 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:49.041 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6243 2024-09-04T17:46:49.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.041 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:49.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:49.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423721+0000", 2024-09-04T17:46:49.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:49.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423721+0000", 2024-09-04T17:46:49.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:49.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423722+0000", 2024-09-04T17:46:49.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0470000000000001e-06 2024-09-04T17:46:49.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:49.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423727+0000", 2024-09-04T17:46:49.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0699999999999997e-06 2024-09-04T17:46:49.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:49.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423744+0000", 2024-09-04T17:46:49.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6659000000000001e-05 2024-09-04T17:46:49.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:49.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423756+0000", 2024-09-04T17:46:49.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2284e-05 2024-09-04T17:46:49.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:49.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.455553+0000", 2024-09-04T17:46:49.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.031797032000000003 2024-09-04T17:46:49.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:49.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.455716+0000", 2024-09-04T17:46:49.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016253700000000001 2024-09-04T17:46:49.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:49.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.456387+0000", 2024-09-04T17:46:49.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00067150300000000005 2024-09-04T17:46:49.045 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.045 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:49.045 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.045 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6244 3.17 3:ed419000:::rbd_data.1134cfbce83d.000000000000000b:6 [read 688128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:49.045 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423775+0000", 2024-09-04T17:46:49.045 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.565899911999999, 2024-09-04T17:46:49.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.033129498, 2024-09-04T17:46:49.045 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:49.045 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:49.045 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:49.045 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:49.046 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:49.046 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:49.046 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6244 2024-09-04T17:46:49.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.046 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:49.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:49.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423775+0000", 2024-09-04T17:46:49.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:49.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423775+0000", 2024-09-04T17:46:49.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:49.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423776+0000", 2024-09-04T17:46:49.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1230000000000001e-06 2024-09-04T17:46:49.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:49.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423782+0000", 2024-09-04T17:46:49.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4929999999999998e-06 2024-09-04T17:46:49.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:49.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423799+0000", 2024-09-04T17:46:49.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7254000000000001e-05 2024-09-04T17:46:49.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:49.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423811+0000", 2024-09-04T17:46:49.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2553e-05 2024-09-04T17:46:49.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:49.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.456421+0000", 2024-09-04T17:46:49.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032609552 2024-09-04T17:46:49.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:49.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.456512+0000", 2024-09-04T17:46:49.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0593999999999994e-05 2024-09-04T17:46:49.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:49.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.456904+0000", 2024-09-04T17:46:49.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00039292899999999998 2024-09-04T17:46:49.049 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.049 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:49.049 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.050 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6245 3.17 3:ed419000:::rbd_data.1134cfbce83d.000000000000000b:6 [read 1048576~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:49.050 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423830+0000", 2024-09-04T17:46:49.050 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.565844906000002, 2024-09-04T17:46:49.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.033577937000000002, 2024-09-04T17:46:49.050 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:49.050 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:49.050 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:49.050 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:49.050 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:49.050 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:49.050 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6245 2024-09-04T17:46:49.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.051 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:49.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:49.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423830+0000", 2024-09-04T17:46:49.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:49.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423830+0000", 2024-09-04T17:46:49.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:49.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423831+0000", 2024-09-04T17:46:49.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.164e-06 2024-09-04T17:46:49.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:49.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423847+0000", 2024-09-04T17:46:49.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5573000000000002e-05 2024-09-04T17:46:49.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:49.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423865+0000", 2024-09-04T17:46:49.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8715999999999998e-05 2024-09-04T17:46:49.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:49.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423880+0000", 2024-09-04T17:46:49.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4107000000000001e-05 2024-09-04T17:46:49.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:49.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.456951+0000", 2024-09-04T17:46:49.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.03307181 2024-09-04T17:46:49.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:49.054 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.457039+0000", 2024-09-04T17:46:49.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7795999999999998e-05 2024-09-04T17:46:49.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:49.054 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.457408+0000", 2024-09-04T17:46:49.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000368771 2024-09-04T17:46:49.054 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.054 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:49.054 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.054 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6246 3.17 3:ed419000:::rbd_data.1134cfbce83d.000000000000000b:6 [read 1409024~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:49.055 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423897+0000", 2024-09-04T17:46:49.055 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.565777566999998, 2024-09-04T17:46:49.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034015888000000001, 2024-09-04T17:46:49.055 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:49.055 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:49.055 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:49.055 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:49.055 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:49.055 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:49.055 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6246 2024-09-04T17:46:49.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.055 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:49.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:49.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423897+0000", 2024-09-04T17:46:49.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:49.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423897+0000", 2024-09-04T17:46:49.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:49.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423899+0000", 2024-09-04T17:46:49.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3200000000000001e-06 2024-09-04T17:46:49.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:49.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423904+0000", 2024-09-04T17:46:49.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4369999999999998e-06 2024-09-04T17:46:49.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:49.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423922+0000", 2024-09-04T17:46:49.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8196999999999999e-05 2024-09-04T17:46:49.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:49.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423935+0000", 2024-09-04T17:46:49.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.314e-05 2024-09-04T17:46:49.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:49.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.457457+0000", 2024-09-04T17:46:49.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.033521663 2024-09-04T17:46:49.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:49.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.457536+0000", 2024-09-04T17:46:49.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8795999999999996e-05 2024-09-04T17:46:49.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:49.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.457913+0000", 2024-09-04T17:46:49.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00037733500000000002 2024-09-04T17:46:49.059 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.059 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:49.059 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.059 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6247 3.17 3:ed419000:::rbd_data.1134cfbce83d.000000000000000b:6 [read 1769472~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:49.059 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423953+0000", 2024-09-04T17:46:49.059 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.565721383000003, 2024-09-04T17:46:49.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0344287, 2024-09-04T17:46:49.059 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:49.060 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:49.060 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:49.060 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:49.060 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:49.060 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:49.060 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6247 2024-09-04T17:46:49.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.060 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:49.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.060 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:49.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423953+0000", 2024-09-04T17:46:49.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:49.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423953+0000", 2024-09-04T17:46:49.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:49.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423955+0000", 2024-09-04T17:46:49.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0529999999999999e-06 2024-09-04T17:46:49.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:49.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423960+0000", 2024-09-04T17:46:49.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0470000000000001e-06 2024-09-04T17:46:49.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:49.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423978+0000", 2024-09-04T17:46:49.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8119999999999999e-05 2024-09-04T17:46:49.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:49.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423991+0000", 2024-09-04T17:46:49.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2813000000000001e-05 2024-09-04T17:46:49.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:49.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.457947+0000", 2024-09-04T17:46:49.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.033956865000000003 2024-09-04T17:46:49.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:49.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.458039+0000", 2024-09-04T17:46:49.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1834000000000002e-05 2024-09-04T17:46:49.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:49.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.458382+0000", 2024-09-04T17:46:49.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00034296799999999997 2024-09-04T17:46:49.063 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.064 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:49.064 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.064 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6305 3.17 3:ee7cd06a:::rbd_data.113431586065.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:49.064 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.455350+0000", 2024-09-04T17:46:49.064 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.534324830999999, 2024-09-04T17:46:49.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029276888000000001, 2024-09-04T17:46:49.064 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:49.064 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:49.064 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:49.064 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:49.064 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:49.065 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:49.065 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6305 2024-09-04T17:46:49.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.065 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:49.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:49.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.455350+0000", 2024-09-04T17:46:49.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:49.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.455350+0000", 2024-09-04T17:46:49.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:49.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.455353+0000", 2024-09-04T17:46:49.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6400000000000001e-06 2024-09-04T17:46:49.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:49.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459745+0000", 2024-09-04T17:46:49.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0043925880000000002 2024-09-04T17:46:49.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:49.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459775+0000", 2024-09-04T17:46:49.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9254000000000001e-05 2024-09-04T17:46:49.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:49.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459789+0000", 2024-09-04T17:46:49.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4731e-05 2024-09-04T17:46:49.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:49.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459885+0000", 2024-09-04T17:46:49.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5502999999999997e-05 2024-09-04T17:46:49.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:49.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.460040+0000", 2024-09-04T17:46:49.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015491499999999999 2024-09-04T17:46:49.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5", 2024-09-04T17:46:49.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.460596+0000", 2024-09-04T17:46:49.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00055657500000000002 2024-09-04T17:46:49.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:49.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484484+0000", 2024-09-04T17:46:49.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.023887950000000002 2024-09-04T17:46:49.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:46:49.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484510+0000", 2024-09-04T17:46:49.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.561e-05 2024-09-04T17:46:49.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:49.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484569+0000", 2024-09-04T17:46:49.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9049999999999999e-05 2024-09-04T17:46:49.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:49.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484627+0000", 2024-09-04T17:46:49.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8072000000000002e-05 2024-09-04T17:46:49.070 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.070 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:49.070 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.070 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6306 3.17 3:ee7cd06a:::rbd_data.113431586065.0000000000000005:head [truncate 1572864] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:49.070 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.459834+0000", 2024-09-04T17:46:49.070 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.529841153, 2024-09-04T17:46:49.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029166788999999999, 2024-09-04T17:46:49.070 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:49.070 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:49.070 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:49.071 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:49.071 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:49.071 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:49.071 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6306 2024-09-04T17:46:49.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.071 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:49.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:49.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459834+0000", 2024-09-04T17:46:49.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:49.072 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459834+0000", 2024-09-04T17:46:49.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:49.072 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459836+0000", 2024-09-04T17:46:49.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.835e-06 2024-09-04T17:46:49.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:49.072 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459841+0000", 2024-09-04T17:46:49.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.728e-06 2024-09-04T17:46:49.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:49.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459851+0000", 2024-09-04T17:46:49.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8570000000000007e-06 2024-09-04T17:46:49.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:49.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459859+0000", 2024-09-04T17:46:49.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4970000000000004e-06 2024-09-04T17:46:49.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:49.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.468789+0000", 2024-09-04T17:46:49.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0089301699999999994 2024-09-04T17:46:49.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:49.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.468890+0000", 2024-09-04T17:46:49.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000100963 2024-09-04T17:46:49.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5", 2024-09-04T17:46:49.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.469090+0000", 2024-09-04T17:46:49.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000199774 2024-09-04T17:46:49.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:49.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484704+0000", 2024-09-04T17:46:49.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015614836999999999 2024-09-04T17:46:49.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:46:49.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.488833+0000", 2024-09-04T17:46:49.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004128357 2024-09-04T17:46:49.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:49.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.488916+0000", 2024-09-04T17:46:49.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2928000000000003e-05 2024-09-04T17:46:49.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:49.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.489001+0000", 2024-09-04T17:46:49.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4843000000000001e-05 2024-09-04T17:46:49.076 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.076 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:49.076 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.076 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6307 3.17 3:eb6e082d:::rbd_data.113431586065.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:49.076 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.459871+0000", 2024-09-04T17:46:49.077 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.529804192999997, 2024-09-04T17:46:49.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032107122000000002, 2024-09-04T17:46:49.077 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:49.077 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:49.077 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:49.077 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:49.077 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:49.077 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:49.077 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6307 2024-09-04T17:46:49.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.077 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:49.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.077 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:49.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459871+0000", 2024-09-04T17:46:49.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:49.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459871+0000", 2024-09-04T17:46:49.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:49.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459871+0000", 2024-09-04T17:46:49.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9699999999999995e-07 2024-09-04T17:46:49.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:49.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.463429+0000", 2024-09-04T17:46:49.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0035577930000000001 2024-09-04T17:46:49.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:49.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.463463+0000", 2024-09-04T17:46:49.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3460000000000002e-05 2024-09-04T17:46:49.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:49.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.463483+0000", 2024-09-04T17:46:49.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0327999999999999e-05 2024-09-04T17:46:49.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:49.080 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.477551+0000", 2024-09-04T17:46:49.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014068259 2024-09-04T17:46:49.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:49.080 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.477748+0000", 2024-09-04T17:46:49.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000197004 2024-09-04T17:46:49.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5", 2024-09-04T17:46:49.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.478346+0000", 2024-09-04T17:46:49.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00059796299999999997 2024-09-04T17:46:49.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:46:49.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.489016+0000", 2024-09-04T17:46:49.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010669781 2024-09-04T17:46:49.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:49.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.491871+0000", 2024-09-04T17:46:49.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028546219999999998 2024-09-04T17:46:49.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:49.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.491931+0000", 2024-09-04T17:46:49.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0736999999999998e-05 2024-09-04T17:46:49.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:49.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.491978+0000", 2024-09-04T17:46:49.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6378000000000002e-05 2024-09-04T17:46:49.082 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.082 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:49.083 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.083 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6308 3.17 3:eb6e082d:::rbd_data.113431586065.0000000000000007:head [truncate 1572864] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:46:49.083 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.463520+0000", 2024-09-04T17:46:49.083 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.526154683999998, 2024-09-04T17:46:49.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029348833000000001, 2024-09-04T17:46:49.083 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:49.083 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:49.083 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:49.083 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:49.083 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:49.083 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:49.083 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6308 2024-09-04T17:46:49.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.084 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:49.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:49.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.463520+0000", 2024-09-04T17:46:49.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:49.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.463520+0000", 2024-09-04T17:46:49.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:49.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.463522+0000", 2024-09-04T17:46:49.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1150000000000001e-06 2024-09-04T17:46:49.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:49.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.463527+0000", 2024-09-04T17:46:49.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5170000000000001e-06 2024-09-04T17:46:49.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:49.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.463555+0000", 2024-09-04T17:46:49.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8594000000000001e-05 2024-09-04T17:46:49.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:49.086 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.463563+0000", 2024-09-04T17:46:49.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4340000000000003e-06 2024-09-04T17:46:49.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:49.086 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484756+0000", 2024-09-04T17:46:49.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021193081999999999 2024-09-04T17:46:49.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:49.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484863+0000", 2024-09-04T17:46:49.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010721999999999999 2024-09-04T17:46:49.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.087 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5", 2024-09-04T17:46:49.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.485067+0000", 2024-09-04T17:46:49.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000204 2024-09-04T17:46:49.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.087 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:46:49.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.489593+0000", 2024-09-04T17:46:49.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045261529999999998 2024-09-04T17:46:49.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:49.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.492787+0000", 2024-09-04T17:46:49.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031940039999999999 2024-09-04T17:46:49.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:49.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.492834+0000", 2024-09-04T17:46:49.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6281e-05 2024-09-04T17:46:49.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:49.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.492869+0000", 2024-09-04T17:46:49.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5432999999999998e-05 2024-09-04T17:46:49.089 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.089 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:49.089 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.089 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:41996 3.6 3:6229bf0a:::rbd_data.113418f5e708.000000000000001f:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-04T17:46:49.089 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:46.113777+0000", 2024-09-04T17:46:49.089 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.875897868, 2024-09-04T17:46:49.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029978818000000001, 2024-09-04T17:46:49.089 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:49.089 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:49.089 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:49.089 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:49.090 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:49.090 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:49.090 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 41996 2024-09-04T17:46:49.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.090 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:49.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:49.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:46.113777+0000", 2024-09-04T17:46:49.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:49.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:46.113777+0000", 2024-09-04T17:46:49.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:49.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:46.113779+0000", 2024-09-04T17:46:49.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0150000000000002e-06 2024-09-04T17:46:49.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:49.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:46.113786+0000", 2024-09-04T17:46:49.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3000000000000004e-06 2024-09-04T17:46:49.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:49.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:46.113803+0000", 2024-09-04T17:46:49.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6711999999999999e-05 2024-09-04T17:46:49.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:49.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:46.113813+0000", 2024-09-04T17:46:49.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7589999999999992e-06 2024-09-04T17:46:49.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:49.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:46.113902+0000", 2024-09-04T17:46:49.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8741999999999998e-05 2024-09-04T17:46:49.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:49.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:46.114007+0000", 2024-09-04T17:46:49.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000105647 2024-09-04T17:46:49.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 1", 2024-09-04T17:46:49.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:46.120514+0000", 2024-09-04T17:46:49.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0065064429999999998 2024-09-04T17:46:49.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:49.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:46.143620+0000", 2024-09-04T17:46:49.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.023106073000000001 2024-09-04T17:46:49.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:46:49.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:46.143643+0000", 2024-09-04T17:46:49.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2954000000000001e-05 2024-09-04T17:46:49.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:49.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:46.143703+0000", 2024-09-04T17:46:49.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0102999999999997e-05 2024-09-04T17:46:49.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:49.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:46.143756+0000", 2024-09-04T17:46:49.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3069999999999998e-05 2024-09-04T17:46:49.095 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.095 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:49.095 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.095 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.095 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:49.095 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:49.096 INFO:teuthology.orchestra.run.smithi092.stdout:2742 read 0x7f42e6b thru 0x7f4da29 (0xabbf bytes) 2024-09-04T17:46:49.096 INFO:teuthology.orchestra.run.smithi092.stdout:2744 writesame 0x7c8dce3 thru 0x7c99d24 data_size 0x27(0xc042 bytes) 2024-09-04T17:46:49.097 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-04T17:46:49.098 INFO:teuthology.orchestra.run.smithi092.stdout:2745 punch from 0x4be2c80 to 0x4be80f6, (0x5476 bytes) 2024-09-04T17:46:49.098 INFO:teuthology.orchestra.run.smithi092.stdout:2746 trunc from 0xdeaaf06 to 0x3ebbbdc 2024-09-04T17:46:49.098 INFO:teuthology.orchestra.run.smithi092.stdout:2747 write 0xb2feb2 thru 0xb39f42 (0xa091 bytes) 2024-09-04T17:46:49.098 INFO:teuthology.orchestra.run.smithi092.stdout:2750 clone 21 order 23 su 131072 sc 2 2024-09-04T17:46:49.098 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:49.097+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:49.197 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:46:49.195+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:46:49.276 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:49.276 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:46:49.276 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:46:49.277 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:49.291 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-04T17:46:49.298 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:49.297+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:49.399 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:49.395+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:49.454 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:49.454 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:46:49.454 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:46:49.454 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:46:49.455 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:49.468 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-04T17:46:49.500 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:49.501+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:49.504 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is unset 2024-09-04T17:46:49.601 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:49.599+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:49.626 INFO:teuthology.orchestra.run.smithi092.stdout:leaving image image_client.0-clone20 intact 2024-09-04T17:46:49.633 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:49.633 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:46:49.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:46:49.633 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:46:49.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.634 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15462 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 835584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:49.634 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.220338+0000", 2024-09-04T17:46:49.634 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 31.411602794, 2024-09-04T17:46:49.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051751278999999997, 2024-09-04T17:46:49.634 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:49.634 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:49.634 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:49.635 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:49.635 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:49.635 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:49.635 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15462 2024-09-04T17:46:49.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.635 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:49.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:49.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.220338+0000", 2024-09-04T17:46:49.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:49.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.220338+0000", 2024-09-04T17:46:49.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:49.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.220600+0000", 2024-09-04T17:46:49.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026198900000000002 2024-09-04T17:46:49.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:49.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.220617+0000", 2024-09-04T17:46:49.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6956000000000002e-05 2024-09-04T17:46:49.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:49.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.220627+0000", 2024-09-04T17:46:49.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0492999999999999e-05 2024-09-04T17:46:49.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:49.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.220635+0000", 2024-09-04T17:46:49.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0059999999999992e-06 2024-09-04T17:46:49.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:49.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.271853+0000", 2024-09-04T17:46:49.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051218020000000003 2024-09-04T17:46:49.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:49.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.271941+0000", 2024-09-04T17:46:49.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7802999999999999e-05 2024-09-04T17:46:49.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:49.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.272089+0000", 2024-09-04T17:46:49.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014801199999999999 2024-09-04T17:46:49.641 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.641 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:49.641 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.641 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15500 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 294912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:49.641 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.228278+0000", 2024-09-04T17:46:49.641 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 31.403662796999999, 2024-09-04T17:46:49.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051770927000000001, 2024-09-04T17:46:49.642 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:49.642 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:49.642 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:49.642 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:49.642 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:49.643 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:49.643 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15500 2024-09-04T17:46:49.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.643 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:49.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:49.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.228278+0000", 2024-09-04T17:46:49.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.644 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:49.644 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.228278+0000", 2024-09-04T17:46:49.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.644 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:49.644 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234002+0000", 2024-09-04T17:46:49.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005724392 2024-09-04T17:46:49.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:49.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234009+0000", 2024-09-04T17:46:49.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1300000000000003e-06 2024-09-04T17:46:49.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:49.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234024+0000", 2024-09-04T17:46:49.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4063000000000001e-05 2024-09-04T17:46:49.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:49.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234035+0000", 2024-09-04T17:46:49.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1124000000000001e-05 2024-09-04T17:46:49.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:49.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.279808+0000", 2024-09-04T17:46:49.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045772992999999998 2024-09-04T17:46:49.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:49.647 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.279894+0000", 2024-09-04T17:46:49.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6174999999999994e-05 2024-09-04T17:46:49.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.647 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:49.647 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.280049+0000", 2024-09-04T17:46:49.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015505 2024-09-04T17:46:49.647 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.647 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:49.647 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.648 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15524 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 1540096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:49.648 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.234831+0000", 2024-09-04T17:46:49.648 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 31.397109920999998, 2024-09-04T17:46:49.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051893631000000003, 2024-09-04T17:46:49.648 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:49.648 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:49.648 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:49.648 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:49.648 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:49.648 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:49.649 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15524 2024-09-04T17:46:49.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.649 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:49.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.649 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:49.649 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234831+0000", 2024-09-04T17:46:49.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:49.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234831+0000", 2024-09-04T17:46:49.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:49.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234831+0000", 2024-09-04T17:46:49.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2699999999999999e-07 2024-09-04T17:46:49.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.652 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:49.652 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234833+0000", 2024-09-04T17:46:49.652 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9810000000000002e-06 2024-09-04T17:46:49.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:49.653 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234839+0000", 2024-09-04T17:46:49.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1519999999999997e-06 2024-09-04T17:46:49.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:49.653 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234845+0000", 2024-09-04T17:46:49.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1109999999999999e-06 2024-09-04T17:46:49.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:49.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.286526+0000", 2024-09-04T17:46:49.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051681373000000003 2024-09-04T17:46:49.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:49.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.286594+0000", 2024-09-04T17:46:49.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8073999999999999e-05 2024-09-04T17:46:49.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:49.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.286724+0000", 2024-09-04T17:46:49.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013041300000000001 2024-09-04T17:46:49.655 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.656 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:49.656 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.656 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15525 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 335872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:49.656 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.234851+0000", 2024-09-04T17:46:49.656 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 31.397089396999998, 2024-09-04T17:46:49.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052146248999999999, 2024-09-04T17:46:49.656 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:49.656 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:49.656 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:49.656 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:49.656 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:49.657 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:49.657 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15525 2024-09-04T17:46:49.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.657 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:49.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.657 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:49.657 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234851+0000", 2024-09-04T17:46:49.657 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.657 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:49.657 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234851+0000", 2024-09-04T17:46:49.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.658 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.658 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.658 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:49.658 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234852+0000", 2024-09-04T17:46:49.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0999999999999999e-07 2024-09-04T17:46:49.658 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.658 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.658 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:49.659 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234854+0000", 2024-09-04T17:46:49.659 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9929999999999998e-06 2024-09-04T17:46:49.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.659 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.659 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:49.659 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234860+0000", 2024-09-04T17:46:49.659 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1489999999999999e-06 2024-09-04T17:46:49.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.659 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.659 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:49.659 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234865+0000", 2024-09-04T17:46:49.659 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8600000000000001e-06 2024-09-04T17:46:49.660 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.660 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:49.660 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.286770+0000", 2024-09-04T17:46:49.660 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051904807999999997 2024-09-04T17:46:49.660 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.660 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:49.660 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.286852+0000", 2024-09-04T17:46:49.660 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2076000000000005e-05 2024-09-04T17:46:49.660 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.660 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:49.661 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.286998+0000", 2024-09-04T17:46:49.661 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014595299999999999 2024-09-04T17:46:49.661 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.661 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:49.661 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.661 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.661 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.661 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15526 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 811008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:49.661 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.234871+0000", 2024-09-04T17:46:49.661 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 31.397069628000001, 2024-09-04T17:46:49.661 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052383817999999999, 2024-09-04T17:46:49.661 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:49.661 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:49.662 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:49.662 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:49.662 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:49.662 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:49.662 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15526 2024-09-04T17:46:49.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.662 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:49.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.662 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:49.662 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234871+0000", 2024-09-04T17:46:49.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:49.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234871+0000", 2024-09-04T17:46:49.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:49.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234872+0000", 2024-09-04T17:46:49.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4299999999999998e-07 2024-09-04T17:46:49.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:49.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234874+0000", 2024-09-04T17:46:49.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2220000000000001e-06 2024-09-04T17:46:49.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:49.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234880+0000", 2024-09-04T17:46:49.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4169999999999997e-06 2024-09-04T17:46:49.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:49.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234886+0000", 2024-09-04T17:46:49.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8150000000000002e-06 2024-09-04T17:46:49.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:49.665 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287040+0000", 2024-09-04T17:46:49.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052153928000000002 2024-09-04T17:46:49.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.665 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:49.665 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287118+0000", 2024-09-04T17:46:49.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8102999999999994e-05 2024-09-04T17:46:49.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.665 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:49.665 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287255+0000", 2024-09-04T17:46:49.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013689 2024-09-04T17:46:49.665 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.666 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:49.666 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.666 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.666 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15527 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 372736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:49.666 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.234906+0000", 2024-09-04T17:46:49.666 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 31.397034312999999, 2024-09-04T17:46:49.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052620742999999998, 2024-09-04T17:46:49.666 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:49.666 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:49.666 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:49.666 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:49.666 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:49.667 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:49.667 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15527 2024-09-04T17:46:49.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.667 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:49.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.667 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:49.667 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234906+0000", 2024-09-04T17:46:49.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.667 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:49.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234906+0000", 2024-09-04T17:46:49.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.668 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:49.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234907+0000", 2024-09-04T17:46:49.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9100000000000004e-07 2024-09-04T17:46:49.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.669 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:49.669 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234910+0000", 2024-09-04T17:46:49.669 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7599999999999998e-06 2024-09-04T17:46:49.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.669 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:49.669 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234917+0000", 2024-09-04T17:46:49.669 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.861e-06 2024-09-04T17:46:49.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.670 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:49.670 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234923+0000", 2024-09-04T17:46:49.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4130000000000003e-06 2024-09-04T17:46:49.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.670 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:49.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287302+0000", 2024-09-04T17:46:49.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052379440999999999 2024-09-04T17:46:49.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:49.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287382+0000", 2024-09-04T17:46:49.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9406999999999994e-05 2024-09-04T17:46:49.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.672 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:49.672 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287527+0000", 2024-09-04T17:46:49.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014537 2024-09-04T17:46:49.672 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.672 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:49.672 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.673 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.673 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15528 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 847872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:49.673 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.234931+0000", 2024-09-04T17:46:49.673 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 31.397009395000001, 2024-09-04T17:46:49.673 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052862919000000001, 2024-09-04T17:46:49.673 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:49.673 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:49.673 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:49.674 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:49.674 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:49.674 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:49.674 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15528 2024-09-04T17:46:49.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.674 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:49.674 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.674 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:49.675 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234931+0000", 2024-09-04T17:46:49.675 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.675 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.675 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:49.675 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234931+0000", 2024-09-04T17:46:49.675 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.676 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:49.676 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234932+0000", 2024-09-04T17:46:49.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6100000000000001e-07 2024-09-04T17:46:49.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.676 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:49.677 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234934+0000", 2024-09-04T17:46:49.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3309999999999998e-06 2024-09-04T17:46:49.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:49.677 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234941+0000", 2024-09-04T17:46:49.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4869999999999999e-06 2024-09-04T17:46:49.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:49.677 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234946+0000", 2024-09-04T17:46:49.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5879999999999997e-06 2024-09-04T17:46:49.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.678 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:49.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287571+0000", 2024-09-04T17:46:49.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052624769000000002 2024-09-04T17:46:49.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.678 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:49.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287651+0000", 2024-09-04T17:46:49.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9975000000000006e-05 2024-09-04T17:46:49.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.679 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:49.679 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287794+0000", 2024-09-04T17:46:49.679 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000143308 2024-09-04T17:46:49.679 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.679 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:49.679 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.679 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15529 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 409600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:49.679 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.234954+0000", 2024-09-04T17:46:49.679 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 31.396986523999999, 2024-09-04T17:46:49.679 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053149889999999998, 2024-09-04T17:46:49.679 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:49.680 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:49.680 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:49.680 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:49.680 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:49.680 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:49.680 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15529 2024-09-04T17:46:49.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.680 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:49.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.680 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:49.680 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234954+0000", 2024-09-04T17:46:49.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:49.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234954+0000", 2024-09-04T17:46:49.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:49.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234955+0000", 2024-09-04T17:46:49.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3799999999999997e-07 2024-09-04T17:46:49.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:49.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234957+0000", 2024-09-04T17:46:49.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3309999999999998e-06 2024-09-04T17:46:49.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:49.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234964+0000", 2024-09-04T17:46:49.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5780000000000004e-06 2024-09-04T17:46:49.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:49.683 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234969+0000", 2024-09-04T17:46:49.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0629999999999996e-06 2024-09-04T17:46:49.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.683 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:49.683 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287839+0000", 2024-09-04T17:46:49.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052870385999999998 2024-09-04T17:46:49.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.683 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:49.684 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287929+0000", 2024-09-04T17:46:49.684 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9514999999999999e-05 2024-09-04T17:46:49.684 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.684 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.684 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:49.684 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288104+0000", 2024-09-04T17:46:49.684 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000175379 2024-09-04T17:46:49.684 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.684 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:49.685 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.685 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15530 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 884736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:49.685 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.234976+0000", 2024-09-04T17:46:49.685 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 31.396965106, 2024-09-04T17:46:49.685 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053414593000000003, 2024-09-04T17:46:49.685 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:49.685 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:49.685 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:49.685 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:49.685 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:49.685 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:49.686 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15530 2024-09-04T17:46:49.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.686 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:49.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:49.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234976+0000", 2024-09-04T17:46:49.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:49.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234976+0000", 2024-09-04T17:46:49.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:49.687 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234976+0000", 2024-09-04T17:46:49.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5299999999999999e-07 2024-09-04T17:46:49.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:49.687 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234978+0000", 2024-09-04T17:46:49.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0329999999999998e-06 2024-09-04T17:46:49.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:49.687 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234984+0000", 2024-09-04T17:46:49.688 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0249999999999999e-06 2024-09-04T17:46:49.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.688 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.688 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:49.688 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234990+0000", 2024-09-04T17:46:49.688 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7579999999999998e-06 2024-09-04T17:46:49.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.688 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.688 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:49.688 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288149+0000", 2024-09-04T17:46:49.689 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053158864 2024-09-04T17:46:49.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.689 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:49.689 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288236+0000", 2024-09-04T17:46:49.689 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7306999999999996e-05 2024-09-04T17:46:49.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.689 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:49.689 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288390+0000", 2024-09-04T17:46:49.689 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000154153 2024-09-04T17:46:49.689 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.689 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:49.690 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.690 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15531 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 446464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:49.690 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.234997+0000", 2024-09-04T17:46:49.690 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 31.396944123000001, 2024-09-04T17:46:49.690 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053721086000000001, 2024-09-04T17:46:49.690 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:49.690 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:49.690 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:49.690 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:49.690 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:49.690 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:49.691 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15531 2024-09-04T17:46:49.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.691 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:49.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.691 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:49.691 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234997+0000", 2024-09-04T17:46:49.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.691 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:49.691 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234997+0000", 2024-09-04T17:46:49.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.692 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.692 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:49.692 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234997+0000", 2024-09-04T17:46:49.692 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8200000000000001e-07 2024-09-04T17:46:49.692 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.692 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.692 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:49.692 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234999+0000", 2024-09-04T17:46:49.692 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.058e-06 2024-09-04T17:46:49.692 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.692 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.692 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:49.692 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235005+0000", 2024-09-04T17:46:49.693 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0700000000000003e-06 2024-09-04T17:46:49.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.693 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:49.693 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235012+0000", 2024-09-04T17:46:49.693 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7020000000000004e-06 2024-09-04T17:46:49.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.693 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:49.693 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288441+0000", 2024-09-04T17:46:49.693 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053429343999999997 2024-09-04T17:46:49.694 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.694 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.694 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:49.694 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288535+0000", 2024-09-04T17:46:49.694 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3406000000000007e-05 2024-09-04T17:46:49.694 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.695 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.695 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:49.695 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288718+0000", 2024-09-04T17:46:49.695 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000183124 2024-09-04T17:46:49.695 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.695 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:49.695 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.695 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.695 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15532 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 921600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:49.695 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235020+0000", 2024-09-04T17:46:49.695 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 31.396920496, 2024-09-04T17:46:49.696 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053966157000000001, 2024-09-04T17:46:49.696 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:49.696 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:49.696 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:49.696 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:49.696 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:49.696 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:49.696 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15532 2024-09-04T17:46:49.696 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.696 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:49.696 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.696 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:49.697 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235020+0000", 2024-09-04T17:46:49.697 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.697 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:49.697 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235020+0000", 2024-09-04T17:46:49.697 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.697 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:49.697 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235021+0000", 2024-09-04T17:46:49.697 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9799999999999999e-07 2024-09-04T17:46:49.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.698 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.698 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:49.698 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235023+0000", 2024-09-04T17:46:49.698 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1560000000000002e-06 2024-09-04T17:46:49.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.698 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.698 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:49.698 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235029+0000", 2024-09-04T17:46:49.698 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0270000000000001e-06 2024-09-04T17:46:49.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.699 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.699 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:49.699 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235034+0000", 2024-09-04T17:46:49.699 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0000000000000004e-06 2024-09-04T17:46:49.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.699 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.699 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:49.699 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288760+0000", 2024-09-04T17:46:49.699 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053726077999999997 2024-09-04T17:46:49.700 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.700 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.700 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:49.700 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288845+0000", 2024-09-04T17:46:49.700 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5242999999999997e-05 2024-09-04T17:46:49.700 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.700 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.700 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:49.700 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288986+0000", 2024-09-04T17:46:49.700 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014125499999999999 2024-09-04T17:46:49.701 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.701 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:49.701 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.701 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.701 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15533 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 483328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:49.701 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235041+0000", 2024-09-04T17:46:49.701 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 31.396899421000001, 2024-09-04T17:46:49.701 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054242994000000003, 2024-09-04T17:46:49.701 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:49.701 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:49.701 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:49.702 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:49.702 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:49.702 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:49.702 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15533 2024-09-04T17:46:49.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.702 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:49.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:49.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235041+0000", 2024-09-04T17:46:49.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:49.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235041+0000", 2024-09-04T17:46:49.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.703 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:49.703 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235042+0000", 2024-09-04T17:46:49.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5499999999999999e-07 2024-09-04T17:46:49.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.703 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:49.703 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235044+0000", 2024-09-04T17:46:49.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2259999999999999e-06 2024-09-04T17:46:49.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.703 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:49.704 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235051+0000", 2024-09-04T17:46:49.704 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4440000000000002e-06 2024-09-04T17:46:49.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.704 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:49.704 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235056+0000", 2024-09-04T17:46:49.704 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8740000000000003e-06 2024-09-04T17:46:49.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.704 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:49.704 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289028+0000", 2024-09-04T17:46:49.704 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053971742000000003 2024-09-04T17:46:49.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.705 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.705 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:49.705 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289108+0000", 2024-09-04T17:46:49.705 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0502999999999998e-05 2024-09-04T17:46:49.705 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.705 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.705 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:49.705 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289284+0000", 2024-09-04T17:46:49.705 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017584999999999999 2024-09-04T17:46:49.705 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.705 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:49.705 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.705 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.706 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.706 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15534 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 958464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:49.706 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235068+0000", 2024-09-04T17:46:49.706 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 31.396872721000001, 2024-09-04T17:46:49.706 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054505452000000003, 2024-09-04T17:46:49.706 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:49.706 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:49.706 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:49.706 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:49.706 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:49.706 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:49.706 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15534 2024-09-04T17:46:49.706 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.706 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:49.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.707 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:49.707 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235068+0000", 2024-09-04T17:46:49.707 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.707 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.707 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:49.707 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235068+0000", 2024-09-04T17:46:49.707 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.707 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.708 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:49.708 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235068+0000", 2024-09-04T17:46:49.708 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7899999999999999e-07 2024-09-04T17:46:49.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.708 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.708 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:49.708 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235071+0000", 2024-09-04T17:46:49.708 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4109999999999999e-06 2024-09-04T17:46:49.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.708 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.708 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:49.708 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235078+0000", 2024-09-04T17:46:49.709 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0489999999999998e-06 2024-09-04T17:46:49.709 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.709 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.709 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:49.709 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235083+0000", 2024-09-04T17:46:49.709 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2249999999999999e-06 2024-09-04T17:46:49.709 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.709 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.709 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:49.709 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289328+0000", 2024-09-04T17:46:49.709 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054244894000000002 2024-09-04T17:46:49.709 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.709 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.710 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:49.710 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289417+0000", 2024-09-04T17:46:49.710 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8988999999999998e-05 2024-09-04T17:46:49.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.710 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.710 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:49.710 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289573+0000", 2024-09-04T17:46:49.710 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000156405 2024-09-04T17:46:49.710 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.710 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:49.710 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.710 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.710 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15535 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 520192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:49.711 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235173+0000", 2024-09-04T17:46:49.711 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 31.396768192, 2024-09-04T17:46:49.711 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054718671000000003, 2024-09-04T17:46:49.711 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:49.711 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:49.711 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:49.711 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:49.711 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:49.711 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:49.711 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15535 2024-09-04T17:46:49.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.711 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:49.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.712 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:49.712 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235173+0000", 2024-09-04T17:46:49.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.712 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:49.712 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235173+0000", 2024-09-04T17:46:49.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.712 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:49.712 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235173+0000", 2024-09-04T17:46:49.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2600000000000002e-07 2024-09-04T17:46:49.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.713 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:49.713 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235177+0000", 2024-09-04T17:46:49.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.625e-06 2024-09-04T17:46:49.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.713 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:49.713 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235188+0000", 2024-09-04T17:46:49.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1464000000000001e-05 2024-09-04T17:46:49.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.714 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.714 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:49.714 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235195+0000", 2024-09-04T17:46:49.714 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5910000000000001e-06 2024-09-04T17:46:49.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.714 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.714 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:49.714 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289623+0000", 2024-09-04T17:46:49.714 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054427595000000002 2024-09-04T17:46:49.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.714 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.714 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:49.715 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289718+0000", 2024-09-04T17:46:49.715 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5958999999999994e-05 2024-09-04T17:46:49.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.715 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:49.715 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289891+0000", 2024-09-04T17:46:49.715 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017271100000000001 2024-09-04T17:46:49.715 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.715 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:49.715 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.715 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15536 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 995328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:49.716 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235206+0000", 2024-09-04T17:46:49.716 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 31.396735051, 2024-09-04T17:46:49.716 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054984065999999998, 2024-09-04T17:46:49.716 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:49.716 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:49.716 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:49.716 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:49.716 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:49.716 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:49.716 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15536 2024-09-04T17:46:49.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.716 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:49.716 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.716 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:49.717 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235206+0000", 2024-09-04T17:46:49.717 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.717 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:49.717 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235206+0000", 2024-09-04T17:46:49.717 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.717 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:49.717 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235206+0000", 2024-09-04T17:46:49.717 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5300000000000004e-07 2024-09-04T17:46:49.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.718 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:49.718 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235209+0000", 2024-09-04T17:46:49.718 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6089999999999999e-06 2024-09-04T17:46:49.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.718 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:49.718 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235216+0000", 2024-09-04T17:46:49.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.277e-06 2024-09-04T17:46:49.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.719 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.719 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:49.719 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235221+0000", 2024-09-04T17:46:49.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2530000000000004e-06 2024-09-04T17:46:49.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.720 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:49.720 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289936+0000", 2024-09-04T17:46:49.720 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054714888000000003 2024-09-04T17:46:49.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.720 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:49.720 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290035+0000", 2024-09-04T17:46:49.721 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8652000000000003e-05 2024-09-04T17:46:49.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.721 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.721 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:49.721 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290190+0000", 2024-09-04T17:46:49.721 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015483399999999999 2024-09-04T17:46:49.721 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.721 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:49.721 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.721 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.721 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15537 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 1323008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:49.722 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235264+0000", 2024-09-04T17:46:49.722 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 31.396676982999999, 2024-09-04T17:46:49.722 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055255669, 2024-09-04T17:46:49.722 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:49.722 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:49.722 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:49.722 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:49.722 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:49.722 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:49.722 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15537 2024-09-04T17:46:49.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.722 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:49.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.722 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:49.723 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235264+0000", 2024-09-04T17:46:49.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:49.723 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235264+0000", 2024-09-04T17:46:49.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:49.723 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235264+0000", 2024-09-04T17:46:49.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4499999999999997e-07 2024-09-04T17:46:49.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.724 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:49.724 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235267+0000", 2024-09-04T17:46:49.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9210000000000001e-06 2024-09-04T17:46:49.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.724 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:49.724 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235276+0000", 2024-09-04T17:46:49.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2309999999999996e-06 2024-09-04T17:46:49.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.724 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:49.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235281+0000", 2024-09-04T17:46:49.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8560000000000003e-06 2024-09-04T17:46:49.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:49.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290246+0000", 2024-09-04T17:46:49.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054964528999999998 2024-09-04T17:46:49.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:49.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290344+0000", 2024-09-04T17:46:49.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8073999999999996e-05 2024-09-04T17:46:49.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.726 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:49.726 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290519+0000", 2024-09-04T17:46:49.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000175413 2024-09-04T17:46:49.726 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.726 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:49.726 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.726 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15538 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 1032192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:49.726 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235290+0000", 2024-09-04T17:46:49.726 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 31.396651074000001, 2024-09-04T17:46:49.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055518932, 2024-09-04T17:46:49.727 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:49.727 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:49.727 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:49.727 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:49.727 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:49.727 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:49.727 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15538 2024-09-04T17:46:49.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.727 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:49.727 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.727 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:49.727 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235290+0000", 2024-09-04T17:46:49.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:49.728 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235290+0000", 2024-09-04T17:46:49.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:49.728 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235290+0000", 2024-09-04T17:46:49.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3399999999999999e-07 2024-09-04T17:46:49.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:49.729 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235292+0000", 2024-09-04T17:46:49.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2639999999999998e-06 2024-09-04T17:46:49.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.729 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:49.729 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235299+0000", 2024-09-04T17:46:49.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2029999999999997e-06 2024-09-04T17:46:49.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.729 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:49.729 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235305+0000", 2024-09-04T17:46:49.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7469999999999999e-06 2024-09-04T17:46:49.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.730 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:49.730 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290562+0000", 2024-09-04T17:46:49.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055256580999999999 2024-09-04T17:46:49.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.730 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:49.730 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290651+0000", 2024-09-04T17:46:49.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8953e-05 2024-09-04T17:46:49.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.730 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:49.730 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290809+0000", 2024-09-04T17:46:49.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015765000000000001 2024-09-04T17:46:49.731 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.731 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:49.731 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.731 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15539 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 1359872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:49.731 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235312+0000", 2024-09-04T17:46:49.731 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 31.396628333999999, 2024-09-04T17:46:49.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055809865, 2024-09-04T17:46:49.731 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:49.732 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:49.732 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:49.732 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:49.732 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:49.732 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:49.732 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15539 2024-09-04T17:46:49.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.732 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:49.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:49.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235312+0000", 2024-09-04T17:46:49.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:49.733 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235312+0000", 2024-09-04T17:46:49.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:49.733 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235313+0000", 2024-09-04T17:46:49.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9000000000000002e-07 2024-09-04T17:46:49.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:49.733 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235315+0000", 2024-09-04T17:46:49.734 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1270000000000001e-06 2024-09-04T17:46:49.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.734 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:49.734 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235320+0000", 2024-09-04T17:46:49.734 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5430000000000001e-06 2024-09-04T17:46:49.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.734 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:49.734 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235326+0000", 2024-09-04T17:46:49.734 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2039999999999996e-06 2024-09-04T17:46:49.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.735 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:49.735 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290858+0000", 2024-09-04T17:46:49.735 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055532440000000002 2024-09-04T17:46:49.735 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.735 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.735 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:49.735 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290953+0000", 2024-09-04T17:46:49.735 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5031000000000005e-05 2024-09-04T17:46:49.735 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.735 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.735 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:49.735 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.291122+0000", 2024-09-04T17:46:49.735 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016913 2024-09-04T17:46:49.735 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.736 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:49.736 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.736 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15540 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 1396736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:49.736 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235333+0000", 2024-09-04T17:46:49.736 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 31.396607827, 2024-09-04T17:46:49.736 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056080528999999997, 2024-09-04T17:46:49.736 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:49.736 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:49.736 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:49.737 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:49.737 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:49.737 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:49.737 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15540 2024-09-04T17:46:49.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.737 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:49.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:49.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235333+0000", 2024-09-04T17:46:49.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:49.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235333+0000", 2024-09-04T17:46:49.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:49.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235333+0000", 2024-09-04T17:46:49.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0999999999999999e-07 2024-09-04T17:46:49.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:49.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235336+0000", 2024-09-04T17:46:49.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.278e-06 2024-09-04T17:46:49.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.739 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:49.739 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235341+0000", 2024-09-04T17:46:49.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8359999999999997e-06 2024-09-04T17:46:49.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.739 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.739 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:49.739 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235346+0000", 2024-09-04T17:46:49.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4329999999999997e-06 2024-09-04T17:46:49.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.739 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.739 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:49.739 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.291167+0000", 2024-09-04T17:46:49.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055821603999999997 2024-09-04T17:46:49.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.740 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.740 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:49.740 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.291257+0000", 2024-09-04T17:46:49.740 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9349000000000001e-05 2024-09-04T17:46:49.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.740 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.740 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:49.740 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.291413+0000", 2024-09-04T17:46:49.740 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015661900000000001 2024-09-04T17:46:49.740 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.740 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:49.740 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.741 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15541 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 1433600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:49.741 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235355+0000", 2024-09-04T17:46:49.741 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 31.396585639000001, 2024-09-04T17:46:49.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056371783000000002, 2024-09-04T17:46:49.741 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:49.741 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:49.741 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:49.741 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:49.741 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:49.741 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:49.742 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15541 2024-09-04T17:46:49.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.742 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:49.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:49.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235355+0000", 2024-09-04T17:46:49.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:49.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235355+0000", 2024-09-04T17:46:49.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:49.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.743 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:49.743 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235356+0000", 2024-09-04T17:46:49.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3099999999999998e-07 2024-09-04T17:46:49.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.743 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:49.743 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235358+0000", 2024-09-04T17:46:49.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3599999999999999e-06 2024-09-04T17:46:49.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.743 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:49.743 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235364+0000", 2024-09-04T17:46:49.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5760000000000002e-06 2024-09-04T17:46:49.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.744 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:49.744 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235370+0000", 2024-09-04T17:46:49.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0819999999999998e-06 2024-09-04T17:46:49.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.744 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:49.744 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.291462+0000", 2024-09-04T17:46:49.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056092301999999997 2024-09-04T17:46:49.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.744 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:49.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.291559+0000", 2024-09-04T17:46:49.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6988e-05 2024-09-04T17:46:49.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:49.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:49.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:49.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.291727+0000", 2024-09-04T17:46:49.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016804400000000001 2024-09-04T17:46:49.745 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.745 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:49.745 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.745 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:49.745 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:49.745 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:49.747 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-04T17:46:49.747 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:49.745+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:49.794 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #19, image image_client.0-clone19 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent20 2024-09-04T17:46:49.847 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:49.843+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:49.894 INFO:teuthology.orchestra.run.smithi092.stdout:2751 trunc from 0x3ebbbdc to 0x4174171 2024-09-04T17:46:49.915 INFO:teuthology.orchestra.run.smithi092.stdout:2753 read 0x1cfeaee thru 0x1d0e185 (0xf698 bytes) 2024-09-04T17:46:49.916 INFO:teuthology.orchestra.run.smithi092.stdout:2754 writesame 0x3ae376b thru 0x3af1f37 data_size 0x1(0xe7cd bytes) 2024-09-04T17:46:49.916 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:49.916 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:46:49.916 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:46:49.916 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:49.931 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-04T17:46:49.948 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:49.949+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:49.974 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:49.974 INFO:teuthology.orchestra.run.smithi092.stdout:2757 read 0xba6a36 thru 0xba9caf (0x327a bytes) 2024-09-04T17:46:49.974 INFO:teuthology.orchestra.run.smithi092.stdout:2758 write 0x3100c6b thru 0x31059a9 (0x4d3f bytes) 2024-09-04T17:46:49.995 INFO:teuthology.orchestra.run.smithi092.stdout:2759 punch from 0x48959a to 0x48c283, (0x2ce9 bytes) 2024-09-04T17:46:49.996 INFO:teuthology.orchestra.run.smithi092.stdout:2760 read 0x1f8688f thru 0x1f8bd3f (0x54b1 bytes) 2024-09-04T17:46:50.011 INFO:teuthology.orchestra.run.smithi092.stdout:2761 writesame 0x57fc2b0 thru 0x5804559 data_size 0x32(0x82aa bytes) 2024-09-04T17:46:50.011 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:50.011 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:50.011 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:50.015 INFO:teuthology.orchestra.run.smithi092.stdout:2767 trunc from 0x580455a to 0x3491766 2024-09-04T17:46:50.032 INFO:teuthology.orchestra.run.smithi092.stdout:2768 punch from 0x1fe146f to 0x1fed710, (0xc2a1 bytes) 2024-09-04T17:46:50.049 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:50.049+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:50.060 INFO:teuthology.orchestra.run.smithi092.stdout:2769 write 0xba1e506 thru 0xba2450e (0x6009 bytes) 2024-09-04T17:46:50.067 INFO:teuthology.orchestra.run.smithi092.stdout:2770 writesame 0x40db0cb thru 0x40db96d data_size 0xb(0x8a3 bytes) 2024-09-04T17:46:50.071 INFO:teuthology.orchestra.run.smithi092.stdout:2771 write 0x3c20dad thru 0x3c2bcc0 (0xaf14 bytes) 2024-09-04T17:46:50.074 INFO:teuthology.orchestra.run.smithi092.stdout:2772 write 0x21d9670 thru 0x21e46a9 (0xb03a bytes) 2024-09-04T17:46:50.117 INFO:teuthology.orchestra.run.smithi092.stdout:2774 read 0x6a9cc0e thru 0x6aa2b49 (0x5f3c bytes) 2024-09-04T17:46:50.117 INFO:teuthology.orchestra.run.smithi092.stdout:2775 punch from 0x3c71cb2 to 0x3c7fdfd, (0xe14b bytes) 2024-09-04T17:46:50.117 INFO:teuthology.orchestra.run.smithi092.stdout:2776 read 0x1ff7280 thru 0x1fff370 (0x80f1 bytes) 2024-09-04T17:46:50.118 INFO:teuthology.orchestra.run.smithi092.stdout:2777 writesame 0x16f27ca thru 0x16f4265 data_size 0x1a(0x1a9c bytes) 2024-09-04T17:46:50.125 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:50.125 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:46:50.126 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:46:50.126 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:46:50.126 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:50.138 INFO:teuthology.orchestra.run.smithi092.stdout:2778 punch from 0x4def59e to 0x4dfaa47, (0xb4a9 bytes) 2024-09-04T17:46:50.138 INFO:teuthology.orchestra.run.smithi092.stdout:2780 writesame 0x5fa4443 thru 0x5fb09ea data_size 0x19(0xc5a8 bytes) 2024-09-04T17:46:50.143 INFO:teuthology.orchestra.run.smithi092.stdout:2781 punch from 0x51be753 to 0x51c9c80, (0xb52d bytes) 2024-09-04T17:46:50.143 INFO:teuthology.orchestra.run.smithi092.stdout:2782 punch from 0x5cc87b8 to 0x5cd0672, (0x7eba bytes) 2024-09-04T17:46:50.143 INFO:teuthology.orchestra.run.smithi092.stdout:2784 read 0x81e8349 thru 0x81f6a26 (0xe6de bytes) 2024-09-04T17:46:50.144 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-04T17:46:50.147 INFO:teuthology.orchestra.run.smithi092.stdout:2785 trunc from 0xba2450f to 0x967388a 2024-09-04T17:46:50.150 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:50.149+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:50.168 INFO:teuthology.orchestra.run.smithi092.stdout:2787 trunc from 0x967388a to 0x617df05 2024-09-04T17:46:50.178 INFO:teuthology.orchestra.run.smithi092.stdout:2793 write 0x5f72225 thru 0x5f7fffe (0xddda bytes) 2024-09-04T17:46:50.181 INFO:teuthology.orchestra.run.smithi092.stdout:2794 write 0x42a8861 thru 0x42b49b7 (0xc157 bytes) 2024-09-04T17:46:50.207 INFO:teuthology.orchestra.run.smithi092.stdout:2795 write 0xbb6ab63 thru 0xbb78a44 (0xdee2 bytes) 2024-09-04T17:46:50.211 INFO:teuthology.orchestra.run.smithi092.stdout:2797 read 0xa483914 thru 0xa484833 (0xf20 bytes) 2024-09-04T17:46:50.229 INFO:teuthology.orchestra.run.smithi092.stdout:2798 trunc from 0xbb78a45 to 0x977be2 2024-09-04T17:46:50.251 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:50.247+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:50.282 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:50.282 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:46:50.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:46:50.282 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:46:50.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.282 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:4264 3.9 e30/24 3:970e51a0:::rbd_data.1134cfbce83d.0000000000000009:head v 30'83, mlcod=30'81)", 2024-09-04T17:46:50.282 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:07.027227+0000", 2024-09-04T17:46:50.282 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 43.254761352000003, 2024-09-04T17:46:50.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0096959590000000005, 2024-09-04T17:46:50.282 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:50.282 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:50.283 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:50.283 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:50.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:50.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.027227+0000", 2024-09-04T17:46:50.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:50.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:50.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.027227+0000", 2024-09-04T17:46:50.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:50.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:50.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.027229+0000", 2024-09-04T17:46:50.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4839999999999998e-06 2024-09-04T17:46:50.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:50.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.031984+0000", 2024-09-04T17:46:50.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047544789999999998 2024-09-04T17:46:50.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:50.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.032019+0000", 2024-09-04T17:46:50.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5253999999999998e-05 2024-09-04T17:46:50.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:50.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.032036+0000", 2024-09-04T17:46:50.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6713000000000001e-05 2024-09-04T17:46:50.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:50.286 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.032129+0000", 2024-09-04T17:46:50.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2961000000000007e-05 2024-09-04T17:46:50.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:50.286 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.032154+0000", 2024-09-04T17:46:50.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5148999999999999e-05 2024-09-04T17:46:50.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:50.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.036778+0000", 2024-09-04T17:46:50.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046239159999999996 2024-09-04T17:46:50.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:50.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.036923+0000", 2024-09-04T17:46:50.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014500299999999999 2024-09-04T17:46:50.287 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:50.287 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:50.287 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:50.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.287 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:4281 3.4 e30/24 3:215ebead:::rbd_header.1134cfbce83d:head v 30'24, mlcod=30'22)", 2024-09-04T17:46:50.288 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:07.073105+0000", 2024-09-04T17:46:50.288 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 43.208882588000002, 2024-09-04T17:46:50.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0032176499999999999, 2024-09-04T17:46:50.288 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:50.288 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:50.288 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:50.288 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:50.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:50.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.073105+0000", 2024-09-04T17:46:50.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:50.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:50.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.073105+0000", 2024-09-04T17:46:50.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:50.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:50.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.073107+0000", 2024-09-04T17:46:50.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1260000000000001e-06 2024-09-04T17:46:50.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:50.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.073115+0000", 2024-09-04T17:46:50.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0430000000000008e-06 2024-09-04T17:46:50.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:50.290 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.073133+0000", 2024-09-04T17:46:50.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8308000000000002e-05 2024-09-04T17:46:50.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:50.290 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.073144+0000", 2024-09-04T17:46:50.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1219999999999999e-05 2024-09-04T17:46:50.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:50.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.073218+0000", 2024-09-04T17:46:50.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3464e-05 2024-09-04T17:46:50.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:50.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.073244+0000", 2024-09-04T17:46:50.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6308e-05 2024-09-04T17:46:50.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:50.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.076286+0000", 2024-09-04T17:46:50.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00304163 2024-09-04T17:46:50.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:50.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.076323+0000", 2024-09-04T17:46:50.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7551e-05 2024-09-04T17:46:50.292 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:50.292 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:50.292 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:50.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.293 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:4399 3.9 e31/24 3:970e51a0:::rbd_data.1134cfbce83d.0000000000000009:head v 31'87, mlcod=30'84)", 2024-09-04T17:46:50.293 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:07.343236+0000", 2024-09-04T17:46:50.293 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.938751668000002, 2024-09-04T17:46:50.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0036134539999999999, 2024-09-04T17:46:50.293 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:50.293 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:50.293 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:50.293 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:50.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:50.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.343236+0000", 2024-09-04T17:46:50.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:50.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:50.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.343236+0000", 2024-09-04T17:46:50.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:50.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:50.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.343237+0000", 2024-09-04T17:46:50.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4000000000000001e-07 2024-09-04T17:46:50.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:50.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.343250+0000", 2024-09-04T17:46:50.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2573e-05 2024-09-04T17:46:50.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:50.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.343279+0000", 2024-09-04T17:46:50.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9263000000000001e-05 2024-09-04T17:46:50.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:50.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.343299+0000", 2024-09-04T17:46:50.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9720000000000001e-05 2024-09-04T17:46:50.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.296 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:50.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.343389+0000", 2024-09-04T17:46:50.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9924000000000002e-05 2024-09-04T17:46:50.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.296 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:50.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.343413+0000", 2024-09-04T17:46:50.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4227999999999999e-05 2024-09-04T17:46:50.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:50.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.346796+0000", 2024-09-04T17:46:50.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0033831310000000002 2024-09-04T17:46:50.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:50.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.346850+0000", 2024-09-04T17:46:50.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3875000000000002e-05 2024-09-04T17:46:50.297 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:50.297 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:50.298 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:50.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.298 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:4881 3.4 e31/24 3:25597cfa:::rbd_data.1134cfbce83d.0000000000000006:head v 31'28, mlcod=31'26)", 2024-09-04T17:46:50.298 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:07.381336+0000", 2024-09-04T17:46:50.298 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.900652039000001, 2024-09-04T17:46:50.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013330164, 2024-09-04T17:46:50.298 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:50.298 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:50.298 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:50.298 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:50.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:50.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.381336+0000", 2024-09-04T17:46:50.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:50.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:50.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.381336+0000", 2024-09-04T17:46:50.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:50.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:50.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.381339+0000", 2024-09-04T17:46:50.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.287e-06 2024-09-04T17:46:50.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.300 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:50.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.384162+0000", 2024-09-04T17:46:50.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028228649999999999 2024-09-04T17:46:50.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.300 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:50.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.384209+0000", 2024-09-04T17:46:50.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6768999999999999e-05 2024-09-04T17:46:50.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.300 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:50.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.384225+0000", 2024-09-04T17:46:50.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5987000000000001e-05 2024-09-04T17:46:50.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:50.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.384372+0000", 2024-09-04T17:46:50.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000146809 2024-09-04T17:46:50.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:50.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.384409+0000", 2024-09-04T17:46:50.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6909999999999997e-05 2024-09-04T17:46:50.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:50.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.394535+0000", 2024-09-04T17:46:50.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010126575000000001 2024-09-04T17:46:50.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:50.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.394666+0000", 2024-09-04T17:46:50.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013096200000000001 2024-09-04T17:46:50.302 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:50.302 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:50.302 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:50.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.302 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:4884 3.4 e31/24 3:25597cfa:::rbd_data.1134cfbce83d.0000000000000006:head v 31'30, mlcod=31'26)", 2024-09-04T17:46:50.302 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:07.386970+0000", 2024-09-04T17:46:50.303 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.895018104999998, 2024-09-04T17:46:50.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0088306779999999998, 2024-09-04T17:46:50.303 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:50.303 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:50.303 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:50.303 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:50.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.303 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:50.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.386970+0000", 2024-09-04T17:46:50.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:50.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.303 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:50.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.386970+0000", 2024-09-04T17:46:50.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:50.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:50.304 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.386973+0000", 2024-09-04T17:46:50.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9229999999999998e-06 2024-09-04T17:46:50.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:50.304 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.386993+0000", 2024-09-04T17:46:50.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.054e-05 2024-09-04T17:46:50.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:50.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.387029+0000", 2024-09-04T17:46:50.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5753000000000001e-05 2024-09-04T17:46:50.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:50.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.387051+0000", 2024-09-04T17:46:50.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1552999999999998e-05 2024-09-04T17:46:50.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:50.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.388844+0000", 2024-09-04T17:46:50.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001793712 2024-09-04T17:46:50.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:50.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.388866+0000", 2024-09-04T17:46:50.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1123000000000001e-05 2024-09-04T17:46:50.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:50.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.395711+0000", 2024-09-04T17:46:50.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0068458859999999998 2024-09-04T17:46:50.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:50.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.395801+0000", 2024-09-04T17:46:50.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9188000000000001e-05 2024-09-04T17:46:50.307 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:50.307 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:50.307 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:50.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.307 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:4997 3.9 e31/24 3:940f1d52:::rbd_data.1134cfbce83d.0000000000000010:head v 31'89, mlcod=30'85)", 2024-09-04T17:46:50.307 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:07.400336+0000", 2024-09-04T17:46:50.307 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.881652453000001, 2024-09-04T17:46:50.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003411533, 2024-09-04T17:46:50.307 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:50.308 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:50.308 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:50.308 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:50.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.308 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:50.308 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.400336+0000", 2024-09-04T17:46:50.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:50.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.308 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:50.308 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.400336+0000", 2024-09-04T17:46:50.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:50.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.309 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:50.309 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.400338+0000", 2024-09-04T17:46:50.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3489999999999999e-06 2024-09-04T17:46:50.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.309 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:50.309 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.400353+0000", 2024-09-04T17:46:50.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5091e-05 2024-09-04T17:46:50.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:50.310 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.400379+0000", 2024-09-04T17:46:50.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5528999999999999e-05 2024-09-04T17:46:50.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:50.310 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.400407+0000", 2024-09-04T17:46:50.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8104000000000001e-05 2024-09-04T17:46:50.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:50.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.400502+0000", 2024-09-04T17:46:50.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5402999999999994e-05 2024-09-04T17:46:50.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:50.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.400519+0000", 2024-09-04T17:46:50.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7329e-05 2024-09-04T17:46:50.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.312 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:50.312 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.403698+0000", 2024-09-04T17:46:50.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031787270000000001 2024-09-04T17:46:50.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.312 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:50.312 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.403747+0000", 2024-09-04T17:46:50.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9001000000000001e-05 2024-09-04T17:46:50.312 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:50.313 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:50.313 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:50.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.313 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:5000 3.9 e31/24 3:94906c79:::rbd_data.1134cfbce83d.0000000000000012:head v 31'91, mlcod=30'85)", 2024-09-04T17:46:50.313 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:07.402507+0000", 2024-09-04T17:46:50.313 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.879480725000001, 2024-09-04T17:46:50.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004039735, 2024-09-04T17:46:50.313 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:50.313 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:50.314 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:50.314 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:50.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.314 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:50.314 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.402507+0000", 2024-09-04T17:46:50.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:50.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.314 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:50.314 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.402507+0000", 2024-09-04T17:46:50.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:50.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.315 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:50.315 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.402509+0000", 2024-09-04T17:46:50.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5939999999999999e-06 2024-09-04T17:46:50.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.315 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:50.315 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.402521+0000", 2024-09-04T17:46:50.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1786e-05 2024-09-04T17:46:50.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.315 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:50.315 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.402543+0000", 2024-09-04T17:46:50.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2158e-05 2024-09-04T17:46:50.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:50.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.402555+0000", 2024-09-04T17:46:50.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2034e-05 2024-09-04T17:46:50.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:50.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.402660+0000", 2024-09-04T17:46:50.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010535900000000001 2024-09-04T17:46:50.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:50.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.402696+0000", 2024-09-04T17:46:50.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5488000000000003e-05 2024-09-04T17:46:50.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:50.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.406510+0000", 2024-09-04T17:46:50.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0038142390000000001 2024-09-04T17:46:50.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:50.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.406547+0000", 2024-09-04T17:46:50.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7076999999999998e-05 2024-09-04T17:46:50.318 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:50.318 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:50.318 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:50.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.318 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:6311 3.9 e32/24 3:941d6897:::rbd_data.113431586065.0000000000000008:head v 32'97, mlcod=32'95)", 2024-09-04T17:46:50.318 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.479986+0000", 2024-09-04T17:46:50.318 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 41.802001828000002, 2024-09-04T17:46:50.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018493539, 2024-09-04T17:46:50.318 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:50.318 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:50.318 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:50.318 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:50.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:50.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.479986+0000", 2024-09-04T17:46:50.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:50.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:50.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.479986+0000", 2024-09-04T17:46:50.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:50.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:50.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.479988+0000", 2024-09-04T17:46:50.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.193e-06 2024-09-04T17:46:50.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:50.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484109+0000", 2024-09-04T17:46:50.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041201650000000003 2024-09-04T17:46:50.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:50.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484139+0000", 2024-09-04T17:46:50.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.082e-05 2024-09-04T17:46:50.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:50.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484153+0000", 2024-09-04T17:46:50.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3456e-05 2024-09-04T17:46:50.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:50.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484250+0000", 2024-09-04T17:46:50.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7638000000000003e-05 2024-09-04T17:46:50.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:50.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484282+0000", 2024-09-04T17:46:50.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1511999999999999e-05 2024-09-04T17:46:50.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:50.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.498411+0000", 2024-09-04T17:46:50.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014128870999999999 2024-09-04T17:46:50.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:50.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.498480+0000", 2024-09-04T17:46:50.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8884e-05 2024-09-04T17:46:50.322 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:50.323 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:50.323 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:50.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.323 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:6312 3.9 e32/24 3:941d6897:::rbd_data.113431586065.0000000000000008:head v 32'99, mlcod=32'95)", 2024-09-04T17:46:50.323 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.486036+0000", 2024-09-04T17:46:50.323 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 41.79595175, 2024-09-04T17:46:50.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013432773, 2024-09-04T17:46:50.323 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:50.324 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:50.324 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:50.324 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:50.324 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.324 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:50.324 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.486036+0000", 2024-09-04T17:46:50.324 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:50.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.324 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.324 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:50.324 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.486036+0000", 2024-09-04T17:46:50.324 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:50.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:50.325 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.486038+0000", 2024-09-04T17:46:50.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.536e-06 2024-09-04T17:46:50.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:50.325 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.486062+0000", 2024-09-04T17:46:50.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3672e-05 2024-09-04T17:46:50.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:50.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.486078+0000", 2024-09-04T17:46:50.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.613e-05 2024-09-04T17:46:50.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:50.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.486086+0000", 2024-09-04T17:46:50.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8849999999999999e-06 2024-09-04T17:46:50.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:50.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.491843+0000", 2024-09-04T17:46:50.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057572040000000001 2024-09-04T17:46:50.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:50.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.491876+0000", 2024-09-04T17:46:50.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3515e-05 2024-09-04T17:46:50.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:50.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.499410+0000", 2024-09-04T17:46:50.328 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0075333780000000003 2024-09-04T17:46:50.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.328 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.328 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:50.328 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.499469+0000", 2024-09-04T17:46:50.328 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9453000000000002e-05 2024-09-04T17:46:50.328 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:50.328 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:50.328 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:50.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.328 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.329 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:12747 3.1f e41/37 3:fb91fe3f:::rbd_object_map.1134ef3a5a7d:head v 41'125, mlcod=41'122)", 2024-09-04T17:46:50.329 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:17.330322+0000", 2024-09-04T17:46:50.329 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.951666299000003, 2024-09-04T17:46:50.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004205762, 2024-09-04T17:46:50.329 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:50.329 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:50.329 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:50.329 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:50.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.329 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:50.329 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.330322+0000", 2024-09-04T17:46:50.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:50.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.330 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:50.330 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.330322+0000", 2024-09-04T17:46:50.330 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:50.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.330 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:50.330 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.330323+0000", 2024-09-04T17:46:50.330 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9500000000000001e-07 2024-09-04T17:46:50.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.330 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:50.330 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.330329+0000", 2024-09-04T17:46:50.330 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2040000000000002e-06 2024-09-04T17:46:50.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.331 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:50.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.330341+0000", 2024-09-04T17:46:50.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1727e-05 2024-09-04T17:46:50.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.331 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:50.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.330348+0000", 2024-09-04T17:46:50.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.187e-06 2024-09-04T17:46:50.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.331 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:50.332 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.330455+0000", 2024-09-04T17:46:50.332 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107311 2024-09-04T17:46:50.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.332 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.332 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:50.332 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.330495+0000", 2024-09-04T17:46:50.332 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0191999999999997e-05 2024-09-04T17:46:50.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.332 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.332 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:50.332 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.334480+0000", 2024-09-04T17:46:50.332 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039848410000000003 2024-09-04T17:46:50.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.333 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.333 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:50.333 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.334528+0000", 2024-09-04T17:46:50.333 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7404999999999998e-05 2024-09-04T17:46:50.333 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:50.333 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:50.333 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:50.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.335 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:12748 3.1f e41/37 3:fb91fe3f:::rbd_object_map.1134ef3a5a7d:head v 41'126, mlcod=41'122)", 2024-09-04T17:46:50.335 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:17.332613+0000", 2024-09-04T17:46:50.335 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.949375320000001, 2024-09-04T17:46:50.335 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0044275870000000002, 2024-09-04T17:46:50.335 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:50.335 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:50.335 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:50.335 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:50.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.335 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:50.335 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.332613+0000", 2024-09-04T17:46:50.336 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:50.336 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.336 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.336 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:50.336 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.332613+0000", 2024-09-04T17:46:50.336 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:50.336 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.336 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.336 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:50.336 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.332614+0000", 2024-09-04T17:46:50.336 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.051e-06 2024-09-04T17:46:50.336 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.336 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.337 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:50.337 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.332620+0000", 2024-09-04T17:46:50.337 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1680000000000001e-06 2024-09-04T17:46:50.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.337 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:50.337 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.332635+0000", 2024-09-04T17:46:50.337 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5543e-05 2024-09-04T17:46:50.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.337 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:50.337 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.332643+0000", 2024-09-04T17:46:50.337 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.537e-06 2024-09-04T17:46:50.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.338 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:50.338 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.332717+0000", 2024-09-04T17:46:50.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3701999999999993e-05 2024-09-04T17:46:50.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.338 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:50.338 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.332786+0000", 2024-09-04T17:46:50.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9669e-05 2024-09-04T17:46:50.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.338 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:50.338 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.336923+0000", 2024-09-04T17:46:50.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041365259999999997 2024-09-04T17:46:50.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.339 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:50.339 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.337040+0000", 2024-09-04T17:46:50.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000117391 2024-09-04T17:46:50.339 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:50.339 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:50.339 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:50.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.339 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:12749 3.1f e41/37 3:fb91fe3f:::rbd_object_map.1134ef3a5a7d:head v 41'127, mlcod=41'122)", 2024-09-04T17:46:50.339 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:17.335529+0000", 2024-09-04T17:46:50.340 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.946459279000003, 2024-09-04T17:46:50.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052795899999999998, 2024-09-04T17:46:50.340 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:50.340 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:50.340 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:50.340 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:50.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.340 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:50.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.335529+0000", 2024-09-04T17:46:50.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:50.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.340 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:50.341 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.335529+0000", 2024-09-04T17:46:50.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:50.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.341 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:50.341 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.335535+0000", 2024-09-04T17:46:50.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4060000000000002e-06 2024-09-04T17:46:50.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.341 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:50.341 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.335546+0000", 2024-09-04T17:46:50.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0652000000000001e-05 2024-09-04T17:46:50.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.342 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:50.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.335571+0000", 2024-09-04T17:46:50.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5148e-05 2024-09-04T17:46:50.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.342 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:50.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.335583+0000", 2024-09-04T17:46:50.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2315e-05 2024-09-04T17:46:50.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.342 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:50.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.335699+0000", 2024-09-04T17:46:50.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000115443 2024-09-04T17:46:50.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.343 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:50.343 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.335736+0000", 2024-09-04T17:46:50.343 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6779999999999997e-05 2024-09-04T17:46:50.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.343 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:50.343 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.340722+0000", 2024-09-04T17:46:50.343 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0049860390000000003 2024-09-04T17:46:50.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.343 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:50.343 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.340808+0000", 2024-09-04T17:46:50.343 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6806999999999997e-05 2024-09-04T17:46:50.344 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:50.344 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:50.344 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:50.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.344 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:12750 3.1f e41/37 3:fb91fe3f:::rbd_object_map.1134ef3a5a7d:head v 41'128, mlcod=41'123)", 2024-09-04T17:46:50.344 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:17.337581+0000", 2024-09-04T17:46:50.344 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.944406735000001, 2024-09-04T17:46:50.344 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045169579999999997, 2024-09-04T17:46:50.344 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:50.344 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:50.344 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:50.344 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:50.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.345 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:50.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.337581+0000", 2024-09-04T17:46:50.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:50.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.345 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:50.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.337581+0000", 2024-09-04T17:46:50.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:50.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.345 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:50.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.337583+0000", 2024-09-04T17:46:50.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5200000000000001e-06 2024-09-04T17:46:50.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:50.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.337595+0000", 2024-09-04T17:46:50.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2395000000000001e-05 2024-09-04T17:46:50.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:50.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.337620+0000", 2024-09-04T17:46:50.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4794e-05 2024-09-04T17:46:50.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:50.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.337634+0000", 2024-09-04T17:46:50.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4241e-05 2024-09-04T17:46:50.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:50.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.337731+0000", 2024-09-04T17:46:50.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6246e-05 2024-09-04T17:46:50.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:50.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.337762+0000", 2024-09-04T17:46:50.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1279999999999999e-05 2024-09-04T17:46:50.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:50.348 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.342050+0000", 2024-09-04T17:46:50.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0042879729999999996 2024-09-04T17:46:50.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:50.348 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.342098+0000", 2024-09-04T17:46:50.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8508999999999999e-05 2024-09-04T17:46:50.348 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:50.348 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:50.348 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:50.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.349 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:12756 3.0 e41/24 3:0074e6f8:::rbd_data.1134ef3a5a7d.000000000000000e:head v 41'24, mlcod=40'20)", 2024-09-04T17:46:50.349 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:17.339316+0000", 2024-09-04T17:46:50.349 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.942672512000001, 2024-09-04T17:46:50.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052778790000000001, 2024-09-04T17:46:50.349 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:50.349 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:50.349 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:50.349 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:50.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:50.350 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.339316+0000", 2024-09-04T17:46:50.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:50.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:50.350 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.339316+0000", 2024-09-04T17:46:50.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:50.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:50.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.339317+0000", 2024-09-04T17:46:50.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.13e-06 2024-09-04T17:46:50.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:50.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.339328+0000", 2024-09-04T17:46:50.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1742e-05 2024-09-04T17:46:50.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:50.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.339358+0000", 2024-09-04T17:46:50.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9149000000000002e-05 2024-09-04T17:46:50.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:50.352 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.339379+0000", 2024-09-04T17:46:50.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1288e-05 2024-09-04T17:46:50.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:50.352 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.339454+0000", 2024-09-04T17:46:50.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4703000000000006e-05 2024-09-04T17:46:50.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:50.352 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.339506+0000", 2024-09-04T17:46:50.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2188999999999998e-05 2024-09-04T17:46:50.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:50.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.344475+0000", 2024-09-04T17:46:50.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0049691919999999999 2024-09-04T17:46:50.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:50.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.344593+0000", 2024-09-04T17:46:50.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000118486 2024-09-04T17:46:50.354 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:50.354 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:50.354 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:50.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.354 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:27957 3.1f e50/37 3:ff7930c6:::rbd_data.1134eedad849.0000000000000380:head v 50'169, mlcod=50'165)", 2024-09-04T17:46:50.354 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:30.598047+0000", 2024-09-04T17:46:50.354 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.683940951, 2024-09-04T17:46:50.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0042527729999999996, 2024-09-04T17:46:50.354 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:50.354 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:50.354 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:50.355 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:50.355 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.355 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:50.355 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.598047+0000", 2024-09-04T17:46:50.355 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:50.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.355 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.355 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:50.355 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.598047+0000", 2024-09-04T17:46:50.355 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:50.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.355 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.355 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:50.355 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.598048+0000", 2024-09-04T17:46:50.356 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7800000000000001e-07 2024-09-04T17:46:50.356 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.356 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.356 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:50.356 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.598055+0000", 2024-09-04T17:46:50.356 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1069999999999999e-06 2024-09-04T17:46:50.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.357 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:50.357 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.598067+0000", 2024-09-04T17:46:50.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2278e-05 2024-09-04T17:46:50.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.357 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:50.357 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.598072+0000", 2024-09-04T17:46:50.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8439999999999997e-06 2024-09-04T17:46:50.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:50.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.598114+0000", 2024-09-04T17:46:50.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1931000000000002e-05 2024-09-04T17:46:50.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:50.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.598133+0000", 2024-09-04T17:46:50.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9102000000000001e-05 2024-09-04T17:46:50.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:50.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.602251+0000", 2024-09-04T17:46:50.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041175079999999998 2024-09-04T17:46:50.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:50.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.602300+0000", 2024-09-04T17:46:50.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9224999999999997e-05 2024-09-04T17:46:50.359 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:50.359 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:50.359 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:50.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.360 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:27958 3.1f e50/37 3:fa2ba8de:::rbd_data.1134eedad849.0000000000000384:head v 50'170, mlcod=50'166)", 2024-09-04T17:46:50.360 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:30.598748+0000", 2024-09-04T17:46:50.360 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.683240469000001, 2024-09-04T17:46:50.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0035807859999999999, 2024-09-04T17:46:50.360 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:50.360 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:50.360 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:50.360 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:50.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:50.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.598748+0000", 2024-09-04T17:46:50.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:50.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:50.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.598748+0000", 2024-09-04T17:46:50.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:50.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:50.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.598748+0000", 2024-09-04T17:46:50.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2399999999999997e-07 2024-09-04T17:46:50.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:50.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.598756+0000", 2024-09-04T17:46:50.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4089999999999997e-06 2024-09-04T17:46:50.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.362 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:50.362 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.598767+0000", 2024-09-04T17:46:50.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1449e-05 2024-09-04T17:46:50.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:50.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.598772+0000", 2024-09-04T17:46:50.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7790000000000003e-06 2024-09-04T17:46:50.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:50.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.598831+0000", 2024-09-04T17:46:50.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9091000000000001e-05 2024-09-04T17:46:50.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:50.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.598843+0000", 2024-09-04T17:46:50.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1691999999999999e-05 2024-09-04T17:46:50.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:50.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.602302+0000", 2024-09-04T17:46:50.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0034595609999999999 2024-09-04T17:46:50.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:50.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.602328+0000", 2024-09-04T17:46:50.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5981000000000001e-05 2024-09-04T17:46:50.367 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:50.367 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:50.367 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:50.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.368 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "pg_update_log_missing(3.0 epoch 50/24 rep_tid 378 entries 50'73 (0'0) error 3:01dff485:::rbd_data.1134eedad849.000000000000038b:head by client.4404.0:27967 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 50'71)", 2024-09-04T17:46:50.368 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:30.600491+0000", 2024-09-04T17:46:50.368 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.681496578000001, 2024-09-04T17:46:50.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045188329999999999, 2024-09-04T17:46:50.368 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:50.369 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:50.369 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-04T17:46:50.369 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:50.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:50.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.600491+0000", 2024-09-04T17:46:50.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:50.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:50.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.600491+0000", 2024-09-04T17:46:50.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:50.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:50.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.600492+0000", 2024-09-04T17:46:50.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3500000000000005e-07 2024-09-04T17:46:50.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:50.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.600510+0000", 2024-09-04T17:46:50.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7533999999999998e-05 2024-09-04T17:46:50.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:50.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.600536+0000", 2024-09-04T17:46:50.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5822000000000001e-05 2024-09-04T17:46:50.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:50.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.600556+0000", 2024-09-04T17:46:50.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9913000000000001e-05 2024-09-04T17:46:50.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:50.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.600644+0000", 2024-09-04T17:46:50.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8282999999999995e-05 2024-09-04T17:46:50.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:50.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.605010+0000", 2024-09-04T17:46:50.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0043664460000000004 2024-09-04T17:46:50.376 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:50.376 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:50.377 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:50.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.377 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:34444 3.1f e56/37 3:fcc2bddb:::rbd_data.11342e8f3384.0000000000000004:head v 56'189, mlcod=54'187)", 2024-09-04T17:46:50.377 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:36.632281+0000", 2024-09-04T17:46:50.377 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.649707284, 2024-09-04T17:46:50.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0036502420000000002, 2024-09-04T17:46:50.378 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:50.378 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:50.378 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:50.378 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:50.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:50.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.632281+0000", 2024-09-04T17:46:50.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:50.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:50.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.632281+0000", 2024-09-04T17:46:50.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:50.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:50.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.632282+0000", 2024-09-04T17:46:50.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5300000000000003e-07 2024-09-04T17:46:50.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:50.382 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.633663+0000", 2024-09-04T17:46:50.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0013816169999999999 2024-09-04T17:46:50.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:50.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.633682+0000", 2024-09-04T17:46:50.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8624e-05 2024-09-04T17:46:50.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:50.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.633691+0000", 2024-09-04T17:46:50.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3260000000000002e-06 2024-09-04T17:46:50.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:50.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.633766+0000", 2024-09-04T17:46:50.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5075999999999997e-05 2024-09-04T17:46:50.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:50.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.633793+0000", 2024-09-04T17:46:50.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6939000000000001e-05 2024-09-04T17:46:50.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:50.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.635872+0000", 2024-09-04T17:46:50.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020786820000000001 2024-09-04T17:46:50.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:50.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.635931+0000", 2024-09-04T17:46:50.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9125000000000001e-05 2024-09-04T17:46:50.387 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:50.388 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:50.388 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:50.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.388 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:37418 3.1f e58/37 3:ff114f66:::rbd_data.1134c5a43465.00000000000000da:head v 58'200, mlcod=58'198)", 2024-09-04T17:46:50.388 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:39.727221+0000", 2024-09-04T17:46:50.389 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.554766598, 2024-09-04T17:46:50.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040617930000000002, 2024-09-04T17:46:50.389 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:50.389 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:50.389 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:50.390 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:50.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:50.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.727221+0000", 2024-09-04T17:46:50.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:50.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:50.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.727221+0000", 2024-09-04T17:46:50.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:50.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:50.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.727223+0000", 2024-09-04T17:46:50.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.88e-06 2024-09-04T17:46:50.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:50.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.727239+0000", 2024-09-04T17:46:50.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5962000000000001e-05 2024-09-04T17:46:50.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:50.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.727278+0000", 2024-09-04T17:46:50.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8844999999999998e-05 2024-09-04T17:46:50.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:50.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.727295+0000", 2024-09-04T17:46:50.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7192000000000001e-05 2024-09-04T17:46:50.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:50.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.727474+0000", 2024-09-04T17:46:50.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000178176 2024-09-04T17:46:50.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:50.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.727536+0000", 2024-09-04T17:46:50.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2212999999999995e-05 2024-09-04T17:46:50.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:50.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.731172+0000", 2024-09-04T17:46:50.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0036367050000000001 2024-09-04T17:46:50.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:50.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.731283+0000", 2024-09-04T17:46:50.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011082 2024-09-04T17:46:50.398 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:50.399 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:50.399 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:50.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.399 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:37420 3.9 e58/24 3:940e2e4e:::rbd_data.1134c5a43465.00000000000000dc:head v 58'215, mlcod=58'213)", 2024-09-04T17:46:50.400 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:39.728565+0000", 2024-09-04T17:46:50.400 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.553423251, 2024-09-04T17:46:50.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039183309999999997, 2024-09-04T17:46:50.400 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:50.400 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:50.400 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:50.401 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:50.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:50.401 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.728565+0000", 2024-09-04T17:46:50.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:50.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:50.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.728565+0000", 2024-09-04T17:46:50.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:50.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:50.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.728567+0000", 2024-09-04T17:46:50.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7230000000000001e-06 2024-09-04T17:46:50.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:50.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.728579+0000", 2024-09-04T17:46:50.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2274e-05 2024-09-04T17:46:50.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:50.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.728613+0000", 2024-09-04T17:46:50.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4326000000000003e-05 2024-09-04T17:46:50.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:50.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.728628+0000", 2024-09-04T17:46:50.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4882000000000001e-05 2024-09-04T17:46:50.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:50.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.728793+0000", 2024-09-04T17:46:50.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016519000000000001 2024-09-04T17:46:50.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:50.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.728839+0000", 2024-09-04T17:46:50.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5343000000000003e-05 2024-09-04T17:46:50.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:50.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.732380+0000", 2024-09-04T17:46:50.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0035412849999999999 2024-09-04T17:46:50.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:50.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:50.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:50.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.732483+0000", 2024-09-04T17:46:50.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010330799999999999 2024-09-04T17:46:50.408 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:50.408 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:50.408 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:50.408 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:50.408 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:50.408 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:50.410 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-04T17:46:50.411 INFO:teuthology.orchestra.run.smithi092.stdout:2799 read 0x8f6213 thru 0x90619d (0xff8b bytes) 2024-09-04T17:46:50.412 INFO:teuthology.orchestra.run.smithi092.stdout:2800 write 0xbfd8eba thru 0xbfdeaa4 (0x5beb bytes) 2024-09-04T17:46:50.412 INFO:teuthology.orchestra.run.smithi092.stdout:2801 write 0x1e0a6af thru 0x1e11be6 (0x7538 bytes) 2024-09-04T17:46:50.412 INFO:teuthology.orchestra.run.smithi092.stdout:2803 write 0x95415a thru 0x95f87a (0xb721 bytes) 2024-09-04T17:46:50.412 INFO:teuthology.orchestra.run.smithi092.stdout:2804 write 0x93831bb thru 0x938832b (0x5171 bytes) 2024-09-04T17:46:50.412 INFO:teuthology.orchestra.run.smithi092.stdout:2805 trunc from 0xbfdeaa5 to 0x8e5a8fe 2024-09-04T17:46:50.412 INFO:teuthology.orchestra.run.smithi092.stdout:2806 read 0x4c7c744 thru 0x4c7e445 (0x1d02 bytes) 2024-09-04T17:46:50.412 INFO:teuthology.orchestra.run.smithi092.stdout:2807 write 0x985d502 thru 0x985e9ff (0x14fe bytes) 2024-09-04T17:46:50.412 INFO:teuthology.orchestra.run.smithi092.stdout:2808 write 0x91cb6a2 thru 0x91d2975 (0x72d4 bytes) 2024-09-04T17:46:50.412 INFO:teuthology.orchestra.run.smithi092.stdout:2809 read 0xd00664 thru 0xd0fc6a (0xf607 bytes) 2024-09-04T17:46:50.412 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:50.413 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:50.409+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:50.424 INFO:teuthology.orchestra.run.smithi092.stdout:2811 write 0xd2522f8 thru 0xd25be3e (0x9b47 bytes) 2024-09-04T17:46:50.429 INFO:teuthology.orchestra.run.smithi092.stdout:2812 write 0xa7b2b63 thru 0xa7c1223 (0xe6c1 bytes) 2024-09-04T17:46:50.436 INFO:teuthology.orchestra.run.smithi092.stdout:2813 punch from 0xc3f9c63 to 0xc402980, (0x8d1d bytes) 2024-09-04T17:46:50.436 INFO:teuthology.orchestra.run.smithi092.stdout:2816 write 0x1e810a thru 0x1eed42 (0x6c39 bytes) 2024-09-04T17:46:50.439 INFO:teuthology.orchestra.run.smithi092.stdout:2817 punch from 0x6c904d0 to 0x6c9caf9, (0xc629 bytes) 2024-09-04T17:46:50.439 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:50.439 INFO:teuthology.orchestra.run.smithi092.stdout:2819 punch from 0x7c5ffe8 to 0x7c61c4d, (0x1c65 bytes) 2024-09-04T17:46:50.439 INFO:teuthology.orchestra.run.smithi092.stdout:2820 write 0x2c503b8 thru 0x2c5c115 (0xbd5e bytes) 2024-09-04T17:46:50.443 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:50.443 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:50.443 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:50.443 INFO:teuthology.orchestra.run.smithi092.stdout:2825 read 0x75dcb8b thru 0x75ddacc (0xf42 bytes) 2024-09-04T17:46:50.458 INFO:teuthology.orchestra.run.smithi092.stdout:2827 trunc from 0xd25be3f to 0x5d18754 2024-09-04T17:46:50.478 INFO:teuthology.orchestra.run.smithi092.stdout:2828 trunc from 0x5d18754 to 0x7f6f7fb 2024-09-04T17:46:50.487 INFO:teuthology.orchestra.run.smithi092.stdout:2830 punch from 0x2c8eb35 to 0x2c94b53, (0x601e bytes) 2024-09-04T17:46:50.488 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:50.488 INFO:teuthology.orchestra.run.smithi092.stdout:2833 writesame 0x25ff22a thru 0x260bd33 data_size 0x3(0xcb0a bytes) 2024-09-04T17:46:50.503 INFO:teuthology.orchestra.run.smithi092.stdout:2834 read 0x7e4966f thru 0x7e55256 (0xbbe8 bytes) 2024-09-04T17:46:50.503 INFO:teuthology.orchestra.run.smithi092.stdout:2835 read 0x7a23db9 thru 0x7a246f7 (0x93f bytes) 2024-09-04T17:46:50.511 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:50.509+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:50.524 INFO:teuthology.orchestra.run.smithi092.stdout:2836 write 0xd40cc27 thru 0xd419b66 (0xcf40 bytes) 2024-09-04T17:46:50.538 INFO:teuthology.orchestra.run.smithi092.stdout:2837 trunc from 0xd419b67 to 0x9881fb9 2024-09-04T17:46:50.542 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:50.542 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:46:50.542 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:46:50.543 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:50.559 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-04T17:46:50.559 INFO:teuthology.orchestra.run.smithi092.stdout:2840 write 0x49c145d thru 0x49c6c0b (0x57af bytes) 2024-09-04T17:46:50.564 INFO:teuthology.orchestra.run.smithi092.stdout:2842 writesame 0x3b464b6 thru 0x3b537a0 data_size 0x1(0xd2eb bytes) 2024-09-04T17:46:50.600 INFO:teuthology.orchestra.run.smithi092.stdout:2843 read 0x72c4709 thru 0x72d26c7 (0xdfbf bytes) 2024-09-04T17:46:50.601 INFO:teuthology.orchestra.run.smithi092.stdout:2844 writesame 0x2162511 thru 0x2166917 data_size 0x1b(0x4407 bytes) 2024-09-04T17:46:50.603 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:50.603 INFO:teuthology.orchestra.run.smithi092.stdout:2848 punch from 0x682fe46 to 0x683ab23, (0xacdd bytes) 2024-09-04T17:46:50.603 INFO:teuthology.orchestra.run.smithi092.stdout:2849 punch from 0x2f27dfe to 0x2f3516d, (0xd36f bytes) 2024-09-04T17:46:50.603 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:50.623 INFO:teuthology.orchestra.run.smithi092.stdout:2852 writesame 0xd3e6084 thru 0xd3e6cfe data_size 0x5(0xc7b bytes) 2024-09-04T17:46:50.623 INFO:teuthology.orchestra.run.smithi092.stdout:2854 read 0x15b4659 thru 0x15ba032 (0x59da bytes) 2024-09-04T17:46:50.623 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:50.644 INFO:teuthology.orchestra.run.smithi092.stdout:2856 trunc from 0xd3e6cff to 0x539e79b 2024-09-04T17:46:50.648 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-04T17:46:50.649 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:50.649+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:50.655 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:50.655 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:50.655 INFO:teuthology.orchestra.run.smithi092.stdout:2861 read 0x150fdfc thru 0x151f9b5 (0xfbba bytes) 2024-09-04T17:46:50.656 INFO:teuthology.orchestra.run.smithi092.stdout:2862 punch from 0x11c7d5b to 0x11d42d0, (0xc575 bytes) 2024-09-04T17:46:50.672 INFO:teuthology.orchestra.run.smithi092.stdout:2864 write 0x753540e thru 0x753abd0 (0x57c3 bytes) 2024-09-04T17:46:50.676 INFO:teuthology.orchestra.run.smithi092.stdout:2866 write 0x72d0358 thru 0x72d9caa (0x9953 bytes) 2024-09-04T17:46:50.680 INFO:teuthology.orchestra.run.smithi092.stdout:2868 read 0x6b87828 thru 0x6b8b573 (0x3d4c bytes) 2024-09-04T17:46:50.700 INFO:teuthology.orchestra.run.smithi092.stdout:2869 writesame 0xb9db017 thru 0xb9e273a data_size 0x19(0x7724 bytes) 2024-09-04T17:46:50.700 INFO:teuthology.orchestra.run.smithi092.stdout:2870 read 0x8dc5256 thru 0x8dcff52 (0xacfd bytes) 2024-09-04T17:46:50.700 INFO:teuthology.orchestra.run.smithi092.stdout:2871 trunc from 0xb9e273b to 0xbea121d 2024-09-04T17:46:50.713 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:46:50.711+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:46:50.719 INFO:teuthology.orchestra.run.smithi092.stdout:2873 trunc from 0xbea121d to 0x72f2910 2024-09-04T17:46:50.723 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:50.724 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:46:50.724 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:46:50.724 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:46:50.724 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:50.736 INFO:teuthology.orchestra.run.smithi092.stdout:2874 trunc from 0x72f2910 to 0x4630cbc 2024-09-04T17:46:50.739 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-04T17:46:50.783 INFO:teuthology.orchestra.run.smithi092.stdout:2875 writesame 0xebec20a thru 0xebf105c data_size 0x1(0x4e53 bytes) 2024-09-04T17:46:50.783 INFO:teuthology.orchestra.run.smithi092.stdout:2876 writesame 0xe17b02 thru 0xe21c8d data_size 0x62(0xa18c bytes) 2024-09-04T17:46:50.803 INFO:teuthology.orchestra.run.smithi092.stdout:2877 trunc from 0xebf105d to 0x258cfb0 2024-09-04T17:46:50.812 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:46:50.813 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:46:50.813 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:50.811+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:50.830 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:46:50.830 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:46:50.836 INFO:teuthology.orchestra.run.smithi092.stdout:2879 write 0xbf345e2 thru 0xbf40e40 (0xc85f bytes) 2024-09-04T17:46:50.840 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:50.844 INFO:teuthology.orchestra.run.smithi092.stdout:2881 trunc from 0xbf40e41 to 0xe9f4566 2024-09-04T17:46:50.849 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-04T17:46:50.850 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-04T17:46:50.851 INFO:teuthology.orchestra.run.smithi092.stdout:2882 read 0x3a7c0e1 thru 0x3a8ac42 (0xeb62 bytes) 2024-09-04T17:46:50.851 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:50.851 INFO:teuthology.orchestra.run.smithi092.stdout:2885 write 0x238ba0f thru 0x238fb92 (0x4184 bytes) 2024-09-04T17:46:50.853 INFO:teuthology.orchestra.run.smithi092.stdout:2887 read 0x8fef7af thru 0x8ffbeef (0xc741 bytes) 2024-09-04T17:46:50.859 INFO:teuthology.orchestra.run.smithi092.stdout:2888 write 0xed90398 thru 0xed9a32c (0x9f95 bytes) 2024-09-04T17:46:50.863 INFO:teuthology.orchestra.run.smithi092.stdout:2889 punch from 0xebd21ba to 0xebe1edf, (0xfd25 bytes) 2024-09-04T17:46:50.863 INFO:teuthology.orchestra.run.smithi092.stdout:2890 punch from 0x467d6c9 to 0x4683405, (0x5d3c bytes) 2024-09-04T17:46:50.863 INFO:teuthology.orchestra.run.smithi092.stdout:2892 read 0x6d91139 thru 0x6d929a6 (0x186e bytes) 2024-09-04T17:46:50.863 INFO:teuthology.orchestra.run.smithi092.stdout:2893 read 0x1bd2c9a thru 0x1be094a (0xdcb1 bytes) 2024-09-04T17:46:50.864 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:50.864 INFO:teuthology.orchestra.run.smithi092.stdout:2897 clone 22 order 19 su 65536 sc 5 2024-09-04T17:46:50.868 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-04T17:46:50.868 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-04T17:46:50.887 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:46:50.887 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:46:50.906 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:46:50.906 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:46:50.906 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-04T17:46:50.910 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:50.910 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:50.913 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:50.911+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:50.925 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-04T17:46:50.926 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:46:50.926 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:46:50.929 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-04T17:46:50.929 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-04T17:46:51.015 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:51.013+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:51.049 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:51.050 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:51.064 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-04T17:46:51.116 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:46:51.115+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:46:51.218 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:51.217+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:51.232 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:51.232 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:51.247 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-04T17:46:51.318 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:51.315+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:51.411 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:51.411 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:51.419 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:51.417+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:51.425 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-04T17:46:51.520 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:51.521+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:51.529 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-04T17:46:51.599 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:51.599 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:51.614 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-04T17:46:51.656 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:51.657+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:51.658 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone21 from 0xed9a32d (overlap 0x977be2) to 0xf25fd 2024-09-04T17:46:51.722 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:51.719+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:51.782 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:51.783 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:51.783 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #20, image image_client.0-clone20 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent21 2024-09-04T17:46:51.797 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-04T17:46:51.823 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:51.819+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:51.919 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:51.919 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:51.925 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:51.923+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:51.933 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-04T17:46:51.989 INFO:teuthology.orchestra.run.smithi092.stdout:2898 read 0x176426d thru 0x176fa54 (0xb7e8 bytes) 2024-09-04T17:46:51.990 INFO:teuthology.orchestra.run.smithi092.stdout:2899 read 0x6f2bccb thru 0x6f36561 (0xa897 bytes) 2024-09-04T17:46:51.991 INFO:teuthology.orchestra.run.smithi092.stdout:2900 writesame 0xbb8c1fd thru 0xbb920b7 data_size 0x1(0x5ebb bytes) 2024-09-04T17:46:52.026 INFO:teuthology.orchestra.run.smithi092.stdout:2901 read 0x4acc11 thru 0x4afff3 (0x33e3 bytes) 2024-09-04T17:46:52.026 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:52.025+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:52.027 INFO:teuthology.orchestra.run.smithi092.stdout:2902 read 0x7c543f2 thru 0x7c56131 (0x1d40 bytes) 2024-09-04T17:46:52.027 INFO:teuthology.orchestra.run.smithi092.stdout:2905 punch from 0x7b99f51 to 0x7ba07a4, (0x6853 bytes) 2024-09-04T17:46:52.027 INFO:teuthology.orchestra.run.smithi092.stdout:2906 write 0x5eadf3a thru 0x5eb9150 (0xb217 bytes) 2024-09-04T17:46:52.037 INFO:teuthology.orchestra.run.smithi092.stdout:2907 write 0x1e1d981 thru 0x1e27781 (0x9e01 bytes) 2024-09-04T17:46:52.048 INFO:teuthology.orchestra.run.smithi092.stdout:2910 read 0x626ae9f thru 0x6273873 (0x89d5 bytes) 2024-09-04T17:46:52.048 INFO:teuthology.orchestra.run.smithi092.stdout:2915 write 0x841535 thru 0x84da2c (0xc4f8 bytes) 2024-09-04T17:46:52.057 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:52.057 INFO:teuthology.orchestra.run.smithi092.stdout:2917 read 0x4a92f5 thru 0x4abf16 (0x2c22 bytes) 2024-09-04T17:46:52.058 INFO:teuthology.orchestra.run.smithi092.stdout:2918 write 0x359c1d3 thru 0x35aad90 (0xebbe bytes) 2024-09-04T17:46:52.068 INFO:teuthology.orchestra.run.smithi092.stdout:2919 read 0x6da57ba thru 0x6db3af9 (0xe340 bytes) 2024-09-04T17:46:52.068 INFO:teuthology.orchestra.run.smithi092.stdout:2920 write 0x6ad8ca7 thru 0x6ae5355 (0xc6af bytes) 2024-09-04T17:46:52.075 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:52.075 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:52.079 INFO:teuthology.orchestra.run.smithi092.stdout:2921 read 0x193bed0 thru 0x194781a (0xb94b bytes) 2024-09-04T17:46:52.080 INFO:teuthology.orchestra.run.smithi092.stdout:2922 read 0xd41ebc thru 0xd44e59 (0x2f9e bytes) 2024-09-04T17:46:52.081 INFO:teuthology.orchestra.run.smithi092.stdout:2924 punch from 0x1fcf18e to 0x1fdc29b, (0xd10d bytes) 2024-09-04T17:46:52.081 INFO:teuthology.orchestra.run.smithi092.stdout:2925 punch from 0xb0c883d to 0xb0d7ed7, (0xf69a bytes) 2024-09-04T17:46:52.081 INFO:teuthology.orchestra.run.smithi092.stdout:2926 read 0x8faf7c5 thru 0x8fb2dc6 (0x3602 bytes) 2024-09-04T17:46:52.081 INFO:teuthology.orchestra.run.smithi092.stdout:2927 read 0x3f516e2 thru 0x3f51ebf (0x7de bytes) 2024-09-04T17:46:52.081 INFO:teuthology.orchestra.run.smithi092.stdout:2928 writesame 0xe3ec7e0 thru 0xe3fa3e1 data_size 0x7e(0xdc02 bytes) 2024-09-04T17:46:52.091 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-04T17:46:52.092 INFO:teuthology.orchestra.run.smithi092.stdout:2929 read 0x2c94cd9 thru 0x2c9aa74 (0x5d9c bytes) 2024-09-04T17:46:52.093 INFO:teuthology.orchestra.run.smithi092.stdout:2930 write 0x72e2950 thru 0x72e718f (0x4840 bytes) 2024-09-04T17:46:52.100 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:52.100 INFO:teuthology.orchestra.run.smithi092.stdout:2932 write 0x8fc5b6d thru 0x8fcf550 (0x99e4 bytes) 2024-09-04T17:46:52.105 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [6] live_osds: [0, 1, 2, 4, 5, 7, 3] 2024-09-04T17:46:52.105 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-04T17:46:52.105 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-04T17:46:52.105 INFO:tasks.thrashosds.thrasher:Setting osd 4 primary_affinity to 0.000000 2024-09-04T17:46:52.105 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 0 2024-09-04T17:46:52.106 INFO:teuthology.orchestra.run.smithi092.stdout:2935 write 0x3ca3197 thru 0x3ca5925 (0x278f bytes) 2024-09-04T17:46:52.111 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:52.118 INFO:teuthology.orchestra.run.smithi092.stdout:2939 trunc from 0xed9a32d to 0xb97c1b0 2024-09-04T17:46:52.127 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:46:52.123+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:46:52.161 INFO:teuthology.orchestra.run.smithi092.stdout:2941 write 0x855366f thru 0x855b86f (0x8201 bytes) 2024-09-04T17:46:52.169 INFO:teuthology.orchestra.run.smithi092.stdout:2942 write 0xa10e288 thru 0xa1192f3 (0xb06c bytes) 2024-09-04T17:46:52.178 INFO:teuthology.orchestra.run.smithi092.stdout:2943 punch from 0x2024227 to 0x202c13e, (0x7f17 bytes) 2024-09-04T17:46:52.178 INFO:teuthology.orchestra.run.smithi092.stdout:2944 write 0x61625be thru 0x6163421 (0xe64 bytes) 2024-09-04T17:46:52.183 INFO:teuthology.orchestra.run.smithi092.stdout:2945 write 0x6ab5c13 thru 0x6abe853 (0x8c41 bytes) 2024-09-04T17:46:52.203 INFO:teuthology.orchestra.run.smithi092.stdout:2946 write 0xcd22c7d thru 0xcd3151f (0xe8a3 bytes) 2024-09-04T17:46:52.211 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:52.211 INFO:teuthology.orchestra.run.smithi092.stdout:2948 punch from 0xe058ab to 0xe06d78, (0x14cd bytes) 2024-09-04T17:46:52.211 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:52.211 INFO:teuthology.orchestra.run.smithi092.stdout:2950 punch from 0x78062dc to 0x78063c9, (0xed bytes) 2024-09-04T17:46:52.212 INFO:teuthology.orchestra.run.smithi092.stdout:2951 write 0x1d7f148 thru 0x1d8be08 (0xccc1 bytes) 2024-09-04T17:46:52.220 INFO:teuthology.orchestra.run.smithi092.stdout:2952 write 0x2222f7f thru 0x2223293 (0x315 bytes) 2024-09-04T17:46:52.227 INFO:teuthology.orchestra.run.smithi092.stdout:2953 read 0x4c42d33 thru 0x4c4a94d (0x7c1b bytes) 2024-09-04T17:46:52.227 INFO:teuthology.orchestra.run.smithi092.stdout:2954 writesame 0xa03681d thru 0xa040376 data_size 0x29(0x9b5a bytes) 2024-09-04T17:46:52.228 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:52.223+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:52.230 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:52.230 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:52.235 INFO:teuthology.orchestra.run.smithi092.stdout:2955 write 0xb2d84be thru 0xb2e5aad (0xd5f0 bytes) 2024-09-04T17:46:52.243 INFO:teuthology.orchestra.run.smithi092.stdout:2957 write 0x934c655 thru 0x935ad61 (0xe70d bytes) 2024-09-04T17:46:52.250 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-04T17:46:52.252 INFO:teuthology.orchestra.run.smithi092.stdout:2958 punch from 0xa26ed14 to 0xa2703b6, (0x16a2 bytes) 2024-09-04T17:46:52.252 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:52.252 INFO:teuthology.orchestra.run.smithi092.stdout:2960 read 0x59bf41a thru 0x59c8264 (0x8e4b bytes) 2024-09-04T17:46:52.252 INFO:teuthology.orchestra.run.smithi092.stdout:2961 writesame 0xb0d47b thru 0xb0fb2c data_size 0x27(0x26b2 bytes) 2024-09-04T17:46:52.257 INFO:teuthology.orchestra.run.smithi092.stdout:2962 read 0x66a4ea3 thru 0x66b2702 (0xd860 bytes) 2024-09-04T17:46:52.258 INFO:teuthology.orchestra.run.smithi092.stdout:2963 flatten 2024-09-04T17:46:52.329 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:52.329+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:52.401 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:52.401 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:52.407 INFO:teuthology.orchestra.run.smithi092.stdout:2964 read 0x6c09e32 thru 0x6c0d5ce (0x379d bytes) 2024-09-04T17:46:52.415 INFO:teuthology.orchestra.run.smithi092.stdout:2965 trunc from 0xcd31520 to 0x78b0516 2024-09-04T17:46:52.419 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-04T17:46:52.429 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:52.429+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:52.443 INFO:teuthology.orchestra.run.smithi092.stdout:2966 read 0x6855dd0 thru 0x6862a08 (0xcc39 bytes) 2024-09-04T17:46:52.443 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:52.443 INFO:teuthology.orchestra.run.smithi092.stdout:2968 read 0x39a80ba thru 0x39adfe9 (0x5f30 bytes) 2024-09-04T17:46:52.465 INFO:teuthology.orchestra.run.smithi092.stdout:2971 write 0xd1a09ad thru 0xd1a3f13 (0x3567 bytes) 2024-09-04T17:46:52.477 INFO:teuthology.orchestra.run.smithi092.stdout:2972 trunc from 0xd1a3f14 to 0x929f740 2024-09-04T17:46:52.496 INFO:teuthology.orchestra.run.smithi092.stdout:2973 punch from 0x2d2d828 to 0x2d3d6b6, (0xfe8e bytes) 2024-09-04T17:46:52.497 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:52.497 INFO:teuthology.orchestra.run.smithi092.stdout:2975 write 0x13c4e77 thru 0x13cbe6b (0x6ff5 bytes) 2024-09-04T17:46:52.509 INFO:teuthology.orchestra.run.smithi092.stdout:2976 trunc from 0x929f740 to 0x4d6ea42 2024-09-04T17:46:52.513 INFO:teuthology.orchestra.run.smithi092.stderr:set osd.4 primary-affinity to 0 (802) 2024-09-04T17:46:52.517 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is unset 2024-09-04T17:46:52.530 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:52.529+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:52.570 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:52.570 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:52.584 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-04T17:46:52.676 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:52.675+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:52.731 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:52.727+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:52.748 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:52.748 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:52.762 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-04T17:46:52.832 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:52.833+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:52.927 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:52.927 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:52.930 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-04T17:46:52.933 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:52.933+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:52.942 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-04T17:46:53.034 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:53.031+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:53.071 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:46:53.071 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:46:53.089 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:46:53.090 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:46:53.107 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-04T17:46:53.108 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-04T17:46:53.109 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:53.109 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:46:53.109 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:46:53.109 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:53.123 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-04T17:46:53.124 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-04T17:46:53.125 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-04T17:46:53.135 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:53.133+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:53.145 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:46:53.145 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:46:53.166 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:46:53.166 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:46:53.166 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-04T17:46:53.187 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:46:53.187 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:46:53.191 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-04T17:46:53.191 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-04T17:46:53.235 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:53.231+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:53.238 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:53.238 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:46:53.238 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:46:53.238 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:46:53.238 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:53.252 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-04T17:46:53.340 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:46:53.339+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:46:53.414 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:53.415 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:46:53.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:46:53.415 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:46:53.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.415 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15462 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 835584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:53.415 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.220338+0000", 2024-09-04T17:46:53.415 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 35.194398735999997, 2024-09-04T17:46:53.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051751278999999997, 2024-09-04T17:46:53.415 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:53.415 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:53.415 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:53.415 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:53.415 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:53.416 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:53.416 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15462 2024-09-04T17:46:53.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.416 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:53.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:53.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.220338+0000", 2024-09-04T17:46:53.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:53.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:53.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.220338+0000", 2024-09-04T17:46:53.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:53.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:53.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.220600+0000", 2024-09-04T17:46:53.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026198900000000002 2024-09-04T17:46:53.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:53.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.220617+0000", 2024-09-04T17:46:53.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6956000000000002e-05 2024-09-04T17:46:53.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:53.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.220627+0000", 2024-09-04T17:46:53.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0492999999999999e-05 2024-09-04T17:46:53.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:53.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.220635+0000", 2024-09-04T17:46:53.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0059999999999992e-06 2024-09-04T17:46:53.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:53.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.271853+0000", 2024-09-04T17:46:53.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051218020000000003 2024-09-04T17:46:53.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:53.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.271941+0000", 2024-09-04T17:46:53.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7802999999999999e-05 2024-09-04T17:46:53.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:53.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.272089+0000", 2024-09-04T17:46:53.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014801199999999999 2024-09-04T17:46:53.419 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:53.419 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:53.419 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:53.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.419 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15500 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 294912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:53.420 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.228278+0000", 2024-09-04T17:46:53.420 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 35.186458739000003, 2024-09-04T17:46:53.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051770927000000001, 2024-09-04T17:46:53.420 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:53.420 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:53.420 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:53.420 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:53.420 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:53.420 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:53.420 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15500 2024-09-04T17:46:53.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.420 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:53.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:53.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.228278+0000", 2024-09-04T17:46:53.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:53.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:53.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.228278+0000", 2024-09-04T17:46:53.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:53.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:53.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234002+0000", 2024-09-04T17:46:53.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005724392 2024-09-04T17:46:53.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:53.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234009+0000", 2024-09-04T17:46:53.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1300000000000003e-06 2024-09-04T17:46:53.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:53.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234024+0000", 2024-09-04T17:46:53.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4063000000000001e-05 2024-09-04T17:46:53.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:53.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234035+0000", 2024-09-04T17:46:53.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1124000000000001e-05 2024-09-04T17:46:53.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:53.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.279808+0000", 2024-09-04T17:46:53.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045772992999999998 2024-09-04T17:46:53.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:53.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.279894+0000", 2024-09-04T17:46:53.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6174999999999994e-05 2024-09-04T17:46:53.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:53.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.280049+0000", 2024-09-04T17:46:53.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015505 2024-09-04T17:46:53.424 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:53.425 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:53.425 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:53.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.425 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15524 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 1540096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:53.425 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.234831+0000", 2024-09-04T17:46:53.425 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 35.179905863000002, 2024-09-04T17:46:53.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051893631000000003, 2024-09-04T17:46:53.425 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:53.426 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:53.426 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:53.426 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:53.426 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:53.426 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:53.426 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15524 2024-09-04T17:46:53.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.426 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:53.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:53.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234831+0000", 2024-09-04T17:46:53.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:53.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:53.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234831+0000", 2024-09-04T17:46:53.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:53.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:53.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234831+0000", 2024-09-04T17:46:53.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2699999999999999e-07 2024-09-04T17:46:53.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:53.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234833+0000", 2024-09-04T17:46:53.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9810000000000002e-06 2024-09-04T17:46:53.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:53.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234839+0000", 2024-09-04T17:46:53.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1519999999999997e-06 2024-09-04T17:46:53.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:53.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234845+0000", 2024-09-04T17:46:53.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1109999999999999e-06 2024-09-04T17:46:53.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:53.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.286526+0000", 2024-09-04T17:46:53.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051681373000000003 2024-09-04T17:46:53.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:53.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.286594+0000", 2024-09-04T17:46:53.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8073999999999999e-05 2024-09-04T17:46:53.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:53.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.286724+0000", 2024-09-04T17:46:53.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013041300000000001 2024-09-04T17:46:53.429 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:53.430 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:53.430 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:53.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.430 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15525 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 335872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:53.430 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.234851+0000", 2024-09-04T17:46:53.430 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 35.179885339000002, 2024-09-04T17:46:53.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052146248999999999, 2024-09-04T17:46:53.430 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:53.430 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:53.430 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:53.430 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:53.430 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:53.431 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:53.431 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15525 2024-09-04T17:46:53.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.431 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:53.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:53.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234851+0000", 2024-09-04T17:46:53.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:53.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:53.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234851+0000", 2024-09-04T17:46:53.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:53.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:53.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234852+0000", 2024-09-04T17:46:53.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0999999999999999e-07 2024-09-04T17:46:53.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:53.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234854+0000", 2024-09-04T17:46:53.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9929999999999998e-06 2024-09-04T17:46:53.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:53.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234860+0000", 2024-09-04T17:46:53.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1489999999999999e-06 2024-09-04T17:46:53.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:53.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234865+0000", 2024-09-04T17:46:53.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8600000000000001e-06 2024-09-04T17:46:53.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:53.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.286770+0000", 2024-09-04T17:46:53.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051904807999999997 2024-09-04T17:46:53.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.434 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:53.434 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.286852+0000", 2024-09-04T17:46:53.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2076000000000005e-05 2024-09-04T17:46:53.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.434 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:53.434 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.286998+0000", 2024-09-04T17:46:53.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014595299999999999 2024-09-04T17:46:53.434 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:53.434 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:53.435 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:53.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.435 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15526 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 811008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:53.435 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.234871+0000", 2024-09-04T17:46:53.435 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 35.179865569999997, 2024-09-04T17:46:53.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052383817999999999, 2024-09-04T17:46:53.435 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:53.435 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:53.435 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:53.435 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:53.435 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:53.435 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:53.435 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15526 2024-09-04T17:46:53.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.436 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:53.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:53.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234871+0000", 2024-09-04T17:46:53.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:53.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:53.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234871+0000", 2024-09-04T17:46:53.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:53.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:53.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234872+0000", 2024-09-04T17:46:53.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4299999999999998e-07 2024-09-04T17:46:53.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:53.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234874+0000", 2024-09-04T17:46:53.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2220000000000001e-06 2024-09-04T17:46:53.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:53.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234880+0000", 2024-09-04T17:46:53.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4169999999999997e-06 2024-09-04T17:46:53.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:53.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234886+0000", 2024-09-04T17:46:53.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8150000000000002e-06 2024-09-04T17:46:53.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:53.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287040+0000", 2024-09-04T17:46:53.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052153928000000002 2024-09-04T17:46:53.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.439 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:53.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287118+0000", 2024-09-04T17:46:53.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8102999999999994e-05 2024-09-04T17:46:53.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.439 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:53.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287255+0000", 2024-09-04T17:46:53.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013689 2024-09-04T17:46:53.439 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:53.439 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:53.439 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:53.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.440 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15527 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 372736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:53.440 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.234906+0000", 2024-09-04T17:46:53.440 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 35.179830254999999, 2024-09-04T17:46:53.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052620742999999998, 2024-09-04T17:46:53.440 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:53.440 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:53.440 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:53.440 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:53.440 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:53.440 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:53.440 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15527 2024-09-04T17:46:53.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.441 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:53.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:53.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234906+0000", 2024-09-04T17:46:53.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:53.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:53.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234906+0000", 2024-09-04T17:46:53.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:53.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:53.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234907+0000", 2024-09-04T17:46:53.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9100000000000004e-07 2024-09-04T17:46:53.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:53.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234910+0000", 2024-09-04T17:46:53.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7599999999999998e-06 2024-09-04T17:46:53.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:53.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234917+0000", 2024-09-04T17:46:53.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.861e-06 2024-09-04T17:46:53.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:53.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234923+0000", 2024-09-04T17:46:53.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4130000000000003e-06 2024-09-04T17:46:53.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:53.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287302+0000", 2024-09-04T17:46:53.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052379440999999999 2024-09-04T17:46:53.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:53.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287382+0000", 2024-09-04T17:46:53.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9406999999999994e-05 2024-09-04T17:46:53.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:53.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287527+0000", 2024-09-04T17:46:53.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014537 2024-09-04T17:46:53.444 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:53.444 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:53.444 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:53.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.445 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15528 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 847872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:53.445 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.234931+0000", 2024-09-04T17:46:53.445 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 35.179805336999998, 2024-09-04T17:46:53.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052862919000000001, 2024-09-04T17:46:53.445 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:53.445 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:53.445 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:53.445 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:53.445 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:53.445 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:53.445 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15528 2024-09-04T17:46:53.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.446 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:53.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:53.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234931+0000", 2024-09-04T17:46:53.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:53.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:53.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234931+0000", 2024-09-04T17:46:53.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:53.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:53.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234932+0000", 2024-09-04T17:46:53.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6100000000000001e-07 2024-09-04T17:46:53.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:53.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234934+0000", 2024-09-04T17:46:53.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3309999999999998e-06 2024-09-04T17:46:53.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:53.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234941+0000", 2024-09-04T17:46:53.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4869999999999999e-06 2024-09-04T17:46:53.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:53.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234946+0000", 2024-09-04T17:46:53.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5879999999999997e-06 2024-09-04T17:46:53.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:53.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287571+0000", 2024-09-04T17:46:53.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052624769000000002 2024-09-04T17:46:53.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:53.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287651+0000", 2024-09-04T17:46:53.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9975000000000006e-05 2024-09-04T17:46:53.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:53.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287794+0000", 2024-09-04T17:46:53.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000143308 2024-09-04T17:46:53.449 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:53.449 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:53.449 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:53.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.449 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15529 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 409600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:53.449 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.234954+0000", 2024-09-04T17:46:53.449 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 35.179782465999999, 2024-09-04T17:46:53.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053149889999999998, 2024-09-04T17:46:53.450 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:53.450 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:53.450 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:53.450 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:53.450 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:53.450 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:53.450 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15529 2024-09-04T17:46:53.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.450 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:53.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.450 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:53.450 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234954+0000", 2024-09-04T17:46:53.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:53.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.451 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:53.451 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234954+0000", 2024-09-04T17:46:53.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:53.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.451 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:53.451 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234955+0000", 2024-09-04T17:46:53.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3799999999999997e-07 2024-09-04T17:46:53.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.451 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:53.451 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234957+0000", 2024-09-04T17:46:53.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3309999999999998e-06 2024-09-04T17:46:53.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:53.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234964+0000", 2024-09-04T17:46:53.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5780000000000004e-06 2024-09-04T17:46:53.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:53.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234969+0000", 2024-09-04T17:46:53.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0629999999999996e-06 2024-09-04T17:46:53.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.453 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:53.453 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287839+0000", 2024-09-04T17:46:53.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052870385999999998 2024-09-04T17:46:53.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.453 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:53.453 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287929+0000", 2024-09-04T17:46:53.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9514999999999999e-05 2024-09-04T17:46:53.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.453 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:53.454 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288104+0000", 2024-09-04T17:46:53.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000175379 2024-09-04T17:46:53.454 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:53.454 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:53.454 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:53.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.454 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15530 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 884736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:53.454 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.234976+0000", 2024-09-04T17:46:53.454 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 35.179761048000003, 2024-09-04T17:46:53.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053414593000000003, 2024-09-04T17:46:53.454 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:53.454 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:53.454 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:53.455 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:53.455 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:53.455 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:53.455 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15530 2024-09-04T17:46:53.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.455 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:53.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:53.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234976+0000", 2024-09-04T17:46:53.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:53.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:53.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234976+0000", 2024-09-04T17:46:53.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:53.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:53.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234976+0000", 2024-09-04T17:46:53.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5299999999999999e-07 2024-09-04T17:46:53.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:53.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234978+0000", 2024-09-04T17:46:53.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0329999999999998e-06 2024-09-04T17:46:53.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:53.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234984+0000", 2024-09-04T17:46:53.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0249999999999999e-06 2024-09-04T17:46:53.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:53.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234990+0000", 2024-09-04T17:46:53.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7579999999999998e-06 2024-09-04T17:46:53.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:53.458 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288149+0000", 2024-09-04T17:46:53.458 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053158864 2024-09-04T17:46:53.458 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.458 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.458 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:53.458 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288236+0000", 2024-09-04T17:46:53.458 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7306999999999996e-05 2024-09-04T17:46:53.458 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.458 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.458 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:53.458 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288390+0000", 2024-09-04T17:46:53.458 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000154153 2024-09-04T17:46:53.458 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:53.459 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:53.459 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:53.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.459 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15531 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 446464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:53.459 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.234997+0000", 2024-09-04T17:46:53.459 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 35.179740064999997, 2024-09-04T17:46:53.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053721086000000001, 2024-09-04T17:46:53.459 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:53.459 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:53.459 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:53.459 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:53.459 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:53.460 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:53.460 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15531 2024-09-04T17:46:53.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.460 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:53.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:53.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234997+0000", 2024-09-04T17:46:53.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:53.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:53.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234997+0000", 2024-09-04T17:46:53.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:53.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:53.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234997+0000", 2024-09-04T17:46:53.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8200000000000001e-07 2024-09-04T17:46:53.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:53.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234999+0000", 2024-09-04T17:46:53.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.058e-06 2024-09-04T17:46:53.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.462 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:53.462 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235005+0000", 2024-09-04T17:46:53.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0700000000000003e-06 2024-09-04T17:46:53.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.462 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:53.462 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235012+0000", 2024-09-04T17:46:53.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7020000000000004e-06 2024-09-04T17:46:53.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.462 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:53.463 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288441+0000", 2024-09-04T17:46:53.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053429343999999997 2024-09-04T17:46:53.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.463 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:53.463 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288535+0000", 2024-09-04T17:46:53.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3406000000000007e-05 2024-09-04T17:46:53.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.463 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:53.463 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288718+0000", 2024-09-04T17:46:53.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000183124 2024-09-04T17:46:53.464 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:53.464 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:53.464 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:53.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.464 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15532 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 921600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:53.464 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235020+0000", 2024-09-04T17:46:53.464 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 35.179716438, 2024-09-04T17:46:53.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053966157000000001, 2024-09-04T17:46:53.464 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:53.464 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:53.465 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:53.465 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:53.465 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:53.465 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:53.465 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15532 2024-09-04T17:46:53.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.465 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:53.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:53.465 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235020+0000", 2024-09-04T17:46:53.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:53.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:53.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235020+0000", 2024-09-04T17:46:53.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:53.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:53.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235021+0000", 2024-09-04T17:46:53.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9799999999999999e-07 2024-09-04T17:46:53.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:53.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235023+0000", 2024-09-04T17:46:53.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1560000000000002e-06 2024-09-04T17:46:53.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.467 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.467 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:53.467 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235029+0000", 2024-09-04T17:46:53.467 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0270000000000001e-06 2024-09-04T17:46:53.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.467 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.467 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:53.467 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235034+0000", 2024-09-04T17:46:53.467 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0000000000000004e-06 2024-09-04T17:46:53.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.467 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.468 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:53.468 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288760+0000", 2024-09-04T17:46:53.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053726077999999997 2024-09-04T17:46:53.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.468 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:53.468 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288845+0000", 2024-09-04T17:46:53.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5242999999999997e-05 2024-09-04T17:46:53.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.468 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:53.468 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288986+0000", 2024-09-04T17:46:53.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014125499999999999 2024-09-04T17:46:53.469 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:53.469 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:53.469 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:53.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.469 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15533 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 483328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:53.469 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235041+0000", 2024-09-04T17:46:53.469 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 35.179695363, 2024-09-04T17:46:53.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054242994000000003, 2024-09-04T17:46:53.469 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:53.469 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:53.469 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:53.470 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:53.470 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:53.470 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:53.470 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15533 2024-09-04T17:46:53.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.470 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:53.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.470 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:53.470 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235041+0000", 2024-09-04T17:46:53.470 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:53.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:53.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235041+0000", 2024-09-04T17:46:53.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:53.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:53.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235042+0000", 2024-09-04T17:46:53.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5499999999999999e-07 2024-09-04T17:46:53.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:53.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235044+0000", 2024-09-04T17:46:53.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2259999999999999e-06 2024-09-04T17:46:53.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.472 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:53.472 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235051+0000", 2024-09-04T17:46:53.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4440000000000002e-06 2024-09-04T17:46:53.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.472 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:53.472 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235056+0000", 2024-09-04T17:46:53.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8740000000000003e-06 2024-09-04T17:46:53.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.472 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:53.472 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289028+0000", 2024-09-04T17:46:53.473 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053971742000000003 2024-09-04T17:46:53.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.473 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:53.473 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289108+0000", 2024-09-04T17:46:53.473 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0502999999999998e-05 2024-09-04T17:46:53.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.473 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:53.473 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289284+0000", 2024-09-04T17:46:53.473 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017584999999999999 2024-09-04T17:46:53.473 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:53.473 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:53.474 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:53.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.474 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15534 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 958464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:53.474 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235068+0000", 2024-09-04T17:46:53.474 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 35.179668663000001, 2024-09-04T17:46:53.474 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054505452000000003, 2024-09-04T17:46:53.474 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:53.474 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:53.474 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:53.474 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:53.475 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:53.475 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:53.475 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15534 2024-09-04T17:46:53.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.475 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:53.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.475 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:53.475 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235068+0000", 2024-09-04T17:46:53.475 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:53.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.475 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:53.475 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235068+0000", 2024-09-04T17:46:53.476 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:53.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:53.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235068+0000", 2024-09-04T17:46:53.476 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7899999999999999e-07 2024-09-04T17:46:53.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:53.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235071+0000", 2024-09-04T17:46:53.476 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4109999999999999e-06 2024-09-04T17:46:53.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.477 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:53.477 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235078+0000", 2024-09-04T17:46:53.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0489999999999998e-06 2024-09-04T17:46:53.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.477 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:53.477 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235083+0000", 2024-09-04T17:46:53.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2249999999999999e-06 2024-09-04T17:46:53.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.478 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:53.478 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289328+0000", 2024-09-04T17:46:53.478 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054244894000000002 2024-09-04T17:46:53.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.478 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.478 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:53.478 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289417+0000", 2024-09-04T17:46:53.478 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8988999999999998e-05 2024-09-04T17:46:53.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.478 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.478 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:53.478 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289573+0000", 2024-09-04T17:46:53.479 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000156405 2024-09-04T17:46:53.479 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:53.479 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:53.479 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:53.479 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.479 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15535 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 520192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:53.479 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235173+0000", 2024-09-04T17:46:53.479 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 35.179564134000003, 2024-09-04T17:46:53.479 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054718671000000003, 2024-09-04T17:46:53.479 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:53.479 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:53.480 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:53.480 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:53.480 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:53.480 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:53.480 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15535 2024-09-04T17:46:53.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.480 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:53.480 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.480 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:53.480 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235173+0000", 2024-09-04T17:46:53.481 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:53.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.481 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:53.481 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235173+0000", 2024-09-04T17:46:53.481 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:53.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.481 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:53.481 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235173+0000", 2024-09-04T17:46:53.481 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2600000000000002e-07 2024-09-04T17:46:53.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.482 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.482 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:53.482 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235177+0000", 2024-09-04T17:46:53.482 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.625e-06 2024-09-04T17:46:53.482 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.482 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.482 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:53.482 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235188+0000", 2024-09-04T17:46:53.482 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1464000000000001e-05 2024-09-04T17:46:53.482 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.482 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.482 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:53.482 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235195+0000", 2024-09-04T17:46:53.483 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5910000000000001e-06 2024-09-04T17:46:53.483 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.483 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.483 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:53.483 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289623+0000", 2024-09-04T17:46:53.483 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054427595000000002 2024-09-04T17:46:53.483 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.483 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.483 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:53.483 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289718+0000", 2024-09-04T17:46:53.483 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5958999999999994e-05 2024-09-04T17:46:53.483 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.483 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.483 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:53.484 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289891+0000", 2024-09-04T17:46:53.484 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017271100000000001 2024-09-04T17:46:53.484 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:53.484 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:53.484 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:53.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.484 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.484 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15536 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 995328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:53.484 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235206+0000", 2024-09-04T17:46:53.484 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 35.179530993, 2024-09-04T17:46:53.484 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054984065999999998, 2024-09-04T17:46:53.484 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:53.484 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:53.485 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:53.485 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:53.485 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:53.485 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:53.485 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15536 2024-09-04T17:46:53.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.485 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:53.485 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.485 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:53.485 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235206+0000", 2024-09-04T17:46:53.485 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:53.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.485 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.486 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:53.486 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235206+0000", 2024-09-04T17:46:53.486 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:53.486 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.486 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.486 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:53.486 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235206+0000", 2024-09-04T17:46:53.486 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5300000000000004e-07 2024-09-04T17:46:53.486 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.486 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.486 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:53.486 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235209+0000", 2024-09-04T17:46:53.486 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6089999999999999e-06 2024-09-04T17:46:53.486 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.487 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.487 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:53.487 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235216+0000", 2024-09-04T17:46:53.487 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.277e-06 2024-09-04T17:46:53.487 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.487 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.487 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:53.487 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235221+0000", 2024-09-04T17:46:53.487 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2530000000000004e-06 2024-09-04T17:46:53.487 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.487 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.487 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:53.487 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289936+0000", 2024-09-04T17:46:53.488 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054714888000000003 2024-09-04T17:46:53.488 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.488 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.488 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:53.488 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290035+0000", 2024-09-04T17:46:53.488 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8652000000000003e-05 2024-09-04T17:46:53.488 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.488 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.488 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:53.488 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290190+0000", 2024-09-04T17:46:53.488 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015483399999999999 2024-09-04T17:46:53.488 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:53.488 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:53.489 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:53.489 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.489 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.489 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15537 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 1323008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:53.489 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235264+0000", 2024-09-04T17:46:53.489 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 35.179472924999999, 2024-09-04T17:46:53.489 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055255669, 2024-09-04T17:46:53.489 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:53.489 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:53.489 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:53.489 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:53.489 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:53.489 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:53.489 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15537 2024-09-04T17:46:53.490 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.490 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:53.490 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.490 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:53.490 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235264+0000", 2024-09-04T17:46:53.490 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:53.490 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.490 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.490 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:53.490 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235264+0000", 2024-09-04T17:46:53.490 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:53.490 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.490 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.491 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:53.491 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235264+0000", 2024-09-04T17:46:53.491 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4499999999999997e-07 2024-09-04T17:46:53.491 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.491 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.491 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:53.491 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235267+0000", 2024-09-04T17:46:53.491 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9210000000000001e-06 2024-09-04T17:46:53.491 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.491 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.491 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:53.491 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235276+0000", 2024-09-04T17:46:53.491 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2309999999999996e-06 2024-09-04T17:46:53.491 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.492 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.492 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:53.492 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235281+0000", 2024-09-04T17:46:53.492 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8560000000000003e-06 2024-09-04T17:46:53.492 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.493 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.493 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:53.493 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290246+0000", 2024-09-04T17:46:53.493 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054964528999999998 2024-09-04T17:46:53.493 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.493 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.493 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:53.493 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290344+0000", 2024-09-04T17:46:53.493 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8073999999999996e-05 2024-09-04T17:46:53.493 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.493 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.493 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:53.493 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290519+0000", 2024-09-04T17:46:53.494 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000175413 2024-09-04T17:46:53.494 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:53.494 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:53.494 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:53.494 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.494 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.494 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15538 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 1032192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:53.494 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235290+0000", 2024-09-04T17:46:53.494 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 35.179447015999997, 2024-09-04T17:46:53.494 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055518932, 2024-09-04T17:46:53.494 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:53.494 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:53.494 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:53.495 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:53.495 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:53.495 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:53.495 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15538 2024-09-04T17:46:53.495 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.495 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:53.495 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.495 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:53.495 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235290+0000", 2024-09-04T17:46:53.495 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:53.495 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.495 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.495 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:53.495 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235290+0000", 2024-09-04T17:46:53.496 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:53.496 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.496 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.496 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:53.496 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235290+0000", 2024-09-04T17:46:53.496 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3399999999999999e-07 2024-09-04T17:46:53.496 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.496 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.496 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:53.496 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235292+0000", 2024-09-04T17:46:53.496 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2639999999999998e-06 2024-09-04T17:46:53.496 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.497 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.497 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:53.497 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235299+0000", 2024-09-04T17:46:53.497 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2029999999999997e-06 2024-09-04T17:46:53.497 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.497 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.497 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:53.497 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235305+0000", 2024-09-04T17:46:53.497 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7469999999999999e-06 2024-09-04T17:46:53.497 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.497 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.497 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:53.498 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290562+0000", 2024-09-04T17:46:53.498 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055256580999999999 2024-09-04T17:46:53.498 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.498 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.498 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:53.498 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290651+0000", 2024-09-04T17:46:53.498 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8953e-05 2024-09-04T17:46:53.498 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.498 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.498 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:53.498 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290809+0000", 2024-09-04T17:46:53.498 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015765000000000001 2024-09-04T17:46:53.499 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:53.499 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:53.499 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:53.499 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.499 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.499 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15539 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 1359872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:53.499 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235312+0000", 2024-09-04T17:46:53.499 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 35.179424275999999, 2024-09-04T17:46:53.499 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055809865, 2024-09-04T17:46:53.499 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:53.499 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:53.499 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:53.499 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:53.500 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:53.500 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:53.500 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15539 2024-09-04T17:46:53.500 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.500 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:53.500 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.500 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:53.500 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235312+0000", 2024-09-04T17:46:53.500 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:53.500 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.500 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.500 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:53.500 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235312+0000", 2024-09-04T17:46:53.501 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:53.501 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.501 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.501 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:53.501 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235313+0000", 2024-09-04T17:46:53.501 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9000000000000002e-07 2024-09-04T17:46:53.501 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.501 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.501 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:53.501 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235315+0000", 2024-09-04T17:46:53.501 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1270000000000001e-06 2024-09-04T17:46:53.501 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.501 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.502 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:53.502 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235320+0000", 2024-09-04T17:46:53.502 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5430000000000001e-06 2024-09-04T17:46:53.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.502 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:53.502 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235326+0000", 2024-09-04T17:46:53.502 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2039999999999996e-06 2024-09-04T17:46:53.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.502 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:53.502 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290858+0000", 2024-09-04T17:46:53.502 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055532440000000002 2024-09-04T17:46:53.503 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.503 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.503 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:53.503 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290953+0000", 2024-09-04T17:46:53.503 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5031000000000005e-05 2024-09-04T17:46:53.503 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.503 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.503 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:53.503 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.291122+0000", 2024-09-04T17:46:53.503 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016913 2024-09-04T17:46:53.503 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:53.503 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:53.503 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:53.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.504 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.504 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15540 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 1396736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:53.504 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235333+0000", 2024-09-04T17:46:53.504 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 35.179403768999997, 2024-09-04T17:46:53.504 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056080528999999997, 2024-09-04T17:46:53.504 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:53.504 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:53.504 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:53.504 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:53.504 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:53.504 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:53.504 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15540 2024-09-04T17:46:53.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.505 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:53.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.505 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:53.505 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235333+0000", 2024-09-04T17:46:53.505 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:53.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.505 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:53.505 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235333+0000", 2024-09-04T17:46:53.505 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:53.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.506 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:53.506 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235333+0000", 2024-09-04T17:46:53.506 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0999999999999999e-07 2024-09-04T17:46:53.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.506 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.506 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:53.506 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235336+0000", 2024-09-04T17:46:53.506 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.278e-06 2024-09-04T17:46:53.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.506 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.506 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:53.506 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235341+0000", 2024-09-04T17:46:53.506 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8359999999999997e-06 2024-09-04T17:46:53.507 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.507 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.507 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:53.507 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235346+0000", 2024-09-04T17:46:53.507 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4329999999999997e-06 2024-09-04T17:46:53.507 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.507 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.507 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:53.507 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.291167+0000", 2024-09-04T17:46:53.507 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055821603999999997 2024-09-04T17:46:53.507 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.507 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.507 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:53.508 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.291257+0000", 2024-09-04T17:46:53.508 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9349000000000001e-05 2024-09-04T17:46:53.508 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.508 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.508 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:53.508 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.291413+0000", 2024-09-04T17:46:53.508 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015661900000000001 2024-09-04T17:46:53.508 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:53.508 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:53.508 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:53.508 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.509 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.509 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15541 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 1433600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:53.509 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235355+0000", 2024-09-04T17:46:53.509 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 35.179381581000001, 2024-09-04T17:46:53.509 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056371783000000002, 2024-09-04T17:46:53.509 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:53.509 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:53.509 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:46:53.510 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:53.510 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:53.510 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:53.510 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15541 2024-09-04T17:46:53.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.510 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:53.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.510 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:53.510 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235355+0000", 2024-09-04T17:46:53.510 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:53.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.510 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:53.511 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235355+0000", 2024-09-04T17:46:53.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:53.511 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.511 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.511 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:53.511 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235356+0000", 2024-09-04T17:46:53.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3099999999999998e-07 2024-09-04T17:46:53.511 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.511 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.511 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:53.511 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235358+0000", 2024-09-04T17:46:53.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3599999999999999e-06 2024-09-04T17:46:53.511 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.512 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.512 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:53.512 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235364+0000", 2024-09-04T17:46:53.512 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5760000000000002e-06 2024-09-04T17:46:53.512 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.512 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.512 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:53.512 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235370+0000", 2024-09-04T17:46:53.512 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0819999999999998e-06 2024-09-04T17:46:53.512 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.512 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.512 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:53.513 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.291462+0000", 2024-09-04T17:46:53.513 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056092301999999997 2024-09-04T17:46:53.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.513 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.513 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:53.513 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.291559+0000", 2024-09-04T17:46:53.513 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6988e-05 2024-09-04T17:46:53.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:53.513 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:53.513 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:53.513 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.291727+0000", 2024-09-04T17:46:53.513 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016804400000000001 2024-09-04T17:46:53.513 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:53.514 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:53.514 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:53.514 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:53.514 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:53.514 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:53.515 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-04T17:46:53.516 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:53.513+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:53.592 INFO:teuthology.orchestra.run.smithi092.stdout:2977 write 0x9c3124e thru 0x9c3c67b (0xb42e bytes) 2024-09-04T17:46:53.600 INFO:teuthology.orchestra.run.smithi092.stdout:2978 punch from 0x8200ccd to 0x820b02d, (0xa360 bytes) 2024-09-04T17:46:53.600 INFO:teuthology.orchestra.run.smithi092.stdout:2979 write 0x45f2fb8 thru 0x45fdba5 (0xabee bytes) 2024-09-04T17:46:53.606 INFO:teuthology.orchestra.run.smithi092.stdout:2980 read 0x2dc85e0 thru 0x2dd602f (0xda50 bytes) 2024-09-04T17:46:53.606 INFO:teuthology.orchestra.run.smithi092.stdout:2981 punch from 0x1f8d992 to 0x1f96fa7, (0x9615 bytes) 2024-09-04T17:46:53.616 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:53.613+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:53.623 INFO:teuthology.orchestra.run.smithi092.stdout:2982 write 0xb163822 thru 0xb16dd16 (0xa4f5 bytes) 2024-09-04T17:46:53.629 INFO:teuthology.orchestra.run.smithi092.stdout:2983 punch from 0x5c200d6 to 0x5c2834b, (0x8275 bytes) 2024-09-04T17:46:53.629 INFO:teuthology.orchestra.run.smithi092.stdout:2984 write 0x9b8898f thru 0x9b8b054 (0x26c6 bytes) 2024-09-04T17:46:53.635 INFO:teuthology.orchestra.run.smithi092.stdout:2985 punch from 0x6ac04 to 0x72075, (0x7471 bytes) 2024-09-04T17:46:53.635 INFO:teuthology.orchestra.run.smithi092.stdout:2986 writesame 0x122d0d9 thru 0x12379a0 data_size 0x2c(0xa8c8 bytes) 2024-09-04T17:46:53.643 INFO:teuthology.orchestra.run.smithi092.stdout:2987 write 0x58f4df4 thru 0x58f7d49 (0x2f56 bytes) 2024-09-04T17:46:53.668 INFO:teuthology.orchestra.run.smithi092.stdout:2988 trunc from 0xb16dd17 to 0x31ddc5e 2024-09-04T17:46:53.686 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:53.686 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:46:53.687 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:46:53.687 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:53.696 INFO:teuthology.orchestra.run.smithi092.stdout:2989 trunc from 0x31ddc5e to 0x21d4de8 2024-09-04T17:46:53.703 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-04T17:46:53.733 INFO:teuthology.orchestra.run.smithi092.stdout:2990 trunc from 0x21d4de8 to 0xa59466d 2024-09-04T17:46:53.733 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:53.731+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:53.745 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:53.745 INFO:teuthology.orchestra.run.smithi092.stdout:2994 write 0x8c7a306 thru 0x8c82143 (0x7e3e bytes) 2024-09-04T17:46:53.751 INFO:teuthology.orchestra.run.smithi092.stdout:2996 write 0x8de15a2 thru 0x8de8fe7 (0x7a46 bytes) 2024-09-04T17:46:53.758 INFO:teuthology.orchestra.run.smithi092.stdout:2999 read 0x254a01f thru 0x2551a94 (0x7a76 bytes) 2024-09-04T17:46:53.758 INFO:teuthology.orchestra.run.smithi092.stdout:3001 read 0x817f148 thru 0x818dddc (0xec95 bytes) 2024-09-04T17:46:53.759 INFO:teuthology.orchestra.run.smithi092.stdout:3002 write 0x3911fe5 thru 0x3920179 (0xe195 bytes) 2024-09-04T17:46:53.766 INFO:teuthology.orchestra.run.smithi092.stdout:3003 trunc from 0xa59466d to 0xa7f7c8f 2024-09-04T17:46:53.796 INFO:teuthology.orchestra.run.smithi092.stdout:3004 write 0xda5dddc thru 0xda5fbf0 (0x1e15 bytes) 2024-09-04T17:46:53.804 INFO:teuthology.orchestra.run.smithi092.stdout:3006 write 0x8f644dd thru 0x8f6ed73 (0xa897 bytes) 2024-09-04T17:46:53.817 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:53.817+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:53.825 INFO:teuthology.orchestra.run.smithi092.stdout:3007 write 0xec2a50d thru 0xec2bfc3 (0x1ab7 bytes) 2024-09-04T17:46:53.831 INFO:teuthology.orchestra.run.smithi092.stdout:3009 read 0xc75620b thru 0xc761da6 (0xbb9c bytes) 2024-09-04T17:46:53.831 INFO:teuthology.orchestra.run.smithi092.stdout:3010 punch from 0x4befa3b to 0x4bf3a68, (0x402d bytes) 2024-09-04T17:46:53.831 INFO:teuthology.orchestra.run.smithi092.stdout:3012 write 0x18c444f thru 0x18c450f (0xc1 bytes) 2024-09-04T17:46:53.837 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:53.838 INFO:teuthology.orchestra.run.smithi092.stdout:3014 trunc from 0xec2bfc4 to 0xe87e347 2024-09-04T17:46:53.858 INFO:teuthology.orchestra.run.smithi092.stdout:3016 read 0x62afaf4 thru 0x62b7cd0 (0x81dd bytes) 2024-09-04T17:46:53.858 INFO:teuthology.orchestra.run.smithi092.stdout:3017 write 0x49165e7 thru 0x491efba (0x89d4 bytes) 2024-09-04T17:46:53.864 INFO:teuthology.orchestra.run.smithi092.stdout:3019 write 0xe52eb93 thru 0xe537616 (0x8a84 bytes) 2024-09-04T17:46:53.871 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:53.871 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:46:53.871 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:46:53.871 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:46:53.871 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:53.872 INFO:teuthology.orchestra.run.smithi092.stdout:3020 read 0x37bd836 thru 0x37be817 (0xfe2 bytes) 2024-09-04T17:46:53.872 INFO:teuthology.orchestra.run.smithi092.stdout:3021 read 0x5c235c4 thru 0x5c291e9 (0x5c26 bytes) 2024-09-04T17:46:53.872 INFO:teuthology.orchestra.run.smithi092.stdout:3022 writesame 0x9f01061 thru 0x9f06b38 data_size 0x66(0x5ad8 bytes) 2024-09-04T17:46:53.878 INFO:teuthology.orchestra.run.smithi092.stdout:3023 read 0xd60b91b thru 0xd61a158 (0xe83e bytes) 2024-09-04T17:46:53.878 INFO:teuthology.orchestra.run.smithi092.stdout:3024 read 0x51155a1 thru 0x5122d15 (0xd775 bytes) 2024-09-04T17:46:53.879 INFO:teuthology.orchestra.run.smithi092.stdout:3025 read 0x709b953 thru 0x70a7467 (0xbb15 bytes) 2024-09-04T17:46:53.879 INFO:teuthology.orchestra.run.smithi092.stdout:3027 read 0x32e38b thru 0x33ae0f (0xca85 bytes) 2024-09-04T17:46:53.879 INFO:teuthology.orchestra.run.smithi092.stdout:3028 read 0x53ef4ae thru 0x53f7b04 (0x8657 bytes) 2024-09-04T17:46:53.879 INFO:teuthology.orchestra.run.smithi092.stdout:3029 punch from 0x9bd24f4 to 0x9bd9d15, (0x7821 bytes) 2024-09-04T17:46:53.879 INFO:teuthology.orchestra.run.smithi092.stdout:3030 read 0x7ccb4f9 thru 0x7cd08fb (0x5403 bytes) 2024-09-04T17:46:53.879 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:53.879 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:53.879 INFO:teuthology.orchestra.run.smithi092.stdout:3035 write 0xc37f794 thru 0xc38baa4 (0xc311 bytes) 2024-09-04T17:46:53.887 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:53.894 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:53.896 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-04T17:46:53.901 INFO:teuthology.orchestra.run.smithi092.stdout:3039 writesame 0xe9e2e16 thru 0xe9e7209 data_size 0x2(0x43f4 bytes) 2024-09-04T17:46:53.901 INFO:teuthology.orchestra.run.smithi092.stdout:3040 read 0x218be27 thru 0x218faa6 (0x3c80 bytes) 2024-09-04T17:46:53.902 INFO:teuthology.orchestra.run.smithi092.stdout:3042 writesame 0x41cc150 thru 0x41daba5 data_size 0x23(0xea56 bytes) 2024-09-04T17:46:53.910 INFO:teuthology.orchestra.run.smithi092.stdout:3043 writesame 0xbcfd7d7 thru 0xbd03ae5 data_size 0x3(0x630f bytes) 2024-09-04T17:46:53.918 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:53.917+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:53.922 INFO:teuthology.orchestra.run.smithi092.stdout:3044 write 0x3a00517 thru 0x3a01847 (0x1331 bytes) 2024-09-04T17:46:53.930 INFO:teuthology.orchestra.run.smithi092.stdout:3045 read 0x460f03f thru 0x461d7b9 (0xe77b bytes) 2024-09-04T17:46:53.930 INFO:teuthology.orchestra.run.smithi092.stdout:3046 punch from 0x58fbc58 to 0x58fee4d, (0x31f5 bytes) 2024-09-04T17:46:53.930 INFO:teuthology.orchestra.run.smithi092.stdout:3047 punch from 0x6141fd1 to 0x6150687, (0xe6b6 bytes) 2024-09-04T17:46:53.930 INFO:teuthology.orchestra.run.smithi092.stdout:3049 write 0xb826734 thru 0xb82fd46 (0x9613 bytes) 2024-09-04T17:46:53.936 INFO:teuthology.orchestra.run.smithi092.stdout:3050 write 0x6c17522 thru 0x6c1815f (0xc3e bytes) 2024-09-04T17:46:53.954 INFO:teuthology.orchestra.run.smithi092.stdout:3054 trunc from 0xe9e720a to 0x812a60c 2024-09-04T17:46:53.980 INFO:teuthology.orchestra.run.smithi092.stdout:3055 punch from 0x1018b6c to 0x101dd68, (0x51fc bytes) 2024-09-04T17:46:53.984 INFO:teuthology.orchestra.run.smithi092.stdout:3056 trunc from 0x812a60c to 0xad7374d 2024-09-04T17:46:53.996 INFO:teuthology.orchestra.run.smithi092.stdout:3057 write 0x7408700 thru 0x7414c5b (0xc55c bytes) 2024-09-04T17:46:54.019 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:54.017+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:54.019 INFO:teuthology.orchestra.run.smithi092.stdout:3058 trunc from 0xad7374d to 0x38976a 2024-09-04T17:46:54.042 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:54.043 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:46:54.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:46:54.043 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:46:54.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.043 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:4264 3.9 e30/24 3:970e51a0:::rbd_data.1134cfbce83d.0000000000000009:head v 30'83, mlcod=30'81)", 2024-09-04T17:46:54.043 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:07.027227+0000", 2024-09-04T17:46:54.043 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.014224132000002, 2024-09-04T17:46:54.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0096959590000000005, 2024-09-04T17:46:54.043 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:54.043 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:54.043 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:54.044 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:54.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:54.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.027227+0000", 2024-09-04T17:46:54.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:54.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:54.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.027227+0000", 2024-09-04T17:46:54.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:54.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:54.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.027229+0000", 2024-09-04T17:46:54.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4839999999999998e-06 2024-09-04T17:46:54.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:54.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.031984+0000", 2024-09-04T17:46:54.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047544789999999998 2024-09-04T17:46:54.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:54.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.032019+0000", 2024-09-04T17:46:54.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5253999999999998e-05 2024-09-04T17:46:54.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:54.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.032036+0000", 2024-09-04T17:46:54.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6713000000000001e-05 2024-09-04T17:46:54.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:54.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.032129+0000", 2024-09-04T17:46:54.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2961000000000007e-05 2024-09-04T17:46:54.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:54.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.032154+0000", 2024-09-04T17:46:54.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5148999999999999e-05 2024-09-04T17:46:54.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:54.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.036778+0000", 2024-09-04T17:46:54.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046239159999999996 2024-09-04T17:46:54.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:54.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.036923+0000", 2024-09-04T17:46:54.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014500299999999999 2024-09-04T17:46:54.047 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:54.047 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:54.048 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:54.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.048 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:4399 3.9 e31/24 3:970e51a0:::rbd_data.1134cfbce83d.0000000000000009:head v 31'87, mlcod=30'84)", 2024-09-04T17:46:54.048 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:07.343236+0000", 2024-09-04T17:46:54.048 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 46.698214448000002, 2024-09-04T17:46:54.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0036134539999999999, 2024-09-04T17:46:54.048 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:54.048 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:54.048 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:54.048 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:54.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:54.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.343236+0000", 2024-09-04T17:46:54.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:54.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:54.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.343236+0000", 2024-09-04T17:46:54.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:54.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:54.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.343237+0000", 2024-09-04T17:46:54.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4000000000000001e-07 2024-09-04T17:46:54.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:54.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.343250+0000", 2024-09-04T17:46:54.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2573e-05 2024-09-04T17:46:54.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:54.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.343279+0000", 2024-09-04T17:46:54.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9263000000000001e-05 2024-09-04T17:46:54.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:54.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.343299+0000", 2024-09-04T17:46:54.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9720000000000001e-05 2024-09-04T17:46:54.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:54.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.343389+0000", 2024-09-04T17:46:54.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9924000000000002e-05 2024-09-04T17:46:54.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:54.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.343413+0000", 2024-09-04T17:46:54.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4227999999999999e-05 2024-09-04T17:46:54.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:54.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.346796+0000", 2024-09-04T17:46:54.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0033831310000000002 2024-09-04T17:46:54.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:54.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.346850+0000", 2024-09-04T17:46:54.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3875000000000002e-05 2024-09-04T17:46:54.052 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:54.052 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:54.052 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:54.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.053 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:4881 3.4 e31/24 3:25597cfa:::rbd_data.1134cfbce83d.0000000000000006:head v 31'28, mlcod=31'26)", 2024-09-04T17:46:54.053 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:07.381336+0000", 2024-09-04T17:46:54.053 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 46.660114819, 2024-09-04T17:46:54.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013330164, 2024-09-04T17:46:54.053 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:54.053 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:54.053 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:54.053 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:54.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:54.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.381336+0000", 2024-09-04T17:46:54.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:54.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:54.054 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.381336+0000", 2024-09-04T17:46:54.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:54.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:54.054 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.381339+0000", 2024-09-04T17:46:54.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.287e-06 2024-09-04T17:46:54.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:54.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.384162+0000", 2024-09-04T17:46:54.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028228649999999999 2024-09-04T17:46:54.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:54.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.384209+0000", 2024-09-04T17:46:54.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6768999999999999e-05 2024-09-04T17:46:54.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:54.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.384225+0000", 2024-09-04T17:46:54.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5987000000000001e-05 2024-09-04T17:46:54.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:54.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.384372+0000", 2024-09-04T17:46:54.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000146809 2024-09-04T17:46:54.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:54.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.384409+0000", 2024-09-04T17:46:54.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6909999999999997e-05 2024-09-04T17:46:54.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:54.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.394535+0000", 2024-09-04T17:46:54.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010126575000000001 2024-09-04T17:46:54.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:54.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.394666+0000", 2024-09-04T17:46:54.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013096200000000001 2024-09-04T17:46:54.057 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:54.057 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:54.057 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:54.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.058 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:4884 3.4 e31/24 3:25597cfa:::rbd_data.1134cfbce83d.0000000000000006:head v 31'30, mlcod=31'26)", 2024-09-04T17:46:54.058 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:07.386970+0000", 2024-09-04T17:46:54.058 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 46.654480884999998, 2024-09-04T17:46:54.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0088306779999999998, 2024-09-04T17:46:54.058 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:54.058 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:54.058 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:54.058 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:54.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:54.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.386970+0000", 2024-09-04T17:46:54.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:54.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:54.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.386970+0000", 2024-09-04T17:46:54.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:54.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:54.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.386973+0000", 2024-09-04T17:46:54.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9229999999999998e-06 2024-09-04T17:46:54.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:54.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.386993+0000", 2024-09-04T17:46:54.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.054e-05 2024-09-04T17:46:54.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.060 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:54.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.387029+0000", 2024-09-04T17:46:54.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5753000000000001e-05 2024-09-04T17:46:54.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.060 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:54.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.387051+0000", 2024-09-04T17:46:54.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1552999999999998e-05 2024-09-04T17:46:54.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:54.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.388844+0000", 2024-09-04T17:46:54.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001793712 2024-09-04T17:46:54.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:54.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.388866+0000", 2024-09-04T17:46:54.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1123000000000001e-05 2024-09-04T17:46:54.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:54.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.395711+0000", 2024-09-04T17:46:54.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0068458859999999998 2024-09-04T17:46:54.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:54.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.395801+0000", 2024-09-04T17:46:54.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9188000000000001e-05 2024-09-04T17:46:54.062 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:54.062 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:54.062 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:54.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.062 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:5000 3.9 e31/24 3:94906c79:::rbd_data.1134cfbce83d.0000000000000012:head v 31'91, mlcod=30'85)", 2024-09-04T17:46:54.062 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:07.402507+0000", 2024-09-04T17:46:54.062 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 46.638943505, 2024-09-04T17:46:54.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004039735, 2024-09-04T17:46:54.063 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:54.063 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:54.063 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:54.063 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:54.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:54.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.402507+0000", 2024-09-04T17:46:54.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:54.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:54.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.402507+0000", 2024-09-04T17:46:54.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:54.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:54.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.402509+0000", 2024-09-04T17:46:54.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5939999999999999e-06 2024-09-04T17:46:54.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:54.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.402521+0000", 2024-09-04T17:46:54.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1786e-05 2024-09-04T17:46:54.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:54.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.402543+0000", 2024-09-04T17:46:54.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2158e-05 2024-09-04T17:46:54.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:54.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.402555+0000", 2024-09-04T17:46:54.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2034e-05 2024-09-04T17:46:54.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:54.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.402660+0000", 2024-09-04T17:46:54.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010535900000000001 2024-09-04T17:46:54.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:54.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.402696+0000", 2024-09-04T17:46:54.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5488000000000003e-05 2024-09-04T17:46:54.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:54.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.406510+0000", 2024-09-04T17:46:54.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0038142390000000001 2024-09-04T17:46:54.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:54.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.406547+0000", 2024-09-04T17:46:54.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7076999999999998e-05 2024-09-04T17:46:54.067 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:54.067 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:54.067 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:54.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.067 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:6311 3.9 e32/24 3:941d6897:::rbd_data.113431586065.0000000000000008:head v 32'97, mlcod=32'95)", 2024-09-04T17:46:54.067 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.479986+0000", 2024-09-04T17:46:54.067 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 45.561464608000001, 2024-09-04T17:46:54.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018493539, 2024-09-04T17:46:54.067 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:54.068 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:54.068 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:54.068 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:54.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:54.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.479986+0000", 2024-09-04T17:46:54.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:54.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:54.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.479986+0000", 2024-09-04T17:46:54.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:54.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:54.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.479988+0000", 2024-09-04T17:46:54.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.193e-06 2024-09-04T17:46:54.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:54.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484109+0000", 2024-09-04T17:46:54.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041201650000000003 2024-09-04T17:46:54.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:54.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484139+0000", 2024-09-04T17:46:54.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.082e-05 2024-09-04T17:46:54.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:54.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484153+0000", 2024-09-04T17:46:54.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3456e-05 2024-09-04T17:46:54.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:54.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484250+0000", 2024-09-04T17:46:54.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7638000000000003e-05 2024-09-04T17:46:54.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:54.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484282+0000", 2024-09-04T17:46:54.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1511999999999999e-05 2024-09-04T17:46:54.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:54.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.498411+0000", 2024-09-04T17:46:54.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014128870999999999 2024-09-04T17:46:54.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:54.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.498480+0000", 2024-09-04T17:46:54.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8884e-05 2024-09-04T17:46:54.071 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:54.072 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:54.072 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:54.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.072 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:6312 3.9 e32/24 3:941d6897:::rbd_data.113431586065.0000000000000008:head v 32'99, mlcod=32'95)", 2024-09-04T17:46:54.072 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.486036+0000", 2024-09-04T17:46:54.072 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 45.55541453, 2024-09-04T17:46:54.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013432773, 2024-09-04T17:46:54.072 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:54.073 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:54.073 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:54.073 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:54.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:54.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.486036+0000", 2024-09-04T17:46:54.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:54.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:54.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.486036+0000", 2024-09-04T17:46:54.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:54.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:54.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.486038+0000", 2024-09-04T17:46:54.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.536e-06 2024-09-04T17:46:54.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:54.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.486062+0000", 2024-09-04T17:46:54.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3672e-05 2024-09-04T17:46:54.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:54.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.486078+0000", 2024-09-04T17:46:54.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.613e-05 2024-09-04T17:46:54.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:54.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.486086+0000", 2024-09-04T17:46:54.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8849999999999999e-06 2024-09-04T17:46:54.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:54.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.491843+0000", 2024-09-04T17:46:54.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057572040000000001 2024-09-04T17:46:54.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:54.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.491876+0000", 2024-09-04T17:46:54.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3515e-05 2024-09-04T17:46:54.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:54.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.499410+0000", 2024-09-04T17:46:54.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0075333780000000003 2024-09-04T17:46:54.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:54.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.499469+0000", 2024-09-04T17:46:54.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9453000000000002e-05 2024-09-04T17:46:54.077 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:54.077 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:54.077 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:54.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.077 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:12747 3.1f e41/37 3:fb91fe3f:::rbd_object_map.1134ef3a5a7d:head v 41'125, mlcod=41'122)", 2024-09-04T17:46:54.077 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:17.330322+0000", 2024-09-04T17:46:54.077 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.711129079000003, 2024-09-04T17:46:54.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004205762, 2024-09-04T17:46:54.077 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:54.077 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:54.077 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:54.078 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:54.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:54.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.330322+0000", 2024-09-04T17:46:54.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:54.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:54.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.330322+0000", 2024-09-04T17:46:54.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:54.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:54.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.330323+0000", 2024-09-04T17:46:54.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9500000000000001e-07 2024-09-04T17:46:54.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:54.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.330329+0000", 2024-09-04T17:46:54.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2040000000000002e-06 2024-09-04T17:46:54.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:54.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.330341+0000", 2024-09-04T17:46:54.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1727e-05 2024-09-04T17:46:54.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:54.080 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.330348+0000", 2024-09-04T17:46:54.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.187e-06 2024-09-04T17:46:54.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:54.080 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.330455+0000", 2024-09-04T17:46:54.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107311 2024-09-04T17:46:54.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:54.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.330495+0000", 2024-09-04T17:46:54.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0191999999999997e-05 2024-09-04T17:46:54.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:54.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.334480+0000", 2024-09-04T17:46:54.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039848410000000003 2024-09-04T17:46:54.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:54.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.334528+0000", 2024-09-04T17:46:54.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7404999999999998e-05 2024-09-04T17:46:54.082 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:54.082 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:54.082 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:54.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.082 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:12748 3.1f e41/37 3:fb91fe3f:::rbd_object_map.1134ef3a5a7d:head v 41'126, mlcod=41'122)", 2024-09-04T17:46:54.082 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:17.332613+0000", 2024-09-04T17:46:54.082 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.708838100000001, 2024-09-04T17:46:54.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0044275870000000002, 2024-09-04T17:46:54.082 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:54.082 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:54.082 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:54.082 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:54.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.083 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:54.083 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.332613+0000", 2024-09-04T17:46:54.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:54.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.083 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:54.083 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.332613+0000", 2024-09-04T17:46:54.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:54.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:54.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.332614+0000", 2024-09-04T17:46:54.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.051e-06 2024-09-04T17:46:54.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:54.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.332620+0000", 2024-09-04T17:46:54.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1680000000000001e-06 2024-09-04T17:46:54.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:54.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.332635+0000", 2024-09-04T17:46:54.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5543e-05 2024-09-04T17:46:54.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:54.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.332643+0000", 2024-09-04T17:46:54.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.537e-06 2024-09-04T17:46:54.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:54.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.332717+0000", 2024-09-04T17:46:54.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3701999999999993e-05 2024-09-04T17:46:54.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:54.086 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.332786+0000", 2024-09-04T17:46:54.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9669e-05 2024-09-04T17:46:54.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:54.086 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.336923+0000", 2024-09-04T17:46:54.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041365259999999997 2024-09-04T17:46:54.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:54.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.337040+0000", 2024-09-04T17:46:54.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000117391 2024-09-04T17:46:54.087 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:54.087 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:54.087 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:54.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.087 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:12749 3.1f e41/37 3:fb91fe3f:::rbd_object_map.1134ef3a5a7d:head v 41'127, mlcod=41'122)", 2024-09-04T17:46:54.087 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:17.335529+0000", 2024-09-04T17:46:54.087 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.705922059000002, 2024-09-04T17:46:54.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052795899999999998, 2024-09-04T17:46:54.088 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:54.088 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:54.088 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:54.088 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:54.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:54.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.335529+0000", 2024-09-04T17:46:54.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:54.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:54.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.335529+0000", 2024-09-04T17:46:54.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:54.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:54.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.335535+0000", 2024-09-04T17:46:54.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4060000000000002e-06 2024-09-04T17:46:54.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:54.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.335546+0000", 2024-09-04T17:46:54.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0652000000000001e-05 2024-09-04T17:46:54.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:54.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.335571+0000", 2024-09-04T17:46:54.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5148e-05 2024-09-04T17:46:54.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:54.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.335583+0000", 2024-09-04T17:46:54.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2315e-05 2024-09-04T17:46:54.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:54.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.335699+0000", 2024-09-04T17:46:54.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000115443 2024-09-04T17:46:54.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:54.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.335736+0000", 2024-09-04T17:46:54.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6779999999999997e-05 2024-09-04T17:46:54.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:54.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.340722+0000", 2024-09-04T17:46:54.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0049860390000000003 2024-09-04T17:46:54.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:54.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.340808+0000", 2024-09-04T17:46:54.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6806999999999997e-05 2024-09-04T17:46:54.092 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:54.092 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:54.092 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:54.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.093 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:12750 3.1f e41/37 3:fb91fe3f:::rbd_object_map.1134ef3a5a7d:head v 41'128, mlcod=41'123)", 2024-09-04T17:46:54.093 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:17.337581+0000", 2024-09-04T17:46:54.093 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.703869515000001, 2024-09-04T17:46:54.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045169579999999997, 2024-09-04T17:46:54.093 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:54.093 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:54.093 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:54.093 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:54.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:54.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.337581+0000", 2024-09-04T17:46:54.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:54.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:54.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.337581+0000", 2024-09-04T17:46:54.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:54.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:54.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.337583+0000", 2024-09-04T17:46:54.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5200000000000001e-06 2024-09-04T17:46:54.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:54.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.337595+0000", 2024-09-04T17:46:54.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2395000000000001e-05 2024-09-04T17:46:54.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:54.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.337620+0000", 2024-09-04T17:46:54.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4794e-05 2024-09-04T17:46:54.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:54.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.337634+0000", 2024-09-04T17:46:54.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4241e-05 2024-09-04T17:46:54.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:54.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.337731+0000", 2024-09-04T17:46:54.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6246e-05 2024-09-04T17:46:54.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:54.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.337762+0000", 2024-09-04T17:46:54.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1279999999999999e-05 2024-09-04T17:46:54.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:54.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.342050+0000", 2024-09-04T17:46:54.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0042879729999999996 2024-09-04T17:46:54.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:54.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.342098+0000", 2024-09-04T17:46:54.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8508999999999999e-05 2024-09-04T17:46:54.097 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:54.097 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:54.097 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:54.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.098 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:12756 3.0 e41/24 3:0074e6f8:::rbd_data.1134ef3a5a7d.000000000000000e:head v 41'24, mlcod=40'20)", 2024-09-04T17:46:54.098 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:17.339316+0000", 2024-09-04T17:46:54.098 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.702135292000001, 2024-09-04T17:46:54.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052778790000000001, 2024-09-04T17:46:54.098 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:54.098 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:54.098 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:54.098 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:54.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:54.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.339316+0000", 2024-09-04T17:46:54.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:54.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:54.099 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.339316+0000", 2024-09-04T17:46:54.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:54.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:54.099 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.339317+0000", 2024-09-04T17:46:54.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.13e-06 2024-09-04T17:46:54.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:54.099 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.339328+0000", 2024-09-04T17:46:54.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1742e-05 2024-09-04T17:46:54.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:54.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.339358+0000", 2024-09-04T17:46:54.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9149000000000002e-05 2024-09-04T17:46:54.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:54.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.339379+0000", 2024-09-04T17:46:54.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1288e-05 2024-09-04T17:46:54.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:54.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.339454+0000", 2024-09-04T17:46:54.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4703000000000006e-05 2024-09-04T17:46:54.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:54.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.339506+0000", 2024-09-04T17:46:54.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2188999999999998e-05 2024-09-04T17:46:54.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:54.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.344475+0000", 2024-09-04T17:46:54.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0049691919999999999 2024-09-04T17:46:54.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:54.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.344593+0000", 2024-09-04T17:46:54.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000118486 2024-09-04T17:46:54.102 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:54.102 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:54.102 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:54.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.103 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:27957 3.1f e50/37 3:ff7930c6:::rbd_data.1134eedad849.0000000000000380:head v 50'169, mlcod=50'165)", 2024-09-04T17:46:54.103 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:30.598047+0000", 2024-09-04T17:46:54.103 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 23.443403731, 2024-09-04T17:46:54.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0042527729999999996, 2024-09-04T17:46:54.103 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:54.103 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:54.103 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:54.103 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:54.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.103 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:54.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.598047+0000", 2024-09-04T17:46:54.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:54.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.104 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:54.104 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.598047+0000", 2024-09-04T17:46:54.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:54.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.104 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:54.104 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.598048+0000", 2024-09-04T17:46:54.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7800000000000001e-07 2024-09-04T17:46:54.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.104 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:54.105 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.598055+0000", 2024-09-04T17:46:54.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1069999999999999e-06 2024-09-04T17:46:54.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.105 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:54.105 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.598067+0000", 2024-09-04T17:46:54.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2278e-05 2024-09-04T17:46:54.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.105 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:54.105 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.598072+0000", 2024-09-04T17:46:54.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8439999999999997e-06 2024-09-04T17:46:54.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:54.106 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.598114+0000", 2024-09-04T17:46:54.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1931000000000002e-05 2024-09-04T17:46:54.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:54.106 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.598133+0000", 2024-09-04T17:46:54.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9102000000000001e-05 2024-09-04T17:46:54.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:54.106 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.602251+0000", 2024-09-04T17:46:54.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041175079999999998 2024-09-04T17:46:54.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:54.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.602300+0000", 2024-09-04T17:46:54.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9224999999999997e-05 2024-09-04T17:46:54.107 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:54.107 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:54.107 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:54.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.107 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:27958 3.1f e50/37 3:fa2ba8de:::rbd_data.1134eedad849.0000000000000384:head v 50'170, mlcod=50'166)", 2024-09-04T17:46:54.107 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:30.598748+0000", 2024-09-04T17:46:54.107 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 23.442703249000001, 2024-09-04T17:46:54.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0035807859999999999, 2024-09-04T17:46:54.108 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:54.108 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:54.108 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:54.108 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:54.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:54.108 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.598748+0000", 2024-09-04T17:46:54.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:54.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:54.109 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.598748+0000", 2024-09-04T17:46:54.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:54.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.109 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:54.109 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.598748+0000", 2024-09-04T17:46:54.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2399999999999997e-07 2024-09-04T17:46:54.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.109 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:54.109 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.598756+0000", 2024-09-04T17:46:54.110 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4089999999999997e-06 2024-09-04T17:46:54.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.110 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.110 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:54.110 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.598767+0000", 2024-09-04T17:46:54.110 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1449e-05 2024-09-04T17:46:54.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.110 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.110 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:54.110 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.598772+0000", 2024-09-04T17:46:54.110 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7790000000000003e-06 2024-09-04T17:46:54.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.110 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.111 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:54.111 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.598831+0000", 2024-09-04T17:46:54.111 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9091000000000001e-05 2024-09-04T17:46:54.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.111 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:54.111 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.598843+0000", 2024-09-04T17:46:54.111 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1691999999999999e-05 2024-09-04T17:46:54.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.111 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:54.111 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.602302+0000", 2024-09-04T17:46:54.112 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0034595609999999999 2024-09-04T17:46:54.112 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.112 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.112 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:54.112 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.602328+0000", 2024-09-04T17:46:54.112 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5981000000000001e-05 2024-09-04T17:46:54.112 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:54.112 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:54.112 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:54.112 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.112 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.112 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "pg_update_log_missing(3.0 epoch 50/24 rep_tid 378 entries 50'73 (0'0) error 3:01dff485:::rbd_data.1134eedad849.000000000000038b:head by client.4404.0:27967 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 50'71)", 2024-09-04T17:46:54.113 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:30.600491+0000", 2024-09-04T17:46:54.113 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 23.440959358000001, 2024-09-04T17:46:54.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045188329999999999, 2024-09-04T17:46:54.113 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:54.113 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:54.113 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-04T17:46:54.113 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:54.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:54.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.600491+0000", 2024-09-04T17:46:54.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:54.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:54.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.600491+0000", 2024-09-04T17:46:54.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:54.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:54.117 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.600492+0000", 2024-09-04T17:46:54.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3500000000000005e-07 2024-09-04T17:46:54.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:54.117 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.600510+0000", 2024-09-04T17:46:54.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7533999999999998e-05 2024-09-04T17:46:54.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:54.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.600536+0000", 2024-09-04T17:46:54.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5822000000000001e-05 2024-09-04T17:46:54.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.118 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:54.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.600556+0000", 2024-09-04T17:46:54.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9913000000000001e-05 2024-09-04T17:46:54.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.118 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:54.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.600644+0000", 2024-09-04T17:46:54.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8282999999999995e-05 2024-09-04T17:46:54.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:54.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.605010+0000", 2024-09-04T17:46:54.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0043664460000000004 2024-09-04T17:46:54.119 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:54.119 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:54.119 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:54.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.119 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:34444 3.1f e56/37 3:fcc2bddb:::rbd_data.11342e8f3384.0000000000000004:head v 56'189, mlcod=54'187)", 2024-09-04T17:46:54.119 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:36.632281+0000", 2024-09-04T17:46:54.119 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.409170064000001, 2024-09-04T17:46:54.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0036502420000000002, 2024-09-04T17:46:54.120 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:54.120 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:54.120 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:54.120 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:54.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.120 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:54.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.632281+0000", 2024-09-04T17:46:54.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:54.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.120 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:54.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.632281+0000", 2024-09-04T17:46:54.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:54.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.121 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:54.121 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.632282+0000", 2024-09-04T17:46:54.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5300000000000003e-07 2024-09-04T17:46:54.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.121 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:54.121 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.633663+0000", 2024-09-04T17:46:54.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0013816169999999999 2024-09-04T17:46:54.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.121 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:54.121 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.633682+0000", 2024-09-04T17:46:54.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8624e-05 2024-09-04T17:46:54.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:54.122 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.633691+0000", 2024-09-04T17:46:54.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3260000000000002e-06 2024-09-04T17:46:54.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:54.122 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.633766+0000", 2024-09-04T17:46:54.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5075999999999997e-05 2024-09-04T17:46:54.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.123 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:54.123 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.633793+0000", 2024-09-04T17:46:54.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6939000000000001e-05 2024-09-04T17:46:54.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.123 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:54.123 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.635872+0000", 2024-09-04T17:46:54.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020786820000000001 2024-09-04T17:46:54.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.123 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:54.123 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.635931+0000", 2024-09-04T17:46:54.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9125000000000001e-05 2024-09-04T17:46:54.124 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:54.124 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:54.124 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:54.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.124 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:37418 3.1f e58/37 3:ff114f66:::rbd_data.1134c5a43465.00000000000000da:head v 58'200, mlcod=58'198)", 2024-09-04T17:46:54.124 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:39.727221+0000", 2024-09-04T17:46:54.124 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.314229378, 2024-09-04T17:46:54.124 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040617930000000002, 2024-09-04T17:46:54.124 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:54.124 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:54.124 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:54.125 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:54.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.125 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:54.125 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.727221+0000", 2024-09-04T17:46:54.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:54.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.125 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:54.125 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.727221+0000", 2024-09-04T17:46:54.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:54.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.125 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:54.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.727223+0000", 2024-09-04T17:46:54.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.88e-06 2024-09-04T17:46:54.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.126 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:54.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.727239+0000", 2024-09-04T17:46:54.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5962000000000001e-05 2024-09-04T17:46:54.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.126 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:54.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.727278+0000", 2024-09-04T17:46:54.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8844999999999998e-05 2024-09-04T17:46:54.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:54.127 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.727295+0000", 2024-09-04T17:46:54.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7192000000000001e-05 2024-09-04T17:46:54.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:54.127 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.727474+0000", 2024-09-04T17:46:54.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000178176 2024-09-04T17:46:54.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:54.127 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.727536+0000", 2024-09-04T17:46:54.128 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2212999999999995e-05 2024-09-04T17:46:54.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.128 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:54.128 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.731172+0000", 2024-09-04T17:46:54.128 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0036367050000000001 2024-09-04T17:46:54.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.128 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:54.128 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.731283+0000", 2024-09-04T17:46:54.128 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011082 2024-09-04T17:46:54.128 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:54.128 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:54.129 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:54.129 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.129 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:37420 3.9 e58/24 3:940e2e4e:::rbd_data.1134c5a43465.00000000000000dc:head v 58'215, mlcod=58'213)", 2024-09-04T17:46:54.129 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:39.728565+0000", 2024-09-04T17:46:54.129 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.312886031, 2024-09-04T17:46:54.129 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039183309999999997, 2024-09-04T17:46:54.129 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:54.129 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:54.129 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:54.129 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:54.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.130 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:54.130 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.728565+0000", 2024-09-04T17:46:54.130 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:54.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.130 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.130 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:54.130 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.728565+0000", 2024-09-04T17:46:54.130 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:54.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.130 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.130 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:54.130 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.728567+0000", 2024-09-04T17:46:54.130 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7230000000000001e-06 2024-09-04T17:46:54.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:54.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.728579+0000", 2024-09-04T17:46:54.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2274e-05 2024-09-04T17:46:54.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:54.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.728613+0000", 2024-09-04T17:46:54.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4326000000000003e-05 2024-09-04T17:46:54.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:54.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.728628+0000", 2024-09-04T17:46:54.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4882000000000001e-05 2024-09-04T17:46:54.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.132 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:54.132 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.728793+0000", 2024-09-04T17:46:54.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016519000000000001 2024-09-04T17:46:54.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.132 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:54.132 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.728839+0000", 2024-09-04T17:46:54.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5343000000000003e-05 2024-09-04T17:46:54.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.133 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:54.133 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.732380+0000", 2024-09-04T17:46:54.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0035412849999999999 2024-09-04T17:46:54.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.133 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:54.133 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.732483+0000", 2024-09-04T17:46:54.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010330799999999999 2024-09-04T17:46:54.133 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:54.133 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:54.133 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:54.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.134 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:46594 3.9 3:97706e40:::rbd_data.1134326b938e.0000000000000013:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-04T17:46:54.134 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:54.025375+0000", 2024-09-04T17:46:54.134 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.016076162000000001, 2024-09-04T17:46:54.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0082115499999999998, 2024-09-04T17:46:54.134 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:54.134 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:54.134 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:54.134 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:54.134 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:54.134 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:54.134 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 46594 2024-09-04T17:46:54.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.134 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:54.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.135 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:54.135 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.025375+0000", 2024-09-04T17:46:54.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:54.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.135 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:54.135 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.025375+0000", 2024-09-04T17:46:54.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:54.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.135 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:54.135 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.025379+0000", 2024-09-04T17:46:54.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0439999999999997e-06 2024-09-04T17:46:54.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:54.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.025393+0000", 2024-09-04T17:46:54.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4456e-05 2024-09-04T17:46:54.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:54.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.025443+0000", 2024-09-04T17:46:54.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9904000000000002e-05 2024-09-04T17:46:54.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:54.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.025467+0000", 2024-09-04T17:46:54.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4419000000000001e-05 2024-09-04T17:46:54.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:54.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.025637+0000", 2024-09-04T17:46:54.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016905699999999999 2024-09-04T17:46:54.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:54.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.025936+0000", 2024-09-04T17:46:54.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00029941499999999998 2024-09-04T17:46:54.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-04T17:46:54.138 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.026229+0000", 2024-09-04T17:46:54.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00029280499999999999 2024-09-04T17:46:54.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:54.138 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.029491+0000", 2024-09-04T17:46:54.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003262596 2024-09-04T17:46:54.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:46:54.138 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.033531+0000", 2024-09-04T17:46:54.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040393779999999997 2024-09-04T17:46:54.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.139 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:54.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.033558+0000", 2024-09-04T17:46:54.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7432000000000001e-05 2024-09-04T17:46:54.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.139 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:54.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.033586+0000", 2024-09-04T17:46:54.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8044000000000001e-05 2024-09-04T17:46:54.139 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:54.139 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:54.139 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:54.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.140 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:46600 3.9 3:966a74f3:::rbd_data.1134326b938e.000000000000002e:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-04T17:46:54.140 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:54.030201+0000", 2024-09-04T17:46:54.140 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.011249507000000001, 2024-09-04T17:46:54.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066468320000000001, 2024-09-04T17:46:54.140 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:54.140 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:54.140 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:54.140 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:54.140 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:54.140 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:54.140 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 46600 2024-09-04T17:46:54.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.141 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:54.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:54.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.030201+0000", 2024-09-04T17:46:54.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:54.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:54.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.030201+0000", 2024-09-04T17:46:54.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:54.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.142 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:54.142 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.030204+0000", 2024-09-04T17:46:54.142 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.627e-06 2024-09-04T17:46:54.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.142 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:54.142 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.030219+0000", 2024-09-04T17:46:54.142 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4746000000000001e-05 2024-09-04T17:46:54.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.142 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:54.142 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.030254+0000", 2024-09-04T17:46:54.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5024999999999998e-05 2024-09-04T17:46:54.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:54.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.030275+0000", 2024-09-04T17:46:54.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1058999999999999e-05 2024-09-04T17:46:54.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:54.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.033026+0000", 2024-09-04T17:46:54.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0027510619999999999 2024-09-04T17:46:54.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:54.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.033104+0000", 2024-09-04T17:46:54.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7909000000000002e-05 2024-09-04T17:46:54.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-04T17:46:54.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.033202+0000", 2024-09-04T17:46:54.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7776999999999996e-05 2024-09-04T17:46:54.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:46:54.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.034081+0000", 2024-09-04T17:46:54.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00087908699999999997 2024-09-04T17:46:54.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:54.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.036762+0000", 2024-09-04T17:46:54.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0026814080000000001 2024-09-04T17:46:54.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:54.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.036811+0000", 2024-09-04T17:46:54.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9197999999999999e-05 2024-09-04T17:46:54.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:54.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:54.146 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:54.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.036848+0000", 2024-09-04T17:46:54.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6934000000000002e-05 2024-09-04T17:46:54.146 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:54.146 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:54.146 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:54.146 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:54.146 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:54.146 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:54.147 INFO:teuthology.orchestra.run.smithi092.stdout:3060 read 0x175a2c thru 0x17b119 (0x56ee bytes) 2024-09-04T17:46:54.148 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:54.149 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-04T17:46:54.150 INFO:teuthology.orchestra.run.smithi092.stdout:3062 trunc from 0x38976a to 0xb6e8d56 2024-09-04T17:46:54.150 INFO:teuthology.orchestra.run.smithi092.stdout:3064 read 0x276f8df thru 0x2779668 (0x9d8a bytes) 2024-09-04T17:46:54.150 INFO:teuthology.orchestra.run.smithi092.stdout:3065 punch from 0x67547c8 to 0x675e9a4, (0xa1dc bytes) 2024-09-04T17:46:54.150 INFO:teuthology.orchestra.run.smithi092.stdout:3066 write 0xe6b13e9 thru 0xe6b73c6 (0x5fde bytes) 2024-09-04T17:46:54.150 INFO:teuthology.orchestra.run.smithi092.stdout:3068 writesame 0xa7d0061 thru 0xa7d724f data_size 0x1(0x71ef bytes) 2024-09-04T17:46:54.150 INFO:teuthology.orchestra.run.smithi092.stdout:3069 punch from 0xbec5b1b to 0xbec9c88, (0x416d bytes) 2024-09-04T17:46:54.150 INFO:teuthology.orchestra.run.smithi092.stdout:3070 write 0x2e02ed7 thru 0x2e0b1ea (0x8314 bytes) 2024-09-04T17:46:54.150 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:54.147+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:54.153 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:54.153 INFO:teuthology.orchestra.run.smithi092.stdout:3072 write 0x87989af thru 0x87a097c (0x7fce bytes) 2024-09-04T17:46:54.161 INFO:teuthology.orchestra.run.smithi092.stdout:3073 write 0x154bf3c thru 0x155937a (0xd43f bytes) 2024-09-04T17:46:54.171 INFO:teuthology.orchestra.run.smithi092.stdout:3074 read 0xde6bb60 thru 0xde78ef4 (0xd395 bytes) 2024-09-04T17:46:54.171 INFO:teuthology.orchestra.run.smithi092.stdout:3075 punch from 0x51f8ce5 to 0x520120a, (0x8525 bytes) 2024-09-04T17:46:54.171 INFO:teuthology.orchestra.run.smithi092.stdout:3076 write 0x966eeff thru 0x96778db (0x89dd bytes) 2024-09-04T17:46:54.179 INFO:teuthology.orchestra.run.smithi092.stdout:3077 write 0x679e7a0 thru 0x67a6a50 (0x82b1 bytes) 2024-09-04T17:46:54.187 INFO:teuthology.orchestra.run.smithi092.stdout:3078 read 0x53e2823 thru 0x53e7ef7 (0x56d5 bytes) 2024-09-04T17:46:54.187 INFO:teuthology.orchestra.run.smithi092.stdout:3079 punch from 0x42fb49a to 0x43022f8, (0x6e5e bytes) 2024-09-04T17:46:54.187 INFO:teuthology.orchestra.run.smithi092.stdout:3080 punch from 0x83e9613 to 0x83f07ec, (0x71d9 bytes) 2024-09-04T17:46:54.187 INFO:teuthology.orchestra.run.smithi092.stdout:3081 read 0x45da173 thru 0x45dac34 (0xac2 bytes) 2024-09-04T17:46:54.187 INFO:teuthology.orchestra.run.smithi092.stdout:3084 read 0x2c695c thru 0x2d15e4 (0xac89 bytes) 2024-09-04T17:46:54.187 INFO:teuthology.orchestra.run.smithi092.stdout:3086 punch from 0x2a2f033 to 0x2a3ad97, (0xbd64 bytes) 2024-09-04T17:46:54.187 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:54.188 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:54.188 INFO:teuthology.orchestra.run.smithi092.stdout:3090 read 0x891faa thru 0x894a76 (0x2acd bytes) 2024-09-04T17:46:54.188 INFO:teuthology.orchestra.run.smithi092.stdout:3092 punch from 0xdc6ff67 to 0xdc74a67, (0x4b00 bytes) 2024-09-04T17:46:54.188 INFO:teuthology.orchestra.run.smithi092.stdout:3093 punch from 0x4a3f8e5 to 0x4a46609, (0x6d24 bytes) 2024-09-04T17:46:54.188 INFO:teuthology.orchestra.run.smithi092.stdout:3094 punch from 0x28df607 to 0x28e7d25, (0x871e bytes) 2024-09-04T17:46:54.188 INFO:teuthology.orchestra.run.smithi092.stdout:3095 read 0x7cfa930 thru 0x7d0537c (0xaa4d bytes) 2024-09-04T17:46:54.188 INFO:teuthology.orchestra.run.smithi092.stdout:3096 writesame 0x7300713 thru 0x730efda data_size 0x9c(0xe8c8 bytes) 2024-09-04T17:46:54.193 INFO:teuthology.orchestra.run.smithi092.stdout:3098 write 0xcd30a44 thru 0xcd39703 (0x8cc0 bytes) 2024-09-04T17:46:54.200 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:54.200 INFO:teuthology.orchestra.run.smithi092.stdout:3100 punch from 0x6af6fc5 to 0x6afd645, (0x6680 bytes) 2024-09-04T17:46:54.200 INFO:teuthology.orchestra.run.smithi092.stdout:3104 read 0x57005e2 thru 0x570443d (0x3e5c bytes) 2024-09-04T17:46:54.201 INFO:teuthology.orchestra.run.smithi092.stdout:3105 trunc from 0xe6b73c7 to 0xeb2aa20 2024-09-04T17:46:54.211 INFO:teuthology.orchestra.run.smithi092.stdout:3106 write 0x979e906 thru 0x97a7412 (0x8b0d bytes) 2024-09-04T17:46:54.244 INFO:teuthology.orchestra.run.smithi092.stdout:3108 trunc from 0xeb2aa20 to 0x1a338f6 2024-09-04T17:46:54.250 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:54.249+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:54.284 INFO:teuthology.orchestra.run.smithi092.stdout:3109 writesame 0x3075c50 thru 0x30845f6 data_size 0x7(0xe9a7 bytes) 2024-09-04T17:46:54.284 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:54.291 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:54.292 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:46:54.292 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:46:54.292 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:54.306 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-04T17:46:54.307 INFO:teuthology.orchestra.run.smithi092.stdout:3112 trunc from 0x30845f7 to 0xb9e48d8 2024-09-04T17:46:54.318 INFO:teuthology.orchestra.run.smithi092.stdout:3113 punch from 0x5243ab0 to 0x52441de, (0x72e bytes) 2024-09-04T17:46:54.319 INFO:teuthology.orchestra.run.smithi092.stdout:3114 writesame 0x9ff3141 thru 0x9fffbb1 data_size 0x19(0xca71 bytes) 2024-09-04T17:46:54.327 INFO:teuthology.orchestra.run.smithi092.stdout:3115 read 0x2d786b4 thru 0x2d87fed (0xf93a bytes) 2024-09-04T17:46:54.327 INFO:teuthology.orchestra.run.smithi092.stdout:3116 punch from 0x9277982 to 0x9277d44, (0x3c2 bytes) 2024-09-04T17:46:54.327 INFO:teuthology.orchestra.run.smithi092.stdout:3117 punch from 0x9ee4095 to 0x9ef3037, (0xefa2 bytes) 2024-09-04T17:46:54.327 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:54.327 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:54.344 INFO:teuthology.orchestra.run.smithi092.stdout:3121 write 0xe50a72f thru 0xe51344a (0x8d1c bytes) 2024-09-04T17:46:54.349 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:54.347+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:54.352 INFO:teuthology.orchestra.run.smithi092.stdout:3122 write 0xa2bb2f1 thru 0xa2c1a87 (0x6797 bytes) 2024-09-04T17:46:54.361 INFO:teuthology.orchestra.run.smithi092.stdout:3123 write 0xbed4ba thru 0xbfc56f (0xf0b6 bytes) 2024-09-04T17:46:54.369 INFO:teuthology.orchestra.run.smithi092.stdout:3124 write 0x3704fbc thru 0x3705382 (0x3c7 bytes) 2024-09-04T17:46:54.376 INFO:teuthology.orchestra.run.smithi092.stdout:3125 write 0xd9a510b thru 0xd9b1ef4 (0xcdea bytes) 2024-09-04T17:46:54.406 INFO:teuthology.orchestra.run.smithi092.stdout:3126 trunc from 0xe51344b to 0x50f92df 2024-09-04T17:46:54.448 INFO:teuthology.orchestra.run.smithi092.stdout:3127 write 0x82d47ec thru 0x82da60a (0x5e1f bytes) 2024-09-04T17:46:54.450 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:54.447+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:54.455 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:54.455 INFO:teuthology.orchestra.run.smithi092.stdout:3129 writesame 0x5ff0ac6 thru 0x5ff3667 data_size 0x5(0x2ba2 bytes) 2024-09-04T17:46:54.464 INFO:teuthology.orchestra.run.smithi092.stdout:3131 read 0x2d94c6 thru 0x2e1b9d (0x86d8 bytes) 2024-09-04T17:46:54.464 INFO:teuthology.orchestra.run.smithi092.stdout:3132 punch from 0x764863d to 0x7650f26, (0x88e9 bytes) 2024-09-04T17:46:54.464 INFO:teuthology.orchestra.run.smithi092.stdout:3133 write 0x5044f07 thru 0x50503e0 (0xb4da bytes) 2024-09-04T17:46:54.477 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:54.477 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:46:54.477 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:46:54.477 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:46:54.477 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:54.492 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-04T17:46:54.497 INFO:teuthology.orchestra.run.smithi092.stdout:3134 write 0xcb7ac83 thru 0xcb854af (0xa82d bytes) 2024-09-04T17:46:54.506 INFO:teuthology.orchestra.run.smithi092.stdout:3135 read 0x9a0f3ac thru 0x9a15144 (0x5d99 bytes) 2024-09-04T17:46:54.506 INFO:teuthology.orchestra.run.smithi092.stdout:3136 write 0x68ff229 thru 0x6903e49 (0x4c21 bytes) 2024-09-04T17:46:54.514 INFO:teuthology.orchestra.run.smithi092.stdout:3138 read 0x1a1feda thru 0x1a2d2ea (0xd411 bytes) 2024-09-04T17:46:54.514 INFO:teuthology.orchestra.run.smithi092.stdout:3141 writesame 0x1bed3cf thru 0x1bf7080 data_size 0x1f(0x9cb2 bytes) 2024-09-04T17:46:54.524 INFO:teuthology.orchestra.run.smithi092.stdout:3142 punch from 0x95f122d to 0x95f5710, (0x44e3 bytes) 2024-09-04T17:46:54.524 INFO:teuthology.orchestra.run.smithi092.stdout:3143 write 0x6d4bebb thru 0x6d50711 (0x4857 bytes) 2024-09-04T17:46:54.549 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-04T17:46:54.551 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:54.547+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:54.560 INFO:teuthology.orchestra.run.smithi092.stdout:3144 trunc from 0xcb854b0 to 0x2e697d1 2024-09-04T17:46:54.610 INFO:teuthology.orchestra.run.smithi092.stdout:3146 write 0x9258220 thru 0x926609f (0xde80 bytes) 2024-09-04T17:46:54.617 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:54.617 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:54.625 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:54.626 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:46:54.626 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:46:54.626 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:46:54.626 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.626 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:12722 3.1b 3:d8152af1:::rbd_data.1134ef3a5a7d.00000000000000bd:head [delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-04T17:46:54.626 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:16.421965+0000", 2024-09-04T17:46:54.626 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 38.202055063000003, 2024-09-04T17:46:54.626 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.890002811, 2024-09-04T17:46:54.626 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:54.626 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:54.626 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:54.626 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:54.626 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:54.627 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:54.627 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 12722 2024-09-04T17:46:54.627 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.627 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:54.627 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.627 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:54.627 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421965+0000", 2024-09-04T17:46:54.627 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:54.627 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.627 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.627 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:54.627 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421965+0000", 2024-09-04T17:46:54.627 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:54.628 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.628 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.628 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:54.628 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421966+0000", 2024-09-04T17:46:54.628 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.102e-06 2024-09-04T17:46:54.628 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.628 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.628 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:54.628 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421971+0000", 2024-09-04T17:46:54.628 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9989999999999999e-06 2024-09-04T17:46:54.628 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.628 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.628 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:54.628 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421981+0000", 2024-09-04T17:46:54.629 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.7380000000000006e-06 2024-09-04T17:46:54.629 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.629 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.629 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:54.629 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421989+0000", 2024-09-04T17:46:54.629 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.1890000000000007e-06 2024-09-04T17:46:54.629 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.629 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.629 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:54.629 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.422050+0000", 2024-09-04T17:46:54.629 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.0702000000000003e-05 2024-09-04T17:46:54.629 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.630 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.630 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for peered", 2024-09-04T17:46:54.630 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.422059+0000", 2024-09-04T17:46:54.630 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5440000000000004e-06 2024-09-04T17:46:54.630 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.630 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.630 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:54.630 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.308156+0000", 2024-09-04T17:46:54.630 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.88609676199999998 2024-09-04T17:46:54.630 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.630 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.630 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:54.630 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.308398+0000", 2024-09-04T17:46:54.631 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024215400000000001 2024-09-04T17:46:54.631 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.631 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.631 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 7", 2024-09-04T17:46:54.631 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.308795+0000", 2024-09-04T17:46:54.631 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000396704 2024-09-04T17:46:54.631 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.631 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.631 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:54.631 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.310831+0000", 2024-09-04T17:46:54.631 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0020364200000000002 2024-09-04T17:46:54.631 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.631 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.631 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:54.632 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.311818+0000", 2024-09-04T17:46:54.632 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00098649799999999993 2024-09-04T17:46:54.632 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.632 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.632 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:54.632 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.311905+0000", 2024-09-04T17:46:54.632 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7070999999999994e-05 2024-09-04T17:46:54.632 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.632 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.632 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:54.632 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.311968+0000", 2024-09-04T17:46:54.632 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.2928000000000005e-05 2024-09-04T17:46:54.633 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:54.633 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:54.633 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:54.633 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.633 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.633 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37359 3.f e58/24 3:f29e50a5:::rbd_data.1134c5a43465.0000000000000021:head v 58'135, mlcod=58'133)", 2024-09-04T17:46:54.633 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:46:54.633 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 15.006724459000001, 2024-09-04T17:46:54.633 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053483197000000003, 2024-09-04T17:46:54.633 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:54.633 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:54.633 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:54.633 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:54.634 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.634 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:54.634 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:46:54.634 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:54.634 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.634 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.634 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:54.634 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:46:54.634 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:54.634 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.634 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.634 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:54.634 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:46:54.634 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1699999999999997e-07 2024-09-04T17:46:54.635 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.635 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.635 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:54.635 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617338+0000", 2024-09-04T17:46:54.635 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.1168999999999998e-05 2024-09-04T17:46:54.635 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.635 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.635 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:54.635 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617351+0000", 2024-09-04T17:46:54.635 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3094e-05 2024-09-04T17:46:54.635 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.635 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.635 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:54.635 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617359+0000", 2024-09-04T17:46:54.636 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7970000000000006e-06 2024-09-04T17:46:54.636 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.636 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.636 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:54.636 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617417+0000", 2024-09-04T17:46:54.636 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7079e-05 2024-09-04T17:46:54.636 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.636 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.636 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:54.636 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617438+0000", 2024-09-04T17:46:54.636 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1654999999999998e-05 2024-09-04T17:46:54.636 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.636 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.637 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:54.637 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670738+0000", 2024-09-04T17:46:54.637 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053300142000000002 2024-09-04T17:46:54.637 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.637 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.637 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:54.637 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670779+0000", 2024-09-04T17:46:54.637 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0343999999999999e-05 2024-09-04T17:46:54.637 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:54.637 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:54.637 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:54.637 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.637 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.638 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37358 3.1d e58/24 3:bd3215f3:::rbd_data.1134c5a43465.0000000000000020:head v 58'91, mlcod=58'89)", 2024-09-04T17:46:54.638 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617502+0000", 2024-09-04T17:46:54.638 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 15.006517741, 2024-09-04T17:46:54.638 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053505641999999999, 2024-09-04T17:46:54.638 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:54.638 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:54.638 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:54.638 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:54.638 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.638 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:54.638 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617502+0000", 2024-09-04T17:46:54.638 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:54.638 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.638 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.639 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:54.639 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617502+0000", 2024-09-04T17:46:54.639 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:54.639 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.639 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.639 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:54.639 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617503+0000", 2024-09-04T17:46:54.639 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.184e-06 2024-09-04T17:46:54.639 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.639 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.639 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:54.639 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617526+0000", 2024-09-04T17:46:54.639 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2099999999999998e-05 2024-09-04T17:46:54.639 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.640 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.640 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:54.640 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617540+0000", 2024-09-04T17:46:54.640 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4741e-05 2024-09-04T17:46:54.640 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.640 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.640 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:54.640 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617550+0000", 2024-09-04T17:46:54.640 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4259999999999992e-06 2024-09-04T17:46:54.640 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.640 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.640 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:54.640 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617613+0000", 2024-09-04T17:46:54.641 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.3625000000000002e-05 2024-09-04T17:46:54.641 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.641 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.641 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:54.641 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617637+0000", 2024-09-04T17:46:54.641 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.3604999999999998e-05 2024-09-04T17:46:54.641 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.641 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.641 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:54.642 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670967+0000", 2024-09-04T17:46:54.642 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053329881000000003 2024-09-04T17:46:54.642 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.642 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.642 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:54.642 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671008+0000", 2024-09-04T17:46:54.642 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.108e-05 2024-09-04T17:46:54.643 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:54.643 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:54.643 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:54.643 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.643 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.643 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37361 3.1c e58/24 3:3c7f8976:::rbd_data.1134c5a43465.0000000000000023:head v 58'159, mlcod=58'157)", 2024-09-04T17:46:54.643 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617578+0000", 2024-09-04T17:46:54.643 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 15.006441891, 2024-09-04T17:46:54.643 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053990367999999997, 2024-09-04T17:46:54.643 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:54.643 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:54.643 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:54.643 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:54.644 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.644 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:54.644 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617578+0000", 2024-09-04T17:46:54.644 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:54.644 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.644 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.644 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:54.644 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617578+0000", 2024-09-04T17:46:54.644 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:54.644 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.644 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.644 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:54.644 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617579+0000", 2024-09-04T17:46:54.645 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7299999999999996e-07 2024-09-04T17:46:54.645 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.645 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.645 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:54.645 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617610+0000", 2024-09-04T17:46:54.645 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0951000000000002e-05 2024-09-04T17:46:54.645 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.645 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.645 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:54.645 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:46:54.645 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0002e-05 2024-09-04T17:46:54.645 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.645 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.646 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:54.646 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617626+0000", 2024-09-04T17:46:54.646 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6660000000000002e-06 2024-09-04T17:46:54.646 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.646 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.646 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:54.646 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670548+0000", 2024-09-04T17:46:54.646 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.052921943999999999 2024-09-04T17:46:54.646 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.646 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.646 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:54.646 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670574+0000", 2024-09-04T17:46:54.646 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5405000000000002e-05 2024-09-04T17:46:54.647 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.647 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.647 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:54.647 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671522+0000", 2024-09-04T17:46:54.647 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00094815299999999997 2024-09-04T17:46:54.647 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.647 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.647 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:54.647 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671568+0000", 2024-09-04T17:46:54.647 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6674000000000001e-05 2024-09-04T17:46:54.647 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:54.647 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:54.647 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:54.647 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.647 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.648 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37357 3.15 e58/49 3:a897312a:::rbd_data.1134c5a43465.000000000000001f:head v 58'117, mlcod=58'115)", 2024-09-04T17:46:54.648 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:46:54.648 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 15.006399553, 2024-09-04T17:46:54.648 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053734297, 2024-09-04T17:46:54.648 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:54.648 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:54.648 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:54.648 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:54.648 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.648 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:54.648 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:46:54.648 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:54.648 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.648 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.649 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:54.649 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:46:54.649 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:54.649 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.649 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.649 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:54.649 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617621+0000", 2024-09-04T17:46:54.649 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.7300000000000004e-07 2024-09-04T17:46:54.649 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.649 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.649 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:54.649 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617642+0000", 2024-09-04T17:46:54.649 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0185000000000001e-05 2024-09-04T17:46:54.650 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.650 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.650 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:54.650 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617654+0000", 2024-09-04T17:46:54.650 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2745000000000001e-05 2024-09-04T17:46:54.650 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.650 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.650 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:54.650 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617662+0000", 2024-09-04T17:46:54.650 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.0499999999999992e-06 2024-09-04T17:46:54.650 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.650 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.651 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:54.651 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617727+0000", 2024-09-04T17:46:54.651 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.4923000000000002e-05 2024-09-04T17:46:54.651 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.651 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.651 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:54.651 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617753+0000", 2024-09-04T17:46:54.651 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5899999999999999e-05 2024-09-04T17:46:54.651 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.651 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.651 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:54.651 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671298+0000", 2024-09-04T17:46:54.651 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053545276000000003 2024-09-04T17:46:54.651 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.652 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.652 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:54.652 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671355+0000", 2024-09-04T17:46:54.652 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6245000000000001e-05 2024-09-04T17:46:54.652 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:54.652 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:54.652 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:54.652 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.652 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.652 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44687 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:46:54.652 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253012+0000", 2024-09-04T17:46:54.652 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.3710074859999999, 2024-09-04T17:46:54.652 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051128777, 2024-09-04T17:46:54.653 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:54.653 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:54.653 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:54.653 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:54.653 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:54.653 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:54.653 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44687 2024-09-04T17:46:54.653 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.653 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:54.653 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.653 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:54.653 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253012+0000", 2024-09-04T17:46:54.653 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:54.653 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.654 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.654 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:54.654 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253012+0000", 2024-09-04T17:46:54.654 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:54.654 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.654 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.654 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:54.654 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253014+0000", 2024-09-04T17:46:54.654 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.24e-06 2024-09-04T17:46:54.654 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.654 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.654 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:54.654 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253019+0000", 2024-09-04T17:46:54.655 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4569999999999996e-06 2024-09-04T17:46:54.655 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.655 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.655 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:54.655 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253035+0000", 2024-09-04T17:46:54.655 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6291999999999999e-05 2024-09-04T17:46:54.655 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.655 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.655 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:54.655 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253050+0000", 2024-09-04T17:46:54.655 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4423000000000001e-05 2024-09-04T17:46:54.655 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.655 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.656 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:54.656 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304115+0000", 2024-09-04T17:46:54.656 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051064778999999998 2024-09-04T17:46:54.656 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.656 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.656 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:54.656 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304141+0000", 2024-09-04T17:46:54.656 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6585999999999999e-05 2024-09-04T17:46:54.656 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:54.656 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:54.656 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:54.656 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.656 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.657 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44688 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:46:54.657 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253067+0000", 2024-09-04T17:46:54.657 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.370952934, 2024-09-04T17:46:54.657 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051116979, 2024-09-04T17:46:54.657 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:54.657 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:54.657 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:54.657 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:54.657 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:54.657 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:54.657 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44688 2024-09-04T17:46:54.657 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.657 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:54.658 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.658 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:54.658 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253067+0000", 2024-09-04T17:46:54.658 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:54.658 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.658 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.658 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:54.658 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253067+0000", 2024-09-04T17:46:54.658 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:54.658 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.658 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.658 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:54.658 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253068+0000", 2024-09-04T17:46:54.658 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0729999999999999e-06 2024-09-04T17:46:54.659 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.659 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.659 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:54.659 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253073+0000", 2024-09-04T17:46:54.659 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2259999999999996e-06 2024-09-04T17:46:54.659 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.659 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.659 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:54.659 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253091+0000", 2024-09-04T17:46:54.659 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7255999999999999e-05 2024-09-04T17:46:54.659 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.659 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.659 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:54.660 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253103+0000", 2024-09-04T17:46:54.660 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2632999999999999e-05 2024-09-04T17:46:54.660 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.660 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.660 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:54.660 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304173+0000", 2024-09-04T17:46:54.660 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051069537999999998 2024-09-04T17:46:54.660 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.660 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.660 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:54.660 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304184+0000", 2024-09-04T17:46:54.660 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1253000000000001e-05 2024-09-04T17:46:54.660 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:54.661 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:54.661 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:54.661 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.661 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.661 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44689 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:46:54.661 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253121+0000", 2024-09-04T17:46:54.661 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.3708987969999997, 2024-09-04T17:46:54.661 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051192595, 2024-09-04T17:46:54.661 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:54.661 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:54.661 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:54.661 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:54.661 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:54.662 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:54.662 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44689 2024-09-04T17:46:54.662 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.662 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:54.662 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.662 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:54.662 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253121+0000", 2024-09-04T17:46:54.662 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:54.662 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.662 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.662 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:54.662 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253121+0000", 2024-09-04T17:46:54.662 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:54.662 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.663 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.663 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:54.663 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253122+0000", 2024-09-04T17:46:54.663 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.15e-06 2024-09-04T17:46:54.663 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.663 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.663 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:54.663 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253128+0000", 2024-09-04T17:46:54.663 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2569999999999998e-06 2024-09-04T17:46:54.663 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.663 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.663 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:54.663 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253145+0000", 2024-09-04T17:46:54.663 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7176e-05 2024-09-04T17:46:54.664 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.664 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.664 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:54.664 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253158+0000", 2024-09-04T17:46:54.664 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3139000000000001e-05 2024-09-04T17:46:54.664 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.664 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.664 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:54.664 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304287+0000", 2024-09-04T17:46:54.664 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051129149999999998 2024-09-04T17:46:54.664 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.664 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.664 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:54.665 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304314+0000", 2024-09-04T17:46:54.665 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6723000000000001e-05 2024-09-04T17:46:54.665 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:54.665 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:54.665 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:54.665 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.665 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.665 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44690 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:46:54.665 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253177+0000", 2024-09-04T17:46:54.665 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.3708426720000002, 2024-09-04T17:46:54.665 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051169263, 2024-09-04T17:46:54.665 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:54.665 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:54.665 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:54.666 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:54.666 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:54.666 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:54.666 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44690 2024-09-04T17:46:54.666 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.666 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:54.666 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.666 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:54.666 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253177+0000", 2024-09-04T17:46:54.666 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:54.666 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.666 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.666 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:54.667 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253177+0000", 2024-09-04T17:46:54.667 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:54.667 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.667 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.667 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:54.667 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253178+0000", 2024-09-04T17:46:54.667 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1000000000000001e-06 2024-09-04T17:46:54.667 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.667 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.667 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:54.667 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253184+0000", 2024-09-04T17:46:54.667 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2089999999999996e-06 2024-09-04T17:46:54.667 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.667 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.668 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:54.668 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253200+0000", 2024-09-04T17:46:54.668 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6087e-05 2024-09-04T17:46:54.668 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.668 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.668 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:54.668 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253231+0000", 2024-09-04T17:46:54.668 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0874999999999998e-05 2024-09-04T17:46:54.668 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.668 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.668 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:54.668 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304337+0000", 2024-09-04T17:46:54.668 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051106430000000001 2024-09-04T17:46:54.668 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.668 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.669 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:54.669 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304347+0000", 2024-09-04T17:46:54.669 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5619999999999993e-06 2024-09-04T17:46:54.669 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:54.669 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:54.669 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:54.669 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.669 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.669 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44691 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:46:54.669 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253249+0000", 2024-09-04T17:46:54.669 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.3707706379999998, 2024-09-04T17:46:54.669 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051195630999999998, 2024-09-04T17:46:54.669 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:54.669 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:54.670 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:54.670 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:54.670 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:54.670 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:54.670 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44691 2024-09-04T17:46:54.670 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.670 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:54.670 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.670 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:54.670 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253249+0000", 2024-09-04T17:46:54.670 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:54.670 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.670 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.670 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:54.671 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253249+0000", 2024-09-04T17:46:54.671 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:54.671 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.671 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.671 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:54.671 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253251+0000", 2024-09-04T17:46:54.671 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2330000000000001e-06 2024-09-04T17:46:54.671 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.671 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.671 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:54.671 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253256+0000", 2024-09-04T17:46:54.671 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6099999999999997e-06 2024-09-04T17:46:54.671 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.672 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.672 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:54.672 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253274+0000", 2024-09-04T17:46:54.672 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7952999999999999e-05 2024-09-04T17:46:54.672 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.672 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.672 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:54.672 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253287+0000", 2024-09-04T17:46:54.672 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2839e-05 2024-09-04T17:46:54.672 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.672 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.672 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:54.672 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304396+0000", 2024-09-04T17:46:54.673 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051108990999999999 2024-09-04T17:46:54.673 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.673 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.673 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:54.673 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304445+0000", 2024-09-04T17:46:54.673 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9005000000000003e-05 2024-09-04T17:46:54.673 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:54.673 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:54.673 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:54.673 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.673 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.673 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44692 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:46:54.673 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253337+0000", 2024-09-04T17:46:54.674 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.3706830649999997, 2024-09-04T17:46:54.674 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051138909000000003, 2024-09-04T17:46:54.674 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:54.674 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:54.674 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:54.674 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:54.674 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:54.674 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:54.674 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44692 2024-09-04T17:46:54.674 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.674 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:54.674 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.674 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:54.674 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253337+0000", 2024-09-04T17:46:54.675 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:54.675 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.675 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.675 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:54.675 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253337+0000", 2024-09-04T17:46:54.675 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:54.675 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.675 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.675 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:54.675 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253338+0000", 2024-09-04T17:46:54.675 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.463e-06 2024-09-04T17:46:54.675 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.675 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.676 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:54.676 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253357+0000", 2024-09-04T17:46:54.676 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9043e-05 2024-09-04T17:46:54.676 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.676 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.676 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:54.676 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253379+0000", 2024-09-04T17:46:54.676 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1243e-05 2024-09-04T17:46:54.676 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.676 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.676 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:54.676 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253396+0000", 2024-09-04T17:46:54.676 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7059e-05 2024-09-04T17:46:54.677 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.677 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.677 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:54.677 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304466+0000", 2024-09-04T17:46:54.677 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051070624000000002 2024-09-04T17:46:54.677 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.677 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.677 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:54.677 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304476+0000", 2024-09-04T17:46:54.677 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4769999999999992e-06 2024-09-04T17:46:54.677 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:54.677 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:54.677 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:54.678 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.678 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.678 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44693 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:46:54.678 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253415+0000", 2024-09-04T17:46:54.678 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.3706046619999999, 2024-09-04T17:46:54.678 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051141949999999999, 2024-09-04T17:46:54.678 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:54.678 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:54.678 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:54.678 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:54.678 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:54.678 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:54.678 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44693 2024-09-04T17:46:54.678 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.679 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:54.679 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.679 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:54.679 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253415+0000", 2024-09-04T17:46:54.679 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:54.679 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.679 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.679 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:54.679 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253415+0000", 2024-09-04T17:46:54.679 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:54.679 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.679 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.679 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:54.679 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253416+0000", 2024-09-04T17:46:54.680 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1799999999999999e-06 2024-09-04T17:46:54.680 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.680 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.680 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:54.680 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253422+0000", 2024-09-04T17:46:54.680 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.5940000000000001e-06 2024-09-04T17:46:54.680 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.680 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.680 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:54.680 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253438+0000", 2024-09-04T17:46:54.680 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6008999999999999e-05 2024-09-04T17:46:54.680 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.680 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.680 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:54.681 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253452+0000", 2024-09-04T17:46:54.681 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3543000000000001e-05 2024-09-04T17:46:54.681 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.681 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.681 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:54.681 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304533+0000", 2024-09-04T17:46:54.681 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051081609 2024-09-04T17:46:54.681 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.681 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.681 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:54.681 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304557+0000", 2024-09-04T17:46:54.681 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4015e-05 2024-09-04T17:46:54.681 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:54.681 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:54.682 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:54.682 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.682 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.682 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44694 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:46:54.682 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253469+0000", 2024-09-04T17:46:54.682 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.3705505369999997, 2024-09-04T17:46:54.682 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051121486000000001, 2024-09-04T17:46:54.682 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:54.682 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:54.682 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:54.682 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:54.682 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:54.682 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:54.683 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44694 2024-09-04T17:46:54.683 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.683 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:54.683 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.683 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:54.683 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253469+0000", 2024-09-04T17:46:54.683 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:54.683 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.683 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.683 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:54.683 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253469+0000", 2024-09-04T17:46:54.683 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:54.683 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.683 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.683 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:54.684 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253471+0000", 2024-09-04T17:46:54.684 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1400000000000001e-06 2024-09-04T17:46:54.684 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.684 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.684 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:54.684 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253476+0000", 2024-09-04T17:46:54.684 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7370000000000003e-06 2024-09-04T17:46:54.684 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.684 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.684 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:54.684 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253493+0000", 2024-09-04T17:46:54.684 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6549e-05 2024-09-04T17:46:54.684 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.684 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.685 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:54.685 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253506+0000", 2024-09-04T17:46:54.685 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3406e-05 2024-09-04T17:46:54.685 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.685 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.685 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:54.685 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304582+0000", 2024-09-04T17:46:54.685 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051075534999999998 2024-09-04T17:46:54.685 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.685 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.685 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:54.685 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304591+0000", 2024-09-04T17:46:54.685 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1190000000000003e-06 2024-09-04T17:46:54.686 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:54.686 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:54.686 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:54.686 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.686 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.686 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44695 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:46:54.686 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253552+0000", 2024-09-04T17:46:54.686 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.3704677829999996, 2024-09-04T17:46:54.686 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051068872000000001, 2024-09-04T17:46:54.686 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:54.686 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:54.686 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:54.686 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:54.686 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:54.687 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:54.687 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44695 2024-09-04T17:46:54.687 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.687 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:54.687 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.687 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:54.687 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253552+0000", 2024-09-04T17:46:54.687 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:54.687 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.687 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.688 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:54.688 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253552+0000", 2024-09-04T17:46:54.688 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:54.688 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.688 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.688 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:54.688 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253553+0000", 2024-09-04T17:46:54.688 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1960000000000001e-06 2024-09-04T17:46:54.688 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.688 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.688 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:54.688 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253560+0000", 2024-09-04T17:46:54.689 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.2040000000000002e-06 2024-09-04T17:46:54.689 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.689 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.689 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:54.689 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253578+0000", 2024-09-04T17:46:54.689 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8241999999999999e-05 2024-09-04T17:46:54.689 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.689 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.689 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:54.689 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253591+0000", 2024-09-04T17:46:54.689 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3453e-05 2024-09-04T17:46:54.689 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.689 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.690 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:54.690 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304614+0000", 2024-09-04T17:46:54.690 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051022456000000001 2024-09-04T17:46:54.690 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.690 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.690 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:54.690 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304621+0000", 2024-09-04T17:46:54.690 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3209999999999999e-06 2024-09-04T17:46:54.690 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:54.690 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:54.690 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:54.690 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.690 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.690 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44696 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:46:54.690 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253609+0000", 2024-09-04T17:46:54.691 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.3704112820000001, 2024-09-04T17:46:54.691 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051115351000000003, 2024-09-04T17:46:54.691 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:54.691 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:54.691 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:54.691 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:54.691 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:54.691 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:54.691 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44696 2024-09-04T17:46:54.691 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.691 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:54.691 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.691 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:54.691 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253609+0000", 2024-09-04T17:46:54.691 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:54.692 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.692 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.692 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:54.692 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253609+0000", 2024-09-04T17:46:54.692 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:54.692 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.692 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.692 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:54.692 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253610+0000", 2024-09-04T17:46:54.692 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2300000000000001e-06 2024-09-04T17:46:54.692 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.692 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.692 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:54.692 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253615+0000", 2024-09-04T17:46:54.693 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7260000000000002e-06 2024-09-04T17:46:54.693 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.693 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.693 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:54.693 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253632+0000", 2024-09-04T17:46:54.693 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7039999999999999e-05 2024-09-04T17:46:54.693 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.693 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.693 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:54.693 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253666+0000", 2024-09-04T17:46:54.693 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4662000000000001e-05 2024-09-04T17:46:54.693 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.693 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.694 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:54.694 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304677+0000", 2024-09-04T17:46:54.694 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051011017999999998 2024-09-04T17:46:54.694 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.694 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.694 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:54.694 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304724+0000", 2024-09-04T17:46:54.694 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6675000000000003e-05 2024-09-04T17:46:54.694 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:54.694 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:54.694 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:54.694 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.694 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.695 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44697 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:46:54.695 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253684+0000", 2024-09-04T17:46:54.695 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.3703357880000002, 2024-09-04T17:46:54.695 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051073107, 2024-09-04T17:46:54.695 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:54.695 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:54.695 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:54.695 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:54.695 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:54.695 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:54.695 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44697 2024-09-04T17:46:54.695 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.695 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:54.696 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.696 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:54.696 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253684+0000", 2024-09-04T17:46:54.696 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:54.696 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.696 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.696 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:54.696 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253684+0000", 2024-09-04T17:46:54.696 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:54.696 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.696 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.696 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:54.696 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253685+0000", 2024-09-04T17:46:54.696 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0699999999999999e-06 2024-09-04T17:46:54.696 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.697 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.697 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:54.697 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253690+0000", 2024-09-04T17:46:54.697 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1959999999999999e-06 2024-09-04T17:46:54.697 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.697 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.697 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:54.697 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253707+0000", 2024-09-04T17:46:54.697 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6946e-05 2024-09-04T17:46:54.697 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.697 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.697 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:54.697 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253720+0000", 2024-09-04T17:46:54.697 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3076e-05 2024-09-04T17:46:54.698 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.698 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.698 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:54.698 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304745+0000", 2024-09-04T17:46:54.698 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051024760000000002 2024-09-04T17:46:54.698 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.698 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.698 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:54.698 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304757+0000", 2024-09-04T17:46:54.698 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2058999999999999e-05 2024-09-04T17:46:54.698 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:54.698 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:54.698 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:54.698 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.699 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.699 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44698 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:46:54.699 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253737+0000", 2024-09-04T17:46:54.699 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.3702834370000003, 2024-09-04T17:46:54.699 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051176078, 2024-09-04T17:46:54.699 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:54.699 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:54.699 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:54.699 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:54.699 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:54.699 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:54.699 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44698 2024-09-04T17:46:54.699 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.699 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:54.700 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.700 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:54.700 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253737+0000", 2024-09-04T17:46:54.700 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:54.700 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.700 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.700 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:54.700 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253737+0000", 2024-09-04T17:46:54.700 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:54.700 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.700 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.700 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:54.701 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253738+0000", 2024-09-04T17:46:54.701 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0300000000000001e-06 2024-09-04T17:46:54.701 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.701 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.701 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:54.701 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253752+0000", 2024-09-04T17:46:54.701 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4358999999999999e-05 2024-09-04T17:46:54.701 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.701 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.701 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:54.701 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253771+0000", 2024-09-04T17:46:54.701 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8879999999999999e-05 2024-09-04T17:46:54.701 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.702 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.702 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:54.702 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253785+0000", 2024-09-04T17:46:54.702 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3852000000000001e-05 2024-09-04T17:46:54.702 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.702 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.702 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:54.702 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304876+0000", 2024-09-04T17:46:54.702 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051091105999999997 2024-09-04T17:46:54.702 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.702 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.702 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:54.702 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304913+0000", 2024-09-04T17:46:54.702 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.6850999999999997e-05 2024-09-04T17:46:54.703 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:54.703 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:54.703 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:54.703 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.703 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.703 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44699 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:46:54.703 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253804+0000", 2024-09-04T17:46:54.703 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.370215913, 2024-09-04T17:46:54.703 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051172250000000002, 2024-09-04T17:46:54.703 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:54.703 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:54.703 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:54.703 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:54.703 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:54.704 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:54.704 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44699 2024-09-04T17:46:54.704 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.704 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:54.704 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.704 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:54.704 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253804+0000", 2024-09-04T17:46:54.704 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:54.704 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.704 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.704 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:54.704 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253804+0000", 2024-09-04T17:46:54.704 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:54.705 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.705 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.705 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:54.705 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253805+0000", 2024-09-04T17:46:54.705 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.08e-06 2024-09-04T17:46:54.705 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.705 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.705 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:54.705 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253810+0000", 2024-09-04T17:46:54.705 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1830000000000002e-06 2024-09-04T17:46:54.705 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.705 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.705 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:54.705 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253828+0000", 2024-09-04T17:46:54.706 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7662000000000001e-05 2024-09-04T17:46:54.706 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.706 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.706 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:54.706 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253841+0000", 2024-09-04T17:46:54.706 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3203999999999999e-05 2024-09-04T17:46:54.706 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.706 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.706 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:54.706 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304966+0000", 2024-09-04T17:46:54.706 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051125273999999998 2024-09-04T17:46:54.706 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.706 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.707 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:54.707 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304976+0000", 2024-09-04T17:46:54.707 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.8470000000000008e-06 2024-09-04T17:46:54.707 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:54.707 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:54.707 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:54.707 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.707 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.707 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44700 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:46:54.707 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253858+0000", 2024-09-04T17:46:54.707 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.3701622589999998, 2024-09-04T17:46:54.707 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051218957000000002, 2024-09-04T17:46:54.707 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:54.707 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:54.708 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:54.708 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:54.708 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:54.708 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:54.708 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44700 2024-09-04T17:46:54.708 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.708 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:54.708 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.708 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:54.708 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253858+0000", 2024-09-04T17:46:54.708 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:54.708 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.708 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.709 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:54.709 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253858+0000", 2024-09-04T17:46:54.709 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:54.709 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.709 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.709 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:54.709 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253859+0000", 2024-09-04T17:46:54.709 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.217e-06 2024-09-04T17:46:54.709 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.709 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.709 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:54.709 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253864+0000", 2024-09-04T17:46:54.709 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7600000000000002e-06 2024-09-04T17:46:54.710 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.710 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.710 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:54.710 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253880+0000", 2024-09-04T17:46:54.710 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6501999999999999e-05 2024-09-04T17:46:54.710 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.710 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.710 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:54.710 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253899+0000", 2024-09-04T17:46:54.710 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8907e-05 2024-09-04T17:46:54.710 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.710 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.710 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:54.710 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305052+0000", 2024-09-04T17:46:54.711 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051152989000000003 2024-09-04T17:46:54.711 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.711 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.711 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:54.711 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305077+0000", 2024-09-04T17:46:54.711 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4581999999999999e-05 2024-09-04T17:46:54.711 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:54.711 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:54.711 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:54.711 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.711 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.711 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44701 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:46:54.711 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253917+0000", 2024-09-04T17:46:54.711 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.3701029069999997, 2024-09-04T17:46:54.712 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051231113000000002, 2024-09-04T17:46:54.712 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:54.712 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:54.712 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:54.712 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:54.712 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:54.712 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:54.712 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44701 2024-09-04T17:46:54.712 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.713 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:54.713 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.713 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:54.713 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253917+0000", 2024-09-04T17:46:54.713 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:54.713 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.713 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.713 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:54.713 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253917+0000", 2024-09-04T17:46:54.713 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:54.713 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.713 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.713 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:54.713 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253918+0000", 2024-09-04T17:46:54.714 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.136e-06 2024-09-04T17:46:54.714 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.714 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.714 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:54.714 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253924+0000", 2024-09-04T17:46:54.714 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.3070000000000002e-06 2024-09-04T17:46:54.714 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.714 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.714 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:54.714 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253940+0000", 2024-09-04T17:46:54.714 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6612e-05 2024-09-04T17:46:54.714 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.714 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.715 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:54.715 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253954+0000", 2024-09-04T17:46:54.715 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3709999999999999e-05 2024-09-04T17:46:54.715 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.715 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.715 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:54.715 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305136+0000", 2024-09-04T17:46:54.715 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051181935999999997 2024-09-04T17:46:54.715 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:54.715 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:54.715 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:54.715 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305148+0000", 2024-09-04T17:46:54.715 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2412000000000001e-05 2024-09-04T17:46:54.716 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:54.716 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:54.716 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:54.716 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:54.716 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:54.716 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:54.718 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-04T17:46:54.718 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:54.717+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:54.720 INFO:teuthology.orchestra.run.smithi092.stdout:3152 write 0xa629b1c thru 0xa631764 (0x7c49 bytes) 2024-09-04T17:46:54.720 INFO:teuthology.orchestra.run.smithi092.stdout:3155 write 0xe68f3a1 thru 0xe695ea7 (0x6b07 bytes) 2024-09-04T17:46:54.720 INFO:teuthology.orchestra.run.smithi092.stdout:3157 read 0x2138c05 thru 0x2146d50 (0xe14c bytes) 2024-09-04T17:46:54.720 INFO:teuthology.orchestra.run.smithi092.stdout:3158 read 0x3c1a4b1 thru 0x3c1d786 (0x32d6 bytes) 2024-09-04T17:46:54.720 INFO:teuthology.orchestra.run.smithi092.stdout:3159 read 0x991eca0 thru 0x9927df7 (0x9158 bytes) 2024-09-04T17:46:54.720 INFO:teuthology.orchestra.run.smithi092.stdout:3160 write 0xb287280 thru 0xb28c90b (0x568c bytes) 2024-09-04T17:46:54.720 INFO:teuthology.orchestra.run.smithi092.stdout:3164 read 0xcd0bf83 thru 0xcd15221 (0x929f bytes) 2024-09-04T17:46:54.720 INFO:teuthology.orchestra.run.smithi092.stdout:3165 writesame 0x9766435 thru 0x976e74a data_size 0x2a(0x8316 bytes) 2024-09-04T17:46:54.720 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:54.720 INFO:teuthology.orchestra.run.smithi092.stdout:3170 read 0x2491ac9 thru 0x249d922 (0xbe5a bytes) 2024-09-04T17:46:54.720 INFO:teuthology.orchestra.run.smithi092.stdout:3171 read 0x63cb7c8 thru 0x63d0fa2 (0x57db bytes) 2024-09-04T17:46:54.720 INFO:teuthology.orchestra.run.smithi092.stdout:3174 punch from 0xbbe02fe to 0xbbec2d6, (0xbfd8 bytes) 2024-09-04T17:46:54.721 INFO:teuthology.orchestra.run.smithi092.stdout:3175 read 0x9677dd2 thru 0x9680f58 (0x9187 bytes) 2024-09-04T17:46:54.721 INFO:teuthology.orchestra.run.smithi092.stdout:3176 write 0xbbba264 thru 0xbbc8293 (0xe030 bytes) 2024-09-04T17:46:54.721 INFO:teuthology.orchestra.run.smithi092.stdout:3177 punch from 0xdb31467 to 0xdb36a98, (0x5631 bytes) 2024-09-04T17:46:54.721 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:54.721 INFO:teuthology.orchestra.run.smithi092.stdout:3180 write 0x78cf843 thru 0x78cfbc8 (0x386 bytes) 2024-09-04T17:46:54.721 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:54.721 INFO:teuthology.orchestra.run.smithi092.stdout:3182 punch from 0xb3b78e8 to 0xb3b8306, (0xa1e bytes) 2024-09-04T17:46:54.721 INFO:teuthology.orchestra.run.smithi092.stdout:3183 read 0x45eebb6 thru 0x45fdbff (0xf04a bytes) 2024-09-04T17:46:54.721 INFO:teuthology.orchestra.run.smithi092.stdout:3184 read 0x2497d8 thru 0x257cd2 (0xe4fb bytes) 2024-09-04T17:46:54.721 INFO:teuthology.orchestra.run.smithi092.stdout:3186 clone 23 order 21 su 1048576 sc 15 2024-09-04T17:46:54.818 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:54.817+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:54.890 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:54.890 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:46:54.890 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:46:54.890 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:54.904 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-04T17:46:54.919 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:54.919+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:55.020 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:55.021+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:55.073 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:55.074 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:46:55.074 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:46:55.074 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:46:55.074 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:55.088 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-04T17:46:55.121 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:55.119+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:55.192 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-04T17:46:55.222 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:55.221+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:55.260 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:55.260 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:46:55.260 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:46:55.260 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:46:55.260 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.261 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8831 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2453504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:55.261 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:46:55.261 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 42.200975970000002, 2024-09-04T17:46:55.261 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14731111799999999, 2024-09-04T17:46:55.261 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:55.261 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:55.261 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:55.262 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:55.262 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:55.262 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:55.262 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8831 2024-09-04T17:46:55.262 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.262 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:55.263 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.263 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:55.263 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:46:55.263 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:55.263 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.263 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.264 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:55.264 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:46:55.264 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:55.264 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.264 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.264 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:55.265 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057811+0000", 2024-09-04T17:46:55.265 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.083e-06 2024-09-04T17:46:55.265 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.266 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.266 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:55.266 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057814+0000", 2024-09-04T17:46:55.266 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.407e-06 2024-09-04T17:46:55.266 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.266 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.266 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:55.266 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057825+0000", 2024-09-04T17:46:55.266 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.17e-05 2024-09-04T17:46:55.266 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.266 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.266 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:55.266 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057836+0000", 2024-09-04T17:46:55.266 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0283e-05 2024-09-04T17:46:55.267 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.267 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.267 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:55.267 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.204762+0000", 2024-09-04T17:46:55.267 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.146926205 2024-09-04T17:46:55.267 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.267 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.267 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:55.267 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.204895+0000", 2024-09-04T17:46:55.267 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013328999999999999 2024-09-04T17:46:55.267 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.267 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.268 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:55.268 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205121+0000", 2024-09-04T17:46:55.268 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00022614999999999999 2024-09-04T17:46:55.268 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:55.268 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:55.268 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:55.269 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.269 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.269 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8832 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2490368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:55.269 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:46:55.269 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 42.200937936999999, 2024-09-04T17:46:55.269 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14770012299999999, 2024-09-04T17:46:55.269 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:55.270 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:55.270 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:55.270 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:55.270 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:55.270 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:55.271 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8832 2024-09-04T17:46:55.271 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.271 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:55.271 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.271 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:55.271 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:46:55.271 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:55.272 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.272 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.272 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:55.272 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:46:55.272 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:55.272 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.272 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.272 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:55.273 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057849+0000", 2024-09-04T17:46:55.273 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6700000000000003e-07 2024-09-04T17:46:55.273 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.273 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.273 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:55.273 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057852+0000", 2024-09-04T17:46:55.273 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6359999999999998e-06 2024-09-04T17:46:55.273 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.274 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.274 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:55.274 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057860+0000", 2024-09-04T17:46:55.274 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8140000000000007e-06 2024-09-04T17:46:55.274 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.274 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.274 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:55.274 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057868+0000", 2024-09-04T17:46:55.274 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1999999999999997e-06 2024-09-04T17:46:55.275 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.275 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.275 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:55.275 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205208+0000", 2024-09-04T17:46:55.275 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14734067300000001 2024-09-04T17:46:55.275 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.275 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.275 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:55.276 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205347+0000", 2024-09-04T17:46:55.276 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013914900000000001 2024-09-04T17:46:55.276 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.276 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.276 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:55.276 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205548+0000", 2024-09-04T17:46:55.276 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000200984 2024-09-04T17:46:55.277 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:55.277 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:55.277 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:55.277 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.277 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.277 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8833 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2527232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:55.277 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:46:55.277 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 42.200906066999998, 2024-09-04T17:46:55.277 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148062572, 2024-09-04T17:46:55.278 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:55.278 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:55.278 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:55.278 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:55.278 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:55.278 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:55.278 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8833 2024-09-04T17:46:55.278 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.279 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:55.279 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.279 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:55.279 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:46:55.279 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:55.279 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.279 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.279 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:55.279 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:46:55.279 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:55.279 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.279 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.280 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:55.280 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057881+0000", 2024-09-04T17:46:55.280 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4300000000000002e-07 2024-09-04T17:46:55.280 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.280 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.280 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:55.280 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057893+0000", 2024-09-04T17:46:55.280 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1973999999999999e-05 2024-09-04T17:46:55.280 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.280 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.280 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:55.280 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057906+0000", 2024-09-04T17:46:55.280 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2716e-05 2024-09-04T17:46:55.281 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.281 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.281 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:55.281 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057913+0000", 2024-09-04T17:46:55.281 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3930000000000002e-06 2024-09-04T17:46:55.281 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.281 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.281 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:55.281 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205621+0000", 2024-09-04T17:46:55.281 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14770788800000001 2024-09-04T17:46:55.281 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.281 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.282 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:55.282 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205740+0000", 2024-09-04T17:46:55.282 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000118936 2024-09-04T17:46:55.282 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.282 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.282 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:55.282 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205943+0000", 2024-09-04T17:46:55.282 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020292199999999999 2024-09-04T17:46:55.282 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:55.282 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:55.282 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:55.283 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.283 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.283 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8834 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2564096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:55.283 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:46:55.283 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 42.200862788000002, 2024-09-04T17:46:55.283 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14846753200000001, 2024-09-04T17:46:55.283 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:55.283 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:55.283 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:55.283 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:55.283 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:55.283 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:55.284 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8834 2024-09-04T17:46:55.284 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.284 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:55.284 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.284 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:55.284 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:46:55.284 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:55.284 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.284 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.284 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:55.284 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:46:55.285 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:55.285 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.285 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.285 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:55.285 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057925+0000", 2024-09-04T17:46:55.285 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0270000000000001e-06 2024-09-04T17:46:55.285 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.285 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.285 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:55.285 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057936+0000", 2024-09-04T17:46:55.286 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.112e-05 2024-09-04T17:46:55.286 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.286 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.286 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:55.286 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057946+0000", 2024-09-04T17:46:55.286 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.009e-05 2024-09-04T17:46:55.286 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.286 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.286 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:55.286 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057953+0000", 2024-09-04T17:46:55.286 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.6229999999999997e-06 2024-09-04T17:46:55.287 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.287 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.287 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:55.287 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206027+0000", 2024-09-04T17:46:55.287 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148073591 2024-09-04T17:46:55.287 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.287 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.287 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:55.287 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206161+0000", 2024-09-04T17:46:55.287 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013411599999999999 2024-09-04T17:46:55.287 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.287 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.287 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:55.287 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206391+0000", 2024-09-04T17:46:55.288 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000229965 2024-09-04T17:46:55.288 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:55.288 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:55.288 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:55.288 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.288 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.288 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8835 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2600960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:55.288 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:46:55.288 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 42.200823137999997, 2024-09-04T17:46:55.289 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14882941599999999, 2024-09-04T17:46:55.289 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:55.289 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:55.289 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:55.290 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:55.290 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:55.290 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:55.290 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8835 2024-09-04T17:46:55.290 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.290 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:55.290 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.291 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:55.291 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:46:55.291 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:55.291 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.291 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.291 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:55.291 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:46:55.291 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:55.291 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.291 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.291 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:55.292 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058544+0000", 2024-09-04T17:46:55.292 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00058061899999999999 2024-09-04T17:46:55.292 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.292 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.292 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:55.292 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058550+0000", 2024-09-04T17:46:55.292 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.1129999999999997e-06 2024-09-04T17:46:55.292 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.292 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.292 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:55.292 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058566+0000", 2024-09-04T17:46:55.292 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5820000000000001e-05 2024-09-04T17:46:55.292 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.293 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.293 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:55.293 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058578+0000", 2024-09-04T17:46:55.293 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2537e-05 2024-09-04T17:46:55.293 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.293 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.293 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:55.293 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206470+0000", 2024-09-04T17:46:55.293 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14789153799999999 2024-09-04T17:46:55.294 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.294 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.294 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:55.294 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206589+0000", 2024-09-04T17:46:55.294 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000118837 2024-09-04T17:46:55.294 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.294 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.294 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:55.295 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206793+0000", 2024-09-04T17:46:55.296 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000203952 2024-09-04T17:46:55.296 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:55.296 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:55.296 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:55.296 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.296 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.296 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8836 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2654208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:55.296 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:46:55.296 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 42.200190450000001, 2024-09-04T17:46:55.296 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14862433999999999, 2024-09-04T17:46:55.296 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:55.296 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:55.297 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:55.297 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:55.297 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:55.297 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:55.297 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8836 2024-09-04T17:46:55.297 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.297 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:55.297 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.297 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:55.298 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:46:55.298 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:55.298 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.298 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.298 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:55.298 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:46:55.298 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:55.298 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.298 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.298 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:55.298 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062663+0000", 2024-09-04T17:46:55.298 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0040670619999999998 2024-09-04T17:46:55.298 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.299 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.299 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:55.299 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062668+0000", 2024-09-04T17:46:55.299 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4199999999999998e-06 2024-09-04T17:46:55.299 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.299 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.299 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:55.299 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062689+0000", 2024-09-04T17:46:55.299 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0356999999999998e-05 2024-09-04T17:46:55.299 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.299 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.299 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:55.299 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062726+0000", 2024-09-04T17:46:55.299 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7302999999999998e-05 2024-09-04T17:46:55.300 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.300 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.300 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:55.300 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206875+0000", 2024-09-04T17:46:55.300 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14414883100000001 2024-09-04T17:46:55.300 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.300 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.300 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:55.300 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.207016+0000", 2024-09-04T17:46:55.300 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000140822 2024-09-04T17:46:55.300 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.300 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.300 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:55.301 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.207220+0000", 2024-09-04T17:46:55.301 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020454499999999999 2024-09-04T17:46:55.301 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:55.301 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:55.301 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:55.301 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.301 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.301 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8844 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2928640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:55.301 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:46:55.301 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 42.195826367000002, 2024-09-04T17:46:55.301 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14752153100000001, 2024-09-04T17:46:55.301 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:55.302 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:55.302 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:55.302 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:55.302 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:55.302 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:55.302 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8844 2024-09-04T17:46:55.302 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.302 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:55.302 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.302 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:55.302 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:46:55.303 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:55.303 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.303 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.303 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:55.303 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:46:55.303 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:55.303 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.303 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.303 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:55.303 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062961+0000", 2024-09-04T17:46:55.303 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6700000000000003e-07 2024-09-04T17:46:55.303 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.304 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.304 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:55.304 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062973+0000", 2024-09-04T17:46:55.304 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2109999999999999e-05 2024-09-04T17:46:55.304 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.304 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.304 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:55.304 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062982+0000", 2024-09-04T17:46:55.304 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.6900000000000004e-06 2024-09-04T17:46:55.304 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.304 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.305 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:55.305 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062990+0000", 2024-09-04T17:46:55.305 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1899999999999998e-06 2024-09-04T17:46:55.305 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.305 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.305 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:55.305 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210090+0000", 2024-09-04T17:46:55.305 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14710026300000001 2024-09-04T17:46:55.305 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.305 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.305 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:55.305 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210225+0000", 2024-09-04T17:46:55.305 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013532400000000001 2024-09-04T17:46:55.306 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.306 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.306 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:55.306 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210481+0000", 2024-09-04T17:46:55.306 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00025628700000000001 2024-09-04T17:46:55.306 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:55.306 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:55.306 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:55.306 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.306 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.306 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8845 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2965504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:55.306 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:46:55.306 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 42.195786644000002, 2024-09-04T17:46:55.306 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147916819, 2024-09-04T17:46:55.307 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:55.307 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:55.307 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:55.307 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:55.307 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:55.307 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:55.307 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8845 2024-09-04T17:46:55.307 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.307 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:55.307 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.307 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:55.307 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:46:55.307 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:55.308 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.308 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.308 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:55.308 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:46:55.308 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:55.308 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.308 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.308 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:55.308 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:46:55.308 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.0399999999999995e-07 2024-09-04T17:46:55.308 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.308 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.308 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:55.309 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063003+0000", 2024-09-04T17:46:55.309 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6529999999999998e-06 2024-09-04T17:46:55.309 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.309 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.309 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:55.309 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063012+0000", 2024-09-04T17:46:55.309 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9500000000000007e-06 2024-09-04T17:46:55.309 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.309 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.309 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:55.309 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063027+0000", 2024-09-04T17:46:55.309 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5443000000000001e-05 2024-09-04T17:46:55.309 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.310 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.310 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:55.310 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210571+0000", 2024-09-04T17:46:55.310 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14754353100000001 2024-09-04T17:46:55.310 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.310 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.310 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:55.310 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210714+0000", 2024-09-04T17:46:55.310 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014295599999999999 2024-09-04T17:46:55.310 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.310 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.310 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:55.310 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210916+0000", 2024-09-04T17:46:55.311 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020258200000000001 2024-09-04T17:46:55.311 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:55.311 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:55.311 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:55.311 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.311 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.311 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8846 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3002368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:55.311 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:46:55.311 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 42.195749106999997, 2024-09-04T17:46:55.311 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14828011599999999, 2024-09-04T17:46:55.311 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:55.311 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:55.311 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:55.311 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:55.312 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:55.312 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:55.312 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8846 2024-09-04T17:46:55.312 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.312 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:55.312 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.312 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:55.312 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:46:55.312 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:55.313 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.313 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.313 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:55.313 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:46:55.313 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:55.313 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.313 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.313 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:55.313 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063038+0000", 2024-09-04T17:46:55.313 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.23e-07 2024-09-04T17:46:55.313 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.313 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.313 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:55.314 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063051+0000", 2024-09-04T17:46:55.314 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2757e-05 2024-09-04T17:46:55.314 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.314 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.314 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:55.314 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063070+0000", 2024-09-04T17:46:55.314 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9663000000000001e-05 2024-09-04T17:46:55.314 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.314 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.314 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:55.315 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063079+0000", 2024-09-04T17:46:55.315 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.0929999999999992e-06 2024-09-04T17:46:55.315 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.315 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.315 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:55.315 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210989+0000", 2024-09-04T17:46:55.315 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14790984500000001 2024-09-04T17:46:55.315 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.315 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.315 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:55.315 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211109+0000", 2024-09-04T17:46:55.316 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000120056 2024-09-04T17:46:55.316 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.316 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.316 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:55.316 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211317+0000", 2024-09-04T17:46:55.316 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020797900000000001 2024-09-04T17:46:55.316 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:55.316 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:55.316 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:55.316 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.316 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.316 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8847 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3039232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:55.316 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:46:55.317 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 42.195674674999999, 2024-09-04T17:46:55.317 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14859408900000001, 2024-09-04T17:46:55.317 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:55.317 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:55.317 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:55.317 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:55.317 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:55.317 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:55.317 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8847 2024-09-04T17:46:55.317 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.317 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:55.317 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.317 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:55.318 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:46:55.318 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:55.318 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.318 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.318 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:55.318 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:46:55.318 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:55.318 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.318 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.318 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:55.318 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063113+0000", 2024-09-04T17:46:55.318 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.204e-06 2024-09-04T17:46:55.319 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.319 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.319 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:55.319 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063116+0000", 2024-09-04T17:46:55.319 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.3330000000000001e-06 2024-09-04T17:46:55.319 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.319 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.319 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:55.319 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063126+0000", 2024-09-04T17:46:55.319 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0166999999999999e-05 2024-09-04T17:46:55.319 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.319 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.320 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:55.320 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063134+0000", 2024-09-04T17:46:55.320 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4329999999999998e-06 2024-09-04T17:46:55.320 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.320 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.320 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:55.320 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211392+0000", 2024-09-04T17:46:55.320 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148258109 2024-09-04T17:46:55.320 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.320 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.320 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:55.320 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211506+0000", 2024-09-04T17:46:55.320 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000113813 2024-09-04T17:46:55.321 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.321 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.321 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:55.321 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211706+0000", 2024-09-04T17:46:55.321 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020002999999999999 2024-09-04T17:46:55.321 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:55.321 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:55.321 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:55.321 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.321 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.321 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8848 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:55.321 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:46:55.321 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 42.195642337999999, 2024-09-04T17:46:55.322 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14907967599999999, 2024-09-04T17:46:55.322 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:55.322 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:55.322 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:55.322 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:55.322 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:55.322 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:55.322 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8848 2024-09-04T17:46:55.322 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.322 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:55.322 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.322 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:55.322 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:46:55.322 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:55.323 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.323 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.323 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:55.323 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:46:55.323 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:55.323 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.323 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.323 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:55.323 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070589+0000", 2024-09-04T17:46:55.323 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.007445386 2024-09-04T17:46:55.323 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.323 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.323 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:55.324 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070600+0000", 2024-09-04T17:46:55.324 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0346999999999999e-05 2024-09-04T17:46:55.324 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.324 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.324 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:55.324 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070622+0000", 2024-09-04T17:46:55.324 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2289999999999998e-05 2024-09-04T17:46:55.324 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.324 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.324 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:55.324 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070636+0000", 2024-09-04T17:46:55.324 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4093e-05 2024-09-04T17:46:55.324 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.325 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.325 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:55.325 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211815+0000", 2024-09-04T17:46:55.325 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14117874699999999 2024-09-04T17:46:55.325 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.325 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.325 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:55.325 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211983+0000", 2024-09-04T17:46:55.325 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00016852800000000001 2024-09-04T17:46:55.325 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.325 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.325 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:55.325 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.212224+0000", 2024-09-04T17:46:55.326 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024028499999999999 2024-09-04T17:46:55.326 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:55.326 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:55.326 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:55.326 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.326 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.326 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8862 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2019328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:55.326 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:46:55.326 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 42.187592870000003, 2024-09-04T17:46:55.326 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14755236399999999, 2024-09-04T17:46:55.326 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:55.326 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:55.326 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:55.327 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:55.327 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:55.327 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:55.327 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8862 2024-09-04T17:46:55.327 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.327 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:55.327 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.327 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:55.327 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:46:55.327 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:55.327 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.327 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.327 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:55.328 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:46:55.328 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:55.328 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.328 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.328 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:55.328 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071194+0000", 2024-09-04T17:46:55.328 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.5000000000000002e-07 2024-09-04T17:46:55.328 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.328 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.328 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:55.329 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071197+0000", 2024-09-04T17:46:55.329 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5900000000000002e-06 2024-09-04T17:46:55.329 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.329 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.329 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:55.329 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071206+0000", 2024-09-04T17:46:55.329 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9700000000000005e-06 2024-09-04T17:46:55.329 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.329 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.329 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:55.329 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071213+0000", 2024-09-04T17:46:55.329 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.9070000000000001e-06 2024-09-04T17:46:55.329 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.329 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.330 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:55.330 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218396+0000", 2024-09-04T17:46:55.330 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147183273 2024-09-04T17:46:55.330 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.330 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.330 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:55.330 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218538+0000", 2024-09-04T17:46:55.330 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014194199999999999 2024-09-04T17:46:55.330 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.330 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.330 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:55.330 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218746+0000", 2024-09-04T17:46:55.330 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000208032 2024-09-04T17:46:55.330 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:55.331 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:55.331 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:55.331 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.331 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.331 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8863 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2056192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:55.331 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:46:55.331 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 42.187564559999998, 2024-09-04T17:46:55.331 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14792281099999999, 2024-09-04T17:46:55.331 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:55.331 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:55.332 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:55.332 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:55.332 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:55.332 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:55.332 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8863 2024-09-04T17:46:55.332 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.332 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:55.332 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.332 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:55.332 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:46:55.332 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:55.332 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.332 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.333 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:55.333 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:46:55.333 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:55.333 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.333 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.333 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:55.333 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:46:55.333 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1299999999999999e-07 2024-09-04T17:46:55.333 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.333 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.333 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:55.333 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071225+0000", 2024-09-04T17:46:55.333 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6199999999999999e-06 2024-09-04T17:46:55.334 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.334 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.334 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:55.334 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071234+0000", 2024-09-04T17:46:55.334 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8769999999999999e-06 2024-09-04T17:46:55.334 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.334 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.334 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:55.334 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071251+0000", 2024-09-04T17:46:55.334 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.738e-05 2024-09-04T17:46:55.334 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.334 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.334 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:55.334 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218798+0000", 2024-09-04T17:46:55.335 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14754645 2024-09-04T17:46:55.335 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.335 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.335 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:55.335 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218926+0000", 2024-09-04T17:46:55.335 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000128446 2024-09-04T17:46:55.335 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.335 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.335 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:55.335 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219145+0000", 2024-09-04T17:46:55.335 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021832500000000001 2024-09-04T17:46:55.335 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:55.335 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:55.336 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:55.336 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.336 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.336 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8864 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2093056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:55.336 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:46:55.336 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 42.187522983999997, 2024-09-04T17:46:55.336 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148277309, 2024-09-04T17:46:55.336 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:55.336 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:55.336 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:55.336 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:55.336 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:55.336 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:55.337 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8864 2024-09-04T17:46:55.337 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.337 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:55.337 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.337 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:55.337 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:46:55.337 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:55.337 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.337 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.337 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:55.337 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:46:55.337 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:55.337 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.337 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.338 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:55.338 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071264+0000", 2024-09-04T17:46:55.338 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3e-07 2024-09-04T17:46:55.338 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.338 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.338 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:55.338 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071267+0000", 2024-09-04T17:46:55.338 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0139999999999999e-06 2024-09-04T17:46:55.338 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.338 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.338 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:55.338 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071276+0000", 2024-09-04T17:46:55.338 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1689999999999998e-06 2024-09-04T17:46:55.339 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.339 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.339 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:55.339 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071283+0000", 2024-09-04T17:46:55.339 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2269999999999996e-06 2024-09-04T17:46:55.339 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.339 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.339 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:55.339 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219195+0000", 2024-09-04T17:46:55.339 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147911868 2024-09-04T17:46:55.339 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.340 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.340 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:55.340 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219324+0000", 2024-09-04T17:46:55.340 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00012835900000000001 2024-09-04T17:46:55.340 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.340 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.340 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:55.340 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219541+0000", 2024-09-04T17:46:55.340 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000216942 2024-09-04T17:46:55.340 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:55.340 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:55.340 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:55.341 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.341 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.341 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8865 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2109440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:55.341 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:46:55.341 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 42.187493523999997, 2024-09-04T17:46:55.341 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14865747300000001, 2024-09-04T17:46:55.341 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:55.341 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:55.341 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:55.341 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:55.341 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:55.341 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:55.341 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8865 2024-09-04T17:46:55.342 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.342 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:55.342 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.342 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:55.342 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:46:55.342 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:55.342 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.342 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.342 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:55.342 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:46:55.342 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:55.342 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.342 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.342 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:55.343 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:46:55.343 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7999999999999995e-07 2024-09-04T17:46:55.343 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.343 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.343 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:55.343 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071296+0000", 2024-09-04T17:46:55.343 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6900000000000001e-06 2024-09-04T17:46:55.343 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.343 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.343 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:55.343 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071305+0000", 2024-09-04T17:46:55.344 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.5499999999999995e-06 2024-09-04T17:46:55.344 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.344 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.344 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:55.344 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071312+0000", 2024-09-04T17:46:55.344 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1559999999999998e-06 2024-09-04T17:46:55.344 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.344 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.344 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:55.344 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219591+0000", 2024-09-04T17:46:55.344 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14827944900000001 2024-09-04T17:46:55.344 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.345 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.345 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:55.345 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219731+0000", 2024-09-04T17:46:55.345 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013931600000000001 2024-09-04T17:46:55.345 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.345 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.345 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:55.345 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219950+0000", 2024-09-04T17:46:55.345 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021963199999999999 2024-09-04T17:46:55.345 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:55.345 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:55.345 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:55.345 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.345 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.346 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8866 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2146304~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:55.346 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:46:55.346 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 42.187465291000002, 2024-09-04T17:46:55.346 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14906117899999999, 2024-09-04T17:46:55.346 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:55.346 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:55.346 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:55.346 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:55.346 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:55.346 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:55.346 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8866 2024-09-04T17:46:55.346 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.346 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:55.347 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.347 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:55.347 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:46:55.347 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:55.347 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.347 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.347 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:55.347 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:46:55.347 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:55.347 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.347 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.347 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:55.347 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071322+0000", 2024-09-04T17:46:55.347 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7999999999999995e-07 2024-09-04T17:46:55.348 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.348 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.348 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:55.348 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071338+0000", 2024-09-04T17:46:55.348 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5872999999999999e-05 2024-09-04T17:46:55.348 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.348 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.348 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:55.348 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071347+0000", 2024-09-04T17:46:55.348 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4669999999999993e-06 2024-09-04T17:46:55.348 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.349 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.349 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:55.349 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071354+0000", 2024-09-04T17:46:55.349 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2099999999999996e-06 2024-09-04T17:46:55.349 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.349 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.349 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:55.349 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220008+0000", 2024-09-04T17:46:55.349 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148653642 2024-09-04T17:46:55.349 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.349 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.350 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:55.350 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220155+0000", 2024-09-04T17:46:55.350 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014726600000000001 2024-09-04T17:46:55.350 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.350 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.350 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:55.350 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220382+0000", 2024-09-04T17:46:55.350 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00022704100000000001 2024-09-04T17:46:55.350 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:55.350 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:55.350 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:55.350 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.350 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.351 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8867 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2183168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:55.351 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:46:55.351 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 42.187421938, 2024-09-04T17:46:55.351 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.149445831, 2024-09-04T17:46:55.351 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:55.351 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:55.351 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:55.351 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:55.351 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:55.351 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:55.351 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8867 2024-09-04T17:46:55.351 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.351 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:55.351 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.352 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:55.352 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:46:55.352 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:55.352 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.352 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.352 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:55.352 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:46:55.352 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:55.352 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.352 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.352 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:55.352 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078653+0000", 2024-09-04T17:46:55.352 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0072890430000000003 2024-09-04T17:46:55.353 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.353 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.353 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:55.353 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078664+0000", 2024-09-04T17:46:55.353 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0307e-05 2024-09-04T17:46:55.353 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.353 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.353 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:55.353 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078690+0000", 2024-09-04T17:46:55.353 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6780000000000001e-05 2024-09-04T17:46:55.353 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.353 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.353 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:55.354 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078706+0000", 2024-09-04T17:46:55.354 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5676999999999999e-05 2024-09-04T17:46:55.354 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.354 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.354 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:55.354 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220467+0000", 2024-09-04T17:46:55.354 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14176103500000001 2024-09-04T17:46:55.354 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.354 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.354 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:55.354 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220595+0000", 2024-09-04T17:46:55.354 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000127427 2024-09-04T17:46:55.354 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.355 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.355 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:55.355 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220810+0000", 2024-09-04T17:46:55.355 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021556200000000001 2024-09-04T17:46:55.355 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:55.355 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:55.355 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:55.355 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.355 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.355 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8880 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2658304~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:55.355 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.079158+0000", 2024-09-04T17:46:55.355 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 42.179628659000002, 2024-09-04T17:46:55.355 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14718968499999999, 2024-09-04T17:46:55.356 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:55.356 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:55.356 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:55.356 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:55.356 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:55.356 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:55.356 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8880 2024-09-04T17:46:55.356 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.356 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:55.356 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.356 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:55.356 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079158+0000", 2024-09-04T17:46:55.356 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:55.357 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.357 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.357 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:55.357 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079158+0000", 2024-09-04T17:46:55.357 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:55.357 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.357 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.357 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:55.357 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079158+0000", 2024-09-04T17:46:55.357 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3e-07 2024-09-04T17:46:55.357 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.357 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.358 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:55.358 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079161+0000", 2024-09-04T17:46:55.358 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.57e-06 2024-09-04T17:46:55.358 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.358 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.358 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:55.358 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079170+0000", 2024-09-04T17:46:55.358 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.0170000000000002e-06 2024-09-04T17:46:55.358 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.358 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.358 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:55.358 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079177+0000", 2024-09-04T17:46:55.358 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.0530000000000001e-06 2024-09-04T17:46:55.359 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.359 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.359 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:55.359 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.225946+0000", 2024-09-04T17:46:55.359 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14676936500000001 2024-09-04T17:46:55.359 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.359 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.359 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:55.359 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226098+0000", 2024-09-04T17:46:55.359 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000151805 2024-09-04T17:46:55.359 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.359 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.359 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:55.359 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226347+0000", 2024-09-04T17:46:55.360 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024914499999999999 2024-09-04T17:46:55.360 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:55.360 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:55.360 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:55.360 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.360 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.360 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8881 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2695168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:55.360 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:46:55.360 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 42.179599869, 2024-09-04T17:46:55.360 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147543547, 2024-09-04T17:46:55.360 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:55.360 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:55.360 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:55.361 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:55.361 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:55.361 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:55.361 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8881 2024-09-04T17:46:55.361 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.361 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:55.361 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.361 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:55.361 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:46:55.361 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:55.361 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.361 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.361 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:55.362 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:46:55.362 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:55.362 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.362 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.362 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:55.362 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079187+0000", 2024-09-04T17:46:55.362 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7700000000000004e-07 2024-09-04T17:46:55.362 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.362 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.362 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:55.362 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079198+0000", 2024-09-04T17:46:55.362 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0849999999999999e-05 2024-09-04T17:46:55.362 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.362 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.363 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:55.363 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079207+0000", 2024-09-04T17:46:55.363 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5599999999999999e-06 2024-09-04T17:46:55.363 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.363 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.363 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:55.363 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079224+0000", 2024-09-04T17:46:55.363 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6387000000000001e-05 2024-09-04T17:46:55.363 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.363 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.363 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:55.363 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226414+0000", 2024-09-04T17:46:55.363 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14718975000000001 2024-09-04T17:46:55.363 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.364 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.364 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:55.364 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226534+0000", 2024-09-04T17:46:55.364 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000120288 2024-09-04T17:46:55.364 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.364 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.364 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:55.364 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226730+0000", 2024-09-04T17:46:55.364 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000196035 2024-09-04T17:46:55.364 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:55.364 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:55.364 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:55.364 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.365 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.365 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8882 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2732032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:55.365 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:46:55.365 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 42.179551654999997, 2024-09-04T17:46:55.365 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14795022399999999, 2024-09-04T17:46:55.365 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:55.365 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:55.365 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:55.365 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:55.365 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:55.365 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:55.365 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8882 2024-09-04T17:46:55.366 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.366 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:55.366 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.366 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:55.366 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:46:55.366 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:55.366 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.366 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.368 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:55.368 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:46:55.368 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:55.368 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.368 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.368 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:55.368 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086588+0000", 2024-09-04T17:46:55.369 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0073530959999999999 2024-09-04T17:46:55.369 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.369 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.369 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:55.369 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086598+0000", 2024-09-04T17:46:55.369 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0122999999999999e-05 2024-09-04T17:46:55.369 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.369 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.369 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:55.369 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086626+0000", 2024-09-04T17:46:55.369 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.7914000000000001e-05 2024-09-04T17:46:55.369 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.370 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.370 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:55.370 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086641+0000", 2024-09-04T17:46:55.370 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5270000000000001e-05 2024-09-04T17:46:55.370 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.370 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.370 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:55.370 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226801+0000", 2024-09-04T17:46:55.370 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.140160321 2024-09-04T17:46:55.370 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.370 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.370 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:55.370 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226951+0000", 2024-09-04T17:46:55.370 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014965200000000001 2024-09-04T17:46:55.371 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:55.371 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:55.371 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:55.371 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.227185+0000", 2024-09-04T17:46:55.371 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00023384799999999999 2024-09-04T17:46:55.371 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:55.371 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:55.371 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:55.371 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:55.371 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:55.371 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:55.372 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:46:55.373 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:46:55.373 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:46:55.374 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:46:55.374 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-04T17:46:55.374 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:55.371+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:55.377 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-04T17:46:55.377 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-04T17:46:55.394 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-04T17:46:55.394 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-04T17:46:55.414 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:46:55.415 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:46:55.433 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:46:55.434 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:46:55.434 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-04T17:46:55.452 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:46:55.452 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:46:55.457 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-04T17:46:55.457 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-04T17:46:55.500 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:55.499+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:55.532 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is set 2024-09-04T17:46:55.547 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:55.547 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:55.562 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-04T17:46:55.608 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:55.609+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:55.675 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:55.671+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:55.685 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:55.685 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:55.699 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-04T17:46:55.777 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:46:55.775+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:46:55.865 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:55.865 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:55.878 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:55.875+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:55.880 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-04T17:46:55.909 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #21, image image_client.0-clone21 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent22 2024-09-04T17:46:55.917 INFO:teuthology.orchestra.run.smithi092.stdout:3187 write 0xe2155b thru 0xe2e5cb (0xd071 bytes) 2024-09-04T17:46:55.932 INFO:teuthology.orchestra.run.smithi092.stdout:3188 write 0xe43c64c thru 0xe4484fc (0xbeb1 bytes) 2024-09-04T17:46:55.937 INFO:teuthology.orchestra.run.smithi092.stdout:3189 punch from 0x76e4f5e to 0x76eb3c8, (0x646a bytes) 2024-09-04T17:46:55.937 INFO:teuthology.orchestra.run.smithi092.stdout:3192 write 0xe3e1178 thru 0xe3efd5f (0xebe8 bytes) 2024-09-04T17:46:55.944 INFO:teuthology.orchestra.run.smithi092.stdout:3193 read 0x56c22fe thru 0x56d173c (0xf43f bytes) 2024-09-04T17:46:55.944 INFO:teuthology.orchestra.run.smithi092.stdout:3194 punch from 0x937552 to 0x943d47, (0xc7f5 bytes) 2024-09-04T17:46:55.944 INFO:teuthology.orchestra.run.smithi092.stdout:3195 read 0x150f860 thru 0x15125f0 (0x2d91 bytes) 2024-09-04T17:46:55.945 INFO:teuthology.orchestra.run.smithi092.stdout:3196 read 0x2dc27b0 thru 0x2dcc9f9 (0xa24a bytes) 2024-09-04T17:46:55.945 INFO:teuthology.orchestra.run.smithi092.stdout:3198 write 0x82955d6 thru 0x829a24b (0x4c76 bytes) 2024-09-04T17:46:55.953 INFO:teuthology.orchestra.run.smithi092.stdout:3200 write 0xad3315a thru 0xad3b2cd (0x8174 bytes) 2024-09-04T17:46:55.959 INFO:teuthology.orchestra.run.smithi092.stdout:3201 writesame 0x4f5d948 thru 0x4f63ec8 data_size 0x1(0x6581 bytes) 2024-09-04T17:46:55.981 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:55.979+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:55.994 INFO:teuthology.orchestra.run.smithi092.stdout:3202 writesame 0xb82ba49 thru 0xb8355f2 data_size 0x19(0x9baa bytes) 2024-09-04T17:46:56.003 INFO:teuthology.orchestra.run.smithi092.stdout:3203 writesame 0x41ce514 thru 0x41d3bcf data_size 0x2(0x56bc bytes) 2024-09-04T17:46:56.012 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:56.012 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:56.025 INFO:teuthology.orchestra.run.smithi092.stdout:3204 trunc from 0xe695ea8 to 0xd079634 2024-09-04T17:46:56.029 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-04T17:46:56.054 INFO:teuthology.orchestra.run.smithi092.stdout:3206 writesame 0x71503da thru 0x7157695 data_size 0x7(0x72bc bytes) 2024-09-04T17:46:56.062 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:56.063 INFO:teuthology.orchestra.run.smithi092.stdout:3208 read 0xba886bf thru 0xba921f5 (0x9b37 bytes) 2024-09-04T17:46:56.063 INFO:teuthology.orchestra.run.smithi092.stdout:3209 write 0x707f1f0 thru 0x70882a2 (0x90b3 bytes) 2024-09-04T17:46:56.076 INFO:teuthology.orchestra.run.smithi092.stdout:3210 trunc from 0xd079634 to 0x9f69a11 2024-09-04T17:46:56.082 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:56.079+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:56.121 INFO:teuthology.orchestra.run.smithi092.stdout:3211 read 0x4dc20ee thru 0x4dc3be1 (0x1af4 bytes) 2024-09-04T17:46:56.121 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:56.121 INFO:teuthology.orchestra.run.smithi092.stdout:3214 punch from 0x39386e9 to 0x393b271, (0x2b88 bytes) 2024-09-04T17:46:56.121 INFO:teuthology.orchestra.run.smithi092.stdout:3215 punch from 0x525d886 to 0x525dac8, (0x242 bytes) 2024-09-04T17:46:56.121 INFO:teuthology.orchestra.run.smithi092.stdout:3216 write 0x9d78b98 thru 0x9d824ba (0x9923 bytes) 2024-09-04T17:46:56.124 INFO:teuthology.orchestra.run.smithi092.stdout:3217 punch from 0x3d0b95d to 0x3d18dd5, (0xd478 bytes) 2024-09-04T17:46:56.132 INFO:teuthology.orchestra.run.smithi092.stdout:3219 write 0x9f779a0 thru 0x9f858e7 (0xdf48 bytes) 2024-09-04T17:46:56.134 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:56.143 INFO:teuthology.orchestra.run.smithi092.stdout:3221 write 0xa7befd4 thru 0xa7c66a7 (0x76d4 bytes) 2024-09-04T17:46:56.147 INFO:teuthology.orchestra.run.smithi092.stdout:3223 read 0x1805ca1 thru 0x18127f1 (0xcb51 bytes) 2024-09-04T17:46:56.147 INFO:teuthology.orchestra.run.smithi092.stdout:3224 write 0x6b120a4 thru 0x6b13563 (0x14c0 bytes) 2024-09-04T17:46:56.154 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:56.162 INFO:teuthology.orchestra.run.smithi092.stdout:3226 write 0xa87012d thru 0xa877e91 (0x7d65 bytes) 2024-09-04T17:46:56.166 INFO:teuthology.orchestra.run.smithi092.stdout:3227 write 0x1a3c49f thru 0x1a428bd (0x641f bytes) 2024-09-04T17:46:56.177 INFO:teuthology.orchestra.run.smithi092.stdout:3228 clone 24 order 19 su 16384 sc 4 2024-09-04T17:46:56.184 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:56.181+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:56.185 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:56.185 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:56.199 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-04T17:46:56.284 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:56.283+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:56.362 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:56.362 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:56.376 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-04T17:46:56.420 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:46:56.419+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:46:56.485 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:46:56.483+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:46:56.547 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:56.547 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:56.565 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-04T17:46:56.608 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:56.607+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:56.665 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone23 from 0xa877e92 (overlap 0x9f69a11) to 0x8f78aa8 2024-09-04T17:46:56.688 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:56.683+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:56.741 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:56.741 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:56.758 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-04T17:46:56.789 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:56.789+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:56.865 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #22, image image_client.0-clone22 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent23 2024-09-04T17:46:56.889 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:56.887+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:56.928 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:56.928 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:56.946 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-04T17:46:56.990 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:56.987+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:57.091 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:57.087+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:57.112 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:57.112 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:57.133 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-04T17:46:57.228 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:57.227+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:57.292 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:57.291+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:57.300 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:57.300 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:57.317 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-04T17:46:57.320 INFO:teuthology.orchestra.run.smithi092.stdout:3229 write 0xd0a8b37 thru 0xd0a9cb3 (0x117d bytes) 2024-09-04T17:46:57.326 INFO:teuthology.orchestra.run.smithi092.stdout:3230 trunc from 0xd0a9cb4 to 0xc067e5b 2024-09-04T17:46:57.339 INFO:teuthology.orchestra.run.smithi092.stdout:3231 write 0x88b83b thru 0x88e1d6 (0x299c bytes) 2024-09-04T17:46:57.346 INFO:teuthology.orchestra.run.smithi092.stdout:3232 punch from 0x9c677f to 0x9d395e, (0xd1df bytes) 2024-09-04T17:46:57.346 INFO:teuthology.orchestra.run.smithi092.stdout:3233 read 0x6349a6f thru 0x63535af (0x9b41 bytes) 2024-09-04T17:46:57.346 INFO:teuthology.orchestra.run.smithi092.stdout:3234 read 0x9fa7550 thru 0x9fb57cd (0xe27e bytes) 2024-09-04T17:46:57.347 INFO:teuthology.orchestra.run.smithi092.stdout:3235 write 0x3b9249 thru 0x3c494c (0xb704 bytes) 2024-09-04T17:46:57.358 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:57.358 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:57.358 INFO:teuthology.orchestra.run.smithi092.stdout:3238 write 0x5ca51b6 thru 0x5cb4562 (0xf3ad bytes) 2024-09-04T17:46:57.393 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:57.391+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:57.393 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:57.394 INFO:teuthology.orchestra.run.smithi092.stdout:3240 read 0x3f9e5fa thru 0x3f9f79f (0x11a6 bytes) 2024-09-04T17:46:57.412 INFO:teuthology.orchestra.run.smithi092.stdout:3242 trunc from 0xc067e5b to 0xe7aff0 2024-09-04T17:46:57.458 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-04T17:46:57.459 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:57.459 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:57.474 INFO:teuthology.orchestra.run.smithi092.stdout:3243 write 0x2aa27bc thru 0x2aaa123 (0x7968 bytes) 2024-09-04T17:46:57.477 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-04T17:46:57.492 INFO:teuthology.orchestra.run.smithi092.stdout:3244 trunc from 0x2aaa124 to 0xa633ca2 2024-09-04T17:46:57.494 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:57.491+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:57.509 INFO:teuthology.orchestra.run.smithi092.stdout:3245 write 0xcbc0eee thru 0xcbcbfc0 (0xb0d3 bytes) 2024-09-04T17:46:57.516 INFO:teuthology.orchestra.run.smithi092.stdout:3246 read 0x3bff158 thru 0x3c06b4a (0x79f3 bytes) 2024-09-04T17:46:57.516 INFO:teuthology.orchestra.run.smithi092.stdout:3247 read 0x57d3d09 thru 0x57df47d (0xb775 bytes) 2024-09-04T17:46:57.516 INFO:teuthology.orchestra.run.smithi092.stdout:3248 clone 25 order 22 su 2097152 sc 5 2024-09-04T17:46:57.550 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [6] live_osds: [0, 1, 2, 4, 5, 7, 3] 2024-09-04T17:46:57.550 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-04T17:46:57.550 INFO:tasks.thrashosds.thrasher:Reweighting osd 2 to 0.6636290505305606 2024-09-04T17:46:57.550 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 0.6636290505305606 2024-09-04T17:46:57.561 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-04T17:46:57.600 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:57.599+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:57.628 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:46:57.629 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:46:57.632 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:46:57.633 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:46:57.648 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-04T17:46:57.654 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:46:57.654 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:46:57.669 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone24 from 0xcbcbfc1 (overlap 0xe7aff0) to 0xd084be 2024-09-04T17:46:57.675 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-04T17:46:57.675 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-04T17:46:57.695 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:57.691+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:57.697 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-04T17:46:57.697 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-04T17:46:57.732 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:46:57.732 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:46:57.753 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:46:57.754 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:46:57.754 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-04T17:46:57.780 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:46:57.780 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:46:57.784 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-04T17:46:57.785 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-04T17:46:57.793 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #23, image image_client.0-clone23 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent24 2024-09-04T17:46:57.796 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:46:57.795+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:46:57.811 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:57.811 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:46:57.811 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:46:57.812 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:46:57.812 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:57.831 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-04T17:46:57.898 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:57.897+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:57.991 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:46:57.991 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:46:57.992 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:46:57.992 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:46:57.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:57.992 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:4264 3.9 e30/24 3:970e51a0:::rbd_data.1134cfbce83d.0000000000000009:head v 30'83, mlcod=30'81)", 2024-09-04T17:46:57.992 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:07.027227+0000", 2024-09-04T17:46:57.992 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.963547105000004, 2024-09-04T17:46:57.992 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0096959590000000005, 2024-09-04T17:46:57.992 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:57.992 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:57.992 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:57.992 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:57.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:57.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:57.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.027227+0000", 2024-09-04T17:46:57.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:57.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:57.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:57.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:57.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.027227+0000", 2024-09-04T17:46:57.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:57.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:57.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:57.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:57.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.027229+0000", 2024-09-04T17:46:57.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4839999999999998e-06 2024-09-04T17:46:57.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:57.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:57.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:57.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.031984+0000", 2024-09-04T17:46:57.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047544789999999998 2024-09-04T17:46:57.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:57.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:57.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:57.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.032019+0000", 2024-09-04T17:46:57.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5253999999999998e-05 2024-09-04T17:46:57.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:57.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:57.995 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:57.995 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.032036+0000", 2024-09-04T17:46:57.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6713000000000001e-05 2024-09-04T17:46:57.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:57.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:57.995 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:57.995 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.032129+0000", 2024-09-04T17:46:57.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2961000000000007e-05 2024-09-04T17:46:57.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:57.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:57.995 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:57.996 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.032154+0000", 2024-09-04T17:46:57.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5148999999999999e-05 2024-09-04T17:46:57.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:57.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:57.996 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:57.996 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.036778+0000", 2024-09-04T17:46:57.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046239159999999996 2024-09-04T17:46:57.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:57.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:57.996 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:57.996 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.036923+0000", 2024-09-04T17:46:57.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014500299999999999 2024-09-04T17:46:57.997 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:57.997 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:57.997 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:57.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:57.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:57.997 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:4399 3.9 e31/24 3:970e51a0:::rbd_data.1134cfbce83d.0000000000000009:head v 31'87, mlcod=30'84)", 2024-09-04T17:46:57.997 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:07.343236+0000", 2024-09-04T17:46:57.997 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.647537421000003, 2024-09-04T17:46:57.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0036134539999999999, 2024-09-04T17:46:57.997 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:57.997 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:57.997 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:57.998 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:57.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:57.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:57.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.343236+0000", 2024-09-04T17:46:57.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:57.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:57.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:57.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:57.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.343236+0000", 2024-09-04T17:46:57.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:57.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:57.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:57.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:57.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.343237+0000", 2024-09-04T17:46:57.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4000000000000001e-07 2024-09-04T17:46:57.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:57.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:57.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:57.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.343250+0000", 2024-09-04T17:46:57.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2573e-05 2024-09-04T17:46:57.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:57.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:57.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:57.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.343279+0000", 2024-09-04T17:46:57.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9263000000000001e-05 2024-09-04T17:46:57.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.343299+0000", 2024-09-04T17:46:58.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9720000000000001e-05 2024-09-04T17:46:58.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:58.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.343389+0000", 2024-09-04T17:46:58.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9924000000000002e-05 2024-09-04T17:46:58.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:58.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.343413+0000", 2024-09-04T17:46:58.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4227999999999999e-05 2024-09-04T17:46:58.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:58.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.346796+0000", 2024-09-04T17:46:58.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0033831310000000002 2024-09-04T17:46:58.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:58.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.346850+0000", 2024-09-04T17:46:58.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3875000000000002e-05 2024-09-04T17:46:58.001 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:58.001 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:58.002 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:58.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.002 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:4881 3.4 e31/24 3:25597cfa:::rbd_data.1134cfbce83d.0000000000000006:head v 31'28, mlcod=31'26)", 2024-09-04T17:46:58.002 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:07.381336+0000", 2024-09-04T17:46:58.002 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.609437792000001, 2024-09-04T17:46:58.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013330164, 2024-09-04T17:46:58.002 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:58.002 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:58.002 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:58.002 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:58.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:58.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.381336+0000", 2024-09-04T17:46:58.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:58.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:58.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.381336+0000", 2024-09-04T17:46:58.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:58.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:58.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.381339+0000", 2024-09-04T17:46:58.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.287e-06 2024-09-04T17:46:58.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:58.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.384162+0000", 2024-09-04T17:46:58.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028228649999999999 2024-09-04T17:46:58.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:58.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.384209+0000", 2024-09-04T17:46:58.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6768999999999999e-05 2024-09-04T17:46:58.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.384225+0000", 2024-09-04T17:46:58.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5987000000000001e-05 2024-09-04T17:46:58.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:58.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.384372+0000", 2024-09-04T17:46:58.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000146809 2024-09-04T17:46:58.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:58.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.384409+0000", 2024-09-04T17:46:58.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6909999999999997e-05 2024-09-04T17:46:58.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:58.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.394535+0000", 2024-09-04T17:46:58.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010126575000000001 2024-09-04T17:46:58.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.006 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:58.006 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.394666+0000", 2024-09-04T17:46:58.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013096200000000001 2024-09-04T17:46:58.006 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:58.006 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:58.006 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:58.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.006 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:4884 3.4 e31/24 3:25597cfa:::rbd_data.1134cfbce83d.0000000000000006:head v 31'30, mlcod=31'26)", 2024-09-04T17:46:58.006 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:07.386970+0000", 2024-09-04T17:46:58.007 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.603803857999999, 2024-09-04T17:46:58.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0088306779999999998, 2024-09-04T17:46:58.007 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:58.007 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:58.007 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:58.007 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:58.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:58.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.386970+0000", 2024-09-04T17:46:58.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:58.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:58.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.386970+0000", 2024-09-04T17:46:58.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:58.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:58.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.386973+0000", 2024-09-04T17:46:58.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9229999999999998e-06 2024-09-04T17:46:58.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:58.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.386993+0000", 2024-09-04T17:46:58.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.054e-05 2024-09-04T17:46:58.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:58.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.387029+0000", 2024-09-04T17:46:58.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5753000000000001e-05 2024-09-04T17:46:58.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.387051+0000", 2024-09-04T17:46:58.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1552999999999998e-05 2024-09-04T17:46:58.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:58.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.388844+0000", 2024-09-04T17:46:58.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001793712 2024-09-04T17:46:58.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:58.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.388866+0000", 2024-09-04T17:46:58.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1123000000000001e-05 2024-09-04T17:46:58.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:58.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.395711+0000", 2024-09-04T17:46:58.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0068458859999999998 2024-09-04T17:46:58.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:58.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.395801+0000", 2024-09-04T17:46:58.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9188000000000001e-05 2024-09-04T17:46:58.011 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:58.011 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:58.011 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:58.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.011 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:5000 3.9 e31/24 3:94906c79:::rbd_data.1134cfbce83d.0000000000000012:head v 31'91, mlcod=30'85)", 2024-09-04T17:46:58.011 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:07.402507+0000", 2024-09-04T17:46:58.012 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.588266478000001, 2024-09-04T17:46:58.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004039735, 2024-09-04T17:46:58.012 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:58.012 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:58.012 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:58.012 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:58.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:58.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.402507+0000", 2024-09-04T17:46:58.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:58.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:58.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.402507+0000", 2024-09-04T17:46:58.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:58.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:58.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.402509+0000", 2024-09-04T17:46:58.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5939999999999999e-06 2024-09-04T17:46:58.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:58.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.402521+0000", 2024-09-04T17:46:58.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1786e-05 2024-09-04T17:46:58.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:58.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.402543+0000", 2024-09-04T17:46:58.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2158e-05 2024-09-04T17:46:58.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.402555+0000", 2024-09-04T17:46:58.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2034e-05 2024-09-04T17:46:58.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:58.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.402660+0000", 2024-09-04T17:46:58.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010535900000000001 2024-09-04T17:46:58.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:58.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.402696+0000", 2024-09-04T17:46:58.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5488000000000003e-05 2024-09-04T17:46:58.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:58.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.406510+0000", 2024-09-04T17:46:58.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0038142390000000001 2024-09-04T17:46:58.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.016 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:58.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:07.406547+0000", 2024-09-04T17:46:58.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7076999999999998e-05 2024-09-04T17:46:58.016 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:58.016 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:58.016 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:58.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.016 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:6311 3.9 e32/24 3:941d6897:::rbd_data.113431586065.0000000000000008:head v 32'97, mlcod=32'95)", 2024-09-04T17:46:58.017 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.479986+0000", 2024-09-04T17:46:58.017 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 49.510787581000002, 2024-09-04T17:46:58.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018493539, 2024-09-04T17:46:58.017 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:58.017 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:58.017 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:58.017 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:58.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:58.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.479986+0000", 2024-09-04T17:46:58.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:58.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:58.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.479986+0000", 2024-09-04T17:46:58.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:58.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:58.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.479988+0000", 2024-09-04T17:46:58.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.193e-06 2024-09-04T17:46:58.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:58.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484109+0000", 2024-09-04T17:46:58.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041201650000000003 2024-09-04T17:46:58.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:58.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484139+0000", 2024-09-04T17:46:58.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.082e-05 2024-09-04T17:46:58.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484153+0000", 2024-09-04T17:46:58.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3456e-05 2024-09-04T17:46:58.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:58.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484250+0000", 2024-09-04T17:46:58.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7638000000000003e-05 2024-09-04T17:46:58.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:58.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484282+0000", 2024-09-04T17:46:58.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1511999999999999e-05 2024-09-04T17:46:58.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:58.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.498411+0000", 2024-09-04T17:46:58.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014128870999999999 2024-09-04T17:46:58.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.021 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:58.021 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.498480+0000", 2024-09-04T17:46:58.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8884e-05 2024-09-04T17:46:58.021 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:58.021 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:58.021 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:58.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.021 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:6312 3.9 e32/24 3:941d6897:::rbd_data.113431586065.0000000000000008:head v 32'99, mlcod=32'95)", 2024-09-04T17:46:58.021 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.486036+0000", 2024-09-04T17:46:58.021 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 49.504737503000001, 2024-09-04T17:46:58.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013432773, 2024-09-04T17:46:58.021 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:58.022 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:58.022 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:58.022 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:58.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:58.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.486036+0000", 2024-09-04T17:46:58.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:58.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:58.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.486036+0000", 2024-09-04T17:46:58.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:58.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:58.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.486038+0000", 2024-09-04T17:46:58.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.536e-06 2024-09-04T17:46:58.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:58.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.486062+0000", 2024-09-04T17:46:58.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3672e-05 2024-09-04T17:46:58.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:58.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.486078+0000", 2024-09-04T17:46:58.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.613e-05 2024-09-04T17:46:58.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.486086+0000", 2024-09-04T17:46:58.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8849999999999999e-06 2024-09-04T17:46:58.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:58.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.491843+0000", 2024-09-04T17:46:58.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057572040000000001 2024-09-04T17:46:58.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:58.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.491876+0000", 2024-09-04T17:46:58.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3515e-05 2024-09-04T17:46:58.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:58.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.499410+0000", 2024-09-04T17:46:58.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0075333780000000003 2024-09-04T17:46:58.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:58.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.499469+0000", 2024-09-04T17:46:58.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9453000000000002e-05 2024-09-04T17:46:58.026 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:58.026 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:58.026 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:58.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.026 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:12747 3.1f e41/37 3:fb91fe3f:::rbd_object_map.1134ef3a5a7d:head v 41'125, mlcod=41'122)", 2024-09-04T17:46:58.026 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:17.330322+0000", 2024-09-04T17:46:58.026 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.660452051999997, 2024-09-04T17:46:58.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004205762, 2024-09-04T17:46:58.026 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:58.026 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:58.026 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:58.026 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:58.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:58.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.330322+0000", 2024-09-04T17:46:58.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:58.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:58.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.330322+0000", 2024-09-04T17:46:58.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:58.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:58.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.330323+0000", 2024-09-04T17:46:58.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9500000000000001e-07 2024-09-04T17:46:58.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:58.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.330329+0000", 2024-09-04T17:46:58.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2040000000000002e-06 2024-09-04T17:46:58.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:58.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.330341+0000", 2024-09-04T17:46:58.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1727e-05 2024-09-04T17:46:58.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.330348+0000", 2024-09-04T17:46:58.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.187e-06 2024-09-04T17:46:58.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:58.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.330455+0000", 2024-09-04T17:46:58.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107311 2024-09-04T17:46:58.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:58.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.330495+0000", 2024-09-04T17:46:58.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0191999999999997e-05 2024-09-04T17:46:58.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:58.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.334480+0000", 2024-09-04T17:46:58.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039848410000000003 2024-09-04T17:46:58.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:58.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.334528+0000", 2024-09-04T17:46:58.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7404999999999998e-05 2024-09-04T17:46:58.030 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:58.030 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:58.030 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:58.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.031 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:12748 3.1f e41/37 3:fb91fe3f:::rbd_object_map.1134ef3a5a7d:head v 41'126, mlcod=41'122)", 2024-09-04T17:46:58.031 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:17.332613+0000", 2024-09-04T17:46:58.031 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.658161073000002, 2024-09-04T17:46:58.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0044275870000000002, 2024-09-04T17:46:58.031 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:58.031 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:58.031 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:58.031 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:58.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:58.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.332613+0000", 2024-09-04T17:46:58.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:58.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:58.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.332613+0000", 2024-09-04T17:46:58.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:58.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:58.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.332614+0000", 2024-09-04T17:46:58.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.051e-06 2024-09-04T17:46:58.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:58.033 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.332620+0000", 2024-09-04T17:46:58.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1680000000000001e-06 2024-09-04T17:46:58.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:58.033 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.332635+0000", 2024-09-04T17:46:58.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5543e-05 2024-09-04T17:46:58.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.033 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.332643+0000", 2024-09-04T17:46:58.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.537e-06 2024-09-04T17:46:58.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:58.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.332717+0000", 2024-09-04T17:46:58.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3701999999999993e-05 2024-09-04T17:46:58.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:58.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.332786+0000", 2024-09-04T17:46:58.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9669e-05 2024-09-04T17:46:58.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:58.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.336923+0000", 2024-09-04T17:46:58.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041365259999999997 2024-09-04T17:46:58.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:58.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.337040+0000", 2024-09-04T17:46:58.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000117391 2024-09-04T17:46:58.036 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:58.036 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:58.036 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:58.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.037 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:12749 3.1f e41/37 3:fb91fe3f:::rbd_object_map.1134ef3a5a7d:head v 41'127, mlcod=41'122)", 2024-09-04T17:46:58.037 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:17.335529+0000", 2024-09-04T17:46:58.037 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.655245032000003, 2024-09-04T17:46:58.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052795899999999998, 2024-09-04T17:46:58.037 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:58.037 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:58.037 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:58.037 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:58.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.037 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:58.037 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.335529+0000", 2024-09-04T17:46:58.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:58.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:58.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.335529+0000", 2024-09-04T17:46:58.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:58.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:58.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.335535+0000", 2024-09-04T17:46:58.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4060000000000002e-06 2024-09-04T17:46:58.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:58.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.335546+0000", 2024-09-04T17:46:58.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0652000000000001e-05 2024-09-04T17:46:58.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:58.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.335571+0000", 2024-09-04T17:46:58.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5148e-05 2024-09-04T17:46:58.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.335583+0000", 2024-09-04T17:46:58.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2315e-05 2024-09-04T17:46:58.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:58.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.335699+0000", 2024-09-04T17:46:58.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000115443 2024-09-04T17:46:58.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:58.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.335736+0000", 2024-09-04T17:46:58.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6779999999999997e-05 2024-09-04T17:46:58.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:58.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.340722+0000", 2024-09-04T17:46:58.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0049860390000000003 2024-09-04T17:46:58.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:58.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.340808+0000", 2024-09-04T17:46:58.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6806999999999997e-05 2024-09-04T17:46:58.041 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:58.041 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:58.041 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:58.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.041 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:12750 3.1f e41/37 3:fb91fe3f:::rbd_object_map.1134ef3a5a7d:head v 41'128, mlcod=41'123)", 2024-09-04T17:46:58.042 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:17.337581+0000", 2024-09-04T17:46:58.042 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.653192488000002, 2024-09-04T17:46:58.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045169579999999997, 2024-09-04T17:46:58.042 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:58.042 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:58.042 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:58.042 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:58.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:58.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.337581+0000", 2024-09-04T17:46:58.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:58.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:58.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.337581+0000", 2024-09-04T17:46:58.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:58.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:58.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.337583+0000", 2024-09-04T17:46:58.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5200000000000001e-06 2024-09-04T17:46:58.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:58.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.337595+0000", 2024-09-04T17:46:58.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2395000000000001e-05 2024-09-04T17:46:58.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:58.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.337620+0000", 2024-09-04T17:46:58.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4794e-05 2024-09-04T17:46:58.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.337634+0000", 2024-09-04T17:46:58.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4241e-05 2024-09-04T17:46:58.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:58.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.337731+0000", 2024-09-04T17:46:58.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6246e-05 2024-09-04T17:46:58.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:58.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.337762+0000", 2024-09-04T17:46:58.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1279999999999999e-05 2024-09-04T17:46:58.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:58.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.342050+0000", 2024-09-04T17:46:58.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0042879729999999996 2024-09-04T17:46:58.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:58.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.342098+0000", 2024-09-04T17:46:58.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8508999999999999e-05 2024-09-04T17:46:58.046 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:58.046 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:58.046 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:58.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.047 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:12756 3.0 e41/24 3:0074e6f8:::rbd_data.1134ef3a5a7d.000000000000000e:head v 41'24, mlcod=40'20)", 2024-09-04T17:46:58.047 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:17.339316+0000", 2024-09-04T17:46:58.047 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.651458265000002, 2024-09-04T17:46:58.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052778790000000001, 2024-09-04T17:46:58.047 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:58.047 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:58.047 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:58.047 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:58.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:58.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.339316+0000", 2024-09-04T17:46:58.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:58.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:58.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.339316+0000", 2024-09-04T17:46:58.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:58.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:58.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.339317+0000", 2024-09-04T17:46:58.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.13e-06 2024-09-04T17:46:58.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:58.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.339328+0000", 2024-09-04T17:46:58.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1742e-05 2024-09-04T17:46:58.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:58.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.339358+0000", 2024-09-04T17:46:58.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9149000000000002e-05 2024-09-04T17:46:58.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.339379+0000", 2024-09-04T17:46:58.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1288e-05 2024-09-04T17:46:58.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:58.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.339454+0000", 2024-09-04T17:46:58.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4703000000000006e-05 2024-09-04T17:46:58.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:58.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.339506+0000", 2024-09-04T17:46:58.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2188999999999998e-05 2024-09-04T17:46:58.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:58.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.344475+0000", 2024-09-04T17:46:58.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0049691919999999999 2024-09-04T17:46:58.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:58.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:17.344593+0000", 2024-09-04T17:46:58.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000118486 2024-09-04T17:46:58.051 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:58.051 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:58.051 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:58.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.051 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:27957 3.1f e50/37 3:ff7930c6:::rbd_data.1134eedad849.0000000000000380:head v 50'169, mlcod=50'165)", 2024-09-04T17:46:58.051 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:30.598047+0000", 2024-09-04T17:46:58.051 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.392726704000001, 2024-09-04T17:46:58.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0042527729999999996, 2024-09-04T17:46:58.052 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:58.052 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:58.052 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:58.052 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:58.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:58.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.598047+0000", 2024-09-04T17:46:58.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:58.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:58.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.598047+0000", 2024-09-04T17:46:58.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:58.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:58.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.598048+0000", 2024-09-04T17:46:58.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7800000000000001e-07 2024-09-04T17:46:58.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:58.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.598055+0000", 2024-09-04T17:46:58.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1069999999999999e-06 2024-09-04T17:46:58.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:58.054 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.598067+0000", 2024-09-04T17:46:58.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2278e-05 2024-09-04T17:46:58.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.054 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.598072+0000", 2024-09-04T17:46:58.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8439999999999997e-06 2024-09-04T17:46:58.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:58.054 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.598114+0000", 2024-09-04T17:46:58.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1931000000000002e-05 2024-09-04T17:46:58.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:58.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.598133+0000", 2024-09-04T17:46:58.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9102000000000001e-05 2024-09-04T17:46:58.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:58.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.602251+0000", 2024-09-04T17:46:58.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041175079999999998 2024-09-04T17:46:58.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:58.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.602300+0000", 2024-09-04T17:46:58.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9224999999999997e-05 2024-09-04T17:46:58.056 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:58.056 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:58.056 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:58.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.056 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "pg_update_log_missing(3.0 epoch 50/24 rep_tid 378 entries 50'73 (0'0) error 3:01dff485:::rbd_data.1134eedad849.000000000000038b:head by client.4404.0:27967 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 50'71)", 2024-09-04T17:46:58.056 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:30.600491+0000", 2024-09-04T17:46:58.056 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.390282331000002, 2024-09-04T17:46:58.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045188329999999999, 2024-09-04T17:46:58.056 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:58.056 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:58.056 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-04T17:46:58.056 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:58.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:58.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.600491+0000", 2024-09-04T17:46:58.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:58.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:58.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.600491+0000", 2024-09-04T17:46:58.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:58.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:58.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.600492+0000", 2024-09-04T17:46:58.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3500000000000005e-07 2024-09-04T17:46:58.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:58.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.600510+0000", 2024-09-04T17:46:58.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7533999999999998e-05 2024-09-04T17:46:58.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:58.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.600536+0000", 2024-09-04T17:46:58.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5822000000000001e-05 2024-09-04T17:46:58.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.600556+0000", 2024-09-04T17:46:58.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9913000000000001e-05 2024-09-04T17:46:58.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:58.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.600644+0000", 2024-09-04T17:46:58.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8282999999999995e-05 2024-09-04T17:46:58.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:58.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:30.605010+0000", 2024-09-04T17:46:58.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0043664460000000004 2024-09-04T17:46:58.059 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:58.060 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:58.060 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:58.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.060 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:34444 3.1f e56/37 3:fcc2bddb:::rbd_data.11342e8f3384.0000000000000004:head v 56'189, mlcod=54'187)", 2024-09-04T17:46:58.060 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:36.632281+0000", 2024-09-04T17:46:58.060 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 21.358493036999999, 2024-09-04T17:46:58.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0036502420000000002, 2024-09-04T17:46:58.060 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:58.060 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:58.060 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:58.060 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:58.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:58.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.632281+0000", 2024-09-04T17:46:58.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:58.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:58.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.632281+0000", 2024-09-04T17:46:58.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:58.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:58.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.632282+0000", 2024-09-04T17:46:58.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5300000000000003e-07 2024-09-04T17:46:58.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:58.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.633663+0000", 2024-09-04T17:46:58.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0013816169999999999 2024-09-04T17:46:58.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:58.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.633682+0000", 2024-09-04T17:46:58.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8624e-05 2024-09-04T17:46:58.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.633691+0000", 2024-09-04T17:46:58.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3260000000000002e-06 2024-09-04T17:46:58.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:58.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.633766+0000", 2024-09-04T17:46:58.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5075999999999997e-05 2024-09-04T17:46:58.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:58.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.633793+0000", 2024-09-04T17:46:58.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6939000000000001e-05 2024-09-04T17:46:58.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:58.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.635872+0000", 2024-09-04T17:46:58.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020786820000000001 2024-09-04T17:46:58.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:58.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.635931+0000", 2024-09-04T17:46:58.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9125000000000001e-05 2024-09-04T17:46:58.064 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:58.065 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:58.065 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:58.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.065 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:37418 3.1f e58/37 3:ff114f66:::rbd_data.1134c5a43465.00000000000000da:head v 58'200, mlcod=58'198)", 2024-09-04T17:46:58.065 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:39.727221+0000", 2024-09-04T17:46:58.065 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.263552351000001, 2024-09-04T17:46:58.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040617930000000002, 2024-09-04T17:46:58.065 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:58.065 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:58.065 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:58.065 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:58.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:58.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.727221+0000", 2024-09-04T17:46:58.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:58.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:58.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.727221+0000", 2024-09-04T17:46:58.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:58.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:58.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.727223+0000", 2024-09-04T17:46:58.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.88e-06 2024-09-04T17:46:58.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:58.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.727239+0000", 2024-09-04T17:46:58.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5962000000000001e-05 2024-09-04T17:46:58.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:58.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.727278+0000", 2024-09-04T17:46:58.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8844999999999998e-05 2024-09-04T17:46:58.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.727295+0000", 2024-09-04T17:46:58.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7192000000000001e-05 2024-09-04T17:46:58.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:58.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.727474+0000", 2024-09-04T17:46:58.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000178176 2024-09-04T17:46:58.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:58.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.727536+0000", 2024-09-04T17:46:58.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2212999999999995e-05 2024-09-04T17:46:58.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:58.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.731172+0000", 2024-09-04T17:46:58.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0036367050000000001 2024-09-04T17:46:58.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:58.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.731283+0000", 2024-09-04T17:46:58.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011082 2024-09-04T17:46:58.069 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:58.069 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:58.069 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:58.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.070 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:37420 3.9 e58/24 3:940e2e4e:::rbd_data.1134c5a43465.00000000000000dc:head v 58'215, mlcod=58'213)", 2024-09-04T17:46:58.070 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:39.728565+0000", 2024-09-04T17:46:58.070 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.262209003999999, 2024-09-04T17:46:58.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039183309999999997, 2024-09-04T17:46:58.070 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:58.070 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:58.070 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:58.070 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:58.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:58.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.728565+0000", 2024-09-04T17:46:58.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:58.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:58.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.728565+0000", 2024-09-04T17:46:58.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:58.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:58.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.728567+0000", 2024-09-04T17:46:58.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7230000000000001e-06 2024-09-04T17:46:58.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:58.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.728579+0000", 2024-09-04T17:46:58.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2274e-05 2024-09-04T17:46:58.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:58.072 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.728613+0000", 2024-09-04T17:46:58.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4326000000000003e-05 2024-09-04T17:46:58.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.072 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.728628+0000", 2024-09-04T17:46:58.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4882000000000001e-05 2024-09-04T17:46:58.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:58.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.728793+0000", 2024-09-04T17:46:58.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016519000000000001 2024-09-04T17:46:58.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:58.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.728839+0000", 2024-09-04T17:46:58.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5343000000000003e-05 2024-09-04T17:46:58.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:58.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.732380+0000", 2024-09-04T17:46:58.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0035412849999999999 2024-09-04T17:46:58.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:58.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.732483+0000", 2024-09-04T17:46:58.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010330799999999999 2024-09-04T17:46:58.074 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:58.074 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:58.074 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:58.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.074 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:46594 3.9 3:97706e40:::rbd_data.1134326b938e.0000000000000013:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-04T17:46:58.074 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:54.025375+0000", 2024-09-04T17:46:58.074 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.9653991350000002, 2024-09-04T17:46:58.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0082115499999999998, 2024-09-04T17:46:58.074 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:58.075 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:58.075 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:58.075 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:58.075 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:58.075 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:58.075 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 46594 2024-09-04T17:46:58.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.075 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:58.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:58.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.025375+0000", 2024-09-04T17:46:58.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:58.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:58.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.025375+0000", 2024-09-04T17:46:58.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:58.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:58.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.025379+0000", 2024-09-04T17:46:58.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0439999999999997e-06 2024-09-04T17:46:58.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:58.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.025393+0000", 2024-09-04T17:46:58.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4456e-05 2024-09-04T17:46:58.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.077 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:58.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.025443+0000", 2024-09-04T17:46:58.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9904000000000002e-05 2024-09-04T17:46:58.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.077 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.025467+0000", 2024-09-04T17:46:58.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4419000000000001e-05 2024-09-04T17:46:58.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:58.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.025637+0000", 2024-09-04T17:46:58.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016905699999999999 2024-09-04T17:46:58.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:58.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.025936+0000", 2024-09-04T17:46:58.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00029941499999999998 2024-09-04T17:46:58.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-04T17:46:58.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.026229+0000", 2024-09-04T17:46:58.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00029280499999999999 2024-09-04T17:46:58.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:58.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.029491+0000", 2024-09-04T17:46:58.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003262596 2024-09-04T17:46:58.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:46:58.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.033531+0000", 2024-09-04T17:46:58.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040393779999999997 2024-09-04T17:46:58.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:58.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.033558+0000", 2024-09-04T17:46:58.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7432000000000001e-05 2024-09-04T17:46:58.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:58.080 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.033586+0000", 2024-09-04T17:46:58.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8044000000000001e-05 2024-09-04T17:46:58.080 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:58.080 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:58.080 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:58.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.080 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:46600 3.9 3:966a74f3:::rbd_data.1134326b938e.000000000000002e:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-04T17:46:58.080 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:54.030201+0000", 2024-09-04T17:46:58.081 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.9605724799999997, 2024-09-04T17:46:58.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066468320000000001, 2024-09-04T17:46:58.081 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:58.081 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:58.081 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:58.081 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:46:58.081 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:46:58.081 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:58.081 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 46600 2024-09-04T17:46:58.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.081 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:58.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:58.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.030201+0000", 2024-09-04T17:46:58.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:58.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:58.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.030201+0000", 2024-09-04T17:46:58.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:58.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:58.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.030204+0000", 2024-09-04T17:46:58.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.627e-06 2024-09-04T17:46:58.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.083 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:58.083 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.030219+0000", 2024-09-04T17:46:58.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4746000000000001e-05 2024-09-04T17:46:58.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.083 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:58.083 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.030254+0000", 2024-09-04T17:46:58.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5024999999999998e-05 2024-09-04T17:46:58.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.083 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.083 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.030275+0000", 2024-09-04T17:46:58.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1058999999999999e-05 2024-09-04T17:46:58.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:58.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.033026+0000", 2024-09-04T17:46:58.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0027510619999999999 2024-09-04T17:46:58.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:58.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.033104+0000", 2024-09-04T17:46:58.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7909000000000002e-05 2024-09-04T17:46:58.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-04T17:46:58.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.033202+0000", 2024-09-04T17:46:58.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7776999999999996e-05 2024-09-04T17:46:58.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:46:58.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.034081+0000", 2024-09-04T17:46:58.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00087908699999999997 2024-09-04T17:46:58.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:58.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.036762+0000", 2024-09-04T17:46:58.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0026814080000000001 2024-09-04T17:46:58.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:58.086 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.036811+0000", 2024-09-04T17:46:58.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9197999999999999e-05 2024-09-04T17:46:58.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:58.086 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.036848+0000", 2024-09-04T17:46:58.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6934000000000002e-05 2024-09-04T17:46:58.086 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:58.087 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:58.087 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:58.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.087 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4404.0:46816 3.0 e70/24 3:043ced34:::rbd_data.1134326b938e.000000000000006e:head v 70'189, mlcod=70'187)", 2024-09-04T17:46:58.087 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:54.567985+0000", 2024-09-04T17:46:58.087 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.422788846, 2024-09-04T17:46:58.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0036409739999999999, 2024-09-04T17:46:58.087 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:46:58.087 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:46:58.087 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:58.087 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:46:58.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:46:58.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.567985+0000", 2024-09-04T17:46:58.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:58.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:46:58.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.567985+0000", 2024-09-04T17:46:58.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:46:58.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:46:58.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.567988+0000", 2024-09-04T17:46:58.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1530000000000001e-06 2024-09-04T17:46:58.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:46:58.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.568012+0000", 2024-09-04T17:46:58.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4326000000000001e-05 2024-09-04T17:46:58.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:46:58.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.568060+0000", 2024-09-04T17:46:58.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7067999999999997e-05 2024-09-04T17:46:58.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.568082+0000", 2024-09-04T17:46:58.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2671999999999999e-05 2024-09-04T17:46:58.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:46:58.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.568225+0000", 2024-09-04T17:46:58.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014271800000000001 2024-09-04T17:46:58.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:46:58.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.568280+0000", 2024-09-04T17:46:58.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5481e-05 2024-09-04T17:46:58.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:46:58.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.571563+0000", 2024-09-04T17:46:58.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0032823520000000001 2024-09-04T17:46:58.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:46:58.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:46:58.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:46:58.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.571626+0000", 2024-09-04T17:46:58.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3204e-05 2024-09-04T17:46:58.091 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:58.091 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:58.091 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:58.091 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:46:58.092 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:46:58.092 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:46:58.093 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-04T17:46:58.094 INFO:teuthology.orchestra.run.smithi092.stdout:3249 write 0x289e390 thru 0x28ad280 (0xeef1 bytes) 2024-09-04T17:46:58.094 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:46:58.091+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:46:58.103 INFO:teuthology.orchestra.run.smithi092.stdout:3251 writesame 0x503429b thru 0x503bfd4 data_size 0x4e(0x7d3a bytes) 2024-09-04T17:46:58.110 INFO:teuthology.orchestra.run.smithi092.stdout:3252 write 0xa32eb33 thru 0xa33da31 (0xeeff bytes) 2024-09-04T17:46:58.117 INFO:teuthology.orchestra.run.smithi092.stdout:3253 punch from 0x370a757 to 0x370ed24, (0x45cd bytes) 2024-09-04T17:46:58.117 INFO:teuthology.orchestra.run.smithi092.stdout:3254 writesame 0x6c74ea3 thru 0x6c7ff1f data_size 0x1(0xb07d bytes) 2024-09-04T17:46:58.178 INFO:teuthology.orchestra.run.smithi092.stdout:3255 read 0xb9f8c7c thru 0xba01bad (0x8f32 bytes) 2024-09-04T17:46:58.178 INFO:teuthology.orchestra.run.smithi092.stdout:3256 write 0x7f88e98 thru 0x7f8dbae (0x4d17 bytes) 2024-09-04T17:46:58.186 INFO:teuthology.orchestra.run.smithi092.stdout:3258 write 0x8aff608 thru 0x8b0b453 (0xbe4c bytes) 2024-09-04T17:46:58.193 INFO:teuthology.orchestra.run.smithi092.stdout:3259 read 0xb3a977c thru 0xb3b3a7d (0xa302 bytes) 2024-09-04T17:46:58.193 INFO:teuthology.orchestra.run.smithi092.stdout:3260 write 0x2238511 thru 0x22449f0 (0xc4e0 bytes) 2024-09-04T17:46:58.194 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:58.193+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:58.201 INFO:teuthology.orchestra.run.smithi092.stdout:3261 punch from 0x4a7df01 to 0x4a8bfc0, (0xe0bf bytes) 2024-09-04T17:46:58.201 INFO:teuthology.orchestra.run.smithi092.stdout:3263 write 0x6ec08f0 thru 0x6ec9f64 (0x9675 bytes) 2024-09-04T17:46:58.231 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:58.231 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:46:58.231 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:46:58.231 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:58.231 INFO:teuthology.orchestra.run.smithi092.stdout:3264 trunc from 0xcbcbfc1 to 0x205eb94 2024-09-04T17:46:58.249 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-04T17:46:58.282 INFO:teuthology.orchestra.run.smithi092.stdout:3265 write 0x455afff thru 0x4564e61 (0x9e63 bytes) 2024-09-04T17:46:58.288 INFO:teuthology.orchestra.run.smithi092.stdout:3266 punch from 0x18ab069 to 0x18b8106, (0xd09d bytes) 2024-09-04T17:46:58.294 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:46:58.293+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:46:58.294 INFO:teuthology.orchestra.run.smithi092.stdout:3267 trunc from 0x4564e62 to 0x1eb0193 2024-09-04T17:46:58.330 INFO:teuthology.orchestra.run.smithi092.stdout:3268 trunc from 0x1eb0193 to 0x5908d58 2024-09-04T17:46:58.342 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:58.351 INFO:teuthology.orchestra.run.smithi092.stdout:3271 trunc from 0x5908d58 to 0x2940171 2024-09-04T17:46:58.395 INFO:teuthology.orchestra.run.smithi092.stdout:3273 write 0x9e5811a thru 0x9e5e869 (0x6750 bytes) 2024-09-04T17:46:58.395 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:46:58.391+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:46:58.413 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:58.413 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:46:58.413 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:46:58.413 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:46:58.413 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:58.415 INFO:teuthology.orchestra.run.smithi092.stdout:3274 trunc from 0x9e5e86a to 0x5653bca 2024-09-04T17:46:58.428 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-04T17:46:58.437 INFO:teuthology.orchestra.run.smithi092.stdout:3277 punch from 0x15097c4 to 0x151347c, (0x9cb8 bytes) 2024-09-04T17:46:58.465 INFO:teuthology.orchestra.run.smithi092.stdout:3278 write 0xb3d219d thru 0xb3e0acb (0xe92f bytes) 2024-09-04T17:46:58.471 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:58.472 INFO:teuthology.orchestra.run.smithi092.stdout:3282 writesame 0x1cb18fb thru 0x1cb4cd0 data_size 0x5(0x33d6 bytes) 2024-09-04T17:46:58.480 INFO:teuthology.orchestra.run.smithi092.stdout:3283 trunc from 0xb3e0acc to 0xdc0525f 2024-09-04T17:46:58.493 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:58.494 INFO:teuthology.orchestra.run.smithi092.stdout:3286 read 0xc41517a thru 0xc4152aa (0x131 bytes) 2024-09-04T17:46:58.494 INFO:teuthology.orchestra.run.smithi092.stdout:3287 write 0x82303b5 thru 0x823bd86 (0xb9d2 bytes) 2024-09-04T17:46:58.496 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:58.497+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:58.500 INFO:teuthology.orchestra.run.smithi092.stdout:3288 punch from 0x368622a to 0x3691afd, (0xb8d3 bytes) 2024-09-04T17:46:58.500 INFO:teuthology.orchestra.run.smithi092.stdout:3290 punch from 0xa76de95 to 0xa7734e0, (0x564b bytes) 2024-09-04T17:46:58.500 INFO:teuthology.orchestra.run.smithi092.stdout:3291 writesame 0xd842856 thru 0xd849e09 data_size 0x6c(0x75b4 bytes) 2024-09-04T17:46:58.527 INFO:teuthology.orchestra.run.smithi092.stdout:3292 trunc from 0xdc0525f to 0x61ebdc8 2024-09-04T17:46:58.551 INFO:teuthology.orchestra.run.smithi092.stdout:3295 read 0x4c9a493 thru 0x4c9f35c (0x4eca bytes) 2024-09-04T17:46:58.551 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:58.556 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:58.556 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:46:58.557 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:46:58.557 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:46:58.557 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.557 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:12722 3.1b 3:d8152af1:::rbd_data.1134ef3a5a7d.00000000000000bd:head [delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-04T17:46:58.557 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:16.421965+0000", 2024-09-04T17:46:58.557 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 42.133402945, 2024-09-04T17:46:58.557 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.890002811, 2024-09-04T17:46:58.557 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:58.557 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:58.557 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:58.557 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:58.557 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:58.558 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:58.558 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 12722 2024-09-04T17:46:58.558 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.558 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:58.558 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.558 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:58.558 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421965+0000", 2024-09-04T17:46:58.558 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.558 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.558 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.558 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:58.558 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421965+0000", 2024-09-04T17:46:58.558 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.559 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.559 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.559 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:58.559 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421966+0000", 2024-09-04T17:46:58.559 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.102e-06 2024-09-04T17:46:58.559 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.559 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.559 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:58.559 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421971+0000", 2024-09-04T17:46:58.559 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9989999999999999e-06 2024-09-04T17:46:58.559 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.559 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.559 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:58.560 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421981+0000", 2024-09-04T17:46:58.560 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.7380000000000006e-06 2024-09-04T17:46:58.560 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.560 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.560 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.560 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421989+0000", 2024-09-04T17:46:58.560 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.1890000000000007e-06 2024-09-04T17:46:58.560 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.560 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.560 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:58.560 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.422050+0000", 2024-09-04T17:46:58.560 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.0702000000000003e-05 2024-09-04T17:46:58.560 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.561 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.561 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for peered", 2024-09-04T17:46:58.561 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.422059+0000", 2024-09-04T17:46:58.561 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5440000000000004e-06 2024-09-04T17:46:58.561 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.561 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.561 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:58.561 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.308156+0000", 2024-09-04T17:46:58.561 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.88609676199999998 2024-09-04T17:46:58.561 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.561 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.561 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:58.562 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.308398+0000", 2024-09-04T17:46:58.562 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024215400000000001 2024-09-04T17:46:58.562 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.562 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.562 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 7", 2024-09-04T17:46:58.562 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.308795+0000", 2024-09-04T17:46:58.562 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000396704 2024-09-04T17:46:58.562 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.562 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.562 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:58.562 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.310831+0000", 2024-09-04T17:46:58.562 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0020364200000000002 2024-09-04T17:46:58.563 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.563 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.563 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:58.563 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.311818+0000", 2024-09-04T17:46:58.563 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00098649799999999993 2024-09-04T17:46:58.563 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.563 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.563 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:58.563 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.311905+0000", 2024-09-04T17:46:58.564 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7070999999999994e-05 2024-09-04T17:46:58.564 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.564 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.564 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:58.564 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.311968+0000", 2024-09-04T17:46:58.564 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.2928000000000005e-05 2024-09-04T17:46:58.564 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.564 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:58.564 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.564 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.565 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.565 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37359 3.f e58/24 3:f29e50a5:::rbd_data.1134c5a43465.0000000000000021:head v 58'135, mlcod=58'133)", 2024-09-04T17:46:58.565 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:46:58.565 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 18.938072341000002, 2024-09-04T17:46:58.565 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053483197000000003, 2024-09-04T17:46:58.565 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:58.565 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:58.565 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:58.565 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:58.565 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.565 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:58.565 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:46:58.566 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.566 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.566 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.566 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:58.566 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:46:58.566 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.566 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.566 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.566 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:58.566 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:46:58.566 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1699999999999997e-07 2024-09-04T17:46:58.566 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.566 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.567 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:58.567 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617338+0000", 2024-09-04T17:46:58.567 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.1168999999999998e-05 2024-09-04T17:46:58.567 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.567 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.567 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:58.567 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617351+0000", 2024-09-04T17:46:58.567 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3094e-05 2024-09-04T17:46:58.567 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.567 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.567 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.567 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617359+0000", 2024-09-04T17:46:58.567 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7970000000000006e-06 2024-09-04T17:46:58.567 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.568 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.568 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:58.568 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617417+0000", 2024-09-04T17:46:58.568 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7079e-05 2024-09-04T17:46:58.568 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.568 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.568 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:58.568 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617438+0000", 2024-09-04T17:46:58.568 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1654999999999998e-05 2024-09-04T17:46:58.568 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.568 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.568 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:58.568 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670738+0000", 2024-09-04T17:46:58.569 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053300142000000002 2024-09-04T17:46:58.569 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.569 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.569 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:58.569 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670779+0000", 2024-09-04T17:46:58.569 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0343999999999999e-05 2024-09-04T17:46:58.569 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.569 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:58.569 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.569 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.569 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.569 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37358 3.1d e58/24 3:bd3215f3:::rbd_data.1134c5a43465.0000000000000020:head v 58'91, mlcod=58'89)", 2024-09-04T17:46:58.569 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617502+0000", 2024-09-04T17:46:58.570 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 18.937865623, 2024-09-04T17:46:58.570 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053505641999999999, 2024-09-04T17:46:58.570 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:58.570 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:58.570 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:58.570 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:58.570 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.570 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:58.570 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617502+0000", 2024-09-04T17:46:58.570 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.570 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.570 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.570 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:58.570 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617502+0000", 2024-09-04T17:46:58.570 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.571 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.571 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.571 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:58.571 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617503+0000", 2024-09-04T17:46:58.571 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.184e-06 2024-09-04T17:46:58.571 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.571 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.571 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:58.571 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617526+0000", 2024-09-04T17:46:58.571 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2099999999999998e-05 2024-09-04T17:46:58.571 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.571 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.571 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:58.571 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617540+0000", 2024-09-04T17:46:58.572 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4741e-05 2024-09-04T17:46:58.572 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.572 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.572 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.572 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617550+0000", 2024-09-04T17:46:58.572 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4259999999999992e-06 2024-09-04T17:46:58.572 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.572 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.572 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:58.572 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617613+0000", 2024-09-04T17:46:58.572 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.3625000000000002e-05 2024-09-04T17:46:58.572 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.572 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.572 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:58.573 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617637+0000", 2024-09-04T17:46:58.573 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.3604999999999998e-05 2024-09-04T17:46:58.573 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.573 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.573 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:58.573 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670967+0000", 2024-09-04T17:46:58.573 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053329881000000003 2024-09-04T17:46:58.573 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.573 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.573 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:58.573 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671008+0000", 2024-09-04T17:46:58.573 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.108e-05 2024-09-04T17:46:58.574 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.574 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:58.574 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.574 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.574 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.574 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37361 3.1c e58/24 3:3c7f8976:::rbd_data.1134c5a43465.0000000000000023:head v 58'159, mlcod=58'157)", 2024-09-04T17:46:58.574 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617578+0000", 2024-09-04T17:46:58.574 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 18.937789772999999, 2024-09-04T17:46:58.574 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053990367999999997, 2024-09-04T17:46:58.574 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:58.574 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:58.574 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:58.574 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:58.574 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.575 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:58.575 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617578+0000", 2024-09-04T17:46:58.575 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.575 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.575 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.575 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:58.575 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617578+0000", 2024-09-04T17:46:58.575 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.575 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.575 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.575 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:58.575 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617579+0000", 2024-09-04T17:46:58.575 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7299999999999996e-07 2024-09-04T17:46:58.576 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.576 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.576 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:58.576 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617610+0000", 2024-09-04T17:46:58.576 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0951000000000002e-05 2024-09-04T17:46:58.576 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.576 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.576 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:58.576 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:46:58.576 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0002e-05 2024-09-04T17:46:58.576 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.576 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.576 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.576 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617626+0000", 2024-09-04T17:46:58.577 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6660000000000002e-06 2024-09-04T17:46:58.577 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.577 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.577 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:58.577 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670548+0000", 2024-09-04T17:46:58.577 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.052921943999999999 2024-09-04T17:46:58.577 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.577 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.577 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:58.577 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670574+0000", 2024-09-04T17:46:58.577 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5405000000000002e-05 2024-09-04T17:46:58.577 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.577 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.577 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:58.578 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671522+0000", 2024-09-04T17:46:58.578 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00094815299999999997 2024-09-04T17:46:58.578 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.578 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.578 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:58.578 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671568+0000", 2024-09-04T17:46:58.578 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6674000000000001e-05 2024-09-04T17:46:58.578 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.578 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:58.578 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.578 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.578 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.578 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37357 3.15 e58/49 3:a897312a:::rbd_data.1134c5a43465.000000000000001f:head v 58'117, mlcod=58'115)", 2024-09-04T17:46:58.578 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:46:58.579 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 18.937747434999999, 2024-09-04T17:46:58.579 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053734297, 2024-09-04T17:46:58.579 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:58.579 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:58.579 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:58.579 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:58.579 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.579 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:58.579 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:46:58.579 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.579 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.579 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.579 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:58.579 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:46:58.579 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.580 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.580 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.580 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:58.580 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617621+0000", 2024-09-04T17:46:58.580 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.7300000000000004e-07 2024-09-04T17:46:58.580 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.580 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.580 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:58.580 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617642+0000", 2024-09-04T17:46:58.580 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0185000000000001e-05 2024-09-04T17:46:58.580 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.580 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.580 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:58.580 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617654+0000", 2024-09-04T17:46:58.581 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2745000000000001e-05 2024-09-04T17:46:58.581 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.581 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.581 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.581 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617662+0000", 2024-09-04T17:46:58.581 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.0499999999999992e-06 2024-09-04T17:46:58.581 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.581 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.581 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:58.581 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617727+0000", 2024-09-04T17:46:58.581 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.4923000000000002e-05 2024-09-04T17:46:58.581 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.581 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.582 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:58.582 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617753+0000", 2024-09-04T17:46:58.582 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5899999999999999e-05 2024-09-04T17:46:58.582 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.582 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.582 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:58.582 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671298+0000", 2024-09-04T17:46:58.582 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053545276000000003 2024-09-04T17:46:58.582 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.582 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.582 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:58.582 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671355+0000", 2024-09-04T17:46:58.582 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6245000000000001e-05 2024-09-04T17:46:58.582 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.582 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:58.583 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.583 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.583 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.583 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44687 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:46:58.583 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253012+0000", 2024-09-04T17:46:58.583 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 8.3023553680000006, 2024-09-04T17:46:58.583 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051128777, 2024-09-04T17:46:58.583 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:58.583 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:58.583 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:58.583 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:58.583 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:58.583 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:58.583 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44687 2024-09-04T17:46:58.583 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.584 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:58.584 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.584 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:58.584 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253012+0000", 2024-09-04T17:46:58.584 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.584 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.584 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.584 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:58.585 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253012+0000", 2024-09-04T17:46:58.585 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.585 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.585 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.585 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:58.585 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253014+0000", 2024-09-04T17:46:58.585 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.24e-06 2024-09-04T17:46:58.585 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.585 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.585 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:58.585 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253019+0000", 2024-09-04T17:46:58.585 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4569999999999996e-06 2024-09-04T17:46:58.586 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.586 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.586 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:58.586 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253035+0000", 2024-09-04T17:46:58.586 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6291999999999999e-05 2024-09-04T17:46:58.586 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.586 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.586 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.586 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253050+0000", 2024-09-04T17:46:58.586 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4423000000000001e-05 2024-09-04T17:46:58.586 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.586 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.586 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:58.587 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304115+0000", 2024-09-04T17:46:58.587 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051064778999999998 2024-09-04T17:46:58.587 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.587 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.587 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:58.587 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304141+0000", 2024-09-04T17:46:58.587 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6585999999999999e-05 2024-09-04T17:46:58.587 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.587 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:58.587 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.587 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.587 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.588 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44688 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:46:58.588 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253067+0000", 2024-09-04T17:46:58.588 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 8.3023008160000007, 2024-09-04T17:46:58.588 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051116979, 2024-09-04T17:46:58.588 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:58.588 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:58.588 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:58.588 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:58.588 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:58.588 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:58.588 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44688 2024-09-04T17:46:58.588 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.589 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:58.589 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.589 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:58.589 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253067+0000", 2024-09-04T17:46:58.589 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.589 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.589 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.589 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:58.589 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253067+0000", 2024-09-04T17:46:58.590 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.590 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.590 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.590 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:58.590 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253068+0000", 2024-09-04T17:46:58.590 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0729999999999999e-06 2024-09-04T17:46:58.590 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.590 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.590 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:58.590 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253073+0000", 2024-09-04T17:46:58.590 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2259999999999996e-06 2024-09-04T17:46:58.590 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.590 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.591 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:58.591 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253091+0000", 2024-09-04T17:46:58.591 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7255999999999999e-05 2024-09-04T17:46:58.591 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.591 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.591 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.591 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253103+0000", 2024-09-04T17:46:58.591 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2632999999999999e-05 2024-09-04T17:46:58.591 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.591 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.591 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:58.591 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304173+0000", 2024-09-04T17:46:58.591 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051069537999999998 2024-09-04T17:46:58.591 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.592 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.592 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:58.592 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304184+0000", 2024-09-04T17:46:58.592 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1253000000000001e-05 2024-09-04T17:46:58.592 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.592 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:58.592 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.592 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.592 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.592 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44689 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:46:58.592 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253121+0000", 2024-09-04T17:46:58.592 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 8.3022466789999996, 2024-09-04T17:46:58.592 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051192595, 2024-09-04T17:46:58.593 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:58.593 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:58.593 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:58.593 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:58.593 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:58.593 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:58.593 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44689 2024-09-04T17:46:58.593 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.593 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:58.593 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.593 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:58.593 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253121+0000", 2024-09-04T17:46:58.593 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.594 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.594 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.594 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:58.594 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253121+0000", 2024-09-04T17:46:58.594 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.594 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.594 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.594 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:58.594 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253122+0000", 2024-09-04T17:46:58.594 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.15e-06 2024-09-04T17:46:58.594 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.594 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.594 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:58.595 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253128+0000", 2024-09-04T17:46:58.595 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2569999999999998e-06 2024-09-04T17:46:58.595 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.595 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.595 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:58.595 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253145+0000", 2024-09-04T17:46:58.595 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7176e-05 2024-09-04T17:46:58.595 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.595 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.595 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.595 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253158+0000", 2024-09-04T17:46:58.595 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3139000000000001e-05 2024-09-04T17:46:58.595 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.595 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.596 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:58.596 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304287+0000", 2024-09-04T17:46:58.596 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051129149999999998 2024-09-04T17:46:58.596 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.596 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.596 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:58.596 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304314+0000", 2024-09-04T17:46:58.596 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6723000000000001e-05 2024-09-04T17:46:58.596 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.596 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:58.596 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.596 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.596 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.597 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44690 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:46:58.597 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253177+0000", 2024-09-04T17:46:58.597 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 8.3021905539999992, 2024-09-04T17:46:58.597 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051169263, 2024-09-04T17:46:58.597 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:58.597 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:58.597 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:58.597 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:58.597 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:58.597 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:58.597 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44690 2024-09-04T17:46:58.597 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.597 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:58.598 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.598 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:58.598 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253177+0000", 2024-09-04T17:46:58.598 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.598 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.598 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.598 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:58.598 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253177+0000", 2024-09-04T17:46:58.598 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.598 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.598 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.598 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:58.598 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253178+0000", 2024-09-04T17:46:58.598 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1000000000000001e-06 2024-09-04T17:46:58.599 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.599 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.599 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:58.599 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253184+0000", 2024-09-04T17:46:58.599 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2089999999999996e-06 2024-09-04T17:46:58.599 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.599 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.599 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:58.599 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253200+0000", 2024-09-04T17:46:58.599 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6087e-05 2024-09-04T17:46:58.599 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.599 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.599 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.600 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253231+0000", 2024-09-04T17:46:58.600 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0874999999999998e-05 2024-09-04T17:46:58.600 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.600 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.600 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:58.600 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304337+0000", 2024-09-04T17:46:58.600 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051106430000000001 2024-09-04T17:46:58.600 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.600 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.600 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:58.600 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304347+0000", 2024-09-04T17:46:58.600 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5619999999999993e-06 2024-09-04T17:46:58.600 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.600 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:58.601 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.601 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.601 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.601 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44691 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:46:58.601 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253249+0000", 2024-09-04T17:46:58.601 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 8.3021185200000005, 2024-09-04T17:46:58.601 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051195630999999998, 2024-09-04T17:46:58.601 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:58.601 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:58.601 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:58.601 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:58.601 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:58.601 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:58.602 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44691 2024-09-04T17:46:58.602 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.602 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:58.602 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.602 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:58.602 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253249+0000", 2024-09-04T17:46:58.602 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.602 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.602 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.602 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:58.602 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253249+0000", 2024-09-04T17:46:58.602 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.602 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.603 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.603 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:58.603 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253251+0000", 2024-09-04T17:46:58.603 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2330000000000001e-06 2024-09-04T17:46:58.603 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.603 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.603 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:58.603 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253256+0000", 2024-09-04T17:46:58.603 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6099999999999997e-06 2024-09-04T17:46:58.603 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.603 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.603 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:58.603 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253274+0000", 2024-09-04T17:46:58.604 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7952999999999999e-05 2024-09-04T17:46:58.604 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.604 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.604 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.604 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253287+0000", 2024-09-04T17:46:58.604 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2839e-05 2024-09-04T17:46:58.604 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.604 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.604 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:58.604 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304396+0000", 2024-09-04T17:46:58.604 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051108990999999999 2024-09-04T17:46:58.604 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.605 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.605 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:58.605 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304445+0000", 2024-09-04T17:46:58.605 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9005000000000003e-05 2024-09-04T17:46:58.605 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.605 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:58.605 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.605 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.605 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.605 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44692 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:46:58.605 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253337+0000", 2024-09-04T17:46:58.605 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 8.3020309470000004, 2024-09-04T17:46:58.606 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051138909000000003, 2024-09-04T17:46:58.606 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:58.606 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:58.606 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:58.606 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:58.606 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:58.606 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:58.606 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44692 2024-09-04T17:46:58.606 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.606 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:58.606 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.606 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:58.606 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253337+0000", 2024-09-04T17:46:58.607 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.607 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.607 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.607 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:58.607 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253337+0000", 2024-09-04T17:46:58.607 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.607 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.607 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.607 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:58.607 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253338+0000", 2024-09-04T17:46:58.607 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.463e-06 2024-09-04T17:46:58.607 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.607 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.608 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:58.608 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253357+0000", 2024-09-04T17:46:58.608 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9043e-05 2024-09-04T17:46:58.608 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.608 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.608 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:58.608 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253379+0000", 2024-09-04T17:46:58.608 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1243e-05 2024-09-04T17:46:58.608 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.608 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.608 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.608 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253396+0000", 2024-09-04T17:46:58.609 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7059e-05 2024-09-04T17:46:58.609 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.609 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.609 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:58.609 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304466+0000", 2024-09-04T17:46:58.609 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051070624000000002 2024-09-04T17:46:58.609 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.609 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.609 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:58.609 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304476+0000", 2024-09-04T17:46:58.609 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4769999999999992e-06 2024-09-04T17:46:58.609 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.609 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:58.610 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.610 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.610 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.610 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44693 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:46:58.610 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253415+0000", 2024-09-04T17:46:58.610 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 8.3019525440000006, 2024-09-04T17:46:58.610 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051141949999999999, 2024-09-04T17:46:58.610 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:58.610 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:58.610 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:58.610 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:58.610 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:58.610 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:58.610 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44693 2024-09-04T17:46:58.611 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.611 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:58.611 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.611 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:58.611 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253415+0000", 2024-09-04T17:46:58.611 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.611 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.611 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.611 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:58.611 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253415+0000", 2024-09-04T17:46:58.611 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.611 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.612 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.612 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:58.612 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253416+0000", 2024-09-04T17:46:58.612 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1799999999999999e-06 2024-09-04T17:46:58.612 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.612 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.612 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:58.612 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253422+0000", 2024-09-04T17:46:58.612 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.5940000000000001e-06 2024-09-04T17:46:58.612 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.612 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.612 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:58.612 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253438+0000", 2024-09-04T17:46:58.613 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6008999999999999e-05 2024-09-04T17:46:58.613 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.613 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.613 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.613 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253452+0000", 2024-09-04T17:46:58.613 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3543000000000001e-05 2024-09-04T17:46:58.613 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.613 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.613 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:58.613 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304533+0000", 2024-09-04T17:46:58.613 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051081609 2024-09-04T17:46:58.613 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.614 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.614 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:58.614 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304557+0000", 2024-09-04T17:46:58.614 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4015e-05 2024-09-04T17:46:58.614 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.614 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:58.614 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.614 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.614 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.614 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44694 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:46:58.614 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253469+0000", 2024-09-04T17:46:58.614 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 8.3018984190000005, 2024-09-04T17:46:58.614 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051121486000000001, 2024-09-04T17:46:58.615 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:58.615 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:58.615 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:58.615 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:58.615 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:58.615 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:58.615 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44694 2024-09-04T17:46:58.615 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.615 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:58.615 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.615 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:58.615 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253469+0000", 2024-09-04T17:46:58.615 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.615 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.616 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.616 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:58.616 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253469+0000", 2024-09-04T17:46:58.616 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.616 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.616 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.616 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:58.616 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253471+0000", 2024-09-04T17:46:58.616 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1400000000000001e-06 2024-09-04T17:46:58.616 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.616 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.616 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:58.616 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253476+0000", 2024-09-04T17:46:58.617 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7370000000000003e-06 2024-09-04T17:46:58.617 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.617 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.617 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:58.617 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253493+0000", 2024-09-04T17:46:58.617 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6549e-05 2024-09-04T17:46:58.617 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.617 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.617 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.617 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253506+0000", 2024-09-04T17:46:58.617 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3406e-05 2024-09-04T17:46:58.617 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.617 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.617 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:58.618 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304582+0000", 2024-09-04T17:46:58.618 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051075534999999998 2024-09-04T17:46:58.618 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.618 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.618 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:58.618 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304591+0000", 2024-09-04T17:46:58.618 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1190000000000003e-06 2024-09-04T17:46:58.618 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.618 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:58.618 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.618 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.618 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.619 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44695 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:46:58.619 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253552+0000", 2024-09-04T17:46:58.619 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 8.3018156649999995, 2024-09-04T17:46:58.619 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051068872000000001, 2024-09-04T17:46:58.619 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:58.619 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:58.619 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:58.619 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:58.619 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:58.619 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:58.619 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44695 2024-09-04T17:46:58.619 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.619 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:58.619 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.620 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:58.620 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253552+0000", 2024-09-04T17:46:58.620 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.620 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.620 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.620 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:58.620 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253552+0000", 2024-09-04T17:46:58.620 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.620 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.620 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.620 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:58.620 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253553+0000", 2024-09-04T17:46:58.621 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1960000000000001e-06 2024-09-04T17:46:58.621 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.621 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.621 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:58.621 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253560+0000", 2024-09-04T17:46:58.621 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.2040000000000002e-06 2024-09-04T17:46:58.621 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.621 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.621 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:58.621 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253578+0000", 2024-09-04T17:46:58.621 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8241999999999999e-05 2024-09-04T17:46:58.621 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.621 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.622 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.622 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253591+0000", 2024-09-04T17:46:58.622 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3453e-05 2024-09-04T17:46:58.622 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.622 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.622 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:58.622 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304614+0000", 2024-09-04T17:46:58.622 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051022456000000001 2024-09-04T17:46:58.622 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.622 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.622 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:58.622 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304621+0000", 2024-09-04T17:46:58.622 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3209999999999999e-06 2024-09-04T17:46:58.622 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.623 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:58.623 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.623 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.623 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.623 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44696 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:46:58.623 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253609+0000", 2024-09-04T17:46:58.623 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 8.3017591639999999, 2024-09-04T17:46:58.623 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051115351000000003, 2024-09-04T17:46:58.623 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:58.623 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:58.623 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:58.623 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:58.623 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:58.623 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:58.624 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44696 2024-09-04T17:46:58.624 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.624 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:58.624 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.624 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:58.624 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253609+0000", 2024-09-04T17:46:58.624 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.624 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.624 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.624 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:58.624 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253609+0000", 2024-09-04T17:46:58.624 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.624 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.625 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.625 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:58.625 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253610+0000", 2024-09-04T17:46:58.625 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2300000000000001e-06 2024-09-04T17:46:58.625 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.625 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.625 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:58.625 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253615+0000", 2024-09-04T17:46:58.625 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7260000000000002e-06 2024-09-04T17:46:58.625 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.625 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.625 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:58.625 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253632+0000", 2024-09-04T17:46:58.626 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7039999999999999e-05 2024-09-04T17:46:58.626 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.626 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.626 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.626 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253666+0000", 2024-09-04T17:46:58.626 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4662000000000001e-05 2024-09-04T17:46:58.626 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.626 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.626 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:58.626 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304677+0000", 2024-09-04T17:46:58.626 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051011017999999998 2024-09-04T17:46:58.626 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.626 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.627 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:58.627 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304724+0000", 2024-09-04T17:46:58.627 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6675000000000003e-05 2024-09-04T17:46:58.627 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.627 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:58.627 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.627 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.627 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.627 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44697 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:46:58.627 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253684+0000", 2024-09-04T17:46:58.627 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 8.3016836699999992, 2024-09-04T17:46:58.627 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051073107, 2024-09-04T17:46:58.628 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:58.628 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:58.628 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:58.628 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:58.628 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:58.628 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:58.628 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44697 2024-09-04T17:46:58.628 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.628 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:58.628 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.628 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:58.628 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253684+0000", 2024-09-04T17:46:58.628 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.628 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.629 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.629 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:58.629 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253684+0000", 2024-09-04T17:46:58.629 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.629 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.629 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.629 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:58.629 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253685+0000", 2024-09-04T17:46:58.629 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0699999999999999e-06 2024-09-04T17:46:58.629 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.629 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.630 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:58.630 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253690+0000", 2024-09-04T17:46:58.630 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1959999999999999e-06 2024-09-04T17:46:58.630 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.630 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.630 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:58.630 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253707+0000", 2024-09-04T17:46:58.630 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6946e-05 2024-09-04T17:46:58.630 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.630 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.630 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.630 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253720+0000", 2024-09-04T17:46:58.630 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3076e-05 2024-09-04T17:46:58.631 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.631 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.631 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:58.631 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304745+0000", 2024-09-04T17:46:58.631 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051024760000000002 2024-09-04T17:46:58.631 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.631 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.631 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:58.631 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304757+0000", 2024-09-04T17:46:58.631 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2058999999999999e-05 2024-09-04T17:46:58.631 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.631 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:58.631 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.632 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.632 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.632 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44698 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:46:58.632 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253737+0000", 2024-09-04T17:46:58.632 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 8.3016313190000002, 2024-09-04T17:46:58.632 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051176078, 2024-09-04T17:46:58.632 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:58.632 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:58.632 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:58.632 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:58.632 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:58.632 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:58.632 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44698 2024-09-04T17:46:58.632 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.633 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:58.633 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.633 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:58.633 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253737+0000", 2024-09-04T17:46:58.633 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.633 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.633 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.633 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:58.633 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253737+0000", 2024-09-04T17:46:58.633 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.633 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.633 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.633 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:58.634 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253738+0000", 2024-09-04T17:46:58.634 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0300000000000001e-06 2024-09-04T17:46:58.634 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.634 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.634 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:58.634 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253752+0000", 2024-09-04T17:46:58.634 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4358999999999999e-05 2024-09-04T17:46:58.634 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.634 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.634 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:58.634 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253771+0000", 2024-09-04T17:46:58.634 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8879999999999999e-05 2024-09-04T17:46:58.634 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.634 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.635 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.635 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253785+0000", 2024-09-04T17:46:58.635 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3852000000000001e-05 2024-09-04T17:46:58.635 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.635 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.635 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:58.635 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304876+0000", 2024-09-04T17:46:58.635 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051091105999999997 2024-09-04T17:46:58.635 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.635 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.635 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:58.635 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304913+0000", 2024-09-04T17:46:58.635 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.6850999999999997e-05 2024-09-04T17:46:58.636 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.636 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:58.636 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.636 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.636 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.636 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44699 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:46:58.636 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253804+0000", 2024-09-04T17:46:58.636 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 8.3015637949999999, 2024-09-04T17:46:58.636 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051172250000000002, 2024-09-04T17:46:58.636 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:58.636 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:58.636 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:58.637 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:58.637 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:58.637 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:58.637 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44699 2024-09-04T17:46:58.637 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.637 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:58.637 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.637 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:58.637 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253804+0000", 2024-09-04T17:46:58.637 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.637 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.637 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.637 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:58.638 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253804+0000", 2024-09-04T17:46:58.638 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.638 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.638 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.638 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:58.638 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253805+0000", 2024-09-04T17:46:58.638 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.08e-06 2024-09-04T17:46:58.638 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.638 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.638 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:58.638 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253810+0000", 2024-09-04T17:46:58.638 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1830000000000002e-06 2024-09-04T17:46:58.638 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.639 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.639 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:58.639 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253828+0000", 2024-09-04T17:46:58.639 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7662000000000001e-05 2024-09-04T17:46:58.639 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.639 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.639 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.639 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253841+0000", 2024-09-04T17:46:58.639 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3203999999999999e-05 2024-09-04T17:46:58.639 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.639 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.639 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:58.639 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304966+0000", 2024-09-04T17:46:58.640 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051125273999999998 2024-09-04T17:46:58.640 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.640 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.640 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:58.640 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304976+0000", 2024-09-04T17:46:58.640 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.8470000000000008e-06 2024-09-04T17:46:58.640 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.640 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:58.640 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.640 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.640 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.640 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44700 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:46:58.641 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253858+0000", 2024-09-04T17:46:58.641 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 8.3015101409999996, 2024-09-04T17:46:58.641 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051218957000000002, 2024-09-04T17:46:58.641 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:58.641 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:58.641 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:58.641 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:58.641 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:58.641 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:58.641 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44700 2024-09-04T17:46:58.641 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.641 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:58.641 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.641 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:58.642 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253858+0000", 2024-09-04T17:46:58.642 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.642 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.642 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.642 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:58.642 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253858+0000", 2024-09-04T17:46:58.642 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.642 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.642 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.642 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:58.642 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253859+0000", 2024-09-04T17:46:58.642 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.217e-06 2024-09-04T17:46:58.642 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.642 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.643 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:58.643 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253864+0000", 2024-09-04T17:46:58.643 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7600000000000002e-06 2024-09-04T17:46:58.643 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.643 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.643 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:58.643 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253880+0000", 2024-09-04T17:46:58.643 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6501999999999999e-05 2024-09-04T17:46:58.643 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.643 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.643 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.643 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253899+0000", 2024-09-04T17:46:58.643 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8907e-05 2024-09-04T17:46:58.644 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.644 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.644 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:58.644 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305052+0000", 2024-09-04T17:46:58.644 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051152989000000003 2024-09-04T17:46:58.644 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.644 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.644 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:58.644 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305077+0000", 2024-09-04T17:46:58.644 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4581999999999999e-05 2024-09-04T17:46:58.644 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.644 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:58.644 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.644 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.645 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.645 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44701 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:46:58.645 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253917+0000", 2024-09-04T17:46:58.645 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 8.3014507890000004, 2024-09-04T17:46:58.645 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051231113000000002, 2024-09-04T17:46:58.645 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:58.645 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:58.645 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:58.645 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:58.645 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:58.645 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:58.645 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44701 2024-09-04T17:46:58.645 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.646 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:58.646 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.646 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:58.646 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253917+0000", 2024-09-04T17:46:58.646 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.646 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.646 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.646 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:58.646 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253917+0000", 2024-09-04T17:46:58.648 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.648 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.648 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.648 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:58.648 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253918+0000", 2024-09-04T17:46:58.648 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.136e-06 2024-09-04T17:46:58.648 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.648 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.648 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:58.648 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253924+0000", 2024-09-04T17:46:58.649 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.3070000000000002e-06 2024-09-04T17:46:58.649 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.649 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.649 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:58.649 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253940+0000", 2024-09-04T17:46:58.649 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6612e-05 2024-09-04T17:46:58.649 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.649 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.649 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.649 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253954+0000", 2024-09-04T17:46:58.649 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3709999999999999e-05 2024-09-04T17:46:58.649 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.649 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.650 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:58.650 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305136+0000", 2024-09-04T17:46:58.650 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051181935999999997 2024-09-04T17:46:58.650 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.650 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.650 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:58.650 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305148+0000", 2024-09-04T17:46:58.650 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2412000000000001e-05 2024-09-04T17:46:58.650 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.650 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:58.650 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.650 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.650 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:58.651 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:58.651 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.2 to 0.663629 (a9e3) 2024-09-04T17:46:58.652 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is set 2024-09-04T17:46:58.652 INFO:teuthology.orchestra.run.smithi092.stdout:3297 writesame 0x67e1d5b thru 0x67e4e80 data_size 0x1b(0x3126 bytes) 2024-09-04T17:46:58.653 INFO:teuthology.orchestra.run.smithi092.stdout:3298 trunc from 0x67e4e81 to 0x240df8b 2024-09-04T17:46:58.653 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-04T17:46:58.655 INFO:teuthology.orchestra.run.smithi092.stdout:3300 writesame 0x2fd8de8 thru 0x2fe6759 data_size 0xd(0xd972 bytes) 2024-09-04T17:46:58.655 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:58.655 INFO:teuthology.orchestra.run.smithi092.stdout:3302 write 0x9ba493d thru 0x9ba8194 (0x3858 bytes) 2024-09-04T17:46:58.656 INFO:teuthology.orchestra.run.smithi092.stdout:3303 punch from 0x875a6ca to 0x876842a, (0xdd60 bytes) 2024-09-04T17:46:58.656 INFO:teuthology.orchestra.run.smithi092.stdout:3305 read 0x17085cf thru 0x170deeb (0x591d bytes) 2024-09-04T17:46:58.656 INFO:teuthology.orchestra.run.smithi092.stdout:3306 trunc from 0x9ba8195 to 0xbaacdd1 2024-09-04T17:46:58.656 INFO:teuthology.orchestra.run.smithi092.stdout:3308 write 0xea7acae thru 0xea8069a (0x59ed bytes) 2024-09-04T17:46:58.656 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:46:58.647+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:46:58.658 INFO:teuthology.orchestra.run.smithi092.stdout:3309 read 0x6fdad2c thru 0x6fdc246 (0x151b bytes) 2024-09-04T17:46:58.658 INFO:teuthology.orchestra.run.smithi092.stdout:3310 read 0xc27fcfb thru 0xc280c61 (0xf67 bytes) 2024-09-04T17:46:58.658 INFO:teuthology.orchestra.run.smithi092.stdout:3311 writesame 0x11d960d thru 0x11dc625 data_size 0x1(0x3019 bytes) 2024-09-04T17:46:58.687 INFO:teuthology.orchestra.run.smithi092.stdout:3312 trunc from 0xea8069b to 0x7bc3153 2024-09-04T17:46:58.735 INFO:teuthology.orchestra.run.smithi092.stdout:3313 writesame 0xe632d8b thru 0xe6390f7 data_size 0x1(0x636d bytes) 2024-09-04T17:46:58.735 INFO:teuthology.orchestra.run.smithi092.stdout:3314 punch from 0x15d6721 to 0x15e0224, (0x9b03 bytes) 2024-09-04T17:46:58.735 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:46:58.741 INFO:teuthology.orchestra.run.smithi092.stdout:3317 trunc from 0xe6390f8 to 0xb6bc0b2 2024-09-04T17:46:58.753 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:58.751+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:58.753 INFO:teuthology.orchestra.run.smithi092.stdout:3319 write 0x8fa5d07 thru 0x8fa7464 (0x175e bytes) 2024-09-04T17:46:58.763 INFO:teuthology.orchestra.run.smithi092.stdout:3321 write 0x2392b8c thru 0x23994a5 (0x691a bytes) 2024-09-04T17:46:58.766 INFO:teuthology.orchestra.run.smithi092.stdout:3322 write 0xa24377 thru 0xa2f4e5 (0xb16f bytes) 2024-09-04T17:46:58.830 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:58.831 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:46:58.831 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.831 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:48759 3.1f 3.4d333f9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-04T17:46:58.831 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:58.770317+0000", 2024-09-04T17:46:58.831 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.059867614999999999, 2024-09-04T17:46:58.831 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.059889812000000001, 2024-09-04T17:46:58.832 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:58.832 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:58.832 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "waiting for peered", 2024-09-04T17:46:58.832 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:58.832 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:58.832 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:58.832 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 48759 2024-09-04T17:46:58.833 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.833 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:58.833 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.833 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:58.833 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770317+0000", 2024-09-04T17:46:58.833 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.833 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.833 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.834 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:58.834 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770317+0000", 2024-09-04T17:46:58.834 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.834 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.834 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.834 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:58.834 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770317+0000", 2024-09-04T17:46:58.834 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.6599999999999995e-07 2024-09-04T17:46:58.835 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.835 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.835 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:58.835 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770321+0000", 2024-09-04T17:46:58.835 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.6100000000000002e-06 2024-09-04T17:46:58.835 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.835 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.835 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:58.836 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770349+0000", 2024-09-04T17:46:58.836 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.7780000000000002e-05 2024-09-04T17:46:58.836 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.836 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.836 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.836 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770357+0000", 2024-09-04T17:46:58.836 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.6489999999999997e-06 2024-09-04T17:46:58.836 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.837 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.837 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:58.837 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770407+0000", 2024-09-04T17:46:58.837 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0037000000000002e-05 2024-09-04T17:46:58.837 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.837 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.837 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for peered", 2024-09-04T17:46:58.837 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770415+0000", 2024-09-04T17:46:58.837 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.6850000000000001e-06 2024-09-04T17:46:58.837 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.837 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:58.837 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.838 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.838 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.838 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:48762 3.1f 3.4d333f9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-04T17:46:58.838 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:58.770389+0000", 2024-09-04T17:46:58.838 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.059795490999999999, 2024-09-04T17:46:58.838 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.059857868000000002, 2024-09-04T17:46:58.838 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:58.838 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:58.839 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "waiting for peered", 2024-09-04T17:46:58.839 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:58.839 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:58.839 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:58.839 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 48762 2024-09-04T17:46:58.839 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.839 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:58.839 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.839 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:58.839 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770389+0000", 2024-09-04T17:46:58.839 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.840 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.840 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.840 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:58.840 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770389+0000", 2024-09-04T17:46:58.840 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.840 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.840 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.840 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:58.840 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770389+0000", 2024-09-04T17:46:58.840 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.4399999999999999e-07 2024-09-04T17:46:58.840 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.840 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.841 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:58.841 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770393+0000", 2024-09-04T17:46:58.841 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.9169999999999999e-06 2024-09-04T17:46:58.841 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.841 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.841 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:58.841 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770403+0000", 2024-09-04T17:46:58.841 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0081000000000001e-05 2024-09-04T17:46:58.841 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.842 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.842 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.842 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770411+0000", 2024-09-04T17:46:58.842 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.7929999999999998e-06 2024-09-04T17:46:58.842 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.842 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.842 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:58.842 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770474+0000", 2024-09-04T17:46:58.842 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.3046000000000006e-05 2024-09-04T17:46:58.842 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.842 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.842 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for peered", 2024-09-04T17:46:58.842 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770482+0000", 2024-09-04T17:46:58.843 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.0630000000000006e-06 2024-09-04T17:46:58.843 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.843 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:58.843 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.843 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.843 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.843 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:48766 3.1f 3.4d333f9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-04T17:46:58.843 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:58.770439+0000", 2024-09-04T17:46:58.843 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.059744773000000001, 2024-09-04T17:46:58.843 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.059849918000000002, 2024-09-04T17:46:58.843 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:58.844 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:58.844 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "waiting for peered", 2024-09-04T17:46:58.844 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:58.844 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:58.844 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:58.844 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 48766 2024-09-04T17:46:58.844 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.844 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:58.844 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.844 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:58.844 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770439+0000", 2024-09-04T17:46:58.845 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.845 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.845 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.845 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:58.845 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770439+0000", 2024-09-04T17:46:58.845 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.845 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.845 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.845 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:58.845 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770440+0000", 2024-09-04T17:46:58.845 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3600000000000003e-07 2024-09-04T17:46:58.845 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.846 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.846 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:58.846 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770444+0000", 2024-09-04T17:46:58.846 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4740000000000001e-06 2024-09-04T17:46:58.846 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.846 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.846 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:58.846 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770453+0000", 2024-09-04T17:46:58.846 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.0710000000000001e-06 2024-09-04T17:46:58.846 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.846 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.846 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.847 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770460+0000", 2024-09-04T17:46:58.847 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1439999999999997e-06 2024-09-04T17:46:58.847 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.847 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.847 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:58.847 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770509+0000", 2024-09-04T17:46:58.847 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9431000000000002e-05 2024-09-04T17:46:58.847 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.847 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.847 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for peered", 2024-09-04T17:46:58.847 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770517+0000", 2024-09-04T17:46:58.847 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3000000000000004e-06 2024-09-04T17:46:58.848 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.848 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:58.848 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.848 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.848 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.848 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:48769 3.1f 3.4d333f9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-04T17:46:58.848 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:58.770471+0000", 2024-09-04T17:46:58.848 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.059712719999999997, 2024-09-04T17:46:58.848 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.059849050000000001, 2024-09-04T17:46:58.848 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:58.848 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:58.848 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "waiting for peered", 2024-09-04T17:46:58.848 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:58.849 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:58.849 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:58.849 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 48769 2024-09-04T17:46:58.849 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.849 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:58.849 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.849 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:58.849 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770471+0000", 2024-09-04T17:46:58.849 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.849 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.849 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.849 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:58.849 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770471+0000", 2024-09-04T17:46:58.850 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.850 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.850 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.850 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:58.850 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770472+0000", 2024-09-04T17:46:58.850 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.4600000000000004e-07 2024-09-04T17:46:58.850 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.850 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.850 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:58.850 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770475+0000", 2024-09-04T17:46:58.850 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.225e-06 2024-09-04T17:46:58.850 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.851 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.851 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:58.851 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770485+0000", 2024-09-04T17:46:58.851 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4439999999999997e-06 2024-09-04T17:46:58.851 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.851 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.851 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.851 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770505+0000", 2024-09-04T17:46:58.851 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0075999999999999e-05 2024-09-04T17:46:58.851 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.851 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.851 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:58.852 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770542+0000", 2024-09-04T17:46:58.852 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7598999999999997e-05 2024-09-04T17:46:58.852 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.852 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.852 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for peered", 2024-09-04T17:46:58.852 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770550+0000", 2024-09-04T17:46:58.852 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.5100000000000001e-06 2024-09-04T17:46:58.852 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.852 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:58.852 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.853 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.853 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.853 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:48775 3.1f 3.4d333f9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-04T17:46:58.853 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:58.770521+0000", 2024-09-04T17:46:58.853 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.059663344, 2024-09-04T17:46:58.853 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.059831122, 2024-09-04T17:46:58.853 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:58.853 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:58.853 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "waiting for peered", 2024-09-04T17:46:58.853 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:58.853 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:58.853 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:58.853 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 48775 2024-09-04T17:46:58.853 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.854 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:58.854 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.854 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:58.854 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770521+0000", 2024-09-04T17:46:58.854 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.854 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.854 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.854 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:58.854 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770521+0000", 2024-09-04T17:46:58.854 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.854 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.855 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.855 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:58.855 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770522+0000", 2024-09-04T17:46:58.855 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.0800000000000004e-07 2024-09-04T17:46:58.855 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.855 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.855 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:58.855 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770525+0000", 2024-09-04T17:46:58.855 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.472e-06 2024-09-04T17:46:58.855 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.855 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.855 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:58.855 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770535+0000", 2024-09-04T17:46:58.856 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.6879999999999994e-06 2024-09-04T17:46:58.856 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.856 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.856 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.856 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770544+0000", 2024-09-04T17:46:58.856 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1400000000000006e-06 2024-09-04T17:46:58.856 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.856 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.856 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:58.856 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770578+0000", 2024-09-04T17:46:58.856 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4078999999999997e-05 2024-09-04T17:46:58.856 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.857 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.857 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for peered", 2024-09-04T17:46:58.857 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770586+0000", 2024-09-04T17:46:58.857 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.7239999999999992e-06 2024-09-04T17:46:58.857 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.857 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:58.857 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.857 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.857 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.857 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:48777 3.1f 3.4d333f9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-04T17:46:58.857 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:58.770555+0000", 2024-09-04T17:46:58.857 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.059629046999999998, 2024-09-04T17:46:58.857 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.059827483000000001, 2024-09-04T17:46:58.858 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:58.858 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:58.858 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "waiting for peered", 2024-09-04T17:46:58.858 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:58.858 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:58.858 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:58.858 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 48777 2024-09-04T17:46:58.858 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.858 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:58.858 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.858 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:58.858 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770555+0000", 2024-09-04T17:46:58.858 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.859 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.859 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.859 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:58.859 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770555+0000", 2024-09-04T17:46:58.859 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.859 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.859 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.859 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:58.859 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770556+0000", 2024-09-04T17:46:58.859 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1600000000000001e-07 2024-09-04T17:46:58.859 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.859 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.859 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:58.860 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770559+0000", 2024-09-04T17:46:58.860 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.5659999999999998e-06 2024-09-04T17:46:58.860 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.860 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.860 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:58.860 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770569+0000", 2024-09-04T17:46:58.860 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.2979999999999997e-06 2024-09-04T17:46:58.860 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.860 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.860 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.860 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770576+0000", 2024-09-04T17:46:58.860 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.148e-06 2024-09-04T17:46:58.860 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.861 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.861 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:58.861 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770613+0000", 2024-09-04T17:46:58.861 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7422999999999997e-05 2024-09-04T17:46:58.861 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.861 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.861 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for peered", 2024-09-04T17:46:58.861 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770621+0000", 2024-09-04T17:46:58.861 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.6389999999999992e-06 2024-09-04T17:46:58.861 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.861 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:58.861 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.861 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.861 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.862 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:48784 3.1f 3.4d333f9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-04T17:46:58.862 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:58.770681+0000", 2024-09-04T17:46:58.862 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.059503514, 2024-09-04T17:46:58.862 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.059732585999999997, 2024-09-04T17:46:58.862 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:58.862 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:58.862 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "waiting for peered", 2024-09-04T17:46:58.862 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:58.862 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:58.862 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:58.862 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 48784 2024-09-04T17:46:58.862 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.863 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:58.863 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.863 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:58.863 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770681+0000", 2024-09-04T17:46:58.863 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.863 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.863 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.863 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:58.863 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770681+0000", 2024-09-04T17:46:58.863 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.863 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.863 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.863 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:58.864 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770682+0000", 2024-09-04T17:46:58.864 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.13e-06 2024-09-04T17:46:58.864 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.864 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.864 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:58.864 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770687+0000", 2024-09-04T17:46:58.864 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9559999999999996e-06 2024-09-04T17:46:58.864 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.864 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.864 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:58.864 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770703+0000", 2024-09-04T17:46:58.864 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6325e-05 2024-09-04T17:46:58.864 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.864 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.865 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.865 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770713+0000", 2024-09-04T17:46:58.865 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5910000000000002e-06 2024-09-04T17:46:58.865 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.865 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.865 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:58.865 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770771+0000", 2024-09-04T17:46:58.865 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.8053000000000001e-05 2024-09-04T17:46:58.866 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.866 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.866 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for peered", 2024-09-04T17:46:58.866 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770779+0000", 2024-09-04T17:46:58.866 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.4710000000000007e-06 2024-09-04T17:46:58.866 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.866 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:58.866 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.866 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.866 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.866 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:48785 3.1f 3.4d333f9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-04T17:46:58.866 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:58.770733+0000", 2024-09-04T17:46:58.867 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.059450930999999999, 2024-09-04T17:46:58.867 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.059711749000000001, 2024-09-04T17:46:58.867 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:58.867 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:58.867 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "waiting for peered", 2024-09-04T17:46:58.867 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:58.867 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:58.867 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:58.867 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 48785 2024-09-04T17:46:58.867 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.867 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:58.867 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.867 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:58.867 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770733+0000", 2024-09-04T17:46:58.868 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.868 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.868 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.868 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:58.868 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770733+0000", 2024-09-04T17:46:58.868 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.868 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.868 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.868 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:58.868 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770734+0000", 2024-09-04T17:46:58.868 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.2900000000000002e-07 2024-09-04T17:46:58.868 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.868 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.868 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:58.869 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770748+0000", 2024-09-04T17:46:58.869 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4439e-05 2024-09-04T17:46:58.869 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.869 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.869 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:58.869 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770760+0000", 2024-09-04T17:46:58.869 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1717e-05 2024-09-04T17:46:58.869 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.869 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.869 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.869 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770777+0000", 2024-09-04T17:46:58.870 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6424999999999999e-05 2024-09-04T17:46:58.870 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.870 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.870 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:58.870 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770835+0000", 2024-09-04T17:46:58.870 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.8581e-05 2024-09-04T17:46:58.870 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.870 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.870 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for peered", 2024-09-04T17:46:58.870 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770843+0000", 2024-09-04T17:46:58.870 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.0069999999999997e-06 2024-09-04T17:46:58.870 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.871 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:58.871 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.871 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.871 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.871 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:48791 3.1f 3.4d333f9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-04T17:46:58.871 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:58.770809+0000", 2024-09-04T17:46:58.871 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.059375285999999999, 2024-09-04T17:46:58.871 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.059666916, 2024-09-04T17:46:58.871 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:58.871 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:58.871 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "waiting for peered", 2024-09-04T17:46:58.871 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:58.871 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:58.872 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:58.872 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 48791 2024-09-04T17:46:58.872 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.872 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:58.872 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.872 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:58.872 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770809+0000", 2024-09-04T17:46:58.872 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.872 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.872 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.872 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:58.872 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770809+0000", 2024-09-04T17:46:58.872 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.873 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.873 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.873 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:58.873 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770810+0000", 2024-09-04T17:46:58.873 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.1100000000000005e-07 2024-09-04T17:46:58.873 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.873 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.873 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:58.873 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770813+0000", 2024-09-04T17:46:58.873 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.1209999999999998e-06 2024-09-04T17:46:58.873 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.873 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.873 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:58.874 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770823+0000", 2024-09-04T17:46:58.874 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0108999999999999e-05 2024-09-04T17:46:58.874 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.874 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.874 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.874 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770831+0000", 2024-09-04T17:46:58.874 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.2640000000000008e-06 2024-09-04T17:46:58.874 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.874 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.874 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:58.874 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770873+0000", 2024-09-04T17:46:58.874 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2318999999999999e-05 2024-09-04T17:46:58.875 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.875 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.875 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for peered", 2024-09-04T17:46:58.875 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770881+0000", 2024-09-04T17:46:58.875 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.858e-06 2024-09-04T17:46:58.875 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.875 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:58.875 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.875 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.875 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.875 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:48793 3.1f 3.4d333f9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-04T17:46:58.875 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:58.770844+0000", 2024-09-04T17:46:58.875 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.059340346000000002, 2024-09-04T17:46:58.876 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.059662422, 2024-09-04T17:46:58.876 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:58.876 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:58.876 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "waiting for peered", 2024-09-04T17:46:58.876 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:58.876 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:58.876 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:58.876 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 48793 2024-09-04T17:46:58.876 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.876 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:58.876 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.876 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:58.877 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770844+0000", 2024-09-04T17:46:58.877 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.877 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.877 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.877 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:58.877 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770844+0000", 2024-09-04T17:46:58.877 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.877 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.877 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.877 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:58.877 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770845+0000", 2024-09-04T17:46:58.877 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.33e-07 2024-09-04T17:46:58.877 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.877 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.878 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:58.878 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770848+0000", 2024-09-04T17:46:58.878 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.303e-06 2024-09-04T17:46:58.878 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.878 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.878 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:58.878 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770857+0000", 2024-09-04T17:46:58.878 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9379999999999998e-06 2024-09-04T17:46:58.878 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.878 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.879 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.879 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770876+0000", 2024-09-04T17:46:58.879 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9191e-05 2024-09-04T17:46:58.879 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.879 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.879 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:58.879 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770945+0000", 2024-09-04T17:46:58.879 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.9225000000000002e-05 2024-09-04T17:46:58.879 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.879 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.879 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for peered", 2024-09-04T17:46:58.880 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770954+0000", 2024-09-04T17:46:58.880 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.5520000000000005e-06 2024-09-04T17:46:58.880 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.880 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:58.880 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.880 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.880 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.880 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:48797 3.1f 3.4d333f9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-04T17:46:58.880 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:58.770925+0000", 2024-09-04T17:46:58.880 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.059259306999999997, 2024-09-04T17:46:58.880 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.059611801999999998, 2024-09-04T17:46:58.880 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:58.880 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:58.881 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "waiting for peered", 2024-09-04T17:46:58.881 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:58.881 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:58.881 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:58.881 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 48797 2024-09-04T17:46:58.881 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.881 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:58.881 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.881 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:58.881 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770925+0000", 2024-09-04T17:46:58.881 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.881 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.882 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.882 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:58.882 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770925+0000", 2024-09-04T17:46:58.882 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.882 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.882 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.882 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:58.882 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770926+0000", 2024-09-04T17:46:58.882 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.3300000000000001e-07 2024-09-04T17:46:58.882 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.882 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.882 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:58.882 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770929+0000", 2024-09-04T17:46:58.883 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.5200000000000002e-06 2024-09-04T17:46:58.883 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.883 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.883 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:58.883 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770939+0000", 2024-09-04T17:46:58.883 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.7180000000000008e-06 2024-09-04T17:46:58.883 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.883 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.883 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.883 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770946+0000", 2024-09-04T17:46:58.883 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2339999999999997e-06 2024-09-04T17:46:58.883 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.883 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.884 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:58.884 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770981+0000", 2024-09-04T17:46:58.884 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.5154999999999998e-05 2024-09-04T17:46:58.884 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.884 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.884 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for peered", 2024-09-04T17:46:58.884 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770989+0000", 2024-09-04T17:46:58.884 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.7840000000000004e-06 2024-09-04T17:46:58.884 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.884 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:58.884 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.884 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.885 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.885 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:48798 3.1f 3.4d333f9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-04T17:46:58.885 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:58.770958+0000", 2024-09-04T17:46:58.885 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.059226254999999998, 2024-09-04T17:46:58.885 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.059609421000000003, 2024-09-04T17:46:58.885 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:58.885 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:58.885 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "waiting for peered", 2024-09-04T17:46:58.885 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:58.885 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:58.885 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:58.885 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 48798 2024-09-04T17:46:58.885 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.886 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:58.886 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.886 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:58.886 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770958+0000", 2024-09-04T17:46:58.886 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.886 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.886 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.886 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:58.886 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770958+0000", 2024-09-04T17:46:58.886 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.886 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.886 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.887 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:58.887 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770958+0000", 2024-09-04T17:46:58.887 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6100000000000001e-07 2024-09-04T17:46:58.887 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.887 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.887 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:58.887 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770962+0000", 2024-09-04T17:46:58.887 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.2799999999999999e-06 2024-09-04T17:46:58.887 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.887 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.887 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:58.887 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770982+0000", 2024-09-04T17:46:58.887 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9936999999999999e-05 2024-09-04T17:46:58.887 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.888 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.888 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.888 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770991+0000", 2024-09-04T17:46:58.888 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.918e-06 2024-09-04T17:46:58.888 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.888 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.888 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:58.888 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771037+0000", 2024-09-04T17:46:58.888 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6465999999999999e-05 2024-09-04T17:46:58.888 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.888 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.888 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for peered", 2024-09-04T17:46:58.889 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771044+0000", 2024-09-04T17:46:58.889 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4680000000000003e-06 2024-09-04T17:46:58.889 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.889 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:58.889 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.889 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.889 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.889 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:48802 3.1f 3.4d333f9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-04T17:46:58.889 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:58.771012+0000", 2024-09-04T17:46:58.889 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.059172291000000002, 2024-09-04T17:46:58.889 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.059586427999999997, 2024-09-04T17:46:58.889 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:58.889 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:58.890 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "waiting for peered", 2024-09-04T17:46:58.890 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:58.890 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:58.890 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:58.890 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 48802 2024-09-04T17:46:58.890 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.890 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:58.890 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.891 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:58.891 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771012+0000", 2024-09-04T17:46:58.891 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.891 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.891 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.891 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:58.891 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771012+0000", 2024-09-04T17:46:58.891 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.891 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.891 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.891 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:58.892 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771013+0000", 2024-09-04T17:46:58.892 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.0400000000000005e-07 2024-09-04T17:46:58.892 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.892 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.892 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:58.892 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771017+0000", 2024-09-04T17:46:58.892 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.206e-06 2024-09-04T17:46:58.892 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.892 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.892 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:58.892 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771027+0000", 2024-09-04T17:46:58.892 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.7540000000000001e-06 2024-09-04T17:46:58.892 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.893 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.893 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.893 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771034+0000", 2024-09-04T17:46:58.893 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3939999999999998e-06 2024-09-04T17:46:58.893 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.893 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.893 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:58.893 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771071+0000", 2024-09-04T17:46:58.893 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7015999999999999e-05 2024-09-04T17:46:58.893 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.893 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.893 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for peered", 2024-09-04T17:46:58.893 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771078+0000", 2024-09-04T17:46:58.894 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.0979999999999996e-06 2024-09-04T17:46:58.894 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.894 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:58.894 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.894 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.894 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.894 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:48806 3.1f 3.4d333f9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-04T17:46:58.894 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:58.771079+0000", 2024-09-04T17:46:58.894 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.059104652000000001, 2024-09-04T17:46:58.894 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.059549700999999997, 2024-09-04T17:46:58.894 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:58.894 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:58.894 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "waiting for peered", 2024-09-04T17:46:58.894 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:58.895 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:58.895 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:58.895 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 48806 2024-09-04T17:46:58.895 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.895 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:58.895 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.895 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:58.895 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771079+0000", 2024-09-04T17:46:58.895 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.895 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.895 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.895 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:58.895 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771079+0000", 2024-09-04T17:46:58.895 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.895 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.896 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.896 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:58.896 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771080+0000", 2024-09-04T17:46:58.896 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.3900000000000004e-07 2024-09-04T17:46:58.896 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.896 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.896 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:58.896 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771084+0000", 2024-09-04T17:46:58.896 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.732e-06 2024-09-04T17:46:58.896 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.896 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.896 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:58.896 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771105+0000", 2024-09-04T17:46:58.896 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0925e-05 2024-09-04T17:46:58.896 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.897 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.897 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.897 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771114+0000", 2024-09-04T17:46:58.897 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.6810000000000004e-06 2024-09-04T17:46:58.897 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.897 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.897 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:58.897 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771191+0000", 2024-09-04T17:46:58.897 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.7815e-05 2024-09-04T17:46:58.897 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.897 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.897 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for peered", 2024-09-04T17:46:58.897 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771207+0000", 2024-09-04T17:46:58.898 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5637999999999998e-05 2024-09-04T17:46:58.898 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.898 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:58.898 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.898 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.898 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.898 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:48807 3.1f 3.4d333f9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-04T17:46:58.898 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:58.771141+0000", 2024-09-04T17:46:58.898 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.059042717000000002, 2024-09-04T17:46:58.898 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.059522296000000002, 2024-09-04T17:46:58.898 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:58.898 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:58.898 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "waiting for peered", 2024-09-04T17:46:58.899 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:58.899 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:58.899 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:58.899 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 48807 2024-09-04T17:46:58.899 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.899 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:58.899 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.899 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:58.899 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771141+0000", 2024-09-04T17:46:58.899 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.899 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.899 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.899 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:58.899 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771141+0000", 2024-09-04T17:46:58.900 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.900 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.900 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.900 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:58.900 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771142+0000", 2024-09-04T17:46:58.900 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.4000000000000001e-07 2024-09-04T17:46:58.900 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.900 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.900 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:58.900 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771145+0000", 2024-09-04T17:46:58.900 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.8909999999999999e-06 2024-09-04T17:46:58.900 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.901 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.901 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:58.901 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771154+0000", 2024-09-04T17:46:58.901 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.3619999999999995e-06 2024-09-04T17:46:58.901 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.901 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.901 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.901 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771161+0000", 2024-09-04T17:46:58.901 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.0090000000000001e-06 2024-09-04T17:46:58.901 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.901 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.901 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:58.901 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771246+0000", 2024-09-04T17:46:58.901 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.4272999999999998e-05 2024-09-04T17:46:58.902 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.902 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.902 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for peered", 2024-09-04T17:46:58.902 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771258+0000", 2024-09-04T17:46:58.902 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2218999999999999e-05 2024-09-04T17:46:58.902 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.902 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:58.902 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.902 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.902 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.902 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:48814 3.1f 3.4d333f9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-04T17:46:58.902 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:58.771263+0000", 2024-09-04T17:46:58.903 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.058921379000000003, 2024-09-04T17:46:58.903 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.059431306000000003, 2024-09-04T17:46:58.903 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:58.903 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:58.903 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "waiting for peered", 2024-09-04T17:46:58.903 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:58.903 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:58.903 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:58.903 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 48814 2024-09-04T17:46:58.903 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.903 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:58.903 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.904 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:58.904 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771263+0000", 2024-09-04T17:46:58.904 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.904 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.904 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.904 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:58.904 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771263+0000", 2024-09-04T17:46:58.904 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.904 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.904 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.904 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:58.904 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771264+0000", 2024-09-04T17:46:58.904 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.47e-07 2024-09-04T17:46:58.904 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.904 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.905 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:58.905 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771268+0000", 2024-09-04T17:46:58.905 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2440000000000004e-06 2024-09-04T17:46:58.905 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.905 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.905 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:58.905 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771277+0000", 2024-09-04T17:46:58.905 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.3030000000000005e-06 2024-09-04T17:46:58.905 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.905 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.905 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.905 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771283+0000", 2024-09-04T17:46:58.905 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6319999999999996e-06 2024-09-04T17:46:58.906 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.906 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.906 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:58.906 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771528+0000", 2024-09-04T17:46:58.906 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000245252 2024-09-04T17:46:58.906 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.906 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.906 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for peered", 2024-09-04T17:46:58.906 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771539+0000", 2024-09-04T17:46:58.906 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0584000000000001e-05 2024-09-04T17:46:58.906 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.906 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:58.906 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.907 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.907 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.907 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:48815 3.1f 3.4d333f9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-04T17:46:58.907 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:58.771477+0000", 2024-09-04T17:46:58.907 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.058706861999999999, 2024-09-04T17:46:58.907 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.059247016, 2024-09-04T17:46:58.907 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:58.907 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:58.907 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "waiting for peered", 2024-09-04T17:46:58.907 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:58.907 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:58.907 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:58.908 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 48815 2024-09-04T17:46:58.908 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.908 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:58.908 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.908 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:58.908 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771477+0000", 2024-09-04T17:46:58.908 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.908 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.908 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.908 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:58.908 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771477+0000", 2024-09-04T17:46:58.908 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.908 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.909 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.909 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:58.909 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771478+0000", 2024-09-04T17:46:58.909 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.8999999999999996e-07 2024-09-04T17:46:58.909 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.909 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.909 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:58.909 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771482+0000", 2024-09-04T17:46:58.909 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7400000000000002e-06 2024-09-04T17:46:58.909 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.909 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.909 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:58.910 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771500+0000", 2024-09-04T17:46:58.910 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8668000000000002e-05 2024-09-04T17:46:58.910 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.910 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.910 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.910 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771507+0000", 2024-09-04T17:46:58.910 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.5180000000000001e-06 2024-09-04T17:46:58.910 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.910 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.910 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:58.910 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771572+0000", 2024-09-04T17:46:58.910 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.5139000000000001e-05 2024-09-04T17:46:58.911 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.911 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.911 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for peered", 2024-09-04T17:46:58.911 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771580+0000", 2024-09-04T17:46:58.911 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.9030000000000004e-06 2024-09-04T17:46:58.911 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.911 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:58.911 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.911 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.911 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.911 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:48821 3.1f 3.4d333f9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-04T17:46:58.911 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:58.771522+0000", 2024-09-04T17:46:58.911 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.058661948999999998, 2024-09-04T17:46:58.912 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.059232102000000002, 2024-09-04T17:46:58.912 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:58.912 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:58.912 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "waiting for peered", 2024-09-04T17:46:58.912 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:58.912 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:58.912 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:58.912 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 48821 2024-09-04T17:46:58.912 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.912 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:58.912 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.913 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:58.913 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771522+0000", 2024-09-04T17:46:58.913 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.913 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.913 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.913 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:58.913 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771522+0000", 2024-09-04T17:46:58.913 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.913 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.913 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.913 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:58.913 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771523+0000", 2024-09-04T17:46:58.914 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1600000000000001e-07 2024-09-04T17:46:58.914 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.914 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.914 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:58.914 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771525+0000", 2024-09-04T17:46:58.914 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4200000000000001e-06 2024-09-04T17:46:58.914 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.914 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.914 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:58.914 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771531+0000", 2024-09-04T17:46:58.914 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.1410000000000002e-06 2024-09-04T17:46:58.914 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.914 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.915 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.915 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771537+0000", 2024-09-04T17:46:58.915 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.9150000000000001e-06 2024-09-04T17:46:58.915 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.915 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.915 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:58.915 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771608+0000", 2024-09-04T17:46:58.915 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1273999999999996e-05 2024-09-04T17:46:58.915 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.915 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.915 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for peered", 2024-09-04T17:46:58.915 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771616+0000", 2024-09-04T17:46:58.916 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.5619999999999998e-06 2024-09-04T17:46:58.916 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.916 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:58.916 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.916 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.916 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.916 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:48823 3.1f 3.4d333f9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-04T17:46:58.916 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:58.771545+0000", 2024-09-04T17:46:58.916 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.058639297, 2024-09-04T17:46:58.916 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.059239285000000003, 2024-09-04T17:46:58.916 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:58.916 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:58.916 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "waiting for peered", 2024-09-04T17:46:58.917 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:58.917 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:58.917 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:58.917 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 48823 2024-09-04T17:46:58.917 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.917 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:58.917 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.917 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:58.917 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771545+0000", 2024-09-04T17:46:58.917 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.917 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.917 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.917 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:58.918 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771545+0000", 2024-09-04T17:46:58.918 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.918 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.918 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.918 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:58.918 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771545+0000", 2024-09-04T17:46:58.918 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.72e-07 2024-09-04T17:46:58.918 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.918 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.918 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:58.918 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771547+0000", 2024-09-04T17:46:58.918 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7740000000000001e-06 2024-09-04T17:46:58.919 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.919 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.919 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:58.919 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771552+0000", 2024-09-04T17:46:58.919 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9849999999999997e-06 2024-09-04T17:46:58.919 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.919 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.919 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.919 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771559+0000", 2024-09-04T17:46:58.920 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1520000000000003e-06 2024-09-04T17:46:58.920 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.920 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.920 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:58.920 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771648+0000", 2024-09-04T17:46:58.920 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8919999999999996e-05 2024-09-04T17:46:58.920 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.920 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.920 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for peered", 2024-09-04T17:46:58.920 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771655+0000", 2024-09-04T17:46:58.920 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6150000000000002e-06 2024-09-04T17:46:58.920 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.921 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:58.921 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.921 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.921 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.921 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:48830 3.1f 3.4d333f9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-04T17:46:58.921 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:58.771607+0000", 2024-09-04T17:46:58.921 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.058577110000000002, 2024-09-04T17:46:58.921 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.059206900999999999, 2024-09-04T17:46:58.921 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:58.921 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:58.921 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "waiting for peered", 2024-09-04T17:46:58.921 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:58.922 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:58.922 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:58.922 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 48830 2024-09-04T17:46:58.922 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.922 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:58.922 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.922 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:58.922 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771607+0000", 2024-09-04T17:46:58.922 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.922 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.922 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.922 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:58.923 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771607+0000", 2024-09-04T17:46:58.923 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.923 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.923 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.923 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:58.923 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771608+0000", 2024-09-04T17:46:58.923 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.1099999999999995e-07 2024-09-04T17:46:58.923 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.923 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.923 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:58.923 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771611+0000", 2024-09-04T17:46:58.923 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0620000000000001e-06 2024-09-04T17:46:58.924 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.924 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.924 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:58.924 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771618+0000", 2024-09-04T17:46:58.924 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.9090000000000003e-06 2024-09-04T17:46:58.924 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.924 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.924 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.924 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771623+0000", 2024-09-04T17:46:58.924 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.152e-06 2024-09-04T17:46:58.924 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.925 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.925 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:58.925 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771679+0000", 2024-09-04T17:46:58.925 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6711e-05 2024-09-04T17:46:58.925 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.925 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.925 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for peered", 2024-09-04T17:46:58.925 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771687+0000", 2024-09-04T17:46:58.925 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4000000000000003e-06 2024-09-04T17:46:58.925 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.925 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:58.925 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.925 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.926 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.926 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:48831 3.1f 3.4d333f9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-04T17:46:58.926 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:58.771640+0000", 2024-09-04T17:46:58.926 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.058544324000000002, 2024-09-04T17:46:58.926 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.059203770000000003, 2024-09-04T17:46:58.926 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:58.926 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:58.926 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "waiting for peered", 2024-09-04T17:46:58.926 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:58.926 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:58.926 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:58.926 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 48831 2024-09-04T17:46:58.926 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.927 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:58.927 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.927 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:58.927 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771640+0000", 2024-09-04T17:46:58.927 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.927 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.927 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.927 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:58.927 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771640+0000", 2024-09-04T17:46:58.927 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.927 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.927 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.927 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:58.927 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771640+0000", 2024-09-04T17:46:58.928 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.5600000000000001e-07 2024-09-04T17:46:58.928 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.928 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.928 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:58.928 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771642+0000", 2024-09-04T17:46:58.928 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.911e-06 2024-09-04T17:46:58.928 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.928 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.928 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:58.928 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771647+0000", 2024-09-04T17:46:58.928 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2610000000000001e-06 2024-09-04T17:46:58.928 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.928 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.929 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.929 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771651+0000", 2024-09-04T17:46:58.929 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.1659999999999996e-06 2024-09-04T17:46:58.929 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.929 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.929 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:58.929 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771712+0000", 2024-09-04T17:46:58.929 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.0489999999999999e-05 2024-09-04T17:46:58.929 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.929 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.929 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for peered", 2024-09-04T17:46:58.929 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771719+0000", 2024-09-04T17:46:58.929 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3309999999999998e-06 2024-09-04T17:46:58.930 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.930 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:58.930 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.930 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.930 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.930 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:48838 3.1f 3.4d333f9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-04T17:46:58.930 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:58.771726+0000", 2024-09-04T17:46:58.930 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.058458595000000002, 2024-09-04T17:46:58.930 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.059147984000000001, 2024-09-04T17:46:58.930 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:58.930 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:58.930 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "waiting for peered", 2024-09-04T17:46:58.930 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:58.931 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:58.931 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:58.931 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 48838 2024-09-04T17:46:58.931 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.931 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:58.931 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.931 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:58.931 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771726+0000", 2024-09-04T17:46:58.931 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.931 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.931 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.931 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:58.931 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771726+0000", 2024-09-04T17:46:58.931 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.931 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.932 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.932 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:58.932 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771726+0000", 2024-09-04T17:46:58.932 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.51e-07 2024-09-04T17:46:58.932 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.932 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.932 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:58.932 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771728+0000", 2024-09-04T17:46:58.932 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4159999999999998e-06 2024-09-04T17:46:58.932 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.932 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.932 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:58.932 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771735+0000", 2024-09-04T17:46:58.932 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6150000000000002e-06 2024-09-04T17:46:58.933 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.933 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.933 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.933 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771740+0000", 2024-09-04T17:46:58.933 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4660000000000001e-06 2024-09-04T17:46:58.933 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.933 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.933 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:58.933 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771784+0000", 2024-09-04T17:46:58.933 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4855999999999999e-05 2024-09-04T17:46:58.933 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.933 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.933 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for peered", 2024-09-04T17:46:58.934 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771793+0000", 2024-09-04T17:46:58.934 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.1249999999999993e-06 2024-09-04T17:46:58.934 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.934 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:58.934 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.934 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.934 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.934 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:48839 3.1f 3.4d333f9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-04T17:46:58.934 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:58.771769+0000", 2024-09-04T17:46:58.934 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.058415177999999998, 2024-09-04T17:46:58.934 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.059134415000000003, 2024-09-04T17:46:58.934 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:58.934 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:58.935 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "waiting for peered", 2024-09-04T17:46:58.935 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:58.935 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:58.935 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:58.935 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 48839 2024-09-04T17:46:58.935 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.935 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:58.935 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.935 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:58.935 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771769+0000", 2024-09-04T17:46:58.935 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.935 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.935 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.936 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:58.936 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771769+0000", 2024-09-04T17:46:58.936 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.936 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.936 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.936 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:58.936 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771770+0000", 2024-09-04T17:46:58.936 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.7000000000000003e-07 2024-09-04T17:46:58.936 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.936 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.936 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:58.936 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771774+0000", 2024-09-04T17:46:58.936 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.3680000000000004e-06 2024-09-04T17:46:58.936 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.937 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.937 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:58.937 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771785+0000", 2024-09-04T17:46:58.937 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0387999999999999e-05 2024-09-04T17:46:58.937 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.937 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.937 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.937 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771791+0000", 2024-09-04T17:46:58.937 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.3969999999999999e-06 2024-09-04T17:46:58.937 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.937 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.937 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:58.937 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771848+0000", 2024-09-04T17:46:58.937 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7116e-05 2024-09-04T17:46:58.938 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.938 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.938 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for peered", 2024-09-04T17:46:58.938 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771856+0000", 2024-09-04T17:46:58.938 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.7239999999999992e-06 2024-09-04T17:46:58.938 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.938 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:58.938 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.938 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.938 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.938 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:48843 3.1f 3.4d333f9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-04T17:46:58.938 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:58.771841+0000", 2024-09-04T17:46:58.938 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.05834276, 2024-09-04T17:46:58.938 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.059092815999999999, 2024-09-04T17:46:58.939 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:58.939 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:58.939 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "waiting for peered", 2024-09-04T17:46:58.939 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:58.939 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:58.939 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:58.939 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 48843 2024-09-04T17:46:58.939 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.939 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:58.939 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.939 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:58.939 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771841+0000", 2024-09-04T17:46:58.939 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.939 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.940 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.940 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:58.940 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771841+0000", 2024-09-04T17:46:58.940 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.940 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.940 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.940 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:58.940 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771842+0000", 2024-09-04T17:46:58.940 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.0100000000000003e-07 2024-09-04T17:46:58.940 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.940 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.940 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:58.940 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771847+0000", 2024-09-04T17:46:58.940 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.775e-06 2024-09-04T17:46:58.941 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.941 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.941 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:58.941 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771858+0000", 2024-09-04T17:46:58.941 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1309e-05 2024-09-04T17:46:58.941 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.941 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.941 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.941 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771865+0000", 2024-09-04T17:46:58.941 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.0210000000000002e-06 2024-09-04T17:46:58.941 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.941 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.941 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:58.941 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771909+0000", 2024-09-04T17:46:58.942 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.3364e-05 2024-09-04T17:46:58.942 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.942 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.942 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for peered", 2024-09-04T17:46:58.942 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771916+0000", 2024-09-04T17:46:58.942 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4950000000000002e-06 2024-09-04T17:46:58.942 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.942 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:58.942 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.942 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.942 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.942 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:48847 3.1f 3.4d333f9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-04T17:46:58.942 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:58.771922+0000", 2024-09-04T17:46:58.942 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.058262361999999998, 2024-09-04T17:46:58.942 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.059042424000000003, 2024-09-04T17:46:58.943 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:58.943 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:58.943 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "waiting for peered", 2024-09-04T17:46:58.943 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:58.943 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:58.943 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:58.943 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 48847 2024-09-04T17:46:58.943 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.943 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:58.943 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.943 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:58.943 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771922+0000", 2024-09-04T17:46:58.943 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.943 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.944 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.944 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:58.944 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771922+0000", 2024-09-04T17:46:58.944 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.944 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.944 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.944 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:58.944 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771922+0000", 2024-09-04T17:46:58.944 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.0900000000000001e-07 2024-09-04T17:46:58.944 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.944 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.944 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:58.945 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771926+0000", 2024-09-04T17:46:58.945 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.3129999999999999e-06 2024-09-04T17:46:58.945 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.945 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.945 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:58.945 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771935+0000", 2024-09-04T17:46:58.945 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8789999999999992e-06 2024-09-04T17:46:58.945 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.945 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.945 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.945 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771940+0000", 2024-09-04T17:46:58.945 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0189999999999997e-06 2024-09-04T17:46:58.945 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.946 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.946 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:58.946 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771986+0000", 2024-09-04T17:46:58.946 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6726e-05 2024-09-04T17:46:58.946 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.946 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.946 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for peered", 2024-09-04T17:46:58.946 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.771995+0000", 2024-09-04T17:46:58.946 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.1450000000000008e-06 2024-09-04T17:46:58.946 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.946 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:58.946 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.946 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.946 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.947 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:48745 3.1f 3.4d333f9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-04T17:46:58.947 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:58.770102+0000", 2024-09-04T17:46:58.947 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.060082462000000003, 2024-09-04T17:46:58.947 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.060892671000000002, 2024-09-04T17:46:58.947 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:58.947 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:58.947 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "waiting for peered", 2024-09-04T17:46:58.947 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:58.947 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:58.947 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:58.947 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 48745 2024-09-04T17:46:58.947 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.947 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:58.948 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.948 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:58.948 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770102+0000", 2024-09-04T17:46:58.948 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.948 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.948 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.948 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:58.948 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770102+0000", 2024-09-04T17:46:58.948 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.948 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.948 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.948 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:58.948 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770105+0000", 2024-09-04T17:46:58.949 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.8820000000000001e-06 2024-09-04T17:46:58.949 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.949 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.949 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:58.949 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770114+0000", 2024-09-04T17:46:58.949 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.6670000000000008e-06 2024-09-04T17:46:58.949 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.949 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.949 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:58.949 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770139+0000", 2024-09-04T17:46:58.949 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5017000000000001e-05 2024-09-04T17:46:58.950 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.950 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.950 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.950 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770152+0000", 2024-09-04T17:46:58.950 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2812e-05 2024-09-04T17:46:58.950 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.950 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.950 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:58.950 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770254+0000", 2024-09-04T17:46:58.950 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00010236199999999999 2024-09-04T17:46:58.950 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.950 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.950 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for peered", 2024-09-04T17:46:58.951 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770281+0000", 2024-09-04T17:46:58.951 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.658e-05 2024-09-04T17:46:58.951 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.951 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:58.951 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.951 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.951 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.951 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:48750 3.1f 3.4d333f9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-04T17:46:58.951 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:58.770188+0000", 2024-09-04T17:46:58.951 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.059996368000000001, 2024-09-04T17:46:58.951 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.060836851999999997, 2024-09-04T17:46:58.951 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:58.951 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:58.952 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "waiting for peered", 2024-09-04T17:46:58.952 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:58.952 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:58.952 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:58.952 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 48750 2024-09-04T17:46:58.952 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.952 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:58.952 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.952 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:58.952 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770188+0000", 2024-09-04T17:46:58.952 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.952 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.952 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.953 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:58.953 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770188+0000", 2024-09-04T17:46:58.953 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.953 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.953 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.953 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:58.953 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770189+0000", 2024-09-04T17:46:58.953 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.8400000000000003e-07 2024-09-04T17:46:58.953 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.953 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.953 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:58.953 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770193+0000", 2024-09-04T17:46:58.953 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9200000000000003e-06 2024-09-04T17:46:58.954 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.954 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.954 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:58.954 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770226+0000", 2024-09-04T17:46:58.954 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.2641000000000001e-05 2024-09-04T17:46:58.954 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.954 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.954 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.954 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770236+0000", 2024-09-04T17:46:58.954 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.7159999999999998e-06 2024-09-04T17:46:58.954 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.954 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.954 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:58.954 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770316+0000", 2024-09-04T17:46:58.954 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.0193999999999998e-05 2024-09-04T17:46:58.955 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.955 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.955 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for peered", 2024-09-04T17:46:58.955 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770324+0000", 2024-09-04T17:46:58.955 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.7039999999999994e-06 2024-09-04T17:46:58.955 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.955 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:58.955 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.955 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.955 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.955 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:48753 3.1f 3.4d333f9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-04T17:46:58.955 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:58.770249+0000", 2024-09-04T17:46:58.955 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.059935554000000002, 2024-09-04T17:46:58.955 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.060806078, 2024-09-04T17:46:58.955 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:58.956 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:58.956 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "waiting for peered", 2024-09-04T17:46:58.956 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:58.956 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:58.956 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:58.956 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 48753 2024-09-04T17:46:58.956 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.956 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:58.956 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.956 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:58.956 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770249+0000", 2024-09-04T17:46:58.956 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.956 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.957 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.957 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:58.957 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770249+0000", 2024-09-04T17:46:58.957 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:58.957 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.957 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.957 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:58.957 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770249+0000", 2024-09-04T17:46:58.957 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.9100000000000003e-07 2024-09-04T17:46:58.957 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.957 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.957 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:58.957 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770253+0000", 2024-09-04T17:46:58.958 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.1289999999999998e-06 2024-09-04T17:46:58.958 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.958 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.958 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:58.958 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770278+0000", 2024-09-04T17:46:58.958 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4732e-05 2024-09-04T17:46:58.958 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.958 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.958 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:58.958 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770287+0000", 2024-09-04T17:46:58.958 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.5250000000000005e-06 2024-09-04T17:46:58.958 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.959 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.959 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:58.959 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770350+0000", 2024-09-04T17:46:58.959 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.3600000000000001e-05 2024-09-04T17:46:58.959 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:58.959 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:58.959 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for peered", 2024-09-04T17:46:58.959 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:58.770358+0000", 2024-09-04T17:46:58.959 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4420000000000001e-06 2024-09-04T17:46:58.959 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.959 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:58.959 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.959 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:58.959 INFO:teuthology.orchestra.run.smithi190.stdout: ], 2024-09-04T17:46:58.960 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 28 2024-09-04T17:46:58.960 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:58.961 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-04T17:46:58.962 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:58.961+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:59.062 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:59.059+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:59.134 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:59.135 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:46:59.135 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:46:59.135 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:46:59.135 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:59.149 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-04T17:46:59.163 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:59.159+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:59.264 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:59.259+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:59.271 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:59.271 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:46:59.271 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:46:59.271 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:46:59.271 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.271 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8831 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2453504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:59.271 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:46:59.271 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 46.213623343000002, 2024-09-04T17:46:59.271 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14731111799999999, 2024-09-04T17:46:59.271 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:59.272 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:59.272 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:59.272 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:59.272 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:59.272 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:59.272 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8831 2024-09-04T17:46:59.272 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.272 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:59.272 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.272 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:59.272 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:46:59.272 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.273 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.273 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.273 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:59.273 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:46:59.273 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.273 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.273 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.273 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:59.273 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057811+0000", 2024-09-04T17:46:59.273 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.083e-06 2024-09-04T17:46:59.273 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.273 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.274 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:59.274 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057814+0000", 2024-09-04T17:46:59.274 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.407e-06 2024-09-04T17:46:59.274 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.274 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.274 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:59.274 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057825+0000", 2024-09-04T17:46:59.274 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.17e-05 2024-09-04T17:46:59.274 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.274 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.274 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:59.274 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057836+0000", 2024-09-04T17:46:59.274 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0283e-05 2024-09-04T17:46:59.275 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.275 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.275 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:59.275 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.204762+0000", 2024-09-04T17:46:59.275 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.146926205 2024-09-04T17:46:59.275 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.275 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.275 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:59.275 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.204895+0000", 2024-09-04T17:46:59.275 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013328999999999999 2024-09-04T17:46:59.275 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.275 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.275 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:59.276 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205121+0000", 2024-09-04T17:46:59.276 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00022614999999999999 2024-09-04T17:46:59.276 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.276 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:59.276 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.276 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.276 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.276 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8832 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2490368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:59.276 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:46:59.276 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 46.213585309999999, 2024-09-04T17:46:59.276 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14770012299999999, 2024-09-04T17:46:59.276 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:59.276 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:59.277 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:59.277 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:59.277 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:59.277 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:59.277 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8832 2024-09-04T17:46:59.277 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.277 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:59.277 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.277 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:59.277 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:46:59.277 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.277 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.277 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.278 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:59.278 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:46:59.278 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.278 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.278 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.278 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:59.278 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057849+0000", 2024-09-04T17:46:59.278 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6700000000000003e-07 2024-09-04T17:46:59.278 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.278 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.278 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:59.278 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057852+0000", 2024-09-04T17:46:59.279 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6359999999999998e-06 2024-09-04T17:46:59.279 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.279 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.279 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:59.279 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057860+0000", 2024-09-04T17:46:59.279 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8140000000000007e-06 2024-09-04T17:46:59.279 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.279 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.279 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:59.279 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057868+0000", 2024-09-04T17:46:59.279 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1999999999999997e-06 2024-09-04T17:46:59.279 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.279 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.280 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:59.280 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205208+0000", 2024-09-04T17:46:59.280 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14734067300000001 2024-09-04T17:46:59.280 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.280 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.280 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:59.280 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205347+0000", 2024-09-04T17:46:59.280 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013914900000000001 2024-09-04T17:46:59.280 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.280 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.280 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:59.280 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205548+0000", 2024-09-04T17:46:59.280 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000200984 2024-09-04T17:46:59.280 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.281 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:59.281 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.281 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.281 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.281 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8833 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2527232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:59.281 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:46:59.281 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 46.213553439999998, 2024-09-04T17:46:59.281 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148062572, 2024-09-04T17:46:59.281 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:59.281 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:59.281 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:59.281 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:59.282 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:59.282 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:59.282 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8833 2024-09-04T17:46:59.282 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.282 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:59.282 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.282 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:59.282 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:46:59.282 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.282 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.282 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.282 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:59.282 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:46:59.282 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.283 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.283 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.283 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:59.283 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057881+0000", 2024-09-04T17:46:59.283 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4300000000000002e-07 2024-09-04T17:46:59.283 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.283 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.283 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:59.283 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057893+0000", 2024-09-04T17:46:59.283 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1973999999999999e-05 2024-09-04T17:46:59.283 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.283 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.283 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:59.284 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057906+0000", 2024-09-04T17:46:59.284 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2716e-05 2024-09-04T17:46:59.284 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.284 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.284 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:59.284 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057913+0000", 2024-09-04T17:46:59.284 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3930000000000002e-06 2024-09-04T17:46:59.284 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.284 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.284 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:59.284 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205621+0000", 2024-09-04T17:46:59.284 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14770788800000001 2024-09-04T17:46:59.284 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.285 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.285 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:59.285 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205740+0000", 2024-09-04T17:46:59.285 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000118936 2024-09-04T17:46:59.285 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.285 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.285 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:59.285 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205943+0000", 2024-09-04T17:46:59.285 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020292199999999999 2024-09-04T17:46:59.285 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.285 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:59.285 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.285 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.286 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.286 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8834 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2564096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:59.286 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:46:59.286 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 46.213510161000002, 2024-09-04T17:46:59.286 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14846753200000001, 2024-09-04T17:46:59.286 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:59.286 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:59.286 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:59.286 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:59.286 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:59.286 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:59.286 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8834 2024-09-04T17:46:59.286 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.287 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:59.287 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.287 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:59.287 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:46:59.287 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.287 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.287 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.287 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:59.287 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:46:59.287 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.287 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.287 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.287 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:59.288 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057925+0000", 2024-09-04T17:46:59.288 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0270000000000001e-06 2024-09-04T17:46:59.288 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.288 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.288 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:59.288 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057936+0000", 2024-09-04T17:46:59.288 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.112e-05 2024-09-04T17:46:59.288 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.288 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.288 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:59.288 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057946+0000", 2024-09-04T17:46:59.288 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.009e-05 2024-09-04T17:46:59.289 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.289 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.289 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:59.289 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057953+0000", 2024-09-04T17:46:59.289 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.6229999999999997e-06 2024-09-04T17:46:59.289 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.289 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.289 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:59.289 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206027+0000", 2024-09-04T17:46:59.289 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148073591 2024-09-04T17:46:59.289 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.289 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.289 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:59.290 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206161+0000", 2024-09-04T17:46:59.290 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013411599999999999 2024-09-04T17:46:59.290 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.290 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.290 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:59.290 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206391+0000", 2024-09-04T17:46:59.290 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000229965 2024-09-04T17:46:59.290 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.290 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:59.290 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.290 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.290 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.290 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8835 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2600960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:59.290 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:46:59.291 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 46.213470510999997, 2024-09-04T17:46:59.291 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14882941599999999, 2024-09-04T17:46:59.291 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:59.291 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:59.291 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:59.291 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:59.291 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:59.291 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:59.291 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8835 2024-09-04T17:46:59.291 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.291 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:59.291 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.291 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:59.292 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:46:59.292 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.292 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.292 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.292 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:59.292 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:46:59.292 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.292 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.292 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.292 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:59.292 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058544+0000", 2024-09-04T17:46:59.292 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00058061899999999999 2024-09-04T17:46:59.292 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.293 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.293 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:59.293 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058550+0000", 2024-09-04T17:46:59.293 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.1129999999999997e-06 2024-09-04T17:46:59.293 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.293 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.293 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:59.293 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058566+0000", 2024-09-04T17:46:59.293 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5820000000000001e-05 2024-09-04T17:46:59.293 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.293 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.293 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:59.293 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058578+0000", 2024-09-04T17:46:59.294 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2537e-05 2024-09-04T17:46:59.294 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.294 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.294 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:59.294 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206470+0000", 2024-09-04T17:46:59.294 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14789153799999999 2024-09-04T17:46:59.294 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.294 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.294 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:59.294 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206589+0000", 2024-09-04T17:46:59.294 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000118837 2024-09-04T17:46:59.294 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.294 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.294 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:59.295 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206793+0000", 2024-09-04T17:46:59.295 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000203952 2024-09-04T17:46:59.295 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.295 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:59.295 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.295 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.295 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.295 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8836 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2654208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:59.295 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:46:59.295 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 46.212837823000001, 2024-09-04T17:46:59.295 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14862433999999999, 2024-09-04T17:46:59.295 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:59.295 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:59.295 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:59.296 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:59.296 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:59.296 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:59.296 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8836 2024-09-04T17:46:59.296 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.296 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:59.296 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.296 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:59.296 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:46:59.296 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.296 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.296 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.296 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:59.297 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:46:59.297 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.297 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.297 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.297 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:59.297 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062663+0000", 2024-09-04T17:46:59.297 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0040670619999999998 2024-09-04T17:46:59.297 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.297 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.297 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:59.297 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062668+0000", 2024-09-04T17:46:59.297 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4199999999999998e-06 2024-09-04T17:46:59.297 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.298 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.298 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:59.298 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062689+0000", 2024-09-04T17:46:59.298 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0356999999999998e-05 2024-09-04T17:46:59.298 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.298 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.298 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:59.298 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062726+0000", 2024-09-04T17:46:59.298 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7302999999999998e-05 2024-09-04T17:46:59.298 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.298 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.298 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:59.298 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206875+0000", 2024-09-04T17:46:59.299 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14414883100000001 2024-09-04T17:46:59.299 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.299 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.299 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:59.299 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.207016+0000", 2024-09-04T17:46:59.299 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000140822 2024-09-04T17:46:59.299 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.299 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.299 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:59.299 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.207220+0000", 2024-09-04T17:46:59.299 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020454499999999999 2024-09-04T17:46:59.299 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.299 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:59.300 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.300 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.300 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.300 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8844 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2928640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:59.300 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:46:59.300 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 46.208473740000002, 2024-09-04T17:46:59.300 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14752153100000001, 2024-09-04T17:46:59.300 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:59.300 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:59.300 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:59.300 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:59.300 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:59.300 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:59.301 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8844 2024-09-04T17:46:59.301 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.301 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:59.301 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.301 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:59.301 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:46:59.301 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.301 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.301 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.301 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:59.301 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:46:59.301 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.301 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.302 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.302 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:59.302 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062961+0000", 2024-09-04T17:46:59.302 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6700000000000003e-07 2024-09-04T17:46:59.302 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.302 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.302 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:59.302 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062973+0000", 2024-09-04T17:46:59.302 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2109999999999999e-05 2024-09-04T17:46:59.302 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.302 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.302 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:59.303 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062982+0000", 2024-09-04T17:46:59.303 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.6900000000000004e-06 2024-09-04T17:46:59.303 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.303 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.303 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:59.303 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062990+0000", 2024-09-04T17:46:59.303 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1899999999999998e-06 2024-09-04T17:46:59.303 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.303 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.303 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:59.303 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210090+0000", 2024-09-04T17:46:59.303 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14710026300000001 2024-09-04T17:46:59.303 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.304 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.304 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:59.304 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210225+0000", 2024-09-04T17:46:59.304 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013532400000000001 2024-09-04T17:46:59.304 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.304 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.304 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:59.304 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210481+0000", 2024-09-04T17:46:59.304 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00025628700000000001 2024-09-04T17:46:59.304 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.304 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:59.304 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.304 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.305 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.305 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8845 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2965504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:59.305 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:46:59.305 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 46.208434017000002, 2024-09-04T17:46:59.305 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147916819, 2024-09-04T17:46:59.305 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:59.305 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:59.305 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:59.305 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:59.305 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:59.305 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:59.305 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8845 2024-09-04T17:46:59.305 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.306 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:59.306 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.306 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:59.306 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:46:59.306 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.306 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.306 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.306 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:59.306 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:46:59.306 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.306 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.306 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.306 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:59.307 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:46:59.307 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.0399999999999995e-07 2024-09-04T17:46:59.307 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.307 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.307 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:59.307 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063003+0000", 2024-09-04T17:46:59.307 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6529999999999998e-06 2024-09-04T17:46:59.307 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.307 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.307 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:59.307 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063012+0000", 2024-09-04T17:46:59.307 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9500000000000007e-06 2024-09-04T17:46:59.307 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.308 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.308 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:59.308 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063027+0000", 2024-09-04T17:46:59.308 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5443000000000001e-05 2024-09-04T17:46:59.308 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.308 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.308 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:59.308 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210571+0000", 2024-09-04T17:46:59.308 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14754353100000001 2024-09-04T17:46:59.308 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.308 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.308 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:59.308 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210714+0000", 2024-09-04T17:46:59.309 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014295599999999999 2024-09-04T17:46:59.309 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.309 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.309 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:59.309 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210916+0000", 2024-09-04T17:46:59.309 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020258200000000001 2024-09-04T17:46:59.309 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.309 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:59.309 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.309 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.309 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.309 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8846 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3002368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:59.309 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:46:59.310 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 46.208396479999998, 2024-09-04T17:46:59.310 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14828011599999999, 2024-09-04T17:46:59.310 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:59.310 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:59.310 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:59.310 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:59.310 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:59.310 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:59.310 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8846 2024-09-04T17:46:59.310 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.310 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:59.310 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.310 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:59.311 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:46:59.311 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.311 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.311 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.311 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:59.311 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:46:59.311 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.311 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.311 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.311 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:59.311 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063038+0000", 2024-09-04T17:46:59.311 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.23e-07 2024-09-04T17:46:59.311 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.311 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.312 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:59.312 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063051+0000", 2024-09-04T17:46:59.312 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2757e-05 2024-09-04T17:46:59.312 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.312 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.312 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:59.312 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063070+0000", 2024-09-04T17:46:59.312 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9663000000000001e-05 2024-09-04T17:46:59.312 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.312 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.312 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:59.312 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063079+0000", 2024-09-04T17:46:59.313 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.0929999999999992e-06 2024-09-04T17:46:59.313 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.313 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.313 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:59.313 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210989+0000", 2024-09-04T17:46:59.313 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14790984500000001 2024-09-04T17:46:59.313 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.313 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.313 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:59.313 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211109+0000", 2024-09-04T17:46:59.313 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000120056 2024-09-04T17:46:59.313 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.313 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.314 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:59.314 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211317+0000", 2024-09-04T17:46:59.314 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020797900000000001 2024-09-04T17:46:59.314 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.314 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:59.314 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.314 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.314 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.314 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8847 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3039232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:59.314 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:46:59.314 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 46.208322047999999, 2024-09-04T17:46:59.314 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14859408900000001, 2024-09-04T17:46:59.314 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:59.314 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:59.315 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:59.315 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:59.315 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:59.315 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:59.315 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8847 2024-09-04T17:46:59.315 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.315 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:59.315 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.315 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:59.315 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:46:59.315 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.315 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.315 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.316 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:59.316 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:46:59.316 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.316 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.316 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.316 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:59.316 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063113+0000", 2024-09-04T17:46:59.316 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.204e-06 2024-09-04T17:46:59.316 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.316 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.316 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:59.316 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063116+0000", 2024-09-04T17:46:59.316 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.3330000000000001e-06 2024-09-04T17:46:59.317 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.317 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.317 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:59.317 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063126+0000", 2024-09-04T17:46:59.317 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0166999999999999e-05 2024-09-04T17:46:59.317 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.317 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.317 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:59.317 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063134+0000", 2024-09-04T17:46:59.317 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4329999999999998e-06 2024-09-04T17:46:59.317 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.317 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.317 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:59.318 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211392+0000", 2024-09-04T17:46:59.318 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148258109 2024-09-04T17:46:59.318 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.318 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.318 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:59.318 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211506+0000", 2024-09-04T17:46:59.318 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000113813 2024-09-04T17:46:59.318 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.318 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.318 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:59.318 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211706+0000", 2024-09-04T17:46:59.318 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020002999999999999 2024-09-04T17:46:59.318 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.319 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:59.319 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.319 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.319 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.319 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8848 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:59.319 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:46:59.319 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 46.208289710999999, 2024-09-04T17:46:59.319 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14907967599999999, 2024-09-04T17:46:59.319 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:59.319 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:59.319 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:59.319 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:59.319 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:59.320 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:59.320 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8848 2024-09-04T17:46:59.320 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.320 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:59.320 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.320 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:59.320 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:46:59.320 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.320 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.320 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.320 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:59.320 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:46:59.321 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.321 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.321 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.321 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:59.321 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070589+0000", 2024-09-04T17:46:59.321 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.007445386 2024-09-04T17:46:59.321 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.321 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.321 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:59.321 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070600+0000", 2024-09-04T17:46:59.321 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0346999999999999e-05 2024-09-04T17:46:59.321 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.321 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.322 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:59.322 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070622+0000", 2024-09-04T17:46:59.322 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2289999999999998e-05 2024-09-04T17:46:59.322 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.322 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.322 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:59.322 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070636+0000", 2024-09-04T17:46:59.322 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4093e-05 2024-09-04T17:46:59.322 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.322 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.322 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:59.322 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211815+0000", 2024-09-04T17:46:59.322 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14117874699999999 2024-09-04T17:46:59.323 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.323 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.323 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:59.323 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211983+0000", 2024-09-04T17:46:59.323 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00016852800000000001 2024-09-04T17:46:59.323 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.323 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.323 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:59.323 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.212224+0000", 2024-09-04T17:46:59.323 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024028499999999999 2024-09-04T17:46:59.323 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.323 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:59.323 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.323 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.324 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.324 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8862 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2019328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:59.324 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:46:59.324 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 46.200240243000003, 2024-09-04T17:46:59.324 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14755236399999999, 2024-09-04T17:46:59.324 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:59.324 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:59.324 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:59.324 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:59.324 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:59.324 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:59.324 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8862 2024-09-04T17:46:59.324 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.325 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:59.325 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.325 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:59.325 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:46:59.325 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.325 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.325 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.325 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:59.325 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:46:59.325 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.325 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.325 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.325 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:59.326 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071194+0000", 2024-09-04T17:46:59.326 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.5000000000000002e-07 2024-09-04T17:46:59.326 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.326 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.326 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:59.326 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071197+0000", 2024-09-04T17:46:59.326 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5900000000000002e-06 2024-09-04T17:46:59.326 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.326 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.326 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:59.326 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071206+0000", 2024-09-04T17:46:59.326 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9700000000000005e-06 2024-09-04T17:46:59.326 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.327 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.327 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:59.327 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071213+0000", 2024-09-04T17:46:59.327 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.9070000000000001e-06 2024-09-04T17:46:59.327 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.327 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.327 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:59.327 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218396+0000", 2024-09-04T17:46:59.327 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147183273 2024-09-04T17:46:59.327 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.327 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.327 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:59.327 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218538+0000", 2024-09-04T17:46:59.327 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014194199999999999 2024-09-04T17:46:59.328 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.328 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.328 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:59.328 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218746+0000", 2024-09-04T17:46:59.328 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000208032 2024-09-04T17:46:59.328 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.328 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:59.328 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.328 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.328 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.328 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8863 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2056192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:59.328 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:46:59.329 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 46.200211932999999, 2024-09-04T17:46:59.329 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14792281099999999, 2024-09-04T17:46:59.329 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:59.329 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:59.329 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:59.329 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:59.329 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:59.329 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:59.329 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8863 2024-09-04T17:46:59.329 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.329 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:59.329 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.329 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:59.330 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:46:59.330 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.330 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.330 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.330 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:59.330 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:46:59.330 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.330 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.330 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.330 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:59.330 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:46:59.330 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1299999999999999e-07 2024-09-04T17:46:59.330 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.330 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.331 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:59.331 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071225+0000", 2024-09-04T17:46:59.331 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6199999999999999e-06 2024-09-04T17:46:59.331 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.331 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.331 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:59.331 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071234+0000", 2024-09-04T17:46:59.331 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8769999999999999e-06 2024-09-04T17:46:59.331 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.331 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.331 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:59.331 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071251+0000", 2024-09-04T17:46:59.331 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.738e-05 2024-09-04T17:46:59.332 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.332 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.332 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:59.332 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218798+0000", 2024-09-04T17:46:59.332 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14754645 2024-09-04T17:46:59.332 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.332 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.332 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:59.332 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218926+0000", 2024-09-04T17:46:59.332 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000128446 2024-09-04T17:46:59.332 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.332 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.332 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:59.333 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219145+0000", 2024-09-04T17:46:59.333 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021832500000000001 2024-09-04T17:46:59.333 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.333 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:59.333 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.333 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.333 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.333 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8864 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2093056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:59.333 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:46:59.333 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 46.200170356999998, 2024-09-04T17:46:59.333 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148277309, 2024-09-04T17:46:59.333 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:59.333 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:59.334 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:59.334 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:59.334 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:59.334 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:59.334 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8864 2024-09-04T17:46:59.334 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.334 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:59.334 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.334 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:59.334 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:46:59.334 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.334 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.334 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.335 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:59.335 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:46:59.335 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.335 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.335 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.335 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:59.335 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071264+0000", 2024-09-04T17:46:59.335 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3e-07 2024-09-04T17:46:59.335 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.335 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.335 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:59.335 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071267+0000", 2024-09-04T17:46:59.335 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0139999999999999e-06 2024-09-04T17:46:59.336 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.336 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.336 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:59.336 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071276+0000", 2024-09-04T17:46:59.336 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1689999999999998e-06 2024-09-04T17:46:59.336 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.336 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.336 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:59.336 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071283+0000", 2024-09-04T17:46:59.336 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2269999999999996e-06 2024-09-04T17:46:59.336 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.336 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.336 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:59.337 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219195+0000", 2024-09-04T17:46:59.337 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147911868 2024-09-04T17:46:59.337 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.337 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.337 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:59.337 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219324+0000", 2024-09-04T17:46:59.337 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00012835900000000001 2024-09-04T17:46:59.337 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.337 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.337 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:59.337 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219541+0000", 2024-09-04T17:46:59.337 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000216942 2024-09-04T17:46:59.337 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.338 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:59.338 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.338 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.338 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.338 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8865 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2109440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:59.338 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:46:59.338 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 46.200140896999997, 2024-09-04T17:46:59.338 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14865747300000001, 2024-09-04T17:46:59.338 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:59.338 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:59.338 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:59.338 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:59.339 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:59.339 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:59.339 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8865 2024-09-04T17:46:59.339 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.339 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:59.339 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.339 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:59.339 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:46:59.339 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.339 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.339 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.339 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:59.339 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:46:59.339 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.340 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.340 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.340 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:59.340 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:46:59.340 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7999999999999995e-07 2024-09-04T17:46:59.340 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.340 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.340 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:59.340 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071296+0000", 2024-09-04T17:46:59.340 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6900000000000001e-06 2024-09-04T17:46:59.340 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.340 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.341 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:59.341 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071305+0000", 2024-09-04T17:46:59.341 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.5499999999999995e-06 2024-09-04T17:46:59.341 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.341 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.341 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:59.341 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071312+0000", 2024-09-04T17:46:59.341 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1559999999999998e-06 2024-09-04T17:46:59.341 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.341 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.341 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:59.341 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219591+0000", 2024-09-04T17:46:59.341 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14827944900000001 2024-09-04T17:46:59.342 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.342 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.342 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:59.342 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219731+0000", 2024-09-04T17:46:59.342 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013931600000000001 2024-09-04T17:46:59.342 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.342 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.342 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:59.342 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219950+0000", 2024-09-04T17:46:59.342 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021963199999999999 2024-09-04T17:46:59.342 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.342 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:59.343 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.343 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.343 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.343 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8866 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2146304~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:59.343 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:46:59.343 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 46.200112664000002, 2024-09-04T17:46:59.343 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14906117899999999, 2024-09-04T17:46:59.343 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:59.343 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:59.343 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:59.343 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:59.343 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:59.344 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:59.344 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8866 2024-09-04T17:46:59.344 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.344 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:59.344 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.344 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:59.344 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:46:59.344 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.344 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.344 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.344 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:59.344 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:46:59.344 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.345 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.345 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.345 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:59.345 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071322+0000", 2024-09-04T17:46:59.345 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7999999999999995e-07 2024-09-04T17:46:59.345 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.345 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.345 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:59.345 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071338+0000", 2024-09-04T17:46:59.345 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5872999999999999e-05 2024-09-04T17:46:59.345 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.345 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.346 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:59.346 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071347+0000", 2024-09-04T17:46:59.346 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4669999999999993e-06 2024-09-04T17:46:59.346 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.346 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.346 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:59.346 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071354+0000", 2024-09-04T17:46:59.346 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2099999999999996e-06 2024-09-04T17:46:59.346 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.346 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.346 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:59.346 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220008+0000", 2024-09-04T17:46:59.346 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148653642 2024-09-04T17:46:59.347 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.347 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.347 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:59.347 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220155+0000", 2024-09-04T17:46:59.347 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014726600000000001 2024-09-04T17:46:59.347 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.347 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.347 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:59.347 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220382+0000", 2024-09-04T17:46:59.347 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00022704100000000001 2024-09-04T17:46:59.347 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.347 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:59.348 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.348 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.348 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.348 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8867 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2183168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:59.348 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:46:59.348 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 46.200069311, 2024-09-04T17:46:59.348 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.149445831, 2024-09-04T17:46:59.348 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:59.348 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:59.348 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:59.348 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:59.348 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:59.348 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:59.349 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8867 2024-09-04T17:46:59.349 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.349 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:59.349 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.349 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:59.349 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:46:59.349 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.349 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.349 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.349 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:59.349 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:46:59.349 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.349 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.350 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.350 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:59.350 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078653+0000", 2024-09-04T17:46:59.350 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0072890430000000003 2024-09-04T17:46:59.350 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.350 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.350 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:59.350 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078664+0000", 2024-09-04T17:46:59.350 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0307e-05 2024-09-04T17:46:59.350 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.350 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.350 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:59.350 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078690+0000", 2024-09-04T17:46:59.351 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6780000000000001e-05 2024-09-04T17:46:59.351 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.351 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.351 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:59.351 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078706+0000", 2024-09-04T17:46:59.351 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5676999999999999e-05 2024-09-04T17:46:59.351 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.351 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.351 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:59.351 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220467+0000", 2024-09-04T17:46:59.351 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14176103500000001 2024-09-04T17:46:59.351 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.351 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.352 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:59.352 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220595+0000", 2024-09-04T17:46:59.352 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000127427 2024-09-04T17:46:59.352 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.352 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.352 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:59.352 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220810+0000", 2024-09-04T17:46:59.352 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021556200000000001 2024-09-04T17:46:59.352 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.352 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:59.352 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.352 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.352 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.353 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8880 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2658304~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:59.353 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.079158+0000", 2024-09-04T17:46:59.353 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 46.192276032000002, 2024-09-04T17:46:59.353 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14718968499999999, 2024-09-04T17:46:59.353 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:59.353 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:59.353 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:59.353 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:59.353 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:59.353 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:59.353 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8880 2024-09-04T17:46:59.353 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.353 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:59.354 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.354 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:59.354 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079158+0000", 2024-09-04T17:46:59.354 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.354 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.354 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.354 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:59.354 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079158+0000", 2024-09-04T17:46:59.354 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.354 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.354 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.354 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:59.354 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079158+0000", 2024-09-04T17:46:59.355 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3e-07 2024-09-04T17:46:59.355 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.355 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.355 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:59.355 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079161+0000", 2024-09-04T17:46:59.355 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.57e-06 2024-09-04T17:46:59.355 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.355 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.355 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:59.355 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079170+0000", 2024-09-04T17:46:59.355 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.0170000000000002e-06 2024-09-04T17:46:59.355 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.355 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.356 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:59.356 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079177+0000", 2024-09-04T17:46:59.356 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.0530000000000001e-06 2024-09-04T17:46:59.356 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.356 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.356 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:59.356 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.225946+0000", 2024-09-04T17:46:59.356 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14676936500000001 2024-09-04T17:46:59.356 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.356 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.356 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:59.356 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226098+0000", 2024-09-04T17:46:59.356 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000151805 2024-09-04T17:46:59.357 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.357 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.357 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:59.357 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226347+0000", 2024-09-04T17:46:59.357 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024914499999999999 2024-09-04T17:46:59.357 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.357 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:59.357 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.357 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.357 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.357 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8881 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2695168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:59.357 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:46:59.357 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 46.192247242000001, 2024-09-04T17:46:59.358 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147543547, 2024-09-04T17:46:59.358 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:59.358 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:59.358 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:59.358 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:59.358 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:59.358 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:59.358 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8881 2024-09-04T17:46:59.358 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.358 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:59.358 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.358 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:59.358 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:46:59.358 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.359 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.359 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.359 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:59.359 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:46:59.359 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.359 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.359 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.359 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:59.359 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079187+0000", 2024-09-04T17:46:59.359 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7700000000000004e-07 2024-09-04T17:46:59.359 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.359 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.359 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:59.360 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079198+0000", 2024-09-04T17:46:59.360 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0849999999999999e-05 2024-09-04T17:46:59.360 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.360 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.360 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:59.360 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079207+0000", 2024-09-04T17:46:59.360 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5599999999999999e-06 2024-09-04T17:46:59.360 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.360 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.360 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:59.360 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079224+0000", 2024-09-04T17:46:59.360 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6387000000000001e-05 2024-09-04T17:46:59.360 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.361 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.361 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:59.361 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226414+0000", 2024-09-04T17:46:59.361 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14718975000000001 2024-09-04T17:46:59.361 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.361 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.361 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:59.361 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226534+0000", 2024-09-04T17:46:59.361 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000120288 2024-09-04T17:46:59.361 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.361 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.361 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:59.361 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226730+0000", 2024-09-04T17:46:59.361 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000196035 2024-09-04T17:46:59.362 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.362 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:59.362 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.362 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.362 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.362 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8882 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2732032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:46:59.362 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:46:59.362 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 46.192199027999997, 2024-09-04T17:46:59.362 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14795022399999999, 2024-09-04T17:46:59.362 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:59.362 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:59.362 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:59.362 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:59.363 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:59.363 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:59.363 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8882 2024-09-04T17:46:59.363 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.363 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:59.363 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.363 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:59.363 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:46:59.363 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.363 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.363 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.363 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:59.363 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:46:59.364 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.364 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.364 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.364 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:59.364 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086588+0000", 2024-09-04T17:46:59.364 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0073530959999999999 2024-09-04T17:46:59.364 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.364 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.364 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:59.364 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086598+0000", 2024-09-04T17:46:59.364 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0122999999999999e-05 2024-09-04T17:46:59.364 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.364 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.365 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:59.365 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086626+0000", 2024-09-04T17:46:59.365 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.7914000000000001e-05 2024-09-04T17:46:59.365 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.365 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.365 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:59.365 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086641+0000", 2024-09-04T17:46:59.365 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5270000000000001e-05 2024-09-04T17:46:59.365 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.365 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.365 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:59.365 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226801+0000", 2024-09-04T17:46:59.365 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.140160321 2024-09-04T17:46:59.366 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.366 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.366 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:59.366 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226951+0000", 2024-09-04T17:46:59.366 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014965200000000001 2024-09-04T17:46:59.366 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.366 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.366 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:59.366 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.227185+0000", 2024-09-04T17:46:59.366 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00023384799999999999 2024-09-04T17:46:59.366 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.366 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:59.366 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.367 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.367 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:59.367 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:59.368 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-04T17:46:59.369 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:46:59.369+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:46:59.468 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:59.469+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:59.541 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:59.542 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:46:59.542 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:46:59.542 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:59.555 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-04T17:46:59.569 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:46:59.567+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:46:59.671 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:46:59.671+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:46:59.724 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:59.725 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:46:59.725 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:46:59.725 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:46:59.725 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:59.739 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-04T17:46:59.771 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:46:59.767+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:46:59.786 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-04T17:46:59.861 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:46:59.861 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:46:59.861 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:46:59.861 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:46:59.861 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.862 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:1430 3.f 3:f58475cb:::rbd_data.11346251f094.0000000000000074:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-04T17:46:59.862 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:02.778704+0000", 2024-09-04T17:46:59.862 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 57.083035269, 2024-09-04T17:46:59.862 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.058567801000000003, 2024-09-04T17:46:59.862 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:59.862 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:59.862 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:59.862 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:59.862 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:59.862 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:59.862 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 1430 2024-09-04T17:46:59.862 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.862 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:59.863 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.863 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:59.863 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.778704+0000", 2024-09-04T17:46:59.863 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.863 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.863 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.863 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:59.863 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.778704+0000", 2024-09-04T17:46:59.863 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.864 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.864 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.864 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:59.864 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.778707+0000", 2024-09-04T17:46:59.864 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.8329999999999998e-06 2024-09-04T17:46:59.864 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.864 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.864 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:59.864 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.778718+0000", 2024-09-04T17:46:59.864 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.152e-05 2024-09-04T17:46:59.864 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.864 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.865 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:59.865 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.778750+0000", 2024-09-04T17:46:59.865 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.1696999999999998e-05 2024-09-04T17:46:59.865 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.865 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.865 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:59.865 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.778765+0000", 2024-09-04T17:46:59.865 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4800000000000001e-05 2024-09-04T17:46:59.865 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.865 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.865 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:59.865 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.778894+0000", 2024-09-04T17:46:59.865 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00012928500000000001 2024-09-04T17:46:59.866 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.866 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.866 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:59.866 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.779030+0000", 2024-09-04T17:46:59.866 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013566899999999999 2024-09-04T17:46:59.866 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.866 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.866 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 4", 2024-09-04T17:46:59.866 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.788794+0000", 2024-09-04T17:46:59.866 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.009764478 2024-09-04T17:46:59.866 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.866 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.866 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:59.866 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.836969+0000", 2024-09-04T17:46:59.867 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.048174626999999998 2024-09-04T17:46:59.867 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.867 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.867 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:59.867 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.837090+0000", 2024-09-04T17:46:59.867 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000120627 2024-09-04T17:46:59.867 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.867 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.867 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:59.867 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.837195+0000", 2024-09-04T17:46:59.867 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000105206 2024-09-04T17:46:59.867 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.867 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.868 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:59.868 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.837272+0000", 2024-09-04T17:46:59.868 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.7058999999999995e-05 2024-09-04T17:46:59.868 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.868 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:59.868 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.868 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.868 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.868 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14369 3.d 3:b763de10:::rbd_data.11345bd5898b.0000000000000002:a [read 77824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:59.868 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.182808+0000", 2024-09-04T17:46:59.868 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 41.678931300999999, 2024-09-04T17:46:59.868 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042057997999999999, 2024-09-04T17:46:59.868 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:59.868 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:59.869 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:59.869 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:59.869 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:59.869 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:59.869 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14369 2024-09-04T17:46:59.869 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.869 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:59.869 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.869 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:59.869 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182808+0000", 2024-09-04T17:46:59.869 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.869 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.869 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.870 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:59.870 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182808+0000", 2024-09-04T17:46:59.870 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.870 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.870 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.870 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:59.870 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182809+0000", 2024-09-04T17:46:59.870 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.063e-06 2024-09-04T17:46:59.870 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.870 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.870 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:59.870 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182814+0000", 2024-09-04T17:46:59.870 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1900000000000003e-06 2024-09-04T17:46:59.871 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.871 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.871 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:59.871 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182832+0000", 2024-09-04T17:46:59.871 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7791999999999998e-05 2024-09-04T17:46:59.871 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.871 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.871 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:59.871 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182849+0000", 2024-09-04T17:46:59.872 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7265000000000002e-05 2024-09-04T17:46:59.872 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.872 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.872 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:59.872 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.224431+0000", 2024-09-04T17:46:59.872 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.041581448 2024-09-04T17:46:59.872 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.872 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.872 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:59.872 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.224615+0000", 2024-09-04T17:46:59.872 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00018389200000000001 2024-09-04T17:46:59.872 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.873 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.873 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:59.873 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.224866+0000", 2024-09-04T17:46:59.873 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00025134799999999998 2024-09-04T17:46:59.873 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.873 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:59.873 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.873 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.873 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.873 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14370 3.d 3:b763de10:::rbd_data.11345bd5898b.0000000000000002:a [read 114688~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:59.873 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.182867+0000", 2024-09-04T17:46:59.873 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 41.678871999000002, 2024-09-04T17:46:59.873 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042368301999999997, 2024-09-04T17:46:59.874 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:59.874 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:59.874 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:59.874 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:59.874 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:59.874 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:59.874 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14370 2024-09-04T17:46:59.874 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.874 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:59.874 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.874 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:59.874 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182867+0000", 2024-09-04T17:46:59.874 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.875 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.875 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.875 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:59.875 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182867+0000", 2024-09-04T17:46:59.875 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.875 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.875 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.875 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:59.875 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182868+0000", 2024-09-04T17:46:59.875 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.006e-06 2024-09-04T17:46:59.875 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.875 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.875 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:59.875 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182875+0000", 2024-09-04T17:46:59.876 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.3990000000000001e-06 2024-09-04T17:46:59.876 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.876 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.876 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:59.876 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182892+0000", 2024-09-04T17:46:59.876 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7096e-05 2024-09-04T17:46:59.876 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.876 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.876 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:59.876 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182905+0000", 2024-09-04T17:46:59.876 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3258000000000001e-05 2024-09-04T17:46:59.876 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.876 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.877 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:59.877 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.224926+0000", 2024-09-04T17:46:59.877 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042020891999999997 2024-09-04T17:46:59.877 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.877 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.877 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:59.877 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.225064+0000", 2024-09-04T17:46:59.877 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000138307 2024-09-04T17:46:59.877 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.877 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.877 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:59.877 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.225236+0000", 2024-09-04T17:46:59.878 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000171344 2024-09-04T17:46:59.878 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.878 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:59.878 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.878 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.878 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.878 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14377 3.d 3:b763de10:::rbd_data.11345bd5898b.0000000000000002:a [read 225280~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:59.878 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.184021+0000", 2024-09-04T17:46:59.878 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 41.677718392999999, 2024-09-04T17:46:59.878 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042298810999999999, 2024-09-04T17:46:59.878 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:59.878 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:59.878 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:59.879 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:59.879 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:59.879 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:59.879 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14377 2024-09-04T17:46:59.879 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.879 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:59.879 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.879 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:59.879 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184021+0000", 2024-09-04T17:46:59.879 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.879 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.879 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.879 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:59.880 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184021+0000", 2024-09-04T17:46:59.880 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.880 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.880 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.880 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:59.880 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184022+0000", 2024-09-04T17:46:59.880 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5099999999999999e-06 2024-09-04T17:46:59.880 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.880 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.880 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:59.880 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184028+0000", 2024-09-04T17:46:59.880 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.852e-06 2024-09-04T17:46:59.881 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.881 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.881 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:59.881 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184046+0000", 2024-09-04T17:46:59.881 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7822e-05 2024-09-04T17:46:59.881 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.881 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.881 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:59.881 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184059+0000", 2024-09-04T17:46:59.881 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3372e-05 2024-09-04T17:46:59.881 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.881 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.881 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:59.882 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.226009+0000", 2024-09-04T17:46:59.882 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.041949187999999998 2024-09-04T17:46:59.882 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.882 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.882 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:59.882 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.226142+0000", 2024-09-04T17:46:59.882 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013381299999999999 2024-09-04T17:46:59.882 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.882 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.882 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:59.882 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.226320+0000", 2024-09-04T17:46:59.882 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00017725400000000001 2024-09-04T17:46:59.882 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.883 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:59.883 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.883 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.883 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.883 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14381 3.d 3:b763de10:::rbd_data.11345bd5898b.0000000000000002:a [read 552960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:59.883 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.184135+0000", 2024-09-04T17:46:59.883 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 41.677604340999999, 2024-09-04T17:46:59.883 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042553818, 2024-09-04T17:46:59.883 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:59.883 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:59.883 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:59.883 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:59.884 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:59.884 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:59.884 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14381 2024-09-04T17:46:59.884 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.884 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:59.884 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.884 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:59.884 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184135+0000", 2024-09-04T17:46:59.884 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.884 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.884 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.884 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:59.884 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184135+0000", 2024-09-04T17:46:59.885 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.885 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.885 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.885 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:59.885 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184136+0000", 2024-09-04T17:46:59.885 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1200000000000001e-06 2024-09-04T17:46:59.885 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.885 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.885 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:59.885 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184142+0000", 2024-09-04T17:46:59.885 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4840000000000003e-06 2024-09-04T17:46:59.885 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.886 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.886 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:59.886 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184157+0000", 2024-09-04T17:46:59.886 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5727999999999999e-05 2024-09-04T17:46:59.886 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.886 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.886 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:59.886 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184194+0000", 2024-09-04T17:46:59.886 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7049999999999999e-05 2024-09-04T17:46:59.886 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.886 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.886 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:59.886 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.226376+0000", 2024-09-04T17:46:59.887 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042181933999999997 2024-09-04T17:46:59.887 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.887 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.887 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:59.887 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.226522+0000", 2024-09-04T17:46:59.887 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014602899999999999 2024-09-04T17:46:59.887 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.887 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.887 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:59.887 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.226689+0000", 2024-09-04T17:46:59.887 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00016647299999999999 2024-09-04T17:46:59.887 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.888 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:59.888 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.888 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.888 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.888 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14383 3.d 3:b763de10:::rbd_data.11345bd5898b.0000000000000002:a [read 589824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:59.888 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.184302+0000", 2024-09-04T17:46:59.888 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 41.677437341999997, 2024-09-04T17:46:59.888 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042760362000000003, 2024-09-04T17:46:59.888 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:59.888 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:59.888 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:59.888 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:59.888 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:59.889 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:59.889 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14383 2024-09-04T17:46:59.889 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.889 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:59.889 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.889 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:59.889 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184302+0000", 2024-09-04T17:46:59.889 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.889 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.889 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.889 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:59.889 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184302+0000", 2024-09-04T17:46:59.889 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.890 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.890 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.890 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:59.890 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184303+0000", 2024-09-04T17:46:59.890 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.206e-06 2024-09-04T17:46:59.890 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.890 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.890 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:59.890 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184340+0000", 2024-09-04T17:46:59.890 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7005999999999998e-05 2024-09-04T17:46:59.890 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.890 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.891 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:59.891 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184382+0000", 2024-09-04T17:46:59.891 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2345999999999997e-05 2024-09-04T17:46:59.891 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.891 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.891 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:59.891 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184397+0000", 2024-09-04T17:46:59.891 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4729000000000001e-05 2024-09-04T17:46:59.891 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.891 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.891 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:59.891 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.226749+0000", 2024-09-04T17:46:59.891 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042352278 2024-09-04T17:46:59.892 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.892 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.892 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:59.892 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.226889+0000", 2024-09-04T17:46:59.892 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000139376 2024-09-04T17:46:59.892 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.892 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.892 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:59.892 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.227062+0000", 2024-09-04T17:46:59.892 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00017342099999999999 2024-09-04T17:46:59.892 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.892 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:59.893 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.893 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.893 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.893 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14385 3.d 3:b763de10:::rbd_data.11345bd5898b.0000000000000002:a [read 626688~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:59.893 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.184481+0000", 2024-09-04T17:46:59.893 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 41.677258678999998, 2024-09-04T17:46:59.893 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042873850999999998, 2024-09-04T17:46:59.893 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:59.893 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:59.893 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:59.893 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:59.893 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:59.893 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:59.894 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14385 2024-09-04T17:46:59.894 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.894 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:59.894 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.894 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:59.894 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184481+0000", 2024-09-04T17:46:59.894 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.894 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.894 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.894 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:59.894 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184481+0000", 2024-09-04T17:46:59.894 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.895 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.895 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.895 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:59.895 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184482+0000", 2024-09-04T17:46:59.895 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2130000000000001e-06 2024-09-04T17:46:59.895 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.895 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.895 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:59.895 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184487+0000", 2024-09-04T17:46:59.895 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4260000000000002e-06 2024-09-04T17:46:59.895 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.895 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.895 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:59.896 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184505+0000", 2024-09-04T17:46:59.896 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7659000000000002e-05 2024-09-04T17:46:59.896 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.896 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.896 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:59.896 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184518+0000", 2024-09-04T17:46:59.896 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2982e-05 2024-09-04T17:46:59.896 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.896 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.896 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:59.896 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.227120+0000", 2024-09-04T17:46:59.896 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042601858999999999 2024-09-04T17:46:59.896 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.896 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.897 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:59.897 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.227196+0000", 2024-09-04T17:46:59.897 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.6540999999999998e-05 2024-09-04T17:46:59.897 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.897 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.897 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:59.897 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.227354+0000", 2024-09-04T17:46:59.897 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000158171 2024-09-04T17:46:59.897 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.897 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:59.897 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.897 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.898 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.898 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14389 3.d 3:b763de10:::rbd_data.11345bd5898b.0000000000000002:a [read 663552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:59.898 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.184555+0000", 2024-09-04T17:46:59.898 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 41.677184543000003, 2024-09-04T17:46:59.898 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043553640999999997, 2024-09-04T17:46:59.898 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:59.898 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:59.898 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:46:59.898 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:59.898 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:59.898 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:59.898 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14389 2024-09-04T17:46:59.898 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.898 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:59.899 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.899 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:59.899 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184555+0000", 2024-09-04T17:46:59.899 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.899 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.899 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.899 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:59.899 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184555+0000", 2024-09-04T17:46:59.899 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.899 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.899 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.899 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:59.899 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184556+0000", 2024-09-04T17:46:59.900 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.353e-06 2024-09-04T17:46:59.900 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.900 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.900 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:59.900 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184562+0000", 2024-09-04T17:46:59.900 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7899999999999996e-06 2024-09-04T17:46:59.900 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.900 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.900 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:59.900 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184580+0000", 2024-09-04T17:46:59.900 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7682000000000001e-05 2024-09-04T17:46:59.900 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.901 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.901 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:59.901 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184593+0000", 2024-09-04T17:46:59.901 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3625000000000001e-05 2024-09-04T17:46:59.901 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.901 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.901 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:59.901 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.227413+0000", 2024-09-04T17:46:59.901 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042819572 2024-09-04T17:46:59.901 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.901 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.901 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:59.902 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.227552+0000", 2024-09-04T17:46:59.902 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000138842 2024-09-04T17:46:59.902 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.902 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.902 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:59.902 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228108+0000", 2024-09-04T17:46:59.902 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00055677700000000005 2024-09-04T17:46:59.902 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.902 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:59.902 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.902 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.902 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.902 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14392 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:59.903 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.184613+0000", 2024-09-04T17:46:59.903 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 41.677126281, 2024-09-04T17:46:59.903 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043677522000000003, 2024-09-04T17:46:59.903 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:59.903 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:59.903 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:59.903 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:59.903 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:59.903 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:59.903 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14392 2024-09-04T17:46:59.903 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.903 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:59.903 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.904 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:59.904 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184613+0000", 2024-09-04T17:46:59.904 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.904 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.904 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.904 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:59.904 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184613+0000", 2024-09-04T17:46:59.904 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.904 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.904 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.904 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:59.904 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184614+0000", 2024-09-04T17:46:59.904 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.35e-06 2024-09-04T17:46:59.905 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.905 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.905 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:59.905 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184620+0000", 2024-09-04T17:46:59.905 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.5589999999999996e-06 2024-09-04T17:46:59.905 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.905 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.905 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:59.905 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184671+0000", 2024-09-04T17:46:59.905 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0615000000000002e-05 2024-09-04T17:46:59.905 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.905 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.905 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:59.906 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184684+0000", 2024-09-04T17:46:59.906 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3596e-05 2024-09-04T17:46:59.906 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.906 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.906 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:59.906 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228277+0000", 2024-09-04T17:46:59.906 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043592509000000002 2024-09-04T17:46:59.906 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.906 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.906 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:59.906 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228291+0000", 2024-09-04T17:46:59.906 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3893000000000001e-05 2024-09-04T17:46:59.906 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.907 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:59.907 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.907 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.907 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.907 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14395 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:59.907 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.184708+0000", 2024-09-04T17:46:59.907 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 41.67703109, 2024-09-04T17:46:59.907 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043643612999999998, 2024-09-04T17:46:59.907 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:59.907 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:59.907 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:59.907 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:59.907 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:59.908 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:59.908 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14395 2024-09-04T17:46:59.908 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.908 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:59.908 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.908 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:59.908 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184708+0000", 2024-09-04T17:46:59.908 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.908 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.908 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.908 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:59.908 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184708+0000", 2024-09-04T17:46:59.909 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.909 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.909 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.909 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:59.909 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184709+0000", 2024-09-04T17:46:59.909 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2359999999999999e-06 2024-09-04T17:46:59.909 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.909 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.909 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:59.909 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184716+0000", 2024-09-04T17:46:59.909 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.3500000000000002e-06 2024-09-04T17:46:59.909 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.910 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.910 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:59.910 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184733+0000", 2024-09-04T17:46:59.910 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6775000000000001e-05 2024-09-04T17:46:59.910 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.910 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.910 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:59.910 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184777+0000", 2024-09-04T17:46:59.910 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4183e-05 2024-09-04T17:46:59.910 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.910 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.910 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:59.910 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228339+0000", 2024-09-04T17:46:59.910 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043562030000000002 2024-09-04T17:46:59.911 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.911 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.911 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:59.911 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228352+0000", 2024-09-04T17:46:59.911 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3039e-05 2024-09-04T17:46:59.911 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.911 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:59.911 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.911 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.911 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.911 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14398 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:59.911 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.184847+0000", 2024-09-04T17:46:59.912 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 41.676892443, 2024-09-04T17:46:59.912 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043564139000000002, 2024-09-04T17:46:59.912 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:59.912 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:59.912 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:59.912 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:59.912 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:59.912 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:59.912 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14398 2024-09-04T17:46:59.912 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.912 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:59.912 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.912 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:59.913 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184847+0000", 2024-09-04T17:46:59.913 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.913 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.913 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.913 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:59.913 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184847+0000", 2024-09-04T17:46:59.913 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.913 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.913 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.913 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:59.913 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184848+0000", 2024-09-04T17:46:59.913 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.28e-06 2024-09-04T17:46:59.913 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.914 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.914 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:59.914 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184861+0000", 2024-09-04T17:46:59.914 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3125000000000001e-05 2024-09-04T17:46:59.914 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.914 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.914 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:59.914 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184884+0000", 2024-09-04T17:46:59.914 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2357999999999998e-05 2024-09-04T17:46:59.914 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.914 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.914 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:59.914 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184900+0000", 2024-09-04T17:46:59.915 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6872e-05 2024-09-04T17:46:59.915 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.915 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.915 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:59.915 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228398+0000", 2024-09-04T17:46:59.915 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043497313000000003 2024-09-04T17:46:59.915 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.915 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.915 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:59.915 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228411+0000", 2024-09-04T17:46:59.915 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3191e-05 2024-09-04T17:46:59.915 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.916 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:59.916 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.916 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.916 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.916 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14399 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:59.916 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.184925+0000", 2024-09-04T17:46:59.916 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 41.676814127, 2024-09-04T17:46:59.916 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043544799000000002, 2024-09-04T17:46:59.916 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:59.916 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:59.916 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:59.916 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:59.916 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:59.917 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:59.917 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14399 2024-09-04T17:46:59.917 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.917 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:59.917 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.917 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:59.917 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184925+0000", 2024-09-04T17:46:59.917 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.917 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.917 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.917 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:59.917 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184925+0000", 2024-09-04T17:46:59.917 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.918 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.918 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.918 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:59.918 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184927+0000", 2024-09-04T17:46:59.918 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7e-06 2024-09-04T17:46:59.918 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.918 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.918 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:59.918 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184949+0000", 2024-09-04T17:46:59.918 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1880999999999999e-05 2024-09-04T17:46:59.918 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.918 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.918 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:59.919 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184969+0000", 2024-09-04T17:46:59.919 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0758e-05 2024-09-04T17:46:59.919 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.919 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.919 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:59.919 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184983+0000", 2024-09-04T17:46:59.919 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3336000000000001e-05 2024-09-04T17:46:59.919 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.919 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.919 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:59.919 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228456+0000", 2024-09-04T17:46:59.919 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043472960999999997 2024-09-04T17:46:59.919 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.920 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.920 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:59.920 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228470+0000", 2024-09-04T17:46:59.920 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4163e-05 2024-09-04T17:46:59.920 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.920 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:59.920 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.920 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.920 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.920 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14400 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:59.920 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.185003+0000", 2024-09-04T17:46:59.920 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 41.676736691000002, 2024-09-04T17:46:59.920 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043532659000000001, 2024-09-04T17:46:59.921 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:59.921 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:59.921 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:59.921 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:59.921 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:59.921 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:59.921 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14400 2024-09-04T17:46:59.921 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.921 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:59.921 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.921 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:59.921 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185003+0000", 2024-09-04T17:46:59.922 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.922 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.922 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.922 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:59.922 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185003+0000", 2024-09-04T17:46:59.922 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.922 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.922 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.922 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:59.922 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185004+0000", 2024-09-04T17:46:59.922 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.294e-06 2024-09-04T17:46:59.922 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.922 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.922 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:59.923 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185010+0000", 2024-09-04T17:46:59.923 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7690000000000002e-06 2024-09-04T17:46:59.923 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.923 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.923 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:59.923 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185029+0000", 2024-09-04T17:46:59.923 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9100999999999999e-05 2024-09-04T17:46:59.923 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.923 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.923 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:59.923 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185043+0000", 2024-09-04T17:46:59.923 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3862e-05 2024-09-04T17:46:59.924 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.924 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.924 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:59.924 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228510+0000", 2024-09-04T17:46:59.924 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043467766999999997 2024-09-04T17:46:59.924 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.924 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.924 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:59.924 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228535+0000", 2024-09-04T17:46:59.924 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4865999999999999e-05 2024-09-04T17:46:59.924 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.924 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:59.924 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.925 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.925 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.925 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14401 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:59.925 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.185062+0000", 2024-09-04T17:46:59.925 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 41.676677425999998, 2024-09-04T17:46:59.925 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043556096000000002, 2024-09-04T17:46:59.925 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:59.925 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:59.925 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:59.925 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:59.925 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:59.925 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:59.925 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14401 2024-09-04T17:46:59.926 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.926 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:59.926 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.926 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:59.926 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185062+0000", 2024-09-04T17:46:59.926 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.926 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.926 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.926 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:59.926 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185062+0000", 2024-09-04T17:46:59.926 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.926 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.926 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.926 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:59.927 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185063+0000", 2024-09-04T17:46:59.927 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.409e-06 2024-09-04T17:46:59.927 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.927 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.927 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:59.927 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185069+0000", 2024-09-04T17:46:59.927 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.5400000000000003e-06 2024-09-04T17:46:59.927 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.927 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.927 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:59.927 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185087+0000", 2024-09-04T17:46:59.927 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8195000000000001e-05 2024-09-04T17:46:59.927 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.928 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.928 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:59.928 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185101+0000", 2024-09-04T17:46:59.928 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3769e-05 2024-09-04T17:46:59.928 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.928 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.928 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:59.928 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228604+0000", 2024-09-04T17:46:59.928 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043503537000000002 2024-09-04T17:46:59.928 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.928 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.928 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:59.928 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228618+0000", 2024-09-04T17:46:59.928 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3645999999999999e-05 2024-09-04T17:46:59.929 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.929 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:59.929 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.929 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.929 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.929 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14402 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:59.929 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.185121+0000", 2024-09-04T17:46:59.929 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 41.676618132000002, 2024-09-04T17:46:59.929 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043551542999999998, 2024-09-04T17:46:59.929 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:59.929 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:59.929 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:59.929 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:59.930 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:59.930 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:59.930 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14402 2024-09-04T17:46:59.930 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.930 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:59.930 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.930 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:59.930 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185121+0000", 2024-09-04T17:46:59.930 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.930 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.930 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.930 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:59.930 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185121+0000", 2024-09-04T17:46:59.931 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.931 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.931 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.931 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:59.931 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185122+0000", 2024-09-04T17:46:59.931 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1769999999999999e-06 2024-09-04T17:46:59.931 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.931 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.931 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:59.931 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185128+0000", 2024-09-04T17:46:59.931 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7790000000000001e-06 2024-09-04T17:46:59.931 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.931 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.931 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:59.932 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185146+0000", 2024-09-04T17:46:59.932 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8114999999999999e-05 2024-09-04T17:46:59.932 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.932 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.932 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:59.932 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185160+0000", 2024-09-04T17:46:59.932 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4158999999999999e-05 2024-09-04T17:46:59.932 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.932 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.932 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:59.932 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228659+0000", 2024-09-04T17:46:59.932 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043498715 2024-09-04T17:46:59.933 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.933 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.933 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:59.933 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228673+0000", 2024-09-04T17:46:59.933 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3597999999999999e-05 2024-09-04T17:46:59.933 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.933 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:59.933 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.933 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.933 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.933 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14403 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:59.933 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.185183+0000", 2024-09-04T17:46:59.933 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 41.676556083999998, 2024-09-04T17:46:59.934 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043541467, 2024-09-04T17:46:59.934 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:59.934 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:59.934 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:59.934 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:59.934 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:59.934 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:59.934 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14403 2024-09-04T17:46:59.934 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.934 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:59.934 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.934 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:59.934 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185183+0000", 2024-09-04T17:46:59.935 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.935 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.935 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.935 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:59.935 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185183+0000", 2024-09-04T17:46:59.935 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.935 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.935 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.935 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:59.935 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185184+0000", 2024-09-04T17:46:59.935 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.156e-06 2024-09-04T17:46:59.935 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.935 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.936 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:59.936 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185190+0000", 2024-09-04T17:46:59.936 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1800000000000004e-06 2024-09-04T17:46:59.936 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.936 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.936 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:59.936 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185207+0000", 2024-09-04T17:46:59.936 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7288000000000001e-05 2024-09-04T17:46:59.936 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.936 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.936 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:59.936 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185221+0000", 2024-09-04T17:46:59.936 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3729e-05 2024-09-04T17:46:59.936 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.937 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.937 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:59.937 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228711+0000", 2024-09-04T17:46:59.937 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043490595 2024-09-04T17:46:59.937 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.937 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.937 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:59.937 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228725+0000", 2024-09-04T17:46:59.937 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3519e-05 2024-09-04T17:46:59.937 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.937 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:59.937 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.937 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.938 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.938 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14404 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:59.938 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.185239+0000", 2024-09-04T17:46:59.938 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 41.676499946, 2024-09-04T17:46:59.938 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043542099000000001, 2024-09-04T17:46:59.938 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:59.938 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:59.938 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:59.938 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:59.938 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:59.938 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:59.938 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14404 2024-09-04T17:46:59.938 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.939 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:59.939 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.939 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:59.939 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185239+0000", 2024-09-04T17:46:59.939 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.939 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.939 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.939 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:59.939 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185239+0000", 2024-09-04T17:46:59.939 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.939 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.939 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.939 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:59.940 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185240+0000", 2024-09-04T17:46:59.940 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.173e-06 2024-09-04T17:46:59.940 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.940 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.940 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:59.940 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185246+0000", 2024-09-04T17:46:59.940 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7529999999999998e-06 2024-09-04T17:46:59.940 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.940 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.940 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:59.940 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185264+0000", 2024-09-04T17:46:59.940 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7872000000000001e-05 2024-09-04T17:46:59.940 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.941 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.941 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:59.941 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185278+0000", 2024-09-04T17:46:59.941 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3532e-05 2024-09-04T17:46:59.941 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.941 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.941 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:59.941 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228768+0000", 2024-09-04T17:46:59.941 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043490586999999997 2024-09-04T17:46:59.941 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.941 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.941 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:59.942 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228781+0000", 2024-09-04T17:46:59.942 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3182e-05 2024-09-04T17:46:59.942 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.942 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:59.942 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.942 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.942 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.942 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14405 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:59.942 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.185297+0000", 2024-09-04T17:46:59.942 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 41.676442754999997, 2024-09-04T17:46:59.942 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043535049999999999, 2024-09-04T17:46:59.942 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:59.942 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:59.943 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:59.943 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:59.943 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:59.943 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:59.943 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14405 2024-09-04T17:46:59.943 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.943 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:59.943 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.943 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:59.943 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185297+0000", 2024-09-04T17:46:59.943 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.943 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.943 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.944 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:59.944 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185297+0000", 2024-09-04T17:46:59.944 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.944 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.944 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.944 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:59.944 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185298+0000", 2024-09-04T17:46:59.944 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.22e-06 2024-09-04T17:46:59.944 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.944 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.944 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:59.944 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185308+0000", 2024-09-04T17:46:59.945 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0173000000000001e-05 2024-09-04T17:46:59.945 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.945 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.945 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:59.945 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185329+0000", 2024-09-04T17:46:59.945 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1087999999999999e-05 2024-09-04T17:46:59.945 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.945 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.945 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:59.945 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185345+0000", 2024-09-04T17:46:59.945 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6035000000000001e-05 2024-09-04T17:46:59.945 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.946 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.946 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:59.946 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228822+0000", 2024-09-04T17:46:59.946 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043477427999999999 2024-09-04T17:46:59.946 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.946 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.946 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:59.946 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228832+0000", 2024-09-04T17:46:59.946 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1060000000000006e-06 2024-09-04T17:46:59.946 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.946 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:59.946 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.946 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.947 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.947 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14411 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:46:59.947 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.186171+0000", 2024-09-04T17:46:59.947 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 41.675567932, 2024-09-04T17:46:59.947 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.04271581, 2024-09-04T17:46:59.947 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:59.947 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:59.947 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:46:59.947 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:59.947 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:59.947 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:59.947 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14411 2024-09-04T17:46:59.947 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.948 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:59.948 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.948 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:59.948 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.186171+0000", 2024-09-04T17:46:59.948 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.948 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.948 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.948 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:59.948 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.186171+0000", 2024-09-04T17:46:59.948 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.948 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.948 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.948 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:59.949 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.186173+0000", 2024-09-04T17:46:59.949 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.012e-06 2024-09-04T17:46:59.949 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.949 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.949 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:59.949 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.186183+0000", 2024-09-04T17:46:59.949 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.7729999999999994e-06 2024-09-04T17:46:59.949 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.949 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.949 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:59.949 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.186205+0000", 2024-09-04T17:46:59.949 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2280999999999999e-05 2024-09-04T17:46:59.949 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.950 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.950 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:59.950 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.186224+0000", 2024-09-04T17:46:59.950 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8695e-05 2024-09-04T17:46:59.950 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.950 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.950 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:59.950 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228873+0000", 2024-09-04T17:46:59.950 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042648798000000002 2024-09-04T17:46:59.950 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.950 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.950 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:59.950 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228887+0000", 2024-09-04T17:46:59.951 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4251e-05 2024-09-04T17:46:59.951 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.951 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:59.951 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.951 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.951 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.951 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:37359 3.f 3:f29e50a5:::rbd_data.1134c5a43465.0000000000000021:head [stat,write 62464~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-04T17:46:59.951 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.616887+0000", 2024-09-04T17:46:59.951 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 20.244852015999999, 2024-09-04T17:46:59.951 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.054183104000000003, 2024-09-04T17:46:59.951 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:46:59.951 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:46:59.951 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:46:59.951 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:46:59.952 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:46:59.952 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:46:59.952 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 37359 2024-09-04T17:46:59.952 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.952 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:46:59.952 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.952 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:46:59.952 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.616887+0000", 2024-09-04T17:46:59.952 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.952 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.952 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.952 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:46:59.952 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.616887+0000", 2024-09-04T17:46:59.953 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:46:59.953 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.953 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.953 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:46:59.953 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.616889+0000", 2024-09-04T17:46:59.953 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.362e-06 2024-09-04T17:46:59.953 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.953 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.953 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:46:59.953 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.616904+0000", 2024-09-04T17:46:59.953 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.507e-05 2024-09-04T17:46:59.953 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.953 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.954 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:46:59.954 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.616920+0000", 2024-09-04T17:46:59.954 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6385999999999999e-05 2024-09-04T17:46:59.954 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.954 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.954 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:46:59.954 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.616929+0000", 2024-09-04T17:46:59.954 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8540000000000003e-06 2024-09-04T17:46:59.954 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.954 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.954 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:46:59.954 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.616983+0000", 2024-09-04T17:46:59.954 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4218999999999997e-05 2024-09-04T17:46:59.955 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.955 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.955 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:46:59.955 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617065+0000", 2024-09-04T17:46:59.955 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.1864000000000001e-05 2024-09-04T17:46:59.955 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.955 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.955 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 4", 2024-09-04T17:46:59.955 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617205+0000", 2024-09-04T17:46:59.955 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013962600000000001 2024-09-04T17:46:59.955 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.955 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.955 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:46:59.955 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.618670+0000", 2024-09-04T17:46:59.956 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.001465647 2024-09-04T17:46:59.956 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.956 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.956 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:46:59.956 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670962+0000", 2024-09-04T17:46:59.956 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.052292000999999998 2024-09-04T17:46:59.956 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.956 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.956 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:46:59.956 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671011+0000", 2024-09-04T17:46:59.956 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9036000000000003e-05 2024-09-04T17:46:59.956 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:46:59.957 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:46:59.957 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:46:59.957 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671070+0000", 2024-09-04T17:46:59.957 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.9039000000000002e-05 2024-09-04T17:46:59.957 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.957 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:59.957 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.957 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:46:59.957 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:46:59.957 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:46:59.958 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:46:59.959 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:46:59.959 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:46:59.959 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:46:59.960 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-04T17:46:59.960 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:46:59.957+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:46:59.975 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-04T17:46:59.975 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-04T17:46:59.993 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-04T17:46:59.993 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-04T17:47:00.016 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:47:00.016 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:47:00.037 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:47:00.037 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:47:00.037 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-04T17:47:00.058 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:47:00.058 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:47:00.059 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:00.055+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:00.061 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-04T17:47:00.062 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-04T17:47:00.078 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:00.078 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:00.092 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-04T17:47:00.180 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:00.179+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:00.257 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:00.258 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:00.261 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:00.261+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:00.272 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-04T17:47:00.362 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:47:00.361+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:47:00.438 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:00.438 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:00.452 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-04T17:47:00.462 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:00.461+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:00.563 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:47:00.561+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:47:00.574 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:00.575 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:00.589 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-04T17:47:00.655 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-04T17:47:00.668 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:00.668 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:47:00.667+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:47:00.684 INFO:teuthology.orchestra.run.smithi092.stdout:3324 trunc from 0xb6bc0b2 to 0x188f7e5 2024-09-04T17:47:00.717 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:00.718 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:00.722 INFO:teuthology.orchestra.run.smithi092.stdout:3325 read 0x13225c1 thru 0x13318ac (0xf2ec bytes) 2024-09-04T17:47:00.722 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:00.734 INFO:teuthology.orchestra.run.smithi092.stdout:3327 write 0x408b4b5 thru 0x409998e (0xe4da bytes) 2024-09-04T17:47:00.738 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-04T17:47:00.743 INFO:teuthology.orchestra.run.smithi092.stdout:3328 punch from 0x17e57de to 0x17e6ba1, (0x13c3 bytes) 2024-09-04T17:47:00.743 INFO:teuthology.orchestra.run.smithi092.stdout:3329 write 0x3c2067a thru 0x3c2de32 (0xd7b9 bytes) 2024-09-04T17:47:00.751 INFO:teuthology.orchestra.run.smithi092.stdout:3330 read 0x2802812 thru 0x2805505 (0x2cf4 bytes) 2024-09-04T17:47:00.751 INFO:teuthology.orchestra.run.smithi092.stdout:3332 read 0x24f6cba thru 0x24f93bc (0x2703 bytes) 2024-09-04T17:47:00.762 INFO:teuthology.orchestra.run.smithi092.stdout:3333 writesame 0x537936f thru 0x5383506 data_size 0x4(0xa198 bytes) 2024-09-04T17:47:00.762 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:00.767 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:00.763+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:00.773 INFO:teuthology.orchestra.run.smithi092.stdout:3335 writesame 0x7679516 thru 0x7689053 data_size 0x1(0xfb3e bytes) 2024-09-04T17:47:00.776 INFO:teuthology.orchestra.run.smithi092.stdout:3337 trunc from 0x7689054 to 0x546e83b 2024-09-04T17:47:00.784 INFO:teuthology.orchestra.run.smithi092.stdout:3338 read 0x30754cc thru 0x30761a9 (0xcde bytes) 2024-09-04T17:47:00.785 INFO:teuthology.orchestra.run.smithi092.stdout:3340 read 0x17de2bc thru 0x17eb3b5 (0xd0fa bytes) 2024-09-04T17:47:00.785 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:00.795 INFO:teuthology.orchestra.run.smithi092.stdout:3342 write 0x64a5cb8 thru 0x64b59f4 (0xfd3d bytes) 2024-09-04T17:47:00.803 INFO:teuthology.orchestra.run.smithi092.stdout:3343 read 0x491ccfa thru 0x4926a67 (0x9d6e bytes) 2024-09-04T17:47:00.803 INFO:teuthology.orchestra.run.smithi092.stdout:3344 read 0x29f478e thru 0x2a00b5f (0xc3d2 bytes) 2024-09-04T17:47:00.803 INFO:teuthology.orchestra.run.smithi092.stdout:3345 read 0x1011204 thru 0x1017034 (0x5e31 bytes) 2024-09-04T17:47:00.803 INFO:teuthology.orchestra.run.smithi092.stdout:3348 read 0x3f61634 thru 0x3f69ed2 (0x889f bytes) 2024-09-04T17:47:00.804 INFO:teuthology.orchestra.run.smithi092.stdout:3351 read 0x1b48250 thru 0x1b4b1c0 (0x2f71 bytes) 2024-09-04T17:47:00.814 INFO:teuthology.orchestra.run.smithi092.stdout:3352 writesame 0x7492c03 thru 0x749a946 data_size 0x2(0x7d44 bytes) 2024-09-04T17:47:00.814 INFO:teuthology.orchestra.run.smithi092.stdout:3353 punch from 0x35caf7f to 0x35cec10, (0x3c91 bytes) 2024-09-04T17:47:00.832 INFO:teuthology.orchestra.run.smithi092.stdout:3354 writesame 0xe9ab2c5 thru 0xe9b7b0c data_size 0x88(0xc848 bytes) 2024-09-04T17:47:00.832 INFO:teuthology.orchestra.run.smithi092.stdout:3355 write 0xa76cdb thru 0xa867df (0xfb05 bytes) 2024-09-04T17:47:00.836 INFO:teuthology.orchestra.run.smithi092.stdout:3356 write 0x9b49a60 thru 0x9b4c21f (0x27c0 bytes) 2024-09-04T17:47:00.842 INFO:teuthology.orchestra.run.smithi092.stdout:3358 punch from 0xb53ad0a to 0xb546d47, (0xc03d bytes) 2024-09-04T17:47:00.842 INFO:teuthology.orchestra.run.smithi092.stdout:3359 read 0x49ec8dc thru 0x49fb776 (0xee9b bytes) 2024-09-04T17:47:00.842 INFO:teuthology.orchestra.run.smithi092.stdout:3360 read 0x43ba703 thru 0x43bfc9a (0x5598 bytes) 2024-09-04T17:47:00.842 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:00.850 INFO:teuthology.orchestra.run.smithi092.stdout:3362 write 0xec55289 thru 0xec5aeb5 (0x5c2d bytes) 2024-09-04T17:47:00.858 INFO:teuthology.orchestra.run.smithi092.stdout:3364 read 0xaabfc6f thru 0xaac9377 (0x9709 bytes) 2024-09-04T17:47:00.858 INFO:teuthology.orchestra.run.smithi092.stdout:3365 read 0x9d1bce3 thru 0x9d27da9 (0xc0c7 bytes) 2024-09-04T17:47:00.858 INFO:teuthology.orchestra.run.smithi092.stdout:3367 write 0xe0e2206 thru 0xe0ed29b (0xb096 bytes) 2024-09-04T17:47:00.864 INFO:teuthology.orchestra.run.smithi092.stdout:3368 write 0x50228c2 thru 0x5030b40 (0xe27f bytes) 2024-09-04T17:47:00.866 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:00.865+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:00.871 INFO:teuthology.orchestra.run.smithi092.stdout:3369 read 0xa72e13b thru 0xa7340e2 (0x5fa8 bytes) 2024-09-04T17:47:00.871 INFO:teuthology.orchestra.run.smithi092.stdout:3370 writesame 0xa2c1edd thru 0xa2c5c0c data_size 0x2c(0x3d30 bytes) 2024-09-04T17:47:00.877 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:00.877 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:00.878 INFO:teuthology.orchestra.run.smithi092.stdout:3371 write 0x68eb120 thru 0x68eecae (0x3b8f bytes) 2024-09-04T17:47:00.883 INFO:teuthology.orchestra.run.smithi092.stdout:3372 read 0x2b57b9c thru 0x2b639a5 (0xbe0a bytes) 2024-09-04T17:47:00.892 INFO:teuthology.orchestra.run.smithi092.stdout:3374 trunc from 0xec5aeb6 to 0x7b10389 2024-09-04T17:47:00.896 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-04T17:47:00.910 INFO:teuthology.orchestra.run.smithi092.stdout:3375 write 0x17a753 thru 0x17f27b (0x4b29 bytes) 2024-09-04T17:47:00.914 INFO:teuthology.orchestra.run.smithi092.stdout:3376 read 0x352bba9 thru 0x353414c (0x85a4 bytes) 2024-09-04T17:47:00.914 INFO:teuthology.orchestra.run.smithi092.stdout:3377 write 0x6bf4e9 thru 0x6cd406 (0xdf1e bytes) 2024-09-04T17:47:00.918 INFO:teuthology.orchestra.run.smithi092.stdout:3380 punch from 0x3b514ef to 0x3b5d108, (0xbc19 bytes) 2024-09-04T17:47:00.918 INFO:teuthology.orchestra.run.smithi092.stdout:3383 read 0x60405e5 thru 0x604a01d (0x9a39 bytes) 2024-09-04T17:47:00.920 INFO:teuthology.orchestra.run.smithi092.stdout:3384 trunc from 0x7b10389 to 0x5c03357 2024-09-04T17:47:00.947 INFO:teuthology.orchestra.run.smithi092.stdout:3385 trunc from 0x5c03357 to 0xba48390 2024-09-04T17:47:00.954 INFO:teuthology.orchestra.run.smithi092.stdout:3386 read 0xa405a05 thru 0xa40b8b5 (0x5eb1 bytes) 2024-09-04T17:47:00.954 INFO:teuthology.orchestra.run.smithi092.stdout:3388 write 0x9766f82 thru 0x976b343 (0x43c2 bytes) 2024-09-04T17:47:00.961 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:00.961 INFO:teuthology.orchestra.run.smithi092.stdout:3390 writesame 0x5ca009c thru 0x5ca8b25 data_size 0x45(0x8a8a bytes) 2024-09-04T17:47:00.966 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:00.965+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:00.968 INFO:teuthology.orchestra.run.smithi092.stdout:3393 writesame 0x8ac43cb thru 0x8acc191 data_size 0x1(0x7dc7 bytes) 2024-09-04T17:47:01.017 INFO:teuthology.orchestra.run.smithi092.stdout:3394 writesame 0x37672a2 thru 0x3768d84 data_size 0x1(0x1ae3 bytes) 2024-09-04T17:47:01.026 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:01.026 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:01.039 INFO:teuthology.orchestra.run.smithi092.stdout:3396 trunc from 0xba48390 to 0x6093f5f 2024-09-04T17:47:01.044 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-04T17:47:01.058 INFO:teuthology.orchestra.run.smithi092.stdout:3397 read 0x2bf18e2 thru 0x2bf8ac4 (0x71e3 bytes) 2024-09-04T17:47:01.058 INFO:teuthology.orchestra.run.smithi092.stdout:3398 write 0x587f188 thru 0x588c6e7 (0xd560 bytes) 2024-09-04T17:47:01.068 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:01.065+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:01.082 INFO:teuthology.orchestra.run.smithi092.stdout:3399 writesame 0xc6c22c7 thru 0xc6c6165 data_size 0x17(0x3e9f bytes) 2024-09-04T17:47:01.082 INFO:teuthology.orchestra.run.smithi092.stdout:3402 write 0x2dbe707 thru 0x2dc2f65 (0x485f bytes) 2024-09-04T17:47:01.085 INFO:teuthology.orchestra.run.smithi092.stdout:3403 write 0x3fa15b6 thru 0x3fa6ba3 (0x55ee bytes) 2024-09-04T17:47:01.094 INFO:teuthology.orchestra.run.smithi092.stdout:3404 read 0x3c7cc20 thru 0x3c86f7e (0xa35f bytes) 2024-09-04T17:47:01.095 INFO:teuthology.orchestra.run.smithi092.stdout:3405 read 0xa57e04e thru 0xa58d93f (0xf8f2 bytes) 2024-09-04T17:47:01.095 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:01.095 INFO:teuthology.orchestra.run.smithi092.stdout:3407 write 0x2e51dea thru 0x2e598a8 (0x7abf bytes) 2024-09-04T17:47:01.102 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:01.103 INFO:teuthology.orchestra.run.smithi092.stdout:3410 read 0x617c6db thru 0x61856a1 (0x8fc7 bytes) 2024-09-04T17:47:01.103 INFO:teuthology.orchestra.run.smithi092.stdout:3411 read 0xc347001 thru 0xc34ede2 (0x7de2 bytes) 2024-09-04T17:47:01.104 INFO:teuthology.orchestra.run.smithi092.stdout:3412 trunc from 0xc6c6166 to 0xaff5cc3 2024-09-04T17:47:01.113 INFO:teuthology.orchestra.run.smithi092.stdout:3413 read 0x79338b9 thru 0x793e4d0 (0xac18 bytes) 2024-09-04T17:47:01.113 INFO:teuthology.orchestra.run.smithi092.stdout:3414 writesame 0x192dbe0 thru 0x1932409 data_size 0x3(0x482a bytes) 2024-09-04T17:47:01.122 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:01.122 INFO:teuthology.orchestra.run.smithi092.stdout:3416 read 0x848bd58 thru 0x8498c0d (0xceb6 bytes) 2024-09-04T17:47:01.122 INFO:teuthology.orchestra.run.smithi092.stdout:3417 write 0x7ac2cc0 thru 0x7ac4e65 (0x21a6 bytes) 2024-09-04T17:47:01.129 INFO:teuthology.orchestra.run.smithi092.stdout:3418 writesame 0xa4348ca thru 0xa43a69c data_size 0x1(0x5dd3 bytes) 2024-09-04T17:47:01.167 INFO:teuthology.orchestra.run.smithi092.stdout:3420 writesame 0x4e85d34 thru 0x4e8c66b data_size 0x68(0x6938 bytes) 2024-09-04T17:47:01.168 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:47:01.163+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:47:01.176 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:01.176 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:01.187 INFO:teuthology.orchestra.run.smithi092.stdout:3421 trunc from 0xaff5cc3 to 0x5029bd1 2024-09-04T17:47:01.195 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-04T17:47:01.223 INFO:teuthology.orchestra.run.smithi092.stdout:3422 read 0x2e0d342 thru 0x2e12593 (0x5252 bytes) 2024-09-04T17:47:01.224 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:01.224 INFO:teuthology.orchestra.run.smithi092.stdout:3424 punch from 0xd5c2c0 to 0xd5c594, (0x2d4 bytes) 2024-09-04T17:47:01.233 INFO:teuthology.orchestra.run.smithi092.stdout:3425 trunc from 0x5029bd1 to 0x9648679 2024-09-04T17:47:01.258 INFO:teuthology.orchestra.run.smithi092.stdout:3428 write 0xe3e34c7 thru 0xe3eaff8 (0x7b32 bytes) 2024-09-04T17:47:01.266 INFO:teuthology.orchestra.run.smithi092.stdout:3429 punch from 0xa82a3f to 0xa83fbf, (0x1580 bytes) 2024-09-04T17:47:01.267 INFO:teuthology.orchestra.run.smithi092.stdout:3432 writesame 0x4bc1904 thru 0x4bc200d data_size 0x11(0x70a bytes) 2024-09-04T17:47:01.269 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:01.269+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:01.270 INFO:teuthology.orchestra.run.smithi092.stdout:3433 read 0xda56d00 thru 0xda59518 (0x2819 bytes) 2024-09-04T17:47:01.270 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:01.279 INFO:teuthology.orchestra.run.smithi092.stdout:3436 trunc from 0xe3eaff9 to 0x83f8c32 2024-09-04T17:47:01.299 INFO:teuthology.orchestra.run.smithi092.stdout:3437 write 0x19ef5c1 thru 0x19f0367 (0xda7 bytes) 2024-09-04T17:47:01.303 INFO:teuthology.orchestra.run.smithi092.stdout:3438 punch from 0x3c6a275 to 0x3c7591f, (0xb6aa bytes) 2024-09-04T17:47:01.328 INFO:teuthology.orchestra.run.smithi092.stdout:3439 writesame 0xeb36c74 thru 0xeb3ab97 data_size 0x12(0x3f24 bytes) 2024-09-04T17:47:01.328 INFO:teuthology.orchestra.run.smithi092.stdout:3441 punch from 0x45bd245 to 0x45cb325, (0xe0e0 bytes) 2024-09-04T17:47:01.329 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:01.329 INFO:teuthology.orchestra.run.smithi092.stdout:3444 writesame 0x4b96265 thru 0x4b9b797 data_size 0x1(0x5533 bytes) 2024-09-04T17:47:01.356 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:01.356 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:01.358 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:01.358 INFO:teuthology.orchestra.run.smithi092.stdout:3447 write 0xa56014d thru 0xa563531 (0x33e5 bytes) 2024-09-04T17:47:01.365 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:01.365 INFO:teuthology.orchestra.run.smithi092.stdout:3450 read 0xe6aa869 thru 0xe6b855d (0xdcf5 bytes) 2024-09-04T17:47:01.366 INFO:teuthology.orchestra.run.smithi092.stdout:3451 write 0x903e69a thru 0x9042069 (0x39d0 bytes) 2024-09-04T17:47:01.370 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-04T17:47:01.371 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:01.367+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:01.372 INFO:teuthology.orchestra.run.smithi092.stdout:3452 punch from 0xd7a4daf to 0xd7a9eef, (0x5140 bytes) 2024-09-04T17:47:01.372 INFO:teuthology.orchestra.run.smithi092.stdout:3454 punch from 0x85389f4 to 0x85486b2, (0xfcbe bytes) 2024-09-04T17:47:01.390 INFO:teuthology.orchestra.run.smithi092.stdout:3456 trunc from 0xeb3ab98 to 0x38c9087 2024-09-04T17:47:01.413 INFO:teuthology.orchestra.run.smithi092.stdout:3457 trunc from 0x38c9087 to 0x3c54662 2024-09-04T17:47:01.447 INFO:teuthology.orchestra.run.smithi092.stdout:3458 write 0xb4dd36e thru 0xb4ea6ad (0xd340 bytes) 2024-09-04T17:47:01.456 INFO:teuthology.orchestra.run.smithi092.stdout:3460 read 0x94fa830 thru 0x94faedf (0x6b0 bytes) 2024-09-04T17:47:01.457 INFO:teuthology.orchestra.run.smithi092.stdout:3462 write 0xc2883b thru 0xc345e6 (0xbdac bytes) 2024-09-04T17:47:01.471 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:01.469+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:01.490 INFO:teuthology.orchestra.run.smithi092.stdout:3463 write 0xe034acb thru 0xe039d2c (0x5262 bytes) 2024-09-04T17:47:01.497 INFO:teuthology.orchestra.run.smithi092.stdout:3464 write 0x5960d44 thru 0x59694d8 (0x8795 bytes) 2024-09-04T17:47:01.500 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:01.500 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:01.503 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:01.503 INFO:teuthology.orchestra.run.smithi092.stdout:3466 punch from 0x5b31c4 to 0x5b9358, (0x6194 bytes) 2024-09-04T17:47:01.503 INFO:teuthology.orchestra.run.smithi092.stdout:3468 read 0x5b2143f thru 0x5b27395 (0x5f57 bytes) 2024-09-04T17:47:01.503 INFO:teuthology.orchestra.run.smithi092.stdout:3469 write 0x71cddd5 thru 0x71d4e60 (0x708c bytes) 2024-09-04T17:47:01.510 INFO:teuthology.orchestra.run.smithi092.stdout:3470 write 0x124cd1d thru 0x125bf5f (0xf243 bytes) 2024-09-04T17:47:01.517 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-04T17:47:01.526 INFO:teuthology.orchestra.run.smithi092.stdout:3471 read 0x3ed3292 thru 0x3ee0353 (0xd0c2 bytes) 2024-09-04T17:47:01.526 INFO:teuthology.orchestra.run.smithi092.stdout:3473 read 0x7f861b2 thru 0x7f945e2 (0xe431 bytes) 2024-09-04T17:47:01.526 INFO:teuthology.orchestra.run.smithi092.stdout:3474 write 0xa43d58f thru 0xa44532d (0x7d9f bytes) 2024-09-04T17:47:01.533 INFO:teuthology.orchestra.run.smithi092.stdout:3475 write 0x96833e5 thru 0x968b993 (0x85af bytes) 2024-09-04T17:47:01.541 INFO:teuthology.orchestra.run.smithi092.stdout:3476 read 0xaf550c8 thru 0xaf606ed (0xb626 bytes) 2024-09-04T17:47:01.563 INFO:teuthology.orchestra.run.smithi092.stdout:3477 trunc from 0xe039d2d to 0x12beeb0 2024-09-04T17:47:01.572 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:47:01.573+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:47:01.587 INFO:teuthology.orchestra.run.smithi092.stdout:3478 read 0x3d140f thru 0x3d8f98 (0x7b8a bytes) 2024-09-04T17:47:01.611 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is unset 2024-09-04T17:47:01.614 INFO:teuthology.orchestra.run.smithi092.stdout:3479 write 0x95e36c5 thru 0x95e78bf (0x41fb bytes) 2024-09-04T17:47:01.633 INFO:teuthology.orchestra.run.smithi092.stdout:3480 writesame 0x9b2ce95 thru 0x9b32a7a data_size 0x9(0x5be6 bytes) 2024-09-04T17:47:01.633 INFO:teuthology.orchestra.run.smithi092.stdout:3482 punch from 0x7f2f791 to 0x7f349c0, (0x522f bytes) 2024-09-04T17:47:01.633 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:01.633 INFO:teuthology.orchestra.run.smithi092.stdout:3485 write 0x4c851a9 thru 0x4c85a80 (0x8d8 bytes) 2024-09-04T17:47:01.640 INFO:teuthology.orchestra.run.smithi092.stdout:3486 read 0x5bf1c59 thru 0x5c00156 (0xe4fe bytes) 2024-09-04T17:47:01.640 INFO:teuthology.orchestra.run.smithi092.stdout:3487 punch from 0x7832fd1 to 0x783a6de, (0x770d bytes) 2024-09-04T17:47:01.640 INFO:teuthology.orchestra.run.smithi092.stdout:3488 read 0x6524598 thru 0x652cbfc (0x8665 bytes) 2024-09-04T17:47:01.643 INFO:teuthology.orchestra.run.smithi092.stdout:3489 trunc from 0x9b32a7b to 0xd1260cc 2024-09-04T17:47:01.651 INFO:teuthology.orchestra.run.smithi092.stdout:3491 punch from 0x58b9454 to 0x58c1273, (0x7e1f bytes) 2024-09-04T17:47:01.651 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:01.661 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:01.661 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:01.669 INFO:teuthology.orchestra.run.smithi092.stdout:3493 trunc from 0xd1260cc to 0x392521 2024-09-04T17:47:01.672 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:01.671+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:01.681 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-04T17:47:01.704 INFO:teuthology.orchestra.run.smithi092.stdout:3495 punch from 0x16373f to 0x16d161, (0x9a22 bytes) 2024-09-04T17:47:01.704 INFO:teuthology.orchestra.run.smithi092.stdout:3497 punch from 0x259a36 to 0x261ee2, (0x84ac bytes) 2024-09-04T17:47:01.718 INFO:teuthology.orchestra.run.smithi092.stdout:3498 write 0x4bbde49 thru 0x4bc0baf (0x2d67 bytes) 2024-09-04T17:47:01.727 INFO:teuthology.orchestra.run.smithi092.stdout:3499 read 0x34b9512 thru 0x34bfd50 (0x683f bytes) 2024-09-04T17:47:01.727 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:01.727 INFO:teuthology.orchestra.run.smithi092.stdout:3504 read 0x76a645 thru 0x7756c0 (0xb07c bytes) 2024-09-04T17:47:01.732 INFO:teuthology.orchestra.run.smithi092.stdout:3505 trunc from 0x4bc0bb0 to 0x14ab12d 2024-09-04T17:47:01.749 INFO:teuthology.orchestra.run.smithi092.stdout:3506 read 0xc5459 thru 0xcc7cc (0x7374 bytes) 2024-09-04T17:47:01.773 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:01.773+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:01.776 INFO:teuthology.orchestra.run.smithi092.stdout:3508 write 0xc8f5382 thru 0xc8fcf4a (0x7bc9 bytes) 2024-09-04T17:47:01.783 INFO:teuthology.orchestra.run.smithi092.stdout:3510 punch from 0x2322abd to 0x232516c, (0x26af bytes) 2024-09-04T17:47:01.784 INFO:teuthology.orchestra.run.smithi092.stdout:3511 write 0x1b6d8b8 thru 0x1b6dfbd (0x706 bytes) 2024-09-04T17:47:01.790 INFO:teuthology.orchestra.run.smithi092.stdout:3512 write 0x28a2702 thru 0x28a7174 (0x4a73 bytes) 2024-09-04T17:47:01.797 INFO:teuthology.orchestra.run.smithi092.stdout:3513 write 0x2bcc685 thru 0x2bd0039 (0x39b5 bytes) 2024-09-04T17:47:01.804 INFO:teuthology.orchestra.run.smithi092.stdout:3514 read 0xc2c9a95 thru 0xc2c9ca1 (0x20d bytes) 2024-09-04T17:47:01.815 INFO:teuthology.orchestra.run.smithi092.stdout:3516 write 0xc95f0cd thru 0xc96c330 (0xd264 bytes) 2024-09-04T17:47:01.834 INFO:teuthology.orchestra.run.smithi092.stdout:3517 trunc from 0xc96c331 to 0x2ce3a2f 2024-09-04T17:47:01.835 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:01.835 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:01.851 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-04T17:47:01.859 INFO:teuthology.orchestra.run.smithi092.stdout:3518 writesame 0x18ad7a1 thru 0x18b6b97 data_size 0x1(0x93f7 bytes) 2024-09-04T17:47:01.873 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:01.871+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:01.941 INFO:teuthology.orchestra.run.smithi092.stdout:3519 writesame 0xc14724b thru 0xc152246 data_size 0x7(0xaffc bytes) 2024-09-04T17:47:01.962 INFO:teuthology.orchestra.run.smithi092.stdout:3520 trunc from 0xc152247 to 0x35f286f 2024-09-04T17:47:01.974 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:47:01.973+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:47:01.977 INFO:teuthology.orchestra.run.smithi092.stdout:3521 read 0xebb105 thru 0xebfb7d (0x4a79 bytes) 2024-09-04T17:47:01.977 INFO:teuthology.orchestra.run.smithi092.stdout:3522 read 0x9b538 thru 0x9c958 (0x1421 bytes) 2024-09-04T17:47:01.979 INFO:teuthology.orchestra.run.smithi092.stdout:3523 trunc from 0x35f286f to 0x2e25d37 2024-09-04T17:47:01.989 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:01.990 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:02.004 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-04T17:47:02.022 INFO:teuthology.orchestra.run.smithi092.stdout:3524 writesame 0x8eaa111 thru 0x8eb07cc data_size 0x19(0x66bc bytes) 2024-09-04T17:47:02.023 INFO:teuthology.orchestra.run.smithi092.stdout:3525 punch from 0x8a9d11c to 0x8aa4a1d, (0x7901 bytes) 2024-09-04T17:47:02.047 INFO:teuthology.orchestra.run.smithi092.stdout:3526 write 0xd78ee9a thru 0xd79df75 (0xf0dc bytes) 2024-09-04T17:47:02.054 INFO:teuthology.orchestra.run.smithi092.stdout:3527 read 0x8f1a639 thru 0x8f2176f (0x7137 bytes) 2024-09-04T17:47:02.055 INFO:teuthology.orchestra.run.smithi092.stdout:3528 write 0x6ab0561 thru 0x6abd63a (0xd0da bytes) 2024-09-04T17:47:02.062 INFO:teuthology.orchestra.run.smithi092.stdout:3530 read 0x1b363ff thru 0x1b43603 (0xd205 bytes) 2024-09-04T17:47:02.062 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-04T17:47:02.064 INFO:teuthology.orchestra.run.smithi092.stdout:3531 read 0x215229f thru 0x215dc58 (0xb9ba bytes) 2024-09-04T17:47:02.064 INFO:teuthology.orchestra.run.smithi092.stdout:3532 write 0xd238973 thru 0xd23be40 (0x34ce bytes) 2024-09-04T17:47:02.070 INFO:teuthology.orchestra.run.smithi092.stdout:3533 write 0x65b229 thru 0x668dab (0xdb83 bytes) 2024-09-04T17:47:02.075 INFO:teuthology.orchestra.run.smithi092.stdout:3536 punch from 0xcf11f90 to 0xcf1cfe1, (0xb051 bytes) 2024-09-04T17:47:02.075 INFO:teuthology.orchestra.run.smithi092.stdout:3537 punch from 0x7c3e1ab to 0x7c3ea4a, (0x89f bytes) 2024-09-04T17:47:02.075 INFO:teuthology.orchestra.run.smithi092.stdout:3538 read 0x2c198a7 thru 0x2c1ba5e (0x21b8 bytes) 2024-09-04T17:47:02.075 INFO:teuthology.orchestra.run.smithi092.stdout:3539 write 0xd6441f1 thru 0xd6515f4 (0xd404 bytes) 2024-09-04T17:47:02.075 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:02.071+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:02.098 INFO:teuthology.orchestra.run.smithi092.stdout:3540 trunc from 0xd79df76 to 0x60cddcf 2024-09-04T17:47:02.115 INFO:teuthology.orchestra.run.smithi092.stdout:3543 punch from 0x2fa4fd9 to 0x2fa96f0, (0x4717 bytes) 2024-09-04T17:47:02.141 INFO:teuthology.orchestra.run.smithi092.stdout:3544 write 0xc4312e4 thru 0xc439924 (0x8641 bytes) 2024-09-04T17:47:02.166 INFO:teuthology.orchestra.run.smithi092.stdout:3545 trunc from 0xc439925 to 0x3f42944 2024-09-04T17:47:02.176 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:47:02.171+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:47:02.177 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:02.178 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:02.185 INFO:teuthology.orchestra.run.smithi092.stdout:3548 read 0x355808d thru 0x356377a (0xb6ee bytes) 2024-09-04T17:47:02.192 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-04T17:47:02.202 INFO:teuthology.orchestra.run.smithi092.stdout:3550 trunc from 0x3f42944 to 0xba8f240 2024-09-04T17:47:02.209 INFO:teuthology.orchestra.run.smithi092.stdout:3551 write 0x6e3d423 thru 0x6e4959b (0xc179 bytes) 2024-09-04T17:47:02.214 INFO:teuthology.orchestra.run.smithi092.stdout:3552 write 0x80aca35 thru 0x80b6064 (0x9630 bytes) 2024-09-04T17:47:02.220 INFO:teuthology.orchestra.run.smithi092.stdout:3553 read 0x70691f2 thru 0x706effb (0x5e0a bytes) 2024-09-04T17:47:02.220 INFO:teuthology.orchestra.run.smithi092.stdout:3555 punch from 0xb4b9c44 to 0xb4c8e97, (0xf253 bytes) 2024-09-04T17:47:02.220 INFO:teuthology.orchestra.run.smithi092.stdout:3556 clone 26 order 23 su 32768 sc 14 2024-09-04T17:47:02.227 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:47:02.227 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:47:02.244 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:47:02.244 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:47:02.260 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-04T17:47:02.260 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-04T17:47:02.276 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-04T17:47:02.277 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-04T17:47:02.277 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:02.275+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:02.295 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:47:02.295 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:47:02.316 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:47:02.316 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:47:02.317 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-04T17:47:02.320 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:02.320 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:47:02.320 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:47:02.320 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:47:02.320 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.320 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:12722 3.1b 3:d8152af1:::rbd_data.1134ef3a5a7d.00000000000000bd:head [delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-04T17:47:02.320 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:16.421965+0000", 2024-09-04T17:47:02.320 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 45.898023189, 2024-09-04T17:47:02.320 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.890002811, 2024-09-04T17:47:02.320 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:02.320 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:02.321 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:02.321 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:02.321 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:02.321 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:02.321 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 12722 2024-09-04T17:47:02.321 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.321 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:02.321 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.321 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:02.322 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421965+0000", 2024-09-04T17:47:02.322 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.322 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.322 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.322 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:02.322 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421965+0000", 2024-09-04T17:47:02.322 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.322 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.322 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.322 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:02.323 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421966+0000", 2024-09-04T17:47:02.323 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.102e-06 2024-09-04T17:47:02.323 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.323 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.323 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:02.323 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421971+0000", 2024-09-04T17:47:02.323 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9989999999999999e-06 2024-09-04T17:47:02.323 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.323 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.323 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:02.323 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421981+0000", 2024-09-04T17:47:02.323 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.7380000000000006e-06 2024-09-04T17:47:02.323 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.323 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.324 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:02.324 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421989+0000", 2024-09-04T17:47:02.324 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.1890000000000007e-06 2024-09-04T17:47:02.324 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.324 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.324 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:02.324 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.422050+0000", 2024-09-04T17:47:02.324 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.0702000000000003e-05 2024-09-04T17:47:02.324 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.324 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.324 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for peered", 2024-09-04T17:47:02.324 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.422059+0000", 2024-09-04T17:47:02.324 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5440000000000004e-06 2024-09-04T17:47:02.325 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.325 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.325 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:02.325 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.308156+0000", 2024-09-04T17:47:02.325 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.88609676199999998 2024-09-04T17:47:02.325 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.325 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.325 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:02.325 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.308398+0000", 2024-09-04T17:47:02.325 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024215400000000001 2024-09-04T17:47:02.325 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.325 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.325 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 7", 2024-09-04T17:47:02.326 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.308795+0000", 2024-09-04T17:47:02.326 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000396704 2024-09-04T17:47:02.326 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.326 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.326 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:47:02.326 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.310831+0000", 2024-09-04T17:47:02.326 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0020364200000000002 2024-09-04T17:47:02.326 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.326 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.326 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:02.326 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.311818+0000", 2024-09-04T17:47:02.326 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00098649799999999993 2024-09-04T17:47:02.326 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.327 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.327 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:47:02.327 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.311905+0000", 2024-09-04T17:47:02.327 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7070999999999994e-05 2024-09-04T17:47:02.327 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.327 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.327 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:02.327 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.311968+0000", 2024-09-04T17:47:02.327 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.2928000000000005e-05 2024-09-04T17:47:02.327 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.327 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:02.327 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.327 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.327 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.328 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37359 3.f e58/24 3:f29e50a5:::rbd_data.1134c5a43465.0000000000000021:head v 58'135, mlcod=58'133)", 2024-09-04T17:47:02.328 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:47:02.328 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.702692585000001, 2024-09-04T17:47:02.328 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053483197000000003, 2024-09-04T17:47:02.328 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:02.328 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:02.328 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:02.328 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:02.328 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.329 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:02.329 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:47:02.329 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.329 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.329 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.329 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:02.329 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:47:02.329 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.329 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.329 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.329 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:02.329 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:47:02.329 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1699999999999997e-07 2024-09-04T17:47:02.330 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.330 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.330 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:02.330 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617338+0000", 2024-09-04T17:47:02.330 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.1168999999999998e-05 2024-09-04T17:47:02.330 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.330 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.330 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:02.330 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617351+0000", 2024-09-04T17:47:02.330 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3094e-05 2024-09-04T17:47:02.330 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.330 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.330 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:02.331 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617359+0000", 2024-09-04T17:47:02.331 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7970000000000006e-06 2024-09-04T17:47:02.331 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.331 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.331 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:02.331 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617417+0000", 2024-09-04T17:47:02.331 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7079e-05 2024-09-04T17:47:02.331 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.331 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.331 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:02.331 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617438+0000", 2024-09-04T17:47:02.331 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1654999999999998e-05 2024-09-04T17:47:02.331 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.331 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.332 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:47:02.332 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670738+0000", 2024-09-04T17:47:02.332 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053300142000000002 2024-09-04T17:47:02.332 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.332 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.332 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:02.332 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670779+0000", 2024-09-04T17:47:02.332 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0343999999999999e-05 2024-09-04T17:47:02.332 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.332 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:02.332 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.332 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.332 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.333 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37358 3.1d e58/24 3:bd3215f3:::rbd_data.1134c5a43465.0000000000000020:head v 58'91, mlcod=58'89)", 2024-09-04T17:47:02.333 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617502+0000", 2024-09-04T17:47:02.333 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.702485867, 2024-09-04T17:47:02.333 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053505641999999999, 2024-09-04T17:47:02.333 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:02.333 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:02.333 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:02.333 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:02.333 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.333 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:02.333 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617502+0000", 2024-09-04T17:47:02.333 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.333 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.333 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.334 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:02.334 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617502+0000", 2024-09-04T17:47:02.334 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.334 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.334 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.334 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:02.334 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617503+0000", 2024-09-04T17:47:02.334 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.184e-06 2024-09-04T17:47:02.334 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.334 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.334 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:02.334 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617526+0000", 2024-09-04T17:47:02.334 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2099999999999998e-05 2024-09-04T17:47:02.334 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.335 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.335 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:02.335 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617540+0000", 2024-09-04T17:47:02.335 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4741e-05 2024-09-04T17:47:02.335 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.335 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.335 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:02.335 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617550+0000", 2024-09-04T17:47:02.335 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4259999999999992e-06 2024-09-04T17:47:02.335 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.335 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.335 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:02.335 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617613+0000", 2024-09-04T17:47:02.336 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.3625000000000002e-05 2024-09-04T17:47:02.336 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.336 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.336 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:02.336 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617637+0000", 2024-09-04T17:47:02.336 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.3604999999999998e-05 2024-09-04T17:47:02.336 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.336 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.336 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:47:02.336 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670967+0000", 2024-09-04T17:47:02.336 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053329881000000003 2024-09-04T17:47:02.336 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.336 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.337 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:02.337 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671008+0000", 2024-09-04T17:47:02.337 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.108e-05 2024-09-04T17:47:02.337 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.337 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:02.337 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.337 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.337 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.337 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37361 3.1c e58/24 3:3c7f8976:::rbd_data.1134c5a43465.0000000000000023:head v 58'159, mlcod=58'157)", 2024-09-04T17:47:02.337 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617578+0000", 2024-09-04T17:47:02.337 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.702410016999998, 2024-09-04T17:47:02.337 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053990367999999997, 2024-09-04T17:47:02.337 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:02.337 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:02.338 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:02.338 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:02.338 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.338 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:02.338 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617578+0000", 2024-09-04T17:47:02.338 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.338 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.338 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.338 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:02.338 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617578+0000", 2024-09-04T17:47:02.338 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.338 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.338 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.339 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:02.339 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617579+0000", 2024-09-04T17:47:02.339 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7299999999999996e-07 2024-09-04T17:47:02.339 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.339 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.339 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:02.339 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617610+0000", 2024-09-04T17:47:02.339 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0951000000000002e-05 2024-09-04T17:47:02.339 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.339 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.339 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:02.339 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:47:02.340 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0002e-05 2024-09-04T17:47:02.340 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.340 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.340 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:02.340 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617626+0000", 2024-09-04T17:47:02.340 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6660000000000002e-06 2024-09-04T17:47:02.340 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.340 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.340 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:02.340 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670548+0000", 2024-09-04T17:47:02.340 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.052921943999999999 2024-09-04T17:47:02.341 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.341 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.341 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:02.341 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670574+0000", 2024-09-04T17:47:02.341 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5405000000000002e-05 2024-09-04T17:47:02.341 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.341 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.341 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:47:02.341 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671522+0000", 2024-09-04T17:47:02.341 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00094815299999999997 2024-09-04T17:47:02.341 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.341 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.341 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:02.342 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671568+0000", 2024-09-04T17:47:02.342 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6674000000000001e-05 2024-09-04T17:47:02.342 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.342 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:02.342 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.342 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.342 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.342 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37357 3.15 e58/49 3:a897312a:::rbd_data.1134c5a43465.000000000000001f:head v 58'117, mlcod=58'115)", 2024-09-04T17:47:02.342 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:47:02.342 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.702367678999998, 2024-09-04T17:47:02.342 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053734297, 2024-09-04T17:47:02.342 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:02.342 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:02.343 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:02.343 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:02.343 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.343 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:02.343 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:47:02.343 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.343 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.343 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.343 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:02.343 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:47:02.343 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.343 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.343 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.343 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:02.344 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617621+0000", 2024-09-04T17:47:02.344 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.7300000000000004e-07 2024-09-04T17:47:02.344 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.344 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.344 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:02.344 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617642+0000", 2024-09-04T17:47:02.344 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0185000000000001e-05 2024-09-04T17:47:02.344 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.344 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.344 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:02.344 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617654+0000", 2024-09-04T17:47:02.344 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2745000000000001e-05 2024-09-04T17:47:02.344 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.345 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.345 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:02.345 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617662+0000", 2024-09-04T17:47:02.345 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.0499999999999992e-06 2024-09-04T17:47:02.345 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.345 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.345 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:02.345 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617727+0000", 2024-09-04T17:47:02.345 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.4923000000000002e-05 2024-09-04T17:47:02.345 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.345 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.345 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:02.345 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617753+0000", 2024-09-04T17:47:02.345 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5899999999999999e-05 2024-09-04T17:47:02.346 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.346 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.346 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:47:02.346 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671298+0000", 2024-09-04T17:47:02.346 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053545276000000003 2024-09-04T17:47:02.346 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.346 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.346 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:02.346 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671355+0000", 2024-09-04T17:47:02.346 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6245000000000001e-05 2024-09-04T17:47:02.346 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.346 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:02.346 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.346 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.347 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.347 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44687 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:02.347 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253012+0000", 2024-09-04T17:47:02.347 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 12.066975612, 2024-09-04T17:47:02.347 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051128777, 2024-09-04T17:47:02.347 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:02.347 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:02.347 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:02.347 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:02.347 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:02.347 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:02.347 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44687 2024-09-04T17:47:02.347 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.347 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:02.348 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.348 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:02.348 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253012+0000", 2024-09-04T17:47:02.348 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.348 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.348 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.348 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:02.348 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253012+0000", 2024-09-04T17:47:02.348 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.348 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.348 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.348 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:02.348 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253014+0000", 2024-09-04T17:47:02.349 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.24e-06 2024-09-04T17:47:02.349 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.349 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.349 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:02.349 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253019+0000", 2024-09-04T17:47:02.349 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4569999999999996e-06 2024-09-04T17:47:02.349 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.349 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.349 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:02.349 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253035+0000", 2024-09-04T17:47:02.349 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6291999999999999e-05 2024-09-04T17:47:02.349 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.349 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.349 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:02.350 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253050+0000", 2024-09-04T17:47:02.350 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4423000000000001e-05 2024-09-04T17:47:02.350 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.350 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.350 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:02.350 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304115+0000", 2024-09-04T17:47:02.350 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051064778999999998 2024-09-04T17:47:02.350 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.350 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.350 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:02.350 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304141+0000", 2024-09-04T17:47:02.350 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6585999999999999e-05 2024-09-04T17:47:02.350 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.351 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:02.351 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.351 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.351 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.351 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44688 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:02.351 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253067+0000", 2024-09-04T17:47:02.351 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 12.06692106, 2024-09-04T17:47:02.351 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051116979, 2024-09-04T17:47:02.351 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:02.351 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:02.351 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:02.351 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:02.351 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:02.351 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:02.352 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44688 2024-09-04T17:47:02.352 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.352 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:02.352 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.352 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:02.352 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253067+0000", 2024-09-04T17:47:02.352 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.352 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.352 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.352 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:02.352 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253067+0000", 2024-09-04T17:47:02.352 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.353 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.353 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.353 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:02.353 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253068+0000", 2024-09-04T17:47:02.353 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0729999999999999e-06 2024-09-04T17:47:02.353 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.353 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.353 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:02.353 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253073+0000", 2024-09-04T17:47:02.353 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2259999999999996e-06 2024-09-04T17:47:02.353 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.353 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.353 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:02.354 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253091+0000", 2024-09-04T17:47:02.354 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7255999999999999e-05 2024-09-04T17:47:02.354 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.354 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.354 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:02.354 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253103+0000", 2024-09-04T17:47:02.354 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2632999999999999e-05 2024-09-04T17:47:02.354 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.354 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.354 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:02.354 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304173+0000", 2024-09-04T17:47:02.354 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051069537999999998 2024-09-04T17:47:02.354 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.354 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.355 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:02.355 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304184+0000", 2024-09-04T17:47:02.355 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1253000000000001e-05 2024-09-04T17:47:02.355 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.355 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:02.355 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.355 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.355 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.355 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44689 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:02.355 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253121+0000", 2024-09-04T17:47:02.355 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 12.066866922999999, 2024-09-04T17:47:02.355 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051192595, 2024-09-04T17:47:02.355 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:02.356 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:02.356 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:02.356 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:02.356 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:02.356 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:02.356 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44689 2024-09-04T17:47:02.356 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.356 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:02.356 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.356 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:02.356 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253121+0000", 2024-09-04T17:47:02.356 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.356 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.356 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.357 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:02.357 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253121+0000", 2024-09-04T17:47:02.357 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.357 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.357 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.357 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:02.357 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253122+0000", 2024-09-04T17:47:02.357 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.15e-06 2024-09-04T17:47:02.357 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.357 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.357 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:02.357 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253128+0000", 2024-09-04T17:47:02.357 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2569999999999998e-06 2024-09-04T17:47:02.358 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.358 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.358 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:02.358 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253145+0000", 2024-09-04T17:47:02.358 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7176e-05 2024-09-04T17:47:02.358 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.358 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.358 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:02.358 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253158+0000", 2024-09-04T17:47:02.358 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3139000000000001e-05 2024-09-04T17:47:02.358 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.358 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.358 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:02.359 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304287+0000", 2024-09-04T17:47:02.359 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051129149999999998 2024-09-04T17:47:02.359 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.359 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.359 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:02.359 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304314+0000", 2024-09-04T17:47:02.359 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6723000000000001e-05 2024-09-04T17:47:02.359 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.359 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:02.359 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.359 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.359 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.359 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44690 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:02.360 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253177+0000", 2024-09-04T17:47:02.360 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 12.066810798000001, 2024-09-04T17:47:02.360 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051169263, 2024-09-04T17:47:02.360 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:02.360 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:02.360 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:02.360 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:02.360 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:02.360 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:02.360 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44690 2024-09-04T17:47:02.360 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.360 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:02.360 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.361 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:02.361 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253177+0000", 2024-09-04T17:47:02.361 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.361 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.361 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.361 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:02.361 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253177+0000", 2024-09-04T17:47:02.361 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.361 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.361 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.361 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:02.361 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253178+0000", 2024-09-04T17:47:02.361 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1000000000000001e-06 2024-09-04T17:47:02.362 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.362 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.362 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:02.362 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253184+0000", 2024-09-04T17:47:02.362 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2089999999999996e-06 2024-09-04T17:47:02.362 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.362 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.362 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:02.362 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253200+0000", 2024-09-04T17:47:02.362 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6087e-05 2024-09-04T17:47:02.362 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.362 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.362 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:02.362 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253231+0000", 2024-09-04T17:47:02.363 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0874999999999998e-05 2024-09-04T17:47:02.363 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.363 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.363 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:02.363 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304337+0000", 2024-09-04T17:47:02.363 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051106430000000001 2024-09-04T17:47:02.363 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.363 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.363 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:02.363 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304347+0000", 2024-09-04T17:47:02.363 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5619999999999993e-06 2024-09-04T17:47:02.363 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.363 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:02.363 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.364 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.364 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.364 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44691 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:02.364 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253249+0000", 2024-09-04T17:47:02.364 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 12.066738764, 2024-09-04T17:47:02.364 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051195630999999998, 2024-09-04T17:47:02.364 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:02.364 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:02.364 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:02.364 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:02.364 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:02.364 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:02.365 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44691 2024-09-04T17:47:02.365 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.365 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:02.365 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.365 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:02.365 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253249+0000", 2024-09-04T17:47:02.365 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.365 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.365 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.365 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:02.365 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253249+0000", 2024-09-04T17:47:02.365 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.365 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.366 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.366 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:02.366 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253251+0000", 2024-09-04T17:47:02.366 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2330000000000001e-06 2024-09-04T17:47:02.366 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.366 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.366 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:02.366 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253256+0000", 2024-09-04T17:47:02.366 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6099999999999997e-06 2024-09-04T17:47:02.366 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.366 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.366 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:02.366 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253274+0000", 2024-09-04T17:47:02.367 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7952999999999999e-05 2024-09-04T17:47:02.367 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.367 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.367 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:02.367 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253287+0000", 2024-09-04T17:47:02.367 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2839e-05 2024-09-04T17:47:02.367 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.367 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.367 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:02.367 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304396+0000", 2024-09-04T17:47:02.367 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051108990999999999 2024-09-04T17:47:02.367 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.367 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.367 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:02.368 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304445+0000", 2024-09-04T17:47:02.368 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9005000000000003e-05 2024-09-04T17:47:02.368 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.368 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:02.368 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.368 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.368 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.368 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44692 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:02.368 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253337+0000", 2024-09-04T17:47:02.368 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 12.066651191, 2024-09-04T17:47:02.368 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051138909000000003, 2024-09-04T17:47:02.368 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:02.368 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:02.369 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:02.369 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:02.369 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:02.369 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:02.369 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44692 2024-09-04T17:47:02.369 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.369 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:02.369 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.369 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:02.369 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253337+0000", 2024-09-04T17:47:02.369 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.369 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.369 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.370 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:02.370 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253337+0000", 2024-09-04T17:47:02.370 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.370 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.370 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.370 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:02.370 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253338+0000", 2024-09-04T17:47:02.370 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.463e-06 2024-09-04T17:47:02.370 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.370 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.370 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:02.370 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253357+0000", 2024-09-04T17:47:02.370 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9043e-05 2024-09-04T17:47:02.371 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.371 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.371 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:02.371 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253379+0000", 2024-09-04T17:47:02.371 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1243e-05 2024-09-04T17:47:02.371 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.371 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.371 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:02.371 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253396+0000", 2024-09-04T17:47:02.371 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7059e-05 2024-09-04T17:47:02.371 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.371 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.371 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:02.371 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304466+0000", 2024-09-04T17:47:02.372 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051070624000000002 2024-09-04T17:47:02.372 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.372 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.372 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:02.372 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304476+0000", 2024-09-04T17:47:02.372 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4769999999999992e-06 2024-09-04T17:47:02.372 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.372 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:02.372 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.372 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.372 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.372 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44693 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:02.372 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253415+0000", 2024-09-04T17:47:02.373 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 12.066572788, 2024-09-04T17:47:02.373 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051141949999999999, 2024-09-04T17:47:02.373 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:02.373 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:02.373 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:02.373 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:02.373 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:02.373 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:02.373 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44693 2024-09-04T17:47:02.373 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.373 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:02.373 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.373 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:02.373 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253415+0000", 2024-09-04T17:47:02.374 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.374 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.374 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.374 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:02.374 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253415+0000", 2024-09-04T17:47:02.374 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.374 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.374 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.374 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:02.374 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253416+0000", 2024-09-04T17:47:02.374 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1799999999999999e-06 2024-09-04T17:47:02.374 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.375 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.375 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:02.375 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253422+0000", 2024-09-04T17:47:02.375 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.5940000000000001e-06 2024-09-04T17:47:02.375 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.375 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.375 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:02.375 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253438+0000", 2024-09-04T17:47:02.375 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6008999999999999e-05 2024-09-04T17:47:02.375 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.375 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.375 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:02.375 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253452+0000", 2024-09-04T17:47:02.376 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3543000000000001e-05 2024-09-04T17:47:02.376 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.376 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.376 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:02.376 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304533+0000", 2024-09-04T17:47:02.376 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051081609 2024-09-04T17:47:02.376 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.376 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.376 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:02.376 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304557+0000", 2024-09-04T17:47:02.376 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4015e-05 2024-09-04T17:47:02.376 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.376 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:02.376 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.377 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.377 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.377 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44694 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:02.377 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253469+0000", 2024-09-04T17:47:02.377 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 12.066518663, 2024-09-04T17:47:02.377 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051121486000000001, 2024-09-04T17:47:02.377 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:02.377 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:02.377 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:02.377 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:02.377 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:02.377 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:02.377 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44694 2024-09-04T17:47:02.378 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.378 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:02.378 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.378 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:02.378 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253469+0000", 2024-09-04T17:47:02.378 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.378 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.378 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.378 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:02.378 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253469+0000", 2024-09-04T17:47:02.378 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.378 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.378 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.378 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:02.378 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253471+0000", 2024-09-04T17:47:02.379 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1400000000000001e-06 2024-09-04T17:47:02.379 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.379 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.379 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:02.379 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253476+0000", 2024-09-04T17:47:02.379 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7370000000000003e-06 2024-09-04T17:47:02.379 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.379 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.379 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:02.379 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253493+0000", 2024-09-04T17:47:02.379 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6549e-05 2024-09-04T17:47:02.379 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.379 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.380 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:02.380 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253506+0000", 2024-09-04T17:47:02.380 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3406e-05 2024-09-04T17:47:02.380 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.380 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.380 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:02.380 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304582+0000", 2024-09-04T17:47:02.380 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051075534999999998 2024-09-04T17:47:02.380 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.380 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.380 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:02.380 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304591+0000", 2024-09-04T17:47:02.380 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1190000000000003e-06 2024-09-04T17:47:02.381 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.381 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:02.381 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.381 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.381 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.381 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44695 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:02.381 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253552+0000", 2024-09-04T17:47:02.381 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 12.066435909000001, 2024-09-04T17:47:02.381 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051068872000000001, 2024-09-04T17:47:02.381 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:02.381 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:02.381 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:02.381 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:02.381 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:02.382 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:02.382 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44695 2024-09-04T17:47:02.382 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.382 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:02.382 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.382 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:02.382 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253552+0000", 2024-09-04T17:47:02.382 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.382 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.382 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.382 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:02.382 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253552+0000", 2024-09-04T17:47:02.383 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.383 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.383 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.383 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:02.383 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253553+0000", 2024-09-04T17:47:02.383 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1960000000000001e-06 2024-09-04T17:47:02.383 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.383 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.383 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:02.383 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253560+0000", 2024-09-04T17:47:02.383 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.2040000000000002e-06 2024-09-04T17:47:02.383 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.383 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.383 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:02.384 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253578+0000", 2024-09-04T17:47:02.384 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8241999999999999e-05 2024-09-04T17:47:02.384 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.384 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.384 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:02.384 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253591+0000", 2024-09-04T17:47:02.384 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3453e-05 2024-09-04T17:47:02.384 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.384 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.384 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:02.384 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304614+0000", 2024-09-04T17:47:02.384 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051022456000000001 2024-09-04T17:47:02.384 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.385 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.385 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:02.385 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304621+0000", 2024-09-04T17:47:02.385 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3209999999999999e-06 2024-09-04T17:47:02.385 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.385 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:02.385 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.385 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.385 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.385 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44696 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:02.385 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253609+0000", 2024-09-04T17:47:02.385 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 12.066379408, 2024-09-04T17:47:02.385 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051115351000000003, 2024-09-04T17:47:02.386 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:02.386 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:02.386 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:02.386 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:02.386 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:02.386 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:02.386 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44696 2024-09-04T17:47:02.386 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.386 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:02.386 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.386 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:02.386 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253609+0000", 2024-09-04T17:47:02.386 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.387 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.387 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.387 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:02.387 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253609+0000", 2024-09-04T17:47:02.387 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.387 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.387 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.387 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:02.387 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253610+0000", 2024-09-04T17:47:02.387 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2300000000000001e-06 2024-09-04T17:47:02.387 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.387 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.387 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:02.387 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253615+0000", 2024-09-04T17:47:02.388 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7260000000000002e-06 2024-09-04T17:47:02.388 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.388 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.388 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:02.388 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253632+0000", 2024-09-04T17:47:02.388 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7039999999999999e-05 2024-09-04T17:47:02.388 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.388 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.388 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:02.388 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253666+0000", 2024-09-04T17:47:02.388 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4662000000000001e-05 2024-09-04T17:47:02.388 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.388 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.389 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:02.389 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304677+0000", 2024-09-04T17:47:02.389 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051011017999999998 2024-09-04T17:47:02.389 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.389 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.389 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:02.389 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304724+0000", 2024-09-04T17:47:02.389 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6675000000000003e-05 2024-09-04T17:47:02.389 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.389 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:02.389 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.389 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.389 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.390 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44697 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:02.390 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253684+0000", 2024-09-04T17:47:02.390 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 12.066303914000001, 2024-09-04T17:47:02.390 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051073107, 2024-09-04T17:47:02.390 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:02.390 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:02.390 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:02.390 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:02.390 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:02.390 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:02.390 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44697 2024-09-04T17:47:02.390 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.390 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:02.391 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.391 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:02.391 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253684+0000", 2024-09-04T17:47:02.391 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.391 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.391 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.391 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:02.391 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253684+0000", 2024-09-04T17:47:02.391 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.391 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.391 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.391 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:02.391 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253685+0000", 2024-09-04T17:47:02.391 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0699999999999999e-06 2024-09-04T17:47:02.392 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.392 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.392 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:02.392 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253690+0000", 2024-09-04T17:47:02.392 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1959999999999999e-06 2024-09-04T17:47:02.392 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.392 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.392 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:02.392 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253707+0000", 2024-09-04T17:47:02.392 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6946e-05 2024-09-04T17:47:02.392 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.392 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.392 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:02.393 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253720+0000", 2024-09-04T17:47:02.393 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3076e-05 2024-09-04T17:47:02.393 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.393 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.393 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:02.393 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304745+0000", 2024-09-04T17:47:02.393 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051024760000000002 2024-09-04T17:47:02.393 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.393 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.393 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:02.393 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304757+0000", 2024-09-04T17:47:02.393 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2058999999999999e-05 2024-09-04T17:47:02.393 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.393 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:02.394 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.394 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.394 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.394 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44698 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:02.394 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253737+0000", 2024-09-04T17:47:02.394 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 12.066251563, 2024-09-04T17:47:02.394 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051176078, 2024-09-04T17:47:02.394 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:02.394 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:02.394 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:02.394 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:02.394 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:02.394 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:02.394 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44698 2024-09-04T17:47:02.395 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.395 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:02.395 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.395 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:02.395 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253737+0000", 2024-09-04T17:47:02.395 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.395 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.395 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.395 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:02.395 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253737+0000", 2024-09-04T17:47:02.395 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.395 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.395 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.396 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:02.396 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253738+0000", 2024-09-04T17:47:02.396 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0300000000000001e-06 2024-09-04T17:47:02.396 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.396 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.396 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:02.396 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253752+0000", 2024-09-04T17:47:02.396 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4358999999999999e-05 2024-09-04T17:47:02.396 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.396 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.396 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:02.396 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253771+0000", 2024-09-04T17:47:02.396 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8879999999999999e-05 2024-09-04T17:47:02.397 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.397 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.397 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:02.397 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253785+0000", 2024-09-04T17:47:02.397 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3852000000000001e-05 2024-09-04T17:47:02.397 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.397 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.397 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:02.397 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304876+0000", 2024-09-04T17:47:02.397 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051091105999999997 2024-09-04T17:47:02.397 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.397 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.398 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:02.398 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304913+0000", 2024-09-04T17:47:02.398 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.6850999999999997e-05 2024-09-04T17:47:02.398 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.398 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:02.398 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.398 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.398 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.398 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44699 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:02.398 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253804+0000", 2024-09-04T17:47:02.398 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 12.066184038999999, 2024-09-04T17:47:02.398 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051172250000000002, 2024-09-04T17:47:02.398 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:02.399 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:02.399 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:02.399 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:02.399 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:02.399 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:02.399 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44699 2024-09-04T17:47:02.399 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.399 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:02.399 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.399 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:02.399 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253804+0000", 2024-09-04T17:47:02.399 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.399 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.399 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.400 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:02.400 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253804+0000", 2024-09-04T17:47:02.400 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.400 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.400 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.400 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:02.400 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253805+0000", 2024-09-04T17:47:02.400 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.08e-06 2024-09-04T17:47:02.400 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.400 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.400 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:02.400 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253810+0000", 2024-09-04T17:47:02.400 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1830000000000002e-06 2024-09-04T17:47:02.400 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.401 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.401 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:02.401 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253828+0000", 2024-09-04T17:47:02.401 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7662000000000001e-05 2024-09-04T17:47:02.401 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.401 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.401 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:02.401 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253841+0000", 2024-09-04T17:47:02.401 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3203999999999999e-05 2024-09-04T17:47:02.401 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.401 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.401 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:02.401 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304966+0000", 2024-09-04T17:47:02.402 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051125273999999998 2024-09-04T17:47:02.402 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.402 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.402 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:02.402 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304976+0000", 2024-09-04T17:47:02.402 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.8470000000000008e-06 2024-09-04T17:47:02.402 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.402 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:02.402 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.402 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.402 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.402 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44700 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:02.402 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253858+0000", 2024-09-04T17:47:02.403 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 12.066130384999999, 2024-09-04T17:47:02.403 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051218957000000002, 2024-09-04T17:47:02.403 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:02.403 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:02.403 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:02.403 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:02.403 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:02.403 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:02.403 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44700 2024-09-04T17:47:02.403 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.403 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:02.403 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.403 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:02.403 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253858+0000", 2024-09-04T17:47:02.404 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.404 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.404 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.404 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:02.404 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253858+0000", 2024-09-04T17:47:02.404 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.404 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.404 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.404 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:02.404 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253859+0000", 2024-09-04T17:47:02.404 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.217e-06 2024-09-04T17:47:02.404 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.404 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.405 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:02.405 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253864+0000", 2024-09-04T17:47:02.405 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7600000000000002e-06 2024-09-04T17:47:02.405 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.405 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.405 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:02.405 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253880+0000", 2024-09-04T17:47:02.405 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6501999999999999e-05 2024-09-04T17:47:02.405 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.405 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.405 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:02.405 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253899+0000", 2024-09-04T17:47:02.405 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8907e-05 2024-09-04T17:47:02.405 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.406 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.406 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:02.406 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305052+0000", 2024-09-04T17:47:02.406 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051152989000000003 2024-09-04T17:47:02.406 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.406 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.406 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:02.406 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305077+0000", 2024-09-04T17:47:02.406 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4581999999999999e-05 2024-09-04T17:47:02.406 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.406 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:02.406 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.406 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.406 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.407 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44701 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:02.407 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253917+0000", 2024-09-04T17:47:02.407 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 12.066071033, 2024-09-04T17:47:02.407 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051231113000000002, 2024-09-04T17:47:02.407 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:02.407 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:02.407 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:02.407 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:02.407 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:02.407 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:02.407 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44701 2024-09-04T17:47:02.407 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.407 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:02.407 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.408 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:02.408 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253917+0000", 2024-09-04T17:47:02.408 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.408 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.408 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.408 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:02.408 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253917+0000", 2024-09-04T17:47:02.408 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.408 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.408 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.408 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:02.408 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253918+0000", 2024-09-04T17:47:02.408 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.136e-06 2024-09-04T17:47:02.409 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.409 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.409 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:02.409 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253924+0000", 2024-09-04T17:47:02.409 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.3070000000000002e-06 2024-09-04T17:47:02.409 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.409 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.409 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:02.409 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253940+0000", 2024-09-04T17:47:02.409 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6612e-05 2024-09-04T17:47:02.409 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.409 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.409 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:02.409 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253954+0000", 2024-09-04T17:47:02.410 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3709999999999999e-05 2024-09-04T17:47:02.410 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.410 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.410 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:02.410 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305136+0000", 2024-09-04T17:47:02.410 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051181935999999997 2024-09-04T17:47:02.410 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.410 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.410 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:02.410 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305148+0000", 2024-09-04T17:47:02.410 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2412000000000001e-05 2024-09-04T17:47:02.410 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.410 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:02.411 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.411 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.411 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:02.411 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:02.412 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:47:02.413 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-04T17:47:02.413 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-04T17:47:02.413 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:02.413+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:02.414 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:47:02.414 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-04T17:47:02.514 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:02.511+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:02.583 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:02.583 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:47:02.583 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:47:02.583 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:02.597 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-04T17:47:02.640 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:02.641+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:02.716 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:02.713+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:02.725 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:02.725 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:47:02.725 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:47:02.725 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:47:02.725 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:02.741 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-04T17:47:02.795 INFO:teuthology.orchestra.run.smithi092.stdout:flattening image image_client.0-clone25 2024-09-04T17:47:02.817 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:02.815+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:02.909 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:02.909 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:47:02.909 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:47:02.910 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:47:02.910 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.910 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8831 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2453504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:02.910 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:47:02.910 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 49.850496548999999, 2024-09-04T17:47:02.910 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14731111799999999, 2024-09-04T17:47:02.910 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:02.910 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:02.910 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:02.910 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:02.910 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:02.910 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:02.911 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8831 2024-09-04T17:47:02.911 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.911 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:02.911 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.911 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:02.911 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:47:02.911 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.911 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.911 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.911 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:02.911 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:47:02.911 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.911 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.911 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.912 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:02.912 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057811+0000", 2024-09-04T17:47:02.912 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.083e-06 2024-09-04T17:47:02.912 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.912 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.912 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:02.912 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057814+0000", 2024-09-04T17:47:02.912 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.407e-06 2024-09-04T17:47:02.912 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.912 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.912 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:02.912 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057825+0000", 2024-09-04T17:47:02.912 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.17e-05 2024-09-04T17:47:02.913 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.913 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.913 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:02.913 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057836+0000", 2024-09-04T17:47:02.913 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0283e-05 2024-09-04T17:47:02.913 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.913 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.913 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:02.913 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.204762+0000", 2024-09-04T17:47:02.913 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.146926205 2024-09-04T17:47:02.913 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.913 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.913 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:02.914 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.204895+0000", 2024-09-04T17:47:02.914 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013328999999999999 2024-09-04T17:47:02.914 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.914 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.914 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:02.914 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205121+0000", 2024-09-04T17:47:02.914 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00022614999999999999 2024-09-04T17:47:02.914 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.914 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:02.914 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.914 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.914 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.914 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8832 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2490368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:02.915 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:47:02.915 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 49.850458516000003, 2024-09-04T17:47:02.915 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14770012299999999, 2024-09-04T17:47:02.915 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:02.915 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:02.915 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:02.915 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:02.915 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:02.915 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:02.915 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8832 2024-09-04T17:47:02.915 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.915 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:02.915 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.916 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:02.916 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:47:02.916 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.916 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.916 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.916 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:02.916 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:47:02.916 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.916 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.916 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.916 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:02.916 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057849+0000", 2024-09-04T17:47:02.916 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6700000000000003e-07 2024-09-04T17:47:02.916 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.917 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.917 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:02.917 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057852+0000", 2024-09-04T17:47:02.917 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6359999999999998e-06 2024-09-04T17:47:02.917 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.917 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.917 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:02.917 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057860+0000", 2024-09-04T17:47:02.917 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8140000000000007e-06 2024-09-04T17:47:02.917 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.917 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.917 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:02.917 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057868+0000", 2024-09-04T17:47:02.918 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1999999999999997e-06 2024-09-04T17:47:02.918 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.918 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.918 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:02.918 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205208+0000", 2024-09-04T17:47:02.918 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14734067300000001 2024-09-04T17:47:02.918 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.918 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.918 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:02.918 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205347+0000", 2024-09-04T17:47:02.918 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013914900000000001 2024-09-04T17:47:02.918 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.918 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.919 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:02.919 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205548+0000", 2024-09-04T17:47:02.919 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000200984 2024-09-04T17:47:02.919 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.919 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:02.919 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.919 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.919 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.919 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8833 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2527232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:02.919 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:47:02.919 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 49.850426646000003, 2024-09-04T17:47:02.919 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148062572, 2024-09-04T17:47:02.919 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:02.919 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:02.920 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:02.920 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:02.920 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:02.920 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:02.920 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8833 2024-09-04T17:47:02.920 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.920 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:02.920 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.920 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:02.920 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:47:02.921 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.921 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.921 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.921 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:02.921 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:47:02.921 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.921 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.921 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.921 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:02.921 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057881+0000", 2024-09-04T17:47:02.921 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4300000000000002e-07 2024-09-04T17:47:02.921 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.921 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.922 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:02.922 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057893+0000", 2024-09-04T17:47:02.922 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1973999999999999e-05 2024-09-04T17:47:02.922 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.922 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.922 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:02.922 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057906+0000", 2024-09-04T17:47:02.922 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2716e-05 2024-09-04T17:47:02.922 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.922 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.922 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:02.922 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057913+0000", 2024-09-04T17:47:02.922 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3930000000000002e-06 2024-09-04T17:47:02.922 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.923 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.923 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:02.923 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205621+0000", 2024-09-04T17:47:02.923 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14770788800000001 2024-09-04T17:47:02.923 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.923 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.923 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:02.923 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205740+0000", 2024-09-04T17:47:02.923 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000118936 2024-09-04T17:47:02.923 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.923 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.923 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:02.923 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205943+0000", 2024-09-04T17:47:02.923 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020292199999999999 2024-09-04T17:47:02.924 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.924 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:02.924 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.924 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.924 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.924 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8834 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2564096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:02.924 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:47:02.924 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 49.850383366999999, 2024-09-04T17:47:02.924 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14846753200000001, 2024-09-04T17:47:02.924 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:02.924 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:02.924 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:02.925 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:02.925 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:02.925 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:02.925 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8834 2024-09-04T17:47:02.925 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.925 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:02.925 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.925 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:02.925 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:47:02.926 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.926 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.926 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.926 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:02.926 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:47:02.926 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.926 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.926 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.926 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:02.926 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057925+0000", 2024-09-04T17:47:02.926 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0270000000000001e-06 2024-09-04T17:47:02.926 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.926 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.927 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:02.927 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057936+0000", 2024-09-04T17:47:02.927 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.112e-05 2024-09-04T17:47:02.927 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.927 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.927 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:02.927 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057946+0000", 2024-09-04T17:47:02.927 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.009e-05 2024-09-04T17:47:02.927 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.927 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.927 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:02.927 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057953+0000", 2024-09-04T17:47:02.927 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.6229999999999997e-06 2024-09-04T17:47:02.928 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.928 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.928 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:02.928 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206027+0000", 2024-09-04T17:47:02.928 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148073591 2024-09-04T17:47:02.928 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.928 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.928 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:02.928 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206161+0000", 2024-09-04T17:47:02.928 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013411599999999999 2024-09-04T17:47:02.928 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.928 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.928 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:02.929 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206391+0000", 2024-09-04T17:47:02.929 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000229965 2024-09-04T17:47:02.929 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.929 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:02.929 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.929 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.929 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.929 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8835 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2600960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:02.929 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:47:02.929 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 49.850343717000001, 2024-09-04T17:47:02.929 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14882941599999999, 2024-09-04T17:47:02.929 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:02.929 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:02.929 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:02.930 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:02.930 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:02.930 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:02.930 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8835 2024-09-04T17:47:02.930 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.930 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:02.930 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.930 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:02.930 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:47:02.930 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.930 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.930 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.930 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:02.930 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:47:02.931 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.931 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.931 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.931 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:02.931 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058544+0000", 2024-09-04T17:47:02.931 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00058061899999999999 2024-09-04T17:47:02.931 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.931 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.931 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:02.931 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058550+0000", 2024-09-04T17:47:02.931 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.1129999999999997e-06 2024-09-04T17:47:02.931 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.931 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.932 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:02.932 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058566+0000", 2024-09-04T17:47:02.932 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5820000000000001e-05 2024-09-04T17:47:02.932 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.932 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.932 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:02.932 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058578+0000", 2024-09-04T17:47:02.932 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2537e-05 2024-09-04T17:47:02.932 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.932 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.932 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:02.932 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206470+0000", 2024-09-04T17:47:02.932 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14789153799999999 2024-09-04T17:47:02.933 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.933 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.933 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:02.933 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206589+0000", 2024-09-04T17:47:02.933 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000118837 2024-09-04T17:47:02.933 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.933 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.933 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:02.933 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206793+0000", 2024-09-04T17:47:02.933 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000203952 2024-09-04T17:47:02.933 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.933 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:02.933 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.933 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.934 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.934 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8836 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2654208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:02.934 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:47:02.934 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 49.849711028999998, 2024-09-04T17:47:02.934 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14862433999999999, 2024-09-04T17:47:02.934 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:02.934 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:02.934 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:02.934 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:02.934 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:02.934 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:02.934 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8836 2024-09-04T17:47:02.934 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.935 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:02.935 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.935 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:02.935 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:47:02.935 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.935 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.935 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.935 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:02.935 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:47:02.935 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.935 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.935 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.935 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:02.935 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062663+0000", 2024-09-04T17:47:02.936 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0040670619999999998 2024-09-04T17:47:02.936 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.936 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.936 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:02.936 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062668+0000", 2024-09-04T17:47:02.936 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4199999999999998e-06 2024-09-04T17:47:02.936 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.936 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.936 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:02.936 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062689+0000", 2024-09-04T17:47:02.936 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0356999999999998e-05 2024-09-04T17:47:02.936 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.936 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.937 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:02.937 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062726+0000", 2024-09-04T17:47:02.937 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7302999999999998e-05 2024-09-04T17:47:02.937 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.937 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.937 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:02.937 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206875+0000", 2024-09-04T17:47:02.937 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14414883100000001 2024-09-04T17:47:02.937 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.937 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.937 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:02.937 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.207016+0000", 2024-09-04T17:47:02.937 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000140822 2024-09-04T17:47:02.937 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.938 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.938 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:02.938 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.207220+0000", 2024-09-04T17:47:02.938 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020454499999999999 2024-09-04T17:47:02.938 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.938 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:02.938 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.938 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.938 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.938 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8844 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2928640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:02.938 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:47:02.938 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 49.845346945999999, 2024-09-04T17:47:02.938 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14752153100000001, 2024-09-04T17:47:02.938 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:02.938 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:02.939 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:02.939 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:02.939 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:02.939 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:02.939 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8844 2024-09-04T17:47:02.939 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.939 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:02.939 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.939 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:02.939 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:47:02.939 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.939 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.939 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.939 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:02.940 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:47:02.940 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.940 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.940 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.940 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:02.940 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062961+0000", 2024-09-04T17:47:02.940 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6700000000000003e-07 2024-09-04T17:47:02.940 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.940 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.940 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:02.940 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062973+0000", 2024-09-04T17:47:02.940 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2109999999999999e-05 2024-09-04T17:47:02.940 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.940 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.941 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:02.941 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062982+0000", 2024-09-04T17:47:02.941 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.6900000000000004e-06 2024-09-04T17:47:02.941 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.941 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.941 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:02.941 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062990+0000", 2024-09-04T17:47:02.941 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1899999999999998e-06 2024-09-04T17:47:02.941 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.941 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.941 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:02.941 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210090+0000", 2024-09-04T17:47:02.941 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14710026300000001 2024-09-04T17:47:02.941 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.942 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.942 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:02.942 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210225+0000", 2024-09-04T17:47:02.942 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013532400000000001 2024-09-04T17:47:02.942 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.942 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.942 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:02.942 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210481+0000", 2024-09-04T17:47:02.942 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00025628700000000001 2024-09-04T17:47:02.942 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.942 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:02.942 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.942 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.943 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.943 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8845 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2965504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:02.943 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:47:02.943 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 49.845307222999999, 2024-09-04T17:47:02.943 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147916819, 2024-09-04T17:47:02.943 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:02.943 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:02.943 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:02.943 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:02.943 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:02.943 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:02.943 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8845 2024-09-04T17:47:02.943 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.943 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:02.944 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.944 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:02.944 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:47:02.944 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.944 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.944 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.944 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:02.944 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:47:02.944 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.944 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.944 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.944 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:02.944 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:47:02.945 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.0399999999999995e-07 2024-09-04T17:47:02.945 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.945 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.945 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:02.945 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063003+0000", 2024-09-04T17:47:02.945 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6529999999999998e-06 2024-09-04T17:47:02.945 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.945 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.945 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:02.945 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063012+0000", 2024-09-04T17:47:02.945 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9500000000000007e-06 2024-09-04T17:47:02.945 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.945 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.946 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:02.946 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063027+0000", 2024-09-04T17:47:02.946 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5443000000000001e-05 2024-09-04T17:47:02.946 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.946 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.946 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:02.946 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210571+0000", 2024-09-04T17:47:02.946 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14754353100000001 2024-09-04T17:47:02.946 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.947 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.948 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:02.948 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210714+0000", 2024-09-04T17:47:02.948 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014295599999999999 2024-09-04T17:47:02.948 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.948 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.948 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:02.948 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210916+0000", 2024-09-04T17:47:02.948 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020258200000000001 2024-09-04T17:47:02.948 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.948 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:02.948 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.948 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.949 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.949 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8846 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3002368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:02.949 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:47:02.949 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 49.845269686000002, 2024-09-04T17:47:02.949 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14828011599999999, 2024-09-04T17:47:02.949 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:02.949 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:02.949 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:02.949 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:02.949 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:02.949 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:02.949 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8846 2024-09-04T17:47:02.949 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.950 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:02.950 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.950 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:02.950 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:47:02.950 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.950 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.950 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.950 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:02.950 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:47:02.950 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.950 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.950 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.950 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:02.950 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063038+0000", 2024-09-04T17:47:02.950 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.23e-07 2024-09-04T17:47:02.950 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.951 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.951 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:02.951 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063051+0000", 2024-09-04T17:47:02.951 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2757e-05 2024-09-04T17:47:02.951 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.951 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.951 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:02.951 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063070+0000", 2024-09-04T17:47:02.951 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9663000000000001e-05 2024-09-04T17:47:02.951 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.951 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.951 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:02.951 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063079+0000", 2024-09-04T17:47:02.951 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.0929999999999992e-06 2024-09-04T17:47:02.952 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.952 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.952 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:02.952 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210989+0000", 2024-09-04T17:47:02.952 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14790984500000001 2024-09-04T17:47:02.952 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.952 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.952 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:02.952 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211109+0000", 2024-09-04T17:47:02.952 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000120056 2024-09-04T17:47:02.952 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.952 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.952 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:02.952 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211317+0000", 2024-09-04T17:47:02.953 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020797900000000001 2024-09-04T17:47:02.953 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.953 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:02.953 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.953 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.953 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.953 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8847 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3039232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:02.953 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:47:02.953 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 49.845195253999997, 2024-09-04T17:47:02.953 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14859408900000001, 2024-09-04T17:47:02.953 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:02.953 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:02.953 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:02.953 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:02.954 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:02.954 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:02.954 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8847 2024-09-04T17:47:02.954 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.954 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:02.954 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.954 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:02.954 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:47:02.954 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.954 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.954 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.954 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:02.954 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:47:02.954 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.955 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.955 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.955 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:02.955 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063113+0000", 2024-09-04T17:47:02.955 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.204e-06 2024-09-04T17:47:02.955 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.955 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.955 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:02.955 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063116+0000", 2024-09-04T17:47:02.955 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.3330000000000001e-06 2024-09-04T17:47:02.955 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.955 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.955 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:02.955 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063126+0000", 2024-09-04T17:47:02.956 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0166999999999999e-05 2024-09-04T17:47:02.956 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.956 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.956 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:02.956 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063134+0000", 2024-09-04T17:47:02.956 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4329999999999998e-06 2024-09-04T17:47:02.956 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.956 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.956 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:02.956 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211392+0000", 2024-09-04T17:47:02.956 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148258109 2024-09-04T17:47:02.956 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.956 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.957 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:02.957 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211506+0000", 2024-09-04T17:47:02.957 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000113813 2024-09-04T17:47:02.957 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.957 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.957 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:02.957 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211706+0000", 2024-09-04T17:47:02.957 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020002999999999999 2024-09-04T17:47:02.957 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.957 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:02.957 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.957 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.957 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.958 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8848 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:02.958 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:47:02.958 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 49.845162917000003, 2024-09-04T17:47:02.958 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14907967599999999, 2024-09-04T17:47:02.958 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:02.958 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:02.958 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:02.958 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:02.958 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:02.958 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:02.958 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8848 2024-09-04T17:47:02.958 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.958 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:02.959 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.959 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:02.959 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:47:02.959 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.959 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.959 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.959 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:02.959 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:47:02.959 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.959 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.959 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.959 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:02.959 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070589+0000", 2024-09-04T17:47:02.959 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.007445386 2024-09-04T17:47:02.960 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.960 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.960 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:02.960 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070600+0000", 2024-09-04T17:47:02.960 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0346999999999999e-05 2024-09-04T17:47:02.960 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.960 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.960 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:02.960 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070622+0000", 2024-09-04T17:47:02.960 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2289999999999998e-05 2024-09-04T17:47:02.960 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.960 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.960 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:02.961 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070636+0000", 2024-09-04T17:47:02.961 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4093e-05 2024-09-04T17:47:02.961 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.961 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.961 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:02.961 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211815+0000", 2024-09-04T17:47:02.961 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14117874699999999 2024-09-04T17:47:02.961 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.961 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.961 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:02.961 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211983+0000", 2024-09-04T17:47:02.961 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00016852800000000001 2024-09-04T17:47:02.961 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.962 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.962 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:02.962 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.212224+0000", 2024-09-04T17:47:02.962 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024028499999999999 2024-09-04T17:47:02.962 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.962 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:02.962 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.962 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.962 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.962 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8862 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2019328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:02.962 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:47:02.962 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 49.837113449, 2024-09-04T17:47:02.963 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14755236399999999, 2024-09-04T17:47:02.963 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:02.963 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:02.963 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:02.963 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:02.963 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:02.963 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:02.963 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8862 2024-09-04T17:47:02.963 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.963 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:02.963 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.963 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:02.963 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:47:02.963 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.964 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.964 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.964 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:02.964 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:47:02.964 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.964 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.964 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.964 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:02.964 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071194+0000", 2024-09-04T17:47:02.964 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.5000000000000002e-07 2024-09-04T17:47:02.964 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.964 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.964 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:02.964 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071197+0000", 2024-09-04T17:47:02.965 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5900000000000002e-06 2024-09-04T17:47:02.965 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.965 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.965 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:02.965 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071206+0000", 2024-09-04T17:47:02.965 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9700000000000005e-06 2024-09-04T17:47:02.965 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.965 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.965 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:02.965 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071213+0000", 2024-09-04T17:47:02.965 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.9070000000000001e-06 2024-09-04T17:47:02.965 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.965 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.966 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:02.966 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218396+0000", 2024-09-04T17:47:02.966 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147183273 2024-09-04T17:47:02.966 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.966 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.966 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:02.966 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218538+0000", 2024-09-04T17:47:02.966 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014194199999999999 2024-09-04T17:47:02.966 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.966 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.966 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:02.966 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218746+0000", 2024-09-04T17:47:02.966 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000208032 2024-09-04T17:47:02.966 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.967 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:02.967 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.967 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.967 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.967 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8863 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2056192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:02.967 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:47:02.967 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 49.837085139000003, 2024-09-04T17:47:02.967 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14792281099999999, 2024-09-04T17:47:02.967 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:02.967 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:02.967 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:02.967 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:02.967 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:02.967 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:02.968 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8863 2024-09-04T17:47:02.968 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.968 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:02.968 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.968 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:02.968 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:47:02.968 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.968 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.968 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.968 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:02.968 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:47:02.968 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.968 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.969 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.969 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:02.969 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:47:02.969 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1299999999999999e-07 2024-09-04T17:47:02.969 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.969 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.969 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:02.969 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071225+0000", 2024-09-04T17:47:02.969 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6199999999999999e-06 2024-09-04T17:47:02.969 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.969 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.969 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:02.969 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071234+0000", 2024-09-04T17:47:02.970 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8769999999999999e-06 2024-09-04T17:47:02.970 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.970 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.970 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:02.970 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071251+0000", 2024-09-04T17:47:02.970 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.738e-05 2024-09-04T17:47:02.970 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.970 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.970 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:02.970 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218798+0000", 2024-09-04T17:47:02.970 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14754645 2024-09-04T17:47:02.970 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.970 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.970 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:02.971 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218926+0000", 2024-09-04T17:47:02.971 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000128446 2024-09-04T17:47:02.971 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.971 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.971 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:02.971 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219145+0000", 2024-09-04T17:47:02.971 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021832500000000001 2024-09-04T17:47:02.971 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.971 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:02.971 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.971 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.971 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.971 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8864 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2093056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:02.971 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:47:02.972 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 49.837043563000002, 2024-09-04T17:47:02.972 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148277309, 2024-09-04T17:47:02.972 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:02.972 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:02.972 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:02.972 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:02.972 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:02.972 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:02.972 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8864 2024-09-04T17:47:02.972 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.972 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:02.972 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.972 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:02.973 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:47:02.973 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.973 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.973 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.973 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:02.973 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:47:02.973 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.973 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.973 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.973 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:02.973 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071264+0000", 2024-09-04T17:47:02.973 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3e-07 2024-09-04T17:47:02.973 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.973 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.974 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:02.974 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071267+0000", 2024-09-04T17:47:02.974 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0139999999999999e-06 2024-09-04T17:47:02.974 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.974 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.974 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:02.974 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071276+0000", 2024-09-04T17:47:02.974 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1689999999999998e-06 2024-09-04T17:47:02.974 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.974 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.974 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:02.974 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071283+0000", 2024-09-04T17:47:02.974 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2269999999999996e-06 2024-09-04T17:47:02.974 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.975 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.975 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:02.975 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219195+0000", 2024-09-04T17:47:02.975 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147911868 2024-09-04T17:47:02.975 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.975 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.975 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:02.975 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219324+0000", 2024-09-04T17:47:02.975 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00012835900000000001 2024-09-04T17:47:02.975 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.975 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.975 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:02.975 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219541+0000", 2024-09-04T17:47:02.975 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000216942 2024-09-04T17:47:02.976 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.976 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:02.976 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.976 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.976 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.976 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8865 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2109440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:02.976 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:47:02.976 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 49.837014103000001, 2024-09-04T17:47:02.976 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14865747300000001, 2024-09-04T17:47:02.976 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:02.976 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:02.976 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:02.977 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:02.977 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:02.977 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:02.977 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8865 2024-09-04T17:47:02.977 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.977 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:02.977 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.977 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:02.977 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:47:02.977 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.977 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.977 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.977 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:02.977 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:47:02.978 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.978 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.978 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.978 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:02.978 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:47:02.978 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7999999999999995e-07 2024-09-04T17:47:02.978 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.978 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.978 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:02.978 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071296+0000", 2024-09-04T17:47:02.978 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6900000000000001e-06 2024-09-04T17:47:02.978 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.978 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.978 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:02.979 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071305+0000", 2024-09-04T17:47:02.979 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.5499999999999995e-06 2024-09-04T17:47:02.979 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.979 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.979 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:02.979 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071312+0000", 2024-09-04T17:47:02.979 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1559999999999998e-06 2024-09-04T17:47:02.979 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.979 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.979 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:02.979 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219591+0000", 2024-09-04T17:47:02.979 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14827944900000001 2024-09-04T17:47:02.979 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.979 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.980 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:02.980 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219731+0000", 2024-09-04T17:47:02.980 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013931600000000001 2024-09-04T17:47:02.980 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.980 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.980 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:02.980 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219950+0000", 2024-09-04T17:47:02.980 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021963199999999999 2024-09-04T17:47:02.980 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.980 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:02.980 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.980 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.980 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.981 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8866 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2146304~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:02.981 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:47:02.981 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 49.836985869999999, 2024-09-04T17:47:02.981 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14906117899999999, 2024-09-04T17:47:02.981 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:02.981 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:02.981 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:02.981 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:02.981 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:02.981 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:02.981 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8866 2024-09-04T17:47:02.981 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.981 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:02.981 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.982 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:02.982 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:47:02.982 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.982 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.982 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.982 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:02.982 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:47:02.982 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.982 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.982 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.982 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:02.982 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071322+0000", 2024-09-04T17:47:02.982 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7999999999999995e-07 2024-09-04T17:47:02.983 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.983 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.983 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:02.983 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071338+0000", 2024-09-04T17:47:02.983 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5872999999999999e-05 2024-09-04T17:47:02.983 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.983 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.983 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:02.983 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071347+0000", 2024-09-04T17:47:02.983 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4669999999999993e-06 2024-09-04T17:47:02.983 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.983 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.983 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:02.983 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071354+0000", 2024-09-04T17:47:02.984 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2099999999999996e-06 2024-09-04T17:47:02.984 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.984 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.984 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:02.984 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220008+0000", 2024-09-04T17:47:02.984 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148653642 2024-09-04T17:47:02.984 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.984 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.984 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:02.984 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220155+0000", 2024-09-04T17:47:02.984 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014726600000000001 2024-09-04T17:47:02.984 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.984 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.985 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:02.985 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220382+0000", 2024-09-04T17:47:02.985 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00022704100000000001 2024-09-04T17:47:02.985 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.985 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:02.985 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.985 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.985 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.985 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8867 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2183168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:02.985 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:47:02.985 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 49.836942516999997, 2024-09-04T17:47:02.985 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.149445831, 2024-09-04T17:47:02.985 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:02.985 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:02.986 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:02.986 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:02.986 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:02.986 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:02.986 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8867 2024-09-04T17:47:02.986 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.986 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:02.986 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.986 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:02.986 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:47:02.986 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.986 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.986 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.986 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:02.987 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:47:02.987 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.987 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.987 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.987 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:02.987 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078653+0000", 2024-09-04T17:47:02.987 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0072890430000000003 2024-09-04T17:47:02.987 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.987 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.987 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:02.987 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078664+0000", 2024-09-04T17:47:02.987 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0307e-05 2024-09-04T17:47:02.987 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.987 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.988 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:02.988 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078690+0000", 2024-09-04T17:47:02.988 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6780000000000001e-05 2024-09-04T17:47:02.988 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.988 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.988 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:02.988 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078706+0000", 2024-09-04T17:47:02.988 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5676999999999999e-05 2024-09-04T17:47:02.988 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.988 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.988 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:02.988 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220467+0000", 2024-09-04T17:47:02.988 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14176103500000001 2024-09-04T17:47:02.989 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.989 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.989 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:02.989 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220595+0000", 2024-09-04T17:47:02.989 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000127427 2024-09-04T17:47:02.989 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.989 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.989 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:02.989 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220810+0000", 2024-09-04T17:47:02.989 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021556200000000001 2024-09-04T17:47:02.989 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.989 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:02.989 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.990 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.990 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.990 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8880 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2658304~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:02.990 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.079158+0000", 2024-09-04T17:47:02.990 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 49.829149237999999, 2024-09-04T17:47:02.990 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14718968499999999, 2024-09-04T17:47:02.990 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:02.990 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:02.990 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:02.990 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:02.990 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:02.990 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:02.990 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8880 2024-09-04T17:47:02.990 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.991 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:02.991 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.991 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:02.991 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079158+0000", 2024-09-04T17:47:02.991 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.991 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.991 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.991 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:02.991 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079158+0000", 2024-09-04T17:47:02.991 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.991 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.991 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.991 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:02.992 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079158+0000", 2024-09-04T17:47:02.992 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3e-07 2024-09-04T17:47:02.992 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.992 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.992 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:02.992 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079161+0000", 2024-09-04T17:47:02.992 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.57e-06 2024-09-04T17:47:02.992 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.992 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.992 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:02.992 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079170+0000", 2024-09-04T17:47:02.992 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.0170000000000002e-06 2024-09-04T17:47:02.992 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.992 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.993 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:02.993 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079177+0000", 2024-09-04T17:47:02.993 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.0530000000000001e-06 2024-09-04T17:47:02.993 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.993 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.993 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:02.993 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.225946+0000", 2024-09-04T17:47:02.993 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14676936500000001 2024-09-04T17:47:02.993 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.993 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.993 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:02.993 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226098+0000", 2024-09-04T17:47:02.993 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000151805 2024-09-04T17:47:02.994 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.994 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.994 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:02.994 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226347+0000", 2024-09-04T17:47:02.994 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024914499999999999 2024-09-04T17:47:02.994 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.994 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:02.994 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.994 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.994 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.994 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8881 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2695168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:02.994 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:47:02.994 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 49.829120447999998, 2024-09-04T17:47:02.994 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147543547, 2024-09-04T17:47:02.995 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:02.995 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:02.995 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:02.995 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:02.995 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:02.995 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:02.995 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8881 2024-09-04T17:47:02.995 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.995 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:02.995 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.995 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:02.995 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:47:02.995 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.995 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.996 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.996 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:02.996 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:47:02.996 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:02.996 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.996 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.996 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:02.996 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079187+0000", 2024-09-04T17:47:02.996 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7700000000000004e-07 2024-09-04T17:47:02.996 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.996 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.996 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:02.996 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079198+0000", 2024-09-04T17:47:02.997 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0849999999999999e-05 2024-09-04T17:47:02.997 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.997 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.997 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:02.997 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079207+0000", 2024-09-04T17:47:02.997 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5599999999999999e-06 2024-09-04T17:47:02.997 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.997 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.997 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:02.997 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079224+0000", 2024-09-04T17:47:02.997 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6387000000000001e-05 2024-09-04T17:47:02.997 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.997 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.997 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:02.998 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226414+0000", 2024-09-04T17:47:02.998 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14718975000000001 2024-09-04T17:47:02.998 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.998 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.998 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:02.998 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226534+0000", 2024-09-04T17:47:02.998 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000120288 2024-09-04T17:47:02.998 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.998 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.998 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:02.998 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226730+0000", 2024-09-04T17:47:02.998 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000196035 2024-09-04T17:47:02.998 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.999 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:02.999 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:02.999 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:02.999 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:02.999 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8882 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2732032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:02.999 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:47:02.999 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 49.829072234000002, 2024-09-04T17:47:02.999 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14795022399999999, 2024-09-04T17:47:02.999 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:02.999 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:02.999 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:02.999 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:02.999 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:03.000 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:03.000 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8882 2024-09-04T17:47:03.000 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.000 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:03.000 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.000 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:03.000 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:47:03.000 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:03.000 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.000 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.000 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:03.000 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:47:03.000 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:03.001 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.001 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.001 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:03.001 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086588+0000", 2024-09-04T17:47:03.001 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0073530959999999999 2024-09-04T17:47:03.001 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.001 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.001 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:03.001 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086598+0000", 2024-09-04T17:47:03.001 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0122999999999999e-05 2024-09-04T17:47:03.001 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.001 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.001 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:03.001 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086626+0000", 2024-09-04T17:47:03.002 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.7914000000000001e-05 2024-09-04T17:47:03.002 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.002 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.002 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:03.002 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086641+0000", 2024-09-04T17:47:03.002 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5270000000000001e-05 2024-09-04T17:47:03.002 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.002 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.002 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:03.002 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226801+0000", 2024-09-04T17:47:03.002 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.140160321 2024-09-04T17:47:03.002 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.002 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.002 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:03.003 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226951+0000", 2024-09-04T17:47:03.003 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014965200000000001 2024-09-04T17:47:03.003 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.003 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.003 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:03.003 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.227185+0000", 2024-09-04T17:47:03.003 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00023384799999999999 2024-09-04T17:47:03.003 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:03.003 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:03.003 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:03.003 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:03.003 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:03.003 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:03.005 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-04T17:47:03.005 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:03.003+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:03.060 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #24, image image_client.0-clone24 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent25 2024-09-04T17:47:03.106 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:47:03.103+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:47:03.122 INFO:teuthology.orchestra.run.smithi092.stdout:3557 read 0x6875fc5 thru 0x687d392 (0x73ce bytes) 2024-09-04T17:47:03.138 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:03.138 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:47:03.138 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:47:03.138 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:03.146 INFO:teuthology.orchestra.run.smithi092.stdout:3559 write 0xecdc5e0 thru 0xecdda12 (0x1433 bytes) 2024-09-04T17:47:03.152 INFO:teuthology.orchestra.run.smithi092.stdout:3560 read 0x74cdd10 thru 0x74d1fb5 (0x42a6 bytes) 2024-09-04T17:47:03.152 INFO:teuthology.orchestra.run.smithi092.stdout:3562 write 0x528feb6 thru 0x529ee6a (0xefb5 bytes) 2024-09-04T17:47:03.153 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-04T17:47:03.194 INFO:teuthology.orchestra.run.smithi092.stdout:3563 trunc from 0xecdda13 to 0x80e12f9 2024-09-04T17:47:03.212 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:47:03.213+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:47:03.270 INFO:teuthology.orchestra.run.smithi092.stdout:3565 write 0xb1819dd thru 0xb189206 (0x782a bytes) 2024-09-04T17:47:03.273 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:03.277 INFO:teuthology.orchestra.run.smithi092.stdout:3568 trunc from 0xb189207 to 0xca61734 2024-09-04T17:47:03.290 INFO:teuthology.orchestra.run.smithi092.stdout:3569 flatten 2024-09-04T17:47:03.296 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:03.296 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:47:03.296 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:47:03.296 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:47:03.296 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:03.307 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:47:03.303+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:47:03.319 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-04T17:47:03.375 INFO:teuthology.orchestra.run.smithi092.stdout:3570 read 0x366beed thru 0x367b20d (0xf321 bytes) 2024-09-04T17:47:03.376 INFO:teuthology.orchestra.run.smithi092.stdout:3572 write 0xb6dd54f thru 0xb6de56c (0x101e bytes) 2024-09-04T17:47:03.378 INFO:teuthology.orchestra.run.smithi092.stdout:3573 read 0x46289ce thru 0x4630aee (0x8121 bytes) 2024-09-04T17:47:03.386 INFO:teuthology.orchestra.run.smithi092.stdout:3574 trunc from 0xca61734 to 0xa255503 2024-09-04T17:47:03.400 INFO:teuthology.orchestra.run.smithi092.stdout:3575 punch from 0x9fae114 to 0x9faf69b, (0x1587 bytes) 2024-09-04T17:47:03.400 INFO:teuthology.orchestra.run.smithi092.stdout:3576 read 0x1f5374e thru 0x1f61033 (0xd8e6 bytes) 2024-09-04T17:47:03.401 INFO:teuthology.orchestra.run.smithi092.stdout:3577 read 0x30a03c1 thru 0x30a0a25 (0x665 bytes) 2024-09-04T17:47:03.402 INFO:teuthology.orchestra.run.smithi092.stdout:3579 read 0xa00dc99 thru 0xa018f02 (0xb26a bytes) 2024-09-04T17:47:03.402 INFO:teuthology.orchestra.run.smithi092.stdout:3580 write 0xcdd5df thru 0xce1eee (0x4910 bytes) 2024-09-04T17:47:03.404 INFO:teuthology.orchestra.run.smithi092.stdout:3581 read 0x4269e87 thru 0x4277423 (0xd59d bytes) 2024-09-04T17:47:03.405 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:03.407 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:47:03.403+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:47:03.408 INFO:teuthology.orchestra.run.smithi092.stdout:3583 trunc from 0xa255503 to 0xb312ad4 2024-09-04T17:47:03.432 INFO:teuthology.orchestra.run.smithi092.stdout:3584 write 0xc711392 thru 0xc711e76 (0xae5 bytes) 2024-09-04T17:47:03.437 INFO:teuthology.orchestra.run.smithi092.stdout:3585 trunc from 0xc711e77 to 0xd7cb013 2024-09-04T17:47:03.447 INFO:teuthology.orchestra.run.smithi092.stdout:3586 read 0x89d108b thru 0x89d1df3 (0xd69 bytes) 2024-09-04T17:47:03.448 INFO:teuthology.orchestra.run.smithi092.stdout:3587 write 0x9b3f1a0 thru 0x9b47d05 (0x8b66 bytes) 2024-09-04T17:47:03.451 INFO:teuthology.orchestra.run.smithi092.stdout:3588 read 0x2981151 thru 0x29846d8 (0x3588 bytes) 2024-09-04T17:47:03.451 INFO:teuthology.orchestra.run.smithi092.stdout:3589 read 0xa39bd69 thru 0xa39cdf7 (0x108f bytes) 2024-09-04T17:47:03.452 INFO:teuthology.orchestra.run.smithi092.stdout:3591 write 0x84fcdb3 thru 0x850a44a (0xd698 bytes) 2024-09-04T17:47:03.456 INFO:teuthology.orchestra.run.smithi092.stdout:3592 writesame 0x3d8ae19 thru 0x3d90ad2 data_size 0x1a(0x5cba bytes) 2024-09-04T17:47:03.459 INFO:teuthology.orchestra.run.smithi092.stdout:3594 read 0x7090197 thru 0x709ebfc (0xea66 bytes) 2024-09-04T17:47:03.460 INFO:teuthology.orchestra.run.smithi092.stdout:3595 punch from 0x3547405 to 0x354fcb2, (0x88ad bytes) 2024-09-04T17:47:03.461 INFO:teuthology.orchestra.run.smithi092.stdout:3597 read 0x187e3b4 thru 0x1887852 (0x949f bytes) 2024-09-04T17:47:03.461 INFO:teuthology.orchestra.run.smithi092.stdout:3601 read 0x87900c1 thru 0x8793c11 (0x3b51 bytes) 2024-09-04T17:47:03.479 INFO:teuthology.orchestra.run.smithi092.stdout:3603 trunc from 0xd7cb013 to 0x5431b9a 2024-09-04T17:47:03.482 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:03.482 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:47:03.482 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:47:03.482 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:47:03.482 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.482 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:1430 3.f 3:f58475cb:::rbd_data.11346251f094.0000000000000074:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-04T17:47:03.482 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:02.778704+0000", 2024-09-04T17:47:03.482 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 60.702347361000001, 2024-09-04T17:47:03.482 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.058567801000000003, 2024-09-04T17:47:03.482 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:03.482 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:03.483 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:03.483 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:03.483 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:03.483 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:03.483 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 1430 2024-09-04T17:47:03.483 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.483 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:03.483 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.483 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:03.483 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.778704+0000", 2024-09-04T17:47:03.483 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:03.483 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.483 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.484 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:03.484 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.778704+0000", 2024-09-04T17:47:03.484 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:03.484 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.484 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.484 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:03.484 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.778707+0000", 2024-09-04T17:47:03.484 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.8329999999999998e-06 2024-09-04T17:47:03.484 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.484 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.484 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:03.484 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.778718+0000", 2024-09-04T17:47:03.485 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.152e-05 2024-09-04T17:47:03.485 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.485 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.485 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:03.485 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.778750+0000", 2024-09-04T17:47:03.485 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.1696999999999998e-05 2024-09-04T17:47:03.485 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.485 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.485 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:03.485 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.778765+0000", 2024-09-04T17:47:03.485 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4800000000000001e-05 2024-09-04T17:47:03.485 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.485 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.486 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:03.486 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.778894+0000", 2024-09-04T17:47:03.486 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00012928500000000001 2024-09-04T17:47:03.486 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.486 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.486 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:03.486 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.779030+0000", 2024-09-04T17:47:03.486 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013566899999999999 2024-09-04T17:47:03.486 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.486 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.486 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 4", 2024-09-04T17:47:03.486 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.788794+0000", 2024-09-04T17:47:03.486 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.009764478 2024-09-04T17:47:03.487 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.487 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.487 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:47:03.487 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.836969+0000", 2024-09-04T17:47:03.487 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.048174626999999998 2024-09-04T17:47:03.487 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.487 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.487 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:03.487 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.837090+0000", 2024-09-04T17:47:03.487 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000120627 2024-09-04T17:47:03.487 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.487 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.487 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:47:03.487 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.837195+0000", 2024-09-04T17:47:03.488 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000105206 2024-09-04T17:47:03.488 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.488 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.488 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:03.488 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.837272+0000", 2024-09-04T17:47:03.488 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.7058999999999995e-05 2024-09-04T17:47:03.488 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:03.488 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:03.488 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:03.488 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.488 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.488 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14369 3.d 3:b763de10:::rbd_data.11345bd5898b.0000000000000002:a [read 77824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:03.488 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.182808+0000", 2024-09-04T17:47:03.489 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 45.298243393, 2024-09-04T17:47:03.489 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042057997999999999, 2024-09-04T17:47:03.489 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:03.489 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:03.489 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:03.489 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:03.489 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:03.489 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:03.489 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14369 2024-09-04T17:47:03.489 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.489 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:03.489 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.489 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:03.490 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182808+0000", 2024-09-04T17:47:03.490 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:03.490 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.490 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.490 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:03.490 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182808+0000", 2024-09-04T17:47:03.490 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:03.490 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.490 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.490 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:03.490 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182809+0000", 2024-09-04T17:47:03.490 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.063e-06 2024-09-04T17:47:03.490 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.490 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.491 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:03.491 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182814+0000", 2024-09-04T17:47:03.491 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1900000000000003e-06 2024-09-04T17:47:03.491 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.491 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.491 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:03.491 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182832+0000", 2024-09-04T17:47:03.491 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7791999999999998e-05 2024-09-04T17:47:03.491 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.491 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.491 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:03.491 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182849+0000", 2024-09-04T17:47:03.491 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7265000000000002e-05 2024-09-04T17:47:03.491 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.492 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.492 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:03.492 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.224431+0000", 2024-09-04T17:47:03.492 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.041581448 2024-09-04T17:47:03.492 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.492 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.492 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:03.492 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.224615+0000", 2024-09-04T17:47:03.492 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00018389200000000001 2024-09-04T17:47:03.492 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.492 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.492 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:03.493 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.224866+0000", 2024-09-04T17:47:03.493 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00025134799999999998 2024-09-04T17:47:03.493 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:03.493 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:03.493 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:03.493 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.493 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.493 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14370 3.d 3:b763de10:::rbd_data.11345bd5898b.0000000000000002:a [read 114688~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:03.493 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.182867+0000", 2024-09-04T17:47:03.493 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 45.298184091000003, 2024-09-04T17:47:03.493 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042368301999999997, 2024-09-04T17:47:03.493 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:03.493 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:03.494 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:03.494 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:03.494 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:03.494 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:03.494 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14370 2024-09-04T17:47:03.494 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.494 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:03.494 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.494 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:03.494 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182867+0000", 2024-09-04T17:47:03.494 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:03.494 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.494 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.495 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:03.495 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182867+0000", 2024-09-04T17:47:03.495 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:03.495 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.495 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.495 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:03.495 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182868+0000", 2024-09-04T17:47:03.495 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.006e-06 2024-09-04T17:47:03.495 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.495 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.495 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:03.495 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182875+0000", 2024-09-04T17:47:03.495 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.3990000000000001e-06 2024-09-04T17:47:03.496 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.496 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.496 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:03.496 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182892+0000", 2024-09-04T17:47:03.496 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7096e-05 2024-09-04T17:47:03.496 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.496 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.496 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:03.496 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182905+0000", 2024-09-04T17:47:03.496 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3258000000000001e-05 2024-09-04T17:47:03.496 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.496 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.496 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:03.497 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.224926+0000", 2024-09-04T17:47:03.497 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042020891999999997 2024-09-04T17:47:03.497 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.497 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.497 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:03.497 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.225064+0000", 2024-09-04T17:47:03.497 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000138307 2024-09-04T17:47:03.497 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.497 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.497 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:03.497 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.225236+0000", 2024-09-04T17:47:03.497 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000171344 2024-09-04T17:47:03.497 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:03.498 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:03.498 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:03.498 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.498 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.498 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14377 3.d 3:b763de10:::rbd_data.11345bd5898b.0000000000000002:a [read 225280~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:03.498 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.184021+0000", 2024-09-04T17:47:03.498 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 45.297030485000001, 2024-09-04T17:47:03.498 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042298810999999999, 2024-09-04T17:47:03.498 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:03.498 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:03.498 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:03.498 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:03.499 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:03.499 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:03.499 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14377 2024-09-04T17:47:03.499 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.499 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:03.499 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.499 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:03.499 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184021+0000", 2024-09-04T17:47:03.499 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:03.499 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.499 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.499 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:03.499 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184021+0000", 2024-09-04T17:47:03.499 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:03.500 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.500 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.500 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:03.500 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184022+0000", 2024-09-04T17:47:03.500 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5099999999999999e-06 2024-09-04T17:47:03.500 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.500 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.500 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:03.500 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184028+0000", 2024-09-04T17:47:03.500 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.852e-06 2024-09-04T17:47:03.500 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.500 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.500 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:03.501 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184046+0000", 2024-09-04T17:47:03.501 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7822e-05 2024-09-04T17:47:03.501 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.501 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.501 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:03.501 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184059+0000", 2024-09-04T17:47:03.501 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3372e-05 2024-09-04T17:47:03.501 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.501 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.501 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:03.501 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.226009+0000", 2024-09-04T17:47:03.501 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.041949187999999998 2024-09-04T17:47:03.501 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.502 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.502 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:03.502 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.226142+0000", 2024-09-04T17:47:03.502 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013381299999999999 2024-09-04T17:47:03.502 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.502 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.502 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:03.502 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.226320+0000", 2024-09-04T17:47:03.502 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00017725400000000001 2024-09-04T17:47:03.502 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:03.502 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:03.502 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:03.502 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.502 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.503 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14381 3.d 3:b763de10:::rbd_data.11345bd5898b.0000000000000002:a [read 552960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:03.503 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.184135+0000", 2024-09-04T17:47:03.503 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 45.296916433, 2024-09-04T17:47:03.503 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042553818, 2024-09-04T17:47:03.503 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:03.503 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:03.503 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:03.503 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:03.503 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:03.503 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:03.503 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14381 2024-09-04T17:47:03.503 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.503 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:03.504 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.504 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:03.504 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184135+0000", 2024-09-04T17:47:03.504 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:03.504 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.504 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.504 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:03.504 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184135+0000", 2024-09-04T17:47:03.504 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:03.504 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.504 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.504 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:03.504 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184136+0000", 2024-09-04T17:47:03.505 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1200000000000001e-06 2024-09-04T17:47:03.505 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.505 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.505 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:03.505 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184142+0000", 2024-09-04T17:47:03.505 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4840000000000003e-06 2024-09-04T17:47:03.505 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.505 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.505 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:03.505 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184157+0000", 2024-09-04T17:47:03.505 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5727999999999999e-05 2024-09-04T17:47:03.505 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.506 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.506 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:03.506 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184194+0000", 2024-09-04T17:47:03.506 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7049999999999999e-05 2024-09-04T17:47:03.506 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.506 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.506 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:03.506 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.226376+0000", 2024-09-04T17:47:03.506 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042181933999999997 2024-09-04T17:47:03.506 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.506 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.506 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:03.506 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.226522+0000", 2024-09-04T17:47:03.506 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014602899999999999 2024-09-04T17:47:03.507 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.507 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.507 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:03.507 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.226689+0000", 2024-09-04T17:47:03.507 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00016647299999999999 2024-09-04T17:47:03.507 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:03.507 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:03.507 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:03.507 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.507 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.507 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14383 3.d 3:b763de10:::rbd_data.11345bd5898b.0000000000000002:a [read 589824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:03.507 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.184302+0000", 2024-09-04T17:47:03.508 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 45.296749433999999, 2024-09-04T17:47:03.508 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042760362000000003, 2024-09-04T17:47:03.508 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:03.508 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:03.508 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:03.508 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:03.508 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:03.508 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:03.508 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14383 2024-09-04T17:47:03.508 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.508 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:03.508 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.508 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:03.509 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184302+0000", 2024-09-04T17:47:03.509 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:03.509 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.509 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.509 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:03.509 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184302+0000", 2024-09-04T17:47:03.509 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:03.509 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.509 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.509 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:03.509 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184303+0000", 2024-09-04T17:47:03.509 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.206e-06 2024-09-04T17:47:03.509 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.509 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.510 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:03.510 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184340+0000", 2024-09-04T17:47:03.510 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7005999999999998e-05 2024-09-04T17:47:03.510 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.510 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.510 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:03.510 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184382+0000", 2024-09-04T17:47:03.510 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2345999999999997e-05 2024-09-04T17:47:03.510 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.510 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.510 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:03.510 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184397+0000", 2024-09-04T17:47:03.510 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4729000000000001e-05 2024-09-04T17:47:03.510 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.510 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.511 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:03.511 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.226749+0000", 2024-09-04T17:47:03.511 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042352278 2024-09-04T17:47:03.511 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.511 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.511 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:03.511 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.226889+0000", 2024-09-04T17:47:03.511 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000139376 2024-09-04T17:47:03.511 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.511 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.511 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:03.511 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.227062+0000", 2024-09-04T17:47:03.511 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00017342099999999999 2024-09-04T17:47:03.511 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:03.511 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:03.512 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:03.512 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.512 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.512 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14385 3.d 3:b763de10:::rbd_data.11345bd5898b.0000000000000002:a [read 626688~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:03.512 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.184481+0000", 2024-09-04T17:47:03.512 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 45.296570770999999, 2024-09-04T17:47:03.512 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042873850999999998, 2024-09-04T17:47:03.512 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:03.512 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:03.512 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:03.512 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:03.512 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:03.512 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:03.512 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14385 2024-09-04T17:47:03.513 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.513 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:03.513 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.513 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:03.513 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184481+0000", 2024-09-04T17:47:03.513 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:03.513 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.513 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.513 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:03.513 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184481+0000", 2024-09-04T17:47:03.513 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:03.513 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.513 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.513 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:03.514 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184482+0000", 2024-09-04T17:47:03.514 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2130000000000001e-06 2024-09-04T17:47:03.514 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.514 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.514 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:03.514 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184487+0000", 2024-09-04T17:47:03.514 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4260000000000002e-06 2024-09-04T17:47:03.514 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.514 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.514 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:03.514 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184505+0000", 2024-09-04T17:47:03.514 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7659000000000002e-05 2024-09-04T17:47:03.514 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.514 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.515 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:03.515 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184518+0000", 2024-09-04T17:47:03.515 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2982e-05 2024-09-04T17:47:03.515 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.515 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.515 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:03.515 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.227120+0000", 2024-09-04T17:47:03.515 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042601858999999999 2024-09-04T17:47:03.515 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.515 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.515 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:03.515 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.227196+0000", 2024-09-04T17:47:03.515 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.6540999999999998e-05 2024-09-04T17:47:03.515 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.516 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.516 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:03.516 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.227354+0000", 2024-09-04T17:47:03.516 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000158171 2024-09-04T17:47:03.516 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:03.516 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:03.516 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:03.516 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.516 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.516 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14389 3.d 3:b763de10:::rbd_data.11345bd5898b.0000000000000002:a [read 663552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:03.516 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.184555+0000", 2024-09-04T17:47:03.516 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 45.296496634999997, 2024-09-04T17:47:03.516 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043553640999999997, 2024-09-04T17:47:03.517 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:03.517 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:03.517 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:03.517 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:03.517 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:03.517 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:03.517 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14389 2024-09-04T17:47:03.517 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.517 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:03.517 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.517 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:03.517 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184555+0000", 2024-09-04T17:47:03.517 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:03.518 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.518 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.518 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:03.518 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184555+0000", 2024-09-04T17:47:03.518 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:03.518 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.518 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.518 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:03.518 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184556+0000", 2024-09-04T17:47:03.518 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.353e-06 2024-09-04T17:47:03.518 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.518 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.518 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:03.518 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184562+0000", 2024-09-04T17:47:03.519 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7899999999999996e-06 2024-09-04T17:47:03.519 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.519 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.519 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:03.519 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184580+0000", 2024-09-04T17:47:03.519 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7682000000000001e-05 2024-09-04T17:47:03.519 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.519 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.519 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:03.519 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184593+0000", 2024-09-04T17:47:03.519 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3625000000000001e-05 2024-09-04T17:47:03.519 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.519 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.519 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:03.520 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.227413+0000", 2024-09-04T17:47:03.520 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042819572 2024-09-04T17:47:03.520 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.520 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.520 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:03.520 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.227552+0000", 2024-09-04T17:47:03.520 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000138842 2024-09-04T17:47:03.520 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.520 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.520 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:03.520 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228108+0000", 2024-09-04T17:47:03.520 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00055677700000000005 2024-09-04T17:47:03.520 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:03.521 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:03.521 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:03.521 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.521 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.521 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14392 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:03.521 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.184613+0000", 2024-09-04T17:47:03.521 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 45.296438373000001, 2024-09-04T17:47:03.521 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043677522000000003, 2024-09-04T17:47:03.521 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:03.521 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:03.521 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:03.521 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:03.521 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:03.521 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:03.522 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14392 2024-09-04T17:47:03.522 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.522 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:03.522 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.522 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:03.522 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184613+0000", 2024-09-04T17:47:03.522 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:03.522 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.522 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.522 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:03.522 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184613+0000", 2024-09-04T17:47:03.522 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:03.522 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.522 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.523 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:03.523 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184614+0000", 2024-09-04T17:47:03.523 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.35e-06 2024-09-04T17:47:03.523 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.523 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.523 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:03.523 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184620+0000", 2024-09-04T17:47:03.523 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.5589999999999996e-06 2024-09-04T17:47:03.523 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.523 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.523 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:03.523 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184671+0000", 2024-09-04T17:47:03.523 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0615000000000002e-05 2024-09-04T17:47:03.523 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.524 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.524 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:03.524 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184684+0000", 2024-09-04T17:47:03.524 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3596e-05 2024-09-04T17:47:03.524 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.524 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.524 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:03.524 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228277+0000", 2024-09-04T17:47:03.524 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043592509000000002 2024-09-04T17:47:03.524 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.524 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.524 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:03.524 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228291+0000", 2024-09-04T17:47:03.524 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3893000000000001e-05 2024-09-04T17:47:03.525 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:03.525 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:03.525 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:03.525 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.525 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.525 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14395 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:03.525 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.184708+0000", 2024-09-04T17:47:03.525 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 45.296343182000001, 2024-09-04T17:47:03.525 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043643612999999998, 2024-09-04T17:47:03.525 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:03.525 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:03.525 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:03.525 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:03.526 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:03.526 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:03.526 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14395 2024-09-04T17:47:03.526 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.526 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:03.526 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.526 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:03.526 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184708+0000", 2024-09-04T17:47:03.526 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:03.526 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.526 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.526 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:03.526 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184708+0000", 2024-09-04T17:47:03.527 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:03.527 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.527 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.527 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:03.527 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184709+0000", 2024-09-04T17:47:03.527 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2359999999999999e-06 2024-09-04T17:47:03.527 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.527 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.527 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:03.528 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184716+0000", 2024-09-04T17:47:03.528 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.3500000000000002e-06 2024-09-04T17:47:03.528 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.528 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.528 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:03.528 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184733+0000", 2024-09-04T17:47:03.528 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6775000000000001e-05 2024-09-04T17:47:03.528 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.528 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.528 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:03.528 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184777+0000", 2024-09-04T17:47:03.528 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4183e-05 2024-09-04T17:47:03.529 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.529 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.529 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:03.529 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228339+0000", 2024-09-04T17:47:03.529 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043562030000000002 2024-09-04T17:47:03.529 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.529 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.529 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:03.529 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228352+0000", 2024-09-04T17:47:03.529 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3039e-05 2024-09-04T17:47:03.529 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:03.529 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:03.529 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:03.530 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.530 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.530 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14398 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:03.530 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.184847+0000", 2024-09-04T17:47:03.530 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 45.296204535000001, 2024-09-04T17:47:03.530 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043564139000000002, 2024-09-04T17:47:03.530 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:03.530 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:03.530 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:03.530 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:03.530 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:03.530 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:03.530 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14398 2024-09-04T17:47:03.530 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.531 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:03.531 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.531 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:03.531 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184847+0000", 2024-09-04T17:47:03.531 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:03.531 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.531 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.531 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:03.531 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184847+0000", 2024-09-04T17:47:03.531 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:03.531 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.531 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.531 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:03.531 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184848+0000", 2024-09-04T17:47:03.532 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.28e-06 2024-09-04T17:47:03.532 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.532 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.532 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:03.532 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184861+0000", 2024-09-04T17:47:03.532 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3125000000000001e-05 2024-09-04T17:47:03.532 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.532 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.532 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:03.532 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184884+0000", 2024-09-04T17:47:03.532 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2357999999999998e-05 2024-09-04T17:47:03.532 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.532 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.533 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:03.533 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184900+0000", 2024-09-04T17:47:03.533 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6872e-05 2024-09-04T17:47:03.533 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.533 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.533 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:03.533 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228398+0000", 2024-09-04T17:47:03.533 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043497313000000003 2024-09-04T17:47:03.533 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.533 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.533 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:03.533 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228411+0000", 2024-09-04T17:47:03.533 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3191e-05 2024-09-04T17:47:03.534 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:03.534 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:03.534 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:03.534 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.534 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.534 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14399 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:03.534 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.184925+0000", 2024-09-04T17:47:03.534 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 45.296126219000001, 2024-09-04T17:47:03.534 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043544799000000002, 2024-09-04T17:47:03.534 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:03.534 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:03.534 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:03.534 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:03.534 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:03.535 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:03.535 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14399 2024-09-04T17:47:03.535 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.535 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:03.535 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.535 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:03.535 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184925+0000", 2024-09-04T17:47:03.535 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:03.535 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.535 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.535 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:03.535 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184925+0000", 2024-09-04T17:47:03.535 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:03.535 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.536 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.536 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:03.536 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184927+0000", 2024-09-04T17:47:03.536 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7e-06 2024-09-04T17:47:03.536 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.536 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.536 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:03.536 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184949+0000", 2024-09-04T17:47:03.536 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1880999999999999e-05 2024-09-04T17:47:03.536 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.536 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.536 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:03.536 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184969+0000", 2024-09-04T17:47:03.537 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0758e-05 2024-09-04T17:47:03.537 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.537 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.537 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:03.537 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184983+0000", 2024-09-04T17:47:03.537 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3336000000000001e-05 2024-09-04T17:47:03.537 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.537 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.537 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:03.537 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228456+0000", 2024-09-04T17:47:03.537 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043472960999999997 2024-09-04T17:47:03.537 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.537 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.538 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:03.538 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228470+0000", 2024-09-04T17:47:03.538 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4163e-05 2024-09-04T17:47:03.538 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:03.538 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:03.538 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:03.538 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.538 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.538 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14400 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:03.538 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.185003+0000", 2024-09-04T17:47:03.538 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 45.296048783000003, 2024-09-04T17:47:03.538 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043532659000000001, 2024-09-04T17:47:03.538 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:03.538 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:03.539 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:03.539 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:03.539 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:03.539 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:03.539 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14400 2024-09-04T17:47:03.539 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.539 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:03.539 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.539 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:03.539 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185003+0000", 2024-09-04T17:47:03.539 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:03.539 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.539 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.539 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:03.539 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185003+0000", 2024-09-04T17:47:03.539 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:03.540 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.540 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.540 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:03.540 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185004+0000", 2024-09-04T17:47:03.540 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.294e-06 2024-09-04T17:47:03.540 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.540 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.540 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:03.540 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185010+0000", 2024-09-04T17:47:03.540 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7690000000000002e-06 2024-09-04T17:47:03.540 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.540 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.540 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:03.541 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185029+0000", 2024-09-04T17:47:03.541 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9100999999999999e-05 2024-09-04T17:47:03.541 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.541 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.541 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:03.541 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185043+0000", 2024-09-04T17:47:03.541 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3862e-05 2024-09-04T17:47:03.541 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.541 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.541 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:03.541 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228510+0000", 2024-09-04T17:47:03.541 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043467766999999997 2024-09-04T17:47:03.541 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.541 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.542 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:03.542 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228535+0000", 2024-09-04T17:47:03.542 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4865999999999999e-05 2024-09-04T17:47:03.542 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:03.542 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:03.542 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:03.542 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.542 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.542 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14401 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:03.542 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.185062+0000", 2024-09-04T17:47:03.542 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 45.295989517999999, 2024-09-04T17:47:03.542 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043556096000000002, 2024-09-04T17:47:03.542 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:03.543 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:03.543 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:03.543 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:03.543 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:03.543 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:03.543 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14401 2024-09-04T17:47:03.543 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.543 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:03.543 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.543 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:03.543 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185062+0000", 2024-09-04T17:47:03.543 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:03.543 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.544 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.544 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:03.544 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185062+0000", 2024-09-04T17:47:03.544 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:03.544 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.544 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.544 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:03.544 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185063+0000", 2024-09-04T17:47:03.544 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.409e-06 2024-09-04T17:47:03.544 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.544 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.544 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:03.544 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185069+0000", 2024-09-04T17:47:03.545 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.5400000000000003e-06 2024-09-04T17:47:03.545 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.545 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.545 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:03.545 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185087+0000", 2024-09-04T17:47:03.545 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8195000000000001e-05 2024-09-04T17:47:03.545 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.545 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.545 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:03.545 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185101+0000", 2024-09-04T17:47:03.545 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3769e-05 2024-09-04T17:47:03.545 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.545 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.546 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:03.546 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228604+0000", 2024-09-04T17:47:03.546 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043503537000000002 2024-09-04T17:47:03.546 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.546 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.546 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:03.546 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228618+0000", 2024-09-04T17:47:03.546 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3645999999999999e-05 2024-09-04T17:47:03.546 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:03.546 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:03.546 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:03.546 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.546 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.547 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14402 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:03.547 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.185121+0000", 2024-09-04T17:47:03.547 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 45.295930224000003, 2024-09-04T17:47:03.547 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043551542999999998, 2024-09-04T17:47:03.547 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:03.547 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:03.547 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:03.547 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:03.547 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:03.547 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:03.547 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14402 2024-09-04T17:47:03.547 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.547 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:03.548 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.548 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:03.548 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185121+0000", 2024-09-04T17:47:03.548 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:03.548 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.548 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.548 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:03.548 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185121+0000", 2024-09-04T17:47:03.548 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:03.548 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.548 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.548 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:03.548 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185122+0000", 2024-09-04T17:47:03.549 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1769999999999999e-06 2024-09-04T17:47:03.549 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.549 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.549 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:03.549 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185128+0000", 2024-09-04T17:47:03.549 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7790000000000001e-06 2024-09-04T17:47:03.549 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.549 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.549 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:03.549 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185146+0000", 2024-09-04T17:47:03.549 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8114999999999999e-05 2024-09-04T17:47:03.549 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.549 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.550 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:03.550 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185160+0000", 2024-09-04T17:47:03.550 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4158999999999999e-05 2024-09-04T17:47:03.550 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.550 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.550 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:03.550 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228659+0000", 2024-09-04T17:47:03.550 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043498715 2024-09-04T17:47:03.550 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.550 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.550 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:03.550 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228673+0000", 2024-09-04T17:47:03.550 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3597999999999999e-05 2024-09-04T17:47:03.550 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:03.551 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:03.551 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:03.551 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.551 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.551 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14403 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:03.551 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.185183+0000", 2024-09-04T17:47:03.551 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 45.295868175999999, 2024-09-04T17:47:03.551 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043541467, 2024-09-04T17:47:03.551 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:03.551 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:03.551 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:03.551 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:03.551 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:03.552 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:03.552 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14403 2024-09-04T17:47:03.552 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.552 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:03.552 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.552 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:03.552 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185183+0000", 2024-09-04T17:47:03.552 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:03.552 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.552 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.552 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:03.552 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185183+0000", 2024-09-04T17:47:03.552 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:03.552 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.553 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.553 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:03.553 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185184+0000", 2024-09-04T17:47:03.553 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.156e-06 2024-09-04T17:47:03.553 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.553 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.553 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:03.553 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185190+0000", 2024-09-04T17:47:03.553 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1800000000000004e-06 2024-09-04T17:47:03.553 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.553 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.553 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:03.553 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185207+0000", 2024-09-04T17:47:03.554 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7288000000000001e-05 2024-09-04T17:47:03.554 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.554 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.554 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:03.554 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185221+0000", 2024-09-04T17:47:03.554 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3729e-05 2024-09-04T17:47:03.554 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.554 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.554 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:03.554 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228711+0000", 2024-09-04T17:47:03.554 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043490595 2024-09-04T17:47:03.554 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.554 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.555 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:03.555 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228725+0000", 2024-09-04T17:47:03.555 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3519e-05 2024-09-04T17:47:03.555 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:03.555 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:03.555 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:03.555 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.555 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.555 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14404 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:03.555 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.185239+0000", 2024-09-04T17:47:03.555 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 45.295812038000001, 2024-09-04T17:47:03.555 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043542099000000001, 2024-09-04T17:47:03.555 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:03.555 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:03.556 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:03.556 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:03.556 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:03.556 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:03.556 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14404 2024-09-04T17:47:03.556 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.556 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:03.556 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.556 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:03.556 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185239+0000", 2024-09-04T17:47:03.556 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:03.556 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.556 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.556 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:03.557 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185239+0000", 2024-09-04T17:47:03.557 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:03.557 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.557 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.557 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:03.557 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185240+0000", 2024-09-04T17:47:03.557 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.173e-06 2024-09-04T17:47:03.557 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.557 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.557 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:03.557 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185246+0000", 2024-09-04T17:47:03.557 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7529999999999998e-06 2024-09-04T17:47:03.557 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.558 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.558 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:03.558 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185264+0000", 2024-09-04T17:47:03.558 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7872000000000001e-05 2024-09-04T17:47:03.558 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.558 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.558 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:03.558 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185278+0000", 2024-09-04T17:47:03.558 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3532e-05 2024-09-04T17:47:03.559 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.560 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.560 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:03.560 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228768+0000", 2024-09-04T17:47:03.560 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043490586999999997 2024-09-04T17:47:03.560 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.560 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.560 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:03.560 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228781+0000", 2024-09-04T17:47:03.560 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3182e-05 2024-09-04T17:47:03.560 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:03.560 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:03.560 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:03.561 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.561 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.561 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14405 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:03.561 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.185297+0000", 2024-09-04T17:47:03.561 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 45.295754846999998, 2024-09-04T17:47:03.561 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043535049999999999, 2024-09-04T17:47:03.561 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:03.561 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:03.561 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:03.561 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:03.561 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:03.561 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:03.561 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14405 2024-09-04T17:47:03.562 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.562 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:03.562 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.562 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:03.562 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185297+0000", 2024-09-04T17:47:03.562 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:03.562 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.562 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.562 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:03.562 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185297+0000", 2024-09-04T17:47:03.562 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:03.562 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.562 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.563 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:03.563 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185298+0000", 2024-09-04T17:47:03.563 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.22e-06 2024-09-04T17:47:03.563 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.563 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.563 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:03.563 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185308+0000", 2024-09-04T17:47:03.563 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0173000000000001e-05 2024-09-04T17:47:03.563 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.563 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.563 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:03.563 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185329+0000", 2024-09-04T17:47:03.563 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1087999999999999e-05 2024-09-04T17:47:03.564 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.564 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.564 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:03.564 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185345+0000", 2024-09-04T17:47:03.564 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6035000000000001e-05 2024-09-04T17:47:03.564 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.564 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.564 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:03.564 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228822+0000", 2024-09-04T17:47:03.564 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043477427999999999 2024-09-04T17:47:03.564 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.564 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.564 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:03.565 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228832+0000", 2024-09-04T17:47:03.565 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1060000000000006e-06 2024-09-04T17:47:03.565 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:03.565 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:03.565 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:03.565 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.565 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.565 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14411 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:03.565 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.186171+0000", 2024-09-04T17:47:03.565 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 45.294880024000001, 2024-09-04T17:47:03.565 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.04271581, 2024-09-04T17:47:03.565 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:03.565 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:03.566 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:03.566 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:03.566 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:03.566 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:03.566 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14411 2024-09-04T17:47:03.566 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.566 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:03.566 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.566 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:03.566 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.186171+0000", 2024-09-04T17:47:03.566 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:03.566 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.566 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.567 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:03.567 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.186171+0000", 2024-09-04T17:47:03.567 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:03.567 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.567 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.567 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:03.567 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.186173+0000", 2024-09-04T17:47:03.567 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.012e-06 2024-09-04T17:47:03.567 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.567 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.567 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:03.567 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.186183+0000", 2024-09-04T17:47:03.567 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.7729999999999994e-06 2024-09-04T17:47:03.567 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.568 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.568 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:03.568 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.186205+0000", 2024-09-04T17:47:03.568 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2280999999999999e-05 2024-09-04T17:47:03.568 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.568 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.568 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:03.568 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.186224+0000", 2024-09-04T17:47:03.568 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8695e-05 2024-09-04T17:47:03.568 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.568 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.568 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:03.568 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228873+0000", 2024-09-04T17:47:03.569 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042648798000000002 2024-09-04T17:47:03.569 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.569 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.569 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:03.569 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228887+0000", 2024-09-04T17:47:03.569 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4251e-05 2024-09-04T17:47:03.569 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:03.569 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:03.569 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:03.569 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.569 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.569 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:37359 3.f 3:f29e50a5:::rbd_data.1134c5a43465.0000000000000021:head [stat,write 62464~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-04T17:47:03.569 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.616887+0000", 2024-09-04T17:47:03.569 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 23.864164108000001, 2024-09-04T17:47:03.570 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.054183104000000003, 2024-09-04T17:47:03.570 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:03.570 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:03.570 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:03.570 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:03.570 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:03.570 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:03.570 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 37359 2024-09-04T17:47:03.570 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.570 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:03.570 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.570 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:03.570 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.616887+0000", 2024-09-04T17:47:03.571 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:03.571 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.571 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.571 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:03.571 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.616887+0000", 2024-09-04T17:47:03.571 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:03.571 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.571 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.571 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:03.571 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.616889+0000", 2024-09-04T17:47:03.571 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.362e-06 2024-09-04T17:47:03.571 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.571 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.572 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:03.572 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.616904+0000", 2024-09-04T17:47:03.572 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.507e-05 2024-09-04T17:47:03.572 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.572 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.572 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:03.572 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.616920+0000", 2024-09-04T17:47:03.572 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6385999999999999e-05 2024-09-04T17:47:03.572 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.572 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.572 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:03.572 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.616929+0000", 2024-09-04T17:47:03.572 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8540000000000003e-06 2024-09-04T17:47:03.573 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.573 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.573 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:03.573 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.616983+0000", 2024-09-04T17:47:03.573 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4218999999999997e-05 2024-09-04T17:47:03.573 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.573 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.573 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:03.573 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617065+0000", 2024-09-04T17:47:03.573 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.1864000000000001e-05 2024-09-04T17:47:03.573 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.573 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.573 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 4", 2024-09-04T17:47:03.574 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617205+0000", 2024-09-04T17:47:03.574 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013962600000000001 2024-09-04T17:47:03.574 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.574 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.574 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:47:03.574 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.618670+0000", 2024-09-04T17:47:03.574 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.001465647 2024-09-04T17:47:03.574 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.574 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.574 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:03.574 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670962+0000", 2024-09-04T17:47:03.574 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.052292000999999998 2024-09-04T17:47:03.574 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.574 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.575 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:47:03.575 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671011+0000", 2024-09-04T17:47:03.575 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9036000000000003e-05 2024-09-04T17:47:03.575 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:03.575 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:03.575 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:03.575 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671070+0000", 2024-09-04T17:47:03.575 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.9039000000000002e-05 2024-09-04T17:47:03.575 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:03.575 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:03.575 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:03.575 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:03.575 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:03.575 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:03.577 INFO:teuthology.orchestra.run.smithi092.stdout:3605 writesame 0x52f80a4 thru 0x5307f12 data_size 0x7(0xfe6f bytes) 2024-09-04T17:47:03.577 INFO:teuthology.orchestra.run.smithi092.stdout:3606 write 0x7faab07 thru 0x7fb8d49 (0xe243 bytes) 2024-09-04T17:47:03.577 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-04T17:47:03.578 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:03.575+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:03.579 INFO:teuthology.orchestra.run.smithi092.stdout:3607 writesame 0xcd35e8f thru 0xcd403fc data_size 0x9a(0xa56e bytes) 2024-09-04T17:47:03.579 INFO:teuthology.orchestra.run.smithi092.stdout:3608 read 0xbd1bf0a thru 0xbd2aa50 (0xeb47 bytes) 2024-09-04T17:47:03.579 INFO:teuthology.orchestra.run.smithi092.stdout:3609 punch from 0x7b6dcaf to 0x7b78460, (0xa7b1 bytes) 2024-09-04T17:47:03.579 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:03.579 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:03.579 INFO:teuthology.orchestra.run.smithi092.stdout:3613 write 0x3bbf9bb thru 0x3bcf1fe (0xf844 bytes) 2024-09-04T17:47:03.582 INFO:teuthology.orchestra.run.smithi092.stdout:3614 writesame 0x8f2db9d thru 0x8f3106e data_size 0x1(0x34d2 bytes) 2024-09-04T17:47:03.595 INFO:teuthology.orchestra.run.smithi092.stdout:3615 punch from 0x9bd9448 to 0x9bdcff3, (0x3bab bytes) 2024-09-04T17:47:03.619 INFO:teuthology.orchestra.run.smithi092.stdout:3616 trunc from 0xcd403fd to 0x366fb93 2024-09-04T17:47:03.634 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-04T17:47:03.646 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:03.646 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:03.656 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [6] live_osds: [0, 1, 2, 4, 5, 7, 3] 2024-09-04T17:47:03.656 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-04T17:47:03.656 INFO:tasks.thrashosds.thrasher:Removing osd 3, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-04T17:47:03.656 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 3 2024-09-04T17:47:03.657 INFO:teuthology.orchestra.run.smithi092.stdout:3622 write 0x45ac1bd thru 0x45aef40 (0x2d84 bytes) 2024-09-04T17:47:03.672 INFO:teuthology.orchestra.run.smithi092.stdout:3623 writesame 0x5bf6f4e thru 0x5bff0d3 data_size 0x3b(0x8186 bytes) 2024-09-04T17:47:03.674 INFO:teuthology.orchestra.run.smithi092.stdout:3624 punch from 0x5069f69 to 0x506d221, (0x32b8 bytes) 2024-09-04T17:47:03.677 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:03.677+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:03.687 INFO:teuthology.orchestra.run.smithi092.stdout:3625 write 0x873f1db thru 0x8744692 (0x54b8 bytes) 2024-09-04T17:47:03.706 INFO:teuthology.orchestra.run.smithi092.stdout:3626 write 0xaff363d thru 0xaff9065 (0x5a29 bytes) 2024-09-04T17:47:03.711 INFO:teuthology.orchestra.run.smithi092.stdout:3628 writesame 0x2dd3e48 thru 0x2de0a70 data_size 0x1(0xcc29 bytes) 2024-09-04T17:47:03.731 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:03.731 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:47:03.732 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:47:03.732 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:03.754 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-04T17:47:03.763 INFO:teuthology.orchestra.run.smithi092.stdout:3629 write 0x94cb9ec thru 0x94cdd0f (0x2324 bytes) 2024-09-04T17:47:03.765 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:03.765 INFO:teuthology.orchestra.run.smithi092.stdout:3632 clone 27 order 24 su 8388608 sc 9 2024-09-04T17:47:03.812 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:03.811+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:03.879 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:03.877+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:03.884 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:03.884 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:47:03.884 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:47:03.884 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:47:03.884 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:03.898 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-04T17:47:03.984 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:03.983+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:03.995 INFO:tasks.ceph.mon.a.smithi092.stderr:2024-09-04T17:47:03.991+0000 7f8f2f46f640 -1 mon.a@0(leader).osd e78 definitely_dead 0 2024-09-04T17:47:04.055 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:04.056 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:47:04.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:47:04.056 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:47:04.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.056 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:27242 3.b 3.34414deb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-04T17:47:04.056 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:28.402347+0000", 2024-09-04T17:47:04.056 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 35.653050862999997, 2024-09-04T17:47:04.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.93265329299999999, 2024-09-04T17:47:04.056 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:04.056 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:04.056 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-04T17:47:04.056 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:04.056 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:04.057 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:04.057 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 27242 2024-09-04T17:47:04.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.057 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:04.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:04.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402347+0000", 2024-09-04T17:47:04.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:04.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:04.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402347+0000", 2024-09-04T17:47:04.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:04.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:04.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402349+0000", 2024-09-04T17:47:04.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.471e-06 2024-09-04T17:47:04.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:04.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402355+0000", 2024-09-04T17:47:04.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8399999999999997e-06 2024-09-04T17:47:04.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:04.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402364+0000", 2024-09-04T17:47:04.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1269999999999992e-06 2024-09-04T17:47:04.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:04.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402373+0000", 2024-09-04T17:47:04.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7909999999999993e-06 2024-09-04T17:47:04.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:04.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402460+0000", 2024-09-04T17:47:04.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6686000000000003e-05 2024-09-04T17:47:04.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.060 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-04T17:47:04.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402473+0000", 2024-09-04T17:47:04.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3281e-05 2024-09-04T17:47:04.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.060 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:04.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:29.334989+0000", 2024-09-04T17:47:04.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.93251549099999997 2024-09-04T17:47:04.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.060 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:04.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:29.335000+0000", 2024-09-04T17:47:04.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1606e-05 2024-09-04T17:47:04.061 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:04.061 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:04.061 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:04.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.061 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:33552 3.7 3:e79ad1db:::rbd_data.11342e8f3384.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-04T17:47:04.061 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:35.999748+0000", 2024-09-04T17:47:04.061 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.055649812999999, 2024-09-04T17:47:04.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045683823999999998, 2024-09-04T17:47:04.061 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:04.061 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:04.061 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:04.062 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:04.062 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:04.062 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:04.062 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 33552 2024-09-04T17:47:04.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.062 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:04.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:04.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999748+0000", 2024-09-04T17:47:04.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:04.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:04.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999748+0000", 2024-09-04T17:47:04.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:04.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:04.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999750+0000", 2024-09-04T17:47:04.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1299999999999999e-06 2024-09-04T17:47:04.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:04.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999759+0000", 2024-09-04T17:47:04.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4649999999999994e-06 2024-09-04T17:47:04.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:04.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999778+0000", 2024-09-04T17:47:04.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8833000000000001e-05 2024-09-04T17:47:04.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:04.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999786+0000", 2024-09-04T17:47:04.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7369999999999995e-06 2024-09-04T17:47:04.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:04.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999856+0000", 2024-09-04T17:47:04.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9247999999999998e-05 2024-09-04T17:47:04.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:04.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999973+0000", 2024-09-04T17:47:04.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000117921 2024-09-04T17:47:04.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 7", 2024-09-04T17:47:04.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.005955+0000", 2024-09-04T17:47:04.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059820560000000004 2024-09-04T17:47:04.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:04.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.045297+0000", 2024-09-04T17:47:04.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.039341650999999998 2024-09-04T17:47:04.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:04.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.045343+0000", 2024-09-04T17:47:04.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5899000000000002e-05 2024-09-04T17:47:04.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:04.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.045388+0000", 2024-09-04T17:47:04.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4601999999999997e-05 2024-09-04T17:47:04.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:04.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.045432+0000", 2024-09-04T17:47:04.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4282000000000001e-05 2024-09-04T17:47:04.067 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:04.067 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:04.067 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:04.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.067 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:37357 3.15 3:a897312a:::rbd_data.1134c5a43465.000000000000001f:head [stat,write 62464~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-04T17:47:04.067 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:39.616724+0000", 2024-09-04T17:47:04.067 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 24.438674405, 2024-09-04T17:47:04.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054965854000000001, 2024-09-04T17:47:04.067 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:04.067 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:04.067 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:04.068 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:04.068 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:04.068 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:04.068 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 37357 2024-09-04T17:47:04.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.068 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:04.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:04.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616724+0000", 2024-09-04T17:47:04.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:04.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:04.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616724+0000", 2024-09-04T17:47:04.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:04.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:04.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616725+0000", 2024-09-04T17:47:04.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.815e-06 2024-09-04T17:47:04.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:04.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616739+0000", 2024-09-04T17:47:04.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3723000000000001e-05 2024-09-04T17:47:04.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:04.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616755+0000", 2024-09-04T17:47:04.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5964000000000002e-05 2024-09-04T17:47:04.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:04.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616763+0000", 2024-09-04T17:47:04.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3869999999999995e-06 2024-09-04T17:47:04.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:04.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616825+0000", 2024-09-04T17:47:04.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1854999999999996e-05 2024-09-04T17:47:04.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:04.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616895+0000", 2024-09-04T17:47:04.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9597999999999993e-05 2024-09-04T17:47:04.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-04T17:47:04.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616991+0000", 2024-09-04T17:47:04.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5828000000000005e-05 2024-09-04T17:47:04.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:04.072 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.617659+0000", 2024-09-04T17:47:04.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00066821600000000001 2024-09-04T17:47:04.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:04.072 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.671482+0000", 2024-09-04T17:47:04.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053823533999999999 2024-09-04T17:47:04.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:04.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.671579+0000", 2024-09-04T17:47:04.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6657999999999995e-05 2024-09-04T17:47:04.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:04.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.671689+0000", 2024-09-04T17:47:04.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000110276 2024-09-04T17:47:04.073 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:04.073 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:04.073 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:04.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.073 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:43873 3.15 3:abd207ee:::rbd_data.113410ddd206.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:04.074 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:49.932940+0000", 2024-09-04T17:47:04.074 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.122457852, 2024-09-04T17:47:04.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042740514, 2024-09-04T17:47:04.074 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:04.074 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:04.074 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:04.074 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:04.074 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:04.074 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:04.074 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 43873 2024-09-04T17:47:04.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.074 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:04.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:04.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.932940+0000", 2024-09-04T17:47:04.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:04.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:04.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.932940+0000", 2024-09-04T17:47:04.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:04.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:04.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.932941+0000", 2024-09-04T17:47:04.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.404e-06 2024-09-04T17:47:04.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:04.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.933065+0000", 2024-09-04T17:47:04.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012361199999999999 2024-09-04T17:47:04.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:04.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.933078+0000", 2024-09-04T17:47:04.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3182e-05 2024-09-04T17:47:04.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:04.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.933085+0000", 2024-09-04T17:47:04.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4500000000000001e-06 2024-09-04T17:47:04.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.077 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:04.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.933134+0000", 2024-09-04T17:47:04.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9150999999999997e-05 2024-09-04T17:47:04.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.077 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:04.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.933242+0000", 2024-09-04T17:47:04.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010781 2024-09-04T17:47:04.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-04T17:47:04.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.938358+0000", 2024-09-04T17:47:04.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051160349999999997 2024-09-04T17:47:04.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:04.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.975343+0000", 2024-09-04T17:47:04.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036985663000000002 2024-09-04T17:47:04.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:04.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.975392+0000", 2024-09-04T17:47:04.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9044e-05 2024-09-04T17:47:04.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:04.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.975548+0000", 2024-09-04T17:47:04.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015572900000000001 2024-09-04T17:47:04.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:04.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.975681+0000", 2024-09-04T17:47:04.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013243400000000001 2024-09-04T17:47:04.079 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:04.079 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:04.079 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:04.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.080 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:46649 3.7 3:e0749ed5:::rbd_data.1134326b938e.000000000000014f:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-04T17:47:04.080 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:54.114730+0000", 2024-09-04T17:47:04.080 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.9406677200000004, 2024-09-04T17:47:04.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032972972000000003, 2024-09-04T17:47:04.080 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:04.080 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:04.080 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:04.080 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:04.080 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:04.080 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:04.080 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 46649 2024-09-04T17:47:04.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.081 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:04.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:04.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114730+0000", 2024-09-04T17:47:04.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:04.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:04.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114730+0000", 2024-09-04T17:47:04.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:04.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:04.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114731+0000", 2024-09-04T17:47:04.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.147e-06 2024-09-04T17:47:04.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:04.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114737+0000", 2024-09-04T17:47:04.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6380000000000001e-06 2024-09-04T17:47:04.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:04.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114751+0000", 2024-09-04T17:47:04.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4012999999999999e-05 2024-09-04T17:47:04.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:04.083 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114758+0000", 2024-09-04T17:47:04.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1539999999999996e-06 2024-09-04T17:47:04.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.083 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:04.083 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114876+0000", 2024-09-04T17:47:04.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000117421 2024-09-04T17:47:04.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.083 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:04.083 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.115173+0000", 2024-09-04T17:47:04.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00029750799999999998 2024-09-04T17:47:04.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 7", 2024-09-04T17:47:04.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.120273+0000", 2024-09-04T17:47:04.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051002900000000004 2024-09-04T17:47:04.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:04.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.147571+0000", 2024-09-04T17:47:04.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027297906 2024-09-04T17:47:04.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:04.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.147588+0000", 2024-09-04T17:47:04.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6807999999999999e-05 2024-09-04T17:47:04.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:04.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.147637+0000", 2024-09-04T17:47:04.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8466e-05 2024-09-04T17:47:04.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:04.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.147703+0000", 2024-09-04T17:47:04.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6620999999999998e-05 2024-09-04T17:47:04.085 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:04.085 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:04.085 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:04.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.086 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49801 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 720896~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:04.086 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.075094+0000", 2024-09-04T17:47:04.086 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.98030373299999995, 2024-09-04T17:47:04.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017824974, 2024-09-04T17:47:04.086 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:04.086 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:04.086 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:04.086 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:04.086 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:04.086 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:04.086 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49801 2024-09-04T17:47:04.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.087 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:04.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.087 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:04.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075094+0000", 2024-09-04T17:47:04.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:04.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.087 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:04.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075094+0000", 2024-09-04T17:47:04.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:04.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.087 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:04.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075095+0000", 2024-09-04T17:47:04.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.017e-06 2024-09-04T17:47:04.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:04.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075101+0000", 2024-09-04T17:47:04.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5029999999999997e-06 2024-09-04T17:47:04.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:04.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075119+0000", 2024-09-04T17:47:04.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8133000000000001e-05 2024-09-04T17:47:04.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:04.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075132+0000", 2024-09-04T17:47:04.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3305999999999999e-05 2024-09-04T17:47:04.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:04.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.092677+0000", 2024-09-04T17:47:04.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017544466000000002 2024-09-04T17:47:04.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:04.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.092782+0000", 2024-09-04T17:47:04.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000105284 2024-09-04T17:47:04.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:04.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.092919+0000", 2024-09-04T17:47:04.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013726499999999999 2024-09-04T17:47:04.090 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:04.090 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:04.090 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:04.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.090 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49809 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 163840~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:04.090 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077146+0000", 2024-09-04T17:47:04.090 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.97825229000000002, 2024-09-04T17:47:04.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017984541999999999, 2024-09-04T17:47:04.091 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:04.091 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:04.091 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:04.091 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:04.091 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:04.091 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:04.091 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49809 2024-09-04T17:47:04.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.091 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:04.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:04.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077146+0000", 2024-09-04T17:47:04.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:04.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:04.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077146+0000", 2024-09-04T17:47:04.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:04.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:04.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077147+0000", 2024-09-04T17:47:04.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0330000000000001e-06 2024-09-04T17:47:04.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:04.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077164+0000", 2024-09-04T17:47:04.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7810000000000001e-05 2024-09-04T17:47:04.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:04.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077187+0000", 2024-09-04T17:47:04.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2246e-05 2024-09-04T17:47:04.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:04.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077203+0000", 2024-09-04T17:47:04.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6392000000000002e-05 2024-09-04T17:47:04.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:04.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.094885+0000", 2024-09-04T17:47:04.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017681589000000001 2024-09-04T17:47:04.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:04.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.094991+0000", 2024-09-04T17:47:04.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010668899999999999 2024-09-04T17:47:04.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:04.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095130+0000", 2024-09-04T17:47:04.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013878299999999999 2024-09-04T17:47:04.094 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:04.095 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:04.095 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:04.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.095 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49810 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 229376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:04.095 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077225+0000", 2024-09-04T17:47:04.095 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.978172449, 2024-09-04T17:47:04.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018218793, 2024-09-04T17:47:04.095 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:04.095 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:04.095 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:04.095 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:04.095 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:04.095 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:04.096 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49810 2024-09-04T17:47:04.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.096 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:04.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:04.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077225+0000", 2024-09-04T17:47:04.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:04.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:04.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077225+0000", 2024-09-04T17:47:04.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:04.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:04.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077227+0000", 2024-09-04T17:47:04.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1829999999999999e-06 2024-09-04T17:47:04.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:04.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077233+0000", 2024-09-04T17:47:04.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3899999999999998e-06 2024-09-04T17:47:04.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:04.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077253+0000", 2024-09-04T17:47:04.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0009000000000001e-05 2024-09-04T17:47:04.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:04.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077270+0000", 2024-09-04T17:47:04.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7336000000000001e-05 2024-09-04T17:47:04.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:04.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095177+0000", 2024-09-04T17:47:04.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017907101000000002 2024-09-04T17:47:04.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:04.099 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095299+0000", 2024-09-04T17:47:04.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000121749 2024-09-04T17:47:04.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:04.099 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095444+0000", 2024-09-04T17:47:04.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014502499999999999 2024-09-04T17:47:04.099 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:04.099 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:04.099 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:04.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.099 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49811 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 294912~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:04.099 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077288+0000", 2024-09-04T17:47:04.100 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.97811013499999999, 2024-09-04T17:47:04.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018443620000000001, 2024-09-04T17:47:04.100 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:04.100 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:04.100 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:04.100 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:04.100 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:04.100 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:04.100 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49811 2024-09-04T17:47:04.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.100 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:04.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:04.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077288+0000", 2024-09-04T17:47:04.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:04.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:04.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077288+0000", 2024-09-04T17:47:04.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:04.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:04.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077289+0000", 2024-09-04T17:47:04.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6199999999999999e-06 2024-09-04T17:47:04.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:04.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077295+0000", 2024-09-04T17:47:04.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1499999999999998e-06 2024-09-04T17:47:04.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:04.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077312+0000", 2024-09-04T17:47:04.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7353000000000001e-05 2024-09-04T17:47:04.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.103 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:04.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077324+0000", 2024-09-04T17:47:04.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2593e-05 2024-09-04T17:47:04.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.103 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:04.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095490+0000", 2024-09-04T17:47:04.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018165976 2024-09-04T17:47:04.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.103 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:04.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095597+0000", 2024-09-04T17:47:04.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010629900000000001 2024-09-04T17:47:04.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.104 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:04.104 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095731+0000", 2024-09-04T17:47:04.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013462899999999999 2024-09-04T17:47:04.104 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:04.104 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:04.104 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:04.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.104 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49813 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 360448~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:04.104 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077349+0000", 2024-09-04T17:47:04.104 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.97804893299999995, 2024-09-04T17:47:04.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018780305000000001, 2024-09-04T17:47:04.105 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:04.105 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:04.105 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:04.105 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:04.105 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:04.105 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:04.105 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49813 2024-09-04T17:47:04.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.105 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:04.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.105 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:04.105 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077349+0000", 2024-09-04T17:47:04.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:04.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:04.106 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077349+0000", 2024-09-04T17:47:04.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:04.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:04.106 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077350+0000", 2024-09-04T17:47:04.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0330000000000001e-06 2024-09-04T17:47:04.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:04.106 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077355+0000", 2024-09-04T17:47:04.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0699999999999997e-06 2024-09-04T17:47:04.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:04.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077376+0000", 2024-09-04T17:47:04.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0846e-05 2024-09-04T17:47:04.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:04.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077388+0000", 2024-09-04T17:47:04.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2353e-05 2024-09-04T17:47:04.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:04.108 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095819+0000", 2024-09-04T17:47:04.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01843057 2024-09-04T17:47:04.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:04.108 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095960+0000", 2024-09-04T17:47:04.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000140965 2024-09-04T17:47:04.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:04.108 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096129+0000", 2024-09-04T17:47:04.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016946800000000001 2024-09-04T17:47:04.108 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:04.109 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:04.109 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:04.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.109 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49814 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 425984~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:04.109 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077407+0000", 2024-09-04T17:47:04.109 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.97799043200000002, 2024-09-04T17:47:04.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019041563000000001, 2024-09-04T17:47:04.109 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:04.109 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:04.109 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:04.109 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:04.109 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:04.109 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:04.110 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49814 2024-09-04T17:47:04.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.110 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:04.110 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.110 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:04.110 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077407+0000", 2024-09-04T17:47:04.110 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:04.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.110 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.110 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:04.110 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077407+0000", 2024-09-04T17:47:04.110 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:04.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.111 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:04.111 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077409+0000", 2024-09-04T17:47:04.111 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2330000000000001e-06 2024-09-04T17:47:04.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.111 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:04.111 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077415+0000", 2024-09-04T17:47:04.111 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2729999999999999e-06 2024-09-04T17:47:04.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.111 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:04.111 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077436+0000", 2024-09-04T17:47:04.111 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1209999999999999e-05 2024-09-04T17:47:04.112 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.112 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.112 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:04.112 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077448+0000", 2024-09-04T17:47:04.112 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2225999999999999e-05 2024-09-04T17:47:04.112 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.112 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.112 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:04.112 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096183+0000", 2024-09-04T17:47:04.112 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018735004 2024-09-04T17:47:04.112 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.112 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.112 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:04.112 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096297+0000", 2024-09-04T17:47:04.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000113962 2024-09-04T17:47:04.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:04.113 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096449+0000", 2024-09-04T17:47:04.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000151655 2024-09-04T17:47:04.113 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:04.113 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:04.113 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:04.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.113 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49815 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 491520~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:04.113 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077471+0000", 2024-09-04T17:47:04.114 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.97792719400000006, 2024-09-04T17:47:04.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019266536000000001, 2024-09-04T17:47:04.114 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:04.114 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:04.114 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:04.114 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:04.114 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:04.114 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:04.114 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49815 2024-09-04T17:47:04.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.114 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:04.114 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.114 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:04.115 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077471+0000", 2024-09-04T17:47:04.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:04.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.115 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:04.115 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077471+0000", 2024-09-04T17:47:04.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:04.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.115 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:04.115 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077472+0000", 2024-09-04T17:47:04.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.17e-06 2024-09-04T17:47:04.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:04.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077478+0000", 2024-09-04T17:47:04.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0399999999999998e-06 2024-09-04T17:47:04.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:04.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077500+0000", 2024-09-04T17:47:04.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1959000000000001e-05 2024-09-04T17:47:04.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:04.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077513+0000", 2024-09-04T17:47:04.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2823000000000001e-05 2024-09-04T17:47:04.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:04.117 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096493+0000", 2024-09-04T17:47:04.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018980701999999999 2024-09-04T17:47:04.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:04.117 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096603+0000", 2024-09-04T17:47:04.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010942000000000001 2024-09-04T17:47:04.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:04.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096737+0000", 2024-09-04T17:47:04.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013442200000000001 2024-09-04T17:47:04.118 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:04.118 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:04.118 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:04.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.118 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49816 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 557056~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:04.118 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077532+0000", 2024-09-04T17:47:04.118 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.97786545300000005, 2024-09-04T17:47:04.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019604331999999999, 2024-09-04T17:47:04.118 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:04.118 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:04.118 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:04.119 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:04.119 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:04.119 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:04.119 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49816 2024-09-04T17:47:04.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.119 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:04.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:04.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077532+0000", 2024-09-04T17:47:04.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:04.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:04.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077532+0000", 2024-09-04T17:47:04.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:04.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.120 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:04.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077534+0000", 2024-09-04T17:47:04.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1200000000000001e-06 2024-09-04T17:47:04.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.120 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:04.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077545+0000", 2024-09-04T17:47:04.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1673e-05 2024-09-04T17:47:04.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.121 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:04.121 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077569+0000", 2024-09-04T17:47:04.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3555999999999999e-05 2024-09-04T17:47:04.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.121 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:04.121 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077584+0000", 2024-09-04T17:47:04.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4883e-05 2024-09-04T17:47:04.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.121 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:04.121 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096825+0000", 2024-09-04T17:47:04.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019241734 2024-09-04T17:47:04.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:04.122 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096964+0000", 2024-09-04T17:47:04.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000138085 2024-09-04T17:47:04.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:04.122 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097137+0000", 2024-09-04T17:47:04.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017328100000000001 2024-09-04T17:47:04.122 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:04.122 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:04.122 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:04.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.123 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49817 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 622592~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:04.123 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077603+0000", 2024-09-04T17:47:04.123 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.97779489500000005, 2024-09-04T17:47:04.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01986928, 2024-09-04T17:47:04.123 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:04.123 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:04.123 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:04.123 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:04.123 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:04.123 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:04.123 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49817 2024-09-04T17:47:04.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.123 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:04.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.124 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:04.124 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077603+0000", 2024-09-04T17:47:04.124 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:04.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.124 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:04.124 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077603+0000", 2024-09-04T17:47:04.124 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:04.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.124 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:04.124 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077604+0000", 2024-09-04T17:47:04.124 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7699999999999992e-07 2024-09-04T17:47:04.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.125 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:04.125 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077609+0000", 2024-09-04T17:47:04.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0830000000000003e-06 2024-09-04T17:47:04.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.125 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:04.125 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077630+0000", 2024-09-04T17:47:04.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0438999999999999e-05 2024-09-04T17:47:04.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.125 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:04.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077664+0000", 2024-09-04T17:47:04.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4579000000000002e-05 2024-09-04T17:47:04.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.126 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:04.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097193+0000", 2024-09-04T17:47:04.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019529174 2024-09-04T17:47:04.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.126 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:04.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097323+0000", 2024-09-04T17:47:04.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000129696 2024-09-04T17:47:04.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:04.127 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097472+0000", 2024-09-04T17:47:04.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014933200000000001 2024-09-04T17:47:04.127 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:04.127 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:04.127 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:04.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.127 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49818 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 688128~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:04.127 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077683+0000", 2024-09-04T17:47:04.128 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.97771453100000005, 2024-09-04T17:47:04.128 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020086333000000001, 2024-09-04T17:47:04.128 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:04.128 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:04.128 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:04.128 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:04.128 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:04.128 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:04.128 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49818 2024-09-04T17:47:04.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.128 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:04.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.129 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:04.129 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077683+0000", 2024-09-04T17:47:04.129 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:04.129 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.129 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:04.129 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077683+0000", 2024-09-04T17:47:04.129 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:04.129 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.129 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:04.129 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077684+0000", 2024-09-04T17:47:04.129 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4e-07 2024-09-04T17:47:04.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.130 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.130 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:04.130 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077690+0000", 2024-09-04T17:47:04.130 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4729999999999999e-06 2024-09-04T17:47:04.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.130 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.130 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:04.130 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077710+0000", 2024-09-04T17:47:04.130 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0149999999999999e-05 2024-09-04T17:47:04.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.130 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.130 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:04.130 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077723+0000", 2024-09-04T17:47:04.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3271999999999999e-05 2024-09-04T17:47:04.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:04.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097521+0000", 2024-09-04T17:47:04.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019797981999999999 2024-09-04T17:47:04.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:04.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097628+0000", 2024-09-04T17:47:04.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010665700000000001 2024-09-04T17:47:04.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:04.132 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097770+0000", 2024-09-04T17:47:04.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014185900000000001 2024-09-04T17:47:04.132 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:04.132 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:04.132 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:04.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.132 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49820 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 753664~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:04.132 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077744+0000", 2024-09-04T17:47:04.132 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.97765360000000001, 2024-09-04T17:47:04.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020364786999999999, 2024-09-04T17:47:04.132 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:04.132 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:04.133 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:04.133 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:04.133 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:04.133 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:04.133 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49820 2024-09-04T17:47:04.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.133 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:04.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.133 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:04.133 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077744+0000", 2024-09-04T17:47:04.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:04.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.133 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:04.134 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077744+0000", 2024-09-04T17:47:04.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:04.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.134 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:04.134 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077745+0000", 2024-09-04T17:47:04.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0270000000000001e-06 2024-09-04T17:47:04.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.134 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:04.134 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077751+0000", 2024-09-04T17:47:04.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.626e-06 2024-09-04T17:47:04.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.135 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:04.135 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077771+0000", 2024-09-04T17:47:04.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0469999999999999e-05 2024-09-04T17:47:04.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.135 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:04.135 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077785+0000", 2024-09-04T17:47:04.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3129000000000001e-05 2024-09-04T17:47:04.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.135 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:04.135 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097816+0000", 2024-09-04T17:47:04.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020031917999999999 2024-09-04T17:47:04.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:04.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097939+0000", 2024-09-04T17:47:04.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000122533 2024-09-04T17:47:04.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:04.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098109+0000", 2024-09-04T17:47:04.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017008400000000001 2024-09-04T17:47:04.136 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:04.136 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:04.136 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:04.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.137 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49821 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 819200~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:04.137 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077807+0000", 2024-09-04T17:47:04.137 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.97759138800000001, 2024-09-04T17:47:04.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020702928999999998, 2024-09-04T17:47:04.137 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:04.137 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:04.137 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:04.137 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:04.137 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:04.137 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:04.137 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49821 2024-09-04T17:47:04.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.138 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:04.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:04.138 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077807+0000", 2024-09-04T17:47:04.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:04.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:04.138 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077807+0000", 2024-09-04T17:47:04.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:04.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:04.138 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077808+0000", 2024-09-04T17:47:04.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.463e-06 2024-09-04T17:47:04.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.139 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:04.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077813+0000", 2024-09-04T17:47:04.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7530000000000001e-06 2024-09-04T17:47:04.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.139 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:04.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077832+0000", 2024-09-04T17:47:04.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.946e-05 2024-09-04T17:47:04.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.140 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:04.140 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077846+0000", 2024-09-04T17:47:04.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3786e-05 2024-09-04T17:47:04.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.140 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:04.140 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098201+0000", 2024-09-04T17:47:04.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020354797000000001 2024-09-04T17:47:04.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:04.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098341+0000", 2024-09-04T17:47:04.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000140042 2024-09-04T17:47:04.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:04.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098509+0000", 2024-09-04T17:47:04.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016862800000000001 2024-09-04T17:47:04.141 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:04.141 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:04.141 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:04.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.142 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49823 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 884736~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:04.142 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077865+0000", 2024-09-04T17:47:04.142 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.97753294400000001, 2024-09-04T17:47:04.142 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021008723, 2024-09-04T17:47:04.142 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:04.142 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:04.142 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:04.142 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:04.142 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:04.142 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:04.142 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49823 2024-09-04T17:47:04.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.142 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:04.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:04.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077865+0000", 2024-09-04T17:47:04.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:04.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:04.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077865+0000", 2024-09-04T17:47:04.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:04.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:04.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077866+0000", 2024-09-04T17:47:04.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.164e-06 2024-09-04T17:47:04.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:04.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077872+0000", 2024-09-04T17:47:04.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3959999999999997e-06 2024-09-04T17:47:04.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:04.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077892+0000", 2024-09-04T17:47:04.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0970000000000001e-05 2024-09-04T17:47:04.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:04.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077905+0000", 2024-09-04T17:47:04.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2312e-05 2024-09-04T17:47:04.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:04.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098579+0000", 2024-09-04T17:47:04.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020674486999999998 2024-09-04T17:47:04.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:04.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098705+0000", 2024-09-04T17:47:04.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012541900000000001 2024-09-04T17:47:04.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.146 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:04.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098874+0000", 2024-09-04T17:47:04.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000168975 2024-09-04T17:47:04.146 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:04.146 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:04.146 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:04.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.146 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49824 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 950272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:04.146 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077923+0000", 2024-09-04T17:47:04.146 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.97747525199999996, 2024-09-04T17:47:04.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021246287999999999, 2024-09-04T17:47:04.147 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:04.147 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:04.147 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:04.147 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:04.147 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:04.147 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:04.147 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49824 2024-09-04T17:47:04.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.147 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:04.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.147 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:04.147 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077923+0000", 2024-09-04T17:47:04.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:04.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.148 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:04.148 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077923+0000", 2024-09-04T17:47:04.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:04.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.148 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:04.148 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077924+0000", 2024-09-04T17:47:04.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0100000000000001e-06 2024-09-04T17:47:04.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:04.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077944+0000", 2024-09-04T17:47:04.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0449e-05 2024-09-04T17:47:04.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:04.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077965+0000", 2024-09-04T17:47:04.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0479000000000002e-05 2024-09-04T17:47:04.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:04.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077990+0000", 2024-09-04T17:47:04.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5582999999999999e-05 2024-09-04T17:47:04.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:04.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098924+0000", 2024-09-04T17:47:04.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020933395 2024-09-04T17:47:04.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:04.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.099032+0000", 2024-09-04T17:47:04.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010846299999999999 2024-09-04T17:47:04.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:04.151 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.099169+0000", 2024-09-04T17:47:04.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000136909 2024-09-04T17:47:04.151 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:04.151 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:04.151 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:04.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.151 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49826 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 1015808~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:04.151 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.078010+0000", 2024-09-04T17:47:04.151 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.97738764499999997, 2024-09-04T17:47:04.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021584564000000001, 2024-09-04T17:47:04.151 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:04.151 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:04.151 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:04.152 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:04.152 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:04.152 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:04.152 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49826 2024-09-04T17:47:04.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.152 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:04.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.152 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:04.152 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078010+0000", 2024-09-04T17:47:04.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:04.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.152 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:04.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078010+0000", 2024-09-04T17:47:04.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:04.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:04.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078011+0000", 2024-09-04T17:47:04.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1340000000000001e-06 2024-09-04T17:47:04.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:04.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078017+0000", 2024-09-04T17:47:04.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.682e-06 2024-09-04T17:47:04.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:04.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078036+0000", 2024-09-04T17:47:04.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9357000000000001e-05 2024-09-04T17:47:04.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:04.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078050+0000", 2024-09-04T17:47:04.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3122000000000001e-05 2024-09-04T17:47:04.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:04.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.099271+0000", 2024-09-04T17:47:04.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021221772999999999 2024-09-04T17:47:04.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:04.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.099410+0000", 2024-09-04T17:47:04.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000138415 2024-09-04T17:47:04.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:04.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:04.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:04.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.099595+0000", 2024-09-04T17:47:04.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000185081 2024-09-04T17:47:04.155 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:04.155 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:04.155 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:04.155 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:04.156 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:04.156 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:04.157 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:04.155+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:04.157 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-04T17:47:04.257 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:47:04.255+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:47:04.328 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:04.328 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:47:04.328 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:47:04.328 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:04.342 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-04T17:47:04.359 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:04.357+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:04.415 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-04T17:47:04.472 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:04.470+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:04.507 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:04.507 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:47:04.507 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:47:04.507 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:47:04.507 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:04.521 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-04T17:47:04.561 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:04.561+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:04.568 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:47:04.568 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:47:04.585 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:47:04.585 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:47:04.602 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-04T17:47:04.602 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-04T17:47:04.619 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-04T17:47:04.619 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-04T17:47:04.627 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is unset 2024-09-04T17:47:04.628 INFO:teuthology.orchestra.run.smithi092.stderr:marked out osd.3. 2024-09-04T17:47:04.650 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:47:04.651 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:47:04.662 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:04.661+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:04.675 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:47:04.675 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:47:04.675 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-04T17:47:04.692 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:04.693 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:04.697 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:47:04.697 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:47:04.700 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-04T17:47:04.700 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-04T17:47:04.707 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-04T17:47:04.763 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:04.761+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:04.864 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:04.864 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:04.865 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:47:04.862+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:47:04.879 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-04T17:47:04.965 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:04.962+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:05.041 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:05.041 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:05.055 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-04T17:47:05.065 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:47:05.065+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:47:05.166 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:05.162+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:05.222 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:05.222 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:05.236 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-04T17:47:05.266 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:05.265+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:05.367 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:05.362+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:05.399 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:05.399 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:05.414 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-04T17:47:05.504 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:05.502+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:05.569 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:05.569+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:05.579 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:05.579 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:05.593 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-04T17:47:05.670 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:05.669+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:05.769 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:05.769 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:05.770 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:05.769+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:05.783 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-04T17:47:05.871 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:05.866+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:05.917 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:05.918 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:05.931 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-04T17:47:05.973 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:05.970+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:06.049 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:06.050 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:06.064 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-04T17:47:06.108 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:06.109+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:06.174 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:06.173+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:06.186 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:06.187 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:06.201 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-04T17:47:06.276 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:47:06.274+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:47:06.370 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:06.370 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:06.377 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:06.374+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:06.384 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-04T17:47:06.478 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:06.474+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:06.506 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:06.506 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:06.520 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-04T17:47:06.578 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:06.578+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:06.648 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:06.648 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:06.663 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-04T17:47:06.665 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-04T17:47:06.701 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-04T17:47:06.708 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:06.706+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:06.780 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:06.777+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:06.806 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:06.806 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:06.824 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-04T17:47:06.881 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:47:06.878+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:47:06.917 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:47:06.917 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:47:06.946 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:47:06.947 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:47:06.947 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #25, image image_client.0-clone25 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent26 2024-09-04T17:47:06.975 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-04T17:47:06.975 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-04T17:47:06.981 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:06.981+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:06.987 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:06.987 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:06.998 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-04T17:47:06.998 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-04T17:47:07.002 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-04T17:47:07.034 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:47:07.034 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:47:07.065 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:47:07.066 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:47:07.066 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-04T17:47:07.088 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:07.089+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:07.089 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:47:07.090 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:47:07.094 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-04T17:47:07.095 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-04T17:47:07.183 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:07.184 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:47:07.184 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:47:07.184 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:47:07.184 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.184 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:1430 3.f 3:f58475cb:::rbd_data.11346251f094.0000000000000074:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-04T17:47:07.184 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:02.778704+0000", 2024-09-04T17:47:07.184 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 64.403646429000005, 2024-09-04T17:47:07.184 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.058567801000000003, 2024-09-04T17:47:07.184 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:07.184 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:07.184 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:07.184 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:07.185 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:07.185 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:07.185 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 1430 2024-09-04T17:47:07.185 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.185 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:07.185 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.185 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:07.185 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.778704+0000", 2024-09-04T17:47:07.185 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:07.185 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.185 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.185 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:07.185 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.778704+0000", 2024-09-04T17:47:07.185 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:07.186 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.186 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.186 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:07.186 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.778707+0000", 2024-09-04T17:47:07.186 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.8329999999999998e-06 2024-09-04T17:47:07.186 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.186 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.186 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:07.186 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.778718+0000", 2024-09-04T17:47:07.186 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.152e-05 2024-09-04T17:47:07.186 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.186 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.186 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:07.187 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.778750+0000", 2024-09-04T17:47:07.187 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.1696999999999998e-05 2024-09-04T17:47:07.187 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.187 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.187 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:07.187 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.778765+0000", 2024-09-04T17:47:07.187 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4800000000000001e-05 2024-09-04T17:47:07.187 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.187 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.187 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:07.187 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.778894+0000", 2024-09-04T17:47:07.187 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00012928500000000001 2024-09-04T17:47:07.187 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.188 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.188 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:07.188 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.779030+0000", 2024-09-04T17:47:07.188 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013566899999999999 2024-09-04T17:47:07.188 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.188 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.188 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 4", 2024-09-04T17:47:07.188 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.788794+0000", 2024-09-04T17:47:07.188 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.009764478 2024-09-04T17:47:07.188 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.188 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.188 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:47:07.188 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.836969+0000", 2024-09-04T17:47:07.189 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.048174626999999998 2024-09-04T17:47:07.189 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.189 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.189 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:07.189 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.837090+0000", 2024-09-04T17:47:07.189 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000120627 2024-09-04T17:47:07.189 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.189 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.189 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:47:07.189 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.837195+0000", 2024-09-04T17:47:07.189 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000105206 2024-09-04T17:47:07.189 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.189 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.189 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:07.190 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:02.837272+0000", 2024-09-04T17:47:07.190 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.7058999999999995e-05 2024-09-04T17:47:07.190 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:07.190 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:07.190 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:07.190 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.190 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.190 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14370 3.d 3:b763de10:::rbd_data.11345bd5898b.0000000000000002:a [read 114688~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:07.190 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.182867+0000", 2024-09-04T17:47:07.190 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 48.999483159, 2024-09-04T17:47:07.190 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042368301999999997, 2024-09-04T17:47:07.190 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:07.191 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:07.191 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:07.191 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:07.191 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:07.191 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:07.191 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14370 2024-09-04T17:47:07.191 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.191 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:07.191 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.191 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:07.191 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182867+0000", 2024-09-04T17:47:07.191 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:07.191 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.192 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.192 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:07.192 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182867+0000", 2024-09-04T17:47:07.192 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:07.192 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.192 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.192 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:07.192 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182868+0000", 2024-09-04T17:47:07.192 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.006e-06 2024-09-04T17:47:07.193 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.193 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.193 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:07.193 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182875+0000", 2024-09-04T17:47:07.193 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.3990000000000001e-06 2024-09-04T17:47:07.193 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.193 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.193 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:07.193 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182892+0000", 2024-09-04T17:47:07.193 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7096e-05 2024-09-04T17:47:07.193 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.193 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.193 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:07.194 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.182905+0000", 2024-09-04T17:47:07.194 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3258000000000001e-05 2024-09-04T17:47:07.194 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.194 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.194 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:07.194 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.224926+0000", 2024-09-04T17:47:07.194 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042020891999999997 2024-09-04T17:47:07.194 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.194 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.194 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:07.194 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.225064+0000", 2024-09-04T17:47:07.194 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000138307 2024-09-04T17:47:07.194 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.195 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.195 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:07.195 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.225236+0000", 2024-09-04T17:47:07.195 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000171344 2024-09-04T17:47:07.195 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:07.195 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:07.195 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:07.195 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.195 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.195 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14377 3.d 3:b763de10:::rbd_data.11345bd5898b.0000000000000002:a [read 225280~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:07.195 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.184021+0000", 2024-09-04T17:47:07.195 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 48.998329552999998, 2024-09-04T17:47:07.195 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042298810999999999, 2024-09-04T17:47:07.196 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:07.196 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:07.196 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:07.196 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:07.196 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:07.196 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:07.196 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14377 2024-09-04T17:47:07.196 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.196 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:07.196 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.196 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:07.196 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184021+0000", 2024-09-04T17:47:07.196 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:07.196 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.197 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.197 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:07.197 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184021+0000", 2024-09-04T17:47:07.197 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:07.197 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.197 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.197 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:07.197 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184022+0000", 2024-09-04T17:47:07.197 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5099999999999999e-06 2024-09-04T17:47:07.197 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.197 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.197 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:07.197 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184028+0000", 2024-09-04T17:47:07.197 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.852e-06 2024-09-04T17:47:07.198 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.198 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.198 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:07.198 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184046+0000", 2024-09-04T17:47:07.198 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7822e-05 2024-09-04T17:47:07.198 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.198 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.198 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:07.198 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184059+0000", 2024-09-04T17:47:07.198 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3372e-05 2024-09-04T17:47:07.198 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.198 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.198 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:07.198 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.226009+0000", 2024-09-04T17:47:07.199 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.041949187999999998 2024-09-04T17:47:07.199 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.199 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.199 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:07.199 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.226142+0000", 2024-09-04T17:47:07.199 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013381299999999999 2024-09-04T17:47:07.199 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.199 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.199 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:07.199 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.226320+0000", 2024-09-04T17:47:07.199 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00017725400000000001 2024-09-04T17:47:07.199 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:07.199 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:07.200 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:07.200 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.200 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.200 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14381 3.d 3:b763de10:::rbd_data.11345bd5898b.0000000000000002:a [read 552960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:07.200 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.184135+0000", 2024-09-04T17:47:07.200 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 48.998215500999997, 2024-09-04T17:47:07.200 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042553818, 2024-09-04T17:47:07.200 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:07.200 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:07.200 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:07.200 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:07.200 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:07.200 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:07.201 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14381 2024-09-04T17:47:07.201 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.201 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:07.201 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.201 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:07.201 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184135+0000", 2024-09-04T17:47:07.201 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:07.201 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.201 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.201 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:07.201 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184135+0000", 2024-09-04T17:47:07.201 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:07.201 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.201 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.202 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:07.202 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184136+0000", 2024-09-04T17:47:07.202 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1200000000000001e-06 2024-09-04T17:47:07.202 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.202 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.202 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:07.202 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184142+0000", 2024-09-04T17:47:07.202 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4840000000000003e-06 2024-09-04T17:47:07.202 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.202 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.202 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:07.202 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184157+0000", 2024-09-04T17:47:07.202 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5727999999999999e-05 2024-09-04T17:47:07.202 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.203 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.203 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:07.203 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184194+0000", 2024-09-04T17:47:07.203 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7049999999999999e-05 2024-09-04T17:47:07.203 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.203 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.203 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:07.203 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.226376+0000", 2024-09-04T17:47:07.203 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042181933999999997 2024-09-04T17:47:07.203 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.203 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.203 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:07.203 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.226522+0000", 2024-09-04T17:47:07.204 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014602899999999999 2024-09-04T17:47:07.204 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.204 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.204 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:07.204 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.226689+0000", 2024-09-04T17:47:07.204 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00016647299999999999 2024-09-04T17:47:07.204 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:07.204 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:07.204 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:07.204 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.204 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.204 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14383 3.d 3:b763de10:::rbd_data.11345bd5898b.0000000000000002:a [read 589824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:07.204 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.184302+0000", 2024-09-04T17:47:07.205 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 48.998048502000003, 2024-09-04T17:47:07.205 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042760362000000003, 2024-09-04T17:47:07.205 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:07.205 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:07.205 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:07.205 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:07.205 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:07.205 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:07.205 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14383 2024-09-04T17:47:07.205 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.205 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:07.205 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.205 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:07.205 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184302+0000", 2024-09-04T17:47:07.206 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:07.206 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.206 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.206 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:07.206 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184302+0000", 2024-09-04T17:47:07.206 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:07.206 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.206 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.206 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:07.206 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184303+0000", 2024-09-04T17:47:07.206 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.206e-06 2024-09-04T17:47:07.206 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.206 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.206 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:07.207 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184340+0000", 2024-09-04T17:47:07.207 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7005999999999998e-05 2024-09-04T17:47:07.207 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.207 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.207 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:07.207 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184382+0000", 2024-09-04T17:47:07.207 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2345999999999997e-05 2024-09-04T17:47:07.207 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.207 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.207 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:07.207 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184397+0000", 2024-09-04T17:47:07.207 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4729000000000001e-05 2024-09-04T17:47:07.207 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.207 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.208 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:07.208 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.226749+0000", 2024-09-04T17:47:07.208 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042352278 2024-09-04T17:47:07.208 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.208 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.208 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:07.208 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.226889+0000", 2024-09-04T17:47:07.208 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000139376 2024-09-04T17:47:07.208 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.208 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.208 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:07.208 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.227062+0000", 2024-09-04T17:47:07.208 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00017342099999999999 2024-09-04T17:47:07.208 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:07.209 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:07.209 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:07.209 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.209 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.209 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14385 3.d 3:b763de10:::rbd_data.11345bd5898b.0000000000000002:a [read 626688~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:07.209 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.184481+0000", 2024-09-04T17:47:07.209 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 48.997869839000003, 2024-09-04T17:47:07.209 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042873850999999998, 2024-09-04T17:47:07.209 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:07.209 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:07.209 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:07.209 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:07.209 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:07.210 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:07.210 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14385 2024-09-04T17:47:07.210 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.210 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:07.210 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.210 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:07.210 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184481+0000", 2024-09-04T17:47:07.210 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:07.210 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.210 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.210 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:07.210 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184481+0000", 2024-09-04T17:47:07.210 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:07.210 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.211 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.211 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:07.211 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184482+0000", 2024-09-04T17:47:07.211 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2130000000000001e-06 2024-09-04T17:47:07.211 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.211 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.211 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:07.211 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184487+0000", 2024-09-04T17:47:07.211 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4260000000000002e-06 2024-09-04T17:47:07.211 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.211 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.211 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:07.211 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184505+0000", 2024-09-04T17:47:07.211 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7659000000000002e-05 2024-09-04T17:47:07.212 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.212 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.212 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:07.212 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184518+0000", 2024-09-04T17:47:07.212 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2982e-05 2024-09-04T17:47:07.212 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.212 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.212 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:07.212 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.227120+0000", 2024-09-04T17:47:07.212 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042601858999999999 2024-09-04T17:47:07.212 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.212 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.213 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:07.213 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.227196+0000", 2024-09-04T17:47:07.213 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.6540999999999998e-05 2024-09-04T17:47:07.213 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.213 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.213 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:07.213 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.227354+0000", 2024-09-04T17:47:07.213 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000158171 2024-09-04T17:47:07.213 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:07.213 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:07.213 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:07.213 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.213 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.213 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14389 3.d 3:b763de10:::rbd_data.11345bd5898b.0000000000000002:a [read 663552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:07.214 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.184555+0000", 2024-09-04T17:47:07.214 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 48.997795703000001, 2024-09-04T17:47:07.214 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043553640999999997, 2024-09-04T17:47:07.214 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:07.214 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:07.214 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:07.214 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:07.214 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:07.214 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:07.214 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14389 2024-09-04T17:47:07.214 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.214 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:07.214 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.215 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:07.215 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184555+0000", 2024-09-04T17:47:07.215 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:07.215 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.215 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.215 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:07.215 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184555+0000", 2024-09-04T17:47:07.215 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:07.215 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.215 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.215 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:07.215 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184556+0000", 2024-09-04T17:47:07.215 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.353e-06 2024-09-04T17:47:07.216 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.216 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.216 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:07.216 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184562+0000", 2024-09-04T17:47:07.216 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7899999999999996e-06 2024-09-04T17:47:07.216 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.216 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.216 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:07.216 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184580+0000", 2024-09-04T17:47:07.216 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7682000000000001e-05 2024-09-04T17:47:07.216 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.216 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.216 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:07.217 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184593+0000", 2024-09-04T17:47:07.217 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3625000000000001e-05 2024-09-04T17:47:07.217 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.217 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.217 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:07.217 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.227413+0000", 2024-09-04T17:47:07.217 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042819572 2024-09-04T17:47:07.217 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.217 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.217 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:07.217 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.227552+0000", 2024-09-04T17:47:07.217 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000138842 2024-09-04T17:47:07.217 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.218 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.218 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:07.218 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228108+0000", 2024-09-04T17:47:07.218 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00055677700000000005 2024-09-04T17:47:07.218 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:07.218 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:07.218 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:07.218 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.218 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.218 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14392 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:07.218 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.184613+0000", 2024-09-04T17:47:07.218 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 48.997737440999998, 2024-09-04T17:47:07.218 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043677522000000003, 2024-09-04T17:47:07.218 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:07.219 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:07.219 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:07.219 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:07.219 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:07.219 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:07.219 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14392 2024-09-04T17:47:07.219 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.219 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:07.219 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.219 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:07.219 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184613+0000", 2024-09-04T17:47:07.219 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:07.219 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.219 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.220 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:07.220 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184613+0000", 2024-09-04T17:47:07.220 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:07.220 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.220 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.220 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:07.220 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184614+0000", 2024-09-04T17:47:07.220 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.35e-06 2024-09-04T17:47:07.220 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.220 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.220 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:07.220 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184620+0000", 2024-09-04T17:47:07.220 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.5589999999999996e-06 2024-09-04T17:47:07.221 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.221 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.221 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:07.221 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184671+0000", 2024-09-04T17:47:07.221 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0615000000000002e-05 2024-09-04T17:47:07.221 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.221 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.221 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:07.221 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184684+0000", 2024-09-04T17:47:07.221 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3596e-05 2024-09-04T17:47:07.221 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.221 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.221 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:07.221 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228277+0000", 2024-09-04T17:47:07.222 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043592509000000002 2024-09-04T17:47:07.222 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.222 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.222 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:07.222 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228291+0000", 2024-09-04T17:47:07.222 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3893000000000001e-05 2024-09-04T17:47:07.222 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:07.222 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:07.222 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:07.222 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.222 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.222 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14395 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:07.222 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.184708+0000", 2024-09-04T17:47:07.223 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 48.997642249999998, 2024-09-04T17:47:07.223 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043643612999999998, 2024-09-04T17:47:07.223 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:07.223 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:07.223 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:07.223 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:07.223 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:07.223 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:07.223 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14395 2024-09-04T17:47:07.223 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.223 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:07.223 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.223 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:07.223 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184708+0000", 2024-09-04T17:47:07.224 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:07.224 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.224 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.224 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:07.224 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184708+0000", 2024-09-04T17:47:07.224 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:07.224 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.224 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.224 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:07.224 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184709+0000", 2024-09-04T17:47:07.224 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2359999999999999e-06 2024-09-04T17:47:07.224 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.224 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.224 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:07.225 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184716+0000", 2024-09-04T17:47:07.225 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.3500000000000002e-06 2024-09-04T17:47:07.225 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.225 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.225 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:07.225 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184733+0000", 2024-09-04T17:47:07.225 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6775000000000001e-05 2024-09-04T17:47:07.225 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.225 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.225 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:07.225 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184777+0000", 2024-09-04T17:47:07.225 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4183e-05 2024-09-04T17:47:07.225 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.225 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.226 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:07.226 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228339+0000", 2024-09-04T17:47:07.226 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043562030000000002 2024-09-04T17:47:07.226 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.226 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.226 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:07.226 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228352+0000", 2024-09-04T17:47:07.226 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3039e-05 2024-09-04T17:47:07.226 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:07.226 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:07.226 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:07.226 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.226 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.227 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14398 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:07.227 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.184847+0000", 2024-09-04T17:47:07.227 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 48.997503602999998, 2024-09-04T17:47:07.227 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043564139000000002, 2024-09-04T17:47:07.227 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:07.227 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:07.227 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:07.227 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:07.227 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:07.227 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:07.227 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14398 2024-09-04T17:47:07.227 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.227 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:07.227 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.228 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:07.228 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184847+0000", 2024-09-04T17:47:07.228 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:07.228 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.228 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.228 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:07.228 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184847+0000", 2024-09-04T17:47:07.228 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:07.228 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.228 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.228 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:07.228 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184848+0000", 2024-09-04T17:47:07.228 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.28e-06 2024-09-04T17:47:07.229 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.229 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.229 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:07.229 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184861+0000", 2024-09-04T17:47:07.229 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3125000000000001e-05 2024-09-04T17:47:07.229 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.229 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.229 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:07.229 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184884+0000", 2024-09-04T17:47:07.229 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2357999999999998e-05 2024-09-04T17:47:07.229 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.229 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.229 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:07.229 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184900+0000", 2024-09-04T17:47:07.230 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6872e-05 2024-09-04T17:47:07.230 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.230 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.230 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:07.230 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228398+0000", 2024-09-04T17:47:07.230 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043497313000000003 2024-09-04T17:47:07.230 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.230 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.230 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:07.230 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228411+0000", 2024-09-04T17:47:07.230 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3191e-05 2024-09-04T17:47:07.230 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:07.230 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:07.231 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:07.231 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.231 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.231 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14399 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:07.231 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.184925+0000", 2024-09-04T17:47:07.231 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 48.997425286999999, 2024-09-04T17:47:07.231 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043544799000000002, 2024-09-04T17:47:07.231 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:07.231 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:07.231 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:07.231 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:07.231 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:07.231 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:07.231 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14399 2024-09-04T17:47:07.232 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.232 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:07.232 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.232 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:07.232 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184925+0000", 2024-09-04T17:47:07.232 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:07.232 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.232 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.232 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:07.232 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184925+0000", 2024-09-04T17:47:07.232 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:07.232 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.232 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.233 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:07.233 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184927+0000", 2024-09-04T17:47:07.233 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7e-06 2024-09-04T17:47:07.233 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.233 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.233 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:07.233 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184949+0000", 2024-09-04T17:47:07.233 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1880999999999999e-05 2024-09-04T17:47:07.233 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.233 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.233 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:07.233 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184969+0000", 2024-09-04T17:47:07.233 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0758e-05 2024-09-04T17:47:07.233 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.234 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.234 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:07.234 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.184983+0000", 2024-09-04T17:47:07.234 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3336000000000001e-05 2024-09-04T17:47:07.234 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.234 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.234 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:07.234 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228456+0000", 2024-09-04T17:47:07.234 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043472960999999997 2024-09-04T17:47:07.234 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.234 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.234 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:07.234 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228470+0000", 2024-09-04T17:47:07.234 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4163e-05 2024-09-04T17:47:07.235 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:07.235 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:07.235 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:07.235 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.235 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.235 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14400 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:07.235 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.185003+0000", 2024-09-04T17:47:07.235 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 48.997347851000001, 2024-09-04T17:47:07.235 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043532659000000001, 2024-09-04T17:47:07.235 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:07.235 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:07.235 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:07.235 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:07.236 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:07.236 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:07.236 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14400 2024-09-04T17:47:07.236 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.236 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:07.236 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.236 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:07.236 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185003+0000", 2024-09-04T17:47:07.236 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:07.236 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.236 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.236 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:07.236 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185003+0000", 2024-09-04T17:47:07.236 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:07.237 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.237 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.237 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:07.237 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185004+0000", 2024-09-04T17:47:07.237 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.294e-06 2024-09-04T17:47:07.237 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.237 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.237 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:07.237 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185010+0000", 2024-09-04T17:47:07.237 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7690000000000002e-06 2024-09-04T17:47:07.237 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.237 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.237 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:07.237 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185029+0000", 2024-09-04T17:47:07.238 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9100999999999999e-05 2024-09-04T17:47:07.238 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.238 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.238 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:07.238 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185043+0000", 2024-09-04T17:47:07.238 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3862e-05 2024-09-04T17:47:07.238 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.238 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.238 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:07.238 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228510+0000", 2024-09-04T17:47:07.239 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043467766999999997 2024-09-04T17:47:07.239 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.239 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.239 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:07.239 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228535+0000", 2024-09-04T17:47:07.239 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4865999999999999e-05 2024-09-04T17:47:07.239 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:07.239 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:07.239 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:07.239 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.239 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.239 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14401 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:07.239 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.185062+0000", 2024-09-04T17:47:07.239 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 48.997288586000003, 2024-09-04T17:47:07.240 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043556096000000002, 2024-09-04T17:47:07.240 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:07.240 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:07.240 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:07.240 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:07.240 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:07.240 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:07.240 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14401 2024-09-04T17:47:07.240 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.240 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:07.240 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.240 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:07.240 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185062+0000", 2024-09-04T17:47:07.241 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:07.241 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.241 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.241 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:07.241 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185062+0000", 2024-09-04T17:47:07.241 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:07.241 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.241 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.241 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:07.241 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185063+0000", 2024-09-04T17:47:07.241 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.409e-06 2024-09-04T17:47:07.241 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.241 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.241 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:07.242 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185069+0000", 2024-09-04T17:47:07.242 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.5400000000000003e-06 2024-09-04T17:47:07.242 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.242 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.242 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:07.242 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185087+0000", 2024-09-04T17:47:07.242 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8195000000000001e-05 2024-09-04T17:47:07.242 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.242 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.243 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:07.243 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185101+0000", 2024-09-04T17:47:07.243 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3769e-05 2024-09-04T17:47:07.243 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.243 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.243 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:07.243 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228604+0000", 2024-09-04T17:47:07.243 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043503537000000002 2024-09-04T17:47:07.243 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.243 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.243 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:07.243 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228618+0000", 2024-09-04T17:47:07.243 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3645999999999999e-05 2024-09-04T17:47:07.243 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:07.244 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:07.244 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:07.244 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.244 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.244 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14402 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:07.244 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.185121+0000", 2024-09-04T17:47:07.244 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 48.997229292, 2024-09-04T17:47:07.244 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043551542999999998, 2024-09-04T17:47:07.244 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:07.244 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:07.244 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:07.244 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:07.244 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:07.245 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:07.245 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14402 2024-09-04T17:47:07.245 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.245 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:07.245 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.245 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:07.245 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185121+0000", 2024-09-04T17:47:07.245 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:07.245 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.245 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.245 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:07.245 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185121+0000", 2024-09-04T17:47:07.245 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:07.246 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.246 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.246 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:07.246 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185122+0000", 2024-09-04T17:47:07.246 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1769999999999999e-06 2024-09-04T17:47:07.246 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.246 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.246 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:07.246 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185128+0000", 2024-09-04T17:47:07.246 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7790000000000001e-06 2024-09-04T17:47:07.246 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.246 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.246 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:07.246 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185146+0000", 2024-09-04T17:47:07.247 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8114999999999999e-05 2024-09-04T17:47:07.247 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.247 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.247 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:07.247 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185160+0000", 2024-09-04T17:47:07.247 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4158999999999999e-05 2024-09-04T17:47:07.247 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.247 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.247 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:07.247 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228659+0000", 2024-09-04T17:47:07.247 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043498715 2024-09-04T17:47:07.247 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.247 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.248 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:07.248 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228673+0000", 2024-09-04T17:47:07.248 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3597999999999999e-05 2024-09-04T17:47:07.248 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:07.248 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:07.248 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:07.248 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.248 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.248 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14403 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:07.248 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.185183+0000", 2024-09-04T17:47:07.248 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 48.997167244000003, 2024-09-04T17:47:07.248 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043541467, 2024-09-04T17:47:07.248 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:07.249 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:07.249 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:07.249 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:07.249 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:07.249 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:07.249 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14403 2024-09-04T17:47:07.249 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.249 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:07.249 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.249 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:07.249 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185183+0000", 2024-09-04T17:47:07.249 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:07.249 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.249 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.250 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:07.250 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185183+0000", 2024-09-04T17:47:07.250 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:07.250 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.250 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.250 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:07.250 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185184+0000", 2024-09-04T17:47:07.250 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.156e-06 2024-09-04T17:47:07.250 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.250 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.250 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:07.250 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185190+0000", 2024-09-04T17:47:07.250 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1800000000000004e-06 2024-09-04T17:47:07.250 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.251 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.251 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:07.251 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185207+0000", 2024-09-04T17:47:07.251 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7288000000000001e-05 2024-09-04T17:47:07.251 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.251 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.252 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:07.252 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185221+0000", 2024-09-04T17:47:07.252 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3729e-05 2024-09-04T17:47:07.252 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.252 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.252 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:07.252 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228711+0000", 2024-09-04T17:47:07.252 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043490595 2024-09-04T17:47:07.252 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.252 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.252 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:07.253 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228725+0000", 2024-09-04T17:47:07.253 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3519e-05 2024-09-04T17:47:07.253 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:07.253 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:07.253 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:07.253 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.253 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.253 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14404 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:07.253 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.185239+0000", 2024-09-04T17:47:07.253 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 48.997111105999998, 2024-09-04T17:47:07.253 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043542099000000001, 2024-09-04T17:47:07.253 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:07.253 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:07.254 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:07.254 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:07.254 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:07.254 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:07.254 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14404 2024-09-04T17:47:07.254 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.254 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:07.254 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.254 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:07.254 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185239+0000", 2024-09-04T17:47:07.254 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:07.254 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.254 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.254 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:07.255 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185239+0000", 2024-09-04T17:47:07.255 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:07.255 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.255 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.255 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:07.255 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185240+0000", 2024-09-04T17:47:07.255 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.173e-06 2024-09-04T17:47:07.255 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.255 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.255 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:07.255 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185246+0000", 2024-09-04T17:47:07.255 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7529999999999998e-06 2024-09-04T17:47:07.255 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.255 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.256 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:07.256 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185264+0000", 2024-09-04T17:47:07.256 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7872000000000001e-05 2024-09-04T17:47:07.256 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.256 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.256 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:07.256 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185278+0000", 2024-09-04T17:47:07.256 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3532e-05 2024-09-04T17:47:07.256 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.256 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.256 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:07.256 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228768+0000", 2024-09-04T17:47:07.256 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043490586999999997 2024-09-04T17:47:07.257 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.257 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.257 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:07.257 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228781+0000", 2024-09-04T17:47:07.257 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3182e-05 2024-09-04T17:47:07.257 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:07.257 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:07.257 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:07.257 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.257 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.257 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14405 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:07.257 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.185297+0000", 2024-09-04T17:47:07.257 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 48.997053915000002, 2024-09-04T17:47:07.257 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043535049999999999, 2024-09-04T17:47:07.258 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:07.258 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:07.258 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:07.258 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:07.258 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:07.258 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:07.258 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14405 2024-09-04T17:47:07.258 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.258 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:07.258 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.258 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:07.258 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185297+0000", 2024-09-04T17:47:07.258 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:07.259 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.259 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.259 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:07.259 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185297+0000", 2024-09-04T17:47:07.259 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:07.259 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.259 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.259 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:07.259 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185298+0000", 2024-09-04T17:47:07.259 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.22e-06 2024-09-04T17:47:07.259 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.259 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.259 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:07.260 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185308+0000", 2024-09-04T17:47:07.260 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0173000000000001e-05 2024-09-04T17:47:07.260 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.260 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.260 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:07.260 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185329+0000", 2024-09-04T17:47:07.260 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1087999999999999e-05 2024-09-04T17:47:07.260 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.260 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.260 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:07.260 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.185345+0000", 2024-09-04T17:47:07.260 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6035000000000001e-05 2024-09-04T17:47:07.260 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.261 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.261 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:07.261 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228822+0000", 2024-09-04T17:47:07.261 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.043477427999999999 2024-09-04T17:47:07.261 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.261 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.261 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:07.261 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228832+0000", 2024-09-04T17:47:07.261 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1060000000000006e-06 2024-09-04T17:47:07.261 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:07.261 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:07.261 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:07.261 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.262 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.262 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:14411 3.d 3.87bc6ed (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:07.262 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:18.186171+0000", 2024-09-04T17:47:07.262 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 48.996179091999998, 2024-09-04T17:47:07.262 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.04271581, 2024-09-04T17:47:07.262 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:07.262 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:07.262 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:07.262 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:07.262 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:07.262 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:07.262 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 14411 2024-09-04T17:47:07.262 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.262 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:07.263 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.263 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:07.263 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.186171+0000", 2024-09-04T17:47:07.263 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:07.263 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.263 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.263 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:07.263 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.186171+0000", 2024-09-04T17:47:07.263 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:07.263 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.263 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.263 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:07.263 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.186173+0000", 2024-09-04T17:47:07.263 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.012e-06 2024-09-04T17:47:07.264 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.264 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.264 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:07.264 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.186183+0000", 2024-09-04T17:47:07.264 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.7729999999999994e-06 2024-09-04T17:47:07.264 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.264 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.264 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:07.264 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.186205+0000", 2024-09-04T17:47:07.264 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2280999999999999e-05 2024-09-04T17:47:07.264 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.264 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.264 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:07.265 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.186224+0000", 2024-09-04T17:47:07.265 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8695e-05 2024-09-04T17:47:07.265 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.265 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.265 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:07.265 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228873+0000", 2024-09-04T17:47:07.265 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.042648798000000002 2024-09-04T17:47:07.265 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.265 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.265 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:07.265 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:18.228887+0000", 2024-09-04T17:47:07.265 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4251e-05 2024-09-04T17:47:07.265 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:07.265 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:07.266 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:07.266 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.266 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.266 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:37359 3.f 3:f29e50a5:::rbd_data.1134c5a43465.0000000000000021:head [stat,write 62464~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-04T17:47:07.266 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.616887+0000", 2024-09-04T17:47:07.266 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 27.565463176000002, 2024-09-04T17:47:07.266 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.054183104000000003, 2024-09-04T17:47:07.266 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:07.266 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:07.266 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:07.266 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:07.266 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:07.266 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:07.267 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 37359 2024-09-04T17:47:07.267 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.267 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:07.267 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.267 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:07.267 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.616887+0000", 2024-09-04T17:47:07.267 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:07.267 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.267 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.267 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:07.267 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.616887+0000", 2024-09-04T17:47:07.267 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:07.267 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.267 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.268 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:07.268 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.616889+0000", 2024-09-04T17:47:07.268 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.362e-06 2024-09-04T17:47:07.268 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.268 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.268 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:07.268 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.616904+0000", 2024-09-04T17:47:07.268 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.507e-05 2024-09-04T17:47:07.268 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.268 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.268 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:07.268 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.616920+0000", 2024-09-04T17:47:07.268 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6385999999999999e-05 2024-09-04T17:47:07.269 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.269 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.269 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:07.269 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.616929+0000", 2024-09-04T17:47:07.269 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8540000000000003e-06 2024-09-04T17:47:07.269 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.269 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.269 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:07.269 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.616983+0000", 2024-09-04T17:47:07.269 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4218999999999997e-05 2024-09-04T17:47:07.269 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.269 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.269 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:07.269 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617065+0000", 2024-09-04T17:47:07.270 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.1864000000000001e-05 2024-09-04T17:47:07.270 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.270 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.270 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 4", 2024-09-04T17:47:07.270 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617205+0000", 2024-09-04T17:47:07.270 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013962600000000001 2024-09-04T17:47:07.270 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.270 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.270 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:47:07.270 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.618670+0000", 2024-09-04T17:47:07.270 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.001465647 2024-09-04T17:47:07.270 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.270 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.270 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:07.271 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670962+0000", 2024-09-04T17:47:07.271 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.052292000999999998 2024-09-04T17:47:07.271 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.271 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.271 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:47:07.271 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671011+0000", 2024-09-04T17:47:07.271 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9036000000000003e-05 2024-09-04T17:47:07.271 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.271 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.271 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:07.271 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671070+0000", 2024-09-04T17:47:07.271 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.9039000000000002e-05 2024-09-04T17:47:07.271 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:07.271 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:07.272 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:07.272 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.272 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.272 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:51399 3.11 3:8a70365a:::rbd_data.11341db94a06.000000000000000b:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:07.272 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:03.713851+0000", 2024-09-04T17:47:07.272 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 3.4684992480000001, 2024-09-04T17:47:07.272 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051250498999999998, 2024-09-04T17:47:07.272 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:07.272 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:07.272 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:07.272 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:07.272 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:07.272 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:07.273 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 51399 2024-09-04T17:47:07.273 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.273 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:07.273 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.273 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:07.273 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:03.713851+0000", 2024-09-04T17:47:07.273 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:07.273 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.273 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.273 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:07.273 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:03.713851+0000", 2024-09-04T17:47:07.273 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:07.273 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.273 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.274 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:07.274 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:03.713854+0000", 2024-09-04T17:47:07.274 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.8569999999999999e-06 2024-09-04T17:47:07.274 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.274 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.274 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:07.274 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:03.713866+0000", 2024-09-04T17:47:07.274 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2422999999999999e-05 2024-09-04T17:47:07.274 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.274 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.274 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:07.274 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:03.713898+0000", 2024-09-04T17:47:07.274 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.1628999999999998e-05 2024-09-04T17:47:07.275 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.275 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.275 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:07.275 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:03.713913+0000", 2024-09-04T17:47:07.275 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.506e-05 2024-09-04T17:47:07.275 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.275 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.275 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:07.275 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:03.714028+0000", 2024-09-04T17:47:07.275 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00011502600000000001 2024-09-04T17:47:07.275 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.275 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.275 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:07.275 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:03.714261+0000", 2024-09-04T17:47:07.276 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00023331 2024-09-04T17:47:07.276 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.276 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.276 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 4", 2024-09-04T17:47:07.276 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:03.724871+0000", 2024-09-04T17:47:07.276 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.010609766 2024-09-04T17:47:07.276 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.276 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.276 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:07.276 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:03.764764+0000", 2024-09-04T17:47:07.276 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.039893177000000002 2024-09-04T17:47:07.276 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.276 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.277 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:47:07.277 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:03.764822+0000", 2024-09-04T17:47:07.277 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7584999999999998e-05 2024-09-04T17:47:07.277 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.277 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.277 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:47:07.277 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:03.764969+0000", 2024-09-04T17:47:07.277 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014708899999999999 2024-09-04T17:47:07.277 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:07.277 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:07.277 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:07.277 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:03.765102+0000", 2024-09-04T17:47:07.277 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013257699999999999 2024-09-04T17:47:07.277 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:07.278 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:07.278 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:07.278 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:07.278 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:07.278 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:07.278 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:07.181+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:07.279 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-04T17:47:07.294 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:07.294 INFO:teuthology.orchestra.run.smithi092.stdout:3638 writesame 0x4b00072 thru 0x4b0eb9b data_size 0x1f(0xeb2a bytes) 2024-09-04T17:47:07.321 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:47:07.321+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:47:07.331 INFO:teuthology.orchestra.run.smithi092.stdout:3639 clone 28 order 24 su 262144 sc 9 2024-09-04T17:47:07.385 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:07.382+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:07.402 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:07.402 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:47:07.402 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:47:07.403 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:07.416 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-04T17:47:07.485 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:07.485+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:07.581 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:07.581 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:47:07.581 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:47:07.581 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:47:07.581 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:07.586 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:07.585+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:07.595 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-04T17:47:07.639 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is set 2024-09-04T17:47:07.716 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:07.714+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:07.723 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:07.723 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:47:07.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:47:07.723 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:47:07.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.723 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:27242 3.b 3.34414deb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-04T17:47:07.724 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:28.402347+0000", 2024-09-04T17:47:07.724 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 39.319392847000003, 2024-09-04T17:47:07.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.93265329299999999, 2024-09-04T17:47:07.724 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:07.724 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:07.724 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-04T17:47:07.724 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:07.724 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:07.724 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:07.724 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 27242 2024-09-04T17:47:07.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.724 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:07.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:07.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402347+0000", 2024-09-04T17:47:07.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:07.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:07.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402347+0000", 2024-09-04T17:47:07.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:07.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:07.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402349+0000", 2024-09-04T17:47:07.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.471e-06 2024-09-04T17:47:07.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.726 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:07.726 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402355+0000", 2024-09-04T17:47:07.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8399999999999997e-06 2024-09-04T17:47:07.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.726 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:07.726 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402364+0000", 2024-09-04T17:47:07.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1269999999999992e-06 2024-09-04T17:47:07.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.726 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:07.727 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402373+0000", 2024-09-04T17:47:07.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7909999999999993e-06 2024-09-04T17:47:07.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.727 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.727 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:07.727 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402460+0000", 2024-09-04T17:47:07.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6686000000000003e-05 2024-09-04T17:47:07.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.727 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.727 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-04T17:47:07.727 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402473+0000", 2024-09-04T17:47:07.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3281e-05 2024-09-04T17:47:07.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:07.728 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:29.334989+0000", 2024-09-04T17:47:07.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.93251549099999997 2024-09-04T17:47:07.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:07.728 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:29.335000+0000", 2024-09-04T17:47:07.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1606e-05 2024-09-04T17:47:07.728 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:07.728 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:07.728 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:07.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.729 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:33552 3.7 3:e79ad1db:::rbd_data.11342e8f3384.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-04T17:47:07.729 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:35.999748+0000", 2024-09-04T17:47:07.729 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 31.721991797000001, 2024-09-04T17:47:07.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045683823999999998, 2024-09-04T17:47:07.729 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:07.729 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:07.729 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:07.729 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:07.729 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:07.729 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:07.729 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 33552 2024-09-04T17:47:07.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.730 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:07.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.730 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:07.730 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999748+0000", 2024-09-04T17:47:07.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:07.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.730 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:07.730 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999748+0000", 2024-09-04T17:47:07.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:07.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.730 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:07.730 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999750+0000", 2024-09-04T17:47:07.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1299999999999999e-06 2024-09-04T17:47:07.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.731 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:07.731 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999759+0000", 2024-09-04T17:47:07.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4649999999999994e-06 2024-09-04T17:47:07.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.731 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:07.731 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999778+0000", 2024-09-04T17:47:07.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8833000000000001e-05 2024-09-04T17:47:07.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:07.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999786+0000", 2024-09-04T17:47:07.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7369999999999995e-06 2024-09-04T17:47:07.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:07.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999856+0000", 2024-09-04T17:47:07.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9247999999999998e-05 2024-09-04T17:47:07.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:07.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999973+0000", 2024-09-04T17:47:07.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000117921 2024-09-04T17:47:07.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 7", 2024-09-04T17:47:07.733 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.005955+0000", 2024-09-04T17:47:07.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059820560000000004 2024-09-04T17:47:07.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:07.733 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.045297+0000", 2024-09-04T17:47:07.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.039341650999999998 2024-09-04T17:47:07.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:07.734 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.045343+0000", 2024-09-04T17:47:07.734 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5899000000000002e-05 2024-09-04T17:47:07.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.734 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:07.734 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.045388+0000", 2024-09-04T17:47:07.734 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4601999999999997e-05 2024-09-04T17:47:07.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.734 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:07.734 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.045432+0000", 2024-09-04T17:47:07.734 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4282000000000001e-05 2024-09-04T17:47:07.735 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:07.735 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:07.735 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:07.735 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.735 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.735 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:37357 3.15 3:a897312a:::rbd_data.1134c5a43465.000000000000001f:head [stat,write 62464~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-04T17:47:07.735 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:39.616724+0000", 2024-09-04T17:47:07.735 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.105016388999999, 2024-09-04T17:47:07.735 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054965854000000001, 2024-09-04T17:47:07.735 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:07.735 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:07.735 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:07.735 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:07.736 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:07.736 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:07.736 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 37357 2024-09-04T17:47:07.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.736 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:07.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.736 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:07.736 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616724+0000", 2024-09-04T17:47:07.736 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:07.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.736 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:07.736 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616724+0000", 2024-09-04T17:47:07.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:07.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:07.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616725+0000", 2024-09-04T17:47:07.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.815e-06 2024-09-04T17:47:07.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:07.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616739+0000", 2024-09-04T17:47:07.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3723000000000001e-05 2024-09-04T17:47:07.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:07.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616755+0000", 2024-09-04T17:47:07.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5964000000000002e-05 2024-09-04T17:47:07.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:07.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616763+0000", 2024-09-04T17:47:07.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3869999999999995e-06 2024-09-04T17:47:07.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:07.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616825+0000", 2024-09-04T17:47:07.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1854999999999996e-05 2024-09-04T17:47:07.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.739 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.739 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:07.739 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616895+0000", 2024-09-04T17:47:07.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9597999999999993e-05 2024-09-04T17:47:07.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.739 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.739 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-04T17:47:07.739 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616991+0000", 2024-09-04T17:47:07.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5828000000000005e-05 2024-09-04T17:47:07.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.739 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.739 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:07.740 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.617659+0000", 2024-09-04T17:47:07.740 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00066821600000000001 2024-09-04T17:47:07.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.740 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.740 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:07.740 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.671482+0000", 2024-09-04T17:47:07.740 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053823533999999999 2024-09-04T17:47:07.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.740 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.740 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:07.740 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.671579+0000", 2024-09-04T17:47:07.740 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6657999999999995e-05 2024-09-04T17:47:07.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:07.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.671689+0000", 2024-09-04T17:47:07.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000110276 2024-09-04T17:47:07.741 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:07.741 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:07.741 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:07.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.742 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:43873 3.15 3:abd207ee:::rbd_data.113410ddd206.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:07.742 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:49.932940+0000", 2024-09-04T17:47:07.742 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.788799835999999, 2024-09-04T17:47:07.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042740514, 2024-09-04T17:47:07.742 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:07.742 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:07.742 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:07.742 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:07.742 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:07.742 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:07.742 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 43873 2024-09-04T17:47:07.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.742 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:07.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.743 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:07.743 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.932940+0000", 2024-09-04T17:47:07.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:07.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.743 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:07.743 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.932940+0000", 2024-09-04T17:47:07.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:07.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.743 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:07.743 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.932941+0000", 2024-09-04T17:47:07.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.404e-06 2024-09-04T17:47:07.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.744 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:07.744 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.933065+0000", 2024-09-04T17:47:07.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012361199999999999 2024-09-04T17:47:07.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.744 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:07.744 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.933078+0000", 2024-09-04T17:47:07.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3182e-05 2024-09-04T17:47:07.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:07.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.933085+0000", 2024-09-04T17:47:07.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4500000000000001e-06 2024-09-04T17:47:07.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:07.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.933134+0000", 2024-09-04T17:47:07.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9150999999999997e-05 2024-09-04T17:47:07.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:07.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.933242+0000", 2024-09-04T17:47:07.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010781 2024-09-04T17:47:07.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-04T17:47:07.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.938358+0000", 2024-09-04T17:47:07.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051160349999999997 2024-09-04T17:47:07.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:07.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.975343+0000", 2024-09-04T17:47:07.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036985663000000002 2024-09-04T17:47:07.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:07.747 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.975392+0000", 2024-09-04T17:47:07.747 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9044e-05 2024-09-04T17:47:07.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.747 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.747 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:07.747 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.975548+0000", 2024-09-04T17:47:07.747 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015572900000000001 2024-09-04T17:47:07.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.747 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.747 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:07.747 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.975681+0000", 2024-09-04T17:47:07.747 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013243400000000001 2024-09-04T17:47:07.747 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:07.748 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:07.748 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:07.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.748 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.748 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:46649 3.7 3:e0749ed5:::rbd_data.1134326b938e.000000000000014f:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-04T17:47:07.748 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:54.114730+0000", 2024-09-04T17:47:07.748 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.607009703999999, 2024-09-04T17:47:07.748 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032972972000000003, 2024-09-04T17:47:07.748 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:07.748 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:07.748 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:07.748 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:07.749 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:07.749 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:07.749 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 46649 2024-09-04T17:47:07.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.749 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:07.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.749 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:07.749 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114730+0000", 2024-09-04T17:47:07.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:07.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.749 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:07.749 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114730+0000", 2024-09-04T17:47:07.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:07.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:07.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114731+0000", 2024-09-04T17:47:07.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.147e-06 2024-09-04T17:47:07.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:07.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114737+0000", 2024-09-04T17:47:07.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6380000000000001e-06 2024-09-04T17:47:07.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:07.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114751+0000", 2024-09-04T17:47:07.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4012999999999999e-05 2024-09-04T17:47:07.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:07.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114758+0000", 2024-09-04T17:47:07.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1539999999999996e-06 2024-09-04T17:47:07.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:07.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114876+0000", 2024-09-04T17:47:07.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000117421 2024-09-04T17:47:07.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.752 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:07.752 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.115173+0000", 2024-09-04T17:47:07.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00029750799999999998 2024-09-04T17:47:07.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.752 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 7", 2024-09-04T17:47:07.752 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.120273+0000", 2024-09-04T17:47:07.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051002900000000004 2024-09-04T17:47:07.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.752 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:07.753 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.147571+0000", 2024-09-04T17:47:07.753 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027297906 2024-09-04T17:47:07.753 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.753 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:07.753 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.147588+0000", 2024-09-04T17:47:07.753 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6807999999999999e-05 2024-09-04T17:47:07.753 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.753 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:07.753 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.147637+0000", 2024-09-04T17:47:07.753 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8466e-05 2024-09-04T17:47:07.753 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:07.754 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.147703+0000", 2024-09-04T17:47:07.754 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6620999999999998e-05 2024-09-04T17:47:07.754 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:07.754 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:07.754 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:07.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.754 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49801 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 720896~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:07.754 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.075094+0000", 2024-09-04T17:47:07.754 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.6466457170000002, 2024-09-04T17:47:07.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017824974, 2024-09-04T17:47:07.755 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:07.755 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:07.755 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:07.755 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:07.755 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:07.755 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:07.755 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49801 2024-09-04T17:47:07.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.755 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:07.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:07.756 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075094+0000", 2024-09-04T17:47:07.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:07.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.756 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:07.756 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075094+0000", 2024-09-04T17:47:07.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:07.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.756 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:07.756 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075095+0000", 2024-09-04T17:47:07.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.017e-06 2024-09-04T17:47:07.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.757 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:07.757 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075101+0000", 2024-09-04T17:47:07.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5029999999999997e-06 2024-09-04T17:47:07.757 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.757 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:07.757 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075119+0000", 2024-09-04T17:47:07.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8133000000000001e-05 2024-09-04T17:47:07.757 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.758 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:07.758 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075132+0000", 2024-09-04T17:47:07.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3305999999999999e-05 2024-09-04T17:47:07.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.758 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:07.758 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.092677+0000", 2024-09-04T17:47:07.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017544466000000002 2024-09-04T17:47:07.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:07.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.092782+0000", 2024-09-04T17:47:07.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000105284 2024-09-04T17:47:07.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:07.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.092919+0000", 2024-09-04T17:47:07.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013726499999999999 2024-09-04T17:47:07.759 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:07.759 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:07.760 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:07.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.760 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49809 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 163840~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:07.760 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077146+0000", 2024-09-04T17:47:07.760 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.6445942740000001, 2024-09-04T17:47:07.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017984541999999999, 2024-09-04T17:47:07.760 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:07.760 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:07.760 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:07.760 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:07.760 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:07.760 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:07.761 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49809 2024-09-04T17:47:07.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.761 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:07.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:07.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077146+0000", 2024-09-04T17:47:07.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:07.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:07.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077146+0000", 2024-09-04T17:47:07.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:07.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.762 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:07.762 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077147+0000", 2024-09-04T17:47:07.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0330000000000001e-06 2024-09-04T17:47:07.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.762 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:07.762 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077164+0000", 2024-09-04T17:47:07.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7810000000000001e-05 2024-09-04T17:47:07.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.763 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:07.763 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077187+0000", 2024-09-04T17:47:07.763 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2246e-05 2024-09-04T17:47:07.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.763 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:07.763 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077203+0000", 2024-09-04T17:47:07.763 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6392000000000002e-05 2024-09-04T17:47:07.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.763 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:07.763 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.094885+0000", 2024-09-04T17:47:07.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017681589000000001 2024-09-04T17:47:07.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:07.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.094991+0000", 2024-09-04T17:47:07.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010668899999999999 2024-09-04T17:47:07.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:07.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095130+0000", 2024-09-04T17:47:07.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013878299999999999 2024-09-04T17:47:07.764 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:07.764 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:07.765 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:07.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.765 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49810 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 229376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:07.765 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077225+0000", 2024-09-04T17:47:07.765 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.6445144330000003, 2024-09-04T17:47:07.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018218793, 2024-09-04T17:47:07.765 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:07.765 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:07.765 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:07.765 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:07.765 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:07.766 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:07.766 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49810 2024-09-04T17:47:07.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.766 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:07.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.766 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:07.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077225+0000", 2024-09-04T17:47:07.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:07.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.766 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:07.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077225+0000", 2024-09-04T17:47:07.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:07.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.767 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:07.767 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077227+0000", 2024-09-04T17:47:07.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1829999999999999e-06 2024-09-04T17:47:07.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.767 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:07.767 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077233+0000", 2024-09-04T17:47:07.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3899999999999998e-06 2024-09-04T17:47:07.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.767 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:07.768 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077253+0000", 2024-09-04T17:47:07.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0009000000000001e-05 2024-09-04T17:47:07.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.768 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:07.768 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077270+0000", 2024-09-04T17:47:07.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7336000000000001e-05 2024-09-04T17:47:07.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.768 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:07.768 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095177+0000", 2024-09-04T17:47:07.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017907101000000002 2024-09-04T17:47:07.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:07.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095299+0000", 2024-09-04T17:47:07.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000121749 2024-09-04T17:47:07.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:07.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095444+0000", 2024-09-04T17:47:07.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014502499999999999 2024-09-04T17:47:07.769 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:07.769 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:07.769 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:07.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.770 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49811 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 294912~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:07.770 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077288+0000", 2024-09-04T17:47:07.770 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.6444521190000003, 2024-09-04T17:47:07.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018443620000000001, 2024-09-04T17:47:07.770 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:07.770 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:07.770 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:07.770 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:07.770 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:07.770 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:07.771 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49811 2024-09-04T17:47:07.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.771 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:07.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:07.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077288+0000", 2024-09-04T17:47:07.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:07.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:07.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077288+0000", 2024-09-04T17:47:07.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:07.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.772 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:07.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077289+0000", 2024-09-04T17:47:07.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6199999999999999e-06 2024-09-04T17:47:07.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.772 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:07.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077295+0000", 2024-09-04T17:47:07.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1499999999999998e-06 2024-09-04T17:47:07.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.772 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:07.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077312+0000", 2024-09-04T17:47:07.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7353000000000001e-05 2024-09-04T17:47:07.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.773 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:07.773 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077324+0000", 2024-09-04T17:47:07.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2593e-05 2024-09-04T17:47:07.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.773 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:07.773 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095490+0000", 2024-09-04T17:47:07.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018165976 2024-09-04T17:47:07.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:07.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095597+0000", 2024-09-04T17:47:07.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010629900000000001 2024-09-04T17:47:07.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:07.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095731+0000", 2024-09-04T17:47:07.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013462899999999999 2024-09-04T17:47:07.774 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:07.774 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:07.774 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:07.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.775 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49813 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 360448~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:07.775 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077349+0000", 2024-09-04T17:47:07.775 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.644390917, 2024-09-04T17:47:07.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018780305000000001, 2024-09-04T17:47:07.775 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:07.775 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:07.775 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:07.775 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:07.775 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:07.775 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:07.775 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49813 2024-09-04T17:47:07.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.776 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:07.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:07.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077349+0000", 2024-09-04T17:47:07.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:07.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:07.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077349+0000", 2024-09-04T17:47:07.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:07.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:07.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077350+0000", 2024-09-04T17:47:07.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0330000000000001e-06 2024-09-04T17:47:07.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.777 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:07.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077355+0000", 2024-09-04T17:47:07.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0699999999999997e-06 2024-09-04T17:47:07.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.777 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:07.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077376+0000", 2024-09-04T17:47:07.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0846e-05 2024-09-04T17:47:07.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:07.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077388+0000", 2024-09-04T17:47:07.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2353e-05 2024-09-04T17:47:07.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:07.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095819+0000", 2024-09-04T17:47:07.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01843057 2024-09-04T17:47:07.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:07.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095960+0000", 2024-09-04T17:47:07.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000140965 2024-09-04T17:47:07.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:07.779 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096129+0000", 2024-09-04T17:47:07.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016946800000000001 2024-09-04T17:47:07.779 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:07.779 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:07.779 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:07.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.780 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49814 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 425984~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:07.780 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077407+0000", 2024-09-04T17:47:07.780 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.6443324160000001, 2024-09-04T17:47:07.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019041563000000001, 2024-09-04T17:47:07.780 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:07.780 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:07.780 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:07.780 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:07.780 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:07.780 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:07.780 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49814 2024-09-04T17:47:07.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.781 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:07.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:07.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077407+0000", 2024-09-04T17:47:07.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:07.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:07.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077407+0000", 2024-09-04T17:47:07.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:07.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:07.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077409+0000", 2024-09-04T17:47:07.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2330000000000001e-06 2024-09-04T17:47:07.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:07.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077415+0000", 2024-09-04T17:47:07.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2729999999999999e-06 2024-09-04T17:47:07.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:07.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077436+0000", 2024-09-04T17:47:07.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1209999999999999e-05 2024-09-04T17:47:07.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:07.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077448+0000", 2024-09-04T17:47:07.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2225999999999999e-05 2024-09-04T17:47:07.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:07.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096183+0000", 2024-09-04T17:47:07.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018735004 2024-09-04T17:47:07.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:07.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096297+0000", 2024-09-04T17:47:07.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000113962 2024-09-04T17:47:07.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:07.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096449+0000", 2024-09-04T17:47:07.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000151655 2024-09-04T17:47:07.784 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:07.784 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:07.784 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:07.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.785 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49815 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 491520~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:07.785 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077471+0000", 2024-09-04T17:47:07.785 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.6442691780000001, 2024-09-04T17:47:07.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019266536000000001, 2024-09-04T17:47:07.785 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:07.785 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:07.785 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:07.785 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:07.785 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:07.785 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:07.785 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49815 2024-09-04T17:47:07.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.786 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:07.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:07.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077471+0000", 2024-09-04T17:47:07.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:07.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:07.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077471+0000", 2024-09-04T17:47:07.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:07.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:07.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077472+0000", 2024-09-04T17:47:07.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.17e-06 2024-09-04T17:47:07.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:07.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077478+0000", 2024-09-04T17:47:07.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0399999999999998e-06 2024-09-04T17:47:07.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:07.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077500+0000", 2024-09-04T17:47:07.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1959000000000001e-05 2024-09-04T17:47:07.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:07.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077513+0000", 2024-09-04T17:47:07.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2823000000000001e-05 2024-09-04T17:47:07.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:07.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096493+0000", 2024-09-04T17:47:07.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018980701999999999 2024-09-04T17:47:07.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:07.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096603+0000", 2024-09-04T17:47:07.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010942000000000001 2024-09-04T17:47:07.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:07.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096737+0000", 2024-09-04T17:47:07.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013442200000000001 2024-09-04T17:47:07.789 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:07.789 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:07.789 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:07.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.790 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49816 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 557056~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:07.790 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077532+0000", 2024-09-04T17:47:07.790 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.6442074370000004, 2024-09-04T17:47:07.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019604331999999999, 2024-09-04T17:47:07.790 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:07.790 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:07.790 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:07.790 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:07.790 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:07.790 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:07.790 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49816 2024-09-04T17:47:07.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.790 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:07.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:07.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077532+0000", 2024-09-04T17:47:07.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:07.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:07.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077532+0000", 2024-09-04T17:47:07.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:07.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:07.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077534+0000", 2024-09-04T17:47:07.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1200000000000001e-06 2024-09-04T17:47:07.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:07.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077545+0000", 2024-09-04T17:47:07.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1673e-05 2024-09-04T17:47:07.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:07.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077569+0000", 2024-09-04T17:47:07.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3555999999999999e-05 2024-09-04T17:47:07.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:07.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077584+0000", 2024-09-04T17:47:07.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4883e-05 2024-09-04T17:47:07.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:07.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096825+0000", 2024-09-04T17:47:07.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019241734 2024-09-04T17:47:07.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:07.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096964+0000", 2024-09-04T17:47:07.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000138085 2024-09-04T17:47:07.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:07.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097137+0000", 2024-09-04T17:47:07.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017328100000000001 2024-09-04T17:47:07.794 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:07.794 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:07.794 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:07.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.794 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49817 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 622592~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:07.794 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077603+0000", 2024-09-04T17:47:07.795 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.6441368790000004, 2024-09-04T17:47:07.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01986928, 2024-09-04T17:47:07.795 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:07.795 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:07.795 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:07.795 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:07.795 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:07.795 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:07.795 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49817 2024-09-04T17:47:07.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.795 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:07.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:07.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077603+0000", 2024-09-04T17:47:07.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:07.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:07.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077603+0000", 2024-09-04T17:47:07.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:07.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:07.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077604+0000", 2024-09-04T17:47:07.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7699999999999992e-07 2024-09-04T17:47:07.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:07.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077609+0000", 2024-09-04T17:47:07.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0830000000000003e-06 2024-09-04T17:47:07.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:07.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077630+0000", 2024-09-04T17:47:07.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0438999999999999e-05 2024-09-04T17:47:07.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:07.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077664+0000", 2024-09-04T17:47:07.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4579000000000002e-05 2024-09-04T17:47:07.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:07.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097193+0000", 2024-09-04T17:47:07.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019529174 2024-09-04T17:47:07.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:07.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097323+0000", 2024-09-04T17:47:07.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000129696 2024-09-04T17:47:07.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:07.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097472+0000", 2024-09-04T17:47:07.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014933200000000001 2024-09-04T17:47:07.799 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:07.799 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:07.799 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:07.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.799 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49818 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 688128~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:07.799 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077683+0000", 2024-09-04T17:47:07.799 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.6440565149999999, 2024-09-04T17:47:07.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020086333000000001, 2024-09-04T17:47:07.799 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:07.799 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:07.800 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:07.800 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:07.800 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:07.800 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:07.800 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49818 2024-09-04T17:47:07.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.800 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:07.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:07.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077683+0000", 2024-09-04T17:47:07.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:07.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:07.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077683+0000", 2024-09-04T17:47:07.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:07.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:07.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077684+0000", 2024-09-04T17:47:07.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4e-07 2024-09-04T17:47:07.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:07.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077690+0000", 2024-09-04T17:47:07.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4729999999999999e-06 2024-09-04T17:47:07.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:07.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077710+0000", 2024-09-04T17:47:07.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0149999999999999e-05 2024-09-04T17:47:07.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:07.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077723+0000", 2024-09-04T17:47:07.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3271999999999999e-05 2024-09-04T17:47:07.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:07.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097521+0000", 2024-09-04T17:47:07.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019797981999999999 2024-09-04T17:47:07.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:07.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097628+0000", 2024-09-04T17:47:07.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010665700000000001 2024-09-04T17:47:07.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:07.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097770+0000", 2024-09-04T17:47:07.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014185900000000001 2024-09-04T17:47:07.803 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:07.804 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:07.804 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:07.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.804 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49820 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 753664~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:07.804 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077744+0000", 2024-09-04T17:47:07.804 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.6439955839999998, 2024-09-04T17:47:07.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020364786999999999, 2024-09-04T17:47:07.804 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:07.804 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:07.804 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:07.804 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:07.805 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:07.805 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:07.805 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49820 2024-09-04T17:47:07.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.805 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:07.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:07.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077744+0000", 2024-09-04T17:47:07.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:07.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:07.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077744+0000", 2024-09-04T17:47:07.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:07.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:07.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077745+0000", 2024-09-04T17:47:07.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0270000000000001e-06 2024-09-04T17:47:07.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:07.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077751+0000", 2024-09-04T17:47:07.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.626e-06 2024-09-04T17:47:07.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:07.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077771+0000", 2024-09-04T17:47:07.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0469999999999999e-05 2024-09-04T17:47:07.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:07.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077785+0000", 2024-09-04T17:47:07.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3129000000000001e-05 2024-09-04T17:47:07.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:07.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097816+0000", 2024-09-04T17:47:07.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020031917999999999 2024-09-04T17:47:07.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:07.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097939+0000", 2024-09-04T17:47:07.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000122533 2024-09-04T17:47:07.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:07.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098109+0000", 2024-09-04T17:47:07.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017008400000000001 2024-09-04T17:47:07.808 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:07.809 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:07.809 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:07.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.809 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49821 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 819200~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:07.809 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077807+0000", 2024-09-04T17:47:07.809 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.6439333720000002, 2024-09-04T17:47:07.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020702928999999998, 2024-09-04T17:47:07.809 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:07.809 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:07.809 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:07.809 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:07.809 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:07.810 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:07.810 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49821 2024-09-04T17:47:07.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.810 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:07.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:07.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077807+0000", 2024-09-04T17:47:07.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:07.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:07.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077807+0000", 2024-09-04T17:47:07.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:07.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:07.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077808+0000", 2024-09-04T17:47:07.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.463e-06 2024-09-04T17:47:07.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:07.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077813+0000", 2024-09-04T17:47:07.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7530000000000001e-06 2024-09-04T17:47:07.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:07.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077832+0000", 2024-09-04T17:47:07.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.946e-05 2024-09-04T17:47:07.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:07.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077846+0000", 2024-09-04T17:47:07.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3786e-05 2024-09-04T17:47:07.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:07.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098201+0000", 2024-09-04T17:47:07.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020354797000000001 2024-09-04T17:47:07.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:07.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098341+0000", 2024-09-04T17:47:07.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000140042 2024-09-04T17:47:07.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:07.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098509+0000", 2024-09-04T17:47:07.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016862800000000001 2024-09-04T17:47:07.813 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:07.813 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:07.814 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:07.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.814 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49823 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 884736~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:07.814 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077865+0000", 2024-09-04T17:47:07.814 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.6438749279999998, 2024-09-04T17:47:07.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021008723, 2024-09-04T17:47:07.814 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:07.814 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:07.814 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:07.814 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:07.814 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:07.815 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:07.815 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49823 2024-09-04T17:47:07.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.815 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:07.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:07.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077865+0000", 2024-09-04T17:47:07.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:07.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:07.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077865+0000", 2024-09-04T17:47:07.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:07.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:07.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077866+0000", 2024-09-04T17:47:07.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.164e-06 2024-09-04T17:47:07.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:07.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077872+0000", 2024-09-04T17:47:07.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3959999999999997e-06 2024-09-04T17:47:07.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:07.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077892+0000", 2024-09-04T17:47:07.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0970000000000001e-05 2024-09-04T17:47:07.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:07.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077905+0000", 2024-09-04T17:47:07.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2312e-05 2024-09-04T17:47:07.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:07.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098579+0000", 2024-09-04T17:47:07.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020674486999999998 2024-09-04T17:47:07.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:07.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098705+0000", 2024-09-04T17:47:07.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012541900000000001 2024-09-04T17:47:07.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:07.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098874+0000", 2024-09-04T17:47:07.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000168975 2024-09-04T17:47:07.818 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:07.818 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:07.818 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:07.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.819 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49824 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 950272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:07.819 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077923+0000", 2024-09-04T17:47:07.819 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.6438172360000003, 2024-09-04T17:47:07.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021246287999999999, 2024-09-04T17:47:07.819 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:07.819 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:07.819 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:07.819 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:07.819 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:07.819 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:07.819 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49824 2024-09-04T17:47:07.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.820 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:07.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:07.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077923+0000", 2024-09-04T17:47:07.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:07.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:07.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077923+0000", 2024-09-04T17:47:07.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:07.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:07.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077924+0000", 2024-09-04T17:47:07.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0100000000000001e-06 2024-09-04T17:47:07.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:07.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077944+0000", 2024-09-04T17:47:07.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0449e-05 2024-09-04T17:47:07.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:07.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077965+0000", 2024-09-04T17:47:07.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0479000000000002e-05 2024-09-04T17:47:07.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:07.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077990+0000", 2024-09-04T17:47:07.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5582999999999999e-05 2024-09-04T17:47:07.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:07.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098924+0000", 2024-09-04T17:47:07.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020933395 2024-09-04T17:47:07.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:07.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.099032+0000", 2024-09-04T17:47:07.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010846299999999999 2024-09-04T17:47:07.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:07.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.099169+0000", 2024-09-04T17:47:07.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000136909 2024-09-04T17:47:07.823 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:07.823 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:07.823 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:07.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.823 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49826 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 1015808~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:07.823 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.078010+0000", 2024-09-04T17:47:07.824 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.6437296290000001, 2024-09-04T17:47:07.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021584564000000001, 2024-09-04T17:47:07.824 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:07.824 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:07.824 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:07.824 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:07.824 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:07.824 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:07.824 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49826 2024-09-04T17:47:07.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.824 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:07.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:07.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078010+0000", 2024-09-04T17:47:07.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:07.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:07.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078010+0000", 2024-09-04T17:47:07.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:07.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:07.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078011+0000", 2024-09-04T17:47:07.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1340000000000001e-06 2024-09-04T17:47:07.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:07.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078017+0000", 2024-09-04T17:47:07.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.682e-06 2024-09-04T17:47:07.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:07.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078036+0000", 2024-09-04T17:47:07.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9357000000000001e-05 2024-09-04T17:47:07.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:07.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078050+0000", 2024-09-04T17:47:07.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3122000000000001e-05 2024-09-04T17:47:07.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:07.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.099271+0000", 2024-09-04T17:47:07.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021221772999999999 2024-09-04T17:47:07.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:07.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.099410+0000", 2024-09-04T17:47:07.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000138415 2024-09-04T17:47:07.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:07.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:07.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:07.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.099595+0000", 2024-09-04T17:47:07.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000185081 2024-09-04T17:47:07.828 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:07.828 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:07.828 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:07.828 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:07.828 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:07.828 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:07.829 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:07.826+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:07.830 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-04T17:47:07.930 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:07.929+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:07.995 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:07.995 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:47:07.995 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:47:07.995 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:08.008 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-04T17:47:08.031 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:08.029+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:08.125 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:08.125 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:47:08.125 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:47:08.125 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:47:08.125 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:08.132 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:08.126+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:08.139 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-04T17:47:08.232 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:08.230+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:08.304 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:08.304 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:47:08.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:47:08.305 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:47:08.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.305 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6103 3.1f 3:fec7e474:::rbd_data.1134cfbce83d.0000000000000003:6 [read 1736704~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:47:08.305 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.421848+0000", 2024-09-04T17:47:08.305 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 59.881279823, 2024-09-04T17:47:08.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027313884, 2024-09-04T17:47:08.305 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:08.305 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:08.305 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:08.306 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:08.306 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:08.306 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:08.306 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6103 2024-09-04T17:47:08.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.306 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:08.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:08.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.421848+0000", 2024-09-04T17:47:08.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:08.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.421848+0000", 2024-09-04T17:47:08.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:08.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.421849+0000", 2024-09-04T17:47:08.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.144e-06 2024-09-04T17:47:08.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:08.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.421855+0000", 2024-09-04T17:47:08.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.003e-06 2024-09-04T17:47:08.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.308 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:08.308 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.421872+0000", 2024-09-04T17:47:08.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7193e-05 2024-09-04T17:47:08.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.308 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:08.308 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.421886+0000", 2024-09-04T17:47:08.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.331e-05 2024-09-04T17:47:08.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.308 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:08.308 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.444849+0000", 2024-09-04T17:47:08.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.022962755000000001 2024-09-04T17:47:08.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.309 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:08.309 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.444990+0000", 2024-09-04T17:47:08.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014129999999999999 2024-09-04T17:47:08.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.309 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:08.309 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.449162+0000", 2024-09-04T17:47:08.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041721789999999998 2024-09-04T17:47:08.309 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.309 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:08.310 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.310 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6141 3.17 3:ed419000:::rbd_data.1134cfbce83d.000000000000000b:6 [sparse-read 983040~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:47:08.310 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.422660+0000", 2024-09-04T17:47:08.310 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 59.880467572000001, 2024-09-04T17:47:08.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025710566000000001, 2024-09-04T17:47:08.310 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:08.310 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:08.310 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:08.310 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:08.310 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:08.310 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:08.311 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6141 2024-09-04T17:47:08.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.311 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:08.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:08.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422660+0000", 2024-09-04T17:47:08.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:08.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422660+0000", 2024-09-04T17:47:08.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.312 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:08.312 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422662+0000", 2024-09-04T17:47:08.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.313e-06 2024-09-04T17:47:08.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.312 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:08.312 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422691+0000", 2024-09-04T17:47:08.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9207e-05 2024-09-04T17:47:08.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.312 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:08.312 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422711+0000", 2024-09-04T17:47:08.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0225999999999999e-05 2024-09-04T17:47:08.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:08.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422727+0000", 2024-09-04T17:47:08.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5982999999999999e-05 2024-09-04T17:47:08.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:08.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.447767+0000", 2024-09-04T17:47:08.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025039558 2024-09-04T17:47:08.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:08.314 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.447871+0000", 2024-09-04T17:47:08.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104287 2024-09-04T17:47:08.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.314 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:08.314 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.448371+0000", 2024-09-04T17:47:08.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00049999199999999999 2024-09-04T17:47:08.314 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.314 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:08.314 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.314 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6142 3.17 3:ed419000:::rbd_data.1134cfbce83d.000000000000000b:6 [sparse-read 1343488~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:47:08.315 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.422750+0000", 2024-09-04T17:47:08.315 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 59.88037834, 2024-09-04T17:47:08.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026566215000000001, 2024-09-04T17:47:08.315 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:08.315 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:08.315 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:08.315 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:08.315 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:08.315 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:08.315 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6142 2024-09-04T17:47:08.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.316 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:08.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:08.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422750+0000", 2024-09-04T17:47:08.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:08.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422750+0000", 2024-09-04T17:47:08.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:08.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422751+0000", 2024-09-04T17:47:08.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6700000000000001e-06 2024-09-04T17:47:08.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.318 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:08.318 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422757+0000", 2024-09-04T17:47:08.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9669999999999998e-06 2024-09-04T17:47:08.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.318 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:08.318 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422775+0000", 2024-09-04T17:47:08.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8073000000000001e-05 2024-09-04T17:47:08.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.318 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:08.318 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422789+0000", 2024-09-04T17:47:08.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3709999999999999e-05 2024-09-04T17:47:08.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:08.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.448420+0000", 2024-09-04T17:47:08.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025630627999999999 2024-09-04T17:47:08.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:08.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.448590+0000", 2024-09-04T17:47:08.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000170213 2024-09-04T17:47:08.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:08.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.449316+0000", 2024-09-04T17:47:08.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00072595400000000001 2024-09-04T17:47:08.320 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.320 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:08.320 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.320 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6143 3.17 3:ed419000:::rbd_data.1134cfbce83d.000000000000000b:6 [read 1736704~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:47:08.320 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.422807+0000", 2024-09-04T17:47:08.320 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 59.880321010000003, 2024-09-04T17:47:08.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027271169000000001, 2024-09-04T17:47:08.320 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:08.320 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:08.320 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:08.321 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:08.321 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:08.321 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:08.321 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6143 2024-09-04T17:47:08.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.321 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:08.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:08.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422807+0000", 2024-09-04T17:47:08.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:08.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422807+0000", 2024-09-04T17:47:08.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:08.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422809+0000", 2024-09-04T17:47:08.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6139999999999999e-06 2024-09-04T17:47:08.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:08.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422814+0000", 2024-09-04T17:47:08.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1800000000000004e-06 2024-09-04T17:47:08.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.323 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:08.323 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422831+0000", 2024-09-04T17:47:08.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7433000000000001e-05 2024-09-04T17:47:08.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.323 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:08.323 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.422844+0000", 2024-09-04T17:47:08.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3216e-05 2024-09-04T17:47:08.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.323 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:08.323 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.449397+0000", 2024-09-04T17:47:08.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026552572 2024-09-04T17:47:08.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.324 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.324 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:08.324 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.449526+0000", 2024-09-04T17:47:08.324 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000129207 2024-09-04T17:47:08.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.324 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.324 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:08.324 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.450078+0000", 2024-09-04T17:47:08.324 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00055194700000000001 2024-09-04T17:47:08.324 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.324 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:08.324 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.325 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6203 3.1f 3:fec7e474:::rbd_data.1134cfbce83d.0000000000000003:6 [read 327680~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:47:08.325 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423445+0000", 2024-09-04T17:47:08.325 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 59.879682598000002, 2024-09-04T17:47:08.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027179207, 2024-09-04T17:47:08.325 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:08.325 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:08.325 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:08.325 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:08.325 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:08.325 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:08.325 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6203 2024-09-04T17:47:08.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.326 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:08.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:08.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423445+0000", 2024-09-04T17:47:08.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:08.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423445+0000", 2024-09-04T17:47:08.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:08.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423446+0000", 2024-09-04T17:47:08.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.077e-06 2024-09-04T17:47:08.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:08.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423452+0000", 2024-09-04T17:47:08.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.113e-06 2024-09-04T17:47:08.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:08.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423468+0000", 2024-09-04T17:47:08.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6685999999999999e-05 2024-09-04T17:47:08.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.328 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.328 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:08.328 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423481+0000", 2024-09-04T17:47:08.328 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2673999999999999e-05 2024-09-04T17:47:08.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.328 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.328 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:08.328 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.449779+0000", 2024-09-04T17:47:08.328 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026297838 2024-09-04T17:47:08.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.328 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.328 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:08.328 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.450020+0000", 2024-09-04T17:47:08.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024114399999999999 2024-09-04T17:47:08.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.329 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:08.329 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.450625+0000", 2024-09-04T17:47:08.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00060467500000000005 2024-09-04T17:47:08.329 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.329 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:08.329 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.329 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6204 3.1f 3:fec7e474:::rbd_data.1134cfbce83d.0000000000000003:6 [read 688128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:47:08.329 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423498+0000", 2024-09-04T17:47:08.330 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 59.879629661999999, 2024-09-04T17:47:08.330 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029262579, 2024-09-04T17:47:08.330 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:08.330 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:08.330 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:08.330 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:08.330 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:08.330 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:08.330 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6204 2024-09-04T17:47:08.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.330 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:08.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.330 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:08.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423498+0000", 2024-09-04T17:47:08.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.331 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:08.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423498+0000", 2024-09-04T17:47:08.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.331 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:08.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423500+0000", 2024-09-04T17:47:08.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.28e-06 2024-09-04T17:47:08.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.332 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.332 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:08.332 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423506+0000", 2024-09-04T17:47:08.332 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0059999999999998e-06 2024-09-04T17:47:08.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.332 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.332 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:08.332 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423524+0000", 2024-09-04T17:47:08.332 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8683999999999999e-05 2024-09-04T17:47:08.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.332 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.332 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:08.332 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423537+0000", 2024-09-04T17:47:08.333 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3030000000000001e-05 2024-09-04T17:47:08.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.333 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.333 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:08.333 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.450697+0000", 2024-09-04T17:47:08.333 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027159793000000002 2024-09-04T17:47:08.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.333 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.333 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:08.333 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.450835+0000", 2024-09-04T17:47:08.333 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013773799999999999 2024-09-04T17:47:08.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.333 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.334 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:08.334 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.452761+0000", 2024-09-04T17:47:08.334 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019260480000000001 2024-09-04T17:47:08.334 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.334 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:08.334 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.334 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6205 3.1f 3:fec7e474:::rbd_data.1134cfbce83d.0000000000000003:6 [read 1048576~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:47:08.334 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423555+0000", 2024-09-04T17:47:08.334 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 59.879572678999999, 2024-09-04T17:47:08.334 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029905979999999999, 2024-09-04T17:47:08.334 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:08.334 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:08.335 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:08.335 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:08.335 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:08.335 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:08.335 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6205 2024-09-04T17:47:08.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.335 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:08.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.335 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:08.335 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423555+0000", 2024-09-04T17:47:08.335 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.336 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:08.336 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423555+0000", 2024-09-04T17:47:08.336 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.336 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.336 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.336 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:08.336 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423556+0000", 2024-09-04T17:47:08.336 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1599999999999999e-06 2024-09-04T17:47:08.336 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.336 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.336 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:08.336 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423562+0000", 2024-09-04T17:47:08.336 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1270000000000002e-06 2024-09-04T17:47:08.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.337 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:08.337 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423579+0000", 2024-09-04T17:47:08.337 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6986e-05 2024-09-04T17:47:08.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.337 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:08.337 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423591+0000", 2024-09-04T17:47:08.337 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2544000000000001e-05 2024-09-04T17:47:08.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.338 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:08.338 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.452819+0000", 2024-09-04T17:47:08.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029227418000000002 2024-09-04T17:47:08.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.338 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:08.338 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.452985+0000", 2024-09-04T17:47:08.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000166675 2024-09-04T17:47:08.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.338 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:08.338 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.453461+0000", 2024-09-04T17:47:08.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00047606999999999999 2024-09-04T17:47:08.339 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.339 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:08.339 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.339 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6206 3.1f 3:fec7e474:::rbd_data.1134cfbce83d.0000000000000003:6 [sparse-read 1409024~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:47:08.339 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423608+0000", 2024-09-04T17:47:08.339 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 59.879520046000003, 2024-09-04T17:47:08.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.031860753999999998, 2024-09-04T17:47:08.339 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:08.339 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:08.339 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:08.339 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:08.339 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:08.340 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:08.340 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6206 2024-09-04T17:47:08.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.340 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:08.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.340 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:08.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423608+0000", 2024-09-04T17:47:08.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.340 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:08.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423608+0000", 2024-09-04T17:47:08.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.341 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:08.341 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423609+0000", 2024-09-04T17:47:08.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0300000000000001e-06 2024-09-04T17:47:08.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.341 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:08.341 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423614+0000", 2024-09-04T17:47:08.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0200000000000002e-06 2024-09-04T17:47:08.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.341 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:08.341 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423630+0000", 2024-09-04T17:47:08.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6495999999999999e-05 2024-09-04T17:47:08.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.342 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:08.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423643+0000", 2024-09-04T17:47:08.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2053e-05 2024-09-04T17:47:08.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.342 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:08.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.453527+0000", 2024-09-04T17:47:08.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029884796000000002 2024-09-04T17:47:08.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.342 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:08.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.453632+0000", 2024-09-04T17:47:08.343 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104515 2024-09-04T17:47:08.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.343 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:08.343 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.455469+0000", 2024-09-04T17:47:08.343 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001836844 2024-09-04T17:47:08.343 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.343 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:08.343 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.343 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6207 3.1f 3:fec7e474:::rbd_data.1134cfbce83d.0000000000000003:6 [sparse-read 1769472~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:47:08.343 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423662+0000", 2024-09-04T17:47:08.343 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 59.879465582999998, 2024-09-04T17:47:08.344 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.033074294999999997, 2024-09-04T17:47:08.344 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:08.344 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:08.344 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:08.344 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:08.344 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:08.344 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:08.344 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6207 2024-09-04T17:47:08.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.344 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:08.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.344 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:08.344 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423662+0000", 2024-09-04T17:47:08.344 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.345 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:08.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423662+0000", 2024-09-04T17:47:08.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.345 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:08.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423664+0000", 2024-09-04T17:47:08.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4929999999999999e-06 2024-09-04T17:47:08.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.345 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:08.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423669+0000", 2024-09-04T17:47:08.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3800000000000002e-06 2024-09-04T17:47:08.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:08.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423686+0000", 2024-09-04T17:47:08.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7010000000000001e-05 2024-09-04T17:47:08.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:08.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423704+0000", 2024-09-04T17:47:08.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7710000000000002e-05 2024-09-04T17:47:08.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:08.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.455985+0000", 2024-09-04T17:47:08.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032281272999999999 2024-09-04T17:47:08.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:08.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.456196+0000", 2024-09-04T17:47:08.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000210335 2024-09-04T17:47:08.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:08.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.456737+0000", 2024-09-04T17:47:08.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00054109399999999999 2024-09-04T17:47:08.348 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.348 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:08.348 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.348 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6243 3.17 3:ed419000:::rbd_data.1134cfbce83d.000000000000000b:6 [sparse-read 294912~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:47:08.348 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423721+0000", 2024-09-04T17:47:08.348 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 59.879406797000001, 2024-09-04T17:47:08.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032666132000000001, 2024-09-04T17:47:08.348 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:08.348 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:08.348 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:08.348 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:08.349 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:08.349 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:08.349 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6243 2024-09-04T17:47:08.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.349 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:08.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:08.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423721+0000", 2024-09-04T17:47:08.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:08.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423721+0000", 2024-09-04T17:47:08.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:08.350 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423722+0000", 2024-09-04T17:47:08.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0470000000000001e-06 2024-09-04T17:47:08.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:08.350 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423727+0000", 2024-09-04T17:47:08.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0699999999999997e-06 2024-09-04T17:47:08.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:08.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423744+0000", 2024-09-04T17:47:08.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6659000000000001e-05 2024-09-04T17:47:08.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:08.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423756+0000", 2024-09-04T17:47:08.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2284e-05 2024-09-04T17:47:08.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:08.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.455553+0000", 2024-09-04T17:47:08.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.031797032000000003 2024-09-04T17:47:08.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:08.352 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.455716+0000", 2024-09-04T17:47:08.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016253700000000001 2024-09-04T17:47:08.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:08.352 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.456387+0000", 2024-09-04T17:47:08.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00067150300000000005 2024-09-04T17:47:08.352 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.352 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:08.352 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.352 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6244 3.17 3:ed419000:::rbd_data.1134cfbce83d.000000000000000b:6 [read 688128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:47:08.353 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423775+0000", 2024-09-04T17:47:08.353 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 59.879353014000003, 2024-09-04T17:47:08.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.033129498, 2024-09-04T17:47:08.353 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:08.353 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:08.353 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:08.353 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:08.353 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:08.353 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:08.353 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6244 2024-09-04T17:47:08.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.353 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:08.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:08.354 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423775+0000", 2024-09-04T17:47:08.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.354 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:08.354 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423775+0000", 2024-09-04T17:47:08.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.354 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:08.354 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423776+0000", 2024-09-04T17:47:08.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1230000000000001e-06 2024-09-04T17:47:08.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.355 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:08.355 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423782+0000", 2024-09-04T17:47:08.355 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4929999999999998e-06 2024-09-04T17:47:08.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.355 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.355 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:08.355 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423799+0000", 2024-09-04T17:47:08.355 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7254000000000001e-05 2024-09-04T17:47:08.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.355 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.355 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:08.355 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423811+0000", 2024-09-04T17:47:08.355 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2553e-05 2024-09-04T17:47:08.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.356 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.356 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:08.356 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.456421+0000", 2024-09-04T17:47:08.356 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032609552 2024-09-04T17:47:08.356 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.356 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.356 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:08.356 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.456512+0000", 2024-09-04T17:47:08.356 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0593999999999994e-05 2024-09-04T17:47:08.356 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.356 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.356 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:08.356 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.456904+0000", 2024-09-04T17:47:08.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00039292899999999998 2024-09-04T17:47:08.357 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.357 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:08.357 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.357 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6245 3.17 3:ed419000:::rbd_data.1134cfbce83d.000000000000000b:6 [read 1048576~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:47:08.357 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423830+0000", 2024-09-04T17:47:08.357 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 59.879298007999999, 2024-09-04T17:47:08.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.033577937000000002, 2024-09-04T17:47:08.357 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:08.357 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:08.357 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:08.358 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:08.358 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:08.358 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:08.358 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6245 2024-09-04T17:47:08.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.358 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:08.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:08.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423830+0000", 2024-09-04T17:47:08.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:08.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423830+0000", 2024-09-04T17:47:08.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:08.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423831+0000", 2024-09-04T17:47:08.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.164e-06 2024-09-04T17:47:08.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:08.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423847+0000", 2024-09-04T17:47:08.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5573000000000002e-05 2024-09-04T17:47:08.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:08.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423865+0000", 2024-09-04T17:47:08.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8715999999999998e-05 2024-09-04T17:47:08.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:08.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423880+0000", 2024-09-04T17:47:08.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4107000000000001e-05 2024-09-04T17:47:08.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:08.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.456951+0000", 2024-09-04T17:47:08.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.03307181 2024-09-04T17:47:08.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:08.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.457039+0000", 2024-09-04T17:47:08.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7795999999999998e-05 2024-09-04T17:47:08.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:08.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.457408+0000", 2024-09-04T17:47:08.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000368771 2024-09-04T17:47:08.361 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.361 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:08.361 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.362 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6246 3.17 3:ed419000:::rbd_data.1134cfbce83d.000000000000000b:6 [read 1409024~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:47:08.362 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423897+0000", 2024-09-04T17:47:08.362 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 59.879230669000002, 2024-09-04T17:47:08.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034015888000000001, 2024-09-04T17:47:08.362 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:08.362 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:08.362 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:08.362 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:08.362 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:08.362 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:08.362 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6246 2024-09-04T17:47:08.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.363 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:08.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:08.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423897+0000", 2024-09-04T17:47:08.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:08.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423897+0000", 2024-09-04T17:47:08.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:08.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423899+0000", 2024-09-04T17:47:08.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3200000000000001e-06 2024-09-04T17:47:08.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:08.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423904+0000", 2024-09-04T17:47:08.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4369999999999998e-06 2024-09-04T17:47:08.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:08.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423922+0000", 2024-09-04T17:47:08.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8196999999999999e-05 2024-09-04T17:47:08.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:08.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423935+0000", 2024-09-04T17:47:08.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.314e-05 2024-09-04T17:47:08.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:08.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.457457+0000", 2024-09-04T17:47:08.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.033521663 2024-09-04T17:47:08.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:08.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.457536+0000", 2024-09-04T17:47:08.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8795999999999996e-05 2024-09-04T17:47:08.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:08.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.457913+0000", 2024-09-04T17:47:08.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00037733500000000002 2024-09-04T17:47:08.366 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.366 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:08.366 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.366 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6247 3.17 3:ed419000:::rbd_data.1134cfbce83d.000000000000000b:6 [read 1769472~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:47:08.366 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.423953+0000", 2024-09-04T17:47:08.366 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 59.879174485, 2024-09-04T17:47:08.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0344287, 2024-09-04T17:47:08.366 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:08.367 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:08.367 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:08.367 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:08.367 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:08.367 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:08.367 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6247 2024-09-04T17:47:08.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.367 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:08.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:08.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423953+0000", 2024-09-04T17:47:08.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:08.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423953+0000", 2024-09-04T17:47:08.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:08.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423955+0000", 2024-09-04T17:47:08.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0529999999999999e-06 2024-09-04T17:47:08.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:08.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423960+0000", 2024-09-04T17:47:08.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0470000000000001e-06 2024-09-04T17:47:08.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:08.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423978+0000", 2024-09-04T17:47:08.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8119999999999999e-05 2024-09-04T17:47:08.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:08.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.423991+0000", 2024-09-04T17:47:08.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2813000000000001e-05 2024-09-04T17:47:08.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:08.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.457947+0000", 2024-09-04T17:47:08.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.033956865000000003 2024-09-04T17:47:08.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:08.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.458039+0000", 2024-09-04T17:47:08.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1834000000000002e-05 2024-09-04T17:47:08.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:08.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.458382+0000", 2024-09-04T17:47:08.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00034296799999999997 2024-09-04T17:47:08.370 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.370 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:08.370 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.371 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6305 3.17 3:ee7cd06a:::rbd_data.113431586065.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:47:08.371 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.455350+0000", 2024-09-04T17:47:08.371 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 59.847777933000003, 2024-09-04T17:47:08.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029276888000000001, 2024-09-04T17:47:08.371 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:08.371 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:08.371 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:08.371 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:08.371 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:08.371 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:08.371 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6305 2024-09-04T17:47:08.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.372 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:08.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:08.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.455350+0000", 2024-09-04T17:47:08.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:08.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.455350+0000", 2024-09-04T17:47:08.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:08.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.455353+0000", 2024-09-04T17:47:08.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6400000000000001e-06 2024-09-04T17:47:08.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:08.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459745+0000", 2024-09-04T17:47:08.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0043925880000000002 2024-09-04T17:47:08.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:08.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459775+0000", 2024-09-04T17:47:08.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9254000000000001e-05 2024-09-04T17:47:08.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:08.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459789+0000", 2024-09-04T17:47:08.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4731e-05 2024-09-04T17:47:08.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:08.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459885+0000", 2024-09-04T17:47:08.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5502999999999997e-05 2024-09-04T17:47:08.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:08.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.460040+0000", 2024-09-04T17:47:08.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015491499999999999 2024-09-04T17:47:08.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5", 2024-09-04T17:47:08.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.460596+0000", 2024-09-04T17:47:08.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00055657500000000002 2024-09-04T17:47:08.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:08.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484484+0000", 2024-09-04T17:47:08.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.023887950000000002 2024-09-04T17:47:08.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:08.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484510+0000", 2024-09-04T17:47:08.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.561e-05 2024-09-04T17:47:08.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:08.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484569+0000", 2024-09-04T17:47:08.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9049999999999999e-05 2024-09-04T17:47:08.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:08.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484627+0000", 2024-09-04T17:47:08.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8072000000000002e-05 2024-09-04T17:47:08.376 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.376 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:08.376 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.377 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6306 3.17 3:ee7cd06a:::rbd_data.113431586065.0000000000000005:head [truncate 1572864] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:47:08.377 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.459834+0000", 2024-09-04T17:47:08.377 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 59.843294255000004, 2024-09-04T17:47:08.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029166788999999999, 2024-09-04T17:47:08.377 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:08.377 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:08.377 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:08.377 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:08.377 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:08.377 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:08.377 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6306 2024-09-04T17:47:08.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.378 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:08.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.378 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:08.378 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459834+0000", 2024-09-04T17:47:08.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.378 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:08.378 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459834+0000", 2024-09-04T17:47:08.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.378 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:08.378 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459836+0000", 2024-09-04T17:47:08.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.835e-06 2024-09-04T17:47:08.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:08.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459841+0000", 2024-09-04T17:47:08.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.728e-06 2024-09-04T17:47:08.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:08.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459851+0000", 2024-09-04T17:47:08.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8570000000000007e-06 2024-09-04T17:47:08.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:08.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459859+0000", 2024-09-04T17:47:08.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4970000000000004e-06 2024-09-04T17:47:08.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:08.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.468789+0000", 2024-09-04T17:47:08.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0089301699999999994 2024-09-04T17:47:08.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:08.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.468890+0000", 2024-09-04T17:47:08.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000100963 2024-09-04T17:47:08.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5", 2024-09-04T17:47:08.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.469090+0000", 2024-09-04T17:47:08.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000199774 2024-09-04T17:47:08.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:08.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484704+0000", 2024-09-04T17:47:08.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015614836999999999 2024-09-04T17:47:08.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:08.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.488833+0000", 2024-09-04T17:47:08.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004128357 2024-09-04T17:47:08.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:08.382 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.488916+0000", 2024-09-04T17:47:08.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2928000000000003e-05 2024-09-04T17:47:08.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:08.382 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.489001+0000", 2024-09-04T17:47:08.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4843000000000001e-05 2024-09-04T17:47:08.382 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.382 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:08.382 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.383 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6307 3.17 3:eb6e082d:::rbd_data.113431586065.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:47:08.383 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.459871+0000", 2024-09-04T17:47:08.383 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 59.843257295000001, 2024-09-04T17:47:08.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032107122000000002, 2024-09-04T17:47:08.383 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:08.383 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:08.383 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:08.383 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:08.383 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:08.383 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:08.383 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6307 2024-09-04T17:47:08.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.384 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:08.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:08.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459871+0000", 2024-09-04T17:47:08.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:08.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459871+0000", 2024-09-04T17:47:08.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:08.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.459871+0000", 2024-09-04T17:47:08.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9699999999999995e-07 2024-09-04T17:47:08.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:08.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.463429+0000", 2024-09-04T17:47:08.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0035577930000000001 2024-09-04T17:47:08.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:08.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.463463+0000", 2024-09-04T17:47:08.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3460000000000002e-05 2024-09-04T17:47:08.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:08.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.463483+0000", 2024-09-04T17:47:08.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0327999999999999e-05 2024-09-04T17:47:08.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:08.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.477551+0000", 2024-09-04T17:47:08.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014068259 2024-09-04T17:47:08.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:08.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.477748+0000", 2024-09-04T17:47:08.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000197004 2024-09-04T17:47:08.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5", 2024-09-04T17:47:08.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.478346+0000", 2024-09-04T17:47:08.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00059796299999999997 2024-09-04T17:47:08.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:08.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.489016+0000", 2024-09-04T17:47:08.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010669781 2024-09-04T17:47:08.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:08.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.491871+0000", 2024-09-04T17:47:08.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028546219999999998 2024-09-04T17:47:08.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:08.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.491931+0000", 2024-09-04T17:47:08.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0736999999999998e-05 2024-09-04T17:47:08.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:08.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.491978+0000", 2024-09-04T17:47:08.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6378000000000002e-05 2024-09-04T17:47:08.388 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.388 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:08.389 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.389 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:6308 3.17 3:eb6e082d:::rbd_data.113431586065.0000000000000007:head [truncate 1572864] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-04T17:47:08.389 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:08.463520+0000", 2024-09-04T17:47:08.389 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 59.839607786000002, 2024-09-04T17:47:08.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029348833000000001, 2024-09-04T17:47:08.389 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:08.389 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:08.389 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:08.389 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:08.389 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:08.389 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:08.389 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 6308 2024-09-04T17:47:08.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.390 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:08.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:08.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.463520+0000", 2024-09-04T17:47:08.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:08.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.463520+0000", 2024-09-04T17:47:08.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:08.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.463522+0000", 2024-09-04T17:47:08.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1150000000000001e-06 2024-09-04T17:47:08.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:08.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.463527+0000", 2024-09-04T17:47:08.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5170000000000001e-06 2024-09-04T17:47:08.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:08.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.463555+0000", 2024-09-04T17:47:08.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8594000000000001e-05 2024-09-04T17:47:08.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:08.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.463563+0000", 2024-09-04T17:47:08.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4340000000000003e-06 2024-09-04T17:47:08.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:08.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484756+0000", 2024-09-04T17:47:08.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021193081999999999 2024-09-04T17:47:08.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:08.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.484863+0000", 2024-09-04T17:47:08.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010721999999999999 2024-09-04T17:47:08.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 5", 2024-09-04T17:47:08.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.485067+0000", 2024-09-04T17:47:08.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000204 2024-09-04T17:47:08.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:08.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.489593+0000", 2024-09-04T17:47:08.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045261529999999998 2024-09-04T17:47:08.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:08.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.492787+0000", 2024-09-04T17:47:08.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031940039999999999 2024-09-04T17:47:08.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:08.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.492834+0000", 2024-09-04T17:47:08.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6281e-05 2024-09-04T17:47:08.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:08.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:08.492869+0000", 2024-09-04T17:47:08.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5432999999999998e-05 2024-09-04T17:47:08.394 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.395 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:08.395 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.395 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:41996 3.6 3:6229bf0a:::rbd_data.113418f5e708.000000000000001f:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-04T17:47:08.395 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:46.113777+0000", 2024-09-04T17:47:08.395 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.18935097, 2024-09-04T17:47:08.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.029978818000000001, 2024-09-04T17:47:08.395 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:08.395 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:08.395 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:08.395 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:08.395 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:08.395 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:08.396 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 41996 2024-09-04T17:47:08.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.396 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:08.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:08.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:46.113777+0000", 2024-09-04T17:47:08.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:08.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:46.113777+0000", 2024-09-04T17:47:08.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:08.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:46.113779+0000", 2024-09-04T17:47:08.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0150000000000002e-06 2024-09-04T17:47:08.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:08.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:46.113786+0000", 2024-09-04T17:47:08.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3000000000000004e-06 2024-09-04T17:47:08.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:08.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:46.113803+0000", 2024-09-04T17:47:08.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6711999999999999e-05 2024-09-04T17:47:08.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:08.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:46.113813+0000", 2024-09-04T17:47:08.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7589999999999992e-06 2024-09-04T17:47:08.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:08.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:46.113902+0000", 2024-09-04T17:47:08.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8741999999999998e-05 2024-09-04T17:47:08.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:08.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:46.114007+0000", 2024-09-04T17:47:08.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000105647 2024-09-04T17:47:08.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 1", 2024-09-04T17:47:08.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:46.120514+0000", 2024-09-04T17:47:08.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0065064429999999998 2024-09-04T17:47:08.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:08.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:46.143620+0000", 2024-09-04T17:47:08.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.023106073000000001 2024-09-04T17:47:08.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:08.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:46.143643+0000", 2024-09-04T17:47:08.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2954000000000001e-05 2024-09-04T17:47:08.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:08.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:46.143703+0000", 2024-09-04T17:47:08.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0102999999999997e-05 2024-09-04T17:47:08.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:08.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:46.143756+0000", 2024-09-04T17:47:08.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3069999999999998e-05 2024-09-04T17:47:08.400 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.400 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:08.401 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.401 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:48444 3.12 3:4ff80fb5:::rbd_data.11346a2b0bc9.000000000000001d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-04T17:47:08.401 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:58.122601+0000", 2024-09-04T17:47:08.401 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.180527118000001, 2024-09-04T17:47:08.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057065713999999997, 2024-09-04T17:47:08.401 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:08.401 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:08.401 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:08.401 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:08.401 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:08.401 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:08.401 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 48444 2024-09-04T17:47:08.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.402 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:08.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:08.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122601+0000", 2024-09-04T17:47:08.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:08.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122601+0000", 2024-09-04T17:47:08.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:08.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122603+0000", 2024-09-04T17:47:08.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3499999999999999e-06 2024-09-04T17:47:08.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:08.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122614+0000", 2024-09-04T17:47:08.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.063e-05 2024-09-04T17:47:08.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:08.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122643+0000", 2024-09-04T17:47:08.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9434999999999999e-05 2024-09-04T17:47:08.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:08.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122657+0000", 2024-09-04T17:47:08.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3920000000000001e-05 2024-09-04T17:47:08.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:08.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122767+0000", 2024-09-04T17:47:08.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010981599999999999 2024-09-04T17:47:08.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:08.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122989+0000", 2024-09-04T17:47:08.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022158200000000001 2024-09-04T17:47:08.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 3", 2024-09-04T17:47:08.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.133349+0000", 2024-09-04T17:47:08.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010360729000000001 2024-09-04T17:47:08.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:08.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.179423+0000", 2024-09-04T17:47:08.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046073408000000003 2024-09-04T17:47:08.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:08.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.179453+0000", 2024-09-04T17:47:08.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0229000000000001e-05 2024-09-04T17:47:08.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:08.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.179545+0000", 2024-09-04T17:47:08.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1580000000000001e-05 2024-09-04T17:47:08.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:08.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.179667+0000", 2024-09-04T17:47:08.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012203500000000001 2024-09-04T17:47:08.406 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.406 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:08.406 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.406 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.407 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:08.407 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:08.408 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-04T17:47:08.408 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:47:08.406+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:47:08.509 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:08.506+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:08.579 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:08.579 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:47:08.579 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:47:08.579 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:08.594 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-04T17:47:08.636 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:47:08.634+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:47:08.711 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:08.709+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:08.735 INFO:teuthology.orchestra.run.smithi092.stdout:leaving image image_client.0-clone27 intact 2024-09-04T17:47:08.776 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:08.776 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:47:08.776 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:47:08.776 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:47:08.776 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:08.793 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-04T17:47:08.844 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:08.842+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:08.882 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #26, image image_client.0-clone26 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent27 2024-09-04T17:47:08.913 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:08.910+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:08.923 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:08.923 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:47:08.923 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:47:08.923 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:47:08.923 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.923 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15462 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 835584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:08.923 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.220338+0000", 2024-09-04T17:47:08.923 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.701288355999999, 2024-09-04T17:47:08.924 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051751278999999997, 2024-09-04T17:47:08.924 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:08.924 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:08.924 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:08.924 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:08.924 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:08.924 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:08.924 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15462 2024-09-04T17:47:08.924 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.924 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:08.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.924 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:08.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.220338+0000", 2024-09-04T17:47:08.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:08.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.220338+0000", 2024-09-04T17:47:08.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:08.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.220600+0000", 2024-09-04T17:47:08.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026198900000000002 2024-09-04T17:47:08.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.926 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.926 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:08.926 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.220617+0000", 2024-09-04T17:47:08.926 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6956000000000002e-05 2024-09-04T17:47:08.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.926 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.926 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:08.926 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.220627+0000", 2024-09-04T17:47:08.926 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0492999999999999e-05 2024-09-04T17:47:08.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.927 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.927 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:08.927 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.220635+0000", 2024-09-04T17:47:08.927 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0059999999999992e-06 2024-09-04T17:47:08.927 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.927 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.927 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:08.927 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.271853+0000", 2024-09-04T17:47:08.927 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051218020000000003 2024-09-04T17:47:08.927 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.927 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.927 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:08.928 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.271941+0000", 2024-09-04T17:47:08.928 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7802999999999999e-05 2024-09-04T17:47:08.928 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.928 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.928 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:08.928 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.272089+0000", 2024-09-04T17:47:08.928 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014801199999999999 2024-09-04T17:47:08.928 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.928 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:08.928 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.928 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.928 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.928 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15500 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 294912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:08.928 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.228278+0000", 2024-09-04T17:47:08.929 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.693348358999998, 2024-09-04T17:47:08.929 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051770927000000001, 2024-09-04T17:47:08.929 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:08.929 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:08.929 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:08.929 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:08.929 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:08.929 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:08.929 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15500 2024-09-04T17:47:08.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.929 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:08.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.929 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:08.930 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.228278+0000", 2024-09-04T17:47:08.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.930 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:08.930 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.228278+0000", 2024-09-04T17:47:08.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.930 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:08.930 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234002+0000", 2024-09-04T17:47:08.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005724392 2024-09-04T17:47:08.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.931 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.931 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:08.931 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234009+0000", 2024-09-04T17:47:08.931 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1300000000000003e-06 2024-09-04T17:47:08.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.931 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.931 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:08.931 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234024+0000", 2024-09-04T17:47:08.931 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4063000000000001e-05 2024-09-04T17:47:08.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.931 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.931 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:08.931 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234035+0000", 2024-09-04T17:47:08.931 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1124000000000001e-05 2024-09-04T17:47:08.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.932 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:08.932 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.279808+0000", 2024-09-04T17:47:08.932 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045772992999999998 2024-09-04T17:47:08.932 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.932 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:08.932 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.279894+0000", 2024-09-04T17:47:08.932 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6174999999999994e-05 2024-09-04T17:47:08.932 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.932 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:08.932 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.280049+0000", 2024-09-04T17:47:08.932 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015505 2024-09-04T17:47:08.932 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.933 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:08.933 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.933 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.933 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15524 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 1540096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:08.933 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.234831+0000", 2024-09-04T17:47:08.933 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.686795482999997, 2024-09-04T17:47:08.933 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051893631000000003, 2024-09-04T17:47:08.933 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:08.933 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:08.933 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:08.933 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:08.934 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:08.934 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:08.934 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15524 2024-09-04T17:47:08.934 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.934 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:08.934 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.934 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:08.934 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234831+0000", 2024-09-04T17:47:08.934 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.934 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.934 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.934 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:08.934 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234831+0000", 2024-09-04T17:47:08.934 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.935 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:08.935 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234831+0000", 2024-09-04T17:47:08.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2699999999999999e-07 2024-09-04T17:47:08.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.935 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:08.935 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234833+0000", 2024-09-04T17:47:08.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9810000000000002e-06 2024-09-04T17:47:08.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.935 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:08.935 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234839+0000", 2024-09-04T17:47:08.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1519999999999997e-06 2024-09-04T17:47:08.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.936 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:08.936 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234845+0000", 2024-09-04T17:47:08.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1109999999999999e-06 2024-09-04T17:47:08.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.936 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:08.936 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.286526+0000", 2024-09-04T17:47:08.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051681373000000003 2024-09-04T17:47:08.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.936 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:08.936 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.286594+0000", 2024-09-04T17:47:08.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8073999999999999e-05 2024-09-04T17:47:08.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.937 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:08.937 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.286724+0000", 2024-09-04T17:47:08.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013041300000000001 2024-09-04T17:47:08.937 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.937 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:08.937 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.937 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15525 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 335872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:08.937 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.234851+0000", 2024-09-04T17:47:08.938 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.686774958999997, 2024-09-04T17:47:08.938 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052146248999999999, 2024-09-04T17:47:08.938 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:08.938 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:08.938 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:08.938 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:08.938 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:08.938 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:08.938 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15525 2024-09-04T17:47:08.938 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.938 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:08.938 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.938 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:08.938 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234851+0000", 2024-09-04T17:47:08.938 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.939 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:08.939 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234851+0000", 2024-09-04T17:47:08.939 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.939 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:08.939 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234852+0000", 2024-09-04T17:47:08.939 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0999999999999999e-07 2024-09-04T17:47:08.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.939 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:08.939 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234854+0000", 2024-09-04T17:47:08.939 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9929999999999998e-06 2024-09-04T17:47:08.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.940 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.940 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:08.940 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234860+0000", 2024-09-04T17:47:08.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1489999999999999e-06 2024-09-04T17:47:08.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.940 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.940 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:08.940 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234865+0000", 2024-09-04T17:47:08.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8600000000000001e-06 2024-09-04T17:47:08.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.940 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.940 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:08.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.286770+0000", 2024-09-04T17:47:08.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051904807999999997 2024-09-04T17:47:08.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:08.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.286852+0000", 2024-09-04T17:47:08.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2076000000000005e-05 2024-09-04T17:47:08.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:08.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.286998+0000", 2024-09-04T17:47:08.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014595299999999999 2024-09-04T17:47:08.941 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.942 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:08.942 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.942 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.942 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15526 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 811008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:08.942 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.234871+0000", 2024-09-04T17:47:08.942 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.68675519, 2024-09-04T17:47:08.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052383817999999999, 2024-09-04T17:47:08.942 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:08.942 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:08.942 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:08.942 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:08.942 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:08.943 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:08.943 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15526 2024-09-04T17:47:08.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.943 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:08.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.943 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:08.943 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234871+0000", 2024-09-04T17:47:08.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.943 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:08.943 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234871+0000", 2024-09-04T17:47:08.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:08.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234872+0000", 2024-09-04T17:47:08.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4299999999999998e-07 2024-09-04T17:47:08.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:08.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234874+0000", 2024-09-04T17:47:08.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2220000000000001e-06 2024-09-04T17:47:08.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:08.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234880+0000", 2024-09-04T17:47:08.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4169999999999997e-06 2024-09-04T17:47:08.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:08.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234886+0000", 2024-09-04T17:47:08.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8150000000000002e-06 2024-09-04T17:47:08.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:08.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287040+0000", 2024-09-04T17:47:08.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052153928000000002 2024-09-04T17:47:08.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:08.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287118+0000", 2024-09-04T17:47:08.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8102999999999994e-05 2024-09-04T17:47:08.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:08.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287255+0000", 2024-09-04T17:47:08.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013689 2024-09-04T17:47:08.946 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.946 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:08.946 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.947 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15527 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 372736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:08.947 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.234906+0000", 2024-09-04T17:47:08.947 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.686719875000001, 2024-09-04T17:47:08.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052620742999999998, 2024-09-04T17:47:08.947 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:08.947 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:08.947 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:08.947 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:08.947 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:08.947 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:08.947 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15527 2024-09-04T17:47:08.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.947 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:08.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:08.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234906+0000", 2024-09-04T17:47:08.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:08.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234906+0000", 2024-09-04T17:47:08.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:08.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234907+0000", 2024-09-04T17:47:08.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9100000000000004e-07 2024-09-04T17:47:08.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:08.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234910+0000", 2024-09-04T17:47:08.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7599999999999998e-06 2024-09-04T17:47:08.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:08.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234917+0000", 2024-09-04T17:47:08.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.861e-06 2024-09-04T17:47:08.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:08.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234923+0000", 2024-09-04T17:47:08.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4130000000000003e-06 2024-09-04T17:47:08.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:08.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287302+0000", 2024-09-04T17:47:08.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052379440999999999 2024-09-04T17:47:08.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:08.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287382+0000", 2024-09-04T17:47:08.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9406999999999994e-05 2024-09-04T17:47:08.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:08.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287527+0000", 2024-09-04T17:47:08.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014537 2024-09-04T17:47:08.951 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.951 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:08.951 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.951 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15528 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 847872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:08.951 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.234931+0000", 2024-09-04T17:47:08.951 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.686694957, 2024-09-04T17:47:08.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052862919000000001, 2024-09-04T17:47:08.951 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:08.951 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:08.951 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:08.952 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:08.952 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:08.952 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:08.952 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15528 2024-09-04T17:47:08.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.952 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:08.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:08.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234931+0000", 2024-09-04T17:47:08.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:08.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234931+0000", 2024-09-04T17:47:08.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:08.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234932+0000", 2024-09-04T17:47:08.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6100000000000001e-07 2024-09-04T17:47:08.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:08.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234934+0000", 2024-09-04T17:47:08.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3309999999999998e-06 2024-09-04T17:47:08.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.954 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:08.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234941+0000", 2024-09-04T17:47:08.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4869999999999999e-06 2024-09-04T17:47:08.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.954 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:08.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234946+0000", 2024-09-04T17:47:08.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5879999999999997e-06 2024-09-04T17:47:08.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.954 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:08.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287571+0000", 2024-09-04T17:47:08.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052624769000000002 2024-09-04T17:47:08.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:08.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287651+0000", 2024-09-04T17:47:08.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9975000000000006e-05 2024-09-04T17:47:08.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:08.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287794+0000", 2024-09-04T17:47:08.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000143308 2024-09-04T17:47:08.955 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.955 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:08.955 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.956 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15529 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 409600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:08.956 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.234954+0000", 2024-09-04T17:47:08.956 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.686672086000002, 2024-09-04T17:47:08.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053149889999999998, 2024-09-04T17:47:08.956 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:08.956 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:08.956 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:08.956 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:08.956 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:08.956 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:08.956 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15529 2024-09-04T17:47:08.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.957 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:08.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.957 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:08.957 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234954+0000", 2024-09-04T17:47:08.957 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.957 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:08.957 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234954+0000", 2024-09-04T17:47:08.957 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.957 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:08.957 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234955+0000", 2024-09-04T17:47:08.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3799999999999997e-07 2024-09-04T17:47:08.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:08.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234957+0000", 2024-09-04T17:47:08.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3309999999999998e-06 2024-09-04T17:47:08.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:08.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234964+0000", 2024-09-04T17:47:08.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5780000000000004e-06 2024-09-04T17:47:08.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:08.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234969+0000", 2024-09-04T17:47:08.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0629999999999996e-06 2024-09-04T17:47:08.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.959 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:08.959 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287839+0000", 2024-09-04T17:47:08.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052870385999999998 2024-09-04T17:47:08.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.959 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:08.959 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287929+0000", 2024-09-04T17:47:08.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9514999999999999e-05 2024-09-04T17:47:08.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.959 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:08.960 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288104+0000", 2024-09-04T17:47:08.960 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000175379 2024-09-04T17:47:08.960 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.960 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:08.960 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.960 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.960 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15530 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 884736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:08.960 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.234976+0000", 2024-09-04T17:47:08.960 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.686650667999999, 2024-09-04T17:47:08.960 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053414593000000003, 2024-09-04T17:47:08.960 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:08.960 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:08.960 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:08.960 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:08.961 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:08.961 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:08.961 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15530 2024-09-04T17:47:08.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.961 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:08.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:08.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234976+0000", 2024-09-04T17:47:08.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:08.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234976+0000", 2024-09-04T17:47:08.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:08.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234976+0000", 2024-09-04T17:47:08.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5299999999999999e-07 2024-09-04T17:47:08.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:08.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234978+0000", 2024-09-04T17:47:08.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0329999999999998e-06 2024-09-04T17:47:08.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:08.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234984+0000", 2024-09-04T17:47:08.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0249999999999999e-06 2024-09-04T17:47:08.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:08.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234990+0000", 2024-09-04T17:47:08.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7579999999999998e-06 2024-09-04T17:47:08.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:08.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288149+0000", 2024-09-04T17:47:08.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053158864 2024-09-04T17:47:08.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:08.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288236+0000", 2024-09-04T17:47:08.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7306999999999996e-05 2024-09-04T17:47:08.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.964 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.964 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:08.964 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288390+0000", 2024-09-04T17:47:08.964 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000154153 2024-09-04T17:47:08.964 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.964 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:08.964 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.964 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.964 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15531 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 446464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:08.964 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.234997+0000", 2024-09-04T17:47:08.964 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.686629685, 2024-09-04T17:47:08.965 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053721086000000001, 2024-09-04T17:47:08.965 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:08.965 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:08.965 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:08.965 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:08.965 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:08.965 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:08.965 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15531 2024-09-04T17:47:08.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.965 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:08.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.965 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:08.965 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234997+0000", 2024-09-04T17:47:08.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:08.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234997+0000", 2024-09-04T17:47:08.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:08.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234997+0000", 2024-09-04T17:47:08.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8200000000000001e-07 2024-09-04T17:47:08.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:08.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234999+0000", 2024-09-04T17:47:08.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.058e-06 2024-09-04T17:47:08.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:08.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235005+0000", 2024-09-04T17:47:08.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0700000000000003e-06 2024-09-04T17:47:08.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:08.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235012+0000", 2024-09-04T17:47:08.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7020000000000004e-06 2024-09-04T17:47:08.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:08.968 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288441+0000", 2024-09-04T17:47:08.968 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053429343999999997 2024-09-04T17:47:08.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.968 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:08.968 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288535+0000", 2024-09-04T17:47:08.968 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3406000000000007e-05 2024-09-04T17:47:08.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.968 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:08.968 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288718+0000", 2024-09-04T17:47:08.968 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000183124 2024-09-04T17:47:08.968 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.968 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:08.969 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.969 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15532 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 921600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:08.969 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235020+0000", 2024-09-04T17:47:08.969 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.686606058000002, 2024-09-04T17:47:08.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053966157000000001, 2024-09-04T17:47:08.969 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:08.969 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:08.969 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:08.969 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:08.969 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:08.969 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:08.970 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15532 2024-09-04T17:47:08.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.970 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:08.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.970 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:08.970 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235020+0000", 2024-09-04T17:47:08.970 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.970 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:08.970 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235020+0000", 2024-09-04T17:47:08.970 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.971 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:08.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235021+0000", 2024-09-04T17:47:08.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9799999999999999e-07 2024-09-04T17:47:08.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.971 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:08.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235023+0000", 2024-09-04T17:47:08.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1560000000000002e-06 2024-09-04T17:47:08.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.971 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:08.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235029+0000", 2024-09-04T17:47:08.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0270000000000001e-06 2024-09-04T17:47:08.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:08.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235034+0000", 2024-09-04T17:47:08.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0000000000000004e-06 2024-09-04T17:47:08.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:08.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288760+0000", 2024-09-04T17:47:08.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053726077999999997 2024-09-04T17:47:08.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:08.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288845+0000", 2024-09-04T17:47:08.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5242999999999997e-05 2024-09-04T17:47:08.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.973 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.973 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:08.973 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288986+0000", 2024-09-04T17:47:08.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014125499999999999 2024-09-04T17:47:08.973 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.973 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:08.973 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.973 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.973 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15533 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 483328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:08.973 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235041+0000", 2024-09-04T17:47:08.973 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.686584983000003, 2024-09-04T17:47:08.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054242994000000003, 2024-09-04T17:47:08.974 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:08.974 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:08.974 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:08.974 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:08.974 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:08.974 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:08.974 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15533 2024-09-04T17:47:08.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.974 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:08.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.974 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:08.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235041+0000", 2024-09-04T17:47:08.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:08.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235041+0000", 2024-09-04T17:47:08.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:08.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235042+0000", 2024-09-04T17:47:08.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5499999999999999e-07 2024-09-04T17:47:08.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:08.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235044+0000", 2024-09-04T17:47:08.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2259999999999999e-06 2024-09-04T17:47:08.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:08.976 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235051+0000", 2024-09-04T17:47:08.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4440000000000002e-06 2024-09-04T17:47:08.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:08.976 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235056+0000", 2024-09-04T17:47:08.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8740000000000003e-06 2024-09-04T17:47:08.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:08.976 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289028+0000", 2024-09-04T17:47:08.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053971742000000003 2024-09-04T17:47:08.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.977 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:08.977 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289108+0000", 2024-09-04T17:47:08.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0502999999999998e-05 2024-09-04T17:47:08.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.977 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:08.977 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289284+0000", 2024-09-04T17:47:08.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017584999999999999 2024-09-04T17:47:08.977 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.977 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:08.977 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.978 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15534 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 958464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:08.978 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235068+0000", 2024-09-04T17:47:08.978 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.686558282999997, 2024-09-04T17:47:08.978 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054505452000000003, 2024-09-04T17:47:08.978 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:08.978 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:08.978 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:08.978 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:08.978 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:08.978 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:08.978 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15534 2024-09-04T17:47:08.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.978 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:08.978 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.978 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:08.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235068+0000", 2024-09-04T17:47:08.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:08.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235068+0000", 2024-09-04T17:47:08.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:08.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235068+0000", 2024-09-04T17:47:08.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7899999999999999e-07 2024-09-04T17:47:08.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:08.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235071+0000", 2024-09-04T17:47:08.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4109999999999999e-06 2024-09-04T17:47:08.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:08.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235078+0000", 2024-09-04T17:47:08.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0489999999999998e-06 2024-09-04T17:47:08.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:08.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235083+0000", 2024-09-04T17:47:08.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2249999999999999e-06 2024-09-04T17:47:08.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:08.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289328+0000", 2024-09-04T17:47:08.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054244894000000002 2024-09-04T17:47:08.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:08.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289417+0000", 2024-09-04T17:47:08.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8988999999999998e-05 2024-09-04T17:47:08.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:08.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289573+0000", 2024-09-04T17:47:08.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000156405 2024-09-04T17:47:08.981 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.982 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:08.982 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.982 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.982 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.982 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15535 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 520192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:08.982 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235173+0000", 2024-09-04T17:47:08.982 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.686453753999999, 2024-09-04T17:47:08.982 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054718671000000003, 2024-09-04T17:47:08.982 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:08.982 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:08.982 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:08.982 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:08.982 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:08.982 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:08.982 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15535 2024-09-04T17:47:08.982 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.983 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:08.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.983 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:08.983 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235173+0000", 2024-09-04T17:47:08.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.983 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:08.983 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235173+0000", 2024-09-04T17:47:08.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.983 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:08.983 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235173+0000", 2024-09-04T17:47:08.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2600000000000002e-07 2024-09-04T17:47:08.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:08.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235177+0000", 2024-09-04T17:47:08.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.625e-06 2024-09-04T17:47:08.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:08.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235188+0000", 2024-09-04T17:47:08.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1464000000000001e-05 2024-09-04T17:47:08.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:08.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235195+0000", 2024-09-04T17:47:08.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5910000000000001e-06 2024-09-04T17:47:08.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:08.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289623+0000", 2024-09-04T17:47:08.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054427595000000002 2024-09-04T17:47:08.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:08.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289718+0000", 2024-09-04T17:47:08.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5958999999999994e-05 2024-09-04T17:47:08.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:08.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289891+0000", 2024-09-04T17:47:08.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017271100000000001 2024-09-04T17:47:08.986 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.986 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:08.986 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.986 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.986 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.986 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15536 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 995328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:08.986 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235206+0000", 2024-09-04T17:47:08.986 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.686420613000003, 2024-09-04T17:47:08.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054984065999999998, 2024-09-04T17:47:08.986 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:08.986 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:08.986 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:08.986 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:08.987 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:08.987 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:08.987 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15536 2024-09-04T17:47:08.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.987 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:08.987 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.987 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:08.987 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235206+0000", 2024-09-04T17:47:08.987 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.987 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.987 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:08.987 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235206+0000", 2024-09-04T17:47:08.987 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.987 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.988 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:08.988 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235206+0000", 2024-09-04T17:47:08.988 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5300000000000004e-07 2024-09-04T17:47:08.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.988 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:08.988 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235209+0000", 2024-09-04T17:47:08.988 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6089999999999999e-06 2024-09-04T17:47:08.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.988 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:08.988 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235216+0000", 2024-09-04T17:47:08.988 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.277e-06 2024-09-04T17:47:08.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.989 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.989 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:08.989 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235221+0000", 2024-09-04T17:47:08.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2530000000000004e-06 2024-09-04T17:47:08.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.989 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.989 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:08.989 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289936+0000", 2024-09-04T17:47:08.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054714888000000003 2024-09-04T17:47:08.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.989 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.989 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:08.989 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290035+0000", 2024-09-04T17:47:08.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8652000000000003e-05 2024-09-04T17:47:08.990 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.990 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.990 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:08.990 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290190+0000", 2024-09-04T17:47:08.990 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015483399999999999 2024-09-04T17:47:08.990 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.990 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:08.990 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.990 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.990 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.990 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15537 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 1323008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:08.990 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235264+0000", 2024-09-04T17:47:08.990 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.686362545000001, 2024-09-04T17:47:08.990 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055255669, 2024-09-04T17:47:08.990 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:08.991 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:08.991 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:08.991 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:08.991 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:08.991 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:08.991 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15537 2024-09-04T17:47:08.991 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.991 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:08.991 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.991 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:08.991 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235264+0000", 2024-09-04T17:47:08.991 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.991 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.991 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.991 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:08.992 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235264+0000", 2024-09-04T17:47:08.992 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.992 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:08.992 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235264+0000", 2024-09-04T17:47:08.992 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4499999999999997e-07 2024-09-04T17:47:08.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.992 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:08.992 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235267+0000", 2024-09-04T17:47:08.992 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9210000000000001e-06 2024-09-04T17:47:08.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:08.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235276+0000", 2024-09-04T17:47:08.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2309999999999996e-06 2024-09-04T17:47:08.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:08.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235281+0000", 2024-09-04T17:47:08.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8560000000000003e-06 2024-09-04T17:47:08.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:08.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290246+0000", 2024-09-04T17:47:08.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054964528999999998 2024-09-04T17:47:08.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:08.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290344+0000", 2024-09-04T17:47:08.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8073999999999996e-05 2024-09-04T17:47:08.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:08.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290519+0000", 2024-09-04T17:47:08.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000175413 2024-09-04T17:47:08.994 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.994 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:08.994 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.994 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15538 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 1032192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:08.995 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235290+0000", 2024-09-04T17:47:08.995 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.686336636, 2024-09-04T17:47:08.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055518932, 2024-09-04T17:47:08.995 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:08.995 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:08.995 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:08.995 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:08.995 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:08.995 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:08.995 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15538 2024-09-04T17:47:08.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.995 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:08.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.996 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:08.996 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235290+0000", 2024-09-04T17:47:08.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.996 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:08.996 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235290+0000", 2024-09-04T17:47:08.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:08.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.996 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:08.996 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235290+0000", 2024-09-04T17:47:08.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3399999999999999e-07 2024-09-04T17:47:08.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:08.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235292+0000", 2024-09-04T17:47:08.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2639999999999998e-06 2024-09-04T17:47:08.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:08.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235299+0000", 2024-09-04T17:47:08.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2029999999999997e-06 2024-09-04T17:47:08.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:08.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235305+0000", 2024-09-04T17:47:08.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7469999999999999e-06 2024-09-04T17:47:08.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:08.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290562+0000", 2024-09-04T17:47:08.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055256580999999999 2024-09-04T17:47:08.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:08.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290651+0000", 2024-09-04T17:47:08.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8953e-05 2024-09-04T17:47:08.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:08.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290809+0000", 2024-09-04T17:47:08.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015765000000000001 2024-09-04T17:47:08.999 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.999 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:08.999 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:08.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:08.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:08.999 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15539 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 1359872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:08.999 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235312+0000", 2024-09-04T17:47:08.999 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.686313896000001, 2024-09-04T17:47:08.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055809865, 2024-09-04T17:47:08.999 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:08.999 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:08.999 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:08.999 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:08.999 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:08.999 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:08.999 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15539 2024-09-04T17:47:09.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:09.000 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:09.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:09.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:09.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235312+0000", 2024-09-04T17:47:09.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:09.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:09.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:09.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:09.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235312+0000", 2024-09-04T17:47:09.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:09.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:09.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:09.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:09.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235313+0000", 2024-09-04T17:47:09.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9000000000000002e-07 2024-09-04T17:47:09.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:09.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:09.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:09.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235315+0000", 2024-09-04T17:47:09.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1270000000000001e-06 2024-09-04T17:47:09.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:09.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:09.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:09.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235320+0000", 2024-09-04T17:47:09.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5430000000000001e-06 2024-09-04T17:47:09.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:09.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:09.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:09.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235326+0000", 2024-09-04T17:47:09.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2039999999999996e-06 2024-09-04T17:47:09.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:09.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:09.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:09.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290858+0000", 2024-09-04T17:47:09.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055532440000000002 2024-09-04T17:47:09.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:09.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:09.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:09.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290953+0000", 2024-09-04T17:47:09.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5031000000000005e-05 2024-09-04T17:47:09.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:09.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:09.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:09.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.291122+0000", 2024-09-04T17:47:09.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016913 2024-09-04T17:47:09.003 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:09.003 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:09.003 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:09.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:09.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:09.003 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15540 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 1396736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:09.003 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235333+0000", 2024-09-04T17:47:09.003 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.686293388999999, 2024-09-04T17:47:09.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056080528999999997, 2024-09-04T17:47:09.003 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:09.003 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:09.003 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:09.003 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:09.004 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:09.004 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:09.004 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15540 2024-09-04T17:47:09.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:09.004 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:09.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:09.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:09.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235333+0000", 2024-09-04T17:47:09.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:09.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:09.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:09.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:09.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235333+0000", 2024-09-04T17:47:09.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:09.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:09.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:09.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:09.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235333+0000", 2024-09-04T17:47:09.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0999999999999999e-07 2024-09-04T17:47:09.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:09.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:09.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:09.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235336+0000", 2024-09-04T17:47:09.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.278e-06 2024-09-04T17:47:09.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:09.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:09.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:09.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235341+0000", 2024-09-04T17:47:09.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8359999999999997e-06 2024-09-04T17:47:09.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:09.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:09.006 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:09.006 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235346+0000", 2024-09-04T17:47:09.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4329999999999997e-06 2024-09-04T17:47:09.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:09.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:09.006 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:09.006 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.291167+0000", 2024-09-04T17:47:09.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055821603999999997 2024-09-04T17:47:09.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:09.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:09.006 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:09.006 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.291257+0000", 2024-09-04T17:47:09.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9349000000000001e-05 2024-09-04T17:47:09.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:09.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:09.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:09.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.291413+0000", 2024-09-04T17:47:09.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015661900000000001 2024-09-04T17:47:09.007 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:09.007 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:09.007 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:09.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:09.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:09.007 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15541 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 1433600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:09.007 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235355+0000", 2024-09-04T17:47:09.007 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.686271200999997, 2024-09-04T17:47:09.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056371783000000002, 2024-09-04T17:47:09.008 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:09.008 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:09.008 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:09.008 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:09.008 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:09.008 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:09.008 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15541 2024-09-04T17:47:09.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:09.008 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:09.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:09.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:09.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235355+0000", 2024-09-04T17:47:09.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:09.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:09.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:09.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:09.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235355+0000", 2024-09-04T17:47:09.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:09.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:09.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:09.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:09.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235356+0000", 2024-09-04T17:47:09.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3099999999999998e-07 2024-09-04T17:47:09.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:09.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:09.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:09.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235358+0000", 2024-09-04T17:47:09.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3599999999999999e-06 2024-09-04T17:47:09.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:09.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:09.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:09.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235364+0000", 2024-09-04T17:47:09.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5760000000000002e-06 2024-09-04T17:47:09.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:09.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:09.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:09.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235370+0000", 2024-09-04T17:47:09.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0819999999999998e-06 2024-09-04T17:47:09.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:09.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:09.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:09.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.291462+0000", 2024-09-04T17:47:09.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056092301999999997 2024-09-04T17:47:09.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:09.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:09.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:09.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.291559+0000", 2024-09-04T17:47:09.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6988e-05 2024-09-04T17:47:09.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:09.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:09.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:09.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.291727+0000", 2024-09-04T17:47:09.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016804400000000001 2024-09-04T17:47:09.011 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:09.011 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:09.011 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:09.011 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:09.011 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:09.012 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:09.013 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-04T17:47:09.056 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:09.054+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:09.096 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-04T17:47:09.115 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:09.113+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:09.185 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:09.185 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:47:09.185 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:47:09.185 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:09.200 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-04T17:47:09.224 INFO:teuthology.orchestra.run.smithi092.stdout:3640 writesame 0xed74d7a thru 0xed7d769 data_size 0x8(0x89f0 bytes) 2024-09-04T17:47:09.226 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:09.222+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:09.235 INFO:teuthology.orchestra.run.smithi092.stdout:3641 writesame 0xa10663a thru 0xa10952d data_size 0x2(0x2ef4 bytes) 2024-09-04T17:47:09.248 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:47:09.248 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:47:09.260 INFO:teuthology.orchestra.run.smithi092.stdout:3644 trunc from 0xed7d76a to 0x4ee01fa 2024-09-04T17:47:09.266 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:47:09.266 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:47:09.300 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-04T17:47:09.301 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-04T17:47:09.317 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:09.314+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:09.324 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-04T17:47:09.325 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-04T17:47:09.356 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:09.356 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:09.359 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:47:09.359 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:47:09.383 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-04T17:47:09.404 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:47:09.405 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:47:09.405 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-04T17:47:09.423 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:47:09.423 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:47:09.423 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:09.418+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:09.427 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-04T17:47:09.428 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-04T17:47:09.518 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:09.518 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:09.519 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:47:09.514+0000 7f6761fe9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26426) UID: 0 2024-09-04T17:47:09.532 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-04T17:47:09.619 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:47:09.617+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:47:09.663 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 6, 7] out_osds: [3] dead_osds: [6] live_osds: [0, 1, 2, 4, 5, 7, 3] 2024-09-04T17:47:09.663 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-04T17:47:09.663 INFO:tasks.thrashosds.thrasher:Killing osd 2, live_osds are [0, 1, 2, 4, 5, 7, 3] 2024-09-04T17:47:09.664 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-04T17:47:09.664 INFO:teuthology.orchestra.run:waiting for 300 2024-09-04T17:47:09.669 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-04T17:47:09.693 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:09.694 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:09.720 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:09.718+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:09.732 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-04T17:47:09.737 INFO:tasks.ceph.osd.2:Stopped 2024-09-04T17:47:09.737 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.2 2024-09-04T17:47:09.737 INFO:tasks.thrashosds.thrasher:remote for osd 2 is ubuntu@smithi092.front.sepia.ceph.com 2024-09-04T17:47:09.737 INFO:tasks.thrashosds.thrasher:remote for exp osd 6 is ubuntu@smithi190.front.sepia.ceph.com 2024-09-04T17:47:09.737 DEBUG:teuthology.orchestra.run.smithi190:> type ceph-objectstore-tool 2024-09-04T17:47:09.780 INFO:teuthology.orchestra.run.smithi190.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-04T17:47:09.780 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-6 --journal-path /var/lib/ceph/osd/ceph-6/journal --op list-pgs 2024-09-04T17:47:09.824 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:09.825+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:09.899 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:09.899 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:09.916 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-04T17:47:09.960 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:09.961+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:10.023 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:10.018+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:10.036 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:10.036 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:10.052 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-04T17:47:10.136 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:10.137+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:10.184 INFO:teuthology.orchestra.run.smithi190.stdout:3.d 2024-09-04T17:47:10.184 INFO:teuthology.orchestra.run.smithi190.stdout:3.a 2024-09-04T17:47:10.184 INFO:teuthology.orchestra.run.smithi190.stdout:3.1e 2024-09-04T17:47:10.184 INFO:teuthology.orchestra.run.smithi190.stdout:3.10 2024-09-04T17:47:10.184 INFO:teuthology.orchestra.run.smithi190.stdout:2.7 2024-09-04T17:47:10.184 INFO:teuthology.orchestra.run.smithi190.stdout:2.6 2024-09-04T17:47:10.218 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:10.218 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:10.226 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:10.222+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:10.232 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-04T17:47:10.326 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:47:10.325+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:47:10.398 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:10.398 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:10.412 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-04T17:47:10.427 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:10.422+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:10.528 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:10.529+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:10.579 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:10.579 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:10.593 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-04T17:47:10.636 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:47:10.637+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:47:10.674 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is set 2024-09-04T17:47:10.723 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-6 --journal-path /var/lib/ceph/osd/ceph-6/journal --op export --pgid 3.1e --file /var/log/ceph/exp.3.1e.6 2024-09-04T17:47:10.731 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:10.731 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:10.745 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-04T17:47:10.768 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:10.769+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:10.832 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:10.830+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:10.907 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:10.907 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:10.921 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-04T17:47:10.968 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:10.966+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:11.032 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:47:11.033+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:47:11.095 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:11.095 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:11.110 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-04T17:47:11.113 INFO:teuthology.orchestra.run.smithi190.stderr:Exporting 3.1e info 3.1e( v 58'338 (0'0,58'338] local-lis/les=51/52 n=25 ec=24/24 lis/c=51/51 les/c/f=52/52/0 sis=51) 2024-09-04T17:47:11.118 INFO:teuthology.orchestra.run.smithi190.stderr:Read #3:7817bfb4:::rbd_data.113417f3df25.0000000000000001:5# 2024-09-04T17:47:11.128 INFO:teuthology.orchestra.run.smithi190.stderr:Read #3:7817bfb4:::rbd_data.113417f3df25.0000000000000001:head# 2024-09-04T17:47:11.132 INFO:teuthology.orchestra.run.smithi190.stderr:Read #3:7847ffdf:::rbd_object_map.11346251f094:head# 2024-09-04T17:47:11.132 INFO:teuthology.orchestra.run.smithi190.stderr:Read #3:7854e172:::rbd_data.1134fdde05d4.000000000000003a:head# 2024-09-04T17:47:11.132 INFO:teuthology.orchestra.run.smithi190.stderr:Read #3:786fb296:::rbd_data.1134fdde05d4.0000000000000017:head# 2024-09-04T17:47:11.133 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:47:11.133+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:47:11.133 INFO:teuthology.orchestra.run.smithi190.stderr:Read #3:789a8067:::rbd_data.1134cfbce83d.0000000000000005:6# 2024-09-04T17:47:11.137 INFO:teuthology.orchestra.run.smithi190.stderr:Read #3:789a8067:::rbd_data.1134cfbce83d.0000000000000005:head# 2024-09-04T17:47:11.137 INFO:teuthology.orchestra.run.smithi190.stderr:Read #3:797dbd38:::rbd_data.1134b705d747.00000000000000a4:head# 2024-09-04T17:47:11.137 INFO:teuthology.orchestra.run.smithi190.stderr:Read #3:7a6b822e:::rbd_id.image_client.0-clone15:head# 2024-09-04T17:47:11.137 INFO:teuthology.orchestra.run.smithi190.stderr:Read #3:7ad00c32:::rbd_data.1134b705d747.00000000000000bd:head# 2024-09-04T17:47:11.138 INFO:teuthology.orchestra.run.smithi190.stderr:Read #3:7b072538:::rbd_data.1134fdde05d4.0000000000000000:head# 2024-09-04T17:47:11.138 INFO:teuthology.orchestra.run.smithi190.stderr:Read #3:7b30cfab:::rbd_data.1134b8120a27.0000000000000053:head# 2024-09-04T17:47:11.139 INFO:teuthology.orchestra.run.smithi190.stderr:Read #3:7b6319ad:::rbd_data.1134fdde05d4.0000000000000178:head# 2024-09-04T17:47:11.139 INFO:teuthology.orchestra.run.smithi190.stderr:Read #3:7bb542e8:::rbd_data.11348d83498b.0000000000000003:c# 2024-09-04T17:47:11.148 INFO:teuthology.orchestra.run.smithi190.stderr:Read #3:7bb542e8:::rbd_data.11348d83498b.0000000000000003:head# 2024-09-04T17:47:11.150 INFO:teuthology.orchestra.run.smithi190.stderr:Read #3:7bbed1a3:::rbd_data.11345bd5898b.0000000000000003:head# 2024-09-04T17:47:11.156 INFO:teuthology.orchestra.run.smithi190.stderr:Read #3:7c3a39b0:::rbd_data.1134eedad849.0000000000000341:head# 2024-09-04T17:47:11.157 INFO:teuthology.orchestra.run.smithi190.stderr:Read #3:7c7e60eb:::rbd_data.1134eedad849.0000000000000337:head# 2024-09-04T17:47:11.157 INFO:teuthology.orchestra.run.smithi190.stderr:Read #3:7cb367d5:::rbd_data.11346251f094.0000000000000000:head# 2024-09-04T17:47:11.157 INFO:teuthology.orchestra.run.smithi190.stderr:Read #3:7cb6c77e:::rbd_data.1134b705d747.00000000000000cc:head# 2024-09-04T17:47:11.157 INFO:teuthology.orchestra.run.smithi190.stderr:Read #3:7dc7c0f5:::rbd_data.1134eedad849.000000000000037d:head# 2024-09-04T17:47:11.157 INFO:teuthology.orchestra.run.smithi190.stderr:Read #3:7ebdf3d9:::rbd_data.1134eedad849.00000000000002a0:head# 2024-09-04T17:47:11.157 INFO:teuthology.orchestra.run.smithi190.stderr:Read #3:7fd84f05:::rbd_data.11342e8f3384.0000000000000012:13# 2024-09-04T17:47:11.158 INFO:teuthology.orchestra.run.smithi190.stderr:Read #3:7fd84f05:::rbd_data.11342e8f3384.0000000000000012:head# 2024-09-04T17:47:11.158 INFO:teuthology.orchestra.run.smithi190.stderr:Read #3:7feb9299:::rbd_data.1134b705d747.0000000000000087:head# 2024-09-04T17:47:11.158 INFO:teuthology.orchestra.run.smithi190.stderr:Export successful 2024-09-04T17:47:11.232 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:11.232 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:11.234 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:11.230+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:11.247 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-04T17:47:11.335 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:47:11.333+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:47:11.407 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:11.407 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:11.422 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-04T17:47:11.429 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-04T17:47:11.435 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:11.433+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:11.535 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:11.533+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:11.544 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:11.545 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:11.561 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-04T17:47:11.607 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:47:11.607 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:47:11.624 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:47:11.625 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:47:11.625 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-04T17:47:11.637 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:11.634+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:11.641 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-04T17:47:11.641 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-04T17:47:11.662 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:47:11.662 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:47:11.678 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:11.678 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:47:11.678 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:47:11.678 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:11.684 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:47:11.684 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:47:11.688 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-04T17:47:11.696 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-04T17:47:11.708 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-6 --journal-path /var/lib/ceph/osd/ceph-6/journal --force --op remove --pgid 3.1e 2024-09-04T17:47:11.710 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:47:11.710 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:47:11.713 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-04T17:47:11.713 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-04T17:47:11.728 INFO:teuthology.orchestra.run.smithi092.stdout:3645 write 0x59de86d thru 0x59e1991 (0x3125 bytes) 2024-09-04T17:47:11.739 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:11.734+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:11.741 INFO:teuthology.orchestra.run.smithi092.stdout:3646 trunc from 0x59e1992 to 0xdc4f3b9 2024-09-04T17:47:11.755 INFO:teuthology.orchestra.run.smithi092.stdout:3647 trunc from 0xdc4f3b9 to 0x7d98713 2024-09-04T17:47:11.764 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:11.764 INFO:teuthology.orchestra.run.smithi092.stdout:3649 read 0x456ddf6 thru 0x45710df (0x32ea bytes) 2024-09-04T17:47:11.765 INFO:teuthology.orchestra.run.smithi092.stdout:3650 write 0x159f30c thru 0x15a3089 (0x3d7e bytes) 2024-09-04T17:47:11.767 INFO:teuthology.orchestra.run.smithi092.stdout:3651 read 0x9ff8ef thru 0xa069ed (0x70ff bytes) 2024-09-04T17:47:11.768 INFO:teuthology.orchestra.run.smithi092.stdout:3652 read 0x32e47f7 thru 0x32ed136 (0x8940 bytes) 2024-09-04T17:47:11.776 INFO:teuthology.orchestra.run.smithi092.stdout:3653 trunc from 0x7d98713 to 0xba252d8 2024-09-04T17:47:11.788 INFO:teuthology.orchestra.run.smithi092.stdout:3654 trunc from 0xba252d8 to 0x7f72d27 2024-09-04T17:47:11.816 INFO:teuthology.orchestra.run.smithi092.stdout:3655 writesame 0xd468fbd thru 0xd471764 data_size 0xc(0x87a8 bytes) 2024-09-04T17:47:11.816 INFO:teuthology.orchestra.run.smithi092.stdout:3656 write 0x354e46b thru 0x35562f9 (0x7e8f bytes) 2024-09-04T17:47:11.818 INFO:teuthology.orchestra.run.smithi092.stdout:3657 write 0x7449022 thru 0x745291f (0x98fe bytes) 2024-09-04T17:47:11.820 INFO:teuthology.orchestra.run.smithi092.stdout:3658 writesame 0x1a58afb thru 0x1a5fc84 data_size 0x1(0x718a bytes) 2024-09-04T17:47:11.838 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:11.838 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:47:11.838 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:47:11.838 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:47:11.838 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:11.839 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:11.834+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:11.848 INFO:teuthology.orchestra.run.smithi092.stdout:3662 punch from 0x3da77b6 to 0x3dabe79, (0x46c3 bytes) 2024-09-04T17:47:11.848 INFO:teuthology.orchestra.run.smithi092.stdout:3663 write 0x2e025bc thru 0x2e059f9 (0x343e bytes) 2024-09-04T17:47:11.851 INFO:teuthology.orchestra.run.smithi092.stdout:3666 read 0x9efcaaa thru 0x9efcef3 (0x44a bytes) 2024-09-04T17:47:11.851 INFO:teuthology.orchestra.run.smithi092.stdout:3667 punch from 0xc2917b1 to 0xc29f091, (0xd8e0 bytes) 2024-09-04T17:47:11.852 INFO:teuthology.orchestra.run.smithi092.stdout:3669 punch from 0x7fdd8c2 to 0x7fde7fa, (0xf38 bytes) 2024-09-04T17:47:11.852 INFO:teuthology.orchestra.run.smithi092.stdout:3672 read 0x6411196 thru 0x6415a9d (0x4908 bytes) 2024-09-04T17:47:11.852 INFO:teuthology.orchestra.run.smithi092.stdout:3673 write 0x4744376 thru 0x474944e (0x50d9 bytes) 2024-09-04T17:47:11.856 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-04T17:47:11.856 INFO:teuthology.orchestra.run.smithi092.stdout:3674 read 0x77ab8fb thru 0x77b1922 (0x6028 bytes) 2024-09-04T17:47:11.857 INFO:teuthology.orchestra.run.smithi092.stdout:3675 write 0x79f6ca2 thru 0x7a03c6e (0xcfcd bytes) 2024-09-04T17:47:11.859 INFO:teuthology.orchestra.run.smithi092.stdout:3676 punch from 0xb106d02 to 0xb10b24b, (0x4549 bytes) 2024-09-04T17:47:11.859 INFO:teuthology.orchestra.run.smithi092.stdout:3677 write 0xa904e7d thru 0xa906836 (0x19ba bytes) 2024-09-04T17:47:11.865 INFO:teuthology.orchestra.run.smithi092.stdout:3679 writesame 0xc50f585 thru 0xc515ceb data_size 0x1(0x6767 bytes) 2024-09-04T17:47:11.878 INFO:teuthology.orchestra.run.smithi092.stdout:3680 punch from 0x3420faa to 0x3427374, (0x63ca bytes) 2024-09-04T17:47:11.879 INFO:teuthology.orchestra.run.smithi092.stdout:3683 read 0xb2418e thru 0xb257fa (0x166d bytes) 2024-09-04T17:47:11.879 INFO:teuthology.orchestra.run.smithi092.stdout:3684 write 0x431e10f thru 0x43283d4 (0xa2c6 bytes) 2024-09-04T17:47:11.882 INFO:teuthology.orchestra.run.smithi092.stdout:3685 read 0xe31527 thru 0xe33164 (0x1c3e bytes) 2024-09-04T17:47:11.882 INFO:teuthology.orchestra.run.smithi092.stdout:3686 read 0x132f300 thru 0x133da4a (0xe74b bytes) 2024-09-04T17:47:11.884 INFO:teuthology.orchestra.run.smithi092.stdout:3688 punch from 0x74ea320 to 0x74f851e, (0xe1fe bytes) 2024-09-04T17:47:11.884 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:11.901 INFO:teuthology.orchestra.run.smithi092.stdout:3690 trunc from 0xd471765 to 0x21bac32 2024-09-04T17:47:11.931 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:11.932 INFO:teuthology.orchestra.run.smithi092.stdout:3692 trunc from 0x21bac32 to 0x2bbf4cf 2024-09-04T17:47:11.940 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:11.938+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:11.941 INFO:teuthology.orchestra.run.smithi092.stdout:3693 punch from 0x2884fef to 0x288b4ee, (0x64ff bytes) 2024-09-04T17:47:11.941 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:11.941 INFO:teuthology.orchestra.run.smithi092.stdout:3696 read 0x19944eb thru 0x199a80d (0x6323 bytes) 2024-09-04T17:47:11.942 INFO:teuthology.orchestra.run.smithi092.stdout:3697 punch from 0x17816c0 to 0x1785b8b, (0x44cb bytes) 2024-09-04T17:47:11.942 INFO:teuthology.orchestra.run.smithi092.stdout:3700 read 0x10cb8d5 thru 0x10d6a1a (0xb146 bytes) 2024-09-04T17:47:11.943 INFO:teuthology.orchestra.run.smithi092.stdout:3701 write 0x1081c6b thru 0x10876f7 (0x5a8d bytes) 2024-09-04T17:47:11.959 INFO:teuthology.orchestra.run.smithi092.stdout:3702 trunc from 0x2bbf4cf to 0xa1745aa 2024-09-04T17:47:11.970 INFO:teuthology.orchestra.run.smithi092.stdout:3703 read 0x4047b29 thru 0x4053f92 (0xc46a bytes) 2024-09-04T17:47:11.972 INFO:teuthology.orchestra.run.smithi092.stdout:3704 writesame 0x55f3ce2 thru 0x5603a32 data_size 0x1(0xfd51 bytes) 2024-09-04T17:47:12.005 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:12.005 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:47:12.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:47:12.005 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:47:12.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.005 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15462 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 835584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:12.005 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.220338+0000", 2024-09-04T17:47:12.005 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 53.781366019000004, 2024-09-04T17:47:12.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051751278999999997, 2024-09-04T17:47:12.006 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:12.006 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:12.006 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:12.006 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:12.006 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:12.006 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:12.006 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15462 2024-09-04T17:47:12.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.006 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:12.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.006 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:12.006 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.220338+0000", 2024-09-04T17:47:12.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:12.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:12.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.220338+0000", 2024-09-04T17:47:12.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:12.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:12.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.220600+0000", 2024-09-04T17:47:12.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026198900000000002 2024-09-04T17:47:12.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:12.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.220617+0000", 2024-09-04T17:47:12.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6956000000000002e-05 2024-09-04T17:47:12.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:12.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.220627+0000", 2024-09-04T17:47:12.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0492999999999999e-05 2024-09-04T17:47:12.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:12.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.220635+0000", 2024-09-04T17:47:12.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0059999999999992e-06 2024-09-04T17:47:12.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:12.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.271853+0000", 2024-09-04T17:47:12.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051218020000000003 2024-09-04T17:47:12.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:12.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.271941+0000", 2024-09-04T17:47:12.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7802999999999999e-05 2024-09-04T17:47:12.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:12.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.272089+0000", 2024-09-04T17:47:12.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014801199999999999 2024-09-04T17:47:12.010 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:12.010 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:12.010 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:12.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.010 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15500 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 294912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:12.010 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.228278+0000", 2024-09-04T17:47:12.010 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 53.773426022000002, 2024-09-04T17:47:12.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051770927000000001, 2024-09-04T17:47:12.010 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:12.011 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:12.011 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:12.011 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:12.011 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:12.011 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:12.011 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15500 2024-09-04T17:47:12.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.011 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:12.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:12.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.228278+0000", 2024-09-04T17:47:12.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:12.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:12.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.228278+0000", 2024-09-04T17:47:12.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:12.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:12.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234002+0000", 2024-09-04T17:47:12.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005724392 2024-09-04T17:47:12.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:12.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234009+0000", 2024-09-04T17:47:12.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1300000000000003e-06 2024-09-04T17:47:12.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:12.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234024+0000", 2024-09-04T17:47:12.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4063000000000001e-05 2024-09-04T17:47:12.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:12.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234035+0000", 2024-09-04T17:47:12.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1124000000000001e-05 2024-09-04T17:47:12.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:12.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.279808+0000", 2024-09-04T17:47:12.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045772992999999998 2024-09-04T17:47:12.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:12.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.279894+0000", 2024-09-04T17:47:12.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6174999999999994e-05 2024-09-04T17:47:12.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:12.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.280049+0000", 2024-09-04T17:47:12.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015505 2024-09-04T17:47:12.015 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:12.015 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:12.015 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:12.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.015 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15524 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 1540096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:12.015 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.234831+0000", 2024-09-04T17:47:12.015 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 53.766873146000002, 2024-09-04T17:47:12.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051893631000000003, 2024-09-04T17:47:12.015 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:12.015 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:12.016 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:12.016 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:12.016 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:12.016 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:12.016 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15524 2024-09-04T17:47:12.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.016 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:12.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.016 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:12.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234831+0000", 2024-09-04T17:47:12.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:12.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:12.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234831+0000", 2024-09-04T17:47:12.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:12.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:12.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234831+0000", 2024-09-04T17:47:12.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2699999999999999e-07 2024-09-04T17:47:12.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:12.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234833+0000", 2024-09-04T17:47:12.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9810000000000002e-06 2024-09-04T17:47:12.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:12.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234839+0000", 2024-09-04T17:47:12.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1519999999999997e-06 2024-09-04T17:47:12.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:12.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234845+0000", 2024-09-04T17:47:12.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1109999999999999e-06 2024-09-04T17:47:12.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:12.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.286526+0000", 2024-09-04T17:47:12.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051681373000000003 2024-09-04T17:47:12.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:12.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.286594+0000", 2024-09-04T17:47:12.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8073999999999999e-05 2024-09-04T17:47:12.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:12.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.286724+0000", 2024-09-04T17:47:12.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013041300000000001 2024-09-04T17:47:12.019 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:12.019 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:12.019 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:12.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.020 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15525 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 335872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:12.020 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.234851+0000", 2024-09-04T17:47:12.020 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 53.766852622000002, 2024-09-04T17:47:12.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052146248999999999, 2024-09-04T17:47:12.020 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:12.020 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:12.020 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:12.020 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:12.020 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:12.020 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:12.021 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15525 2024-09-04T17:47:12.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.021 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:12.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.021 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:12.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234851+0000", 2024-09-04T17:47:12.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:12.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:12.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234851+0000", 2024-09-04T17:47:12.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:12.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:12.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234852+0000", 2024-09-04T17:47:12.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0999999999999999e-07 2024-09-04T17:47:12.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:12.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234854+0000", 2024-09-04T17:47:12.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9929999999999998e-06 2024-09-04T17:47:12.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:12.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234860+0000", 2024-09-04T17:47:12.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1489999999999999e-06 2024-09-04T17:47:12.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:12.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234865+0000", 2024-09-04T17:47:12.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8600000000000001e-06 2024-09-04T17:47:12.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:12.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.286770+0000", 2024-09-04T17:47:12.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.051904807999999997 2024-09-04T17:47:12.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:12.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.286852+0000", 2024-09-04T17:47:12.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2076000000000005e-05 2024-09-04T17:47:12.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:12.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.286998+0000", 2024-09-04T17:47:12.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014595299999999999 2024-09-04T17:47:12.025 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:12.025 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:12.025 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:12.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.025 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15526 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 811008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:12.026 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.234871+0000", 2024-09-04T17:47:12.026 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 53.766832852999997, 2024-09-04T17:47:12.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052383817999999999, 2024-09-04T17:47:12.026 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:12.026 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:12.026 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:12.026 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:12.026 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:12.026 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:12.026 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15526 2024-09-04T17:47:12.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.026 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:12.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:12.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234871+0000", 2024-09-04T17:47:12.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:12.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:12.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234871+0000", 2024-09-04T17:47:12.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:12.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:12.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234872+0000", 2024-09-04T17:47:12.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4299999999999998e-07 2024-09-04T17:47:12.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:12.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234874+0000", 2024-09-04T17:47:12.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2220000000000001e-06 2024-09-04T17:47:12.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:12.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234880+0000", 2024-09-04T17:47:12.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4169999999999997e-06 2024-09-04T17:47:12.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:12.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234886+0000", 2024-09-04T17:47:12.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8150000000000002e-06 2024-09-04T17:47:12.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:12.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287040+0000", 2024-09-04T17:47:12.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052153928000000002 2024-09-04T17:47:12.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:12.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287118+0000", 2024-09-04T17:47:12.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8102999999999994e-05 2024-09-04T17:47:12.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:12.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287255+0000", 2024-09-04T17:47:12.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013689 2024-09-04T17:47:12.030 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:12.030 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:12.030 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:12.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.030 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15527 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 372736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:12.030 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.234906+0000", 2024-09-04T17:47:12.030 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 53.766797537999999, 2024-09-04T17:47:12.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052620742999999998, 2024-09-04T17:47:12.030 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:12.030 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:12.031 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:12.031 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:12.031 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:12.031 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:12.031 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15527 2024-09-04T17:47:12.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.031 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:12.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:12.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234906+0000", 2024-09-04T17:47:12.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:12.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:12.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234906+0000", 2024-09-04T17:47:12.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:12.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:12.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234907+0000", 2024-09-04T17:47:12.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9100000000000004e-07 2024-09-04T17:47:12.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:12.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234910+0000", 2024-09-04T17:47:12.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7599999999999998e-06 2024-09-04T17:47:12.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:12.033 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234917+0000", 2024-09-04T17:47:12.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.861e-06 2024-09-04T17:47:12.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:12.033 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234923+0000", 2024-09-04T17:47:12.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4130000000000003e-06 2024-09-04T17:47:12.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:12.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287302+0000", 2024-09-04T17:47:12.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052379440999999999 2024-09-04T17:47:12.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:12.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287382+0000", 2024-09-04T17:47:12.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9406999999999994e-05 2024-09-04T17:47:12.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:12.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287527+0000", 2024-09-04T17:47:12.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014537 2024-09-04T17:47:12.035 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:12.035 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:12.035 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:12.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.035 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15528 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 847872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:12.035 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.234931+0000", 2024-09-04T17:47:12.036 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 53.766772619999998, 2024-09-04T17:47:12.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052862919000000001, 2024-09-04T17:47:12.036 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:12.036 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:12.036 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:12.036 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:12.036 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:12.036 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:12.036 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15528 2024-09-04T17:47:12.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.036 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:12.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.037 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:12.037 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234931+0000", 2024-09-04T17:47:12.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:12.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.037 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:12.037 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234931+0000", 2024-09-04T17:47:12.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:12.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.037 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:12.037 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234932+0000", 2024-09-04T17:47:12.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6100000000000001e-07 2024-09-04T17:47:12.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:12.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234934+0000", 2024-09-04T17:47:12.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3309999999999998e-06 2024-09-04T17:47:12.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:12.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234941+0000", 2024-09-04T17:47:12.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4869999999999999e-06 2024-09-04T17:47:12.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:12.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234946+0000", 2024-09-04T17:47:12.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5879999999999997e-06 2024-09-04T17:47:12.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:12.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287571+0000", 2024-09-04T17:47:12.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052624769000000002 2024-09-04T17:47:12.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:12.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287651+0000", 2024-09-04T17:47:12.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9975000000000006e-05 2024-09-04T17:47:12.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:12.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287794+0000", 2024-09-04T17:47:12.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000143308 2024-09-04T17:47:12.040 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:12.040 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:12.040 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:12.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.040 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15529 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 409600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:12.040 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.234954+0000", 2024-09-04T17:47:12.040 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 53.766749748999999, 2024-09-04T17:47:12.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053149889999999998, 2024-09-04T17:47:12.040 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:12.041 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:12.041 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:12.041 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:12.041 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:12.041 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:12.041 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15529 2024-09-04T17:47:12.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.041 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:12.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:12.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234954+0000", 2024-09-04T17:47:12.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:12.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:12.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234954+0000", 2024-09-04T17:47:12.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:12.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:12.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234955+0000", 2024-09-04T17:47:12.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3799999999999997e-07 2024-09-04T17:47:12.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:12.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234957+0000", 2024-09-04T17:47:12.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3309999999999998e-06 2024-09-04T17:47:12.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:12.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234964+0000", 2024-09-04T17:47:12.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5780000000000004e-06 2024-09-04T17:47:12.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:12.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234969+0000", 2024-09-04T17:47:12.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0629999999999996e-06 2024-09-04T17:47:12.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:12.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287839+0000", 2024-09-04T17:47:12.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052870385999999998 2024-09-04T17:47:12.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:12.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.287929+0000", 2024-09-04T17:47:12.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9514999999999999e-05 2024-09-04T17:47:12.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:12.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288104+0000", 2024-09-04T17:47:12.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000175379 2024-09-04T17:47:12.045 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:12.045 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:12.045 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:12.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.045 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15530 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 884736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:12.045 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.234976+0000", 2024-09-04T17:47:12.045 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 53.766728331000003, 2024-09-04T17:47:12.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053414593000000003, 2024-09-04T17:47:12.045 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:12.045 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:12.046 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:12.046 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:12.046 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:12.046 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:12.046 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15530 2024-09-04T17:47:12.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.046 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:12.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:12.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234976+0000", 2024-09-04T17:47:12.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:12.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:12.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234976+0000", 2024-09-04T17:47:12.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:12.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:12.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234976+0000", 2024-09-04T17:47:12.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5299999999999999e-07 2024-09-04T17:47:12.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:12.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234978+0000", 2024-09-04T17:47:12.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0329999999999998e-06 2024-09-04T17:47:12.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:12.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234984+0000", 2024-09-04T17:47:12.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0249999999999999e-06 2024-09-04T17:47:12.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:12.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234990+0000", 2024-09-04T17:47:12.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7579999999999998e-06 2024-09-04T17:47:12.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:12.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288149+0000", 2024-09-04T17:47:12.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053158864 2024-09-04T17:47:12.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:12.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288236+0000", 2024-09-04T17:47:12.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7306999999999996e-05 2024-09-04T17:47:12.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:12.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288390+0000", 2024-09-04T17:47:12.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000154153 2024-09-04T17:47:12.050 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:12.050 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:12.050 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:12.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.050 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15531 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 446464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:12.050 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.234997+0000", 2024-09-04T17:47:12.050 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 53.766707347999997, 2024-09-04T17:47:12.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053721086000000001, 2024-09-04T17:47:12.050 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:12.050 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:12.050 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:12.051 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:12.051 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:12.051 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:12.051 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15531 2024-09-04T17:47:12.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.051 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:12.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:12.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234997+0000", 2024-09-04T17:47:12.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:12.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:12.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234997+0000", 2024-09-04T17:47:12.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:12.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:12.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234997+0000", 2024-09-04T17:47:12.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8200000000000001e-07 2024-09-04T17:47:12.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:12.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.234999+0000", 2024-09-04T17:47:12.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.058e-06 2024-09-04T17:47:12.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:12.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235005+0000", 2024-09-04T17:47:12.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0700000000000003e-06 2024-09-04T17:47:12.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:12.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235012+0000", 2024-09-04T17:47:12.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7020000000000004e-06 2024-09-04T17:47:12.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:12.054 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288441+0000", 2024-09-04T17:47:12.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053429343999999997 2024-09-04T17:47:12.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:12.054 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288535+0000", 2024-09-04T17:47:12.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3406000000000007e-05 2024-09-04T17:47:12.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:12.054 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288718+0000", 2024-09-04T17:47:12.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000183124 2024-09-04T17:47:12.054 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:12.054 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:12.055 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:12.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.055 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15532 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 921600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:12.055 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235020+0000", 2024-09-04T17:47:12.055 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 53.766683721, 2024-09-04T17:47:12.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053966157000000001, 2024-09-04T17:47:12.055 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:12.055 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:12.055 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:12.055 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:12.055 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:12.055 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:12.056 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15532 2024-09-04T17:47:12.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.056 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:12.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:12.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235020+0000", 2024-09-04T17:47:12.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:12.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:12.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235020+0000", 2024-09-04T17:47:12.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:12.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:12.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235021+0000", 2024-09-04T17:47:12.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9799999999999999e-07 2024-09-04T17:47:12.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:12.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235023+0000", 2024-09-04T17:47:12.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1560000000000002e-06 2024-09-04T17:47:12.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:12.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235029+0000", 2024-09-04T17:47:12.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0270000000000001e-06 2024-09-04T17:47:12.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:12.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235034+0000", 2024-09-04T17:47:12.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0000000000000004e-06 2024-09-04T17:47:12.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:12.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288760+0000", 2024-09-04T17:47:12.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053726077999999997 2024-09-04T17:47:12.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:12.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288845+0000", 2024-09-04T17:47:12.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5242999999999997e-05 2024-09-04T17:47:12.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:12.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.288986+0000", 2024-09-04T17:47:12.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014125499999999999 2024-09-04T17:47:12.059 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:12.059 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:12.059 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:12.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.060 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15533 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 483328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:12.060 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235041+0000", 2024-09-04T17:47:12.060 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 53.766662646, 2024-09-04T17:47:12.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054242994000000003, 2024-09-04T17:47:12.060 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:12.060 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:12.060 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:12.060 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:12.060 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:12.060 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:12.060 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15533 2024-09-04T17:47:12.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.060 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:12.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:12.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235041+0000", 2024-09-04T17:47:12.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:12.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:12.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235041+0000", 2024-09-04T17:47:12.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:12.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:12.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235042+0000", 2024-09-04T17:47:12.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5499999999999999e-07 2024-09-04T17:47:12.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:12.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235044+0000", 2024-09-04T17:47:12.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2259999999999999e-06 2024-09-04T17:47:12.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:12.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235051+0000", 2024-09-04T17:47:12.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4440000000000002e-06 2024-09-04T17:47:12.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:12.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235056+0000", 2024-09-04T17:47:12.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8740000000000003e-06 2024-09-04T17:47:12.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:12.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289028+0000", 2024-09-04T17:47:12.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053971742000000003 2024-09-04T17:47:12.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:12.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289108+0000", 2024-09-04T17:47:12.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0502999999999998e-05 2024-09-04T17:47:12.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:12.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289284+0000", 2024-09-04T17:47:12.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017584999999999999 2024-09-04T17:47:12.064 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:12.064 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:12.065 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:12.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.065 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15534 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 958464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:12.065 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235068+0000", 2024-09-04T17:47:12.065 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 53.766635946000001, 2024-09-04T17:47:12.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054505452000000003, 2024-09-04T17:47:12.065 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:12.065 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:12.065 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:12.065 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:12.065 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:12.065 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:12.065 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15534 2024-09-04T17:47:12.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.066 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:12.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:12.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235068+0000", 2024-09-04T17:47:12.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:12.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:12.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235068+0000", 2024-09-04T17:47:12.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:12.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:12.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235068+0000", 2024-09-04T17:47:12.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7899999999999999e-07 2024-09-04T17:47:12.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:12.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235071+0000", 2024-09-04T17:47:12.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4109999999999999e-06 2024-09-04T17:47:12.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:12.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235078+0000", 2024-09-04T17:47:12.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0489999999999998e-06 2024-09-04T17:47:12.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:12.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235083+0000", 2024-09-04T17:47:12.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2249999999999999e-06 2024-09-04T17:47:12.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:12.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289328+0000", 2024-09-04T17:47:12.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054244894000000002 2024-09-04T17:47:12.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:12.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289417+0000", 2024-09-04T17:47:12.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8988999999999998e-05 2024-09-04T17:47:12.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:12.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289573+0000", 2024-09-04T17:47:12.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000156405 2024-09-04T17:47:12.069 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:12.069 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:12.069 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:12.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.069 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15535 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 520192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:12.069 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235173+0000", 2024-09-04T17:47:12.070 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 53.766531417000003, 2024-09-04T17:47:12.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054718671000000003, 2024-09-04T17:47:12.070 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:12.070 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:12.070 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:12.070 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:12.070 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:12.070 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:12.070 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15535 2024-09-04T17:47:12.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.070 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:12.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:12.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235173+0000", 2024-09-04T17:47:12.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:12.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:12.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235173+0000", 2024-09-04T17:47:12.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:12.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:12.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235173+0000", 2024-09-04T17:47:12.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2600000000000002e-07 2024-09-04T17:47:12.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:12.072 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235177+0000", 2024-09-04T17:47:12.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.625e-06 2024-09-04T17:47:12.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:12.072 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235188+0000", 2024-09-04T17:47:12.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1464000000000001e-05 2024-09-04T17:47:12.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:12.072 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235195+0000", 2024-09-04T17:47:12.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5910000000000001e-06 2024-09-04T17:47:12.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:12.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289623+0000", 2024-09-04T17:47:12.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054427595000000002 2024-09-04T17:47:12.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:12.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289718+0000", 2024-09-04T17:47:12.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5958999999999994e-05 2024-09-04T17:47:12.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:12.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289891+0000", 2024-09-04T17:47:12.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017271100000000001 2024-09-04T17:47:12.074 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:12.074 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:12.074 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:12.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.074 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15536 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 995328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:12.074 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235206+0000", 2024-09-04T17:47:12.074 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 53.766498276, 2024-09-04T17:47:12.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054984065999999998, 2024-09-04T17:47:12.074 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:12.074 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:12.075 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:12.075 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:12.075 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:12.075 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:12.075 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15536 2024-09-04T17:47:12.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.075 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:12.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:12.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235206+0000", 2024-09-04T17:47:12.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:12.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:12.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235206+0000", 2024-09-04T17:47:12.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:12.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:12.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235206+0000", 2024-09-04T17:47:12.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5300000000000004e-07 2024-09-04T17:47:12.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:12.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235209+0000", 2024-09-04T17:47:12.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6089999999999999e-06 2024-09-04T17:47:12.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.077 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:12.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235216+0000", 2024-09-04T17:47:12.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.277e-06 2024-09-04T17:47:12.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.077 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:12.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235221+0000", 2024-09-04T17:47:12.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2530000000000004e-06 2024-09-04T17:47:12.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.077 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:12.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.289936+0000", 2024-09-04T17:47:12.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054714888000000003 2024-09-04T17:47:12.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:12.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290035+0000", 2024-09-04T17:47:12.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8652000000000003e-05 2024-09-04T17:47:12.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:12.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290190+0000", 2024-09-04T17:47:12.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015483399999999999 2024-09-04T17:47:12.078 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:12.078 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:12.078 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:12.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.079 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15537 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 1323008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:12.079 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235264+0000", 2024-09-04T17:47:12.079 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 53.766440207999999, 2024-09-04T17:47:12.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055255669, 2024-09-04T17:47:12.079 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:12.079 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:12.079 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:12.079 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:12.079 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:12.079 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:12.079 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15537 2024-09-04T17:47:12.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.080 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:12.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:12.080 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235264+0000", 2024-09-04T17:47:12.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:12.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:12.080 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235264+0000", 2024-09-04T17:47:12.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:12.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:12.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235264+0000", 2024-09-04T17:47:12.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4499999999999997e-07 2024-09-04T17:47:12.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:12.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235267+0000", 2024-09-04T17:47:12.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9210000000000001e-06 2024-09-04T17:47:12.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:12.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235276+0000", 2024-09-04T17:47:12.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2309999999999996e-06 2024-09-04T17:47:12.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:12.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235281+0000", 2024-09-04T17:47:12.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8560000000000003e-06 2024-09-04T17:47:12.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:12.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290246+0000", 2024-09-04T17:47:12.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054964528999999998 2024-09-04T17:47:12.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:12.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290344+0000", 2024-09-04T17:47:12.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8073999999999996e-05 2024-09-04T17:47:12.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.083 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:12.083 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290519+0000", 2024-09-04T17:47:12.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000175413 2024-09-04T17:47:12.083 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:12.083 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:12.083 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:12.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.083 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15538 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 1032192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:12.083 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235290+0000", 2024-09-04T17:47:12.084 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 53.766414298999997, 2024-09-04T17:47:12.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055518932, 2024-09-04T17:47:12.084 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:12.084 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:12.084 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:12.084 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:12.084 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:12.084 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:12.084 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15538 2024-09-04T17:47:12.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.084 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:12.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:12.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235290+0000", 2024-09-04T17:47:12.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:12.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:12.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235290+0000", 2024-09-04T17:47:12.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:12.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:12.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235290+0000", 2024-09-04T17:47:12.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3399999999999999e-07 2024-09-04T17:47:12.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:12.086 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235292+0000", 2024-09-04T17:47:12.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2639999999999998e-06 2024-09-04T17:47:12.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:12.086 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235299+0000", 2024-09-04T17:47:12.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2029999999999997e-06 2024-09-04T17:47:12.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:12.086 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235305+0000", 2024-09-04T17:47:12.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7469999999999999e-06 2024-09-04T17:47:12.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.087 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:12.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290562+0000", 2024-09-04T17:47:12.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055256580999999999 2024-09-04T17:47:12.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.087 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:12.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290651+0000", 2024-09-04T17:47:12.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8953e-05 2024-09-04T17:47:12.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.087 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:12.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290809+0000", 2024-09-04T17:47:12.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015765000000000001 2024-09-04T17:47:12.088 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:12.088 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:12.088 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:12.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.088 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15539 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 1359872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:12.088 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235312+0000", 2024-09-04T17:47:12.088 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 53.766391558999999, 2024-09-04T17:47:12.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055809865, 2024-09-04T17:47:12.088 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:12.088 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:12.089 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:12.089 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:12.089 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:12.089 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:12.089 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15539 2024-09-04T17:47:12.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.089 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:12.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:12.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235312+0000", 2024-09-04T17:47:12.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:12.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:12.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235312+0000", 2024-09-04T17:47:12.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:12.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:12.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235313+0000", 2024-09-04T17:47:12.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9000000000000002e-07 2024-09-04T17:47:12.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:12.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235315+0000", 2024-09-04T17:47:12.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1270000000000001e-06 2024-09-04T17:47:12.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:12.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235320+0000", 2024-09-04T17:47:12.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5430000000000001e-06 2024-09-04T17:47:12.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:12.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235326+0000", 2024-09-04T17:47:12.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2039999999999996e-06 2024-09-04T17:47:12.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:12.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290858+0000", 2024-09-04T17:47:12.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055532440000000002 2024-09-04T17:47:12.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:12.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.290953+0000", 2024-09-04T17:47:12.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5031000000000005e-05 2024-09-04T17:47:12.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:12.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.291122+0000", 2024-09-04T17:47:12.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016913 2024-09-04T17:47:12.092 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:12.093 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:12.093 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:12.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.093 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15540 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 1396736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:12.093 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235333+0000", 2024-09-04T17:47:12.093 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 53.766371051999997, 2024-09-04T17:47:12.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056080528999999997, 2024-09-04T17:47:12.093 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:12.093 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:12.093 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:12.093 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:12.093 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:12.094 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:12.094 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15540 2024-09-04T17:47:12.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.094 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:12.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:12.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235333+0000", 2024-09-04T17:47:12.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:12.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:12.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235333+0000", 2024-09-04T17:47:12.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:12.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:12.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235333+0000", 2024-09-04T17:47:12.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0999999999999999e-07 2024-09-04T17:47:12.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:12.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235336+0000", 2024-09-04T17:47:12.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.278e-06 2024-09-04T17:47:12.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:12.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235341+0000", 2024-09-04T17:47:12.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8359999999999997e-06 2024-09-04T17:47:12.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:12.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235346+0000", 2024-09-04T17:47:12.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4329999999999997e-06 2024-09-04T17:47:12.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:12.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.291167+0000", 2024-09-04T17:47:12.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055821603999999997 2024-09-04T17:47:12.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:12.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.291257+0000", 2024-09-04T17:47:12.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9349000000000001e-05 2024-09-04T17:47:12.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:12.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.291413+0000", 2024-09-04T17:47:12.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015661900000000001 2024-09-04T17:47:12.097 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:12.097 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:12.097 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:12.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.098 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:15541 3.18 3:1ec31dec:::rbd_data.11345bd5898b.0000000000000000:a [read 1433600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-04T17:47:12.098 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:18.235355+0000", 2024-09-04T17:47:12.098 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 53.766348864000001, 2024-09-04T17:47:12.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056371783000000002, 2024-09-04T17:47:12.098 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:12.098 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:12.098 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:12.098 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:12.098 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:12.098 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:12.098 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 15541 2024-09-04T17:47:12.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.098 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:12.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:12.099 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235355+0000", 2024-09-04T17:47:12.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:12.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:12.099 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235355+0000", 2024-09-04T17:47:12.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:12.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:12.099 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235356+0000", 2024-09-04T17:47:12.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3099999999999998e-07 2024-09-04T17:47:12.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:12.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235358+0000", 2024-09-04T17:47:12.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3599999999999999e-06 2024-09-04T17:47:12.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:12.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235364+0000", 2024-09-04T17:47:12.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5760000000000002e-06 2024-09-04T17:47:12.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:12.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.235370+0000", 2024-09-04T17:47:12.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0819999999999998e-06 2024-09-04T17:47:12.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:12.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.291462+0000", 2024-09-04T17:47:12.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056092301999999997 2024-09-04T17:47:12.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:12.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.291559+0000", 2024-09-04T17:47:12.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6988e-05 2024-09-04T17:47:12.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:12.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:12.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:12.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:18.291727+0000", 2024-09-04T17:47:12.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016804400000000001 2024-09-04T17:47:12.102 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:12.102 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:12.102 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:12.102 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:12.102 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:12.102 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:12.104 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-04T17:47:12.105 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:12.105 INFO:teuthology.orchestra.run.smithi092.stdout:3706 read 0x2bd1cfb thru 0x2bdfeae (0xe1b4 bytes) 2024-09-04T17:47:12.105 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:12.105 INFO:teuthology.orchestra.run.smithi092.stdout:3709 trunc from 0xa1745aa to 0x7d40033 2024-09-04T17:47:12.105 INFO:teuthology.orchestra.run.smithi092.stdout:3710 write 0x15dbd7f thru 0x15e3916 (0x7b98 bytes) 2024-09-04T17:47:12.105 INFO:teuthology.orchestra.run.smithi092.stdout:3711 clone 29 order 21 su 8192 sc 13 2024-09-04T17:47:12.105 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:12.098+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:12.205 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:47:12.205+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:47:12.235 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:12.236 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:47:12.236 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:47:12.236 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:12.249 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-04T17:47:12.306 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:12.302+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:12.406 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:12.405+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:12.417 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:12.417 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:47:12.417 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:47:12.417 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:47:12.417 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:12.431 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-04T17:47:12.520 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:12.521+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:12.608 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:12.609 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:47:12.609 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:47:12.609 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:47:12.609 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.609 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:12722 3.1b 3:d8152af1:::rbd_data.1134ef3a5a7d.00000000000000bd:head [delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-04T17:47:12.609 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:16.421965+0000", 2024-09-04T17:47:12.609 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 56.186142515, 2024-09-04T17:47:12.609 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.890002811, 2024-09-04T17:47:12.609 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:12.609 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:12.609 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:12.610 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:12.610 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:12.610 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:12.610 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 12722 2024-09-04T17:47:12.610 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.610 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:12.610 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.610 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:12.610 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421965+0000", 2024-09-04T17:47:12.610 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:12.610 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.610 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.610 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:12.611 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421965+0000", 2024-09-04T17:47:12.611 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:12.611 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.611 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.611 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:12.611 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421966+0000", 2024-09-04T17:47:12.611 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.102e-06 2024-09-04T17:47:12.611 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.611 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.611 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:12.611 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421971+0000", 2024-09-04T17:47:12.611 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9989999999999999e-06 2024-09-04T17:47:12.611 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.611 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.612 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:12.612 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421981+0000", 2024-09-04T17:47:12.612 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.7380000000000006e-06 2024-09-04T17:47:12.612 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.612 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.612 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:12.612 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421989+0000", 2024-09-04T17:47:12.612 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.1890000000000007e-06 2024-09-04T17:47:12.612 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.612 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.612 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:12.612 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.422050+0000", 2024-09-04T17:47:12.612 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.0702000000000003e-05 2024-09-04T17:47:12.613 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.613 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.613 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for peered", 2024-09-04T17:47:12.613 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.422059+0000", 2024-09-04T17:47:12.613 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5440000000000004e-06 2024-09-04T17:47:12.613 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.613 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.613 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:12.613 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.308156+0000", 2024-09-04T17:47:12.613 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.88609676199999998 2024-09-04T17:47:12.613 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.613 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.613 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:12.614 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.308398+0000", 2024-09-04T17:47:12.614 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024215400000000001 2024-09-04T17:47:12.614 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.614 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.614 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 7", 2024-09-04T17:47:12.614 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.308795+0000", 2024-09-04T17:47:12.614 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000396704 2024-09-04T17:47:12.614 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.614 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.614 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:47:12.614 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.310831+0000", 2024-09-04T17:47:12.614 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0020364200000000002 2024-09-04T17:47:12.614 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.615 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.615 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:12.615 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.311818+0000", 2024-09-04T17:47:12.615 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00098649799999999993 2024-09-04T17:47:12.615 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.615 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.615 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:47:12.615 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.311905+0000", 2024-09-04T17:47:12.615 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7070999999999994e-05 2024-09-04T17:47:12.615 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.615 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.615 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:12.615 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.311968+0000", 2024-09-04T17:47:12.615 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.2928000000000005e-05 2024-09-04T17:47:12.616 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:12.616 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:12.616 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:12.616 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.616 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.616 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37359 3.f e58/24 3:f29e50a5:::rbd_data.1134c5a43465.0000000000000021:head v 58'135, mlcod=58'133)", 2024-09-04T17:47:12.616 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:47:12.616 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 32.990811911000002, 2024-09-04T17:47:12.616 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053483197000000003, 2024-09-04T17:47:12.616 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:12.616 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:12.616 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:12.617 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:12.617 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.617 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:12.617 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:47:12.617 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:12.617 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.617 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.617 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:12.617 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:47:12.617 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:12.617 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.617 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.617 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:12.618 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:47:12.618 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1699999999999997e-07 2024-09-04T17:47:12.618 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.618 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.618 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:12.618 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617338+0000", 2024-09-04T17:47:12.618 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.1168999999999998e-05 2024-09-04T17:47:12.618 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.618 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.618 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:12.618 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617351+0000", 2024-09-04T17:47:12.618 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3094e-05 2024-09-04T17:47:12.618 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.619 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.619 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:12.619 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617359+0000", 2024-09-04T17:47:12.619 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7970000000000006e-06 2024-09-04T17:47:12.619 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.619 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.619 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:12.619 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617417+0000", 2024-09-04T17:47:12.619 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7079e-05 2024-09-04T17:47:12.619 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.619 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.619 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:12.619 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617438+0000", 2024-09-04T17:47:12.619 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1654999999999998e-05 2024-09-04T17:47:12.620 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.620 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.620 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:47:12.620 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670738+0000", 2024-09-04T17:47:12.620 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053300142000000002 2024-09-04T17:47:12.620 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.620 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.620 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:12.620 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670779+0000", 2024-09-04T17:47:12.620 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0343999999999999e-05 2024-09-04T17:47:12.620 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:12.620 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:12.620 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:12.621 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.621 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.621 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37358 3.1d e58/24 3:bd3215f3:::rbd_data.1134c5a43465.0000000000000020:head v 58'91, mlcod=58'89)", 2024-09-04T17:47:12.621 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617502+0000", 2024-09-04T17:47:12.621 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 32.990605193, 2024-09-04T17:47:12.621 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053505641999999999, 2024-09-04T17:47:12.621 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:12.621 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:12.621 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:12.621 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:12.621 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.621 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:12.621 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617502+0000", 2024-09-04T17:47:12.622 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:12.622 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.622 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.622 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:12.622 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617502+0000", 2024-09-04T17:47:12.622 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:12.622 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.622 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.622 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:12.622 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617503+0000", 2024-09-04T17:47:12.622 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.184e-06 2024-09-04T17:47:12.622 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.622 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.622 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:12.623 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617526+0000", 2024-09-04T17:47:12.623 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2099999999999998e-05 2024-09-04T17:47:12.623 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.623 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.623 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:12.623 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617540+0000", 2024-09-04T17:47:12.623 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4741e-05 2024-09-04T17:47:12.623 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.623 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.623 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:12.623 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617550+0000", 2024-09-04T17:47:12.623 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4259999999999992e-06 2024-09-04T17:47:12.623 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.623 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.624 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:12.624 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617613+0000", 2024-09-04T17:47:12.624 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.3625000000000002e-05 2024-09-04T17:47:12.624 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.624 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.624 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:12.624 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617637+0000", 2024-09-04T17:47:12.624 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.3604999999999998e-05 2024-09-04T17:47:12.624 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.624 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.624 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:47:12.624 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670967+0000", 2024-09-04T17:47:12.624 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053329881000000003 2024-09-04T17:47:12.625 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.625 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.625 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:12.625 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671008+0000", 2024-09-04T17:47:12.625 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.108e-05 2024-09-04T17:47:12.625 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:12.625 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:12.625 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:12.625 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.625 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.625 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37361 3.1c e58/24 3:3c7f8976:::rbd_data.1134c5a43465.0000000000000023:head v 58'159, mlcod=58'157)", 2024-09-04T17:47:12.625 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617578+0000", 2024-09-04T17:47:12.625 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 32.990529342999999, 2024-09-04T17:47:12.626 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053990367999999997, 2024-09-04T17:47:12.626 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:12.626 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:12.626 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:12.626 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:12.626 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.626 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:12.626 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617578+0000", 2024-09-04T17:47:12.626 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:12.626 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.626 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.626 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:12.626 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617578+0000", 2024-09-04T17:47:12.626 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:12.627 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.627 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.627 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:12.627 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617579+0000", 2024-09-04T17:47:12.627 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7299999999999996e-07 2024-09-04T17:47:12.627 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.627 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.627 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:12.627 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617610+0000", 2024-09-04T17:47:12.627 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0951000000000002e-05 2024-09-04T17:47:12.627 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.627 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.627 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:12.627 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:47:12.628 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0002e-05 2024-09-04T17:47:12.628 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.628 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.628 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:12.628 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617626+0000", 2024-09-04T17:47:12.628 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6660000000000002e-06 2024-09-04T17:47:12.628 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.628 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.628 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:12.628 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670548+0000", 2024-09-04T17:47:12.628 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.052921943999999999 2024-09-04T17:47:12.628 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.629 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.629 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:12.629 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670574+0000", 2024-09-04T17:47:12.629 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5405000000000002e-05 2024-09-04T17:47:12.629 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.629 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.629 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:47:12.629 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671522+0000", 2024-09-04T17:47:12.629 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00094815299999999997 2024-09-04T17:47:12.629 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.629 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.629 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:12.629 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671568+0000", 2024-09-04T17:47:12.630 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6674000000000001e-05 2024-09-04T17:47:12.630 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:12.630 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:12.630 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:12.630 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.630 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.630 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37357 3.15 e58/49 3:a897312a:::rbd_data.1134c5a43465.000000000000001f:head v 58'117, mlcod=58'115)", 2024-09-04T17:47:12.630 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:47:12.630 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 32.990487004999999, 2024-09-04T17:47:12.630 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053734297, 2024-09-04T17:47:12.630 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:12.630 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:12.630 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:12.630 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:12.631 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.631 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:12.631 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:47:12.631 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:12.631 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.631 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.631 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:12.631 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:47:12.631 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:12.631 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.631 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.631 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:12.631 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617621+0000", 2024-09-04T17:47:12.632 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.7300000000000004e-07 2024-09-04T17:47:12.632 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.632 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.632 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:12.632 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617642+0000", 2024-09-04T17:47:12.632 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0185000000000001e-05 2024-09-04T17:47:12.632 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.632 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.632 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:12.632 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617654+0000", 2024-09-04T17:47:12.633 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2745000000000001e-05 2024-09-04T17:47:12.633 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.633 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.633 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:12.633 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617662+0000", 2024-09-04T17:47:12.633 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.0499999999999992e-06 2024-09-04T17:47:12.633 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.633 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.633 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:12.633 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617727+0000", 2024-09-04T17:47:12.633 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.4923000000000002e-05 2024-09-04T17:47:12.633 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.633 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.634 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:12.634 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617753+0000", 2024-09-04T17:47:12.634 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5899999999999999e-05 2024-09-04T17:47:12.634 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.634 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.634 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:47:12.634 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671298+0000", 2024-09-04T17:47:12.634 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053545276000000003 2024-09-04T17:47:12.634 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.634 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.635 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:12.635 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671355+0000", 2024-09-04T17:47:12.635 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6245000000000001e-05 2024-09-04T17:47:12.635 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:12.635 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:12.635 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:12.635 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.635 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.635 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44687 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:12.635 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253012+0000", 2024-09-04T17:47:12.636 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.355094938000001, 2024-09-04T17:47:12.636 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051128777, 2024-09-04T17:47:12.636 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:12.636 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:12.636 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:12.636 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:12.636 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:12.636 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:12.636 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44687 2024-09-04T17:47:12.636 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.636 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:12.636 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.636 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:12.637 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253012+0000", 2024-09-04T17:47:12.637 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:12.637 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.637 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.637 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:12.637 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253012+0000", 2024-09-04T17:47:12.637 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:12.638 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.638 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.638 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:12.638 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253014+0000", 2024-09-04T17:47:12.638 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.24e-06 2024-09-04T17:47:12.638 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.638 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.639 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:12.639 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253019+0000", 2024-09-04T17:47:12.639 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4569999999999996e-06 2024-09-04T17:47:12.639 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.639 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.639 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:12.640 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253035+0000", 2024-09-04T17:47:12.640 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6291999999999999e-05 2024-09-04T17:47:12.640 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.640 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.640 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:12.640 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253050+0000", 2024-09-04T17:47:12.641 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4423000000000001e-05 2024-09-04T17:47:12.641 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.641 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.641 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:12.641 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304115+0000", 2024-09-04T17:47:12.642 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051064778999999998 2024-09-04T17:47:12.642 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.642 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.642 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:12.642 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304141+0000", 2024-09-04T17:47:12.642 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6585999999999999e-05 2024-09-04T17:47:12.642 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:12.643 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:12.643 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:12.643 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.643 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.643 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44688 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:12.643 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253067+0000", 2024-09-04T17:47:12.644 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.355040385999999, 2024-09-04T17:47:12.644 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051116979, 2024-09-04T17:47:12.644 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:12.644 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:12.644 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:12.645 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:12.645 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:12.645 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:12.645 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44688 2024-09-04T17:47:12.645 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.645 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:12.646 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.646 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:12.646 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253067+0000", 2024-09-04T17:47:12.646 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:12.646 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.647 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.647 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:12.647 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253067+0000", 2024-09-04T17:47:12.647 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:12.647 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.647 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.648 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:12.648 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253068+0000", 2024-09-04T17:47:12.648 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0729999999999999e-06 2024-09-04T17:47:12.648 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.648 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.648 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:12.649 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253073+0000", 2024-09-04T17:47:12.649 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2259999999999996e-06 2024-09-04T17:47:12.649 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.649 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.649 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:12.650 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253091+0000", 2024-09-04T17:47:12.650 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7255999999999999e-05 2024-09-04T17:47:12.650 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.650 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.650 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:12.650 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253103+0000", 2024-09-04T17:47:12.650 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2632999999999999e-05 2024-09-04T17:47:12.651 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.651 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.651 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:12.651 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304173+0000", 2024-09-04T17:47:12.651 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051069537999999998 2024-09-04T17:47:12.652 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.652 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.652 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:12.652 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304184+0000", 2024-09-04T17:47:12.652 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1253000000000001e-05 2024-09-04T17:47:12.652 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:12.653 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:12.653 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:12.653 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.653 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.653 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44689 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:12.653 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253121+0000", 2024-09-04T17:47:12.654 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.354986249, 2024-09-04T17:47:12.654 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051192595, 2024-09-04T17:47:12.654 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:12.654 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:12.654 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:12.654 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:12.655 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:12.655 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:12.655 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44689 2024-09-04T17:47:12.655 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.655 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:12.655 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.656 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:12.656 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253121+0000", 2024-09-04T17:47:12.656 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:12.656 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.656 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.657 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:12.657 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253121+0000", 2024-09-04T17:47:12.657 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:12.657 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.657 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.657 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:12.658 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253122+0000", 2024-09-04T17:47:12.658 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.15e-06 2024-09-04T17:47:12.658 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.658 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.658 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:12.659 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253128+0000", 2024-09-04T17:47:12.659 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2569999999999998e-06 2024-09-04T17:47:12.659 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.659 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.659 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:12.659 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253145+0000", 2024-09-04T17:47:12.660 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7176e-05 2024-09-04T17:47:12.660 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.660 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.660 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:12.660 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253158+0000", 2024-09-04T17:47:12.660 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3139000000000001e-05 2024-09-04T17:47:12.661 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.661 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.661 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:12.661 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304287+0000", 2024-09-04T17:47:12.661 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051129149999999998 2024-09-04T17:47:12.661 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.662 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.662 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:12.662 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304314+0000", 2024-09-04T17:47:12.662 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6723000000000001e-05 2024-09-04T17:47:12.662 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:12.663 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:12.663 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:12.663 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.663 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.663 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44690 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:12.663 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253177+0000", 2024-09-04T17:47:12.664 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.354930123999999, 2024-09-04T17:47:12.664 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051169263, 2024-09-04T17:47:12.664 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:12.664 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:12.664 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:12.665 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:12.665 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:12.665 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:12.665 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44690 2024-09-04T17:47:12.665 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.665 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:12.666 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.666 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:12.666 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253177+0000", 2024-09-04T17:47:12.666 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:12.666 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.667 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.667 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:12.667 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253177+0000", 2024-09-04T17:47:12.667 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:12.667 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.667 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.668 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:12.668 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253178+0000", 2024-09-04T17:47:12.668 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1000000000000001e-06 2024-09-04T17:47:12.668 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.668 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.669 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:12.669 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253184+0000", 2024-09-04T17:47:12.669 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2089999999999996e-06 2024-09-04T17:47:12.669 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.669 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.669 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:12.670 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253200+0000", 2024-09-04T17:47:12.670 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6087e-05 2024-09-04T17:47:12.670 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.670 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.670 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:12.670 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253231+0000", 2024-09-04T17:47:12.671 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0874999999999998e-05 2024-09-04T17:47:12.671 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.671 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.671 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:12.671 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304337+0000", 2024-09-04T17:47:12.671 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051106430000000001 2024-09-04T17:47:12.672 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.672 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.672 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:12.672 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304347+0000", 2024-09-04T17:47:12.672 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5619999999999993e-06 2024-09-04T17:47:12.672 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:12.673 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:12.673 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:12.673 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.673 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.673 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44691 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:12.674 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253249+0000", 2024-09-04T17:47:12.674 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.35485809, 2024-09-04T17:47:12.674 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051195630999999998, 2024-09-04T17:47:12.674 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:12.674 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:12.674 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:12.675 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:12.675 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:12.675 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:12.675 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44691 2024-09-04T17:47:12.675 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.675 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:12.676 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.676 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:12.676 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253249+0000", 2024-09-04T17:47:12.676 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:12.676 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.676 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.677 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:12.677 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253249+0000", 2024-09-04T17:47:12.677 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:12.677 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.677 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.677 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:12.678 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253251+0000", 2024-09-04T17:47:12.678 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2330000000000001e-06 2024-09-04T17:47:12.678 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.678 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.678 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:12.678 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253256+0000", 2024-09-04T17:47:12.679 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6099999999999997e-06 2024-09-04T17:47:12.679 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.679 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.679 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:12.679 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253274+0000", 2024-09-04T17:47:12.679 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7952999999999999e-05 2024-09-04T17:47:12.680 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.680 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.680 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:12.680 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253287+0000", 2024-09-04T17:47:12.680 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2839e-05 2024-09-04T17:47:12.680 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.680 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.680 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:12.681 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304396+0000", 2024-09-04T17:47:12.681 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051108990999999999 2024-09-04T17:47:12.681 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.681 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.681 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:12.681 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304445+0000", 2024-09-04T17:47:12.681 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9005000000000003e-05 2024-09-04T17:47:12.681 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:12.681 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:12.681 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:12.681 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.681 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.681 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44692 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:12.682 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253337+0000", 2024-09-04T17:47:12.682 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.354770516999999, 2024-09-04T17:47:12.682 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051138909000000003, 2024-09-04T17:47:12.682 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:12.682 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:12.682 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:12.682 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:12.682 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:12.682 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:12.682 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44692 2024-09-04T17:47:12.682 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.682 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:12.682 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.682 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:12.683 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253337+0000", 2024-09-04T17:47:12.683 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:12.683 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.683 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.683 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:12.683 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253337+0000", 2024-09-04T17:47:12.683 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:12.683 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.683 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.683 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:12.683 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253338+0000", 2024-09-04T17:47:12.683 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.463e-06 2024-09-04T17:47:12.683 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.683 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.684 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:12.684 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253357+0000", 2024-09-04T17:47:12.684 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9043e-05 2024-09-04T17:47:12.684 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.684 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.684 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:12.684 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253379+0000", 2024-09-04T17:47:12.684 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1243e-05 2024-09-04T17:47:12.684 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.684 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.684 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:12.684 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253396+0000", 2024-09-04T17:47:12.684 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7059e-05 2024-09-04T17:47:12.685 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.685 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.685 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:12.685 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304466+0000", 2024-09-04T17:47:12.685 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051070624000000002 2024-09-04T17:47:12.685 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.685 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.685 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:12.685 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304476+0000", 2024-09-04T17:47:12.685 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4769999999999992e-06 2024-09-04T17:47:12.685 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:12.685 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:12.685 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:12.686 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.686 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.686 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44693 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:12.686 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253415+0000", 2024-09-04T17:47:12.686 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.354692113999999, 2024-09-04T17:47:12.686 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051141949999999999, 2024-09-04T17:47:12.686 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:12.686 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:12.686 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:12.686 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:12.686 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:12.686 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:12.686 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44693 2024-09-04T17:47:12.687 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.687 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:12.687 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.687 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:12.687 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253415+0000", 2024-09-04T17:47:12.687 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:12.687 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.687 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.687 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:12.687 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253415+0000", 2024-09-04T17:47:12.687 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:12.687 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.688 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.688 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:12.688 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253416+0000", 2024-09-04T17:47:12.688 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1799999999999999e-06 2024-09-04T17:47:12.688 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.688 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.688 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:12.688 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253422+0000", 2024-09-04T17:47:12.688 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.5940000000000001e-06 2024-09-04T17:47:12.688 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.688 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.688 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:12.688 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253438+0000", 2024-09-04T17:47:12.689 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6008999999999999e-05 2024-09-04T17:47:12.689 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.689 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.689 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:12.689 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253452+0000", 2024-09-04T17:47:12.689 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3543000000000001e-05 2024-09-04T17:47:12.689 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.689 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.689 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:12.689 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304533+0000", 2024-09-04T17:47:12.689 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051081609 2024-09-04T17:47:12.689 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.689 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.690 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:12.690 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304557+0000", 2024-09-04T17:47:12.690 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4015e-05 2024-09-04T17:47:12.690 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:12.690 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:12.690 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:12.690 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.690 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.690 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44694 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:12.690 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253469+0000", 2024-09-04T17:47:12.690 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.354637989, 2024-09-04T17:47:12.690 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051121486000000001, 2024-09-04T17:47:12.691 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:12.691 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:12.691 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:12.691 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:12.691 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:12.691 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:12.691 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44694 2024-09-04T17:47:12.691 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.691 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:12.691 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.691 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:12.691 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253469+0000", 2024-09-04T17:47:12.691 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:12.691 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.692 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.692 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:12.692 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253469+0000", 2024-09-04T17:47:12.692 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:12.692 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.692 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.692 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:12.692 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253471+0000", 2024-09-04T17:47:12.692 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1400000000000001e-06 2024-09-04T17:47:12.692 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.692 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.692 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:12.692 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253476+0000", 2024-09-04T17:47:12.693 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7370000000000003e-06 2024-09-04T17:47:12.693 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.693 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.693 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:12.693 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253493+0000", 2024-09-04T17:47:12.693 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6549e-05 2024-09-04T17:47:12.693 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.693 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.693 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:12.693 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253506+0000", 2024-09-04T17:47:12.693 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3406e-05 2024-09-04T17:47:12.693 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.693 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.693 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:12.694 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304582+0000", 2024-09-04T17:47:12.694 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051075534999999998 2024-09-04T17:47:12.694 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.694 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.694 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:12.694 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304591+0000", 2024-09-04T17:47:12.694 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1190000000000003e-06 2024-09-04T17:47:12.694 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:12.695 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:12.695 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:12.695 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.695 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.695 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44695 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:12.695 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253552+0000", 2024-09-04T17:47:12.695 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.354555234999999, 2024-09-04T17:47:12.695 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051068872000000001, 2024-09-04T17:47:12.695 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:12.695 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:12.695 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:12.695 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:12.695 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:12.695 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:12.696 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44695 2024-09-04T17:47:12.696 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.696 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:12.696 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.696 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:12.696 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253552+0000", 2024-09-04T17:47:12.696 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:12.696 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.696 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.696 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:12.696 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253552+0000", 2024-09-04T17:47:12.696 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:12.697 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.697 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.697 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:12.697 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253553+0000", 2024-09-04T17:47:12.697 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1960000000000001e-06 2024-09-04T17:47:12.697 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.697 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.697 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:12.697 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253560+0000", 2024-09-04T17:47:12.697 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.2040000000000002e-06 2024-09-04T17:47:12.697 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.697 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.697 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:12.697 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253578+0000", 2024-09-04T17:47:12.698 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8241999999999999e-05 2024-09-04T17:47:12.698 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.698 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.698 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:12.698 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253591+0000", 2024-09-04T17:47:12.698 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3453e-05 2024-09-04T17:47:12.698 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.698 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.698 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:12.698 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304614+0000", 2024-09-04T17:47:12.698 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051022456000000001 2024-09-04T17:47:12.698 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.698 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.699 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:12.699 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304621+0000", 2024-09-04T17:47:12.699 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3209999999999999e-06 2024-09-04T17:47:12.699 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:12.699 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:12.699 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:12.699 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.699 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.699 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44696 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:12.699 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253609+0000", 2024-09-04T17:47:12.699 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.354498734, 2024-09-04T17:47:12.699 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051115351000000003, 2024-09-04T17:47:12.699 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:12.700 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:12.700 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:12.700 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:12.700 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:12.700 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:12.700 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44696 2024-09-04T17:47:12.700 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.700 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:12.700 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.700 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:12.700 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253609+0000", 2024-09-04T17:47:12.700 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:12.700 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.701 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.701 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:12.701 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253609+0000", 2024-09-04T17:47:12.701 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:12.701 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.701 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.701 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:12.701 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253610+0000", 2024-09-04T17:47:12.701 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2300000000000001e-06 2024-09-04T17:47:12.701 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.701 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.701 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:12.701 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253615+0000", 2024-09-04T17:47:12.701 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7260000000000002e-06 2024-09-04T17:47:12.702 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.702 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.702 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:12.702 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253632+0000", 2024-09-04T17:47:12.702 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7039999999999999e-05 2024-09-04T17:47:12.702 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.702 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.702 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:12.702 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253666+0000", 2024-09-04T17:47:12.702 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4662000000000001e-05 2024-09-04T17:47:12.702 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.702 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.702 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:12.703 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304677+0000", 2024-09-04T17:47:12.703 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051011017999999998 2024-09-04T17:47:12.703 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.703 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.703 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:12.703 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304724+0000", 2024-09-04T17:47:12.703 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6675000000000003e-05 2024-09-04T17:47:12.703 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:12.703 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:12.703 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:12.703 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.703 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.703 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44697 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:12.703 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253684+0000", 2024-09-04T17:47:12.704 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.354423239999999, 2024-09-04T17:47:12.704 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051073107, 2024-09-04T17:47:12.704 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:12.704 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:12.704 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:12.704 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:12.704 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:12.704 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:12.704 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44697 2024-09-04T17:47:12.704 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.704 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:12.704 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.704 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:12.704 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253684+0000", 2024-09-04T17:47:12.705 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:12.705 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.705 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.705 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:12.705 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253684+0000", 2024-09-04T17:47:12.705 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:12.705 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.705 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.705 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:12.705 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253685+0000", 2024-09-04T17:47:12.705 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0699999999999999e-06 2024-09-04T17:47:12.705 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.705 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.705 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:12.706 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253690+0000", 2024-09-04T17:47:12.706 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1959999999999999e-06 2024-09-04T17:47:12.706 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.706 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.706 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:12.706 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253707+0000", 2024-09-04T17:47:12.706 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6946e-05 2024-09-04T17:47:12.706 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.706 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.706 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:12.707 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253720+0000", 2024-09-04T17:47:12.707 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3076e-05 2024-09-04T17:47:12.707 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.707 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.707 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:12.707 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304745+0000", 2024-09-04T17:47:12.707 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051024760000000002 2024-09-04T17:47:12.707 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.707 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.707 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:12.707 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304757+0000", 2024-09-04T17:47:12.707 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2058999999999999e-05 2024-09-04T17:47:12.707 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:12.708 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:12.708 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:12.708 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.708 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.708 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44698 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:12.708 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253737+0000", 2024-09-04T17:47:12.708 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.354370888999998, 2024-09-04T17:47:12.708 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051176078, 2024-09-04T17:47:12.708 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:12.708 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:12.708 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:12.708 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:12.708 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:12.708 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:12.709 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44698 2024-09-04T17:47:12.709 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.709 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:12.709 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.709 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:12.709 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253737+0000", 2024-09-04T17:47:12.709 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:12.709 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.709 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.709 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:12.709 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253737+0000", 2024-09-04T17:47:12.709 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:12.709 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.710 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.710 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:12.710 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253738+0000", 2024-09-04T17:47:12.710 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0300000000000001e-06 2024-09-04T17:47:12.710 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.710 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.710 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:12.710 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253752+0000", 2024-09-04T17:47:12.710 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4358999999999999e-05 2024-09-04T17:47:12.710 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.710 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.710 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:12.711 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253771+0000", 2024-09-04T17:47:12.711 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8879999999999999e-05 2024-09-04T17:47:12.711 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.711 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.711 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:12.711 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253785+0000", 2024-09-04T17:47:12.711 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3852000000000001e-05 2024-09-04T17:47:12.711 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.711 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.711 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:12.711 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304876+0000", 2024-09-04T17:47:12.711 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051091105999999997 2024-09-04T17:47:12.711 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.712 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.712 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:12.712 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304913+0000", 2024-09-04T17:47:12.712 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.6850999999999997e-05 2024-09-04T17:47:12.712 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:12.712 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:12.712 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:12.712 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.712 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.712 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44699 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:12.712 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253804+0000", 2024-09-04T17:47:12.712 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.354303365, 2024-09-04T17:47:12.712 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051172250000000002, 2024-09-04T17:47:12.713 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:12.713 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:12.713 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:12.713 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:12.713 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:12.713 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:12.713 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44699 2024-09-04T17:47:12.713 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.713 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:12.713 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.713 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:12.713 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253804+0000", 2024-09-04T17:47:12.713 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:12.714 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.714 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.714 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:12.714 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253804+0000", 2024-09-04T17:47:12.714 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:12.714 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.714 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.714 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:12.714 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253805+0000", 2024-09-04T17:47:12.714 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.08e-06 2024-09-04T17:47:12.714 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.714 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.714 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:12.714 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253810+0000", 2024-09-04T17:47:12.715 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1830000000000002e-06 2024-09-04T17:47:12.715 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.715 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.715 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:12.715 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253828+0000", 2024-09-04T17:47:12.715 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7662000000000001e-05 2024-09-04T17:47:12.715 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.715 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.715 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:12.715 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253841+0000", 2024-09-04T17:47:12.715 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3203999999999999e-05 2024-09-04T17:47:12.715 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.715 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.716 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:12.716 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304966+0000", 2024-09-04T17:47:12.716 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051125273999999998 2024-09-04T17:47:12.716 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.716 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.716 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:12.716 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304976+0000", 2024-09-04T17:47:12.716 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.8470000000000008e-06 2024-09-04T17:47:12.716 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:12.716 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:12.716 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:12.716 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.716 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.716 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44700 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:12.717 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253858+0000", 2024-09-04T17:47:12.717 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.354249711000001, 2024-09-04T17:47:12.717 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051218957000000002, 2024-09-04T17:47:12.717 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:12.717 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:12.717 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:12.717 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:12.717 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:12.717 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:12.717 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44700 2024-09-04T17:47:12.717 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.717 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:12.717 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.718 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:12.718 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253858+0000", 2024-09-04T17:47:12.718 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:12.718 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.718 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.718 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:12.718 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253858+0000", 2024-09-04T17:47:12.718 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:12.718 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.718 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.718 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:12.718 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253859+0000", 2024-09-04T17:47:12.718 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.217e-06 2024-09-04T17:47:12.719 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.719 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.719 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:12.719 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253864+0000", 2024-09-04T17:47:12.719 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7600000000000002e-06 2024-09-04T17:47:12.719 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.719 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.719 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:12.719 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253880+0000", 2024-09-04T17:47:12.719 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6501999999999999e-05 2024-09-04T17:47:12.719 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.719 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.719 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:12.719 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253899+0000", 2024-09-04T17:47:12.720 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8907e-05 2024-09-04T17:47:12.720 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.720 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.720 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:12.720 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305052+0000", 2024-09-04T17:47:12.720 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051152989000000003 2024-09-04T17:47:12.720 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.720 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.720 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:12.720 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305077+0000", 2024-09-04T17:47:12.720 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4581999999999999e-05 2024-09-04T17:47:12.720 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:12.720 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:12.721 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:12.721 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.721 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.721 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44701 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:12.721 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253917+0000", 2024-09-04T17:47:12.721 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.354190359, 2024-09-04T17:47:12.721 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051231113000000002, 2024-09-04T17:47:12.721 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:12.721 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:12.721 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:12.721 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:12.721 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:12.721 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:12.721 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44701 2024-09-04T17:47:12.722 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.722 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:12.722 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.722 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:12.722 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253917+0000", 2024-09-04T17:47:12.722 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:12.722 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.722 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.722 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:12.722 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253917+0000", 2024-09-04T17:47:12.722 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:12.722 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.722 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.722 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:12.723 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253918+0000", 2024-09-04T17:47:12.723 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.136e-06 2024-09-04T17:47:12.723 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.723 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.723 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:12.723 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253924+0000", 2024-09-04T17:47:12.723 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.3070000000000002e-06 2024-09-04T17:47:12.723 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.723 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.723 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:12.723 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253940+0000", 2024-09-04T17:47:12.723 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6612e-05 2024-09-04T17:47:12.723 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.723 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.724 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:12.724 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253954+0000", 2024-09-04T17:47:12.724 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3709999999999999e-05 2024-09-04T17:47:12.724 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.724 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.724 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:12.724 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305136+0000", 2024-09-04T17:47:12.724 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051181935999999997 2024-09-04T17:47:12.724 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:12.724 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:12.724 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:12.724 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305148+0000", 2024-09-04T17:47:12.724 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2412000000000001e-05 2024-09-04T17:47:12.725 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:12.725 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:12.725 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:12.725 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:12.725 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:12.725 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:12.725 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-04T17:47:12.726 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:12.606+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:12.727 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-04T17:47:12.728 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:12.722+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:12.812 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone28 from 0x7d40033 (overlap 0x21bac32) to 0x179c556 2024-09-04T17:47:12.827 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:12.822+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:12.902 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:12.902 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:47:12.902 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:47:12.902 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:12.917 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-04T17:47:12.927 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:12.922+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:12.938 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #27, image image_client.0-clone27 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent28 2024-09-04T17:47:13.029 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:13.026+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:13.043 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:13.043 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:47:13.043 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:47:13.043 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:47:13.044 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:13.057 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-04T17:47:13.130 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:13.129+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:13.173 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:13.173 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:47:13.173 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:47:13.173 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:47:13.173 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.173 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8831 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2453504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:13.173 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:47:13.174 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 60.114277627, 2024-09-04T17:47:13.174 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14731111799999999, 2024-09-04T17:47:13.174 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:13.174 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:13.174 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:13.174 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:13.174 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:13.174 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:13.174 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8831 2024-09-04T17:47:13.174 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.174 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:13.174 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.174 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:13.175 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:47:13.175 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.175 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.175 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.175 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:13.175 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:47:13.175 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.175 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.175 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.175 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:13.175 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057811+0000", 2024-09-04T17:47:13.175 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.083e-06 2024-09-04T17:47:13.175 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.175 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.176 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:13.176 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057814+0000", 2024-09-04T17:47:13.176 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.407e-06 2024-09-04T17:47:13.176 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.176 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.176 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:13.176 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057825+0000", 2024-09-04T17:47:13.176 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.17e-05 2024-09-04T17:47:13.176 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.176 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.176 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:13.176 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057836+0000", 2024-09-04T17:47:13.177 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0283e-05 2024-09-04T17:47:13.177 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.177 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.177 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:13.177 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.204762+0000", 2024-09-04T17:47:13.177 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.146926205 2024-09-04T17:47:13.177 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.177 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.177 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:13.177 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.204895+0000", 2024-09-04T17:47:13.177 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013328999999999999 2024-09-04T17:47:13.178 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.178 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.178 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:13.178 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205121+0000", 2024-09-04T17:47:13.178 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00022614999999999999 2024-09-04T17:47:13.178 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.178 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:13.178 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.178 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.178 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.178 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8832 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2490368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:13.178 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:47:13.179 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 60.114239593999997, 2024-09-04T17:47:13.179 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14770012299999999, 2024-09-04T17:47:13.179 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:13.179 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:13.179 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:13.179 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:13.179 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:13.179 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:13.179 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8832 2024-09-04T17:47:13.179 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.179 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:13.179 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.180 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:13.180 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:47:13.180 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.180 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.180 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.180 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:13.180 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:47:13.180 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.180 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.180 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.180 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:13.180 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057849+0000", 2024-09-04T17:47:13.180 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6700000000000003e-07 2024-09-04T17:47:13.180 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.181 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.181 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:13.181 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057852+0000", 2024-09-04T17:47:13.181 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6359999999999998e-06 2024-09-04T17:47:13.181 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.181 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.181 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:13.181 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057860+0000", 2024-09-04T17:47:13.181 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8140000000000007e-06 2024-09-04T17:47:13.181 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.181 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.181 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:13.181 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057868+0000", 2024-09-04T17:47:13.182 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1999999999999997e-06 2024-09-04T17:47:13.182 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.182 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.182 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:13.182 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205208+0000", 2024-09-04T17:47:13.182 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14734067300000001 2024-09-04T17:47:13.182 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.182 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.182 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:13.182 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205347+0000", 2024-09-04T17:47:13.182 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013914900000000001 2024-09-04T17:47:13.182 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.182 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.182 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:13.183 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205548+0000", 2024-09-04T17:47:13.183 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000200984 2024-09-04T17:47:13.183 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.183 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:13.183 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.183 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.183 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.183 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8833 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2527232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:13.183 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:47:13.183 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 60.114207724000003, 2024-09-04T17:47:13.183 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148062572, 2024-09-04T17:47:13.183 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:13.183 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:13.183 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:13.184 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:13.184 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:13.184 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:13.184 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8833 2024-09-04T17:47:13.184 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.184 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:13.184 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.184 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:13.184 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:47:13.184 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.184 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.184 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.185 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:13.185 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:47:13.185 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.185 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.185 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.185 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:13.185 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057881+0000", 2024-09-04T17:47:13.185 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4300000000000002e-07 2024-09-04T17:47:13.185 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.185 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.185 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:13.185 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057893+0000", 2024-09-04T17:47:13.185 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1973999999999999e-05 2024-09-04T17:47:13.186 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.186 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.186 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:13.186 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057906+0000", 2024-09-04T17:47:13.186 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2716e-05 2024-09-04T17:47:13.186 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.186 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.186 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:13.186 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057913+0000", 2024-09-04T17:47:13.186 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3930000000000002e-06 2024-09-04T17:47:13.186 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.186 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.186 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:13.186 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205621+0000", 2024-09-04T17:47:13.187 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14770788800000001 2024-09-04T17:47:13.187 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.187 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.187 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:13.187 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205740+0000", 2024-09-04T17:47:13.187 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000118936 2024-09-04T17:47:13.187 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.187 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.187 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:13.187 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205943+0000", 2024-09-04T17:47:13.187 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020292199999999999 2024-09-04T17:47:13.187 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.187 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:13.188 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.188 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.188 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.188 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8834 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2564096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:13.188 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:47:13.188 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 60.114164445, 2024-09-04T17:47:13.188 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14846753200000001, 2024-09-04T17:47:13.188 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:13.188 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:13.188 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:13.188 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:13.188 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:13.188 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:13.188 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8834 2024-09-04T17:47:13.189 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.189 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:13.189 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.189 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:13.189 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:47:13.189 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.189 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.189 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.189 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:13.189 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:47:13.189 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.189 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.189 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.190 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:13.190 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057925+0000", 2024-09-04T17:47:13.190 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0270000000000001e-06 2024-09-04T17:47:13.190 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.190 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.190 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:13.190 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057936+0000", 2024-09-04T17:47:13.190 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.112e-05 2024-09-04T17:47:13.190 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.190 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.190 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:13.190 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057946+0000", 2024-09-04T17:47:13.191 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.009e-05 2024-09-04T17:47:13.191 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.191 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.191 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:13.191 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057953+0000", 2024-09-04T17:47:13.191 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.6229999999999997e-06 2024-09-04T17:47:13.191 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.191 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.191 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:13.191 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206027+0000", 2024-09-04T17:47:13.191 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148073591 2024-09-04T17:47:13.191 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.191 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.192 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:13.192 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206161+0000", 2024-09-04T17:47:13.192 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013411599999999999 2024-09-04T17:47:13.192 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.192 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.192 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:13.192 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206391+0000", 2024-09-04T17:47:13.192 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000229965 2024-09-04T17:47:13.192 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.192 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:13.192 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.192 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.192 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.192 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8835 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2600960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:13.192 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:47:13.193 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 60.114124795000002, 2024-09-04T17:47:13.193 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14882941599999999, 2024-09-04T17:47:13.193 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:13.193 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:13.193 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:13.193 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:13.196 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:13.196 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:13.196 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8835 2024-09-04T17:47:13.196 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.196 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:13.196 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.196 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:13.196 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:47:13.196 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.196 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.196 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.196 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:13.197 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:47:13.197 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.197 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.197 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.197 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:13.197 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058544+0000", 2024-09-04T17:47:13.197 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00058061899999999999 2024-09-04T17:47:13.197 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.197 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.197 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:13.197 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058550+0000", 2024-09-04T17:47:13.197 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.1129999999999997e-06 2024-09-04T17:47:13.198 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.198 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.198 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:13.198 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058566+0000", 2024-09-04T17:47:13.198 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5820000000000001e-05 2024-09-04T17:47:13.198 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.198 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.198 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:13.198 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058578+0000", 2024-09-04T17:47:13.198 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2537e-05 2024-09-04T17:47:13.198 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.198 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.198 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:13.199 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206470+0000", 2024-09-04T17:47:13.199 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14789153799999999 2024-09-04T17:47:13.199 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.199 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.199 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:13.199 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206589+0000", 2024-09-04T17:47:13.199 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000118837 2024-09-04T17:47:13.199 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.199 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.199 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:13.199 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206793+0000", 2024-09-04T17:47:13.199 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000203952 2024-09-04T17:47:13.199 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.199 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:13.200 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.200 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.200 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.200 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8836 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2654208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:13.200 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:47:13.200 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 60.113492106999999, 2024-09-04T17:47:13.200 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14862433999999999, 2024-09-04T17:47:13.200 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:13.200 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:13.200 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:13.200 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:13.200 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:13.200 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:13.201 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8836 2024-09-04T17:47:13.201 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.201 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:13.201 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.201 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:13.201 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:47:13.201 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.201 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.201 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.201 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:13.201 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:47:13.201 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.202 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.202 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.202 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:13.202 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062663+0000", 2024-09-04T17:47:13.202 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0040670619999999998 2024-09-04T17:47:13.202 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.202 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.202 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:13.202 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062668+0000", 2024-09-04T17:47:13.202 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4199999999999998e-06 2024-09-04T17:47:13.202 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.203 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.203 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:13.203 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062689+0000", 2024-09-04T17:47:13.203 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0356999999999998e-05 2024-09-04T17:47:13.203 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.203 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.203 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:13.203 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062726+0000", 2024-09-04T17:47:13.203 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7302999999999998e-05 2024-09-04T17:47:13.203 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.203 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.203 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:13.203 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206875+0000", 2024-09-04T17:47:13.204 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14414883100000001 2024-09-04T17:47:13.204 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.204 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.204 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:13.204 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.207016+0000", 2024-09-04T17:47:13.204 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000140822 2024-09-04T17:47:13.204 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.204 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.204 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:13.204 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.207220+0000", 2024-09-04T17:47:13.204 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020454499999999999 2024-09-04T17:47:13.204 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.204 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:13.204 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.205 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.205 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.205 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8844 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2928640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:13.205 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:47:13.205 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 60.109128024, 2024-09-04T17:47:13.205 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14752153100000001, 2024-09-04T17:47:13.205 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:13.205 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:13.205 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:13.205 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:13.205 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:13.205 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:13.205 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8844 2024-09-04T17:47:13.205 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.206 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:13.206 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.206 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:13.206 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:47:13.206 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.206 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.206 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.206 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:13.206 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:47:13.206 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.206 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.206 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.206 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:13.207 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062961+0000", 2024-09-04T17:47:13.207 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6700000000000003e-07 2024-09-04T17:47:13.207 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.207 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.207 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:13.207 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062973+0000", 2024-09-04T17:47:13.207 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2109999999999999e-05 2024-09-04T17:47:13.207 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.207 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.207 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:13.207 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062982+0000", 2024-09-04T17:47:13.207 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.6900000000000004e-06 2024-09-04T17:47:13.208 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.208 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.208 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:13.208 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062990+0000", 2024-09-04T17:47:13.208 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1899999999999998e-06 2024-09-04T17:47:13.208 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.208 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.208 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:13.208 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210090+0000", 2024-09-04T17:47:13.209 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14710026300000001 2024-09-04T17:47:13.209 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.209 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.209 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:13.209 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210225+0000", 2024-09-04T17:47:13.209 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013532400000000001 2024-09-04T17:47:13.209 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.209 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.209 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:13.209 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210481+0000", 2024-09-04T17:47:13.209 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00025628700000000001 2024-09-04T17:47:13.209 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.209 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:13.210 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.210 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.210 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.210 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8845 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2965504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:13.210 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:47:13.210 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 60.109088301, 2024-09-04T17:47:13.210 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147916819, 2024-09-04T17:47:13.210 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:13.210 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:13.210 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:13.210 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:13.210 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:13.210 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:13.210 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8845 2024-09-04T17:47:13.211 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.211 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:13.211 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.211 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:13.211 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:47:13.211 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.211 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.211 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.211 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:13.211 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:47:13.211 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.211 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.211 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.212 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:13.212 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:47:13.212 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.0399999999999995e-07 2024-09-04T17:47:13.212 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.212 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.212 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:13.212 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063003+0000", 2024-09-04T17:47:13.212 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6529999999999998e-06 2024-09-04T17:47:13.212 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.212 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.212 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:13.212 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063012+0000", 2024-09-04T17:47:13.212 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9500000000000007e-06 2024-09-04T17:47:13.213 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.213 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.213 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:13.213 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063027+0000", 2024-09-04T17:47:13.213 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5443000000000001e-05 2024-09-04T17:47:13.213 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.213 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.213 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:13.213 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210571+0000", 2024-09-04T17:47:13.213 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14754353100000001 2024-09-04T17:47:13.213 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.213 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.213 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:13.214 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210714+0000", 2024-09-04T17:47:13.214 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014295599999999999 2024-09-04T17:47:13.214 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.214 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.214 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:13.214 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210916+0000", 2024-09-04T17:47:13.214 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020258200000000001 2024-09-04T17:47:13.214 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.214 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:13.214 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.214 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.214 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.215 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8846 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3002368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:13.215 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:47:13.215 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 60.109050764000003, 2024-09-04T17:47:13.215 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14828011599999999, 2024-09-04T17:47:13.215 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:13.215 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:13.215 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:13.215 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:13.215 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:13.215 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:13.215 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8846 2024-09-04T17:47:13.215 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.215 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:13.215 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.216 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:13.216 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:47:13.216 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.216 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.216 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.216 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:13.216 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:47:13.216 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.216 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.216 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.216 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:13.216 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063038+0000", 2024-09-04T17:47:13.217 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.23e-07 2024-09-04T17:47:13.217 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.217 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.217 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:13.217 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063051+0000", 2024-09-04T17:47:13.217 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2757e-05 2024-09-04T17:47:13.217 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.217 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.217 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:13.217 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063070+0000", 2024-09-04T17:47:13.217 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9663000000000001e-05 2024-09-04T17:47:13.217 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.217 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.217 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:13.218 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063079+0000", 2024-09-04T17:47:13.218 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.0929999999999992e-06 2024-09-04T17:47:13.218 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.218 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.218 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:13.218 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210989+0000", 2024-09-04T17:47:13.218 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14790984500000001 2024-09-04T17:47:13.218 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.218 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.218 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:13.218 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211109+0000", 2024-09-04T17:47:13.218 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000120056 2024-09-04T17:47:13.218 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.218 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.219 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:13.219 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211317+0000", 2024-09-04T17:47:13.219 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020797900000000001 2024-09-04T17:47:13.219 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.219 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:13.219 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.219 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.219 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.219 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8847 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3039232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:13.219 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:47:13.219 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 60.108976331999997, 2024-09-04T17:47:13.219 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14859408900000001, 2024-09-04T17:47:13.219 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:13.219 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:13.219 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:13.220 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:13.220 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:13.220 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:13.220 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8847 2024-09-04T17:47:13.220 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.220 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:13.220 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.220 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:13.220 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:47:13.220 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.220 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.220 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.220 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:13.220 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:47:13.221 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.221 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.221 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.221 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:13.221 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063113+0000", 2024-09-04T17:47:13.221 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.204e-06 2024-09-04T17:47:13.221 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.221 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.221 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:13.221 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063116+0000", 2024-09-04T17:47:13.221 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.3330000000000001e-06 2024-09-04T17:47:13.221 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.221 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.221 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:13.222 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063126+0000", 2024-09-04T17:47:13.222 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0166999999999999e-05 2024-09-04T17:47:13.222 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.222 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.222 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:13.222 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063134+0000", 2024-09-04T17:47:13.222 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4329999999999998e-06 2024-09-04T17:47:13.222 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.222 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.222 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:13.222 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211392+0000", 2024-09-04T17:47:13.222 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148258109 2024-09-04T17:47:13.222 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.222 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.222 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:13.223 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211506+0000", 2024-09-04T17:47:13.223 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000113813 2024-09-04T17:47:13.223 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.223 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.223 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:13.223 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211706+0000", 2024-09-04T17:47:13.223 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020002999999999999 2024-09-04T17:47:13.223 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.223 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:13.223 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.223 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.223 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.223 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8848 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:13.223 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:47:13.224 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 60.108943994999997, 2024-09-04T17:47:13.224 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14907967599999999, 2024-09-04T17:47:13.224 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:13.224 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:13.224 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:13.224 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:13.224 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:13.224 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:13.224 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8848 2024-09-04T17:47:13.224 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.224 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:13.224 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.224 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:13.225 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:47:13.225 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.225 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.225 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.225 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:13.225 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:47:13.225 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.225 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.225 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.225 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:13.225 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070589+0000", 2024-09-04T17:47:13.225 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.007445386 2024-09-04T17:47:13.225 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.225 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.226 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:13.226 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070600+0000", 2024-09-04T17:47:13.226 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0346999999999999e-05 2024-09-04T17:47:13.226 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.226 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.226 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:13.226 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070622+0000", 2024-09-04T17:47:13.226 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2289999999999998e-05 2024-09-04T17:47:13.226 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.226 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.226 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:13.226 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070636+0000", 2024-09-04T17:47:13.226 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4093e-05 2024-09-04T17:47:13.226 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.227 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.227 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:13.227 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211815+0000", 2024-09-04T17:47:13.227 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14117874699999999 2024-09-04T17:47:13.227 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.227 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.227 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:13.227 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211983+0000", 2024-09-04T17:47:13.227 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00016852800000000001 2024-09-04T17:47:13.227 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.227 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.227 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:13.227 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.212224+0000", 2024-09-04T17:47:13.228 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024028499999999999 2024-09-04T17:47:13.228 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.228 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:13.228 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.228 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.228 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.228 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8862 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2019328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:13.228 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:47:13.228 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 60.100894527000001, 2024-09-04T17:47:13.229 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14755236399999999, 2024-09-04T17:47:13.229 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:13.229 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:13.229 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:13.229 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:13.229 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:13.229 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:13.229 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8862 2024-09-04T17:47:13.229 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.229 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:13.229 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.230 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:13.230 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:47:13.230 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.230 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.230 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.230 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:13.230 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:47:13.230 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.230 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.230 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.230 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:13.230 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071194+0000", 2024-09-04T17:47:13.230 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.5000000000000002e-07 2024-09-04T17:47:13.231 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.231 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.231 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:13.231 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071197+0000", 2024-09-04T17:47:13.231 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5900000000000002e-06 2024-09-04T17:47:13.231 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.231 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.231 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:13.231 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071206+0000", 2024-09-04T17:47:13.231 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9700000000000005e-06 2024-09-04T17:47:13.231 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.231 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.231 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:13.232 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071213+0000", 2024-09-04T17:47:13.232 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.9070000000000001e-06 2024-09-04T17:47:13.232 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.232 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.232 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:13.232 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218396+0000", 2024-09-04T17:47:13.232 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147183273 2024-09-04T17:47:13.232 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.232 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.232 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:13.232 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218538+0000", 2024-09-04T17:47:13.232 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014194199999999999 2024-09-04T17:47:13.232 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.233 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.233 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:13.233 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218746+0000", 2024-09-04T17:47:13.233 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000208032 2024-09-04T17:47:13.233 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.233 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:13.233 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.233 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.233 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.233 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8863 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2056192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:13.233 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:47:13.233 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 60.100866216999997, 2024-09-04T17:47:13.233 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14792281099999999, 2024-09-04T17:47:13.233 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:13.234 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:13.234 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:13.234 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:13.234 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:13.234 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:13.234 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8863 2024-09-04T17:47:13.234 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.234 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:13.234 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.234 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:13.234 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:47:13.234 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.234 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.234 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.235 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:13.235 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:47:13.235 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.235 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.235 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.235 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:13.235 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:47:13.235 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1299999999999999e-07 2024-09-04T17:47:13.235 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.235 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.235 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:13.235 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071225+0000", 2024-09-04T17:47:13.235 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6199999999999999e-06 2024-09-04T17:47:13.236 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.236 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.236 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:13.236 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071234+0000", 2024-09-04T17:47:13.236 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8769999999999999e-06 2024-09-04T17:47:13.236 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.236 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.236 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:13.236 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071251+0000", 2024-09-04T17:47:13.236 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.738e-05 2024-09-04T17:47:13.236 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.236 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.237 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:13.237 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218798+0000", 2024-09-04T17:47:13.237 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14754645 2024-09-04T17:47:13.237 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.237 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.237 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:13.237 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218926+0000", 2024-09-04T17:47:13.237 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000128446 2024-09-04T17:47:13.237 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.237 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.237 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:13.237 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219145+0000", 2024-09-04T17:47:13.237 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021832500000000001 2024-09-04T17:47:13.238 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.238 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:13.238 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.238 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.238 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.238 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8864 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2093056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:13.238 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:47:13.238 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 60.100824641000003, 2024-09-04T17:47:13.238 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148277309, 2024-09-04T17:47:13.238 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:13.238 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:13.238 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:13.238 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:13.239 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:13.239 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:13.239 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8864 2024-09-04T17:47:13.239 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.239 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:13.239 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.239 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:13.239 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:47:13.239 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.239 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.239 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.239 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:13.239 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:47:13.240 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.240 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.240 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.240 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:13.240 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071264+0000", 2024-09-04T17:47:13.240 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3e-07 2024-09-04T17:47:13.240 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.240 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.240 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:13.240 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071267+0000", 2024-09-04T17:47:13.240 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0139999999999999e-06 2024-09-04T17:47:13.240 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.240 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.241 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:13.241 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071276+0000", 2024-09-04T17:47:13.241 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1689999999999998e-06 2024-09-04T17:47:13.241 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.241 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.241 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:13.241 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071283+0000", 2024-09-04T17:47:13.241 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2269999999999996e-06 2024-09-04T17:47:13.241 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.241 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.241 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:13.241 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219195+0000", 2024-09-04T17:47:13.241 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147911868 2024-09-04T17:47:13.242 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.242 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.242 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:13.242 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219324+0000", 2024-09-04T17:47:13.242 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00012835900000000001 2024-09-04T17:47:13.242 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.242 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.242 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:13.242 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219541+0000", 2024-09-04T17:47:13.242 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000216942 2024-09-04T17:47:13.242 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.242 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:13.243 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.243 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.243 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.243 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8865 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2109440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:13.243 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:47:13.243 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 60.100795181000002, 2024-09-04T17:47:13.243 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14865747300000001, 2024-09-04T17:47:13.243 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:13.243 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:13.243 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:13.243 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:13.244 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:13.244 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:13.244 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8865 2024-09-04T17:47:13.244 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.244 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:13.244 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.244 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:13.244 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:47:13.244 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.244 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.244 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.244 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:13.245 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:47:13.245 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.245 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.245 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.245 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:13.245 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:47:13.245 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7999999999999995e-07 2024-09-04T17:47:13.245 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.245 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.245 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:13.245 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071296+0000", 2024-09-04T17:47:13.246 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6900000000000001e-06 2024-09-04T17:47:13.246 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.246 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.246 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:13.246 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071305+0000", 2024-09-04T17:47:13.246 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.5499999999999995e-06 2024-09-04T17:47:13.246 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.246 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.246 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:13.246 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071312+0000", 2024-09-04T17:47:13.246 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1559999999999998e-06 2024-09-04T17:47:13.246 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.246 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.247 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:13.247 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219591+0000", 2024-09-04T17:47:13.247 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14827944900000001 2024-09-04T17:47:13.247 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.247 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.247 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:13.247 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219731+0000", 2024-09-04T17:47:13.247 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013931600000000001 2024-09-04T17:47:13.247 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.247 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.247 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:13.247 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219950+0000", 2024-09-04T17:47:13.247 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021963199999999999 2024-09-04T17:47:13.248 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.248 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:13.248 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.248 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.248 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.248 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8866 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2146304~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:13.248 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:47:13.248 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 60.100766948, 2024-09-04T17:47:13.248 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14906117899999999, 2024-09-04T17:47:13.248 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:13.248 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:13.248 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:13.249 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:13.249 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:13.249 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:13.249 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8866 2024-09-04T17:47:13.249 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.250 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:13.250 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.250 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:13.250 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:47:13.250 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.250 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.250 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.250 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:13.250 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:47:13.250 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.250 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.251 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.251 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:13.251 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071322+0000", 2024-09-04T17:47:13.251 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7999999999999995e-07 2024-09-04T17:47:13.251 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.251 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.251 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:13.251 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071338+0000", 2024-09-04T17:47:13.251 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5872999999999999e-05 2024-09-04T17:47:13.251 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.251 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.252 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:13.252 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071347+0000", 2024-09-04T17:47:13.252 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4669999999999993e-06 2024-09-04T17:47:13.252 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.252 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.252 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:13.252 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071354+0000", 2024-09-04T17:47:13.252 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2099999999999996e-06 2024-09-04T17:47:13.252 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.252 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.253 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:13.253 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220008+0000", 2024-09-04T17:47:13.253 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148653642 2024-09-04T17:47:13.253 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.253 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.253 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:13.253 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220155+0000", 2024-09-04T17:47:13.253 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014726600000000001 2024-09-04T17:47:13.253 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.253 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.253 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:13.253 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220382+0000", 2024-09-04T17:47:13.253 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00022704100000000001 2024-09-04T17:47:13.254 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.254 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:13.254 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.254 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.254 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.254 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8867 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2183168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:13.254 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:47:13.254 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 60.100723594999998, 2024-09-04T17:47:13.254 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.149445831, 2024-09-04T17:47:13.254 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:13.254 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:13.254 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:13.254 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:13.254 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:13.255 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:13.255 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8867 2024-09-04T17:47:13.255 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.255 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:13.255 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.255 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:13.255 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:47:13.255 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.255 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.255 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.255 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:13.255 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:47:13.255 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.255 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.256 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.256 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:13.256 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078653+0000", 2024-09-04T17:47:13.256 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0072890430000000003 2024-09-04T17:47:13.256 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.256 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.256 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:13.256 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078664+0000", 2024-09-04T17:47:13.256 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0307e-05 2024-09-04T17:47:13.256 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.256 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.256 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:13.256 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078690+0000", 2024-09-04T17:47:13.256 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6780000000000001e-05 2024-09-04T17:47:13.257 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.257 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.257 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:13.257 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078706+0000", 2024-09-04T17:47:13.257 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5676999999999999e-05 2024-09-04T17:47:13.257 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.257 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.257 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:13.257 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220467+0000", 2024-09-04T17:47:13.257 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14176103500000001 2024-09-04T17:47:13.257 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.257 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.257 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:13.258 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220595+0000", 2024-09-04T17:47:13.258 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000127427 2024-09-04T17:47:13.258 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.258 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.258 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:13.258 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220810+0000", 2024-09-04T17:47:13.258 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021556200000000001 2024-09-04T17:47:13.258 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.258 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:13.258 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.258 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.258 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.258 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8880 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2658304~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:13.259 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.079158+0000", 2024-09-04T17:47:13.259 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 60.092930316, 2024-09-04T17:47:13.259 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14718968499999999, 2024-09-04T17:47:13.259 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:13.259 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:13.259 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:13.259 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:13.259 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:13.259 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:13.259 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8880 2024-09-04T17:47:13.259 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.259 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:13.259 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.260 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:13.260 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079158+0000", 2024-09-04T17:47:13.260 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.260 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.260 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.260 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:13.260 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079158+0000", 2024-09-04T17:47:13.260 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.260 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.260 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.260 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:13.260 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079158+0000", 2024-09-04T17:47:13.260 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3e-07 2024-09-04T17:47:13.261 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.261 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.261 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:13.261 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079161+0000", 2024-09-04T17:47:13.261 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.57e-06 2024-09-04T17:47:13.261 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.261 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.261 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:13.261 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079170+0000", 2024-09-04T17:47:13.261 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.0170000000000002e-06 2024-09-04T17:47:13.261 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.261 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.261 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:13.261 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079177+0000", 2024-09-04T17:47:13.262 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.0530000000000001e-06 2024-09-04T17:47:13.262 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.262 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.262 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:13.262 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.225946+0000", 2024-09-04T17:47:13.262 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14676936500000001 2024-09-04T17:47:13.262 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.262 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.262 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:13.262 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226098+0000", 2024-09-04T17:47:13.262 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000151805 2024-09-04T17:47:13.262 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.262 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.263 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:13.263 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226347+0000", 2024-09-04T17:47:13.263 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024914499999999999 2024-09-04T17:47:13.263 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.263 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:13.263 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.263 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.263 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.263 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8881 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2695168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:13.263 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:47:13.263 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 60.092901525999999, 2024-09-04T17:47:13.263 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147543547, 2024-09-04T17:47:13.263 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:13.264 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:13.264 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:13.264 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:13.264 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:13.264 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:13.264 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8881 2024-09-04T17:47:13.264 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.264 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:13.264 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.264 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:13.264 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:47:13.264 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.264 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.265 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.265 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:13.265 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:47:13.265 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.265 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.265 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.265 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:13.265 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079187+0000", 2024-09-04T17:47:13.265 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7700000000000004e-07 2024-09-04T17:47:13.265 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.265 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.265 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:13.265 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079198+0000", 2024-09-04T17:47:13.266 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0849999999999999e-05 2024-09-04T17:47:13.266 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.266 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.266 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:13.266 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079207+0000", 2024-09-04T17:47:13.266 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5599999999999999e-06 2024-09-04T17:47:13.266 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.266 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.266 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:13.266 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079224+0000", 2024-09-04T17:47:13.266 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6387000000000001e-05 2024-09-04T17:47:13.266 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.266 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.266 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:13.267 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226414+0000", 2024-09-04T17:47:13.267 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14718975000000001 2024-09-04T17:47:13.267 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.267 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.267 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:13.267 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226534+0000", 2024-09-04T17:47:13.267 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000120288 2024-09-04T17:47:13.267 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.267 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.267 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:13.267 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226730+0000", 2024-09-04T17:47:13.267 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000196035 2024-09-04T17:47:13.267 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.268 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:13.268 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.268 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.268 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.268 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8882 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2732032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:13.268 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:47:13.268 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 60.092853312000003, 2024-09-04T17:47:13.268 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14795022399999999, 2024-09-04T17:47:13.268 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:13.268 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:13.268 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:13.268 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:13.268 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:13.269 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:13.269 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8882 2024-09-04T17:47:13.269 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.269 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:13.269 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.269 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:13.269 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:47:13.269 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.269 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.269 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.269 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:13.269 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:47:13.269 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.270 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.270 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.270 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:13.270 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086588+0000", 2024-09-04T17:47:13.270 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0073530959999999999 2024-09-04T17:47:13.270 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.270 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.270 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:13.270 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086598+0000", 2024-09-04T17:47:13.270 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0122999999999999e-05 2024-09-04T17:47:13.270 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.270 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.270 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:13.270 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086626+0000", 2024-09-04T17:47:13.271 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.7914000000000001e-05 2024-09-04T17:47:13.271 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.271 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.271 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:13.271 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086641+0000", 2024-09-04T17:47:13.271 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5270000000000001e-05 2024-09-04T17:47:13.271 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.271 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.271 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:13.271 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226801+0000", 2024-09-04T17:47:13.271 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.140160321 2024-09-04T17:47:13.271 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.271 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.272 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:13.272 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226951+0000", 2024-09-04T17:47:13.272 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014965200000000001 2024-09-04T17:47:13.272 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.272 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.272 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:13.272 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.227185+0000", 2024-09-04T17:47:13.272 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00023384799999999999 2024-09-04T17:47:13.272 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.272 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:13.272 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.272 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.272 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:13.273 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:13.274 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-04T17:47:13.274 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:13.273+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:13.337 INFO:teuthology.orchestra.run.smithi190.stdout: marking collection for removal 2024-09-04T17:47:13.337 INFO:teuthology.orchestra.run.smithi190.stdout:setting '_remove' omap key 2024-09-04T17:47:13.337 INFO:teuthology.orchestra.run.smithi190.stdout:finish_remove_pgs 3.1e_head removing 3.1e 2024-09-04T17:47:13.342 INFO:teuthology.orchestra.run.smithi190.stdout:Remove successful 2024-09-04T17:47:13.345 INFO:teuthology.orchestra.run.smithi092.stdout:3712 read 0x27b485b thru 0x27bc4d8 (0x7c7e bytes) 2024-09-04T17:47:13.347 INFO:teuthology.orchestra.run.smithi092.stdout:3713 writesame 0x4ee7084 thru 0x4eea69a data_size 0x1(0x3617 bytes) 2024-09-04T17:47:13.374 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:13.370+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:13.420 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:13.420 INFO:teuthology.orchestra.run.smithi092.stdout:3715 read 0x15dcbeb thru 0x15e0436 (0x384c bytes) 2024-09-04T17:47:13.422 INFO:teuthology.orchestra.run.smithi092.stdout:3716 read 0x385ceef thru 0x385fd35 (0x2e47 bytes) 2024-09-04T17:47:13.423 INFO:teuthology.orchestra.run.smithi092.stdout:3717 punch from 0x39d29a4 to 0x39d8d5d, (0x63b9 bytes) 2024-09-04T17:47:13.423 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:13.433 INFO:teuthology.orchestra.run.smithi092.stdout:3719 trunc from 0x7d40033 to 0xe00ff2b 2024-09-04T17:47:13.441 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:13.452 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:13.452 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:47:13.452 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:47:13.452 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:13.459 INFO:teuthology.orchestra.run.smithi092.stdout:3722 trunc from 0xe00ff2b to 0x1f7c7cf 2024-09-04T17:47:13.466 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-04T17:47:13.476 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:13.470+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:13.577 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:13.574+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:13.677 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:13.677+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:13.682 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:13.682 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:47:13.682 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:47:13.682 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:47:13.682 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:13.701 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-04T17:47:13.705 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is unset 2024-09-04T17:47:13.714 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-04T17:47:13.734 INFO:teuthology.orchestra.run.smithi092.stdout:3723 punch from 0x1894c84 to 0x18a3d63, (0xf0df bytes) 2024-09-04T17:47:13.734 INFO:teuthology.orchestra.run.smithi092.stdout:3726 read 0x1d02876 thru 0x1d06258 (0x39e3 bytes) 2024-09-04T17:47:13.735 INFO:teuthology.orchestra.run.smithi092.stdout:3727 punch from 0x14f291f to 0x1501b4b, (0xf22c bytes) 2024-09-04T17:47:13.735 INFO:teuthology.orchestra.run.smithi092.stdout:3728 read 0x593119 thru 0x59f307 (0xc1ef bytes) 2024-09-04T17:47:13.742 INFO:teuthology.orchestra.run.smithi092.stdout:3729 trunc from 0x1f7c7cf to 0x5d8a815 2024-09-04T17:47:13.748 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:13.748 INFO:teuthology.orchestra.run.smithi092.stdout:3731 write 0x3d144de thru 0x3d170d0 (0x2bf3 bytes) 2024-09-04T17:47:13.751 INFO:teuthology.orchestra.run.smithi092.stdout:3732 write 0x5a33803 thru 0x5a40106 (0xc904 bytes) 2024-09-04T17:47:13.767 INFO:teuthology.orchestra.run.smithi092.stdout:3733 read 0x58aef9e thru 0x58bda6b (0xeace bytes) 2024-09-04T17:47:13.779 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:13.774+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:13.784 INFO:teuthology.orchestra.run.smithi092.stdout:3734 write 0xc6d6699 thru 0xc6debc5 (0x852d bytes) 2024-09-04T17:47:13.804 INFO:teuthology.orchestra.run.smithi092.stdout:3736 writesame 0xdda1c90 thru 0xddae96a data_size 0x3(0xccdb bytes) 2024-09-04T17:47:13.804 INFO:teuthology.orchestra.run.smithi092.stdout:3737 punch from 0x6533ed9 to 0x653f454, (0xb57b bytes) 2024-09-04T17:47:13.804 INFO:teuthology.orchestra.run.smithi092.stdout:3738 writesame 0x14a655f thru 0x14af4f0 data_size 0x2f(0x8f92 bytes) 2024-09-04T17:47:13.880 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:13.878+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:13.885 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:47:13.885 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:47:13.893 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-2 --journal-path /var/lib/ceph/osd/ceph-2/journal --op list-pgs 2024-09-04T17:47:13.899 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:13.899 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:47:13.899 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:47:13.899 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:47:13.899 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.899 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60913 3.7 3:e4932581:::rbd_data.11341dfde0c5.0000000000000002:1f [read 2629632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:13.899 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822759+0000", 2024-09-04T17:47:13.899 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.076556373999999996, 2024-09-04T17:47:13.899 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.069093719999999997, 2024-09-04T17:47:13.900 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:13.900 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:13.900 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:13.900 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:13.900 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:13.900 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:13.900 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60913 2024-09-04T17:47:13.900 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.900 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:13.900 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.900 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:13.900 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822759+0000", 2024-09-04T17:47:13.901 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.901 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.901 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.901 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:13.901 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822759+0000", 2024-09-04T17:47:13.901 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.901 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.901 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.901 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:13.902 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822760+0000", 2024-09-04T17:47:13.902 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.8800000000000002e-07 2024-09-04T17:47:13.902 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.902 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.902 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:13.902 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822763+0000", 2024-09-04T17:47:13.902 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4759999999999998e-06 2024-09-04T17:47:13.902 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.902 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.902 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:13.902 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822772+0000", 2024-09-04T17:47:13.902 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.6600000000000001e-06 2024-09-04T17:47:13.903 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.903 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.903 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:13.903 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822777+0000", 2024-09-04T17:47:13.903 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4940000000000003e-06 2024-09-04T17:47:13.903 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.903 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.903 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:13.903 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.891482+0000", 2024-09-04T17:47:13.903 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.068704383999999993 2024-09-04T17:47:13.903 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.903 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.903 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:13.903 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.891576+0000", 2024-09-04T17:47:13.904 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.3851000000000007e-05 2024-09-04T17:47:13.904 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.904 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.904 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:13.904 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.891853+0000", 2024-09-04T17:47:13.904 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00027716699999999998 2024-09-04T17:47:13.904 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.904 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:13.904 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.904 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.904 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.904 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60914 3.7 3:e4932581:::rbd_data.11341dfde0c5.0000000000000002:1f [read 2736128~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:13.904 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822786+0000", 2024-09-04T17:47:13.905 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.076528985999999993, 2024-09-04T17:47:13.905 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.069382541000000006, 2024-09-04T17:47:13.905 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:13.905 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:13.905 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:13.905 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:13.905 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:13.905 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:13.905 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60914 2024-09-04T17:47:13.905 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.905 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:13.905 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.905 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:13.906 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822786+0000", 2024-09-04T17:47:13.906 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.906 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.906 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.906 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:13.906 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822786+0000", 2024-09-04T17:47:13.906 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.906 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.906 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.906 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:13.906 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822787+0000", 2024-09-04T17:47:13.906 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6300000000000005e-07 2024-09-04T17:47:13.906 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.907 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.907 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:13.907 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822789+0000", 2024-09-04T17:47:13.907 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1890000000000001e-06 2024-09-04T17:47:13.907 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.907 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.907 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:13.907 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822796+0000", 2024-09-04T17:47:13.907 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6549999999999998e-06 2024-09-04T17:47:13.907 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.907 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.907 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:13.907 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822801+0000", 2024-09-04T17:47:13.908 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6460000000000001e-06 2024-09-04T17:47:13.908 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.908 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.908 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:13.908 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.891875+0000", 2024-09-04T17:47:13.908 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.069074021999999999 2024-09-04T17:47:13.908 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.908 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.908 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:13.908 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.891937+0000", 2024-09-04T17:47:13.908 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.2173000000000003e-05 2024-09-04T17:47:13.908 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.908 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.908 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:13.909 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.892169+0000", 2024-09-04T17:47:13.909 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00023229300000000001 2024-09-04T17:47:13.909 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.909 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:13.909 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.909 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.909 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.909 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60915 3.7 3:e4932581:::rbd_data.11341dfde0c5.0000000000000002:1f [read 2842624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:13.909 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822810+0000", 2024-09-04T17:47:13.909 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.076505813000000006, 2024-09-04T17:47:13.909 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.069729901999999996, 2024-09-04T17:47:13.909 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:13.909 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:13.910 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:13.910 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:13.910 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:13.910 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:13.910 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60915 2024-09-04T17:47:13.910 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.910 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:13.910 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.910 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:13.910 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822810+0000", 2024-09-04T17:47:13.910 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.910 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.910 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.911 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:13.911 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822810+0000", 2024-09-04T17:47:13.911 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.911 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.911 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.911 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:13.911 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822810+0000", 2024-09-04T17:47:13.911 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4799999999999998e-07 2024-09-04T17:47:13.911 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.911 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.911 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:13.911 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822812+0000", 2024-09-04T17:47:13.911 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2649999999999999e-06 2024-09-04T17:47:13.912 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.912 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.912 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:13.912 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822819+0000", 2024-09-04T17:47:13.912 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6329999999999999e-06 2024-09-04T17:47:13.912 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.912 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.912 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:13.912 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822824+0000", 2024-09-04T17:47:13.912 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6240000000000001e-06 2024-09-04T17:47:13.912 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.912 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.912 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:13.912 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.892215+0000", 2024-09-04T17:47:13.913 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.069391447999999994 2024-09-04T17:47:13.913 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.913 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.913 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:13.913 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.892299+0000", 2024-09-04T17:47:13.913 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.3955999999999994e-05 2024-09-04T17:47:13.913 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.913 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.913 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:13.914 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.892540+0000", 2024-09-04T17:47:13.914 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000240428 2024-09-04T17:47:13.914 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.914 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:13.914 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.914 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.914 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.914 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60916 3.7 3:e4932581:::rbd_data.11341dfde0c5.0000000000000002:1f [read 2981888~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:13.914 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822831+0000", 2024-09-04T17:47:13.915 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.076484664999999993, 2024-09-04T17:47:13.915 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.072833690000000006, 2024-09-04T17:47:13.915 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:13.915 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:13.916 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:13.916 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:13.916 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:13.916 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:13.916 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60916 2024-09-04T17:47:13.916 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.916 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:13.916 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.916 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:13.916 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822831+0000", 2024-09-04T17:47:13.917 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.917 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.917 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.917 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:13.917 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822831+0000", 2024-09-04T17:47:13.917 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.917 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.917 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.917 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:13.917 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822831+0000", 2024-09-04T17:47:13.917 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.5999999999999999e-07 2024-09-04T17:47:13.917 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.917 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.918 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:13.918 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822833+0000", 2024-09-04T17:47:13.918 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.199e-06 2024-09-04T17:47:13.918 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.918 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.918 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:13.918 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822842+0000", 2024-09-04T17:47:13.918 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.337e-06 2024-09-04T17:47:13.918 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.918 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.918 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:13.918 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822847+0000", 2024-09-04T17:47:13.918 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.989e-06 2024-09-04T17:47:13.919 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.919 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.919 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:13.919 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.892571+0000", 2024-09-04T17:47:13.919 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.069723848000000005 2024-09-04T17:47:13.919 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.919 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.919 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:13.919 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.892628+0000", 2024-09-04T17:47:13.919 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6892999999999999e-05 2024-09-04T17:47:13.919 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.919 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.919 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:13.919 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.895664+0000", 2024-09-04T17:47:13.919 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0030369640000000001 2024-09-04T17:47:13.920 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.920 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:13.920 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.920 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.920 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.920 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60917 3.7 3:e4932581:::rbd_data.11341dfde0c5.0000000000000002:1f [read 3088384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:13.920 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822854+0000", 2024-09-04T17:47:13.920 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.076461093999999993, 2024-09-04T17:47:13.920 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073161601000000007, 2024-09-04T17:47:13.920 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:13.920 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:13.920 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:13.921 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:13.921 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:13.921 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:13.921 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60917 2024-09-04T17:47:13.921 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.921 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:13.921 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.921 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:13.921 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822854+0000", 2024-09-04T17:47:13.921 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.921 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.921 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.922 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:13.922 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822854+0000", 2024-09-04T17:47:13.922 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.922 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.922 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.922 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:13.922 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822855+0000", 2024-09-04T17:47:13.922 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2399999999999999e-07 2024-09-04T17:47:13.922 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.922 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.922 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:13.922 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822857+0000", 2024-09-04T17:47:13.922 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.345e-06 2024-09-04T17:47:13.922 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.923 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.923 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:13.923 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822861+0000", 2024-09-04T17:47:13.923 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2180000000000001e-06 2024-09-04T17:47:13.923 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.923 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.923 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:13.923 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822864+0000", 2024-09-04T17:47:13.923 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.931e-06 2024-09-04T17:47:13.923 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.923 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.923 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:13.923 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.895702+0000", 2024-09-04T17:47:13.923 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.072838146000000006 2024-09-04T17:47:13.924 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.924 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.924 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:13.924 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.895765+0000", 2024-09-04T17:47:13.924 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.2209e-05 2024-09-04T17:47:13.924 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.924 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.924 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:13.924 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896016+0000", 2024-09-04T17:47:13.924 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00025132799999999999 2024-09-04T17:47:13.924 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.924 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:13.924 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.925 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.925 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.925 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60918 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 16384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:13.925 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822870+0000", 2024-09-04T17:47:13.925 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.076445926999999997, 2024-09-04T17:47:13.925 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073283713, 2024-09-04T17:47:13.925 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:13.925 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:13.925 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:13.925 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:13.925 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:13.925 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:13.925 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60918 2024-09-04T17:47:13.926 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.926 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:13.926 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.926 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:13.926 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822870+0000", 2024-09-04T17:47:13.926 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.926 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.926 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.926 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:13.926 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822870+0000", 2024-09-04T17:47:13.926 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.926 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.926 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.926 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:13.927 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822870+0000", 2024-09-04T17:47:13.927 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.3900000000000001e-07 2024-09-04T17:47:13.927 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.927 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.927 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:13.927 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822871+0000", 2024-09-04T17:47:13.927 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1310000000000001e-06 2024-09-04T17:47:13.927 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.927 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.927 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:13.927 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822875+0000", 2024-09-04T17:47:13.927 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4309999999999996e-06 2024-09-04T17:47:13.927 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.928 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.928 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:13.928 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822879+0000", 2024-09-04T17:47:13.928 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.3610000000000001e-06 2024-09-04T17:47:13.928 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.928 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.928 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:13.928 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896045+0000", 2024-09-04T17:47:13.928 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073166300000000004 2024-09-04T17:47:13.928 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.928 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.928 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:13.928 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896094+0000", 2024-09-04T17:47:13.929 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9227000000000001e-05 2024-09-04T17:47:13.929 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.929 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.929 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:13.929 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896153+0000", 2024-09-04T17:47:13.929 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.9024000000000003e-05 2024-09-04T17:47:13.929 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.929 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:13.929 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.929 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.929 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.929 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60919 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:13.929 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822883+0000", 2024-09-04T17:47:13.929 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.076432772999999996, 2024-09-04T17:47:13.930 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073555613000000006, 2024-09-04T17:47:13.930 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:13.930 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:13.930 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:13.930 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:13.930 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:13.930 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:13.930 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60919 2024-09-04T17:47:13.930 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.930 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:13.930 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.930 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:13.930 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822883+0000", 2024-09-04T17:47:13.931 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.931 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.931 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.931 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:13.931 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822883+0000", 2024-09-04T17:47:13.931 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.931 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.931 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.931 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:13.931 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822883+0000", 2024-09-04T17:47:13.931 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2399999999999999e-07 2024-09-04T17:47:13.931 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.931 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.931 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:13.932 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822884+0000", 2024-09-04T17:47:13.932 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.184e-06 2024-09-04T17:47:13.932 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.932 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.932 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:13.932 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822887+0000", 2024-09-04T17:47:13.932 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.331e-06 2024-09-04T17:47:13.932 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.932 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.932 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:13.932 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822890+0000", 2024-09-04T17:47:13.932 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5689999999999999e-06 2024-09-04T17:47:13.932 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.932 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.933 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:13.933 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896180+0000", 2024-09-04T17:47:13.933 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073290496999999996 2024-09-04T17:47:13.933 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.933 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.933 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:13.933 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896228+0000", 2024-09-04T17:47:13.933 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7410000000000002e-05 2024-09-04T17:47:13.933 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.933 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.933 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:13.933 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896438+0000", 2024-09-04T17:47:13.933 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021039799999999999 2024-09-04T17:47:13.933 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.934 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:13.934 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.934 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.934 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.934 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60920 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 229376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:13.934 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822905+0000", 2024-09-04T17:47:13.934 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.076410586000000003, 2024-09-04T17:47:13.934 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073655074000000001, 2024-09-04T17:47:13.934 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:13.934 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:13.934 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:13.934 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:13.934 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:13.934 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:13.935 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60920 2024-09-04T17:47:13.935 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.935 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:13.935 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.935 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:13.935 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822905+0000", 2024-09-04T17:47:13.935 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.935 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.935 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.935 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:13.935 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822905+0000", 2024-09-04T17:47:13.935 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.935 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.935 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.936 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:13.936 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822905+0000", 2024-09-04T17:47:13.936 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.77e-07 2024-09-04T17:47:13.936 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.936 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.936 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:13.936 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822907+0000", 2024-09-04T17:47:13.936 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9520000000000001e-06 2024-09-04T17:47:13.936 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.936 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.936 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:13.936 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822912+0000", 2024-09-04T17:47:13.936 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.028e-06 2024-09-04T17:47:13.937 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.937 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.937 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:13.937 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822915+0000", 2024-09-04T17:47:13.937 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.066e-06 2024-09-04T17:47:13.937 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.937 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.937 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:13.937 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896465+0000", 2024-09-04T17:47:13.937 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073549613999999999 2024-09-04T17:47:13.937 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.937 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.937 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:13.937 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896507+0000", 2024-09-04T17:47:13.938 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2040999999999999e-05 2024-09-04T17:47:13.938 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.938 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.938 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:13.938 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896560+0000", 2024-09-04T17:47:13.938 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2995999999999999e-05 2024-09-04T17:47:13.938 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.938 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:13.938 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.938 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.938 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.938 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60921 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 262144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:13.938 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822920+0000", 2024-09-04T17:47:13.938 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.076395789000000006, 2024-09-04T17:47:13.939 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073904394999999998, 2024-09-04T17:47:13.939 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:13.939 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:13.939 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:13.939 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:13.939 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:13.939 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:13.939 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60921 2024-09-04T17:47:13.939 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.939 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:13.939 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.939 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:13.939 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822920+0000", 2024-09-04T17:47:13.939 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.940 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.940 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.940 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:13.940 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822920+0000", 2024-09-04T17:47:13.940 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.940 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.940 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.940 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:13.940 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822920+0000", 2024-09-04T17:47:13.940 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4999999999999999e-07 2024-09-04T17:47:13.940 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.940 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.940 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:13.940 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822921+0000", 2024-09-04T17:47:13.941 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.113e-06 2024-09-04T17:47:13.941 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.941 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.941 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:13.941 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822924+0000", 2024-09-04T17:47:13.941 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.354e-06 2024-09-04T17:47:13.941 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.941 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.941 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:13.941 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822927+0000", 2024-09-04T17:47:13.941 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6199999999999999e-06 2024-09-04T17:47:13.941 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.941 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.941 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:13.942 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896586+0000", 2024-09-04T17:47:13.942 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073658976000000001 2024-09-04T17:47:13.942 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.942 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.942 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:13.942 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896626+0000", 2024-09-04T17:47:13.942 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0484000000000001e-05 2024-09-04T17:47:13.942 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.942 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.942 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:13.942 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896824+0000", 2024-09-04T17:47:13.942 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000197598 2024-09-04T17:47:13.942 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.942 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:13.943 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.943 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.943 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.943 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60922 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:13.943 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822930+0000", 2024-09-04T17:47:13.943 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.076385214000000007, 2024-09-04T17:47:13.943 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074178872000000007, 2024-09-04T17:47:13.943 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:13.943 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:13.943 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:13.943 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:13.943 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:13.943 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:13.943 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60922 2024-09-04T17:47:13.944 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.944 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:13.944 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.944 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:13.944 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822930+0000", 2024-09-04T17:47:13.944 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.944 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.944 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.944 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:13.944 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822930+0000", 2024-09-04T17:47:13.944 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.944 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.944 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.945 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:13.945 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822930+0000", 2024-09-04T17:47:13.945 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.36e-07 2024-09-04T17:47:13.945 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.945 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.945 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:13.945 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822932+0000", 2024-09-04T17:47:13.945 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.567e-06 2024-09-04T17:47:13.945 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.946 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.946 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:13.946 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822938+0000", 2024-09-04T17:47:13.946 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4840000000000003e-06 2024-09-04T17:47:13.946 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.946 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.946 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:13.946 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822942+0000", 2024-09-04T17:47:13.946 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2039999999999999e-06 2024-09-04T17:47:13.946 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.946 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.946 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:13.946 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896850+0000", 2024-09-04T17:47:13.947 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073907941000000005 2024-09-04T17:47:13.947 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.947 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.947 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:13.947 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896891+0000", 2024-09-04T17:47:13.947 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0964999999999998e-05 2024-09-04T17:47:13.947 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.947 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.947 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:13.947 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897109+0000", 2024-09-04T17:47:13.947 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021847500000000001 2024-09-04T17:47:13.947 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.948 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:13.948 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.948 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.948 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.948 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60923 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 475136~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:13.948 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822949+0000", 2024-09-04T17:47:13.948 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.076366924000000003, 2024-09-04T17:47:13.948 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074426928000000003, 2024-09-04T17:47:13.948 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:13.948 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:13.948 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:13.948 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:13.948 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:13.949 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:13.949 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60923 2024-09-04T17:47:13.949 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.949 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:13.949 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.949 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:13.949 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822949+0000", 2024-09-04T17:47:13.949 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.949 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.949 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.949 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:13.949 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822949+0000", 2024-09-04T17:47:13.949 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.949 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.950 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.950 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:13.950 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822949+0000", 2024-09-04T17:47:13.950 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7399999999999999e-07 2024-09-04T17:47:13.950 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.950 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.950 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:13.950 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822951+0000", 2024-09-04T17:47:13.950 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1940000000000001e-06 2024-09-04T17:47:13.950 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.950 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.950 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:13.950 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822957+0000", 2024-09-04T17:47:13.951 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.0650000000000004e-06 2024-09-04T17:47:13.951 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.951 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.951 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:13.951 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822960+0000", 2024-09-04T17:47:13.951 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.8009999999999999e-06 2024-09-04T17:47:13.951 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.951 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.951 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:13.951 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897138+0000", 2024-09-04T17:47:13.951 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074177990999999999 2024-09-04T17:47:13.951 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.951 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.951 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:13.952 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897179+0000", 2024-09-04T17:47:13.952 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.1057999999999999e-05 2024-09-04T17:47:13.952 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.952 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.952 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:13.952 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897375+0000", 2024-09-04T17:47:13.952 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019644500000000001 2024-09-04T17:47:13.952 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.952 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:13.952 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.952 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.952 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.952 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60924 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:13.953 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822964+0000", 2024-09-04T17:47:13.953 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.076351739000000002, 2024-09-04T17:47:13.953 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074531902999999997, 2024-09-04T17:47:13.953 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:13.953 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:13.953 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:13.953 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:13.953 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:13.953 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:13.953 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60924 2024-09-04T17:47:13.953 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.953 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:13.953 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.953 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:13.954 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822964+0000", 2024-09-04T17:47:13.954 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.954 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.954 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.954 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:13.954 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822964+0000", 2024-09-04T17:47:13.954 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.954 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.954 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.954 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:13.954 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822964+0000", 2024-09-04T17:47:13.954 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4999999999999999e-07 2024-09-04T17:47:13.954 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.954 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.955 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:13.955 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822966+0000", 2024-09-04T17:47:13.955 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.917e-06 2024-09-04T17:47:13.955 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.955 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.955 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:13.955 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822969+0000", 2024-09-04T17:47:13.955 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.315e-06 2024-09-04T17:47:13.955 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.955 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.955 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:13.955 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822972+0000", 2024-09-04T17:47:13.955 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.599e-06 2024-09-04T17:47:13.955 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.956 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.956 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:13.956 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897398+0000", 2024-09-04T17:47:13.956 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074426168000000001 2024-09-04T17:47:13.956 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.956 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.956 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:13.956 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897441+0000", 2024-09-04T17:47:13.956 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.3158000000000003e-05 2024-09-04T17:47:13.956 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.956 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.956 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:13.956 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897496+0000", 2024-09-04T17:47:13.956 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4496000000000001e-05 2024-09-04T17:47:13.957 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.957 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:13.957 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.957 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.957 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.957 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60925 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 720896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:13.957 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822975+0000", 2024-09-04T17:47:13.957 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.076340303999999998, 2024-09-04T17:47:13.957 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074635272000000003, 2024-09-04T17:47:13.957 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:13.957 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:13.957 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:13.957 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:13.958 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:13.958 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:13.958 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60925 2024-09-04T17:47:13.958 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.958 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:13.958 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.958 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:13.958 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822975+0000", 2024-09-04T17:47:13.958 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.958 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.958 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.958 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:13.958 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822975+0000", 2024-09-04T17:47:13.958 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.958 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.959 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.959 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:13.959 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822975+0000", 2024-09-04T17:47:13.959 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.34e-07 2024-09-04T17:47:13.959 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.959 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.959 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:13.959 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822976+0000", 2024-09-04T17:47:13.959 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0359999999999999e-06 2024-09-04T17:47:13.959 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.959 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.959 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:13.959 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822980+0000", 2024-09-04T17:47:13.960 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.242e-06 2024-09-04T17:47:13.960 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.960 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.960 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:13.960 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822983+0000", 2024-09-04T17:47:13.960 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.8830000000000002e-06 2024-09-04T17:47:13.960 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.960 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.960 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:13.960 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897519+0000", 2024-09-04T17:47:13.960 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074536366000000007 2024-09-04T17:47:13.960 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.960 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.960 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:13.961 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897560+0000", 2024-09-04T17:47:13.961 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0677999999999999e-05 2024-09-04T17:47:13.961 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.961 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.961 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:13.961 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897610+0000", 2024-09-04T17:47:13.961 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0832999999999999e-05 2024-09-04T17:47:13.961 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.961 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:13.961 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.961 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.961 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.961 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60926 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 860160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:13.961 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822986+0000", 2024-09-04T17:47:13.962 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.076329705999999997, 2024-09-04T17:47:13.962 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074736413000000002, 2024-09-04T17:47:13.962 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:13.962 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:13.962 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:13.962 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:13.962 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:13.962 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:13.962 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60926 2024-09-04T17:47:13.962 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.962 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:13.962 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.962 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:13.962 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822986+0000", 2024-09-04T17:47:13.963 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.963 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.963 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.963 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:13.963 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822986+0000", 2024-09-04T17:47:13.963 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.963 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.963 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.963 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:13.963 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822986+0000", 2024-09-04T17:47:13.963 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5600000000000002e-07 2024-09-04T17:47:13.963 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.963 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.964 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:13.964 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822987+0000", 2024-09-04T17:47:13.964 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.082e-06 2024-09-04T17:47:13.964 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.964 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.964 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:13.964 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822990+0000", 2024-09-04T17:47:13.964 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.2219999999999998e-06 2024-09-04T17:47:13.964 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.964 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.964 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:13.964 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822994+0000", 2024-09-04T17:47:13.964 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.19e-06 2024-09-04T17:47:13.965 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.965 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.965 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:13.965 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897632+0000", 2024-09-04T17:47:13.965 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074638205999999999 2024-09-04T17:47:13.965 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.965 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.965 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:13.965 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897672+0000", 2024-09-04T17:47:13.965 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.9873999999999998e-05 2024-09-04T17:47:13.965 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.965 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.965 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:13.966 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897722+0000", 2024-09-04T17:47:13.966 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0583e-05 2024-09-04T17:47:13.966 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.966 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:13.966 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.966 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.966 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.966 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60927 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 966656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:13.966 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823000+0000", 2024-09-04T17:47:13.966 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.076315523999999996, 2024-09-04T17:47:13.966 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074834036000000007, 2024-09-04T17:47:13.966 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:13.966 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:13.966 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:13.967 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:13.967 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:13.967 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:13.967 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60927 2024-09-04T17:47:13.967 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.967 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:13.967 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.967 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:13.967 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823000+0000", 2024-09-04T17:47:13.967 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.967 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.967 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.967 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:13.967 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823000+0000", 2024-09-04T17:47:13.968 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.968 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.968 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.968 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:13.968 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823000+0000", 2024-09-04T17:47:13.968 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4099999999999999e-07 2024-09-04T17:47:13.968 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.968 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.968 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:13.968 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823003+0000", 2024-09-04T17:47:13.969 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0810000000000001e-06 2024-09-04T17:47:13.969 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.969 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.969 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:13.969 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823011+0000", 2024-09-04T17:47:13.969 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7090000000000008e-06 2024-09-04T17:47:13.969 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.969 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.969 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:13.969 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823017+0000", 2024-09-04T17:47:13.969 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.3890000000000004e-06 2024-09-04T17:47:13.969 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.969 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.970 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:13.970 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897745+0000", 2024-09-04T17:47:13.970 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074728252999999994 2024-09-04T17:47:13.970 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.970 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.970 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:13.970 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897785+0000", 2024-09-04T17:47:13.970 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.9830999999999999e-05 2024-09-04T17:47:13.970 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.970 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.970 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:13.970 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897834+0000", 2024-09-04T17:47:13.970 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9231999999999999e-05 2024-09-04T17:47:13.971 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.971 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:13.971 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.971 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.971 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.971 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60928 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:13.971 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823024+0000", 2024-09-04T17:47:13.971 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.076291593000000005, 2024-09-04T17:47:13.971 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075067479000000006, 2024-09-04T17:47:13.971 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:13.971 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:13.971 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:13.971 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:13.971 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:13.971 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:13.972 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60928 2024-09-04T17:47:13.972 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.972 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:13.972 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.972 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:13.972 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823024+0000", 2024-09-04T17:47:13.972 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.972 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.972 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.972 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:13.972 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823024+0000", 2024-09-04T17:47:13.972 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.972 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.973 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.973 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:13.973 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823024+0000", 2024-09-04T17:47:13.973 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6499999999999999e-07 2024-09-04T17:47:13.973 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.973 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.973 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:13.973 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823027+0000", 2024-09-04T17:47:13.973 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.3099999999999999e-06 2024-09-04T17:47:13.973 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.973 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.973 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:13.973 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823033+0000", 2024-09-04T17:47:13.973 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7429999999999996e-06 2024-09-04T17:47:13.974 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.974 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.974 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:13.974 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823038+0000", 2024-09-04T17:47:13.974 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.561e-06 2024-09-04T17:47:13.974 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.974 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.974 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:13.974 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897855+0000", 2024-09-04T17:47:13.974 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074816740000000007 2024-09-04T17:47:13.974 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.974 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.974 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:13.974 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897894+0000", 2024-09-04T17:47:13.975 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.9085999999999998e-05 2024-09-04T17:47:13.975 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.975 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.975 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:13.975 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898091+0000", 2024-09-04T17:47:13.975 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000197574 2024-09-04T17:47:13.975 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.975 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:13.975 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.975 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.975 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.975 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60929 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1212416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:13.975 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823042+0000", 2024-09-04T17:47:13.976 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.076273150999999997, 2024-09-04T17:47:13.976 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075301251, 2024-09-04T17:47:13.976 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:13.976 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:13.976 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:13.976 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:13.976 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:13.976 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:13.976 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60929 2024-09-04T17:47:13.976 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.976 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:13.976 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.976 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:13.976 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823042+0000", 2024-09-04T17:47:13.977 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.977 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.977 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.977 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:13.977 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823042+0000", 2024-09-04T17:47:13.977 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.977 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.977 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.977 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:13.977 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823043+0000", 2024-09-04T17:47:13.977 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.1100000000000002e-07 2024-09-04T17:47:13.977 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.977 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.978 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:13.978 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823045+0000", 2024-09-04T17:47:13.978 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.933e-06 2024-09-04T17:47:13.978 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.978 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.978 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:13.978 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823051+0000", 2024-09-04T17:47:13.978 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.9569999999999999e-06 2024-09-04T17:47:13.978 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.978 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.978 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:13.978 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823055+0000", 2024-09-04T17:47:13.978 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4479999999999996e-06 2024-09-04T17:47:13.978 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.979 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.979 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:13.979 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898117+0000", 2024-09-04T17:47:13.979 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075062344000000003 2024-09-04T17:47:13.979 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.979 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.979 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:13.979 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898157+0000", 2024-09-04T17:47:13.979 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0095999999999997e-05 2024-09-04T17:47:13.979 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.979 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.979 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:13.979 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898344+0000", 2024-09-04T17:47:13.979 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000186162 2024-09-04T17:47:13.980 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.980 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:13.980 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.980 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.980 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.980 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60930 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1351680~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:13.980 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823061+0000", 2024-09-04T17:47:13.980 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.076254271999999998, 2024-09-04T17:47:13.980 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075545507999999997, 2024-09-04T17:47:13.980 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:13.980 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:13.980 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:13.980 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:13.980 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:13.981 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:13.981 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60930 2024-09-04T17:47:13.981 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.981 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:13.981 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.981 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:13.981 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823061+0000", 2024-09-04T17:47:13.981 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.981 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.981 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.981 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:13.981 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823061+0000", 2024-09-04T17:47:13.981 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.981 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.982 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.982 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:13.982 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823378+0000", 2024-09-04T17:47:13.982 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00031631299999999998 2024-09-04T17:47:13.982 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.982 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.982 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:13.982 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823381+0000", 2024-09-04T17:47:13.982 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4020000000000002e-06 2024-09-04T17:47:13.982 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.982 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.982 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:13.982 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823389+0000", 2024-09-04T17:47:13.982 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.1429999999999998e-06 2024-09-04T17:47:13.983 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.983 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.983 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:13.983 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823395+0000", 2024-09-04T17:47:13.983 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.0059999999999998e-06 2024-09-04T17:47:13.983 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.983 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.983 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:13.983 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898368+0000", 2024-09-04T17:47:13.983 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074973032999999994 2024-09-04T17:47:13.983 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.983 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.983 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:13.983 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898410+0000", 2024-09-04T17:47:13.984 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2243000000000002e-05 2024-09-04T17:47:13.984 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.984 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.984 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:13.984 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898607+0000", 2024-09-04T17:47:13.984 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019636799999999999 2024-09-04T17:47:13.984 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.984 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:13.984 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.984 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.984 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.984 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60931 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1458176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:13.984 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823407+0000", 2024-09-04T17:47:13.984 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.075908197999999996, 2024-09-04T17:47:13.985 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075460334000000004, 2024-09-04T17:47:13.985 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:13.985 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:13.985 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:13.985 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:13.985 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:13.985 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:13.985 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60931 2024-09-04T17:47:13.985 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.985 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:13.985 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.985 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:13.985 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823407+0000", 2024-09-04T17:47:13.985 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.986 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.986 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.986 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:13.986 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823407+0000", 2024-09-04T17:47:13.986 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.986 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.986 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.986 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:13.986 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823408+0000", 2024-09-04T17:47:13.986 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9999999999999998e-07 2024-09-04T17:47:13.986 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.986 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.986 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:13.986 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823415+0000", 2024-09-04T17:47:13.987 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.5510000000000002e-06 2024-09-04T17:47:13.987 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.987 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.987 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:13.987 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823423+0000", 2024-09-04T17:47:13.987 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2760000000000003e-06 2024-09-04T17:47:13.987 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.987 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.987 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:13.987 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823426+0000", 2024-09-04T17:47:13.987 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.2839999999999998e-06 2024-09-04T17:47:13.987 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.987 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.988 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:13.988 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898631+0000", 2024-09-04T17:47:13.988 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075205045999999998 2024-09-04T17:47:13.988 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.988 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.988 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:13.988 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898672+0000", 2024-09-04T17:47:13.988 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0614000000000001e-05 2024-09-04T17:47:13.988 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.988 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.988 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:13.988 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898868+0000", 2024-09-04T17:47:13.988 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019606300000000001 2024-09-04T17:47:13.989 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.989 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:13.989 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.989 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.989 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.989 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60932 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1564672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:13.989 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823434+0000", 2024-09-04T17:47:13.989 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.075881899000000003, 2024-09-04T17:47:13.989 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075693393999999997, 2024-09-04T17:47:13.989 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:13.989 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:13.989 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:13.989 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:13.989 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:13.990 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:13.990 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60932 2024-09-04T17:47:13.990 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.990 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:13.990 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.990 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:13.990 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823434+0000", 2024-09-04T17:47:13.990 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.990 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.990 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.990 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:13.990 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823434+0000", 2024-09-04T17:47:13.990 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:13.990 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.991 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.991 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:13.991 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.830290+0000", 2024-09-04T17:47:13.991 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0068563039999999997 2024-09-04T17:47:13.991 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.991 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.991 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:13.991 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.830297+0000", 2024-09-04T17:47:13.991 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4760000000000001e-06 2024-09-04T17:47:13.991 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.991 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.991 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:13.991 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.830308+0000", 2024-09-04T17:47:13.991 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0927999999999999e-05 2024-09-04T17:47:13.992 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.992 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.992 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:13.992 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.830317+0000", 2024-09-04T17:47:13.992 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7280000000000001e-06 2024-09-04T17:47:13.992 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.992 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.992 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:13.992 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898892+0000", 2024-09-04T17:47:13.992 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.068574722000000005 2024-09-04T17:47:13.992 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.992 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.992 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:13.993 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898934+0000", 2024-09-04T17:47:13.993 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2249000000000002e-05 2024-09-04T17:47:13.993 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:13.993 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:13.993 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:13.993 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.899127+0000", 2024-09-04T17:47:13.993 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019298699999999999 2024-09-04T17:47:13.993 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.993 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:13.993 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.993 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:13.993 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:13.993 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:13.994 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:47:13.995 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:47:13.995 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-04T17:47:13.996 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-04T17:47:13.996 INFO:teuthology.orchestra.run.smithi092.stdout:3739 read 0xb23614a thru 0xb2384be (0x2375 bytes) 2024-09-04T17:47:13.996 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:13.997 INFO:teuthology.orchestra.run.smithi092.stdout:3742 read 0x1949100 thru 0x1950347 (0x7248 bytes) 2024-09-04T17:47:13.997 INFO:teuthology.orchestra.run.smithi092.stdout:3745 punch from 0x2c4fd74 to 0x2c57629, (0x78b5 bytes) 2024-09-04T17:47:13.997 INFO:teuthology.orchestra.run.smithi092.stdout:3746 writesame 0xb84d80b thru 0xb851d3e data_size 0x2b(0x4534 bytes) 2024-09-04T17:47:13.997 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-04T17:47:13.997 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-04T17:47:13.998 INFO:teuthology.orchestra.run.smithi092.stdout:3747 writesame 0x4abdeef thru 0x4ac2c77 data_size 0x1(0x4d89 bytes) 2024-09-04T17:47:13.998 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:47:13.998 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:47:13.998 INFO:teuthology.orchestra.run.smithi092.stdout:3748 trunc from 0xddae96b to 0x2c93766 2024-09-04T17:47:13.999 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:47:13.999 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:47:13.999 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-04T17:47:13.999 INFO:teuthology.orchestra.run.smithi092.stdout:3749 punch from 0x289b029 to 0x289fbd2, (0x4ba9 bytes) 2024-09-04T17:47:14.000 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:47:14.000 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:47:14.000 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:13.990+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:14.001 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-04T17:47:14.001 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-04T17:47:14.001 INFO:teuthology.orchestra.run.smithi092.stdout:3751 write 0xd250a85 thru 0xd259839 (0x8db5 bytes) 2024-09-04T17:47:14.009 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:14.009 INFO:teuthology.orchestra.run.smithi092.stdout:3753 read 0x34fbf8b thru 0x34fc5e6 (0x65c bytes) 2024-09-04T17:47:14.009 INFO:teuthology.orchestra.run.smithi092.stdout:3754 read 0x65c80e5 thru 0x65d4e1a (0xcd36 bytes) 2024-09-04T17:47:14.012 INFO:teuthology.orchestra.run.smithi092.stdout:3755 read 0x59df887 thru 0x59e0330 (0xaaa bytes) 2024-09-04T17:47:14.012 INFO:teuthology.orchestra.run.smithi092.stdout:3757 writesame 0x27bc103 thru 0x27ca70b data_size 0x1(0xe609 bytes) 2024-09-04T17:47:14.021 INFO:teuthology.orchestra.run.smithi092.stdout:3758 write 0xa3b2047 thru 0xa3b5012 (0x2fcc bytes) 2024-09-04T17:47:14.027 INFO:teuthology.orchestra.run.smithi092.stdout:3759 read 0x10ed0e thru 0x118d09 (0x9ffc bytes) 2024-09-04T17:47:14.029 INFO:teuthology.orchestra.run.smithi092.stdout:3760 read 0x3d473ef thru 0x3d51cdd (0xa8ef bytes) 2024-09-04T17:47:14.029 INFO:teuthology.orchestra.run.smithi092.stdout:3763 flatten 2024-09-04T17:47:14.095 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:14.093+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:14.163 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:14.163 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:14.182 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-04T17:47:14.197 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:14.196+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:14.250 INFO:teuthology.orchestra.run.smithi092.stdout:3764 writesame 0x4208bb1 thru 0x4218a66 data_size 0x1(0xfeb6 bytes) 2024-09-04T17:47:14.283 INFO:teuthology.orchestra.run.smithi092.stdout:3765 read 0x5cd35a7 thru 0x5cdaf77 (0x79d1 bytes) 2024-09-04T17:47:14.283 INFO:teuthology.orchestra.run.smithi092.stdout:3766 clone 30 order 24 su 4194304 sc 2 2024-09-04T17:47:14.298 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:14.294+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:14.324 INFO:teuthology.orchestra.run.smithi092.stdout:3.9 2024-09-04T17:47:14.325 INFO:teuthology.orchestra.run.smithi092.stdout:3.1f 2024-09-04T17:47:14.325 INFO:teuthology.orchestra.run.smithi092.stdout:3.1e 2024-09-04T17:47:14.325 INFO:teuthology.orchestra.run.smithi092.stdout:2.7 2024-09-04T17:47:14.325 INFO:teuthology.orchestra.run.smithi092.stdout:3.5 2024-09-04T17:47:14.325 INFO:teuthology.orchestra.run.smithi092.stdout:3.18 2024-09-04T17:47:14.325 INFO:teuthology.orchestra.run.smithi092.stdout:2.3 2024-09-04T17:47:14.325 INFO:teuthology.orchestra.run.smithi092.stdout:3.0 2024-09-04T17:47:14.325 INFO:teuthology.orchestra.run.smithi092.stdout:2.1 2024-09-04T17:47:14.340 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:14.340 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:14.352 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-04T17:47:14.398 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:14.394+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:14.499 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:14.496+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:14.514 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:14.515 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:14.528 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-04T17:47:14.600 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:14.598+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:14.630 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-6 --journal-path /var/lib/ceph/osd/ceph-6/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.3.1e.6 2024-09-04T17:47:14.648 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:14.648 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:14.662 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-04T17:47:14.704 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:14.702+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:14.804 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:14.804+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:14.828 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:14.829 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:14.843 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-04T17:47:14.905 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:14.904+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:14.973 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:14.973 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:14.990 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-04T17:47:15.036 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:15.034+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:15.039 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #28, image image_client.0-clone28 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent29 2024-09-04T17:47:15.096 INFO:teuthology.orchestra.run.smithi092.stdout:3768 read 0x9a0175c thru 0x9a0af15 (0x97ba bytes) 2024-09-04T17:47:15.097 INFO:teuthology.orchestra.run.smithi092.stdout:3769 punch from 0x25aec3e to 0x25bb856, (0xcc18 bytes) 2024-09-04T17:47:15.105 INFO:teuthology.orchestra.run.smithi092.stdout:3770 writesame 0xbe9fb64 thru 0xbea358f data_size 0x44(0x3a2c bytes) 2024-09-04T17:47:15.107 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:47:15.104+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:47:15.113 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:15.113 INFO:teuthology.orchestra.run.smithi092.stdout:3772 punch from 0x4fe64c7 to 0x4fea1ec, (0x3d25 bytes) 2024-09-04T17:47:15.113 INFO:teuthology.orchestra.run.smithi092.stdout:3773 write 0x3fbf847 thru 0x3fc76a1 (0x7e5b bytes) 2024-09-04T17:47:15.125 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:15.126 INFO:teuthology.orchestra.run.smithi092.stdout:3776 read 0x6bf48ac thru 0x6c0225e (0xd9b3 bytes) 2024-09-04T17:47:15.126 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:15.126 INFO:teuthology.orchestra.run.smithi092.stdout:3780 write 0x4b5ab96 thru 0x4b68f46 (0xe3b1 bytes) 2024-09-04T17:47:15.126 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:15.126 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:15.136 INFO:teuthology.orchestra.run.smithi092.stdout:3781 write 0x4e25cc1 thru 0x4e27ddc (0x211c bytes) 2024-09-04T17:47:15.141 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-04T17:47:15.145 INFO:teuthology.orchestra.run.smithi092.stdout:3783 write 0x4f8ca5c thru 0x4f941c5 (0x776a bytes) 2024-09-04T17:47:15.147 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:15.147 INFO:teuthology.orchestra.run.smithi092.stdout:3786 writesame 0x7c2c2bc thru 0x7c2f1cc data_size 0x1(0x2f11 bytes) 2024-09-04T17:47:15.168 INFO:teuthology.orchestra.run.smithi092.stdout:3787 write 0x536660a thru 0x536770f (0x1106 bytes) 2024-09-04T17:47:15.171 INFO:teuthology.orchestra.run.smithi092.stdout:3788 write 0xa2f5ee2 thru 0xa30529a (0xf3b9 bytes) 2024-09-04T17:47:15.180 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:15.180 INFO:teuthology.orchestra.run.smithi092.stdout:3791 trunc from 0xd25983a to 0xd3a55c1 2024-09-04T17:47:15.193 INFO:teuthology.orchestra.run.smithi092.stdout:3792 read 0x7a58a7f thru 0x7a67911 (0xee93 bytes) 2024-09-04T17:47:15.193 INFO:teuthology.orchestra.run.smithi092.stdout:3793 write 0x122d111 thru 0x123cb19 (0xfa09 bytes) 2024-09-04T17:47:15.210 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:15.206+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:15.270 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:15.271 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:15.286 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-04T17:47:15.327 INFO:teuthology.orchestra.run.smithi092.stdout:3794 write 0x15f42f8 thru 0x1601884 (0xd58d bytes) 2024-09-04T17:47:15.328 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:15.326+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:15.410 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:15.406+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:15.413 INFO:teuthology.orchestra.run.smithi092.stdout:3795 read 0xf6057f thru 0xf68b66 (0x85e8 bytes) 2024-09-04T17:47:15.427 INFO:teuthology.orchestra.run.smithi092.stdout:3796 trunc from 0xd3a55c1 to 0x3786256 2024-09-04T17:47:15.459 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:15.473 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:15.473 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:15.477 INFO:teuthology.orchestra.run.smithi092.stdout:3799 writesame 0x6978285 thru 0x6980952 data_size 0x91(0x86ce bytes) 2024-09-04T17:47:15.477 INFO:teuthology.orchestra.run.smithi092.stdout:3800 punch from 0x27a0e01 to 0x27aeba7, (0xdda6 bytes) 2024-09-04T17:47:15.477 INFO:teuthology.orchestra.run.smithi092.stdout:3801 punch from 0x686de37 to 0x687b621, (0xd7ea bytes) 2024-09-04T17:47:15.490 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-04T17:47:15.497 INFO:teuthology.orchestra.run.smithi092.stdout:3802 write 0xb21df4b thru 0xb22bfcd (0xe083 bytes) 2024-09-04T17:47:15.509 INFO:teuthology.orchestra.run.smithi092.stdout:3803 trunc from 0xb22bfce to 0x6894c9c 2024-09-04T17:47:15.524 INFO:teuthology.orchestra.run.smithi092.stdout:3807 write 0x6376a33 thru 0x638599e (0xef6c bytes) 2024-09-04T17:47:15.528 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:15.536 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:15.536+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:15.550 INFO:teuthology.orchestra.run.smithi092.stdout:3811 write 0xc08fee0 thru 0xc0926d8 (0x27f9 bytes) 2024-09-04T17:47:15.555 INFO:teuthology.orchestra.run.smithi092.stdout:3812 read 0x5fbb2f6 thru 0x5fc24e1 (0x71ec bytes) 2024-09-04T17:47:15.570 INFO:teuthology.orchestra.run.smithi092.stdout:3813 write 0xda8d778 thru 0xda97877 (0xa100 bytes) 2024-09-04T17:47:15.574 INFO:teuthology.orchestra.run.smithi092.stdout:3814 writesame 0x99aa789 thru 0x99b55d0 data_size 0x68(0xae48 bytes) 2024-09-04T17:47:15.579 INFO:teuthology.orchestra.run.smithi092.stdout:3816 punch from 0x47451fc to 0x4750a91, (0xb895 bytes) 2024-09-04T17:47:15.579 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:15.579 INFO:teuthology.orchestra.run.smithi092.stdout:3818 read 0x691ef6 thru 0x69b4a4 (0x95af bytes) 2024-09-04T17:47:15.581 INFO:teuthology.orchestra.run.smithi092.stdout:3819 writesame 0xa8a0a38 thru 0xa8a626d data_size 0x7(0x5836 bytes) 2024-09-04T17:47:15.600 INFO:teuthology.orchestra.run.smithi092.stdout:3821 trunc from 0xda97878 to 0x935469f 2024-09-04T17:47:15.611 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:15.608+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:15.620 INFO:teuthology.orchestra.run.smithi092.stdout:3822 write 0x6b4ccf8 thru 0x6b4fa6b (0x2d74 bytes) 2024-09-04T17:47:15.623 INFO:teuthology.orchestra.run.smithi092.stdout:3824 write 0x458c751 thru 0x45979f8 (0xb2a8 bytes) 2024-09-04T17:47:15.625 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:15.625 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:15.627 INFO:teuthology.orchestra.run.smithi092.stdout:3825 write 0x16de9b7 thru 0x16e536f (0x69b9 bytes) 2024-09-04T17:47:15.631 INFO:teuthology.orchestra.run.smithi092.stdout:3826 write 0x2acc625 thru 0x2ad2c42 (0x661e bytes) 2024-09-04T17:47:15.634 INFO:teuthology.orchestra.run.smithi092.stdout:3828 write 0x49f4905 thru 0x49fad8e (0x648a bytes) 2024-09-04T17:47:15.639 INFO:teuthology.orchestra.run.smithi092.stdout:3829 read 0x8b2c10a thru 0x8b3570e (0x9605 bytes) 2024-09-04T17:47:15.644 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-04T17:47:15.647 INFO:teuthology.orchestra.run.smithi092.stdout:3830 trunc from 0x935469f to 0x6711bf8 2024-09-04T17:47:15.674 INFO:teuthology.orchestra.run.smithi092.stdout:3831 trunc from 0x6711bf8 to 0x943a4cd 2024-09-04T17:47:15.685 INFO:teuthology.orchestra.run.smithi092.stdout:3832 punch from 0x85576b4 to 0x856121a, (0x9b66 bytes) 2024-09-04T17:47:15.707 INFO:teuthology.orchestra.run.smithi092.stdout:3833 writesame 0xe007e84 thru 0xe00c1b9 data_size 0x7(0x4336 bytes) 2024-09-04T17:47:15.708 INFO:teuthology.orchestra.run.smithi092.stdout:3834 read 0xdfb0f99 thru 0xdfb6360 (0x53c8 bytes) 2024-09-04T17:47:15.708 INFO:teuthology.orchestra.run.smithi092.stdout:3835 punch from 0x4b17ba9 to 0x4b22ffa, (0xb451 bytes) 2024-09-04T17:47:15.708 INFO:teuthology.orchestra.run.smithi092.stdout:3836 write 0xd63cfb1 thru 0xd640a88 (0x3ad8 bytes) 2024-09-04T17:47:15.711 INFO:teuthology.orchestra.run.smithi092.stdout:3837 read 0x6a2277 thru 0x6a3f48 (0x1cd2 bytes) 2024-09-04T17:47:15.713 INFO:teuthology.orchestra.run.smithi092.stdout:3839 write 0xaa047c thru 0xaa1877 (0x13fc bytes) 2024-09-04T17:47:15.715 INFO:teuthology.orchestra.run.smithi092.stdout:3840 punch from 0x8b6545a to 0x8b6a8c5, (0x546b bytes) 2024-09-04T17:47:15.715 INFO:teuthology.orchestra.run.smithi092.stdout:3841 read 0x88075b5 thru 0x8807e43 (0x88f bytes) 2024-09-04T17:47:15.716 INFO:teuthology.orchestra.run.smithi092.stdout:3842 read 0x424057b thru 0x424a398 (0x9e1e bytes) 2024-09-04T17:47:15.716 INFO:teuthology.orchestra.run.smithi092.stdout:3843 read 0x32d8efb thru 0x32e84f8 (0xf5fe bytes) 2024-09-04T17:47:15.718 INFO:teuthology.orchestra.run.smithi092.stdout:3844 writesame 0x2b40a2b thru 0x2b50347 data_size 0x1(0xf91d bytes) 2024-09-04T17:47:15.731 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-04T17:47:15.736 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:15.736+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:15.796 INFO:teuthology.orchestra.run.smithi092.stdout:3845 read 0xc8bbae4 thru 0xc8c5ed2 (0xa3ef bytes) 2024-09-04T17:47:15.812 INFO:teuthology.orchestra.run.smithi092.stdout:3846 trunc from 0xe00c1ba to 0x74f3a6b 2024-09-04T17:47:15.812 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:15.810+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:15.813 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:15.813 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:15.831 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-04T17:47:15.856 INFO:teuthology.orchestra.run.smithi092.stdout:3847 writesame 0xecabe06 thru 0xecb6a11 data_size 0x9a(0xac0c bytes) 2024-09-04T17:47:15.856 INFO:teuthology.orchestra.run.smithi092.stdout:3848 punch from 0x4556e35 to 0x4558da5, (0x1f70 bytes) 2024-09-04T17:47:15.856 INFO:teuthology.orchestra.run.smithi092.stdout:3849 read 0x5bc1c9 thru 0x5bc2cc (0x104 bytes) 2024-09-04T17:47:15.857 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:15.857 INFO:teuthology.orchestra.run.smithi092.stdout:3851 write 0xa9ac50b thru 0xa9b84ca (0xbfc0 bytes) 2024-09-04T17:47:15.860 INFO:teuthology.orchestra.run.smithi092.stdout:3852 read 0x273e835 thru 0x2749632 (0xadfe bytes) 2024-09-04T17:47:15.882 INFO:teuthology.orchestra.run.smithi092.stdout:3853 trunc from 0xecb6a12 to 0x4c9fdc 2024-09-04T17:47:15.909 INFO:teuthology.orchestra.run.smithi092.stdout:3854 write 0x1cb0a35 thru 0x1cb716a (0x6736 bytes) 2024-09-04T17:47:15.910 INFO:teuthology.orchestra.run.smithi092.stdout:3855 read 0x955e1f thru 0x961c62 (0xbe44 bytes) 2024-09-04T17:47:15.913 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:15.910+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:15.941 INFO:teuthology.orchestra.run.smithi092.stdout:3856 write 0xd5e27f3 thru 0xd5ef235 (0xca43 bytes) 2024-09-04T17:47:15.945 INFO:teuthology.orchestra.run.smithi092.stdout:3858 writesame 0x8d50caa thru 0x8d55810 data_size 0x1(0x4b67 bytes) 2024-09-04T17:47:15.966 INFO:teuthology.orchestra.run.smithi092.stdout:3859 write 0x9d232dc thru 0x9d2d4b2 (0xa1d7 bytes) 2024-09-04T17:47:15.983 INFO:teuthology.orchestra.run.smithi092.stdout:3860 trunc from 0xd5ef236 to 0x5d80dc9 2024-09-04T17:47:15.997 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:15.997 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:16.002 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-04T17:47:16.002 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:16.011 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-04T17:47:16.017 INFO:teuthology.orchestra.run.smithi092.stdout:3862 write 0x9482922 thru 0x948a07f (0x775e bytes) 2024-09-04T17:47:16.017 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:16.014+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:16.022 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:16.022 INFO:teuthology.orchestra.run.smithi092.stdout:3865 punch from 0xd5aa0e to 0xd66df9, (0xc3eb bytes) 2024-09-04T17:47:16.022 INFO:teuthology.orchestra.run.smithi092.stdout:3867 read 0x3d46be8 thru 0x3d51823 (0xac3c bytes) 2024-09-04T17:47:16.025 INFO:teuthology.orchestra.run.smithi092.stdout:3868 trunc from 0x948a080 to 0x8160d5a 2024-09-04T17:47:16.039 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:16.039 INFO:teuthology.orchestra.run.smithi092.stdout:3872 writesame 0x68d1c2a thru 0x68d7e00 data_size 0x45(0x61d7 bytes) 2024-09-04T17:47:16.044 INFO:teuthology.orchestra.run.smithi092.stdout:3874 punch from 0x29b7606 to 0x29c27a0, (0xb19a bytes) 2024-09-04T17:47:16.044 INFO:teuthology.orchestra.run.smithi092.stdout:3875 read 0x71b5d8c thru 0x71bb13b (0x53b0 bytes) 2024-09-04T17:47:16.055 INFO:teuthology.orchestra.run.smithi092.stdout:3876 write 0xa3ecae4 thru 0xa3f0289 (0x37a6 bytes) 2024-09-04T17:47:16.062 INFO:teuthology.orchestra.run.smithi092.stdout:3877 trunc from 0xa3f028a to 0x7c07c93 2024-09-04T17:47:16.074 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:16.074 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:16.074 INFO:teuthology.orchestra.run.smithi092.stdout:3881 read 0x2a92e07 thru 0x2a9c4b1 (0x96ab bytes) 2024-09-04T17:47:16.074 INFO:teuthology.orchestra.run.smithi092.stdout:3883 write 0x4f2b2a1 thru 0x4f2e572 (0x32d2 bytes) 2024-09-04T17:47:16.087 INFO:teuthology.orchestra.run.smithi092.stdout:3884 writesame 0x9d85bde thru 0x9d94bde data_size 0x1(0xf001 bytes) 2024-09-04T17:47:16.087 INFO:teuthology.orchestra.run.smithi092.stdout:3885 read 0x367bde8 thru 0x3686627 (0xa840 bytes) 2024-09-04T17:47:16.087 INFO:teuthology.orchestra.run.smithi092.stdout:3886 punch from 0x875a952 to 0x87665bb, (0xbc69 bytes) 2024-09-04T17:47:16.087 INFO:teuthology.orchestra.run.smithi092.stdout:3888 punch from 0x75c9b14 to 0x75cb320, (0x180c bytes) 2024-09-04T17:47:16.088 INFO:teuthology.orchestra.run.smithi092.stdout:3889 write 0x3c4cf74 thru 0x3c50422 (0x34af bytes) 2024-09-04T17:47:16.091 INFO:teuthology.orchestra.run.smithi092.stdout:3891 read 0x6c85c0d thru 0x6c8e392 (0x8786 bytes) 2024-09-04T17:47:16.091 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:16.091 INFO:teuthology.orchestra.run.smithi092.stdout:3895 read 0x37598e1 thru 0x3768e63 (0xf583 bytes) 2024-09-04T17:47:16.091 INFO:teuthology.orchestra.run.smithi092.stdout:3897 punch from 0x1336b95 to 0x133fc77, (0x90e2 bytes) 2024-09-04T17:47:16.092 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:16.092 INFO:teuthology.orchestra.run.smithi092.stdout:3900 writesame 0x52c069 thru 0x539076 data_size 0xb(0xd00e bytes) 2024-09-04T17:47:16.095 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:16.111 INFO:teuthology.orchestra.run.smithi092.stdout:3903 writesame 0xbfa878d thru 0xbfaafc3 data_size 0x1d(0x2837 bytes) 2024-09-04T17:47:16.111 INFO:teuthology.orchestra.run.smithi092.stdout:3904 read 0x88e62aa thru 0x88f2de4 (0xcb3b bytes) 2024-09-04T17:47:16.111 INFO:teuthology.orchestra.run.smithi092.stdout:3905 writesame 0x3e73cc4 thru 0x3e78513 data_size 0x10(0x4850 bytes) 2024-09-04T17:47:16.114 INFO:teuthology.orchestra.run.smithi092.stdout:3906 read 0xbd17cee thru 0xbd19688 (0x199b bytes) 2024-09-04T17:47:16.114 INFO:teuthology.orchestra.run.smithi092.stdout:3907 read 0xbd6a7ad thru 0xbd781c1 (0xda15 bytes) 2024-09-04T17:47:16.115 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:16.110+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:16.115 INFO:teuthology.orchestra.run.smithi092.stdout:3909 writesame 0x5d7babc thru 0x5d8b5b2 data_size 0x1(0xfaf7 bytes) 2024-09-04T17:47:16.140 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:16.140 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:16.152 INFO:teuthology.orchestra.run.smithi092.stdout:3910 write 0x8324c40 thru 0x8327670 (0x2a31 bytes) 2024-09-04T17:47:16.156 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:47:16.156 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:47:16.158 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-04T17:47:16.158 INFO:teuthology.orchestra.run.smithi092.stdout:3911 trunc from 0xbfaafc4 to 0xde29f44 2024-09-04T17:47:16.165 INFO:teuthology.orchestra.run.smithi092.stdout:3913 writesame 0x8e9cd21 thru 0x8ea3b83 data_size 0xb(0x6e63 bytes) 2024-09-04T17:47:16.169 INFO:teuthology.orchestra.run.smithi092.stdout:3914 punch from 0xce32a58 to 0xce39906, (0x6eae bytes) 2024-09-04T17:47:16.176 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:47:16.176 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:47:16.177 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-04T17:47:16.178 INFO:teuthology.orchestra.run.smithi092.stdout:3915 trunc from 0xde29f44 to 0x8ac43cf 2024-09-04T17:47:16.188 INFO:teuthology.orchestra.run.smithi190.stdout:get_pg_num_history pg_num_history pg_num_history(e58 pg_nums {1={16=1},2={19=8},3={24=32}} deleted_pools ) 2024-09-04T17:47:16.188 INFO:teuthology.orchestra.run.smithi190.stdout:Importing pgid 3.1e 2024-09-04T17:47:16.188 INFO:teuthology.orchestra.run.smithi190.stdout:Write #3:7817bfb4:::rbd_data.113417f3df25.0000000000000001:5# 2024-09-04T17:47:16.188 INFO:teuthology.orchestra.run.smithi092.stdout:3917 punch from 0x35f677e to 0x35fecce, (0x8550 bytes) 2024-09-04T17:47:16.188 INFO:teuthology.orchestra.run.smithi092.stdout:3919 writesame 0x28ffe98 thru 0x2907705 data_size 0x5(0x786e bytes) 2024-09-04T17:47:16.200 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-04T17:47:16.200 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-04T17:47:16.207 INFO:teuthology.orchestra.run.smithi092.stdout:3920 write 0xd65173e thru 0xd653910 (0x21d3 bytes) 2024-09-04T17:47:16.215 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:47:16.212+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:47:16.217 INFO:teuthology.orchestra.run.smithi190.stdout:Write #3:7817bfb4:::rbd_data.113417f3df25.0000000000000001:head# 2024-09-04T17:47:16.218 INFO:teuthology.orchestra.run.smithi190.stdout:snapset 5=[]:{5=[5]} 2024-09-04T17:47:16.218 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:47:16.218 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:47:16.219 INFO:teuthology.orchestra.run.smithi092.stdout:3921 write 0xb8def26 thru 0xb8ec278 (0xd353 bytes) 2024-09-04T17:47:16.225 INFO:teuthology.orchestra.run.smithi190.stdout:Write #3:7847ffdf:::rbd_object_map.11346251f094:head# 2024-09-04T17:47:16.225 INFO:teuthology.orchestra.run.smithi190.stdout:snapset 0=[]:{} 2024-09-04T17:47:16.225 INFO:teuthology.orchestra.run.smithi092.stdout:3922 read 0x29844a7 thru 0x298f76b (0xb2c5 bytes) 2024-09-04T17:47:16.225 INFO:teuthology.orchestra.run.smithi092.stdout:3925 read 0xa11307b thru 0xa116939 (0x38bf bytes) 2024-09-04T17:47:16.226 INFO:teuthology.orchestra.run.smithi092.stdout:3926 write 0x26413c thru 0x27194d (0xd812 bytes) 2024-09-04T17:47:16.228 INFO:teuthology.orchestra.run.smithi190.stdout:Write #3:7854e172:::rbd_data.1134fdde05d4.000000000000003a:head# 2024-09-04T17:47:16.228 INFO:teuthology.orchestra.run.smithi190.stdout:snapset 0=[]:{} 2024-09-04T17:47:16.228 INFO:teuthology.orchestra.run.smithi190.stdout:Write #3:786fb296:::rbd_data.1134fdde05d4.0000000000000017:head# 2024-09-04T17:47:16.228 INFO:teuthology.orchestra.run.smithi190.stdout:snapset 0=[]:{} 2024-09-04T17:47:16.229 INFO:teuthology.orchestra.run.smithi092.stdout:3927 read 0x3b3a736 thru 0x3b43030 (0x88fb bytes) 2024-09-04T17:47:16.229 INFO:teuthology.orchestra.run.smithi092.stdout:3928 read 0x34f75df thru 0x34fbb54 (0x4576 bytes) 2024-09-04T17:47:16.229 INFO:teuthology.orchestra.run.smithi190.stdout:Write #3:789a8067:::rbd_data.1134cfbce83d.0000000000000005:6# 2024-09-04T17:47:16.230 INFO:teuthology.orchestra.run.smithi092.stdout:3930 write 0xd031e51 thru 0xd036ae8 (0x4c98 bytes) 2024-09-04T17:47:16.232 INFO:teuthology.orchestra.run.smithi190.stdout:Write #3:789a8067:::rbd_data.1134cfbce83d.0000000000000005:head# 2024-09-04T17:47:16.232 INFO:teuthology.orchestra.run.smithi190.stdout:snapset 6=[]:{6=[6]} 2024-09-04T17:47:16.233 INFO:teuthology.orchestra.run.smithi092.stdout:3933 clone 31 order 25 su 262144 sc 10 2024-09-04T17:47:16.233 INFO:teuthology.orchestra.run.smithi190.stdout:Write #3:797dbd38:::rbd_data.1134b705d747.00000000000000a4:head# 2024-09-04T17:47:16.233 INFO:teuthology.orchestra.run.smithi190.stdout:snapset 11=[]:{} 2024-09-04T17:47:16.233 INFO:teuthology.orchestra.run.smithi190.stdout:Write #3:7a6b822e:::rbd_id.image_client.0-clone15:head# 2024-09-04T17:47:16.234 INFO:teuthology.orchestra.run.smithi190.stdout:snapset 0=[]:{} 2024-09-04T17:47:16.234 INFO:teuthology.orchestra.run.smithi190.stdout:Write #3:7ad00c32:::rbd_data.1134b705d747.00000000000000bd:head# 2024-09-04T17:47:16.234 INFO:teuthology.orchestra.run.smithi190.stdout:snapset 11=[]:{} 2024-09-04T17:47:16.234 INFO:teuthology.orchestra.run.smithi190.stdout:Write #3:7b072538:::rbd_data.1134fdde05d4.0000000000000000:head# 2024-09-04T17:47:16.234 INFO:teuthology.orchestra.run.smithi190.stdout:snapset 0=[]:{} 2024-09-04T17:47:16.235 INFO:teuthology.orchestra.run.smithi190.stdout:Write #3:7b30cfab:::rbd_data.1134b8120a27.0000000000000053:head# 2024-09-04T17:47:16.235 INFO:teuthology.orchestra.run.smithi190.stdout:snapset 0=[]:{} 2024-09-04T17:47:16.236 INFO:teuthology.orchestra.run.smithi190.stdout:Write #3:7b6319ad:::rbd_data.1134fdde05d4.0000000000000178:head# 2024-09-04T17:47:16.236 INFO:teuthology.orchestra.run.smithi190.stdout:snapset 0=[]:{} 2024-09-04T17:47:16.236 INFO:teuthology.orchestra.run.smithi190.stdout:Write #3:7bb542e8:::rbd_data.11348d83498b.0000000000000003:c# 2024-09-04T17:47:16.237 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:47:16.237 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:47:16.237 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-04T17:47:16.257 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:47:16.257 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:47:16.258 INFO:teuthology.orchestra.run.smithi190.stdout:Write #3:7bb542e8:::rbd_data.11348d83498b.0000000000000003:head# 2024-09-04T17:47:16.258 INFO:teuthology.orchestra.run.smithi190.stdout:snapset c=[]:{c=[c]} 2024-09-04T17:47:16.258 INFO:teuthology.orchestra.run.smithi190.stdout:Write #3:7bbed1a3:::rbd_data.11345bd5898b.0000000000000003:head# 2024-09-04T17:47:16.258 INFO:teuthology.orchestra.run.smithi190.stdout:snapset 0=[]:{} 2024-09-04T17:47:16.258 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-04T17:47:16.259 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-04T17:47:16.267 INFO:teuthology.orchestra.run.smithi190.stdout:Write #3:7c3a39b0:::rbd_data.1134eedad849.0000000000000341:head# 2024-09-04T17:47:16.267 INFO:teuthology.orchestra.run.smithi190.stdout:snapset 10=[]:{} 2024-09-04T17:47:16.268 INFO:teuthology.orchestra.run.smithi190.stdout:Write #3:7c7e60eb:::rbd_data.1134eedad849.0000000000000337:head# 2024-09-04T17:47:16.268 INFO:teuthology.orchestra.run.smithi190.stdout:snapset 10=[]:{} 2024-09-04T17:47:16.268 INFO:teuthology.orchestra.run.smithi190.stdout:Write #3:7cb367d5:::rbd_data.11346251f094.0000000000000000:head# 2024-09-04T17:47:16.268 INFO:teuthology.orchestra.run.smithi190.stdout:snapset 0=[]:{} 2024-09-04T17:47:16.269 INFO:teuthology.orchestra.run.smithi190.stdout:Write #3:7cb6c77e:::rbd_data.1134b705d747.00000000000000cc:head# 2024-09-04T17:47:16.269 INFO:teuthology.orchestra.run.smithi190.stdout:snapset 11=[]:{} 2024-09-04T17:47:16.269 INFO:teuthology.orchestra.run.smithi190.stdout:Write #3:7dc7c0f5:::rbd_data.1134eedad849.000000000000037d:head# 2024-09-04T17:47:16.269 INFO:teuthology.orchestra.run.smithi190.stdout:snapset 10=[]:{} 2024-09-04T17:47:16.269 INFO:teuthology.orchestra.run.smithi190.stdout:Write #3:7ebdf3d9:::rbd_data.1134eedad849.00000000000002a0:head# 2024-09-04T17:47:16.270 INFO:teuthology.orchestra.run.smithi190.stdout:snapset 10=[]:{} 2024-09-04T17:47:16.270 INFO:teuthology.orchestra.run.smithi190.stdout:Write #3:7fd84f05:::rbd_data.11342e8f3384.0000000000000012:13# 2024-09-04T17:47:16.272 INFO:teuthology.orchestra.run.smithi190.stdout:Write #3:7fd84f05:::rbd_data.11342e8f3384.0000000000000012:head# 2024-09-04T17:47:16.272 INFO:teuthology.orchestra.run.smithi190.stdout:snapset 13=[]:{13=[13]} 2024-09-04T17:47:16.273 INFO:teuthology.orchestra.run.smithi190.stdout:Write #3:7feb9299:::rbd_data.1134b705d747.0000000000000087:head# 2024-09-04T17:47:16.273 INFO:teuthology.orchestra.run.smithi190.stdout:snapset 11=[]:{} 2024-09-04T17:47:16.273 INFO:teuthology.orchestra.run.smithi190.stdout:write_pg epoch 58 info 3.1e( v 58'338 (0'0,58'338] local-lis/les=51/52 n=25 ec=24/24 lis/c=51/51 les/c/f=52/52/0 sis=51) 2024-09-04T17:47:16.275 INFO:teuthology.orchestra.run.smithi190.stdout:Import successful 2024-09-04T17:47:16.290 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:16.291 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:47:16.291 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:47:16.291 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:47:16.291 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:16.305 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-04T17:47:16.352 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:16.352+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:16.418 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:16.418+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:16.479 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:16.479 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:47:16.480 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:47:16.480 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:47:16.480 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.480 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8831 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2453504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:16.480 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:47:16.480 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 63.420650664999997, 2024-09-04T17:47:16.480 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14731111799999999, 2024-09-04T17:47:16.480 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:16.480 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:16.480 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:16.480 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:16.480 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:16.481 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:16.481 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8831 2024-09-04T17:47:16.481 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.481 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:16.481 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.481 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:16.481 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:47:16.481 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:16.481 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.481 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.481 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:16.481 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:47:16.481 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:16.482 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.482 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.482 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:16.482 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057811+0000", 2024-09-04T17:47:16.482 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.083e-06 2024-09-04T17:47:16.482 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.482 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.482 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:16.482 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057814+0000", 2024-09-04T17:47:16.482 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.407e-06 2024-09-04T17:47:16.482 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.482 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.482 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:16.483 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057825+0000", 2024-09-04T17:47:16.483 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.17e-05 2024-09-04T17:47:16.483 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.483 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.483 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:16.483 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057836+0000", 2024-09-04T17:47:16.483 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0283e-05 2024-09-04T17:47:16.483 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.483 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.483 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:16.483 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.204762+0000", 2024-09-04T17:47:16.483 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.146926205 2024-09-04T17:47:16.483 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.483 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.484 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:16.484 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.204895+0000", 2024-09-04T17:47:16.484 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013328999999999999 2024-09-04T17:47:16.484 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.484 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.484 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:16.484 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205121+0000", 2024-09-04T17:47:16.484 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00022614999999999999 2024-09-04T17:47:16.484 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:16.484 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:16.484 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:16.484 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.484 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.485 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8832 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2490368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:16.485 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:47:16.485 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 63.420612632000001, 2024-09-04T17:47:16.485 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14770012299999999, 2024-09-04T17:47:16.485 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:16.485 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:16.485 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:16.485 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:16.485 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:16.485 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:16.485 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8832 2024-09-04T17:47:16.485 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.486 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:16.486 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.486 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:16.486 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:47:16.486 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:16.486 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.486 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.486 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:16.486 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:47:16.486 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:16.486 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.486 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.486 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:16.487 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057849+0000", 2024-09-04T17:47:16.487 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6700000000000003e-07 2024-09-04T17:47:16.487 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.487 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.487 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:16.487 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057852+0000", 2024-09-04T17:47:16.487 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6359999999999998e-06 2024-09-04T17:47:16.487 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.487 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.487 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:16.487 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057860+0000", 2024-09-04T17:47:16.487 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8140000000000007e-06 2024-09-04T17:47:16.487 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.488 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.488 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:16.488 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057868+0000", 2024-09-04T17:47:16.488 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1999999999999997e-06 2024-09-04T17:47:16.488 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.488 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.488 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:16.488 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205208+0000", 2024-09-04T17:47:16.488 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14734067300000001 2024-09-04T17:47:16.488 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.488 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.488 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:16.488 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205347+0000", 2024-09-04T17:47:16.489 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013914900000000001 2024-09-04T17:47:16.489 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.489 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.489 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:16.489 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205548+0000", 2024-09-04T17:47:16.489 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000200984 2024-09-04T17:47:16.489 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:16.489 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:16.489 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:16.489 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.489 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.489 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8833 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2527232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:16.489 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:47:16.490 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 63.420580762, 2024-09-04T17:47:16.490 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148062572, 2024-09-04T17:47:16.490 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:16.490 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:16.490 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:16.490 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:16.490 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:16.490 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:16.490 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8833 2024-09-04T17:47:16.490 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.490 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:16.490 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.490 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:16.491 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:47:16.491 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:16.491 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.491 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.491 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:16.491 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:47:16.491 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:16.491 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.491 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.491 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:16.491 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057881+0000", 2024-09-04T17:47:16.491 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4300000000000002e-07 2024-09-04T17:47:16.491 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.492 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.492 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:16.492 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057893+0000", 2024-09-04T17:47:16.492 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1973999999999999e-05 2024-09-04T17:47:16.492 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.492 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.492 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:16.492 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057906+0000", 2024-09-04T17:47:16.492 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2716e-05 2024-09-04T17:47:16.492 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.492 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.492 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:16.493 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057913+0000", 2024-09-04T17:47:16.493 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3930000000000002e-06 2024-09-04T17:47:16.493 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.493 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.493 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:16.493 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205621+0000", 2024-09-04T17:47:16.493 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14770788800000001 2024-09-04T17:47:16.493 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.493 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.493 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:16.493 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205740+0000", 2024-09-04T17:47:16.494 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000118936 2024-09-04T17:47:16.494 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.494 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.494 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:16.494 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205943+0000", 2024-09-04T17:47:16.494 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020292199999999999 2024-09-04T17:47:16.494 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:16.494 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:16.494 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:16.494 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.494 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.494 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8834 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2564096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:16.494 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:47:16.495 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 63.420537482999997, 2024-09-04T17:47:16.495 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14846753200000001, 2024-09-04T17:47:16.495 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:16.495 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:16.495 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:16.495 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:16.495 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:16.495 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:16.495 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8834 2024-09-04T17:47:16.495 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.495 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:16.495 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.495 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:16.496 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:47:16.496 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:16.496 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.496 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.496 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:16.496 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:47:16.496 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:16.496 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.496 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.496 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:16.496 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057925+0000", 2024-09-04T17:47:16.496 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0270000000000001e-06 2024-09-04T17:47:16.496 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.497 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.497 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:16.497 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057936+0000", 2024-09-04T17:47:16.497 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.112e-05 2024-09-04T17:47:16.497 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.497 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.497 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:16.497 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057946+0000", 2024-09-04T17:47:16.497 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.009e-05 2024-09-04T17:47:16.497 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.497 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.497 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:16.497 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057953+0000", 2024-09-04T17:47:16.498 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.6229999999999997e-06 2024-09-04T17:47:16.498 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.498 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.498 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:16.498 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206027+0000", 2024-09-04T17:47:16.498 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148073591 2024-09-04T17:47:16.498 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.498 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.498 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:16.498 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206161+0000", 2024-09-04T17:47:16.498 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013411599999999999 2024-09-04T17:47:16.498 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.498 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.498 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:16.499 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206391+0000", 2024-09-04T17:47:16.499 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000229965 2024-09-04T17:47:16.499 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:16.499 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:16.499 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:16.499 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.499 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.499 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8835 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2600960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:16.499 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:47:16.499 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 63.420497832999999, 2024-09-04T17:47:16.499 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14882941599999999, 2024-09-04T17:47:16.499 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:16.499 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:16.500 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:16.500 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:16.500 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:16.500 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:16.500 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8835 2024-09-04T17:47:16.500 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.500 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:16.500 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.500 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:16.500 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:47:16.500 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:16.500 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.500 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.501 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:16.501 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:47:16.501 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:16.501 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.501 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.501 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:16.501 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058544+0000", 2024-09-04T17:47:16.501 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00058061899999999999 2024-09-04T17:47:16.501 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.501 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.501 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:16.501 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058550+0000", 2024-09-04T17:47:16.501 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.1129999999999997e-06 2024-09-04T17:47:16.501 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.502 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.502 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:16.502 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058566+0000", 2024-09-04T17:47:16.502 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5820000000000001e-05 2024-09-04T17:47:16.502 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.502 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.502 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:16.502 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058578+0000", 2024-09-04T17:47:16.502 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2537e-05 2024-09-04T17:47:16.502 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.502 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.502 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:16.502 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206470+0000", 2024-09-04T17:47:16.503 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14789153799999999 2024-09-04T17:47:16.503 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.503 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.503 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:16.503 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206589+0000", 2024-09-04T17:47:16.503 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000118837 2024-09-04T17:47:16.503 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.503 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.503 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:16.503 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206793+0000", 2024-09-04T17:47:16.503 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000203952 2024-09-04T17:47:16.504 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:16.504 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:16.504 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:16.504 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.504 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.504 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8836 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2654208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:16.504 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:47:16.504 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 63.419865145000003, 2024-09-04T17:47:16.504 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14862433999999999, 2024-09-04T17:47:16.504 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:16.504 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:16.505 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:16.505 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:16.505 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:16.505 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:16.505 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8836 2024-09-04T17:47:16.505 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.505 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:16.505 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.505 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:16.505 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:47:16.505 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:16.505 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.505 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.506 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:16.506 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:47:16.506 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:16.506 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.506 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.506 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:16.506 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062663+0000", 2024-09-04T17:47:16.506 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0040670619999999998 2024-09-04T17:47:16.506 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.506 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.506 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:16.506 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062668+0000", 2024-09-04T17:47:16.506 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4199999999999998e-06 2024-09-04T17:47:16.506 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.507 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.507 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:16.507 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062689+0000", 2024-09-04T17:47:16.507 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0356999999999998e-05 2024-09-04T17:47:16.507 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.507 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.507 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:16.507 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062726+0000", 2024-09-04T17:47:16.507 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7302999999999998e-05 2024-09-04T17:47:16.507 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.507 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.507 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:16.507 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206875+0000", 2024-09-04T17:47:16.508 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14414883100000001 2024-09-04T17:47:16.508 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.508 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.508 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:16.508 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.207016+0000", 2024-09-04T17:47:16.508 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000140822 2024-09-04T17:47:16.508 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.508 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.508 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:16.508 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.207220+0000", 2024-09-04T17:47:16.508 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020454499999999999 2024-09-04T17:47:16.508 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:16.508 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:16.509 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:16.509 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.509 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.509 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8844 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2928640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:16.509 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:47:16.509 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 63.415501061999997, 2024-09-04T17:47:16.509 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14752153100000001, 2024-09-04T17:47:16.509 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:16.509 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:16.509 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:16.509 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:16.509 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:16.509 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:16.510 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8844 2024-09-04T17:47:16.510 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.510 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:16.510 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.510 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:16.510 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:47:16.510 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:16.510 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.510 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.510 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:16.510 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:47:16.510 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:16.510 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.511 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.511 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:16.511 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062961+0000", 2024-09-04T17:47:16.511 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6700000000000003e-07 2024-09-04T17:47:16.511 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.511 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.511 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:16.511 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062973+0000", 2024-09-04T17:47:16.511 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2109999999999999e-05 2024-09-04T17:47:16.511 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.511 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.511 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:16.511 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062982+0000", 2024-09-04T17:47:16.511 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.6900000000000004e-06 2024-09-04T17:47:16.512 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.512 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.512 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:16.512 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062990+0000", 2024-09-04T17:47:16.512 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1899999999999998e-06 2024-09-04T17:47:16.512 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.512 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.512 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:16.512 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210090+0000", 2024-09-04T17:47:16.512 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14710026300000001 2024-09-04T17:47:16.512 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.512 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.512 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:16.513 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210225+0000", 2024-09-04T17:47:16.513 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013532400000000001 2024-09-04T17:47:16.513 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.513 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.513 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:16.513 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210481+0000", 2024-09-04T17:47:16.513 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00025628700000000001 2024-09-04T17:47:16.513 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:16.513 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:16.513 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:16.513 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.513 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.513 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8845 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2965504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:16.514 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:47:16.514 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 63.415461338999997, 2024-09-04T17:47:16.514 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147916819, 2024-09-04T17:47:16.514 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:16.514 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:16.514 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:16.514 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:16.514 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:16.514 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:16.514 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8845 2024-09-04T17:47:16.514 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.514 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:16.514 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.515 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:16.515 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:47:16.515 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:16.515 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.515 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.515 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:16.515 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:47:16.515 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:16.515 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.515 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.515 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:16.515 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:47:16.515 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.0399999999999995e-07 2024-09-04T17:47:16.516 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.516 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.516 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:16.516 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063003+0000", 2024-09-04T17:47:16.516 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6529999999999998e-06 2024-09-04T17:47:16.516 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.516 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.516 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:16.516 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063012+0000", 2024-09-04T17:47:16.516 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9500000000000007e-06 2024-09-04T17:47:16.516 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.516 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.516 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:16.517 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063027+0000", 2024-09-04T17:47:16.517 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5443000000000001e-05 2024-09-04T17:47:16.517 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.517 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.517 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:16.517 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210571+0000", 2024-09-04T17:47:16.517 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14754353100000001 2024-09-04T17:47:16.517 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.517 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.517 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:16.517 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210714+0000", 2024-09-04T17:47:16.517 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014295599999999999 2024-09-04T17:47:16.517 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.517 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.518 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:16.518 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210916+0000", 2024-09-04T17:47:16.518 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020258200000000001 2024-09-04T17:47:16.518 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:16.518 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:16.518 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:16.518 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.518 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.518 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8846 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3002368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:16.518 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:47:16.518 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 63.415423801999999, 2024-09-04T17:47:16.518 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14828011599999999, 2024-09-04T17:47:16.518 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:16.519 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:16.519 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:16.519 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:16.519 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:16.519 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:16.519 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8846 2024-09-04T17:47:16.519 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.519 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:16.519 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.519 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:16.519 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:47:16.519 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:16.519 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.519 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.520 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:16.520 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:47:16.520 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:16.520 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.520 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.520 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:16.520 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063038+0000", 2024-09-04T17:47:16.520 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.23e-07 2024-09-04T17:47:16.520 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.520 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.520 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:16.520 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063051+0000", 2024-09-04T17:47:16.520 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2757e-05 2024-09-04T17:47:16.521 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.521 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.521 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:16.521 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063070+0000", 2024-09-04T17:47:16.521 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9663000000000001e-05 2024-09-04T17:47:16.521 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.521 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.521 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:16.521 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063079+0000", 2024-09-04T17:47:16.521 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.0929999999999992e-06 2024-09-04T17:47:16.521 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.521 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.521 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:16.522 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210989+0000", 2024-09-04T17:47:16.522 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14790984500000001 2024-09-04T17:47:16.522 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.522 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.522 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:16.522 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211109+0000", 2024-09-04T17:47:16.522 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000120056 2024-09-04T17:47:16.522 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.522 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.522 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:16.522 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211317+0000", 2024-09-04T17:47:16.522 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020797900000000001 2024-09-04T17:47:16.522 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:16.522 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:16.523 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:16.523 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.523 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.523 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8847 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3039232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:16.523 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:47:16.523 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 63.415349370000001, 2024-09-04T17:47:16.523 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14859408900000001, 2024-09-04T17:47:16.523 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:16.523 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:16.523 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:16.523 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:16.523 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:16.523 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:16.523 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8847 2024-09-04T17:47:16.524 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.524 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:16.524 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.524 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:16.524 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:47:16.524 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:16.524 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.524 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.524 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:16.524 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:47:16.524 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:16.524 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.524 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.525 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:16.525 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063113+0000", 2024-09-04T17:47:16.525 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.204e-06 2024-09-04T17:47:16.525 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.525 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.525 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:16.525 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063116+0000", 2024-09-04T17:47:16.525 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.3330000000000001e-06 2024-09-04T17:47:16.525 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.525 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.525 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:16.525 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063126+0000", 2024-09-04T17:47:16.525 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0166999999999999e-05 2024-09-04T17:47:16.526 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.526 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.526 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:16.526 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063134+0000", 2024-09-04T17:47:16.526 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4329999999999998e-06 2024-09-04T17:47:16.526 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.526 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.526 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:16.526 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211392+0000", 2024-09-04T17:47:16.526 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148258109 2024-09-04T17:47:16.526 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.526 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.526 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:16.527 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211506+0000", 2024-09-04T17:47:16.527 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000113813 2024-09-04T17:47:16.527 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.527 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.527 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:16.527 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211706+0000", 2024-09-04T17:47:16.527 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020002999999999999 2024-09-04T17:47:16.527 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:16.527 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:16.527 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:16.527 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.528 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.528 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8848 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:16.528 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:47:16.528 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 63.415317033000001, 2024-09-04T17:47:16.528 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14907967599999999, 2024-09-04T17:47:16.528 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:16.528 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:16.528 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:16.528 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:16.528 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:16.528 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:16.528 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8848 2024-09-04T17:47:16.528 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.529 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:16.529 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.529 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:16.529 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:47:16.529 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:16.529 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.529 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.529 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:16.529 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:47:16.529 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:16.529 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.529 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.529 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:16.530 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070589+0000", 2024-09-04T17:47:16.530 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.007445386 2024-09-04T17:47:16.530 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.530 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.530 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:16.530 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070600+0000", 2024-09-04T17:47:16.530 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0346999999999999e-05 2024-09-04T17:47:16.530 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.530 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.530 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:16.530 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070622+0000", 2024-09-04T17:47:16.530 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2289999999999998e-05 2024-09-04T17:47:16.530 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.530 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.531 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:16.531 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070636+0000", 2024-09-04T17:47:16.531 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4093e-05 2024-09-04T17:47:16.531 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.531 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.531 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:16.531 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211815+0000", 2024-09-04T17:47:16.531 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14117874699999999 2024-09-04T17:47:16.531 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.531 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.531 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:16.531 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211983+0000", 2024-09-04T17:47:16.531 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00016852800000000001 2024-09-04T17:47:16.532 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.532 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.532 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:16.532 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.212224+0000", 2024-09-04T17:47:16.532 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024028499999999999 2024-09-04T17:47:16.532 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:16.532 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:16.532 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:16.532 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.532 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.532 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8862 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2019328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:16.532 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:47:16.532 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 63.407267564999998, 2024-09-04T17:47:16.533 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14755236399999999, 2024-09-04T17:47:16.533 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:16.533 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:16.533 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:16.533 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:16.533 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:16.533 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:16.533 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8862 2024-09-04T17:47:16.533 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.533 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:16.533 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.533 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:16.533 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:47:16.533 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:16.534 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.534 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.534 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:16.534 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:47:16.534 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:16.534 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.534 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.534 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:16.534 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071194+0000", 2024-09-04T17:47:16.534 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.5000000000000002e-07 2024-09-04T17:47:16.534 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.534 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.534 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:16.535 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071197+0000", 2024-09-04T17:47:16.535 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5900000000000002e-06 2024-09-04T17:47:16.535 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.535 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.535 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:16.535 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071206+0000", 2024-09-04T17:47:16.535 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9700000000000005e-06 2024-09-04T17:47:16.535 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.535 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.535 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:16.535 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071213+0000", 2024-09-04T17:47:16.535 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.9070000000000001e-06 2024-09-04T17:47:16.535 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.535 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.536 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:16.536 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218396+0000", 2024-09-04T17:47:16.536 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147183273 2024-09-04T17:47:16.536 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.536 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.536 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:16.536 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218538+0000", 2024-09-04T17:47:16.536 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014194199999999999 2024-09-04T17:47:16.536 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.536 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.536 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:16.536 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218746+0000", 2024-09-04T17:47:16.536 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000208032 2024-09-04T17:47:16.537 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:16.537 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:16.537 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:16.537 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.537 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.537 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8863 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2056192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:16.537 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:47:16.537 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 63.407239255, 2024-09-04T17:47:16.537 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14792281099999999, 2024-09-04T17:47:16.537 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:16.537 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:16.537 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:16.537 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:16.538 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:16.538 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:16.538 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8863 2024-09-04T17:47:16.538 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.538 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:16.538 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.538 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:16.538 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:47:16.538 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:16.538 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.538 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.538 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:16.538 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:47:16.539 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:16.539 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.539 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.539 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:16.539 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:47:16.539 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1299999999999999e-07 2024-09-04T17:47:16.539 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.539 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.539 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:16.539 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071225+0000", 2024-09-04T17:47:16.539 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6199999999999999e-06 2024-09-04T17:47:16.539 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.539 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.540 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:16.540 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071234+0000", 2024-09-04T17:47:16.540 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8769999999999999e-06 2024-09-04T17:47:16.540 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.540 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.540 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:16.540 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071251+0000", 2024-09-04T17:47:16.540 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.738e-05 2024-09-04T17:47:16.540 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.540 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.540 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:16.540 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218798+0000", 2024-09-04T17:47:16.540 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14754645 2024-09-04T17:47:16.541 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.541 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.541 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:16.541 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218926+0000", 2024-09-04T17:47:16.541 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000128446 2024-09-04T17:47:16.541 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.541 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.541 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:16.541 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219145+0000", 2024-09-04T17:47:16.541 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021832500000000001 2024-09-04T17:47:16.541 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:16.541 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:16.541 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:16.542 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.542 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.542 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8864 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2093056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:16.542 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:47:16.542 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 63.407197678999999, 2024-09-04T17:47:16.542 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148277309, 2024-09-04T17:47:16.542 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:16.542 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:16.542 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:16.542 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:16.542 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:16.542 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:16.542 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8864 2024-09-04T17:47:16.543 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.543 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:16.543 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.543 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:16.543 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:47:16.543 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:16.543 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.543 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.543 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:16.543 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:47:16.543 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:16.543 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.543 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.543 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:16.544 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071264+0000", 2024-09-04T17:47:16.544 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3e-07 2024-09-04T17:47:16.544 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.544 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.544 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:16.544 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071267+0000", 2024-09-04T17:47:16.544 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0139999999999999e-06 2024-09-04T17:47:16.544 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.544 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.544 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:16.544 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071276+0000", 2024-09-04T17:47:16.544 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1689999999999998e-06 2024-09-04T17:47:16.544 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.545 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.545 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:16.545 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071283+0000", 2024-09-04T17:47:16.545 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2269999999999996e-06 2024-09-04T17:47:16.545 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.545 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.545 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:16.545 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219195+0000", 2024-09-04T17:47:16.545 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147911868 2024-09-04T17:47:16.545 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.545 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.545 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:16.545 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219324+0000", 2024-09-04T17:47:16.546 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00012835900000000001 2024-09-04T17:47:16.546 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.546 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.546 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:16.546 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219541+0000", 2024-09-04T17:47:16.546 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000216942 2024-09-04T17:47:16.546 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:16.546 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:16.546 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:16.546 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.546 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.546 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8865 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2109440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:16.546 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:47:16.547 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 63.407168218999999, 2024-09-04T17:47:16.547 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14865747300000001, 2024-09-04T17:47:16.547 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:16.547 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:16.547 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:16.547 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:16.547 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:16.547 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:16.547 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8865 2024-09-04T17:47:16.547 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.547 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:16.547 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.547 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:16.547 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:47:16.548 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:16.548 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.548 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.548 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:16.548 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:47:16.548 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:16.548 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.548 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.548 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:16.548 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:47:16.548 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7999999999999995e-07 2024-09-04T17:47:16.548 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.548 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.549 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:16.549 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071296+0000", 2024-09-04T17:47:16.549 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6900000000000001e-06 2024-09-04T17:47:16.549 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.549 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.549 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:16.549 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071305+0000", 2024-09-04T17:47:16.549 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.5499999999999995e-06 2024-09-04T17:47:16.549 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.549 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.549 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:16.549 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071312+0000", 2024-09-04T17:47:16.549 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1559999999999998e-06 2024-09-04T17:47:16.549 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.550 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.550 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:16.550 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219591+0000", 2024-09-04T17:47:16.550 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14827944900000001 2024-09-04T17:47:16.550 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.550 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.550 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:16.550 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219731+0000", 2024-09-04T17:47:16.550 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013931600000000001 2024-09-04T17:47:16.550 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.550 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.550 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:16.550 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219950+0000", 2024-09-04T17:47:16.551 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021963199999999999 2024-09-04T17:47:16.551 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:16.551 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:16.551 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:16.551 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.551 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.551 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8866 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2146304~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:16.551 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:47:16.551 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 63.407139985999997, 2024-09-04T17:47:16.551 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14906117899999999, 2024-09-04T17:47:16.551 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:16.551 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:16.551 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:16.551 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:16.552 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:16.552 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:16.552 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8866 2024-09-04T17:47:16.552 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.552 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:16.552 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.552 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:16.552 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:47:16.552 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:16.552 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.552 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.552 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:16.552 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:47:16.553 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:16.553 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.553 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.553 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:16.553 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071322+0000", 2024-09-04T17:47:16.553 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7999999999999995e-07 2024-09-04T17:47:16.553 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.553 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.553 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:16.553 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071338+0000", 2024-09-04T17:47:16.553 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5872999999999999e-05 2024-09-04T17:47:16.553 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.553 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.553 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:16.554 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071347+0000", 2024-09-04T17:47:16.554 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4669999999999993e-06 2024-09-04T17:47:16.554 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.554 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.554 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:16.554 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071354+0000", 2024-09-04T17:47:16.554 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2099999999999996e-06 2024-09-04T17:47:16.554 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.554 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.554 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:16.554 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220008+0000", 2024-09-04T17:47:16.554 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148653642 2024-09-04T17:47:16.555 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.555 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.555 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:16.555 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220155+0000", 2024-09-04T17:47:16.555 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014726600000000001 2024-09-04T17:47:16.555 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.555 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.555 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:16.555 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220382+0000", 2024-09-04T17:47:16.555 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00022704100000000001 2024-09-04T17:47:16.555 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:16.555 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:16.555 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:16.556 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.556 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.556 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8867 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2183168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:16.556 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:47:16.556 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 63.407096633000002, 2024-09-04T17:47:16.556 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.149445831, 2024-09-04T17:47:16.556 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:16.556 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:16.556 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:16.556 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:16.556 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:16.556 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:16.556 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8867 2024-09-04T17:47:16.557 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.557 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:16.557 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.557 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:16.557 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:47:16.557 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:16.557 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.557 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.557 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:16.557 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:47:16.557 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:16.557 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.557 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.557 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:16.558 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078653+0000", 2024-09-04T17:47:16.558 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0072890430000000003 2024-09-04T17:47:16.558 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.558 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.558 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:16.558 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078664+0000", 2024-09-04T17:47:16.558 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0307e-05 2024-09-04T17:47:16.558 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.558 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.558 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:16.558 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078690+0000", 2024-09-04T17:47:16.558 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6780000000000001e-05 2024-09-04T17:47:16.558 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.559 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.559 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:16.559 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078706+0000", 2024-09-04T17:47:16.559 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5676999999999999e-05 2024-09-04T17:47:16.559 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.559 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.559 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:16.559 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220467+0000", 2024-09-04T17:47:16.559 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14176103500000001 2024-09-04T17:47:16.559 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.559 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.559 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:16.559 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220595+0000", 2024-09-04T17:47:16.559 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000127427 2024-09-04T17:47:16.560 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.560 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.560 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:16.560 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220810+0000", 2024-09-04T17:47:16.560 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021556200000000001 2024-09-04T17:47:16.560 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:16.560 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:16.560 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:16.560 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.560 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.560 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8880 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2658304~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:16.560 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.079158+0000", 2024-09-04T17:47:16.560 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 63.399303353999997, 2024-09-04T17:47:16.561 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14718968499999999, 2024-09-04T17:47:16.561 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:16.561 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:16.561 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:16.561 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:16.561 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:16.561 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:16.561 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8880 2024-09-04T17:47:16.561 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.561 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:16.561 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.561 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:16.562 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079158+0000", 2024-09-04T17:47:16.562 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:16.562 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.562 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.562 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:16.562 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079158+0000", 2024-09-04T17:47:16.562 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:16.562 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.562 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.562 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:16.562 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079158+0000", 2024-09-04T17:47:16.562 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3e-07 2024-09-04T17:47:16.562 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.563 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.563 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:16.563 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079161+0000", 2024-09-04T17:47:16.563 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.57e-06 2024-09-04T17:47:16.563 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.563 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.563 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:16.563 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079170+0000", 2024-09-04T17:47:16.563 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.0170000000000002e-06 2024-09-04T17:47:16.563 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.563 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.563 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:16.563 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079177+0000", 2024-09-04T17:47:16.563 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.0530000000000001e-06 2024-09-04T17:47:16.564 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.564 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.564 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:16.564 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.225946+0000", 2024-09-04T17:47:16.564 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14676936500000001 2024-09-04T17:47:16.564 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.564 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.564 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:16.564 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226098+0000", 2024-09-04T17:47:16.564 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000151805 2024-09-04T17:47:16.564 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.564 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.564 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:16.565 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226347+0000", 2024-09-04T17:47:16.565 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024914499999999999 2024-09-04T17:47:16.565 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:16.565 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:16.565 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:16.565 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.565 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.565 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8881 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2695168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:16.565 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:47:16.565 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 63.399274564000002, 2024-09-04T17:47:16.565 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147543547, 2024-09-04T17:47:16.565 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:16.565 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:16.565 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:16.566 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:16.566 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:16.566 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:16.566 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8881 2024-09-04T17:47:16.566 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.566 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:16.566 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.566 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:16.566 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:47:16.566 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:16.566 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.566 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.566 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:16.567 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:47:16.567 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:16.567 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.567 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.567 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:16.567 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079187+0000", 2024-09-04T17:47:16.567 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7700000000000004e-07 2024-09-04T17:47:16.567 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.567 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.567 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:16.567 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079198+0000", 2024-09-04T17:47:16.567 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0849999999999999e-05 2024-09-04T17:47:16.567 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.567 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.568 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:16.568 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079207+0000", 2024-09-04T17:47:16.568 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5599999999999999e-06 2024-09-04T17:47:16.568 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.568 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.568 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:16.568 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079224+0000", 2024-09-04T17:47:16.568 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6387000000000001e-05 2024-09-04T17:47:16.568 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.568 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.568 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:16.568 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226414+0000", 2024-09-04T17:47:16.568 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14718975000000001 2024-09-04T17:47:16.569 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.569 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.569 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:16.569 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226534+0000", 2024-09-04T17:47:16.569 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000120288 2024-09-04T17:47:16.569 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.569 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.569 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:16.569 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226730+0000", 2024-09-04T17:47:16.569 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000196035 2024-09-04T17:47:16.569 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:16.569 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:16.569 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:16.570 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.570 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.570 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8882 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2732032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:16.570 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:47:16.570 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 63.399226349999999, 2024-09-04T17:47:16.570 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14795022399999999, 2024-09-04T17:47:16.570 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:16.570 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:16.570 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:16.570 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:16.570 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:16.570 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:16.570 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8882 2024-09-04T17:47:16.570 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.571 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:16.571 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.571 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:16.571 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:47:16.571 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:16.571 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.571 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.571 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:16.571 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:47:16.571 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:16.571 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.571 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.571 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:16.571 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086588+0000", 2024-09-04T17:47:16.572 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0073530959999999999 2024-09-04T17:47:16.572 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.572 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.572 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:16.572 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086598+0000", 2024-09-04T17:47:16.572 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0122999999999999e-05 2024-09-04T17:47:16.572 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.572 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.572 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:16.572 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086626+0000", 2024-09-04T17:47:16.572 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.7914000000000001e-05 2024-09-04T17:47:16.572 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.572 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.573 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:16.573 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086641+0000", 2024-09-04T17:47:16.573 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5270000000000001e-05 2024-09-04T17:47:16.573 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.573 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.573 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:16.573 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226801+0000", 2024-09-04T17:47:16.573 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.140160321 2024-09-04T17:47:16.573 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.573 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.573 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:16.573 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226951+0000", 2024-09-04T17:47:16.573 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014965200000000001 2024-09-04T17:47:16.573 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:16.574 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:16.574 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:16.574 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.227185+0000", 2024-09-04T17:47:16.574 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00023384799999999999 2024-09-04T17:47:16.574 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:16.574 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:16.574 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:16.574 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:16.574 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:16.574 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:16.576 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-04T17:47:16.576 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:16.570+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:16.677 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:16.674+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:16.703 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:16.703 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:47:16.703 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:47:16.703 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:16.717 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-04T17:47:16.743 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is unset 2024-09-04T17:47:16.777 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:47:16.776+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:47:16.837 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f /var/log/ceph/exp.3.1e.6 2024-09-04T17:47:16.843 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:16.844 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:47:16.844 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:47:16.844 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:47:16.844 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:16.860 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-04T17:47:16.861 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone30 from 0xd653911 (overlap 0x4c9fdc) to 0x3d4cb0 2024-09-04T17:47:16.909 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:16.908+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:16.979 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:16.976+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:16.980 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #29, image image_client.0-clone29 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent30 2024-09-04T17:47:17.032 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:17.032 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:47:17.032 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:47:17.032 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:47:17.032 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.032 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60921 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 262144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:17.033 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822920+0000", 2024-09-04T17:47:17.033 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 3.2095033580000001, 2024-09-04T17:47:17.033 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073904394999999998, 2024-09-04T17:47:17.033 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:17.033 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:17.033 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:17.033 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:17.033 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:17.033 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:17.033 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60921 2024-09-04T17:47:17.033 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.033 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:17.033 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.034 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:17.034 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822920+0000", 2024-09-04T17:47:17.034 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:17.034 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.034 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.034 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:17.034 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822920+0000", 2024-09-04T17:47:17.034 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:17.034 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.034 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.034 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:17.034 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822920+0000", 2024-09-04T17:47:17.034 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4999999999999999e-07 2024-09-04T17:47:17.035 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.035 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.035 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:17.035 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822921+0000", 2024-09-04T17:47:17.035 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.113e-06 2024-09-04T17:47:17.035 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.035 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.035 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:17.035 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822924+0000", 2024-09-04T17:47:17.035 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.354e-06 2024-09-04T17:47:17.035 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.035 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.035 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:17.035 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822927+0000", 2024-09-04T17:47:17.036 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6199999999999999e-06 2024-09-04T17:47:17.036 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.036 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.036 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:17.036 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896586+0000", 2024-09-04T17:47:17.036 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073658976000000001 2024-09-04T17:47:17.036 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.036 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.036 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:17.036 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896626+0000", 2024-09-04T17:47:17.036 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0484000000000001e-05 2024-09-04T17:47:17.036 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.036 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.037 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:17.037 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896824+0000", 2024-09-04T17:47:17.037 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000197598 2024-09-04T17:47:17.037 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:17.037 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:17.037 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:17.037 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.037 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.037 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60922 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:17.037 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822930+0000", 2024-09-04T17:47:17.038 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 3.209492783, 2024-09-04T17:47:17.038 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074178872000000007, 2024-09-04T17:47:17.038 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:17.038 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:17.038 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:17.038 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:17.038 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:17.038 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:17.038 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60922 2024-09-04T17:47:17.038 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.038 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:17.038 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.038 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:17.038 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822930+0000", 2024-09-04T17:47:17.039 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:17.039 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.039 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.039 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:17.039 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822930+0000", 2024-09-04T17:47:17.039 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:17.039 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.039 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.039 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:17.039 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822930+0000", 2024-09-04T17:47:17.039 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.36e-07 2024-09-04T17:47:17.039 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.039 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.040 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:17.040 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822932+0000", 2024-09-04T17:47:17.040 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.567e-06 2024-09-04T17:47:17.040 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.040 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.040 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:17.040 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822938+0000", 2024-09-04T17:47:17.040 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4840000000000003e-06 2024-09-04T17:47:17.040 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.040 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.040 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:17.040 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822942+0000", 2024-09-04T17:47:17.041 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2039999999999999e-06 2024-09-04T17:47:17.041 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.041 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.041 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:17.041 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896850+0000", 2024-09-04T17:47:17.041 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073907941000000005 2024-09-04T17:47:17.041 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.041 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.041 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:17.041 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896891+0000", 2024-09-04T17:47:17.041 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0964999999999998e-05 2024-09-04T17:47:17.041 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.041 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.041 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:17.042 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897109+0000", 2024-09-04T17:47:17.042 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021847500000000001 2024-09-04T17:47:17.042 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:17.042 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:17.042 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:17.042 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.042 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.042 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60923 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 475136~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:17.042 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822949+0000", 2024-09-04T17:47:17.042 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 3.2094744930000001, 2024-09-04T17:47:17.042 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074426928000000003, 2024-09-04T17:47:17.042 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:17.042 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:17.043 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:17.043 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:17.043 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:17.043 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:17.043 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60923 2024-09-04T17:47:17.043 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.043 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:17.043 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.043 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:17.043 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822949+0000", 2024-09-04T17:47:17.043 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:17.043 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.043 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.043 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:17.044 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822949+0000", 2024-09-04T17:47:17.044 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:17.044 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.044 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.044 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:17.044 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822949+0000", 2024-09-04T17:47:17.044 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7399999999999999e-07 2024-09-04T17:47:17.044 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.044 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.044 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:17.044 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822951+0000", 2024-09-04T17:47:17.044 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1940000000000001e-06 2024-09-04T17:47:17.044 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.045 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.045 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:17.045 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822957+0000", 2024-09-04T17:47:17.045 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.0650000000000004e-06 2024-09-04T17:47:17.045 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.045 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.045 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:17.045 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822960+0000", 2024-09-04T17:47:17.045 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.8009999999999999e-06 2024-09-04T17:47:17.045 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.045 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.045 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:17.045 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897138+0000", 2024-09-04T17:47:17.046 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074177990999999999 2024-09-04T17:47:17.046 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.046 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.046 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:17.046 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897179+0000", 2024-09-04T17:47:17.046 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.1057999999999999e-05 2024-09-04T17:47:17.046 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.046 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.046 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:17.046 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897375+0000", 2024-09-04T17:47:17.046 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019644500000000001 2024-09-04T17:47:17.046 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:17.046 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:17.046 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:17.047 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.047 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.047 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60924 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:17.047 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822964+0000", 2024-09-04T17:47:17.047 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 3.209459308, 2024-09-04T17:47:17.047 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074531902999999997, 2024-09-04T17:47:17.047 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:17.047 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:17.047 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:17.047 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:17.047 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:17.047 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:17.048 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60924 2024-09-04T17:47:17.048 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.048 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:17.048 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.048 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:17.048 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822964+0000", 2024-09-04T17:47:17.048 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:17.048 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.048 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.048 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:17.048 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822964+0000", 2024-09-04T17:47:17.048 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:17.048 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.049 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.049 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:17.049 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822964+0000", 2024-09-04T17:47:17.049 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4999999999999999e-07 2024-09-04T17:47:17.049 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.049 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.049 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:17.049 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822966+0000", 2024-09-04T17:47:17.049 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.917e-06 2024-09-04T17:47:17.049 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.049 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.049 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:17.049 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822969+0000", 2024-09-04T17:47:17.050 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.315e-06 2024-09-04T17:47:17.050 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.050 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.050 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:17.050 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822972+0000", 2024-09-04T17:47:17.050 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.599e-06 2024-09-04T17:47:17.050 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.050 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.050 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:17.050 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897398+0000", 2024-09-04T17:47:17.050 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074426168000000001 2024-09-04T17:47:17.050 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.050 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.051 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:17.051 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897441+0000", 2024-09-04T17:47:17.051 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.3158000000000003e-05 2024-09-04T17:47:17.051 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.051 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.051 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:17.051 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897496+0000", 2024-09-04T17:47:17.051 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4496000000000001e-05 2024-09-04T17:47:17.051 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:17.051 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:17.051 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:17.051 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.051 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.051 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60925 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 720896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:17.052 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822975+0000", 2024-09-04T17:47:17.052 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 3.2094478730000002, 2024-09-04T17:47:17.052 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074635272000000003, 2024-09-04T17:47:17.052 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:17.052 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:17.052 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:17.052 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:17.052 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:17.052 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:17.052 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60925 2024-09-04T17:47:17.052 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.052 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:17.053 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.053 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:17.053 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822975+0000", 2024-09-04T17:47:17.053 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:17.053 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.053 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.053 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:17.053 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822975+0000", 2024-09-04T17:47:17.053 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:17.053 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.053 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.053 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:17.053 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822975+0000", 2024-09-04T17:47:17.054 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.34e-07 2024-09-04T17:47:17.054 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.054 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.054 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:17.055 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822976+0000", 2024-09-04T17:47:17.055 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0359999999999999e-06 2024-09-04T17:47:17.055 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.055 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.055 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:17.055 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822980+0000", 2024-09-04T17:47:17.055 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.242e-06 2024-09-04T17:47:17.055 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.055 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.055 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:17.055 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822983+0000", 2024-09-04T17:47:17.055 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.8830000000000002e-06 2024-09-04T17:47:17.055 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.056 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.056 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:17.056 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897519+0000", 2024-09-04T17:47:17.056 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074536366000000007 2024-09-04T17:47:17.056 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.056 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.056 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:17.056 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897560+0000", 2024-09-04T17:47:17.056 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0677999999999999e-05 2024-09-04T17:47:17.056 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.056 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.056 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:17.056 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897610+0000", 2024-09-04T17:47:17.057 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0832999999999999e-05 2024-09-04T17:47:17.057 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:17.057 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:17.057 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:17.057 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.057 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.057 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60926 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 860160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:17.057 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822986+0000", 2024-09-04T17:47:17.057 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 3.209437275, 2024-09-04T17:47:17.057 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074736413000000002, 2024-09-04T17:47:17.057 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:17.057 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:17.057 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:17.058 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:17.058 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:17.058 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:17.058 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60926 2024-09-04T17:47:17.058 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.058 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:17.058 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.058 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:17.058 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822986+0000", 2024-09-04T17:47:17.058 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:17.058 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.058 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.058 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:17.058 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822986+0000", 2024-09-04T17:47:17.059 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:17.059 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.059 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.059 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:17.059 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822986+0000", 2024-09-04T17:47:17.059 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5600000000000002e-07 2024-09-04T17:47:17.059 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.059 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.059 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:17.059 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822987+0000", 2024-09-04T17:47:17.059 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.082e-06 2024-09-04T17:47:17.059 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.059 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.060 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:17.060 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822990+0000", 2024-09-04T17:47:17.060 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.2219999999999998e-06 2024-09-04T17:47:17.060 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.060 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.060 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:17.060 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822994+0000", 2024-09-04T17:47:17.060 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.19e-06 2024-09-04T17:47:17.060 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.060 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.060 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:17.060 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897632+0000", 2024-09-04T17:47:17.061 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074638205999999999 2024-09-04T17:47:17.061 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.061 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.061 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:17.061 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897672+0000", 2024-09-04T17:47:17.061 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.9873999999999998e-05 2024-09-04T17:47:17.061 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.061 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.061 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:17.061 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897722+0000", 2024-09-04T17:47:17.061 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0583e-05 2024-09-04T17:47:17.061 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:17.061 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:17.061 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:17.062 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.062 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.062 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60927 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 966656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:17.062 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823000+0000", 2024-09-04T17:47:17.062 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 3.2094230929999998, 2024-09-04T17:47:17.062 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074834036000000007, 2024-09-04T17:47:17.062 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:17.062 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:17.062 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:17.062 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:17.062 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:17.062 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:17.062 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60927 2024-09-04T17:47:17.062 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.063 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:17.063 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.063 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:17.063 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823000+0000", 2024-09-04T17:47:17.063 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:17.063 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.063 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.063 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:17.063 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823000+0000", 2024-09-04T17:47:17.063 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:17.063 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.063 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.064 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:17.064 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823000+0000", 2024-09-04T17:47:17.064 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4099999999999999e-07 2024-09-04T17:47:17.064 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.064 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.064 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:17.064 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823003+0000", 2024-09-04T17:47:17.064 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0810000000000001e-06 2024-09-04T17:47:17.064 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.064 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.064 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:17.065 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823011+0000", 2024-09-04T17:47:17.065 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7090000000000008e-06 2024-09-04T17:47:17.065 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.065 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.065 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:17.065 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823017+0000", 2024-09-04T17:47:17.065 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.3890000000000004e-06 2024-09-04T17:47:17.065 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.065 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.066 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:17.066 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897745+0000", 2024-09-04T17:47:17.066 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074728252999999994 2024-09-04T17:47:17.066 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.066 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.066 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:17.066 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897785+0000", 2024-09-04T17:47:17.067 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.9830999999999999e-05 2024-09-04T17:47:17.067 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.067 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.067 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:17.067 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897834+0000", 2024-09-04T17:47:17.067 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9231999999999999e-05 2024-09-04T17:47:17.067 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:17.067 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:17.068 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:17.068 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.068 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.068 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60928 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:17.068 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823024+0000", 2024-09-04T17:47:17.068 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 3.209399162, 2024-09-04T17:47:17.068 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075067479000000006, 2024-09-04T17:47:17.068 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:17.068 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:17.068 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:17.068 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:17.068 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:17.069 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:17.069 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60928 2024-09-04T17:47:17.069 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.069 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:17.069 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.069 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:17.069 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823024+0000", 2024-09-04T17:47:17.069 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:17.069 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.069 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.069 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:17.069 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823024+0000", 2024-09-04T17:47:17.069 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:17.070 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.070 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.070 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:17.070 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823024+0000", 2024-09-04T17:47:17.070 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6499999999999999e-07 2024-09-04T17:47:17.070 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.070 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.070 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:17.070 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823027+0000", 2024-09-04T17:47:17.070 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.3099999999999999e-06 2024-09-04T17:47:17.070 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.070 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.070 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:17.070 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823033+0000", 2024-09-04T17:47:17.071 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7429999999999996e-06 2024-09-04T17:47:17.071 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.071 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.071 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:17.071 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823038+0000", 2024-09-04T17:47:17.071 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.561e-06 2024-09-04T17:47:17.071 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.071 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.071 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:17.071 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897855+0000", 2024-09-04T17:47:17.071 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074816740000000007 2024-09-04T17:47:17.071 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.071 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.072 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:17.072 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897894+0000", 2024-09-04T17:47:17.072 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.9085999999999998e-05 2024-09-04T17:47:17.072 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.072 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.072 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:17.072 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898091+0000", 2024-09-04T17:47:17.072 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000197574 2024-09-04T17:47:17.072 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:17.072 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:17.072 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:17.073 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.073 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.073 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60929 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1212416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:17.073 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823042+0000", 2024-09-04T17:47:17.073 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 3.20938072, 2024-09-04T17:47:17.073 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075301251, 2024-09-04T17:47:17.073 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:17.073 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:17.073 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:17.073 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:17.073 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:17.073 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:17.073 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60929 2024-09-04T17:47:17.074 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.074 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:17.074 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.074 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:17.074 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823042+0000", 2024-09-04T17:47:17.074 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:17.074 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.074 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.074 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:17.074 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823042+0000", 2024-09-04T17:47:17.074 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:17.076 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.076 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.076 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:17.076 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823043+0000", 2024-09-04T17:47:17.077 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.1100000000000002e-07 2024-09-04T17:47:17.077 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.077 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.077 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:17.077 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823045+0000", 2024-09-04T17:47:17.077 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.933e-06 2024-09-04T17:47:17.077 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.077 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.077 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:17.077 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823051+0000", 2024-09-04T17:47:17.077 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.9569999999999999e-06 2024-09-04T17:47:17.077 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.077 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.078 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:17.078 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823055+0000", 2024-09-04T17:47:17.078 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4479999999999996e-06 2024-09-04T17:47:17.078 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.078 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.078 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:17.078 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898117+0000", 2024-09-04T17:47:17.078 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075062344000000003 2024-09-04T17:47:17.078 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.078 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.078 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:17.078 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898157+0000", 2024-09-04T17:47:17.078 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0095999999999997e-05 2024-09-04T17:47:17.079 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.079 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.079 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:17.079 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898344+0000", 2024-09-04T17:47:17.079 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000186162 2024-09-04T17:47:17.079 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:17.079 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:17.079 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:17.079 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.079 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.079 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60930 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1351680~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:17.079 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823061+0000", 2024-09-04T17:47:17.079 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 3.2093618409999998, 2024-09-04T17:47:17.080 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075545507999999997, 2024-09-04T17:47:17.080 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:17.080 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:17.080 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:17.080 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:17.080 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:17.080 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:17.080 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60930 2024-09-04T17:47:17.080 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.080 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:17.080 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.080 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:17.080 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823061+0000", 2024-09-04T17:47:17.080 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:17.081 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.081 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.081 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:17.081 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823061+0000", 2024-09-04T17:47:17.081 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:17.081 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.081 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.081 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:17.081 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823378+0000", 2024-09-04T17:47:17.081 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00031631299999999998 2024-09-04T17:47:17.081 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.082 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.082 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:17.082 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823381+0000", 2024-09-04T17:47:17.082 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4020000000000002e-06 2024-09-04T17:47:17.082 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.082 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.082 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:17.082 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823389+0000", 2024-09-04T17:47:17.082 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.1429999999999998e-06 2024-09-04T17:47:17.082 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.083 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.083 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:17.083 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823395+0000", 2024-09-04T17:47:17.083 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.0059999999999998e-06 2024-09-04T17:47:17.083 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.083 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.083 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:17.083 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898368+0000", 2024-09-04T17:47:17.083 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074973032999999994 2024-09-04T17:47:17.083 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.083 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.083 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:17.083 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898410+0000", 2024-09-04T17:47:17.084 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2243000000000002e-05 2024-09-04T17:47:17.084 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.084 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.084 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:17.084 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898607+0000", 2024-09-04T17:47:17.084 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019636799999999999 2024-09-04T17:47:17.084 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:17.084 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:17.084 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:17.084 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.084 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.084 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60931 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1458176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:17.084 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823407+0000", 2024-09-04T17:47:17.085 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 3.2090157669999999, 2024-09-04T17:47:17.085 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075460334000000004, 2024-09-04T17:47:17.085 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:17.085 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:17.085 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:17.085 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:17.085 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:17.085 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:17.085 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60931 2024-09-04T17:47:17.085 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.085 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:17.085 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.085 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:17.086 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823407+0000", 2024-09-04T17:47:17.086 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:17.086 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.086 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.086 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:17.086 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823407+0000", 2024-09-04T17:47:17.086 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:17.086 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.086 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.086 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:17.086 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823408+0000", 2024-09-04T17:47:17.086 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9999999999999998e-07 2024-09-04T17:47:17.086 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.086 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.087 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:17.087 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823415+0000", 2024-09-04T17:47:17.087 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.5510000000000002e-06 2024-09-04T17:47:17.087 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.087 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.087 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:17.087 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823423+0000", 2024-09-04T17:47:17.087 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2760000000000003e-06 2024-09-04T17:47:17.087 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.087 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.087 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:17.087 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823426+0000", 2024-09-04T17:47:17.087 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.2839999999999998e-06 2024-09-04T17:47:17.087 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.088 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.088 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:17.088 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898631+0000", 2024-09-04T17:47:17.088 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075205045999999998 2024-09-04T17:47:17.088 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.088 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.088 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:17.088 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898672+0000", 2024-09-04T17:47:17.088 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0614000000000001e-05 2024-09-04T17:47:17.088 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.088 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.088 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:17.089 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898868+0000", 2024-09-04T17:47:17.089 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019606300000000001 2024-09-04T17:47:17.089 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:17.089 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:17.089 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:17.089 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.089 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.089 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60932 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1564672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:17.089 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823434+0000", 2024-09-04T17:47:17.089 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 3.208989468, 2024-09-04T17:47:17.089 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075693393999999997, 2024-09-04T17:47:17.089 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:17.090 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:17.090 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:17.090 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:17.090 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:17.090 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:17.090 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60932 2024-09-04T17:47:17.090 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.090 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:17.090 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.090 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:17.090 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823434+0000", 2024-09-04T17:47:17.090 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:17.090 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.090 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.091 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:17.091 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823434+0000", 2024-09-04T17:47:17.091 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:17.091 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.091 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.091 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:17.091 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.830290+0000", 2024-09-04T17:47:17.091 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0068563039999999997 2024-09-04T17:47:17.091 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.091 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.091 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:17.091 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.830297+0000", 2024-09-04T17:47:17.091 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4760000000000001e-06 2024-09-04T17:47:17.091 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.092 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.092 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:17.092 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.830308+0000", 2024-09-04T17:47:17.092 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0927999999999999e-05 2024-09-04T17:47:17.092 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.092 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.092 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:17.092 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.830317+0000", 2024-09-04T17:47:17.092 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7280000000000001e-06 2024-09-04T17:47:17.092 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.092 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.092 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:17.092 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898892+0000", 2024-09-04T17:47:17.093 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.068574722000000005 2024-09-04T17:47:17.093 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.093 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.093 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:17.093 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898934+0000", 2024-09-04T17:47:17.093 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2249000000000002e-05 2024-09-04T17:47:17.093 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.093 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.093 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:17.093 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.899127+0000", 2024-09-04T17:47:17.093 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019298699999999999 2024-09-04T17:47:17.093 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:17.093 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:17.093 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:17.094 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.094 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.094 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61182 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 868352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:17.094 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839249+0000", 2024-09-04T17:47:17.094 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 3.193173576, 2024-09-04T17:47:17.094 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073680151999999999, 2024-09-04T17:47:17.094 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:17.094 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:17.094 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:17.094 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:17.094 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:17.094 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:17.094 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61182 2024-09-04T17:47:17.094 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.095 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:17.095 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.095 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:17.095 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839249+0000", 2024-09-04T17:47:17.095 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:17.095 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.095 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.095 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:17.095 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839249+0000", 2024-09-04T17:47:17.096 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:17.096 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.096 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.096 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:17.096 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839250+0000", 2024-09-04T17:47:17.096 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4999999999999998e-07 2024-09-04T17:47:17.096 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.096 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.096 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:17.096 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839257+0000", 2024-09-04T17:47:17.096 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7700000000000004e-06 2024-09-04T17:47:17.097 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.097 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.097 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:17.097 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839263+0000", 2024-09-04T17:47:17.097 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.246e-06 2024-09-04T17:47:17.097 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.097 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.097 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:17.097 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839268+0000", 2024-09-04T17:47:17.097 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.814e-06 2024-09-04T17:47:17.097 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.097 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.097 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:17.098 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.912831+0000", 2024-09-04T17:47:17.098 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073563294000000001 2024-09-04T17:47:17.098 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.098 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.098 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:17.098 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.912876+0000", 2024-09-04T17:47:17.098 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4742e-05 2024-09-04T17:47:17.098 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.098 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.098 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:17.098 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.912930+0000", 2024-09-04T17:47:17.098 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.3936e-05 2024-09-04T17:47:17.099 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:17.099 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:17.099 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:17.099 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.099 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.099 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61183 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:17.099 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839275+0000", 2024-09-04T17:47:17.099 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 3.193148383, 2024-09-04T17:47:17.099 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073781886000000005, 2024-09-04T17:47:17.099 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:17.099 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:17.099 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:17.100 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:17.100 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:17.100 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:17.100 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61183 2024-09-04T17:47:17.100 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.100 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:17.100 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.100 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:17.100 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839275+0000", 2024-09-04T17:47:17.100 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:17.100 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.100 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.101 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:17.101 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839275+0000", 2024-09-04T17:47:17.101 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:17.101 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.101 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.101 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:17.101 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839275+0000", 2024-09-04T17:47:17.101 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6899999999999997e-07 2024-09-04T17:47:17.101 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.101 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.101 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:17.101 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839277+0000", 2024-09-04T17:47:17.101 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9530000000000002e-06 2024-09-04T17:47:17.102 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.102 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.102 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:17.102 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839283+0000", 2024-09-04T17:47:17.102 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.9329999999999998e-06 2024-09-04T17:47:17.102 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.102 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.102 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:17.102 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839288+0000", 2024-09-04T17:47:17.102 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4100000000000001e-06 2024-09-04T17:47:17.102 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.102 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.102 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:17.103 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.912959+0000", 2024-09-04T17:47:17.103 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073671511999999995 2024-09-04T17:47:17.103 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.103 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.103 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:17.103 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913001+0000", 2024-09-04T17:47:17.103 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.1984999999999999e-05 2024-09-04T17:47:17.103 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.103 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.103 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:17.103 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913057+0000", 2024-09-04T17:47:17.103 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.5523999999999999e-05 2024-09-04T17:47:17.103 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:17.104 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:17.104 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:17.104 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.104 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.104 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61184 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1114112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:17.104 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839294+0000", 2024-09-04T17:47:17.104 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 3.1931291709999998, 2024-09-04T17:47:17.104 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073995263000000006, 2024-09-04T17:47:17.104 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:17.104 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:17.104 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:17.104 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:17.104 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:17.104 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:17.105 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61184 2024-09-04T17:47:17.105 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.105 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:17.105 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.105 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:17.105 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839294+0000", 2024-09-04T17:47:17.105 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:17.105 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.105 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.105 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:17.105 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839294+0000", 2024-09-04T17:47:17.105 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:17.105 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.106 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.106 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:17.106 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839294+0000", 2024-09-04T17:47:17.106 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.77e-07 2024-09-04T17:47:17.106 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.106 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.106 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:17.106 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839296+0000", 2024-09-04T17:47:17.106 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9999999999999999e-06 2024-09-04T17:47:17.106 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.106 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.106 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:17.107 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839302+0000", 2024-09-04T17:47:17.107 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7230000000000001e-06 2024-09-04T17:47:17.107 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.107 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.107 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:17.107 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839314+0000", 2024-09-04T17:47:17.107 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1826e-05 2024-09-04T17:47:17.107 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.107 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.107 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:17.107 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913084+0000", 2024-09-04T17:47:17.107 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073770215 2024-09-04T17:47:17.108 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.108 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.108 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:17.108 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913127+0000", 2024-09-04T17:47:17.108 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2982999999999998e-05 2024-09-04T17:47:17.108 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.108 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.108 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:17.108 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913289+0000", 2024-09-04T17:47:17.108 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000162139 2024-09-04T17:47:17.108 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:17.108 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:17.109 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:17.109 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.109 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.109 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61185 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1220608~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:17.109 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839321+0000", 2024-09-04T17:47:17.109 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 3.193101704, 2024-09-04T17:47:17.109 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074206062000000003, 2024-09-04T17:47:17.109 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:17.109 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:17.109 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:17.109 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:17.109 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:17.109 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:17.110 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61185 2024-09-04T17:47:17.110 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.110 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:17.110 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.110 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:17.110 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839321+0000", 2024-09-04T17:47:17.110 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:17.110 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.110 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.110 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:17.110 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839321+0000", 2024-09-04T17:47:17.110 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:17.110 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.111 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.111 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:17.111 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839322+0000", 2024-09-04T17:47:17.111 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.5400000000000001e-07 2024-09-04T17:47:17.111 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.111 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.111 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:17.111 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839324+0000", 2024-09-04T17:47:17.111 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2539999999999999e-06 2024-09-04T17:47:17.111 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.111 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.111 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:17.111 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839331+0000", 2024-09-04T17:47:17.111 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2139999999999999e-06 2024-09-04T17:47:17.112 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.112 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.112 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:17.112 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839336+0000", 2024-09-04T17:47:17.112 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7029999999999997e-06 2024-09-04T17:47:17.112 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.112 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.112 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:17.112 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913320+0000", 2024-09-04T17:47:17.112 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073984016 2024-09-04T17:47:17.112 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.112 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.112 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:17.113 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913366+0000", 2024-09-04T17:47:17.113 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.5704000000000002e-05 2024-09-04T17:47:17.113 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.113 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.113 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:17.113 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913527+0000", 2024-09-04T17:47:17.113 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00016161699999999999 2024-09-04T17:47:17.113 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:17.113 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:17.113 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:17.113 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.113 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.113 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61186 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1359872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:17.114 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839344+0000", 2024-09-04T17:47:17.114 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 3.1930793620000002, 2024-09-04T17:47:17.114 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074450673999999994, 2024-09-04T17:47:17.114 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:17.114 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:17.114 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:17.114 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:17.114 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:17.114 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:17.114 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61186 2024-09-04T17:47:17.114 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.114 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:17.114 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.115 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:17.115 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839344+0000", 2024-09-04T17:47:17.115 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:17.115 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.115 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.115 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:17.115 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839344+0000", 2024-09-04T17:47:17.115 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:17.115 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.115 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.115 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:17.115 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839344+0000", 2024-09-04T17:47:17.115 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4799999999999999e-07 2024-09-04T17:47:17.115 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.115 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.116 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:17.116 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839346+0000", 2024-09-04T17:47:17.116 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0889999999999998e-06 2024-09-04T17:47:17.116 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.116 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.116 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:17.116 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839352+0000", 2024-09-04T17:47:17.116 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6389999999999997e-06 2024-09-04T17:47:17.117 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.117 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.117 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:17.117 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839356+0000", 2024-09-04T17:47:17.117 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.233e-06 2024-09-04T17:47:17.117 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.117 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.118 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:17.118 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913557+0000", 2024-09-04T17:47:17.118 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074200944000000005 2024-09-04T17:47:17.118 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.118 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.118 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:17.118 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913601+0000", 2024-09-04T17:47:17.119 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4350000000000001e-05 2024-09-04T17:47:17.119 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.119 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.119 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:17.119 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913794+0000", 2024-09-04T17:47:17.119 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019297100000000001 2024-09-04T17:47:17.119 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:17.119 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:17.119 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:17.119 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.119 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.119 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61187 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1466368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:17.119 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839364+0000", 2024-09-04T17:47:17.120 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 3.1930595130000001, 2024-09-04T17:47:17.120 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074692334999999999, 2024-09-04T17:47:17.120 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:17.120 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:17.120 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:17.120 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:17.120 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:17.120 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:17.120 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61187 2024-09-04T17:47:17.120 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.120 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:17.120 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.120 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:17.121 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839364+0000", 2024-09-04T17:47:17.121 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:17.121 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.121 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.121 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:17.121 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839364+0000", 2024-09-04T17:47:17.121 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:17.121 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.121 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.121 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:17.121 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839364+0000", 2024-09-04T17:47:17.121 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.75e-07 2024-09-04T17:47:17.121 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.122 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.122 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:17.122 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839366+0000", 2024-09-04T17:47:17.122 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0990000000000001e-06 2024-09-04T17:47:17.122 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.122 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.122 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:17.122 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839372+0000", 2024-09-04T17:47:17.122 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.2609999999999999e-06 2024-09-04T17:47:17.122 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.122 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.122 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:17.122 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839389+0000", 2024-09-04T17:47:17.122 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6489000000000001e-05 2024-09-04T17:47:17.123 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.123 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.123 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:17.123 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913824+0000", 2024-09-04T17:47:17.123 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074434811000000004 2024-09-04T17:47:17.123 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.123 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.123 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:17.123 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913868+0000", 2024-09-04T17:47:17.123 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4768000000000002e-05 2024-09-04T17:47:17.123 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.123 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.123 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:17.123 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914056+0000", 2024-09-04T17:47:17.124 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00018743199999999999 2024-09-04T17:47:17.124 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:17.124 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:17.124 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:17.124 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.124 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.124 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61188 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1605632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:17.124 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839399+0000", 2024-09-04T17:47:17.124 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 3.1930236170000001, 2024-09-04T17:47:17.124 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074975095000000005, 2024-09-04T17:47:17.124 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:17.124 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:17.124 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:17.125 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:17.125 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:17.125 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:17.125 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61188 2024-09-04T17:47:17.125 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.125 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:17.125 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.125 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:17.125 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839399+0000", 2024-09-04T17:47:17.125 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:17.125 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.125 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.126 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:17.126 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839399+0000", 2024-09-04T17:47:17.126 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:17.126 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.126 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.126 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:17.126 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839400+0000", 2024-09-04T17:47:17.126 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0100000000000005e-07 2024-09-04T17:47:17.126 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.126 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.126 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:17.126 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839402+0000", 2024-09-04T17:47:17.126 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.491e-06 2024-09-04T17:47:17.126 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.127 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.127 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:17.127 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839409+0000", 2024-09-04T17:47:17.127 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.0450000000000003e-06 2024-09-04T17:47:17.127 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.127 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.127 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:17.127 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839414+0000", 2024-09-04T17:47:17.127 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.87e-06 2024-09-04T17:47:17.127 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.127 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.127 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:17.128 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914086+0000", 2024-09-04T17:47:17.128 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074671596000000007 2024-09-04T17:47:17.128 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.128 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.128 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:17.128 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914130+0000", 2024-09-04T17:47:17.128 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4579999999999997e-05 2024-09-04T17:47:17.128 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.128 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.128 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:17.128 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914375+0000", 2024-09-04T17:47:17.128 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024401199999999999 2024-09-04T17:47:17.129 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:17.129 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:17.129 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:17.129 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.129 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.129 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61189 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1712128~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:17.129 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839424+0000", 2024-09-04T17:47:17.129 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 3.1929994160000001, 2024-09-04T17:47:17.129 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075247680999999997, 2024-09-04T17:47:17.129 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:17.129 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:17.129 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:17.130 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:17.130 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:17.130 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:17.130 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61189 2024-09-04T17:47:17.130 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.130 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:17.130 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.130 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:17.130 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839424+0000", 2024-09-04T17:47:17.130 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:17.130 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.130 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.131 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:17.131 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839424+0000", 2024-09-04T17:47:17.131 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:17.131 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.131 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.131 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:17.131 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.846320+0000", 2024-09-04T17:47:17.131 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0068965210000000001 2024-09-04T17:47:17.131 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.131 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.131 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:17.131 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.846328+0000", 2024-09-04T17:47:17.131 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.8189999999999992e-06 2024-09-04T17:47:17.132 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.132 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.132 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:17.132 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.846341+0000", 2024-09-04T17:47:17.132 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3338e-05 2024-09-04T17:47:17.132 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.132 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.132 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:17.132 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.846352+0000", 2024-09-04T17:47:17.132 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0479000000000001e-05 2024-09-04T17:47:17.132 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.132 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.133 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:17.133 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914405+0000", 2024-09-04T17:47:17.133 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.068053049000000004 2024-09-04T17:47:17.133 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.133 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.133 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:17.133 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914451+0000", 2024-09-04T17:47:17.133 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6545999999999998e-05 2024-09-04T17:47:17.133 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:17.133 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:17.133 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:17.133 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914671+0000", 2024-09-04T17:47:17.134 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000219929 2024-09-04T17:47:17.134 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:17.134 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:17.134 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:17.134 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:17.134 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:17.134 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:17.136 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-04T17:47:17.136 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:17.132+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:17.236 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:17.236+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:17.272 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:17.273 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:47:17.273 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:47:17.273 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:17.288 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-04T17:47:17.337 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:17.336+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:17.409 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:17.410 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:47:17.410 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:47:17.410 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:47:17.410 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:17.425 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-04T17:47:17.438 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:47:17.436+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:47:17.454 INFO:teuthology.orchestra.run.smithi092.stdout:3934 read 0x409d9a8 thru 0x40aafdb (0xd634 bytes) 2024-09-04T17:47:17.470 INFO:teuthology.orchestra.run.smithi092.stdout:3935 trunc from 0xd653911 to 0x26ef45f 2024-09-04T17:47:17.539 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:17.534+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:17.547 INFO:teuthology.orchestra.run.smithi092.stdout:3936 punch from 0x345dfc to 0x34d3a2, (0x75a6 bytes) 2024-09-04T17:47:17.547 INFO:teuthology.orchestra.run.smithi092.stdout:3937 write 0x1d3baf6 thru 0x1d46fee (0xb4f9 bytes) 2024-09-04T17:47:17.549 INFO:teuthology.orchestra.run.smithi092.stdout:3938 punch from 0xd96f28 to 0xd9e1ec, (0x72c4 bytes) 2024-09-04T17:47:17.549 INFO:teuthology.orchestra.run.smithi092.stdout:3939 punch from 0xb75cdf to 0xb75e62, (0x183 bytes) 2024-09-04T17:47:17.549 INFO:teuthology.orchestra.run.smithi092.stdout:3940 read 0x58000d thru 0x58247e (0x2472 bytes) 2024-09-04T17:47:17.550 INFO:teuthology.orchestra.run.smithi092.stdout:3941 punch from 0x87eede to 0x88655c, (0x767e bytes) 2024-09-04T17:47:17.550 INFO:teuthology.orchestra.run.smithi092.stdout:3942 read 0x1b756a1 thru 0x1b83057 (0xd9b7 bytes) 2024-09-04T17:47:17.550 INFO:teuthology.orchestra.run.smithi092.stdout:3945 read 0xec3e6e thru 0xed3625 (0xf7b8 bytes) 2024-09-04T17:47:17.551 INFO:teuthology.orchestra.run.smithi092.stdout:3946 write 0x945c4b thru 0x951237 (0xb5ed bytes) 2024-09-04T17:47:17.564 INFO:teuthology.orchestra.run.smithi092.stdout:3947 trunc from 0x26ef45f to 0xa4bc3d4 2024-09-04T17:47:17.566 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:17.567 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:47:17.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:47:17.567 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:47:17.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.567 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:27242 3.b 3.34414deb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-04T17:47:17.567 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:28.402347+0000", 2024-09-04T17:47:17.567 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 49.163974412000002, 2024-09-04T17:47:17.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.93265329299999999, 2024-09-04T17:47:17.567 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:17.567 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:17.567 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-04T17:47:17.568 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:17.568 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:17.568 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:17.568 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 27242 2024-09-04T17:47:17.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.568 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:17.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:17.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402347+0000", 2024-09-04T17:47:17.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:17.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:17.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402347+0000", 2024-09-04T17:47:17.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:17.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:17.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402349+0000", 2024-09-04T17:47:17.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.471e-06 2024-09-04T17:47:17.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:17.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402355+0000", 2024-09-04T17:47:17.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8399999999999997e-06 2024-09-04T17:47:17.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:17.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402364+0000", 2024-09-04T17:47:17.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1269999999999992e-06 2024-09-04T17:47:17.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:17.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402373+0000", 2024-09-04T17:47:17.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7909999999999993e-06 2024-09-04T17:47:17.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:17.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402460+0000", 2024-09-04T17:47:17.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6686000000000003e-05 2024-09-04T17:47:17.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-04T17:47:17.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402473+0000", 2024-09-04T17:47:17.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3281e-05 2024-09-04T17:47:17.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:17.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:29.334989+0000", 2024-09-04T17:47:17.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.93251549099999997 2024-09-04T17:47:17.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:17.572 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:29.335000+0000", 2024-09-04T17:47:17.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1606e-05 2024-09-04T17:47:17.572 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:17.572 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:17.572 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:17.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.572 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.572 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:33552 3.7 3:e79ad1db:::rbd_data.11342e8f3384.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-04T17:47:17.572 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:35.999748+0000", 2024-09-04T17:47:17.572 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 41.566573362, 2024-09-04T17:47:17.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045683823999999998, 2024-09-04T17:47:17.572 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:17.573 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:17.573 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:17.573 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:17.573 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:17.573 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:17.573 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 33552 2024-09-04T17:47:17.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.573 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:17.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.573 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:17.573 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999748+0000", 2024-09-04T17:47:17.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:17.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:17.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999748+0000", 2024-09-04T17:47:17.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:17.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:17.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999750+0000", 2024-09-04T17:47:17.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1299999999999999e-06 2024-09-04T17:47:17.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:17.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999759+0000", 2024-09-04T17:47:17.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4649999999999994e-06 2024-09-04T17:47:17.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:17.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999778+0000", 2024-09-04T17:47:17.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8833000000000001e-05 2024-09-04T17:47:17.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:17.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999786+0000", 2024-09-04T17:47:17.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7369999999999995e-06 2024-09-04T17:47:17.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.576 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:17.576 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999856+0000", 2024-09-04T17:47:17.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9247999999999998e-05 2024-09-04T17:47:17.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.576 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:17.576 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999973+0000", 2024-09-04T17:47:17.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000117921 2024-09-04T17:47:17.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 7", 2024-09-04T17:47:17.578 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.005955+0000", 2024-09-04T17:47:17.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059820560000000004 2024-09-04T17:47:17.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:17.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.045297+0000", 2024-09-04T17:47:17.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.039341650999999998 2024-09-04T17:47:17.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:17.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.045343+0000", 2024-09-04T17:47:17.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5899000000000002e-05 2024-09-04T17:47:17.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:17.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.045388+0000", 2024-09-04T17:47:17.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4601999999999997e-05 2024-09-04T17:47:17.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:17.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.045432+0000", 2024-09-04T17:47:17.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4282000000000001e-05 2024-09-04T17:47:17.580 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:17.580 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:17.580 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:17.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.580 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:37357 3.15 3:a897312a:::rbd_data.1134c5a43465.000000000000001f:head [stat,write 62464~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-04T17:47:17.580 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:39.616724+0000", 2024-09-04T17:47:17.580 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.949597953999998, 2024-09-04T17:47:17.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054965854000000001, 2024-09-04T17:47:17.581 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:17.581 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:17.581 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:17.581 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:17.581 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:17.581 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:17.581 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 37357 2024-09-04T17:47:17.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.581 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:17.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.581 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:17.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616724+0000", 2024-09-04T17:47:17.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:17.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:17.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616724+0000", 2024-09-04T17:47:17.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:17.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:17.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616725+0000", 2024-09-04T17:47:17.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.815e-06 2024-09-04T17:47:17.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.583 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:17.583 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616739+0000", 2024-09-04T17:47:17.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3723000000000001e-05 2024-09-04T17:47:17.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.583 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:17.583 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616755+0000", 2024-09-04T17:47:17.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5964000000000002e-05 2024-09-04T17:47:17.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.583 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:17.583 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616763+0000", 2024-09-04T17:47:17.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3869999999999995e-06 2024-09-04T17:47:17.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:17.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616825+0000", 2024-09-04T17:47:17.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1854999999999996e-05 2024-09-04T17:47:17.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:17.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616895+0000", 2024-09-04T17:47:17.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9597999999999993e-05 2024-09-04T17:47:17.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-04T17:47:17.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616991+0000", 2024-09-04T17:47:17.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5828000000000005e-05 2024-09-04T17:47:17.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:17.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.617659+0000", 2024-09-04T17:47:17.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00066821600000000001 2024-09-04T17:47:17.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:17.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.671482+0000", 2024-09-04T17:47:17.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053823533999999999 2024-09-04T17:47:17.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:17.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.671579+0000", 2024-09-04T17:47:17.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6657999999999995e-05 2024-09-04T17:47:17.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:17.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.671689+0000", 2024-09-04T17:47:17.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000110276 2024-09-04T17:47:17.586 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:17.586 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:17.586 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:17.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.587 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:43873 3.15 3:abd207ee:::rbd_data.113410ddd206.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:17.587 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:49.932940+0000", 2024-09-04T17:47:17.587 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.633381401000001, 2024-09-04T17:47:17.587 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042740514, 2024-09-04T17:47:17.587 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:17.587 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:17.587 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:17.587 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:17.587 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:17.587 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:17.587 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 43873 2024-09-04T17:47:17.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.588 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:17.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:17.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.932940+0000", 2024-09-04T17:47:17.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:17.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:17.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.932940+0000", 2024-09-04T17:47:17.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:17.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:17.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.932941+0000", 2024-09-04T17:47:17.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.404e-06 2024-09-04T17:47:17.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:17.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.933065+0000", 2024-09-04T17:47:17.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012361199999999999 2024-09-04T17:47:17.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:17.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.933078+0000", 2024-09-04T17:47:17.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3182e-05 2024-09-04T17:47:17.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:17.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.933085+0000", 2024-09-04T17:47:17.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4500000000000001e-06 2024-09-04T17:47:17.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:17.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.933134+0000", 2024-09-04T17:47:17.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9150999999999997e-05 2024-09-04T17:47:17.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.591 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:17.591 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.933242+0000", 2024-09-04T17:47:17.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010781 2024-09-04T17:47:17.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.591 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-04T17:47:17.591 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.938358+0000", 2024-09-04T17:47:17.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051160349999999997 2024-09-04T17:47:17.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.592 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.592 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:17.592 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.975343+0000", 2024-09-04T17:47:17.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036985663000000002 2024-09-04T17:47:17.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.592 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.592 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:17.592 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.975392+0000", 2024-09-04T17:47:17.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9044e-05 2024-09-04T17:47:17.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.592 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.592 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:17.592 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.975548+0000", 2024-09-04T17:47:17.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015572900000000001 2024-09-04T17:47:17.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:17.593 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.975681+0000", 2024-09-04T17:47:17.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013243400000000001 2024-09-04T17:47:17.593 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:17.593 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:17.593 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:17.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.594 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:46649 3.7 3:e0749ed5:::rbd_data.1134326b938e.000000000000014f:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-04T17:47:17.594 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:54.114730+0000", 2024-09-04T17:47:17.594 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 23.451591269000001, 2024-09-04T17:47:17.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032972972000000003, 2024-09-04T17:47:17.594 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:17.594 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:17.594 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:17.594 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:17.594 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:17.594 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:17.594 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 46649 2024-09-04T17:47:17.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.594 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:17.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:17.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114730+0000", 2024-09-04T17:47:17.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:17.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:17.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114730+0000", 2024-09-04T17:47:17.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:17.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.596 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:17.596 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114731+0000", 2024-09-04T17:47:17.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.147e-06 2024-09-04T17:47:17.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.596 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:17.596 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114737+0000", 2024-09-04T17:47:17.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6380000000000001e-06 2024-09-04T17:47:17.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.596 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:17.597 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114751+0000", 2024-09-04T17:47:17.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4012999999999999e-05 2024-09-04T17:47:17.597 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.597 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:17.597 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114758+0000", 2024-09-04T17:47:17.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1539999999999996e-06 2024-09-04T17:47:17.597 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:17.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114876+0000", 2024-09-04T17:47:17.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000117421 2024-09-04T17:47:17.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:17.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.115173+0000", 2024-09-04T17:47:17.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00029750799999999998 2024-09-04T17:47:17.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 7", 2024-09-04T17:47:17.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.120273+0000", 2024-09-04T17:47:17.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051002900000000004 2024-09-04T17:47:17.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:17.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.147571+0000", 2024-09-04T17:47:17.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027297906 2024-09-04T17:47:17.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:17.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.147588+0000", 2024-09-04T17:47:17.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6807999999999999e-05 2024-09-04T17:47:17.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:17.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.147637+0000", 2024-09-04T17:47:17.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8466e-05 2024-09-04T17:47:17.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:17.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.147703+0000", 2024-09-04T17:47:17.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6620999999999998e-05 2024-09-04T17:47:17.600 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:17.600 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:17.601 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:17.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.601 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49801 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 720896~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:17.601 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.075094+0000", 2024-09-04T17:47:17.601 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.491227282000001, 2024-09-04T17:47:17.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017824974, 2024-09-04T17:47:17.601 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:17.601 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:17.601 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:17.601 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:17.601 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:17.601 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:17.602 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49801 2024-09-04T17:47:17.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.602 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:17.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:17.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075094+0000", 2024-09-04T17:47:17.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:17.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:17.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075094+0000", 2024-09-04T17:47:17.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:17.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:17.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075095+0000", 2024-09-04T17:47:17.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.017e-06 2024-09-04T17:47:17.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:17.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075101+0000", 2024-09-04T17:47:17.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5029999999999997e-06 2024-09-04T17:47:17.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:17.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075119+0000", 2024-09-04T17:47:17.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8133000000000001e-05 2024-09-04T17:47:17.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:17.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075132+0000", 2024-09-04T17:47:17.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3305999999999999e-05 2024-09-04T17:47:17.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:17.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.092677+0000", 2024-09-04T17:47:17.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017544466000000002 2024-09-04T17:47:17.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:17.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.092782+0000", 2024-09-04T17:47:17.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000105284 2024-09-04T17:47:17.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:17.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.092919+0000", 2024-09-04T17:47:17.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013726499999999999 2024-09-04T17:47:17.605 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:17.605 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:17.605 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:17.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.605 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49809 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 163840~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:17.605 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077146+0000", 2024-09-04T17:47:17.606 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.489175839, 2024-09-04T17:47:17.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017984541999999999, 2024-09-04T17:47:17.606 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:17.606 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:17.606 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:17.606 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:17.606 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:17.606 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:17.606 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49809 2024-09-04T17:47:17.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.606 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:17.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:17.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077146+0000", 2024-09-04T17:47:17.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:17.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:17.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077146+0000", 2024-09-04T17:47:17.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:17.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:17.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077147+0000", 2024-09-04T17:47:17.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0330000000000001e-06 2024-09-04T17:47:17.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:17.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077164+0000", 2024-09-04T17:47:17.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7810000000000001e-05 2024-09-04T17:47:17.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:17.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077187+0000", 2024-09-04T17:47:17.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2246e-05 2024-09-04T17:47:17.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:17.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077203+0000", 2024-09-04T17:47:17.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6392000000000002e-05 2024-09-04T17:47:17.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:17.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.094885+0000", 2024-09-04T17:47:17.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017681589000000001 2024-09-04T17:47:17.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:17.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.094991+0000", 2024-09-04T17:47:17.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010668899999999999 2024-09-04T17:47:17.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:17.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095130+0000", 2024-09-04T17:47:17.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013878299999999999 2024-09-04T17:47:17.610 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:17.610 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:17.610 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:17.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.610 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49810 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 229376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:17.610 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077225+0000", 2024-09-04T17:47:17.610 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.489095998, 2024-09-04T17:47:17.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018218793, 2024-09-04T17:47:17.610 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:17.610 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:17.610 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:17.610 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:17.610 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:17.611 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:17.611 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49810 2024-09-04T17:47:17.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.611 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:17.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:17.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077225+0000", 2024-09-04T17:47:17.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:17.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:17.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077225+0000", 2024-09-04T17:47:17.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:17.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:17.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077227+0000", 2024-09-04T17:47:17.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1829999999999999e-06 2024-09-04T17:47:17.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:17.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077233+0000", 2024-09-04T17:47:17.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3899999999999998e-06 2024-09-04T17:47:17.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:17.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077253+0000", 2024-09-04T17:47:17.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0009000000000001e-05 2024-09-04T17:47:17.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.613 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:17.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077270+0000", 2024-09-04T17:47:17.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7336000000000001e-05 2024-09-04T17:47:17.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.613 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:17.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095177+0000", 2024-09-04T17:47:17.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017907101000000002 2024-09-04T17:47:17.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.613 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:17.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095299+0000", 2024-09-04T17:47:17.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000121749 2024-09-04T17:47:17.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:17.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095444+0000", 2024-09-04T17:47:17.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014502499999999999 2024-09-04T17:47:17.614 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:17.614 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:17.614 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:17.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.614 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49811 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 294912~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:17.615 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077288+0000", 2024-09-04T17:47:17.615 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.489033684000001, 2024-09-04T17:47:17.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018443620000000001, 2024-09-04T17:47:17.615 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:17.615 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:17.615 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:17.615 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:17.615 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:17.615 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:17.615 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49811 2024-09-04T17:47:17.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.615 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:17.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:17.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077288+0000", 2024-09-04T17:47:17.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:17.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:17.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077288+0000", 2024-09-04T17:47:17.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:17.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:17.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077289+0000", 2024-09-04T17:47:17.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6199999999999999e-06 2024-09-04T17:47:17.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.617 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:17.617 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077295+0000", 2024-09-04T17:47:17.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1499999999999998e-06 2024-09-04T17:47:17.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.617 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:17.617 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077312+0000", 2024-09-04T17:47:17.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7353000000000001e-05 2024-09-04T17:47:17.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.617 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:17.617 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077324+0000", 2024-09-04T17:47:17.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2593e-05 2024-09-04T17:47:17.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.618 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:17.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095490+0000", 2024-09-04T17:47:17.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018165976 2024-09-04T17:47:17.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.618 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:17.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095597+0000", 2024-09-04T17:47:17.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010629900000000001 2024-09-04T17:47:17.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.618 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:17.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095731+0000", 2024-09-04T17:47:17.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013462899999999999 2024-09-04T17:47:17.619 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:17.619 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:17.619 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:17.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.619 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49813 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 360448~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:17.619 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077349+0000", 2024-09-04T17:47:17.619 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.488972481999999, 2024-09-04T17:47:17.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018780305000000001, 2024-09-04T17:47:17.619 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:17.619 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:17.620 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:17.620 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:17.620 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:17.620 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:17.620 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49813 2024-09-04T17:47:17.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.620 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:17.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:17.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077349+0000", 2024-09-04T17:47:17.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:17.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:17.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077349+0000", 2024-09-04T17:47:17.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:17.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:17.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077350+0000", 2024-09-04T17:47:17.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0330000000000001e-06 2024-09-04T17:47:17.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:17.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077355+0000", 2024-09-04T17:47:17.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0699999999999997e-06 2024-09-04T17:47:17.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.622 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:17.622 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077376+0000", 2024-09-04T17:47:17.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0846e-05 2024-09-04T17:47:17.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.622 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:17.622 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077388+0000", 2024-09-04T17:47:17.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2353e-05 2024-09-04T17:47:17.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.622 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:17.622 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095819+0000", 2024-09-04T17:47:17.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01843057 2024-09-04T17:47:17.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.623 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:17.623 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095960+0000", 2024-09-04T17:47:17.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000140965 2024-09-04T17:47:17.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.623 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:17.623 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096129+0000", 2024-09-04T17:47:17.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016946800000000001 2024-09-04T17:47:17.623 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:17.623 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:17.623 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:17.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.624 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49814 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 425984~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:17.624 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077407+0000", 2024-09-04T17:47:17.624 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.488913981, 2024-09-04T17:47:17.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019041563000000001, 2024-09-04T17:47:17.624 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:17.624 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:17.624 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:17.624 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:17.625 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:17.625 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:17.625 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49814 2024-09-04T17:47:17.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.625 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:17.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:17.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077407+0000", 2024-09-04T17:47:17.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:17.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:17.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077407+0000", 2024-09-04T17:47:17.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:17.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:17.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077409+0000", 2024-09-04T17:47:17.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2330000000000001e-06 2024-09-04T17:47:17.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:17.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077415+0000", 2024-09-04T17:47:17.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2729999999999999e-06 2024-09-04T17:47:17.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:17.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077436+0000", 2024-09-04T17:47:17.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1209999999999999e-05 2024-09-04T17:47:17.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:17.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077448+0000", 2024-09-04T17:47:17.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2225999999999999e-05 2024-09-04T17:47:17.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:17.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096183+0000", 2024-09-04T17:47:17.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018735004 2024-09-04T17:47:17.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:17.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096297+0000", 2024-09-04T17:47:17.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000113962 2024-09-04T17:47:17.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:17.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096449+0000", 2024-09-04T17:47:17.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000151655 2024-09-04T17:47:17.628 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:17.628 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:17.629 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:17.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.629 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49815 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 491520~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:17.629 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077471+0000", 2024-09-04T17:47:17.629 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.488850743, 2024-09-04T17:47:17.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019266536000000001, 2024-09-04T17:47:17.629 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:17.629 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:17.629 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:17.629 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:17.629 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:17.629 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:17.630 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49815 2024-09-04T17:47:17.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.630 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:17.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:17.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077471+0000", 2024-09-04T17:47:17.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:17.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:17.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077471+0000", 2024-09-04T17:47:17.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:17.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:17.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077472+0000", 2024-09-04T17:47:17.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.17e-06 2024-09-04T17:47:17.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:17.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077478+0000", 2024-09-04T17:47:17.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0399999999999998e-06 2024-09-04T17:47:17.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:17.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077500+0000", 2024-09-04T17:47:17.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1959000000000001e-05 2024-09-04T17:47:17.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:17.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077513+0000", 2024-09-04T17:47:17.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2823000000000001e-05 2024-09-04T17:47:17.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:17.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096493+0000", 2024-09-04T17:47:17.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018980701999999999 2024-09-04T17:47:17.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:17.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096603+0000", 2024-09-04T17:47:17.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010942000000000001 2024-09-04T17:47:17.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:17.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096737+0000", 2024-09-04T17:47:17.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013442200000000001 2024-09-04T17:47:17.633 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:17.633 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:17.633 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:17.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.633 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49816 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 557056~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:17.633 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077532+0000", 2024-09-04T17:47:17.634 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.488789002000001, 2024-09-04T17:47:17.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019604331999999999, 2024-09-04T17:47:17.634 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:17.634 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:17.634 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:17.634 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:17.634 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:17.634 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:17.634 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49816 2024-09-04T17:47:17.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.634 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:17.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:17.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077532+0000", 2024-09-04T17:47:17.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:17.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:17.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077532+0000", 2024-09-04T17:47:17.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:17.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:17.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077534+0000", 2024-09-04T17:47:17.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1200000000000001e-06 2024-09-04T17:47:17.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:17.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077545+0000", 2024-09-04T17:47:17.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1673e-05 2024-09-04T17:47:17.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:17.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077569+0000", 2024-09-04T17:47:17.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3555999999999999e-05 2024-09-04T17:47:17.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:17.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077584+0000", 2024-09-04T17:47:17.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4883e-05 2024-09-04T17:47:17.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:17.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096825+0000", 2024-09-04T17:47:17.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019241734 2024-09-04T17:47:17.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:17.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096964+0000", 2024-09-04T17:47:17.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000138085 2024-09-04T17:47:17.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:17.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097137+0000", 2024-09-04T17:47:17.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017328100000000001 2024-09-04T17:47:17.638 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:17.638 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:17.638 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:17.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.638 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49817 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 622592~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:17.638 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077603+0000", 2024-09-04T17:47:17.638 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.488718444, 2024-09-04T17:47:17.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01986928, 2024-09-04T17:47:17.639 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:17.639 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:17.639 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:17.639 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:17.639 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:17.639 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:17.639 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49817 2024-09-04T17:47:17.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.639 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:17.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:17.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077603+0000", 2024-09-04T17:47:17.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:17.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:17.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077603+0000", 2024-09-04T17:47:17.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:17.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:17.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077604+0000", 2024-09-04T17:47:17.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7699999999999992e-07 2024-09-04T17:47:17.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:17.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077609+0000", 2024-09-04T17:47:17.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0830000000000003e-06 2024-09-04T17:47:17.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:17.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077630+0000", 2024-09-04T17:47:17.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0438999999999999e-05 2024-09-04T17:47:17.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:17.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077664+0000", 2024-09-04T17:47:17.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4579000000000002e-05 2024-09-04T17:47:17.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:17.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097193+0000", 2024-09-04T17:47:17.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019529174 2024-09-04T17:47:17.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:17.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097323+0000", 2024-09-04T17:47:17.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000129696 2024-09-04T17:47:17.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:17.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097472+0000", 2024-09-04T17:47:17.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014933200000000001 2024-09-04T17:47:17.643 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:17.643 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:17.643 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:17.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.643 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49818 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 688128~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:17.643 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077683+0000", 2024-09-04T17:47:17.643 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.488638079999999, 2024-09-04T17:47:17.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020086333000000001, 2024-09-04T17:47:17.643 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:17.643 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:17.643 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:17.644 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:17.644 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:17.644 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:17.644 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49818 2024-09-04T17:47:17.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.644 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:17.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.644 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:17.644 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077683+0000", 2024-09-04T17:47:17.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:17.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.644 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:17.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077683+0000", 2024-09-04T17:47:17.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:17.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:17.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077684+0000", 2024-09-04T17:47:17.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4e-07 2024-09-04T17:47:17.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:17.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077690+0000", 2024-09-04T17:47:17.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4729999999999999e-06 2024-09-04T17:47:17.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:17.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077710+0000", 2024-09-04T17:47:17.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0149999999999999e-05 2024-09-04T17:47:17.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:17.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077723+0000", 2024-09-04T17:47:17.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3271999999999999e-05 2024-09-04T17:47:17.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:17.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097521+0000", 2024-09-04T17:47:17.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019797981999999999 2024-09-04T17:47:17.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.647 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:17.647 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097628+0000", 2024-09-04T17:47:17.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010665700000000001 2024-09-04T17:47:17.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.647 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:17.647 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097770+0000", 2024-09-04T17:47:17.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014185900000000001 2024-09-04T17:47:17.647 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:17.647 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:17.647 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:17.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.648 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49820 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 753664~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:17.648 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077744+0000", 2024-09-04T17:47:17.648 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.488577148999999, 2024-09-04T17:47:17.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020364786999999999, 2024-09-04T17:47:17.648 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:17.648 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:17.648 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:17.648 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:17.648 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:17.648 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:17.648 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49820 2024-09-04T17:47:17.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.648 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:17.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.649 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:17.649 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077744+0000", 2024-09-04T17:47:17.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:17.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.649 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:17.649 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077744+0000", 2024-09-04T17:47:17.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:17.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.649 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:17.649 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077745+0000", 2024-09-04T17:47:17.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0270000000000001e-06 2024-09-04T17:47:17.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:17.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077751+0000", 2024-09-04T17:47:17.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.626e-06 2024-09-04T17:47:17.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:17.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077771+0000", 2024-09-04T17:47:17.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0469999999999999e-05 2024-09-04T17:47:17.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:17.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077785+0000", 2024-09-04T17:47:17.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3129000000000001e-05 2024-09-04T17:47:17.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:17.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097816+0000", 2024-09-04T17:47:17.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020031917999999999 2024-09-04T17:47:17.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:17.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097939+0000", 2024-09-04T17:47:17.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000122533 2024-09-04T17:47:17.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:17.652 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098109+0000", 2024-09-04T17:47:17.652 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017008400000000001 2024-09-04T17:47:17.652 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:17.652 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:17.652 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:17.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.652 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49821 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 819200~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:17.652 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077807+0000", 2024-09-04T17:47:17.652 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.488514937, 2024-09-04T17:47:17.652 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020702928999999998, 2024-09-04T17:47:17.652 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:17.652 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:17.652 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:17.653 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:17.653 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:17.653 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:17.653 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49821 2024-09-04T17:47:17.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.653 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:17.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:17.653 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077807+0000", 2024-09-04T17:47:17.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:17.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:17.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077807+0000", 2024-09-04T17:47:17.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:17.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:17.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077808+0000", 2024-09-04T17:47:17.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.463e-06 2024-09-04T17:47:17.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:17.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077813+0000", 2024-09-04T17:47:17.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7530000000000001e-06 2024-09-04T17:47:17.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:17.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077832+0000", 2024-09-04T17:47:17.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.946e-05 2024-09-04T17:47:17.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:17.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077846+0000", 2024-09-04T17:47:17.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3786e-05 2024-09-04T17:47:17.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:17.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098201+0000", 2024-09-04T17:47:17.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020354797000000001 2024-09-04T17:47:17.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:17.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098341+0000", 2024-09-04T17:47:17.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000140042 2024-09-04T17:47:17.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:17.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098509+0000", 2024-09-04T17:47:17.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016862800000000001 2024-09-04T17:47:17.656 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:17.656 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:17.656 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:17.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.657 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49823 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 884736~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:17.657 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077865+0000", 2024-09-04T17:47:17.657 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.488456492999999, 2024-09-04T17:47:17.657 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021008723, 2024-09-04T17:47:17.657 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:17.657 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:17.657 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:17.657 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:17.657 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:17.657 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:17.657 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49823 2024-09-04T17:47:17.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.657 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:17.658 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.658 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:17.658 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077865+0000", 2024-09-04T17:47:17.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:17.658 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.658 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.658 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:17.658 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077865+0000", 2024-09-04T17:47:17.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:17.658 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.658 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.658 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:17.658 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077866+0000", 2024-09-04T17:47:17.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.164e-06 2024-09-04T17:47:17.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.659 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.659 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:17.659 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077872+0000", 2024-09-04T17:47:17.659 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3959999999999997e-06 2024-09-04T17:47:17.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.659 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.659 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:17.659 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077892+0000", 2024-09-04T17:47:17.659 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0970000000000001e-05 2024-09-04T17:47:17.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.659 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.659 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:17.660 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077905+0000", 2024-09-04T17:47:17.660 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2312e-05 2024-09-04T17:47:17.660 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.660 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:17.660 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098579+0000", 2024-09-04T17:47:17.660 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020674486999999998 2024-09-04T17:47:17.660 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.660 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:17.660 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098705+0000", 2024-09-04T17:47:17.660 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012541900000000001 2024-09-04T17:47:17.660 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.661 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.661 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:17.661 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098874+0000", 2024-09-04T17:47:17.661 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000168975 2024-09-04T17:47:17.661 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:17.661 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:17.661 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:17.661 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.661 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.661 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49824 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 950272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:17.661 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077923+0000", 2024-09-04T17:47:17.661 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.488398801000001, 2024-09-04T17:47:17.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021246287999999999, 2024-09-04T17:47:17.662 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:17.662 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:17.662 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:17.662 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:17.662 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:17.662 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:17.662 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49824 2024-09-04T17:47:17.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.662 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:17.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.662 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:17.662 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077923+0000", 2024-09-04T17:47:17.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:17.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:17.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077923+0000", 2024-09-04T17:47:17.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:17.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:17.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077924+0000", 2024-09-04T17:47:17.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0100000000000001e-06 2024-09-04T17:47:17.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:17.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077944+0000", 2024-09-04T17:47:17.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0449e-05 2024-09-04T17:47:17.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:17.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077965+0000", 2024-09-04T17:47:17.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0479000000000002e-05 2024-09-04T17:47:17.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:17.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077990+0000", 2024-09-04T17:47:17.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5582999999999999e-05 2024-09-04T17:47:17.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.665 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:17.665 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098924+0000", 2024-09-04T17:47:17.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020933395 2024-09-04T17:47:17.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.665 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:17.665 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.099032+0000", 2024-09-04T17:47:17.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010846299999999999 2024-09-04T17:47:17.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.665 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:17.665 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.099169+0000", 2024-09-04T17:47:17.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000136909 2024-09-04T17:47:17.665 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:17.666 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:17.666 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:17.666 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.666 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49826 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 1015808~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:17.666 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.078010+0000", 2024-09-04T17:47:17.666 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.488311194, 2024-09-04T17:47:17.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021584564000000001, 2024-09-04T17:47:17.666 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:17.666 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:17.666 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:17.666 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:17.666 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:17.666 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:17.667 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49826 2024-09-04T17:47:17.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.667 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:17.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.667 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:17.667 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078010+0000", 2024-09-04T17:47:17.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:17.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.667 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:17.667 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078010+0000", 2024-09-04T17:47:17.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:17.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.668 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:17.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078011+0000", 2024-09-04T17:47:17.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1340000000000001e-06 2024-09-04T17:47:17.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.668 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:17.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078017+0000", 2024-09-04T17:47:17.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.682e-06 2024-09-04T17:47:17.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.668 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:17.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078036+0000", 2024-09-04T17:47:17.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9357000000000001e-05 2024-09-04T17:47:17.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.669 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:17.669 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078050+0000", 2024-09-04T17:47:17.669 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3122000000000001e-05 2024-09-04T17:47:17.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.669 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:17.669 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.099271+0000", 2024-09-04T17:47:17.669 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021221772999999999 2024-09-04T17:47:17.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.669 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:17.669 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.099410+0000", 2024-09-04T17:47:17.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000138415 2024-09-04T17:47:17.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:17.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:17.670 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:17.670 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.099595+0000", 2024-09-04T17:47:17.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000185081 2024-09-04T17:47:17.670 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:17.670 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:17.670 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:17.670 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:17.670 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:17.670 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:17.671 INFO:teuthology.orchestra.run.smithi092.stdout:3948 write 0x3a26a39 thru 0x3a274f4 (0xabc bytes) 2024-09-04T17:47:17.672 INFO:teuthology.orchestra.run.smithi092.stdout:3949 write 0x365e0e7 thru 0x3663424 (0x533e bytes) 2024-09-04T17:47:17.672 INFO:teuthology.orchestra.run.smithi092.stdout:3950 read 0x6e9db13 thru 0x6ea732e (0x981c bytes) 2024-09-04T17:47:17.673 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-04T17:47:17.674 INFO:teuthology.orchestra.run.smithi092.stdout:3951 trunc from 0xa4bc3d4 to 0xc5f2a66 2024-09-04T17:47:17.674 INFO:teuthology.orchestra.run.smithi092.stdout:3953 punch from 0x2edfa19 to 0x2eee61d, (0xec04 bytes) 2024-09-04T17:47:17.674 INFO:teuthology.orchestra.run.smithi092.stdout:3954 read 0x47c3c4a thru 0x47c7e40 (0x41f7 bytes) 2024-09-04T17:47:17.674 INFO:teuthology.orchestra.run.smithi092.stdout:3958 writesame 0x3e9d40b thru 0x3ea7391 data_size 0x1(0x9f87 bytes) 2024-09-04T17:47:17.675 INFO:teuthology.orchestra.run.smithi092.stdout:3959 write 0x2da8446 thru 0x2dad8bc (0x5477 bytes) 2024-09-04T17:47:17.675 INFO:teuthology.orchestra.run.smithi092.stdout:3960 write 0x130687f thru 0x130aa27 (0x41a9 bytes) 2024-09-04T17:47:17.675 INFO:teuthology.orchestra.run.smithi092.stdout:3961 read 0xa3dad32 thru 0xa3e190b (0x6bda bytes) 2024-09-04T17:47:17.675 INFO:teuthology.orchestra.run.smithi092.stdout:3962 read 0x1b1ddef thru 0x1b2a171 (0xc383 bytes) 2024-09-04T17:47:17.675 INFO:teuthology.orchestra.run.smithi092.stdout:3963 write 0xca62ea thru 0xcb5ea1 (0xfbb8 bytes) 2024-09-04T17:47:17.675 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:17.675 INFO:teuthology.orchestra.run.smithi092.stdout:3966 write 0xe1eb8d9 thru 0xe1f62e5 (0xaa0d bytes) 2024-09-04T17:47:17.675 INFO:teuthology.orchestra.run.smithi092.stdout:3967 write 0xb2d5e81 thru 0xb2dbe26 (0x5fa6 bytes) 2024-09-04T17:47:17.675 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:17.675 INFO:teuthology.orchestra.run.smithi092.stdout:3969 read 0x2f483ad thru 0x2f4c213 (0x3e67 bytes) 2024-09-04T17:47:17.675 INFO:teuthology.orchestra.run.smithi092.stdout:3970 punch from 0x72df282 to 0x72e2ddd, (0x3b5b bytes) 2024-09-04T17:47:17.675 INFO:teuthology.orchestra.run.smithi092.stdout:3973 punch from 0xf67890 to 0xf76ae6, (0xf256 bytes) 2024-09-04T17:47:17.675 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:17.675 INFO:teuthology.orchestra.run.smithi092.stdout:3975 read 0x96f21a1 thru 0x96f9879 (0x76d9 bytes) 2024-09-04T17:47:17.676 INFO:teuthology.orchestra.run.smithi092.stdout:3976 read 0x6a0c8b6 thru 0x6a1108e (0x47d9 bytes) 2024-09-04T17:47:17.676 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:17.676 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:17.668+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:17.677 INFO:teuthology.orchestra.run.smithi092.stdout:3984 trunc from 0xe1f62e6 to 0x92b589e 2024-09-04T17:47:17.695 INFO:teuthology.orchestra.run.smithi092.stdout:3985 writesame 0xe9ac97 thru 0xea4fbb data_size 0x1(0xa325 bytes) 2024-09-04T17:47:17.727 INFO:teuthology.orchestra.run.smithi092.stdout:3987 writesame 0x560b6df thru 0x561a32c data_size 0x1d(0xec4e bytes) 2024-09-04T17:47:17.732 INFO:teuthology.orchestra.run.smithi092.stdout:3988 read 0x4b03055 thru 0x4b12794 (0xf740 bytes) 2024-09-04T17:47:17.748 INFO:teuthology.orchestra.run.smithi092.stdout:3989 write 0xaa44a9c thru 0xaa4c27b (0x77e0 bytes) 2024-09-04T17:47:17.751 INFO:teuthology.orchestra.run.smithi092.stdout:3990 write 0x4406e90 thru 0x440a166 (0x32d7 bytes) 2024-09-04T17:47:17.754 INFO:teuthology.orchestra.run.smithi092.stdout:3991 read 0x760a503 thru 0x76178bc (0xd3ba bytes) 2024-09-04T17:47:17.754 INFO:teuthology.orchestra.run.smithi092.stdout:3994 writesame 0x47a91a5 thru 0x47b6744 data_size 0x10(0xd5a0 bytes) 2024-09-04T17:47:17.759 INFO:teuthology.orchestra.run.smithi092.stdout:3995 trunc from 0xaa4c27c to 0xb806dac 2024-09-04T17:47:17.772 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:17.770+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:17.772 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:17.772 INFO:teuthology.orchestra.run.smithi092.stdout:3998 read 0x5c4b5c thru 0x5cc527 (0x79cc bytes) 2024-09-04T17:47:17.775 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:17.790 INFO:teuthology.orchestra.run.smithi092.stdout:4001 writesame 0xcda9679 thru 0xcdaa6b0 data_size 0x1(0x1038 bytes) 2024-09-04T17:47:17.792 INFO:teuthology.orchestra.run.smithi092.stdout:4002 write 0x6b51f9b thru 0x6b5a550 (0x85b6 bytes) 2024-09-04T17:47:17.796 INFO:teuthology.orchestra.run.smithi092.stdout:4003 writesame 0x3378aca thru 0x337aa65 data_size 0x22(0x1f9c bytes) 2024-09-04T17:47:17.799 INFO:teuthology.orchestra.run.smithi092.stdout:4005 read 0x4d35c47 thru 0x4d40bcc (0xaf86 bytes) 2024-09-04T17:47:17.801 INFO:teuthology.orchestra.run.smithi092.stdout:4007 writesame 0x68a3b67 thru 0x68aa444 data_size 0x1f(0x68de bytes) 2024-09-04T17:47:17.807 INFO:teuthology.orchestra.run.smithi092.stdout:4009 trunc from 0xcdaa6b1 to 0xabef6de 2024-09-04T17:47:17.813 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:17.813 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:47:17.813 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:47:17.813 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:17.824 INFO:teuthology.orchestra.run.smithi092.stdout:4010 write 0x92ade05 thru 0x92aefe5 (0x11e1 bytes) 2024-09-04T17:47:17.827 INFO:teuthology.orchestra.run.smithi092.stdout:4011 read 0x8cd304c thru 0x8cd9519 (0x64ce bytes) 2024-09-04T17:47:17.828 INFO:teuthology.orchestra.run.smithi092.stdout:4013 punch from 0x7bd38a2 to 0x7bdec8a, (0xb3e8 bytes) 2024-09-04T17:47:17.828 INFO:teuthology.orchestra.run.smithi092.stdout:4014 read 0x29b8398 thru 0x29bd45e (0x50c7 bytes) 2024-09-04T17:47:17.829 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-04T17:47:17.830 INFO:teuthology.orchestra.run.smithi092.stdout:4016 punch from 0x2badfe to 0x2bba0d, (0xc0f bytes) 2024-09-04T17:47:17.830 INFO:teuthology.orchestra.run.smithi092.stdout:4017 write 0x1d511bb thru 0x1d57db5 (0x6bfb bytes) 2024-09-04T17:47:17.846 INFO:teuthology.orchestra.run.smithi092.stdout:4018 writesame 0xb270a45 thru 0xb27c30d data_size 0x1(0xb8c9 bytes) 2024-09-04T17:47:17.848 INFO:teuthology.orchestra.run.smithi092.stdout:4019 read 0x556732e thru 0x556adc4 (0x3a97 bytes) 2024-09-04T17:47:17.848 INFO:teuthology.orchestra.run.smithi092.stdout:4020 read 0x90383c3 thru 0x903fc7d (0x78bb bytes) 2024-09-04T17:47:17.871 INFO:teuthology.orchestra.run.smithi092.stdout:4021 writesame 0xee28dc5 thru 0xee2e7b7 data_size 0x1(0x59f3 bytes) 2024-09-04T17:47:17.873 INFO:teuthology.orchestra.run.smithi092.stdout:4022 read 0x835a1c0 thru 0x835a211 (0x52 bytes) 2024-09-04T17:47:17.873 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:17.870+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:17.887 INFO:teuthology.orchestra.run.smithi092.stdout:4023 trunc from 0xee2e7b8 to 0x95815ed 2024-09-04T17:47:17.903 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:17.918 INFO:teuthology.orchestra.run.smithi092.stdout:4026 trunc from 0x95815ed to 0x1afacb 2024-09-04T17:47:17.943 INFO:teuthology.orchestra.run.smithi092.stdout:4027 read 0xda727 thru 0xe4d50 (0xa62a bytes) 2024-09-04T17:47:17.943 INFO:teuthology.orchestra.run.smithi092.stdout:4028 read 0xcc4be thru 0xd2725 (0x6268 bytes) 2024-09-04T17:47:17.957 INFO:teuthology.orchestra.run.smithi092.stdout:4029 write 0x1513d52 thru 0x151b896 (0x7b45 bytes) 2024-09-04T17:47:17.974 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:17.970+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:17.978 INFO:teuthology.orchestra.run.smithi092.stdout:4030 write 0x4c9d457 thru 0x4ca236f (0x4f19 bytes) 2024-09-04T17:47:17.981 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:17.981 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:47:17.981 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:47:17.982 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:47:17.982 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:17.982 INFO:teuthology.orchestra.run.smithi092.stdout:4031 punch from 0x111823f to 0x11250ab, (0xce6c bytes) 2024-09-04T17:47:17.982 INFO:teuthology.orchestra.run.smithi092.stdout:4032 write 0x42ace7b thru 0x42b5546 (0x86cc bytes) 2024-09-04T17:47:18.003 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-04T17:47:18.007 INFO:teuthology.orchestra.run.smithi092.stdout:4034 write 0xc3c7f05 thru 0xc3cc1bd (0x42b9 bytes) 2024-09-04T17:47:18.009 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:18.009 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:18.023 INFO:teuthology.orchestra.run.smithi092.stdout:4038 write 0xdb46176 thru 0xdb47db6 (0x1c41 bytes) 2024-09-04T17:47:18.028 INFO:teuthology.orchestra.run.smithi092.stdout:4039 read 0x3fabb7d thru 0x3fb2fec (0x7470 bytes) 2024-09-04T17:47:18.029 INFO:teuthology.orchestra.run.smithi092.stdout:4040 write 0x355d56b thru 0x3569906 (0xc39c bytes) 2024-09-04T17:47:18.031 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:18.031 INFO:teuthology.orchestra.run.smithi092.stdout:4042 write 0x3a6c897 thru 0x3a76df8 (0xa562 bytes) 2024-09-04T17:47:18.032 INFO:teuthology.orchestra.run.smithi092.stdout:4043 write 0x3195d01 thru 0x319bbf8 (0x5ef8 bytes) 2024-09-04T17:47:18.038 INFO:teuthology.orchestra.run.smithi092.stdout:4044 write 0x836bfbb thru 0x83726f3 (0x6739 bytes) 2024-09-04T17:47:18.041 INFO:teuthology.orchestra.run.smithi092.stdout:4045 writesame 0x9f4715d thru 0x9f5318b data_size 0x1(0xc02f bytes) 2024-09-04T17:47:18.074 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:18.070+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:18.106 INFO:teuthology.orchestra.run.smithi092.stdout:4046 writesame 0x3b8784d thru 0x3b940da data_size 0x81(0xc88e bytes) 2024-09-04T17:47:18.110 INFO:teuthology.orchestra.run.smithi092.stdout:4047 writesame 0x43e4cd6 thru 0x43f21f4 data_size 0x1(0xd51f bytes) 2024-09-04T17:47:18.134 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:18.134 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:47:18.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:47:18.135 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:47:18.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.135 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:48444 3.12 3:4ff80fb5:::rbd_data.11346a2b0bc9.000000000000001d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-04T17:47:18.135 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:58.122601+0000", 2024-09-04T17:47:18.135 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.010504358999999, 2024-09-04T17:47:18.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057065713999999997, 2024-09-04T17:47:18.135 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:18.135 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:18.135 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:18.135 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:18.135 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:18.135 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:18.135 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 48444 2024-09-04T17:47:18.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.136 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:18.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:18.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122601+0000", 2024-09-04T17:47:18.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:18.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:18.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122601+0000", 2024-09-04T17:47:18.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:18.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:18.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122603+0000", 2024-09-04T17:47:18.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3499999999999999e-06 2024-09-04T17:47:18.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:18.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122614+0000", 2024-09-04T17:47:18.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.063e-05 2024-09-04T17:47:18.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:18.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122643+0000", 2024-09-04T17:47:18.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9434999999999999e-05 2024-09-04T17:47:18.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:18.138 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122657+0000", 2024-09-04T17:47:18.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3920000000000001e-05 2024-09-04T17:47:18.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:18.138 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122767+0000", 2024-09-04T17:47:18.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010981599999999999 2024-09-04T17:47:18.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:18.138 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122989+0000", 2024-09-04T17:47:18.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022158200000000001 2024-09-04T17:47:18.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.139 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 3", 2024-09-04T17:47:18.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.133349+0000", 2024-09-04T17:47:18.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010360729000000001 2024-09-04T17:47:18.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.139 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:18.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.179423+0000", 2024-09-04T17:47:18.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046073408000000003 2024-09-04T17:47:18.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.140 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:18.140 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.179453+0000", 2024-09-04T17:47:18.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0229000000000001e-05 2024-09-04T17:47:18.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.140 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:18.140 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.179545+0000", 2024-09-04T17:47:18.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1580000000000001e-05 2024-09-04T17:47:18.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.140 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:18.140 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.179667+0000", 2024-09-04T17:47:18.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012203500000000001 2024-09-04T17:47:18.140 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:18.141 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:18.141 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:18.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.141 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:55376 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-04T17:47:18.141 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:11.974053+0000", 2024-09-04T17:47:18.141 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.1590525749999996, 2024-09-04T17:47:18.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060514722, 2024-09-04T17:47:18.141 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:18.141 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:18.141 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:18.141 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:18.141 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:18.141 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:18.142 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 55376 2024-09-04T17:47:18.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.142 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:18.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.142 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:18.142 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974053+0000", 2024-09-04T17:47:18.142 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:18.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.142 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:18.142 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974053+0000", 2024-09-04T17:47:18.142 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:18.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:18.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974054+0000", 2024-09-04T17:47:18.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.841e-06 2024-09-04T17:47:18.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:18.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974062+0000", 2024-09-04T17:47:18.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5460000000000003e-06 2024-09-04T17:47:18.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:18.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974077+0000", 2024-09-04T17:47:18.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5389000000000001e-05 2024-09-04T17:47:18.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:18.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974085+0000", 2024-09-04T17:47:18.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7659999999999998e-06 2024-09-04T17:47:18.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:18.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974188+0000", 2024-09-04T17:47:18.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000102521 2024-09-04T17:47:18.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:18.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974321+0000", 2024-09-04T17:47:18.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013324699999999999 2024-09-04T17:47:18.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-04T17:47:18.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.986481+0000", 2024-09-04T17:47:18.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012160450999999999 2024-09-04T17:47:18.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:18.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:12.034415+0000", 2024-09-04T17:47:18.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.047934065999999997 2024-09-04T17:47:18.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:18.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:12.034438+0000", 2024-09-04T17:47:18.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2923999999999999e-05 2024-09-04T17:47:18.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.146 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:18.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:12.034505+0000", 2024-09-04T17:47:18.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6711000000000006e-05 2024-09-04T17:47:18.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.146 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:18.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:12.034567+0000", 2024-09-04T17:47:18.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2260000000000004e-05 2024-09-04T17:47:18.146 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:18.146 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:18.146 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:18.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.147 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61126 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 1212416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:18.147 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819377+0000", 2024-09-04T17:47:18.147 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.3137285470000002, 2024-09-04T17:47:18.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037740846000000002, 2024-09-04T17:47:18.147 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:18.147 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:18.147 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:18.147 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:18.147 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:18.147 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:18.147 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61126 2024-09-04T17:47:18.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.148 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:18.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.148 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:18.148 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819377+0000", 2024-09-04T17:47:18.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:18.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.148 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:18.148 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819377+0000", 2024-09-04T17:47:18.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:18.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.148 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:18.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819377+0000", 2024-09-04T17:47:18.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7299999999999996e-07 2024-09-04T17:47:18.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:18.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819384+0000", 2024-09-04T17:47:18.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0530000000000001e-06 2024-09-04T17:47:18.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:18.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819392+0000", 2024-09-04T17:47:18.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5159999999999997e-06 2024-09-04T17:47:18.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:18.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819397+0000", 2024-09-04T17:47:18.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.254e-06 2024-09-04T17:47:18.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:18.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.856830+0000", 2024-09-04T17:47:18.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037432498000000002 2024-09-04T17:47:18.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:18.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.856880+0000", 2024-09-04T17:47:18.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0284999999999997e-05 2024-09-04T17:47:18.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.151 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:18.151 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.857118+0000", 2024-09-04T17:47:18.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000237667 2024-09-04T17:47:18.151 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:18.151 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:18.151 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:18.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.151 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61127 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 1351680~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:18.151 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819406+0000", 2024-09-04T17:47:18.151 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.3136995059999999, 2024-09-04T17:47:18.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.038078545999999998, 2024-09-04T17:47:18.152 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:18.152 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:18.152 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:18.152 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:18.152 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:18.152 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:18.152 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61127 2024-09-04T17:47:18.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.152 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:18.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.152 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:18.152 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819406+0000", 2024-09-04T17:47:18.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:18.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:18.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819406+0000", 2024-09-04T17:47:18.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:18.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:18.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819406+0000", 2024-09-04T17:47:18.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.06e-07 2024-09-04T17:47:18.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:18.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819409+0000", 2024-09-04T17:47:18.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4059999999999999e-06 2024-09-04T17:47:18.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:18.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819415+0000", 2024-09-04T17:47:18.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6830000000000002e-06 2024-09-04T17:47:18.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:18.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819420+0000", 2024-09-04T17:47:18.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8760000000000004e-06 2024-09-04T17:47:18.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:18.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.857136+0000", 2024-09-04T17:47:18.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037715752999999998 2024-09-04T17:47:18.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:18.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.857206+0000", 2024-09-04T17:47:18.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9887000000000004e-05 2024-09-04T17:47:18.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:18.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.857484+0000", 2024-09-04T17:47:18.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000278435 2024-09-04T17:47:18.155 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:18.156 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:18.156 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:18.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.156 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61128 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 1458176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:18.156 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819489+0000", 2024-09-04T17:47:18.156 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.31361624, 2024-09-04T17:47:18.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.038380478000000003, 2024-09-04T17:47:18.156 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:18.156 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:18.156 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:18.156 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:18.156 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:18.156 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:18.157 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61128 2024-09-04T17:47:18.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.157 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:18.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:18.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819489+0000", 2024-09-04T17:47:18.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:18.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:18.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819489+0000", 2024-09-04T17:47:18.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:18.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:18.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819490+0000", 2024-09-04T17:47:18.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6599999999999996e-07 2024-09-04T17:47:18.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:18.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819493+0000", 2024-09-04T17:47:18.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2279999999999998e-06 2024-09-04T17:47:18.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:18.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819501+0000", 2024-09-04T17:47:18.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4039999999999995e-06 2024-09-04T17:47:18.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.159 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:18.159 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819508+0000", 2024-09-04T17:47:18.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2400000000000004e-06 2024-09-04T17:47:18.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.159 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:18.159 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.857534+0000", 2024-09-04T17:47:18.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.038026272 2024-09-04T17:47:18.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.159 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:18.159 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.857629+0000", 2024-09-04T17:47:18.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5397999999999997e-05 2024-09-04T17:47:18.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.160 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:18.160 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.857869+0000", 2024-09-04T17:47:18.160 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024027 2024-09-04T17:47:18.160 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:18.160 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:18.160 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:18.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.160 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61129 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 1564672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:18.160 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819517+0000", 2024-09-04T17:47:18.160 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.3135884190000002, 2024-09-04T17:47:18.160 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.038772745999999997, 2024-09-04T17:47:18.161 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:18.161 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:18.161 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:18.161 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:18.161 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:18.161 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:18.161 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61129 2024-09-04T17:47:18.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.161 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:18.161 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.161 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:18.161 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819517+0000", 2024-09-04T17:47:18.161 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:18.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:18.162 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819517+0000", 2024-09-04T17:47:18.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:18.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:18.162 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819517+0000", 2024-09-04T17:47:18.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5999999999999999e-07 2024-09-04T17:47:18.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:18.162 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819520+0000", 2024-09-04T17:47:18.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2709999999999999e-06 2024-09-04T17:47:18.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.163 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.163 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:18.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819526+0000", 2024-09-04T17:47:18.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7059999999999998e-06 2024-09-04T17:47:18.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.163 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.163 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:18.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819531+0000", 2024-09-04T17:47:18.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0549999999999999e-06 2024-09-04T17:47:18.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.163 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.163 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:18.164 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.857913+0000", 2024-09-04T17:47:18.164 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.038381954000000003 2024-09-04T17:47:18.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.164 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:18.164 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.858000+0000", 2024-09-04T17:47:18.164 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6963000000000001e-05 2024-09-04T17:47:18.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.164 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:18.164 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.858290+0000", 2024-09-04T17:47:18.164 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028933700000000003 2024-09-04T17:47:18.164 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:18.165 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:18.165 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:18.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.165 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61130 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 1597440~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:18.165 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819539+0000", 2024-09-04T17:47:18.165 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.3135665919999999, 2024-09-04T17:47:18.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.039166595999999998, 2024-09-04T17:47:18.165 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:18.165 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:18.165 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:18.165 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:18.165 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:18.165 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:18.166 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61130 2024-09-04T17:47:18.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.166 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:18.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.166 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:18.166 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819539+0000", 2024-09-04T17:47:18.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:18.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.166 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:18.166 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819539+0000", 2024-09-04T17:47:18.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:18.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:18.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819539+0000", 2024-09-04T17:47:18.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9400000000000005e-07 2024-09-04T17:47:18.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:18.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819542+0000", 2024-09-04T17:47:18.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2630000000000002e-06 2024-09-04T17:47:18.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:18.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819548+0000", 2024-09-04T17:47:18.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3269999999999997e-06 2024-09-04T17:47:18.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.168 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.168 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:18.168 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819553+0000", 2024-09-04T17:47:18.168 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9849999999999997e-06 2024-09-04T17:47:18.168 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.168 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.168 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:18.168 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.858321+0000", 2024-09-04T17:47:18.168 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.038768395999999997 2024-09-04T17:47:18.168 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.168 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.168 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:18.168 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.858406+0000", 2024-09-04T17:47:18.169 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4456999999999995e-05 2024-09-04T17:47:18.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.169 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:18.169 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.858705+0000", 2024-09-04T17:47:18.169 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00029947399999999999 2024-09-04T17:47:18.169 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:18.169 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:18.169 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:18.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.169 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61131 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 1703936~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:18.169 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819560+0000", 2024-09-04T17:47:18.169 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.3135452399999998, 2024-09-04T17:47:18.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.039522082, 2024-09-04T17:47:18.170 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:18.170 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:18.170 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:18.170 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:18.170 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:18.170 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:18.170 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61131 2024-09-04T17:47:18.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.170 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:18.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.170 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:18.170 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819560+0000", 2024-09-04T17:47:18.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:18.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.171 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:18.171 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819560+0000", 2024-09-04T17:47:18.171 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:18.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.171 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:18.171 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819561+0000", 2024-09-04T17:47:18.171 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5300000000000004e-07 2024-09-04T17:47:18.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.171 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:18.171 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819563+0000", 2024-09-04T17:47:18.171 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.187e-06 2024-09-04T17:47:18.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.172 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:18.172 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819569+0000", 2024-09-04T17:47:18.172 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6170000000000003e-06 2024-09-04T17:47:18.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.172 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:18.172 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819574+0000", 2024-09-04T17:47:18.172 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9640000000000002e-06 2024-09-04T17:47:18.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.172 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:18.173 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.858754+0000", 2024-09-04T17:47:18.173 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.039179295000000003 2024-09-04T17:47:18.173 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.173 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.173 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:18.173 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.858800+0000", 2024-09-04T17:47:18.173 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.587e-05 2024-09-04T17:47:18.173 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.173 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.173 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:18.173 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.859082+0000", 2024-09-04T17:47:18.173 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028249599999999997 2024-09-04T17:47:18.173 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:18.173 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:18.174 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:18.174 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.174 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.174 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61132 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 1810432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:18.174 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819581+0000", 2024-09-04T17:47:18.174 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.3135237980000003, 2024-09-04T17:47:18.174 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.039953120000000002, 2024-09-04T17:47:18.174 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:18.174 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:18.174 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:18.174 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:18.174 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:18.174 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:18.174 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61132 2024-09-04T17:47:18.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.175 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:18.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.175 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:18.175 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819581+0000", 2024-09-04T17:47:18.175 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:18.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.175 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:18.175 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819581+0000", 2024-09-04T17:47:18.175 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:18.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.175 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:18.175 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819582+0000", 2024-09-04T17:47:18.176 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3399999999999999e-07 2024-09-04T17:47:18.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.176 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:18.176 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819584+0000", 2024-09-04T17:47:18.176 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1320000000000001e-06 2024-09-04T17:47:18.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.176 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:18.176 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819591+0000", 2024-09-04T17:47:18.176 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3759999999999996e-06 2024-09-04T17:47:18.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.177 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:18.177 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819595+0000", 2024-09-04T17:47:18.177 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7339999999999999e-06 2024-09-04T17:47:18.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.177 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.177 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:18.177 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.859119+0000", 2024-09-04T17:47:18.177 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.039523677 2024-09-04T17:47:18.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.177 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.177 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:18.177 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.859232+0000", 2024-09-04T17:47:18.177 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000113001 2024-09-04T17:47:18.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.178 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.178 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:18.178 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.859535+0000", 2024-09-04T17:47:18.178 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030256599999999999 2024-09-04T17:47:18.178 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:18.178 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:18.178 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:18.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.178 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.178 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61133 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 1843200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:18.178 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819602+0000", 2024-09-04T17:47:18.178 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.3135034900000004, 2024-09-04T17:47:18.178 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.040226171999999998, 2024-09-04T17:47:18.179 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:18.179 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:18.179 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:18.179 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:18.179 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:18.179 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:18.179 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61133 2024-09-04T17:47:18.179 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.179 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:18.179 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.179 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:18.179 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819602+0000", 2024-09-04T17:47:18.179 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:18.179 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.180 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.180 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:18.180 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819602+0000", 2024-09-04T17:47:18.180 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:18.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.180 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.180 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:18.180 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819602+0000", 2024-09-04T17:47:18.180 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.75e-07 2024-09-04T17:47:18.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.180 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.180 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:18.180 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819605+0000", 2024-09-04T17:47:18.180 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.52e-06 2024-09-04T17:47:18.181 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.181 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.181 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:18.181 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819611+0000", 2024-09-04T17:47:18.181 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5200000000000003e-06 2024-09-04T17:47:18.181 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.181 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.181 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:18.181 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819616+0000", 2024-09-04T17:47:18.181 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0939999999999998e-06 2024-09-04T17:47:18.181 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.181 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.181 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:18.182 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.859538+0000", 2024-09-04T17:47:18.182 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.039921660999999997 2024-09-04T17:47:18.182 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.182 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.182 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:18.182 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.859594+0000", 2024-09-04T17:47:18.182 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5671999999999998e-05 2024-09-04T17:47:18.182 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.182 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.182 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:18.182 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.859828+0000", 2024-09-04T17:47:18.182 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023423000000000001 2024-09-04T17:47:18.182 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:18.182 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:18.183 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:18.183 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.183 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61134 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 1949696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:18.183 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819623+0000", 2024-09-04T17:47:18.183 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.3134820129999998, 2024-09-04T17:47:18.183 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.040606862000000001, 2024-09-04T17:47:18.183 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:18.183 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:18.183 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:18.183 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:18.183 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:18.183 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:18.183 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61134 2024-09-04T17:47:18.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.184 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:18.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.184 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:18.184 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819623+0000", 2024-09-04T17:47:18.184 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:18.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.184 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:18.184 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819623+0000", 2024-09-04T17:47:18.184 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:18.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.184 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:18.185 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819624+0000", 2024-09-04T17:47:18.185 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2100000000000002e-07 2024-09-04T17:47:18.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.185 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.185 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:18.185 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819626+0000", 2024-09-04T17:47:18.185 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2460000000000002e-06 2024-09-04T17:47:18.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.185 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.185 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:18.185 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819632+0000", 2024-09-04T17:47:18.185 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1659999999999999e-06 2024-09-04T17:47:18.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.185 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.186 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:18.186 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819637+0000", 2024-09-04T17:47:18.186 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2680000000000002e-06 2024-09-04T17:47:18.186 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.186 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:18.186 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.859872+0000", 2024-09-04T17:47:18.186 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.040234377000000002 2024-09-04T17:47:18.186 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.186 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:18.186 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.859950+0000", 2024-09-04T17:47:18.186 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8128999999999997e-05 2024-09-04T17:47:18.186 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.187 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.187 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:18.187 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.860230+0000", 2024-09-04T17:47:18.187 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028025499999999998 2024-09-04T17:47:18.187 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:18.187 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:18.187 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:18.187 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.187 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.187 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61135 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 2056192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:18.187 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819644+0000", 2024-09-04T17:47:18.187 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.3134609729999998, 2024-09-04T17:47:18.187 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.040925930999999999, 2024-09-04T17:47:18.187 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:18.188 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:18.188 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:18.188 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:18.188 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:18.188 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:18.188 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61135 2024-09-04T17:47:18.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.188 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:18.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.188 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:18.188 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819644+0000", 2024-09-04T17:47:18.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:18.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:18.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819644+0000", 2024-09-04T17:47:18.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:18.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:18.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819645+0000", 2024-09-04T17:47:18.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9399999999999995e-07 2024-09-04T17:47:18.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:18.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819647+0000", 2024-09-04T17:47:18.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.311e-06 2024-09-04T17:47:18.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:18.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819653+0000", 2024-09-04T17:47:18.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3940000000000001e-06 2024-09-04T17:47:18.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:18.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819658+0000", 2024-09-04T17:47:18.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8420000000000004e-06 2024-09-04T17:47:18.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:18.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.860263+0000", 2024-09-04T17:47:18.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.040605109 2024-09-04T17:47:18.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:18.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.860339+0000", 2024-09-04T17:47:18.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5612999999999996e-05 2024-09-04T17:47:18.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:18.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.860570+0000", 2024-09-04T17:47:18.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023116799999999999 2024-09-04T17:47:18.191 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:18.191 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:18.191 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:18.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.192 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61136 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 2195456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:18.192 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819665+0000", 2024-09-04T17:47:18.192 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.3134405190000003, 2024-09-04T17:47:18.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.041275408999999999, 2024-09-04T17:47:18.192 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:18.192 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:18.192 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:18.192 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:18.192 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:18.192 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:18.192 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61136 2024-09-04T17:47:18.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.193 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:18.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:18.193 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819665+0000", 2024-09-04T17:47:18.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:18.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:18.193 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819665+0000", 2024-09-04T17:47:18.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:18.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:18.193 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819665+0000", 2024-09-04T17:47:18.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8599999999999999e-07 2024-09-04T17:47:18.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:18.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819667+0000", 2024-09-04T17:47:18.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9439999999999999e-06 2024-09-04T17:47:18.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:18.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819673+0000", 2024-09-04T17:47:18.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0220000000000001e-06 2024-09-04T17:47:18.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:18.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819677+0000", 2024-09-04T17:47:18.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4139999999999996e-06 2024-09-04T17:47:18.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:18.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.860603+0000", 2024-09-04T17:47:18.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.040925504000000001 2024-09-04T17:47:18.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:18.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.860673+0000", 2024-09-04T17:47:18.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9707000000000002e-05 2024-09-04T17:47:18.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.196 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:18.196 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.860940+0000", 2024-09-04T17:47:18.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026743200000000001 2024-09-04T17:47:18.196 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:18.196 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:18.196 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:18.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.196 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61137 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 2301952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:18.196 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819685+0000", 2024-09-04T17:47:18.196 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.3134197260000002, 2024-09-04T17:47:18.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.041579208999999999, 2024-09-04T17:47:18.196 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:18.197 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:18.197 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:18.197 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:18.197 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:18.197 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:18.197 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61137 2024-09-04T17:47:18.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.197 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:18.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:18.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819685+0000", 2024-09-04T17:47:18.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:18.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:18.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819685+0000", 2024-09-04T17:47:18.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:18.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:18.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819686+0000", 2024-09-04T17:47:18.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3000000000000001e-07 2024-09-04T17:47:18.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:18.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819688+0000", 2024-09-04T17:47:18.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1150000000000001e-06 2024-09-04T17:47:18.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:18.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819694+0000", 2024-09-04T17:47:18.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.75e-06 2024-09-04T17:47:18.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:18.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819699+0000", 2024-09-04T17:47:18.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8110000000000002e-06 2024-09-04T17:47:18.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:18.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.860977+0000", 2024-09-04T17:47:18.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.041277945000000003 2024-09-04T17:47:18.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.200 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:18.200 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.861071+0000", 2024-09-04T17:47:18.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4252000000000005e-05 2024-09-04T17:47:18.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.200 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:18.200 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.861265+0000", 2024-09-04T17:47:18.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019390600000000001 2024-09-04T17:47:18.200 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:18.200 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:18.200 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:18.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.201 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61138 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 2441216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:18.201 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819706+0000", 2024-09-04T17:47:18.201 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.3133993249999998, 2024-09-04T17:47:18.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.041876780000000002, 2024-09-04T17:47:18.201 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:18.201 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:18.201 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:18.201 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:18.201 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:18.201 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:18.201 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61138 2024-09-04T17:47:18.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.202 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:18.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.202 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:18.202 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819706+0000", 2024-09-04T17:47:18.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:18.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.202 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:18.202 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819706+0000", 2024-09-04T17:47:18.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:18.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.202 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:18.202 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819707+0000", 2024-09-04T17:47:18.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7299999999999995e-07 2024-09-04T17:47:18.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:18.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819727+0000", 2024-09-04T17:47:18.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0610000000000001e-05 2024-09-04T17:47:18.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:18.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819735+0000", 2024-09-04T17:47:18.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.123e-06 2024-09-04T17:47:18.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:18.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819741+0000", 2024-09-04T17:47:18.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4009999999999996e-06 2024-09-04T17:47:18.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.204 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:18.204 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.861298+0000", 2024-09-04T17:47:18.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.041557135000000002 2024-09-04T17:47:18.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.204 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:18.204 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.861369+0000", 2024-09-04T17:47:18.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0794e-05 2024-09-04T17:47:18.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.205 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:18.205 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.861583+0000", 2024-09-04T17:47:18.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021404400000000001 2024-09-04T17:47:18.205 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:18.205 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:18.205 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:18.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.205 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61139 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 2547712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:18.205 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819750+0000", 2024-09-04T17:47:18.205 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.3133548609999997, 2024-09-04T17:47:18.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042185699, 2024-09-04T17:47:18.205 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:18.205 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:18.206 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:18.206 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:18.206 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:18.206 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:18.206 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61139 2024-09-04T17:47:18.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.206 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:18.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:18.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819750+0000", 2024-09-04T17:47:18.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:18.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:18.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819750+0000", 2024-09-04T17:47:18.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:18.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.207 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:18.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819751+0000", 2024-09-04T17:47:18.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4000000000000002e-07 2024-09-04T17:47:18.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.207 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:18.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819753+0000", 2024-09-04T17:47:18.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5179999999999999e-06 2024-09-04T17:47:18.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.208 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:18.208 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819760+0000", 2024-09-04T17:47:18.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5440000000000004e-06 2024-09-04T17:47:18.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.208 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:18.208 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819765+0000", 2024-09-04T17:47:18.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8760000000000004e-06 2024-09-04T17:47:18.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.208 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:18.208 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.861622+0000", 2024-09-04T17:47:18.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.041857093999999997 2024-09-04T17:47:18.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.209 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.209 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:18.209 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.861698+0000", 2024-09-04T17:47:18.209 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6080000000000003e-05 2024-09-04T17:47:18.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.209 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.209 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:18.209 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.861936+0000", 2024-09-04T17:47:18.209 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023804699999999999 2024-09-04T17:47:18.209 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:18.209 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:18.209 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:18.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.210 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61140 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 2686976~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:18.210 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819772+0000", 2024-09-04T17:47:18.210 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.3133327269999997, 2024-09-04T17:47:18.210 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042515052999999997, 2024-09-04T17:47:18.210 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:18.210 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:18.210 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:18.210 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:18.210 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:18.210 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:18.210 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61140 2024-09-04T17:47:18.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.210 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:18.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:18.211 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819772+0000", 2024-09-04T17:47:18.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:18.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:18.211 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819772+0000", 2024-09-04T17:47:18.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:18.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:18.211 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819773+0000", 2024-09-04T17:47:18.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2900000000000003e-07 2024-09-04T17:47:18.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.212 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:18.212 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819775+0000", 2024-09-04T17:47:18.212 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3309999999999998e-06 2024-09-04T17:47:18.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.212 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:18.212 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819781+0000", 2024-09-04T17:47:18.212 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.981e-06 2024-09-04T17:47:18.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.212 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:18.212 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819786+0000", 2024-09-04T17:47:18.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5719999999999996e-06 2024-09-04T17:47:18.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.213 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:18.213 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.861966+0000", 2024-09-04T17:47:18.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042179541000000001 2024-09-04T17:47:18.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.213 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:18.213 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862038+0000", 2024-09-04T17:47:18.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2719000000000006e-05 2024-09-04T17:47:18.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.213 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:18.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862288+0000", 2024-09-04T17:47:18.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024928 2024-09-04T17:47:18.214 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:18.214 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:18.214 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:18.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.214 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61141 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 2793472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:18.214 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819909+0000", 2024-09-04T17:47:18.214 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.3131965829999999, 2024-09-04T17:47:18.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042786419999999999, 2024-09-04T17:47:18.214 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:18.214 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:18.214 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:18.215 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:18.215 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:18.215 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:18.215 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61141 2024-09-04T17:47:18.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.215 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:18.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:18.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819909+0000", 2024-09-04T17:47:18.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:18.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:18.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819909+0000", 2024-09-04T17:47:18.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:18.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:18.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819909+0000", 2024-09-04T17:47:18.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6300000000000005e-07 2024-09-04T17:47:18.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:18.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819913+0000", 2024-09-04T17:47:18.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4189999999999998e-06 2024-09-04T17:47:18.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.217 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:18.217 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819920+0000", 2024-09-04T17:47:18.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8619999999999997e-06 2024-09-04T17:47:18.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.217 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:18.217 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819926+0000", 2024-09-04T17:47:18.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9399999999999999e-06 2024-09-04T17:47:18.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.217 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:18.217 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862331+0000", 2024-09-04T17:47:18.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042405348000000002 2024-09-04T17:47:18.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:18.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862410+0000", 2024-09-04T17:47:18.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9078000000000004e-05 2024-09-04T17:47:18.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:18.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862695+0000", 2024-09-04T17:47:18.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028510999999999999 2024-09-04T17:47:18.218 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:18.218 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:18.218 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:18.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.219 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61142 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 2932736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:18.219 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819934+0000", 2024-09-04T17:47:18.219 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.3131709650000003, 2024-09-04T17:47:18.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043101985000000002, 2024-09-04T17:47:18.219 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:18.219 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:18.219 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:18.219 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:18.219 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:18.219 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:18.219 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61142 2024-09-04T17:47:18.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.219 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:18.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:18.220 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819934+0000", 2024-09-04T17:47:18.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:18.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:18.220 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819934+0000", 2024-09-04T17:47:18.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:18.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:18.220 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819935+0000", 2024-09-04T17:47:18.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2024-09-04T17:47:18.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.221 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.221 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:18.221 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819937+0000", 2024-09-04T17:47:18.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5770000000000001e-06 2024-09-04T17:47:18.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.221 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.221 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:18.221 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819943+0000", 2024-09-04T17:47:18.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7950000000000004e-06 2024-09-04T17:47:18.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.221 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.221 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:18.221 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819948+0000", 2024-09-04T17:47:18.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6779999999999999e-06 2024-09-04T17:47:18.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:18.222 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862739+0000", 2024-09-04T17:47:18.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042791454999999999 2024-09-04T17:47:18.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:18.222 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862824+0000", 2024-09-04T17:47:18.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4790999999999995e-05 2024-09-04T17:47:18.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:18.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.863036+0000", 2024-09-04T17:47:18.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021224899999999999 2024-09-04T17:47:18.223 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:18.223 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:18.223 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:18.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.223 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61143 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 3039232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:18.223 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819954+0000", 2024-09-04T17:47:18.223 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.3131508470000002, 2024-09-04T17:47:18.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043485134000000002, 2024-09-04T17:47:18.223 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:18.223 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:18.223 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:18.224 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:18.224 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:18.224 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:18.224 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61143 2024-09-04T17:47:18.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.224 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:18.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:18.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819954+0000", 2024-09-04T17:47:18.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:18.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:18.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819954+0000", 2024-09-04T17:47:18.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:18.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.225 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:18.225 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819955+0000", 2024-09-04T17:47:18.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7300000000000002e-07 2024-09-04T17:47:18.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.225 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:18.225 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819957+0000", 2024-09-04T17:47:18.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.024e-06 2024-09-04T17:47:18.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.225 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:18.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819963+0000", 2024-09-04T17:47:18.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8109999999999999e-06 2024-09-04T17:47:18.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:18.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819967+0000", 2024-09-04T17:47:18.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6149999999999999e-06 2024-09-04T17:47:18.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:18.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.863082+0000", 2024-09-04T17:47:18.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043115078000000001 2024-09-04T17:47:18.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:18.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.863172+0000", 2024-09-04T17:47:18.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0181999999999998e-05 2024-09-04T17:47:18.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:18.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:18.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:18.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.863440+0000", 2024-09-04T17:47:18.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026705099999999997 2024-09-04T17:47:18.227 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:18.227 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:18.227 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:18.227 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:18.227 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:18.227 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:18.229 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-04T17:47:18.230 INFO:teuthology.orchestra.run.smithi092.stdout:4048 trunc from 0xdb47db7 to 0x51bd10b 2024-09-04T17:47:18.230 INFO:teuthology.orchestra.run.smithi092.stdout:4049 write 0x1714608 thru 0x172334c (0xed45 bytes) 2024-09-04T17:47:18.230 INFO:teuthology.orchestra.run.smithi092.stdout:4050 read 0x1cc94fe thru 0x1cce32b (0x4e2e bytes) 2024-09-04T17:47:18.230 INFO:teuthology.orchestra.run.smithi092.stdout:4052 writesame 0x2b2ac79 thru 0x2b396a2 data_size 0x67(0xea2a bytes) 2024-09-04T17:47:18.230 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:18.230 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:18.231 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:18.231 INFO:teuthology.orchestra.run.smithi092.stdout:4057 read 0x35424f0 thru 0x3542a62 (0x573 bytes) 2024-09-04T17:47:18.231 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:18.226+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:18.246 INFO:teuthology.orchestra.run.smithi092.stdout:4058 trunc from 0x51bd10b to 0xe5bb29d 2024-09-04T17:47:18.260 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-04T17:47:18.261 INFO:teuthology.orchestra.run.smithi092.stdout:4060 write 0xd4dc7f thru 0xd5a832 (0xcbb4 bytes) 2024-09-04T17:47:18.263 INFO:teuthology.orchestra.run.smithi092.stdout:4061 writesame 0x2798059 thru 0x279a611 data_size 0x25(0x25b9 bytes) 2024-09-04T17:47:18.266 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:18.272 INFO:teuthology.orchestra.run.smithi092.stdout:4064 trunc from 0xe5bb29d to 0xbb8a4b4 2024-09-04T17:47:18.286 INFO:teuthology.orchestra.run.smithi092.stdout:4066 write 0x62f89b6 thru 0x62fbf19 (0x3564 bytes) 2024-09-04T17:47:18.305 INFO:teuthology.orchestra.run.smithi092.stdout:4067 write 0xc6adaf8 thru 0xc6ba6e4 (0xcbed bytes) 2024-09-04T17:47:18.319 INFO:teuthology.orchestra.run.smithi092.stdout:4068 writesame 0xea1599a thru 0xea17b71 data_size 0x18(0x21d8 bytes) 2024-09-04T17:47:18.322 INFO:teuthology.orchestra.run.smithi092.stdout:4074 writesame 0x77b4f2f thru 0x77c28a8 data_size 0x1b(0xd97a bytes) 2024-09-04T17:47:18.329 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:18.329 INFO:teuthology.orchestra.run.smithi092.stdout:4076 write 0xd032f9a thru 0xd041bc3 (0xec2a bytes) 2024-09-04T17:47:18.329 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:18.326+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:18.331 INFO:teuthology.orchestra.run.smithi092.stdout:4077 writesame 0x5751f82 thru 0x57555bf data_size 0x35(0x363e bytes) 2024-09-04T17:47:18.335 INFO:teuthology.orchestra.run.smithi092.stdout:4078 read 0xbfb8f5d thru 0xbfbea2f (0x5ad3 bytes) 2024-09-04T17:47:18.335 INFO:teuthology.orchestra.run.smithi092.stdout:4079 read 0x29424b2 thru 0x2949540 (0x708f bytes) 2024-09-04T17:47:18.336 INFO:teuthology.orchestra.run.smithi092.stdout:4080 write 0x4e0213f thru 0x4e02684 (0x546 bytes) 2024-09-04T17:47:18.338 INFO:teuthology.orchestra.run.smithi092.stdout:4081 punch from 0x15a04ef to 0x15ad033, (0xcb44 bytes) 2024-09-04T17:47:18.338 INFO:teuthology.orchestra.run.smithi092.stdout:4082 write 0xd3e9595 thru 0xd3f8735 (0xf1a1 bytes) 2024-09-04T17:47:18.341 INFO:teuthology.orchestra.run.smithi092.stdout:4083 read 0xd8b85df thru 0xd8becd1 (0x66f3 bytes) 2024-09-04T17:47:18.342 INFO:teuthology.orchestra.run.smithi092.stdout:4084 write 0xdded412 thru 0xddf3389 (0x5f78 bytes) 2024-09-04T17:47:18.344 INFO:teuthology.orchestra.run.smithi092.stdout:4085 read 0x1d357c7 thru 0x1d37c30 (0x246a bytes) 2024-09-04T17:47:18.358 INFO:teuthology.orchestra.run.smithi092.stdout:4086 trunc from 0xea17b72 to 0x690052c 2024-09-04T17:47:18.375 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:18.375 INFO:teuthology.orchestra.run.smithi092.stdout:4088 writesame 0x2ce16ac thru 0x2ce94de data_size 0x21(0x7e33 bytes) 2024-09-04T17:47:18.387 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:18.387 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:47:18.387 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:47:18.387 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:18.388 INFO:teuthology.orchestra.run.smithi092.stdout:4089 trunc from 0x690052c to 0x283658 2024-09-04T17:47:18.406 INFO:teuthology.orchestra.run.smithi092.stdout:4090 read 0x19c232 thru 0x19c3e7 (0x1b6 bytes) 2024-09-04T17:47:18.407 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-04T17:47:18.410 INFO:teuthology.orchestra.run.smithi092.stdout:4091 trunc from 0x283658 to 0x32a8980 2024-09-04T17:47:18.421 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:18.421 INFO:teuthology.orchestra.run.smithi092.stdout:4093 punch from 0x4d8874 to 0x4dbdd6, (0x3562 bytes) 2024-09-04T17:47:18.421 INFO:teuthology.orchestra.run.smithi092.stdout:4095 read 0x1317c2d thru 0x131da14 (0x5de8 bytes) 2024-09-04T17:47:18.422 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:47:18.422 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:47:18.428 INFO:teuthology.orchestra.run.smithi092.stdout:4096 trunc from 0x32a8980 to 0x8336639 2024-09-04T17:47:18.430 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:18.428+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:18.441 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:47:18.441 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:47:18.441 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-04T17:47:18.454 INFO:teuthology.orchestra.run.smithi092.stdout:4097 write 0xb9e9dd7 thru 0xb9f1ec7 (0x80f1 bytes) 2024-09-04T17:47:18.455 INFO:teuthology.orchestra.run.smithi092.stdout:4098 write 0x496b5a4 thru 0x4975bfd (0xa65a bytes) 2024-09-04T17:47:18.457 INFO:teuthology.orchestra.run.smithi092.stdout:4101 punch from 0x66d7aa1 to 0x66dea07, (0x6f66 bytes) 2024-09-04T17:47:18.457 INFO:teuthology.orchestra.run.smithi092.stdout:4103 read 0x61af1e4 thru 0x61b2bcc (0x39e9 bytes) 2024-09-04T17:47:18.457 INFO:teuthology.orchestra.run.smithi092.stdout:4105 write 0xaacf015 thru 0xaad305f (0x404b bytes) 2024-09-04T17:47:18.458 INFO:teuthology.orchestra.run.smithi092.stdout:4106 read 0x2b8e091 thru 0x2b9507c (0x6fec bytes) 2024-09-04T17:47:18.459 INFO:teuthology.orchestra.run.smithi092.stdout:4107 write 0xaf51158 thru 0xaf5bbbc (0xaa65 bytes) 2024-09-04T17:47:18.460 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-04T17:47:18.460 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-04T17:47:18.463 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:18.463 INFO:teuthology.orchestra.run.smithi092.stdout:4109 writesame 0x40dfcc2 thru 0x40ed3bf data_size 0x3(0xd6fe bytes) 2024-09-04T17:47:18.481 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:47:18.482 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:47:18.502 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:47:18.502 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:47:18.502 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-04T17:47:18.509 INFO:teuthology.orchestra.run.smithi092.stdout:4110 trunc from 0xb9f1ec8 to 0x5d69f3 2024-09-04T17:47:18.526 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:47:18.526 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:47:18.530 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-04T17:47:18.530 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-04T17:47:18.530 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:18.526+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:18.533 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:18.533 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:18.540 INFO:teuthology.orchestra.run.smithi092.stdout:4111 write 0x431f440 thru 0x43212b4 (0x1e75 bytes) 2024-09-04T17:47:18.549 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-04T17:47:18.554 INFO:teuthology.orchestra.run.smithi092.stdout:4113 write 0x4adc524 thru 0x4ae6109 (0x9be6 bytes) 2024-09-04T17:47:18.572 INFO:teuthology.orchestra.run.smithi092.stdout:4114 write 0x6e56968 thru 0x6e5b340 (0x49d9 bytes) 2024-09-04T17:47:18.586 INFO:teuthology.orchestra.run.smithi092.stdout:4117 trunc from 0x6e5b341 to 0xe4643f 2024-09-04T17:47:18.626 INFO:teuthology.orchestra.run.smithi092.stdout:4119 trunc from 0xe4643f to 0xd5af761 2024-09-04T17:47:18.631 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:18.628+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:18.644 INFO:teuthology.orchestra.run.smithi092.stdout:4120 trunc from 0xd5af761 to 0xa3da832 2024-09-04T17:47:18.670 INFO:teuthology.orchestra.run.smithi092.stdout:4122 trunc from 0xa3da832 to 0x5b3eba5 2024-09-04T17:47:18.680 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:18.680 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:18.686 INFO:teuthology.orchestra.run.smithi092.stdout:4123 write 0x273be52 thru 0x273d846 (0x19f5 bytes) 2024-09-04T17:47:18.696 INFO:teuthology.orchestra.run.smithi092.stdout:4124 trunc from 0x5b3eba5 to 0x1f5d48d 2024-09-04T17:47:18.701 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-04T17:47:18.713 INFO:teuthology.orchestra.run.smithi092.stdout:4125 write 0xc31d6a thru 0xc3c4e1 (0xa778 bytes) 2024-09-04T17:47:18.715 INFO:teuthology.orchestra.run.smithi092.stdout:4126 writesame 0x1ae1722 thru 0x1ae2028 data_size 0x1(0x907 bytes) 2024-09-04T17:47:18.730 INFO:teuthology.orchestra.run.smithi092.stdout:4127 write 0x36dc4eb thru 0x36dcc0b (0x721 bytes) 2024-09-04T17:47:18.731 INFO:teuthology.orchestra.run.smithi092.stdout:4128 punch from 0x2b95f7a to 0x2b9eaed, (0x8b73 bytes) 2024-09-04T17:47:18.748 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:47:18.748+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:47:18.750 INFO:teuthology.orchestra.run.smithi092.stdout:4129 trunc from 0x36dcc0c to 0xdb5cf6a 2024-09-04T17:47:18.759 INFO:teuthology.orchestra.run.smithi092.stdout:4130 write 0xfcb1e9 thru 0xfd0d0f (0x5b27 bytes) 2024-09-04T17:47:18.760 INFO:teuthology.orchestra.run.smithi092.stdout:4131 write 0x8e77fbb thru 0x8e7ad87 (0x2dcd bytes) 2024-09-04T17:47:18.763 INFO:teuthology.orchestra.run.smithi092.stdout:4132 read 0x410896a thru 0x410dcd5 (0x536c bytes) 2024-09-04T17:47:18.769 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-04T17:47:18.772 INFO:teuthology.orchestra.run.smithi092.stdout:4133 writesame 0xdf93e1e thru 0xdf9911f data_size 0x32(0x5302 bytes) 2024-09-04T17:47:18.794 INFO:teuthology.orchestra.run.smithi092.stdout:4135 trunc from 0xdf99120 to 0x4bd9a15 2024-09-04T17:47:18.833 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:18.830+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:18.839 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:18.839 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:18.841 INFO:teuthology.orchestra.run.smithi092.stdout:4136 write 0xeb28f3c thru 0xeb2d165 (0x422a bytes) 2024-09-04T17:47:18.854 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-04T17:47:18.869 INFO:teuthology.orchestra.run.smithi092.stdout:4137 trunc from 0xeb2d166 to 0x1667652 2024-09-04T17:47:18.882 INFO:teuthology.orchestra.run.smithi092.stdout:4138 read 0x1cd82c thru 0x1d588f (0x8064 bytes) 2024-09-04T17:47:18.882 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:18.894 INFO:teuthology.orchestra.run.smithi092.stdout:4143 trunc from 0x1667652 to 0x7e230ce 2024-09-04T17:47:18.903 INFO:teuthology.orchestra.run.smithi092.stdout:4144 read 0x6472f08 thru 0x6474d43 (0x1e3c bytes) 2024-09-04T17:47:18.904 INFO:teuthology.orchestra.run.smithi092.stdout:4145 write 0x671f34f thru 0x6728280 (0x8f32 bytes) 2024-09-04T17:47:18.915 INFO:teuthology.orchestra.run.smithi092.stdout:4146 write 0x876ca93 thru 0x877519e (0x870c bytes) 2024-09-04T17:47:18.918 INFO:teuthology.orchestra.run.smithi092.stdout:4147 writesame 0x26fa62c thru 0x2707fce data_size 0xb(0xd9a3 bytes) 2024-09-04T17:47:18.921 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:18.921 INFO:teuthology.orchestra.run.smithi092.stdout:4149 read 0x4d3de45 thru 0x4d3e515 (0x6d1 bytes) 2024-09-04T17:47:18.921 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:18.922 INFO:teuthology.orchestra.run.smithi092.stdout:4151 read 0x701d849 thru 0x7029506 (0xbcbe bytes) 2024-09-04T17:47:18.922 INFO:teuthology.orchestra.run.smithi092.stdout:4153 write 0x3fa8f16 thru 0x3faec8c (0x5d77 bytes) 2024-09-04T17:47:18.933 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:18.930+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:18.934 INFO:teuthology.orchestra.run.smithi092.stdout:4154 write 0x97936d1 thru 0x9797f16 (0x4846 bytes) 2024-09-04T17:47:18.935 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:18.935 INFO:teuthology.orchestra.run.smithi092.stdout:4157 writesame 0x7808dab thru 0x780ad24 data_size 0x4f(0x1f7a bytes) 2024-09-04T17:47:18.937 INFO:teuthology.orchestra.run.smithi092.stdout:4158 write 0x7d685a5 thru 0x7d6a511 (0x1f6d bytes) 2024-09-04T17:47:18.940 INFO:teuthology.orchestra.run.smithi092.stdout:4160 punch from 0x42d55bf to 0x42d7aa6, (0x24e7 bytes) 2024-09-04T17:47:18.940 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:18.940 INFO:teuthology.orchestra.run.smithi092.stdout:4164 read 0x4820561 thru 0x482281a (0x22ba bytes) 2024-09-04T17:47:18.940 INFO:teuthology.orchestra.run.smithi092.stdout:4165 writesame 0x1245009 thru 0x1250c89 data_size 0x29(0xbc81 bytes) 2024-09-04T17:47:18.952 INFO:teuthology.orchestra.run.smithi092.stdout:4166 write 0xae0e566 thru 0xae1bad4 (0xd56f bytes) 2024-09-04T17:47:18.954 INFO:teuthology.orchestra.run.smithi092.stdout:4167 read 0x4d8366a thru 0x4d86a4e (0x33e5 bytes) 2024-09-04T17:47:18.955 INFO:teuthology.orchestra.run.smithi092.stdout:4168 writesame 0x7208350 thru 0x720b93b data_size 0x7(0x35ec bytes) 2024-09-04T17:47:18.956 INFO:teuthology.orchestra.run.smithi092.stdout:4170 write 0x70b4e2a thru 0x70becaa (0x9e81 bytes) 2024-09-04T17:47:18.958 INFO:teuthology.orchestra.run.smithi092.stdout:4171 read 0x407499e thru 0x4078ee4 (0x4547 bytes) 2024-09-04T17:47:18.958 INFO:teuthology.orchestra.run.smithi092.stdout:4172 read 0x7b1e0dc thru 0x7b1fa0b (0x1930 bytes) 2024-09-04T17:47:18.971 INFO:teuthology.orchestra.run.smithi092.stdout:4173 write 0xd42c3d6 thru 0xd4313d6 (0x5001 bytes) 2024-09-04T17:47:18.974 INFO:teuthology.orchestra.run.smithi092.stdout:4174 read 0xc9918e0 thru 0xc9a0b50 (0xf271 bytes) 2024-09-04T17:47:18.974 INFO:teuthology.orchestra.run.smithi092.stdout:4175 read 0x5031fe thru 0x50ed09 (0xbb0c bytes) 2024-09-04T17:47:18.975 INFO:teuthology.orchestra.run.smithi092.stdout:4177 write 0x339c987 thru 0x33aa25c (0xd8d6 bytes) 2024-09-04T17:47:18.978 INFO:teuthology.orchestra.run.smithi092.stdout:4178 writesame 0x59e5767 thru 0x59f2d07 data_size 0x1(0xd5a1 bytes) 2024-09-04T17:47:19.017 INFO:teuthology.orchestra.run.smithi092.stdout:4179 write 0x4d11684 thru 0x4d137c9 (0x2146 bytes) 2024-09-04T17:47:19.019 INFO:teuthology.orchestra.run.smithi092.stdout:4180 read 0x9df4a51 thru 0x9e01def (0xd39f bytes) 2024-09-04T17:47:19.020 INFO:teuthology.orchestra.run.smithi092.stdout:4181 read 0xc2419e9 thru 0xc245e25 (0x443d bytes) 2024-09-04T17:47:19.021 INFO:teuthology.orchestra.run.smithi092.stdout:4182 write 0x336e6ea thru 0x3376a03 (0x831a bytes) 2024-09-04T17:47:19.025 INFO:teuthology.orchestra.run.smithi092.stdout:4185 writesame 0x7e1486a thru 0x7e15b9f data_size 0x1(0x1336 bytes) 2024-09-04T17:47:19.029 INFO:teuthology.orchestra.run.smithi092.stdout:4186 write 0x93b17d5 thru 0x93b40b9 (0x28e5 bytes) 2024-09-04T17:47:19.030 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:19.031 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:19.032 INFO:teuthology.orchestra.run.smithi092.stdout:4188 punch from 0x2bd58fc to 0x2bd93b8, (0x3abc bytes) 2024-09-04T17:47:19.033 INFO:teuthology.orchestra.run.smithi092.stdout:4191 writesame 0xa4d6e52 thru 0xa4e5f27 data_size 0x1d(0xf0d6 bytes) 2024-09-04T17:47:19.034 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:19.030+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:19.036 INFO:teuthology.orchestra.run.smithi092.stdout:4192 punch from 0x6770880 to 0x6771a12, (0x1192 bytes) 2024-09-04T17:47:19.036 INFO:teuthology.orchestra.run.smithi092.stdout:4193 read 0x52c5870 thru 0x52d06dd (0xae6e bytes) 2024-09-04T17:47:19.037 INFO:teuthology.orchestra.run.smithi092.stdout:4195 write 0x21aaf90 thru 0x21b4e04 (0x9e75 bytes) 2024-09-04T17:47:19.047 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-04T17:47:19.052 INFO:teuthology.orchestra.run.smithi092.stdout:4196 write 0xe2139e1 thru 0xe21db70 (0xa190 bytes) 2024-09-04T17:47:19.054 INFO:teuthology.orchestra.run.smithi092.stdout:4199 punch from 0x568e3ac to 0x56985f8, (0xa24c bytes) 2024-09-04T17:47:19.054 INFO:teuthology.orchestra.run.smithi092.stdout:4200 writesame 0x8fdf7bf thru 0x8fe45eb data_size 0x7(0x4e2d bytes) 2024-09-04T17:47:19.060 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:19.060 INFO:teuthology.orchestra.run.smithi092.stdout:4204 write 0x739057 thru 0x73ec5b (0x5c05 bytes) 2024-09-04T17:47:19.062 INFO:teuthology.orchestra.run.smithi092.stdout:4205 writesame 0xc988a99 thru 0xc98e93d data_size 0x1(0x5ea5 bytes) 2024-09-04T17:47:19.094 INFO:teuthology.orchestra.run.smithi092.stdout:4206 trunc from 0xe21db71 to 0xcfd3713 2024-09-04T17:47:19.106 INFO:teuthology.orchestra.run.smithi092.stdout:4207 read 0x8890c44 thru 0x889c9d1 (0xbd8e bytes) 2024-09-04T17:47:19.107 INFO:teuthology.orchestra.run.smithi092.stdout:4210 write 0xcea97b2 thru 0xceb771b (0xdf6a bytes) 2024-09-04T17:47:19.109 INFO:teuthology.orchestra.run.smithi092.stdout:4212 read 0x670d5d8 thru 0x67129aa (0x53d3 bytes) 2024-09-04T17:47:19.110 INFO:teuthology.orchestra.run.smithi092.stdout:4214 read 0xcf14938 thru 0xcf14f42 (0x60b bytes) 2024-09-04T17:47:19.110 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:19.110 INFO:teuthology.orchestra.run.smithi092.stdout:4217 read 0xf18519 thru 0xf26e20 (0xe908 bytes) 2024-09-04T17:47:19.111 INFO:teuthology.orchestra.run.smithi092.stdout:4218 read 0x434c947 thru 0x4359117 (0xc7d1 bytes) 2024-09-04T17:47:19.112 INFO:teuthology.orchestra.run.smithi092.stdout:4221 read 0x8536247 thru 0x8540e73 (0xac2d bytes) 2024-09-04T17:47:19.112 INFO:teuthology.orchestra.run.smithi092.stdout:4222 writesame 0x7e9ebe4 thru 0x7ea5259 data_size 0xa(0x6676 bytes) 2024-09-04T17:47:19.117 INFO:teuthology.orchestra.run.smithi092.stdout:4223 write 0xc109732 thru 0xc11731b (0xdbea bytes) 2024-09-04T17:47:19.121 INFO:teuthology.orchestra.run.smithi092.stdout:4224 writesame 0x36c512e thru 0x36d4429 data_size 0x2(0xf2fc bytes) 2024-09-04T17:47:19.135 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:47:19.132+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:47:19.137 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:19.137 INFO:teuthology.orchestra.run.smithi092.stdout:4227 write 0x979f299 thru 0x97a8594 (0x92fc bytes) 2024-09-04T17:47:19.140 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:19.140 INFO:teuthology.orchestra.run.smithi092.stdout:4230 write 0x634e201 thru 0x635d751 (0xf551 bytes) 2024-09-04T17:47:19.143 INFO:teuthology.orchestra.run.smithi092.stdout:4233 write 0xb807c5c thru 0xb80e452 (0x67f7 bytes) 2024-09-04T17:47:19.146 INFO:teuthology.orchestra.run.smithi092.stdout:4236 read 0x1a3e348 thru 0x1a4cfcd (0xec86 bytes) 2024-09-04T17:47:19.148 INFO:teuthology.orchestra.run.smithi092.stdout:4237 writesame 0x7844543 thru 0x7852f89 data_size 0x5(0xea47 bytes) 2024-09-04T17:47:19.164 INFO:teuthology.orchestra.run.smithi092.stdout:4238 writesame 0x846d83 thru 0x84b5bf data_size 0x1(0x483d bytes) 2024-09-04T17:47:19.178 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:19.178 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:19.179 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:19.179 INFO:teuthology.orchestra.run.smithi092.stdout:4240 read 0xcf79855 thru 0xcf81792 (0x7f3e bytes) 2024-09-04T17:47:19.179 INFO:teuthology.orchestra.run.smithi092.stdout:4241 write 0x9973da1 thru 0x99833ec (0xf64c bytes) 2024-09-04T17:47:19.184 INFO:teuthology.orchestra.run.smithi092.stdout:4243 read 0x2f77215 thru 0x2f78a55 (0x1841 bytes) 2024-09-04T17:47:19.185 INFO:teuthology.orchestra.run.smithi092.stdout:4244 read 0xcbc9698 thru 0xcbd1488 (0x7df1 bytes) 2024-09-04T17:47:19.185 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:19.194 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-04T17:47:19.202 INFO:teuthology.orchestra.run.smithi092.stdout:4246 trunc from 0xcfd3713 to 0x66c2b61 2024-09-04T17:47:19.221 INFO:teuthology.orchestra.run.smithi092.stdout:4248 trunc from 0x66c2b61 to 0x8f60856 2024-09-04T17:47:19.230 INFO:teuthology.orchestra.run.smithi092.stdout:4249 writesame 0x447764 thru 0x44d13a data_size 0x1(0x59d7 bytes) 2024-09-04T17:47:19.237 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:19.234+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:19.242 INFO:teuthology.orchestra.run.smithi092.stdout:4250 write 0x86cd9bb thru 0x86d47b4 (0x6dfa bytes) 2024-09-04T17:47:19.245 INFO:teuthology.orchestra.run.smithi092.stdout:4251 writesame 0x2e84a0 thru 0x2f750c data_size 0x1(0xf06d bytes) 2024-09-04T17:47:19.324 INFO:teuthology.orchestra.run.smithi092.stdout:4253 write 0xa38bea8 thru 0xa3943af (0x8508 bytes) 2024-09-04T17:47:19.327 INFO:teuthology.orchestra.run.smithi092.stdout:4254 read 0x649d5a6 thru 0x64a7670 (0xa0cb bytes) 2024-09-04T17:47:19.328 INFO:teuthology.orchestra.run.smithi092.stdout:4255 writesame 0xa37230a thru 0xa375a98 data_size 0xb(0x378f bytes) 2024-09-04T17:47:19.338 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:19.334+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:19.342 INFO:teuthology.orchestra.run.smithi092.stdout:4256 trunc from 0xa3943b0 to 0xe6af66b 2024-09-04T17:47:19.353 INFO:teuthology.orchestra.run.smithi092.stdout:4257 writesame 0x49a5841 thru 0x49a5b2a data_size 0x1(0x2ea bytes) 2024-09-04T17:47:19.356 INFO:teuthology.orchestra.run.smithi092.stdout:4260 read 0xd321f1 thru 0xd413ba (0xf1ca bytes) 2024-09-04T17:47:19.358 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:19.358 INFO:teuthology.orchestra.run.smithi092.stdout:4262 writesame 0xdbd3d1e thru 0xdbd7293 data_size 0x3(0x3576 bytes) 2024-09-04T17:47:19.365 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:19.365 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:19.365 INFO:teuthology.orchestra.run.smithi092.stdout:4263 writesame 0x748044e thru 0x748234f data_size 0x3f(0x1f02 bytes) 2024-09-04T17:47:19.369 INFO:teuthology.orchestra.run.smithi092.stdout:4266 punch from 0xb525519 to 0xb53139a, (0xbe81 bytes) 2024-09-04T17:47:19.369 INFO:teuthology.orchestra.run.smithi092.stdout:4268 read 0x550bd9b thru 0x5512c65 (0x6ecb bytes) 2024-09-04T17:47:19.370 INFO:teuthology.orchestra.run.smithi092.stdout:4269 read 0xb41c252 thru 0xb427b79 (0xb928 bytes) 2024-09-04T17:47:19.379 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-04T17:47:19.391 INFO:teuthology.orchestra.run.smithi092.stdout:4271 trunc from 0xe6af66b to 0x624cd25 2024-09-04T17:47:19.405 INFO:teuthology.orchestra.run.smithi092.stdout:4272 punch from 0x4b4b48b to 0x4b520eb, (0x6c60 bytes) 2024-09-04T17:47:19.406 INFO:teuthology.orchestra.run.smithi092.stdout:4273 read 0x5378aeb thru 0x537b04b (0x2561 bytes) 2024-09-04T17:47:19.422 INFO:teuthology.orchestra.run.smithi092.stdout:4274 write 0x82290ee thru 0x822ffb0 (0x6ec3 bytes) 2024-09-04T17:47:19.424 INFO:teuthology.orchestra.run.smithi092.stdout:4275 punch from 0x263c220 to 0x26417df, (0x55bf bytes) 2024-09-04T17:47:19.439 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:19.434+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:19.441 INFO:teuthology.orchestra.run.smithi092.stdout:4278 write 0xab724ab thru 0xab80c76 (0xe7cc bytes) 2024-09-04T17:47:19.444 INFO:teuthology.orchestra.run.smithi092.stdout:4279 punch from 0x92f7966 to 0x92fd60a, (0x5ca4 bytes) 2024-09-04T17:47:19.444 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:19.444 INFO:teuthology.orchestra.run.smithi092.stdout:4282 read 0x76e0860 thru 0x76f04ff (0xfca0 bytes) 2024-09-04T17:47:19.449 INFO:teuthology.orchestra.run.smithi092.stdout:4283 trunc from 0xab80c77 to 0x9458998 2024-09-04T17:47:19.461 INFO:teuthology.orchestra.run.smithi092.stdout:4284 read 0x4dce97a thru 0x4dd91e7 (0xa86e bytes) 2024-09-04T17:47:19.461 INFO:teuthology.orchestra.run.smithi092.stdout:4285 read 0x8dd61a0 thru 0x8dd9d7b (0x3bdc bytes) 2024-09-04T17:47:19.482 INFO:teuthology.orchestra.run.smithi092.stdout:4286 writesame 0xd200287 thru 0xd202d5c data_size 0x2(0x2ad6 bytes) 2024-09-04T17:47:19.483 INFO:teuthology.orchestra.run.smithi092.stdout:4287 write 0x56f3d4 thru 0x57992f (0xa55c bytes) 2024-09-04T17:47:19.509 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:19.509 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:19.513 INFO:teuthology.orchestra.run.smithi092.stdout:4288 trunc from 0xd202d5d to 0x24f55ca 2024-09-04T17:47:19.525 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-04T17:47:19.532 INFO:teuthology.orchestra.run.smithi092.stdout:4289 write 0x18b6b49 thru 0x18bc6bc (0x5b74 bytes) 2024-09-04T17:47:19.535 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:19.536 INFO:teuthology.orchestra.run.smithi092.stdout:4291 trunc from 0x24f55ca to 0x2a61d56 2024-09-04T17:47:19.547 INFO:teuthology.orchestra.run.smithi092.stdout:4292 read 0xa0cb8d thru 0xa1b6ca (0xeb3e bytes) 2024-09-04T17:47:19.548 INFO:teuthology.orchestra.run.smithi092.stdout:4294 read 0xde85a3 thru 0xdeef74 (0x69d2 bytes) 2024-09-04T17:47:19.549 INFO:teuthology.orchestra.run.smithi092.stdout:4296 punch from 0x24e68f0 to 0x24f3bfe, (0xd30e bytes) 2024-09-04T17:47:19.572 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:19.572+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:19.588 INFO:teuthology.orchestra.run.smithi092.stdout:4298 write 0xc1143bd thru 0xc120304 (0xbf48 bytes) 2024-09-04T17:47:19.591 INFO:teuthology.orchestra.run.smithi092.stdout:4300 clone 32 order 25 su 2097152 sc 4 2024-09-04T17:47:19.641 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:19.640+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:19.648 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:19.649 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:19.663 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-04T17:47:19.741 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:19.738+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:19.760 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is set 2024-09-04T17:47:19.837 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:19.837 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:19.842 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:47:19.840+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:47:19.851 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-04T17:47:19.887 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone31 from 0xc120305 (overlap 0x1afacb) to 0x5655b 2024-09-04T17:47:19.943 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:19.938+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:20.028 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:20.028 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:20.043 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-04T17:47:20.045 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:20.042+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:20.084 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #30, image image_client.0-clone30 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent31 2024-09-04T17:47:20.109 INFO:teuthology.orchestra.run.smithi092.stdout:4301 trunc from 0xc120305 to 0xc10fdb1 2024-09-04T17:47:20.136 INFO:teuthology.orchestra.run.smithi092.stdout:4302 punch from 0x89a0aae to 0x89a5f35, (0x5487 bytes) 2024-09-04T17:47:20.137 INFO:teuthology.orchestra.run.smithi092.stdout:4304 read 0x740ad85 thru 0x7416296 (0xb512 bytes) 2024-09-04T17:47:20.137 INFO:teuthology.orchestra.run.smithi092.stdout:4305 read 0xa952a6c thru 0xa95f12e (0xc6c3 bytes) 2024-09-04T17:47:20.145 INFO:teuthology.orchestra.run.smithi092.stdout:4308 trunc from 0xc10fdb1 to 0xd25bc22 2024-09-04T17:47:20.145 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:20.142+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:20.151 INFO:teuthology.orchestra.run.smithi092.stdout:4309 write 0x811b83e thru 0x811f0cd (0x3890 bytes) 2024-09-04T17:47:20.154 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:20.154 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:20.165 INFO:teuthology.orchestra.run.smithi092.stdout:4312 trunc from 0xd25bc22 to 0x9632adb 2024-09-04T17:47:20.177 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:20.177 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:20.193 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-04T17:47:20.201 INFO:teuthology.orchestra.run.smithi092.stdout:4313 read 0x5edac8a thru 0x5ee5b20 (0xae97 bytes) 2024-09-04T17:47:20.202 INFO:teuthology.orchestra.run.smithi092.stdout:4315 writesame 0x6957f0b thru 0x6967500 data_size 0x13(0xf5f6 bytes) 2024-09-04T17:47:20.229 INFO:teuthology.orchestra.run.smithi092.stdout:4318 punch from 0x4f9fa67 to 0x4fa58d9, (0x5e72 bytes) 2024-09-04T17:47:20.229 INFO:teuthology.orchestra.run.smithi092.stdout:4319 read 0x2d4c5d2 thru 0x2d51e9d (0x58cc bytes) 2024-09-04T17:47:20.229 INFO:teuthology.orchestra.run.smithi092.stdout:4320 read 0x5d5ea49 thru 0x5d62504 (0x3abc bytes) 2024-09-04T17:47:20.230 INFO:teuthology.orchestra.run.smithi092.stdout:4321 writesame 0x1cdac90 thru 0x1ce7a40 data_size 0x1(0xcdb1 bytes) 2024-09-04T17:47:20.246 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:20.244+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:20.288 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:20.289 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:20.289 INFO:teuthology.orchestra.run.smithi092.stdout:4326 read 0x6822d0d thru 0x6827f0d (0x5201 bytes) 2024-09-04T17:47:20.289 INFO:teuthology.orchestra.run.smithi092.stdout:4328 read 0x13457ea thru 0x13525dd (0xcdf4 bytes) 2024-09-04T17:47:20.290 INFO:teuthology.orchestra.run.smithi092.stdout:4329 read 0x80cb46e thru 0x80cbc0c (0x79f bytes) 2024-09-04T17:47:20.291 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:20.291 INFO:teuthology.orchestra.run.smithi092.stdout:4334 writesame 0x79b2d9 thru 0x7ab225 data_size 0x1(0xff4d bytes) 2024-09-04T17:47:20.340 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:20.340 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:20.346 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:20.342+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:20.351 INFO:teuthology.orchestra.run.smithi092.stdout:4335 write 0x37a096c thru 0x37afdbe (0xf453 bytes) 2024-09-04T17:47:20.353 INFO:teuthology.orchestra.run.smithi092.stdout:4336 punch from 0x28f5d33 to 0x28fd1a4, (0x7471 bytes) 2024-09-04T17:47:20.355 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-04T17:47:20.362 INFO:teuthology.orchestra.run.smithi092.stdout:4338 trunc from 0x9632adb to 0xd44840a 2024-09-04T17:47:20.374 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:20.378 INFO:teuthology.orchestra.run.smithi092.stdout:4340 trunc from 0xd44840a to 0xa863c5c 2024-09-04T17:47:20.396 INFO:teuthology.orchestra.run.smithi092.stdout:4341 read 0x44ee28b thru 0x44efc34 (0x19aa bytes) 2024-09-04T17:47:20.396 INFO:teuthology.orchestra.run.smithi092.stdout:4342 punch from 0x3ac35b2 to 0x3acb349, (0x7d97 bytes) 2024-09-04T17:47:20.403 INFO:teuthology.orchestra.run.smithi092.stdout:4343 trunc from 0xa863c5c to 0x61c2f12 2024-09-04T17:47:20.447 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:20.442+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:20.447 INFO:teuthology.orchestra.run.smithi092.stdout:4345 trunc from 0x61c2f12 to 0x846d324 2024-09-04T17:47:20.460 INFO:teuthology.orchestra.run.smithi092.stdout:4346 read 0x53fde5e thru 0x540ac0c (0xcdaf bytes) 2024-09-04T17:47:20.461 INFO:teuthology.orchestra.run.smithi092.stdout:4347 writesame 0x1e07278 thru 0x1e0c9c0 data_size 0x29(0x5749 bytes) 2024-09-04T17:47:20.463 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:20.463 INFO:teuthology.orchestra.run.smithi092.stdout:4350 write 0x7eb03a2 thru 0x7eb4e62 (0x4ac1 bytes) 2024-09-04T17:47:20.465 INFO:teuthology.orchestra.run.smithi092.stdout:4351 writesame 0x3294c7 thru 0x32b7d1 data_size 0x1(0x230b bytes) 2024-09-04T17:47:20.474 INFO:teuthology.orchestra.run.smithi092.stdout:4352 read 0x5627b98 thru 0x563286e (0xacd7 bytes) 2024-09-04T17:47:20.475 INFO:teuthology.orchestra.run.smithi092.stdout:4355 read 0x822699f thru 0x8228312 (0x1974 bytes) 2024-09-04T17:47:20.475 INFO:teuthology.orchestra.run.smithi092.stdout:4356 write 0x7069432 thru 0x7074cae (0xb87d bytes) 2024-09-04T17:47:20.479 INFO:teuthology.orchestra.run.smithi092.stdout:4359 read 0x2ce09a7 thru 0x2cedd3d (0xd397 bytes) 2024-09-04T17:47:20.503 INFO:teuthology.orchestra.run.smithi092.stdout:4360 write 0xe831546 thru 0xe83c8ea (0xb3a5 bytes) 2024-09-04T17:47:20.503 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:20.503 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:20.510 INFO:teuthology.orchestra.run.smithi092.stdout:4362 read 0xd8b1b55 thru 0xd8b5b3e (0x3fea bytes) 2024-09-04T17:47:20.512 INFO:teuthology.orchestra.run.smithi092.stdout:4364 write 0x8fea8bd thru 0x8ff03cc (0x5b10 bytes) 2024-09-04T17:47:20.519 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-04T17:47:20.520 INFO:teuthology.orchestra.run.smithi092.stdout:4365 trunc from 0xe83c8eb to 0xe3a165b 2024-09-04T17:47:20.531 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-04T17:47:20.535 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:20.536 INFO:teuthology.orchestra.run.smithi092.stdout:4368 punch from 0xbe4e767 to 0xbe5baec, (0xd385 bytes) 2024-09-04T17:47:20.536 INFO:teuthology.orchestra.run.smithi092.stdout:4369 writesame 0x2387b13 thru 0x238a92b data_size 0x1(0x2e19 bytes) 2024-09-04T17:47:20.545 INFO:teuthology.orchestra.run.smithi092.stdout:4371 write 0x5aa90ce thru 0x5ab518b (0xc0be bytes) 2024-09-04T17:47:20.547 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:20.544+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:20.548 INFO:teuthology.orchestra.run.smithi092.stdout:4372 read 0xc01f402 thru 0xc0288e3 (0x94e2 bytes) 2024-09-04T17:47:20.553 INFO:teuthology.orchestra.run.smithi092.stdout:4373 write 0x11a07b7 thru 0x11ace5b (0xc6a5 bytes) 2024-09-04T17:47:20.554 INFO:teuthology.orchestra.run.smithi092.stdout:4376 punch from 0x70c2c58 to 0x70ccc5e, (0xa006 bytes) 2024-09-04T17:47:20.554 INFO:teuthology.orchestra.run.smithi092.stdout:4377 writesame 0x7f3452d thru 0x7f3a0aa data_size 0x31(0x5b7e bytes) 2024-09-04T17:47:20.554 INFO:teuthology.orchestra.run.smithi092.stdout:4378 write 0xd9e9203 thru 0xd9f6c28 (0xda26 bytes) 2024-09-04T17:47:20.558 INFO:teuthology.orchestra.run.smithi092.stdout:4379 writesame 0x6272ed0 thru 0x627d6fc data_size 0x71(0xa82d bytes) 2024-09-04T17:47:20.574 INFO:teuthology.orchestra.run.smithi092.stdout:4380 trunc from 0xe3a165b to 0x5c3e99a 2024-09-04T17:47:20.597 INFO:teuthology.orchestra.run.smithi092.stdout:4381 read 0x5358390 thru 0x5367ca6 (0xf917 bytes) 2024-09-04T17:47:20.617 INFO:teuthology.orchestra.run.smithi092.stdout:4382 write 0xb821a17 thru 0xb82359f (0x1b89 bytes) 2024-09-04T17:47:20.624 INFO:teuthology.orchestra.run.smithi092.stdout:4383 punch from 0x8b2e76e to 0x8b3249d, (0x3d2f bytes) 2024-09-04T17:47:20.624 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:20.624 INFO:teuthology.orchestra.run.smithi092.stdout:4385 read 0x478d300 thru 0x4796ce0 (0x99e1 bytes) 2024-09-04T17:47:20.625 INFO:teuthology.orchestra.run.smithi092.stdout:4386 write 0x7b177d9 thru 0x7b1f422 (0x7c4a bytes) 2024-09-04T17:47:20.627 INFO:teuthology.orchestra.run.smithi092.stdout:4387 writesame 0x722206a thru 0x722b96a data_size 0x1(0x9901 bytes) 2024-09-04T17:47:20.648 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:20.646+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:20.664 INFO:teuthology.orchestra.run.smithi092.stdout:4389 write 0xe5efb88 thru 0xe5fbaa4 (0xbf1d bytes) 2024-09-04T17:47:20.665 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:20.665 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:20.670 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:20.670 INFO:teuthology.orchestra.run.smithi092.stdout:4391 write 0x9741be4 thru 0x974f20d (0xd62a bytes) 2024-09-04T17:47:20.675 INFO:teuthology.orchestra.run.smithi092.stdout:4394 writesame 0x22168c9 thru 0x22241bc data_size 0xa(0xd8f4 bytes) 2024-09-04T17:47:20.679 INFO:teuthology.orchestra.run.smithi092.stdout:4395 writesame 0x54cf741 thru 0x54d565b data_size 0x1(0x5f1b bytes) 2024-09-04T17:47:20.682 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-04T17:47:20.693 INFO:teuthology.orchestra.run.smithi092.stdout:4396 read 0xd4b3938 thru 0xd4b9a03 (0x60cc bytes) 2024-09-04T17:47:20.694 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:20.703 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:47:20.703 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:47:20.707 INFO:teuthology.orchestra.run.smithi092.stdout:4398 trunc from 0xe5fbaa5 to 0x509ccfb 2024-09-04T17:47:20.723 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:47:20.723 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:47:20.723 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-04T17:47:20.729 INFO:teuthology.orchestra.run.smithi092.stdout:4399 read 0x40e73b4 thru 0x40f0588 (0x91d5 bytes) 2024-09-04T17:47:20.729 INFO:teuthology.orchestra.run.smithi092.stdout:4400 writesame 0x480284c thru 0x480eeb9 data_size 0xb(0xc66e bytes) 2024-09-04T17:47:20.732 INFO:teuthology.orchestra.run.smithi092.stdout:4401 read 0x3fe6a0f thru 0x3feaae5 (0x40d7 bytes) 2024-09-04T17:47:20.733 INFO:teuthology.orchestra.run.smithi092.stdout:4403 read 0x1bb8fdb thru 0x1bbf3d8 (0x63fe bytes) 2024-09-04T17:47:20.739 INFO:teuthology.orchestra.run.smithi092.stdout:4405 trunc from 0x509ccfb to 0x119b099 2024-09-04T17:47:20.742 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-04T17:47:20.742 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-04T17:47:20.749 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:20.746+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:20.765 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:47:20.765 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:47:20.769 INFO:teuthology.orchestra.run.smithi092.stdout:4407 trunc from 0x119b099 to 0xaeb0380 2024-09-04T17:47:20.783 INFO:teuthology.orchestra.run.smithi092.stdout:4408 read 0x8feba2b thru 0x8ff2cf9 (0x72cf bytes) 2024-09-04T17:47:20.787 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:47:20.787 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:47:20.787 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-04T17:47:20.801 INFO:teuthology.orchestra.run.smithi092.stdout:4409 write 0xebfbfd8 thru 0xec094fd (0xd526 bytes) 2024-09-04T17:47:20.807 INFO:teuthology.orchestra.run.smithi092.stdout:4410 write 0xfba2b6 thru 0xfbb307 (0x1052 bytes) 2024-09-04T17:47:20.809 INFO:teuthology.orchestra.run.smithi092.stdout:4411 write 0xbdd3ece thru 0xbdde5c2 (0xa6f5 bytes) 2024-09-04T17:47:20.811 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:47:20.811 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:47:20.811 INFO:teuthology.orchestra.run.smithi092.stdout:4413 punch from 0x154bc3a to 0x1550562, (0x4928 bytes) 2024-09-04T17:47:20.811 INFO:teuthology.orchestra.run.smithi092.stdout:4414 write 0xde48173 thru 0xde4dd2d (0x5bbb bytes) 2024-09-04T17:47:20.814 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-04T17:47:20.815 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-04T17:47:20.816 INFO:teuthology.orchestra.run.smithi092.stdout:4415 writesame 0x5ceb6ad thru 0x5cfab3b data_size 0x29(0xf48f bytes) 2024-09-04T17:47:20.818 INFO:teuthology.orchestra.run.smithi092.stdout:4416 write 0x2a3c866 thru 0x2a4286e (0x6009 bytes) 2024-09-04T17:47:20.825 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:20.825 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:47:20.825 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:47:20.825 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:20.826 INFO:teuthology.orchestra.run.smithi092.stdout:4417 trunc from 0xec094fe to 0xb184418 2024-09-04T17:47:20.842 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-04T17:47:20.849 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:47:20.848+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:47:20.851 INFO:teuthology.orchestra.run.smithi092.stdout:4418 write 0xba462de thru 0xba5599e (0xf6c1 bytes) 2024-09-04T17:47:20.854 INFO:teuthology.orchestra.run.smithi092.stdout:4419 write 0xf496fb thru 0xf513d3 (0x7cd9 bytes) 2024-09-04T17:47:20.870 INFO:teuthology.orchestra.run.smithi092.stdout:4420 write 0xe207edc thru 0xe20d2bf (0x53e4 bytes) 2024-09-04T17:47:20.872 INFO:teuthology.orchestra.run.smithi092.stdout:4421 write 0x2b43929 thru 0x2b51888 (0xdf60 bytes) 2024-09-04T17:47:20.875 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:20.875 INFO:teuthology.orchestra.run.smithi092.stdout:4425 write 0x4364539 thru 0x4367ed8 (0x39a0 bytes) 2024-09-04T17:47:20.878 INFO:teuthology.orchestra.run.smithi092.stdout:4426 read 0x8fdb0d6 thru 0x8fe72d4 (0xc1ff bytes) 2024-09-04T17:47:20.895 INFO:teuthology.orchestra.run.smithi092.stdout:4429 trunc from 0xe20d2c0 to 0x261bdba 2024-09-04T17:47:20.928 INFO:teuthology.orchestra.run.smithi092.stdout:4430 writesame 0x320e9e7 thru 0x321e7b9 data_size 0x1(0xfdd3 bytes) 2024-09-04T17:47:20.930 INFO:teuthology.orchestra.run.smithi092.stdout:4432 write 0x1872b6 thru 0x18b7a8 (0x44f3 bytes) 2024-09-04T17:47:20.933 INFO:teuthology.orchestra.run.smithi092.stdout:4433 read 0xac1e1e thru 0xac31c2 (0x13a5 bytes) 2024-09-04T17:47:20.937 INFO:teuthology.orchestra.run.smithi092.stdout:4434 trunc from 0x321e7ba to 0xa329cb 2024-09-04T17:47:20.950 INFO:teuthology.orchestra.run.smithi092.stdout:4435 read 0x7b9cf4 thru 0x7be9f6 (0x4d03 bytes) 2024-09-04T17:47:20.950 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:20.948+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:20.968 INFO:teuthology.orchestra.run.smithi092.stdout:4436 writesame 0x4ef0a28 thru 0x4efe0a6 data_size 0x1(0xd67f bytes) 2024-09-04T17:47:20.970 INFO:teuthology.orchestra.run.smithi092.stdout:4437 punch from 0x3df372d to 0x3dfecf5, (0xb5c8 bytes) 2024-09-04T17:47:20.976 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:20.977 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:47:20.977 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:47:20.977 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:47:20.977 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:20.982 INFO:teuthology.orchestra.run.smithi092.stdout:4438 trunc from 0x4efe0a7 to 0xd5194f3 2024-09-04T17:47:20.994 INFO:teuthology.orchestra.run.smithi092.stdout:4439 write 0xe9712 thru 0xee327 (0x4c16 bytes) 2024-09-04T17:47:20.995 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-04T17:47:20.998 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:20.999 INFO:teuthology.orchestra.run.smithi092.stdout:4441 trunc from 0xd5194f3 to 0xe86d629 2024-09-04T17:47:21.011 INFO:teuthology.orchestra.run.smithi092.stdout:4442 write 0xc8315e9 thru 0xc83c525 (0xaf3d bytes) 2024-09-04T17:47:21.018 INFO:teuthology.orchestra.run.smithi092.stdout:4443 read 0x9b57961 thru 0x9b653c6 (0xda66 bytes) 2024-09-04T17:47:21.019 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:21.019 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:21.019 INFO:teuthology.orchestra.run.smithi092.stdout:4447 read 0x6e5ca07 thru 0x6e6b706 (0xed00 bytes) 2024-09-04T17:47:21.030 INFO:teuthology.orchestra.run.smithi092.stdout:4448 write 0xe967a0e thru 0xe96d95d (0x5f50 bytes) 2024-09-04T17:47:21.033 INFO:teuthology.orchestra.run.smithi092.stdout:4449 writesame 0xd6891f5 thru 0xd6964b1 data_size 0x31(0xd2bd bytes) 2024-09-04T17:47:21.038 INFO:teuthology.orchestra.run.smithi092.stdout:4451 write 0x7254be thru 0x733c8b (0xe7ce bytes) 2024-09-04T17:47:21.039 INFO:teuthology.orchestra.run.smithi092.stdout:4452 punch from 0x7c85003 to 0x7c8af1e, (0x5f1b bytes) 2024-09-04T17:47:21.051 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:21.046+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:21.061 INFO:teuthology.orchestra.run.smithi092.stdout:4453 trunc from 0xe96d95e to 0x1404820 2024-09-04T17:47:21.103 INFO:teuthology.orchestra.run.smithi092.stdout:4454 writesame 0x47b8d64 thru 0x47bb070 data_size 0x3(0x230d bytes) 2024-09-04T17:47:21.121 INFO:teuthology.orchestra.run.smithi092.stdout:4456 write 0x783346b thru 0x783b2d1 (0x7e67 bytes) 2024-09-04T17:47:21.124 INFO:teuthology.orchestra.run.smithi092.stdout:4458 punch from 0x2475fd7 to 0x2485a0f, (0xfa38 bytes) 2024-09-04T17:47:21.130 INFO:teuthology.orchestra.run.smithi092.stdout:4460 trunc from 0x783b2d2 to 0x4acd180 2024-09-04T17:47:21.135 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:21.135 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:47:21.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:47:21.135 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:47:21.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.135 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:48444 3.12 3:4ff80fb5:::rbd_data.11346a2b0bc9.000000000000001d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-04T17:47:21.135 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:58.122601+0000", 2024-09-04T17:47:21.135 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 23.012127, 2024-09-04T17:47:21.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057065713999999997, 2024-09-04T17:47:21.136 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:21.136 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:21.136 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:21.136 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:21.136 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:21.136 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:21.136 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 48444 2024-09-04T17:47:21.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.136 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:21.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:21.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122601+0000", 2024-09-04T17:47:21.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:21.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122601+0000", 2024-09-04T17:47:21.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:21.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122603+0000", 2024-09-04T17:47:21.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3499999999999999e-06 2024-09-04T17:47:21.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:21.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122614+0000", 2024-09-04T17:47:21.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.063e-05 2024-09-04T17:47:21.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:21.138 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122643+0000", 2024-09-04T17:47:21.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9434999999999999e-05 2024-09-04T17:47:21.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:21.138 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122657+0000", 2024-09-04T17:47:21.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3920000000000001e-05 2024-09-04T17:47:21.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.139 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:21.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122767+0000", 2024-09-04T17:47:21.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010981599999999999 2024-09-04T17:47:21.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.139 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:21.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122989+0000", 2024-09-04T17:47:21.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022158200000000001 2024-09-04T17:47:21.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.139 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 3", 2024-09-04T17:47:21.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.133349+0000", 2024-09-04T17:47:21.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010360729000000001 2024-09-04T17:47:21.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.140 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:21.140 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.179423+0000", 2024-09-04T17:47:21.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046073408000000003 2024-09-04T17:47:21.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.140 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:21.140 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.179453+0000", 2024-09-04T17:47:21.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0229000000000001e-05 2024-09-04T17:47:21.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.140 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:21.140 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.179545+0000", 2024-09-04T17:47:21.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1580000000000001e-05 2024-09-04T17:47:21.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:21.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.179667+0000", 2024-09-04T17:47:21.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012203500000000001 2024-09-04T17:47:21.141 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.141 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:21.141 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.141 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:55376 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-04T17:47:21.141 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:11.974053+0000", 2024-09-04T17:47:21.142 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.1606752159999996, 2024-09-04T17:47:21.142 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060514722, 2024-09-04T17:47:21.142 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:21.142 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:21.142 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:21.142 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:21.142 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:21.142 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:21.142 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 55376 2024-09-04T17:47:21.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.142 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:21.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.142 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:21.142 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974053+0000", 2024-09-04T17:47:21.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:21.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974053+0000", 2024-09-04T17:47:21.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:21.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974054+0000", 2024-09-04T17:47:21.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.841e-06 2024-09-04T17:47:21.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:21.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974062+0000", 2024-09-04T17:47:21.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5460000000000003e-06 2024-09-04T17:47:21.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:21.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974077+0000", 2024-09-04T17:47:21.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5389000000000001e-05 2024-09-04T17:47:21.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:21.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974085+0000", 2024-09-04T17:47:21.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7659999999999998e-06 2024-09-04T17:47:21.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:21.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974188+0000", 2024-09-04T17:47:21.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000102521 2024-09-04T17:47:21.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:21.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974321+0000", 2024-09-04T17:47:21.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013324699999999999 2024-09-04T17:47:21.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-04T17:47:21.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.986481+0000", 2024-09-04T17:47:21.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012160450999999999 2024-09-04T17:47:21.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.146 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:21.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:12.034415+0000", 2024-09-04T17:47:21.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.047934065999999997 2024-09-04T17:47:21.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.146 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:21.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:12.034438+0000", 2024-09-04T17:47:21.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2923999999999999e-05 2024-09-04T17:47:21.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.146 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:21.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:12.034505+0000", 2024-09-04T17:47:21.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6711000000000006e-05 2024-09-04T17:47:21.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.147 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:21.147 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:12.034567+0000", 2024-09-04T17:47:21.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2260000000000004e-05 2024-09-04T17:47:21.147 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.147 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:21.147 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.147 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61126 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 1212416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:21.147 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819377+0000", 2024-09-04T17:47:21.147 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 7.3153511880000002, 2024-09-04T17:47:21.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037740846000000002, 2024-09-04T17:47:21.148 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:21.148 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:21.148 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:21.148 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:21.148 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:21.148 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:21.148 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61126 2024-09-04T17:47:21.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.148 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:21.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.148 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:21.148 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819377+0000", 2024-09-04T17:47:21.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:21.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819377+0000", 2024-09-04T17:47:21.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:21.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819377+0000", 2024-09-04T17:47:21.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7299999999999996e-07 2024-09-04T17:47:21.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:21.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819384+0000", 2024-09-04T17:47:21.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0530000000000001e-06 2024-09-04T17:47:21.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:21.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819392+0000", 2024-09-04T17:47:21.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5159999999999997e-06 2024-09-04T17:47:21.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:21.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819397+0000", 2024-09-04T17:47:21.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.254e-06 2024-09-04T17:47:21.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.151 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:21.151 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.856830+0000", 2024-09-04T17:47:21.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037432498000000002 2024-09-04T17:47:21.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.151 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:21.151 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.856880+0000", 2024-09-04T17:47:21.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0284999999999997e-05 2024-09-04T17:47:21.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.151 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:21.151 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.857118+0000", 2024-09-04T17:47:21.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000237667 2024-09-04T17:47:21.151 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.152 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:21.152 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.152 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61127 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 1351680~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:21.152 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819406+0000", 2024-09-04T17:47:21.152 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 7.3153221469999998, 2024-09-04T17:47:21.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.038078545999999998, 2024-09-04T17:47:21.152 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:21.152 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:21.152 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:21.152 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:21.152 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:21.152 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:21.153 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61127 2024-09-04T17:47:21.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.153 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:21.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:21.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819406+0000", 2024-09-04T17:47:21.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:21.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819406+0000", 2024-09-04T17:47:21.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:21.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819406+0000", 2024-09-04T17:47:21.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.06e-07 2024-09-04T17:47:21.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:21.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819409+0000", 2024-09-04T17:47:21.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4059999999999999e-06 2024-09-04T17:47:21.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:21.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819415+0000", 2024-09-04T17:47:21.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6830000000000002e-06 2024-09-04T17:47:21.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:21.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819420+0000", 2024-09-04T17:47:21.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8760000000000004e-06 2024-09-04T17:47:21.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:21.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.857136+0000", 2024-09-04T17:47:21.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.037715752999999998 2024-09-04T17:47:21.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:21.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.857206+0000", 2024-09-04T17:47:21.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9887000000000004e-05 2024-09-04T17:47:21.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.156 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:21.156 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.857484+0000", 2024-09-04T17:47:21.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000278435 2024-09-04T17:47:21.156 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.156 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:21.156 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.156 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61128 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 1458176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:21.156 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819489+0000", 2024-09-04T17:47:21.156 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 7.315238881, 2024-09-04T17:47:21.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.038380478000000003, 2024-09-04T17:47:21.157 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:21.157 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:21.157 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:21.157 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:21.157 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:21.157 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:21.157 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61128 2024-09-04T17:47:21.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.157 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:21.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:21.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819489+0000", 2024-09-04T17:47:21.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:21.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819489+0000", 2024-09-04T17:47:21.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:21.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819490+0000", 2024-09-04T17:47:21.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6599999999999996e-07 2024-09-04T17:47:21.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:21.159 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819493+0000", 2024-09-04T17:47:21.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2279999999999998e-06 2024-09-04T17:47:21.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.159 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:21.159 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819501+0000", 2024-09-04T17:47:21.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4039999999999995e-06 2024-09-04T17:47:21.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.159 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:21.159 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819508+0000", 2024-09-04T17:47:21.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2400000000000004e-06 2024-09-04T17:47:21.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.160 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:21.160 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.857534+0000", 2024-09-04T17:47:21.160 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.038026272 2024-09-04T17:47:21.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.160 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:21.160 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.857629+0000", 2024-09-04T17:47:21.160 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5397999999999997e-05 2024-09-04T17:47:21.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.160 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:21.160 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.857869+0000", 2024-09-04T17:47:21.160 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024027 2024-09-04T17:47:21.160 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.161 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:21.161 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.161 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.161 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61129 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 1564672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:21.161 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819517+0000", 2024-09-04T17:47:21.161 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 7.3152110600000002, 2024-09-04T17:47:21.161 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.038772745999999997, 2024-09-04T17:47:21.161 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:21.161 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:21.161 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:21.161 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:21.161 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:21.161 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:21.162 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61129 2024-09-04T17:47:21.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.162 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:21.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:21.162 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819517+0000", 2024-09-04T17:47:21.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:21.162 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819517+0000", 2024-09-04T17:47:21.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.163 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:21.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819517+0000", 2024-09-04T17:47:21.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5999999999999999e-07 2024-09-04T17:47:21.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.163 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.163 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:21.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819520+0000", 2024-09-04T17:47:21.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2709999999999999e-06 2024-09-04T17:47:21.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.163 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.163 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:21.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819526+0000", 2024-09-04T17:47:21.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7059999999999998e-06 2024-09-04T17:47:21.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.164 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:21.164 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819531+0000", 2024-09-04T17:47:21.164 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0549999999999999e-06 2024-09-04T17:47:21.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.164 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:21.164 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.857913+0000", 2024-09-04T17:47:21.164 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.038381954000000003 2024-09-04T17:47:21.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.164 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:21.165 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.858000+0000", 2024-09-04T17:47:21.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6963000000000001e-05 2024-09-04T17:47:21.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.165 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:21.165 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.858290+0000", 2024-09-04T17:47:21.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028933700000000003 2024-09-04T17:47:21.165 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.169 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:21.169 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.169 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61130 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 1597440~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:21.169 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819539+0000", 2024-09-04T17:47:21.169 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 7.3151892329999999, 2024-09-04T17:47:21.169 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.039166595999999998, 2024-09-04T17:47:21.170 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:21.170 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:21.170 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:21.170 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:21.170 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:21.170 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:21.170 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61130 2024-09-04T17:47:21.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.170 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:21.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.170 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:21.170 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819539+0000", 2024-09-04T17:47:21.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.171 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:21.171 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819539+0000", 2024-09-04T17:47:21.171 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.171 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:21.171 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819539+0000", 2024-09-04T17:47:21.171 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9400000000000005e-07 2024-09-04T17:47:21.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.171 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:21.171 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819542+0000", 2024-09-04T17:47:21.171 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2630000000000002e-06 2024-09-04T17:47:21.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.172 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:21.172 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819548+0000", 2024-09-04T17:47:21.172 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3269999999999997e-06 2024-09-04T17:47:21.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.172 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:21.172 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819553+0000", 2024-09-04T17:47:21.172 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9849999999999997e-06 2024-09-04T17:47:21.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.172 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:21.173 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.858321+0000", 2024-09-04T17:47:21.173 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.038768395999999997 2024-09-04T17:47:21.173 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.173 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.173 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:21.173 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.858406+0000", 2024-09-04T17:47:21.173 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4456999999999995e-05 2024-09-04T17:47:21.173 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.173 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.173 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:21.173 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.858705+0000", 2024-09-04T17:47:21.173 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00029947399999999999 2024-09-04T17:47:21.173 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.173 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:21.174 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.174 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.174 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.174 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61131 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 1703936~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:21.174 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819560+0000", 2024-09-04T17:47:21.174 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 7.3151678809999998, 2024-09-04T17:47:21.174 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.039522082, 2024-09-04T17:47:21.174 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:21.174 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:21.174 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:21.174 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:21.174 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:21.174 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:21.175 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61131 2024-09-04T17:47:21.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.175 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:21.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.175 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:21.175 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819560+0000", 2024-09-04T17:47:21.175 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.175 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:21.175 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819560+0000", 2024-09-04T17:47:21.175 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.176 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:21.176 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819561+0000", 2024-09-04T17:47:21.176 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5300000000000004e-07 2024-09-04T17:47:21.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.176 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:21.176 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819563+0000", 2024-09-04T17:47:21.176 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.187e-06 2024-09-04T17:47:21.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.176 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:21.176 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819569+0000", 2024-09-04T17:47:21.176 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6170000000000003e-06 2024-09-04T17:47:21.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.177 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.177 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:21.177 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819574+0000", 2024-09-04T17:47:21.177 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9640000000000002e-06 2024-09-04T17:47:21.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.177 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.177 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:21.177 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.858754+0000", 2024-09-04T17:47:21.177 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.039179295000000003 2024-09-04T17:47:21.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.178 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.178 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:21.178 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.858800+0000", 2024-09-04T17:47:21.178 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.587e-05 2024-09-04T17:47:21.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.178 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.178 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:21.178 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.859082+0000", 2024-09-04T17:47:21.178 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028249599999999997 2024-09-04T17:47:21.179 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.179 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:21.179 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.179 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.179 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.179 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61132 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 1810432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:21.179 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819581+0000", 2024-09-04T17:47:21.179 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 7.3151464390000003, 2024-09-04T17:47:21.179 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.039953120000000002, 2024-09-04T17:47:21.179 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:21.179 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:21.179 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:21.179 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:21.179 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:21.180 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:21.180 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61132 2024-09-04T17:47:21.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.180 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:21.180 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.180 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:21.180 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819581+0000", 2024-09-04T17:47:21.180 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.180 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.180 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:21.180 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819581+0000", 2024-09-04T17:47:21.180 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.181 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.181 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.181 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:21.181 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819582+0000", 2024-09-04T17:47:21.181 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3399999999999999e-07 2024-09-04T17:47:21.181 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.181 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.181 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:21.181 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819584+0000", 2024-09-04T17:47:21.181 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1320000000000001e-06 2024-09-04T17:47:21.181 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.181 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.181 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:21.181 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819591+0000", 2024-09-04T17:47:21.182 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3759999999999996e-06 2024-09-04T17:47:21.182 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.182 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.182 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:21.182 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819595+0000", 2024-09-04T17:47:21.182 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7339999999999999e-06 2024-09-04T17:47:21.182 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.182 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.182 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:21.182 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.859119+0000", 2024-09-04T17:47:21.182 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.039523677 2024-09-04T17:47:21.182 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.182 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.183 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:21.183 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.859232+0000", 2024-09-04T17:47:21.183 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000113001 2024-09-04T17:47:21.183 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.183 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:21.183 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.859535+0000", 2024-09-04T17:47:21.183 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030256599999999999 2024-09-04T17:47:21.183 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.183 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:21.183 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.183 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.184 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61133 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 1843200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:21.184 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819602+0000", 2024-09-04T17:47:21.184 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 7.3151261309999995, 2024-09-04T17:47:21.184 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.040226171999999998, 2024-09-04T17:47:21.184 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:21.184 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:21.184 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:21.184 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:21.184 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:21.184 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:21.184 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61133 2024-09-04T17:47:21.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.184 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:21.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.185 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:21.185 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819602+0000", 2024-09-04T17:47:21.185 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.185 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.185 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:21.185 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819602+0000", 2024-09-04T17:47:21.185 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.185 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.185 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:21.185 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819602+0000", 2024-09-04T17:47:21.185 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.75e-07 2024-09-04T17:47:21.186 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.186 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:21.186 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819605+0000", 2024-09-04T17:47:21.186 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.52e-06 2024-09-04T17:47:21.186 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.186 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:21.186 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819611+0000", 2024-09-04T17:47:21.186 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5200000000000003e-06 2024-09-04T17:47:21.186 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.186 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:21.186 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819616+0000", 2024-09-04T17:47:21.187 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0939999999999998e-06 2024-09-04T17:47:21.187 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.187 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.187 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:21.187 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.859538+0000", 2024-09-04T17:47:21.187 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.039921660999999997 2024-09-04T17:47:21.187 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.187 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.187 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:21.187 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.859594+0000", 2024-09-04T17:47:21.187 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5671999999999998e-05 2024-09-04T17:47:21.187 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.187 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.187 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:21.188 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.859828+0000", 2024-09-04T17:47:21.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023423000000000001 2024-09-04T17:47:21.188 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.188 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:21.188 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.188 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61134 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 1949696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:21.188 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819623+0000", 2024-09-04T17:47:21.188 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 7.3151046539999998, 2024-09-04T17:47:21.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.040606862000000001, 2024-09-04T17:47:21.188 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:21.188 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:21.189 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:21.189 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:21.189 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:21.189 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:21.189 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61134 2024-09-04T17:47:21.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.189 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:21.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:21.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819623+0000", 2024-09-04T17:47:21.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:21.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819623+0000", 2024-09-04T17:47:21.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:21.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819624+0000", 2024-09-04T17:47:21.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2100000000000002e-07 2024-09-04T17:47:21.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:21.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819626+0000", 2024-09-04T17:47:21.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2460000000000002e-06 2024-09-04T17:47:21.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:21.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819632+0000", 2024-09-04T17:47:21.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1659999999999999e-06 2024-09-04T17:47:21.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:21.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819637+0000", 2024-09-04T17:47:21.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2680000000000002e-06 2024-09-04T17:47:21.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:21.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.859872+0000", 2024-09-04T17:47:21.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.040234377000000002 2024-09-04T17:47:21.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.192 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:21.192 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.859950+0000", 2024-09-04T17:47:21.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8128999999999997e-05 2024-09-04T17:47:21.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.192 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:21.192 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.860230+0000", 2024-09-04T17:47:21.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028025499999999998 2024-09-04T17:47:21.192 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.192 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:21.192 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.193 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61135 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 2056192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:21.193 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819644+0000", 2024-09-04T17:47:21.193 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 7.3150836139999997, 2024-09-04T17:47:21.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.040925930999999999, 2024-09-04T17:47:21.193 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:21.193 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:21.193 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:21.193 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:21.193 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:21.193 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:21.193 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61135 2024-09-04T17:47:21.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.193 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:21.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:21.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819644+0000", 2024-09-04T17:47:21.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:21.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819644+0000", 2024-09-04T17:47:21.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:21.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819645+0000", 2024-09-04T17:47:21.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9399999999999995e-07 2024-09-04T17:47:21.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:21.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819647+0000", 2024-09-04T17:47:21.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.311e-06 2024-09-04T17:47:21.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:21.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819653+0000", 2024-09-04T17:47:21.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3940000000000001e-06 2024-09-04T17:47:21.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:21.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819658+0000", 2024-09-04T17:47:21.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8420000000000004e-06 2024-09-04T17:47:21.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.196 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:21.196 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.860263+0000", 2024-09-04T17:47:21.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.040605109 2024-09-04T17:47:21.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.196 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:21.196 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.860339+0000", 2024-09-04T17:47:21.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5612999999999996e-05 2024-09-04T17:47:21.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:21.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.860570+0000", 2024-09-04T17:47:21.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023116799999999999 2024-09-04T17:47:21.197 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.197 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:21.197 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.197 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61136 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 2195456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:21.197 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819665+0000", 2024-09-04T17:47:21.197 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 7.3150631600000002, 2024-09-04T17:47:21.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.041275408999999999, 2024-09-04T17:47:21.197 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:21.198 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:21.198 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:21.198 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:21.198 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:21.198 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:21.198 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61136 2024-09-04T17:47:21.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.198 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:21.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:21.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819665+0000", 2024-09-04T17:47:21.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:21.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819665+0000", 2024-09-04T17:47:21.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:21.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819665+0000", 2024-09-04T17:47:21.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8599999999999999e-07 2024-09-04T17:47:21.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:21.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819667+0000", 2024-09-04T17:47:21.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9439999999999999e-06 2024-09-04T17:47:21.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.200 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:21.200 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819673+0000", 2024-09-04T17:47:21.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0220000000000001e-06 2024-09-04T17:47:21.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.200 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:21.200 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819677+0000", 2024-09-04T17:47:21.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4139999999999996e-06 2024-09-04T17:47:21.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.200 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:21.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.860603+0000", 2024-09-04T17:47:21.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.040925504000000001 2024-09-04T17:47:21.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:21.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.860673+0000", 2024-09-04T17:47:21.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9707000000000002e-05 2024-09-04T17:47:21.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:21.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.860940+0000", 2024-09-04T17:47:21.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026743200000000001 2024-09-04T17:47:21.201 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.201 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:21.202 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.202 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61137 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 2301952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:21.202 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819685+0000", 2024-09-04T17:47:21.202 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 7.3150423670000002, 2024-09-04T17:47:21.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.041579208999999999, 2024-09-04T17:47:21.202 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:21.202 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:21.202 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:21.202 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:21.202 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:21.202 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:21.203 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61137 2024-09-04T17:47:21.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.203 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:21.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:21.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819685+0000", 2024-09-04T17:47:21.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:21.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819685+0000", 2024-09-04T17:47:21.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.204 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:21.204 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819686+0000", 2024-09-04T17:47:21.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3000000000000001e-07 2024-09-04T17:47:21.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.204 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:21.204 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819688+0000", 2024-09-04T17:47:21.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1150000000000001e-06 2024-09-04T17:47:21.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.204 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:21.204 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819694+0000", 2024-09-04T17:47:21.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.75e-06 2024-09-04T17:47:21.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.205 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:21.205 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819699+0000", 2024-09-04T17:47:21.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8110000000000002e-06 2024-09-04T17:47:21.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.205 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:21.205 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.860977+0000", 2024-09-04T17:47:21.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.041277945000000003 2024-09-04T17:47:21.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.205 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:21.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.861071+0000", 2024-09-04T17:47:21.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4252000000000005e-05 2024-09-04T17:47:21.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:21.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.861265+0000", 2024-09-04T17:47:21.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019390600000000001 2024-09-04T17:47:21.206 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.206 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:21.206 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.206 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61138 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 2441216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:21.207 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819706+0000", 2024-09-04T17:47:21.207 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 7.3150219659999998, 2024-09-04T17:47:21.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.041876780000000002, 2024-09-04T17:47:21.207 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:21.207 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:21.207 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:21.207 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:21.207 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:21.207 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:21.207 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61138 2024-09-04T17:47:21.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.207 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:21.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.207 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:21.208 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819706+0000", 2024-09-04T17:47:21.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.208 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:21.208 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819706+0000", 2024-09-04T17:47:21.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.208 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:21.208 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819707+0000", 2024-09-04T17:47:21.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7299999999999995e-07 2024-09-04T17:47:21.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.209 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:21.209 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819727+0000", 2024-09-04T17:47:21.209 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0610000000000001e-05 2024-09-04T17:47:21.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.209 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.209 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:21.209 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819735+0000", 2024-09-04T17:47:21.209 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.123e-06 2024-09-04T17:47:21.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.209 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.209 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:21.209 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819741+0000", 2024-09-04T17:47:21.209 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4009999999999996e-06 2024-09-04T17:47:21.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.210 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:21.210 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.861298+0000", 2024-09-04T17:47:21.210 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.041557135000000002 2024-09-04T17:47:21.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.210 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:21.210 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.861369+0000", 2024-09-04T17:47:21.210 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0794e-05 2024-09-04T17:47:21.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.210 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:21.210 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.861583+0000", 2024-09-04T17:47:21.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021404400000000001 2024-09-04T17:47:21.211 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.211 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:21.211 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.211 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61139 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 2547712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:21.211 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819750+0000", 2024-09-04T17:47:21.211 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 7.3149775019999996, 2024-09-04T17:47:21.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042185699, 2024-09-04T17:47:21.211 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:21.211 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:21.211 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:21.211 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:21.212 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:21.212 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:21.212 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61139 2024-09-04T17:47:21.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.212 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:21.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.212 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:21.212 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819750+0000", 2024-09-04T17:47:21.212 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.212 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:21.212 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819750+0000", 2024-09-04T17:47:21.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.213 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:21.213 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819751+0000", 2024-09-04T17:47:21.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4000000000000002e-07 2024-09-04T17:47:21.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.213 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:21.213 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819753+0000", 2024-09-04T17:47:21.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5179999999999999e-06 2024-09-04T17:47:21.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.213 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:21.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819760+0000", 2024-09-04T17:47:21.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5440000000000004e-06 2024-09-04T17:47:21.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.214 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:21.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819765+0000", 2024-09-04T17:47:21.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8760000000000004e-06 2024-09-04T17:47:21.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.214 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:21.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.861622+0000", 2024-09-04T17:47:21.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.041857093999999997 2024-09-04T17:47:21.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:21.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.861698+0000", 2024-09-04T17:47:21.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6080000000000003e-05 2024-09-04T17:47:21.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:21.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.861936+0000", 2024-09-04T17:47:21.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023804699999999999 2024-09-04T17:47:21.215 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.215 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:21.215 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.216 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61140 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 2686976~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:21.216 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819772+0000", 2024-09-04T17:47:21.216 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 7.3149553679999997, 2024-09-04T17:47:21.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042515052999999997, 2024-09-04T17:47:21.216 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:21.216 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:21.216 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:21.216 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:21.216 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:21.216 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:21.216 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61140 2024-09-04T17:47:21.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.216 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:21.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.217 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:21.217 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819772+0000", 2024-09-04T17:47:21.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.217 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:21.217 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819772+0000", 2024-09-04T17:47:21.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.217 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:21.217 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819773+0000", 2024-09-04T17:47:21.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2900000000000003e-07 2024-09-04T17:47:21.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:21.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819775+0000", 2024-09-04T17:47:21.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3309999999999998e-06 2024-09-04T17:47:21.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:21.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819781+0000", 2024-09-04T17:47:21.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.981e-06 2024-09-04T17:47:21.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:21.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819786+0000", 2024-09-04T17:47:21.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5719999999999996e-06 2024-09-04T17:47:21.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:21.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.861966+0000", 2024-09-04T17:47:21.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042179541000000001 2024-09-04T17:47:21.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:21.220 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862038+0000", 2024-09-04T17:47:21.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2719000000000006e-05 2024-09-04T17:47:21.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:21.220 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862288+0000", 2024-09-04T17:47:21.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024928 2024-09-04T17:47:21.220 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.220 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:21.220 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.220 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61141 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 2793472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:21.221 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819909+0000", 2024-09-04T17:47:21.221 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 7.3148192239999998, 2024-09-04T17:47:21.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042786419999999999, 2024-09-04T17:47:21.221 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:21.221 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:21.221 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:21.221 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:21.221 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:21.221 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:21.221 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61141 2024-09-04T17:47:21.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.221 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:21.221 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.221 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:21.222 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819909+0000", 2024-09-04T17:47:21.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:21.222 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819909+0000", 2024-09-04T17:47:21.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:21.222 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819909+0000", 2024-09-04T17:47:21.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6300000000000005e-07 2024-09-04T17:47:21.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:21.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819913+0000", 2024-09-04T17:47:21.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4189999999999998e-06 2024-09-04T17:47:21.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.223 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:21.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819920+0000", 2024-09-04T17:47:21.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8619999999999997e-06 2024-09-04T17:47:21.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.223 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:21.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819926+0000", 2024-09-04T17:47:21.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9399999999999999e-06 2024-09-04T17:47:21.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:21.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862331+0000", 2024-09-04T17:47:21.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042405348000000002 2024-09-04T17:47:21.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:21.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862410+0000", 2024-09-04T17:47:21.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9078000000000004e-05 2024-09-04T17:47:21.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:21.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862695+0000", 2024-09-04T17:47:21.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028510999999999999 2024-09-04T17:47:21.225 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.225 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:21.225 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.225 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61142 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 2932736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:21.225 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819934+0000", 2024-09-04T17:47:21.225 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 7.3147936060000003, 2024-09-04T17:47:21.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043101985000000002, 2024-09-04T17:47:21.225 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:21.225 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:21.225 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:21.225 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:21.225 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:21.226 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:21.226 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61142 2024-09-04T17:47:21.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.226 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:21.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:21.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819934+0000", 2024-09-04T17:47:21.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:21.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819934+0000", 2024-09-04T17:47:21.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:21.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819935+0000", 2024-09-04T17:47:21.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2024-09-04T17:47:21.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:21.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819937+0000", 2024-09-04T17:47:21.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5770000000000001e-06 2024-09-04T17:47:21.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:21.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819943+0000", 2024-09-04T17:47:21.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7950000000000004e-06 2024-09-04T17:47:21.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.228 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:21.228 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819948+0000", 2024-09-04T17:47:21.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6779999999999999e-06 2024-09-04T17:47:21.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.228 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:21.228 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862739+0000", 2024-09-04T17:47:21.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042791454999999999 2024-09-04T17:47:21.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.228 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:21.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862824+0000", 2024-09-04T17:47:21.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4790999999999995e-05 2024-09-04T17:47:21.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.229 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:21.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.863036+0000", 2024-09-04T17:47:21.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021224899999999999 2024-09-04T17:47:21.229 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.229 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:21.229 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.229 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61143 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 3039232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:21.229 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819954+0000", 2024-09-04T17:47:21.230 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 7.3147734880000002, 2024-09-04T17:47:21.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043485134000000002, 2024-09-04T17:47:21.230 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:21.230 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:21.230 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:21.230 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:21.230 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:21.230 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:21.230 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61143 2024-09-04T17:47:21.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.230 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:21.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:21.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819954+0000", 2024-09-04T17:47:21.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:21.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819954+0000", 2024-09-04T17:47:21.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:21.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819955+0000", 2024-09-04T17:47:21.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7300000000000002e-07 2024-09-04T17:47:21.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:21.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819957+0000", 2024-09-04T17:47:21.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.024e-06 2024-09-04T17:47:21.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:21.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819963+0000", 2024-09-04T17:47:21.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8109999999999999e-06 2024-09-04T17:47:21.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:21.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819967+0000", 2024-09-04T17:47:21.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6149999999999999e-06 2024-09-04T17:47:21.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:21.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.863082+0000", 2024-09-04T17:47:21.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043115078000000001 2024-09-04T17:47:21.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:21.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.863172+0000", 2024-09-04T17:47:21.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0181999999999998e-05 2024-09-04T17:47:21.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:21.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.863440+0000", 2024-09-04T17:47:21.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026705099999999997 2024-09-04T17:47:21.234 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.234 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:21.234 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.234 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.234 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:21.234 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:21.235 INFO:teuthology.orchestra.run.smithi092.stdout:4461 trunc from 0x4acd180 to 0x20a47e7 2024-09-04T17:47:21.236 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-04T17:47:21.236 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:21.237 INFO:teuthology.orchestra.run.smithi092.stdout:4463 punch from 0x11f98f0 to 0x11fbe06, (0x2516 bytes) 2024-09-04T17:47:21.237 INFO:teuthology.orchestra.run.smithi092.stdout:4464 writesame 0x1dbd83e thru 0x1dbdeba data_size 0x1(0x67d bytes) 2024-09-04T17:47:21.237 INFO:teuthology.orchestra.run.smithi092.stdout:4465 read 0x8c7f4e thru 0x8d0675 (0x8728 bytes) 2024-09-04T17:47:21.237 INFO:teuthology.orchestra.run.smithi092.stdout:4466 write 0x86349cd thru 0x863b4c1 (0x6af5 bytes) 2024-09-04T17:47:21.237 INFO:teuthology.orchestra.run.smithi092.stdout:4467 writesame 0xe66ac22 thru 0xe676337 data_size 0x2b(0xb716 bytes) 2024-09-04T17:47:21.237 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:21.232+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:21.238 INFO:teuthology.orchestra.run.smithi092.stdout:4469 write 0xcc3ecd1 thru 0xcc470b7 (0x83e7 bytes) 2024-09-04T17:47:21.245 INFO:teuthology.orchestra.run.smithi092.stdout:4470 read 0x639d9c7 thru 0x639ebc8 (0x1202 bytes) 2024-09-04T17:47:21.246 INFO:teuthology.orchestra.run.smithi092.stdout:4472 read 0x2c22bff thru 0x2c311c5 (0xe5c7 bytes) 2024-09-04T17:47:21.278 INFO:teuthology.orchestra.run.smithi092.stdout:4473 trunc from 0xe676338 to 0x1fa5eb2 2024-09-04T17:47:21.302 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:21.336 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:21.334+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:21.337 INFO:teuthology.orchestra.run.smithi092.stdout:4475 writesame 0xc3b6a80 thru 0xc3bfb1d data_size 0x6b(0x909e bytes) 2024-09-04T17:47:21.337 INFO:teuthology.orchestra.run.smithi092.stdout:4476 write 0x399b470 thru 0x399c3de (0xf6f bytes) 2024-09-04T17:47:21.340 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:21.341 INFO:teuthology.orchestra.run.smithi092.stdout:4478 read 0xbd0ad39 thru 0xbd1684d (0xbb15 bytes) 2024-09-04T17:47:21.358 INFO:teuthology.orchestra.run.smithi092.stdout:4479 trunc from 0xc3bfb1e to 0x46d8c9b 2024-09-04T17:47:21.374 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:21.374 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:47:21.374 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:47:21.374 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:21.379 INFO:teuthology.orchestra.run.smithi092.stdout:4480 trunc from 0x46d8c9b to 0x1f69c56 2024-09-04T17:47:21.393 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-04T17:47:21.395 INFO:teuthology.orchestra.run.smithi092.stdout:4481 punch from 0x1a4ae13 to 0x1a4b329, (0x516 bytes) 2024-09-04T17:47:21.395 INFO:teuthology.orchestra.run.smithi092.stdout:4482 punch from 0x1332a65 to 0x1333527, (0xac2 bytes) 2024-09-04T17:47:21.395 INFO:teuthology.orchestra.run.smithi092.stdout:4483 punch from 0x134e033 to 0x1351b3f, (0x3b0c bytes) 2024-09-04T17:47:21.397 INFO:teuthology.orchestra.run.smithi092.stdout:4484 trunc from 0x1f69c56 to 0xa589f5 2024-09-04T17:47:21.437 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:21.436+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:21.445 INFO:teuthology.orchestra.run.smithi092.stdout:4485 write 0xd011bea thru 0xd01a2f4 (0x870b bytes) 2024-09-04T17:47:21.452 INFO:teuthology.orchestra.run.smithi092.stdout:4487 writesame 0x13a42ac thru 0x13a60d5 data_size 0x2(0x1e2a bytes) 2024-09-04T17:47:21.466 INFO:teuthology.orchestra.run.smithi092.stdout:4488 trunc from 0xd01a2f5 to 0x6ba6e72 2024-09-04T17:47:21.493 INFO:teuthology.orchestra.run.smithi092.stdout:4489 read 0xd6a187 thru 0xd74a39 (0xa8b3 bytes) 2024-09-04T17:47:21.494 INFO:teuthology.orchestra.run.smithi092.stdout:4490 read 0x8fe6ad thru 0x90c5a7 (0xdefb bytes) 2024-09-04T17:47:21.494 INFO:teuthology.orchestra.run.smithi092.stdout:4491 read 0x4eb6077 thru 0x4eb6cb6 (0xc40 bytes) 2024-09-04T17:47:21.520 INFO:teuthology.orchestra.run.smithi092.stdout:4492 write 0xea13d03 thru 0xea227a6 (0xeaa4 bytes) 2024-09-04T17:47:21.527 INFO:teuthology.orchestra.run.smithi092.stdout:4494 writesame 0xd0bcd60 thru 0xd0c3adc data_size 0x1(0x6d7d bytes) 2024-09-04T17:47:21.538 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:47:21.536+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:47:21.549 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:21.549 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:47:21.549 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:47:21.549 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:47:21.550 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:21.564 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-04T17:47:21.589 INFO:teuthology.orchestra.run.smithi092.stdout:4495 trunc from 0xea227a7 to 0x2a50756 2024-09-04T17:47:21.613 INFO:teuthology.orchestra.run.smithi092.stdout:4496 read 0x9040c8 thru 0x90c609 (0x8542 bytes) 2024-09-04T17:47:21.613 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:21.618 INFO:teuthology.orchestra.run.smithi092.stdout:4498 trunc from 0x2a50756 to 0x5ab2738 2024-09-04T17:47:21.631 INFO:teuthology.orchestra.run.smithi092.stdout:4500 write 0x5278a1f thru 0x527e715 (0x5cf7 bytes) 2024-09-04T17:47:21.639 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:21.634+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:21.645 INFO:teuthology.orchestra.run.smithi092.stdout:4501 trunc from 0x5ab2738 to 0xc11d3a3 2024-09-04T17:47:21.656 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:21.656 INFO:teuthology.orchestra.run.smithi092.stdout:4505 writesame 0x43362c4 thru 0x4346287 data_size 0x2(0xffc4 bytes) 2024-09-04T17:47:21.680 INFO:teuthology.orchestra.run.smithi092.stdout:4506 read 0x8b9134 thru 0x8c04dd (0x73aa bytes) 2024-09-04T17:47:21.681 INFO:teuthology.orchestra.run.smithi092.stdout:4508 writesame 0x78c11a1 thru 0x78cba5e data_size 0x1(0xa8be bytes) 2024-09-04T17:47:21.721 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:21.721 INFO:teuthology.orchestra.run.smithi092.stdout:4511 read 0x7574b67 thru 0x757f6a6 (0xab40 bytes) 2024-09-04T17:47:21.723 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:21.723 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:47:21.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:47:21.723 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:47:21.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.723 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59950 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5365760~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:47:21.724 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654917+0000", 2024-09-04T17:47:21.724 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.0675362039999996, 2024-09-04T17:47:21.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066822450000000005, 2024-09-04T17:47:21.724 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:21.724 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:21.724 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:21.724 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:21.724 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:21.724 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:21.724 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59950 2024-09-04T17:47:21.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.725 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:21.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:21.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654917+0000", 2024-09-04T17:47:21.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:21.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654917+0000", 2024-09-04T17:47:21.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.726 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:21.726 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654918+0000", 2024-09-04T17:47:21.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.29e-07 2024-09-04T17:47:21.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.726 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:21.726 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654919+0000", 2024-09-04T17:47:21.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2589999999999999e-06 2024-09-04T17:47:21.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.727 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.727 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:21.727 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654922+0000", 2024-09-04T17:47:21.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1350000000000001e-06 2024-09-04T17:47:21.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.727 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.727 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:21.727 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654925+0000", 2024-09-04T17:47:21.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1250000000000001e-06 2024-09-04T17:47:21.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:21.728 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721466+0000", 2024-09-04T17:47:21.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066540959999999996 2024-09-04T17:47:21.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:21.728 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721574+0000", 2024-09-04T17:47:21.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010747399999999999 2024-09-04T17:47:21.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:21.729 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721740+0000", 2024-09-04T17:47:21.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016626799999999999 2024-09-04T17:47:21.729 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.729 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:21.729 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.729 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59951 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5472256~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:47:21.729 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654929+0000", 2024-09-04T17:47:21.729 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.0675250189999996, 2024-09-04T17:47:21.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067110534999999999, 2024-09-04T17:47:21.730 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:21.730 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:21.730 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:21.730 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:21.730 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:21.730 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:21.730 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59951 2024-09-04T17:47:21.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.730 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:21.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.731 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:21.731 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654929+0000", 2024-09-04T17:47:21.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.731 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:21.731 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654929+0000", 2024-09-04T17:47:21.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.731 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:21.731 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654929+0000", 2024-09-04T17:47:21.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5100000000000001e-07 2024-09-04T17:47:21.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:21.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654930+0000", 2024-09-04T17:47:21.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2440000000000001e-06 2024-09-04T17:47:21.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:21.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654933+0000", 2024-09-04T17:47:21.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1599999999999998e-06 2024-09-04T17:47:21.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:21.733 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654936+0000", 2024-09-04T17:47:21.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6560000000000001e-06 2024-09-04T17:47:21.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:21.733 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721789+0000", 2024-09-04T17:47:21.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066853250000000003 2024-09-04T17:47:21.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.734 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:21.734 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721885+0000", 2024-09-04T17:47:21.734 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5616000000000001e-05 2024-09-04T17:47:21.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.734 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:21.734 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722039+0000", 2024-09-04T17:47:21.734 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000154258 2024-09-04T17:47:21.734 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.734 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:21.734 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.735 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.735 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59952 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5505024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:47:21.735 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654939+0000", 2024-09-04T17:47:21.735 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.0675141420000003, 2024-09-04T17:47:21.735 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067433742000000005, 2024-09-04T17:47:21.735 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:21.735 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:21.735 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:21.735 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:21.735 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:21.735 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:21.735 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59952 2024-09-04T17:47:21.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.736 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:21.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.736 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:21.736 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654939+0000", 2024-09-04T17:47:21.736 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.736 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:21.736 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654939+0000", 2024-09-04T17:47:21.736 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:21.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654940+0000", 2024-09-04T17:47:21.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3300000000000001e-07 2024-09-04T17:47:21.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:21.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654941+0000", 2024-09-04T17:47:21.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1570000000000001e-06 2024-09-04T17:47:21.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:21.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654944+0000", 2024-09-04T17:47:21.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1319999999999998e-06 2024-09-04T17:47:21.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:21.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654947+0000", 2024-09-04T17:47:21.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5969999999999999e-06 2024-09-04T17:47:21.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.739 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:21.739 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722089+0000", 2024-09-04T17:47:21.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067142347000000005 2024-09-04T17:47:21.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.739 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.739 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:21.739 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722201+0000", 2024-09-04T17:47:21.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011235600000000001 2024-09-04T17:47:21.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.739 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.739 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:21.740 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722373+0000", 2024-09-04T17:47:21.740 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017191999999999999 2024-09-04T17:47:21.740 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.740 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:21.740 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.740 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.740 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59953 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5611520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:47:21.740 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654950+0000", 2024-09-04T17:47:21.740 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.0675035019999992, 2024-09-04T17:47:21.740 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067601852000000004, 2024-09-04T17:47:21.741 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:21.741 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:21.741 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:21.741 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:21.741 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:21.741 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:21.741 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59953 2024-09-04T17:47:21.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.741 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:21.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:21.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654950+0000", 2024-09-04T17:47:21.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:21.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654950+0000", 2024-09-04T17:47:21.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:21.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654950+0000", 2024-09-04T17:47:21.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2499999999999999e-07 2024-09-04T17:47:21.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.743 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:21.743 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654954+0000", 2024-09-04T17:47:21.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.287e-06 2024-09-04T17:47:21.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.743 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:21.743 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654962+0000", 2024-09-04T17:47:21.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4230000000000005e-06 2024-09-04T17:47:21.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.743 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:21.744 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654966+0000", 2024-09-04T17:47:21.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6449999999999998e-06 2024-09-04T17:47:21.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.744 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:21.744 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722417+0000", 2024-09-04T17:47:21.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067451188999999995 2024-09-04T17:47:21.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.744 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:21.744 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722467+0000", 2024-09-04T17:47:21.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0556999999999997e-05 2024-09-04T17:47:21.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:21.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722552+0000", 2024-09-04T17:47:21.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4525999999999997e-05 2024-09-04T17:47:21.745 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.745 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:21.745 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.745 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59954 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5718016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:47:21.746 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654970+0000", 2024-09-04T17:47:21.746 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.0674837410000002, 2024-09-04T17:47:21.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067874285000000006, 2024-09-04T17:47:21.746 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:21.746 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:21.746 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:21.746 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:21.746 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:21.746 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:21.746 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59954 2024-09-04T17:47:21.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.746 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:21.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.747 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:21.747 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654970+0000", 2024-09-04T17:47:21.747 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.747 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.747 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:21.747 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654970+0000", 2024-09-04T17:47:21.747 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.747 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.748 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:21.748 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654970+0000", 2024-09-04T17:47:21.748 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5899999999999998e-07 2024-09-04T17:47:21.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.748 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.748 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:21.748 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654971+0000", 2024-09-04T17:47:21.748 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2929999999999999e-06 2024-09-04T17:47:21.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.748 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.748 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:21.749 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654975+0000", 2024-09-04T17:47:21.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4699999999999998e-06 2024-09-04T17:47:21.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.749 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:21.749 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654978+0000", 2024-09-04T17:47:21.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7930000000000002e-06 2024-09-04T17:47:21.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.749 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:21.749 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722600+0000", 2024-09-04T17:47:21.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067622229000000006 2024-09-04T17:47:21.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:21.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722689+0000", 2024-09-04T17:47:21.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9511000000000004e-05 2024-09-04T17:47:21.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:21.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722844+0000", 2024-09-04T17:47:21.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015473 2024-09-04T17:47:21.751 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.751 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:21.751 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.751 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59955 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5857280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:47:21.751 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654981+0000", 2024-09-04T17:47:21.751 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.0674723910000008, 2024-09-04T17:47:21.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068052360000000006, 2024-09-04T17:47:21.752 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:21.752 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:21.752 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:21.752 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:21.752 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:21.752 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:21.752 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59955 2024-09-04T17:47:21.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.752 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:21.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.752 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:21.753 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654981+0000", 2024-09-04T17:47:21.753 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.753 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.753 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:21.753 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654981+0000", 2024-09-04T17:47:21.753 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.753 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.753 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:21.753 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654981+0000", 2024-09-04T17:47:21.753 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2399999999999999e-07 2024-09-04T17:47:21.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:21.754 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654985+0000", 2024-09-04T17:47:21.754 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0980000000000004e-06 2024-09-04T17:47:21.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:21.754 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654990+0000", 2024-09-04T17:47:21.754 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3429999999999998e-06 2024-09-04T17:47:21.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:21.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654993+0000", 2024-09-04T17:47:21.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8700000000000001e-06 2024-09-04T17:47:21.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:21.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722888+0000", 2024-09-04T17:47:21.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067895239999999996 2024-09-04T17:47:21.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:21.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722948+0000", 2024-09-04T17:47:21.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9750999999999998e-05 2024-09-04T17:47:21.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.756 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:21.756 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.723034+0000", 2024-09-04T17:47:21.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5834000000000005e-05 2024-09-04T17:47:21.756 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.756 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:21.756 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.757 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59956 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5963776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:47:21.757 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654998+0000", 2024-09-04T17:47:21.757 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.0674558619999992, 2024-09-04T17:47:21.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068359518999999994, 2024-09-04T17:47:21.757 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:21.757 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:21.757 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:21.757 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:21.757 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:21.757 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:21.757 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59956 2024-09-04T17:47:21.757 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.758 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:21.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.758 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:21.758 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654998+0000", 2024-09-04T17:47:21.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.758 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:21.758 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654998+0000", 2024-09-04T17:47:21.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:21.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654998+0000", 2024-09-04T17:47:21.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4299999999999999e-07 2024-09-04T17:47:21.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:21.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654999+0000", 2024-09-04T17:47:21.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1799999999999999e-06 2024-09-04T17:47:21.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:21.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.655002+0000", 2024-09-04T17:47:21.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3639999999999999e-06 2024-09-04T17:47:21.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:21.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.655005+0000", 2024-09-04T17:47:21.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7970000000000001e-06 2024-09-04T17:47:21.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:21.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.723085+0000", 2024-09-04T17:47:21.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068080229000000006 2024-09-04T17:47:21.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:21.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.723197+0000", 2024-09-04T17:47:21.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000111344 2024-09-04T17:47:21.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:21.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.723357+0000", 2024-09-04T17:47:21.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016036199999999999 2024-09-04T17:47:21.762 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.762 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:21.762 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.762 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61082 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 2654208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:21.762 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.822664+0000", 2024-09-04T17:47:21.762 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 7.8997894750000004, 2024-09-04T17:47:21.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067072242000000004, 2024-09-04T17:47:21.763 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:21.763 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:21.763 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:21.763 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:21.763 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:21.763 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:21.763 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61082 2024-09-04T17:47:21.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.763 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:21.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.763 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:21.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822664+0000", 2024-09-04T17:47:21.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:21.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822664+0000", 2024-09-04T17:47:21.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:21.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822664+0000", 2024-09-04T17:47:21.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6600000000000003e-07 2024-09-04T17:47:21.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:21.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822666+0000", 2024-09-04T17:47:21.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.536e-06 2024-09-04T17:47:21.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:21.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822669+0000", 2024-09-04T17:47:21.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3759999999999999e-06 2024-09-04T17:47:21.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.766 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:21.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822672+0000", 2024-09-04T17:47:21.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8109999999999999e-06 2024-09-04T17:47:21.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.766 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:21.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889263+0000", 2024-09-04T17:47:21.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066591259999999999 2024-09-04T17:47:21.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.767 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:21.767 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889369+0000", 2024-09-04T17:47:21.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000105362 2024-09-04T17:47:21.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.767 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:21.767 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889736+0000", 2024-09-04T17:47:21.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00036763099999999999 2024-09-04T17:47:21.767 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.767 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:21.767 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.768 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61083 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 2760704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:21.768 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.822678+0000", 2024-09-04T17:47:21.768 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 7.899775741, 2024-09-04T17:47:21.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067475996999999996, 2024-09-04T17:47:21.768 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:21.768 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:21.768 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:21.768 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:21.768 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:21.769 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:21.769 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61083 2024-09-04T17:47:21.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.769 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:21.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:21.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822678+0000", 2024-09-04T17:47:21.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:21.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822678+0000", 2024-09-04T17:47:21.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:21.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822725+0000", 2024-09-04T17:47:21.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7429000000000003e-05 2024-09-04T17:47:21.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:21.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822728+0000", 2024-09-04T17:47:21.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2140000000000001e-06 2024-09-04T17:47:21.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:21.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822737+0000", 2024-09-04T17:47:21.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4370000000000007e-06 2024-09-04T17:47:21.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:21.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822743+0000", 2024-09-04T17:47:21.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7520000000000002e-06 2024-09-04T17:47:21.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.772 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:21.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889785+0000", 2024-09-04T17:47:21.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067042265000000004 2024-09-04T17:47:21.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.772 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:21.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889840+0000", 2024-09-04T17:47:21.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5254000000000003e-05 2024-09-04T17:47:21.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.773 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:21.773 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.890154+0000", 2024-09-04T17:47:21.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000313646 2024-09-04T17:47:21.773 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.773 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:21.773 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.774 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61084 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 2867200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:21.774 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.822753+0000", 2024-09-04T17:47:21.774 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 7.8997008050000002, 2024-09-04T17:47:21.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068497161000000001, 2024-09-04T17:47:21.774 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:21.774 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:21.774 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:21.774 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:21.774 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:21.774 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:21.774 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61084 2024-09-04T17:47:21.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.775 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:21.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:21.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822753+0000", 2024-09-04T17:47:21.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:21.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822753+0000", 2024-09-04T17:47:21.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:21.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822753+0000", 2024-09-04T17:47:21.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9100000000000003e-07 2024-09-04T17:47:21.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:21.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822756+0000", 2024-09-04T17:47:21.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.678e-06 2024-09-04T17:47:21.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.777 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:21.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822763+0000", 2024-09-04T17:47:21.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8220000000000001e-06 2024-09-04T17:47:21.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:21.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822768+0000", 2024-09-04T17:47:21.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1930000000000001e-06 2024-09-04T17:47:21.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:21.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.890246+0000", 2024-09-04T17:47:21.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067477996999999998 2024-09-04T17:47:21.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:21.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.890555+0000", 2024-09-04T17:47:21.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000309143 2024-09-04T17:47:21.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:21.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.891250+0000", 2024-09-04T17:47:21.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00069463700000000001 2024-09-04T17:47:21.783 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.783 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:21.783 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.784 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61285 3.8 3:117cfeb9:::rbd_data.11341dfde0c5.0000000000000006:1f [read 2760704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:21.784 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835200+0000", 2024-09-04T17:47:21.784 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 7.8872537380000001, 2024-09-04T17:47:21.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066899803999999993, 2024-09-04T17:47:21.784 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:21.784 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:21.784 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:21.784 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:21.784 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:21.785 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:21.785 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61285 2024-09-04T17:47:21.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.785 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:21.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.785 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:21.785 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835200+0000", 2024-09-04T17:47:21.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:21.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835200+0000", 2024-09-04T17:47:21.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:21.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835200+0000", 2024-09-04T17:47:21.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0899999999999997e-07 2024-09-04T17:47:21.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:21.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835206+0000", 2024-09-04T17:47:21.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3280000000000002e-06 2024-09-04T17:47:21.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:21.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835211+0000", 2024-09-04T17:47:21.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9230000000000001e-06 2024-09-04T17:47:21.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:21.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835216+0000", 2024-09-04T17:47:21.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9019999999999998e-06 2024-09-04T17:47:21.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:21.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.901719+0000", 2024-09-04T17:47:21.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066502981000000003 2024-09-04T17:47:21.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:21.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.901813+0000", 2024-09-04T17:47:21.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3905000000000003e-05 2024-09-04T17:47:21.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:21.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902100+0000", 2024-09-04T17:47:21.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028645599999999998 2024-09-04T17:47:21.789 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.789 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:21.789 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.790 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61286 3.8 3:117cfeb9:::rbd_data.11341dfde0c5.0000000000000006:1f [read 2867200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:21.790 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835244+0000", 2024-09-04T17:47:21.790 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 7.8872091189999995, 2024-09-04T17:47:21.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067143215000000006, 2024-09-04T17:47:21.790 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:21.790 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:21.790 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:21.790 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:21.790 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:21.791 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:21.791 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61286 2024-09-04T17:47:21.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.791 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:21.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:21.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835244+0000", 2024-09-04T17:47:21.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:21.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835244+0000", 2024-09-04T17:47:21.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:21.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835245+0000", 2024-09-04T17:47:21.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8400000000000004e-07 2024-09-04T17:47:21.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:21.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835248+0000", 2024-09-04T17:47:21.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6850000000000001e-06 2024-09-04T17:47:21.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:21.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835254+0000", 2024-09-04T17:47:21.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.494e-06 2024-09-04T17:47:21.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:21.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835260+0000", 2024-09-04T17:47:21.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0580000000000003e-06 2024-09-04T17:47:21.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:21.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902119+0000", 2024-09-04T17:47:21.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066858318 2024-09-04T17:47:21.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:21.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902163+0000", 2024-09-04T17:47:21.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4753000000000003e-05 2024-09-04T17:47:21.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:21.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902388+0000", 2024-09-04T17:47:21.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022422300000000001 2024-09-04T17:47:21.795 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.795 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:21.795 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.795 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61315 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 98304~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:21.795 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835269+0000", 2024-09-04T17:47:21.796 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 7.8871843269999999, 2024-09-04T17:47:21.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067329396, 2024-09-04T17:47:21.796 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:21.796 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:21.796 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:21.796 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:21.796 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:21.796 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:21.796 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61315 2024-09-04T17:47:21.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.796 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:21.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:21.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835269+0000", 2024-09-04T17:47:21.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:21.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835269+0000", 2024-09-04T17:47:21.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:21.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835270+0000", 2024-09-04T17:47:21.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6599999999999996e-07 2024-09-04T17:47:21.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:21.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835272+0000", 2024-09-04T17:47:21.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.261e-06 2024-09-04T17:47:21.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:21.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835278+0000", 2024-09-04T17:47:21.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.443e-06 2024-09-04T17:47:21.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:21.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835284+0000", 2024-09-04T17:47:21.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3290000000000001e-06 2024-09-04T17:47:21.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:21.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902147+0000", 2024-09-04T17:47:21.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066862695999999999 2024-09-04T17:47:21.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:21.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902248+0000", 2024-09-04T17:47:21.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000101501 2024-09-04T17:47:21.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:21.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902599+0000", 2024-09-04T17:47:21.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00035060000000000001 2024-09-04T17:47:21.801 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.801 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:21.801 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.801 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61316 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 204800~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:21.801 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835291+0000", 2024-09-04T17:47:21.802 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 7.8871623079999997, 2024-09-04T17:47:21.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067632052999999998, 2024-09-04T17:47:21.802 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:21.802 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:21.802 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:21.802 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:21.802 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:21.802 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:21.802 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61316 2024-09-04T17:47:21.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.803 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:21.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:21.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835291+0000", 2024-09-04T17:47:21.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:21.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835291+0000", 2024-09-04T17:47:21.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:21.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835292+0000", 2024-09-04T17:47:21.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1899999999999998e-07 2024-09-04T17:47:21.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:21.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835294+0000", 2024-09-04T17:47:21.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0880000000000002e-06 2024-09-04T17:47:21.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:21.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835300+0000", 2024-09-04T17:47:21.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.224e-06 2024-09-04T17:47:21.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:21.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835305+0000", 2024-09-04T17:47:21.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0730000000000004e-06 2024-09-04T17:47:21.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:21.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902632+0000", 2024-09-04T17:47:21.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067327322999999994 2024-09-04T17:47:21.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:21.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902688+0000", 2024-09-04T17:47:21.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5177999999999999e-05 2024-09-04T17:47:21.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:21.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902923+0000", 2024-09-04T17:47:21.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023574800000000001 2024-09-04T17:47:21.807 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.807 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:21.807 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.807 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61317 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 344064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:21.807 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835312+0000", 2024-09-04T17:47:21.807 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 7.887141304, 2024-09-04T17:47:21.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067918511000000001, 2024-09-04T17:47:21.807 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:21.807 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:21.808 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:21.808 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:21.808 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:21.808 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:21.808 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61317 2024-09-04T17:47:21.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.808 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:21.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:21.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835312+0000", 2024-09-04T17:47:21.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:21.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835312+0000", 2024-09-04T17:47:21.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:21.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835313+0000", 2024-09-04T17:47:21.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9000000000000002e-07 2024-09-04T17:47:21.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:21.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835315+0000", 2024-09-04T17:47:21.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1689999999999999e-06 2024-09-04T17:47:21.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:21.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835321+0000", 2024-09-04T17:47:21.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0619999999999997e-06 2024-09-04T17:47:21.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:21.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835326+0000", 2024-09-04T17:47:21.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0390000000000004e-06 2024-09-04T17:47:21.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:21.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902967+0000", 2024-09-04T17:47:21.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067641520999999996 2024-09-04T17:47:21.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:21.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903067+0000", 2024-09-04T17:47:21.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9426999999999995e-05 2024-09-04T17:47:21.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:21.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903231+0000", 2024-09-04T17:47:21.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016390299999999999 2024-09-04T17:47:21.812 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.812 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:21.812 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.813 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61318 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 450560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:21.813 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835334+0000", 2024-09-04T17:47:21.813 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 7.8871197869999996, 2024-09-04T17:47:21.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068091771999999995, 2024-09-04T17:47:21.813 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:21.813 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:21.813 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:21.813 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:21.813 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:21.813 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:21.814 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61318 2024-09-04T17:47:21.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.814 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:21.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:21.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835334+0000", 2024-09-04T17:47:21.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:21.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835334+0000", 2024-09-04T17:47:21.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:21.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835334+0000", 2024-09-04T17:47:21.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5699999999999998e-07 2024-09-04T17:47:21.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:21.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835337+0000", 2024-09-04T17:47:21.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3889999999999999e-06 2024-09-04T17:47:21.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:21.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835343+0000", 2024-09-04T17:47:21.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1809999999999998e-06 2024-09-04T17:47:21.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:21.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835348+0000", 2024-09-04T17:47:21.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3170000000000001e-06 2024-09-04T17:47:21.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:21.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903270+0000", 2024-09-04T17:47:21.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067922191000000007 2024-09-04T17:47:21.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:21.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903321+0000", 2024-09-04T17:47:21.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0889e-05 2024-09-04T17:47:21.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:21.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903426+0000", 2024-09-04T17:47:21.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104348 2024-09-04T17:47:21.818 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.818 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:21.818 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.818 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61319 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 589824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:21.818 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835357+0000", 2024-09-04T17:47:21.818 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 7.88709697, 2024-09-04T17:47:21.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068372535999999998, 2024-09-04T17:47:21.819 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:21.819 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:21.819 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:21.819 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:21.819 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:21.819 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:21.819 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61319 2024-09-04T17:47:21.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.819 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:21.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:21.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835357+0000", 2024-09-04T17:47:21.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:21.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835357+0000", 2024-09-04T17:47:21.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:21.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835357+0000", 2024-09-04T17:47:21.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.32e-07 2024-09-04T17:47:21.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:21.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835359+0000", 2024-09-04T17:47:21.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1660000000000001e-06 2024-09-04T17:47:21.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:21.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835366+0000", 2024-09-04T17:47:21.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3550000000000001e-06 2024-09-04T17:47:21.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:21.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835371+0000", 2024-09-04T17:47:21.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1050000000000003e-06 2024-09-04T17:47:21.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:21.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903471+0000", 2024-09-04T17:47:21.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068100451000000006 2024-09-04T17:47:21.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:21.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903573+0000", 2024-09-04T17:47:21.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000101691 2024-09-04T17:47:21.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:21.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903729+0000", 2024-09-04T17:47:21.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000156336 2024-09-04T17:47:21.823 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.823 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:21.823 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.824 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61320 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:21.824 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835380+0000", 2024-09-04T17:47:21.824 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 7.8870732050000001, 2024-09-04T17:47:21.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068543058000000004, 2024-09-04T17:47:21.824 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:21.824 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:21.824 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:21.824 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:21.824 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:21.824 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:21.824 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61320 2024-09-04T17:47:21.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.825 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:21.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:21.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835380+0000", 2024-09-04T17:47:21.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:21.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835380+0000", 2024-09-04T17:47:21.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:21.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835381+0000", 2024-09-04T17:47:21.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9999999999999998e-07 2024-09-04T17:47:21.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:21.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835383+0000", 2024-09-04T17:47:21.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2230000000000001e-06 2024-09-04T17:47:21.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:21.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835390+0000", 2024-09-04T17:47:21.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.34e-06 2024-09-04T17:47:21.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:21.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835396+0000", 2024-09-04T17:47:21.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9830000000000001e-06 2024-09-04T17:47:21.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:21.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903765+0000", 2024-09-04T17:47:21.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068368441000000002 2024-09-04T17:47:21.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:21.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903835+0000", 2024-09-04T17:47:21.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0631999999999997e-05 2024-09-04T17:47:21.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:21.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903923+0000", 2024-09-04T17:47:21.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8039000000000002e-05 2024-09-04T17:47:21.829 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.829 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:21.829 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.829 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61321 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 835584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:21.829 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835404+0000", 2024-09-04T17:47:21.829 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 7.8870491459999998, 2024-09-04T17:47:21.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068911043000000005, 2024-09-04T17:47:21.829 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:21.830 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:21.830 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:21.830 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:21.830 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:21.830 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:21.830 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61321 2024-09-04T17:47:21.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.830 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:21.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:21.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835404+0000", 2024-09-04T17:47:21.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:21.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835404+0000", 2024-09-04T17:47:21.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:21.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835405+0000", 2024-09-04T17:47:21.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.89e-07 2024-09-04T17:47:21.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:21.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835407+0000", 2024-09-04T17:47:21.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4439999999999998e-06 2024-09-04T17:47:21.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:21.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835414+0000", 2024-09-04T17:47:21.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4930000000000003e-06 2024-09-04T17:47:21.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:21.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835419+0000", 2024-09-04T17:47:21.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8919999999999999e-06 2024-09-04T17:47:21.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:21.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903957+0000", 2024-09-04T17:47:21.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068538500000000002 2024-09-04T17:47:21.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:21.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904019+0000", 2024-09-04T17:47:21.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1419000000000003e-05 2024-09-04T17:47:21.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:21.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904315+0000", 2024-09-04T17:47:21.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00029680599999999999 2024-09-04T17:47:21.834 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.834 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:21.834 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.835 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61322 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:21.835 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835427+0000", 2024-09-04T17:47:21.835 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 7.8870267600000004, 2024-09-04T17:47:21.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.069550138999999997, 2024-09-04T17:47:21.835 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:21.835 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:21.835 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:21.835 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:21.836 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:21.836 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:21.836 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61322 2024-09-04T17:47:21.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.836 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:21.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:21.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835427+0000", 2024-09-04T17:47:21.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.837 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:21.837 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835427+0000", 2024-09-04T17:47:21.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:21.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.837 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:21.837 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.842157+0000", 2024-09-04T17:47:21.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067298419999999998 2024-09-04T17:47:21.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.837 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:21.837 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.842163+0000", 2024-09-04T17:47:21.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2029999999999997e-06 2024-09-04T17:47:21.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:21.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.842175+0000", 2024-09-04T17:47:21.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2072000000000001e-05 2024-09-04T17:47:21.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:21.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.842183+0000", 2024-09-04T17:47:21.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3850000000000002e-06 2024-09-04T17:47:21.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:21.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904371+0000", 2024-09-04T17:47:21.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.062187754999999997 2024-09-04T17:47:21.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:21.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904467+0000", 2024-09-04T17:47:21.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6117000000000001e-05 2024-09-04T17:47:21.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:21.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:21.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:21.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904977+0000", 2024-09-04T17:47:21.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00050976500000000002 2024-09-04T17:47:21.840 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.840 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:21.840 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.840 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:21.840 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:21.840 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:21.841 INFO:teuthology.orchestra.run.smithi092.stdout:4512 writesame 0x511b711 thru 0x512a8fd data_size 0x1(0xf1ed bytes) 2024-09-04T17:47:21.842 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-04T17:47:21.843 INFO:teuthology.orchestra.run.smithi092.stdout:4513 write 0xaf493a9 thru 0xaf4c85d (0x34b5 bytes) 2024-09-04T17:47:21.843 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:21.843 INFO:teuthology.orchestra.run.smithi092.stdout:4516 writesame 0x33a58bf thru 0x33b11c1 data_size 0x1(0xb903 bytes) 2024-09-04T17:47:21.843 INFO:teuthology.orchestra.run.smithi092.stdout:4517 read 0x305d955 thru 0x306b081 (0xd72d bytes) 2024-09-04T17:47:21.843 INFO:teuthology.orchestra.run.smithi092.stdout:4520 punch from 0xa9d5f7 to 0xaa71fd, (0x9c06 bytes) 2024-09-04T17:47:21.844 INFO:teuthology.orchestra.run.smithi092.stdout:4521 write 0x93aad33 thru 0x93b28db (0x7ba9 bytes) 2024-09-04T17:47:21.844 INFO:teuthology.orchestra.run.smithi092.stdout:4522 write 0x20c85ac thru 0x20d4cd4 (0xc729 bytes) 2024-09-04T17:47:21.844 INFO:teuthology.orchestra.run.smithi092.stdout:4523 writesame 0x1edbd7d thru 0x1eeb96f data_size 0x1(0xfbf3 bytes) 2024-09-04T17:47:21.844 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-04T17:47:21.852 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:21.852 INFO:teuthology.orchestra.run.smithi092.stdout:4525 read 0x87fccd9 thru 0x880abbd (0xdee5 bytes) 2024-09-04T17:47:21.853 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:21.850+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:21.853 INFO:teuthology.orchestra.run.smithi092.stdout:4529 punch from 0x90be318 to 0x90c83c5, (0xa0ad bytes) 2024-09-04T17:47:21.854 INFO:teuthology.orchestra.run.smithi092.stdout:4530 write 0x44b5f33 thru 0x44bfe11 (0x9edf bytes) 2024-09-04T17:47:21.856 INFO:teuthology.orchestra.run.smithi092.stdout:4531 punch from 0x6d693d6 to 0x6d70eff, (0x7b29 bytes) 2024-09-04T17:47:21.856 INFO:teuthology.orchestra.run.smithi092.stdout:4532 read 0x7411836 thru 0x74176e1 (0x5eac bytes) 2024-09-04T17:47:21.863 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 6, 7] out_osds: [3] dead_osds: [6, 2] live_osds: [0, 1, 4, 5, 7, 3] 2024-09-04T17:47:21.863 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-04T17:47:21.863 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-04T17:47:21.863 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-04T17:47:21.866 INFO:teuthology.orchestra.run.smithi092.stdout:4533 trunc from 0xc11d3a3 to 0x86d920b 2024-09-04T17:47:21.878 INFO:teuthology.orchestra.run.smithi092.stdout:4534 punch from 0x6b9ba50 to 0x6ba8d81, (0xd331 bytes) 2024-09-04T17:47:21.885 INFO:teuthology.orchestra.run.smithi092.stdout:4535 trunc from 0x86d920b to 0x45c634e 2024-09-04T17:47:21.925 INFO:teuthology.orchestra.run.smithi092.stdout:4536 read 0x8dc5be thru 0x8e5166 (0x8ba9 bytes) 2024-09-04T17:47:21.925 INFO:teuthology.orchestra.run.smithi092.stdout:4538 punch from 0xc94f76 to 0xca4bcf, (0xfc59 bytes) 2024-09-04T17:47:21.925 INFO:teuthology.orchestra.run.smithi092.stdout:4539 punch from 0xacc7b to 0xb17e3, (0x4b68 bytes) 2024-09-04T17:47:21.946 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:21.944+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:21.951 INFO:teuthology.orchestra.run.smithi092.stdout:4540 write 0xcac5d1e thru 0xcac8b4f (0x2e32 bytes) 2024-09-04T17:47:21.969 INFO:teuthology.orchestra.run.smithi092.stdout:4541 write 0xcc76fce thru 0xcc80339 (0x936c bytes) 2024-09-04T17:47:21.975 INFO:teuthology.orchestra.run.smithi092.stdout:4542 write 0x751d2a0 thru 0x75211c1 (0x3f22 bytes) 2024-09-04T17:47:21.977 INFO:teuthology.orchestra.run.smithi092.stdout:4543 writesame 0xb025fe5 thru 0xb02df65 data_size 0x1(0x7f81 bytes) 2024-09-04T17:47:21.985 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:21.985 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:47:21.985 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:47:21.986 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:21.999 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-04T17:47:22.018 INFO:teuthology.orchestra.run.smithi092.stdout:4544 trunc from 0xcc8033a to 0x9e266bd 2024-09-04T17:47:22.030 INFO:teuthology.orchestra.run.smithi092.stdout:4545 punch from 0x727a028 to 0x7282bf4, (0x8bcc bytes) 2024-09-04T17:47:22.040 INFO:teuthology.orchestra.run.smithi092.stdout:4546 trunc from 0x9e266bd to 0x2f26ca4 2024-09-04T17:47:22.046 INFO:teuthology.orchestra.run.smithi092.stderr:no valid command found; 10 closest matches: 2024-09-04T17:47:22.046 INFO:teuthology.orchestra.run.smithi092.stderr:0 2024-09-04T17:47:22.046 INFO:teuthology.orchestra.run.smithi092.stderr:1 2024-09-04T17:47:22.046 INFO:teuthology.orchestra.run.smithi092.stderr:2 2024-09-04T17:47:22.046 INFO:teuthology.orchestra.run.smithi092.stderr:abort 2024-09-04T17:47:22.046 INFO:teuthology.orchestra.run.smithi092.stderr:assert 2024-09-04T17:47:22.046 INFO:teuthology.orchestra.run.smithi092.stderr:bench [] [] [] [] 2024-09-04T17:47:22.046 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs debug_inject_read_zeros 2024-09-04T17:47:22.046 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs files list 2024-09-04T17:47:22.046 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs stats 2024-09-04T17:47:22.047 INFO:teuthology.orchestra.run.smithi092.stderr:bluestore allocator dump block 2024-09-04T17:47:22.047 INFO:teuthology.orchestra.run.smithi092.stderr:Error EINVAL: invalid command 2024-09-04T17:47:22.047 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:22.042+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:22.049 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-04T17:47:22.050 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-04T17:47:22.050 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-04T17:47:22.060 INFO:teuthology.orchestra.run.smithi092.stdout:4547 punch from 0x2d1ad46 to 0x2d26ddb, (0xc095 bytes) 2024-09-04T17:47:22.060 INFO:teuthology.orchestra.run.smithi092.stdout:4548 read 0x9f709d thru 0x9feb52 (0x7ab6 bytes) 2024-09-04T17:47:22.060 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:22.060 INFO:teuthology.orchestra.run.smithi092.stdout:4550 writesame 0x1b3f75 thru 0x1ba769 data_size 0x3(0x67f5 bytes) 2024-09-04T17:47:22.065 INFO:teuthology.orchestra.run.smithi092.stdout:4551 read 0x2966d9a thru 0x2975a2c (0xec93 bytes) 2024-09-04T17:47:22.091 INFO:teuthology.orchestra.run.smithi092.stdout:4552 write 0xe2cd49b thru 0xe2d0e87 (0x39ed bytes) 2024-09-04T17:47:22.096 INFO:teuthology.orchestra.run.smithi092.stdout:4553 punch from 0x20aac7a to 0x20b0f50, (0x62d6 bytes) 2024-09-04T17:47:22.096 INFO:teuthology.orchestra.run.smithi092.stdout:4554 read 0x80ebdb8 thru 0x80f98cf (0xdb18 bytes) 2024-09-04T17:47:22.096 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:22.096 INFO:teuthology.orchestra.run.smithi092.stdout:4556 read 0x4f201f7 thru 0x4f26c70 (0x6a7a bytes) 2024-09-04T17:47:22.096 INFO:teuthology.orchestra.run.smithi092.stdout:4557 read 0xc0dd1d1 thru 0xc0e1105 (0x3f35 bytes) 2024-09-04T17:47:22.096 INFO:teuthology.orchestra.run.smithi092.stdout:4558 punch from 0x6afb864 to 0x6aff6c0, (0x3e5c bytes) 2024-09-04T17:47:22.096 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:22.106 INFO:teuthology.orchestra.run.smithi092.stdout:4560 write 0xe886f38 thru 0xe892c0a (0xbcd3 bytes) 2024-09-04T17:47:22.113 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:22.114 INFO:teuthology.orchestra.run.smithi092.stdout:4562 write 0xb524998 thru 0xb52c2fd (0x7966 bytes) 2024-09-04T17:47:22.117 INFO:teuthology.orchestra.run.smithi092.stdout:4563 writesame 0x308d65d thru 0x3095345 data_size 0x63(0x7ce9 bytes) 2024-09-04T17:47:22.131 INFO:teuthology.orchestra.run.smithi092.stdout:4564 trunc from 0xe892c0b to 0x7d337cd 2024-09-04T17:47:22.146 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:22.142+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:22.151 INFO:teuthology.orchestra.run.smithi092.stdout:4565 trunc from 0x7d337cd to 0x82ebf31 2024-09-04T17:47:22.167 INFO:teuthology.orchestra.run.smithi092.stdout:4566 trunc from 0x82ebf31 to 0xb4d6073 2024-09-04T17:47:22.168 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:22.168 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:47:22.168 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:47:22.168 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:47:22.168 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:22.177 INFO:teuthology.orchestra.run.smithi092.stdout:4567 read 0x9b4af7c thru 0x9b50613 (0x5698 bytes) 2024-09-04T17:47:22.177 INFO:teuthology.orchestra.run.smithi092.stdout:4568 punch from 0x930c5c6 to 0x93156b7, (0x90f1 bytes) 2024-09-04T17:47:22.177 INFO:teuthology.orchestra.run.smithi092.stdout:4569 punch from 0x16c2dd4 to 0x16d1d3c, (0xef68 bytes) 2024-09-04T17:47:22.177 INFO:teuthology.orchestra.run.smithi092.stdout:4571 read 0x18a8a46 thru 0x18af25c (0x6817 bytes) 2024-09-04T17:47:22.182 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-04T17:47:22.189 INFO:teuthology.orchestra.run.smithi092.stdout:4573 trunc from 0xb4d6073 to 0x3ef8c8f 2024-09-04T17:47:22.198 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:22.204 INFO:teuthology.orchestra.run.smithi092.stdout:4576 trunc from 0x3ef8c8f to 0x8479791 2024-09-04T17:47:22.212 INFO:teuthology.orchestra.run.smithi092.stdout:4577 punch from 0x7aadac0 to 0x7ab5573, (0x7ab3 bytes) 2024-09-04T17:47:22.213 INFO:teuthology.orchestra.run.smithi092.stdout:4578 punch from 0x823b913 to 0x8248a2c, (0xd119 bytes) 2024-09-04T17:47:22.224 INFO:teuthology.orchestra.run.smithi092.stdout:4579 write 0xa99a8ab thru 0xa99e43e (0x3b94 bytes) 2024-09-04T17:47:22.227 INFO:teuthology.orchestra.run.smithi092.stderr:no valid command found; 10 closest matches: 2024-09-04T17:47:22.227 INFO:teuthology.orchestra.run.smithi092.stderr:0 2024-09-04T17:47:22.227 INFO:teuthology.orchestra.run.smithi092.stderr:1 2024-09-04T17:47:22.227 INFO:teuthology.orchestra.run.smithi092.stderr:2 2024-09-04T17:47:22.227 INFO:teuthology.orchestra.run.smithi092.stderr:abort 2024-09-04T17:47:22.227 INFO:teuthology.orchestra.run.smithi092.stderr:assert 2024-09-04T17:47:22.227 INFO:teuthology.orchestra.run.smithi092.stderr:bench [] [] [] [] 2024-09-04T17:47:22.227 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs debug_inject_read_zeros 2024-09-04T17:47:22.227 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs files list 2024-09-04T17:47:22.227 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs stats 2024-09-04T17:47:22.227 INFO:teuthology.orchestra.run.smithi092.stderr:bluestore allocator dump block 2024-09-04T17:47:22.228 INFO:teuthology.orchestra.run.smithi092.stderr:Error EINVAL: invalid command 2024-09-04T17:47:22.230 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-04T17:47:22.231 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-04T17:47:22.231 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-04T17:47:22.234 INFO:teuthology.orchestra.run.smithi092.stdout:4580 trunc from 0xa99e43f to 0xe8cbb79 2024-09-04T17:47:22.240 INFO:teuthology.orchestra.run.smithi092.stdout:4581 read 0xb444597 thru 0xb453513 (0xef7d bytes) 2024-09-04T17:47:22.240 INFO:teuthology.orchestra.run.smithi092.stdout:4582 punch from 0x5fa1a9f to 0x5fac3d8, (0xa939 bytes) 2024-09-04T17:47:22.240 INFO:teuthology.orchestra.run.smithi092.stdout:4583 write 0x48c3866 thru 0x48d1897 (0xe032 bytes) 2024-09-04T17:47:22.242 INFO:teuthology.orchestra.run.smithi092.stdout:4584 write 0xd94fc45 thru 0xd95c972 (0xcd2e bytes) 2024-09-04T17:47:22.244 INFO:teuthology.orchestra.run.smithi092.stdout:4586 clone 33 order 23 su 524288 sc 7 2024-09-04T17:47:22.247 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:22.244+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:22.311 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:22.311 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:47:22.311 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:47:22.311 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:47:22.311 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.311 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:12722 3.1b 3:d8152af1:::rbd_data.1134ef3a5a7d.00000000000000bd:head [delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-04T17:47:22.311 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:16.421965+0000", 2024-09-04T17:47:22.312 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 65.887659286000002, 2024-09-04T17:47:22.312 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.890002811, 2024-09-04T17:47:22.312 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:22.312 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:22.312 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:22.312 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:22.312 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:22.312 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:22.312 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 12722 2024-09-04T17:47:22.312 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.312 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:22.312 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.312 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:22.313 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421965+0000", 2024-09-04T17:47:22.313 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.313 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.313 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.313 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:22.313 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421965+0000", 2024-09-04T17:47:22.313 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.313 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.313 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.313 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:22.313 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421966+0000", 2024-09-04T17:47:22.313 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.102e-06 2024-09-04T17:47:22.313 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.313 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.314 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:22.314 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421971+0000", 2024-09-04T17:47:22.314 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9989999999999999e-06 2024-09-04T17:47:22.314 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.314 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.314 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:22.314 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421981+0000", 2024-09-04T17:47:22.314 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.7380000000000006e-06 2024-09-04T17:47:22.314 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.314 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.314 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:22.314 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421989+0000", 2024-09-04T17:47:22.314 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.1890000000000007e-06 2024-09-04T17:47:22.315 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.315 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.315 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:22.315 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.422050+0000", 2024-09-04T17:47:22.315 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.0702000000000003e-05 2024-09-04T17:47:22.315 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.315 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.315 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for peered", 2024-09-04T17:47:22.315 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.422059+0000", 2024-09-04T17:47:22.315 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5440000000000004e-06 2024-09-04T17:47:22.315 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.315 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.315 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:22.315 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.308156+0000", 2024-09-04T17:47:22.316 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.88609676199999998 2024-09-04T17:47:22.316 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.316 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.316 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:22.316 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.308398+0000", 2024-09-04T17:47:22.316 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024215400000000001 2024-09-04T17:47:22.316 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.316 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.316 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 7", 2024-09-04T17:47:22.316 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.308795+0000", 2024-09-04T17:47:22.316 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000396704 2024-09-04T17:47:22.316 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.317 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.317 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:47:22.317 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.310831+0000", 2024-09-04T17:47:22.317 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0020364200000000002 2024-09-04T17:47:22.317 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.317 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.317 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:22.317 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.311818+0000", 2024-09-04T17:47:22.317 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00098649799999999993 2024-09-04T17:47:22.317 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.318 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.318 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:47:22.318 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.311905+0000", 2024-09-04T17:47:22.318 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7070999999999994e-05 2024-09-04T17:47:22.318 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.318 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.318 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:22.318 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.311968+0000", 2024-09-04T17:47:22.318 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.2928000000000005e-05 2024-09-04T17:47:22.318 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.318 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:22.318 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.318 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.318 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.319 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37359 3.f e58/24 3:f29e50a5:::rbd_data.1134c5a43465.0000000000000021:head v 58'135, mlcod=58'133)", 2024-09-04T17:47:22.319 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:47:22.319 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 42.692328682000003, 2024-09-04T17:47:22.319 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053483197000000003, 2024-09-04T17:47:22.319 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:22.319 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:22.319 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:22.319 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:22.319 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.319 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:22.319 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:47:22.319 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.319 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.320 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.320 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:22.320 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:47:22.320 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.320 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.320 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.320 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:22.320 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:47:22.320 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1699999999999997e-07 2024-09-04T17:47:22.320 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.320 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.320 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:22.320 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617338+0000", 2024-09-04T17:47:22.320 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.1168999999999998e-05 2024-09-04T17:47:22.321 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.321 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.321 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:22.321 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617351+0000", 2024-09-04T17:47:22.321 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3094e-05 2024-09-04T17:47:22.321 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.321 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.321 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:22.321 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617359+0000", 2024-09-04T17:47:22.321 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7970000000000006e-06 2024-09-04T17:47:22.321 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.321 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.321 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:22.322 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617417+0000", 2024-09-04T17:47:22.322 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7079e-05 2024-09-04T17:47:22.322 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.322 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.322 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:22.322 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617438+0000", 2024-09-04T17:47:22.322 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1654999999999998e-05 2024-09-04T17:47:22.322 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.322 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.322 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:47:22.323 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670738+0000", 2024-09-04T17:47:22.323 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053300142000000002 2024-09-04T17:47:22.323 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.323 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.323 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:22.323 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670779+0000", 2024-09-04T17:47:22.323 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0343999999999999e-05 2024-09-04T17:47:22.323 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.323 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:22.323 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.323 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.323 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.324 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37358 3.1d e58/24 3:bd3215f3:::rbd_data.1134c5a43465.0000000000000020:head v 58'91, mlcod=58'89)", 2024-09-04T17:47:22.324 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617502+0000", 2024-09-04T17:47:22.324 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 42.692121964000002, 2024-09-04T17:47:22.324 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053505641999999999, 2024-09-04T17:47:22.324 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:22.324 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:22.324 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:22.324 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:22.324 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.324 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:22.324 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617502+0000", 2024-09-04T17:47:22.324 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.324 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.324 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.325 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:22.325 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617502+0000", 2024-09-04T17:47:22.325 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.325 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.325 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.325 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:22.325 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617503+0000", 2024-09-04T17:47:22.325 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.184e-06 2024-09-04T17:47:22.325 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.325 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.326 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:22.326 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617526+0000", 2024-09-04T17:47:22.326 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2099999999999998e-05 2024-09-04T17:47:22.326 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.326 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.326 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:22.326 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617540+0000", 2024-09-04T17:47:22.326 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4741e-05 2024-09-04T17:47:22.326 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.326 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.326 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:22.327 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617550+0000", 2024-09-04T17:47:22.327 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4259999999999992e-06 2024-09-04T17:47:22.327 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.327 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.327 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:22.327 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617613+0000", 2024-09-04T17:47:22.328 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.3625000000000002e-05 2024-09-04T17:47:22.328 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.328 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.328 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:22.328 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617637+0000", 2024-09-04T17:47:22.328 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.3604999999999998e-05 2024-09-04T17:47:22.328 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.328 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.328 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:47:22.328 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670967+0000", 2024-09-04T17:47:22.328 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053329881000000003 2024-09-04T17:47:22.328 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.328 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.329 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:22.329 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671008+0000", 2024-09-04T17:47:22.329 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.108e-05 2024-09-04T17:47:22.329 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.329 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:22.329 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.329 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.329 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.329 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37361 3.1c e58/24 3:3c7f8976:::rbd_data.1134c5a43465.0000000000000023:head v 58'159, mlcod=58'157)", 2024-09-04T17:47:22.329 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617578+0000", 2024-09-04T17:47:22.329 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 42.692046114, 2024-09-04T17:47:22.329 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053990367999999997, 2024-09-04T17:47:22.330 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:22.330 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:22.330 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:22.330 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:22.330 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.330 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:22.330 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617578+0000", 2024-09-04T17:47:22.330 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.330 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.330 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.330 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:22.330 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617578+0000", 2024-09-04T17:47:22.330 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.330 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.331 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.331 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:22.331 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617579+0000", 2024-09-04T17:47:22.331 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7299999999999996e-07 2024-09-04T17:47:22.331 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.331 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.331 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:22.331 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617610+0000", 2024-09-04T17:47:22.331 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0951000000000002e-05 2024-09-04T17:47:22.331 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.331 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.331 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:22.332 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:47:22.332 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0002e-05 2024-09-04T17:47:22.332 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.332 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.332 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:22.332 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617626+0000", 2024-09-04T17:47:22.332 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6660000000000002e-06 2024-09-04T17:47:22.332 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.332 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.332 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:22.332 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670548+0000", 2024-09-04T17:47:22.332 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.052921943999999999 2024-09-04T17:47:22.332 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.333 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.333 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:22.333 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670574+0000", 2024-09-04T17:47:22.333 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5405000000000002e-05 2024-09-04T17:47:22.333 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.333 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.333 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:47:22.333 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671522+0000", 2024-09-04T17:47:22.333 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00094815299999999997 2024-09-04T17:47:22.333 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.333 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.333 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:22.333 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671568+0000", 2024-09-04T17:47:22.334 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6674000000000001e-05 2024-09-04T17:47:22.336 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.336 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:22.336 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.336 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.336 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.336 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37357 3.15 e58/49 3:a897312a:::rbd_data.1134c5a43465.000000000000001f:head v 58'117, mlcod=58'115)", 2024-09-04T17:47:22.336 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:47:22.336 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 42.692003776, 2024-09-04T17:47:22.337 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053734297, 2024-09-04T17:47:22.337 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:22.337 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:22.337 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:22.337 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:22.337 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.337 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:22.337 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:47:22.337 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.337 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.338 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.338 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:22.338 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:47:22.338 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.338 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.338 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.338 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:22.338 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617621+0000", 2024-09-04T17:47:22.338 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.7300000000000004e-07 2024-09-04T17:47:22.338 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.338 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.338 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:22.338 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617642+0000", 2024-09-04T17:47:22.339 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0185000000000001e-05 2024-09-04T17:47:22.339 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.339 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.339 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:22.339 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617654+0000", 2024-09-04T17:47:22.339 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2745000000000001e-05 2024-09-04T17:47:22.339 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.339 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.339 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:22.339 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617662+0000", 2024-09-04T17:47:22.339 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.0499999999999992e-06 2024-09-04T17:47:22.340 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.340 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.340 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:22.340 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617727+0000", 2024-09-04T17:47:22.340 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.4923000000000002e-05 2024-09-04T17:47:22.340 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.340 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.340 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:22.340 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617753+0000", 2024-09-04T17:47:22.341 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5899999999999999e-05 2024-09-04T17:47:22.341 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.341 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.341 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:47:22.341 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671298+0000", 2024-09-04T17:47:22.341 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053545276000000003 2024-09-04T17:47:22.341 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.341 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.341 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:22.341 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671355+0000", 2024-09-04T17:47:22.341 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6245000000000001e-05 2024-09-04T17:47:22.341 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.341 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:22.342 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.342 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.342 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.342 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44687 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:22.342 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253012+0000", 2024-09-04T17:47:22.342 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 32.056611709000002, 2024-09-04T17:47:22.342 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051128777, 2024-09-04T17:47:22.342 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:22.342 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:22.342 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:22.342 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:22.342 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:22.342 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:22.343 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44687 2024-09-04T17:47:22.343 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.343 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:22.343 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.343 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:22.343 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253012+0000", 2024-09-04T17:47:22.343 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.343 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.343 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.343 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:22.343 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253012+0000", 2024-09-04T17:47:22.343 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.343 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.343 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.344 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:22.344 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253014+0000", 2024-09-04T17:47:22.344 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.24e-06 2024-09-04T17:47:22.344 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.344 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.344 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:22.344 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253019+0000", 2024-09-04T17:47:22.344 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4569999999999996e-06 2024-09-04T17:47:22.344 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.345 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.345 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:22.345 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253035+0000", 2024-09-04T17:47:22.345 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6291999999999999e-05 2024-09-04T17:47:22.345 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.345 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.345 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:22.345 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253050+0000", 2024-09-04T17:47:22.345 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4423000000000001e-05 2024-09-04T17:47:22.345 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.345 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.345 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:22.345 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304115+0000", 2024-09-04T17:47:22.346 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051064778999999998 2024-09-04T17:47:22.346 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.346 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.346 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:22.346 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304141+0000", 2024-09-04T17:47:22.346 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6585999999999999e-05 2024-09-04T17:47:22.346 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.346 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:22.346 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.346 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.346 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.346 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44688 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:22.346 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253067+0000", 2024-09-04T17:47:22.347 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 32.056557157, 2024-09-04T17:47:22.347 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051116979, 2024-09-04T17:47:22.347 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:22.347 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:22.347 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:22.347 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:22.347 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:22.347 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:22.347 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44688 2024-09-04T17:47:22.347 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.348 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:22.348 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.348 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:22.348 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253067+0000", 2024-09-04T17:47:22.348 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.348 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.349 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.349 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:22.349 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253067+0000", 2024-09-04T17:47:22.349 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.349 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.349 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.349 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:22.349 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253068+0000", 2024-09-04T17:47:22.349 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0729999999999999e-06 2024-09-04T17:47:22.349 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.349 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.349 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:22.349 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253073+0000", 2024-09-04T17:47:22.350 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2259999999999996e-06 2024-09-04T17:47:22.350 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.350 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.350 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:22.350 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253091+0000", 2024-09-04T17:47:22.350 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7255999999999999e-05 2024-09-04T17:47:22.350 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.350 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.350 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:22.350 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253103+0000", 2024-09-04T17:47:22.350 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2632999999999999e-05 2024-09-04T17:47:22.350 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.350 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.350 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:22.351 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304173+0000", 2024-09-04T17:47:22.351 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051069537999999998 2024-09-04T17:47:22.351 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.351 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.351 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:22.351 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304184+0000", 2024-09-04T17:47:22.351 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1253000000000001e-05 2024-09-04T17:47:22.351 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.351 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:22.351 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.351 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.351 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.351 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44689 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:22.352 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253121+0000", 2024-09-04T17:47:22.352 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 32.056503020000001, 2024-09-04T17:47:22.352 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051192595, 2024-09-04T17:47:22.352 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:22.352 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:22.352 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:22.352 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:22.352 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:22.352 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:22.352 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44689 2024-09-04T17:47:22.352 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.352 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:22.352 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.353 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:22.353 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253121+0000", 2024-09-04T17:47:22.353 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.353 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.353 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.353 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:22.353 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253121+0000", 2024-09-04T17:47:22.353 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.353 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.353 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.353 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:22.353 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253122+0000", 2024-09-04T17:47:22.353 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.15e-06 2024-09-04T17:47:22.354 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.354 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.354 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:22.354 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253128+0000", 2024-09-04T17:47:22.354 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2569999999999998e-06 2024-09-04T17:47:22.354 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.354 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.354 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:22.354 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253145+0000", 2024-09-04T17:47:22.354 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7176e-05 2024-09-04T17:47:22.354 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.354 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.355 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:22.355 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253158+0000", 2024-09-04T17:47:22.355 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3139000000000001e-05 2024-09-04T17:47:22.355 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.355 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.355 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:22.355 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304287+0000", 2024-09-04T17:47:22.355 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051129149999999998 2024-09-04T17:47:22.355 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.355 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.355 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:22.355 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304314+0000", 2024-09-04T17:47:22.355 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6723000000000001e-05 2024-09-04T17:47:22.355 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.356 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:22.356 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.356 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.356 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.356 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44690 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:22.356 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253177+0000", 2024-09-04T17:47:22.356 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 32.056446895000001, 2024-09-04T17:47:22.356 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051169263, 2024-09-04T17:47:22.356 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:22.356 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:22.356 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:22.356 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:22.357 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:22.357 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:22.357 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44690 2024-09-04T17:47:22.357 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.357 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:22.357 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.357 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:22.357 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253177+0000", 2024-09-04T17:47:22.357 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.357 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.357 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.357 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:22.357 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253177+0000", 2024-09-04T17:47:22.358 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.358 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.358 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.358 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:22.358 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253178+0000", 2024-09-04T17:47:22.358 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1000000000000001e-06 2024-09-04T17:47:22.358 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.358 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.358 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:22.358 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253184+0000", 2024-09-04T17:47:22.358 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2089999999999996e-06 2024-09-04T17:47:22.358 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.358 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.359 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:22.359 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253200+0000", 2024-09-04T17:47:22.359 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6087e-05 2024-09-04T17:47:22.359 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.359 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.359 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:22.359 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253231+0000", 2024-09-04T17:47:22.359 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0874999999999998e-05 2024-09-04T17:47:22.359 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.359 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.359 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:22.359 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304337+0000", 2024-09-04T17:47:22.359 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051106430000000001 2024-09-04T17:47:22.360 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.360 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.360 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:22.360 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304347+0000", 2024-09-04T17:47:22.360 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5619999999999993e-06 2024-09-04T17:47:22.360 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.360 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:22.360 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.360 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.360 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.360 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44691 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:22.360 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253249+0000", 2024-09-04T17:47:22.360 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 32.056374861000002, 2024-09-04T17:47:22.361 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051195630999999998, 2024-09-04T17:47:22.361 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:22.361 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:22.361 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:22.361 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:22.361 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:22.361 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:22.361 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44691 2024-09-04T17:47:22.361 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.361 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:22.361 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.361 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:22.361 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253249+0000", 2024-09-04T17:47:22.362 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.362 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.362 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.362 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:22.362 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253249+0000", 2024-09-04T17:47:22.362 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.362 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.362 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.362 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:22.362 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253251+0000", 2024-09-04T17:47:22.362 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2330000000000001e-06 2024-09-04T17:47:22.362 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.362 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.363 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:22.363 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253256+0000", 2024-09-04T17:47:22.363 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6099999999999997e-06 2024-09-04T17:47:22.363 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.363 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.363 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:22.363 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253274+0000", 2024-09-04T17:47:22.363 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7952999999999999e-05 2024-09-04T17:47:22.363 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.363 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.363 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:22.363 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253287+0000", 2024-09-04T17:47:22.363 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2839e-05 2024-09-04T17:47:22.364 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.364 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.364 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:22.364 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304396+0000", 2024-09-04T17:47:22.364 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051108990999999999 2024-09-04T17:47:22.364 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.364 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.364 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:22.364 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304445+0000", 2024-09-04T17:47:22.364 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9005000000000003e-05 2024-09-04T17:47:22.364 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.364 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:22.364 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.365 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.365 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.365 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44692 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:22.365 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253337+0000", 2024-09-04T17:47:22.365 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 32.056287288, 2024-09-04T17:47:22.365 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051138909000000003, 2024-09-04T17:47:22.365 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:22.365 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:22.365 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:22.365 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:22.365 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:22.365 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:22.365 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44692 2024-09-04T17:47:22.366 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.366 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:22.366 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.366 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:22.366 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253337+0000", 2024-09-04T17:47:22.366 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.366 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.366 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.366 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:22.366 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253337+0000", 2024-09-04T17:47:22.366 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.366 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.366 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.366 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:22.367 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253338+0000", 2024-09-04T17:47:22.367 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.463e-06 2024-09-04T17:47:22.367 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.367 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.367 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:22.367 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253357+0000", 2024-09-04T17:47:22.367 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9043e-05 2024-09-04T17:47:22.367 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.367 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.367 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:22.367 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253379+0000", 2024-09-04T17:47:22.367 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1243e-05 2024-09-04T17:47:22.367 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.368 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.368 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:22.368 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253396+0000", 2024-09-04T17:47:22.368 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7059e-05 2024-09-04T17:47:22.368 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.368 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.368 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:22.368 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304466+0000", 2024-09-04T17:47:22.368 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051070624000000002 2024-09-04T17:47:22.368 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.368 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.368 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:22.369 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304476+0000", 2024-09-04T17:47:22.369 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4769999999999992e-06 2024-09-04T17:47:22.369 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.369 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:22.369 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.369 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.369 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.369 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44693 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:22.369 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253415+0000", 2024-09-04T17:47:22.369 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 32.056208884999997, 2024-09-04T17:47:22.369 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051141949999999999, 2024-09-04T17:47:22.369 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:22.369 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:22.369 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:22.370 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:22.370 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:22.370 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:22.370 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44693 2024-09-04T17:47:22.370 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.370 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:22.370 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.370 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:22.370 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253415+0000", 2024-09-04T17:47:22.370 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.370 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.370 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.370 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:22.371 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253415+0000", 2024-09-04T17:47:22.371 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.371 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.371 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.371 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:22.371 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253416+0000", 2024-09-04T17:47:22.371 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1799999999999999e-06 2024-09-04T17:47:22.371 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.371 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.371 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:22.371 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253422+0000", 2024-09-04T17:47:22.371 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.5940000000000001e-06 2024-09-04T17:47:22.371 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.372 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.372 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:22.372 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253438+0000", 2024-09-04T17:47:22.372 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6008999999999999e-05 2024-09-04T17:47:22.372 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.372 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.372 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:22.372 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253452+0000", 2024-09-04T17:47:22.372 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3543000000000001e-05 2024-09-04T17:47:22.372 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.372 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.372 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:22.373 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304533+0000", 2024-09-04T17:47:22.373 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051081609 2024-09-04T17:47:22.373 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.373 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.373 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:22.373 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304557+0000", 2024-09-04T17:47:22.373 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4015e-05 2024-09-04T17:47:22.373 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.373 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:22.373 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.373 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.373 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.373 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44694 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:22.374 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253469+0000", 2024-09-04T17:47:22.374 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 32.056154759999998, 2024-09-04T17:47:22.374 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051121486000000001, 2024-09-04T17:47:22.374 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:22.374 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:22.374 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:22.374 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:22.374 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:22.374 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:22.374 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44694 2024-09-04T17:47:22.374 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.374 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:22.374 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.375 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:22.375 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253469+0000", 2024-09-04T17:47:22.375 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.375 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.375 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.375 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:22.375 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253469+0000", 2024-09-04T17:47:22.375 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.375 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.375 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.375 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:22.375 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253471+0000", 2024-09-04T17:47:22.375 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1400000000000001e-06 2024-09-04T17:47:22.376 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.376 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.376 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:22.376 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253476+0000", 2024-09-04T17:47:22.376 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7370000000000003e-06 2024-09-04T17:47:22.376 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.376 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.376 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:22.376 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253493+0000", 2024-09-04T17:47:22.376 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6549e-05 2024-09-04T17:47:22.376 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.376 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.376 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:22.377 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253506+0000", 2024-09-04T17:47:22.377 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3406e-05 2024-09-04T17:47:22.377 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.377 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.377 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:22.377 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304582+0000", 2024-09-04T17:47:22.377 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051075534999999998 2024-09-04T17:47:22.377 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.377 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.377 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:22.377 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304591+0000", 2024-09-04T17:47:22.377 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1190000000000003e-06 2024-09-04T17:47:22.377 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.378 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:22.378 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.378 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.378 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.378 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44695 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:22.378 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253552+0000", 2024-09-04T17:47:22.378 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 32.056072006000001, 2024-09-04T17:47:22.378 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051068872000000001, 2024-09-04T17:47:22.378 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:22.378 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:22.378 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:22.378 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:22.379 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:22.379 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:22.379 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44695 2024-09-04T17:47:22.379 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.379 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:22.379 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.379 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:22.379 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253552+0000", 2024-09-04T17:47:22.379 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.379 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.379 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.379 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:22.379 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253552+0000", 2024-09-04T17:47:22.380 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.380 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.380 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.380 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:22.380 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253553+0000", 2024-09-04T17:47:22.380 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1960000000000001e-06 2024-09-04T17:47:22.380 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.380 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.380 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:22.380 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253560+0000", 2024-09-04T17:47:22.380 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.2040000000000002e-06 2024-09-04T17:47:22.380 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.380 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.381 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:22.381 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253578+0000", 2024-09-04T17:47:22.381 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8241999999999999e-05 2024-09-04T17:47:22.381 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.381 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.381 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:22.381 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253591+0000", 2024-09-04T17:47:22.381 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3453e-05 2024-09-04T17:47:22.381 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.381 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.381 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:22.381 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304614+0000", 2024-09-04T17:47:22.381 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051022456000000001 2024-09-04T17:47:22.381 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.382 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.382 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:22.382 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304621+0000", 2024-09-04T17:47:22.382 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3209999999999999e-06 2024-09-04T17:47:22.382 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.382 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:22.382 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.382 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.382 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.382 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44696 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:22.382 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253609+0000", 2024-09-04T17:47:22.382 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 32.056015504999998, 2024-09-04T17:47:22.382 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051115351000000003, 2024-09-04T17:47:22.383 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:22.383 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:22.383 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:22.383 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:22.383 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:22.383 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:22.383 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44696 2024-09-04T17:47:22.383 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.383 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:22.383 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.383 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:22.383 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253609+0000", 2024-09-04T17:47:22.384 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.384 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.384 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.384 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:22.384 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253609+0000", 2024-09-04T17:47:22.384 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.384 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.384 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.384 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:22.384 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253610+0000", 2024-09-04T17:47:22.384 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2300000000000001e-06 2024-09-04T17:47:22.384 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.384 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.385 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:22.385 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253615+0000", 2024-09-04T17:47:22.385 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7260000000000002e-06 2024-09-04T17:47:22.385 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.385 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.385 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:22.385 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253632+0000", 2024-09-04T17:47:22.385 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7039999999999999e-05 2024-09-04T17:47:22.385 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.385 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.385 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:22.385 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253666+0000", 2024-09-04T17:47:22.385 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4662000000000001e-05 2024-09-04T17:47:22.386 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.386 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.386 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:22.386 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304677+0000", 2024-09-04T17:47:22.386 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051011017999999998 2024-09-04T17:47:22.386 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.386 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.386 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:22.386 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304724+0000", 2024-09-04T17:47:22.386 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6675000000000003e-05 2024-09-04T17:47:22.386 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.386 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:22.386 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.387 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.387 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.387 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44697 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:22.387 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253684+0000", 2024-09-04T17:47:22.387 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 32.055940010999997, 2024-09-04T17:47:22.387 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051073107, 2024-09-04T17:47:22.387 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:22.387 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:22.387 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:22.387 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:22.387 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:22.387 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:22.387 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44697 2024-09-04T17:47:22.387 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.388 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:22.388 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.388 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:22.388 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253684+0000", 2024-09-04T17:47:22.388 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.388 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.388 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.388 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:22.388 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253684+0000", 2024-09-04T17:47:22.388 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.388 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.388 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.388 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:22.389 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253685+0000", 2024-09-04T17:47:22.389 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0699999999999999e-06 2024-09-04T17:47:22.389 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.389 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.389 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:22.389 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253690+0000", 2024-09-04T17:47:22.389 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1959999999999999e-06 2024-09-04T17:47:22.389 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.389 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.389 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:22.389 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253707+0000", 2024-09-04T17:47:22.389 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6946e-05 2024-09-04T17:47:22.389 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.390 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.390 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:22.390 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253720+0000", 2024-09-04T17:47:22.390 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3076e-05 2024-09-04T17:47:22.390 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.390 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.390 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:22.390 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304745+0000", 2024-09-04T17:47:22.390 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051024760000000002 2024-09-04T17:47:22.390 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.390 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.390 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:22.390 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304757+0000", 2024-09-04T17:47:22.391 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2058999999999999e-05 2024-09-04T17:47:22.391 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.391 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:22.391 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.391 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.391 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.391 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44698 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:22.391 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253737+0000", 2024-09-04T17:47:22.391 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 32.055887660000003, 2024-09-04T17:47:22.391 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051176078, 2024-09-04T17:47:22.391 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:22.391 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:22.391 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:22.391 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:22.392 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:22.392 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:22.392 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44698 2024-09-04T17:47:22.392 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.392 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:22.392 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.392 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:22.392 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253737+0000", 2024-09-04T17:47:22.392 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.392 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.392 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.392 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:22.393 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253737+0000", 2024-09-04T17:47:22.393 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.393 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.393 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.393 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:22.393 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253738+0000", 2024-09-04T17:47:22.393 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0300000000000001e-06 2024-09-04T17:47:22.393 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.393 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.393 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:22.393 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253752+0000", 2024-09-04T17:47:22.393 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4358999999999999e-05 2024-09-04T17:47:22.393 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.394 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.394 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:22.394 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253771+0000", 2024-09-04T17:47:22.394 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8879999999999999e-05 2024-09-04T17:47:22.394 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.394 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.394 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:22.394 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253785+0000", 2024-09-04T17:47:22.394 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3852000000000001e-05 2024-09-04T17:47:22.394 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.394 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.394 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:22.394 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304876+0000", 2024-09-04T17:47:22.395 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051091105999999997 2024-09-04T17:47:22.395 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.395 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.395 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:22.395 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304913+0000", 2024-09-04T17:47:22.395 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.6850999999999997e-05 2024-09-04T17:47:22.395 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.395 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:22.395 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.395 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.395 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.395 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44699 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:22.395 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253804+0000", 2024-09-04T17:47:22.396 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 32.055820136000001, 2024-09-04T17:47:22.396 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051172250000000002, 2024-09-04T17:47:22.396 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:22.396 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:22.396 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:22.396 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:22.396 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:22.396 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:22.396 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44699 2024-09-04T17:47:22.396 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.396 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:22.396 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.396 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:22.397 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253804+0000", 2024-09-04T17:47:22.397 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.397 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.397 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.397 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:22.397 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253804+0000", 2024-09-04T17:47:22.397 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.397 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.397 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.397 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:22.397 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253805+0000", 2024-09-04T17:47:22.397 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.08e-06 2024-09-04T17:47:22.397 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.398 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.398 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:22.398 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253810+0000", 2024-09-04T17:47:22.398 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1830000000000002e-06 2024-09-04T17:47:22.398 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.398 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.398 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:22.398 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253828+0000", 2024-09-04T17:47:22.398 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7662000000000001e-05 2024-09-04T17:47:22.398 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.398 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.398 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:22.398 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253841+0000", 2024-09-04T17:47:22.399 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3203999999999999e-05 2024-09-04T17:47:22.399 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.399 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.399 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:22.399 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304966+0000", 2024-09-04T17:47:22.399 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051125273999999998 2024-09-04T17:47:22.399 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.399 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.399 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:22.399 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304976+0000", 2024-09-04T17:47:22.399 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.8470000000000008e-06 2024-09-04T17:47:22.399 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.399 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:22.399 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.400 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.400 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.400 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44700 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:22.400 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253858+0000", 2024-09-04T17:47:22.400 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 32.055766482000003, 2024-09-04T17:47:22.400 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051218957000000002, 2024-09-04T17:47:22.400 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:22.400 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:22.400 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:22.400 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:22.400 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:22.400 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:22.401 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44700 2024-09-04T17:47:22.401 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.401 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:22.401 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.401 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:22.401 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253858+0000", 2024-09-04T17:47:22.401 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.401 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.401 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.401 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:22.401 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253858+0000", 2024-09-04T17:47:22.401 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.401 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.401 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.402 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:22.402 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253859+0000", 2024-09-04T17:47:22.402 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.217e-06 2024-09-04T17:47:22.402 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.402 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.402 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:22.402 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253864+0000", 2024-09-04T17:47:22.402 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7600000000000002e-06 2024-09-04T17:47:22.402 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.402 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.402 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:22.402 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253880+0000", 2024-09-04T17:47:22.402 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6501999999999999e-05 2024-09-04T17:47:22.403 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.403 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.403 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:22.403 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253899+0000", 2024-09-04T17:47:22.403 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8907e-05 2024-09-04T17:47:22.403 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.403 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.403 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:22.403 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305052+0000", 2024-09-04T17:47:22.403 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051152989000000003 2024-09-04T17:47:22.403 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.403 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.403 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:22.403 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305077+0000", 2024-09-04T17:47:22.404 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4581999999999999e-05 2024-09-04T17:47:22.404 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.404 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:22.404 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.404 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.404 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.404 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44701 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:22.404 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253917+0000", 2024-09-04T17:47:22.404 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 32.055707130000002, 2024-09-04T17:47:22.404 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051231113000000002, 2024-09-04T17:47:22.404 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:22.404 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:22.404 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:22.405 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:22.405 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:22.405 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:22.405 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44701 2024-09-04T17:47:22.405 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.405 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:22.405 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.405 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:22.405 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253917+0000", 2024-09-04T17:47:22.405 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.405 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.405 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.405 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:22.406 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253917+0000", 2024-09-04T17:47:22.406 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.406 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.406 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.406 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:22.406 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253918+0000", 2024-09-04T17:47:22.406 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.136e-06 2024-09-04T17:47:22.406 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.406 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.406 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:22.406 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253924+0000", 2024-09-04T17:47:22.406 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.3070000000000002e-06 2024-09-04T17:47:22.406 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.407 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.407 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:22.407 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253940+0000", 2024-09-04T17:47:22.407 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6612e-05 2024-09-04T17:47:22.407 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.407 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.407 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:22.407 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253954+0000", 2024-09-04T17:47:22.407 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3709999999999999e-05 2024-09-04T17:47:22.407 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.407 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.407 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:22.407 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305136+0000", 2024-09-04T17:47:22.407 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051181935999999997 2024-09-04T17:47:22.408 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.408 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.408 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:22.408 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305148+0000", 2024-09-04T17:47:22.408 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2412000000000001e-05 2024-09-04T17:47:22.408 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.408 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:22.408 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.408 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.408 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:22.408 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:22.410 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-04T17:47:22.410 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-04T17:47:22.411 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-04T17:47:22.411 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:22.408+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:22.411 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-04T17:47:22.411 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 'trim stale osdmaps' 2024-09-04T17:47:22.511 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:22.506+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:22.579 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:22.579 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:47:22.580 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:47:22.580 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:22.597 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-04T17:47:22.619 INFO:teuthology.orchestra.run.smithi092.stderr:no valid command found; 10 closest matches: 2024-09-04T17:47:22.619 INFO:teuthology.orchestra.run.smithi092.stderr:0 2024-09-04T17:47:22.619 INFO:teuthology.orchestra.run.smithi092.stderr:1 2024-09-04T17:47:22.619 INFO:teuthology.orchestra.run.smithi092.stderr:2 2024-09-04T17:47:22.619 INFO:teuthology.orchestra.run.smithi092.stderr:abort 2024-09-04T17:47:22.619 INFO:teuthology.orchestra.run.smithi092.stderr:assert 2024-09-04T17:47:22.619 INFO:teuthology.orchestra.run.smithi092.stderr:bench [] [] [] [] 2024-09-04T17:47:22.619 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs debug_inject_read_zeros 2024-09-04T17:47:22.619 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs files list 2024-09-04T17:47:22.619 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs stats 2024-09-04T17:47:22.620 INFO:teuthology.orchestra.run.smithi092.stderr:bluestore allocator dump block 2024-09-04T17:47:22.620 INFO:teuthology.orchestra.run.smithi092.stderr:Error EINVAL: invalid command 2024-09-04T17:47:22.622 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-04T17:47:22.622 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-04T17:47:22.623 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 'trim stale osdmaps' 2024-09-04T17:47:22.640 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:22.640+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:22.712 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:22.712+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:22.719 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:22.719 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:47:22.720 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:47:22.720 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:47:22.720 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:22.734 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-04T17:47:22.765 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is set 2024-09-04T17:47:22.816 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-04T17:47:22.820 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:47:22.820+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:47:22.826 INFO:teuthology.orchestra.run.smithi092.stderr:no valid command found; 10 closest matches: 2024-09-04T17:47:22.826 INFO:teuthology.orchestra.run.smithi092.stderr:0 2024-09-04T17:47:22.826 INFO:teuthology.orchestra.run.smithi092.stderr:1 2024-09-04T17:47:22.826 INFO:teuthology.orchestra.run.smithi092.stderr:2 2024-09-04T17:47:22.826 INFO:teuthology.orchestra.run.smithi092.stderr:abort 2024-09-04T17:47:22.826 INFO:teuthology.orchestra.run.smithi092.stderr:assert 2024-09-04T17:47:22.826 INFO:teuthology.orchestra.run.smithi092.stderr:bench [] [] [] [] 2024-09-04T17:47:22.826 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs debug_inject_read_zeros 2024-09-04T17:47:22.826 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs files list 2024-09-04T17:47:22.827 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs stats 2024-09-04T17:47:22.827 INFO:teuthology.orchestra.run.smithi092.stderr:bluestore allocator dump block 2024-09-04T17:47:22.827 INFO:teuthology.orchestra.run.smithi092.stderr:Error EINVAL: invalid command 2024-09-04T17:47:22.829 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-04T17:47:22.830 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-04T17:47:22.830 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 'trim stale osdmaps' 2024-09-04T17:47:22.857 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone32 from 0xe8cbb79 (overlap 0xa329cb) to 0x92d8d0 2024-09-04T17:47:22.906 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:22.906 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:47:22.906 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:47:22.906 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:47:22.906 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.906 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8831 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2453504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:22.906 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:47:22.906 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 69.847289504000003, 2024-09-04T17:47:22.906 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14731111799999999, 2024-09-04T17:47:22.907 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:22.907 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:22.907 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:22.907 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:22.907 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:22.907 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:22.907 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8831 2024-09-04T17:47:22.907 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.907 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:22.907 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.907 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:22.907 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:47:22.907 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.907 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.908 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.908 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:22.908 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:47:22.908 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.908 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.908 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.908 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:22.908 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057811+0000", 2024-09-04T17:47:22.908 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.083e-06 2024-09-04T17:47:22.908 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.908 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.908 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:22.908 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057814+0000", 2024-09-04T17:47:22.908 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.407e-06 2024-09-04T17:47:22.909 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.909 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.909 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:22.909 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057825+0000", 2024-09-04T17:47:22.909 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.17e-05 2024-09-04T17:47:22.909 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.909 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.909 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:22.909 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057836+0000", 2024-09-04T17:47:22.909 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0283e-05 2024-09-04T17:47:22.909 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.909 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.909 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:22.910 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.204762+0000", 2024-09-04T17:47:22.910 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.146926205 2024-09-04T17:47:22.910 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.910 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.910 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:22.910 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.204895+0000", 2024-09-04T17:47:22.910 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013328999999999999 2024-09-04T17:47:22.910 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.910 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.910 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:22.910 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205121+0000", 2024-09-04T17:47:22.910 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00022614999999999999 2024-09-04T17:47:22.910 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.910 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:22.910 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.911 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.911 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.911 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8832 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2490368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:22.911 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:47:22.911 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 69.847251471000007, 2024-09-04T17:47:22.911 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14770012299999999, 2024-09-04T17:47:22.911 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:22.911 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:22.911 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:22.911 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:22.911 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:22.911 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:22.911 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8832 2024-09-04T17:47:22.911 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.912 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:22.912 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.912 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:22.912 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:47:22.912 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.912 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.912 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.912 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:22.912 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:47:22.912 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.912 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.912 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.912 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:22.913 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057849+0000", 2024-09-04T17:47:22.913 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6700000000000003e-07 2024-09-04T17:47:22.913 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.913 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.913 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:22.913 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057852+0000", 2024-09-04T17:47:22.913 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6359999999999998e-06 2024-09-04T17:47:22.913 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.913 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.913 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:22.913 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057860+0000", 2024-09-04T17:47:22.913 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8140000000000007e-06 2024-09-04T17:47:22.913 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.913 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.914 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:22.914 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057868+0000", 2024-09-04T17:47:22.914 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1999999999999997e-06 2024-09-04T17:47:22.914 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.914 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.914 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:22.914 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205208+0000", 2024-09-04T17:47:22.914 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14734067300000001 2024-09-04T17:47:22.914 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.914 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.914 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:22.914 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205347+0000", 2024-09-04T17:47:22.914 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013914900000000001 2024-09-04T17:47:22.914 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.915 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.915 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:22.915 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205548+0000", 2024-09-04T17:47:22.915 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000200984 2024-09-04T17:47:22.915 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.915 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:22.915 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.915 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.915 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.915 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8833 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2527232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:22.915 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:47:22.915 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 69.847219601000006, 2024-09-04T17:47:22.915 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148062572, 2024-09-04T17:47:22.915 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:22.916 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:22.916 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:22.916 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:22.916 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:22.916 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:22.916 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8833 2024-09-04T17:47:22.916 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.916 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:22.916 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.916 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:22.916 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:47:22.916 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.916 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.917 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.917 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:22.917 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:47:22.917 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.917 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.917 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.917 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:22.917 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057881+0000", 2024-09-04T17:47:22.917 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4300000000000002e-07 2024-09-04T17:47:22.917 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.917 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.917 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:22.917 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057893+0000", 2024-09-04T17:47:22.917 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1973999999999999e-05 2024-09-04T17:47:22.918 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.918 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.918 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:22.918 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057906+0000", 2024-09-04T17:47:22.918 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2716e-05 2024-09-04T17:47:22.918 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.918 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.918 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:22.918 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057913+0000", 2024-09-04T17:47:22.918 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3930000000000002e-06 2024-09-04T17:47:22.918 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.918 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.918 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:22.919 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205621+0000", 2024-09-04T17:47:22.919 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14770788800000001 2024-09-04T17:47:22.919 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.919 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.919 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:22.919 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205740+0000", 2024-09-04T17:47:22.919 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000118936 2024-09-04T17:47:22.919 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.919 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.919 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:22.919 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205943+0000", 2024-09-04T17:47:22.919 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020292199999999999 2024-09-04T17:47:22.919 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.919 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:22.920 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.920 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.920 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.920 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8834 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2564096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:22.920 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:47:22.920 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 69.847176321999996, 2024-09-04T17:47:22.920 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14846753200000001, 2024-09-04T17:47:22.920 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:22.920 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:22.920 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:22.920 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:22.920 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:22.920 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:22.921 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8834 2024-09-04T17:47:22.921 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.921 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:22.921 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.921 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:22.921 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:47:22.921 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.921 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.921 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.921 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:22.921 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:47:22.921 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.921 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.922 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.922 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:22.922 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057925+0000", 2024-09-04T17:47:22.922 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0270000000000001e-06 2024-09-04T17:47:22.922 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.922 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.922 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:22.922 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057936+0000", 2024-09-04T17:47:22.922 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.112e-05 2024-09-04T17:47:22.922 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.922 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.922 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:22.922 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057946+0000", 2024-09-04T17:47:22.922 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.009e-05 2024-09-04T17:47:22.923 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.923 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.923 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:22.923 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057953+0000", 2024-09-04T17:47:22.923 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.6229999999999997e-06 2024-09-04T17:47:22.923 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.923 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.923 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:22.923 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206027+0000", 2024-09-04T17:47:22.923 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148073591 2024-09-04T17:47:22.923 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.923 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.923 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:22.923 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206161+0000", 2024-09-04T17:47:22.924 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013411599999999999 2024-09-04T17:47:22.924 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.924 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.924 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:22.924 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206391+0000", 2024-09-04T17:47:22.924 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000229965 2024-09-04T17:47:22.924 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.924 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:22.924 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.924 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.924 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.924 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8835 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2600960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:22.924 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:47:22.924 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 69.847136672000005, 2024-09-04T17:47:22.925 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14882941599999999, 2024-09-04T17:47:22.925 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:22.925 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:22.925 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:22.925 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:22.925 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:22.925 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:22.925 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8835 2024-09-04T17:47:22.925 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.925 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:22.925 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.925 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:22.925 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:47:22.926 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.926 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.926 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.926 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:22.926 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:47:22.926 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.926 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.926 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.926 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:22.926 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058544+0000", 2024-09-04T17:47:22.926 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00058061899999999999 2024-09-04T17:47:22.926 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.926 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.927 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:22.927 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058550+0000", 2024-09-04T17:47:22.927 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.1129999999999997e-06 2024-09-04T17:47:22.927 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.927 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.927 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:22.927 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058566+0000", 2024-09-04T17:47:22.927 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5820000000000001e-05 2024-09-04T17:47:22.927 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.927 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.927 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:22.927 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058578+0000", 2024-09-04T17:47:22.927 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2537e-05 2024-09-04T17:47:22.927 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.928 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.928 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:22.928 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206470+0000", 2024-09-04T17:47:22.928 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14789153799999999 2024-09-04T17:47:22.928 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.928 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.928 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:22.928 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206589+0000", 2024-09-04T17:47:22.928 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000118837 2024-09-04T17:47:22.928 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.928 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.928 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:22.928 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206793+0000", 2024-09-04T17:47:22.929 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000203952 2024-09-04T17:47:22.929 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.929 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:22.929 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.929 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.929 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.929 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8836 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2654208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:22.929 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:47:22.929 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 69.846503983999995, 2024-09-04T17:47:22.929 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14862433999999999, 2024-09-04T17:47:22.929 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:22.929 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:22.929 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:22.930 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:22.930 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:22.930 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:22.930 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8836 2024-09-04T17:47:22.930 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.930 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:22.930 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.930 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:22.930 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:47:22.930 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.930 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.930 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.930 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:22.931 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:47:22.931 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.931 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.931 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.931 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:22.931 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062663+0000", 2024-09-04T17:47:22.931 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0040670619999999998 2024-09-04T17:47:22.931 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.931 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.931 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:22.931 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062668+0000", 2024-09-04T17:47:22.931 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4199999999999998e-06 2024-09-04T17:47:22.931 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.932 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.932 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:22.932 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062689+0000", 2024-09-04T17:47:22.932 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0356999999999998e-05 2024-09-04T17:47:22.932 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.932 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.932 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:22.932 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062726+0000", 2024-09-04T17:47:22.932 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7302999999999998e-05 2024-09-04T17:47:22.932 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.932 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.932 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:22.932 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206875+0000", 2024-09-04T17:47:22.932 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14414883100000001 2024-09-04T17:47:22.933 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.933 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.933 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:22.933 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.207016+0000", 2024-09-04T17:47:22.933 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000140822 2024-09-04T17:47:22.933 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.933 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.933 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:22.933 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.207220+0000", 2024-09-04T17:47:22.933 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020454499999999999 2024-09-04T17:47:22.933 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.933 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:22.933 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.934 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.934 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.934 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8844 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2928640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:22.934 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:47:22.934 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 69.842139900999996, 2024-09-04T17:47:22.934 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14752153100000001, 2024-09-04T17:47:22.934 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:22.934 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:22.934 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:22.934 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:22.934 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:22.934 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:22.934 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8844 2024-09-04T17:47:22.935 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.935 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:22.935 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.935 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:22.935 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:47:22.935 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.935 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.935 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.935 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:22.935 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:47:22.935 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.935 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.935 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.936 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:22.936 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062961+0000", 2024-09-04T17:47:22.936 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6700000000000003e-07 2024-09-04T17:47:22.936 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.936 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.936 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:22.936 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062973+0000", 2024-09-04T17:47:22.936 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2109999999999999e-05 2024-09-04T17:47:22.936 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.936 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.936 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:22.936 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062982+0000", 2024-09-04T17:47:22.936 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.6900000000000004e-06 2024-09-04T17:47:22.937 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.937 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.937 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:22.937 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062990+0000", 2024-09-04T17:47:22.937 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1899999999999998e-06 2024-09-04T17:47:22.937 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.937 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.937 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:22.937 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210090+0000", 2024-09-04T17:47:22.937 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14710026300000001 2024-09-04T17:47:22.937 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.937 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.937 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:22.937 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210225+0000", 2024-09-04T17:47:22.938 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013532400000000001 2024-09-04T17:47:22.938 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.938 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.938 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:22.938 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210481+0000", 2024-09-04T17:47:22.938 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00025628700000000001 2024-09-04T17:47:22.938 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.938 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:22.938 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.938 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.938 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.938 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8845 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2965504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:22.938 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:47:22.938 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 69.842100177999995, 2024-09-04T17:47:22.939 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147916819, 2024-09-04T17:47:22.939 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:22.939 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:22.939 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:22.939 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:22.939 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:22.939 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:22.939 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8845 2024-09-04T17:47:22.939 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.939 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:22.939 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.939 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:22.939 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:47:22.939 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.940 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.940 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.940 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:22.940 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:47:22.940 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.940 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.940 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.940 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:22.940 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:47:22.940 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.0399999999999995e-07 2024-09-04T17:47:22.940 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.940 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.940 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:22.940 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063003+0000", 2024-09-04T17:47:22.941 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6529999999999998e-06 2024-09-04T17:47:22.941 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.941 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.941 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:22.941 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063012+0000", 2024-09-04T17:47:22.941 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9500000000000007e-06 2024-09-04T17:47:22.941 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.941 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.941 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:22.941 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063027+0000", 2024-09-04T17:47:22.941 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5443000000000001e-05 2024-09-04T17:47:22.941 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.941 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.942 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:22.942 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210571+0000", 2024-09-04T17:47:22.942 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14754353100000001 2024-09-04T17:47:22.942 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.942 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.942 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:22.942 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210714+0000", 2024-09-04T17:47:22.942 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014295599999999999 2024-09-04T17:47:22.942 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.942 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.942 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:22.942 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210916+0000", 2024-09-04T17:47:22.942 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020258200000000001 2024-09-04T17:47:22.943 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.943 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:22.943 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.943 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.943 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.943 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8846 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3002368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:22.943 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:47:22.943 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 69.842062640999998, 2024-09-04T17:47:22.943 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14828011599999999, 2024-09-04T17:47:22.943 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:22.943 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:22.943 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:22.943 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:22.943 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:22.944 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:22.944 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8846 2024-09-04T17:47:22.944 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.944 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:22.944 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.944 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:22.944 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:47:22.944 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.944 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.944 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.944 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:22.944 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:47:22.944 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.945 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.945 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.945 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:22.945 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063038+0000", 2024-09-04T17:47:22.945 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.23e-07 2024-09-04T17:47:22.945 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.945 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.945 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:22.945 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063051+0000", 2024-09-04T17:47:22.945 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2757e-05 2024-09-04T17:47:22.945 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.945 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.945 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:22.945 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063070+0000", 2024-09-04T17:47:22.946 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9663000000000001e-05 2024-09-04T17:47:22.946 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.946 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.946 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:22.946 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063079+0000", 2024-09-04T17:47:22.946 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.0929999999999992e-06 2024-09-04T17:47:22.946 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.946 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.946 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:22.946 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210989+0000", 2024-09-04T17:47:22.946 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14790984500000001 2024-09-04T17:47:22.946 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.946 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.946 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:22.947 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211109+0000", 2024-09-04T17:47:22.947 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000120056 2024-09-04T17:47:22.947 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.947 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.947 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:22.947 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211317+0000", 2024-09-04T17:47:22.947 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020797900000000001 2024-09-04T17:47:22.947 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.947 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:22.947 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.947 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.947 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.947 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8847 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3039232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:22.948 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:47:22.948 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 69.841988208999993, 2024-09-04T17:47:22.948 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14859408900000001, 2024-09-04T17:47:22.948 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:22.948 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:22.948 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:22.948 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:22.948 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:22.948 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:22.948 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8847 2024-09-04T17:47:22.948 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.948 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:22.948 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.948 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:22.949 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:47:22.949 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.949 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.949 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.949 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:22.949 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:47:22.949 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.949 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.949 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.949 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:22.949 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063113+0000", 2024-09-04T17:47:22.949 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.204e-06 2024-09-04T17:47:22.949 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.949 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.950 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:22.950 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063116+0000", 2024-09-04T17:47:22.950 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.3330000000000001e-06 2024-09-04T17:47:22.950 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.950 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.950 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:22.950 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063126+0000", 2024-09-04T17:47:22.950 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0166999999999999e-05 2024-09-04T17:47:22.950 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.950 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.950 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:22.950 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063134+0000", 2024-09-04T17:47:22.950 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4329999999999998e-06 2024-09-04T17:47:22.951 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.951 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.951 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:22.951 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211392+0000", 2024-09-04T17:47:22.951 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148258109 2024-09-04T17:47:22.951 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.951 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.951 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:22.951 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211506+0000", 2024-09-04T17:47:22.951 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000113813 2024-09-04T17:47:22.951 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.951 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.951 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:22.951 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211706+0000", 2024-09-04T17:47:22.952 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020002999999999999 2024-09-04T17:47:22.952 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.952 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:22.952 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.952 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.952 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.952 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8848 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:22.952 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:47:22.952 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 69.841955872, 2024-09-04T17:47:22.952 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14907967599999999, 2024-09-04T17:47:22.952 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:22.952 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:22.952 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:22.953 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:22.953 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:22.953 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:22.953 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8848 2024-09-04T17:47:22.953 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.953 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:22.953 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.953 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:22.953 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:47:22.953 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.953 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.953 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.953 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:22.953 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:47:22.954 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.954 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.954 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.954 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:22.954 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070589+0000", 2024-09-04T17:47:22.954 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.007445386 2024-09-04T17:47:22.954 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.954 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.954 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:22.954 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070600+0000", 2024-09-04T17:47:22.954 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0346999999999999e-05 2024-09-04T17:47:22.954 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.954 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.955 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:22.955 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070622+0000", 2024-09-04T17:47:22.955 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2289999999999998e-05 2024-09-04T17:47:22.955 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.955 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.955 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:22.955 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070636+0000", 2024-09-04T17:47:22.955 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4093e-05 2024-09-04T17:47:22.955 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.955 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.955 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:22.955 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211815+0000", 2024-09-04T17:47:22.955 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14117874699999999 2024-09-04T17:47:22.956 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.956 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.956 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:22.956 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211983+0000", 2024-09-04T17:47:22.956 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00016852800000000001 2024-09-04T17:47:22.956 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.956 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.956 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:22.956 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.212224+0000", 2024-09-04T17:47:22.956 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024028499999999999 2024-09-04T17:47:22.956 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.956 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:22.956 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.957 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.957 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.957 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8862 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2019328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:22.957 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:47:22.957 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 69.833906404000004, 2024-09-04T17:47:22.957 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14755236399999999, 2024-09-04T17:47:22.957 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:22.957 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:22.957 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:22.957 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:22.957 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:22.957 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:22.957 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8862 2024-09-04T17:47:22.957 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.958 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:22.958 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.958 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:22.958 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:47:22.958 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.958 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.958 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.958 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:22.958 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:47:22.958 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.958 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.958 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.958 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:22.958 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071194+0000", 2024-09-04T17:47:22.959 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.5000000000000002e-07 2024-09-04T17:47:22.959 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.959 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.959 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:22.959 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071197+0000", 2024-09-04T17:47:22.959 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5900000000000002e-06 2024-09-04T17:47:22.959 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.959 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.959 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:22.959 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071206+0000", 2024-09-04T17:47:22.959 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9700000000000005e-06 2024-09-04T17:47:22.959 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.959 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.959 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:22.960 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071213+0000", 2024-09-04T17:47:22.960 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.9070000000000001e-06 2024-09-04T17:47:22.960 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.960 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.960 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:22.960 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218396+0000", 2024-09-04T17:47:22.960 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147183273 2024-09-04T17:47:22.960 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.960 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.960 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:22.960 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218538+0000", 2024-09-04T17:47:22.960 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014194199999999999 2024-09-04T17:47:22.961 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.961 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.961 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:22.961 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218746+0000", 2024-09-04T17:47:22.961 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000208032 2024-09-04T17:47:22.961 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.961 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:22.961 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.961 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.961 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.961 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8863 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2056192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:22.961 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:47:22.961 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 69.833878093999999, 2024-09-04T17:47:22.961 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14792281099999999, 2024-09-04T17:47:22.962 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:22.962 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:22.962 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:22.962 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:22.962 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:22.962 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:22.962 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8863 2024-09-04T17:47:22.962 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.962 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:22.962 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.962 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:22.962 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:47:22.962 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.962 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.963 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.963 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:22.963 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:47:22.963 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.963 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.963 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.963 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:22.963 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:47:22.963 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1299999999999999e-07 2024-09-04T17:47:22.963 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.963 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.963 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:22.963 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071225+0000", 2024-09-04T17:47:22.963 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6199999999999999e-06 2024-09-04T17:47:22.964 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.964 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.964 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:22.964 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071234+0000", 2024-09-04T17:47:22.964 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8769999999999999e-06 2024-09-04T17:47:22.964 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.964 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.964 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:22.964 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071251+0000", 2024-09-04T17:47:22.964 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.738e-05 2024-09-04T17:47:22.964 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.964 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.964 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:22.965 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218798+0000", 2024-09-04T17:47:22.965 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14754645 2024-09-04T17:47:22.965 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.965 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.965 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:22.965 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218926+0000", 2024-09-04T17:47:22.965 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000128446 2024-09-04T17:47:22.965 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.965 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.965 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:22.965 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219145+0000", 2024-09-04T17:47:22.965 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021832500000000001 2024-09-04T17:47:22.966 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.966 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:22.966 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.966 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.966 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.966 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8864 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2093056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:22.966 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:47:22.966 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 69.833836517999998, 2024-09-04T17:47:22.966 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148277309, 2024-09-04T17:47:22.966 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:22.966 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:22.966 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:22.967 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:22.967 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:22.967 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:22.967 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8864 2024-09-04T17:47:22.967 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.967 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:22.967 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.967 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:22.967 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:47:22.967 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.967 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.967 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.967 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:22.968 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:47:22.968 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.968 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.968 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.968 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:22.968 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071264+0000", 2024-09-04T17:47:22.968 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3e-07 2024-09-04T17:47:22.968 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.968 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.968 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:22.968 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071267+0000", 2024-09-04T17:47:22.968 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0139999999999999e-06 2024-09-04T17:47:22.968 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.968 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.969 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:22.969 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071276+0000", 2024-09-04T17:47:22.969 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1689999999999998e-06 2024-09-04T17:47:22.969 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.969 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.969 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:22.969 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071283+0000", 2024-09-04T17:47:22.969 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2269999999999996e-06 2024-09-04T17:47:22.969 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.969 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.969 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:22.969 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219195+0000", 2024-09-04T17:47:22.969 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147911868 2024-09-04T17:47:22.970 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.970 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.970 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:22.970 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219324+0000", 2024-09-04T17:47:22.970 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00012835900000000001 2024-09-04T17:47:22.970 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.970 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.970 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:22.970 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219541+0000", 2024-09-04T17:47:22.970 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000216942 2024-09-04T17:47:22.970 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.970 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:22.970 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.970 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.971 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.971 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8865 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2109440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:22.971 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:47:22.971 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 69.833807058000005, 2024-09-04T17:47:22.971 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14865747300000001, 2024-09-04T17:47:22.971 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:22.971 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:22.971 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:22.971 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:22.971 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:22.971 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:22.971 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8865 2024-09-04T17:47:22.971 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.972 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:22.972 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.972 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:22.972 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:47:22.972 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.972 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.972 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.972 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:22.972 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:47:22.972 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.972 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.972 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.972 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:22.973 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:47:22.973 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7999999999999995e-07 2024-09-04T17:47:22.973 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.973 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.973 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:22.973 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071296+0000", 2024-09-04T17:47:22.973 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6900000000000001e-06 2024-09-04T17:47:22.973 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.973 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.973 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:22.973 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071305+0000", 2024-09-04T17:47:22.973 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.5499999999999995e-06 2024-09-04T17:47:22.973 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.974 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.974 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:22.974 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071312+0000", 2024-09-04T17:47:22.974 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1559999999999998e-06 2024-09-04T17:47:22.974 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.974 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.974 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:22.974 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219591+0000", 2024-09-04T17:47:22.974 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14827944900000001 2024-09-04T17:47:22.974 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.974 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.975 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:22.975 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219731+0000", 2024-09-04T17:47:22.975 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013931600000000001 2024-09-04T17:47:22.975 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.975 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.975 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:22.975 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219950+0000", 2024-09-04T17:47:22.975 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021963199999999999 2024-09-04T17:47:22.975 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.975 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:22.975 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.975 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.975 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.976 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8866 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2146304~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:22.976 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:47:22.976 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 69.833778824999996, 2024-09-04T17:47:22.976 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14906117899999999, 2024-09-04T17:47:22.976 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:22.976 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:22.976 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:22.976 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:22.976 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:22.976 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:22.976 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8866 2024-09-04T17:47:22.976 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.976 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:22.977 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.977 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:22.977 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:47:22.977 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.977 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.977 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.977 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:22.977 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:47:22.977 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.977 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.977 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.977 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:22.977 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071322+0000", 2024-09-04T17:47:22.978 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7999999999999995e-07 2024-09-04T17:47:22.978 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.978 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.978 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:22.978 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071338+0000", 2024-09-04T17:47:22.978 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5872999999999999e-05 2024-09-04T17:47:22.978 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.978 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.978 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:22.978 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071347+0000", 2024-09-04T17:47:22.978 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4669999999999993e-06 2024-09-04T17:47:22.978 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.978 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.979 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:22.979 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071354+0000", 2024-09-04T17:47:22.979 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2099999999999996e-06 2024-09-04T17:47:22.979 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.979 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.979 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:22.979 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220008+0000", 2024-09-04T17:47:22.979 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148653642 2024-09-04T17:47:22.979 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.979 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.979 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:22.979 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220155+0000", 2024-09-04T17:47:22.980 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014726600000000001 2024-09-04T17:47:22.980 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.980 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.980 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:22.980 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220382+0000", 2024-09-04T17:47:22.980 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00022704100000000001 2024-09-04T17:47:22.980 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.980 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:22.980 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.980 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.980 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.980 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8867 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2183168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:22.981 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:47:22.981 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 69.833735472000001, 2024-09-04T17:47:22.981 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.149445831, 2024-09-04T17:47:22.981 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:22.981 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:22.981 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:22.981 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:22.981 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:22.981 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:22.981 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8867 2024-09-04T17:47:22.981 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.981 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:22.981 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.982 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:22.982 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:47:22.982 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.982 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.982 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.982 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:22.982 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:47:22.982 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.982 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.982 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.982 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:22.982 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078653+0000", 2024-09-04T17:47:22.982 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0072890430000000003 2024-09-04T17:47:22.983 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.983 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.983 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:22.983 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078664+0000", 2024-09-04T17:47:22.983 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0307e-05 2024-09-04T17:47:22.983 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.983 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.983 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:22.983 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078690+0000", 2024-09-04T17:47:22.983 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6780000000000001e-05 2024-09-04T17:47:22.983 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.983 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.984 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:22.984 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078706+0000", 2024-09-04T17:47:22.984 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5676999999999999e-05 2024-09-04T17:47:22.984 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.984 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.984 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:22.984 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220467+0000", 2024-09-04T17:47:22.984 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14176103500000001 2024-09-04T17:47:22.984 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.984 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.984 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:22.984 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220595+0000", 2024-09-04T17:47:22.984 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000127427 2024-09-04T17:47:22.985 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.985 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.985 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:22.985 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220810+0000", 2024-09-04T17:47:22.985 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021556200000000001 2024-09-04T17:47:22.985 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.985 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:22.985 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.985 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.985 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.985 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8880 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2658304~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:22.985 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.079158+0000", 2024-09-04T17:47:22.985 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 69.825942193000003, 2024-09-04T17:47:22.986 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14718968499999999, 2024-09-04T17:47:22.986 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:22.986 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:22.986 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:22.986 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:22.986 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:22.986 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:22.986 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8880 2024-09-04T17:47:22.986 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.986 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:22.986 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.986 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:22.986 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079158+0000", 2024-09-04T17:47:22.987 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.987 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.987 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.987 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:22.987 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079158+0000", 2024-09-04T17:47:22.987 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.987 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.987 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.987 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:22.987 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079158+0000", 2024-09-04T17:47:22.987 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3e-07 2024-09-04T17:47:22.987 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.987 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.987 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:22.988 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079161+0000", 2024-09-04T17:47:22.988 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.57e-06 2024-09-04T17:47:22.988 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.988 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.988 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:22.988 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079170+0000", 2024-09-04T17:47:22.988 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.0170000000000002e-06 2024-09-04T17:47:22.988 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.988 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.988 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:22.988 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079177+0000", 2024-09-04T17:47:22.988 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.0530000000000001e-06 2024-09-04T17:47:22.988 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.988 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.989 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:22.989 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.225946+0000", 2024-09-04T17:47:22.989 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14676936500000001 2024-09-04T17:47:22.989 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.989 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.989 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:22.989 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226098+0000", 2024-09-04T17:47:22.989 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000151805 2024-09-04T17:47:22.989 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.989 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.989 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:22.989 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226347+0000", 2024-09-04T17:47:22.989 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024914499999999999 2024-09-04T17:47:22.990 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.990 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:22.990 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.990 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.990 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.990 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8881 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2695168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:22.990 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:47:22.990 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 69.825913403000001, 2024-09-04T17:47:22.990 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147543547, 2024-09-04T17:47:22.990 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:22.990 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:22.990 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:22.990 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:22.990 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:22.991 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:22.991 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8881 2024-09-04T17:47:22.991 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.991 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:22.991 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.991 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:22.991 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:47:22.991 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.991 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.991 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.991 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:22.991 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:47:22.991 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.992 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.992 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.992 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:22.992 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079187+0000", 2024-09-04T17:47:22.992 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7700000000000004e-07 2024-09-04T17:47:22.992 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.992 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.992 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:22.992 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079198+0000", 2024-09-04T17:47:22.992 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0849999999999999e-05 2024-09-04T17:47:22.992 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.992 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.992 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:22.992 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079207+0000", 2024-09-04T17:47:22.993 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5599999999999999e-06 2024-09-04T17:47:22.993 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.993 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.993 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:22.993 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079224+0000", 2024-09-04T17:47:22.993 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6387000000000001e-05 2024-09-04T17:47:22.993 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.993 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.993 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:22.993 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226414+0000", 2024-09-04T17:47:22.993 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14718975000000001 2024-09-04T17:47:22.993 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.993 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.994 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:22.994 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226534+0000", 2024-09-04T17:47:22.994 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000120288 2024-09-04T17:47:22.994 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.994 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.994 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:22.994 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226730+0000", 2024-09-04T17:47:22.994 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000196035 2024-09-04T17:47:22.994 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.994 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:22.994 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.994 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.994 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.995 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8882 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2732032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:22.995 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:47:22.995 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 69.825865188999998, 2024-09-04T17:47:22.995 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14795022399999999, 2024-09-04T17:47:22.995 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:22.995 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:22.995 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:22.995 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:22.995 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:22.995 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:22.995 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8882 2024-09-04T17:47:22.995 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.995 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:22.995 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.996 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:22.996 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:47:22.996 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.996 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.996 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.996 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:22.996 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:47:22.996 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:22.996 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.996 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.996 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:22.996 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086588+0000", 2024-09-04T17:47:22.996 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0073530959999999999 2024-09-04T17:47:22.996 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.997 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.997 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:22.997 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086598+0000", 2024-09-04T17:47:22.997 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0122999999999999e-05 2024-09-04T17:47:22.997 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.997 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.997 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:22.997 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086626+0000", 2024-09-04T17:47:22.997 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.7914000000000001e-05 2024-09-04T17:47:22.997 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.997 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.997 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:22.997 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086641+0000", 2024-09-04T17:47:22.998 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5270000000000001e-05 2024-09-04T17:47:22.998 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.998 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.998 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:22.998 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226801+0000", 2024-09-04T17:47:22.998 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.140160321 2024-09-04T17:47:22.998 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.998 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.998 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:22.998 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226951+0000", 2024-09-04T17:47:22.998 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014965200000000001 2024-09-04T17:47:22.998 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:22.998 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:22.998 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:22.998 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.227185+0000", 2024-09-04T17:47:22.999 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00023384799999999999 2024-09-04T17:47:22.999 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.999 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:22.999 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.999 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:22.999 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:22.999 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:23.000 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:47:23.000 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:47:23.001 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:47:23.004 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-04T17:47:23.005 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:47:23.005 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-04T17:47:23.005 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-04T17:47:23.006 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-04T17:47:23.006 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-04T17:47:23.006 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 'trim stale osdmaps' 2024-09-04T17:47:23.007 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #31, image image_client.0-clone31 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent32 2024-09-04T17:47:23.007 INFO:teuthology.orchestra.run.smithi092.stdout:4587 punch from 0x17d23f8 to 0x17dddef, (0xb9f7 bytes) 2024-09-04T17:47:23.007 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:22.998+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:23.007 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-04T17:47:23.008 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-04T17:47:23.008 INFO:teuthology.orchestra.run.smithi092.stdout:4588 read 0x5f334cd thru 0x5f3d05a (0x9b8e bytes) 2024-09-04T17:47:23.014 INFO:teuthology.orchestra.run.smithi092.stdout:4589 trunc from 0xe8cbb79 to 0x803938f 2024-09-04T17:47:23.024 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:47:23.025 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:47:23.034 INFO:teuthology.orchestra.run.smithi092.stdout:4594 read 0x585e6f8 thru 0x5863a52 (0x535b bytes) 2024-09-04T17:47:23.034 INFO:teuthology.orchestra.run.smithi092.stdout:4596 read 0x4d0a6c thru 0x4d23ca (0x195f bytes) 2024-09-04T17:47:23.045 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:47:23.045 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:47:23.045 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-04T17:47:23.046 INFO:teuthology.orchestra.run.smithi092.stdout:4597 writesame 0x9ce1003 thru 0x9cedcec data_size 0x15(0xccea bytes) 2024-09-04T17:47:23.048 INFO:teuthology.orchestra.run.smithi092.stdout:4599 read 0x5f13eae thru 0x5f22606 (0xe759 bytes) 2024-09-04T17:47:23.048 INFO:teuthology.orchestra.run.smithi092.stdout:4600 read 0x532ccf8 thru 0x533a305 (0xd60e bytes) 2024-09-04T17:47:23.049 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:23.049 INFO:teuthology.orchestra.run.smithi092.stdout:4604 read 0x91e57ce thru 0x91ea97a (0x51ad bytes) 2024-09-04T17:47:23.049 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:23.060 INFO:teuthology.orchestra.run.smithi092.stdout:4609 trunc from 0x9cedced to 0x3b3495f 2024-09-04T17:47:23.066 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:47:23.067 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:47:23.073 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-04T17:47:23.073 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-04T17:47:23.095 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:23.095 INFO:teuthology.orchestra.run.smithi092.stdout:4614 read 0x264f6e3 thru 0x265885d (0x917b bytes) 2024-09-04T17:47:23.096 INFO:teuthology.orchestra.run.smithi092.stdout:4615 punch from 0x38032b to 0x38744a, (0x711f bytes) 2024-09-04T17:47:23.101 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:23.100+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:23.107 INFO:teuthology.orchestra.run.smithi092.stdout:4616 trunc from 0x3b3495f to 0xaca42fa 2024-09-04T17:47:23.122 INFO:teuthology.orchestra.run.smithi092.stdout:4617 trunc from 0xaca42fa to 0xe554ab8 2024-09-04T17:47:23.143 INFO:teuthology.orchestra.run.smithi092.stdout:4618 trunc from 0xe554ab8 to 0x65c43d2 2024-09-04T17:47:23.158 INFO:teuthology.orchestra.run.smithi092.stdout:4619 write 0x6446300 thru 0x645256a (0xc26b bytes) 2024-09-04T17:47:23.162 INFO:teuthology.orchestra.run.smithi092.stdout:4621 read 0x30fa14e thru 0x30fac5b (0xb0e bytes) 2024-09-04T17:47:23.162 INFO:teuthology.orchestra.run.smithi092.stdout:4622 punch from 0x81193a to 0x816eed, (0x55b3 bytes) 2024-09-04T17:47:23.164 INFO:teuthology.orchestra.run.smithi092.stdout:4623 trunc from 0x65c43d2 to 0x7a54f3f 2024-09-04T17:47:23.180 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:23.181 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:23.193 INFO:teuthology.orchestra.run.smithi092.stdout:4624 write 0xd0139e4 thru 0xd018f15 (0x5532 bytes) 2024-09-04T17:47:23.196 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-04T17:47:23.196 INFO:teuthology.orchestra.run.smithi092.stdout:4626 write 0x66b0d2c thru 0x66b4462 (0x3737 bytes) 2024-09-04T17:47:23.201 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:23.198+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:23.206 INFO:teuthology.orchestra.run.smithi092.stdout:4627 writesame 0xd755907 thru 0xd75b5c0 data_size 0x53(0x5cba bytes) 2024-09-04T17:47:23.207 INFO:teuthology.orchestra.run.smithi092.stdout:4628 read 0xb18780b thru 0xb1959d7 (0xe1cd bytes) 2024-09-04T17:47:23.207 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:23.207 INFO:teuthology.orchestra.run.smithi092.stdout:4630 write 0x58eea41 thru 0x58fb3c8 (0xc988 bytes) 2024-09-04T17:47:23.208 INFO:teuthology.orchestra.run.smithi092.stdout:4631 writesame 0xb35a19 thru 0xb444cd data_size 0x3d(0xeab5 bytes) 2024-09-04T17:47:23.213 INFO:teuthology.orchestra.run.smithi092.stderr:no valid command found; 10 closest matches: 2024-09-04T17:47:23.213 INFO:teuthology.orchestra.run.smithi092.stderr:0 2024-09-04T17:47:23.213 INFO:teuthology.orchestra.run.smithi092.stderr:1 2024-09-04T17:47:23.213 INFO:teuthology.orchestra.run.smithi092.stderr:2 2024-09-04T17:47:23.214 INFO:teuthology.orchestra.run.smithi092.stderr:abort 2024-09-04T17:47:23.214 INFO:teuthology.orchestra.run.smithi092.stderr:assert 2024-09-04T17:47:23.214 INFO:teuthology.orchestra.run.smithi092.stderr:bench [] [] [] [] 2024-09-04T17:47:23.214 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs debug_inject_read_zeros 2024-09-04T17:47:23.214 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs files list 2024-09-04T17:47:23.214 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs stats 2024-09-04T17:47:23.214 INFO:teuthology.orchestra.run.smithi092.stderr:bluestore allocator dump block 2024-09-04T17:47:23.214 INFO:teuthology.orchestra.run.smithi092.stderr:Error EINVAL: invalid command 2024-09-04T17:47:23.215 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-04T17:47:23.216 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-04T17:47:23.219 INFO:teuthology.orchestra.run.smithi092.stdout:4634 read 0xf9f2cf thru 0xfa149a (0x21cc bytes) 2024-09-04T17:47:23.219 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:23.219 INFO:teuthology.orchestra.run.smithi092.stdout:4638 punch from 0x2a65 to 0x37c3, (0xd5e bytes) 2024-09-04T17:47:23.219 INFO:teuthology.orchestra.run.smithi092.stdout:4639 read 0xd71e539 thru 0xd729ddc (0xb8a4 bytes) 2024-09-04T17:47:23.220 INFO:teuthology.orchestra.run.smithi092.stdout:4642 writesame 0xac1152d thru 0xac1cb5a data_size 0x9(0xb62e bytes) 2024-09-04T17:47:23.224 INFO:teuthology.orchestra.run.smithi092.stdout:4644 read 0x7e8a026 thru 0x7e8d83f (0x381a bytes) 2024-09-04T17:47:23.225 INFO:teuthology.orchestra.run.smithi092.stdout:4645 read 0xda99a5 thru 0xdb3c80 (0xa2dc bytes) 2024-09-04T17:47:23.225 INFO:teuthology.orchestra.run.smithi092.stdout:4646 read 0x9183e59 thru 0x9190f92 (0xd13a bytes) 2024-09-04T17:47:23.225 INFO:teuthology.orchestra.run.smithi092.stdout:4647 write 0x416edfa thru 0x41761e2 (0x73e9 bytes) 2024-09-04T17:47:23.228 INFO:teuthology.orchestra.run.smithi092.stdout:4649 read 0x2861805 thru 0x28707bc (0xefb8 bytes) 2024-09-04T17:47:23.229 INFO:teuthology.orchestra.run.smithi092.stdout:4650 clone 34 order 19 su 65536 sc 12 2024-09-04T17:47:23.302 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:47:23.300+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:47:23.360 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:23.360 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:23.375 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-04T17:47:23.421 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:23.420+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:23.504 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:23.500+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:23.546 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:23.546 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:23.560 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-04T17:47:23.605 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:23.602+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:23.690 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:23.690 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:23.704 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-04T17:47:23.705 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:23.704+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:23.807 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:23.804+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:23.882 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:23.882 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:23.896 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-04T17:47:23.941 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:23.938+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:23.971 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone33 from 0xd75b5c1 (overlap 0x3b3495f) to 0x0 2024-09-04T17:47:24.008 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:24.008+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:24.032 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:24.032 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:24.048 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-04T17:47:24.141 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:24.138+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:24.210 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:24.206+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:24.217 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #32, image image_client.0-clone32 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent33 2024-09-04T17:47:24.226 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:24.226 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:24.240 INFO:teuthology.orchestra.run.smithi092.stdout:4651 writesame 0x4e2e6a3 thru 0x4e3a077 data_size 0x1(0xb9d5 bytes) 2024-09-04T17:47:24.244 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-04T17:47:24.316 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:24.316 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:24.310+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:24.335 INFO:teuthology.orchestra.run.smithi092.stdout:4654 trunc from 0xd75b5c1 to 0x2433af8 2024-09-04T17:47:24.380 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:24.397 INFO:teuthology.orchestra.run.smithi092.stdout:4659 trunc from 0x2433af8 to 0xdf19245 2024-09-04T17:47:24.411 INFO:teuthology.orchestra.run.smithi092.stdout:4660 writesame 0x1456125 thru 0x145975f data_size 0x1(0x363b bytes) 2024-09-04T17:47:24.411 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:47:24.408+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:47:24.420 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:24.420 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:24.435 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-04T17:47:24.452 INFO:teuthology.orchestra.run.smithi092.stdout:4661 trunc from 0xdf19245 to 0x473f2f3 2024-09-04T17:47:24.465 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:24.481 INFO:teuthology.orchestra.run.smithi092.stdout:4667 write 0x68fd932 thru 0x68fee91 (0x1560 bytes) 2024-09-04T17:47:24.486 INFO:teuthology.orchestra.run.smithi092.stdout:4668 read 0x672f10c thru 0x67390b2 (0x9fa7 bytes) 2024-09-04T17:47:24.501 INFO:teuthology.orchestra.run.smithi092.stdout:4669 trunc from 0x68fee92 to 0xeb1b282 2024-09-04T17:47:24.512 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:47:24.508+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:47:24.513 INFO:teuthology.orchestra.run.smithi092.stdout:4672 punch from 0x9b31adf to 0x9b3da3a, (0xbf5b bytes) 2024-09-04T17:47:24.513 INFO:teuthology.orchestra.run.smithi092.stdout:4673 read 0x334a0a1 thru 0x3356a44 (0xc9a4 bytes) 2024-09-04T17:47:24.540 INFO:teuthology.orchestra.run.smithi092.stdout:4674 trunc from 0xeb1b282 to 0x7ac539 2024-09-04T17:47:24.585 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:24.585 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:24.601 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-04T17:47:24.607 INFO:teuthology.orchestra.run.smithi092.stdout:4675 read 0x1c47cc thru 0x1c8776 (0x3fab bytes) 2024-09-04T17:47:24.609 INFO:teuthology.orchestra.run.smithi092.stdout:4676 punch from 0x387642 to 0x3949b2, (0xd370 bytes) 2024-09-04T17:47:24.698 INFO:teuthology.orchestra.run.smithi092.stdout:4678 write 0x555d016 thru 0x5564923 (0x790e bytes) 2024-09-04T17:47:24.698 INFO:teuthology.orchestra.run.smithi092.stdout:4679 punch from 0x1c04225 to 0x1c0d6a2, (0x947d bytes) 2024-09-04T17:47:24.699 INFO:teuthology.orchestra.run.smithi092.stdout:4680 read 0x45d02fa thru 0x45d105b (0xd62 bytes) 2024-09-04T17:47:24.699 INFO:teuthology.orchestra.run.smithi092.stdout:4681 write 0xd84026a thru 0xd8404e8 (0x27f bytes) 2024-09-04T17:47:24.699 INFO:teuthology.orchestra.run.smithi092.stdout:4682 read 0x59aeae7 thru 0x59b7f57 (0x9471 bytes) 2024-09-04T17:47:24.699 INFO:teuthology.orchestra.run.smithi092.stdout:4683 trunc from 0xd8404e9 to 0x4a4640d 2024-09-04T17:47:24.699 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:24.694+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:24.732 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:24.732 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:24.749 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-04T17:47:24.749 INFO:teuthology.orchestra.run.smithi092.stdout:4684 write 0xde0c417 thru 0xde0e754 (0x233e bytes) 2024-09-04T17:47:24.756 INFO:teuthology.orchestra.run.smithi092.stdout:4685 punch from 0xcb8c760 to 0xcb98865, (0xc105 bytes) 2024-09-04T17:47:24.756 INFO:teuthology.orchestra.run.smithi092.stdout:4688 read 0xa4e4b19 thru 0xa4eee9e (0xa386 bytes) 2024-09-04T17:47:24.757 INFO:teuthology.orchestra.run.smithi092.stdout:4689 read 0x4d75cfc thru 0x4d7eaf4 (0x8df9 bytes) 2024-09-04T17:47:24.757 INFO:teuthology.orchestra.run.smithi092.stdout:4690 read 0x6942e6b thru 0x69495ec (0x6782 bytes) 2024-09-04T17:47:24.757 INFO:teuthology.orchestra.run.smithi092.stdout:4691 write 0xd541e68 thru 0xd54c00f (0xa1a8 bytes) 2024-09-04T17:47:24.764 INFO:teuthology.orchestra.run.smithi092.stdout:4692 read 0x3f6d8ea thru 0x3f74b90 (0x72a7 bytes) 2024-09-04T17:47:24.764 INFO:teuthology.orchestra.run.smithi092.stdout:4694 read 0x352df1f thru 0x35374f9 (0x95db bytes) 2024-09-04T17:47:24.765 INFO:teuthology.orchestra.run.smithi092.stdout:4695 writesame 0x19d4b67 thru 0x19dc603 data_size 0x1(0x7a9d bytes) 2024-09-04T17:47:24.783 INFO:teuthology.orchestra.run.smithi092.stdout:4696 read 0x4a41e2d thru 0x4a45773 (0x3947 bytes) 2024-09-04T17:47:24.783 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:24.784 INFO:teuthology.orchestra.run.smithi092.stdout:4698 trunc from 0xde0e755 to 0xd5640ca 2024-09-04T17:47:24.787 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-04T17:47:24.799 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:24.796+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:24.809 INFO:teuthology.orchestra.run.smithi092.stdout:4699 read 0xbc4eed7 thru 0xbc59457 (0xa581 bytes) 2024-09-04T17:47:24.809 INFO:teuthology.orchestra.run.smithi092.stdout:4701 trunc from 0xd5640ca to 0xd0d5efe 2024-09-04T17:47:24.832 INFO:teuthology.orchestra.run.smithi092.stdout:4703 write 0xa443b29 thru 0xa44c1ff (0x86d7 bytes) 2024-09-04T17:47:24.853 INFO:teuthology.orchestra.run.smithi092.stdout:4704 writesame 0xd9eebf8 thru 0xd9f5809 data_size 0x3a(0x6c12 bytes) 2024-09-04T17:47:24.853 INFO:teuthology.orchestra.run.smithi092.stdout:4705 write 0x38cecf3 thru 0x38dbb22 (0xce30 bytes) 2024-09-04T17:47:24.863 INFO:teuthology.orchestra.run.smithi092.stdout:4707 trunc from 0xd9f580a to 0xe484fb2 2024-09-04T17:47:24.875 INFO:teuthology.orchestra.run.smithi092.stdout:4708 read 0x96628b2 thru 0x966c9ec (0xa13b bytes) 2024-09-04T17:47:24.875 INFO:teuthology.orchestra.run.smithi092.stdout:4709 read 0xf7f293 thru 0xf8a5aa (0xb318 bytes) 2024-09-04T17:47:24.875 INFO:teuthology.orchestra.run.smithi092.stdout:4710 writesame 0xceed403 thru 0xcef96a7 data_size 0x1(0xc2a5 bytes) 2024-09-04T17:47:24.884 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:24.884 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:24.899 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:24.894+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:24.901 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-04T17:47:24.916 INFO:teuthology.orchestra.run.smithi092.stdout:4712 read 0x3087527 thru 0x309739c (0xfe76 bytes) 2024-09-04T17:47:24.929 INFO:teuthology.orchestra.run.smithi092.stdout:4713 trunc from 0xe484fb2 to 0x56fde0c 2024-09-04T17:47:24.967 INFO:teuthology.orchestra.run.smithi092.stdout:4714 writesame 0x5cfe20f thru 0x5d00e37 data_size 0x55(0x2c29 bytes) 2024-09-04T17:47:24.967 INFO:teuthology.orchestra.run.smithi092.stdout:4715 write 0x7bd500 thru 0x7c88df (0xb3e0 bytes) 2024-09-04T17:47:24.971 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:24.971 INFO:teuthology.orchestra.run.smithi092.stdout:4718 read 0x2ed4ebe thru 0x2ee382e (0xe971 bytes) 2024-09-04T17:47:24.971 INFO:teuthology.orchestra.run.smithi092.stdout:4719 writesame 0x612f3f thru 0x61d292 data_size 0x2(0xa354 bytes) 2024-09-04T17:47:25.001 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:24.998+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:25.002 INFO:teuthology.orchestra.run.smithi092.stdout:4720 read 0x3ea1da8 thru 0x3ea33e1 (0x163a bytes) 2024-09-04T17:47:25.003 INFO:teuthology.orchestra.run.smithi092.stdout:4721 punch from 0x5b91b64 to 0x5ba122c, (0xf6c8 bytes) 2024-09-04T17:47:25.003 INFO:teuthology.orchestra.run.smithi092.stdout:4722 punch from 0x644b85 to 0x65123a, (0xc6b5 bytes) 2024-09-04T17:47:25.029 INFO:teuthology.orchestra.run.smithi092.stdout:4724 write 0xbb3665a thru 0xbb3b5c4 (0x4f6b bytes) 2024-09-04T17:47:25.036 INFO:teuthology.orchestra.run.smithi092.stdout:4725 write 0x14c554e thru 0x14d33a3 (0xde56 bytes) 2024-09-04T17:47:25.037 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:25.037 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:25.047 INFO:teuthology.orchestra.run.smithi092.stdout:4726 write 0x5c9caa6 thru 0x5ca1b04 (0x505f bytes) 2024-09-04T17:47:25.052 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-04T17:47:25.056 INFO:teuthology.orchestra.run.smithi092.stdout:4727 punch from 0xb2eeb18 to 0xb2f4a06, (0x5eee bytes) 2024-09-04T17:47:25.056 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:25.056 INFO:teuthology.orchestra.run.smithi092.stdout:4731 write 0x327d189 thru 0x328938d (0xc205 bytes) 2024-09-04T17:47:25.064 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:25.064 INFO:teuthology.orchestra.run.smithi092.stdout:4733 writesame 0xb2a8ec2 thru 0xb2b6c6a data_size 0x5(0xdda9 bytes) 2024-09-04T17:47:25.074 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-04T17:47:25.089 INFO:teuthology.orchestra.run.smithi092.stdout:4734 writesame 0xc8c3415 thru 0xc8ca220 data_size 0x2(0x6e0c bytes) 2024-09-04T17:47:25.101 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:25.098+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:25.104 INFO:teuthology.orchestra.run.smithi092.stdout:4736 trunc from 0xc8ca221 to 0x4028b5c 2024-09-04T17:47:25.120 INFO:teuthology.orchestra.run.smithi092.stdout:4738 punch from 0x10f7d31 to 0x1104e17, (0xd0e6 bytes) 2024-09-04T17:47:25.120 INFO:teuthology.orchestra.run.smithi092.stdout:4739 read 0x884ce9 thru 0x88c587 (0x789f bytes) 2024-09-04T17:47:25.120 INFO:teuthology.orchestra.run.smithi092.stdout:4740 read 0xecb288 thru 0xed1f6d (0x6ce6 bytes) 2024-09-04T17:47:25.121 INFO:teuthology.orchestra.run.smithi092.stdout:4741 punch from 0x239ba0f to 0x23a2aa0, (0x7091 bytes) 2024-09-04T17:47:25.135 INFO:teuthology.orchestra.run.smithi092.stdout:4743 trunc from 0x4028b5c to 0xcd60ae5 2024-09-04T17:47:25.162 INFO:teuthology.orchestra.run.smithi092.stdout:4744 trunc from 0xcd60ae5 to 0x2de186f 2024-09-04T17:47:25.179 INFO:teuthology.orchestra.run.smithi092.stdout:4745 write 0x66e8ae thru 0x674e4b (0x659e bytes) 2024-09-04T17:47:25.183 INFO:teuthology.orchestra.run.smithi092.stdout:4748 punch from 0x280873d to 0x280b87d, (0x3140 bytes) 2024-09-04T17:47:25.194 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:25.194 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:25.197 INFO:teuthology.orchestra.run.smithi092.stdout:4750 writesame 0x59427e7 thru 0x594a03d data_size 0x3f(0x7857 bytes) 2024-09-04T17:47:25.197 INFO:teuthology.orchestra.run.smithi092.stdout:4751 read 0x1c86fb2 thru 0x1c96121 (0xf170 bytes) 2024-09-04T17:47:25.197 INFO:teuthology.orchestra.run.smithi092.stdout:4753 read 0x2f247b7 thru 0x2f2c546 (0x7d90 bytes) 2024-09-04T17:47:25.198 INFO:teuthology.orchestra.run.smithi092.stdout:4754 read 0x2a432b4 thru 0x2a472ec (0x4039 bytes) 2024-09-04T17:47:25.198 INFO:teuthology.orchestra.run.smithi092.stdout:4755 punch from 0x16d7dd9 to 0x16e692f, (0xeb56 bytes) 2024-09-04T17:47:25.198 INFO:teuthology.orchestra.run.smithi092.stdout:4756 write 0x3250c07 thru 0x32538ae (0x2ca8 bytes) 2024-09-04T17:47:25.202 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:25.200+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:25.203 INFO:teuthology.orchestra.run.smithi092.stdout:4757 read 0x28b57f0 thru 0x28bc9bb (0x71cc bytes) 2024-09-04T17:47:25.209 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-04T17:47:25.217 INFO:teuthology.orchestra.run.smithi092.stdout:4759 writesame 0x8ff23ce thru 0x8ff5c9a data_size 0x25(0x38cd bytes) 2024-09-04T17:47:25.217 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:25.217 INFO:teuthology.orchestra.run.smithi092.stdout:4761 write 0x236ae1e thru 0x23755e6 (0xa7c9 bytes) 2024-09-04T17:47:25.229 INFO:teuthology.orchestra.run.smithi092.stdout:4762 write 0x93a4f0b thru 0x93b06d5 (0xb7cb bytes) 2024-09-04T17:47:25.233 INFO:teuthology.orchestra.run.smithi092.stdout:4763 write 0xa29d35 thru 0xa3676e (0xca3a bytes) 2024-09-04T17:47:25.237 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:25.249 INFO:teuthology.orchestra.run.smithi092.stdout:4767 writesame 0xc98d938 thru 0xc98ea83 data_size 0xc(0x114c bytes) 2024-09-04T17:47:25.249 INFO:teuthology.orchestra.run.smithi092.stdout:4768 punch from 0x9533aeb to 0x953eff3, (0xb508 bytes) 2024-09-04T17:47:25.257 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:47:25.257 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:47:25.258 INFO:teuthology.orchestra.run.smithi092.stdout:4769 trunc from 0xc98ea84 to 0x7551274 2024-09-04T17:47:25.271 INFO:teuthology.orchestra.run.smithi092.stdout:4770 read 0x247170 thru 0x25496d (0xd7fe bytes) 2024-09-04T17:47:25.271 INFO:teuthology.orchestra.run.smithi092.stdout:4771 write 0x580264a thru 0x580bc19 (0x95d0 bytes) 2024-09-04T17:47:25.275 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:47:25.276 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:47:25.277 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:25.277 INFO:teuthology.orchestra.run.smithi092.stdout:4775 read 0x1e59c27 thru 0x1e6891d (0xecf7 bytes) 2024-09-04T17:47:25.277 INFO:teuthology.orchestra.run.smithi092.stdout:4776 read 0x336d01d thru 0x337148e (0x4472 bytes) 2024-09-04T17:47:25.277 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-04T17:47:25.283 INFO:teuthology.orchestra.run.smithi092.stdout:4777 trunc from 0x7551274 to 0x1d6e681 2024-09-04T17:47:25.294 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-04T17:47:25.294 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-04T17:47:25.303 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:25.298+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:25.313 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:47:25.314 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:47:25.314 INFO:teuthology.orchestra.run.smithi092.stdout:4778 write 0x3726cb1 thru 0x3733473 (0xc7c3 bytes) 2024-09-04T17:47:25.321 INFO:teuthology.orchestra.run.smithi092.stdout:4780 read 0x1df77 thru 0x2a055 (0xc0df bytes) 2024-09-04T17:47:25.322 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:25.336 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:47:25.336 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:47:25.337 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-04T17:47:25.342 INFO:teuthology.orchestra.run.smithi092.stdout:4782 write 0xb441626 thru 0xb44d1f4 (0xbbcf bytes) 2024-09-04T17:47:25.356 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:47:25.356 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:47:25.360 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-04T17:47:25.360 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-04T17:47:25.362 INFO:teuthology.orchestra.run.smithi092.stdout:4784 write 0xdc895e3 thru 0xdc8b3b3 (0x1dd1 bytes) 2024-09-04T17:47:25.366 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:25.366 INFO:teuthology.orchestra.run.smithi092.stdout:4786 write 0x8802c8f thru 0x8809adc (0x6e4e bytes) 2024-09-04T17:47:25.370 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:25.370 INFO:teuthology.orchestra.run.smithi092.stdout:4791 write 0x2cbe8e5 thru 0x2cc3b8d (0x52a9 bytes) 2024-09-04T17:47:25.377 INFO:teuthology.orchestra.run.smithi092.stdout:4792 read 0xa63f915 thru 0xa64a645 (0xad31 bytes) 2024-09-04T17:47:25.377 INFO:teuthology.orchestra.run.smithi092.stdout:4793 write 0xdbcdd66 thru 0xdbd9d97 (0xc032 bytes) 2024-09-04T17:47:25.392 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:25.392 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:47:25.392 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:47:25.392 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:47:25.393 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:25.393 INFO:teuthology.orchestra.run.smithi092.stdout:4794 writesame 0xe6b07b9 thru 0xe6bc534 data_size 0x86(0xbd7c bytes) 2024-09-04T17:47:25.393 INFO:teuthology.orchestra.run.smithi092.stdout:4796 write 0x1f15590 thru 0x1f15822 (0x293 bytes) 2024-09-04T17:47:25.398 INFO:teuthology.orchestra.run.smithi092.stdout:4798 read 0xa0b8560 thru 0xa0ba116 (0x1bb7 bytes) 2024-09-04T17:47:25.398 INFO:teuthology.orchestra.run.smithi092.stdout:4800 write 0x5f12118 thru 0x5f1601c (0x3f05 bytes) 2024-09-04T17:47:25.403 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:47:25.400+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:47:25.408 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-04T17:47:25.425 INFO:teuthology.orchestra.run.smithi092.stdout:4801 trunc from 0xe6bc535 to 0x5761545 2024-09-04T17:47:25.443 INFO:teuthology.orchestra.run.smithi092.stdout:4802 read 0x184d6d8 thru 0x1859dfd (0xc726 bytes) 2024-09-04T17:47:25.444 INFO:teuthology.orchestra.run.smithi092.stdout:4804 write 0x2591ece thru 0x2594b84 (0x2cb7 bytes) 2024-09-04T17:47:25.469 INFO:teuthology.orchestra.run.smithi092.stdout:4806 trunc from 0x5761545 to 0xe88d7dc 2024-09-04T17:47:25.480 INFO:teuthology.orchestra.run.smithi092.stdout:4807 read 0xcf2d5b thru 0xcf87ce (0x5a74 bytes) 2024-09-04T17:47:25.480 INFO:teuthology.orchestra.run.smithi092.stdout:4808 read 0xd776dfa thru 0xd782a63 (0xbc6a bytes) 2024-09-04T17:47:25.480 INFO:teuthology.orchestra.run.smithi092.stdout:4810 read 0x7096647 thru 0x7096fd6 (0x990 bytes) 2024-09-04T17:47:25.480 INFO:teuthology.orchestra.run.smithi092.stdout:4812 read 0x913cacf thru 0x914578f (0x8cc1 bytes) 2024-09-04T17:47:25.480 INFO:teuthology.orchestra.run.smithi092.stdout:4813 read 0x326cd73 thru 0x327a563 (0xd7f1 bytes) 2024-09-04T17:47:25.481 INFO:teuthology.orchestra.run.smithi092.stdout:4815 punch from 0x2b5d976 to 0x2b64ad5, (0x715f bytes) 2024-09-04T17:47:25.481 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:25.481 INFO:teuthology.orchestra.run.smithi092.stdout:4817 write 0x7b8e74a thru 0x7b9b844 (0xd0fb bytes) 2024-09-04T17:47:25.487 INFO:teuthology.orchestra.run.smithi092.stdout:4819 write 0xc40cd5d thru 0xc416cca (0x9f6e bytes) 2024-09-04T17:47:25.504 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:25.502+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:25.513 INFO:teuthology.orchestra.run.smithi092.stdout:4821 trunc from 0xe88d7dc to 0x6aee3f8 2024-09-04T17:47:25.531 INFO:teuthology.orchestra.run.smithi092.stdout:4822 punch from 0x3142c96 to 0x31482a7, (0x5611 bytes) 2024-09-04T17:47:25.531 INFO:teuthology.orchestra.run.smithi092.stdout:4823 punch from 0x13b58e4 to 0x13ba5f4, (0x4d10 bytes) 2024-09-04T17:47:25.531 INFO:teuthology.orchestra.run.smithi092.stdout:4824 punch from 0x29e6b76 to 0x29eab92, (0x401c bytes) 2024-09-04T17:47:25.531 INFO:teuthology.orchestra.run.smithi092.stdout:4825 writesame 0x4a845c0 thru 0x4a87ba2 data_size 0x1f(0x35e3 bytes) 2024-09-04T17:47:25.537 INFO:teuthology.orchestra.run.smithi092.stdout:4826 writesame 0x4465f34 thru 0x446fc09 data_size 0x92(0x9cd6 bytes) 2024-09-04T17:47:25.559 INFO:teuthology.orchestra.run.smithi092.stdout:4827 trunc from 0x6aee3f8 to 0xde26a14 2024-09-04T17:47:25.570 INFO:teuthology.orchestra.run.smithi092.stdout:4828 read 0x85ebb5b thru 0x85f8f52 (0xd3f8 bytes) 2024-09-04T17:47:25.571 INFO:teuthology.orchestra.run.smithi092.stdout:4829 read 0xa6c980f thru 0xa6ce725 (0x4f17 bytes) 2024-09-04T17:47:25.571 INFO:teuthology.orchestra.run.smithi092.stdout:4831 punch from 0x459c73b to 0x45a9109, (0xc9ce bytes) 2024-09-04T17:47:25.571 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:25.571 INFO:teuthology.orchestra.run.smithi092.stdout:4834 write 0xd1805ad thru 0xd18f201 (0xec55 bytes) 2024-09-04T17:47:25.576 INFO:teuthology.orchestra.run.smithi092.stdout:4835 read 0x5c31ab5 thru 0x5c3f5be (0xdb0a bytes) 2024-09-04T17:47:25.577 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:25.577 INFO:teuthology.orchestra.run.smithi092.stdout:4838 writesame 0x114b4f7 thru 0x115b3a1 data_size 0x55(0xfeab bytes) 2024-09-04T17:47:25.583 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:25.583 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:47:25.583 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:47:25.583 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:47:25.583 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.583 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:12722 3.1b 3:d8152af1:::rbd_data.1134ef3a5a7d.00000000000000bd:head [delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-04T17:47:25.583 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:16.421965+0000", 2024-09-04T17:47:25.583 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 69.160446457999996, 2024-09-04T17:47:25.584 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.890002811, 2024-09-04T17:47:25.584 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:25.584 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:25.584 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:25.584 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:25.584 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:25.584 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:25.584 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 12722 2024-09-04T17:47:25.584 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.584 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:25.584 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.584 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:25.584 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421965+0000", 2024-09-04T17:47:25.585 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:25.585 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.585 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.585 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:25.585 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421965+0000", 2024-09-04T17:47:25.585 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:25.585 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.585 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.585 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:25.585 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421966+0000", 2024-09-04T17:47:25.585 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.102e-06 2024-09-04T17:47:25.585 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.586 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.586 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:25.586 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421971+0000", 2024-09-04T17:47:25.586 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9989999999999999e-06 2024-09-04T17:47:25.586 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.586 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.586 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:25.586 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421981+0000", 2024-09-04T17:47:25.586 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.7380000000000006e-06 2024-09-04T17:47:25.586 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.586 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.586 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:25.587 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421989+0000", 2024-09-04T17:47:25.587 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.1890000000000007e-06 2024-09-04T17:47:25.587 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.587 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.587 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:25.587 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.422050+0000", 2024-09-04T17:47:25.587 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.0702000000000003e-05 2024-09-04T17:47:25.587 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.587 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.587 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for peered", 2024-09-04T17:47:25.587 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.422059+0000", 2024-09-04T17:47:25.587 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5440000000000004e-06 2024-09-04T17:47:25.587 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.588 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.588 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:25.588 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.308156+0000", 2024-09-04T17:47:25.588 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.88609676199999998 2024-09-04T17:47:25.588 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.588 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.588 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:25.588 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.308398+0000", 2024-09-04T17:47:25.588 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024215400000000001 2024-09-04T17:47:25.588 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.588 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.588 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 7", 2024-09-04T17:47:25.589 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.308795+0000", 2024-09-04T17:47:25.589 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000396704 2024-09-04T17:47:25.589 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.589 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.589 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:47:25.589 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.310831+0000", 2024-09-04T17:47:25.589 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0020364200000000002 2024-09-04T17:47:25.589 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.589 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.589 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:25.589 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.311818+0000", 2024-09-04T17:47:25.589 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00098649799999999993 2024-09-04T17:47:25.589 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.589 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.590 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:47:25.590 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.311905+0000", 2024-09-04T17:47:25.590 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7070999999999994e-05 2024-09-04T17:47:25.590 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.590 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.590 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:25.590 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.311968+0000", 2024-09-04T17:47:25.590 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.2928000000000005e-05 2024-09-04T17:47:25.591 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:25.591 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:25.591 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:25.591 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.591 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.591 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37359 3.f e58/24 3:f29e50a5:::rbd_data.1134c5a43465.0000000000000021:head v 58'135, mlcod=58'133)", 2024-09-04T17:47:25.591 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:47:25.591 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 45.965115853999997, 2024-09-04T17:47:25.591 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053483197000000003, 2024-09-04T17:47:25.591 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:25.591 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:25.591 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:25.591 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:25.592 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.592 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:25.592 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:47:25.592 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:25.592 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.592 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.592 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:25.592 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:47:25.592 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:25.592 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.592 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.592 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:25.592 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:47:25.593 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1699999999999997e-07 2024-09-04T17:47:25.593 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.593 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.593 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:25.593 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617338+0000", 2024-09-04T17:47:25.593 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.1168999999999998e-05 2024-09-04T17:47:25.593 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.593 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.593 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:25.593 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617351+0000", 2024-09-04T17:47:25.593 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3094e-05 2024-09-04T17:47:25.593 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.593 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.594 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:25.594 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617359+0000", 2024-09-04T17:47:25.594 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7970000000000006e-06 2024-09-04T17:47:25.594 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.594 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.594 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:25.594 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617417+0000", 2024-09-04T17:47:25.594 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7079e-05 2024-09-04T17:47:25.594 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.594 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.594 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:25.594 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617438+0000", 2024-09-04T17:47:25.595 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1654999999999998e-05 2024-09-04T17:47:25.595 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.595 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.595 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:47:25.595 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670738+0000", 2024-09-04T17:47:25.595 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053300142000000002 2024-09-04T17:47:25.595 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.595 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.595 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:25.595 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670779+0000", 2024-09-04T17:47:25.595 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0343999999999999e-05 2024-09-04T17:47:25.595 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:25.596 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:25.596 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:25.596 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.596 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.596 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37358 3.1d e58/24 3:bd3215f3:::rbd_data.1134c5a43465.0000000000000020:head v 58'91, mlcod=58'89)", 2024-09-04T17:47:25.596 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617502+0000", 2024-09-04T17:47:25.596 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 45.964909136000003, 2024-09-04T17:47:25.596 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053505641999999999, 2024-09-04T17:47:25.596 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:25.596 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:25.596 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:25.596 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:25.597 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.597 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:25.597 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617502+0000", 2024-09-04T17:47:25.597 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:25.597 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.597 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.597 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:25.597 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617502+0000", 2024-09-04T17:47:25.597 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:25.597 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.597 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.597 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:25.597 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617503+0000", 2024-09-04T17:47:25.598 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.184e-06 2024-09-04T17:47:25.598 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.598 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.598 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:25.598 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617526+0000", 2024-09-04T17:47:25.598 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2099999999999998e-05 2024-09-04T17:47:25.598 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.598 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.598 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:25.598 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617540+0000", 2024-09-04T17:47:25.598 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4741e-05 2024-09-04T17:47:25.598 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.598 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.599 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:25.599 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617550+0000", 2024-09-04T17:47:25.599 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4259999999999992e-06 2024-09-04T17:47:25.599 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.599 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.599 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:25.599 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617613+0000", 2024-09-04T17:47:25.599 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.3625000000000002e-05 2024-09-04T17:47:25.599 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.599 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.599 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:25.599 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617637+0000", 2024-09-04T17:47:25.599 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.3604999999999998e-05 2024-09-04T17:47:25.600 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.600 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.600 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:47:25.600 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670967+0000", 2024-09-04T17:47:25.600 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053329881000000003 2024-09-04T17:47:25.600 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.600 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.600 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:25.600 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671008+0000", 2024-09-04T17:47:25.600 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.108e-05 2024-09-04T17:47:25.600 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:25.600 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:25.601 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:25.601 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.601 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.601 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37361 3.1c e58/24 3:3c7f8976:::rbd_data.1134c5a43465.0000000000000023:head v 58'159, mlcod=58'157)", 2024-09-04T17:47:25.601 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617578+0000", 2024-09-04T17:47:25.601 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 45.964833286000001, 2024-09-04T17:47:25.601 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053990367999999997, 2024-09-04T17:47:25.601 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:25.601 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:25.601 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:25.601 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:25.602 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.602 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:25.602 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617578+0000", 2024-09-04T17:47:25.602 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:25.602 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.602 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.602 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:25.602 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617578+0000", 2024-09-04T17:47:25.602 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:25.603 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.603 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.603 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:25.603 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617579+0000", 2024-09-04T17:47:25.603 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7299999999999996e-07 2024-09-04T17:47:25.603 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.603 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.603 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:25.603 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617610+0000", 2024-09-04T17:47:25.603 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0951000000000002e-05 2024-09-04T17:47:25.603 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.603 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.604 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:25.604 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:47:25.604 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0002e-05 2024-09-04T17:47:25.604 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.604 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.604 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:25.604 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617626+0000", 2024-09-04T17:47:25.604 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6660000000000002e-06 2024-09-04T17:47:25.605 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.605 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.605 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:25.605 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670548+0000", 2024-09-04T17:47:25.605 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.052921943999999999 2024-09-04T17:47:25.605 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.605 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.605 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:25.605 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670574+0000", 2024-09-04T17:47:25.606 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5405000000000002e-05 2024-09-04T17:47:25.606 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.606 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.606 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:47:25.606 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671522+0000", 2024-09-04T17:47:25.606 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00094815299999999997 2024-09-04T17:47:25.606 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.606 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.606 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:25.606 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671568+0000", 2024-09-04T17:47:25.606 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6674000000000001e-05 2024-09-04T17:47:25.607 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:25.607 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:25.607 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:25.607 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.607 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.607 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37357 3.15 e58/49 3:a897312a:::rbd_data.1134c5a43465.000000000000001f:head v 58'117, mlcod=58'115)", 2024-09-04T17:47:25.607 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:47:25.607 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 45.964790948000001, 2024-09-04T17:47:25.607 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053734297, 2024-09-04T17:47:25.607 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:25.607 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:25.607 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:25.608 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:25.608 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.608 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:25.608 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:47:25.608 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:25.608 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.608 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.608 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:25.608 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:47:25.608 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:25.608 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.608 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.608 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:25.609 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617621+0000", 2024-09-04T17:47:25.609 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.7300000000000004e-07 2024-09-04T17:47:25.609 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.609 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.609 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:25.609 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617642+0000", 2024-09-04T17:47:25.609 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0185000000000001e-05 2024-09-04T17:47:25.609 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.609 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.609 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:25.609 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617654+0000", 2024-09-04T17:47:25.609 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2745000000000001e-05 2024-09-04T17:47:25.609 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.610 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.610 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:25.610 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617662+0000", 2024-09-04T17:47:25.610 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.0499999999999992e-06 2024-09-04T17:47:25.610 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.610 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.610 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:25.610 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617727+0000", 2024-09-04T17:47:25.610 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.4923000000000002e-05 2024-09-04T17:47:25.610 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.610 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.610 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:25.611 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617753+0000", 2024-09-04T17:47:25.611 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5899999999999999e-05 2024-09-04T17:47:25.611 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.611 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.611 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:47:25.611 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671298+0000", 2024-09-04T17:47:25.611 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053545276000000003 2024-09-04T17:47:25.611 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.611 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.611 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:25.611 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671355+0000", 2024-09-04T17:47:25.611 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6245000000000001e-05 2024-09-04T17:47:25.611 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:25.612 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:25.612 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:25.612 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.612 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.612 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44687 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:25.612 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253012+0000", 2024-09-04T17:47:25.612 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 35.329398881000003, 2024-09-04T17:47:25.612 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051128777, 2024-09-04T17:47:25.612 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:25.612 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:25.612 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:25.612 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:25.613 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:25.613 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:25.613 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44687 2024-09-04T17:47:25.613 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.613 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:25.613 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.613 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:25.613 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253012+0000", 2024-09-04T17:47:25.613 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:25.613 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.613 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.613 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:25.613 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253012+0000", 2024-09-04T17:47:25.614 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:25.614 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.614 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.614 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:25.614 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253014+0000", 2024-09-04T17:47:25.614 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.24e-06 2024-09-04T17:47:25.615 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.615 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.615 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:25.615 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253019+0000", 2024-09-04T17:47:25.615 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4569999999999996e-06 2024-09-04T17:47:25.615 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.615 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.615 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:25.616 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253035+0000", 2024-09-04T17:47:25.616 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6291999999999999e-05 2024-09-04T17:47:25.616 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.616 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.616 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:25.616 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253050+0000", 2024-09-04T17:47:25.616 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4423000000000001e-05 2024-09-04T17:47:25.616 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.616 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.616 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:25.616 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304115+0000", 2024-09-04T17:47:25.616 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051064778999999998 2024-09-04T17:47:25.617 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.617 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.617 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:25.617 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304141+0000", 2024-09-04T17:47:25.617 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6585999999999999e-05 2024-09-04T17:47:25.617 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:25.617 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:25.617 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:25.617 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.617 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.617 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44688 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:25.617 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253067+0000", 2024-09-04T17:47:25.617 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 35.329344329000001, 2024-09-04T17:47:25.617 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051116979, 2024-09-04T17:47:25.618 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:25.618 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:25.618 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:25.618 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:25.618 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:25.618 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:25.618 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44688 2024-09-04T17:47:25.618 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.618 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:25.618 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.618 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:25.618 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253067+0000", 2024-09-04T17:47:25.618 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:25.619 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.619 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.619 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:25.619 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253067+0000", 2024-09-04T17:47:25.619 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:25.619 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.619 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.619 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:25.619 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253068+0000", 2024-09-04T17:47:25.619 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0729999999999999e-06 2024-09-04T17:47:25.619 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.619 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.619 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:25.619 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253073+0000", 2024-09-04T17:47:25.620 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2259999999999996e-06 2024-09-04T17:47:25.620 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.620 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.620 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:25.620 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253091+0000", 2024-09-04T17:47:25.620 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7255999999999999e-05 2024-09-04T17:47:25.620 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.620 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.620 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:25.620 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253103+0000", 2024-09-04T17:47:25.620 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2632999999999999e-05 2024-09-04T17:47:25.620 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.620 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.621 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:25.621 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304173+0000", 2024-09-04T17:47:25.621 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051069537999999998 2024-09-04T17:47:25.621 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.621 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.621 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:25.621 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304184+0000", 2024-09-04T17:47:25.621 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1253000000000001e-05 2024-09-04T17:47:25.621 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:25.621 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:25.621 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:25.621 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.621 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.622 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44689 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:25.622 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253121+0000", 2024-09-04T17:47:25.622 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 35.329290192000002, 2024-09-04T17:47:25.622 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051192595, 2024-09-04T17:47:25.622 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:25.622 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:25.622 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:25.622 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:25.622 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:25.622 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:25.622 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44689 2024-09-04T17:47:25.622 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.622 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:25.622 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.623 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:25.623 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253121+0000", 2024-09-04T17:47:25.623 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:25.623 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.623 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.623 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:25.623 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253121+0000", 2024-09-04T17:47:25.623 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:25.623 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.623 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.623 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:25.624 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253122+0000", 2024-09-04T17:47:25.624 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.15e-06 2024-09-04T17:47:25.624 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.624 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.624 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:25.624 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253128+0000", 2024-09-04T17:47:25.624 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2569999999999998e-06 2024-09-04T17:47:25.624 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.624 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.624 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:25.624 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253145+0000", 2024-09-04T17:47:25.624 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7176e-05 2024-09-04T17:47:25.624 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.625 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.625 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:25.625 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253158+0000", 2024-09-04T17:47:25.625 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3139000000000001e-05 2024-09-04T17:47:25.625 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.625 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.625 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:25.625 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304287+0000", 2024-09-04T17:47:25.625 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051129149999999998 2024-09-04T17:47:25.625 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.625 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.625 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:25.625 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304314+0000", 2024-09-04T17:47:25.626 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6723000000000001e-05 2024-09-04T17:47:25.626 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:25.626 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:25.626 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:25.626 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.626 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.626 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44690 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:25.626 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253177+0000", 2024-09-04T17:47:25.626 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 35.329234067000002, 2024-09-04T17:47:25.626 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051169263, 2024-09-04T17:47:25.626 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:25.626 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:25.626 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:25.627 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:25.627 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:25.627 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:25.627 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44690 2024-09-04T17:47:25.627 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.627 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:25.627 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.627 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:25.627 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253177+0000", 2024-09-04T17:47:25.627 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:25.627 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.627 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.628 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:25.628 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253177+0000", 2024-09-04T17:47:25.628 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:25.628 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.628 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.628 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:25.628 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253178+0000", 2024-09-04T17:47:25.628 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1000000000000001e-06 2024-09-04T17:47:25.628 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.628 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.628 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:25.629 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253184+0000", 2024-09-04T17:47:25.629 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2089999999999996e-06 2024-09-04T17:47:25.629 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.629 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.629 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:25.629 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253200+0000", 2024-09-04T17:47:25.629 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6087e-05 2024-09-04T17:47:25.629 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.629 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.629 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:25.629 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253231+0000", 2024-09-04T17:47:25.629 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0874999999999998e-05 2024-09-04T17:47:25.629 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.629 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.630 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:25.630 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304337+0000", 2024-09-04T17:47:25.630 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051106430000000001 2024-09-04T17:47:25.630 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.630 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.630 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:25.630 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304347+0000", 2024-09-04T17:47:25.630 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5619999999999993e-06 2024-09-04T17:47:25.630 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:25.630 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:25.630 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:25.630 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.630 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.630 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44691 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:25.631 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253249+0000", 2024-09-04T17:47:25.631 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 35.329162033000003, 2024-09-04T17:47:25.631 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051195630999999998, 2024-09-04T17:47:25.631 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:25.631 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:25.631 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:25.631 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:25.631 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:25.631 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:25.631 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44691 2024-09-04T17:47:25.631 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.631 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:25.631 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.632 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:25.632 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253249+0000", 2024-09-04T17:47:25.632 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:25.632 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.632 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.632 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:25.632 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253249+0000", 2024-09-04T17:47:25.632 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:25.632 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.632 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.632 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:25.632 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253251+0000", 2024-09-04T17:47:25.633 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2330000000000001e-06 2024-09-04T17:47:25.633 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.633 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.633 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:25.633 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253256+0000", 2024-09-04T17:47:25.633 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6099999999999997e-06 2024-09-04T17:47:25.633 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.633 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.633 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:25.633 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253274+0000", 2024-09-04T17:47:25.633 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7952999999999999e-05 2024-09-04T17:47:25.633 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.634 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.634 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:25.634 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253287+0000", 2024-09-04T17:47:25.634 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2839e-05 2024-09-04T17:47:25.634 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.634 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.634 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:25.634 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304396+0000", 2024-09-04T17:47:25.634 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051108990999999999 2024-09-04T17:47:25.634 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.634 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.634 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:25.634 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304445+0000", 2024-09-04T17:47:25.634 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9005000000000003e-05 2024-09-04T17:47:25.635 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:25.635 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:25.635 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:25.635 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.635 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.635 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44692 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:25.635 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253337+0000", 2024-09-04T17:47:25.635 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 35.329074460000001, 2024-09-04T17:47:25.635 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051138909000000003, 2024-09-04T17:47:25.635 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:25.635 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:25.635 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:25.635 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:25.636 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:25.636 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:25.636 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44692 2024-09-04T17:47:25.636 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.636 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:25.636 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.636 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:25.636 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253337+0000", 2024-09-04T17:47:25.636 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:25.636 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.636 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.636 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:25.636 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253337+0000", 2024-09-04T17:47:25.637 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:25.637 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.637 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.637 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:25.637 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253338+0000", 2024-09-04T17:47:25.637 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.463e-06 2024-09-04T17:47:25.637 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.637 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.637 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:25.637 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253357+0000", 2024-09-04T17:47:25.637 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9043e-05 2024-09-04T17:47:25.637 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.637 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.638 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:25.638 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253379+0000", 2024-09-04T17:47:25.638 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1243e-05 2024-09-04T17:47:25.638 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.638 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.638 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:25.638 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253396+0000", 2024-09-04T17:47:25.638 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7059e-05 2024-09-04T17:47:25.638 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.638 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.638 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:25.638 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304466+0000", 2024-09-04T17:47:25.638 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051070624000000002 2024-09-04T17:47:25.639 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.639 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.639 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:25.639 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304476+0000", 2024-09-04T17:47:25.639 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4769999999999992e-06 2024-09-04T17:47:25.639 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:25.639 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:25.639 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:25.639 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.639 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.639 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44693 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:25.639 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253415+0000", 2024-09-04T17:47:25.639 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 35.328996056999998, 2024-09-04T17:47:25.640 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051141949999999999, 2024-09-04T17:47:25.640 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:25.640 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:25.640 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:25.640 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:25.640 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:25.640 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:25.640 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44693 2024-09-04T17:47:25.640 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.640 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:25.640 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.640 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:25.640 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253415+0000", 2024-09-04T17:47:25.641 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:25.641 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.641 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.641 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:25.641 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253415+0000", 2024-09-04T17:47:25.641 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:25.641 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.641 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.641 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:25.641 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253416+0000", 2024-09-04T17:47:25.641 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1799999999999999e-06 2024-09-04T17:47:25.641 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.641 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.641 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:25.642 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253422+0000", 2024-09-04T17:47:25.642 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.5940000000000001e-06 2024-09-04T17:47:25.642 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.642 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.642 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:25.642 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253438+0000", 2024-09-04T17:47:25.642 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6008999999999999e-05 2024-09-04T17:47:25.642 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.642 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.642 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:25.642 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253452+0000", 2024-09-04T17:47:25.642 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3543000000000001e-05 2024-09-04T17:47:25.642 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.643 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.643 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:25.643 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304533+0000", 2024-09-04T17:47:25.643 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051081609 2024-09-04T17:47:25.643 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.643 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.643 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:25.643 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304557+0000", 2024-09-04T17:47:25.643 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4015e-05 2024-09-04T17:47:25.643 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:25.643 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:25.643 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:25.643 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.643 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.644 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44694 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:25.644 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253469+0000", 2024-09-04T17:47:25.644 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 35.328941931999999, 2024-09-04T17:47:25.644 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051121486000000001, 2024-09-04T17:47:25.644 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:25.644 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:25.644 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:25.644 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:25.644 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:25.644 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:25.644 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44694 2024-09-04T17:47:25.644 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.644 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:25.644 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.645 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:25.645 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253469+0000", 2024-09-04T17:47:25.645 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:25.645 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.645 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.645 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:25.645 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253469+0000", 2024-09-04T17:47:25.645 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:25.645 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.645 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.645 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:25.645 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253471+0000", 2024-09-04T17:47:25.645 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1400000000000001e-06 2024-09-04T17:47:25.646 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.646 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.646 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:25.646 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253476+0000", 2024-09-04T17:47:25.646 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7370000000000003e-06 2024-09-04T17:47:25.646 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.646 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.646 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:25.646 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253493+0000", 2024-09-04T17:47:25.646 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6549e-05 2024-09-04T17:47:25.646 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.646 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.646 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:25.646 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253506+0000", 2024-09-04T17:47:25.647 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3406e-05 2024-09-04T17:47:25.647 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.647 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.647 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:25.647 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304582+0000", 2024-09-04T17:47:25.647 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051075534999999998 2024-09-04T17:47:25.647 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.647 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.647 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:25.647 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304591+0000", 2024-09-04T17:47:25.647 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1190000000000003e-06 2024-09-04T17:47:25.647 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:25.647 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:25.648 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:25.648 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.648 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.648 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44695 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:25.648 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253552+0000", 2024-09-04T17:47:25.648 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 35.328859178000002, 2024-09-04T17:47:25.648 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051068872000000001, 2024-09-04T17:47:25.648 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:25.648 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:25.648 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:25.648 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:25.648 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:25.648 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:25.649 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44695 2024-09-04T17:47:25.649 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.649 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:25.649 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.649 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:25.649 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253552+0000", 2024-09-04T17:47:25.649 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:25.649 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.649 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.649 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:25.649 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253552+0000", 2024-09-04T17:47:25.649 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:25.649 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.650 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.650 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:25.650 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253553+0000", 2024-09-04T17:47:25.650 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1960000000000001e-06 2024-09-04T17:47:25.650 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.650 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.650 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:25.650 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253560+0000", 2024-09-04T17:47:25.650 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.2040000000000002e-06 2024-09-04T17:47:25.650 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.650 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.650 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:25.650 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253578+0000", 2024-09-04T17:47:25.651 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8241999999999999e-05 2024-09-04T17:47:25.651 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.651 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.651 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:25.651 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253591+0000", 2024-09-04T17:47:25.651 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3453e-05 2024-09-04T17:47:25.651 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.651 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.651 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:25.651 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304614+0000", 2024-09-04T17:47:25.651 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051022456000000001 2024-09-04T17:47:25.651 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.651 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.652 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:25.652 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304621+0000", 2024-09-04T17:47:25.652 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3209999999999999e-06 2024-09-04T17:47:25.652 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:25.652 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:25.652 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:25.652 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.652 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.652 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44696 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:25.652 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253609+0000", 2024-09-04T17:47:25.652 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 35.328802676999999, 2024-09-04T17:47:25.652 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051115351000000003, 2024-09-04T17:47:25.652 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:25.653 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:25.653 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:25.653 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:25.653 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:25.653 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:25.653 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44696 2024-09-04T17:47:25.653 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.653 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:25.653 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.653 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:25.653 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253609+0000", 2024-09-04T17:47:25.653 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:25.653 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.653 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.654 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:25.654 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253609+0000", 2024-09-04T17:47:25.654 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:25.654 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.654 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.654 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:25.654 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253610+0000", 2024-09-04T17:47:25.654 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2300000000000001e-06 2024-09-04T17:47:25.654 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.654 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.654 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:25.654 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253615+0000", 2024-09-04T17:47:25.654 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7260000000000002e-06 2024-09-04T17:47:25.655 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.655 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.655 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:25.655 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253632+0000", 2024-09-04T17:47:25.655 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7039999999999999e-05 2024-09-04T17:47:25.655 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.655 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.655 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:25.655 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253666+0000", 2024-09-04T17:47:25.655 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4662000000000001e-05 2024-09-04T17:47:25.655 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.655 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.655 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:25.656 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304677+0000", 2024-09-04T17:47:25.656 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051011017999999998 2024-09-04T17:47:25.656 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.656 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.656 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:25.656 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304724+0000", 2024-09-04T17:47:25.656 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6675000000000003e-05 2024-09-04T17:47:25.656 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:25.656 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:25.656 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:25.656 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.656 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.656 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44697 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:25.657 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253684+0000", 2024-09-04T17:47:25.657 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 35.328727182999998, 2024-09-04T17:47:25.657 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051073107, 2024-09-04T17:47:25.657 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:25.657 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:25.657 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:25.657 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:25.657 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:25.657 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:25.657 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44697 2024-09-04T17:47:25.657 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.657 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:25.657 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.657 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:25.658 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253684+0000", 2024-09-04T17:47:25.658 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:25.658 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.658 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.658 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:25.658 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253684+0000", 2024-09-04T17:47:25.658 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:25.658 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.658 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.658 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:25.658 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253685+0000", 2024-09-04T17:47:25.658 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0699999999999999e-06 2024-09-04T17:47:25.658 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.658 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.659 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:25.659 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253690+0000", 2024-09-04T17:47:25.659 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1959999999999999e-06 2024-09-04T17:47:25.659 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.659 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.659 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:25.659 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253707+0000", 2024-09-04T17:47:25.659 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6946e-05 2024-09-04T17:47:25.659 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.659 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.659 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:25.659 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253720+0000", 2024-09-04T17:47:25.659 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3076e-05 2024-09-04T17:47:25.659 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.660 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.660 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:25.660 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304745+0000", 2024-09-04T17:47:25.660 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051024760000000002 2024-09-04T17:47:25.660 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.660 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.660 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:25.660 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304757+0000", 2024-09-04T17:47:25.660 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2058999999999999e-05 2024-09-04T17:47:25.660 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:25.660 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:25.660 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:25.660 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.661 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.661 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44698 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:25.661 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253737+0000", 2024-09-04T17:47:25.661 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 35.328674831999997, 2024-09-04T17:47:25.661 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051176078, 2024-09-04T17:47:25.661 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:25.661 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:25.661 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:25.661 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:25.661 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:25.661 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:25.661 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44698 2024-09-04T17:47:25.661 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.662 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:25.662 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.662 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:25.662 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253737+0000", 2024-09-04T17:47:25.662 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:25.662 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.662 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.662 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:25.662 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253737+0000", 2024-09-04T17:47:25.662 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:25.662 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.662 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.662 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:25.663 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253738+0000", 2024-09-04T17:47:25.663 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0300000000000001e-06 2024-09-04T17:47:25.663 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.663 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.663 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:25.663 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253752+0000", 2024-09-04T17:47:25.663 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4358999999999999e-05 2024-09-04T17:47:25.663 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.663 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.663 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:25.663 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253771+0000", 2024-09-04T17:47:25.663 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8879999999999999e-05 2024-09-04T17:47:25.663 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.664 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.664 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:25.664 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253785+0000", 2024-09-04T17:47:25.664 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3852000000000001e-05 2024-09-04T17:47:25.664 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.664 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.664 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:25.664 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304876+0000", 2024-09-04T17:47:25.664 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051091105999999997 2024-09-04T17:47:25.664 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.664 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.664 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:25.664 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304913+0000", 2024-09-04T17:47:25.665 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.6850999999999997e-05 2024-09-04T17:47:25.665 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:25.665 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:25.665 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:25.665 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.665 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.665 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44699 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:25.665 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253804+0000", 2024-09-04T17:47:25.665 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 35.328607308000002, 2024-09-04T17:47:25.665 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051172250000000002, 2024-09-04T17:47:25.665 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:25.665 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:25.665 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:25.665 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:25.666 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:25.666 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:25.666 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44699 2024-09-04T17:47:25.666 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.666 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:25.666 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.666 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:25.666 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253804+0000", 2024-09-04T17:47:25.666 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:25.666 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.666 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.666 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:25.666 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253804+0000", 2024-09-04T17:47:25.666 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:25.667 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.667 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.667 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:25.667 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253805+0000", 2024-09-04T17:47:25.667 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.08e-06 2024-09-04T17:47:25.667 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.667 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.667 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:25.667 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253810+0000", 2024-09-04T17:47:25.667 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1830000000000002e-06 2024-09-04T17:47:25.667 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.667 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.667 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:25.668 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253828+0000", 2024-09-04T17:47:25.668 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7662000000000001e-05 2024-09-04T17:47:25.668 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.668 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.668 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:25.668 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253841+0000", 2024-09-04T17:47:25.668 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3203999999999999e-05 2024-09-04T17:47:25.668 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.668 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.668 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:25.668 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304966+0000", 2024-09-04T17:47:25.668 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051125273999999998 2024-09-04T17:47:25.668 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.669 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.669 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:25.669 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304976+0000", 2024-09-04T17:47:25.669 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.8470000000000008e-06 2024-09-04T17:47:25.669 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:25.669 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:25.669 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:25.669 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.669 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.669 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44700 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:25.669 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253858+0000", 2024-09-04T17:47:25.669 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 35.328553653999997, 2024-09-04T17:47:25.669 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051218957000000002, 2024-09-04T17:47:25.670 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:25.670 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:25.670 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:25.670 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:25.670 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:25.670 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:25.670 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44700 2024-09-04T17:47:25.670 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.670 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:25.670 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.671 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:25.671 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253858+0000", 2024-09-04T17:47:25.671 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:25.671 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.671 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.671 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:25.671 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253858+0000", 2024-09-04T17:47:25.671 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:25.671 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.671 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.671 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:25.671 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253859+0000", 2024-09-04T17:47:25.672 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.217e-06 2024-09-04T17:47:25.672 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.672 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.672 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:25.672 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253864+0000", 2024-09-04T17:47:25.672 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7600000000000002e-06 2024-09-04T17:47:25.672 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.672 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.672 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:25.672 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253880+0000", 2024-09-04T17:47:25.672 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6501999999999999e-05 2024-09-04T17:47:25.672 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.672 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.673 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:25.673 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253899+0000", 2024-09-04T17:47:25.673 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8907e-05 2024-09-04T17:47:25.673 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.673 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.673 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:25.673 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305052+0000", 2024-09-04T17:47:25.673 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051152989000000003 2024-09-04T17:47:25.673 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.673 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.673 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:25.673 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305077+0000", 2024-09-04T17:47:25.673 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4581999999999999e-05 2024-09-04T17:47:25.673 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:25.674 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:25.674 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:25.674 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.674 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.674 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44701 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:25.674 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253917+0000", 2024-09-04T17:47:25.674 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 35.328494302000003, 2024-09-04T17:47:25.674 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051231113000000002, 2024-09-04T17:47:25.674 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:25.674 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:25.674 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:25.674 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:25.674 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:25.675 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:25.675 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44701 2024-09-04T17:47:25.675 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.675 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:25.675 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.675 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:25.675 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253917+0000", 2024-09-04T17:47:25.675 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:25.675 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.675 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.675 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:25.675 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253917+0000", 2024-09-04T17:47:25.675 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:25.676 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.676 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.676 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:25.676 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253918+0000", 2024-09-04T17:47:25.676 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.136e-06 2024-09-04T17:47:25.676 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.676 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.676 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:25.676 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253924+0000", 2024-09-04T17:47:25.676 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.3070000000000002e-06 2024-09-04T17:47:25.676 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.676 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.676 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:25.677 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253940+0000", 2024-09-04T17:47:25.677 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6612e-05 2024-09-04T17:47:25.677 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.677 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.677 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:25.677 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253954+0000", 2024-09-04T17:47:25.677 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3709999999999999e-05 2024-09-04T17:47:25.677 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.677 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.677 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:25.677 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305136+0000", 2024-09-04T17:47:25.677 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051181935999999997 2024-09-04T17:47:25.677 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:25.677 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:25.678 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:25.678 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305148+0000", 2024-09-04T17:47:25.678 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2412000000000001e-05 2024-09-04T17:47:25.678 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:25.678 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:25.678 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:25.678 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:25.678 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:25.678 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:25.680 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-04T17:47:25.681 INFO:teuthology.orchestra.run.smithi092.stdout:4839 read 0x3ef9ccd thru 0x3f02f92 (0x92c6 bytes) 2024-09-04T17:47:25.681 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:25.681 INFO:teuthology.orchestra.run.smithi092.stdout:4842 read 0xcfacf7a thru 0xcfb066b (0x36f2 bytes) 2024-09-04T17:47:25.681 INFO:teuthology.orchestra.run.smithi092.stdout:4843 write 0x7e1be3d thru 0x7e24519 (0x86dd bytes) 2024-09-04T17:47:25.682 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:25.682 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:25.682 INFO:teuthology.orchestra.run.smithi092.stdout:4846 read 0x98d32cc thru 0x98d62b4 (0x2fe9 bytes) 2024-09-04T17:47:25.682 INFO:teuthology.orchestra.run.smithi092.stdout:4847 write 0x144b9d9 thru 0x14523d2 (0x69fa bytes) 2024-09-04T17:47:25.682 INFO:teuthology.orchestra.run.smithi092.stdout:4848 read 0x3a309aa thru 0x3a36098 (0x56ef bytes) 2024-09-04T17:47:25.682 INFO:teuthology.orchestra.run.smithi092.stdout:4849 read 0x2d60c0a thru 0x2d6d2cd (0xc6c4 bytes) 2024-09-04T17:47:25.682 INFO:teuthology.orchestra.run.smithi092.stdout:4852 trunc from 0xde26a14 to 0x47a17c8 2024-09-04T17:47:25.682 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:25.682 INFO:teuthology.orchestra.run.smithi092.stdout:4856 read 0x61f8a6 thru 0x624077 (0x47d2 bytes) 2024-09-04T17:47:25.682 INFO:teuthology.orchestra.run.smithi092.stdout:4860 writesame 0x5aad105 thru 0x5ab0e2a data_size 0x3(0x3d26 bytes) 2024-09-04T17:47:25.682 INFO:teuthology.orchestra.run.smithi092.stdout:4865 read 0x214d1c0 thru 0x215a3ae (0xd1ef bytes) 2024-09-04T17:47:25.682 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:25.682 INFO:teuthology.orchestra.run.smithi092.stdout:4867 read 0x99d0d8 thru 0x99ea98 (0x19c1 bytes) 2024-09-04T17:47:25.682 INFO:teuthology.orchestra.run.smithi092.stdout:4868 punch from 0x7d3aac to 0x7d4a0b, (0xf5f bytes) 2024-09-04T17:47:25.683 INFO:teuthology.orchestra.run.smithi092.stdout:4870 read 0x95f440 thru 0x967724 (0x82e5 bytes) 2024-09-04T17:47:25.683 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:25.674+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:25.683 INFO:teuthology.orchestra.run.smithi092.stdout:4871 trunc from 0x5ab0e2b to 0xb3b0e54 2024-09-04T17:47:25.695 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:25.695 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:25.695 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:25.695 INFO:teuthology.orchestra.run.smithi092.stdout:4878 write 0x6687ac8 thru 0x669756e (0xfaa7 bytes) 2024-09-04T17:47:25.704 INFO:teuthology.orchestra.run.smithi092.stdout:4879 read 0x51117f0 thru 0x511b071 (0x9882 bytes) 2024-09-04T17:47:25.714 INFO:teuthology.orchestra.run.smithi092.stdout:4880 trunc from 0xb3b0e54 to 0x72e25b9 2024-09-04T17:47:25.726 INFO:teuthology.orchestra.run.smithi092.stdout:4882 write 0x486d5a3 thru 0x487333e (0x5d9c bytes) 2024-09-04T17:47:25.759 INFO:teuthology.orchestra.run.smithi092.stdout:4884 write 0xd5d6f32 thru 0xd5de76d (0x783c bytes) 2024-09-04T17:47:25.764 INFO:teuthology.orchestra.run.smithi092.stdout:4889 write 0x63febce thru 0x640a2d1 (0xb704 bytes) 2024-09-04T17:47:25.774 INFO:teuthology.orchestra.run.smithi092.stdout:4892 punch from 0x129cd3d to 0x12a8ee9, (0xc1ac bytes) 2024-09-04T17:47:25.775 INFO:teuthology.orchestra.run.smithi092.stdout:4893 write 0x19611ba thru 0x196f3cd (0xe214 bytes) 2024-09-04T17:47:25.780 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:25.778+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:25.782 INFO:teuthology.orchestra.run.smithi092.stdout:4894 writesame 0x9a4d9aa thru 0x9a5872a data_size 0x1(0xad81 bytes) 2024-09-04T17:47:25.793 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is unset 2024-09-04T17:47:25.822 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:25.823 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:47:25.823 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:47:25.823 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:25.833 INFO:teuthology.orchestra.run.smithi092.stdout:4895 write 0x233e8e7 thru 0x23438f6 (0x5010 bytes) 2024-09-04T17:47:25.837 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-04T17:47:25.843 INFO:teuthology.orchestra.run.smithi092.stdout:4898 write 0x3cdebfb thru 0x3ce2c6a (0x4070 bytes) 2024-09-04T17:47:25.849 INFO:teuthology.orchestra.run.smithi092.stdout:4902 read 0x7cc4a6d thru 0x7cc5246 (0x7da bytes) 2024-09-04T17:47:25.849 INFO:teuthology.orchestra.run.smithi092.stdout:4903 punch from 0xc7dd5e5 to 0xc7e01fb, (0x2c16 bytes) 2024-09-04T17:47:25.849 INFO:teuthology.orchestra.run.smithi092.stdout:4904 write 0xb639d83 thru 0xb63e03a (0x42b8 bytes) 2024-09-04T17:47:25.858 INFO:teuthology.orchestra.run.smithi092.stdout:4906 write 0xc5c5c5f thru 0xc5c7be2 (0x1f84 bytes) 2024-09-04T17:47:25.865 INFO:teuthology.orchestra.run.smithi092.stdout:4907 trunc from 0xd5de76e to 0xe8c81c1 2024-09-04T17:47:25.881 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:25.880+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:25.906 INFO:teuthology.orchestra.run.smithi092.stdout:4909 trunc from 0xe8c81c1 to 0x21a361a 2024-09-04T17:47:25.971 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:25.971 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:47:25.971 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:47:25.971 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:47:25.971 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:25.982 INFO:teuthology.orchestra.run.smithi092.stdout:4910 writesame 0xe404ac8 thru 0xe411394 data_size 0x73(0xc8cd bytes) 2024-09-04T17:47:25.983 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:25.978+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:25.987 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-04T17:47:25.994 INFO:teuthology.orchestra.run.smithi092.stdout:4911 write 0xeadc9e9 thru 0xeadeb07 (0x211f bytes) 2024-09-04T17:47:26.000 INFO:teuthology.orchestra.run.smithi092.stdout:4912 write 0x3114765 thru 0x311af47 (0x67e3 bytes) 2024-09-04T17:47:26.007 INFO:teuthology.orchestra.run.smithi092.stdout:4913 write 0x9d164cd thru 0x9d24062 (0xdb96 bytes) 2024-09-04T17:47:26.018 INFO:teuthology.orchestra.run.smithi092.stdout:4915 trunc from 0xeadeb08 to 0xd9116c2 2024-09-04T17:47:26.037 INFO:teuthology.orchestra.run.smithi092.stdout:4916 read 0x52c803e thru 0x52d5765 (0xd728 bytes) 2024-09-04T17:47:26.048 INFO:teuthology.orchestra.run.smithi092.stdout:4918 writesame 0xdbe9c5d thru 0xdbf7499 data_size 0x1(0xd83d bytes) 2024-09-04T17:47:26.057 INFO:teuthology.orchestra.run.smithi092.stdout:4919 writesame 0xdf8fdb6 thru 0xdf9438b data_size 0x7(0x45d6 bytes) 2024-09-04T17:47:26.057 INFO:teuthology.orchestra.run.smithi092.stdout:4921 write 0x34189f4 thru 0x34253e5 (0xc9f2 bytes) 2024-09-04T17:47:26.063 INFO:teuthology.orchestra.run.smithi092.stdout:4922 read 0x45729be thru 0x457578c (0x2dcf bytes) 2024-09-04T17:47:26.063 INFO:teuthology.orchestra.run.smithi092.stdout:4923 read 0x418a1ec thru 0x4196046 (0xbe5b bytes) 2024-09-04T17:47:26.063 INFO:teuthology.orchestra.run.smithi092.stdout:4924 read 0xbe79946 thru 0xbe7d1e8 (0x38a3 bytes) 2024-09-04T17:47:26.063 INFO:teuthology.orchestra.run.smithi092.stdout:4925 punch from 0x31d645f to 0x31df438, (0x8fd9 bytes) 2024-09-04T17:47:26.063 INFO:teuthology.orchestra.run.smithi092.stdout:4926 punch from 0x8301beb to 0x8304d9c, (0x31b1 bytes) 2024-09-04T17:47:26.063 INFO:teuthology.orchestra.run.smithi092.stdout:4928 write 0x8bf3fe9 thru 0x8bfac00 (0x6c18 bytes) 2024-09-04T17:47:26.069 INFO:teuthology.orchestra.run.smithi092.stdout:4929 punch from 0xda17c5b to 0xda26d5c, (0xf101 bytes) 2024-09-04T17:47:26.069 INFO:teuthology.orchestra.run.smithi092.stdout:4930 write 0x3850319 thru 0x385e6e0 (0xe3c8 bytes) 2024-09-04T17:47:26.074 INFO:teuthology.orchestra.run.smithi092.stdout:4931 punch from 0xa14e589 to 0xa1555a6, (0x701d bytes) 2024-09-04T17:47:26.074 INFO:teuthology.orchestra.run.smithi092.stdout:4932 read 0x301f01c thru 0x3026cf5 (0x7cda bytes) 2024-09-04T17:47:26.074 INFO:teuthology.orchestra.run.smithi092.stdout:4933 punch from 0x2bf93ee to 0x2c028d6, (0x94e8 bytes) 2024-09-04T17:47:26.074 INFO:teuthology.orchestra.run.smithi092.stdout:4935 write 0x9dcd254 thru 0x9ddcf9d (0xfd4a bytes) 2024-09-04T17:47:26.079 INFO:teuthology.orchestra.run.smithi092.stdout:4936 writesame 0x5c13657 thru 0x5c1b3ae data_size 0x54(0x7d58 bytes) 2024-09-04T17:47:26.085 INFO:teuthology.orchestra.run.smithi092.stdout:4937 punch from 0x2b5e2f8 to 0x2b60c67, (0x296f bytes) 2024-09-04T17:47:26.086 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:47:26.084+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:47:26.114 INFO:teuthology.orchestra.run.smithi092.stdout:4940 trunc from 0xdf9438c to 0x267a409 2024-09-04T17:47:26.136 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:26.136 INFO:teuthology.orchestra.run.smithi092.stdout:4942 read 0x2528137 thru 0x2536209 (0xe0d3 bytes) 2024-09-04T17:47:26.137 INFO:teuthology.orchestra.run.smithi092.stdout:4943 trunc from 0x267a409 to 0x2f94b75 2024-09-04T17:47:26.149 INFO:teuthology.orchestra.run.smithi092.stdout:4944 punch from 0x1bb228e to 0x1bb85a4, (0x6316 bytes) 2024-09-04T17:47:26.149 INFO:teuthology.orchestra.run.smithi092.stdout:4946 read 0x26658c0 thru 0x2670270 (0xa9b1 bytes) 2024-09-04T17:47:26.149 INFO:teuthology.orchestra.run.smithi092.stdout:4948 writesame 0x2a776ca thru 0x2a80208 data_size 0x1(0x8b3f bytes) 2024-09-04T17:47:26.165 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:26.165 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:47:26.165 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:47:26.165 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:47:26.165 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.165 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8831 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2453504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:26.165 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:47:26.165 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 73.106128389000006, 2024-09-04T17:47:26.166 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14731111799999999, 2024-09-04T17:47:26.166 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:26.166 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:26.166 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:26.166 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:26.166 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:26.166 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:26.166 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8831 2024-09-04T17:47:26.166 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.166 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:26.166 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.166 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:26.167 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:47:26.167 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.167 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.167 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.167 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:26.167 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:47:26.167 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.167 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.167 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.167 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:26.167 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057811+0000", 2024-09-04T17:47:26.167 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.083e-06 2024-09-04T17:47:26.167 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.167 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.168 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:26.168 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057814+0000", 2024-09-04T17:47:26.168 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.407e-06 2024-09-04T17:47:26.168 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.168 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.168 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:26.168 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057825+0000", 2024-09-04T17:47:26.168 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.17e-05 2024-09-04T17:47:26.168 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.168 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.168 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:26.168 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057836+0000", 2024-09-04T17:47:26.169 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0283e-05 2024-09-04T17:47:26.169 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.169 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.169 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:26.169 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.204762+0000", 2024-09-04T17:47:26.169 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.146926205 2024-09-04T17:47:26.169 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.169 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.169 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:26.169 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.204895+0000", 2024-09-04T17:47:26.169 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013328999999999999 2024-09-04T17:47:26.169 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.169 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.170 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:26.170 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205121+0000", 2024-09-04T17:47:26.170 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00022614999999999999 2024-09-04T17:47:26.170 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.170 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:26.170 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.170 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.170 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.170 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8832 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2490368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:26.170 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:47:26.170 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 73.106090355999996, 2024-09-04T17:47:26.170 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14770012299999999, 2024-09-04T17:47:26.170 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:26.171 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:26.171 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:26.171 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:26.171 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:26.171 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:26.171 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8832 2024-09-04T17:47:26.171 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.171 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:26.171 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.171 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:26.171 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:47:26.171 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.171 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.172 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.172 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:26.172 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:47:26.172 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.172 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.172 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.172 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:26.172 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057849+0000", 2024-09-04T17:47:26.172 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6700000000000003e-07 2024-09-04T17:47:26.172 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.172 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.172 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:26.172 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057852+0000", 2024-09-04T17:47:26.173 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6359999999999998e-06 2024-09-04T17:47:26.173 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.173 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.173 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:26.173 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057860+0000", 2024-09-04T17:47:26.173 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8140000000000007e-06 2024-09-04T17:47:26.173 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.173 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.173 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:26.173 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057868+0000", 2024-09-04T17:47:26.173 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1999999999999997e-06 2024-09-04T17:47:26.173 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.173 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.174 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:26.174 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205208+0000", 2024-09-04T17:47:26.174 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14734067300000001 2024-09-04T17:47:26.174 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.174 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.174 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:26.174 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205347+0000", 2024-09-04T17:47:26.174 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013914900000000001 2024-09-04T17:47:26.174 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.174 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.174 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:26.174 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205548+0000", 2024-09-04T17:47:26.174 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000200984 2024-09-04T17:47:26.175 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.175 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:26.175 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.175 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.175 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.175 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8833 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2527232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:26.175 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:47:26.175 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 73.106058485999995, 2024-09-04T17:47:26.175 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148062572, 2024-09-04T17:47:26.175 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:26.175 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:26.175 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:26.175 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:26.176 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:26.176 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:26.176 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8833 2024-09-04T17:47:26.176 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.176 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:26.176 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.176 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:26.176 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:47:26.176 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.176 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.176 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.176 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:26.176 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:47:26.176 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.177 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.177 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.177 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:26.177 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057881+0000", 2024-09-04T17:47:26.177 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4300000000000002e-07 2024-09-04T17:47:26.177 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.177 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.177 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:26.177 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057893+0000", 2024-09-04T17:47:26.177 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1973999999999999e-05 2024-09-04T17:47:26.177 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.177 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.178 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:26.178 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057906+0000", 2024-09-04T17:47:26.178 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2716e-05 2024-09-04T17:47:26.178 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.178 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.178 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:26.178 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057913+0000", 2024-09-04T17:47:26.178 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3930000000000002e-06 2024-09-04T17:47:26.178 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.178 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.178 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:26.178 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205621+0000", 2024-09-04T17:47:26.178 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14770788800000001 2024-09-04T17:47:26.178 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.179 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.179 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:26.179 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205740+0000", 2024-09-04T17:47:26.179 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000118936 2024-09-04T17:47:26.179 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.179 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.179 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:26.179 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205943+0000", 2024-09-04T17:47:26.179 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020292199999999999 2024-09-04T17:47:26.179 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.179 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:26.179 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.179 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.180 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.180 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8834 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2564096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:26.180 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:47:26.180 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 73.106015206999999, 2024-09-04T17:47:26.180 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14846753200000001, 2024-09-04T17:47:26.180 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:26.180 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:26.180 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:26.180 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:26.180 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:26.180 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:26.180 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8834 2024-09-04T17:47:26.180 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.181 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:26.181 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.181 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:26.181 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:47:26.181 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.181 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.181 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.181 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:26.181 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:47:26.181 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.181 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.181 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.181 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:26.182 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057925+0000", 2024-09-04T17:47:26.182 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0270000000000001e-06 2024-09-04T17:47:26.182 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.182 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.182 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:26.182 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057936+0000", 2024-09-04T17:47:26.182 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.112e-05 2024-09-04T17:47:26.182 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.182 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.182 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:26.182 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057946+0000", 2024-09-04T17:47:26.182 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.009e-05 2024-09-04T17:47:26.182 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.182 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.183 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:26.183 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057953+0000", 2024-09-04T17:47:26.183 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.6229999999999997e-06 2024-09-04T17:47:26.183 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.183 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.183 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:26.183 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206027+0000", 2024-09-04T17:47:26.183 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148073591 2024-09-04T17:47:26.183 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.183 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.183 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:26.183 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206161+0000", 2024-09-04T17:47:26.183 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013411599999999999 2024-09-04T17:47:26.184 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.184 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.184 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:26.184 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206391+0000", 2024-09-04T17:47:26.184 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000229965 2024-09-04T17:47:26.184 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.184 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:26.184 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.184 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.184 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.184 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8835 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2600960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:26.184 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:47:26.184 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 73.105975556999994, 2024-09-04T17:47:26.184 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14882941599999999, 2024-09-04T17:47:26.185 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:26.185 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:26.185 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:26.185 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:26.185 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:26.185 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:26.185 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8835 2024-09-04T17:47:26.185 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.185 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:26.185 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.185 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:26.185 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:47:26.185 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.186 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.186 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.186 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:26.186 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:47:26.186 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.186 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.186 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.186 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:26.186 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058544+0000", 2024-09-04T17:47:26.186 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00058061899999999999 2024-09-04T17:47:26.186 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.186 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.186 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:26.186 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058550+0000", 2024-09-04T17:47:26.187 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.1129999999999997e-06 2024-09-04T17:47:26.187 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.187 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.187 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:26.187 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058566+0000", 2024-09-04T17:47:26.187 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5820000000000001e-05 2024-09-04T17:47:26.187 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.187 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.187 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:26.187 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058578+0000", 2024-09-04T17:47:26.187 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2537e-05 2024-09-04T17:47:26.187 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.187 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.188 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:26.188 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206470+0000", 2024-09-04T17:47:26.188 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14789153799999999 2024-09-04T17:47:26.188 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.188 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.188 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:26.188 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206589+0000", 2024-09-04T17:47:26.188 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000118837 2024-09-04T17:47:26.188 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.188 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.188 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:26.188 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206793+0000", 2024-09-04T17:47:26.189 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000203952 2024-09-04T17:47:26.189 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.189 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:26.189 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.189 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.189 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.189 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8836 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2654208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:26.189 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:47:26.189 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 73.105342868999998, 2024-09-04T17:47:26.189 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14862433999999999, 2024-09-04T17:47:26.189 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:26.189 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:26.189 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:26.190 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:26.190 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:26.190 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:26.190 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8836 2024-09-04T17:47:26.190 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.190 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:26.190 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.190 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:26.190 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:47:26.190 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.190 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.190 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.190 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:26.191 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:47:26.191 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.191 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.191 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.191 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:26.191 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062663+0000", 2024-09-04T17:47:26.191 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0040670619999999998 2024-09-04T17:47:26.191 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.191 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.191 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:26.191 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062668+0000", 2024-09-04T17:47:26.191 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4199999999999998e-06 2024-09-04T17:47:26.191 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.191 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.192 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:26.192 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062689+0000", 2024-09-04T17:47:26.192 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0356999999999998e-05 2024-09-04T17:47:26.192 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.192 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.192 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:26.192 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062726+0000", 2024-09-04T17:47:26.192 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7302999999999998e-05 2024-09-04T17:47:26.192 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.192 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.192 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:26.192 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206875+0000", 2024-09-04T17:47:26.192 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14414883100000001 2024-09-04T17:47:26.193 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.193 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.193 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:26.193 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.207016+0000", 2024-09-04T17:47:26.193 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000140822 2024-09-04T17:47:26.193 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.193 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.193 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:26.193 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.207220+0000", 2024-09-04T17:47:26.193 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020454499999999999 2024-09-04T17:47:26.193 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.193 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:26.193 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.194 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.194 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.194 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8844 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2928640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:26.194 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:47:26.194 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 73.100978785999999, 2024-09-04T17:47:26.194 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14752153100000001, 2024-09-04T17:47:26.194 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:26.194 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:26.194 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:26.194 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:26.194 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:26.194 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:26.194 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8844 2024-09-04T17:47:26.195 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.195 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:26.195 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.195 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:26.195 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:47:26.195 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.195 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.195 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.195 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:26.195 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:47:26.195 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.195 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.195 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.196 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:26.196 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062961+0000", 2024-09-04T17:47:26.196 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6700000000000003e-07 2024-09-04T17:47:26.196 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.196 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.196 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:26.196 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062973+0000", 2024-09-04T17:47:26.196 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2109999999999999e-05 2024-09-04T17:47:26.196 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.196 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.196 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:26.196 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062982+0000", 2024-09-04T17:47:26.196 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.6900000000000004e-06 2024-09-04T17:47:26.196 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.197 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.197 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:26.197 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062990+0000", 2024-09-04T17:47:26.197 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1899999999999998e-06 2024-09-04T17:47:26.197 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.197 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.197 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:26.197 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210090+0000", 2024-09-04T17:47:26.197 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14710026300000001 2024-09-04T17:47:26.197 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.197 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.197 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:26.197 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210225+0000", 2024-09-04T17:47:26.198 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013532400000000001 2024-09-04T17:47:26.198 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.198 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.198 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:26.198 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210481+0000", 2024-09-04T17:47:26.198 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00025628700000000001 2024-09-04T17:47:26.198 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.198 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:26.198 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.198 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.198 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.198 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8845 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2965504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:26.198 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:47:26.199 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 73.100939062999998, 2024-09-04T17:47:26.199 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147916819, 2024-09-04T17:47:26.199 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:26.199 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:26.199 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:26.199 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:26.199 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:26.199 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:26.199 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8845 2024-09-04T17:47:26.199 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.199 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:26.199 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.199 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:26.200 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:47:26.200 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.200 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.200 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.200 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:26.200 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:47:26.200 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.200 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.200 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.200 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:26.200 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:47:26.200 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.0399999999999995e-07 2024-09-04T17:47:26.200 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.201 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.201 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:26.201 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063003+0000", 2024-09-04T17:47:26.201 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6529999999999998e-06 2024-09-04T17:47:26.201 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.201 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.201 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:26.201 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063012+0000", 2024-09-04T17:47:26.201 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9500000000000007e-06 2024-09-04T17:47:26.202 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.202 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.202 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:26.202 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063027+0000", 2024-09-04T17:47:26.202 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5443000000000001e-05 2024-09-04T17:47:26.202 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.202 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.202 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:26.202 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210571+0000", 2024-09-04T17:47:26.202 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14754353100000001 2024-09-04T17:47:26.202 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.202 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.202 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:26.202 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210714+0000", 2024-09-04T17:47:26.203 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014295599999999999 2024-09-04T17:47:26.203 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.203 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.203 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:26.203 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210916+0000", 2024-09-04T17:47:26.203 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020258200000000001 2024-09-04T17:47:26.203 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.203 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:26.203 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.203 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.203 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.203 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8846 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3002368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:26.203 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:47:26.204 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 73.100901526000001, 2024-09-04T17:47:26.204 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14828011599999999, 2024-09-04T17:47:26.204 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:26.204 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:26.204 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:26.204 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:26.204 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:26.204 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:26.204 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8846 2024-09-04T17:47:26.204 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.204 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:26.204 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.204 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:26.204 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:47:26.205 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.205 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.205 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.205 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:26.205 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:47:26.205 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.205 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.205 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.205 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:26.205 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063038+0000", 2024-09-04T17:47:26.205 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.23e-07 2024-09-04T17:47:26.205 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.206 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.206 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:26.206 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063051+0000", 2024-09-04T17:47:26.206 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2757e-05 2024-09-04T17:47:26.206 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.206 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.206 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:26.206 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063070+0000", 2024-09-04T17:47:26.206 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9663000000000001e-05 2024-09-04T17:47:26.206 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.206 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.206 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:26.206 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063079+0000", 2024-09-04T17:47:26.207 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.0929999999999992e-06 2024-09-04T17:47:26.207 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.207 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.207 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:26.207 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210989+0000", 2024-09-04T17:47:26.207 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14790984500000001 2024-09-04T17:47:26.207 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.207 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.207 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:26.207 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211109+0000", 2024-09-04T17:47:26.207 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000120056 2024-09-04T17:47:26.207 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.207 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.207 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:26.208 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211317+0000", 2024-09-04T17:47:26.208 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020797900000000001 2024-09-04T17:47:26.208 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.208 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:26.208 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.208 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.208 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.208 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8847 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3039232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:26.208 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:47:26.208 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 73.100827093999996, 2024-09-04T17:47:26.208 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14859408900000001, 2024-09-04T17:47:26.208 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:26.208 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:26.208 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:26.209 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:26.209 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:26.209 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:26.209 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8847 2024-09-04T17:47:26.209 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.209 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:26.209 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.209 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:26.209 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:47:26.209 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.209 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.209 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.209 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:26.210 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:47:26.210 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.210 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.210 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.210 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:26.210 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063113+0000", 2024-09-04T17:47:26.210 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.204e-06 2024-09-04T17:47:26.210 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.210 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.210 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:26.210 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063116+0000", 2024-09-04T17:47:26.210 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.3330000000000001e-06 2024-09-04T17:47:26.210 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.211 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.211 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:26.211 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063126+0000", 2024-09-04T17:47:26.211 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0166999999999999e-05 2024-09-04T17:47:26.211 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.211 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.211 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:26.211 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063134+0000", 2024-09-04T17:47:26.211 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4329999999999998e-06 2024-09-04T17:47:26.211 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.211 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.211 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:26.211 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211392+0000", 2024-09-04T17:47:26.211 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148258109 2024-09-04T17:47:26.211 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.212 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.212 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:26.212 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211506+0000", 2024-09-04T17:47:26.212 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000113813 2024-09-04T17:47:26.212 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.212 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.212 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:26.212 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211706+0000", 2024-09-04T17:47:26.212 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020002999999999999 2024-09-04T17:47:26.212 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.213 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:26.213 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.213 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.213 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.213 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8848 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:26.213 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:47:26.213 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 73.100794757000003, 2024-09-04T17:47:26.213 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14907967599999999, 2024-09-04T17:47:26.213 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:26.213 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:26.213 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:26.213 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:26.213 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:26.214 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:26.214 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8848 2024-09-04T17:47:26.214 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.214 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:26.214 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.214 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:26.214 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:47:26.214 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.214 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.214 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.214 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:26.214 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:47:26.214 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.214 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.215 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.215 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:26.215 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070589+0000", 2024-09-04T17:47:26.215 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.007445386 2024-09-04T17:47:26.215 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.215 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.215 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:26.215 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070600+0000", 2024-09-04T17:47:26.215 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0346999999999999e-05 2024-09-04T17:47:26.215 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.215 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.215 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:26.215 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070622+0000", 2024-09-04T17:47:26.215 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2289999999999998e-05 2024-09-04T17:47:26.216 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.216 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.216 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:26.216 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070636+0000", 2024-09-04T17:47:26.216 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4093e-05 2024-09-04T17:47:26.216 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.216 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.216 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:26.216 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211815+0000", 2024-09-04T17:47:26.216 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14117874699999999 2024-09-04T17:47:26.216 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.216 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.216 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:26.217 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211983+0000", 2024-09-04T17:47:26.217 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00016852800000000001 2024-09-04T17:47:26.217 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.217 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.217 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:26.217 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.212224+0000", 2024-09-04T17:47:26.217 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024028499999999999 2024-09-04T17:47:26.217 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.218 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:26.218 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.218 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.218 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.218 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8862 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2019328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:26.218 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:47:26.218 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 73.092745289000007, 2024-09-04T17:47:26.218 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14755236399999999, 2024-09-04T17:47:26.218 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:26.218 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:26.219 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:26.219 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:26.219 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:26.219 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:26.219 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8862 2024-09-04T17:47:26.219 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.219 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:26.219 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.219 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:26.219 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:47:26.219 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.219 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.219 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.219 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:26.220 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:47:26.220 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.220 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.220 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.220 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:26.220 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071194+0000", 2024-09-04T17:47:26.220 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.5000000000000002e-07 2024-09-04T17:47:26.220 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.220 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.220 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:26.220 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071197+0000", 2024-09-04T17:47:26.220 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5900000000000002e-06 2024-09-04T17:47:26.220 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.220 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.221 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:26.221 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071206+0000", 2024-09-04T17:47:26.221 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9700000000000005e-06 2024-09-04T17:47:26.221 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.221 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.221 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:26.221 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071213+0000", 2024-09-04T17:47:26.221 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.9070000000000001e-06 2024-09-04T17:47:26.221 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.221 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.221 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:26.221 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218396+0000", 2024-09-04T17:47:26.221 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147183273 2024-09-04T17:47:26.221 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.222 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.222 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:26.222 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218538+0000", 2024-09-04T17:47:26.222 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014194199999999999 2024-09-04T17:47:26.222 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.222 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.222 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:26.222 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218746+0000", 2024-09-04T17:47:26.222 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000208032 2024-09-04T17:47:26.222 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.222 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:26.222 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.222 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.223 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.223 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8863 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2056192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:26.223 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:47:26.223 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 73.092716979000002, 2024-09-04T17:47:26.223 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14792281099999999, 2024-09-04T17:47:26.223 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:26.223 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:26.223 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:26.223 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:26.223 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:26.223 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:26.223 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8863 2024-09-04T17:47:26.223 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.223 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:26.223 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.224 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:26.224 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:47:26.224 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.224 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.224 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.224 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:26.224 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:47:26.224 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.224 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.224 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.224 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:26.224 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:47:26.225 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1299999999999999e-07 2024-09-04T17:47:26.225 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.225 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.225 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:26.225 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071225+0000", 2024-09-04T17:47:26.225 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6199999999999999e-06 2024-09-04T17:47:26.225 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.225 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.225 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:26.226 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071234+0000", 2024-09-04T17:47:26.226 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8769999999999999e-06 2024-09-04T17:47:26.226 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.226 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.226 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:26.226 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071251+0000", 2024-09-04T17:47:26.226 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.738e-05 2024-09-04T17:47:26.226 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.226 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.226 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:26.226 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218798+0000", 2024-09-04T17:47:26.226 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14754645 2024-09-04T17:47:26.227 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.227 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.227 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:26.227 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218926+0000", 2024-09-04T17:47:26.227 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000128446 2024-09-04T17:47:26.227 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.227 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.227 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:26.227 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219145+0000", 2024-09-04T17:47:26.227 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021832500000000001 2024-09-04T17:47:26.227 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.227 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:26.227 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.227 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.228 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.228 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8864 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2093056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:26.228 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:47:26.228 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 73.092675403000001, 2024-09-04T17:47:26.228 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148277309, 2024-09-04T17:47:26.228 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:26.228 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:26.228 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:26.228 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:26.228 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:26.228 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:26.228 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8864 2024-09-04T17:47:26.229 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.229 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:26.229 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.229 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:26.229 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:47:26.229 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.229 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.229 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.229 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:26.229 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:47:26.229 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.229 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.229 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.229 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:26.230 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071264+0000", 2024-09-04T17:47:26.230 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3e-07 2024-09-04T17:47:26.230 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.230 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.230 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:26.230 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071267+0000", 2024-09-04T17:47:26.230 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0139999999999999e-06 2024-09-04T17:47:26.230 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.230 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.230 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:26.230 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071276+0000", 2024-09-04T17:47:26.230 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1689999999999998e-06 2024-09-04T17:47:26.230 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.231 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.231 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:26.231 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071283+0000", 2024-09-04T17:47:26.231 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2269999999999996e-06 2024-09-04T17:47:26.231 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.231 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.231 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:26.231 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219195+0000", 2024-09-04T17:47:26.231 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147911868 2024-09-04T17:47:26.231 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.231 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.231 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:26.231 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219324+0000", 2024-09-04T17:47:26.232 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00012835900000000001 2024-09-04T17:47:26.232 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.232 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.232 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:26.232 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219541+0000", 2024-09-04T17:47:26.232 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000216942 2024-09-04T17:47:26.232 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.232 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:26.232 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.232 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.232 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.232 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8865 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2109440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:26.232 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:47:26.232 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 73.092645942999994, 2024-09-04T17:47:26.233 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14865747300000001, 2024-09-04T17:47:26.233 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:26.233 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:26.233 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:26.233 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:26.233 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:26.233 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:26.233 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8865 2024-09-04T17:47:26.233 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.233 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:26.233 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.233 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:26.234 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:47:26.234 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.234 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.234 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.234 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:26.234 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:47:26.234 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.234 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.234 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.234 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:26.234 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:47:26.234 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7999999999999995e-07 2024-09-04T17:47:26.234 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.234 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.235 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:26.235 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071296+0000", 2024-09-04T17:47:26.235 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6900000000000001e-06 2024-09-04T17:47:26.235 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.235 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.235 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:26.235 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071305+0000", 2024-09-04T17:47:26.235 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.5499999999999995e-06 2024-09-04T17:47:26.235 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.235 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.235 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:26.235 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071312+0000", 2024-09-04T17:47:26.235 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1559999999999998e-06 2024-09-04T17:47:26.236 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.236 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.236 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:26.236 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219591+0000", 2024-09-04T17:47:26.236 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14827944900000001 2024-09-04T17:47:26.236 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.236 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.236 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:26.236 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219731+0000", 2024-09-04T17:47:26.236 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013931600000000001 2024-09-04T17:47:26.236 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.236 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.236 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:26.236 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219950+0000", 2024-09-04T17:47:26.237 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021963199999999999 2024-09-04T17:47:26.237 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.237 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:26.237 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.237 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.237 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.237 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8866 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2146304~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:26.237 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:47:26.237 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 73.092617709999999, 2024-09-04T17:47:26.237 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14906117899999999, 2024-09-04T17:47:26.237 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:26.237 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:26.237 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:26.238 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:26.238 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:26.238 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:26.238 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8866 2024-09-04T17:47:26.238 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.238 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:26.238 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.238 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:26.238 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:47:26.238 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.238 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.238 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.238 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:26.239 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:47:26.239 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.239 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.239 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.239 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:26.239 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071322+0000", 2024-09-04T17:47:26.239 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7999999999999995e-07 2024-09-04T17:47:26.239 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.239 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.239 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:26.239 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071338+0000", 2024-09-04T17:47:26.239 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5872999999999999e-05 2024-09-04T17:47:26.239 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.240 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.240 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:26.240 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071347+0000", 2024-09-04T17:47:26.240 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4669999999999993e-06 2024-09-04T17:47:26.240 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.240 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.240 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:26.240 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071354+0000", 2024-09-04T17:47:26.240 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2099999999999996e-06 2024-09-04T17:47:26.240 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.240 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.240 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:26.240 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220008+0000", 2024-09-04T17:47:26.241 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148653642 2024-09-04T17:47:26.241 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.241 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.241 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:26.241 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220155+0000", 2024-09-04T17:47:26.241 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014726600000000001 2024-09-04T17:47:26.241 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.241 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.241 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:26.241 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220382+0000", 2024-09-04T17:47:26.241 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00022704100000000001 2024-09-04T17:47:26.241 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.242 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:26.242 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.242 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.242 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.242 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8867 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2183168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:26.242 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:47:26.242 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 73.092574357000004, 2024-09-04T17:47:26.242 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.149445831, 2024-09-04T17:47:26.242 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:26.242 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:26.242 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:26.242 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:26.242 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:26.242 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:26.243 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8867 2024-09-04T17:47:26.243 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.243 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:26.243 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.243 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:26.243 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:47:26.243 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.243 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.243 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.243 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:26.243 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:47:26.243 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.243 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.244 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.244 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:26.244 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078653+0000", 2024-09-04T17:47:26.244 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0072890430000000003 2024-09-04T17:47:26.244 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.244 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.244 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:26.244 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078664+0000", 2024-09-04T17:47:26.244 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0307e-05 2024-09-04T17:47:26.244 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.244 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.244 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:26.245 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078690+0000", 2024-09-04T17:47:26.245 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6780000000000001e-05 2024-09-04T17:47:26.245 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.245 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.245 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:26.245 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078706+0000", 2024-09-04T17:47:26.245 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5676999999999999e-05 2024-09-04T17:47:26.245 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.245 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.245 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:26.245 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220467+0000", 2024-09-04T17:47:26.245 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14176103500000001 2024-09-04T17:47:26.245 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.246 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.246 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:26.246 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220595+0000", 2024-09-04T17:47:26.246 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000127427 2024-09-04T17:47:26.246 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.246 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.246 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:26.246 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220810+0000", 2024-09-04T17:47:26.246 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021556200000000001 2024-09-04T17:47:26.246 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.246 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:26.247 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.247 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.247 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.247 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8880 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2658304~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:26.247 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.079158+0000", 2024-09-04T17:47:26.247 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 73.084781078000006, 2024-09-04T17:47:26.247 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14718968499999999, 2024-09-04T17:47:26.247 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:26.247 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:26.247 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:26.247 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:26.247 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:26.247 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:26.247 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8880 2024-09-04T17:47:26.248 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.248 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:26.248 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.248 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:26.248 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079158+0000", 2024-09-04T17:47:26.249 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.249 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.249 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.249 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:26.249 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079158+0000", 2024-09-04T17:47:26.249 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.249 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.249 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.249 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:26.249 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079158+0000", 2024-09-04T17:47:26.249 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3e-07 2024-09-04T17:47:26.249 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.250 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.250 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:26.250 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079161+0000", 2024-09-04T17:47:26.250 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.57e-06 2024-09-04T17:47:26.250 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.250 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.250 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:26.250 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079170+0000", 2024-09-04T17:47:26.250 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.0170000000000002e-06 2024-09-04T17:47:26.250 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.250 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.250 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:26.250 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079177+0000", 2024-09-04T17:47:26.251 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.0530000000000001e-06 2024-09-04T17:47:26.251 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.251 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.251 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:26.251 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.225946+0000", 2024-09-04T17:47:26.251 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14676936500000001 2024-09-04T17:47:26.251 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.251 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.251 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:26.251 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226098+0000", 2024-09-04T17:47:26.251 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000151805 2024-09-04T17:47:26.251 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.251 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.252 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:26.252 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226347+0000", 2024-09-04T17:47:26.252 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024914499999999999 2024-09-04T17:47:26.252 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.252 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:26.252 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.252 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.252 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.252 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8881 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2695168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:26.252 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:47:26.252 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 73.084752288000004, 2024-09-04T17:47:26.252 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147543547, 2024-09-04T17:47:26.252 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:26.253 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:26.253 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:26.253 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:26.253 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:26.253 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:26.253 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8881 2024-09-04T17:47:26.253 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.253 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:26.253 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.253 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:26.253 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:47:26.253 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.253 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.253 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.253 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:26.254 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:47:26.254 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.254 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.254 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.254 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:26.254 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079187+0000", 2024-09-04T17:47:26.254 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7700000000000004e-07 2024-09-04T17:47:26.254 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.254 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.254 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:26.254 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079198+0000", 2024-09-04T17:47:26.254 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0849999999999999e-05 2024-09-04T17:47:26.254 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.254 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.254 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:26.254 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079207+0000", 2024-09-04T17:47:26.255 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5599999999999999e-06 2024-09-04T17:47:26.255 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.255 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.255 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:26.255 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079224+0000", 2024-09-04T17:47:26.255 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6387000000000001e-05 2024-09-04T17:47:26.255 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.255 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.255 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:26.255 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226414+0000", 2024-09-04T17:47:26.255 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14718975000000001 2024-09-04T17:47:26.255 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.255 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.255 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:26.255 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226534+0000", 2024-09-04T17:47:26.256 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000120288 2024-09-04T17:47:26.256 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.256 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.256 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:26.256 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226730+0000", 2024-09-04T17:47:26.256 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000196035 2024-09-04T17:47:26.256 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.256 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:26.256 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.256 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.256 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.256 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8882 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2732032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:26.256 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:47:26.256 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 73.084704074000001, 2024-09-04T17:47:26.257 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14795022399999999, 2024-09-04T17:47:26.257 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:26.257 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:26.257 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:26.257 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:26.257 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:26.257 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:26.257 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8882 2024-09-04T17:47:26.257 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.257 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:26.257 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.257 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:26.257 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:47:26.257 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.257 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.258 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.258 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:26.258 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:47:26.258 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.258 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.258 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.258 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:26.258 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086588+0000", 2024-09-04T17:47:26.258 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0073530959999999999 2024-09-04T17:47:26.258 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.258 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.258 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:26.258 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086598+0000", 2024-09-04T17:47:26.258 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0122999999999999e-05 2024-09-04T17:47:26.259 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.259 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.259 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:26.259 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086626+0000", 2024-09-04T17:47:26.259 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.7914000000000001e-05 2024-09-04T17:47:26.259 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.259 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.259 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:26.259 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086641+0000", 2024-09-04T17:47:26.259 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5270000000000001e-05 2024-09-04T17:47:26.259 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.259 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.259 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:26.259 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226801+0000", 2024-09-04T17:47:26.259 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.140160321 2024-09-04T17:47:26.259 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.260 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.260 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:26.260 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226951+0000", 2024-09-04T17:47:26.260 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014965200000000001 2024-09-04T17:47:26.260 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.260 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.260 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:26.260 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.227185+0000", 2024-09-04T17:47:26.260 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00023384799999999999 2024-09-04T17:47:26.260 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.260 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:26.260 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.260 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.260 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:26.261 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:26.263 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-04T17:47:26.263 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:26.260+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:26.264 INFO:teuthology.orchestra.run.smithi092.stdout:4950 write 0x8f20ae3 thru 0x8f24104 (0x3622 bytes) 2024-09-04T17:47:26.264 INFO:teuthology.orchestra.run.smithi092.stdout:4951 writesame 0x7e1690f thru 0x7e1a3d6 data_size 0x48(0x3ac8 bytes) 2024-09-04T17:47:26.264 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:26.264 INFO:teuthology.orchestra.run.smithi092.stdout:4954 writesame 0x6855893 thru 0x6856251 data_size 0x5(0x9bf bytes) 2024-09-04T17:47:26.264 INFO:teuthology.orchestra.run.smithi092.stdout:4956 punch from 0x8888f9e to 0x88909a3, (0x7a05 bytes) 2024-09-04T17:47:26.264 INFO:teuthology.orchestra.run.smithi092.stdout:4957 punch from 0x60998cb to 0x609d475, (0x3baa bytes) 2024-09-04T17:47:26.264 INFO:teuthology.orchestra.run.smithi092.stdout:4958 read 0x4e18b35 thru 0x4e24815 (0xbce1 bytes) 2024-09-04T17:47:26.264 INFO:teuthology.orchestra.run.smithi092.stdout:4960 read 0x802bafe thru 0x80342d6 (0x87d9 bytes) 2024-09-04T17:47:26.264 INFO:teuthology.orchestra.run.smithi092.stdout:4961 write 0x87a6122 thru 0x87aa6e5 (0x45c4 bytes) 2024-09-04T17:47:26.264 INFO:teuthology.orchestra.run.smithi092.stdout:4962 punch from 0x5319ebb to 0x53290aa, (0xf1ef bytes) 2024-09-04T17:47:26.265 INFO:teuthology.orchestra.run.smithi092.stdout:4963 read 0x8b4aace thru 0x8b52979 (0x7eac bytes) 2024-09-04T17:47:26.265 INFO:teuthology.orchestra.run.smithi092.stdout:4964 read 0x258125c thru 0x2581ec7 (0xc6c bytes) 2024-09-04T17:47:26.265 INFO:teuthology.orchestra.run.smithi092.stdout:4965 read 0x3d37660 thru 0x3d3a94d (0x32ee bytes) 2024-09-04T17:47:26.265 INFO:teuthology.orchestra.run.smithi092.stdout:4967 trunc from 0x8f24105 to 0xb081371 2024-09-04T17:47:26.277 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:26.278 INFO:teuthology.orchestra.run.smithi092.stdout:4969 read 0x77ccb2e thru 0x77d41e2 (0x76b5 bytes) 2024-09-04T17:47:26.278 INFO:teuthology.orchestra.run.smithi092.stdout:4972 write 0x8299ca8 thru 0x82a6039 (0xc392 bytes) 2024-09-04T17:47:26.285 INFO:teuthology.orchestra.run.smithi092.stdout:4973 write 0x2af146f thru 0x2afb4a6 (0xa038 bytes) 2024-09-04T17:47:26.294 INFO:teuthology.orchestra.run.smithi092.stdout:4974 read 0x15ba527 thru 0x15c4764 (0xa23e bytes) 2024-09-04T17:47:26.295 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:26.301 INFO:teuthology.orchestra.run.smithi092.stdout:4977 trunc from 0xb081371 to 0xd1ab795 2024-09-04T17:47:26.315 INFO:teuthology.orchestra.run.smithi092.stdout:4978 punch from 0x4cde64a to 0x4ce587b, (0x7231 bytes) 2024-09-04T17:47:26.315 INFO:teuthology.orchestra.run.smithi092.stdout:4980 punch from 0xbf6289 to 0xc059b9, (0xf730 bytes) 2024-09-04T17:47:26.315 INFO:teuthology.orchestra.run.smithi092.stdout:4981 writesame 0x33185db thru 0x3323e90 data_size 0x8e(0xb8b6 bytes) 2024-09-04T17:47:26.322 INFO:teuthology.orchestra.run.smithi092.stdout:4983 read 0x40f2631 thru 0x40f5468 (0x2e38 bytes) 2024-09-04T17:47:26.322 INFO:teuthology.orchestra.run.smithi092.stdout:4984 read 0x2e709f6 thru 0x2e7744a (0x6a55 bytes) 2024-09-04T17:47:26.322 INFO:teuthology.orchestra.run.smithi092.stdout:4986 write 0x418f1ba thru 0x4192f5d (0x3da4 bytes) 2024-09-04T17:47:26.329 INFO:teuthology.orchestra.run.smithi092.stdout:4987 read 0x469c5f5 thru 0x46a4f80 (0x898c bytes) 2024-09-04T17:47:26.329 INFO:teuthology.orchestra.run.smithi092.stdout:4988 writesame 0x345341 thru 0x35033c data_size 0xe(0xaffc bytes) 2024-09-04T17:47:26.363 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:47:26.360+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:47:26.374 INFO:teuthology.orchestra.run.smithi092.stdout:4989 trunc from 0xd1ab795 to 0x1e5dc45 2024-09-04T17:47:26.408 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:26.442 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:26.442 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:47:26.442 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.442 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:70115 3.14 3:2c8ccdd1:::rbd_object_map.11341aa245a3:head [call lock.assert_locked in=44b,call rbd.object_map_resize in=29b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-04T17:47:26.442 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:26.441258+0000", 2024-09-04T17:47:26.443 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.0020112519999999998, 2024-09-04T17:47:26.443 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0021028599999999998, 2024-09-04T17:47:26.443 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:26.443 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:26.443 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "waiting for sub ops", 2024-09-04T17:47:26.443 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:26.443 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:26.443 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:26.443 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 70115 2024-09-04T17:47:26.443 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.443 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:26.443 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.443 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:26.443 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:26.441258+0000", 2024-09-04T17:47:26.444 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.444 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.444 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.444 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:26.444 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:26.441258+0000", 2024-09-04T17:47:26.444 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.444 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.444 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.444 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:26.444 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:26.441260+0000", 2024-09-04T17:47:26.444 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.114e-06 2024-09-04T17:47:26.444 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.445 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.445 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:26.445 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:26.441269+0000", 2024-09-04T17:47:26.445 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7749999999999998e-06 2024-09-04T17:47:26.445 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.445 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.445 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:26.445 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:26.441289+0000", 2024-09-04T17:47:26.445 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0234e-05 2024-09-04T17:47:26.445 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.445 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.445 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:26.446 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:26.441300+0000", 2024-09-04T17:47:26.446 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1007e-05 2024-09-04T17:47:26.446 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.446 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.446 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:26.446 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:26.441446+0000", 2024-09-04T17:47:26.446 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000145835 2024-09-04T17:47:26.446 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.446 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.446 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:26.446 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:26.441578+0000", 2024-09-04T17:47:26.447 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013247199999999999 2024-09-04T17:47:26.447 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.447 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.447 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 4", 2024-09-04T17:47:26.447 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:26.441984+0000", 2024-09-04T17:47:26.447 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000406303 2024-09-04T17:47:26.447 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.447 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:26.447 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.447 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.447 INFO:teuthology.orchestra.run.smithi190.stdout: ], 2024-09-04T17:47:26.448 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 1 2024-09-04T17:47:26.448 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:26.451 INFO:teuthology.orchestra.run.smithi092.stdout:4992 write 0xdeccf43 thru 0xded7a56 (0xab14 bytes) 2024-09-04T17:47:26.458 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-04T17:47:26.460 INFO:teuthology.orchestra.run.smithi092.stdout:4993 writesame 0x5f2ab97 thru 0x5f2bd53 data_size 0x1(0x11bd bytes) 2024-09-04T17:47:26.468 INFO:teuthology.orchestra.run.smithi092.stdout:4995 read 0x592aa3f thru 0x59390ec (0xe6ae bytes) 2024-09-04T17:47:26.468 INFO:teuthology.orchestra.run.smithi092.stdout:4996 read 0x5a8f60b thru 0x5a9ae87 (0xb87d bytes) 2024-09-04T17:47:26.468 INFO:teuthology.orchestra.run.smithi092.stdout:4997 read 0x6a03bd3 thru 0x6a11655 (0xda83 bytes) 2024-09-04T17:47:26.468 INFO:teuthology.orchestra.run.smithi092.stdout:4998 punch from 0xaab4ac4 to 0xaac333e, (0xe87a bytes) 2024-09-04T17:47:26.501 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:26.500+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:26.504 INFO:teuthology.orchestra.run.smithi092.stdout:4999 trunc from 0xded7a57 to 0x18bf32 2024-09-04T17:47:26.553 INFO:teuthology.orchestra.run.smithi092.stdout:5000 read 0x65cc2 thru 0x70207 (0xa546 bytes) 2024-09-04T17:47:26.554 INFO:teuthology.orchestra.run.smithi092.stdout:5001 trunc from 0x18bf32 to 0x36523f 2024-09-04T17:47:26.565 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:26.565+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:26.616 INFO:teuthology.orchestra.run.smithi092.stdout:5002 writesame 0xe55d7bc thru 0xe55e05f data_size 0xe(0x8a4 bytes) 2024-09-04T17:47:26.625 INFO:teuthology.orchestra.run.smithi092.stdout:5004 trunc from 0xe55e060 to 0xb2f2c75 2024-09-04T17:47:26.631 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:26.631 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:47:26.631 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:47:26.631 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:47:26.631 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:26.646 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-04T17:47:26.651 INFO:teuthology.orchestra.run.smithi092.stdout:5005 write 0xcad3b95 thru 0xcad7647 (0x3ab3 bytes) 2024-09-04T17:47:26.657 INFO:teuthology.orchestra.run.smithi092.stdout:5007 read 0xd4233 thru 0xd9aaf (0x587d bytes) 2024-09-04T17:47:26.666 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:26.665+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:26.687 INFO:teuthology.orchestra.run.smithi092.stdout:5009 trunc from 0xcad7648 to 0x1371328 2024-09-04T17:47:26.702 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:26.703 INFO:teuthology.orchestra.run.smithi092.stdout:5011 read 0xbf3048 thru 0xbffc3e (0xcbf7 bytes) 2024-09-04T17:47:26.725 INFO:teuthology.orchestra.run.smithi092.stdout:5012 trunc from 0x1371328 to 0xac936a8 2024-09-04T17:47:26.745 INFO:teuthology.orchestra.run.smithi092.stdout:5013 trunc from 0xac936a8 to 0x65e2962 2024-09-04T17:47:26.755 INFO:teuthology.orchestra.run.smithi092.stdout:5014 writesame 0x64d2acc thru 0x64d7ecb data_size 0x70(0x5400 bytes) 2024-09-04T17:47:26.767 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:26.764+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:26.777 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:26.777 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:47:26.778 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:47:26.778 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:47:26.778 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.778 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60921 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 262144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:26.778 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822920+0000", 2024-09-04T17:47:26.778 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 12.954530885, 2024-09-04T17:47:26.778 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073904394999999998, 2024-09-04T17:47:26.778 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:26.778 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:26.778 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:26.778 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:26.778 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:26.779 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:26.779 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60921 2024-09-04T17:47:26.779 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.779 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:26.779 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.779 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:26.779 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822920+0000", 2024-09-04T17:47:26.779 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.779 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.779 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.779 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:26.779 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822920+0000", 2024-09-04T17:47:26.779 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.780 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.780 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.780 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:26.780 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822920+0000", 2024-09-04T17:47:26.780 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4999999999999999e-07 2024-09-04T17:47:26.780 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.780 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.780 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:26.780 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822921+0000", 2024-09-04T17:47:26.780 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.113e-06 2024-09-04T17:47:26.780 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.780 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.780 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:26.781 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822924+0000", 2024-09-04T17:47:26.781 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.354e-06 2024-09-04T17:47:26.781 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.781 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.781 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:26.781 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822927+0000", 2024-09-04T17:47:26.781 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6199999999999999e-06 2024-09-04T17:47:26.781 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.781 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.781 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:26.781 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896586+0000", 2024-09-04T17:47:26.781 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073658976000000001 2024-09-04T17:47:26.781 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.782 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.782 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:26.782 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896626+0000", 2024-09-04T17:47:26.782 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0484000000000001e-05 2024-09-04T17:47:26.782 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.782 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.782 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:26.782 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896824+0000", 2024-09-04T17:47:26.782 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000197598 2024-09-04T17:47:26.782 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.782 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:26.782 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.782 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.783 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.783 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60922 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:26.783 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822930+0000", 2024-09-04T17:47:26.783 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 12.954520309999999, 2024-09-04T17:47:26.783 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074178872000000007, 2024-09-04T17:47:26.783 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:26.783 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:26.783 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:26.783 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:26.783 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:26.783 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:26.783 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60922 2024-09-04T17:47:26.783 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.784 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:26.784 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.784 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:26.784 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822930+0000", 2024-09-04T17:47:26.784 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.784 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.784 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.784 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:26.784 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822930+0000", 2024-09-04T17:47:26.784 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.784 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.784 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.784 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:26.785 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822930+0000", 2024-09-04T17:47:26.785 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.36e-07 2024-09-04T17:47:26.785 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.785 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.785 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:26.785 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822932+0000", 2024-09-04T17:47:26.785 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.567e-06 2024-09-04T17:47:26.785 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.785 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.785 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:26.785 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822938+0000", 2024-09-04T17:47:26.785 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4840000000000003e-06 2024-09-04T17:47:26.785 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.786 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.786 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:26.786 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822942+0000", 2024-09-04T17:47:26.786 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2039999999999999e-06 2024-09-04T17:47:26.786 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.786 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.786 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:26.786 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896850+0000", 2024-09-04T17:47:26.786 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073907941000000005 2024-09-04T17:47:26.786 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.786 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.786 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:26.787 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896891+0000", 2024-09-04T17:47:26.787 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0964999999999998e-05 2024-09-04T17:47:26.787 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.787 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.787 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:26.787 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897109+0000", 2024-09-04T17:47:26.787 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021847500000000001 2024-09-04T17:47:26.787 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.787 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:26.787 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.787 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.787 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.787 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60923 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 475136~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:26.787 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822949+0000", 2024-09-04T17:47:26.788 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 12.95450202, 2024-09-04T17:47:26.788 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074426928000000003, 2024-09-04T17:47:26.788 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:26.788 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:26.788 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:26.788 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:26.788 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:26.788 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:26.788 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60923 2024-09-04T17:47:26.788 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.788 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:26.788 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.788 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:26.789 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822949+0000", 2024-09-04T17:47:26.789 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.789 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.789 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.789 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:26.789 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822949+0000", 2024-09-04T17:47:26.789 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.789 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.789 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.789 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:26.789 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822949+0000", 2024-09-04T17:47:26.789 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7399999999999999e-07 2024-09-04T17:47:26.789 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.789 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.790 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:26.790 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822951+0000", 2024-09-04T17:47:26.790 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1940000000000001e-06 2024-09-04T17:47:26.790 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.790 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.790 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:26.790 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822957+0000", 2024-09-04T17:47:26.790 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.0650000000000004e-06 2024-09-04T17:47:26.790 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.790 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.790 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:26.790 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822960+0000", 2024-09-04T17:47:26.790 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.8009999999999999e-06 2024-09-04T17:47:26.791 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.791 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.791 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:26.791 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897138+0000", 2024-09-04T17:47:26.791 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074177990999999999 2024-09-04T17:47:26.791 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.791 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.791 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:26.791 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897179+0000", 2024-09-04T17:47:26.791 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.1057999999999999e-05 2024-09-04T17:47:26.791 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.791 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.791 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:26.792 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897375+0000", 2024-09-04T17:47:26.792 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019644500000000001 2024-09-04T17:47:26.792 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.792 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:26.792 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.792 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.792 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.792 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60924 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:26.792 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822964+0000", 2024-09-04T17:47:26.792 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 12.954486835000001, 2024-09-04T17:47:26.792 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074531902999999997, 2024-09-04T17:47:26.792 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:26.792 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:26.793 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:26.793 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:26.793 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:26.793 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:26.793 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60924 2024-09-04T17:47:26.793 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.793 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:26.793 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.793 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:26.793 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822964+0000", 2024-09-04T17:47:26.793 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.793 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.793 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.794 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:26.794 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822964+0000", 2024-09-04T17:47:26.794 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.794 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.794 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.794 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:26.794 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822964+0000", 2024-09-04T17:47:26.794 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4999999999999999e-07 2024-09-04T17:47:26.794 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.794 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.794 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:26.794 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822966+0000", 2024-09-04T17:47:26.794 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.917e-06 2024-09-04T17:47:26.795 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.795 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.795 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:26.795 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822969+0000", 2024-09-04T17:47:26.795 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.315e-06 2024-09-04T17:47:26.795 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.795 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.795 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:26.795 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822972+0000", 2024-09-04T17:47:26.795 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.599e-06 2024-09-04T17:47:26.795 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.795 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.795 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:26.795 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897398+0000", 2024-09-04T17:47:26.796 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074426168000000001 2024-09-04T17:47:26.796 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.796 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.796 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:26.796 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897441+0000", 2024-09-04T17:47:26.796 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.3158000000000003e-05 2024-09-04T17:47:26.796 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.796 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.796 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:26.796 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897496+0000", 2024-09-04T17:47:26.796 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4496000000000001e-05 2024-09-04T17:47:26.796 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.797 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:26.797 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.797 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.797 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.797 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60925 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 720896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:26.797 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822975+0000", 2024-09-04T17:47:26.797 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 12.9544754, 2024-09-04T17:47:26.797 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074635272000000003, 2024-09-04T17:47:26.797 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:26.797 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:26.797 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:26.797 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:26.798 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:26.798 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:26.798 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60925 2024-09-04T17:47:26.798 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.798 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:26.798 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.798 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:26.798 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822975+0000", 2024-09-04T17:47:26.798 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.798 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.798 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.798 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:26.798 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822975+0000", 2024-09-04T17:47:26.798 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.799 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.799 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.799 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:26.799 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822975+0000", 2024-09-04T17:47:26.799 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.34e-07 2024-09-04T17:47:26.799 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.799 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.799 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:26.799 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822976+0000", 2024-09-04T17:47:26.799 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0359999999999999e-06 2024-09-04T17:47:26.799 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.799 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.799 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:26.799 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822980+0000", 2024-09-04T17:47:26.799 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.242e-06 2024-09-04T17:47:26.800 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.800 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.800 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:26.800 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822983+0000", 2024-09-04T17:47:26.800 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.8830000000000002e-06 2024-09-04T17:47:26.800 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.800 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.800 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:26.800 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897519+0000", 2024-09-04T17:47:26.800 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074536366000000007 2024-09-04T17:47:26.800 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.800 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.800 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:26.800 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897560+0000", 2024-09-04T17:47:26.801 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0677999999999999e-05 2024-09-04T17:47:26.801 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.801 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.801 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:26.801 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897610+0000", 2024-09-04T17:47:26.801 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0832999999999999e-05 2024-09-04T17:47:26.801 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.801 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:26.801 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.801 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.802 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.802 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60926 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 860160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:26.802 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822986+0000", 2024-09-04T17:47:26.802 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 12.954464802, 2024-09-04T17:47:26.802 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074736413000000002, 2024-09-04T17:47:26.802 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:26.802 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:26.802 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:26.802 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:26.802 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:26.802 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:26.802 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60926 2024-09-04T17:47:26.802 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.803 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:26.803 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.803 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:26.803 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822986+0000", 2024-09-04T17:47:26.803 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.803 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.803 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.803 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:26.803 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822986+0000", 2024-09-04T17:47:26.803 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.803 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.803 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.803 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:26.804 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822986+0000", 2024-09-04T17:47:26.804 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5600000000000002e-07 2024-09-04T17:47:26.804 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.804 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.804 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:26.804 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822987+0000", 2024-09-04T17:47:26.804 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.082e-06 2024-09-04T17:47:26.804 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.804 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.804 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:26.804 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822990+0000", 2024-09-04T17:47:26.804 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.2219999999999998e-06 2024-09-04T17:47:26.804 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.805 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.805 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:26.805 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822994+0000", 2024-09-04T17:47:26.805 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.19e-06 2024-09-04T17:47:26.805 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.805 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.805 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:26.805 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897632+0000", 2024-09-04T17:47:26.805 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074638205999999999 2024-09-04T17:47:26.806 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.806 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.806 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:26.806 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897672+0000", 2024-09-04T17:47:26.806 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.9873999999999998e-05 2024-09-04T17:47:26.806 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.806 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.806 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:26.806 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897722+0000", 2024-09-04T17:47:26.806 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0583e-05 2024-09-04T17:47:26.806 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.806 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:26.806 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.806 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.807 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.807 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60927 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 966656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:26.807 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823000+0000", 2024-09-04T17:47:26.807 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 12.954450619999999, 2024-09-04T17:47:26.807 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074834036000000007, 2024-09-04T17:47:26.807 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:26.807 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:26.807 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:26.807 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:26.808 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:26.808 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:26.808 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60927 2024-09-04T17:47:26.808 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.808 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:26.808 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.808 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:26.808 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823000+0000", 2024-09-04T17:47:26.808 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.808 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.808 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.808 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:26.808 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823000+0000", 2024-09-04T17:47:26.809 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.809 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.809 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.809 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:26.809 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823000+0000", 2024-09-04T17:47:26.809 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4099999999999999e-07 2024-09-04T17:47:26.809 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.809 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.809 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:26.809 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823003+0000", 2024-09-04T17:47:26.810 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0810000000000001e-06 2024-09-04T17:47:26.810 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.810 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.810 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:26.810 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823011+0000", 2024-09-04T17:47:26.810 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7090000000000008e-06 2024-09-04T17:47:26.810 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.810 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.810 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:26.810 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823017+0000", 2024-09-04T17:47:26.810 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.3890000000000004e-06 2024-09-04T17:47:26.811 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.811 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.811 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:26.811 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897745+0000", 2024-09-04T17:47:26.811 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074728252999999994 2024-09-04T17:47:26.811 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.811 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.811 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:26.811 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897785+0000", 2024-09-04T17:47:26.811 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.9830999999999999e-05 2024-09-04T17:47:26.811 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.811 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.811 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:26.812 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897834+0000", 2024-09-04T17:47:26.812 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9231999999999999e-05 2024-09-04T17:47:26.812 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.812 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:26.812 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.812 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.812 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.812 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60928 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:26.812 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823024+0000", 2024-09-04T17:47:26.812 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 12.954426689, 2024-09-04T17:47:26.812 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075067479000000006, 2024-09-04T17:47:26.812 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:26.812 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:26.813 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:26.813 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:26.813 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:26.813 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:26.813 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60928 2024-09-04T17:47:26.813 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.813 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:26.813 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.813 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:26.813 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823024+0000", 2024-09-04T17:47:26.813 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.813 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.813 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.813 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:26.814 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823024+0000", 2024-09-04T17:47:26.814 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.814 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.814 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.814 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:26.814 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823024+0000", 2024-09-04T17:47:26.814 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6499999999999999e-07 2024-09-04T17:47:26.814 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.814 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.814 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:26.814 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823027+0000", 2024-09-04T17:47:26.815 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.3099999999999999e-06 2024-09-04T17:47:26.815 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.815 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.815 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:26.815 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823033+0000", 2024-09-04T17:47:26.815 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7429999999999996e-06 2024-09-04T17:47:26.815 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.815 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.815 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:26.815 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823038+0000", 2024-09-04T17:47:26.815 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.561e-06 2024-09-04T17:47:26.815 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.815 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.816 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:26.816 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897855+0000", 2024-09-04T17:47:26.816 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074816740000000007 2024-09-04T17:47:26.816 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.816 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.816 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:26.816 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897894+0000", 2024-09-04T17:47:26.816 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.9085999999999998e-05 2024-09-04T17:47:26.816 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.816 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.816 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:26.816 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898091+0000", 2024-09-04T17:47:26.816 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000197574 2024-09-04T17:47:26.817 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.817 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:26.817 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.817 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.817 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.817 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60929 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1212416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:26.817 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823042+0000", 2024-09-04T17:47:26.817 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 12.954408247, 2024-09-04T17:47:26.817 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075301251, 2024-09-04T17:47:26.817 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:26.817 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:26.817 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:26.817 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:26.818 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:26.818 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:26.818 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60929 2024-09-04T17:47:26.818 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.818 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:26.818 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.818 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:26.818 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823042+0000", 2024-09-04T17:47:26.818 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.818 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.818 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.818 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:26.818 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823042+0000", 2024-09-04T17:47:26.818 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.819 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.819 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.819 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:26.819 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823043+0000", 2024-09-04T17:47:26.819 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.1100000000000002e-07 2024-09-04T17:47:26.819 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.819 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.819 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:26.819 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823045+0000", 2024-09-04T17:47:26.819 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.933e-06 2024-09-04T17:47:26.819 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.819 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.819 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:26.820 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823051+0000", 2024-09-04T17:47:26.820 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.9569999999999999e-06 2024-09-04T17:47:26.820 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.820 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.820 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:26.820 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823055+0000", 2024-09-04T17:47:26.820 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4479999999999996e-06 2024-09-04T17:47:26.820 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.820 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.821 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:26.821 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898117+0000", 2024-09-04T17:47:26.821 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075062344000000003 2024-09-04T17:47:26.821 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.821 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.821 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:26.821 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898157+0000", 2024-09-04T17:47:26.821 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0095999999999997e-05 2024-09-04T17:47:26.821 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.821 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.821 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:26.821 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898344+0000", 2024-09-04T17:47:26.821 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000186162 2024-09-04T17:47:26.822 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.822 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:26.822 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.822 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.822 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.822 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60930 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1351680~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:26.822 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823061+0000", 2024-09-04T17:47:26.822 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 12.954389367999999, 2024-09-04T17:47:26.822 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075545507999999997, 2024-09-04T17:47:26.822 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:26.822 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:26.822 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:26.822 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:26.822 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:26.823 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:26.823 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60930 2024-09-04T17:47:26.823 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.823 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:26.823 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.823 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:26.823 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823061+0000", 2024-09-04T17:47:26.823 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.823 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.823 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.823 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:26.823 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823061+0000", 2024-09-04T17:47:26.823 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.823 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.824 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.824 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:26.824 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823378+0000", 2024-09-04T17:47:26.824 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00031631299999999998 2024-09-04T17:47:26.824 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.824 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.824 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:26.824 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823381+0000", 2024-09-04T17:47:26.824 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4020000000000002e-06 2024-09-04T17:47:26.824 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.824 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.824 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:26.824 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823389+0000", 2024-09-04T17:47:26.825 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.1429999999999998e-06 2024-09-04T17:47:26.825 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.825 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.825 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:26.825 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823395+0000", 2024-09-04T17:47:26.825 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.0059999999999998e-06 2024-09-04T17:47:26.825 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.825 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.825 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:26.825 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898368+0000", 2024-09-04T17:47:26.825 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074973032999999994 2024-09-04T17:47:26.825 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.825 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.826 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:26.826 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898410+0000", 2024-09-04T17:47:26.826 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2243000000000002e-05 2024-09-04T17:47:26.826 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.826 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.826 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:26.826 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898607+0000", 2024-09-04T17:47:26.826 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019636799999999999 2024-09-04T17:47:26.826 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.826 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:26.826 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.826 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.826 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.827 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60931 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1458176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:26.827 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823407+0000", 2024-09-04T17:47:26.827 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 12.954043294, 2024-09-04T17:47:26.827 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075460334000000004, 2024-09-04T17:47:26.827 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:26.827 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:26.827 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:26.827 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:26.827 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:26.827 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:26.827 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60931 2024-09-04T17:47:26.827 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.827 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:26.827 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.828 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:26.828 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823407+0000", 2024-09-04T17:47:26.828 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.828 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.828 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.828 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:26.828 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823407+0000", 2024-09-04T17:47:26.828 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.828 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.828 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.828 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:26.828 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823408+0000", 2024-09-04T17:47:26.828 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9999999999999998e-07 2024-09-04T17:47:26.829 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.829 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.829 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:26.829 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823415+0000", 2024-09-04T17:47:26.829 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.5510000000000002e-06 2024-09-04T17:47:26.829 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.829 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.829 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:26.829 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823423+0000", 2024-09-04T17:47:26.829 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2760000000000003e-06 2024-09-04T17:47:26.829 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.829 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.830 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:26.830 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823426+0000", 2024-09-04T17:47:26.830 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.2839999999999998e-06 2024-09-04T17:47:26.830 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.830 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.830 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:26.830 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898631+0000", 2024-09-04T17:47:26.830 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075205045999999998 2024-09-04T17:47:26.830 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.830 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.830 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:26.830 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898672+0000", 2024-09-04T17:47:26.830 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0614000000000001e-05 2024-09-04T17:47:26.831 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.831 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.831 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:26.831 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898868+0000", 2024-09-04T17:47:26.831 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019606300000000001 2024-09-04T17:47:26.831 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.831 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:26.831 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.831 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.831 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.831 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60932 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1564672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:26.831 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823434+0000", 2024-09-04T17:47:26.831 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 12.954016995, 2024-09-04T17:47:26.832 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075693393999999997, 2024-09-04T17:47:26.832 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:26.832 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:26.832 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:26.832 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:26.832 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:26.832 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:26.832 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60932 2024-09-04T17:47:26.832 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.832 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:26.832 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.832 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:26.832 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823434+0000", 2024-09-04T17:47:26.833 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.833 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.833 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.833 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:26.833 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823434+0000", 2024-09-04T17:47:26.833 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.833 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.833 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.833 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:26.833 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.830290+0000", 2024-09-04T17:47:26.833 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0068563039999999997 2024-09-04T17:47:26.833 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.833 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.833 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:26.834 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.830297+0000", 2024-09-04T17:47:26.834 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4760000000000001e-06 2024-09-04T17:47:26.834 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.834 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.834 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:26.834 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.830308+0000", 2024-09-04T17:47:26.834 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0927999999999999e-05 2024-09-04T17:47:26.834 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.834 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.834 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:26.834 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.830317+0000", 2024-09-04T17:47:26.834 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7280000000000001e-06 2024-09-04T17:47:26.834 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.834 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.835 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:26.835 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898892+0000", 2024-09-04T17:47:26.835 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.068574722000000005 2024-09-04T17:47:26.835 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.835 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.835 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:26.835 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898934+0000", 2024-09-04T17:47:26.835 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2249000000000002e-05 2024-09-04T17:47:26.835 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.835 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.835 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:26.835 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.899127+0000", 2024-09-04T17:47:26.835 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019298699999999999 2024-09-04T17:47:26.836 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.836 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:26.836 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.836 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.836 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.836 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61182 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 868352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:26.836 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839249+0000", 2024-09-04T17:47:26.836 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 12.938201103000001, 2024-09-04T17:47:26.836 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073680151999999999, 2024-09-04T17:47:26.836 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:26.836 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:26.836 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:26.836 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:26.836 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:26.837 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:26.837 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61182 2024-09-04T17:47:26.837 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.837 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:26.837 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.837 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:26.837 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839249+0000", 2024-09-04T17:47:26.837 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.837 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.837 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.837 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:26.837 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839249+0000", 2024-09-04T17:47:26.837 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.838 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.838 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.838 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:26.838 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839250+0000", 2024-09-04T17:47:26.838 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4999999999999998e-07 2024-09-04T17:47:26.838 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.838 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.838 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:26.838 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839257+0000", 2024-09-04T17:47:26.838 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7700000000000004e-06 2024-09-04T17:47:26.838 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.838 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.838 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:26.839 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839263+0000", 2024-09-04T17:47:26.839 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.246e-06 2024-09-04T17:47:26.839 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.839 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.839 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:26.839 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839268+0000", 2024-09-04T17:47:26.839 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.814e-06 2024-09-04T17:47:26.839 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.839 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.839 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:26.839 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.912831+0000", 2024-09-04T17:47:26.839 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073563294000000001 2024-09-04T17:47:26.839 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.840 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.840 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:26.840 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.912876+0000", 2024-09-04T17:47:26.840 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4742e-05 2024-09-04T17:47:26.840 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.840 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.840 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:26.840 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.912930+0000", 2024-09-04T17:47:26.840 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.3936e-05 2024-09-04T17:47:26.840 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.840 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:26.840 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.840 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.841 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.841 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61183 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:26.841 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839275+0000", 2024-09-04T17:47:26.841 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 12.93817591, 2024-09-04T17:47:26.841 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073781886000000005, 2024-09-04T17:47:26.841 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:26.841 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:26.841 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:26.841 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:26.841 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:26.841 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:26.841 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61183 2024-09-04T17:47:26.841 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.842 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:26.842 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.842 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:26.842 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839275+0000", 2024-09-04T17:47:26.842 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.842 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.842 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.842 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:26.842 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839275+0000", 2024-09-04T17:47:26.842 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.842 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.842 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.842 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:26.843 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839275+0000", 2024-09-04T17:47:26.843 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6899999999999997e-07 2024-09-04T17:47:26.843 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.843 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.843 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:26.843 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839277+0000", 2024-09-04T17:47:26.843 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9530000000000002e-06 2024-09-04T17:47:26.843 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.843 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.843 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:26.843 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839283+0000", 2024-09-04T17:47:26.843 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.9329999999999998e-06 2024-09-04T17:47:26.844 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.844 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.844 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:26.844 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839288+0000", 2024-09-04T17:47:26.844 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4100000000000001e-06 2024-09-04T17:47:26.844 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.844 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.844 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:26.844 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.912959+0000", 2024-09-04T17:47:26.844 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073671511999999995 2024-09-04T17:47:26.844 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.844 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.844 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:26.844 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913001+0000", 2024-09-04T17:47:26.845 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.1984999999999999e-05 2024-09-04T17:47:26.845 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.845 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.845 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:26.845 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913057+0000", 2024-09-04T17:47:26.845 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.5523999999999999e-05 2024-09-04T17:47:26.845 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.845 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:26.845 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.845 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.845 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.845 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61184 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1114112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:26.845 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839294+0000", 2024-09-04T17:47:26.846 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 12.938156698, 2024-09-04T17:47:26.846 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073995263000000006, 2024-09-04T17:47:26.846 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:26.846 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:26.846 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:26.846 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:26.846 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:26.846 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:26.846 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61184 2024-09-04T17:47:26.846 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.846 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:26.846 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.846 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:26.846 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839294+0000", 2024-09-04T17:47:26.847 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.847 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.847 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.847 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:26.847 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839294+0000", 2024-09-04T17:47:26.847 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.847 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.847 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.847 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:26.847 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839294+0000", 2024-09-04T17:47:26.847 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.77e-07 2024-09-04T17:47:26.847 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.847 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.847 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:26.848 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839296+0000", 2024-09-04T17:47:26.848 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9999999999999999e-06 2024-09-04T17:47:26.848 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.848 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.848 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:26.848 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839302+0000", 2024-09-04T17:47:26.848 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7230000000000001e-06 2024-09-04T17:47:26.848 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.848 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.848 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:26.848 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839314+0000", 2024-09-04T17:47:26.848 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1826e-05 2024-09-04T17:47:26.848 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.849 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.849 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:26.849 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913084+0000", 2024-09-04T17:47:26.849 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073770215 2024-09-04T17:47:26.849 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.849 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.849 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:26.849 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913127+0000", 2024-09-04T17:47:26.849 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2982999999999998e-05 2024-09-04T17:47:26.849 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.849 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.849 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:26.850 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913289+0000", 2024-09-04T17:47:26.850 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000162139 2024-09-04T17:47:26.850 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.850 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:26.850 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.850 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.850 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.850 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61185 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1220608~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:26.850 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839321+0000", 2024-09-04T17:47:26.850 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 12.938129231, 2024-09-04T17:47:26.850 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074206062000000003, 2024-09-04T17:47:26.851 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:26.851 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:26.851 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:26.851 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:26.851 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:26.851 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:26.851 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61185 2024-09-04T17:47:26.851 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.851 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:26.851 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.852 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:26.852 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839321+0000", 2024-09-04T17:47:26.852 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.852 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.852 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.852 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:26.852 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839321+0000", 2024-09-04T17:47:26.852 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.852 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.852 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.852 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:26.852 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839322+0000", 2024-09-04T17:47:26.852 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.5400000000000001e-07 2024-09-04T17:47:26.853 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.853 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.853 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:26.853 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839324+0000", 2024-09-04T17:47:26.853 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2539999999999999e-06 2024-09-04T17:47:26.853 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.853 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.853 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:26.853 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839331+0000", 2024-09-04T17:47:26.853 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2139999999999999e-06 2024-09-04T17:47:26.853 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.853 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.853 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:26.854 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839336+0000", 2024-09-04T17:47:26.854 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7029999999999997e-06 2024-09-04T17:47:26.854 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.854 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.854 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:26.854 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913320+0000", 2024-09-04T17:47:26.854 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073984016 2024-09-04T17:47:26.854 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.854 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.854 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:26.854 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913366+0000", 2024-09-04T17:47:26.854 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.5704000000000002e-05 2024-09-04T17:47:26.854 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.855 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.855 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:26.855 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913527+0000", 2024-09-04T17:47:26.855 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00016161699999999999 2024-09-04T17:47:26.855 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.855 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:26.855 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.855 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.855 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.855 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61186 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1359872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:26.855 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839344+0000", 2024-09-04T17:47:26.855 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 12.938106889, 2024-09-04T17:47:26.855 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074450673999999994, 2024-09-04T17:47:26.856 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:26.856 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:26.856 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:26.856 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:26.856 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:26.856 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:26.856 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61186 2024-09-04T17:47:26.856 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.856 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:26.856 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.856 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:26.856 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839344+0000", 2024-09-04T17:47:26.856 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.856 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.857 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.857 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:26.857 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839344+0000", 2024-09-04T17:47:26.857 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.857 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.857 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.857 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:26.857 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839344+0000", 2024-09-04T17:47:26.857 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4799999999999999e-07 2024-09-04T17:47:26.857 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.857 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.857 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:26.857 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839346+0000", 2024-09-04T17:47:26.858 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0889999999999998e-06 2024-09-04T17:47:26.858 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.858 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.858 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:26.858 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839352+0000", 2024-09-04T17:47:26.858 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6389999999999997e-06 2024-09-04T17:47:26.858 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.858 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.858 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:26.858 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839356+0000", 2024-09-04T17:47:26.858 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.233e-06 2024-09-04T17:47:26.858 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.858 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.859 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:26.859 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913557+0000", 2024-09-04T17:47:26.859 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074200944000000005 2024-09-04T17:47:26.859 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.859 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.859 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:26.859 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913601+0000", 2024-09-04T17:47:26.859 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4350000000000001e-05 2024-09-04T17:47:26.859 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.859 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.859 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:26.859 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913794+0000", 2024-09-04T17:47:26.859 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019297100000000001 2024-09-04T17:47:26.860 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.860 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:26.860 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.860 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.860 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.860 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61187 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1466368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:26.860 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839364+0000", 2024-09-04T17:47:26.860 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 12.938087039999999, 2024-09-04T17:47:26.860 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074692334999999999, 2024-09-04T17:47:26.860 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:26.860 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:26.860 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:26.860 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:26.860 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:26.861 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:26.861 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61187 2024-09-04T17:47:26.861 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.861 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:26.861 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.861 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:26.861 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839364+0000", 2024-09-04T17:47:26.861 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.861 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.861 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.861 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:26.861 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839364+0000", 2024-09-04T17:47:26.861 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.862 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.862 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.862 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:26.862 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839364+0000", 2024-09-04T17:47:26.862 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.75e-07 2024-09-04T17:47:26.862 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.862 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.862 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:26.862 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839366+0000", 2024-09-04T17:47:26.862 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0990000000000001e-06 2024-09-04T17:47:26.862 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.862 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.862 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:26.863 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839372+0000", 2024-09-04T17:47:26.863 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.2609999999999999e-06 2024-09-04T17:47:26.863 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.863 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.863 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:26.863 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839389+0000", 2024-09-04T17:47:26.863 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6489000000000001e-05 2024-09-04T17:47:26.863 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.863 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.863 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:26.863 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913824+0000", 2024-09-04T17:47:26.863 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074434811000000004 2024-09-04T17:47:26.863 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.864 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.864 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:26.864 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913868+0000", 2024-09-04T17:47:26.864 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4768000000000002e-05 2024-09-04T17:47:26.864 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.864 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.864 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:26.864 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914056+0000", 2024-09-04T17:47:26.864 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00018743199999999999 2024-09-04T17:47:26.864 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.864 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:26.864 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.864 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.865 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.865 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61188 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1605632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:26.865 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839399+0000", 2024-09-04T17:47:26.865 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 12.938051143999999, 2024-09-04T17:47:26.865 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074975095000000005, 2024-09-04T17:47:26.865 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:26.865 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:26.865 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:26.865 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:26.865 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:26.865 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:26.865 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61188 2024-09-04T17:47:26.865 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.865 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:26.866 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.866 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:26.866 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839399+0000", 2024-09-04T17:47:26.866 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.866 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.866 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.866 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:26.866 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839399+0000", 2024-09-04T17:47:26.866 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.866 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.866 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.866 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:26.866 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839400+0000", 2024-09-04T17:47:26.867 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0100000000000005e-07 2024-09-04T17:47:26.867 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.867 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.867 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:26.867 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839402+0000", 2024-09-04T17:47:26.867 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.491e-06 2024-09-04T17:47:26.867 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.867 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.867 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:26.867 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839409+0000", 2024-09-04T17:47:26.867 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.0450000000000003e-06 2024-09-04T17:47:26.867 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.867 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.867 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:26.868 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839414+0000", 2024-09-04T17:47:26.868 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.87e-06 2024-09-04T17:47:26.868 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.868 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.868 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:26.868 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914086+0000", 2024-09-04T17:47:26.868 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074671596000000007 2024-09-04T17:47:26.868 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.868 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.868 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:26.868 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914130+0000", 2024-09-04T17:47:26.868 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4579999999999997e-05 2024-09-04T17:47:26.868 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.869 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.869 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:26.869 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914375+0000", 2024-09-04T17:47:26.869 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024401199999999999 2024-09-04T17:47:26.869 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.869 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:26.869 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.869 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.869 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.869 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61189 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1712128~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:26.869 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839424+0000", 2024-09-04T17:47:26.869 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 12.938026943000001, 2024-09-04T17:47:26.869 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075247680999999997, 2024-09-04T17:47:26.869 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:26.870 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:26.870 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:26.870 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:26.870 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:26.870 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:26.870 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61189 2024-09-04T17:47:26.870 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.870 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:26.870 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.870 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:26.870 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839424+0000", 2024-09-04T17:47:26.870 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.870 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.871 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.871 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:26.871 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839424+0000", 2024-09-04T17:47:26.871 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:26.871 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.871 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.871 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:26.871 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.846320+0000", 2024-09-04T17:47:26.871 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0068965210000000001 2024-09-04T17:47:26.871 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.871 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.871 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:26.871 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.846328+0000", 2024-09-04T17:47:26.871 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.8189999999999992e-06 2024-09-04T17:47:26.872 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.872 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.872 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:26.872 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.846341+0000", 2024-09-04T17:47:26.872 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3338e-05 2024-09-04T17:47:26.872 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.872 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.872 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:26.872 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.846352+0000", 2024-09-04T17:47:26.872 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0479000000000001e-05 2024-09-04T17:47:26.872 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.872 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.872 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:26.873 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914405+0000", 2024-09-04T17:47:26.873 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.068053049000000004 2024-09-04T17:47:26.873 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.873 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.873 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:26.873 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914451+0000", 2024-09-04T17:47:26.873 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6545999999999998e-05 2024-09-04T17:47:26.873 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:26.873 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:26.873 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:26.873 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914671+0000", 2024-09-04T17:47:26.873 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000219929 2024-09-04T17:47:26.873 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.873 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:26.874 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.874 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:26.874 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:26.874 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:26.876 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-04T17:47:26.876 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:26.872+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:26.877 INFO:teuthology.orchestra.run.smithi092.stdout:5015 write 0xbd406f7 thru 0xbd4947e (0x8d88 bytes) 2024-09-04T17:47:26.877 INFO:teuthology.orchestra.run.smithi092.stdout:5016 read 0x6a6fce2 thru 0x6a76e55 (0x7174 bytes) 2024-09-04T17:47:26.877 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:26.877 INFO:teuthology.orchestra.run.smithi092.stdout:5018 write 0x8e3d6ba thru 0x8e4c38b (0xecd2 bytes) 2024-09-04T17:47:26.877 INFO:teuthology.orchestra.run.smithi092.stdout:5019 read 0x5a86ef thru 0x5a9ec2 (0x17d4 bytes) 2024-09-04T17:47:26.878 INFO:teuthology.orchestra.run.smithi092.stdout:5020 trunc from 0xbd4947f to 0xc763935 2024-09-04T17:47:26.878 INFO:teuthology.orchestra.run.smithi092.stdout:5021 punch from 0x338f0eb to 0x338f598, (0x4ad bytes) 2024-09-04T17:47:26.878 INFO:teuthology.orchestra.run.smithi092.stdout:5022 write 0x5b521a0 thru 0x5b5d049 (0xaeaa bytes) 2024-09-04T17:47:26.878 INFO:teuthology.orchestra.run.smithi092.stdout:5023 trunc from 0xc763935 to 0x9c3c266 2024-09-04T17:47:26.878 INFO:teuthology.orchestra.run.smithi092.stdout:5024 read 0x888ea22 thru 0x889c504 (0xdae3 bytes) 2024-09-04T17:47:26.878 INFO:teuthology.orchestra.run.smithi092.stdout:5025 trunc from 0x9c3c266 to 0x336a06b 2024-09-04T17:47:26.878 INFO:teuthology.orchestra.run.smithi092.stdout:5026 read 0x61469c thru 0x617a07 (0x336c bytes) 2024-09-04T17:47:26.878 INFO:teuthology.orchestra.run.smithi092.stdout:5028 read 0x125a256 thru 0x1268109 (0xdeb4 bytes) 2024-09-04T17:47:26.878 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:26.887 INFO:teuthology.orchestra.run.smithi092.stdout:5031 writesame 0x6d3ff0a thru 0x6d48c51 data_size 0x58(0x8d48 bytes) 2024-09-04T17:47:26.887 INFO:teuthology.orchestra.run.smithi092.stdout:5032 read 0x3c51bd3 thru 0x3c53138 (0x1566 bytes) 2024-09-04T17:47:26.887 INFO:teuthology.orchestra.run.smithi092.stdout:5033 punch from 0x4c472bc to 0x4c5313e, (0xbe82 bytes) 2024-09-04T17:47:26.888 INFO:teuthology.orchestra.run.smithi092.stdout:5034 trunc from 0x6d48c52 to 0x72f1441 2024-09-04T17:47:26.900 INFO:teuthology.orchestra.run.smithi092.stdout:5036 write 0x3ce3dca thru 0x3cf36c9 (0xf900 bytes) 2024-09-04T17:47:26.908 INFO:teuthology.orchestra.run.smithi092.stdout:5037 write 0x600cc9b thru 0x601c932 (0xfc98 bytes) 2024-09-04T17:47:26.937 INFO:teuthology.orchestra.run.smithi092.stdout:5038 writesame 0xa2fa01c thru 0xa300820 data_size 0x1(0x6805 bytes) 2024-09-04T17:47:26.937 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:26.937 INFO:teuthology.orchestra.run.smithi092.stdout:5041 read 0xcef76c thru 0xcfe4a5 (0xed3a bytes) 2024-09-04T17:47:26.937 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:26.937 INFO:teuthology.orchestra.run.smithi092.stdout:5043 writesame 0x4a4aad1 thru 0x4a5a1cd data_size 0x1(0xf6fd bytes) 2024-09-04T17:47:26.976 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:47:26.976+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:47:26.997 INFO:teuthology.orchestra.run.smithi092.stdout:5044 read 0x6f80a14 thru 0x6f8f1e2 (0xe7cf bytes) 2024-09-04T17:47:26.997 INFO:teuthology.orchestra.run.smithi092.stdout:5046 write 0x6933a4c thru 0x693bccc (0x8281 bytes) 2024-09-04T17:47:27.005 INFO:teuthology.orchestra.run.smithi092.stdout:5047 punch from 0x797a33c to 0x797f1da, (0x4e9e bytes) 2024-09-04T17:47:27.005 INFO:teuthology.orchestra.run.smithi092.stdout:5048 read 0x758d5ba thru 0x7598fb0 (0xb9f7 bytes) 2024-09-04T17:47:27.005 INFO:teuthology.orchestra.run.smithi092.stdout:5049 write 0x2d1055 thru 0x2d293b (0x18e7 bytes) 2024-09-04T17:47:27.008 INFO:teuthology.orchestra.run.smithi092.stdout:5050 write 0x6e6e51b thru 0x6e72206 (0x3cec bytes) 2024-09-04T17:47:27.011 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:27.011 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:47:27.011 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:47:27.011 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:27.018 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:27.018 INFO:teuthology.orchestra.run.smithi092.stdout:5054 read 0xc449e7 thru 0xc51f68 (0xd582 bytes) 2024-09-04T17:47:27.018 INFO:teuthology.orchestra.run.smithi092.stdout:5055 writesame 0x208cc74 thru 0x2095076 data_size 0xf(0x8403 bytes) 2024-09-04T17:47:27.026 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-04T17:47:27.047 INFO:teuthology.orchestra.run.smithi092.stdout:5058 write 0xd321bff thru 0xd327c0e (0x6010 bytes) 2024-09-04T17:47:27.054 INFO:teuthology.orchestra.run.smithi092.stdout:5059 punch from 0x4daba84 to 0x4dacedf, (0x145b bytes) 2024-09-04T17:47:27.055 INFO:teuthology.orchestra.run.smithi092.stdout:5061 read 0xbe4a6e4 thru 0xbe4c8a1 (0x21be bytes) 2024-09-04T17:47:27.055 INFO:teuthology.orchestra.run.smithi092.stdout:5062 read 0xa17348b thru 0xa17dfa5 (0xab1b bytes) 2024-09-04T17:47:27.055 INFO:teuthology.orchestra.run.smithi092.stdout:5066 punch from 0x9c5c49 to 0x9ca283, (0x463a bytes) 2024-09-04T17:47:27.055 INFO:teuthology.orchestra.run.smithi092.stdout:5067 read 0x40b6e93 thru 0x40b7d58 (0xec6 bytes) 2024-09-04T17:47:27.055 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:27.055 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:27.055 INFO:teuthology.orchestra.run.smithi092.stdout:5071 read 0x7c407c5 thru 0x7c47272 (0x6aae bytes) 2024-09-04T17:47:27.056 INFO:teuthology.orchestra.run.smithi092.stdout:5072 clone 35 order 23 su 65536 sc 4 2024-09-04T17:47:27.077 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:27.077+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:27.172 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:27.172 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:47:27.172 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:47:27.172 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:47:27.172 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:27.178 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:27.177+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:27.185 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-04T17:47:27.278 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:27.277+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:27.351 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:27.351 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:47:27.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:47:27.351 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:47:27.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.351 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:27242 3.b 3.34414deb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-04T17:47:27.351 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:28.402347+0000", 2024-09-04T17:47:27.351 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 58.947792640999999, 2024-09-04T17:47:27.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.93265329299999999, 2024-09-04T17:47:27.351 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:27.351 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:27.352 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-04T17:47:27.352 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:27.352 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:27.352 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:27.352 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 27242 2024-09-04T17:47:27.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.352 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:27.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:27.352 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402347+0000", 2024-09-04T17:47:27.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:27.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:27.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402347+0000", 2024-09-04T17:47:27.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:27.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:27.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402349+0000", 2024-09-04T17:47:27.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.471e-06 2024-09-04T17:47:27.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:27.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402355+0000", 2024-09-04T17:47:27.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8399999999999997e-06 2024-09-04T17:47:27.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.354 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:27.354 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402364+0000", 2024-09-04T17:47:27.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1269999999999992e-06 2024-09-04T17:47:27.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.354 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:27.354 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402373+0000", 2024-09-04T17:47:27.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7909999999999993e-06 2024-09-04T17:47:27.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.354 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:27.354 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402460+0000", 2024-09-04T17:47:27.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6686000000000003e-05 2024-09-04T17:47:27.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.355 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.355 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-04T17:47:27.355 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402473+0000", 2024-09-04T17:47:27.355 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3281e-05 2024-09-04T17:47:27.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.355 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.355 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:27.355 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:29.334989+0000", 2024-09-04T17:47:27.355 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.93251549099999997 2024-09-04T17:47:27.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.355 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.355 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:27.355 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:29.335000+0000", 2024-09-04T17:47:27.355 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1606e-05 2024-09-04T17:47:27.356 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:27.356 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:27.356 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:27.356 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.356 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.356 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:33552 3.7 3:e79ad1db:::rbd_data.11342e8f3384.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-04T17:47:27.356 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:35.999748+0000", 2024-09-04T17:47:27.356 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.350391590999998, 2024-09-04T17:47:27.356 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045683823999999998, 2024-09-04T17:47:27.356 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:27.356 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:27.356 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:27.356 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:27.357 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:27.357 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:27.357 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 33552 2024-09-04T17:47:27.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.357 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:27.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.357 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:27.357 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999748+0000", 2024-09-04T17:47:27.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:27.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.357 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:27.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999748+0000", 2024-09-04T17:47:27.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:27.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:27.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999750+0000", 2024-09-04T17:47:27.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1299999999999999e-06 2024-09-04T17:47:27.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:27.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999759+0000", 2024-09-04T17:47:27.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4649999999999994e-06 2024-09-04T17:47:27.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:27.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999778+0000", 2024-09-04T17:47:27.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8833000000000001e-05 2024-09-04T17:47:27.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:27.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999786+0000", 2024-09-04T17:47:27.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7369999999999995e-06 2024-09-04T17:47:27.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:27.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999856+0000", 2024-09-04T17:47:27.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9247999999999998e-05 2024-09-04T17:47:27.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:27.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999973+0000", 2024-09-04T17:47:27.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000117921 2024-09-04T17:47:27.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 7", 2024-09-04T17:47:27.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.005955+0000", 2024-09-04T17:47:27.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059820560000000004 2024-09-04T17:47:27.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:27.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.045297+0000", 2024-09-04T17:47:27.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.039341650999999998 2024-09-04T17:47:27.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:27.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.045343+0000", 2024-09-04T17:47:27.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5899000000000002e-05 2024-09-04T17:47:27.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:27.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.045388+0000", 2024-09-04T17:47:27.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4601999999999997e-05 2024-09-04T17:47:27.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:27.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.045432+0000", 2024-09-04T17:47:27.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4282000000000001e-05 2024-09-04T17:47:27.362 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:27.362 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:27.362 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:27.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.362 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:37357 3.15 3:a897312a:::rbd_data.1134c5a43465.000000000000001f:head [stat,write 62464~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-04T17:47:27.362 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:39.616724+0000", 2024-09-04T17:47:27.362 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.733416183000003, 2024-09-04T17:47:27.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054965854000000001, 2024-09-04T17:47:27.362 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:27.362 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:27.362 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:27.362 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:27.363 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:27.363 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:27.363 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 37357 2024-09-04T17:47:27.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.363 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:27.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:27.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616724+0000", 2024-09-04T17:47:27.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:27.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:27.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616724+0000", 2024-09-04T17:47:27.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:27.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:27.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616725+0000", 2024-09-04T17:47:27.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.815e-06 2024-09-04T17:47:27.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:27.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616739+0000", 2024-09-04T17:47:27.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3723000000000001e-05 2024-09-04T17:47:27.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:27.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616755+0000", 2024-09-04T17:47:27.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5964000000000002e-05 2024-09-04T17:47:27.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:27.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616763+0000", 2024-09-04T17:47:27.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3869999999999995e-06 2024-09-04T17:47:27.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:27.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616825+0000", 2024-09-04T17:47:27.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1854999999999996e-05 2024-09-04T17:47:27.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:27.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616895+0000", 2024-09-04T17:47:27.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9597999999999993e-05 2024-09-04T17:47:27.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-04T17:47:27.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616991+0000", 2024-09-04T17:47:27.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5828000000000005e-05 2024-09-04T17:47:27.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:27.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.617659+0000", 2024-09-04T17:47:27.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00066821600000000001 2024-09-04T17:47:27.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:27.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.671482+0000", 2024-09-04T17:47:27.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053823533999999999 2024-09-04T17:47:27.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:27.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.671579+0000", 2024-09-04T17:47:27.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6657999999999995e-05 2024-09-04T17:47:27.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:27.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.671689+0000", 2024-09-04T17:47:27.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000110276 2024-09-04T17:47:27.368 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:27.368 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:27.368 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:27.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.368 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:43873 3.15 3:abd207ee:::rbd_data.113410ddd206.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:27.368 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:49.932940+0000", 2024-09-04T17:47:27.368 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.417199629999999, 2024-09-04T17:47:27.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042740514, 2024-09-04T17:47:27.368 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:27.368 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:27.368 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:27.368 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:27.369 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:27.369 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:27.369 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 43873 2024-09-04T17:47:27.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.369 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:27.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:27.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.932940+0000", 2024-09-04T17:47:27.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:27.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:27.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.932940+0000", 2024-09-04T17:47:27.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:27.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:27.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.932941+0000", 2024-09-04T17:47:27.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.404e-06 2024-09-04T17:47:27.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:27.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.933065+0000", 2024-09-04T17:47:27.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012361199999999999 2024-09-04T17:47:27.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:27.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.933078+0000", 2024-09-04T17:47:27.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3182e-05 2024-09-04T17:47:27.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:27.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.933085+0000", 2024-09-04T17:47:27.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4500000000000001e-06 2024-09-04T17:47:27.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:27.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.933134+0000", 2024-09-04T17:47:27.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9150999999999997e-05 2024-09-04T17:47:27.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:27.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.933242+0000", 2024-09-04T17:47:27.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010781 2024-09-04T17:47:27.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-04T17:47:27.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.938358+0000", 2024-09-04T17:47:27.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051160349999999997 2024-09-04T17:47:27.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:27.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.975343+0000", 2024-09-04T17:47:27.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036985663000000002 2024-09-04T17:47:27.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:27.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.975392+0000", 2024-09-04T17:47:27.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9044e-05 2024-09-04T17:47:27.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:27.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.975548+0000", 2024-09-04T17:47:27.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015572900000000001 2024-09-04T17:47:27.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:27.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.975681+0000", 2024-09-04T17:47:27.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013243400000000001 2024-09-04T17:47:27.374 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:27.374 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:27.374 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:27.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.374 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:46649 3.7 3:e0749ed5:::rbd_data.1134326b938e.000000000000014f:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-04T17:47:27.374 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:54.114730+0000", 2024-09-04T17:47:27.374 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 33.235409498000003, 2024-09-04T17:47:27.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032972972000000003, 2024-09-04T17:47:27.374 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:27.374 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:27.374 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:27.375 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:27.375 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:27.375 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:27.375 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 46649 2024-09-04T17:47:27.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.375 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:27.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:27.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114730+0000", 2024-09-04T17:47:27.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:27.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:27.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114730+0000", 2024-09-04T17:47:27.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:27.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:27.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114731+0000", 2024-09-04T17:47:27.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.147e-06 2024-09-04T17:47:27.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:27.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114737+0000", 2024-09-04T17:47:27.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6380000000000001e-06 2024-09-04T17:47:27.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:27.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114751+0000", 2024-09-04T17:47:27.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4012999999999999e-05 2024-09-04T17:47:27.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:27.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114758+0000", 2024-09-04T17:47:27.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1539999999999996e-06 2024-09-04T17:47:27.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:27.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114876+0000", 2024-09-04T17:47:27.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000117421 2024-09-04T17:47:27.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.378 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:27.378 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.115173+0000", 2024-09-04T17:47:27.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00029750799999999998 2024-09-04T17:47:27.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.378 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 7", 2024-09-04T17:47:27.378 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.120273+0000", 2024-09-04T17:47:27.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051002900000000004 2024-09-04T17:47:27.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.378 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:27.378 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.147571+0000", 2024-09-04T17:47:27.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027297906 2024-09-04T17:47:27.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:27.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.147588+0000", 2024-09-04T17:47:27.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6807999999999999e-05 2024-09-04T17:47:27.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:27.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.147637+0000", 2024-09-04T17:47:27.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8466e-05 2024-09-04T17:47:27.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:27.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.147703+0000", 2024-09-04T17:47:27.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6620999999999998e-05 2024-09-04T17:47:27.380 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:27.380 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:27.380 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:27.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.380 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49801 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 720896~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:27.380 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.075094+0000", 2024-09-04T17:47:27.380 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 24.275045510999998, 2024-09-04T17:47:27.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017824974, 2024-09-04T17:47:27.380 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:27.381 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:27.381 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:27.381 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:27.381 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:27.381 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:27.381 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49801 2024-09-04T17:47:27.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.381 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:27.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:27.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075094+0000", 2024-09-04T17:47:27.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:27.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:27.382 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075094+0000", 2024-09-04T17:47:27.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:27.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:27.382 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075095+0000", 2024-09-04T17:47:27.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.017e-06 2024-09-04T17:47:27.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:27.382 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075101+0000", 2024-09-04T17:47:27.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5029999999999997e-06 2024-09-04T17:47:27.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:27.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075119+0000", 2024-09-04T17:47:27.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8133000000000001e-05 2024-09-04T17:47:27.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:27.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075132+0000", 2024-09-04T17:47:27.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3305999999999999e-05 2024-09-04T17:47:27.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:27.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.092677+0000", 2024-09-04T17:47:27.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017544466000000002 2024-09-04T17:47:27.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:27.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.092782+0000", 2024-09-04T17:47:27.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000105284 2024-09-04T17:47:27.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:27.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.092919+0000", 2024-09-04T17:47:27.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013726499999999999 2024-09-04T17:47:27.385 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:27.385 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:27.385 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:27.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.385 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49809 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 163840~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:27.385 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077146+0000", 2024-09-04T17:47:27.385 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 24.272994067999999, 2024-09-04T17:47:27.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017984541999999999, 2024-09-04T17:47:27.385 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:27.385 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:27.386 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:27.386 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:27.386 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:27.386 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:27.386 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49809 2024-09-04T17:47:27.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.386 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:27.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:27.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077146+0000", 2024-09-04T17:47:27.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:27.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:27.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077146+0000", 2024-09-04T17:47:27.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:27.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:27.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077147+0000", 2024-09-04T17:47:27.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0330000000000001e-06 2024-09-04T17:47:27.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:27.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077164+0000", 2024-09-04T17:47:27.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7810000000000001e-05 2024-09-04T17:47:27.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:27.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077187+0000", 2024-09-04T17:47:27.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2246e-05 2024-09-04T17:47:27.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:27.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077203+0000", 2024-09-04T17:47:27.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6392000000000002e-05 2024-09-04T17:47:27.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:27.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.094885+0000", 2024-09-04T17:47:27.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017681589000000001 2024-09-04T17:47:27.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:27.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.094991+0000", 2024-09-04T17:47:27.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010668899999999999 2024-09-04T17:47:27.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:27.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095130+0000", 2024-09-04T17:47:27.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013878299999999999 2024-09-04T17:47:27.389 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:27.389 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:27.389 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:27.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.390 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49810 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 229376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:27.390 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077225+0000", 2024-09-04T17:47:27.390 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 24.272914227000001, 2024-09-04T17:47:27.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018218793, 2024-09-04T17:47:27.390 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:27.390 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:27.390 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:27.390 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:27.390 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:27.390 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:27.390 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49810 2024-09-04T17:47:27.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.390 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:27.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:27.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077225+0000", 2024-09-04T17:47:27.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:27.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:27.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077225+0000", 2024-09-04T17:47:27.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:27.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:27.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077227+0000", 2024-09-04T17:47:27.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1829999999999999e-06 2024-09-04T17:47:27.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:27.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077233+0000", 2024-09-04T17:47:27.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3899999999999998e-06 2024-09-04T17:47:27.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:27.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077253+0000", 2024-09-04T17:47:27.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0009000000000001e-05 2024-09-04T17:47:27.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:27.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077270+0000", 2024-09-04T17:47:27.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7336000000000001e-05 2024-09-04T17:47:27.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:27.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095177+0000", 2024-09-04T17:47:27.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017907101000000002 2024-09-04T17:47:27.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:27.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095299+0000", 2024-09-04T17:47:27.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000121749 2024-09-04T17:47:27.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:27.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095444+0000", 2024-09-04T17:47:27.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014502499999999999 2024-09-04T17:47:27.393 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:27.393 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:27.394 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:27.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.394 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49811 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 294912~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:27.394 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077288+0000", 2024-09-04T17:47:27.394 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 24.272851913, 2024-09-04T17:47:27.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018443620000000001, 2024-09-04T17:47:27.394 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:27.394 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:27.394 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:27.394 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:27.394 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:27.394 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:27.394 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49811 2024-09-04T17:47:27.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.395 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:27.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:27.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077288+0000", 2024-09-04T17:47:27.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:27.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:27.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077288+0000", 2024-09-04T17:47:27.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:27.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:27.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077289+0000", 2024-09-04T17:47:27.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6199999999999999e-06 2024-09-04T17:47:27.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:27.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077295+0000", 2024-09-04T17:47:27.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1499999999999998e-06 2024-09-04T17:47:27.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:27.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077312+0000", 2024-09-04T17:47:27.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7353000000000001e-05 2024-09-04T17:47:27.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:27.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077324+0000", 2024-09-04T17:47:27.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2593e-05 2024-09-04T17:47:27.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:27.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095490+0000", 2024-09-04T17:47:27.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018165976 2024-09-04T17:47:27.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:27.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095597+0000", 2024-09-04T17:47:27.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010629900000000001 2024-09-04T17:47:27.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:27.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095731+0000", 2024-09-04T17:47:27.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013462899999999999 2024-09-04T17:47:27.398 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:27.398 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:27.398 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:27.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.398 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49813 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 360448~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:27.398 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077349+0000", 2024-09-04T17:47:27.398 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 24.272790710999999, 2024-09-04T17:47:27.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018780305000000001, 2024-09-04T17:47:27.398 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:27.398 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:27.398 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:27.398 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:27.398 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:27.399 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:27.399 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49813 2024-09-04T17:47:27.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.399 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:27.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:27.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077349+0000", 2024-09-04T17:47:27.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:27.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:27.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077349+0000", 2024-09-04T17:47:27.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:27.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:27.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077350+0000", 2024-09-04T17:47:27.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0330000000000001e-06 2024-09-04T17:47:27.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:27.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077355+0000", 2024-09-04T17:47:27.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0699999999999997e-06 2024-09-04T17:47:27.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:27.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077376+0000", 2024-09-04T17:47:27.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0846e-05 2024-09-04T17:47:27.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:27.401 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077388+0000", 2024-09-04T17:47:27.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2353e-05 2024-09-04T17:47:27.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:27.401 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095819+0000", 2024-09-04T17:47:27.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01843057 2024-09-04T17:47:27.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:27.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095960+0000", 2024-09-04T17:47:27.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000140965 2024-09-04T17:47:27.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:27.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096129+0000", 2024-09-04T17:47:27.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016946800000000001 2024-09-04T17:47:27.402 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:27.402 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:27.402 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:27.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.402 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49814 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 425984~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:27.402 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077407+0000", 2024-09-04T17:47:27.403 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 24.272732210000001, 2024-09-04T17:47:27.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019041563000000001, 2024-09-04T17:47:27.403 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:27.403 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:27.403 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:27.403 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:27.403 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:27.403 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:27.403 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49814 2024-09-04T17:47:27.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.403 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:27.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:27.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077407+0000", 2024-09-04T17:47:27.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:27.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:27.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077407+0000", 2024-09-04T17:47:27.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:27.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:27.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077409+0000", 2024-09-04T17:47:27.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2330000000000001e-06 2024-09-04T17:47:27.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:27.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077415+0000", 2024-09-04T17:47:27.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2729999999999999e-06 2024-09-04T17:47:27.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:27.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077436+0000", 2024-09-04T17:47:27.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1209999999999999e-05 2024-09-04T17:47:27.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:27.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077448+0000", 2024-09-04T17:47:27.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2225999999999999e-05 2024-09-04T17:47:27.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:27.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096183+0000", 2024-09-04T17:47:27.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018735004 2024-09-04T17:47:27.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:27.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096297+0000", 2024-09-04T17:47:27.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000113962 2024-09-04T17:47:27.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:27.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096449+0000", 2024-09-04T17:47:27.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000151655 2024-09-04T17:47:27.407 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:27.407 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:27.407 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:27.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.407 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49815 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 491520~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:27.407 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077471+0000", 2024-09-04T17:47:27.407 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 24.272668972000002, 2024-09-04T17:47:27.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019266536000000001, 2024-09-04T17:47:27.407 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:27.407 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:27.407 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:27.407 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:27.407 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:27.408 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:27.408 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49815 2024-09-04T17:47:27.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.408 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:27.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:27.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077471+0000", 2024-09-04T17:47:27.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:27.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:27.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077471+0000", 2024-09-04T17:47:27.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:27.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:27.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077472+0000", 2024-09-04T17:47:27.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.17e-06 2024-09-04T17:47:27.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:27.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077478+0000", 2024-09-04T17:47:27.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0399999999999998e-06 2024-09-04T17:47:27.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:27.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077500+0000", 2024-09-04T17:47:27.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1959000000000001e-05 2024-09-04T17:47:27.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:27.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077513+0000", 2024-09-04T17:47:27.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2823000000000001e-05 2024-09-04T17:47:27.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:27.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096493+0000", 2024-09-04T17:47:27.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018980701999999999 2024-09-04T17:47:27.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:27.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096603+0000", 2024-09-04T17:47:27.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010942000000000001 2024-09-04T17:47:27.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.411 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:27.411 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096737+0000", 2024-09-04T17:47:27.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013442200000000001 2024-09-04T17:47:27.411 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:27.411 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:27.411 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:27.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.411 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49816 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 557056~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:27.411 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077532+0000", 2024-09-04T17:47:27.411 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 24.272607230999999, 2024-09-04T17:47:27.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019604331999999999, 2024-09-04T17:47:27.412 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:27.412 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:27.412 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:27.412 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:27.412 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:27.412 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:27.412 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49816 2024-09-04T17:47:27.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.412 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:27.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:27.412 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077532+0000", 2024-09-04T17:47:27.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:27.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:27.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077532+0000", 2024-09-04T17:47:27.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:27.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:27.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077534+0000", 2024-09-04T17:47:27.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1200000000000001e-06 2024-09-04T17:47:27.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:27.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077545+0000", 2024-09-04T17:47:27.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1673e-05 2024-09-04T17:47:27.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:27.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077569+0000", 2024-09-04T17:47:27.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3555999999999999e-05 2024-09-04T17:47:27.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:27.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077584+0000", 2024-09-04T17:47:27.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4883e-05 2024-09-04T17:47:27.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:27.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096825+0000", 2024-09-04T17:47:27.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019241734 2024-09-04T17:47:27.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:27.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096964+0000", 2024-09-04T17:47:27.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000138085 2024-09-04T17:47:27.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:27.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097137+0000", 2024-09-04T17:47:27.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017328100000000001 2024-09-04T17:47:27.415 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:27.415 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:27.416 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:27.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.416 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49817 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 622592~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:27.416 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077603+0000", 2024-09-04T17:47:27.416 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 24.272536673000001, 2024-09-04T17:47:27.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01986928, 2024-09-04T17:47:27.416 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:27.416 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:27.416 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:27.416 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:27.416 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:27.416 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:27.416 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49817 2024-09-04T17:47:27.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.417 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:27.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:27.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077603+0000", 2024-09-04T17:47:27.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:27.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:27.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077603+0000", 2024-09-04T17:47:27.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:27.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:27.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077604+0000", 2024-09-04T17:47:27.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7699999999999992e-07 2024-09-04T17:47:27.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:27.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077609+0000", 2024-09-04T17:47:27.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0830000000000003e-06 2024-09-04T17:47:27.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:27.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077630+0000", 2024-09-04T17:47:27.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0438999999999999e-05 2024-09-04T17:47:27.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:27.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077664+0000", 2024-09-04T17:47:27.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4579000000000002e-05 2024-09-04T17:47:27.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:27.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097193+0000", 2024-09-04T17:47:27.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019529174 2024-09-04T17:47:27.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:27.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097323+0000", 2024-09-04T17:47:27.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000129696 2024-09-04T17:47:27.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:27.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097472+0000", 2024-09-04T17:47:27.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014933200000000001 2024-09-04T17:47:27.420 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:27.420 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:27.420 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:27.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.420 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49818 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 688128~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:27.420 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077683+0000", 2024-09-04T17:47:27.420 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 24.272456308999999, 2024-09-04T17:47:27.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020086333000000001, 2024-09-04T17:47:27.421 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:27.421 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:27.421 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:27.421 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:27.421 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:27.421 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:27.421 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49818 2024-09-04T17:47:27.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.421 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:27.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:27.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077683+0000", 2024-09-04T17:47:27.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:27.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:27.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077683+0000", 2024-09-04T17:47:27.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:27.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:27.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077684+0000", 2024-09-04T17:47:27.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4e-07 2024-09-04T17:47:27.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:27.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077690+0000", 2024-09-04T17:47:27.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4729999999999999e-06 2024-09-04T17:47:27.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:27.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077710+0000", 2024-09-04T17:47:27.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0149999999999999e-05 2024-09-04T17:47:27.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:27.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077723+0000", 2024-09-04T17:47:27.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3271999999999999e-05 2024-09-04T17:47:27.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:27.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097521+0000", 2024-09-04T17:47:27.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019797981999999999 2024-09-04T17:47:27.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:27.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097628+0000", 2024-09-04T17:47:27.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010665700000000001 2024-09-04T17:47:27.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:27.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097770+0000", 2024-09-04T17:47:27.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014185900000000001 2024-09-04T17:47:27.425 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:27.425 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:27.425 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:27.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.425 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49820 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 753664~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:27.425 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077744+0000", 2024-09-04T17:47:27.425 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 24.272395377999999, 2024-09-04T17:47:27.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020364786999999999, 2024-09-04T17:47:27.425 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:27.425 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:27.425 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:27.425 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:27.425 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:27.426 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:27.426 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49820 2024-09-04T17:47:27.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.426 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:27.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:27.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077744+0000", 2024-09-04T17:47:27.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:27.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:27.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077744+0000", 2024-09-04T17:47:27.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:27.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:27.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077745+0000", 2024-09-04T17:47:27.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0270000000000001e-06 2024-09-04T17:47:27.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:27.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077751+0000", 2024-09-04T17:47:27.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.626e-06 2024-09-04T17:47:27.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:27.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077771+0000", 2024-09-04T17:47:27.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0469999999999999e-05 2024-09-04T17:47:27.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:27.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077785+0000", 2024-09-04T17:47:27.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3129000000000001e-05 2024-09-04T17:47:27.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:27.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097816+0000", 2024-09-04T17:47:27.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020031917999999999 2024-09-04T17:47:27.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:27.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097939+0000", 2024-09-04T17:47:27.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000122533 2024-09-04T17:47:27.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:27.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098109+0000", 2024-09-04T17:47:27.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017008400000000001 2024-09-04T17:47:27.429 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:27.429 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:27.429 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:27.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.429 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49821 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 819200~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:27.429 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077807+0000", 2024-09-04T17:47:27.429 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 24.272333165999999, 2024-09-04T17:47:27.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020702928999999998, 2024-09-04T17:47:27.429 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:27.430 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:27.430 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:27.430 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:27.430 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:27.430 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:27.430 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49821 2024-09-04T17:47:27.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.430 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:27.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.430 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:27.430 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077807+0000", 2024-09-04T17:47:27.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:27.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:27.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077807+0000", 2024-09-04T17:47:27.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:27.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:27.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077808+0000", 2024-09-04T17:47:27.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.463e-06 2024-09-04T17:47:27.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:27.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077813+0000", 2024-09-04T17:47:27.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7530000000000001e-06 2024-09-04T17:47:27.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:27.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077832+0000", 2024-09-04T17:47:27.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.946e-05 2024-09-04T17:47:27.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:27.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077846+0000", 2024-09-04T17:47:27.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3786e-05 2024-09-04T17:47:27.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:27.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098201+0000", 2024-09-04T17:47:27.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020354797000000001 2024-09-04T17:47:27.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:27.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098341+0000", 2024-09-04T17:47:27.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000140042 2024-09-04T17:47:27.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:27.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098509+0000", 2024-09-04T17:47:27.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016862800000000001 2024-09-04T17:47:27.433 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:27.433 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:27.433 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:27.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.434 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49823 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 884736~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:27.434 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077865+0000", 2024-09-04T17:47:27.434 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 24.272274721999999, 2024-09-04T17:47:27.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021008723, 2024-09-04T17:47:27.434 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:27.434 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:27.434 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:27.434 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:27.434 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:27.434 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:27.434 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49823 2024-09-04T17:47:27.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.435 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:27.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:27.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077865+0000", 2024-09-04T17:47:27.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:27.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:27.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077865+0000", 2024-09-04T17:47:27.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:27.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:27.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077866+0000", 2024-09-04T17:47:27.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.164e-06 2024-09-04T17:47:27.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:27.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077872+0000", 2024-09-04T17:47:27.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3959999999999997e-06 2024-09-04T17:47:27.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:27.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077892+0000", 2024-09-04T17:47:27.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0970000000000001e-05 2024-09-04T17:47:27.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:27.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077905+0000", 2024-09-04T17:47:27.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2312e-05 2024-09-04T17:47:27.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:27.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098579+0000", 2024-09-04T17:47:27.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020674486999999998 2024-09-04T17:47:27.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:27.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098705+0000", 2024-09-04T17:47:27.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012541900000000001 2024-09-04T17:47:27.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:27.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098874+0000", 2024-09-04T17:47:27.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000168975 2024-09-04T17:47:27.438 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:27.438 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:27.438 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:27.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.438 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49824 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 950272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:27.438 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077923+0000", 2024-09-04T17:47:27.438 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 24.27221703, 2024-09-04T17:47:27.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021246287999999999, 2024-09-04T17:47:27.438 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:27.438 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:27.439 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:27.439 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:27.439 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:27.439 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:27.439 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49824 2024-09-04T17:47:27.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.439 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:27.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.439 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:27.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077923+0000", 2024-09-04T17:47:27.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:27.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.439 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:27.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077923+0000", 2024-09-04T17:47:27.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:27.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:27.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077924+0000", 2024-09-04T17:47:27.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0100000000000001e-06 2024-09-04T17:47:27.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:27.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077944+0000", 2024-09-04T17:47:27.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0449e-05 2024-09-04T17:47:27.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:27.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077965+0000", 2024-09-04T17:47:27.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0479000000000002e-05 2024-09-04T17:47:27.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:27.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077990+0000", 2024-09-04T17:47:27.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5582999999999999e-05 2024-09-04T17:47:27.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:27.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098924+0000", 2024-09-04T17:47:27.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020933395 2024-09-04T17:47:27.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:27.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.099032+0000", 2024-09-04T17:47:27.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010846299999999999 2024-09-04T17:47:27.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:27.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.099169+0000", 2024-09-04T17:47:27.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000136909 2024-09-04T17:47:27.442 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:27.442 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:27.442 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:27.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.443 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49826 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 1015808~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:27.443 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.078010+0000", 2024-09-04T17:47:27.443 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 24.272129422999999, 2024-09-04T17:47:27.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021584564000000001, 2024-09-04T17:47:27.443 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:27.443 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:27.443 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:27.443 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:27.443 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:27.443 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:27.443 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49826 2024-09-04T17:47:27.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.444 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:27.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:27.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078010+0000", 2024-09-04T17:47:27.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:27.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:27.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078010+0000", 2024-09-04T17:47:27.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:27.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:27.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078011+0000", 2024-09-04T17:47:27.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1340000000000001e-06 2024-09-04T17:47:27.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:27.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078017+0000", 2024-09-04T17:47:27.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.682e-06 2024-09-04T17:47:27.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:27.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078036+0000", 2024-09-04T17:47:27.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9357000000000001e-05 2024-09-04T17:47:27.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:27.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078050+0000", 2024-09-04T17:47:27.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3122000000000001e-05 2024-09-04T17:47:27.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:27.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.099271+0000", 2024-09-04T17:47:27.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021221772999999999 2024-09-04T17:47:27.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:27.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.099410+0000", 2024-09-04T17:47:27.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000138415 2024-09-04T17:47:27.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:27.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:27.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:27.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.099595+0000", 2024-09-04T17:47:27.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000185081 2024-09-04T17:47:27.447 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:27.447 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:27.447 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:27.447 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:27.447 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:27.447 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:27.447 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-04T17:47:27.449 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:27.449+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:27.449 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-04T17:47:27.550 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:27.549+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:27.620 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:27.621 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:47:27.621 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:47:27.621 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:27.634 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-04T17:47:27.636 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:47:27.636 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:47:27.650 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:27.648+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:27.653 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:47:27.653 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:47:27.654 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-04T17:47:27.671 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-04T17:47:27.671 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-04T17:47:27.692 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:47:27.692 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:47:27.712 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:47:27.713 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:47:27.713 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-04T17:47:27.734 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:47:27.734 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:47:27.738 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-04T17:47:27.738 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-04T17:47:27.750 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:27.750 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:27.751 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:27.749+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:27.764 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-04T17:47:27.820 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-04T17:47:27.852 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:27.848+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:27.935 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:27.935 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:27.950 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-04T17:47:27.953 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:27.953+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:27.984 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone34 from 0xd327c0f (overlap 0x18bf32) to 0x27985 2024-09-04T17:47:28.053 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:28.053+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:28.072 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:28.072 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:28.086 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-04T17:47:28.154 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:28.152+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:28.211 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #33, image image_client.0-clone33 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent34 2024-09-04T17:47:28.213 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:28.214 INFO:teuthology.orchestra.run.smithi092.stdout:5075 trunc from 0xd327c0f to 0xcebd68b 2024-09-04T17:47:28.217 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 6, 7] out_osds: [3] dead_osds: [6, 2] live_osds: [0, 1, 4, 5, 7, 3] 2024-09-04T17:47:28.217 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-04T17:47:28.217 INFO:tasks.thrashosds.thrasher:Removing osd 1, in_osds are: [0, 1, 2, 4, 5, 6, 7] 2024-09-04T17:47:28.217 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 1 2024-09-04T17:47:28.250 INFO:teuthology.orchestra.run.smithi092.stdout:5078 trunc from 0xcebd68b to 0x88ca826 2024-09-04T17:47:28.255 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:28.253+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:28.255 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:28.255 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:28.270 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-04T17:47:28.283 INFO:teuthology.orchestra.run.smithi092.stdout:5079 write 0x72db2e3 thru 0x72e01af (0x4ecd bytes) 2024-09-04T17:47:28.296 INFO:teuthology.orchestra.run.smithi092.stdout:5080 read 0x5d024ff thru 0x5d06734 (0x4236 bytes) 2024-09-04T17:47:28.299 INFO:teuthology.orchestra.run.smithi092.stdout:5084 trunc from 0x88ca826 to 0x5e0ea92 2024-09-04T17:47:28.356 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:28.353+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:28.358 INFO:teuthology.orchestra.run.smithi092.stdout:5086 write 0xdc57ef8 thru 0xdc6410a (0xc213 bytes) 2024-09-04T17:47:28.367 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:28.367 INFO:teuthology.orchestra.run.smithi092.stdout:5089 write 0x72a12eb thru 0x72afe97 (0xebad bytes) 2024-09-04T17:47:28.372 INFO:teuthology.orchestra.run.smithi092.stdout:5091 read 0x581b5a5 thru 0x581bc19 (0x675 bytes) 2024-09-04T17:47:28.383 INFO:teuthology.orchestra.run.smithi092.stdout:5092 trunc from 0xdc6410b to 0x5642b4f 2024-09-04T17:47:28.406 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:28.406 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:28.406 INFO:teuthology.orchestra.run.smithi092.stdout:5095 punch from 0xeda92d to 0xede4a8, (0x3b7b bytes) 2024-09-04T17:47:28.409 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:28.409 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:28.412 INFO:teuthology.orchestra.run.smithi092.stdout:5097 trunc from 0x5642b4f to 0xe0254f 2024-09-04T17:47:28.426 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-04T17:47:28.473 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:28.472+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:28.482 INFO:teuthology.orchestra.run.smithi092.stdout:5099 writesame 0xd756ee6 thru 0xd764f58 data_size 0x6b(0xe073 bytes) 2024-09-04T17:47:28.483 INFO:teuthology.orchestra.run.smithi092.stdout:5100 read 0x2ffce0 thru 0x309f9c (0xa2bd bytes) 2024-09-04T17:47:28.483 INFO:teuthology.orchestra.run.smithi092.stdout:5101 write 0xc64f91 thru 0xc7012f (0xb19f bytes) 2024-09-04T17:47:28.485 INFO:teuthology.orchestra.run.smithi092.stdout:5102 write 0x93c0b01 thru 0x93c9a3a (0x8f3a bytes) 2024-09-04T17:47:28.491 INFO:teuthology.orchestra.run.smithi092.stdout:5103 read 0xd0c7ab8 thru 0xd0c92e3 (0x182c bytes) 2024-09-04T17:47:28.492 INFO:teuthology.orchestra.run.smithi092.stdout:5104 read 0x9ff8ffd thru 0xa0044eb (0xb4ef bytes) 2024-09-04T17:47:28.505 INFO:teuthology.orchestra.run.smithi092.stdout:5107 trunc from 0xd764f59 to 0x1f583e1 2024-09-04T17:47:28.525 INFO:teuthology.orchestra.run.smithi092.stdout:5110 punch from 0x6d4282 to 0x6d9a64, (0x57e2 bytes) 2024-09-04T17:47:28.525 INFO:teuthology.orchestra.run.smithi092.stdout:5111 read 0xe7a268 thru 0xe89c0c (0xf9a5 bytes) 2024-09-04T17:47:28.526 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:28.526 INFO:teuthology.orchestra.run.smithi092.stdout:5113 read 0x1e41913 thru 0x1e49303 (0x79f1 bytes) 2024-09-04T17:47:28.527 INFO:teuthology.orchestra.run.smithi092.stdout:5114 read 0x1c6fe50 thru 0x1c7e1a5 (0xe356 bytes) 2024-09-04T17:47:28.542 INFO:tasks.ceph.mon.a.smithi092.stderr:2024-09-04T17:47:28.541+0000 7f8f2f46f640 -1 mon.a@0(leader).osd e94 definitely_dead 0 2024-09-04T17:47:28.554 INFO:teuthology.orchestra.run.smithi092.stdout:5116 write 0xb253ba7 thru 0xb25b922 (0x7d7c bytes) 2024-09-04T17:47:28.557 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:28.557+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:28.560 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:28.568 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:28.568 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:28.573 INFO:teuthology.orchestra.run.smithi092.stdout:5119 write 0xc61b007 thru 0xc6235ad (0x85a7 bytes) 2024-09-04T17:47:28.582 INFO:teuthology.orchestra.run.smithi092.stdout:5120 read 0x4bd58fe thru 0x4bdcc18 (0x731b bytes) 2024-09-04T17:47:28.582 INFO:teuthology.orchestra.run.smithi092.stdout:5121 write 0x4d2a6d8 thru 0x4d370a2 (0xc9cb bytes) 2024-09-04T17:47:28.584 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-04T17:47:28.589 INFO:teuthology.orchestra.run.smithi092.stdout:5122 punch from 0x7ec9aac to 0x7ed835f, (0xe8b3 bytes) 2024-09-04T17:47:28.589 INFO:teuthology.orchestra.run.smithi092.stdout:5123 read 0x265550c thru 0x26568af (0x13a4 bytes) 2024-09-04T17:47:28.589 INFO:teuthology.orchestra.run.smithi092.stdout:5124 punch from 0x27942a3 to 0x27961e6, (0x1f43 bytes) 2024-09-04T17:47:28.609 INFO:teuthology.orchestra.run.smithi092.stdout:5125 trunc from 0xc6235ae to 0x2bc92a6 2024-09-04T17:47:28.630 INFO:teuthology.orchestra.run.smithi092.stdout:5126 punch from 0xe49413 to 0xe5836c, (0xef59 bytes) 2024-09-04T17:47:28.658 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:28.657+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:28.660 INFO:teuthology.orchestra.run.smithi092.stdout:5127 write 0xd33fdd4 thru 0xd3425d7 (0x2804 bytes) 2024-09-04T17:47:28.667 INFO:teuthology.orchestra.run.smithi092.stdout:5128 read 0x4231b39 thru 0x4236c03 (0x50cb bytes) 2024-09-04T17:47:28.667 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:28.667 INFO:teuthology.orchestra.run.smithi092.stdout:5130 read 0xb7215e3 thru 0xb72ff1c (0xe93a bytes) 2024-09-04T17:47:28.667 INFO:teuthology.orchestra.run.smithi092.stdout:5132 read 0x2afc2d thru 0x2b9625 (0x99f9 bytes) 2024-09-04T17:47:28.668 INFO:teuthology.orchestra.run.smithi092.stdout:5133 write 0x72e75d5 thru 0x72e83f2 (0xe1e bytes) 2024-09-04T17:47:28.673 INFO:teuthology.orchestra.run.smithi092.stdout:5134 write 0x7602a38 thru 0x7608864 (0x5e2d bytes) 2024-09-04T17:47:28.682 INFO:teuthology.orchestra.run.smithi092.stdout:5135 trunc from 0xd3425d8 to 0xeb8781c 2024-09-04T17:47:28.694 INFO:teuthology.orchestra.run.smithi092.stdout:5136 write 0xa145388 thru 0xa1505fa (0xb273 bytes) 2024-09-04T17:47:28.702 INFO:teuthology.orchestra.run.smithi092.stdout:5137 write 0xa46e58d thru 0xa478ebc (0xa930 bytes) 2024-09-04T17:47:28.728 INFO:teuthology.orchestra.run.smithi092.stdout:5138 trunc from 0xeb8781c to 0x73a29ef 2024-09-04T17:47:28.759 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:28.756+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:28.760 INFO:teuthology.orchestra.run.smithi092.stdout:5139 trunc from 0x73a29ef to 0x5808732 2024-09-04T17:47:28.771 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:28.771 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:28.781 INFO:teuthology.orchestra.run.smithi092.stdout:5140 read 0x5237b6f thru 0x524585f (0xdcf1 bytes) 2024-09-04T17:47:28.781 INFO:teuthology.orchestra.run.smithi092.stdout:5142 punch from 0x527c1ae to 0x5284cdc, (0x8b2e bytes) 2024-09-04T17:47:28.782 INFO:teuthology.orchestra.run.smithi092.stdout:5143 write 0x15b2a28 thru 0x15b608c (0x3665 bytes) 2024-09-04T17:47:28.785 INFO:teuthology.orchestra.run.smithi092.stdout:5144 read 0xe7184c thru 0xe7d539 (0xbcee bytes) 2024-09-04T17:47:28.787 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-04T17:47:28.787 INFO:teuthology.orchestra.run.smithi092.stdout:5147 read 0x3179133 thru 0x317b199 (0x2067 bytes) 2024-09-04T17:47:28.787 INFO:teuthology.orchestra.run.smithi092.stdout:5149 write 0x1b88289 thru 0x1b957e9 (0xd561 bytes) 2024-09-04T17:47:28.790 INFO:teuthology.orchestra.run.smithi092.stderr:marked out osd.1. 2024-09-04T17:47:28.808 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is unset 2024-09-04T17:47:28.877 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:28.876+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:28.954 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:28.954 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:28.962 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:28.961+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:28.968 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-04T17:47:29.063 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:29.060+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:29.138 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:29.138 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:29.152 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-04T17:47:29.197 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:29.196+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:29.265 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:29.265+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:29.322 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:29.322 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:29.336 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-04T17:47:29.366 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:29.365+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:29.455 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:29.455 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:29.467 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:29.465+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:29.469 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-04T17:47:29.568 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:29.565+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:29.592 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:29.592 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:29.606 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-04T17:47:29.669 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:29.669+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:29.729 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:29.729 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:29.739 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-04T17:47:29.744 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-04T17:47:29.785 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:29.785+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:29.871 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:29.869+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:29.872 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:29.872 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:29.886 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-04T17:47:29.900 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:47:29.901 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:47:29.917 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:47:29.918 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:47:29.918 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-04T17:47:29.934 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-04T17:47:29.934 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-04T17:47:29.954 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:47:29.955 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:47:29.971 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:29.968+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:29.976 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:47:29.976 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:47:29.976 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-04T17:47:29.996 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:47:29.996 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:47:29.999 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-04T17:47:29.999 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-04T17:47:30.002 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:30.002 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:47:30.002 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:47:30.002 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:30.016 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-04T17:47:30.073 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:30.072+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:30.174 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:30.173+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:30.178 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:30.178 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:47:30.178 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:47:30.178 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:47:30.178 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:30.192 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-04T17:47:30.281 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:30.281+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:30.366 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:30.366 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:47:30.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:47:30.366 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:47:30.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.366 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:27242 3.b 3.34414deb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-04T17:47:30.366 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:28.402347+0000", 2024-09-04T17:47:30.366 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 61.961568450999998, 2024-09-04T17:47:30.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.93265329299999999, 2024-09-04T17:47:30.367 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:30.367 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:30.367 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-04T17:47:30.367 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:30.367 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:30.367 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:30.367 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 27242 2024-09-04T17:47:30.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.367 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:30.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:30.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402347+0000", 2024-09-04T17:47:30.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:30.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:30.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402347+0000", 2024-09-04T17:47:30.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:30.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:30.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402349+0000", 2024-09-04T17:47:30.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.471e-06 2024-09-04T17:47:30.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:30.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402355+0000", 2024-09-04T17:47:30.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8399999999999997e-06 2024-09-04T17:47:30.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:30.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402364+0000", 2024-09-04T17:47:30.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1269999999999992e-06 2024-09-04T17:47:30.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:30.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402373+0000", 2024-09-04T17:47:30.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7909999999999993e-06 2024-09-04T17:47:30.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:30.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402460+0000", 2024-09-04T17:47:30.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6686000000000003e-05 2024-09-04T17:47:30.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-04T17:47:30.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402473+0000", 2024-09-04T17:47:30.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3281e-05 2024-09-04T17:47:30.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:30.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:29.334989+0000", 2024-09-04T17:47:30.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.93251549099999997 2024-09-04T17:47:30.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:30.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:29.335000+0000", 2024-09-04T17:47:30.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1606e-05 2024-09-04T17:47:30.371 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:30.371 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:30.371 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:30.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.371 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:33552 3.7 3:e79ad1db:::rbd_data.11342e8f3384.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-04T17:47:30.371 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:35.999748+0000", 2024-09-04T17:47:30.372 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 54.364167401000003, 2024-09-04T17:47:30.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045683823999999998, 2024-09-04T17:47:30.372 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:30.372 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:30.372 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:30.372 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:30.372 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:30.372 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:30.372 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 33552 2024-09-04T17:47:30.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.372 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:30.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:30.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999748+0000", 2024-09-04T17:47:30.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:30.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:30.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999748+0000", 2024-09-04T17:47:30.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:30.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:30.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999750+0000", 2024-09-04T17:47:30.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1299999999999999e-06 2024-09-04T17:47:30.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:30.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999759+0000", 2024-09-04T17:47:30.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4649999999999994e-06 2024-09-04T17:47:30.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:30.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999778+0000", 2024-09-04T17:47:30.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8833000000000001e-05 2024-09-04T17:47:30.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:30.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999786+0000", 2024-09-04T17:47:30.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7369999999999995e-06 2024-09-04T17:47:30.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:30.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999856+0000", 2024-09-04T17:47:30.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9247999999999998e-05 2024-09-04T17:47:30.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:30.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999973+0000", 2024-09-04T17:47:30.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000117921 2024-09-04T17:47:30.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 7", 2024-09-04T17:47:30.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.005955+0000", 2024-09-04T17:47:30.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059820560000000004 2024-09-04T17:47:30.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:30.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.045297+0000", 2024-09-04T17:47:30.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.039341650999999998 2024-09-04T17:47:30.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:30.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.045343+0000", 2024-09-04T17:47:30.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5899000000000002e-05 2024-09-04T17:47:30.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:30.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.045388+0000", 2024-09-04T17:47:30.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4601999999999997e-05 2024-09-04T17:47:30.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:30.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.045432+0000", 2024-09-04T17:47:30.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4282000000000001e-05 2024-09-04T17:47:30.378 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:30.378 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:30.378 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:30.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.378 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:37357 3.15 3:a897312a:::rbd_data.1134c5a43465.000000000000001f:head [stat,write 62464~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-04T17:47:30.378 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:39.616724+0000", 2024-09-04T17:47:30.378 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.747191993000001, 2024-09-04T17:47:30.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054965854000000001, 2024-09-04T17:47:30.378 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:30.378 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:30.378 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:30.378 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:30.379 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:30.379 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:30.379 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 37357 2024-09-04T17:47:30.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.379 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:30.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:30.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616724+0000", 2024-09-04T17:47:30.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:30.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:30.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616724+0000", 2024-09-04T17:47:30.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:30.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:30.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616725+0000", 2024-09-04T17:47:30.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.815e-06 2024-09-04T17:47:30.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:30.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616739+0000", 2024-09-04T17:47:30.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3723000000000001e-05 2024-09-04T17:47:30.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:30.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616755+0000", 2024-09-04T17:47:30.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5964000000000002e-05 2024-09-04T17:47:30.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:30.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616763+0000", 2024-09-04T17:47:30.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3869999999999995e-06 2024-09-04T17:47:30.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:30.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616825+0000", 2024-09-04T17:47:30.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1854999999999996e-05 2024-09-04T17:47:30.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:30.382 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616895+0000", 2024-09-04T17:47:30.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9597999999999993e-05 2024-09-04T17:47:30.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-04T17:47:30.382 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616991+0000", 2024-09-04T17:47:30.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5828000000000005e-05 2024-09-04T17:47:30.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:30.382 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.617659+0000", 2024-09-04T17:47:30.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00066821600000000001 2024-09-04T17:47:30.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:30.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.671482+0000", 2024-09-04T17:47:30.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053823533999999999 2024-09-04T17:47:30.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:30.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.671579+0000", 2024-09-04T17:47:30.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6657999999999995e-05 2024-09-04T17:47:30.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:30.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.671689+0000", 2024-09-04T17:47:30.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000110276 2024-09-04T17:47:30.384 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:30.384 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:30.384 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:30.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.385 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:43873 3.15 3:abd207ee:::rbd_data.113410ddd206.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:30.385 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:49.932940+0000", 2024-09-04T17:47:30.385 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.430975439999997, 2024-09-04T17:47:30.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042740514, 2024-09-04T17:47:30.385 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:30.385 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:30.385 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:30.385 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:30.385 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:30.385 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:30.385 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 43873 2024-09-04T17:47:30.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.385 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:30.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:30.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.932940+0000", 2024-09-04T17:47:30.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:30.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:30.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.932940+0000", 2024-09-04T17:47:30.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:30.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:30.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.932941+0000", 2024-09-04T17:47:30.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.404e-06 2024-09-04T17:47:30.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:30.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.933065+0000", 2024-09-04T17:47:30.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012361199999999999 2024-09-04T17:47:30.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:30.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.933078+0000", 2024-09-04T17:47:30.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3182e-05 2024-09-04T17:47:30.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:30.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.933085+0000", 2024-09-04T17:47:30.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4500000000000001e-06 2024-09-04T17:47:30.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:30.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.933134+0000", 2024-09-04T17:47:30.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9150999999999997e-05 2024-09-04T17:47:30.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:30.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.933242+0000", 2024-09-04T17:47:30.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010781 2024-09-04T17:47:30.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-04T17:47:30.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.938358+0000", 2024-09-04T17:47:30.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051160349999999997 2024-09-04T17:47:30.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:30.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.975343+0000", 2024-09-04T17:47:30.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036985663000000002 2024-09-04T17:47:30.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:30.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.975392+0000", 2024-09-04T17:47:30.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9044e-05 2024-09-04T17:47:30.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:30.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.975548+0000", 2024-09-04T17:47:30.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015572900000000001 2024-09-04T17:47:30.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:30.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.975681+0000", 2024-09-04T17:47:30.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013243400000000001 2024-09-04T17:47:30.390 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:30.390 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:30.390 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:30.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.391 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:46649 3.7 3:e0749ed5:::rbd_data.1134326b938e.000000000000014f:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-04T17:47:30.391 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:54.114730+0000", 2024-09-04T17:47:30.391 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.249185308000001, 2024-09-04T17:47:30.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032972972000000003, 2024-09-04T17:47:30.391 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:30.391 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:30.391 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:30.391 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:30.391 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:30.391 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:30.391 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 46649 2024-09-04T17:47:30.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.392 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:30.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:30.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114730+0000", 2024-09-04T17:47:30.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:30.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:30.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114730+0000", 2024-09-04T17:47:30.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:30.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:30.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114731+0000", 2024-09-04T17:47:30.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.147e-06 2024-09-04T17:47:30.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:30.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114737+0000", 2024-09-04T17:47:30.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6380000000000001e-06 2024-09-04T17:47:30.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:30.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114751+0000", 2024-09-04T17:47:30.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4012999999999999e-05 2024-09-04T17:47:30.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:30.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114758+0000", 2024-09-04T17:47:30.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1539999999999996e-06 2024-09-04T17:47:30.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:30.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114876+0000", 2024-09-04T17:47:30.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000117421 2024-09-04T17:47:30.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:30.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.115173+0000", 2024-09-04T17:47:30.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00029750799999999998 2024-09-04T17:47:30.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 7", 2024-09-04T17:47:30.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.120273+0000", 2024-09-04T17:47:30.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051002900000000004 2024-09-04T17:47:30.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:30.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.147571+0000", 2024-09-04T17:47:30.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027297906 2024-09-04T17:47:30.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:30.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.147588+0000", 2024-09-04T17:47:30.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6807999999999999e-05 2024-09-04T17:47:30.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:30.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.147637+0000", 2024-09-04T17:47:30.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8466e-05 2024-09-04T17:47:30.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:30.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.147703+0000", 2024-09-04T17:47:30.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6620999999999998e-05 2024-09-04T17:47:30.397 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:30.397 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:30.397 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:30.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.397 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49801 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 720896~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:30.397 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.075094+0000", 2024-09-04T17:47:30.397 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.288821321, 2024-09-04T17:47:30.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017824974, 2024-09-04T17:47:30.398 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:30.398 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:30.398 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:30.398 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:30.398 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:30.398 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:30.398 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49801 2024-09-04T17:47:30.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.398 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:30.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:30.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075094+0000", 2024-09-04T17:47:30.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:30.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:30.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075094+0000", 2024-09-04T17:47:30.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:30.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:30.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075095+0000", 2024-09-04T17:47:30.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.017e-06 2024-09-04T17:47:30.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:30.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075101+0000", 2024-09-04T17:47:30.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5029999999999997e-06 2024-09-04T17:47:30.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:30.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075119+0000", 2024-09-04T17:47:30.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8133000000000001e-05 2024-09-04T17:47:30.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:30.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075132+0000", 2024-09-04T17:47:30.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3305999999999999e-05 2024-09-04T17:47:30.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:30.401 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.092677+0000", 2024-09-04T17:47:30.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017544466000000002 2024-09-04T17:47:30.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:30.401 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.092782+0000", 2024-09-04T17:47:30.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000105284 2024-09-04T17:47:30.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:30.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.092919+0000", 2024-09-04T17:47:30.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013726499999999999 2024-09-04T17:47:30.402 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:30.402 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:30.402 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:30.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.402 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49809 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 163840~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:30.402 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077146+0000", 2024-09-04T17:47:30.402 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.286769878000001, 2024-09-04T17:47:30.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017984541999999999, 2024-09-04T17:47:30.402 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:30.403 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:30.403 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:30.403 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:30.403 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:30.403 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:30.403 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49809 2024-09-04T17:47:30.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.403 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:30.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:30.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077146+0000", 2024-09-04T17:47:30.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:30.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:30.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077146+0000", 2024-09-04T17:47:30.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:30.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:30.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077147+0000", 2024-09-04T17:47:30.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0330000000000001e-06 2024-09-04T17:47:30.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:30.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077164+0000", 2024-09-04T17:47:30.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7810000000000001e-05 2024-09-04T17:47:30.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:30.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077187+0000", 2024-09-04T17:47:30.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2246e-05 2024-09-04T17:47:30.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:30.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077203+0000", 2024-09-04T17:47:30.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6392000000000002e-05 2024-09-04T17:47:30.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:30.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.094885+0000", 2024-09-04T17:47:30.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017681589000000001 2024-09-04T17:47:30.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:30.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.094991+0000", 2024-09-04T17:47:30.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010668899999999999 2024-09-04T17:47:30.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:30.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095130+0000", 2024-09-04T17:47:30.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013878299999999999 2024-09-04T17:47:30.406 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:30.406 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:30.406 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:30.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.407 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49810 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 229376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:30.407 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077225+0000", 2024-09-04T17:47:30.407 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.286690037, 2024-09-04T17:47:30.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018218793, 2024-09-04T17:47:30.407 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:30.407 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:30.407 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:30.407 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:30.407 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:30.407 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:30.407 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49810 2024-09-04T17:47:30.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.407 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:30.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:30.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077225+0000", 2024-09-04T17:47:30.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:30.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:30.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077225+0000", 2024-09-04T17:47:30.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:30.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:30.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077227+0000", 2024-09-04T17:47:30.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1829999999999999e-06 2024-09-04T17:47:30.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:30.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077233+0000", 2024-09-04T17:47:30.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3899999999999998e-06 2024-09-04T17:47:30.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:30.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077253+0000", 2024-09-04T17:47:30.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0009000000000001e-05 2024-09-04T17:47:30.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:30.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077270+0000", 2024-09-04T17:47:30.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7336000000000001e-05 2024-09-04T17:47:30.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:30.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095177+0000", 2024-09-04T17:47:30.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017907101000000002 2024-09-04T17:47:30.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:30.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095299+0000", 2024-09-04T17:47:30.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000121749 2024-09-04T17:47:30.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:30.411 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095444+0000", 2024-09-04T17:47:30.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014502499999999999 2024-09-04T17:47:30.411 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:30.411 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:30.411 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:30.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.411 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49811 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 294912~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:30.411 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077288+0000", 2024-09-04T17:47:30.411 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.286627722999999, 2024-09-04T17:47:30.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018443620000000001, 2024-09-04T17:47:30.411 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:30.411 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:30.411 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:30.412 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:30.412 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:30.412 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:30.412 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49811 2024-09-04T17:47:30.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.412 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:30.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:30.412 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077288+0000", 2024-09-04T17:47:30.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:30.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:30.412 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077288+0000", 2024-09-04T17:47:30.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:30.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:30.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077289+0000", 2024-09-04T17:47:30.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6199999999999999e-06 2024-09-04T17:47:30.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:30.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077295+0000", 2024-09-04T17:47:30.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1499999999999998e-06 2024-09-04T17:47:30.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:30.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077312+0000", 2024-09-04T17:47:30.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7353000000000001e-05 2024-09-04T17:47:30.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:30.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077324+0000", 2024-09-04T17:47:30.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2593e-05 2024-09-04T17:47:30.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:30.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095490+0000", 2024-09-04T17:47:30.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018165976 2024-09-04T17:47:30.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:30.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095597+0000", 2024-09-04T17:47:30.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010629900000000001 2024-09-04T17:47:30.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:30.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095731+0000", 2024-09-04T17:47:30.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013462899999999999 2024-09-04T17:47:30.415 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:30.415 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:30.415 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:30.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.416 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49813 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 360448~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:30.416 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077349+0000", 2024-09-04T17:47:30.416 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.286566521000001, 2024-09-04T17:47:30.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018780305000000001, 2024-09-04T17:47:30.416 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:30.416 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:30.416 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:30.416 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:30.416 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:30.416 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:30.416 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49813 2024-09-04T17:47:30.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.416 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:30.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:30.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077349+0000", 2024-09-04T17:47:30.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:30.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:30.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077349+0000", 2024-09-04T17:47:30.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:30.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:30.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077350+0000", 2024-09-04T17:47:30.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0330000000000001e-06 2024-09-04T17:47:30.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:30.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077355+0000", 2024-09-04T17:47:30.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0699999999999997e-06 2024-09-04T17:47:30.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:30.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077376+0000", 2024-09-04T17:47:30.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0846e-05 2024-09-04T17:47:30.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:30.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077388+0000", 2024-09-04T17:47:30.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2353e-05 2024-09-04T17:47:30.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:30.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095819+0000", 2024-09-04T17:47:30.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01843057 2024-09-04T17:47:30.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:30.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095960+0000", 2024-09-04T17:47:30.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000140965 2024-09-04T17:47:30.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:30.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096129+0000", 2024-09-04T17:47:30.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016946800000000001 2024-09-04T17:47:30.420 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:30.420 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:30.420 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:30.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.420 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49814 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 425984~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:30.420 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077407+0000", 2024-09-04T17:47:30.420 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.286508019999999, 2024-09-04T17:47:30.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019041563000000001, 2024-09-04T17:47:30.420 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:30.420 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:30.420 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:30.421 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:30.421 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:30.421 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:30.421 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49814 2024-09-04T17:47:30.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.421 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:30.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:30.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077407+0000", 2024-09-04T17:47:30.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:30.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:30.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077407+0000", 2024-09-04T17:47:30.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:30.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:30.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077409+0000", 2024-09-04T17:47:30.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2330000000000001e-06 2024-09-04T17:47:30.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:30.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077415+0000", 2024-09-04T17:47:30.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2729999999999999e-06 2024-09-04T17:47:30.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:30.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077436+0000", 2024-09-04T17:47:30.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1209999999999999e-05 2024-09-04T17:47:30.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:30.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077448+0000", 2024-09-04T17:47:30.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2225999999999999e-05 2024-09-04T17:47:30.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:30.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096183+0000", 2024-09-04T17:47:30.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018735004 2024-09-04T17:47:30.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:30.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096297+0000", 2024-09-04T17:47:30.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000113962 2024-09-04T17:47:30.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:30.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096449+0000", 2024-09-04T17:47:30.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000151655 2024-09-04T17:47:30.424 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:30.424 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:30.424 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:30.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.425 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49815 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 491520~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:30.425 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077471+0000", 2024-09-04T17:47:30.425 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.286444782, 2024-09-04T17:47:30.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019266536000000001, 2024-09-04T17:47:30.425 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:30.425 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:30.425 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:30.425 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:30.425 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:30.425 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:30.425 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49815 2024-09-04T17:47:30.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.426 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:30.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:30.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077471+0000", 2024-09-04T17:47:30.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:30.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:30.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077471+0000", 2024-09-04T17:47:30.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:30.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:30.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077472+0000", 2024-09-04T17:47:30.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.17e-06 2024-09-04T17:47:30.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:30.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077478+0000", 2024-09-04T17:47:30.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0399999999999998e-06 2024-09-04T17:47:30.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:30.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077500+0000", 2024-09-04T17:47:30.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1959000000000001e-05 2024-09-04T17:47:30.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:30.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077513+0000", 2024-09-04T17:47:30.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2823000000000001e-05 2024-09-04T17:47:30.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:30.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096493+0000", 2024-09-04T17:47:30.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018980701999999999 2024-09-04T17:47:30.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:30.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096603+0000", 2024-09-04T17:47:30.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010942000000000001 2024-09-04T17:47:30.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:30.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096737+0000", 2024-09-04T17:47:30.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013442200000000001 2024-09-04T17:47:30.429 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:30.429 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:30.429 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:30.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.429 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49816 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 557056~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:30.429 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077532+0000", 2024-09-04T17:47:30.429 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.286383041000001, 2024-09-04T17:47:30.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019604331999999999, 2024-09-04T17:47:30.430 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:30.430 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:30.430 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:30.430 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:30.430 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:30.430 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:30.430 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49816 2024-09-04T17:47:30.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.430 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:30.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.430 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:30.430 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077532+0000", 2024-09-04T17:47:30.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:30.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:30.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077532+0000", 2024-09-04T17:47:30.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:30.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:30.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077534+0000", 2024-09-04T17:47:30.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1200000000000001e-06 2024-09-04T17:47:30.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:30.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077545+0000", 2024-09-04T17:47:30.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1673e-05 2024-09-04T17:47:30.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:30.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077569+0000", 2024-09-04T17:47:30.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3555999999999999e-05 2024-09-04T17:47:30.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:30.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077584+0000", 2024-09-04T17:47:30.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4883e-05 2024-09-04T17:47:30.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:30.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096825+0000", 2024-09-04T17:47:30.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019241734 2024-09-04T17:47:30.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:30.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096964+0000", 2024-09-04T17:47:30.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000138085 2024-09-04T17:47:30.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:30.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097137+0000", 2024-09-04T17:47:30.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017328100000000001 2024-09-04T17:47:30.434 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:30.434 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:30.434 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:30.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.434 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49817 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 622592~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:30.434 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077603+0000", 2024-09-04T17:47:30.434 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.286312483, 2024-09-04T17:47:30.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01986928, 2024-09-04T17:47:30.434 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:30.434 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:30.434 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:30.434 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:30.435 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:30.435 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:30.435 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49817 2024-09-04T17:47:30.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.435 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:30.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:30.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077603+0000", 2024-09-04T17:47:30.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:30.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:30.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077603+0000", 2024-09-04T17:47:30.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:30.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:30.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077604+0000", 2024-09-04T17:47:30.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7699999999999992e-07 2024-09-04T17:47:30.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:30.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077609+0000", 2024-09-04T17:47:30.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0830000000000003e-06 2024-09-04T17:47:30.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:30.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077630+0000", 2024-09-04T17:47:30.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0438999999999999e-05 2024-09-04T17:47:30.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:30.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077664+0000", 2024-09-04T17:47:30.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4579000000000002e-05 2024-09-04T17:47:30.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:30.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097193+0000", 2024-09-04T17:47:30.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019529174 2024-09-04T17:47:30.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:30.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097323+0000", 2024-09-04T17:47:30.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000129696 2024-09-04T17:47:30.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:30.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097472+0000", 2024-09-04T17:47:30.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014933200000000001 2024-09-04T17:47:30.438 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:30.438 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:30.438 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:30.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.438 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49818 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 688128~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:30.439 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077683+0000", 2024-09-04T17:47:30.439 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.286232119000001, 2024-09-04T17:47:30.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020086333000000001, 2024-09-04T17:47:30.439 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:30.439 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:30.439 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:30.439 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:30.439 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:30.439 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:30.439 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49818 2024-09-04T17:47:30.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.439 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:30.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:30.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077683+0000", 2024-09-04T17:47:30.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:30.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:30.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077683+0000", 2024-09-04T17:47:30.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:30.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:30.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077684+0000", 2024-09-04T17:47:30.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4e-07 2024-09-04T17:47:30.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:30.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077690+0000", 2024-09-04T17:47:30.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4729999999999999e-06 2024-09-04T17:47:30.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:30.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077710+0000", 2024-09-04T17:47:30.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0149999999999999e-05 2024-09-04T17:47:30.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:30.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077723+0000", 2024-09-04T17:47:30.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3271999999999999e-05 2024-09-04T17:47:30.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:30.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097521+0000", 2024-09-04T17:47:30.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019797981999999999 2024-09-04T17:47:30.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:30.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097628+0000", 2024-09-04T17:47:30.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010665700000000001 2024-09-04T17:47:30.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:30.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097770+0000", 2024-09-04T17:47:30.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014185900000000001 2024-09-04T17:47:30.443 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:30.443 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:30.443 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:30.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.444 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49820 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 753664~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:30.444 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077744+0000", 2024-09-04T17:47:30.444 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.286171188000001, 2024-09-04T17:47:30.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020364786999999999, 2024-09-04T17:47:30.444 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:30.444 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:30.444 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:30.444 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:30.444 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:30.444 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:30.444 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49820 2024-09-04T17:47:30.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.445 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:30.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:30.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077744+0000", 2024-09-04T17:47:30.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:30.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:30.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077744+0000", 2024-09-04T17:47:30.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:30.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:30.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077745+0000", 2024-09-04T17:47:30.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0270000000000001e-06 2024-09-04T17:47:30.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:30.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077751+0000", 2024-09-04T17:47:30.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.626e-06 2024-09-04T17:47:30.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:30.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077771+0000", 2024-09-04T17:47:30.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0469999999999999e-05 2024-09-04T17:47:30.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:30.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077785+0000", 2024-09-04T17:47:30.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3129000000000001e-05 2024-09-04T17:47:30.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:30.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097816+0000", 2024-09-04T17:47:30.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020031917999999999 2024-09-04T17:47:30.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:30.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097939+0000", 2024-09-04T17:47:30.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000122533 2024-09-04T17:47:30.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:30.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098109+0000", 2024-09-04T17:47:30.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017008400000000001 2024-09-04T17:47:30.448 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:30.448 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:30.448 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:30.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.448 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49821 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 819200~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:30.448 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077807+0000", 2024-09-04T17:47:30.449 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.286108976000001, 2024-09-04T17:47:30.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020702928999999998, 2024-09-04T17:47:30.449 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:30.449 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:30.449 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:30.449 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:30.449 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:30.449 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:30.449 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49821 2024-09-04T17:47:30.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.449 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:30.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:30.450 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077807+0000", 2024-09-04T17:47:30.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:30.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.450 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:30.450 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077807+0000", 2024-09-04T17:47:30.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:30.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.450 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:30.450 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077808+0000", 2024-09-04T17:47:30.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.463e-06 2024-09-04T17:47:30.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.451 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:30.451 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077813+0000", 2024-09-04T17:47:30.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7530000000000001e-06 2024-09-04T17:47:30.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.451 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:30.451 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077832+0000", 2024-09-04T17:47:30.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.946e-05 2024-09-04T17:47:30.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.451 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:30.451 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077846+0000", 2024-09-04T17:47:30.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3786e-05 2024-09-04T17:47:30.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:30.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098201+0000", 2024-09-04T17:47:30.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020354797000000001 2024-09-04T17:47:30.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:30.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098341+0000", 2024-09-04T17:47:30.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000140042 2024-09-04T17:47:30.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:30.453 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098509+0000", 2024-09-04T17:47:30.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016862800000000001 2024-09-04T17:47:30.453 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:30.453 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:30.453 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:30.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.453 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49823 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 884736~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:30.453 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077865+0000", 2024-09-04T17:47:30.453 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.286050532000001, 2024-09-04T17:47:30.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021008723, 2024-09-04T17:47:30.453 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:30.453 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:30.454 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:30.454 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:30.454 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:30.454 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:30.454 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49823 2024-09-04T17:47:30.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.454 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:30.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.454 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:30.454 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077865+0000", 2024-09-04T17:47:30.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:30.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:30.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077865+0000", 2024-09-04T17:47:30.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:30.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:30.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077866+0000", 2024-09-04T17:47:30.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.164e-06 2024-09-04T17:47:30.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:30.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077872+0000", 2024-09-04T17:47:30.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3959999999999997e-06 2024-09-04T17:47:30.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:30.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077892+0000", 2024-09-04T17:47:30.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0970000000000001e-05 2024-09-04T17:47:30.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:30.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077905+0000", 2024-09-04T17:47:30.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2312e-05 2024-09-04T17:47:30.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:30.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098579+0000", 2024-09-04T17:47:30.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020674486999999998 2024-09-04T17:47:30.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:30.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098705+0000", 2024-09-04T17:47:30.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012541900000000001 2024-09-04T17:47:30.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:30.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098874+0000", 2024-09-04T17:47:30.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000168975 2024-09-04T17:47:30.457 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:30.458 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:30.458 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:30.458 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.458 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.458 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49824 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 950272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:30.458 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077923+0000", 2024-09-04T17:47:30.458 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.285992839999999, 2024-09-04T17:47:30.458 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021246287999999999, 2024-09-04T17:47:30.458 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:30.458 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:30.458 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:30.458 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:30.459 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:30.459 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:30.459 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49824 2024-09-04T17:47:30.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.459 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:30.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:30.459 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077923+0000", 2024-09-04T17:47:30.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:30.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:30.459 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077923+0000", 2024-09-04T17:47:30.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:30.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:30.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077924+0000", 2024-09-04T17:47:30.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0100000000000001e-06 2024-09-04T17:47:30.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:30.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077944+0000", 2024-09-04T17:47:30.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0449e-05 2024-09-04T17:47:30.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:30.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077965+0000", 2024-09-04T17:47:30.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0479000000000002e-05 2024-09-04T17:47:30.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:30.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077990+0000", 2024-09-04T17:47:30.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5582999999999999e-05 2024-09-04T17:47:30.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:30.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098924+0000", 2024-09-04T17:47:30.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020933395 2024-09-04T17:47:30.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.462 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:30.462 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.099032+0000", 2024-09-04T17:47:30.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010846299999999999 2024-09-04T17:47:30.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.462 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:30.462 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.099169+0000", 2024-09-04T17:47:30.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000136909 2024-09-04T17:47:30.462 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:30.462 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:30.462 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:30.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.463 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49826 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 1015808~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:30.463 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.078010+0000", 2024-09-04T17:47:30.463 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.285905233000001, 2024-09-04T17:47:30.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021584564000000001, 2024-09-04T17:47:30.463 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:30.463 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:30.463 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:30.463 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:30.463 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:30.463 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:30.463 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49826 2024-09-04T17:47:30.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.463 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:30.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.464 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:30.464 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078010+0000", 2024-09-04T17:47:30.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:30.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.464 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:30.464 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078010+0000", 2024-09-04T17:47:30.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:30.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.464 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:30.464 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078011+0000", 2024-09-04T17:47:30.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1340000000000001e-06 2024-09-04T17:47:30.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:30.465 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078017+0000", 2024-09-04T17:47:30.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.682e-06 2024-09-04T17:47:30.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:30.465 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078036+0000", 2024-09-04T17:47:30.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9357000000000001e-05 2024-09-04T17:47:30.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:30.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078050+0000", 2024-09-04T17:47:30.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3122000000000001e-05 2024-09-04T17:47:30.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:30.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.099271+0000", 2024-09-04T17:47:30.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021221772999999999 2024-09-04T17:47:30.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:30.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.099410+0000", 2024-09-04T17:47:30.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000138415 2024-09-04T17:47:30.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:30.467 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:30.467 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:30.467 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.099595+0000", 2024-09-04T17:47:30.467 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000185081 2024-09-04T17:47:30.467 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:30.467 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:30.467 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:30.467 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:30.467 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:30.467 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:30.469 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-04T17:47:30.469 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:30.469+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:30.569 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:30.569+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:30.639 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:30.639 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:47:30.639 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:47:30.639 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:30.653 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-04T17:47:30.670 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:30.668+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:30.771 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:30.768+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:30.822 INFO:teuthology.orchestra.run.smithi092.stdout:5151 write 0x62033ba thru 0x620439b (0xfe2 bytes) 2024-09-04T17:47:30.834 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-04T17:47:30.842 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:30.843 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:47:30.843 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:47:30.843 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:47:30.843 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:30.843 INFO:teuthology.orchestra.run.smithi092.stdout:5153 trunc from 0x620439c to 0x274642 2024-09-04T17:47:30.862 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-04T17:47:30.872 INFO:teuthology.orchestra.run.smithi092.stdout:5154 read 0x38e25 thru 0x39088 (0x264 bytes) 2024-09-04T17:47:30.872 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:30.869+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:30.882 INFO:teuthology.orchestra.run.smithi092.stdout:5155 trunc from 0x274642 to 0x63886bb 2024-09-04T17:47:30.894 INFO:teuthology.orchestra.run.smithi092.stdout:5157 write 0xc93797 thru 0xc9f46a (0xbcd4 bytes) 2024-09-04T17:47:30.897 INFO:teuthology.orchestra.run.smithi092.stdout:5159 read 0xca1e6d thru 0xcaf943 (0xdad7 bytes) 2024-09-04T17:47:30.898 INFO:teuthology.orchestra.run.smithi092.stdout:5160 read 0x46742ae thru 0x467cd73 (0x8ac6 bytes) 2024-09-04T17:47:30.898 INFO:teuthology.orchestra.run.smithi092.stdout:5161 write 0x2488bc5 thru 0x248fe43 (0x727f bytes) 2024-09-04T17:47:30.904 INFO:teuthology.orchestra.run.smithi092.stdout:5163 read 0x45b61ee thru 0x45ba42e (0x4241 bytes) 2024-09-04T17:47:30.905 INFO:teuthology.orchestra.run.smithi092.stdout:5164 trunc from 0x63886bb to 0x5903a10 2024-09-04T17:47:30.916 INFO:teuthology.orchestra.run.smithi092.stdout:5166 read 0x2d8a981 thru 0x2d8d320 (0x29a0 bytes) 2024-09-04T17:47:30.916 INFO:teuthology.orchestra.run.smithi092.stdout:5167 punch from 0xa85ede to 0xa8a905, (0x4a27 bytes) 2024-09-04T17:47:30.921 INFO:teuthology.orchestra.run.smithi092.stdout:5168 trunc from 0x5903a10 to 0x8de83be 2024-09-04T17:47:30.931 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:30.931 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:30.931 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:30.931 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:30.932 INFO:teuthology.orchestra.run.smithi092.stdout:5175 punch from 0x5cc1a9b to 0x5ccdfe1, (0xc546 bytes) 2024-09-04T17:47:30.951 INFO:teuthology.orchestra.run.smithi092.stdout:5177 writesame 0xe04b9c3 thru 0xe056483 data_size 0x9(0xaac1 bytes) 2024-09-04T17:47:30.951 INFO:teuthology.orchestra.run.smithi092.stdout:5179 write 0x6c47dfd thru 0x6c54498 (0xc69c bytes) 2024-09-04T17:47:30.960 INFO:teuthology.orchestra.run.smithi092.stdout:5181 write 0x2545115 thru 0x254f957 (0xa843 bytes) 2024-09-04T17:47:30.962 INFO:teuthology.orchestra.run.smithi092.stdout:5182 write 0xaa6da97 thru 0xaa7296b (0x4ed5 bytes) 2024-09-04T17:47:30.968 INFO:teuthology.orchestra.run.smithi092.stdout:5183 read 0xc068763 thru 0xc0778a9 (0xf147 bytes) 2024-09-04T17:47:30.969 INFO:teuthology.orchestra.run.smithi092.stdout:5184 punch from 0x401e100 to 0x40281be, (0xa0be bytes) 2024-09-04T17:47:30.969 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:30.969 INFO:teuthology.orchestra.run.smithi092.stdout:5186 write 0xadadbad thru 0xadbb176 (0xd5ca bytes) 2024-09-04T17:47:30.971 INFO:teuthology.orchestra.run.smithi092.stdout:5187 write 0x6c9337f thru 0x6ca10f7 (0xdd79 bytes) 2024-09-04T17:47:30.972 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:47:30.972+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:47:30.976 INFO:teuthology.orchestra.run.smithi092.stdout:5188 writesame 0xd33cf3e thru 0xd34c927 data_size 0x3(0xf9ea bytes) 2024-09-04T17:47:30.989 INFO:teuthology.orchestra.run.smithi092.stdout:5189 write 0xab75b9f thru 0xab80c3a (0xb09c bytes) 2024-09-04T17:47:30.994 INFO:teuthology.orchestra.run.smithi092.stdout:5191 punch from 0x6ac0a2b to 0x6aca8df, (0x9eb4 bytes) 2024-09-04T17:47:31.007 INFO:teuthology.orchestra.run.smithi092.stdout:5192 trunc from 0xe056484 to 0x5a045ff 2024-09-04T17:47:31.021 INFO:teuthology.orchestra.run.smithi092.stdout:5193 writesame 0x132a9c1 thru 0x1335483 data_size 0x7(0xaac3 bytes) 2024-09-04T17:47:31.028 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:31.028 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:47:31.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:47:31.028 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:47:31.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.028 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:48444 3.12 3:4ff80fb5:::rbd_data.11346a2b0bc9.000000000000001d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-04T17:47:31.028 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:58.122601+0000", 2024-09-04T17:47:31.028 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.904965318000002, 2024-09-04T17:47:31.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057065713999999997, 2024-09-04T17:47:31.029 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:31.029 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:31.029 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:31.029 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:31.029 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:31.029 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:31.029 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 48444 2024-09-04T17:47:31.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.029 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:31.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:31.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122601+0000", 2024-09-04T17:47:31.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:31.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122601+0000", 2024-09-04T17:47:31.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:31.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122603+0000", 2024-09-04T17:47:31.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3499999999999999e-06 2024-09-04T17:47:31.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:31.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122614+0000", 2024-09-04T17:47:31.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.063e-05 2024-09-04T17:47:31.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:31.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122643+0000", 2024-09-04T17:47:31.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9434999999999999e-05 2024-09-04T17:47:31.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:31.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122657+0000", 2024-09-04T17:47:31.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3920000000000001e-05 2024-09-04T17:47:31.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:31.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122767+0000", 2024-09-04T17:47:31.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010981599999999999 2024-09-04T17:47:31.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:31.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122989+0000", 2024-09-04T17:47:31.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022158200000000001 2024-09-04T17:47:31.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 3", 2024-09-04T17:47:31.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.133349+0000", 2024-09-04T17:47:31.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010360729000000001 2024-09-04T17:47:31.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:31.033 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.179423+0000", 2024-09-04T17:47:31.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046073408000000003 2024-09-04T17:47:31.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:31.033 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.179453+0000", 2024-09-04T17:47:31.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0229000000000001e-05 2024-09-04T17:47:31.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:31.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.179545+0000", 2024-09-04T17:47:31.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1580000000000001e-05 2024-09-04T17:47:31.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:31.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.179667+0000", 2024-09-04T17:47:31.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012203500000000001 2024-09-04T17:47:31.034 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.034 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:31.034 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.034 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:55376 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-04T17:47:31.035 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:11.974053+0000", 2024-09-04T17:47:31.035 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.053513534, 2024-09-04T17:47:31.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060514722, 2024-09-04T17:47:31.035 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:31.035 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:31.035 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:31.035 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:31.035 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:31.035 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:31.035 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 55376 2024-09-04T17:47:31.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.035 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:31.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:31.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974053+0000", 2024-09-04T17:47:31.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:31.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974053+0000", 2024-09-04T17:47:31.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:31.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974054+0000", 2024-09-04T17:47:31.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.841e-06 2024-09-04T17:47:31.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.037 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:31.037 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974062+0000", 2024-09-04T17:47:31.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5460000000000003e-06 2024-09-04T17:47:31.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.037 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:31.037 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974077+0000", 2024-09-04T17:47:31.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5389000000000001e-05 2024-09-04T17:47:31.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.037 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:31.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974085+0000", 2024-09-04T17:47:31.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7659999999999998e-06 2024-09-04T17:47:31.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:31.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974188+0000", 2024-09-04T17:47:31.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000102521 2024-09-04T17:47:31.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:31.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974321+0000", 2024-09-04T17:47:31.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013324699999999999 2024-09-04T17:47:31.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-04T17:47:31.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.986481+0000", 2024-09-04T17:47:31.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012160450999999999 2024-09-04T17:47:31.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:31.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:12.034415+0000", 2024-09-04T17:47:31.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.047934065999999997 2024-09-04T17:47:31.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:31.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:12.034438+0000", 2024-09-04T17:47:31.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2923999999999999e-05 2024-09-04T17:47:31.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:31.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:12.034505+0000", 2024-09-04T17:47:31.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6711000000000006e-05 2024-09-04T17:47:31.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:31.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:12.034567+0000", 2024-09-04T17:47:31.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2260000000000004e-05 2024-09-04T17:47:31.040 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.040 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:31.040 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.041 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61128 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 1458176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:31.041 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819489+0000", 2024-09-04T17:47:31.041 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.208077199000002, 2024-09-04T17:47:31.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.038380478000000003, 2024-09-04T17:47:31.041 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:31.041 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:31.041 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:31.041 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:31.041 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:31.041 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:31.041 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61128 2024-09-04T17:47:31.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.041 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:31.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:31.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819489+0000", 2024-09-04T17:47:31.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:31.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819489+0000", 2024-09-04T17:47:31.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:31.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819490+0000", 2024-09-04T17:47:31.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6599999999999996e-07 2024-09-04T17:47:31.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:31.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819493+0000", 2024-09-04T17:47:31.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2279999999999998e-06 2024-09-04T17:47:31.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:31.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819501+0000", 2024-09-04T17:47:31.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4039999999999995e-06 2024-09-04T17:47:31.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:31.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819508+0000", 2024-09-04T17:47:31.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2400000000000004e-06 2024-09-04T17:47:31.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:31.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.857534+0000", 2024-09-04T17:47:31.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.038026272 2024-09-04T17:47:31.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:31.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.857629+0000", 2024-09-04T17:47:31.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5397999999999997e-05 2024-09-04T17:47:31.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:31.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.857869+0000", 2024-09-04T17:47:31.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024027 2024-09-04T17:47:31.046 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.046 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:31.046 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.046 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61129 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 1564672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:31.046 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819517+0000", 2024-09-04T17:47:31.046 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.208049377999998, 2024-09-04T17:47:31.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.038772745999999997, 2024-09-04T17:47:31.046 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:31.046 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:31.046 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:31.047 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:31.047 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:31.047 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:31.047 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61129 2024-09-04T17:47:31.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.047 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:31.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:31.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819517+0000", 2024-09-04T17:47:31.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:31.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819517+0000", 2024-09-04T17:47:31.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:31.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819517+0000", 2024-09-04T17:47:31.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5999999999999999e-07 2024-09-04T17:47:31.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:31.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819520+0000", 2024-09-04T17:47:31.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2709999999999999e-06 2024-09-04T17:47:31.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:31.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819526+0000", 2024-09-04T17:47:31.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7059999999999998e-06 2024-09-04T17:47:31.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:31.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819531+0000", 2024-09-04T17:47:31.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0549999999999999e-06 2024-09-04T17:47:31.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:31.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.857913+0000", 2024-09-04T17:47:31.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.038381954000000003 2024-09-04T17:47:31.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:31.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.858000+0000", 2024-09-04T17:47:31.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6963000000000001e-05 2024-09-04T17:47:31.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:31.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.858290+0000", 2024-09-04T17:47:31.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028933700000000003 2024-09-04T17:47:31.050 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.050 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:31.050 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.051 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61130 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 1597440~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:31.051 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819539+0000", 2024-09-04T17:47:31.051 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.208027551000001, 2024-09-04T17:47:31.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.039166595999999998, 2024-09-04T17:47:31.051 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:31.051 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:31.051 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:31.051 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:31.051 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:31.051 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:31.051 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61130 2024-09-04T17:47:31.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.052 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:31.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:31.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819539+0000", 2024-09-04T17:47:31.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:31.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819539+0000", 2024-09-04T17:47:31.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:31.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819539+0000", 2024-09-04T17:47:31.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9400000000000005e-07 2024-09-04T17:47:31.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:31.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819542+0000", 2024-09-04T17:47:31.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2630000000000002e-06 2024-09-04T17:47:31.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:31.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819548+0000", 2024-09-04T17:47:31.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3269999999999997e-06 2024-09-04T17:47:31.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:31.054 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819553+0000", 2024-09-04T17:47:31.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9849999999999997e-06 2024-09-04T17:47:31.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:31.054 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.858321+0000", 2024-09-04T17:47:31.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.038768395999999997 2024-09-04T17:47:31.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:31.054 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.858406+0000", 2024-09-04T17:47:31.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4456999999999995e-05 2024-09-04T17:47:31.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:31.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.858705+0000", 2024-09-04T17:47:31.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00029947399999999999 2024-09-04T17:47:31.055 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.055 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:31.055 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.055 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61131 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 1703936~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:31.055 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819560+0000", 2024-09-04T17:47:31.055 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.208006199, 2024-09-04T17:47:31.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.039522082, 2024-09-04T17:47:31.056 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:31.056 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:31.056 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:31.056 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:31.056 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:31.056 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:31.056 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61131 2024-09-04T17:47:31.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.056 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:31.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:31.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819560+0000", 2024-09-04T17:47:31.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:31.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819560+0000", 2024-09-04T17:47:31.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:31.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819561+0000", 2024-09-04T17:47:31.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5300000000000004e-07 2024-09-04T17:47:31.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:31.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819563+0000", 2024-09-04T17:47:31.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.187e-06 2024-09-04T17:47:31.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:31.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819569+0000", 2024-09-04T17:47:31.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6170000000000003e-06 2024-09-04T17:47:31.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:31.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819574+0000", 2024-09-04T17:47:31.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9640000000000002e-06 2024-09-04T17:47:31.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:31.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.858754+0000", 2024-09-04T17:47:31.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.039179295000000003 2024-09-04T17:47:31.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:31.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.858800+0000", 2024-09-04T17:47:31.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.587e-05 2024-09-04T17:47:31.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:31.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.859082+0000", 2024-09-04T17:47:31.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028249599999999997 2024-09-04T17:47:31.060 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.060 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:31.060 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.060 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61132 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 1810432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:31.060 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819581+0000", 2024-09-04T17:47:31.060 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.207984756999998, 2024-09-04T17:47:31.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.039953120000000002, 2024-09-04T17:47:31.060 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:31.060 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:31.060 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:31.060 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:31.061 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:31.061 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:31.061 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61132 2024-09-04T17:47:31.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.061 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:31.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:31.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819581+0000", 2024-09-04T17:47:31.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:31.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819581+0000", 2024-09-04T17:47:31.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:31.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819582+0000", 2024-09-04T17:47:31.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3399999999999999e-07 2024-09-04T17:47:31.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:31.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819584+0000", 2024-09-04T17:47:31.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1320000000000001e-06 2024-09-04T17:47:31.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:31.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819591+0000", 2024-09-04T17:47:31.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3759999999999996e-06 2024-09-04T17:47:31.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:31.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819595+0000", 2024-09-04T17:47:31.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7339999999999999e-06 2024-09-04T17:47:31.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:31.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.859119+0000", 2024-09-04T17:47:31.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.039523677 2024-09-04T17:47:31.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:31.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.859232+0000", 2024-09-04T17:47:31.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000113001 2024-09-04T17:47:31.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:31.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.859535+0000", 2024-09-04T17:47:31.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030256599999999999 2024-09-04T17:47:31.064 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.064 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:31.064 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.065 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61133 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 1843200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:31.065 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819602+0000", 2024-09-04T17:47:31.065 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.207964448999999, 2024-09-04T17:47:31.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.040226171999999998, 2024-09-04T17:47:31.065 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:31.065 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:31.065 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:31.065 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:31.065 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:31.065 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:31.065 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61133 2024-09-04T17:47:31.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.065 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:31.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:31.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819602+0000", 2024-09-04T17:47:31.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:31.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819602+0000", 2024-09-04T17:47:31.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:31.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819602+0000", 2024-09-04T17:47:31.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.75e-07 2024-09-04T17:47:31.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:31.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819605+0000", 2024-09-04T17:47:31.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.52e-06 2024-09-04T17:47:31.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:31.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819611+0000", 2024-09-04T17:47:31.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5200000000000003e-06 2024-09-04T17:47:31.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:31.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819616+0000", 2024-09-04T17:47:31.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0939999999999998e-06 2024-09-04T17:47:31.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:31.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.859538+0000", 2024-09-04T17:47:31.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.039921660999999997 2024-09-04T17:47:31.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:31.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.859594+0000", 2024-09-04T17:47:31.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5671999999999998e-05 2024-09-04T17:47:31.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:31.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.859828+0000", 2024-09-04T17:47:31.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023423000000000001 2024-09-04T17:47:31.069 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.069 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:31.069 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.069 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61134 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 1949696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:31.069 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819623+0000", 2024-09-04T17:47:31.069 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.207942972000001, 2024-09-04T17:47:31.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.040606862000000001, 2024-09-04T17:47:31.070 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:31.070 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:31.070 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:31.070 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:31.070 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:31.070 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:31.070 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61134 2024-09-04T17:47:31.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.070 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:31.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:31.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819623+0000", 2024-09-04T17:47:31.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:31.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819623+0000", 2024-09-04T17:47:31.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:31.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819624+0000", 2024-09-04T17:47:31.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2100000000000002e-07 2024-09-04T17:47:31.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:31.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819626+0000", 2024-09-04T17:47:31.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2460000000000002e-06 2024-09-04T17:47:31.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:31.072 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819632+0000", 2024-09-04T17:47:31.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1659999999999999e-06 2024-09-04T17:47:31.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:31.072 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819637+0000", 2024-09-04T17:47:31.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2680000000000002e-06 2024-09-04T17:47:31.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:31.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.859872+0000", 2024-09-04T17:47:31.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.040234377000000002 2024-09-04T17:47:31.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:31.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.859950+0000", 2024-09-04T17:47:31.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8128999999999997e-05 2024-09-04T17:47:31.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:31.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.860230+0000", 2024-09-04T17:47:31.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028025499999999998 2024-09-04T17:47:31.073 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.074 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:31.074 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.074 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61135 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 2056192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:31.074 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819644+0000", 2024-09-04T17:47:31.074 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.207921932000001, 2024-09-04T17:47:31.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.040925930999999999, 2024-09-04T17:47:31.074 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:31.074 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:31.074 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:31.074 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:31.074 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:31.075 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:31.075 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61135 2024-09-04T17:47:31.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.075 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:31.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:31.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819644+0000", 2024-09-04T17:47:31.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:31.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819644+0000", 2024-09-04T17:47:31.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:31.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819645+0000", 2024-09-04T17:47:31.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9399999999999995e-07 2024-09-04T17:47:31.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:31.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819647+0000", 2024-09-04T17:47:31.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.311e-06 2024-09-04T17:47:31.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.077 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:31.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819653+0000", 2024-09-04T17:47:31.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3940000000000001e-06 2024-09-04T17:47:31.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.077 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:31.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819658+0000", 2024-09-04T17:47:31.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8420000000000004e-06 2024-09-04T17:47:31.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.077 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:31.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.860263+0000", 2024-09-04T17:47:31.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.040605109 2024-09-04T17:47:31.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:31.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.860339+0000", 2024-09-04T17:47:31.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5612999999999996e-05 2024-09-04T17:47:31.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:31.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.860570+0000", 2024-09-04T17:47:31.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023116799999999999 2024-09-04T17:47:31.078 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.078 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:31.078 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.079 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61136 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 2195456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:31.079 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819665+0000", 2024-09-04T17:47:31.079 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.207901478, 2024-09-04T17:47:31.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.041275408999999999, 2024-09-04T17:47:31.079 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:31.079 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:31.079 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:31.079 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:31.079 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:31.079 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:31.079 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61136 2024-09-04T17:47:31.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.079 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:31.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:31.080 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819665+0000", 2024-09-04T17:47:31.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:31.080 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819665+0000", 2024-09-04T17:47:31.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:31.080 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819665+0000", 2024-09-04T17:47:31.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8599999999999999e-07 2024-09-04T17:47:31.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:31.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819667+0000", 2024-09-04T17:47:31.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9439999999999999e-06 2024-09-04T17:47:31.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:31.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819673+0000", 2024-09-04T17:47:31.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0220000000000001e-06 2024-09-04T17:47:31.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:31.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819677+0000", 2024-09-04T17:47:31.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4139999999999996e-06 2024-09-04T17:47:31.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:31.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.860603+0000", 2024-09-04T17:47:31.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.040925504000000001 2024-09-04T17:47:31.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:31.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.860673+0000", 2024-09-04T17:47:31.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9707000000000002e-05 2024-09-04T17:47:31.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.083 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:31.083 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.860940+0000", 2024-09-04T17:47:31.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026743200000000001 2024-09-04T17:47:31.083 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.083 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:31.083 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.083 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61137 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 2301952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:31.083 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819685+0000", 2024-09-04T17:47:31.083 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.207880684999999, 2024-09-04T17:47:31.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.041579208999999999, 2024-09-04T17:47:31.084 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:31.084 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:31.084 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:31.084 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:31.084 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:31.084 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:31.084 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61137 2024-09-04T17:47:31.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.084 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:31.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:31.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819685+0000", 2024-09-04T17:47:31.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:31.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819685+0000", 2024-09-04T17:47:31.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:31.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819686+0000", 2024-09-04T17:47:31.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3000000000000001e-07 2024-09-04T17:47:31.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:31.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819688+0000", 2024-09-04T17:47:31.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1150000000000001e-06 2024-09-04T17:47:31.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:31.086 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819694+0000", 2024-09-04T17:47:31.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.75e-06 2024-09-04T17:47:31.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:31.086 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819699+0000", 2024-09-04T17:47:31.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8110000000000002e-06 2024-09-04T17:47:31.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.087 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:31.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.860977+0000", 2024-09-04T17:47:31.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.041277945000000003 2024-09-04T17:47:31.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.087 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:31.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.861071+0000", 2024-09-04T17:47:31.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4252000000000005e-05 2024-09-04T17:47:31.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.087 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:31.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.861265+0000", 2024-09-04T17:47:31.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019390600000000001 2024-09-04T17:47:31.088 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.088 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:31.088 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.088 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61138 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 2441216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:31.088 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819706+0000", 2024-09-04T17:47:31.088 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.207860283999999, 2024-09-04T17:47:31.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.041876780000000002, 2024-09-04T17:47:31.088 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:31.088 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:31.088 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:31.088 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:31.089 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:31.089 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:31.089 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61138 2024-09-04T17:47:31.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.089 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:31.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:31.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819706+0000", 2024-09-04T17:47:31.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:31.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819706+0000", 2024-09-04T17:47:31.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:31.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819707+0000", 2024-09-04T17:47:31.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7299999999999995e-07 2024-09-04T17:47:31.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:31.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819727+0000", 2024-09-04T17:47:31.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0610000000000001e-05 2024-09-04T17:47:31.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:31.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819735+0000", 2024-09-04T17:47:31.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.123e-06 2024-09-04T17:47:31.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:31.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819741+0000", 2024-09-04T17:47:31.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4009999999999996e-06 2024-09-04T17:47:31.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:31.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.861298+0000", 2024-09-04T17:47:31.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.041557135000000002 2024-09-04T17:47:31.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:31.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.861369+0000", 2024-09-04T17:47:31.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0794e-05 2024-09-04T17:47:31.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:31.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.861583+0000", 2024-09-04T17:47:31.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021404400000000001 2024-09-04T17:47:31.092 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.093 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:31.093 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.093 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61139 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 2547712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:31.093 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819750+0000", 2024-09-04T17:47:31.093 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.20781582, 2024-09-04T17:47:31.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042185699, 2024-09-04T17:47:31.093 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:31.093 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:31.093 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:31.093 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:31.093 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:31.094 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:31.094 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61139 2024-09-04T17:47:31.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.094 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:31.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:31.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819750+0000", 2024-09-04T17:47:31.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:31.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819750+0000", 2024-09-04T17:47:31.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:31.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819751+0000", 2024-09-04T17:47:31.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4000000000000002e-07 2024-09-04T17:47:31.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:31.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819753+0000", 2024-09-04T17:47:31.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5179999999999999e-06 2024-09-04T17:47:31.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:31.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819760+0000", 2024-09-04T17:47:31.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5440000000000004e-06 2024-09-04T17:47:31.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:31.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819765+0000", 2024-09-04T17:47:31.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8760000000000004e-06 2024-09-04T17:47:31.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:31.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.861622+0000", 2024-09-04T17:47:31.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.041857093999999997 2024-09-04T17:47:31.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:31.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.861698+0000", 2024-09-04T17:47:31.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6080000000000003e-05 2024-09-04T17:47:31.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:31.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.861936+0000", 2024-09-04T17:47:31.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023804699999999999 2024-09-04T17:47:31.097 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.097 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:31.097 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.098 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61140 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 2686976~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:31.098 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819772+0000", 2024-09-04T17:47:31.098 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.207793685999999, 2024-09-04T17:47:31.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042515052999999997, 2024-09-04T17:47:31.098 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:31.098 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:31.098 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:31.098 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:31.098 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:31.098 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:31.098 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61140 2024-09-04T17:47:31.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.099 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:31.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:31.099 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819772+0000", 2024-09-04T17:47:31.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:31.099 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819772+0000", 2024-09-04T17:47:31.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:31.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819773+0000", 2024-09-04T17:47:31.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2900000000000003e-07 2024-09-04T17:47:31.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:31.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819775+0000", 2024-09-04T17:47:31.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3309999999999998e-06 2024-09-04T17:47:31.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:31.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819781+0000", 2024-09-04T17:47:31.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.981e-06 2024-09-04T17:47:31.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:31.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819786+0000", 2024-09-04T17:47:31.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5719999999999996e-06 2024-09-04T17:47:31.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:31.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.861966+0000", 2024-09-04T17:47:31.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042179541000000001 2024-09-04T17:47:31.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:31.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862038+0000", 2024-09-04T17:47:31.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2719000000000006e-05 2024-09-04T17:47:31.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:31.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862288+0000", 2024-09-04T17:47:31.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024928 2024-09-04T17:47:31.102 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.102 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:31.102 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.103 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61141 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 2793472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:31.103 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819909+0000", 2024-09-04T17:47:31.103 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.207657542, 2024-09-04T17:47:31.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042786419999999999, 2024-09-04T17:47:31.103 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:31.103 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:31.103 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:31.103 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:31.103 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:31.103 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:31.103 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61141 2024-09-04T17:47:31.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.103 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:31.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.104 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:31.104 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819909+0000", 2024-09-04T17:47:31.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.104 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:31.104 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819909+0000", 2024-09-04T17:47:31.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.104 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:31.104 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819909+0000", 2024-09-04T17:47:31.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6300000000000005e-07 2024-09-04T17:47:31.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.105 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:31.105 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819913+0000", 2024-09-04T17:47:31.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4189999999999998e-06 2024-09-04T17:47:31.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.105 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:31.105 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819920+0000", 2024-09-04T17:47:31.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8619999999999997e-06 2024-09-04T17:47:31.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:31.106 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819926+0000", 2024-09-04T17:47:31.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9399999999999999e-06 2024-09-04T17:47:31.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:31.106 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862331+0000", 2024-09-04T17:47:31.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042405348000000002 2024-09-04T17:47:31.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:31.106 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862410+0000", 2024-09-04T17:47:31.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9078000000000004e-05 2024-09-04T17:47:31.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:31.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862695+0000", 2024-09-04T17:47:31.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028510999999999999 2024-09-04T17:47:31.107 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.107 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:31.107 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.107 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61142 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 2932736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:31.107 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819934+0000", 2024-09-04T17:47:31.108 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.207631924000001, 2024-09-04T17:47:31.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043101985000000002, 2024-09-04T17:47:31.108 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:31.108 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:31.108 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:31.108 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:31.108 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:31.108 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:31.108 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61142 2024-09-04T17:47:31.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.108 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:31.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:31.109 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819934+0000", 2024-09-04T17:47:31.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.109 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:31.109 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819934+0000", 2024-09-04T17:47:31.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.109 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:31.109 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819935+0000", 2024-09-04T17:47:31.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2024-09-04T17:47:31.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.110 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.110 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:31.110 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819937+0000", 2024-09-04T17:47:31.110 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5770000000000001e-06 2024-09-04T17:47:31.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.110 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.110 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:31.110 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819943+0000", 2024-09-04T17:47:31.110 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7950000000000004e-06 2024-09-04T17:47:31.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.110 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.110 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:31.111 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819948+0000", 2024-09-04T17:47:31.111 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6779999999999999e-06 2024-09-04T17:47:31.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.111 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:31.111 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862739+0000", 2024-09-04T17:47:31.111 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042791454999999999 2024-09-04T17:47:31.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.111 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:31.111 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862824+0000", 2024-09-04T17:47:31.111 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4790999999999995e-05 2024-09-04T17:47:31.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.112 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.112 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:31.112 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.863036+0000", 2024-09-04T17:47:31.112 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021224899999999999 2024-09-04T17:47:31.112 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.112 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:31.112 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.112 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.112 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.112 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61143 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 3039232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:31.112 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819954+0000", 2024-09-04T17:47:31.112 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.207611805999999, 2024-09-04T17:47:31.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043485134000000002, 2024-09-04T17:47:31.113 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:31.113 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:31.113 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:31.113 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:31.113 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:31.113 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:31.113 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61143 2024-09-04T17:47:31.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.113 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:31.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:31.113 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819954+0000", 2024-09-04T17:47:31.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.114 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.114 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:31.114 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819954+0000", 2024-09-04T17:47:31.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.114 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.114 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:31.114 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819955+0000", 2024-09-04T17:47:31.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7300000000000002e-07 2024-09-04T17:47:31.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.115 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:31.115 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819957+0000", 2024-09-04T17:47:31.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.024e-06 2024-09-04T17:47:31.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.115 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:31.115 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819963+0000", 2024-09-04T17:47:31.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8109999999999999e-06 2024-09-04T17:47:31.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.115 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:31.115 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819967+0000", 2024-09-04T17:47:31.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6149999999999999e-06 2024-09-04T17:47:31.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:31.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.863082+0000", 2024-09-04T17:47:31.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043115078000000001 2024-09-04T17:47:31.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:31.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.863172+0000", 2024-09-04T17:47:31.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0181999999999998e-05 2024-09-04T17:47:31.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:31.117 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.863440+0000", 2024-09-04T17:47:31.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026705099999999997 2024-09-04T17:47:31.117 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.117 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:31.117 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.117 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:70015 3.6 3:6012e780:::rbd_data.11341aa245a3.000000000000005b:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-04T17:47:31.117 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:26.155497+0000", 2024-09-04T17:47:31.117 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.8720687490000003, 2024-09-04T17:47:31.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.049007124999999999, 2024-09-04T17:47:31.117 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:31.118 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:31.118 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:31.118 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:31.118 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:31.118 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:31.118 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 70015 2024-09-04T17:47:31.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.118 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:31.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.118 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:31.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155497+0000", 2024-09-04T17:47:31.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:31.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155497+0000", 2024-09-04T17:47:31.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:31.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155500+0000", 2024-09-04T17:47:31.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7070000000000001e-06 2024-09-04T17:47:31.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:31.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155513+0000", 2024-09-04T17:47:31.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2989e-05 2024-09-04T17:47:31.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.120 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:31.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155544+0000", 2024-09-04T17:47:31.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0747000000000002e-05 2024-09-04T17:47:31.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.120 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:31.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155560+0000", 2024-09-04T17:47:31.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5866e-05 2024-09-04T17:47:31.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.121 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:31.121 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155675+0000", 2024-09-04T17:47:31.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000114838 2024-09-04T17:47:31.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.121 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:31.121 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155803+0000", 2024-09-04T17:47:31.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012851799999999999 2024-09-04T17:47:31.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 1", 2024-09-04T17:47:31.122 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.163582+0000", 2024-09-04T17:47:31.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0077790960000000001 2024-09-04T17:47:31.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:31.122 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.204055+0000", 2024-09-04T17:47:31.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.040472517 2024-09-04T17:47:31.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:31.122 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.204195+0000", 2024-09-04T17:47:31.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014055200000000001 2024-09-04T17:47:31.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.123 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:31.123 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.204373+0000", 2024-09-04T17:47:31.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000177784 2024-09-04T17:47:31.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.123 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:31.123 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.204505+0000", 2024-09-04T17:47:31.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013151100000000001 2024-09-04T17:47:31.123 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.123 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:31.123 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.124 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:70434 3.12 3:4cd2342b:::rbd_data.11341aa245a3.0000000000000091:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-04T17:47:31.124 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:26.943124+0000", 2024-09-04T17:47:31.124 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.0844419670000001, 2024-09-04T17:47:31.124 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055011831999999997, 2024-09-04T17:47:31.124 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:31.124 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:31.124 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:31.124 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:31.124 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:31.124 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:31.124 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 70434 2024-09-04T17:47:31.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.124 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:31.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.125 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:31.125 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943124+0000", 2024-09-04T17:47:31.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.125 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:31.125 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943124+0000", 2024-09-04T17:47:31.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.125 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:31.125 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943128+0000", 2024-09-04T17:47:31.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8460000000000001e-06 2024-09-04T17:47:31.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.126 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:31.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943142+0000", 2024-09-04T17:47:31.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4089e-05 2024-09-04T17:47:31.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.126 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:31.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943177+0000", 2024-09-04T17:47:31.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5167999999999999e-05 2024-09-04T17:47:31.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.126 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:31.127 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943194+0000", 2024-09-04T17:47:31.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6235e-05 2024-09-04T17:47:31.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:31.127 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943286+0000", 2024-09-04T17:47:31.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2793000000000004e-05 2024-09-04T17:47:31.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:31.127 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943526+0000", 2024-09-04T17:47:31.128 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023950999999999999 2024-09-04T17:47:31.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.128 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 1", 2024-09-04T17:47:31.128 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.952871+0000", 2024-09-04T17:47:31.128 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0093453070000000006 2024-09-04T17:47:31.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.128 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:31.128 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.997968+0000", 2024-09-04T17:47:31.128 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045096613000000001 2024-09-04T17:47:31.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.129 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:31.129 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.998006+0000", 2024-09-04T17:47:31.129 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8257000000000003e-05 2024-09-04T17:47:31.129 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.129 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:31.129 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.998073+0000", 2024-09-04T17:47:31.129 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6803000000000004e-05 2024-09-04T17:47:31.129 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.129 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:31.129 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.998136+0000", 2024-09-04T17:47:31.129 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3211000000000002e-05 2024-09-04T17:47:31.130 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.130 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:31.130 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.130 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.130 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:31.130 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:31.131 INFO:teuthology.orchestra.run.smithi092.stdout:5198 write 0xdc12a06 thru 0xdc2204f (0xf64a bytes) 2024-09-04T17:47:31.132 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-04T17:47:31.134 INFO:teuthology.orchestra.run.smithi092.stdout:5200 read 0x2f339f3 thru 0x2f36395 (0x29a3 bytes) 2024-09-04T17:47:31.134 INFO:teuthology.orchestra.run.smithi092.stdout:5201 read 0x3ca256 thru 0x3d1b92 (0x793d bytes) 2024-09-04T17:47:31.134 INFO:teuthology.orchestra.run.smithi092.stdout:5203 read 0xba6676b thru 0xba6e019 (0x78af bytes) 2024-09-04T17:47:31.134 INFO:teuthology.orchestra.run.smithi092.stdout:5204 write 0x3dc2236 thru 0x3dcd106 (0xaed1 bytes) 2024-09-04T17:47:31.134 INFO:teuthology.orchestra.run.smithi092.stdout:5206 writesame 0xbe435f6 thru 0xbe4ac52 data_size 0x1(0x765d bytes) 2024-09-04T17:47:31.134 INFO:teuthology.orchestra.run.smithi092.stdout:5207 write 0xdf959b8 thru 0xdf9f9f4 (0xa03d bytes) 2024-09-04T17:47:31.134 INFO:teuthology.orchestra.run.smithi092.stdout:5208 writesame 0x2abf492 thru 0x2aca861 data_size 0x54(0xb3d0 bytes) 2024-09-04T17:47:31.134 INFO:teuthology.orchestra.run.smithi092.stdout:5209 punch from 0x8101fff to 0x810391f, (0x1920 bytes) 2024-09-04T17:47:31.134 INFO:teuthology.orchestra.run.smithi092.stdout:5210 read 0xb6f1734 thru 0xb6f9077 (0x7944 bytes) 2024-09-04T17:47:31.134 INFO:teuthology.orchestra.run.smithi092.stdout:5211 punch from 0x1a9b4b1 to 0x1aa1126, (0x5c75 bytes) 2024-09-04T17:47:31.134 INFO:teuthology.orchestra.run.smithi092.stdout:5212 writesame 0x6e61f06 thru 0x6e6ccc7 data_size 0x2(0xadc2 bytes) 2024-09-04T17:47:31.135 INFO:teuthology.orchestra.run.smithi092.stdout:5213 read 0x907fd14 thru 0x908980d (0x9afa bytes) 2024-09-04T17:47:31.135 INFO:teuthology.orchestra.run.smithi092.stdout:5214 write 0x1ca3526 thru 0x1cb1a01 (0xe4dc bytes) 2024-09-04T17:47:31.135 INFO:teuthology.orchestra.run.smithi092.stdout:5215 write 0xe1bea2d thru 0xe1c5cf9 (0x72cd bytes) 2024-09-04T17:47:31.135 INFO:teuthology.orchestra.run.smithi092.stdout:5216 punch from 0x97cafcd to 0x97d0152, (0x5185 bytes) 2024-09-04T17:47:31.135 INFO:teuthology.orchestra.run.smithi092.stdout:5217 writesame 0x8574494 thru 0x8580870 data_size 0x1d(0xc3dd bytes) 2024-09-04T17:47:31.135 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:31.135 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:31.135 INFO:teuthology.orchestra.run.smithi092.stdout:5220 write 0x8a39645 thru 0x8a4215d (0x8b19 bytes) 2024-09-04T17:47:31.135 INFO:teuthology.orchestra.run.smithi092.stdout:5222 read 0xd0e1e08 thru 0xd0e266b (0x864 bytes) 2024-09-04T17:47:31.135 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:31.129+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:31.136 INFO:teuthology.orchestra.run.smithi092.stdout:5223 trunc from 0xe1c5cfa to 0xc1ac477 2024-09-04T17:47:31.149 INFO:teuthology.orchestra.run.smithi092.stdout:5224 trunc from 0xc1ac477 to 0xc7e67ff 2024-09-04T17:47:31.158 INFO:teuthology.orchestra.run.smithi092.stdout:5225 trunc from 0xc7e67ff to 0xab25b0e 2024-09-04T17:47:31.175 INFO:teuthology.orchestra.run.smithi092.stdout:5229 read 0x90fb2ce thru 0x90ffc7d (0x49b0 bytes) 2024-09-04T17:47:31.178 INFO:teuthology.orchestra.run.smithi092.stdout:5232 trunc from 0xab25b0e to 0x91f7b12 2024-09-04T17:47:31.192 INFO:teuthology.orchestra.run.smithi092.stdout:5233 write 0x5e2a664 thru 0x5e2d437 (0x2dd4 bytes) 2024-09-04T17:47:31.196 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:31.196 INFO:teuthology.orchestra.run.smithi092.stdout:5235 read 0x7437da7 thru 0x744241a (0xa674 bytes) 2024-09-04T17:47:31.197 INFO:teuthology.orchestra.run.smithi092.stdout:5236 punch from 0x8dd22eb to 0x8dda8b7, (0x85cc bytes) 2024-09-04T17:47:31.197 INFO:teuthology.orchestra.run.smithi092.stdout:5237 write 0x84c641f thru 0x84c7b78 (0x175a bytes) 2024-09-04T17:47:31.199 INFO:teuthology.orchestra.run.smithi092.stdout:5238 punch from 0xb17b17 to 0xb1a1d1, (0x26ba bytes) 2024-09-04T17:47:31.199 INFO:teuthology.orchestra.run.smithi092.stdout:5239 read 0x663b640 thru 0x6642aab (0x746c bytes) 2024-09-04T17:47:31.199 INFO:teuthology.orchestra.run.smithi092.stdout:5240 punch from 0x7463258 to 0x74650d0, (0x1e78 bytes) 2024-09-04T17:47:31.200 INFO:teuthology.orchestra.run.smithi092.stdout:5241 write 0x8698815 thru 0x86a6c4d (0xe439 bytes) 2024-09-04T17:47:31.212 INFO:teuthology.orchestra.run.smithi092.stdout:5242 trunc from 0x91f7b12 to 0x281cc05 2024-09-04T17:47:31.231 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:31.229+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:31.250 INFO:teuthology.orchestra.run.smithi092.stdout:5243 write 0xb59b23c thru 0xb5a5117 (0x9edc bytes) 2024-09-04T17:47:31.256 INFO:teuthology.orchestra.run.smithi092.stdout:5244 read 0x2bb33ab thru 0x2bb54cd (0x2123 bytes) 2024-09-04T17:47:31.266 INFO:teuthology.orchestra.run.smithi092.stdout:5246 trunc from 0xb5a5118 to 0x5f5ce53 2024-09-04T17:47:31.268 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:31.268 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:47:31.268 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:47:31.268 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:31.283 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-04T17:47:31.283 INFO:teuthology.orchestra.run.smithi092.stdout:5248 read 0x35df620 thru 0x35e8baa (0x958b bytes) 2024-09-04T17:47:31.296 INFO:teuthology.orchestra.run.smithi092.stdout:5249 writesame 0x7fab826 thru 0x7fb1828 data_size 0x3(0x6003 bytes) 2024-09-04T17:47:31.296 INFO:teuthology.orchestra.run.smithi092.stdout:5254 read 0x40104ef thru 0x40159c4 (0x54d6 bytes) 2024-09-04T17:47:31.318 INFO:teuthology.orchestra.run.smithi092.stdout:5255 write 0xec3c06f thru 0xec4adbb (0xed4d bytes) 2024-09-04T17:47:31.325 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:31.325 INFO:teuthology.orchestra.run.smithi092.stdout:5257 write 0x29cc05d thru 0x29d9b59 (0xdafd bytes) 2024-09-04T17:47:31.332 INFO:teuthology.orchestra.run.smithi092.stdout:5258 read 0xd062252 thru 0xd062f8f (0xd3e bytes) 2024-09-04T17:47:31.332 INFO:teuthology.orchestra.run.smithi092.stdout:5261 write 0x81933d4 thru 0x81a3079 (0xfca6 bytes) 2024-09-04T17:47:31.332 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:31.329+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:31.340 INFO:teuthology.orchestra.run.smithi092.stdout:5263 write 0xb21f2b2 thru 0xb22883f (0x958e bytes) 2024-09-04T17:47:31.349 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:31.349 INFO:teuthology.orchestra.run.smithi092.stdout:5265 write 0xd2397e4 thru 0xd244ccf (0xb4ec bytes) 2024-09-04T17:47:31.357 INFO:teuthology.orchestra.run.smithi092.stdout:5267 punch from 0xd365a26 to 0xd371550, (0xbb2a bytes) 2024-09-04T17:47:31.357 INFO:teuthology.orchestra.run.smithi092.stdout:5268 read 0x376ccda thru 0x3776b53 (0x9e7a bytes) 2024-09-04T17:47:31.358 INFO:teuthology.orchestra.run.smithi092.stdout:5269 writesame 0x8a42418 thru 0x8a47836 data_size 0x3b(0x541f bytes) 2024-09-04T17:47:31.363 INFO:teuthology.orchestra.run.smithi092.stdout:5270 write 0xace0fda thru 0xacf088f (0xf8b6 bytes) 2024-09-04T17:47:31.370 INFO:teuthology.orchestra.run.smithi092.stdout:5271 write 0xe387879 thru 0xe3967fe (0xef86 bytes) 2024-09-04T17:47:31.377 INFO:teuthology.orchestra.run.smithi092.stdout:5272 write 0x7b4dcb9 thru 0x7b591aa (0xb4f2 bytes) 2024-09-04T17:47:31.386 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:31.386 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:31.386 INFO:teuthology.orchestra.run.smithi092.stdout:5278 flatten 2024-09-04T17:47:31.397 INFO:teuthology.orchestra.run.smithi092.stdout:5279 read 0x31d5552 thru 0x31dcbbb (0x766a bytes) 2024-09-04T17:47:31.399 INFO:teuthology.orchestra.run.smithi092.stdout:5280 read 0x6fb49c8 thru 0x6fbf705 (0xad3e bytes) 2024-09-04T17:47:31.399 INFO:teuthology.orchestra.run.smithi092.stdout:5281 read 0xb3b2d02 thru 0xb3becf0 (0xbfef bytes) 2024-09-04T17:47:31.400 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:31.400 INFO:teuthology.orchestra.run.smithi092.stdout:5283 write 0x1c6ba55 thru 0x1c737f5 (0x7da1 bytes) 2024-09-04T17:47:31.403 INFO:teuthology.orchestra.run.smithi092.stdout:5284 write 0x361c2da thru 0x3622648 (0x636f bytes) 2024-09-04T17:47:31.408 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:31.408 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:47:31.408 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:47:31.408 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:47:31.408 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:31.412 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:31.412 INFO:teuthology.orchestra.run.smithi092.stdout:5286 write 0xab58d13 thru 0xab6659d (0xd88b bytes) 2024-09-04T17:47:31.415 INFO:teuthology.orchestra.run.smithi092.stdout:5288 write 0xa51983 thru 0xa5e4ce (0xcb4c bytes) 2024-09-04T17:47:31.419 INFO:teuthology.orchestra.run.smithi092.stdout:5290 read 0x2aead03 thru 0x2aef5c3 (0x48c1 bytes) 2024-09-04T17:47:31.419 INFO:teuthology.orchestra.run.smithi092.stdout:5292 read 0xeba5611 thru 0xebab05c (0x5a4c bytes) 2024-09-04T17:47:31.423 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-04T17:47:31.430 INFO:teuthology.orchestra.run.smithi092.stdout:5296 trunc from 0xec4adbc to 0xa149841 2024-09-04T17:47:31.433 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:31.432+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:31.472 INFO:teuthology.orchestra.run.smithi092.stdout:5297 trunc from 0xa149841 to 0x3491527 2024-09-04T17:47:31.502 INFO:teuthology.orchestra.run.smithi092.stdout:5299 read 0x1619ed5 thru 0x161c9cc (0x2af8 bytes) 2024-09-04T17:47:31.526 INFO:teuthology.orchestra.run.smithi092.stdout:5300 write 0x82f1c87 thru 0x82fba42 (0x9dbc bytes) 2024-09-04T17:47:31.534 INFO:teuthology.orchestra.run.smithi092.stdout:5302 write 0x670096 thru 0x67f9be (0xf929 bytes) 2024-09-04T17:47:31.534 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:47:31.532+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:47:31.548 INFO:teuthology.orchestra.run.smithi092.stdout:5303 trunc from 0x82fba43 to 0x2c95d58 2024-09-04T17:47:31.563 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:31.563 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:47:31.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:47:31.563 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:47:31.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.563 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59950 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5365760~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:47:31.563 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654917+0000", 2024-09-04T17:47:31.563 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.905659983, 2024-09-04T17:47:31.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066822450000000005, 2024-09-04T17:47:31.563 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:31.564 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:31.564 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:31.564 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:31.564 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:31.564 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:31.564 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59950 2024-09-04T17:47:31.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.564 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:31.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:31.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654917+0000", 2024-09-04T17:47:31.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.565 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:31.565 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654917+0000", 2024-09-04T17:47:31.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.565 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:31.565 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654918+0000", 2024-09-04T17:47:31.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.29e-07 2024-09-04T17:47:31.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.565 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:31.566 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654919+0000", 2024-09-04T17:47:31.566 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2589999999999999e-06 2024-09-04T17:47:31.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.566 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:31.566 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654922+0000", 2024-09-04T17:47:31.566 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1350000000000001e-06 2024-09-04T17:47:31.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.566 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:31.566 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654925+0000", 2024-09-04T17:47:31.566 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1250000000000001e-06 2024-09-04T17:47:31.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:31.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721466+0000", 2024-09-04T17:47:31.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066540959999999996 2024-09-04T17:47:31.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:31.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721574+0000", 2024-09-04T17:47:31.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010747399999999999 2024-09-04T17:47:31.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:31.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721740+0000", 2024-09-04T17:47:31.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016626799999999999 2024-09-04T17:47:31.568 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.568 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:31.568 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.568 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59951 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5472256~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:47:31.568 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654929+0000", 2024-09-04T17:47:31.568 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.905648798000001, 2024-09-04T17:47:31.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067110534999999999, 2024-09-04T17:47:31.568 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:31.568 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:31.568 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:31.568 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:31.569 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:31.569 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:31.569 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59951 2024-09-04T17:47:31.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.569 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:31.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:31.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654929+0000", 2024-09-04T17:47:31.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:31.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654929+0000", 2024-09-04T17:47:31.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:31.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654929+0000", 2024-09-04T17:47:31.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5100000000000001e-07 2024-09-04T17:47:31.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:31.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654930+0000", 2024-09-04T17:47:31.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2440000000000001e-06 2024-09-04T17:47:31.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:31.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654933+0000", 2024-09-04T17:47:31.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1599999999999998e-06 2024-09-04T17:47:31.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:31.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654936+0000", 2024-09-04T17:47:31.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6560000000000001e-06 2024-09-04T17:47:31.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:31.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721789+0000", 2024-09-04T17:47:31.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066853250000000003 2024-09-04T17:47:31.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.572 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.572 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:31.572 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721885+0000", 2024-09-04T17:47:31.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5616000000000001e-05 2024-09-04T17:47:31.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.572 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.572 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:31.572 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722039+0000", 2024-09-04T17:47:31.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000154258 2024-09-04T17:47:31.572 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.572 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:31.572 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.573 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59952 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5505024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:47:31.573 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654939+0000", 2024-09-04T17:47:31.573 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.905637921, 2024-09-04T17:47:31.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067433742000000005, 2024-09-04T17:47:31.573 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:31.573 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:31.573 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:31.573 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:31.573 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:31.573 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:31.573 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59952 2024-09-04T17:47:31.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.574 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:31.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:31.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654939+0000", 2024-09-04T17:47:31.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:31.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654939+0000", 2024-09-04T17:47:31.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:31.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654940+0000", 2024-09-04T17:47:31.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3300000000000001e-07 2024-09-04T17:47:31.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:31.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654941+0000", 2024-09-04T17:47:31.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1570000000000001e-06 2024-09-04T17:47:31.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:31.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654944+0000", 2024-09-04T17:47:31.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1319999999999998e-06 2024-09-04T17:47:31.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.576 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:31.576 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654947+0000", 2024-09-04T17:47:31.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5969999999999999e-06 2024-09-04T17:47:31.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.576 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:31.576 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722089+0000", 2024-09-04T17:47:31.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067142347000000005 2024-09-04T17:47:31.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.576 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:31.576 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722201+0000", 2024-09-04T17:47:31.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011235600000000001 2024-09-04T17:47:31.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:31.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722373+0000", 2024-09-04T17:47:31.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017191999999999999 2024-09-04T17:47:31.577 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.577 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:31.577 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.577 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59953 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5611520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:47:31.577 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654950+0000", 2024-09-04T17:47:31.578 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.905627281000001, 2024-09-04T17:47:31.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067601852000000004, 2024-09-04T17:47:31.578 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:31.578 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:31.578 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:31.578 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:31.578 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:31.578 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:31.578 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59953 2024-09-04T17:47:31.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.578 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:31.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:31.578 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654950+0000", 2024-09-04T17:47:31.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:31.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654950+0000", 2024-09-04T17:47:31.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:31.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654950+0000", 2024-09-04T17:47:31.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2499999999999999e-07 2024-09-04T17:47:31.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:31.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654954+0000", 2024-09-04T17:47:31.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.287e-06 2024-09-04T17:47:31.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:31.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654962+0000", 2024-09-04T17:47:31.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4230000000000005e-06 2024-09-04T17:47:31.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:31.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654966+0000", 2024-09-04T17:47:31.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6449999999999998e-06 2024-09-04T17:47:31.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.581 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:31.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722417+0000", 2024-09-04T17:47:31.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067451188999999995 2024-09-04T17:47:31.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.581 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:31.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722467+0000", 2024-09-04T17:47:31.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0556999999999997e-05 2024-09-04T17:47:31.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.581 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:31.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722552+0000", 2024-09-04T17:47:31.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4525999999999997e-05 2024-09-04T17:47:31.582 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.582 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:31.582 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.582 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59954 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5718016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:47:31.582 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654970+0000", 2024-09-04T17:47:31.582 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.90560752, 2024-09-04T17:47:31.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067874285000000006, 2024-09-04T17:47:31.582 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:31.583 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:31.583 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:31.583 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:31.583 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:31.583 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:31.583 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59954 2024-09-04T17:47:31.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.583 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:31.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.583 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:31.583 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654970+0000", 2024-09-04T17:47:31.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:31.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654970+0000", 2024-09-04T17:47:31.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:31.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654970+0000", 2024-09-04T17:47:31.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5899999999999998e-07 2024-09-04T17:47:31.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:31.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654971+0000", 2024-09-04T17:47:31.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2929999999999999e-06 2024-09-04T17:47:31.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:31.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654975+0000", 2024-09-04T17:47:31.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4699999999999998e-06 2024-09-04T17:47:31.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:31.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654978+0000", 2024-09-04T17:47:31.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7930000000000002e-06 2024-09-04T17:47:31.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:31.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722600+0000", 2024-09-04T17:47:31.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067622229000000006 2024-09-04T17:47:31.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:31.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722689+0000", 2024-09-04T17:47:31.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9511000000000004e-05 2024-09-04T17:47:31.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:31.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722844+0000", 2024-09-04T17:47:31.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015473 2024-09-04T17:47:31.586 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.586 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:31.586 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.587 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59955 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5857280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:47:31.587 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654981+0000", 2024-09-04T17:47:31.587 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.905596169999999, 2024-09-04T17:47:31.587 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068052360000000006, 2024-09-04T17:47:31.587 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:31.587 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:31.587 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:31.587 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:31.587 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:31.587 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:31.587 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59955 2024-09-04T17:47:31.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.588 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:31.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:31.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654981+0000", 2024-09-04T17:47:31.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:31.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654981+0000", 2024-09-04T17:47:31.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:31.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654981+0000", 2024-09-04T17:47:31.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2399999999999999e-07 2024-09-04T17:47:31.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:31.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654985+0000", 2024-09-04T17:47:31.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0980000000000004e-06 2024-09-04T17:47:31.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:31.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654990+0000", 2024-09-04T17:47:31.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3429999999999998e-06 2024-09-04T17:47:31.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:31.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654993+0000", 2024-09-04T17:47:31.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8700000000000001e-06 2024-09-04T17:47:31.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:31.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722888+0000", 2024-09-04T17:47:31.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067895239999999996 2024-09-04T17:47:31.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:31.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722948+0000", 2024-09-04T17:47:31.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9750999999999998e-05 2024-09-04T17:47:31.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.591 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:31.591 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.723034+0000", 2024-09-04T17:47:31.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5834000000000005e-05 2024-09-04T17:47:31.591 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.591 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:31.591 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.591 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59956 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5963776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:47:31.591 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654998+0000", 2024-09-04T17:47:31.592 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.905579640999999, 2024-09-04T17:47:31.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068359518999999994, 2024-09-04T17:47:31.592 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:31.592 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:31.592 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:31.592 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:31.592 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:31.592 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:31.592 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59956 2024-09-04T17:47:31.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.592 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:31.592 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:31.593 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654998+0000", 2024-09-04T17:47:31.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:31.593 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654998+0000", 2024-09-04T17:47:31.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:31.593 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654998+0000", 2024-09-04T17:47:31.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4299999999999999e-07 2024-09-04T17:47:31.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.594 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:31.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654999+0000", 2024-09-04T17:47:31.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1799999999999999e-06 2024-09-04T17:47:31.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.594 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:31.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.655002+0000", 2024-09-04T17:47:31.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3639999999999999e-06 2024-09-04T17:47:31.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.594 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:31.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.655005+0000", 2024-09-04T17:47:31.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7970000000000001e-06 2024-09-04T17:47:31.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:31.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.723085+0000", 2024-09-04T17:47:31.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068080229000000006 2024-09-04T17:47:31.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:31.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.723197+0000", 2024-09-04T17:47:31.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000111344 2024-09-04T17:47:31.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.596 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:31.596 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.723357+0000", 2024-09-04T17:47:31.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016036199999999999 2024-09-04T17:47:31.596 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.596 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:31.596 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.596 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61082 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 2654208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:31.596 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.822664+0000", 2024-09-04T17:47:31.596 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.737913253999999, 2024-09-04T17:47:31.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067072242000000004, 2024-09-04T17:47:31.597 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:31.597 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:31.597 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:31.597 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:31.597 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:31.597 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:31.597 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61082 2024-09-04T17:47:31.597 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.597 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:31.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.597 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:31.597 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822664+0000", 2024-09-04T17:47:31.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:31.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822664+0000", 2024-09-04T17:47:31.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:31.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822664+0000", 2024-09-04T17:47:31.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6600000000000003e-07 2024-09-04T17:47:31.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:31.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822666+0000", 2024-09-04T17:47:31.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.536e-06 2024-09-04T17:47:31.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:31.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822669+0000", 2024-09-04T17:47:31.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3759999999999999e-06 2024-09-04T17:47:31.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:31.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822672+0000", 2024-09-04T17:47:31.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8109999999999999e-06 2024-09-04T17:47:31.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:31.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889263+0000", 2024-09-04T17:47:31.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066591259999999999 2024-09-04T17:47:31.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:31.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889369+0000", 2024-09-04T17:47:31.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000105362 2024-09-04T17:47:31.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:31.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889736+0000", 2024-09-04T17:47:31.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00036763099999999999 2024-09-04T17:47:31.601 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.601 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:31.601 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.601 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61083 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 2760704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:31.601 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.822678+0000", 2024-09-04T17:47:31.601 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.737899519999999, 2024-09-04T17:47:31.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067475996999999996, 2024-09-04T17:47:31.601 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:31.601 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:31.602 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:31.602 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:31.602 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:31.602 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:31.602 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61083 2024-09-04T17:47:31.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.602 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:31.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:31.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822678+0000", 2024-09-04T17:47:31.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:31.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822678+0000", 2024-09-04T17:47:31.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:31.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822725+0000", 2024-09-04T17:47:31.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7429000000000003e-05 2024-09-04T17:47:31.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:31.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822728+0000", 2024-09-04T17:47:31.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2140000000000001e-06 2024-09-04T17:47:31.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:31.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822737+0000", 2024-09-04T17:47:31.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4370000000000007e-06 2024-09-04T17:47:31.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:31.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822743+0000", 2024-09-04T17:47:31.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7520000000000002e-06 2024-09-04T17:47:31.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:31.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889785+0000", 2024-09-04T17:47:31.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067042265000000004 2024-09-04T17:47:31.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:31.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889840+0000", 2024-09-04T17:47:31.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5254000000000003e-05 2024-09-04T17:47:31.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:31.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.890154+0000", 2024-09-04T17:47:31.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000313646 2024-09-04T17:47:31.605 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.606 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:31.606 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.606 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61084 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 2867200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:31.606 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.822753+0000", 2024-09-04T17:47:31.606 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.737824583999998, 2024-09-04T17:47:31.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068497161000000001, 2024-09-04T17:47:31.606 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:31.606 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:31.606 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:31.606 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:31.606 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:31.607 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:31.607 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61084 2024-09-04T17:47:31.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.607 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:31.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:31.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822753+0000", 2024-09-04T17:47:31.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:31.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822753+0000", 2024-09-04T17:47:31.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:31.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822753+0000", 2024-09-04T17:47:31.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9100000000000003e-07 2024-09-04T17:47:31.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:31.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822756+0000", 2024-09-04T17:47:31.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.678e-06 2024-09-04T17:47:31.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:31.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822763+0000", 2024-09-04T17:47:31.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8220000000000001e-06 2024-09-04T17:47:31.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:31.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822768+0000", 2024-09-04T17:47:31.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1930000000000001e-06 2024-09-04T17:47:31.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:31.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.890246+0000", 2024-09-04T17:47:31.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067477996999999998 2024-09-04T17:47:31.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:31.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.890555+0000", 2024-09-04T17:47:31.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000309143 2024-09-04T17:47:31.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:31.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.891250+0000", 2024-09-04T17:47:31.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00069463700000000001 2024-09-04T17:47:31.610 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.610 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:31.610 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.610 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61285 3.8 3:117cfeb9:::rbd_data.11341dfde0c5.0000000000000006:1f [read 2760704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:31.611 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835200+0000", 2024-09-04T17:47:31.611 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.725377516999998, 2024-09-04T17:47:31.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066899803999999993, 2024-09-04T17:47:31.611 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:31.611 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:31.611 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:31.611 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:31.611 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:31.611 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:31.611 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61285 2024-09-04T17:47:31.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.611 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:31.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:31.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835200+0000", 2024-09-04T17:47:31.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:31.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835200+0000", 2024-09-04T17:47:31.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:31.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835200+0000", 2024-09-04T17:47:31.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0899999999999997e-07 2024-09-04T17:47:31.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.613 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:31.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835206+0000", 2024-09-04T17:47:31.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3280000000000002e-06 2024-09-04T17:47:31.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.613 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:31.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835211+0000", 2024-09-04T17:47:31.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9230000000000001e-06 2024-09-04T17:47:31.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:31.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835216+0000", 2024-09-04T17:47:31.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9019999999999998e-06 2024-09-04T17:47:31.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:31.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.901719+0000", 2024-09-04T17:47:31.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066502981000000003 2024-09-04T17:47:31.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:31.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.901813+0000", 2024-09-04T17:47:31.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3905000000000003e-05 2024-09-04T17:47:31.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:31.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902100+0000", 2024-09-04T17:47:31.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028645599999999998 2024-09-04T17:47:31.615 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.615 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:31.615 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.615 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61286 3.8 3:117cfeb9:::rbd_data.11341dfde0c5.0000000000000006:1f [read 2867200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:31.615 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835244+0000", 2024-09-04T17:47:31.615 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.725332898000001, 2024-09-04T17:47:31.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067143215000000006, 2024-09-04T17:47:31.616 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:31.616 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:31.616 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:31.616 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:31.616 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:31.616 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:31.616 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61286 2024-09-04T17:47:31.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.616 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:31.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:31.617 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835244+0000", 2024-09-04T17:47:31.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.617 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:31.617 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835244+0000", 2024-09-04T17:47:31.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.617 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:31.617 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835245+0000", 2024-09-04T17:47:31.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8400000000000004e-07 2024-09-04T17:47:31.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.618 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:31.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835248+0000", 2024-09-04T17:47:31.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6850000000000001e-06 2024-09-04T17:47:31.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.618 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:31.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835254+0000", 2024-09-04T17:47:31.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.494e-06 2024-09-04T17:47:31.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.618 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:31.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835260+0000", 2024-09-04T17:47:31.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0580000000000003e-06 2024-09-04T17:47:31.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:31.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902119+0000", 2024-09-04T17:47:31.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066858318 2024-09-04T17:47:31.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:31.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902163+0000", 2024-09-04T17:47:31.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4753000000000003e-05 2024-09-04T17:47:31.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:31.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902388+0000", 2024-09-04T17:47:31.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022422300000000001 2024-09-04T17:47:31.620 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.620 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:31.620 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.620 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61315 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 98304~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:31.620 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835269+0000", 2024-09-04T17:47:31.620 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.725308106, 2024-09-04T17:47:31.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067329396, 2024-09-04T17:47:31.621 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:31.621 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:31.621 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:31.621 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:31.621 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:31.621 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:31.621 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61315 2024-09-04T17:47:31.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.621 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:31.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:31.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835269+0000", 2024-09-04T17:47:31.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.622 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:31.622 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835269+0000", 2024-09-04T17:47:31.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.622 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:31.622 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835270+0000", 2024-09-04T17:47:31.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6599999999999996e-07 2024-09-04T17:47:31.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.622 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:31.623 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835272+0000", 2024-09-04T17:47:31.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.261e-06 2024-09-04T17:47:31.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.623 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:31.623 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835278+0000", 2024-09-04T17:47:31.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.443e-06 2024-09-04T17:47:31.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.623 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:31.623 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835284+0000", 2024-09-04T17:47:31.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3290000000000001e-06 2024-09-04T17:47:31.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:31.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902147+0000", 2024-09-04T17:47:31.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066862695999999999 2024-09-04T17:47:31.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:31.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902248+0000", 2024-09-04T17:47:31.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000101501 2024-09-04T17:47:31.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:31.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902599+0000", 2024-09-04T17:47:31.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00035060000000000001 2024-09-04T17:47:31.625 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.625 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:31.625 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.625 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61316 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 204800~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:31.625 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835291+0000", 2024-09-04T17:47:31.625 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.725286087000001, 2024-09-04T17:47:31.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067632052999999998, 2024-09-04T17:47:31.625 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:31.625 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:31.625 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:31.626 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:31.626 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:31.626 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:31.626 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61316 2024-09-04T17:47:31.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.626 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:31.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:31.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835291+0000", 2024-09-04T17:47:31.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:31.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835291+0000", 2024-09-04T17:47:31.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:31.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835292+0000", 2024-09-04T17:47:31.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1899999999999998e-07 2024-09-04T17:47:31.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:31.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835294+0000", 2024-09-04T17:47:31.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0880000000000002e-06 2024-09-04T17:47:31.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:31.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835300+0000", 2024-09-04T17:47:31.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.224e-06 2024-09-04T17:47:31.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:31.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835305+0000", 2024-09-04T17:47:31.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0730000000000004e-06 2024-09-04T17:47:31.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:31.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902632+0000", 2024-09-04T17:47:31.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067327322999999994 2024-09-04T17:47:31.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:31.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902688+0000", 2024-09-04T17:47:31.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5177999999999999e-05 2024-09-04T17:47:31.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:31.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902923+0000", 2024-09-04T17:47:31.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023574800000000001 2024-09-04T17:47:31.629 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.629 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:31.629 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.630 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61317 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 344064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:31.630 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835312+0000", 2024-09-04T17:47:31.630 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.725265083, 2024-09-04T17:47:31.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067918511000000001, 2024-09-04T17:47:31.630 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:31.630 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:31.630 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:31.630 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:31.630 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:31.630 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:31.630 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61317 2024-09-04T17:47:31.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.631 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:31.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:31.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835312+0000", 2024-09-04T17:47:31.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:31.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835312+0000", 2024-09-04T17:47:31.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:31.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835313+0000", 2024-09-04T17:47:31.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9000000000000002e-07 2024-09-04T17:47:31.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:31.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835315+0000", 2024-09-04T17:47:31.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1689999999999999e-06 2024-09-04T17:47:31.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:31.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835321+0000", 2024-09-04T17:47:31.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0619999999999997e-06 2024-09-04T17:47:31.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:31.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835326+0000", 2024-09-04T17:47:31.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0390000000000004e-06 2024-09-04T17:47:31.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:31.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902967+0000", 2024-09-04T17:47:31.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067641520999999996 2024-09-04T17:47:31.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:31.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903067+0000", 2024-09-04T17:47:31.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9426999999999995e-05 2024-09-04T17:47:31.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:31.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903231+0000", 2024-09-04T17:47:31.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016390299999999999 2024-09-04T17:47:31.634 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.634 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:31.634 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.634 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61318 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 450560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:31.634 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835334+0000", 2024-09-04T17:47:31.634 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.725243566, 2024-09-04T17:47:31.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068091771999999995, 2024-09-04T17:47:31.634 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:31.634 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:31.635 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:31.635 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:31.635 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:31.635 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:31.635 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61318 2024-09-04T17:47:31.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.635 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:31.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:31.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835334+0000", 2024-09-04T17:47:31.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:31.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835334+0000", 2024-09-04T17:47:31.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:31.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835334+0000", 2024-09-04T17:47:31.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5699999999999998e-07 2024-09-04T17:47:31.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:31.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835337+0000", 2024-09-04T17:47:31.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3889999999999999e-06 2024-09-04T17:47:31.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:31.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835343+0000", 2024-09-04T17:47:31.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1809999999999998e-06 2024-09-04T17:47:31.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:31.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835348+0000", 2024-09-04T17:47:31.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3170000000000001e-06 2024-09-04T17:47:31.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:31.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903270+0000", 2024-09-04T17:47:31.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067922191000000007 2024-09-04T17:47:31.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:31.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903321+0000", 2024-09-04T17:47:31.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0889e-05 2024-09-04T17:47:31.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:31.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903426+0000", 2024-09-04T17:47:31.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104348 2024-09-04T17:47:31.638 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.638 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:31.638 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.639 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61319 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 589824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:31.639 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835357+0000", 2024-09-04T17:47:31.639 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.725220749000002, 2024-09-04T17:47:31.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068372535999999998, 2024-09-04T17:47:31.639 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:31.639 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:31.639 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:31.639 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:31.639 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:31.639 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:31.639 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61319 2024-09-04T17:47:31.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.639 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:31.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:31.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835357+0000", 2024-09-04T17:47:31.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:31.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835357+0000", 2024-09-04T17:47:31.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:31.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835357+0000", 2024-09-04T17:47:31.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.32e-07 2024-09-04T17:47:31.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:31.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835359+0000", 2024-09-04T17:47:31.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1660000000000001e-06 2024-09-04T17:47:31.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:31.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835366+0000", 2024-09-04T17:47:31.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3550000000000001e-06 2024-09-04T17:47:31.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:31.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835371+0000", 2024-09-04T17:47:31.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1050000000000003e-06 2024-09-04T17:47:31.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:31.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903471+0000", 2024-09-04T17:47:31.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068100451000000006 2024-09-04T17:47:31.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:31.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903573+0000", 2024-09-04T17:47:31.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000101691 2024-09-04T17:47:31.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:31.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903729+0000", 2024-09-04T17:47:31.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000156336 2024-09-04T17:47:31.643 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.643 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:31.643 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.643 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61320 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:31.643 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835380+0000", 2024-09-04T17:47:31.643 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.725196984, 2024-09-04T17:47:31.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068543058000000004, 2024-09-04T17:47:31.644 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:31.644 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:31.644 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:31.644 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:31.644 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:31.644 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:31.644 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61320 2024-09-04T17:47:31.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.644 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:31.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.644 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:31.644 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835380+0000", 2024-09-04T17:47:31.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:31.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835380+0000", 2024-09-04T17:47:31.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:31.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835381+0000", 2024-09-04T17:47:31.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9999999999999998e-07 2024-09-04T17:47:31.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:31.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835383+0000", 2024-09-04T17:47:31.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2230000000000001e-06 2024-09-04T17:47:31.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:31.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835390+0000", 2024-09-04T17:47:31.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.34e-06 2024-09-04T17:47:31.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:31.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835396+0000", 2024-09-04T17:47:31.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9830000000000001e-06 2024-09-04T17:47:31.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.647 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:31.647 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903765+0000", 2024-09-04T17:47:31.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068368441000000002 2024-09-04T17:47:31.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.648 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:31.648 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903835+0000", 2024-09-04T17:47:31.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0631999999999997e-05 2024-09-04T17:47:31.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.648 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:31.648 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903923+0000", 2024-09-04T17:47:31.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8039000000000002e-05 2024-09-04T17:47:31.648 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.648 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:31.648 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.649 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61321 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 835584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:31.649 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835404+0000", 2024-09-04T17:47:31.649 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.725172924999999, 2024-09-04T17:47:31.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068911043000000005, 2024-09-04T17:47:31.649 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:31.649 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:31.649 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:31.649 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:31.649 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:31.649 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:31.649 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61321 2024-09-04T17:47:31.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.650 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:31.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:31.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835404+0000", 2024-09-04T17:47:31.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:31.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835404+0000", 2024-09-04T17:47:31.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:31.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835405+0000", 2024-09-04T17:47:31.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.89e-07 2024-09-04T17:47:31.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:31.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835407+0000", 2024-09-04T17:47:31.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4439999999999998e-06 2024-09-04T17:47:31.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:31.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835414+0000", 2024-09-04T17:47:31.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4930000000000003e-06 2024-09-04T17:47:31.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.652 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:31.652 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835419+0000", 2024-09-04T17:47:31.652 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8919999999999999e-06 2024-09-04T17:47:31.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.652 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:31.652 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903957+0000", 2024-09-04T17:47:31.652 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068538500000000002 2024-09-04T17:47:31.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:31.653 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904019+0000", 2024-09-04T17:47:31.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1419000000000003e-05 2024-09-04T17:47:31.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:31.653 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904315+0000", 2024-09-04T17:47:31.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00029680599999999999 2024-09-04T17:47:31.653 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.653 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:31.653 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.653 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61322 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:31.654 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835427+0000", 2024-09-04T17:47:31.654 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.725150539000001, 2024-09-04T17:47:31.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.069550138999999997, 2024-09-04T17:47:31.654 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:31.654 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:31.654 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:31.654 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:31.654 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:31.654 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:31.654 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61322 2024-09-04T17:47:31.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.654 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:31.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:31.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835427+0000", 2024-09-04T17:47:31.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:31.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835427+0000", 2024-09-04T17:47:31.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:31.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:31.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.842157+0000", 2024-09-04T17:47:31.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067298419999999998 2024-09-04T17:47:31.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:31.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.842163+0000", 2024-09-04T17:47:31.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2029999999999997e-06 2024-09-04T17:47:31.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:31.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.842175+0000", 2024-09-04T17:47:31.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2072000000000001e-05 2024-09-04T17:47:31.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:31.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.842183+0000", 2024-09-04T17:47:31.657 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3850000000000002e-06 2024-09-04T17:47:31.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.657 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:31.657 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904371+0000", 2024-09-04T17:47:31.657 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.062187754999999997 2024-09-04T17:47:31.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.657 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:31.657 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904467+0000", 2024-09-04T17:47:31.657 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6117000000000001e-05 2024-09-04T17:47:31.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:31.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:31.658 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:31.658 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904977+0000", 2024-09-04T17:47:31.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00050976500000000002 2024-09-04T17:47:31.658 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.658 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:31.658 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.658 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:31.658 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:31.658 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:31.659 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:31.659 INFO:teuthology.orchestra.run.smithi092.stdout:5308 writesame 0x1c35c04 thru 0x1c3fa99 data_size 0x2(0x9e96 bytes) 2024-09-04T17:47:31.660 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-04T17:47:31.661 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:31.661 INFO:teuthology.orchestra.run.smithi092.stdout:5310 read 0xbb507c thru 0xbb54c0 (0x445 bytes) 2024-09-04T17:47:31.661 INFO:teuthology.orchestra.run.smithi092.stdout:5313 read 0x2915bce thru 0x29215d3 (0xba06 bytes) 2024-09-04T17:47:31.661 INFO:teuthology.orchestra.run.smithi092.stdout:5314 trunc from 0x2c95d58 to 0x3839ea0 2024-09-04T17:47:31.661 INFO:teuthology.orchestra.run.smithi092.stdout:5315 write 0x68c65a0 thru 0x68caa6e (0x44cf bytes) 2024-09-04T17:47:31.661 INFO:teuthology.orchestra.run.smithi092.stdout:5316 write 0x4d894cb thru 0x4d8bbc9 (0x26ff bytes) 2024-09-04T17:47:31.662 INFO:teuthology.orchestra.run.smithi092.stdout:5317 trunc from 0x68caa6f to 0x97b0b9a 2024-09-04T17:47:31.662 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:31.657+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:31.678 INFO:teuthology.orchestra.run.smithi092.stdout:5319 trunc from 0x97b0b9a to 0x405e2dc 2024-09-04T17:47:31.699 INFO:teuthology.orchestra.run.smithi092.stdout:5321 read 0x2641e89 thru 0x264aceb (0x8e63 bytes) 2024-09-04T17:47:31.738 INFO:teuthology.orchestra.run.smithi092.stdout:5323 write 0xe059269 thru 0xe064bdf (0xb977 bytes) 2024-09-04T17:47:31.747 INFO:teuthology.orchestra.run.smithi092.stdout:5324 punch from 0x1896f68 to 0x18a3b1a, (0xcbb2 bytes) 2024-09-04T17:47:31.747 INFO:teuthology.orchestra.run.smithi092.stdout:5325 write 0x229a75a thru 0x229f55a (0x4e01 bytes) 2024-09-04T17:47:31.749 INFO:teuthology.orchestra.run.smithi092.stdout:5326 write 0x2d8a37b thru 0x2d99401 (0xf087 bytes) 2024-09-04T17:47:31.755 INFO:teuthology.orchestra.run.smithi092.stdout:5327 trunc from 0xe064be0 to 0xcd66375 2024-09-04T17:47:31.760 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:31.756+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:31.774 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:31.774 INFO:teuthology.orchestra.run.smithi092.stdout:5330 write 0x2a41dd1 thru 0x2a44e34 (0x3064 bytes) 2024-09-04T17:47:31.777 INFO:teuthology.orchestra.run.smithi092.stdout:5331 read 0x546dd6d thru 0x5479c6b (0xbeff bytes) 2024-09-04T17:47:31.777 INFO:teuthology.orchestra.run.smithi092.stdout:5332 write 0x4329c14 thru 0x43389aa (0xed97 bytes) 2024-09-04T17:47:31.784 INFO:teuthology.orchestra.run.smithi092.stdout:5334 read 0xb2c0b96 thru 0xb2c68b2 (0x5d1d bytes) 2024-09-04T17:47:31.792 INFO:teuthology.orchestra.run.smithi092.stdout:5335 trunc from 0xcd66375 to 0x9feb169 2024-09-04T17:47:31.802 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:31.802 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:47:31.802 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:47:31.802 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:31.803 INFO:teuthology.orchestra.run.smithi092.stdout:5337 writesame 0x85991cc thru 0x85a0c7e data_size 0x1(0x7ab3 bytes) 2024-09-04T17:47:31.819 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-04T17:47:31.825 INFO:teuthology.orchestra.run.smithi092.stdout:5339 read 0x8a5e1ed thru 0x8a61ec0 (0x3cd4 bytes) 2024-09-04T17:47:31.828 INFO:teuthology.orchestra.run.smithi092.stdout:5340 read 0x90dc41c thru 0x90eb823 (0xf408 bytes) 2024-09-04T17:47:31.828 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:31.828 INFO:teuthology.orchestra.run.smithi092.stdout:5344 write 0x7c5e909 thru 0x7c6963d (0xad35 bytes) 2024-09-04T17:47:31.838 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is set 2024-09-04T17:47:31.841 INFO:teuthology.orchestra.run.smithi092.stdout:5345 punch from 0x19d42e9 to 0x19dd4bd, (0x91d4 bytes) 2024-09-04T17:47:31.841 INFO:teuthology.orchestra.run.smithi092.stdout:5346 write 0x71ef0ef thru 0x71fa1da (0xb0ec bytes) 2024-09-04T17:47:31.846 INFO:teuthology.orchestra.run.smithi092.stdout:5347 write 0x76bcd93 thru 0x76be9c7 (0x1c35 bytes) 2024-09-04T17:47:31.847 INFO:teuthology.orchestra.run.smithi092.stdout:5348 write 0xe7e773 thru 0xe84fc0 (0x684e bytes) 2024-09-04T17:47:31.850 INFO:teuthology.orchestra.run.smithi092.stdout:5349 writesame 0xd4b05e thru 0xd4b296 data_size 0x1(0x239 bytes) 2024-09-04T17:47:31.857 INFO:teuthology.orchestra.run.smithi092.stdout:5351 trunc from 0x9feb169 to 0xcf01f88 2024-09-04T17:47:31.861 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:31.860+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:31.867 INFO:teuthology.orchestra.run.smithi092.stdout:5352 write 0xc04419a thru 0xc051be8 (0xda4f bytes) 2024-09-04T17:47:31.872 INFO:teuthology.orchestra.run.smithi092.stdout:5353 punch from 0x8b44e47 to 0x8b46bb3, (0x1d6c bytes) 2024-09-04T17:47:31.882 INFO:teuthology.orchestra.run.smithi092.stdout:5354 writesame 0xe2b4549 thru 0xe2c3b0a data_size 0x2(0xf5c2 bytes) 2024-09-04T17:47:31.883 INFO:teuthology.orchestra.run.smithi092.stdout:5355 read 0xc5d14e thru 0xc64fe4 (0x7e97 bytes) 2024-09-04T17:47:31.883 INFO:teuthology.orchestra.run.smithi092.stdout:5358 read 0x481d6d5 thru 0x4826997 (0x92c3 bytes) 2024-09-04T17:47:31.883 INFO:teuthology.orchestra.run.smithi092.stdout:5359 clone 36 order 20 su 131072 sc 12 2024-09-04T17:47:31.962 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:47:31.960+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:47:31.987 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:31.987 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:47:31.987 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:47:31.987 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:47:31.987 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:32.000 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-04T17:47:32.002 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-04T17:47:32.093 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:47:32.092+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:47:32.157 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:47:32.158 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:47:32.163 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:32.161+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:32.174 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:47:32.174 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:47:32.174 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-04T17:47:32.176 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:32.176 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:47:32.176 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:47:32.177 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:47:32.177 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.177 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:12722 3.1b 3:d8152af1:::rbd_data.1134ef3a5a7d.00000000000000bd:head [delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-04T17:47:32.177 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:16.421965+0000", 2024-09-04T17:47:32.177 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 75.752528084000005, 2024-09-04T17:47:32.178 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.890002811, 2024-09-04T17:47:32.178 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:32.178 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:32.178 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:32.178 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:32.179 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:32.179 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:32.179 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 12722 2024-09-04T17:47:32.179 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.179 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:32.179 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.179 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:32.179 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421965+0000", 2024-09-04T17:47:32.179 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:32.179 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.180 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.180 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:32.180 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421965+0000", 2024-09-04T17:47:32.180 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:32.180 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.180 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.180 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:32.180 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421966+0000", 2024-09-04T17:47:32.180 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.102e-06 2024-09-04T17:47:32.180 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.180 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.180 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:32.180 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421971+0000", 2024-09-04T17:47:32.181 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9989999999999999e-06 2024-09-04T17:47:32.181 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.181 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.181 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:32.181 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421981+0000", 2024-09-04T17:47:32.181 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.7380000000000006e-06 2024-09-04T17:47:32.181 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.181 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.181 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:32.181 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421989+0000", 2024-09-04T17:47:32.181 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.1890000000000007e-06 2024-09-04T17:47:32.181 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.181 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.182 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:32.182 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.422050+0000", 2024-09-04T17:47:32.182 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.0702000000000003e-05 2024-09-04T17:47:32.182 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.182 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.182 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for peered", 2024-09-04T17:47:32.182 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.422059+0000", 2024-09-04T17:47:32.182 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5440000000000004e-06 2024-09-04T17:47:32.182 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.182 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.182 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:32.182 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.308156+0000", 2024-09-04T17:47:32.182 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.88609676199999998 2024-09-04T17:47:32.182 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.183 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.183 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:32.183 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.308398+0000", 2024-09-04T17:47:32.183 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024215400000000001 2024-09-04T17:47:32.183 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.183 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.183 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 7", 2024-09-04T17:47:32.183 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.308795+0000", 2024-09-04T17:47:32.183 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000396704 2024-09-04T17:47:32.183 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.183 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.184 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:47:32.184 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.310831+0000", 2024-09-04T17:47:32.184 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0020364200000000002 2024-09-04T17:47:32.184 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.184 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.184 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:32.184 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.311818+0000", 2024-09-04T17:47:32.184 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00098649799999999993 2024-09-04T17:47:32.184 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.185 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.185 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:47:32.185 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.311905+0000", 2024-09-04T17:47:32.185 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7070999999999994e-05 2024-09-04T17:47:32.185 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.185 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.185 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:32.185 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.311968+0000", 2024-09-04T17:47:32.185 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.2928000000000005e-05 2024-09-04T17:47:32.185 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:32.185 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:32.185 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:32.185 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.186 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.186 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37359 3.f e58/24 3:f29e50a5:::rbd_data.1134c5a43465.0000000000000021:head v 58'135, mlcod=58'133)", 2024-09-04T17:47:32.186 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:47:32.186 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 52.557197479999999, 2024-09-04T17:47:32.186 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053483197000000003, 2024-09-04T17:47:32.186 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:32.186 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:32.186 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:32.186 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:32.186 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.186 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:32.186 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:47:32.186 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:32.187 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.187 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.187 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:32.187 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:47:32.187 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:32.187 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.187 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.187 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:32.187 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:47:32.187 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1699999999999997e-07 2024-09-04T17:47:32.187 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.187 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.187 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:32.188 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617338+0000", 2024-09-04T17:47:32.188 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.1168999999999998e-05 2024-09-04T17:47:32.188 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.188 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.188 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:32.188 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617351+0000", 2024-09-04T17:47:32.188 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3094e-05 2024-09-04T17:47:32.188 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.188 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.188 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:32.188 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617359+0000", 2024-09-04T17:47:32.188 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7970000000000006e-06 2024-09-04T17:47:32.188 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.189 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.189 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:32.189 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617417+0000", 2024-09-04T17:47:32.189 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7079e-05 2024-09-04T17:47:32.189 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.189 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.189 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:32.189 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617438+0000", 2024-09-04T17:47:32.190 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1654999999999998e-05 2024-09-04T17:47:32.190 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.190 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.190 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:47:32.190 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670738+0000", 2024-09-04T17:47:32.190 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053300142000000002 2024-09-04T17:47:32.190 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.190 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.190 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:32.190 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670779+0000", 2024-09-04T17:47:32.190 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0343999999999999e-05 2024-09-04T17:47:32.190 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:32.190 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:32.191 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:32.191 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.191 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.191 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37358 3.1d e58/24 3:bd3215f3:::rbd_data.1134c5a43465.0000000000000020:head v 58'91, mlcod=58'89)", 2024-09-04T17:47:32.191 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617502+0000", 2024-09-04T17:47:32.191 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 52.556990761999998, 2024-09-04T17:47:32.191 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053505641999999999, 2024-09-04T17:47:32.191 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:32.191 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:32.191 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:32.191 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:32.191 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.191 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:32.191 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617502+0000", 2024-09-04T17:47:32.192 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:32.192 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.192 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.192 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:32.192 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617502+0000", 2024-09-04T17:47:32.192 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:32.192 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.192 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.192 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:32.192 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617503+0000", 2024-09-04T17:47:32.192 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.184e-06 2024-09-04T17:47:32.192 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.193 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.193 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:32.193 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617526+0000", 2024-09-04T17:47:32.193 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2099999999999998e-05 2024-09-04T17:47:32.193 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.193 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.194 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:32.194 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617540+0000", 2024-09-04T17:47:32.194 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4741e-05 2024-09-04T17:47:32.194 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.194 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.194 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:32.195 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617550+0000", 2024-09-04T17:47:32.195 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4259999999999992e-06 2024-09-04T17:47:32.195 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.195 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.195 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:32.195 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617613+0000", 2024-09-04T17:47:32.195 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.3625000000000002e-05 2024-09-04T17:47:32.195 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.195 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.195 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:32.195 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617637+0000", 2024-09-04T17:47:32.195 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.3604999999999998e-05 2024-09-04T17:47:32.196 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.196 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.196 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:47:32.196 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670967+0000", 2024-09-04T17:47:32.196 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053329881000000003 2024-09-04T17:47:32.196 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.196 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.196 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:32.196 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671008+0000", 2024-09-04T17:47:32.196 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.108e-05 2024-09-04T17:47:32.196 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:32.196 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:32.196 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:32.197 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.197 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.197 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37361 3.1c e58/24 3:3c7f8976:::rbd_data.1134c5a43465.0000000000000023:head v 58'159, mlcod=58'157)", 2024-09-04T17:47:32.197 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617578+0000", 2024-09-04T17:47:32.197 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 52.556914912000003, 2024-09-04T17:47:32.197 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053990367999999997, 2024-09-04T17:47:32.197 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:32.197 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:32.197 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:32.197 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:32.197 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.198 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:32.198 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617578+0000", 2024-09-04T17:47:32.198 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:32.198 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.198 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.198 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:32.198 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617578+0000", 2024-09-04T17:47:32.198 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:32.198 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.198 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.198 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:32.198 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617579+0000", 2024-09-04T17:47:32.198 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7299999999999996e-07 2024-09-04T17:47:32.199 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.199 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.199 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:32.199 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617610+0000", 2024-09-04T17:47:32.199 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0951000000000002e-05 2024-09-04T17:47:32.199 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.199 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.199 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:32.199 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:47:32.199 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0002e-05 2024-09-04T17:47:32.199 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.200 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.200 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:32.200 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617626+0000", 2024-09-04T17:47:32.200 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6660000000000002e-06 2024-09-04T17:47:32.200 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.201 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.201 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:32.201 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670548+0000", 2024-09-04T17:47:32.201 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.052921943999999999 2024-09-04T17:47:32.201 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.201 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.201 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:32.201 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670574+0000", 2024-09-04T17:47:32.201 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5405000000000002e-05 2024-09-04T17:47:32.201 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.202 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.202 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:47:32.202 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671522+0000", 2024-09-04T17:47:32.202 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00094815299999999997 2024-09-04T17:47:32.202 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.202 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.202 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:32.202 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671568+0000", 2024-09-04T17:47:32.202 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6674000000000001e-05 2024-09-04T17:47:32.202 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:32.202 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:32.203 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:32.203 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.203 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.203 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37357 3.15 e58/49 3:a897312a:::rbd_data.1134c5a43465.000000000000001f:head v 58'117, mlcod=58'115)", 2024-09-04T17:47:32.203 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:47:32.203 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 52.556872574000003, 2024-09-04T17:47:32.203 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053734297, 2024-09-04T17:47:32.204 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:32.204 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:32.204 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:32.204 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:32.204 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.204 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:32.204 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:47:32.204 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:32.204 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.204 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.204 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:32.204 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:47:32.204 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:32.205 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.205 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.205 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:32.205 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617621+0000", 2024-09-04T17:47:32.205 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.7300000000000004e-07 2024-09-04T17:47:32.205 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.205 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.205 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:32.205 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617642+0000", 2024-09-04T17:47:32.205 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0185000000000001e-05 2024-09-04T17:47:32.205 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.205 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.205 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:32.206 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617654+0000", 2024-09-04T17:47:32.206 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2745000000000001e-05 2024-09-04T17:47:32.206 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.206 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.206 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:32.206 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617662+0000", 2024-09-04T17:47:32.206 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.0499999999999992e-06 2024-09-04T17:47:32.206 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.206 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.206 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:32.207 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617727+0000", 2024-09-04T17:47:32.207 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.4923000000000002e-05 2024-09-04T17:47:32.207 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.207 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.207 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:32.207 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617753+0000", 2024-09-04T17:47:32.207 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5899999999999999e-05 2024-09-04T17:47:32.207 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.207 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.207 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:47:32.207 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671298+0000", 2024-09-04T17:47:32.207 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053545276000000003 2024-09-04T17:47:32.208 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.208 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.208 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:32.208 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671355+0000", 2024-09-04T17:47:32.208 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6245000000000001e-05 2024-09-04T17:47:32.208 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:32.208 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:32.208 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:32.208 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.209 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.209 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44687 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:32.209 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253012+0000", 2024-09-04T17:47:32.209 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 41.921480506999998, 2024-09-04T17:47:32.209 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051128777, 2024-09-04T17:47:32.209 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:32.209 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:32.209 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:32.209 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:32.209 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:32.209 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:32.209 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44687 2024-09-04T17:47:32.209 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.210 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:32.210 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.210 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:32.210 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253012+0000", 2024-09-04T17:47:32.210 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:32.210 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.210 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.210 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:32.210 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253012+0000", 2024-09-04T17:47:32.210 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:32.210 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.210 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.210 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:32.210 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253014+0000", 2024-09-04T17:47:32.211 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.24e-06 2024-09-04T17:47:32.211 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.211 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.211 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:32.211 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253019+0000", 2024-09-04T17:47:32.211 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4569999999999996e-06 2024-09-04T17:47:32.211 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.211 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.212 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:32.212 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253035+0000", 2024-09-04T17:47:32.212 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6291999999999999e-05 2024-09-04T17:47:32.212 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.212 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.212 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:32.212 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253050+0000", 2024-09-04T17:47:32.212 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4423000000000001e-05 2024-09-04T17:47:32.213 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.213 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.213 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:32.213 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304115+0000", 2024-09-04T17:47:32.213 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051064778999999998 2024-09-04T17:47:32.213 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.213 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.213 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:32.213 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304141+0000", 2024-09-04T17:47:32.213 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6585999999999999e-05 2024-09-04T17:47:32.213 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:32.213 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:32.213 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:32.214 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.214 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.214 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44688 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:32.214 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253067+0000", 2024-09-04T17:47:32.214 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 41.921425954999997, 2024-09-04T17:47:32.214 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051116979, 2024-09-04T17:47:32.214 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:32.214 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:32.214 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:32.214 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:32.214 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:32.214 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:32.214 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44688 2024-09-04T17:47:32.215 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.215 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:32.215 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.215 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:32.215 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253067+0000", 2024-09-04T17:47:32.215 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:32.215 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.215 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.215 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:32.215 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253067+0000", 2024-09-04T17:47:32.215 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:32.215 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.215 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.216 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:32.216 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253068+0000", 2024-09-04T17:47:32.216 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0729999999999999e-06 2024-09-04T17:47:32.216 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.216 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.216 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:32.216 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253073+0000", 2024-09-04T17:47:32.216 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2259999999999996e-06 2024-09-04T17:47:32.216 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.216 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.216 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:32.216 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253091+0000", 2024-09-04T17:47:32.216 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7255999999999999e-05 2024-09-04T17:47:32.217 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.217 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.217 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:32.217 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253103+0000", 2024-09-04T17:47:32.217 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2632999999999999e-05 2024-09-04T17:47:32.217 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.217 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.217 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:32.217 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304173+0000", 2024-09-04T17:47:32.217 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051069537999999998 2024-09-04T17:47:32.217 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.217 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.217 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:32.218 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304184+0000", 2024-09-04T17:47:32.218 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1253000000000001e-05 2024-09-04T17:47:32.218 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:32.218 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:32.218 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:32.218 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.218 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.218 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44689 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:32.218 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253121+0000", 2024-09-04T17:47:32.218 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 41.921371817999997, 2024-09-04T17:47:32.218 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051192595, 2024-09-04T17:47:32.218 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:32.218 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:32.219 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:32.219 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:32.219 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:32.219 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:32.219 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44689 2024-09-04T17:47:32.219 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.219 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:32.219 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.219 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:32.219 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253121+0000", 2024-09-04T17:47:32.219 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:32.220 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.220 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.220 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:32.220 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253121+0000", 2024-09-04T17:47:32.220 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:32.220 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.221 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.221 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:32.221 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253122+0000", 2024-09-04T17:47:32.221 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.15e-06 2024-09-04T17:47:32.221 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.221 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.221 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:32.221 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253128+0000", 2024-09-04T17:47:32.221 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2569999999999998e-06 2024-09-04T17:47:32.221 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.222 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.222 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:32.222 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253145+0000", 2024-09-04T17:47:32.222 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7176e-05 2024-09-04T17:47:32.222 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.222 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.223 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:32.223 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253158+0000", 2024-09-04T17:47:32.223 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3139000000000001e-05 2024-09-04T17:47:32.223 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.223 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.223 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:32.223 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304287+0000", 2024-09-04T17:47:32.223 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051129149999999998 2024-09-04T17:47:32.223 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.223 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.224 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:32.224 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304314+0000", 2024-09-04T17:47:32.224 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6723000000000001e-05 2024-09-04T17:47:32.224 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:32.224 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:32.224 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:32.224 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.224 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.224 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44690 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:32.224 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253177+0000", 2024-09-04T17:47:32.224 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 41.921315692999997, 2024-09-04T17:47:32.224 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051169263, 2024-09-04T17:47:32.224 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:32.225 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:32.225 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:32.225 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:32.225 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:32.225 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:32.225 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44690 2024-09-04T17:47:32.225 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.225 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:32.225 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.225 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:32.225 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253177+0000", 2024-09-04T17:47:32.225 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:32.226 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.226 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.226 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:32.226 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253177+0000", 2024-09-04T17:47:32.226 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:32.226 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.226 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.227 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:32.227 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253178+0000", 2024-09-04T17:47:32.227 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1000000000000001e-06 2024-09-04T17:47:32.227 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.227 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.228 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:32.228 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253184+0000", 2024-09-04T17:47:32.228 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2089999999999996e-06 2024-09-04T17:47:32.228 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.228 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.229 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:32.229 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253200+0000", 2024-09-04T17:47:32.229 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6087e-05 2024-09-04T17:47:32.229 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.229 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.230 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:32.230 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253231+0000", 2024-09-04T17:47:32.230 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0874999999999998e-05 2024-09-04T17:47:32.230 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.230 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.230 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:32.230 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304337+0000", 2024-09-04T17:47:32.230 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051106430000000001 2024-09-04T17:47:32.230 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.230 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.231 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:32.231 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304347+0000", 2024-09-04T17:47:32.231 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5619999999999993e-06 2024-09-04T17:47:32.231 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:32.231 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:32.231 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:32.231 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.231 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.231 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44691 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:32.231 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253249+0000", 2024-09-04T17:47:32.231 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 41.921243658999998, 2024-09-04T17:47:32.231 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051195630999999998, 2024-09-04T17:47:32.231 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:32.232 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:32.232 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:32.232 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:32.232 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:32.232 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:32.232 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44691 2024-09-04T17:47:32.232 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.232 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:32.232 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.232 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:32.232 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253249+0000", 2024-09-04T17:47:32.233 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:32.233 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.233 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.233 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:32.233 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253249+0000", 2024-09-04T17:47:32.233 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:32.234 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.234 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.234 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:32.234 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253251+0000", 2024-09-04T17:47:32.234 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2330000000000001e-06 2024-09-04T17:47:32.234 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.234 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.235 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:32.235 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253256+0000", 2024-09-04T17:47:32.235 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6099999999999997e-06 2024-09-04T17:47:32.235 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.235 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.235 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:32.235 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253274+0000", 2024-09-04T17:47:32.236 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7952999999999999e-05 2024-09-04T17:47:32.236 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.236 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.236 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:32.236 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253287+0000", 2024-09-04T17:47:32.237 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2839e-05 2024-09-04T17:47:32.237 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.237 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.237 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:32.237 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304396+0000", 2024-09-04T17:47:32.237 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051108990999999999 2024-09-04T17:47:32.238 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.238 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.238 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:32.238 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304445+0000", 2024-09-04T17:47:32.238 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9005000000000003e-05 2024-09-04T17:47:32.238 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:32.239 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:32.239 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:32.239 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.239 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.239 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44692 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:32.239 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253337+0000", 2024-09-04T17:47:32.240 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 41.921156086000003, 2024-09-04T17:47:32.240 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051138909000000003, 2024-09-04T17:47:32.240 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:32.240 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:32.240 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:32.241 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:32.241 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:32.241 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:32.241 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44692 2024-09-04T17:47:32.241 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.241 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:32.242 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.242 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:32.242 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253337+0000", 2024-09-04T17:47:32.242 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:32.242 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.242 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.243 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:32.243 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253337+0000", 2024-09-04T17:47:32.243 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:32.243 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.243 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.244 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:32.244 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253338+0000", 2024-09-04T17:47:32.244 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.463e-06 2024-09-04T17:47:32.244 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.244 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.245 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:32.245 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253357+0000", 2024-09-04T17:47:32.245 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9043e-05 2024-09-04T17:47:32.245 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.245 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.245 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:32.246 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253379+0000", 2024-09-04T17:47:32.246 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1243e-05 2024-09-04T17:47:32.246 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.246 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.246 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:32.247 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253396+0000", 2024-09-04T17:47:32.247 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7059e-05 2024-09-04T17:47:32.247 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.247 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.247 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:32.247 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304466+0000", 2024-09-04T17:47:32.248 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051070624000000002 2024-09-04T17:47:32.248 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.248 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.248 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:32.248 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304476+0000", 2024-09-04T17:47:32.248 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4769999999999992e-06 2024-09-04T17:47:32.249 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:32.249 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:32.249 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:32.249 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.249 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.249 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44693 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:32.249 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253415+0000", 2024-09-04T17:47:32.250 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 41.921077683, 2024-09-04T17:47:32.250 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051141949999999999, 2024-09-04T17:47:32.250 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:32.250 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:32.250 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:32.250 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:32.251 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:32.251 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:32.251 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44693 2024-09-04T17:47:32.251 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.251 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:32.251 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.251 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:32.251 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253415+0000", 2024-09-04T17:47:32.251 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:32.252 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.252 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.252 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:32.252 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253415+0000", 2024-09-04T17:47:32.252 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:32.252 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.252 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.252 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:32.252 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253416+0000", 2024-09-04T17:47:32.252 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1799999999999999e-06 2024-09-04T17:47:32.252 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.252 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.252 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:32.253 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253422+0000", 2024-09-04T17:47:32.253 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.5940000000000001e-06 2024-09-04T17:47:32.253 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.253 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.253 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:32.253 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253438+0000", 2024-09-04T17:47:32.253 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6008999999999999e-05 2024-09-04T17:47:32.253 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.253 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.253 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:32.253 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253452+0000", 2024-09-04T17:47:32.253 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3543000000000001e-05 2024-09-04T17:47:32.253 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.254 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.254 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:32.254 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304533+0000", 2024-09-04T17:47:32.254 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051081609 2024-09-04T17:47:32.254 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.254 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.254 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:32.254 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304557+0000", 2024-09-04T17:47:32.254 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4015e-05 2024-09-04T17:47:32.254 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:32.254 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:32.254 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:32.254 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.255 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.255 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44694 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:32.255 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253469+0000", 2024-09-04T17:47:32.255 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 41.921023558000002, 2024-09-04T17:47:32.255 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051121486000000001, 2024-09-04T17:47:32.255 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:32.255 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:32.255 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:32.255 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:32.255 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:32.255 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:32.255 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44694 2024-09-04T17:47:32.255 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.255 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:32.256 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.256 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:32.256 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253469+0000", 2024-09-04T17:47:32.256 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:32.256 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.256 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.256 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:32.256 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253469+0000", 2024-09-04T17:47:32.256 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:32.256 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.256 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.256 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:32.257 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253471+0000", 2024-09-04T17:47:32.257 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1400000000000001e-06 2024-09-04T17:47:32.257 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.257 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.257 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:32.257 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253476+0000", 2024-09-04T17:47:32.257 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7370000000000003e-06 2024-09-04T17:47:32.257 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.257 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.257 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:32.257 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253493+0000", 2024-09-04T17:47:32.257 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6549e-05 2024-09-04T17:47:32.258 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.258 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.258 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:32.258 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253506+0000", 2024-09-04T17:47:32.258 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3406e-05 2024-09-04T17:47:32.258 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.258 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.258 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:32.258 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304582+0000", 2024-09-04T17:47:32.258 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051075534999999998 2024-09-04T17:47:32.258 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.258 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.258 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:32.259 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304591+0000", 2024-09-04T17:47:32.259 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1190000000000003e-06 2024-09-04T17:47:32.259 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:32.259 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:32.259 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:32.259 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.259 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.259 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44695 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:32.259 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253552+0000", 2024-09-04T17:47:32.259 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 41.920940803999997, 2024-09-04T17:47:32.259 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051068872000000001, 2024-09-04T17:47:32.259 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:32.259 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:32.259 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:32.260 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:32.260 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:32.260 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:32.260 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44695 2024-09-04T17:47:32.260 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.260 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:32.260 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.260 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:32.260 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253552+0000", 2024-09-04T17:47:32.260 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:32.260 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.260 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.261 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:32.261 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253552+0000", 2024-09-04T17:47:32.261 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:32.261 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.261 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.261 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:32.261 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253553+0000", 2024-09-04T17:47:32.261 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1960000000000001e-06 2024-09-04T17:47:32.261 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.261 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.261 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:32.261 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253560+0000", 2024-09-04T17:47:32.262 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.2040000000000002e-06 2024-09-04T17:47:32.262 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.262 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.262 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:32.262 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253578+0000", 2024-09-04T17:47:32.262 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8241999999999999e-05 2024-09-04T17:47:32.262 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.262 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.262 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:32.262 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253591+0000", 2024-09-04T17:47:32.262 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3453e-05 2024-09-04T17:47:32.262 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.262 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.263 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:32.263 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304614+0000", 2024-09-04T17:47:32.263 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051022456000000001 2024-09-04T17:47:32.263 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.263 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.263 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:32.263 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304621+0000", 2024-09-04T17:47:32.263 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3209999999999999e-06 2024-09-04T17:47:32.263 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:32.263 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:32.263 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:32.263 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.263 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.264 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44696 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:32.264 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253609+0000", 2024-09-04T17:47:32.264 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 41.920884303000001, 2024-09-04T17:47:32.264 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051115351000000003, 2024-09-04T17:47:32.264 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:32.264 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:32.264 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:32.264 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:32.264 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:32.264 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:32.264 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44696 2024-09-04T17:47:32.264 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.264 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:32.265 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.265 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:32.265 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253609+0000", 2024-09-04T17:47:32.265 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:32.265 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.265 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.265 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:32.265 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253609+0000", 2024-09-04T17:47:32.265 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:32.265 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.265 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.265 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:32.266 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253610+0000", 2024-09-04T17:47:32.266 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2300000000000001e-06 2024-09-04T17:47:32.266 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.266 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.266 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:32.266 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253615+0000", 2024-09-04T17:47:32.266 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7260000000000002e-06 2024-09-04T17:47:32.266 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.266 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.266 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:32.266 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253632+0000", 2024-09-04T17:47:32.267 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7039999999999999e-05 2024-09-04T17:47:32.267 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.267 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.267 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:32.267 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253666+0000", 2024-09-04T17:47:32.267 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4662000000000001e-05 2024-09-04T17:47:32.267 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.267 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.268 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:32.268 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304677+0000", 2024-09-04T17:47:32.268 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051011017999999998 2024-09-04T17:47:32.268 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.268 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.269 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:32.269 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304724+0000", 2024-09-04T17:47:32.269 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6675000000000003e-05 2024-09-04T17:47:32.269 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:32.269 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:32.269 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:32.270 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.270 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.270 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44697 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:32.270 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253684+0000", 2024-09-04T17:47:32.270 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 41.920808809, 2024-09-04T17:47:32.270 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051073107, 2024-09-04T17:47:32.271 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:32.271 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:32.271 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:32.271 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:32.271 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:32.271 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:32.271 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44697 2024-09-04T17:47:32.272 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.272 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:32.272 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.272 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:32.272 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253684+0000", 2024-09-04T17:47:32.272 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:32.273 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.273 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.273 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:32.273 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253684+0000", 2024-09-04T17:47:32.273 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:32.273 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.274 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.274 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:32.274 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253685+0000", 2024-09-04T17:47:32.274 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0699999999999999e-06 2024-09-04T17:47:32.274 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.274 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.275 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:32.275 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253690+0000", 2024-09-04T17:47:32.275 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1959999999999999e-06 2024-09-04T17:47:32.275 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.275 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.275 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:32.276 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253707+0000", 2024-09-04T17:47:32.276 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6946e-05 2024-09-04T17:47:32.276 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.276 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.276 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:32.276 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253720+0000", 2024-09-04T17:47:32.277 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3076e-05 2024-09-04T17:47:32.277 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.277 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.277 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:32.277 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304745+0000", 2024-09-04T17:47:32.277 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051024760000000002 2024-09-04T17:47:32.277 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.278 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.278 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:32.278 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304757+0000", 2024-09-04T17:47:32.278 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2058999999999999e-05 2024-09-04T17:47:32.278 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:32.278 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:32.278 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:32.278 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.279 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.279 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44698 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:32.279 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253737+0000", 2024-09-04T17:47:32.279 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 41.920756458, 2024-09-04T17:47:32.279 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051176078, 2024-09-04T17:47:32.279 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:32.279 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:32.280 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:32.280 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:32.280 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:32.280 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:32.280 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44698 2024-09-04T17:47:32.280 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.280 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:32.281 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.281 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:32.281 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253737+0000", 2024-09-04T17:47:32.281 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:32.281 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.281 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.282 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:32.282 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253737+0000", 2024-09-04T17:47:32.282 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:32.282 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.282 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.283 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:32.283 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253738+0000", 2024-09-04T17:47:32.283 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0300000000000001e-06 2024-09-04T17:47:32.283 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.283 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.284 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:32.284 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253752+0000", 2024-09-04T17:47:32.284 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4358999999999999e-05 2024-09-04T17:47:32.284 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.284 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.285 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:32.285 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253771+0000", 2024-09-04T17:47:32.285 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8879999999999999e-05 2024-09-04T17:47:32.285 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.285 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.285 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:32.285 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253785+0000", 2024-09-04T17:47:32.285 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3852000000000001e-05 2024-09-04T17:47:32.286 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.286 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.286 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:32.286 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304876+0000", 2024-09-04T17:47:32.286 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051091105999999997 2024-09-04T17:47:32.286 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.286 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.286 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:32.287 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304913+0000", 2024-09-04T17:47:32.287 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.6850999999999997e-05 2024-09-04T17:47:32.287 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:32.287 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:32.287 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:32.287 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.287 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.287 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44699 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:32.288 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253804+0000", 2024-09-04T17:47:32.288 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 41.920688933999998, 2024-09-04T17:47:32.288 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051172250000000002, 2024-09-04T17:47:32.288 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:32.288 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:32.288 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:32.288 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:32.289 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:32.289 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:32.289 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44699 2024-09-04T17:47:32.289 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.289 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:32.289 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.290 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:32.290 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253804+0000", 2024-09-04T17:47:32.290 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:32.290 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.290 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.290 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:32.291 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253804+0000", 2024-09-04T17:47:32.291 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:32.291 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.291 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.291 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:32.291 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253805+0000", 2024-09-04T17:47:32.291 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.08e-06 2024-09-04T17:47:32.291 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.291 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.291 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:32.291 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253810+0000", 2024-09-04T17:47:32.292 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1830000000000002e-06 2024-09-04T17:47:32.292 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.292 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.292 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:32.292 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253828+0000", 2024-09-04T17:47:32.292 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7662000000000001e-05 2024-09-04T17:47:32.292 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.292 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.292 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:32.292 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253841+0000", 2024-09-04T17:47:32.292 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3203999999999999e-05 2024-09-04T17:47:32.292 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.292 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.293 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:32.293 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304966+0000", 2024-09-04T17:47:32.293 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051125273999999998 2024-09-04T17:47:32.293 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.293 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.293 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:32.293 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304976+0000", 2024-09-04T17:47:32.293 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.8470000000000008e-06 2024-09-04T17:47:32.293 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:32.293 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:32.293 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:32.293 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.293 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.294 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44700 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:32.294 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253858+0000", 2024-09-04T17:47:32.294 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 41.920635279999999, 2024-09-04T17:47:32.294 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051218957000000002, 2024-09-04T17:47:32.294 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:32.294 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:32.294 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:32.294 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:32.294 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:32.294 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:32.294 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44700 2024-09-04T17:47:32.294 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.294 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:32.295 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.295 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:32.295 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253858+0000", 2024-09-04T17:47:32.295 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:32.295 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.295 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.295 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:32.295 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253858+0000", 2024-09-04T17:47:32.295 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:32.295 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.295 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.295 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:32.295 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253859+0000", 2024-09-04T17:47:32.295 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.217e-06 2024-09-04T17:47:32.296 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.296 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.296 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:32.296 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253864+0000", 2024-09-04T17:47:32.296 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7600000000000002e-06 2024-09-04T17:47:32.296 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.296 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.296 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:32.296 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253880+0000", 2024-09-04T17:47:32.296 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6501999999999999e-05 2024-09-04T17:47:32.296 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.296 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.296 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:32.297 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253899+0000", 2024-09-04T17:47:32.297 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8907e-05 2024-09-04T17:47:32.297 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.297 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.297 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:32.297 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305052+0000", 2024-09-04T17:47:32.297 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051152989000000003 2024-09-04T17:47:32.297 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.297 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.297 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:32.297 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305077+0000", 2024-09-04T17:47:32.297 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4581999999999999e-05 2024-09-04T17:47:32.298 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:32.298 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:32.298 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:32.298 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.298 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.298 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44701 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:32.298 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253917+0000", 2024-09-04T17:47:32.298 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 41.920575927999998, 2024-09-04T17:47:32.298 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051231113000000002, 2024-09-04T17:47:32.298 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:32.298 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:32.298 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:32.298 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:32.299 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:32.299 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:32.299 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44701 2024-09-04T17:47:32.299 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.299 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:32.299 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.299 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:32.299 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253917+0000", 2024-09-04T17:47:32.299 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:32.299 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.299 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.299 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:32.300 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253917+0000", 2024-09-04T17:47:32.300 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:32.300 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.300 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.300 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:32.300 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253918+0000", 2024-09-04T17:47:32.300 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.136e-06 2024-09-04T17:47:32.300 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.300 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.300 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:32.300 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253924+0000", 2024-09-04T17:47:32.300 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.3070000000000002e-06 2024-09-04T17:47:32.300 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.301 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.301 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:32.301 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253940+0000", 2024-09-04T17:47:32.301 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6612e-05 2024-09-04T17:47:32.301 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.301 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.302 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:32.302 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253954+0000", 2024-09-04T17:47:32.302 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3709999999999999e-05 2024-09-04T17:47:32.302 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.302 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.302 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:32.302 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305136+0000", 2024-09-04T17:47:32.303 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051181935999999997 2024-09-04T17:47:32.303 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:32.303 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:32.303 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:32.303 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305148+0000", 2024-09-04T17:47:32.303 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2412000000000001e-05 2024-09-04T17:47:32.303 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:32.303 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:32.303 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:32.303 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:32.303 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:32.304 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:32.304 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-04T17:47:32.305 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-04T17:47:32.305 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:47:32.306 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:47:32.306 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-04T17:47:32.307 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:32.304+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:32.307 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:47:32.307 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:47:32.307 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-04T17:47:32.308 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:47:32.308 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-04T17:47:32.308 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:47:32.309 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-04T17:47:32.406 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:32.405+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:32.480 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:32.480 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:32.495 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-04T17:47:32.507 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:32.505+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:32.608 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:32.605+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:32.618 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:32.618 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:32.632 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-04T17:47:32.709 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:32.709+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:32.750 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:32.751 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:32.765 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-04T17:47:32.810 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:32.807+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:32.889 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:32.889 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:32.903 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-04T17:47:32.911 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:32.909+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:33.012 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:33.009+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:33.069 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:33.069 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:33.083 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-04T17:47:33.113 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:33.113+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:33.199 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:33.199 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:33.213 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:33.211+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:33.214 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-04T17:47:33.315 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:33.313+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:33.345 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:33.346 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:33.360 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-04T17:47:33.415 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:33.411+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:33.517 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:33.515+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:33.523 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:33.523 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:33.537 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-04T17:47:33.617 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:33.617+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:33.659 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:33.660 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:33.673 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-04T17:47:33.718 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:33.717+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:33.819 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:33.817+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:33.823 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 4, 5, 6, 7] out_osds: [3, 1] dead_osds: [6, 2] live_osds: [0, 1, 4, 5, 7, 3] 2024-09-04T17:47:33.823 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-04T17:47:33.823 INFO:tasks.thrashosds.thrasher:Reviving osd 2 2024-09-04T17:47:33.823 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-04T17:47:33.823 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-04T17:47:33.825 INFO:tasks.ceph.osd.2:Started 2024-09-04T17:47:33.826 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-04T17:47:33.846 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:33.846 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:33.862 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-04T17:47:33.866 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-04T17:47:33.919 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:47:33.915+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:47:33.948 INFO:teuthology.orchestra.run.smithi092.stderr:no valid command found; 10 closest matches: 2024-09-04T17:47:33.948 INFO:teuthology.orchestra.run.smithi092.stderr:0 2024-09-04T17:47:33.948 INFO:teuthology.orchestra.run.smithi092.stderr:1 2024-09-04T17:47:33.948 INFO:teuthology.orchestra.run.smithi092.stderr:2 2024-09-04T17:47:33.948 INFO:teuthology.orchestra.run.smithi092.stderr:abort 2024-09-04T17:47:33.948 INFO:teuthology.orchestra.run.smithi092.stderr:assert 2024-09-04T17:47:33.948 INFO:teuthology.orchestra.run.smithi092.stderr:config diff 2024-09-04T17:47:33.948 INFO:teuthology.orchestra.run.smithi092.stderr:config diff get 2024-09-04T17:47:33.948 INFO:teuthology.orchestra.run.smithi092.stderr:config get 2024-09-04T17:47:33.949 INFO:teuthology.orchestra.run.smithi092.stderr:config help [] 2024-09-04T17:47:33.949 INFO:teuthology.orchestra.run.smithi092.stderr:config set ... 2024-09-04T17:47:33.949 INFO:teuthology.orchestra.run.smithi092.stderr:admin_socket: invalid command 2024-09-04T17:47:33.950 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-04T17:47:33.951 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2024-09-04T17:47:34.020 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:34.017+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:34.022 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:34.022 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:34.038 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-04T17:47:34.125 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:34.125+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:34.200 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:34.200 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:34.215 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-04T17:47:34.257 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:34.257+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:34.310 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-04T17:47:34.357 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:34.357+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:34.381 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:34.382 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:34.397 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-04T17:47:34.445 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:34.445+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:34.449 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:47:34.449 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:47:34.468 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:47:34.468 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:47:34.468 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-04T17:47:34.485 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-04T17:47:34.485 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-04T17:47:34.506 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:47:34.506 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:47:34.525 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:34.525+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:34.527 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:47:34.527 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:47:34.527 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-04T17:47:34.562 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:47:34.562 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:47:34.568 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-04T17:47:34.568 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-04T17:47:34.571 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:34.571 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:47:34.571 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:47:34.571 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:47:34.571 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:34.585 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-04T17:47:34.626 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:34.623+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:34.727 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:34.725+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:34.758 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:34.758 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:47:34.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:47:34.758 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:47:34.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.758 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59950 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5365760~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:47:34.759 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654917+0000", 2024-09-04T17:47:34.759 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 21.102665441999999, 2024-09-04T17:47:34.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066822450000000005, 2024-09-04T17:47:34.759 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:34.759 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:34.759 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:34.759 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:34.759 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:34.759 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:34.759 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59950 2024-09-04T17:47:34.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.759 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:34.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:34.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654917+0000", 2024-09-04T17:47:34.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:34.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:34.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654917+0000", 2024-09-04T17:47:34.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:34.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:34.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654918+0000", 2024-09-04T17:47:34.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.29e-07 2024-09-04T17:47:34.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:34.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654919+0000", 2024-09-04T17:47:34.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2589999999999999e-06 2024-09-04T17:47:34.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:34.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654922+0000", 2024-09-04T17:47:34.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1350000000000001e-06 2024-09-04T17:47:34.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:34.762 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654925+0000", 2024-09-04T17:47:34.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1250000000000001e-06 2024-09-04T17:47:34.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.762 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:34.762 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721466+0000", 2024-09-04T17:47:34.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066540959999999996 2024-09-04T17:47:34.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.762 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:34.762 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721574+0000", 2024-09-04T17:47:34.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010747399999999999 2024-09-04T17:47:34.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.763 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:34.763 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721740+0000", 2024-09-04T17:47:34.763 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016626799999999999 2024-09-04T17:47:34.763 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:34.763 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:34.763 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:34.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.763 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59951 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5472256~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:47:34.763 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654929+0000", 2024-09-04T17:47:34.763 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 21.102654257000001, 2024-09-04T17:47:34.763 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067110534999999999, 2024-09-04T17:47:34.764 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:34.764 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:34.764 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:34.764 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:34.764 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:34.764 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:34.764 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59951 2024-09-04T17:47:34.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.764 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:34.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:34.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654929+0000", 2024-09-04T17:47:34.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:34.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:34.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654929+0000", 2024-09-04T17:47:34.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:34.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:34.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654929+0000", 2024-09-04T17:47:34.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5100000000000001e-07 2024-09-04T17:47:34.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:34.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654930+0000", 2024-09-04T17:47:34.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2440000000000001e-06 2024-09-04T17:47:34.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.766 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:34.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654933+0000", 2024-09-04T17:47:34.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1599999999999998e-06 2024-09-04T17:47:34.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.766 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:34.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654936+0000", 2024-09-04T17:47:34.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6560000000000001e-06 2024-09-04T17:47:34.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.767 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:34.767 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721789+0000", 2024-09-04T17:47:34.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066853250000000003 2024-09-04T17:47:34.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.767 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:34.767 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721885+0000", 2024-09-04T17:47:34.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5616000000000001e-05 2024-09-04T17:47:34.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.767 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:34.767 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722039+0000", 2024-09-04T17:47:34.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000154258 2024-09-04T17:47:34.768 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:34.768 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:34.768 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:34.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.768 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59952 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5505024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:47:34.768 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654939+0000", 2024-09-04T17:47:34.768 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 21.10264338, 2024-09-04T17:47:34.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067433742000000005, 2024-09-04T17:47:34.769 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:34.769 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:34.769 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:34.769 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:34.769 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:34.769 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:34.769 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59952 2024-09-04T17:47:34.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.769 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:34.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:34.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654939+0000", 2024-09-04T17:47:34.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:34.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:34.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654939+0000", 2024-09-04T17:47:34.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:34.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:34.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654940+0000", 2024-09-04T17:47:34.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3300000000000001e-07 2024-09-04T17:47:34.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:34.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654941+0000", 2024-09-04T17:47:34.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1570000000000001e-06 2024-09-04T17:47:34.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:34.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654944+0000", 2024-09-04T17:47:34.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1319999999999998e-06 2024-09-04T17:47:34.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.772 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:34.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654947+0000", 2024-09-04T17:47:34.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5969999999999999e-06 2024-09-04T17:47:34.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.772 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:34.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722089+0000", 2024-09-04T17:47:34.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067142347000000005 2024-09-04T17:47:34.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.772 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:34.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722201+0000", 2024-09-04T17:47:34.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011235600000000001 2024-09-04T17:47:34.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.773 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:34.773 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722373+0000", 2024-09-04T17:47:34.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017191999999999999 2024-09-04T17:47:34.773 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:34.773 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:34.773 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:34.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.773 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59953 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5611520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:47:34.773 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654950+0000", 2024-09-04T17:47:34.774 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 21.102632740000001, 2024-09-04T17:47:34.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067601852000000004, 2024-09-04T17:47:34.774 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:34.774 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:34.774 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:34.774 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:34.774 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:34.774 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:34.774 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59953 2024-09-04T17:47:34.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.774 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:34.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:34.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654950+0000", 2024-09-04T17:47:34.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:34.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:34.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654950+0000", 2024-09-04T17:47:34.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:34.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:34.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654950+0000", 2024-09-04T17:47:34.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2499999999999999e-07 2024-09-04T17:47:34.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:34.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654954+0000", 2024-09-04T17:47:34.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.287e-06 2024-09-04T17:47:34.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:34.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654962+0000", 2024-09-04T17:47:34.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4230000000000005e-06 2024-09-04T17:47:34.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:34.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654966+0000", 2024-09-04T17:47:34.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6449999999999998e-06 2024-09-04T17:47:34.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.777 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:34.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722417+0000", 2024-09-04T17:47:34.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067451188999999995 2024-09-04T17:47:34.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.777 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:34.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722467+0000", 2024-09-04T17:47:34.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0556999999999997e-05 2024-09-04T17:47:34.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.777 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:34.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722552+0000", 2024-09-04T17:47:34.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4525999999999997e-05 2024-09-04T17:47:34.778 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:34.778 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:34.778 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:34.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.778 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59954 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5718016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:47:34.778 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654970+0000", 2024-09-04T17:47:34.778 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 21.102612979, 2024-09-04T17:47:34.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067874285000000006, 2024-09-04T17:47:34.778 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:34.778 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:34.778 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:34.778 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:34.779 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:34.779 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:34.779 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59954 2024-09-04T17:47:34.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.779 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:34.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:34.779 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654970+0000", 2024-09-04T17:47:34.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:34.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:34.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654970+0000", 2024-09-04T17:47:34.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:34.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.780 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:34.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654970+0000", 2024-09-04T17:47:34.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5899999999999998e-07 2024-09-04T17:47:34.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.780 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:34.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654971+0000", 2024-09-04T17:47:34.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2929999999999999e-06 2024-09-04T17:47:34.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:34.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654975+0000", 2024-09-04T17:47:34.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4699999999999998e-06 2024-09-04T17:47:34.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:34.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654978+0000", 2024-09-04T17:47:34.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7930000000000002e-06 2024-09-04T17:47:34.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:34.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722600+0000", 2024-09-04T17:47:34.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067622229000000006 2024-09-04T17:47:34.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:34.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722689+0000", 2024-09-04T17:47:34.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9511000000000004e-05 2024-09-04T17:47:34.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:34.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722844+0000", 2024-09-04T17:47:34.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015473 2024-09-04T17:47:34.782 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:34.782 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:34.782 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:34.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.783 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59955 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5857280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:47:34.783 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654981+0000", 2024-09-04T17:47:34.783 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 21.102601628999999, 2024-09-04T17:47:34.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068052360000000006, 2024-09-04T17:47:34.783 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:34.783 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:34.783 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:34.783 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:34.783 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:34.783 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:34.784 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59955 2024-09-04T17:47:34.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.784 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:34.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:34.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654981+0000", 2024-09-04T17:47:34.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:34.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:34.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654981+0000", 2024-09-04T17:47:34.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:34.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.785 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:34.785 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654981+0000", 2024-09-04T17:47:34.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2399999999999999e-07 2024-09-04T17:47:34.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.785 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:34.785 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654985+0000", 2024-09-04T17:47:34.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0980000000000004e-06 2024-09-04T17:47:34.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.785 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:34.785 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654990+0000", 2024-09-04T17:47:34.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3429999999999998e-06 2024-09-04T17:47:34.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:34.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654993+0000", 2024-09-04T17:47:34.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8700000000000001e-06 2024-09-04T17:47:34.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:34.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722888+0000", 2024-09-04T17:47:34.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067895239999999996 2024-09-04T17:47:34.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:34.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722948+0000", 2024-09-04T17:47:34.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9750999999999998e-05 2024-09-04T17:47:34.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:34.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.723034+0000", 2024-09-04T17:47:34.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5834000000000005e-05 2024-09-04T17:47:34.787 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:34.787 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:34.787 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:34.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.787 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59956 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5963776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:47:34.788 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654998+0000", 2024-09-04T17:47:34.788 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 21.102585099999999, 2024-09-04T17:47:34.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068359518999999994, 2024-09-04T17:47:34.788 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:34.788 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:34.788 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:34.788 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:34.788 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:34.788 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:34.788 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59956 2024-09-04T17:47:34.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.788 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:34.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:34.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654998+0000", 2024-09-04T17:47:34.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:34.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:34.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654998+0000", 2024-09-04T17:47:34.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:34.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:34.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654998+0000", 2024-09-04T17:47:34.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4299999999999999e-07 2024-09-04T17:47:34.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.790 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:34.790 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654999+0000", 2024-09-04T17:47:34.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1799999999999999e-06 2024-09-04T17:47:34.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.790 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:34.790 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.655002+0000", 2024-09-04T17:47:34.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3639999999999999e-06 2024-09-04T17:47:34.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.790 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:34.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.655005+0000", 2024-09-04T17:47:34.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7970000000000001e-06 2024-09-04T17:47:34.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:34.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.723085+0000", 2024-09-04T17:47:34.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068080229000000006 2024-09-04T17:47:34.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:34.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.723197+0000", 2024-09-04T17:47:34.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000111344 2024-09-04T17:47:34.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:34.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.723357+0000", 2024-09-04T17:47:34.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016036199999999999 2024-09-04T17:47:34.792 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:34.792 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:34.792 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:34.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.792 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61082 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 2654208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:34.792 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.822664+0000", 2024-09-04T17:47:34.792 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.934918712999998, 2024-09-04T17:47:34.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067072242000000004, 2024-09-04T17:47:34.792 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:34.793 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:34.793 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:34.793 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:34.793 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:34.793 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:34.793 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61082 2024-09-04T17:47:34.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.793 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:34.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:34.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822664+0000", 2024-09-04T17:47:34.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:34.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:34.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822664+0000", 2024-09-04T17:47:34.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:34.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:34.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822664+0000", 2024-09-04T17:47:34.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6600000000000003e-07 2024-09-04T17:47:34.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:34.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822666+0000", 2024-09-04T17:47:34.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.536e-06 2024-09-04T17:47:34.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:34.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822669+0000", 2024-09-04T17:47:34.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3759999999999999e-06 2024-09-04T17:47:34.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:34.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822672+0000", 2024-09-04T17:47:34.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8109999999999999e-06 2024-09-04T17:47:34.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:34.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889263+0000", 2024-09-04T17:47:34.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066591259999999999 2024-09-04T17:47:34.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:34.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889369+0000", 2024-09-04T17:47:34.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000105362 2024-09-04T17:47:34.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:34.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889736+0000", 2024-09-04T17:47:34.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00036763099999999999 2024-09-04T17:47:34.796 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:34.796 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:34.797 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:34.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.797 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61083 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 2760704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:34.797 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.822678+0000", 2024-09-04T17:47:34.797 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.934904978999999, 2024-09-04T17:47:34.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067475996999999996, 2024-09-04T17:47:34.797 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:34.797 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:34.797 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:34.797 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:34.797 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:34.797 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:34.798 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61083 2024-09-04T17:47:34.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.798 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:34.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:34.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822678+0000", 2024-09-04T17:47:34.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:34.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:34.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822678+0000", 2024-09-04T17:47:34.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:34.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:34.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822725+0000", 2024-09-04T17:47:34.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7429000000000003e-05 2024-09-04T17:47:34.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:34.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822728+0000", 2024-09-04T17:47:34.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2140000000000001e-06 2024-09-04T17:47:34.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:34.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822737+0000", 2024-09-04T17:47:34.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4370000000000007e-06 2024-09-04T17:47:34.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:34.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822743+0000", 2024-09-04T17:47:34.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7520000000000002e-06 2024-09-04T17:47:34.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:34.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889785+0000", 2024-09-04T17:47:34.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067042265000000004 2024-09-04T17:47:34.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:34.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889840+0000", 2024-09-04T17:47:34.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5254000000000003e-05 2024-09-04T17:47:34.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:34.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.890154+0000", 2024-09-04T17:47:34.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000313646 2024-09-04T17:47:34.801 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:34.801 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:34.801 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:34.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.802 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61084 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 2867200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:34.802 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.822753+0000", 2024-09-04T17:47:34.802 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.934830043000002, 2024-09-04T17:47:34.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068497161000000001, 2024-09-04T17:47:34.802 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:34.802 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:34.802 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:34.802 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:34.802 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:34.802 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:34.802 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61084 2024-09-04T17:47:34.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.802 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:34.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:34.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822753+0000", 2024-09-04T17:47:34.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:34.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:34.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822753+0000", 2024-09-04T17:47:34.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:34.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:34.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822753+0000", 2024-09-04T17:47:34.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9100000000000003e-07 2024-09-04T17:47:34.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:34.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822756+0000", 2024-09-04T17:47:34.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.678e-06 2024-09-04T17:47:34.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:34.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822763+0000", 2024-09-04T17:47:34.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8220000000000001e-06 2024-09-04T17:47:34.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:34.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822768+0000", 2024-09-04T17:47:34.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1930000000000001e-06 2024-09-04T17:47:34.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:34.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.890246+0000", 2024-09-04T17:47:34.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067477996999999998 2024-09-04T17:47:34.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:34.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.890555+0000", 2024-09-04T17:47:34.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000309143 2024-09-04T17:47:34.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:34.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.891250+0000", 2024-09-04T17:47:34.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00069463700000000001 2024-09-04T17:47:34.806 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:34.806 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:34.806 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:34.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.806 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61285 3.8 3:117cfeb9:::rbd_data.11341dfde0c5.0000000000000006:1f [read 2760704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:34.806 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835200+0000", 2024-09-04T17:47:34.806 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.922382976000002, 2024-09-04T17:47:34.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066899803999999993, 2024-09-04T17:47:34.807 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:34.807 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:34.807 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:34.807 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:34.807 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:34.807 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:34.807 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61285 2024-09-04T17:47:34.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.807 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:34.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:34.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835200+0000", 2024-09-04T17:47:34.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:34.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:34.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835200+0000", 2024-09-04T17:47:34.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:34.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:34.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835200+0000", 2024-09-04T17:47:34.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0899999999999997e-07 2024-09-04T17:47:34.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:34.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835206+0000", 2024-09-04T17:47:34.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3280000000000002e-06 2024-09-04T17:47:34.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:34.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835211+0000", 2024-09-04T17:47:34.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9230000000000001e-06 2024-09-04T17:47:34.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:34.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835216+0000", 2024-09-04T17:47:34.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9019999999999998e-06 2024-09-04T17:47:34.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:34.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.901719+0000", 2024-09-04T17:47:34.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066502981000000003 2024-09-04T17:47:34.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:34.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.901813+0000", 2024-09-04T17:47:34.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3905000000000003e-05 2024-09-04T17:47:34.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:34.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902100+0000", 2024-09-04T17:47:34.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028645599999999998 2024-09-04T17:47:34.811 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:34.811 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:34.811 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:34.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.811 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61286 3.8 3:117cfeb9:::rbd_data.11341dfde0c5.0000000000000006:1f [read 2867200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:34.811 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835244+0000", 2024-09-04T17:47:34.811 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.922338357000001, 2024-09-04T17:47:34.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067143215000000006, 2024-09-04T17:47:34.811 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:34.811 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:34.811 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:34.811 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:34.812 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:34.812 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:34.812 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61286 2024-09-04T17:47:34.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.812 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:34.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:34.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835244+0000", 2024-09-04T17:47:34.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:34.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:34.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835244+0000", 2024-09-04T17:47:34.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:34.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:34.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835245+0000", 2024-09-04T17:47:34.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8400000000000004e-07 2024-09-04T17:47:34.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:34.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835248+0000", 2024-09-04T17:47:34.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6850000000000001e-06 2024-09-04T17:47:34.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:34.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835254+0000", 2024-09-04T17:47:34.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.494e-06 2024-09-04T17:47:34.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:34.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835260+0000", 2024-09-04T17:47:34.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0580000000000003e-06 2024-09-04T17:47:34.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:34.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902119+0000", 2024-09-04T17:47:34.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066858318 2024-09-04T17:47:34.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:34.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902163+0000", 2024-09-04T17:47:34.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4753000000000003e-05 2024-09-04T17:47:34.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:34.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902388+0000", 2024-09-04T17:47:34.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022422300000000001 2024-09-04T17:47:34.815 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:34.815 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:34.815 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:34.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.816 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61315 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 98304~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:34.816 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835269+0000", 2024-09-04T17:47:34.816 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.922313565, 2024-09-04T17:47:34.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067329396, 2024-09-04T17:47:34.816 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:34.816 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:34.816 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:34.816 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:34.816 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:34.816 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:34.816 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61315 2024-09-04T17:47:34.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.816 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:34.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:34.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835269+0000", 2024-09-04T17:47:34.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:34.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:34.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835269+0000", 2024-09-04T17:47:34.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:34.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:34.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835270+0000", 2024-09-04T17:47:34.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6599999999999996e-07 2024-09-04T17:47:34.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:34.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835272+0000", 2024-09-04T17:47:34.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.261e-06 2024-09-04T17:47:34.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:34.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835278+0000", 2024-09-04T17:47:34.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.443e-06 2024-09-04T17:47:34.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:34.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835284+0000", 2024-09-04T17:47:34.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3290000000000001e-06 2024-09-04T17:47:34.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:34.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902147+0000", 2024-09-04T17:47:34.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066862695999999999 2024-09-04T17:47:34.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:34.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902248+0000", 2024-09-04T17:47:34.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000101501 2024-09-04T17:47:34.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:34.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902599+0000", 2024-09-04T17:47:34.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00035060000000000001 2024-09-04T17:47:34.820 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:34.820 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:34.820 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:34.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.820 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61316 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 204800~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:34.820 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835291+0000", 2024-09-04T17:47:34.820 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.922291546, 2024-09-04T17:47:34.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067632052999999998, 2024-09-04T17:47:34.821 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:34.821 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:34.821 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:34.821 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:34.821 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:34.821 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:34.821 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61316 2024-09-04T17:47:34.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.821 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:34.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:34.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835291+0000", 2024-09-04T17:47:34.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:34.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:34.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835291+0000", 2024-09-04T17:47:34.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:34.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:34.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835292+0000", 2024-09-04T17:47:34.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1899999999999998e-07 2024-09-04T17:47:34.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:34.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835294+0000", 2024-09-04T17:47:34.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0880000000000002e-06 2024-09-04T17:47:34.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:34.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835300+0000", 2024-09-04T17:47:34.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.224e-06 2024-09-04T17:47:34.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:34.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835305+0000", 2024-09-04T17:47:34.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0730000000000004e-06 2024-09-04T17:47:34.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:34.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902632+0000", 2024-09-04T17:47:34.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067327322999999994 2024-09-04T17:47:34.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:34.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902688+0000", 2024-09-04T17:47:34.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5177999999999999e-05 2024-09-04T17:47:34.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:34.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902923+0000", 2024-09-04T17:47:34.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023574800000000001 2024-09-04T17:47:34.825 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:34.825 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:34.825 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:34.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.825 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61317 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 344064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:34.825 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835312+0000", 2024-09-04T17:47:34.825 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.922270542, 2024-09-04T17:47:34.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067918511000000001, 2024-09-04T17:47:34.825 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:34.825 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:34.825 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:34.825 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:34.826 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:34.826 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:34.826 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61317 2024-09-04T17:47:34.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.826 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:34.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:34.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835312+0000", 2024-09-04T17:47:34.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:34.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:34.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835312+0000", 2024-09-04T17:47:34.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:34.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:34.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835313+0000", 2024-09-04T17:47:34.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9000000000000002e-07 2024-09-04T17:47:34.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:34.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835315+0000", 2024-09-04T17:47:34.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1689999999999999e-06 2024-09-04T17:47:34.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:34.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835321+0000", 2024-09-04T17:47:34.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0619999999999997e-06 2024-09-04T17:47:34.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:34.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835326+0000", 2024-09-04T17:47:34.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0390000000000004e-06 2024-09-04T17:47:34.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:34.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902967+0000", 2024-09-04T17:47:34.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067641520999999996 2024-09-04T17:47:34.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:34.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903067+0000", 2024-09-04T17:47:34.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9426999999999995e-05 2024-09-04T17:47:34.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:34.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903231+0000", 2024-09-04T17:47:34.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016390299999999999 2024-09-04T17:47:34.829 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:34.829 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:34.829 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:34.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.829 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61318 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 450560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:34.830 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835334+0000", 2024-09-04T17:47:34.830 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.922249024999999, 2024-09-04T17:47:34.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068091771999999995, 2024-09-04T17:47:34.830 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:34.830 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:34.830 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:34.830 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:34.830 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:34.830 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:34.830 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61318 2024-09-04T17:47:34.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.830 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:34.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:34.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835334+0000", 2024-09-04T17:47:34.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:34.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:34.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835334+0000", 2024-09-04T17:47:34.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:34.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:34.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835334+0000", 2024-09-04T17:47:34.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5699999999999998e-07 2024-09-04T17:47:34.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:34.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835337+0000", 2024-09-04T17:47:34.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3889999999999999e-06 2024-09-04T17:47:34.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:34.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835343+0000", 2024-09-04T17:47:34.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1809999999999998e-06 2024-09-04T17:47:34.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:34.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835348+0000", 2024-09-04T17:47:34.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3170000000000001e-06 2024-09-04T17:47:34.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:34.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903270+0000", 2024-09-04T17:47:34.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067922191000000007 2024-09-04T17:47:34.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:34.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903321+0000", 2024-09-04T17:47:34.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0889e-05 2024-09-04T17:47:34.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:34.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903426+0000", 2024-09-04T17:47:34.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104348 2024-09-04T17:47:34.834 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:34.834 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:34.834 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:34.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.834 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61319 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 589824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:34.834 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835357+0000", 2024-09-04T17:47:34.834 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.922226208000001, 2024-09-04T17:47:34.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068372535999999998, 2024-09-04T17:47:34.834 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:34.834 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:34.835 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:34.835 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:34.835 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:34.835 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:34.835 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61319 2024-09-04T17:47:34.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.835 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:34.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.835 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:34.835 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835357+0000", 2024-09-04T17:47:34.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:34.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:34.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835357+0000", 2024-09-04T17:47:34.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:34.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:34.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835357+0000", 2024-09-04T17:47:34.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.32e-07 2024-09-04T17:47:34.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:34.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835359+0000", 2024-09-04T17:47:34.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1660000000000001e-06 2024-09-04T17:47:34.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.837 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:34.837 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835366+0000", 2024-09-04T17:47:34.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3550000000000001e-06 2024-09-04T17:47:34.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.837 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:34.837 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835371+0000", 2024-09-04T17:47:34.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1050000000000003e-06 2024-09-04T17:47:34.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.837 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:34.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903471+0000", 2024-09-04T17:47:34.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068100451000000006 2024-09-04T17:47:34.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:34.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903573+0000", 2024-09-04T17:47:34.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000101691 2024-09-04T17:47:34.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:34.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903729+0000", 2024-09-04T17:47:34.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000156336 2024-09-04T17:47:34.838 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:34.839 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:34.839 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:34.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.839 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61320 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:34.839 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835380+0000", 2024-09-04T17:47:34.839 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.922202443, 2024-09-04T17:47:34.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068543058000000004, 2024-09-04T17:47:34.839 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:34.839 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:34.839 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:34.839 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:34.839 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:34.840 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:34.840 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61320 2024-09-04T17:47:34.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.840 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:34.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:34.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835380+0000", 2024-09-04T17:47:34.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:34.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:34.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835380+0000", 2024-09-04T17:47:34.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:34.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:34.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835381+0000", 2024-09-04T17:47:34.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9999999999999998e-07 2024-09-04T17:47:34.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:34.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835383+0000", 2024-09-04T17:47:34.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2230000000000001e-06 2024-09-04T17:47:34.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:34.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835390+0000", 2024-09-04T17:47:34.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.34e-06 2024-09-04T17:47:34.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:34.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835396+0000", 2024-09-04T17:47:34.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9830000000000001e-06 2024-09-04T17:47:34.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:34.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903765+0000", 2024-09-04T17:47:34.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068368441000000002 2024-09-04T17:47:34.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:34.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903835+0000", 2024-09-04T17:47:34.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0631999999999997e-05 2024-09-04T17:47:34.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:34.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903923+0000", 2024-09-04T17:47:34.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8039000000000002e-05 2024-09-04T17:47:34.843 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:34.843 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:34.843 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:34.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.843 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61321 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 835584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:34.843 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835404+0000", 2024-09-04T17:47:34.844 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.922178383999999, 2024-09-04T17:47:34.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068911043000000005, 2024-09-04T17:47:34.844 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:34.844 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:34.844 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:34.844 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:34.844 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:34.844 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:34.844 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61321 2024-09-04T17:47:34.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.844 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:34.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:34.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835404+0000", 2024-09-04T17:47:34.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:34.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:34.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835404+0000", 2024-09-04T17:47:34.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:34.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:34.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835405+0000", 2024-09-04T17:47:34.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.89e-07 2024-09-04T17:47:34.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:34.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835407+0000", 2024-09-04T17:47:34.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4439999999999998e-06 2024-09-04T17:47:34.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:34.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835414+0000", 2024-09-04T17:47:34.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4930000000000003e-06 2024-09-04T17:47:34.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:34.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835419+0000", 2024-09-04T17:47:34.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8919999999999999e-06 2024-09-04T17:47:34.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:34.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903957+0000", 2024-09-04T17:47:34.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068538500000000002 2024-09-04T17:47:34.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:34.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904019+0000", 2024-09-04T17:47:34.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1419000000000003e-05 2024-09-04T17:47:34.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:34.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904315+0000", 2024-09-04T17:47:34.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00029680599999999999 2024-09-04T17:47:34.848 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:34.848 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:34.848 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:34.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.848 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61322 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:34.848 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835427+0000", 2024-09-04T17:47:34.848 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.922155998000001, 2024-09-04T17:47:34.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.069550138999999997, 2024-09-04T17:47:34.848 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:34.848 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:34.848 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:34.848 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:34.849 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:34.849 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:34.849 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61322 2024-09-04T17:47:34.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.849 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:34.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:34.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835427+0000", 2024-09-04T17:47:34.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:34.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:34.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835427+0000", 2024-09-04T17:47:34.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:34.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:34.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.842157+0000", 2024-09-04T17:47:34.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067298419999999998 2024-09-04T17:47:34.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:34.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.842163+0000", 2024-09-04T17:47:34.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2029999999999997e-06 2024-09-04T17:47:34.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:34.851 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.842175+0000", 2024-09-04T17:47:34.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2072000000000001e-05 2024-09-04T17:47:34.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:34.851 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.842183+0000", 2024-09-04T17:47:34.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3850000000000002e-06 2024-09-04T17:47:34.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:34.851 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904371+0000", 2024-09-04T17:47:34.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.062187754999999997 2024-09-04T17:47:34.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:34.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904467+0000", 2024-09-04T17:47:34.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6117000000000001e-05 2024-09-04T17:47:34.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:34.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:34.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:34.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904977+0000", 2024-09-04T17:47:34.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00050976500000000002 2024-09-04T17:47:34.852 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:34.852 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:34.852 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:34.852 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:34.852 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:34.853 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:34.854 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-04T17:47:34.854 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is set 2024-09-04T17:47:34.855 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:34.853+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:34.955 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:47:34.951+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:47:34.984 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:34.984 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:47:34.985 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:47:34.985 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:34.998 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-04T17:47:35.085 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:47:35.083+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:47:35.157 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:35.155+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:35.158 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:35.159 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:47:35.159 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:47:35.159 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:47:35.159 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:35.173 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-04T17:47:35.243 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:47:35.241+0000 7fdf502127c0 -1 Falling back to public interface 2024-09-04T17:47:35.261 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:35.259+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:35.338 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:35.339 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:47:35.339 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:47:35.339 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:47:35.339 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.339 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:12722 3.1b 3:d8152af1:::rbd_data.1134ef3a5a7d.00000000000000bd:head [delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-04T17:47:35.340 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:16.421965+0000", 2024-09-04T17:47:35.340 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 78.914509203999998, 2024-09-04T17:47:35.340 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.890002811, 2024-09-04T17:47:35.340 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:35.340 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:35.341 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:35.341 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:35.341 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:35.341 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:35.341 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 12722 2024-09-04T17:47:35.342 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.342 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:35.342 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.342 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:35.342 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421965+0000", 2024-09-04T17:47:35.342 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:35.343 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.343 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.343 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:35.343 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421965+0000", 2024-09-04T17:47:35.343 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:35.344 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.344 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.344 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:35.344 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421966+0000", 2024-09-04T17:47:35.344 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.102e-06 2024-09-04T17:47:35.345 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.345 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.345 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:35.345 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421971+0000", 2024-09-04T17:47:35.345 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9989999999999999e-06 2024-09-04T17:47:35.345 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.346 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.346 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:35.346 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421981+0000", 2024-09-04T17:47:35.346 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.7380000000000006e-06 2024-09-04T17:47:35.346 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.347 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.347 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:35.347 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421989+0000", 2024-09-04T17:47:35.347 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.1890000000000007e-06 2024-09-04T17:47:35.347 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.347 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.348 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:35.348 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.422050+0000", 2024-09-04T17:47:35.348 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.0702000000000003e-05 2024-09-04T17:47:35.348 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.348 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.349 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for peered", 2024-09-04T17:47:35.349 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.422059+0000", 2024-09-04T17:47:35.349 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5440000000000004e-06 2024-09-04T17:47:35.349 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.349 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.350 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:35.350 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.308156+0000", 2024-09-04T17:47:35.350 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.88609676199999998 2024-09-04T17:47:35.350 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.350 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.351 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:35.351 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.308398+0000", 2024-09-04T17:47:35.351 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024215400000000001 2024-09-04T17:47:35.351 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.351 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.351 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 7", 2024-09-04T17:47:35.352 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.308795+0000", 2024-09-04T17:47:35.352 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000396704 2024-09-04T17:47:35.352 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.352 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.352 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:47:35.353 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.310831+0000", 2024-09-04T17:47:35.353 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0020364200000000002 2024-09-04T17:47:35.353 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.353 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.353 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:35.354 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.311818+0000", 2024-09-04T17:47:35.354 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00098649799999999993 2024-09-04T17:47:35.354 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.354 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.354 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:47:35.354 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.311905+0000", 2024-09-04T17:47:35.355 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7070999999999994e-05 2024-09-04T17:47:35.355 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.355 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.355 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:35.355 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.311968+0000", 2024-09-04T17:47:35.356 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.2928000000000005e-05 2024-09-04T17:47:35.356 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:35.356 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:35.356 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:35.356 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.356 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.357 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37359 3.f e58/24 3:f29e50a5:::rbd_data.1134c5a43465.0000000000000021:head v 58'135, mlcod=58'133)", 2024-09-04T17:47:35.357 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:47:35.357 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 55.719178599999999, 2024-09-04T17:47:35.357 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053483197000000003, 2024-09-04T17:47:35.357 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:35.358 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:35.358 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:35.358 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:35.358 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.358 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:35.359 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:47:35.359 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:35.359 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.359 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.359 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:35.360 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:47:35.360 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:35.360 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.360 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.360 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:35.361 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:47:35.361 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1699999999999997e-07 2024-09-04T17:47:35.361 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.361 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.361 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:35.361 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617338+0000", 2024-09-04T17:47:35.362 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.1168999999999998e-05 2024-09-04T17:47:35.362 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.362 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.362 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:35.362 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617351+0000", 2024-09-04T17:47:35.363 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3094e-05 2024-09-04T17:47:35.363 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.363 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.363 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:35.363 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617359+0000", 2024-09-04T17:47:35.364 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7970000000000006e-06 2024-09-04T17:47:35.364 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.364 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.364 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:35.364 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617417+0000", 2024-09-04T17:47:35.365 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7079e-05 2024-09-04T17:47:35.365 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.365 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.365 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:35.365 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617438+0000", 2024-09-04T17:47:35.366 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1654999999999998e-05 2024-09-04T17:47:35.366 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.366 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.366 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:47:35.366 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670738+0000", 2024-09-04T17:47:35.366 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053300142000000002 2024-09-04T17:47:35.367 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.367 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.367 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:35.367 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670779+0000", 2024-09-04T17:47:35.367 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0343999999999999e-05 2024-09-04T17:47:35.368 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:35.368 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:35.368 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:35.368 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.368 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.369 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37358 3.1d e58/24 3:bd3215f3:::rbd_data.1134c5a43465.0000000000000020:head v 58'91, mlcod=58'89)", 2024-09-04T17:47:35.369 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617502+0000", 2024-09-04T17:47:35.369 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 55.718971881999998, 2024-09-04T17:47:35.369 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053505641999999999, 2024-09-04T17:47:35.369 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:35.369 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:35.370 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:35.370 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:35.370 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.370 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:35.370 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617502+0000", 2024-09-04T17:47:35.371 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:35.371 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.371 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.371 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:35.371 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617502+0000", 2024-09-04T17:47:35.372 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:35.372 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.372 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.372 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:35.372 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617503+0000", 2024-09-04T17:47:35.373 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.184e-06 2024-09-04T17:47:35.373 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.373 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.373 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:35.373 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617526+0000", 2024-09-04T17:47:35.373 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2099999999999998e-05 2024-09-04T17:47:35.374 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.374 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.374 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:35.374 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617540+0000", 2024-09-04T17:47:35.374 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4741e-05 2024-09-04T17:47:35.375 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.375 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.375 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:35.375 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617550+0000", 2024-09-04T17:47:35.375 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4259999999999992e-06 2024-09-04T17:47:35.376 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.376 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.376 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:35.376 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617613+0000", 2024-09-04T17:47:35.376 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.3625000000000002e-05 2024-09-04T17:47:35.376 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.377 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.377 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:35.377 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617637+0000", 2024-09-04T17:47:35.377 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.3604999999999998e-05 2024-09-04T17:47:35.377 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.378 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.378 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:47:35.378 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670967+0000", 2024-09-04T17:47:35.378 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053329881000000003 2024-09-04T17:47:35.378 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.379 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.379 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:35.379 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671008+0000", 2024-09-04T17:47:35.379 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.108e-05 2024-09-04T17:47:35.379 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:35.379 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:35.380 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:35.380 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.380 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.380 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37361 3.1c e58/24 3:3c7f8976:::rbd_data.1134c5a43465.0000000000000023:head v 58'159, mlcod=58'157)", 2024-09-04T17:47:35.380 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617578+0000", 2024-09-04T17:47:35.381 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 55.718896032000004, 2024-09-04T17:47:35.381 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053990367999999997, 2024-09-04T17:47:35.381 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:35.381 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:35.381 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:35.382 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:35.382 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.382 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:35.382 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617578+0000", 2024-09-04T17:47:35.382 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:35.382 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.383 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.383 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:35.383 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617578+0000", 2024-09-04T17:47:35.383 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:35.383 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.384 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.384 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:35.384 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617579+0000", 2024-09-04T17:47:35.384 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7299999999999996e-07 2024-09-04T17:47:35.384 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.385 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.385 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:35.385 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617610+0000", 2024-09-04T17:47:35.385 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0951000000000002e-05 2024-09-04T17:47:35.385 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.385 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.386 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:35.386 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:47:35.386 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0002e-05 2024-09-04T17:47:35.386 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.386 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.387 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:35.387 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617626+0000", 2024-09-04T17:47:35.387 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6660000000000002e-06 2024-09-04T17:47:35.387 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.387 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.388 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:35.388 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670548+0000", 2024-09-04T17:47:35.388 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.052921943999999999 2024-09-04T17:47:35.388 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.388 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.388 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:35.389 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670574+0000", 2024-09-04T17:47:35.389 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5405000000000002e-05 2024-09-04T17:47:35.389 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.389 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.389 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:47:35.390 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671522+0000", 2024-09-04T17:47:35.390 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00094815299999999997 2024-09-04T17:47:35.390 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.390 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.390 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:35.391 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671568+0000", 2024-09-04T17:47:35.391 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6674000000000001e-05 2024-09-04T17:47:35.391 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:35.391 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:35.391 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:35.391 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.392 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.392 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37357 3.15 e58/49 3:a897312a:::rbd_data.1134c5a43465.000000000000001f:head v 58'117, mlcod=58'115)", 2024-09-04T17:47:35.392 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:47:35.392 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 55.718853694000003, 2024-09-04T17:47:35.392 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053734297, 2024-09-04T17:47:35.393 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:35.393 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:35.393 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:35.393 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:35.393 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.394 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:35.394 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:47:35.394 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:35.394 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.394 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.395 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:35.395 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:47:35.395 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:35.395 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.395 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.395 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:35.396 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617621+0000", 2024-09-04T17:47:35.396 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.7300000000000004e-07 2024-09-04T17:47:35.396 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.396 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.396 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:35.397 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617642+0000", 2024-09-04T17:47:35.397 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0185000000000001e-05 2024-09-04T17:47:35.397 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.397 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.397 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:35.398 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617654+0000", 2024-09-04T17:47:35.398 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2745000000000001e-05 2024-09-04T17:47:35.398 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.398 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.398 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:35.399 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617662+0000", 2024-09-04T17:47:35.399 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.0499999999999992e-06 2024-09-04T17:47:35.399 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.399 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.399 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:35.399 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617727+0000", 2024-09-04T17:47:35.400 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.4923000000000002e-05 2024-09-04T17:47:35.400 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.400 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.400 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:35.400 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617753+0000", 2024-09-04T17:47:35.401 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5899999999999999e-05 2024-09-04T17:47:35.401 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.401 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.401 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:47:35.401 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671298+0000", 2024-09-04T17:47:35.402 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053545276000000003 2024-09-04T17:47:35.402 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.402 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.402 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:35.402 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671355+0000", 2024-09-04T17:47:35.402 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6245000000000001e-05 2024-09-04T17:47:35.403 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:35.403 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:35.403 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:35.403 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.403 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.403 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44687 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:35.403 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253012+0000", 2024-09-04T17:47:35.403 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 45.083461626999998, 2024-09-04T17:47:35.403 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051128777, 2024-09-04T17:47:35.403 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:35.403 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:35.403 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:35.403 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:35.404 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:35.404 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:35.404 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44687 2024-09-04T17:47:35.404 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.404 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:35.404 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.404 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:35.404 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253012+0000", 2024-09-04T17:47:35.404 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:35.404 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.404 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.404 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:35.404 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253012+0000", 2024-09-04T17:47:35.404 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:35.405 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.405 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.405 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:35.405 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253014+0000", 2024-09-04T17:47:35.405 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.24e-06 2024-09-04T17:47:35.405 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.405 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.405 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:35.405 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253019+0000", 2024-09-04T17:47:35.405 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4569999999999996e-06 2024-09-04T17:47:35.405 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.405 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.405 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:35.405 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253035+0000", 2024-09-04T17:47:35.406 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6291999999999999e-05 2024-09-04T17:47:35.406 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.406 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.406 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:35.406 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253050+0000", 2024-09-04T17:47:35.406 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4423000000000001e-05 2024-09-04T17:47:35.406 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.406 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.406 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:35.406 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304115+0000", 2024-09-04T17:47:35.406 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051064778999999998 2024-09-04T17:47:35.406 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.406 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.407 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:35.407 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304141+0000", 2024-09-04T17:47:35.407 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6585999999999999e-05 2024-09-04T17:47:35.407 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:35.407 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:35.407 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:35.407 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.407 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.407 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44688 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:35.407 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253067+0000", 2024-09-04T17:47:35.407 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 45.083407074999997, 2024-09-04T17:47:35.407 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051116979, 2024-09-04T17:47:35.407 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:35.408 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:35.408 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:35.408 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:35.408 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:35.408 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:35.408 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44688 2024-09-04T17:47:35.408 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.408 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:35.408 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.408 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:35.408 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253067+0000", 2024-09-04T17:47:35.408 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:35.408 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.408 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.409 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:35.409 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253067+0000", 2024-09-04T17:47:35.409 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:35.409 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.409 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.409 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:35.409 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253068+0000", 2024-09-04T17:47:35.409 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0729999999999999e-06 2024-09-04T17:47:35.409 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.409 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.409 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:35.409 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253073+0000", 2024-09-04T17:47:35.409 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2259999999999996e-06 2024-09-04T17:47:35.410 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.410 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.410 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:35.410 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253091+0000", 2024-09-04T17:47:35.410 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7255999999999999e-05 2024-09-04T17:47:35.410 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.410 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.410 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:35.410 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253103+0000", 2024-09-04T17:47:35.410 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2632999999999999e-05 2024-09-04T17:47:35.410 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.410 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.410 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:35.410 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304173+0000", 2024-09-04T17:47:35.411 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051069537999999998 2024-09-04T17:47:35.411 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.411 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.411 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:35.411 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304184+0000", 2024-09-04T17:47:35.411 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1253000000000001e-05 2024-09-04T17:47:35.411 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:35.411 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:35.411 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:35.411 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.411 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.411 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44689 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:35.411 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253121+0000", 2024-09-04T17:47:35.412 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 45.083352937999997, 2024-09-04T17:47:35.412 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051192595, 2024-09-04T17:47:35.412 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:35.412 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:35.412 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:35.412 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:35.412 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:35.412 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:35.412 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44689 2024-09-04T17:47:35.412 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.412 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:35.412 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.412 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:35.413 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253121+0000", 2024-09-04T17:47:35.413 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:35.413 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.413 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.413 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:35.413 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253121+0000", 2024-09-04T17:47:35.413 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:35.413 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.413 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.413 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:35.413 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253122+0000", 2024-09-04T17:47:35.413 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.15e-06 2024-09-04T17:47:35.413 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.413 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.414 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:35.414 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253128+0000", 2024-09-04T17:47:35.414 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2569999999999998e-06 2024-09-04T17:47:35.414 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.414 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.414 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:35.414 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253145+0000", 2024-09-04T17:47:35.414 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7176e-05 2024-09-04T17:47:35.414 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.414 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.414 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:35.414 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253158+0000", 2024-09-04T17:47:35.414 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3139000000000001e-05 2024-09-04T17:47:35.414 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.415 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.415 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:35.415 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304287+0000", 2024-09-04T17:47:35.415 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051129149999999998 2024-09-04T17:47:35.415 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.415 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.415 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:35.415 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304314+0000", 2024-09-04T17:47:35.415 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6723000000000001e-05 2024-09-04T17:47:35.415 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:35.415 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:35.415 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:35.415 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.416 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.416 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44690 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:35.416 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253177+0000", 2024-09-04T17:47:35.416 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 45.083296812999997, 2024-09-04T17:47:35.416 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051169263, 2024-09-04T17:47:35.416 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:35.416 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:35.416 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:35.416 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:35.416 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:35.416 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:35.416 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44690 2024-09-04T17:47:35.416 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.416 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:35.417 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.417 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:35.417 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253177+0000", 2024-09-04T17:47:35.417 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:35.417 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.417 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.417 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:35.417 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253177+0000", 2024-09-04T17:47:35.417 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:35.417 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.417 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.417 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:35.417 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253178+0000", 2024-09-04T17:47:35.418 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1000000000000001e-06 2024-09-04T17:47:35.418 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.418 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.418 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:35.418 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253184+0000", 2024-09-04T17:47:35.418 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2089999999999996e-06 2024-09-04T17:47:35.418 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.418 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.418 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:35.418 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253200+0000", 2024-09-04T17:47:35.418 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6087e-05 2024-09-04T17:47:35.418 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.418 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.418 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:35.419 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253231+0000", 2024-09-04T17:47:35.419 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0874999999999998e-05 2024-09-04T17:47:35.419 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.419 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.419 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:35.419 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304337+0000", 2024-09-04T17:47:35.419 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051106430000000001 2024-09-04T17:47:35.419 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.419 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.419 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:35.419 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304347+0000", 2024-09-04T17:47:35.419 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5619999999999993e-06 2024-09-04T17:47:35.419 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:35.419 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:35.420 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:35.420 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.420 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.420 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44691 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:35.420 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253249+0000", 2024-09-04T17:47:35.420 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 45.083224778999998, 2024-09-04T17:47:35.420 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051195630999999998, 2024-09-04T17:47:35.420 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:35.420 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:35.420 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:35.420 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:35.420 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:35.420 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:35.421 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44691 2024-09-04T17:47:35.421 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.421 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:35.421 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.421 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:35.421 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253249+0000", 2024-09-04T17:47:35.421 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:35.421 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.421 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.421 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:35.421 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253249+0000", 2024-09-04T17:47:35.421 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:35.421 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.421 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.422 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:35.422 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253251+0000", 2024-09-04T17:47:35.422 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2330000000000001e-06 2024-09-04T17:47:35.422 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.422 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.422 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:35.422 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253256+0000", 2024-09-04T17:47:35.422 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6099999999999997e-06 2024-09-04T17:47:35.422 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.422 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.422 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:35.422 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253274+0000", 2024-09-04T17:47:35.422 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7952999999999999e-05 2024-09-04T17:47:35.423 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.423 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.423 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:35.423 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253287+0000", 2024-09-04T17:47:35.423 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2839e-05 2024-09-04T17:47:35.423 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.423 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.423 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:35.423 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304396+0000", 2024-09-04T17:47:35.423 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051108990999999999 2024-09-04T17:47:35.423 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.423 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.423 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:35.423 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304445+0000", 2024-09-04T17:47:35.424 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9005000000000003e-05 2024-09-04T17:47:35.424 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:35.424 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:35.424 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:35.424 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.424 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.424 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44692 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:35.424 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253337+0000", 2024-09-04T17:47:35.424 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 45.083137206000004, 2024-09-04T17:47:35.424 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051138909000000003, 2024-09-04T17:47:35.424 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:35.424 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:35.424 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:35.425 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:35.425 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:35.425 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:35.425 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44692 2024-09-04T17:47:35.425 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.425 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:35.425 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.425 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:35.425 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253337+0000", 2024-09-04T17:47:35.425 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:35.425 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.425 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.425 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:35.425 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253337+0000", 2024-09-04T17:47:35.426 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:35.426 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.426 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.426 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:35.426 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253338+0000", 2024-09-04T17:47:35.426 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.463e-06 2024-09-04T17:47:35.426 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.426 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.426 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:35.426 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253357+0000", 2024-09-04T17:47:35.426 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9043e-05 2024-09-04T17:47:35.426 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.426 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.426 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:35.427 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253379+0000", 2024-09-04T17:47:35.427 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1243e-05 2024-09-04T17:47:35.427 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.427 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.427 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:35.427 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253396+0000", 2024-09-04T17:47:35.427 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7059e-05 2024-09-04T17:47:35.427 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.427 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.427 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:35.427 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304466+0000", 2024-09-04T17:47:35.427 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051070624000000002 2024-09-04T17:47:35.427 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.428 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.428 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:35.428 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304476+0000", 2024-09-04T17:47:35.428 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4769999999999992e-06 2024-09-04T17:47:35.428 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:35.428 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:35.428 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:35.428 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.428 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.428 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44693 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:35.428 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253415+0000", 2024-09-04T17:47:35.428 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 45.083058803, 2024-09-04T17:47:35.428 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051141949999999999, 2024-09-04T17:47:35.429 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:35.429 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:35.429 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:35.429 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:35.429 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:35.429 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:35.429 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44693 2024-09-04T17:47:35.429 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.429 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:35.429 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.429 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:35.429 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253415+0000", 2024-09-04T17:47:35.429 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:35.429 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.430 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.430 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:35.430 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253415+0000", 2024-09-04T17:47:35.430 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:35.430 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.430 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.430 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:35.430 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253416+0000", 2024-09-04T17:47:35.430 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1799999999999999e-06 2024-09-04T17:47:35.430 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.430 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.430 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:35.430 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253422+0000", 2024-09-04T17:47:35.430 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.5940000000000001e-06 2024-09-04T17:47:35.431 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.431 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.431 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:35.431 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253438+0000", 2024-09-04T17:47:35.431 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6008999999999999e-05 2024-09-04T17:47:35.431 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.431 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.431 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:35.431 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253452+0000", 2024-09-04T17:47:35.431 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3543000000000001e-05 2024-09-04T17:47:35.431 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.431 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.431 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:35.432 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304533+0000", 2024-09-04T17:47:35.432 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051081609 2024-09-04T17:47:35.432 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.432 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.432 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:35.432 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304557+0000", 2024-09-04T17:47:35.432 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4015e-05 2024-09-04T17:47:35.432 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:35.432 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:35.432 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:35.432 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.432 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.432 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44694 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:35.432 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253469+0000", 2024-09-04T17:47:35.433 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 45.083004678000002, 2024-09-04T17:47:35.433 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051121486000000001, 2024-09-04T17:47:35.433 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:35.433 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:35.433 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:35.433 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:35.433 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:35.433 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:35.433 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44694 2024-09-04T17:47:35.433 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.433 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:35.433 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.433 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:35.434 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253469+0000", 2024-09-04T17:47:35.434 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:35.434 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.434 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.434 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:35.434 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253469+0000", 2024-09-04T17:47:35.434 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:35.434 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.434 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.434 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:35.434 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253471+0000", 2024-09-04T17:47:35.434 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1400000000000001e-06 2024-09-04T17:47:35.434 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.435 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.435 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:35.435 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253476+0000", 2024-09-04T17:47:35.435 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7370000000000003e-06 2024-09-04T17:47:35.435 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.435 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.435 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:35.435 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253493+0000", 2024-09-04T17:47:35.435 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6549e-05 2024-09-04T17:47:35.435 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.435 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.435 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:35.435 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253506+0000", 2024-09-04T17:47:35.435 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3406e-05 2024-09-04T17:47:35.436 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.436 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.436 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:35.436 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304582+0000", 2024-09-04T17:47:35.436 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051075534999999998 2024-09-04T17:47:35.436 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.436 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.436 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:35.436 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304591+0000", 2024-09-04T17:47:35.436 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1190000000000003e-06 2024-09-04T17:47:35.436 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:35.436 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:35.436 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:35.437 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.437 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.437 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44695 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:35.437 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253552+0000", 2024-09-04T17:47:35.437 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 45.082921923999997, 2024-09-04T17:47:35.437 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051068872000000001, 2024-09-04T17:47:35.437 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:35.437 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:35.437 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:35.437 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:35.437 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:35.437 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:35.437 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44695 2024-09-04T17:47:35.438 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.438 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:35.438 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.438 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:35.438 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253552+0000", 2024-09-04T17:47:35.438 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:35.438 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.438 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.438 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:35.438 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253552+0000", 2024-09-04T17:47:35.438 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:35.438 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.438 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.439 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:35.439 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253553+0000", 2024-09-04T17:47:35.439 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1960000000000001e-06 2024-09-04T17:47:35.439 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.439 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.439 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:35.439 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253560+0000", 2024-09-04T17:47:35.439 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.2040000000000002e-06 2024-09-04T17:47:35.439 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.439 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.439 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:35.439 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253578+0000", 2024-09-04T17:47:35.439 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8241999999999999e-05 2024-09-04T17:47:35.439 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.440 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.440 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:35.440 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253591+0000", 2024-09-04T17:47:35.440 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3453e-05 2024-09-04T17:47:35.440 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.440 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.440 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:35.440 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304614+0000", 2024-09-04T17:47:35.440 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051022456000000001 2024-09-04T17:47:35.440 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.440 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.440 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:35.440 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304621+0000", 2024-09-04T17:47:35.441 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3209999999999999e-06 2024-09-04T17:47:35.441 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:35.441 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:35.441 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:35.441 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.441 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.441 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44696 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:35.441 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253609+0000", 2024-09-04T17:47:35.441 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 45.082865423000001, 2024-09-04T17:47:35.441 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051115351000000003, 2024-09-04T17:47:35.441 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:35.441 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:35.441 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:35.441 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:35.442 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:35.442 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:35.442 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44696 2024-09-04T17:47:35.442 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.442 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:35.442 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.442 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:35.442 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253609+0000", 2024-09-04T17:47:35.442 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:35.442 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.442 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.442 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:35.442 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253609+0000", 2024-09-04T17:47:35.443 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:35.443 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.443 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.443 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:35.443 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253610+0000", 2024-09-04T17:47:35.443 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2300000000000001e-06 2024-09-04T17:47:35.443 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.443 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.443 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:35.443 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253615+0000", 2024-09-04T17:47:35.443 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7260000000000002e-06 2024-09-04T17:47:35.443 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.443 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.444 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:35.444 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253632+0000", 2024-09-04T17:47:35.444 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7039999999999999e-05 2024-09-04T17:47:35.444 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.444 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.444 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:35.444 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253666+0000", 2024-09-04T17:47:35.444 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4662000000000001e-05 2024-09-04T17:47:35.444 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.444 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.444 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:35.444 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304677+0000", 2024-09-04T17:47:35.444 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051011017999999998 2024-09-04T17:47:35.445 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.445 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.445 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:35.445 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304724+0000", 2024-09-04T17:47:35.445 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6675000000000003e-05 2024-09-04T17:47:35.445 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:35.445 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:35.445 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:35.445 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.445 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.445 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44697 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:35.445 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253684+0000", 2024-09-04T17:47:35.445 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 45.082789929, 2024-09-04T17:47:35.446 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051073107, 2024-09-04T17:47:35.446 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:35.446 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:35.446 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:35.446 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:35.446 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:35.446 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:35.446 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44697 2024-09-04T17:47:35.446 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.446 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:35.446 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.446 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:35.446 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253684+0000", 2024-09-04T17:47:35.447 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:35.447 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.447 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.447 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:35.447 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253684+0000", 2024-09-04T17:47:35.447 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:35.447 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.447 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.447 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:35.447 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253685+0000", 2024-09-04T17:47:35.447 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0699999999999999e-06 2024-09-04T17:47:35.447 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.447 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.447 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:35.448 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253690+0000", 2024-09-04T17:47:35.448 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1959999999999999e-06 2024-09-04T17:47:35.448 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.448 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.448 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:35.448 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253707+0000", 2024-09-04T17:47:35.448 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6946e-05 2024-09-04T17:47:35.448 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.448 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.448 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:35.448 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253720+0000", 2024-09-04T17:47:35.448 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3076e-05 2024-09-04T17:47:35.448 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.448 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.449 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:35.449 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304745+0000", 2024-09-04T17:47:35.449 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051024760000000002 2024-09-04T17:47:35.449 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.449 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.449 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:35.449 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304757+0000", 2024-09-04T17:47:35.449 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2058999999999999e-05 2024-09-04T17:47:35.449 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:35.449 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:35.449 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:35.449 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.449 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.450 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44698 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:35.450 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253737+0000", 2024-09-04T17:47:35.450 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 45.082737578, 2024-09-04T17:47:35.450 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051176078, 2024-09-04T17:47:35.450 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:35.450 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:35.450 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:35.450 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:35.450 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:35.450 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:35.450 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44698 2024-09-04T17:47:35.450 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.450 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:35.450 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.451 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:35.451 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253737+0000", 2024-09-04T17:47:35.451 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:35.451 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.451 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.451 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:35.451 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253737+0000", 2024-09-04T17:47:35.451 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:35.451 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.451 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.451 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:35.451 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253738+0000", 2024-09-04T17:47:35.451 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0300000000000001e-06 2024-09-04T17:47:35.452 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.452 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.452 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:35.452 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253752+0000", 2024-09-04T17:47:35.452 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4358999999999999e-05 2024-09-04T17:47:35.452 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.452 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.452 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:35.452 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253771+0000", 2024-09-04T17:47:35.452 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8879999999999999e-05 2024-09-04T17:47:35.452 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.452 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.452 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:35.453 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253785+0000", 2024-09-04T17:47:35.453 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3852000000000001e-05 2024-09-04T17:47:35.453 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.453 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.453 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:35.453 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304876+0000", 2024-09-04T17:47:35.453 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051091105999999997 2024-09-04T17:47:35.453 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.453 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.453 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:35.453 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304913+0000", 2024-09-04T17:47:35.453 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.6850999999999997e-05 2024-09-04T17:47:35.453 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:35.453 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:35.454 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:35.454 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.454 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.454 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44699 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:35.454 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253804+0000", 2024-09-04T17:47:35.454 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 45.082670053999998, 2024-09-04T17:47:35.454 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051172250000000002, 2024-09-04T17:47:35.454 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:35.454 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:35.454 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:35.454 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:35.454 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:35.455 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:35.455 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44699 2024-09-04T17:47:35.455 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.455 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:35.455 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.455 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:35.455 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253804+0000", 2024-09-04T17:47:35.455 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:35.455 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.455 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.455 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:35.455 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253804+0000", 2024-09-04T17:47:35.455 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:35.455 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.456 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.456 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:35.456 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253805+0000", 2024-09-04T17:47:35.456 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.08e-06 2024-09-04T17:47:35.456 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.456 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.456 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:35.456 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253810+0000", 2024-09-04T17:47:35.456 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1830000000000002e-06 2024-09-04T17:47:35.456 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.456 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.456 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:35.456 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253828+0000", 2024-09-04T17:47:35.457 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7662000000000001e-05 2024-09-04T17:47:35.457 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.457 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.457 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:35.457 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253841+0000", 2024-09-04T17:47:35.457 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3203999999999999e-05 2024-09-04T17:47:35.457 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.457 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.457 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:35.457 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304966+0000", 2024-09-04T17:47:35.457 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051125273999999998 2024-09-04T17:47:35.457 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.457 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.457 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:35.458 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304976+0000", 2024-09-04T17:47:35.458 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.8470000000000008e-06 2024-09-04T17:47:35.458 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:35.458 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:35.458 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:35.458 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.458 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.458 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44700 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:35.458 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253858+0000", 2024-09-04T17:47:35.458 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 45.082616399999999, 2024-09-04T17:47:35.458 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051218957000000002, 2024-09-04T17:47:35.458 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:35.458 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:35.459 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:35.459 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:35.459 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:35.459 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:35.459 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44700 2024-09-04T17:47:35.459 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.459 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:35.459 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.459 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:35.459 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253858+0000", 2024-09-04T17:47:35.459 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:35.459 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.459 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.459 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:35.460 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253858+0000", 2024-09-04T17:47:35.460 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:35.460 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.460 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.460 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:35.460 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253859+0000", 2024-09-04T17:47:35.460 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.217e-06 2024-09-04T17:47:35.460 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.460 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.460 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:35.460 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253864+0000", 2024-09-04T17:47:35.460 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7600000000000002e-06 2024-09-04T17:47:35.460 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.461 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.461 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:35.461 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253880+0000", 2024-09-04T17:47:35.461 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6501999999999999e-05 2024-09-04T17:47:35.461 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.461 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.461 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:35.461 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253899+0000", 2024-09-04T17:47:35.461 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8907e-05 2024-09-04T17:47:35.461 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.461 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.461 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:35.461 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305052+0000", 2024-09-04T17:47:35.461 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051152989000000003 2024-09-04T17:47:35.462 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.462 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.462 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:35.462 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305077+0000", 2024-09-04T17:47:35.462 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4581999999999999e-05 2024-09-04T17:47:35.462 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:35.462 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:35.462 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:35.462 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.462 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.462 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44701 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:35.462 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253917+0000", 2024-09-04T17:47:35.462 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 45.082557047999998, 2024-09-04T17:47:35.463 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051231113000000002, 2024-09-04T17:47:35.463 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:35.463 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:35.463 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:35.463 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:35.463 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:35.463 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:35.463 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44701 2024-09-04T17:47:35.463 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.463 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:35.463 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.463 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:35.463 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253917+0000", 2024-09-04T17:47:35.463 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:35.464 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.464 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.464 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:35.464 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253917+0000", 2024-09-04T17:47:35.464 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:35.464 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.464 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.464 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:35.464 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253918+0000", 2024-09-04T17:47:35.464 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.136e-06 2024-09-04T17:47:35.464 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.464 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.464 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:35.465 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253924+0000", 2024-09-04T17:47:35.465 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.3070000000000002e-06 2024-09-04T17:47:35.465 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.465 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.465 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:35.465 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253940+0000", 2024-09-04T17:47:35.465 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6612e-05 2024-09-04T17:47:35.465 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.465 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.465 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:35.465 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253954+0000", 2024-09-04T17:47:35.465 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3709999999999999e-05 2024-09-04T17:47:35.465 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.465 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.466 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:35.466 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305136+0000", 2024-09-04T17:47:35.466 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051181935999999997 2024-09-04T17:47:35.466 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.466 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.466 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:35.466 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305148+0000", 2024-09-04T17:47:35.466 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2412000000000001e-05 2024-09-04T17:47:35.466 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:35.466 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:35.466 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:35.466 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:35.466 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:35.467 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:35.468 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-04T17:47:35.468 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:35.465+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:35.569 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:35.569+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:35.638 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:35.639 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:47:35.639 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:47:35.639 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:35.654 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-04T17:47:35.697 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:35.695+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:35.771 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:35.769+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:35.772 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:35.772 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:47:35.773 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:47:35.773 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:47:35.773 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:35.787 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-04T17:47:35.877 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:35.875+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:35.952 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:35.953 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:47:35.953 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:47:35.953 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:47:35.953 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.953 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8831 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2453504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:35.953 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:47:35.953 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 82.893042617000006, 2024-09-04T17:47:35.953 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14731111799999999, 2024-09-04T17:47:35.953 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:35.953 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:35.953 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:35.953 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:35.954 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:35.954 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:35.954 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8831 2024-09-04T17:47:35.954 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.954 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:35.954 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.954 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:35.954 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:47:35.954 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:35.954 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.954 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.954 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:35.955 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:47:35.955 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:35.955 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.955 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.955 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:35.955 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057811+0000", 2024-09-04T17:47:35.955 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.083e-06 2024-09-04T17:47:35.955 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.955 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.955 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:35.955 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057814+0000", 2024-09-04T17:47:35.955 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.407e-06 2024-09-04T17:47:35.955 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.956 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.956 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:35.956 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057825+0000", 2024-09-04T17:47:35.956 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.17e-05 2024-09-04T17:47:35.956 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.956 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.956 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:35.956 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057836+0000", 2024-09-04T17:47:35.956 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0283e-05 2024-09-04T17:47:35.956 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.956 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.956 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:35.956 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.204762+0000", 2024-09-04T17:47:35.957 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.146926205 2024-09-04T17:47:35.957 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.957 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.957 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:35.957 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.204895+0000", 2024-09-04T17:47:35.957 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013328999999999999 2024-09-04T17:47:35.957 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.957 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.957 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:35.957 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205121+0000", 2024-09-04T17:47:35.957 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00022614999999999999 2024-09-04T17:47:35.957 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:35.957 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:35.958 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:35.958 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.958 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.958 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8832 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2490368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:35.958 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:47:35.958 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 82.893004583999996, 2024-09-04T17:47:35.958 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14770012299999999, 2024-09-04T17:47:35.958 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:35.958 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:35.958 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:35.958 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:35.958 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:35.958 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:35.958 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8832 2024-09-04T17:47:35.959 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.959 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:35.959 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.959 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:35.959 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:47:35.959 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:35.959 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.959 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.959 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:35.959 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:47:35.959 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:35.959 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.960 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.960 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:35.960 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057849+0000", 2024-09-04T17:47:35.960 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6700000000000003e-07 2024-09-04T17:47:35.960 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.960 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.960 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:35.960 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057852+0000", 2024-09-04T17:47:35.960 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6359999999999998e-06 2024-09-04T17:47:35.960 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.960 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.960 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:35.961 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057860+0000", 2024-09-04T17:47:35.961 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8140000000000007e-06 2024-09-04T17:47:35.961 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.961 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.961 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:35.961 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057868+0000", 2024-09-04T17:47:35.961 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1999999999999997e-06 2024-09-04T17:47:35.961 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.961 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.961 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:35.961 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205208+0000", 2024-09-04T17:47:35.961 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14734067300000001 2024-09-04T17:47:35.961 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.962 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.962 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:35.962 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205347+0000", 2024-09-04T17:47:35.962 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013914900000000001 2024-09-04T17:47:35.962 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.962 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.962 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:35.962 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205548+0000", 2024-09-04T17:47:35.963 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000200984 2024-09-04T17:47:35.963 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:35.963 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:35.963 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:35.963 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.963 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.963 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8833 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2527232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:35.963 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:47:35.963 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 82.892972713999995, 2024-09-04T17:47:35.964 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148062572, 2024-09-04T17:47:35.964 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:35.964 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:35.964 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:35.964 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:35.964 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:35.964 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:35.964 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8833 2024-09-04T17:47:35.964 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.964 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:35.964 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.964 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:35.964 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:47:35.965 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:35.965 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.965 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.965 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:35.965 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:47:35.965 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:35.965 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.965 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.965 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:35.965 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057881+0000", 2024-09-04T17:47:35.965 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4300000000000002e-07 2024-09-04T17:47:35.965 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.965 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.966 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:35.966 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057893+0000", 2024-09-04T17:47:35.966 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1973999999999999e-05 2024-09-04T17:47:35.966 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.966 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.966 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:35.966 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057906+0000", 2024-09-04T17:47:35.966 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2716e-05 2024-09-04T17:47:35.966 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.966 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.966 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:35.966 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057913+0000", 2024-09-04T17:47:35.966 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3930000000000002e-06 2024-09-04T17:47:35.967 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.967 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.967 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:35.967 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205621+0000", 2024-09-04T17:47:35.967 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14770788800000001 2024-09-04T17:47:35.967 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.967 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.967 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:35.967 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205740+0000", 2024-09-04T17:47:35.967 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000118936 2024-09-04T17:47:35.967 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.967 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.967 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:35.967 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205943+0000", 2024-09-04T17:47:35.967 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020292199999999999 2024-09-04T17:47:35.968 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:35.968 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:35.968 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:35.968 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.968 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.968 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8834 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2564096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:35.968 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:47:35.968 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 82.892929434999999, 2024-09-04T17:47:35.968 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14846753200000001, 2024-09-04T17:47:35.968 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:35.968 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:35.968 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:35.969 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:35.969 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:35.969 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:35.969 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8834 2024-09-04T17:47:35.969 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.969 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:35.969 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.969 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:35.969 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:47:35.969 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:35.969 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.969 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.969 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:35.970 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:47:35.970 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:35.970 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.970 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.970 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:35.970 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057925+0000", 2024-09-04T17:47:35.970 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0270000000000001e-06 2024-09-04T17:47:35.970 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.970 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.970 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:35.970 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057936+0000", 2024-09-04T17:47:35.970 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.112e-05 2024-09-04T17:47:35.970 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.971 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.971 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:35.971 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057946+0000", 2024-09-04T17:47:35.971 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.009e-05 2024-09-04T17:47:35.971 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.971 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.971 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:35.971 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057953+0000", 2024-09-04T17:47:35.971 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.6229999999999997e-06 2024-09-04T17:47:35.971 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.971 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.971 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:35.971 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206027+0000", 2024-09-04T17:47:35.971 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148073591 2024-09-04T17:47:35.971 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.972 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.972 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:35.972 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206161+0000", 2024-09-04T17:47:35.972 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013411599999999999 2024-09-04T17:47:35.972 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.972 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.972 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:35.972 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206391+0000", 2024-09-04T17:47:35.972 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000229965 2024-09-04T17:47:35.973 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:35.973 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:35.973 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:35.973 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.973 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.973 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8835 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2600960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:35.973 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:47:35.973 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 82.892889784999994, 2024-09-04T17:47:35.973 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14882941599999999, 2024-09-04T17:47:35.973 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:35.973 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:35.973 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:35.973 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:35.974 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:35.974 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:35.974 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8835 2024-09-04T17:47:35.974 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.974 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:35.974 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.974 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:35.974 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:47:35.974 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:35.974 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.974 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.974 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:35.974 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:47:35.974 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:35.975 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.975 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.975 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:35.975 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058544+0000", 2024-09-04T17:47:35.975 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00058061899999999999 2024-09-04T17:47:35.975 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.975 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.975 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:35.975 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058550+0000", 2024-09-04T17:47:35.975 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.1129999999999997e-06 2024-09-04T17:47:35.975 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.975 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.975 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:35.976 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058566+0000", 2024-09-04T17:47:35.976 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5820000000000001e-05 2024-09-04T17:47:35.976 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.976 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.976 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:35.976 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058578+0000", 2024-09-04T17:47:35.976 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2537e-05 2024-09-04T17:47:35.976 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.976 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.976 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:35.976 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206470+0000", 2024-09-04T17:47:35.976 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14789153799999999 2024-09-04T17:47:35.976 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.976 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.977 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:35.977 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206589+0000", 2024-09-04T17:47:35.977 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000118837 2024-09-04T17:47:35.977 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.977 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.977 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:35.977 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206793+0000", 2024-09-04T17:47:35.977 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000203952 2024-09-04T17:47:35.977 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:35.977 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:35.977 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:35.977 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.978 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.978 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8836 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2654208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:35.978 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:47:35.978 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 82.892257096999998, 2024-09-04T17:47:35.978 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14862433999999999, 2024-09-04T17:47:35.978 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:35.978 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:35.978 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:35.978 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:35.978 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:35.978 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:35.978 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8836 2024-09-04T17:47:35.979 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.979 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:35.979 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.979 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:35.979 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:47:35.979 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:35.979 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.979 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.979 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:35.979 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:47:35.979 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:35.979 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.980 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.980 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:35.980 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062663+0000", 2024-09-04T17:47:35.980 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0040670619999999998 2024-09-04T17:47:35.980 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.980 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.980 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:35.980 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062668+0000", 2024-09-04T17:47:35.980 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4199999999999998e-06 2024-09-04T17:47:35.980 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.980 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.980 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:35.980 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062689+0000", 2024-09-04T17:47:35.981 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0356999999999998e-05 2024-09-04T17:47:35.981 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.981 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.981 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:35.981 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062726+0000", 2024-09-04T17:47:35.981 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7302999999999998e-05 2024-09-04T17:47:35.981 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.981 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.981 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:35.981 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206875+0000", 2024-09-04T17:47:35.981 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14414883100000001 2024-09-04T17:47:35.981 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.981 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.982 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:35.982 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.207016+0000", 2024-09-04T17:47:35.982 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000140822 2024-09-04T17:47:35.982 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.982 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.982 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:35.982 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.207220+0000", 2024-09-04T17:47:35.982 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020454499999999999 2024-09-04T17:47:35.982 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:35.983 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:35.983 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:35.983 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.983 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.983 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8844 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2928640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:35.983 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:47:35.983 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 82.887893013999999, 2024-09-04T17:47:35.983 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14752153100000001, 2024-09-04T17:47:35.983 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:35.983 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:35.983 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:35.983 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:35.983 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:35.983 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:35.983 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8844 2024-09-04T17:47:35.984 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.984 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:35.984 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.984 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:35.984 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:47:35.984 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:35.984 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.984 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.984 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:35.984 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:47:35.984 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:35.984 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.984 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.985 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:35.985 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062961+0000", 2024-09-04T17:47:35.985 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6700000000000003e-07 2024-09-04T17:47:35.985 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.985 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.985 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:35.985 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062973+0000", 2024-09-04T17:47:35.985 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2109999999999999e-05 2024-09-04T17:47:35.985 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.985 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.985 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:35.985 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062982+0000", 2024-09-04T17:47:35.985 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.6900000000000004e-06 2024-09-04T17:47:35.985 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.986 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.986 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:35.986 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062990+0000", 2024-09-04T17:47:35.986 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1899999999999998e-06 2024-09-04T17:47:35.986 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.986 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.986 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:35.986 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210090+0000", 2024-09-04T17:47:35.986 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14710026300000001 2024-09-04T17:47:35.986 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.986 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.986 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:35.986 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210225+0000", 2024-09-04T17:47:35.986 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013532400000000001 2024-09-04T17:47:35.987 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.987 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.987 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:35.987 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210481+0000", 2024-09-04T17:47:35.987 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00025628700000000001 2024-09-04T17:47:35.987 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:35.987 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:35.987 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:35.987 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.987 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.987 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8845 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2965504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:35.987 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:47:35.987 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 82.887853290999999, 2024-09-04T17:47:35.987 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147916819, 2024-09-04T17:47:35.988 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:35.988 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:35.988 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:35.988 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:35.988 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:35.988 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:35.988 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8845 2024-09-04T17:47:35.988 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.988 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:35.988 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.988 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:35.989 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:47:35.989 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:35.989 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.989 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.989 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:35.989 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:47:35.989 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:35.989 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.989 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.989 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:35.989 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:47:35.989 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.0399999999999995e-07 2024-09-04T17:47:35.990 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.990 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.990 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:35.990 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063003+0000", 2024-09-04T17:47:35.990 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6529999999999998e-06 2024-09-04T17:47:35.990 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.990 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.990 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:35.990 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063012+0000", 2024-09-04T17:47:35.990 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9500000000000007e-06 2024-09-04T17:47:35.991 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.991 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.991 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:35.991 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063027+0000", 2024-09-04T17:47:35.991 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5443000000000001e-05 2024-09-04T17:47:35.991 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.991 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.991 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:35.991 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210571+0000", 2024-09-04T17:47:35.992 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14754353100000001 2024-09-04T17:47:35.992 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.992 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.992 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:35.992 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210714+0000", 2024-09-04T17:47:35.992 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014295599999999999 2024-09-04T17:47:35.992 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.992 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.992 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:35.993 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210916+0000", 2024-09-04T17:47:35.993 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020258200000000001 2024-09-04T17:47:35.994 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:35.994 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:35.994 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:35.994 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.994 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.994 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8846 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3002368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:35.995 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:47:35.995 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 82.887815754000002, 2024-09-04T17:47:35.995 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14828011599999999, 2024-09-04T17:47:35.995 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:35.995 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:35.995 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:35.995 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:35.995 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:35.995 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:35.995 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8846 2024-09-04T17:47:35.995 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.995 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:35.995 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.996 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:35.996 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:47:35.996 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:35.996 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.996 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.996 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:35.996 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:47:35.996 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:35.996 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.996 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.996 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:35.996 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063038+0000", 2024-09-04T17:47:35.996 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.23e-07 2024-09-04T17:47:35.997 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.997 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.997 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:35.997 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063051+0000", 2024-09-04T17:47:35.997 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2757e-05 2024-09-04T17:47:35.997 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.997 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.997 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:35.997 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063070+0000", 2024-09-04T17:47:35.997 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9663000000000001e-05 2024-09-04T17:47:35.998 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.998 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.998 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:35.998 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063079+0000", 2024-09-04T17:47:35.998 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.0929999999999992e-06 2024-09-04T17:47:35.998 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.998 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.998 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:35.998 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210989+0000", 2024-09-04T17:47:35.998 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14790984500000001 2024-09-04T17:47:35.999 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.999 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.999 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:35.999 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211109+0000", 2024-09-04T17:47:35.999 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000120056 2024-09-04T17:47:35.999 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:35.999 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:35.999 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:36.000 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211317+0000", 2024-09-04T17:47:36.000 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020797900000000001 2024-09-04T17:47:36.000 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.000 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:36.000 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.000 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.000 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.000 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8847 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3039232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:36.000 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:47:36.000 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 82.887741321999997, 2024-09-04T17:47:36.000 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14859408900000001, 2024-09-04T17:47:36.000 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:36.000 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:36.001 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:36.001 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:36.001 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:36.001 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:36.001 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8847 2024-09-04T17:47:36.001 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.001 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:36.001 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.001 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:36.001 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:47:36.001 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:36.001 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.001 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.002 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:36.002 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:47:36.002 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:36.002 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.002 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.002 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:36.002 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063113+0000", 2024-09-04T17:47:36.002 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.204e-06 2024-09-04T17:47:36.002 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.002 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.002 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:36.002 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063116+0000", 2024-09-04T17:47:36.003 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.3330000000000001e-06 2024-09-04T17:47:36.003 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.003 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.003 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:36.003 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063126+0000", 2024-09-04T17:47:36.003 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0166999999999999e-05 2024-09-04T17:47:36.003 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.003 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.003 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:36.003 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063134+0000", 2024-09-04T17:47:36.003 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4329999999999998e-06 2024-09-04T17:47:36.003 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.003 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.003 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:36.004 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211392+0000", 2024-09-04T17:47:36.004 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148258109 2024-09-04T17:47:36.004 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.004 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.004 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:36.004 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211506+0000", 2024-09-04T17:47:36.004 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000113813 2024-09-04T17:47:36.004 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.004 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.004 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:36.004 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211706+0000", 2024-09-04T17:47:36.004 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020002999999999999 2024-09-04T17:47:36.004 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.005 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:36.005 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.005 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.005 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.005 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8848 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:36.005 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:47:36.005 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 82.887708985000003, 2024-09-04T17:47:36.005 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14907967599999999, 2024-09-04T17:47:36.005 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:36.005 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:36.005 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:36.005 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:36.006 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:36.006 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:36.006 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8848 2024-09-04T17:47:36.006 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.006 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:36.006 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.006 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:36.006 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:47:36.006 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:36.006 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.006 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.006 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:36.006 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:47:36.006 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:36.007 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.007 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.007 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:36.007 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070589+0000", 2024-09-04T17:47:36.007 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.007445386 2024-09-04T17:47:36.007 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.007 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.007 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:36.007 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070600+0000", 2024-09-04T17:47:36.007 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0346999999999999e-05 2024-09-04T17:47:36.007 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.007 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.007 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:36.007 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070622+0000", 2024-09-04T17:47:36.008 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2289999999999998e-05 2024-09-04T17:47:36.008 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.008 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.008 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:36.008 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070636+0000", 2024-09-04T17:47:36.008 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4093e-05 2024-09-04T17:47:36.008 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.008 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.008 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:36.008 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211815+0000", 2024-09-04T17:47:36.008 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14117874699999999 2024-09-04T17:47:36.008 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.008 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.009 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:36.009 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211983+0000", 2024-09-04T17:47:36.009 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00016852800000000001 2024-09-04T17:47:36.009 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.009 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.009 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:36.009 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.212224+0000", 2024-09-04T17:47:36.009 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024028499999999999 2024-09-04T17:47:36.009 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.009 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:36.009 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.009 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.009 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.010 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8862 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2019328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:36.010 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:47:36.010 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 82.879659516999993, 2024-09-04T17:47:36.010 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14755236399999999, 2024-09-04T17:47:36.010 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:36.010 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:36.010 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:36.010 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:36.010 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:36.010 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:36.010 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8862 2024-09-04T17:47:36.010 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.010 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:36.011 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.011 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:36.011 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:47:36.011 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:36.011 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.011 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.011 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:36.011 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:47:36.011 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:36.011 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.011 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.011 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:36.011 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071194+0000", 2024-09-04T17:47:36.011 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.5000000000000002e-07 2024-09-04T17:47:36.012 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.012 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.012 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:36.012 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071197+0000", 2024-09-04T17:47:36.012 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5900000000000002e-06 2024-09-04T17:47:36.012 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.012 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.012 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:36.012 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071206+0000", 2024-09-04T17:47:36.012 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9700000000000005e-06 2024-09-04T17:47:36.012 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.012 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.013 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:36.013 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071213+0000", 2024-09-04T17:47:36.013 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.9070000000000001e-06 2024-09-04T17:47:36.013 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.013 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.013 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:36.013 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218396+0000", 2024-09-04T17:47:36.013 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147183273 2024-09-04T17:47:36.013 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.013 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.013 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:36.013 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218538+0000", 2024-09-04T17:47:36.013 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014194199999999999 2024-09-04T17:47:36.014 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.014 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.014 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:36.014 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218746+0000", 2024-09-04T17:47:36.014 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000208032 2024-09-04T17:47:36.014 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.014 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:36.014 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.014 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.014 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.014 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8863 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2056192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:36.014 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:47:36.015 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 82.879631207000003, 2024-09-04T17:47:36.015 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14792281099999999, 2024-09-04T17:47:36.015 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:36.015 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:36.015 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:36.015 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:36.015 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:36.015 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:36.015 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8863 2024-09-04T17:47:36.015 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.015 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:36.015 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.015 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:36.015 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:47:36.015 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:36.016 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.016 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.016 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:36.016 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:47:36.016 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:36.016 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.016 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.016 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:36.016 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:47:36.016 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1299999999999999e-07 2024-09-04T17:47:36.016 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.016 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.016 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:36.017 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071225+0000", 2024-09-04T17:47:36.017 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6199999999999999e-06 2024-09-04T17:47:36.017 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.017 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.017 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:36.017 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071234+0000", 2024-09-04T17:47:36.017 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8769999999999999e-06 2024-09-04T17:47:36.017 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.017 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.017 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:36.017 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071251+0000", 2024-09-04T17:47:36.017 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.738e-05 2024-09-04T17:47:36.017 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.018 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.018 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:36.018 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218798+0000", 2024-09-04T17:47:36.018 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14754645 2024-09-04T17:47:36.018 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.018 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.018 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:36.018 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218926+0000", 2024-09-04T17:47:36.018 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000128446 2024-09-04T17:47:36.018 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.018 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.018 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:36.018 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219145+0000", 2024-09-04T17:47:36.018 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021832500000000001 2024-09-04T17:47:36.019 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.019 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:36.019 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.019 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.019 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.019 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8864 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2093056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:36.019 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:47:36.019 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 82.879589631000002, 2024-09-04T17:47:36.019 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148277309, 2024-09-04T17:47:36.019 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:36.019 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:36.019 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:36.019 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:36.020 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:36.020 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:36.020 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8864 2024-09-04T17:47:36.020 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.020 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:36.020 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.020 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:36.020 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:47:36.020 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:36.020 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.020 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.020 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:36.021 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:47:36.021 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:36.021 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.021 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.021 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:36.021 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071264+0000", 2024-09-04T17:47:36.021 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3e-07 2024-09-04T17:47:36.021 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.021 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.021 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:36.021 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071267+0000", 2024-09-04T17:47:36.021 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0139999999999999e-06 2024-09-04T17:47:36.021 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.022 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.022 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:36.022 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071276+0000", 2024-09-04T17:47:36.022 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1689999999999998e-06 2024-09-04T17:47:36.022 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.022 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.022 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:36.022 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071283+0000", 2024-09-04T17:47:36.022 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2269999999999996e-06 2024-09-04T17:47:36.022 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.022 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.022 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:36.022 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219195+0000", 2024-09-04T17:47:36.022 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147911868 2024-09-04T17:47:36.023 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.023 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.023 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:36.023 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219324+0000", 2024-09-04T17:47:36.023 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00012835900000000001 2024-09-04T17:47:36.023 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.023 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.023 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:36.023 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219541+0000", 2024-09-04T17:47:36.023 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000216942 2024-09-04T17:47:36.023 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.023 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:36.023 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.023 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.024 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.024 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8865 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2109440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:36.024 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:47:36.024 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 82.879560170999994, 2024-09-04T17:47:36.024 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14865747300000001, 2024-09-04T17:47:36.024 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:36.024 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:36.024 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:36.024 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:36.024 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:36.024 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:36.024 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8865 2024-09-04T17:47:36.024 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.024 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:36.025 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.025 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:36.025 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:47:36.025 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:36.025 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.025 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.025 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:36.025 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:47:36.025 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:36.025 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.025 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.025 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:36.025 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:47:36.025 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7999999999999995e-07 2024-09-04T17:47:36.026 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.026 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.026 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:36.026 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071296+0000", 2024-09-04T17:47:36.026 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6900000000000001e-06 2024-09-04T17:47:36.026 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.026 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.026 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:36.026 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071305+0000", 2024-09-04T17:47:36.026 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.5499999999999995e-06 2024-09-04T17:47:36.026 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.026 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.026 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:36.026 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071312+0000", 2024-09-04T17:47:36.026 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1559999999999998e-06 2024-09-04T17:47:36.026 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.027 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.027 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:36.027 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219591+0000", 2024-09-04T17:47:36.027 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14827944900000001 2024-09-04T17:47:36.027 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.027 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.027 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:36.027 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219731+0000", 2024-09-04T17:47:36.027 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013931600000000001 2024-09-04T17:47:36.027 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.027 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.027 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:36.027 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219950+0000", 2024-09-04T17:47:36.027 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021963199999999999 2024-09-04T17:47:36.028 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.028 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:36.028 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.028 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.028 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.028 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8866 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2146304~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:36.028 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:47:36.028 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 82.879531938, 2024-09-04T17:47:36.028 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14906117899999999, 2024-09-04T17:47:36.028 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:36.028 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:36.028 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:36.028 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:36.028 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:36.029 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:36.029 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8866 2024-09-04T17:47:36.029 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.029 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:36.029 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.029 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:36.029 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:47:36.029 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:36.029 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.029 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.029 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:36.029 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:47:36.029 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:36.029 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.030 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.030 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:36.030 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071322+0000", 2024-09-04T17:47:36.030 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7999999999999995e-07 2024-09-04T17:47:36.030 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.030 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.030 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:36.030 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071338+0000", 2024-09-04T17:47:36.030 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5872999999999999e-05 2024-09-04T17:47:36.030 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.030 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.030 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:36.030 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071347+0000", 2024-09-04T17:47:36.031 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4669999999999993e-06 2024-09-04T17:47:36.031 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.031 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.031 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:36.031 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071354+0000", 2024-09-04T17:47:36.031 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2099999999999996e-06 2024-09-04T17:47:36.031 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.031 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.031 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:36.031 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220008+0000", 2024-09-04T17:47:36.031 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148653642 2024-09-04T17:47:36.031 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.031 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.032 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:36.032 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220155+0000", 2024-09-04T17:47:36.032 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014726600000000001 2024-09-04T17:47:36.032 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.032 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.032 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:36.032 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220382+0000", 2024-09-04T17:47:36.032 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00022704100000000001 2024-09-04T17:47:36.032 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.032 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:36.032 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.032 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.032 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.032 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8867 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2183168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:36.033 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:47:36.033 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 82.879488585000004, 2024-09-04T17:47:36.033 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.149445831, 2024-09-04T17:47:36.033 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:36.033 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:36.033 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:36.033 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:36.033 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:36.033 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:36.033 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8867 2024-09-04T17:47:36.033 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.033 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:36.033 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.034 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:36.034 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:47:36.034 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:36.034 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.034 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.034 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:36.034 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:47:36.034 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:36.034 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.034 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.034 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:36.034 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078653+0000", 2024-09-04T17:47:36.034 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0072890430000000003 2024-09-04T17:47:36.034 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.035 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.035 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:36.035 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078664+0000", 2024-09-04T17:47:36.035 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0307e-05 2024-09-04T17:47:36.035 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.035 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.035 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:36.035 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078690+0000", 2024-09-04T17:47:36.035 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6780000000000001e-05 2024-09-04T17:47:36.035 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.035 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.035 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:36.035 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078706+0000", 2024-09-04T17:47:36.036 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5676999999999999e-05 2024-09-04T17:47:36.036 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.036 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.036 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:36.036 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220467+0000", 2024-09-04T17:47:36.036 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14176103500000001 2024-09-04T17:47:36.036 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.036 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.036 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:36.036 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220595+0000", 2024-09-04T17:47:36.036 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000127427 2024-09-04T17:47:36.036 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.036 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.036 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:36.037 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220810+0000", 2024-09-04T17:47:36.037 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021556200000000001 2024-09-04T17:47:36.037 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.037 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:36.037 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.037 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.037 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.037 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8880 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2658304~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:36.037 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.079158+0000", 2024-09-04T17:47:36.037 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 82.871695306000007, 2024-09-04T17:47:36.037 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14718968499999999, 2024-09-04T17:47:36.037 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:36.037 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:36.037 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:36.038 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:36.038 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:36.038 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:36.038 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8880 2024-09-04T17:47:36.038 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.038 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:36.038 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.038 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:36.038 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079158+0000", 2024-09-04T17:47:36.038 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:36.038 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.038 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.038 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:36.038 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079158+0000", 2024-09-04T17:47:36.039 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:36.039 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.039 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.039 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:36.039 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079158+0000", 2024-09-04T17:47:36.039 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3e-07 2024-09-04T17:47:36.039 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.039 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.039 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:36.039 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079161+0000", 2024-09-04T17:47:36.039 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.57e-06 2024-09-04T17:47:36.039 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.039 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.039 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:36.040 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079170+0000", 2024-09-04T17:47:36.040 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.0170000000000002e-06 2024-09-04T17:47:36.040 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.040 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.040 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:36.040 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079177+0000", 2024-09-04T17:47:36.040 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.0530000000000001e-06 2024-09-04T17:47:36.040 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.040 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.040 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:36.040 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.225946+0000", 2024-09-04T17:47:36.040 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14676936500000001 2024-09-04T17:47:36.040 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.041 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.041 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:36.041 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226098+0000", 2024-09-04T17:47:36.041 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000151805 2024-09-04T17:47:36.041 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.041 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.041 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:36.041 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226347+0000", 2024-09-04T17:47:36.041 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024914499999999999 2024-09-04T17:47:36.041 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.041 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:36.041 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.041 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.042 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.042 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8881 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2695168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:36.042 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:47:36.042 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 82.871666516000005, 2024-09-04T17:47:36.042 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147543547, 2024-09-04T17:47:36.042 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:36.042 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:36.042 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:36.042 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:36.042 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:36.042 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:36.042 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8881 2024-09-04T17:47:36.042 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.042 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:36.043 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.043 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:36.043 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:47:36.043 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:36.043 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.043 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.043 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:36.043 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:47:36.043 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:36.043 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.043 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.043 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:36.043 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079187+0000", 2024-09-04T17:47:36.044 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7700000000000004e-07 2024-09-04T17:47:36.044 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.044 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.044 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:36.044 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079198+0000", 2024-09-04T17:47:36.044 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0849999999999999e-05 2024-09-04T17:47:36.044 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.044 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.044 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:36.044 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079207+0000", 2024-09-04T17:47:36.044 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5599999999999999e-06 2024-09-04T17:47:36.044 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.044 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.045 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:36.045 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079224+0000", 2024-09-04T17:47:36.045 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6387000000000001e-05 2024-09-04T17:47:36.045 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.045 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.045 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:36.045 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226414+0000", 2024-09-04T17:47:36.045 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14718975000000001 2024-09-04T17:47:36.045 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.045 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.045 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:36.045 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226534+0000", 2024-09-04T17:47:36.046 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000120288 2024-09-04T17:47:36.046 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.046 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.046 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:36.046 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226730+0000", 2024-09-04T17:47:36.046 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000196035 2024-09-04T17:47:36.046 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.046 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:36.046 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.046 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.046 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.046 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8882 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2732032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:36.046 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:47:36.047 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 82.871618302000002, 2024-09-04T17:47:36.047 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14795022399999999, 2024-09-04T17:47:36.047 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:36.047 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:36.047 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:36.047 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:36.047 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:36.047 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:36.047 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8882 2024-09-04T17:47:36.047 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.047 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:36.047 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.047 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:36.048 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:47:36.048 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:36.048 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.048 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.048 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:36.048 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:47:36.048 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:36.048 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.048 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.048 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:36.048 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086588+0000", 2024-09-04T17:47:36.048 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0073530959999999999 2024-09-04T17:47:36.048 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.049 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.049 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:36.049 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086598+0000", 2024-09-04T17:47:36.049 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0122999999999999e-05 2024-09-04T17:47:36.049 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.049 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.049 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:36.049 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086626+0000", 2024-09-04T17:47:36.049 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.7914000000000001e-05 2024-09-04T17:47:36.049 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.049 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.049 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:36.049 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086641+0000", 2024-09-04T17:47:36.050 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5270000000000001e-05 2024-09-04T17:47:36.050 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.050 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.050 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:36.050 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226801+0000", 2024-09-04T17:47:36.050 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.140160321 2024-09-04T17:47:36.050 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.050 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.050 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:36.050 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226951+0000", 2024-09-04T17:47:36.050 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014965200000000001 2024-09-04T17:47:36.050 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.050 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.050 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:36.051 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.227185+0000", 2024-09-04T17:47:36.051 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00023384799999999999 2024-09-04T17:47:36.051 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.051 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:36.051 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.051 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.051 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:36.051 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:36.052 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-04T17:47:36.053 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:36.049+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:36.153 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:36.153+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:36.223 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:36.223 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:47:36.223 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:47:36.224 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:36.239 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-04T17:47:36.254 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:36.253+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:36.356 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:36.353+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:36.359 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:36.360 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:47:36.360 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:47:36.360 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:47:36.360 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:36.374 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-04T17:47:36.456 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:36.453+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:36.499 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:36.500 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:47:36.500 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:47:36.500 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:47:36.500 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.500 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60921 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 262144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:36.500 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822920+0000", 2024-09-04T17:47:36.500 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.675226127999998, 2024-09-04T17:47:36.500 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073904394999999998, 2024-09-04T17:47:36.500 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:36.500 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:36.500 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:36.501 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:36.501 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:36.501 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:36.501 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60921 2024-09-04T17:47:36.501 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.501 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:36.501 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.501 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:36.501 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822920+0000", 2024-09-04T17:47:36.501 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:36.501 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.501 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.502 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:36.502 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822920+0000", 2024-09-04T17:47:36.502 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:36.502 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.502 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.502 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:36.502 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822920+0000", 2024-09-04T17:47:36.502 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4999999999999999e-07 2024-09-04T17:47:36.502 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.502 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.502 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:36.502 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822921+0000", 2024-09-04T17:47:36.502 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.113e-06 2024-09-04T17:47:36.502 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.503 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.503 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:36.503 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822924+0000", 2024-09-04T17:47:36.503 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.354e-06 2024-09-04T17:47:36.503 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.503 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.503 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:36.503 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822927+0000", 2024-09-04T17:47:36.503 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6199999999999999e-06 2024-09-04T17:47:36.503 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.503 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.503 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:36.503 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896586+0000", 2024-09-04T17:47:36.503 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073658976000000001 2024-09-04T17:47:36.504 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.504 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.504 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:36.504 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896626+0000", 2024-09-04T17:47:36.504 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0484000000000001e-05 2024-09-04T17:47:36.504 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.504 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.504 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:36.504 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896824+0000", 2024-09-04T17:47:36.504 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000197598 2024-09-04T17:47:36.504 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.504 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:36.505 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.505 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.505 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.505 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60922 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:36.505 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822930+0000", 2024-09-04T17:47:36.505 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.675215553000001, 2024-09-04T17:47:36.505 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074178872000000007, 2024-09-04T17:47:36.505 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:36.505 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:36.505 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:36.505 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:36.505 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:36.505 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:36.506 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60922 2024-09-04T17:47:36.506 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.506 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:36.506 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.506 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:36.506 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822930+0000", 2024-09-04T17:47:36.506 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:36.506 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.506 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.506 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:36.507 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822930+0000", 2024-09-04T17:47:36.507 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:36.507 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.507 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.507 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:36.507 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822930+0000", 2024-09-04T17:47:36.507 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.36e-07 2024-09-04T17:47:36.507 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.507 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.507 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:36.507 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822932+0000", 2024-09-04T17:47:36.507 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.567e-06 2024-09-04T17:47:36.507 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.507 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.507 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:36.508 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822938+0000", 2024-09-04T17:47:36.508 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4840000000000003e-06 2024-09-04T17:47:36.508 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.508 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.508 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:36.508 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822942+0000", 2024-09-04T17:47:36.508 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2039999999999999e-06 2024-09-04T17:47:36.508 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.508 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.508 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:36.508 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896850+0000", 2024-09-04T17:47:36.508 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073907941000000005 2024-09-04T17:47:36.509 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.509 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.509 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:36.509 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896891+0000", 2024-09-04T17:47:36.509 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0964999999999998e-05 2024-09-04T17:47:36.509 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.509 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.509 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:36.509 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897109+0000", 2024-09-04T17:47:36.509 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021847500000000001 2024-09-04T17:47:36.509 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.509 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:36.509 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.510 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.510 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.510 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60923 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 475136~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:36.510 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822949+0000", 2024-09-04T17:47:36.510 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.675197263000001, 2024-09-04T17:47:36.510 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074426928000000003, 2024-09-04T17:47:36.510 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:36.510 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:36.510 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:36.510 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:36.510 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:36.511 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:36.511 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60923 2024-09-04T17:47:36.511 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.511 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:36.511 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.511 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:36.511 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822949+0000", 2024-09-04T17:47:36.511 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:36.511 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.511 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.511 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:36.511 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822949+0000", 2024-09-04T17:47:36.511 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:36.511 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.512 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.512 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:36.512 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822949+0000", 2024-09-04T17:47:36.512 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7399999999999999e-07 2024-09-04T17:47:36.512 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.512 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.512 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:36.512 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822951+0000", 2024-09-04T17:47:36.512 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1940000000000001e-06 2024-09-04T17:47:36.512 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.512 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.512 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:36.513 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822957+0000", 2024-09-04T17:47:36.513 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.0650000000000004e-06 2024-09-04T17:47:36.513 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.513 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.513 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:36.513 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822960+0000", 2024-09-04T17:47:36.513 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.8009999999999999e-06 2024-09-04T17:47:36.513 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.513 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.513 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:36.513 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897138+0000", 2024-09-04T17:47:36.513 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074177990999999999 2024-09-04T17:47:36.513 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.513 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.514 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:36.514 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897179+0000", 2024-09-04T17:47:36.514 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.1057999999999999e-05 2024-09-04T17:47:36.514 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.514 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.514 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:36.514 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897375+0000", 2024-09-04T17:47:36.514 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019644500000000001 2024-09-04T17:47:36.514 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.514 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:36.514 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.515 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.515 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.515 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60924 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:36.515 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822964+0000", 2024-09-04T17:47:36.515 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.675182077999999, 2024-09-04T17:47:36.515 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074531902999999997, 2024-09-04T17:47:36.515 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:36.515 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:36.515 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:36.515 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:36.515 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:36.516 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:36.516 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60924 2024-09-04T17:47:36.516 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.516 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:36.516 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.516 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:36.516 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822964+0000", 2024-09-04T17:47:36.516 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:36.516 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.516 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.516 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:36.516 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822964+0000", 2024-09-04T17:47:36.516 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:36.517 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.517 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.517 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:36.517 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822964+0000", 2024-09-04T17:47:36.517 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4999999999999999e-07 2024-09-04T17:47:36.517 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.517 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.517 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:36.517 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822966+0000", 2024-09-04T17:47:36.517 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.917e-06 2024-09-04T17:47:36.517 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.517 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.518 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:36.518 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822969+0000", 2024-09-04T17:47:36.518 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.315e-06 2024-09-04T17:47:36.518 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.518 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.518 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:36.518 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822972+0000", 2024-09-04T17:47:36.518 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.599e-06 2024-09-04T17:47:36.518 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.518 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.518 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:36.518 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897398+0000", 2024-09-04T17:47:36.518 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074426168000000001 2024-09-04T17:47:36.519 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.519 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.519 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:36.519 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897441+0000", 2024-09-04T17:47:36.519 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.3158000000000003e-05 2024-09-04T17:47:36.519 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.519 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.519 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:36.519 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897496+0000", 2024-09-04T17:47:36.519 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4496000000000001e-05 2024-09-04T17:47:36.519 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.519 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:36.519 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.519 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.520 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.520 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60925 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 720896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:36.520 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822975+0000", 2024-09-04T17:47:36.520 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.675170643000001, 2024-09-04T17:47:36.520 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074635272000000003, 2024-09-04T17:47:36.520 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:36.520 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:36.520 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:36.520 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:36.520 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:36.520 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:36.521 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60925 2024-09-04T17:47:36.521 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.521 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:36.521 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.521 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:36.521 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822975+0000", 2024-09-04T17:47:36.521 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:36.521 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.521 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.521 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:36.521 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822975+0000", 2024-09-04T17:47:36.522 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:36.522 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.522 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.522 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:36.522 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822975+0000", 2024-09-04T17:47:36.522 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.34e-07 2024-09-04T17:47:36.522 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.522 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.522 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:36.522 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822976+0000", 2024-09-04T17:47:36.522 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0359999999999999e-06 2024-09-04T17:47:36.522 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.523 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.523 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:36.523 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822980+0000", 2024-09-04T17:47:36.523 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.242e-06 2024-09-04T17:47:36.523 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.523 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.523 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:36.523 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822983+0000", 2024-09-04T17:47:36.523 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.8830000000000002e-06 2024-09-04T17:47:36.523 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.523 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.524 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:36.524 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897519+0000", 2024-09-04T17:47:36.524 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074536366000000007 2024-09-04T17:47:36.524 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.524 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.524 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:36.524 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897560+0000", 2024-09-04T17:47:36.524 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0677999999999999e-05 2024-09-04T17:47:36.524 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.524 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.524 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:36.525 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897610+0000", 2024-09-04T17:47:36.525 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0832999999999999e-05 2024-09-04T17:47:36.525 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.525 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:36.525 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.525 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.525 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.525 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60926 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 860160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:36.525 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822986+0000", 2024-09-04T17:47:36.525 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.675160044999998, 2024-09-04T17:47:36.525 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074736413000000002, 2024-09-04T17:47:36.526 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:36.526 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:36.526 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:36.526 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:36.526 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:36.526 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:36.526 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60926 2024-09-04T17:47:36.526 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.526 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:36.526 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.526 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:36.526 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822986+0000", 2024-09-04T17:47:36.526 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:36.526 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.527 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.527 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:36.527 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822986+0000", 2024-09-04T17:47:36.527 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:36.527 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.527 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.527 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:36.527 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822986+0000", 2024-09-04T17:47:36.527 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5600000000000002e-07 2024-09-04T17:47:36.527 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.527 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.527 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:36.527 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822987+0000", 2024-09-04T17:47:36.528 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.082e-06 2024-09-04T17:47:36.528 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.528 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.528 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:36.528 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822990+0000", 2024-09-04T17:47:36.528 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.2219999999999998e-06 2024-09-04T17:47:36.528 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.528 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.528 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:36.528 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822994+0000", 2024-09-04T17:47:36.528 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.19e-06 2024-09-04T17:47:36.528 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.529 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.529 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:36.529 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897632+0000", 2024-09-04T17:47:36.529 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074638205999999999 2024-09-04T17:47:36.529 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.529 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.529 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:36.529 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897672+0000", 2024-09-04T17:47:36.529 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.9873999999999998e-05 2024-09-04T17:47:36.529 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.529 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.529 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:36.529 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897722+0000", 2024-09-04T17:47:36.530 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0583e-05 2024-09-04T17:47:36.530 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.530 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:36.530 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.530 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.530 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.530 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60927 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 966656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:36.530 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823000+0000", 2024-09-04T17:47:36.530 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.675145863000001, 2024-09-04T17:47:36.530 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074834036000000007, 2024-09-04T17:47:36.530 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:36.530 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:36.530 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:36.530 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:36.531 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:36.531 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:36.531 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60927 2024-09-04T17:47:36.531 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.531 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:36.531 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.531 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:36.531 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823000+0000", 2024-09-04T17:47:36.531 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:36.531 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.531 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.531 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:36.531 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823000+0000", 2024-09-04T17:47:36.532 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:36.532 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.532 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.532 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:36.532 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823000+0000", 2024-09-04T17:47:36.532 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4099999999999999e-07 2024-09-04T17:47:36.532 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.532 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.532 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:36.532 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823003+0000", 2024-09-04T17:47:36.532 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0810000000000001e-06 2024-09-04T17:47:36.532 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.532 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.533 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:36.533 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823011+0000", 2024-09-04T17:47:36.533 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7090000000000008e-06 2024-09-04T17:47:36.533 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.533 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.533 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:36.533 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823017+0000", 2024-09-04T17:47:36.533 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.3890000000000004e-06 2024-09-04T17:47:36.533 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.533 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.533 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:36.533 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897745+0000", 2024-09-04T17:47:36.534 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074728252999999994 2024-09-04T17:47:36.534 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.534 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.534 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:36.534 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897785+0000", 2024-09-04T17:47:36.534 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.9830999999999999e-05 2024-09-04T17:47:36.534 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.534 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.534 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:36.534 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897834+0000", 2024-09-04T17:47:36.534 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9231999999999999e-05 2024-09-04T17:47:36.535 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.535 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:36.535 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.535 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.535 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.535 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60928 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:36.535 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823024+0000", 2024-09-04T17:47:36.535 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.675121932, 2024-09-04T17:47:36.535 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075067479000000006, 2024-09-04T17:47:36.535 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:36.535 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:36.535 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:36.535 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:36.535 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:36.536 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:36.536 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60928 2024-09-04T17:47:36.536 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.536 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:36.536 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.536 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:36.536 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823024+0000", 2024-09-04T17:47:36.536 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:36.536 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.536 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.536 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:36.536 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823024+0000", 2024-09-04T17:47:36.536 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:36.537 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.537 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.537 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:36.537 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823024+0000", 2024-09-04T17:47:36.537 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6499999999999999e-07 2024-09-04T17:47:36.537 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.537 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.537 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:36.537 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823027+0000", 2024-09-04T17:47:36.537 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.3099999999999999e-06 2024-09-04T17:47:36.538 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.538 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.538 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:36.538 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823033+0000", 2024-09-04T17:47:36.538 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7429999999999996e-06 2024-09-04T17:47:36.538 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.538 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.538 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:36.539 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823038+0000", 2024-09-04T17:47:36.539 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.561e-06 2024-09-04T17:47:36.539 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.539 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.539 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:36.539 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897855+0000", 2024-09-04T17:47:36.539 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074816740000000007 2024-09-04T17:47:36.539 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.539 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.539 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:36.539 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897894+0000", 2024-09-04T17:47:36.539 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.9085999999999998e-05 2024-09-04T17:47:36.539 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.540 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.540 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:36.540 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898091+0000", 2024-09-04T17:47:36.540 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000197574 2024-09-04T17:47:36.540 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.540 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:36.540 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.540 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.540 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.540 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60929 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1212416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:36.540 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823042+0000", 2024-09-04T17:47:36.540 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.675103490000001, 2024-09-04T17:47:36.540 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075301251, 2024-09-04T17:47:36.541 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:36.541 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:36.541 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:36.541 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:36.541 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:36.541 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:36.541 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60929 2024-09-04T17:47:36.541 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.541 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:36.541 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.541 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:36.541 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823042+0000", 2024-09-04T17:47:36.541 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:36.541 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.542 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.542 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:36.542 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823042+0000", 2024-09-04T17:47:36.542 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:36.542 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.542 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.542 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:36.542 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823043+0000", 2024-09-04T17:47:36.542 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.1100000000000002e-07 2024-09-04T17:47:36.542 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.542 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.542 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:36.542 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823045+0000", 2024-09-04T17:47:36.543 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.933e-06 2024-09-04T17:47:36.543 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.543 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.543 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:36.543 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823051+0000", 2024-09-04T17:47:36.543 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.9569999999999999e-06 2024-09-04T17:47:36.543 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.543 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.543 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:36.543 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823055+0000", 2024-09-04T17:47:36.543 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4479999999999996e-06 2024-09-04T17:47:36.543 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.543 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.544 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:36.544 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898117+0000", 2024-09-04T17:47:36.544 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075062344000000003 2024-09-04T17:47:36.544 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.544 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.544 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:36.544 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898157+0000", 2024-09-04T17:47:36.544 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0095999999999997e-05 2024-09-04T17:47:36.544 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.544 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.544 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:36.544 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898344+0000", 2024-09-04T17:47:36.544 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000186162 2024-09-04T17:47:36.545 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.545 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:36.545 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.545 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.545 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.545 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60930 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1351680~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:36.545 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823061+0000", 2024-09-04T17:47:36.545 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.675084610999999, 2024-09-04T17:47:36.545 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075545507999999997, 2024-09-04T17:47:36.545 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:36.545 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:36.545 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:36.545 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:36.546 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:36.546 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:36.546 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60930 2024-09-04T17:47:36.546 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.546 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:36.546 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.546 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:36.546 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823061+0000", 2024-09-04T17:47:36.546 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:36.546 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.546 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.546 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:36.546 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823061+0000", 2024-09-04T17:47:36.547 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:36.547 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.547 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.547 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:36.547 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823378+0000", 2024-09-04T17:47:36.547 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00031631299999999998 2024-09-04T17:47:36.547 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.547 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.547 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:36.547 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823381+0000", 2024-09-04T17:47:36.547 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4020000000000002e-06 2024-09-04T17:47:36.547 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.547 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.548 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:36.548 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823389+0000", 2024-09-04T17:47:36.548 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.1429999999999998e-06 2024-09-04T17:47:36.548 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.548 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.548 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:36.548 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823395+0000", 2024-09-04T17:47:36.548 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.0059999999999998e-06 2024-09-04T17:47:36.549 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.549 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.549 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:36.549 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898368+0000", 2024-09-04T17:47:36.549 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074973032999999994 2024-09-04T17:47:36.549 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.549 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.549 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:36.549 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898410+0000", 2024-09-04T17:47:36.549 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2243000000000002e-05 2024-09-04T17:47:36.549 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.550 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.550 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:36.550 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898607+0000", 2024-09-04T17:47:36.550 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019636799999999999 2024-09-04T17:47:36.550 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.550 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:36.550 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.550 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.550 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.550 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60931 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1458176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:36.550 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823407+0000", 2024-09-04T17:47:36.550 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.674738537, 2024-09-04T17:47:36.550 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075460334000000004, 2024-09-04T17:47:36.550 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:36.551 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:36.551 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:36.551 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:36.551 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:36.551 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:36.551 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60931 2024-09-04T17:47:36.551 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.551 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:36.551 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.551 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:36.551 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823407+0000", 2024-09-04T17:47:36.551 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:36.552 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.552 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.552 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:36.552 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823407+0000", 2024-09-04T17:47:36.552 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:36.552 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.552 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.552 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:36.552 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823408+0000", 2024-09-04T17:47:36.552 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9999999999999998e-07 2024-09-04T17:47:36.552 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.552 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.552 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:36.553 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823415+0000", 2024-09-04T17:47:36.553 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.5510000000000002e-06 2024-09-04T17:47:36.553 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.553 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.553 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:36.553 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823423+0000", 2024-09-04T17:47:36.553 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2760000000000003e-06 2024-09-04T17:47:36.553 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.553 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.553 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:36.553 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823426+0000", 2024-09-04T17:47:36.553 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.2839999999999998e-06 2024-09-04T17:47:36.553 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.554 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.554 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:36.554 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898631+0000", 2024-09-04T17:47:36.554 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075205045999999998 2024-09-04T17:47:36.554 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.554 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.554 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:36.554 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898672+0000", 2024-09-04T17:47:36.554 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0614000000000001e-05 2024-09-04T17:47:36.554 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.554 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.554 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:36.554 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898868+0000", 2024-09-04T17:47:36.555 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019606300000000001 2024-09-04T17:47:36.555 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.555 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:36.555 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.555 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.555 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.555 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60932 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1564672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:36.555 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823434+0000", 2024-09-04T17:47:36.555 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.674712238000001, 2024-09-04T17:47:36.555 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075693393999999997, 2024-09-04T17:47:36.555 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:36.556 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:36.556 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:36.556 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:36.556 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:36.556 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:36.556 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60932 2024-09-04T17:47:36.556 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.556 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:36.556 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.556 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:36.556 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823434+0000", 2024-09-04T17:47:36.556 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:36.557 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.557 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.557 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:36.557 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823434+0000", 2024-09-04T17:47:36.557 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:36.557 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.557 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.557 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:36.557 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.830290+0000", 2024-09-04T17:47:36.557 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0068563039999999997 2024-09-04T17:47:36.557 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.557 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.557 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:36.558 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.830297+0000", 2024-09-04T17:47:36.558 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4760000000000001e-06 2024-09-04T17:47:36.558 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.558 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.558 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:36.558 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.830308+0000", 2024-09-04T17:47:36.558 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0927999999999999e-05 2024-09-04T17:47:36.558 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.558 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.558 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:36.558 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.830317+0000", 2024-09-04T17:47:36.558 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7280000000000001e-06 2024-09-04T17:47:36.559 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.559 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.559 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:36.559 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898892+0000", 2024-09-04T17:47:36.559 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.068574722000000005 2024-09-04T17:47:36.559 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.559 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.559 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:36.559 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898934+0000", 2024-09-04T17:47:36.560 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2249000000000002e-05 2024-09-04T17:47:36.560 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.560 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.560 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:36.560 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.899127+0000", 2024-09-04T17:47:36.560 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019298699999999999 2024-09-04T17:47:36.560 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.560 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:36.560 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.560 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.560 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.560 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61182 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 868352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:36.561 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839249+0000", 2024-09-04T17:47:36.561 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.658896345999999, 2024-09-04T17:47:36.561 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073680151999999999, 2024-09-04T17:47:36.561 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:36.561 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:36.561 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:36.561 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:36.561 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:36.561 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:36.561 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61182 2024-09-04T17:47:36.561 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.561 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:36.561 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.562 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:36.562 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839249+0000", 2024-09-04T17:47:36.562 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:36.562 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.562 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.562 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:36.562 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839249+0000", 2024-09-04T17:47:36.562 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:36.562 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.562 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.562 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:36.562 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839250+0000", 2024-09-04T17:47:36.562 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4999999999999998e-07 2024-09-04T17:47:36.562 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.563 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.563 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:36.563 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839257+0000", 2024-09-04T17:47:36.563 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7700000000000004e-06 2024-09-04T17:47:36.563 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.563 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.563 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:36.563 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839263+0000", 2024-09-04T17:47:36.563 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.246e-06 2024-09-04T17:47:36.563 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.563 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.563 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:36.564 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839268+0000", 2024-09-04T17:47:36.564 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.814e-06 2024-09-04T17:47:36.564 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.564 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.564 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:36.564 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.912831+0000", 2024-09-04T17:47:36.564 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073563294000000001 2024-09-04T17:47:36.564 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.564 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.564 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:36.564 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.912876+0000", 2024-09-04T17:47:36.565 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4742e-05 2024-09-04T17:47:36.565 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.565 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.565 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:36.565 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.912930+0000", 2024-09-04T17:47:36.566 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.3936e-05 2024-09-04T17:47:36.566 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.566 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:36.566 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.566 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.566 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.566 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61183 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:36.566 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839275+0000", 2024-09-04T17:47:36.566 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.658871153, 2024-09-04T17:47:36.566 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073781886000000005, 2024-09-04T17:47:36.567 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:36.567 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:36.567 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:36.567 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:36.567 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:36.567 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:36.567 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61183 2024-09-04T17:47:36.567 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.567 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:36.567 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.567 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:36.567 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839275+0000", 2024-09-04T17:47:36.568 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:36.568 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.568 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.568 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:36.568 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839275+0000", 2024-09-04T17:47:36.568 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:36.568 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.568 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.568 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:36.568 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839275+0000", 2024-09-04T17:47:36.568 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6899999999999997e-07 2024-09-04T17:47:36.568 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.569 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.569 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:36.569 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839277+0000", 2024-09-04T17:47:36.569 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9530000000000002e-06 2024-09-04T17:47:36.569 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.569 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.569 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:36.569 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839283+0000", 2024-09-04T17:47:36.569 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.9329999999999998e-06 2024-09-04T17:47:36.570 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.570 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.570 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:36.570 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839288+0000", 2024-09-04T17:47:36.570 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4100000000000001e-06 2024-09-04T17:47:36.570 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.570 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.570 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:36.570 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.912959+0000", 2024-09-04T17:47:36.570 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073671511999999995 2024-09-04T17:47:36.570 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.571 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.571 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:36.571 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913001+0000", 2024-09-04T17:47:36.571 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.1984999999999999e-05 2024-09-04T17:47:36.571 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.571 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.571 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:36.571 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913057+0000", 2024-09-04T17:47:36.571 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.5523999999999999e-05 2024-09-04T17:47:36.571 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.571 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:36.571 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.572 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.572 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.572 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61184 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1114112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:36.572 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839294+0000", 2024-09-04T17:47:36.572 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.658851941000002, 2024-09-04T17:47:36.572 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073995263000000006, 2024-09-04T17:47:36.572 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:36.572 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:36.572 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:36.572 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:36.572 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:36.572 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:36.573 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61184 2024-09-04T17:47:36.573 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.573 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:36.573 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.573 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:36.573 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839294+0000", 2024-09-04T17:47:36.573 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:36.573 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.573 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.573 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:36.573 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839294+0000", 2024-09-04T17:47:36.573 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:36.573 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.574 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.574 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:36.574 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839294+0000", 2024-09-04T17:47:36.574 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.77e-07 2024-09-04T17:47:36.574 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.574 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.574 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:36.574 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839296+0000", 2024-09-04T17:47:36.574 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9999999999999999e-06 2024-09-04T17:47:36.574 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.574 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.574 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:36.574 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839302+0000", 2024-09-04T17:47:36.575 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7230000000000001e-06 2024-09-04T17:47:36.575 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.575 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.575 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:36.575 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839314+0000", 2024-09-04T17:47:36.575 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1826e-05 2024-09-04T17:47:36.575 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.575 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.575 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:36.575 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913084+0000", 2024-09-04T17:47:36.575 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073770215 2024-09-04T17:47:36.575 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.576 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.576 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:36.576 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913127+0000", 2024-09-04T17:47:36.576 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2982999999999998e-05 2024-09-04T17:47:36.576 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.576 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.576 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:36.576 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913289+0000", 2024-09-04T17:47:36.576 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000162139 2024-09-04T17:47:36.576 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.577 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:36.577 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.577 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.577 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.577 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61185 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1220608~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:36.577 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839321+0000", 2024-09-04T17:47:36.577 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.658824473999999, 2024-09-04T17:47:36.577 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074206062000000003, 2024-09-04T17:47:36.577 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:36.577 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:36.577 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:36.577 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:36.577 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:36.578 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:36.578 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61185 2024-09-04T17:47:36.578 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.578 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:36.578 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.578 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:36.578 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839321+0000", 2024-09-04T17:47:36.578 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:36.578 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.578 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.578 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:36.578 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839321+0000", 2024-09-04T17:47:36.578 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:36.578 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.579 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.579 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:36.579 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839322+0000", 2024-09-04T17:47:36.579 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.5400000000000001e-07 2024-09-04T17:47:36.579 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.579 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.579 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:36.579 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839324+0000", 2024-09-04T17:47:36.579 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2539999999999999e-06 2024-09-04T17:47:36.579 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.579 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.579 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:36.579 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839331+0000", 2024-09-04T17:47:36.580 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2139999999999999e-06 2024-09-04T17:47:36.580 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.580 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.580 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:36.580 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839336+0000", 2024-09-04T17:47:36.580 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7029999999999997e-06 2024-09-04T17:47:36.580 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.580 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.581 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:36.581 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913320+0000", 2024-09-04T17:47:36.581 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073984016 2024-09-04T17:47:36.581 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.581 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.581 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:36.581 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913366+0000", 2024-09-04T17:47:36.581 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.5704000000000002e-05 2024-09-04T17:47:36.581 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.581 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.581 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:36.581 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913527+0000", 2024-09-04T17:47:36.581 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00016161699999999999 2024-09-04T17:47:36.582 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.582 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:36.582 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.582 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.582 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.582 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61186 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1359872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:36.582 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839344+0000", 2024-09-04T17:47:36.582 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.658802132000002, 2024-09-04T17:47:36.582 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074450673999999994, 2024-09-04T17:47:36.582 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:36.582 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:36.583 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:36.583 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:36.583 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:36.583 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:36.583 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61186 2024-09-04T17:47:36.583 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.583 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:36.583 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.583 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:36.583 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839344+0000", 2024-09-04T17:47:36.583 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:36.583 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.583 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.583 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:36.584 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839344+0000", 2024-09-04T17:47:36.584 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:36.584 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.584 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.584 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:36.584 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839344+0000", 2024-09-04T17:47:36.584 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4799999999999999e-07 2024-09-04T17:47:36.584 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.584 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.584 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:36.584 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839346+0000", 2024-09-04T17:47:36.585 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0889999999999998e-06 2024-09-04T17:47:36.585 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.585 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.585 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:36.585 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839352+0000", 2024-09-04T17:47:36.585 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6389999999999997e-06 2024-09-04T17:47:36.585 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.585 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.585 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:36.585 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839356+0000", 2024-09-04T17:47:36.585 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.233e-06 2024-09-04T17:47:36.586 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.586 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.586 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:36.586 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913557+0000", 2024-09-04T17:47:36.586 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074200944000000005 2024-09-04T17:47:36.586 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.586 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.586 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:36.586 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913601+0000", 2024-09-04T17:47:36.586 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4350000000000001e-05 2024-09-04T17:47:36.586 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.586 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.586 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:36.587 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913794+0000", 2024-09-04T17:47:36.587 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019297100000000001 2024-09-04T17:47:36.587 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.587 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:36.587 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.587 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.587 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.587 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61187 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1466368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:36.587 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839364+0000", 2024-09-04T17:47:36.588 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.658782283000001, 2024-09-04T17:47:36.588 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074692334999999999, 2024-09-04T17:47:36.588 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:36.588 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:36.588 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:36.588 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:36.588 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:36.588 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:36.588 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61187 2024-09-04T17:47:36.588 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.588 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:36.589 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.589 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:36.589 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839364+0000", 2024-09-04T17:47:36.589 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:36.589 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.589 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.589 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:36.589 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839364+0000", 2024-09-04T17:47:36.589 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:36.589 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.589 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.589 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:36.589 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839364+0000", 2024-09-04T17:47:36.590 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.75e-07 2024-09-04T17:47:36.590 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.590 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.590 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:36.590 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839366+0000", 2024-09-04T17:47:36.590 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0990000000000001e-06 2024-09-04T17:47:36.590 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.590 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.590 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:36.590 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839372+0000", 2024-09-04T17:47:36.590 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.2609999999999999e-06 2024-09-04T17:47:36.590 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.591 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.591 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:36.591 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839389+0000", 2024-09-04T17:47:36.591 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6489000000000001e-05 2024-09-04T17:47:36.591 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.591 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.591 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:36.591 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913824+0000", 2024-09-04T17:47:36.591 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074434811000000004 2024-09-04T17:47:36.592 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.592 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.592 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:36.592 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913868+0000", 2024-09-04T17:47:36.592 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4768000000000002e-05 2024-09-04T17:47:36.592 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.592 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.592 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:36.592 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914056+0000", 2024-09-04T17:47:36.592 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00018743199999999999 2024-09-04T17:47:36.592 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.592 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:36.592 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.593 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.593 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.593 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61188 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1605632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:36.593 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839399+0000", 2024-09-04T17:47:36.593 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.658746387000001, 2024-09-04T17:47:36.593 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074975095000000005, 2024-09-04T17:47:36.593 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:36.593 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:36.593 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:36.593 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:36.593 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:36.593 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:36.594 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61188 2024-09-04T17:47:36.594 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.594 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:36.594 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.594 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:36.594 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839399+0000", 2024-09-04T17:47:36.594 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:36.594 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.594 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.594 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:36.594 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839399+0000", 2024-09-04T17:47:36.594 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:36.594 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.594 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.595 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:36.595 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839400+0000", 2024-09-04T17:47:36.595 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0100000000000005e-07 2024-09-04T17:47:36.595 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.595 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.595 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:36.595 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839402+0000", 2024-09-04T17:47:36.595 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.491e-06 2024-09-04T17:47:36.595 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.595 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.595 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:36.595 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839409+0000", 2024-09-04T17:47:36.596 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.0450000000000003e-06 2024-09-04T17:47:36.596 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.596 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.596 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:36.596 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839414+0000", 2024-09-04T17:47:36.596 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.87e-06 2024-09-04T17:47:36.596 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.596 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.596 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:36.596 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914086+0000", 2024-09-04T17:47:36.596 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074671596000000007 2024-09-04T17:47:36.597 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.597 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.597 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:36.597 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914130+0000", 2024-09-04T17:47:36.597 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4579999999999997e-05 2024-09-04T17:47:36.597 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.597 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.597 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:36.597 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914375+0000", 2024-09-04T17:47:36.597 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024401199999999999 2024-09-04T17:47:36.597 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.598 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:36.598 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.598 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.598 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.598 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61189 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1712128~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:36.598 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839424+0000", 2024-09-04T17:47:36.598 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.658722185999999, 2024-09-04T17:47:36.598 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075247680999999997, 2024-09-04T17:47:36.598 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:36.598 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:36.598 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:36.598 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:36.598 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:36.599 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:36.599 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61189 2024-09-04T17:47:36.599 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.599 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:36.599 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.599 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:36.599 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839424+0000", 2024-09-04T17:47:36.599 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:36.599 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.599 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.599 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:36.599 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839424+0000", 2024-09-04T17:47:36.600 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:36.600 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.600 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.600 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:36.600 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.846320+0000", 2024-09-04T17:47:36.600 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0068965210000000001 2024-09-04T17:47:36.600 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.600 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.600 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:36.600 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.846328+0000", 2024-09-04T17:47:36.600 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.8189999999999992e-06 2024-09-04T17:47:36.600 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.601 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.601 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:36.601 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.846341+0000", 2024-09-04T17:47:36.601 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3338e-05 2024-09-04T17:47:36.601 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.601 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.601 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:36.601 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.846352+0000", 2024-09-04T17:47:36.601 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0479000000000001e-05 2024-09-04T17:47:36.601 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.602 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.602 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:36.602 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914405+0000", 2024-09-04T17:47:36.602 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.068053049000000004 2024-09-04T17:47:36.602 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.602 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.602 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:36.602 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914451+0000", 2024-09-04T17:47:36.602 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6545999999999998e-05 2024-09-04T17:47:36.602 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:36.602 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:36.603 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:36.603 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914671+0000", 2024-09-04T17:47:36.603 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000219929 2024-09-04T17:47:36.603 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.603 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:36.603 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.603 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:36.603 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:36.603 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:36.604 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-04T17:47:36.605 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-04T17:47:36.606 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:36.603+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:36.706 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:36.703+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:36.733 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:36.733 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:47:36.733 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:47:36.733 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:36.749 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-04T17:47:36.764 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:47:36.764 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:47:36.780 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:47:36.781 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:47:36.781 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-04T17:47:36.798 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-04T17:47:36.798 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-04T17:47:36.806 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:36.805+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:36.819 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:47:36.819 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:47:36.839 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:47:36.840 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:47:36.840 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-04T17:47:36.860 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:47:36.861 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:47:36.865 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:36.865 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:36.866 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-04T17:47:36.867 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-04T17:47:36.875 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-04T17:47:36.881 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-04T17:47:36.926 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:36.925+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:37.008 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:37.005+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:37.041 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:37.041 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:37.059 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-04T17:47:37.145 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:37.145+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:37.211 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:37.207+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:37.217 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:37.217 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:37.231 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-04T17:47:37.317 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:37.317+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:37.390 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:37.390 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:37.404 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-04T17:47:37.413 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:37.411+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:37.514 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:37.513+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:37.566 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:37.566 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:37.580 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-04T17:47:37.614 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:37.611+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:37.715 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:37.713+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:37.745 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:37.745 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:37.759 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-04T17:47:37.845 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:37.845+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:37.885 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is unset 2024-09-04T17:47:37.927 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:37.927 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:37.927 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:37.925+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:37.941 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-04T17:47:38.018 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:38.015+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:38.107 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:38.107 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:38.119 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:38.117+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:38.120 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-04T17:47:38.219 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:38.215+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:38.253 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:38.253 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:38.267 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-04T17:47:38.353 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:38.351+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:38.421 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:38.421+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:38.425 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:38.426 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:38.440 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-04T17:47:38.525 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:47:38.523+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:47:38.602 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:38.602 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:38.616 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-04T17:47:38.623 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:38.621+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:38.723 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:38.721+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:38.784 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:38.785 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:38.799 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-04T17:47:38.845 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:38.843+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:38.867 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-04T17:47:38.916 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:38.916 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:38.926 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:38.925+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:38.930 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-04T17:47:38.951 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-04T17:47:39.016 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:47:39.017 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:47:39.026 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:39.023+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:39.034 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:47:39.035 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:47:39.035 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-04T17:47:39.053 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-04T17:47:39.053 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-04T17:47:39.077 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:47:39.077 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:47:39.078 INFO:teuthology.orchestra.run.smithi092.stderr:no valid command found; 10 closest matches: 2024-09-04T17:47:39.078 INFO:teuthology.orchestra.run.smithi092.stderr:0 2024-09-04T17:47:39.078 INFO:teuthology.orchestra.run.smithi092.stderr:1 2024-09-04T17:47:39.078 INFO:teuthology.orchestra.run.smithi092.stderr:2 2024-09-04T17:47:39.078 INFO:teuthology.orchestra.run.smithi092.stderr:abort 2024-09-04T17:47:39.078 INFO:teuthology.orchestra.run.smithi092.stderr:assert 2024-09-04T17:47:39.079 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs debug_inject_read_zeros 2024-09-04T17:47:39.079 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs files list 2024-09-04T17:47:39.079 INFO:teuthology.orchestra.run.smithi092.stderr:bluefs stats 2024-09-04T17:47:39.079 INFO:teuthology.orchestra.run.smithi092.stderr:bluestore bluefs device info [] 2024-09-04T17:47:39.079 INFO:teuthology.orchestra.run.smithi092.stderr:config diff 2024-09-04T17:47:39.079 INFO:teuthology.orchestra.run.smithi092.stderr:admin_socket: invalid command 2024-09-04T17:47:39.080 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-04T17:47:39.081 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2024-09-04T17:47:39.097 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:47:39.098 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:47:39.098 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-04T17:47:39.101 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:39.101 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:47:39.101 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:47:39.101 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:47:39.101 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.101 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8831 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2453504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:39.101 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:47:39.101 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 86.041359869999994, 2024-09-04T17:47:39.101 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14731111799999999, 2024-09-04T17:47:39.102 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:39.102 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:39.102 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:39.102 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:39.102 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:39.102 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:39.102 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8831 2024-09-04T17:47:39.102 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.102 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:39.102 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.102 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:39.102 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:47:39.102 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.103 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.103 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.103 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:39.103 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:47:39.103 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.103 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.103 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.103 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:39.103 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057811+0000", 2024-09-04T17:47:39.103 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.083e-06 2024-09-04T17:47:39.103 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.103 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.103 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:39.104 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057814+0000", 2024-09-04T17:47:39.104 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.407e-06 2024-09-04T17:47:39.104 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.104 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.104 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:39.104 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057825+0000", 2024-09-04T17:47:39.104 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.17e-05 2024-09-04T17:47:39.104 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.104 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.104 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:39.104 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057836+0000", 2024-09-04T17:47:39.104 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0283e-05 2024-09-04T17:47:39.104 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.105 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.105 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:39.105 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.204762+0000", 2024-09-04T17:47:39.105 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.146926205 2024-09-04T17:47:39.105 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.105 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.105 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:39.105 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.204895+0000", 2024-09-04T17:47:39.105 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013328999999999999 2024-09-04T17:47:39.105 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.105 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.105 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:39.105 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205121+0000", 2024-09-04T17:47:39.106 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00022614999999999999 2024-09-04T17:47:39.106 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.106 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:39.106 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.106 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.106 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.106 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8832 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2490368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:39.106 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:47:39.106 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 86.041321836999998, 2024-09-04T17:47:39.106 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14770012299999999, 2024-09-04T17:47:39.106 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:39.106 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:39.106 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:39.107 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:39.107 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:39.107 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:39.107 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8832 2024-09-04T17:47:39.107 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.107 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:39.107 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.107 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:39.107 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:47:39.107 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.107 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.107 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.107 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:39.107 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:47:39.108 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.108 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.108 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.108 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:39.108 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057849+0000", 2024-09-04T17:47:39.108 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6700000000000003e-07 2024-09-04T17:47:39.108 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.108 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.108 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:39.108 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057852+0000", 2024-09-04T17:47:39.108 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6359999999999998e-06 2024-09-04T17:47:39.108 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.108 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.109 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:39.109 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057860+0000", 2024-09-04T17:47:39.109 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8140000000000007e-06 2024-09-04T17:47:39.109 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.109 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.109 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:39.109 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057868+0000", 2024-09-04T17:47:39.109 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1999999999999997e-06 2024-09-04T17:47:39.109 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.109 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.109 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:39.109 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205208+0000", 2024-09-04T17:47:39.109 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14734067300000001 2024-09-04T17:47:39.110 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.110 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.110 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:39.110 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205347+0000", 2024-09-04T17:47:39.110 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013914900000000001 2024-09-04T17:47:39.110 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.110 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.110 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:39.110 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205548+0000", 2024-09-04T17:47:39.110 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000200984 2024-09-04T17:47:39.110 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.110 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:39.110 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.111 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.111 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.111 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8833 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2527232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:39.111 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:47:39.111 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 86.041289966999997, 2024-09-04T17:47:39.111 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148062572, 2024-09-04T17:47:39.111 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:39.111 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:39.111 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:39.111 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:39.111 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:39.111 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:39.111 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8833 2024-09-04T17:47:39.111 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.112 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:39.112 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.112 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:39.112 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:47:39.112 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.112 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.112 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.112 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:39.112 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:47:39.112 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.112 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.112 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.112 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:39.113 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057881+0000", 2024-09-04T17:47:39.113 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4300000000000002e-07 2024-09-04T17:47:39.113 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.113 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.113 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:39.113 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057893+0000", 2024-09-04T17:47:39.113 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1973999999999999e-05 2024-09-04T17:47:39.113 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.113 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.113 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:39.113 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057906+0000", 2024-09-04T17:47:39.113 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2716e-05 2024-09-04T17:47:39.113 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.114 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.114 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:39.114 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057913+0000", 2024-09-04T17:47:39.114 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3930000000000002e-06 2024-09-04T17:47:39.114 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.114 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.114 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:39.114 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205621+0000", 2024-09-04T17:47:39.114 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14770788800000001 2024-09-04T17:47:39.114 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.114 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.114 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:39.114 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205740+0000", 2024-09-04T17:47:39.115 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000118936 2024-09-04T17:47:39.115 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.115 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.115 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:39.115 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205943+0000", 2024-09-04T17:47:39.115 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020292199999999999 2024-09-04T17:47:39.115 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.115 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:39.115 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.115 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.115 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.116 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8834 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2564096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:39.116 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:47:39.116 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 86.041246688000001, 2024-09-04T17:47:39.116 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14846753200000001, 2024-09-04T17:47:39.116 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:39.116 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:39.116 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:39.116 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:39.116 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:39.116 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:39.116 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8834 2024-09-04T17:47:39.116 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.116 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:39.117 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.117 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:39.117 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:47:39.117 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.117 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.117 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.117 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:39.117 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:47:39.117 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.117 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.117 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.117 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:39.117 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057925+0000", 2024-09-04T17:47:39.118 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0270000000000001e-06 2024-09-04T17:47:39.118 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.118 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.118 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:39.118 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057936+0000", 2024-09-04T17:47:39.118 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.112e-05 2024-09-04T17:47:39.118 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.118 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.118 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:39.118 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057946+0000", 2024-09-04T17:47:39.118 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.009e-05 2024-09-04T17:47:39.118 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.118 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.119 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:39.119 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057953+0000", 2024-09-04T17:47:39.119 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.6229999999999997e-06 2024-09-04T17:47:39.119 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.119 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.119 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:39.119 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206027+0000", 2024-09-04T17:47:39.119 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148073591 2024-09-04T17:47:39.119 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.119 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.119 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:39.119 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206161+0000", 2024-09-04T17:47:39.119 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013411599999999999 2024-09-04T17:47:39.120 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.120 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.120 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:39.120 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206391+0000", 2024-09-04T17:47:39.120 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000229965 2024-09-04T17:47:39.120 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.120 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:39.120 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.120 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.120 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.120 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8835 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2600960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:39.120 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:47:39.120 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 86.041207037999996, 2024-09-04T17:47:39.121 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14882941599999999, 2024-09-04T17:47:39.121 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:39.121 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:39.121 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:39.121 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:39.121 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:39.121 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:39.121 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8835 2024-09-04T17:47:39.121 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.121 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:39.121 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.121 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:39.121 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:47:39.122 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.122 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.122 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.122 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:39.122 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:47:39.122 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.122 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.122 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.122 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:39.122 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058544+0000", 2024-09-04T17:47:39.122 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00058061899999999999 2024-09-04T17:47:39.122 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.122 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.123 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:39.123 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058550+0000", 2024-09-04T17:47:39.123 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.1129999999999997e-06 2024-09-04T17:47:39.123 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.123 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.123 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:39.123 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058566+0000", 2024-09-04T17:47:39.123 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5820000000000001e-05 2024-09-04T17:47:39.123 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.123 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.123 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:39.123 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058578+0000", 2024-09-04T17:47:39.123 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2537e-05 2024-09-04T17:47:39.124 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.124 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.124 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:39.124 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206470+0000", 2024-09-04T17:47:39.124 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14789153799999999 2024-09-04T17:47:39.124 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.124 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.124 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:39.124 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206589+0000", 2024-09-04T17:47:39.124 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000118837 2024-09-04T17:47:39.124 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.124 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.124 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:39.125 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206793+0000", 2024-09-04T17:47:39.125 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000203952 2024-09-04T17:47:39.125 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.125 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:39.125 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.125 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.125 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.125 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8836 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2654208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:39.125 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:47:39.125 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 86.04057435, 2024-09-04T17:47:39.125 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14862433999999999, 2024-09-04T17:47:39.125 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:39.126 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:39.126 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:39.126 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:39.126 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:39.126 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:39.126 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8836 2024-09-04T17:47:39.126 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.126 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:39.126 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.126 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:39.126 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:47:39.126 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.126 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.126 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.127 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:39.127 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:47:39.127 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.127 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.127 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.127 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:39.127 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062663+0000", 2024-09-04T17:47:39.127 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0040670619999999998 2024-09-04T17:47:39.127 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.127 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.127 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:39.127 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062668+0000", 2024-09-04T17:47:39.127 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4199999999999998e-06 2024-09-04T17:47:39.127 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.128 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.128 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:39.128 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062689+0000", 2024-09-04T17:47:39.128 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0356999999999998e-05 2024-09-04T17:47:39.128 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.128 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.128 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:39.128 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062726+0000", 2024-09-04T17:47:39.128 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7302999999999998e-05 2024-09-04T17:47:39.128 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.128 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.128 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:39.129 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206875+0000", 2024-09-04T17:47:39.129 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14414883100000001 2024-09-04T17:47:39.129 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.129 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.129 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:39.129 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.207016+0000", 2024-09-04T17:47:39.129 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000140822 2024-09-04T17:47:39.129 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.129 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.129 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:39.129 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.207220+0000", 2024-09-04T17:47:39.129 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020454499999999999 2024-09-04T17:47:39.129 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.130 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:39.130 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.130 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.130 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.130 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8844 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2928640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:39.130 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:47:39.130 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 86.036210267000001, 2024-09-04T17:47:39.130 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14752153100000001, 2024-09-04T17:47:39.130 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:39.130 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:39.130 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:39.130 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:39.130 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:39.131 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:39.131 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8844 2024-09-04T17:47:39.131 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.131 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:39.131 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.131 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:39.131 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:47:39.131 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.131 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.131 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.131 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:39.131 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:47:39.132 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.132 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.132 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.132 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:39.132 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062961+0000", 2024-09-04T17:47:39.132 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6700000000000003e-07 2024-09-04T17:47:39.132 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.132 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.132 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:39.132 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062973+0000", 2024-09-04T17:47:39.132 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2109999999999999e-05 2024-09-04T17:47:39.132 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.132 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.133 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:39.133 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062982+0000", 2024-09-04T17:47:39.133 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.6900000000000004e-06 2024-09-04T17:47:39.133 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.133 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.133 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:39.133 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062990+0000", 2024-09-04T17:47:39.133 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1899999999999998e-06 2024-09-04T17:47:39.133 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.133 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.133 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:39.133 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210090+0000", 2024-09-04T17:47:39.133 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14710026300000001 2024-09-04T17:47:39.134 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.134 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.134 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:39.134 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210225+0000", 2024-09-04T17:47:39.134 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013532400000000001 2024-09-04T17:47:39.134 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.134 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.134 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:39.134 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210481+0000", 2024-09-04T17:47:39.134 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00025628700000000001 2024-09-04T17:47:39.134 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.134 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:39.134 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.135 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.135 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.135 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8845 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2965504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:39.135 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:47:39.135 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 86.036170544000001, 2024-09-04T17:47:39.135 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147916819, 2024-09-04T17:47:39.135 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:39.135 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:39.135 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:39.135 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:39.135 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:39.135 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:39.135 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8845 2024-09-04T17:47:39.135 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.136 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:39.136 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.136 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:39.136 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:47:39.136 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.136 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.136 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.136 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:39.136 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:47:39.136 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.136 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.136 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.136 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:39.137 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:47:39.137 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.0399999999999995e-07 2024-09-04T17:47:39.137 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.137 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.137 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:39.137 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063003+0000", 2024-09-04T17:47:39.137 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6529999999999998e-06 2024-09-04T17:47:39.137 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.137 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.137 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:39.137 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063012+0000", 2024-09-04T17:47:39.137 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9500000000000007e-06 2024-09-04T17:47:39.138 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.138 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.138 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:39.138 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063027+0000", 2024-09-04T17:47:39.138 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5443000000000001e-05 2024-09-04T17:47:39.138 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.138 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.138 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:39.138 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210571+0000", 2024-09-04T17:47:39.138 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14754353100000001 2024-09-04T17:47:39.138 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.138 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.138 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:39.139 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210714+0000", 2024-09-04T17:47:39.139 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014295599999999999 2024-09-04T17:47:39.139 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.139 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.139 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:39.139 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210916+0000", 2024-09-04T17:47:39.139 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020258200000000001 2024-09-04T17:47:39.139 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.139 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:39.139 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.139 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.139 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.139 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8846 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3002368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:39.139 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:47:39.140 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 86.036133007000004, 2024-09-04T17:47:39.140 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14828011599999999, 2024-09-04T17:47:39.140 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:39.140 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:39.140 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:39.140 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:39.140 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:39.140 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:39.140 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8846 2024-09-04T17:47:39.140 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.140 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:39.140 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.140 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:39.141 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:47:39.141 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.141 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.141 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.141 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:39.141 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:47:39.141 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.141 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.141 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.141 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:39.141 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063038+0000", 2024-09-04T17:47:39.141 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.23e-07 2024-09-04T17:47:39.141 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.142 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.142 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:39.142 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063051+0000", 2024-09-04T17:47:39.142 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2757e-05 2024-09-04T17:47:39.142 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.142 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.142 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:39.142 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063070+0000", 2024-09-04T17:47:39.142 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9663000000000001e-05 2024-09-04T17:47:39.142 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.142 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.142 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:39.142 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063079+0000", 2024-09-04T17:47:39.143 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.0929999999999992e-06 2024-09-04T17:47:39.143 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.143 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.143 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:39.143 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210989+0000", 2024-09-04T17:47:39.143 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14790984500000001 2024-09-04T17:47:39.143 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.143 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.143 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:39.143 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211109+0000", 2024-09-04T17:47:39.143 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000120056 2024-09-04T17:47:39.143 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.143 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.144 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:39.144 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211317+0000", 2024-09-04T17:47:39.144 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020797900000000001 2024-09-04T17:47:39.144 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.144 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:39.144 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.144 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.144 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.144 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8847 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3039232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:39.144 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:47:39.144 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 86.036058574999998, 2024-09-04T17:47:39.144 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14859408900000001, 2024-09-04T17:47:39.144 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:39.145 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:39.145 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:39.145 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:39.145 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:39.145 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:39.145 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8847 2024-09-04T17:47:39.145 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.145 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:39.145 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.145 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:39.145 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:47:39.145 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.145 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.146 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.146 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:39.146 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:47:39.146 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.146 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.146 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.146 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:39.146 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063113+0000", 2024-09-04T17:47:39.146 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.204e-06 2024-09-04T17:47:39.146 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.146 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.146 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:39.146 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063116+0000", 2024-09-04T17:47:39.147 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.3330000000000001e-06 2024-09-04T17:47:39.147 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.147 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.147 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:39.147 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063126+0000", 2024-09-04T17:47:39.147 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0166999999999999e-05 2024-09-04T17:47:39.147 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.147 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.147 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:39.147 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063134+0000", 2024-09-04T17:47:39.147 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4329999999999998e-06 2024-09-04T17:47:39.147 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.147 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.148 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:39.148 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211392+0000", 2024-09-04T17:47:39.148 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148258109 2024-09-04T17:47:39.148 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.148 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.148 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:39.148 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211506+0000", 2024-09-04T17:47:39.148 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000113813 2024-09-04T17:47:39.148 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.148 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.148 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:39.148 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211706+0000", 2024-09-04T17:47:39.149 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020002999999999999 2024-09-04T17:47:39.149 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.149 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:39.149 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.149 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.149 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.149 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8848 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:39.149 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:47:39.149 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 86.036026238000005, 2024-09-04T17:47:39.149 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14907967599999999, 2024-09-04T17:47:39.149 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:39.149 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:39.149 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:39.150 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:39.150 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:39.150 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:39.150 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8848 2024-09-04T17:47:39.150 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.150 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:39.150 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.150 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:39.150 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:47:39.150 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.150 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.150 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.150 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:39.151 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:47:39.151 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.151 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.151 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.151 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:39.151 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070589+0000", 2024-09-04T17:47:39.151 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.007445386 2024-09-04T17:47:39.151 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.151 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.151 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:39.151 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070600+0000", 2024-09-04T17:47:39.151 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0346999999999999e-05 2024-09-04T17:47:39.151 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.152 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.152 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:39.152 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070622+0000", 2024-09-04T17:47:39.152 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2289999999999998e-05 2024-09-04T17:47:39.152 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.152 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.152 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:39.152 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070636+0000", 2024-09-04T17:47:39.152 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4093e-05 2024-09-04T17:47:39.152 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.152 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.152 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:39.152 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211815+0000", 2024-09-04T17:47:39.153 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14117874699999999 2024-09-04T17:47:39.153 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.153 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.153 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:39.153 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211983+0000", 2024-09-04T17:47:39.153 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00016852800000000001 2024-09-04T17:47:39.153 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.153 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.153 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:39.153 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.212224+0000", 2024-09-04T17:47:39.153 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024028499999999999 2024-09-04T17:47:39.153 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.154 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:39.154 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.154 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.154 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.154 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8862 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2019328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:39.154 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:47:39.154 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 86.027976769999995, 2024-09-04T17:47:39.154 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14755236399999999, 2024-09-04T17:47:39.154 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:39.154 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:39.154 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:39.154 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:39.154 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:39.154 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:39.155 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8862 2024-09-04T17:47:39.155 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.155 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:39.155 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.155 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:39.155 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:47:39.155 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.155 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.155 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.155 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:39.155 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:47:39.155 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.156 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.156 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.156 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:39.156 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071194+0000", 2024-09-04T17:47:39.156 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.5000000000000002e-07 2024-09-04T17:47:39.156 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.156 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.156 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:39.156 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071197+0000", 2024-09-04T17:47:39.156 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5900000000000002e-06 2024-09-04T17:47:39.156 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.156 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.156 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:39.157 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071206+0000", 2024-09-04T17:47:39.157 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9700000000000005e-06 2024-09-04T17:47:39.157 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.157 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.157 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:39.157 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071213+0000", 2024-09-04T17:47:39.157 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.9070000000000001e-06 2024-09-04T17:47:39.157 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.157 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.157 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:39.157 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218396+0000", 2024-09-04T17:47:39.157 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147183273 2024-09-04T17:47:39.157 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.158 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.158 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:39.158 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218538+0000", 2024-09-04T17:47:39.158 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014194199999999999 2024-09-04T17:47:39.158 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.158 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.158 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:39.158 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218746+0000", 2024-09-04T17:47:39.158 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000208032 2024-09-04T17:47:39.158 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.158 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:39.158 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.158 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.159 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.159 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8863 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2056192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:39.159 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:47:39.159 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 86.027948460000005, 2024-09-04T17:47:39.159 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14792281099999999, 2024-09-04T17:47:39.159 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:39.159 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:39.159 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:39.159 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:39.159 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:39.159 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:39.159 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8863 2024-09-04T17:47:39.159 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.160 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:39.160 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.160 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:39.160 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:47:39.160 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.160 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.160 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.160 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:39.160 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:47:39.160 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.160 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.160 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.160 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:39.161 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:47:39.161 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1299999999999999e-07 2024-09-04T17:47:39.161 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.161 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.161 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:39.161 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071225+0000", 2024-09-04T17:47:39.161 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6199999999999999e-06 2024-09-04T17:47:39.161 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.161 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.161 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:39.161 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071234+0000", 2024-09-04T17:47:39.161 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8769999999999999e-06 2024-09-04T17:47:39.161 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.162 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.162 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:39.162 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071251+0000", 2024-09-04T17:47:39.162 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.738e-05 2024-09-04T17:47:39.162 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.162 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.162 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:39.162 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218798+0000", 2024-09-04T17:47:39.162 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14754645 2024-09-04T17:47:39.162 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.162 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.162 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:39.163 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218926+0000", 2024-09-04T17:47:39.163 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000128446 2024-09-04T17:47:39.163 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.163 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.163 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:39.163 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219145+0000", 2024-09-04T17:47:39.163 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021832500000000001 2024-09-04T17:47:39.163 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.163 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:39.163 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.163 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.163 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.163 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8864 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2093056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:39.163 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:47:39.164 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 86.027906884000004, 2024-09-04T17:47:39.164 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148277309, 2024-09-04T17:47:39.164 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:39.164 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:39.164 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:39.164 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:39.164 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:39.164 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:39.164 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8864 2024-09-04T17:47:39.164 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.164 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:39.164 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.164 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:39.165 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:47:39.165 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.165 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.165 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.165 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:39.165 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:47:39.165 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.165 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.165 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.165 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:39.165 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071264+0000", 2024-09-04T17:47:39.165 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3e-07 2024-09-04T17:47:39.165 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.166 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.166 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:39.166 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071267+0000", 2024-09-04T17:47:39.166 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0139999999999999e-06 2024-09-04T17:47:39.166 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.166 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.166 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:39.166 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071276+0000", 2024-09-04T17:47:39.166 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1689999999999998e-06 2024-09-04T17:47:39.166 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.166 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.166 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:39.166 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071283+0000", 2024-09-04T17:47:39.167 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2269999999999996e-06 2024-09-04T17:47:39.167 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.167 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.167 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:39.167 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219195+0000", 2024-09-04T17:47:39.167 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147911868 2024-09-04T17:47:39.167 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.167 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.167 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:39.167 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219324+0000", 2024-09-04T17:47:39.167 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00012835900000000001 2024-09-04T17:47:39.167 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.167 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.167 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:39.168 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219541+0000", 2024-09-04T17:47:39.168 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000216942 2024-09-04T17:47:39.168 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.168 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:39.168 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.168 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.168 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.168 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8865 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2109440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:39.168 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:47:39.168 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 86.027877423999996, 2024-09-04T17:47:39.168 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14865747300000001, 2024-09-04T17:47:39.168 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:39.169 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:39.169 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:39.169 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:39.169 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:39.169 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:39.169 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8865 2024-09-04T17:47:39.169 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.169 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:39.169 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.169 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:39.169 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:47:39.169 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.169 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.170 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.170 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:39.170 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:47:39.170 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.170 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.170 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.170 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:39.170 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:47:39.170 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7999999999999995e-07 2024-09-04T17:47:39.170 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.170 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.170 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:39.170 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071296+0000", 2024-09-04T17:47:39.170 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6900000000000001e-06 2024-09-04T17:47:39.171 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.171 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.171 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:39.171 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071305+0000", 2024-09-04T17:47:39.171 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.5499999999999995e-06 2024-09-04T17:47:39.171 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.171 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.171 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:39.171 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071312+0000", 2024-09-04T17:47:39.171 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1559999999999998e-06 2024-09-04T17:47:39.171 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.171 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.171 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:39.172 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219591+0000", 2024-09-04T17:47:39.172 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14827944900000001 2024-09-04T17:47:39.172 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.172 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.172 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:39.172 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219731+0000", 2024-09-04T17:47:39.172 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013931600000000001 2024-09-04T17:47:39.172 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.172 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.172 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:39.172 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219950+0000", 2024-09-04T17:47:39.172 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021963199999999999 2024-09-04T17:47:39.172 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.173 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:39.173 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.173 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.173 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.173 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8866 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2146304~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:39.173 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:47:39.173 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 86.027849191000001, 2024-09-04T17:47:39.173 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14906117899999999, 2024-09-04T17:47:39.173 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:39.173 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:39.174 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:39.174 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:39.174 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:39.174 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:39.174 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8866 2024-09-04T17:47:39.174 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.174 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:39.174 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.174 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:39.174 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:47:39.174 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.174 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.174 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.175 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:39.175 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:47:39.175 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.175 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.175 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.175 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:39.175 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071322+0000", 2024-09-04T17:47:39.175 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7999999999999995e-07 2024-09-04T17:47:39.175 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.175 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.176 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:39.176 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071338+0000", 2024-09-04T17:47:39.176 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5872999999999999e-05 2024-09-04T17:47:39.176 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.176 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.176 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:39.176 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071347+0000", 2024-09-04T17:47:39.176 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4669999999999993e-06 2024-09-04T17:47:39.176 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.176 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.176 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:39.176 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071354+0000", 2024-09-04T17:47:39.177 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2099999999999996e-06 2024-09-04T17:47:39.177 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.177 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.177 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:39.177 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220008+0000", 2024-09-04T17:47:39.177 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148653642 2024-09-04T17:47:39.177 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.177 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.177 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:39.177 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220155+0000", 2024-09-04T17:47:39.177 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014726600000000001 2024-09-04T17:47:39.177 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.177 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.177 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:39.178 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220382+0000", 2024-09-04T17:47:39.178 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00022704100000000001 2024-09-04T17:47:39.178 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.178 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:39.178 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.178 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.178 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.178 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8867 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2183168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:39.178 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:47:39.178 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 86.027805838000006, 2024-09-04T17:47:39.178 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.149445831, 2024-09-04T17:47:39.178 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:39.179 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:39.179 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:39.179 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:39.179 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:39.179 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:39.179 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8867 2024-09-04T17:47:39.179 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.179 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:39.179 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.179 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:39.179 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:47:39.179 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.179 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.179 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.180 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:39.180 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:47:39.180 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.180 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.180 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.180 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:39.180 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078653+0000", 2024-09-04T17:47:39.180 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0072890430000000003 2024-09-04T17:47:39.180 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.180 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.180 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:39.180 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078664+0000", 2024-09-04T17:47:39.180 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0307e-05 2024-09-04T17:47:39.181 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.181 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.181 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:39.181 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078690+0000", 2024-09-04T17:47:39.181 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6780000000000001e-05 2024-09-04T17:47:39.181 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.181 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.181 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:39.181 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078706+0000", 2024-09-04T17:47:39.181 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5676999999999999e-05 2024-09-04T17:47:39.181 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.182 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.182 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:39.182 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220467+0000", 2024-09-04T17:47:39.182 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14176103500000001 2024-09-04T17:47:39.182 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.182 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.182 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:39.182 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220595+0000", 2024-09-04T17:47:39.182 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000127427 2024-09-04T17:47:39.182 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.182 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.182 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:39.182 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220810+0000", 2024-09-04T17:47:39.183 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021556200000000001 2024-09-04T17:47:39.183 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.183 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:39.183 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.183 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.183 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.183 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8880 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2658304~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:39.183 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.079158+0000", 2024-09-04T17:47:39.183 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 86.020012558999994, 2024-09-04T17:47:39.183 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14718968499999999, 2024-09-04T17:47:39.183 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:39.183 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:39.183 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:39.184 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:39.184 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:39.184 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:39.184 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8880 2024-09-04T17:47:39.184 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.184 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:39.184 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.184 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:39.184 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079158+0000", 2024-09-04T17:47:39.184 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.184 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.184 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.184 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:39.185 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079158+0000", 2024-09-04T17:47:39.185 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.185 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.185 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.185 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:39.185 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079158+0000", 2024-09-04T17:47:39.185 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3e-07 2024-09-04T17:47:39.185 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.185 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.185 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:39.185 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079161+0000", 2024-09-04T17:47:39.185 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.57e-06 2024-09-04T17:47:39.185 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.186 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.186 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:39.186 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079170+0000", 2024-09-04T17:47:39.186 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.0170000000000002e-06 2024-09-04T17:47:39.186 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.186 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.186 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:39.186 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079177+0000", 2024-09-04T17:47:39.186 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.0530000000000001e-06 2024-09-04T17:47:39.186 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.186 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.186 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:39.186 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.225946+0000", 2024-09-04T17:47:39.187 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14676936500000001 2024-09-04T17:47:39.187 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.187 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.187 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:39.187 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226098+0000", 2024-09-04T17:47:39.187 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000151805 2024-09-04T17:47:39.187 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.187 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.187 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:39.187 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226347+0000", 2024-09-04T17:47:39.187 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024914499999999999 2024-09-04T17:47:39.187 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.188 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:39.188 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.188 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.188 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.188 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8881 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2695168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:39.188 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:47:39.188 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 86.019983769000007, 2024-09-04T17:47:39.188 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147543547, 2024-09-04T17:47:39.188 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:39.188 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:39.188 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:39.188 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:39.188 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:39.189 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:39.189 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8881 2024-09-04T17:47:39.189 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.189 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:39.189 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.189 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:39.189 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:47:39.189 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.189 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.189 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.189 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:39.189 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:47:39.189 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.190 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.190 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.190 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:39.190 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079187+0000", 2024-09-04T17:47:39.190 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7700000000000004e-07 2024-09-04T17:47:39.190 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.190 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.190 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:39.190 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079198+0000", 2024-09-04T17:47:39.190 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0849999999999999e-05 2024-09-04T17:47:39.190 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.190 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.190 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:39.191 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079207+0000", 2024-09-04T17:47:39.191 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5599999999999999e-06 2024-09-04T17:47:39.191 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.191 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.191 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:39.191 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079224+0000", 2024-09-04T17:47:39.191 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6387000000000001e-05 2024-09-04T17:47:39.191 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.191 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.191 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:39.191 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226414+0000", 2024-09-04T17:47:39.191 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14718975000000001 2024-09-04T17:47:39.191 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.192 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.192 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:39.192 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226534+0000", 2024-09-04T17:47:39.192 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000120288 2024-09-04T17:47:39.192 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.192 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.192 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:39.192 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226730+0000", 2024-09-04T17:47:39.192 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000196035 2024-09-04T17:47:39.192 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.192 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:39.192 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.192 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.193 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.193 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8882 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2732032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:39.193 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:47:39.193 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 86.019935555000004, 2024-09-04T17:47:39.193 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14795022399999999, 2024-09-04T17:47:39.193 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:39.193 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:39.193 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:39.193 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:39.193 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:39.193 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:39.193 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8882 2024-09-04T17:47:39.194 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.194 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:39.194 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.194 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:39.194 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:47:39.194 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.194 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.194 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.194 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:39.194 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:47:39.194 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.194 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.194 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.195 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:39.195 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086588+0000", 2024-09-04T17:47:39.195 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0073530959999999999 2024-09-04T17:47:39.195 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.195 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.195 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:39.195 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086598+0000", 2024-09-04T17:47:39.195 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0122999999999999e-05 2024-09-04T17:47:39.195 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.195 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.195 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:39.195 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086626+0000", 2024-09-04T17:47:39.195 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.7914000000000001e-05 2024-09-04T17:47:39.195 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.196 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.196 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:39.196 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086641+0000", 2024-09-04T17:47:39.196 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5270000000000001e-05 2024-09-04T17:47:39.196 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.196 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.196 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:39.196 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226801+0000", 2024-09-04T17:47:39.196 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.140160321 2024-09-04T17:47:39.196 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.196 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.196 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:39.196 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226951+0000", 2024-09-04T17:47:39.197 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014965200000000001 2024-09-04T17:47:39.197 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.197 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.197 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:39.197 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.227185+0000", 2024-09-04T17:47:39.197 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00023384799999999999 2024-09-04T17:47:39.197 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.197 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:39.197 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.197 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.197 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:39.197 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:39.198 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:47:39.199 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-04T17:47:39.199 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-04T17:47:39.200 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:47:39.200 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-04T17:47:39.200 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:39.197+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:39.300 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:39.295+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:39.364 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:39.364 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:47:39.364 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:47:39.364 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:39.378 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-04T17:47:39.421 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:39.419+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:39.439 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:47:39.437+0000 7fdf502127c0 -1 osd.2 83 log_to_monitors true 2024-09-04T17:47:39.502 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:39.501+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:39.540 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:39.540 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:47:39.540 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:47:39.540 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:47:39.540 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:39.553 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-04T17:47:39.602 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:39.599+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:39.677 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:39.677 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:47:39.677 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:47:39.678 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:47:39.678 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.678 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60921 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 262144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:39.678 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822920+0000", 2024-09-04T17:47:39.678 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 25.852931928, 2024-09-04T17:47:39.678 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073904394999999998, 2024-09-04T17:47:39.678 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:39.678 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:39.678 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:39.678 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:39.678 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:39.678 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:39.679 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60921 2024-09-04T17:47:39.679 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.679 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:39.679 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.679 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:39.679 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822920+0000", 2024-09-04T17:47:39.679 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.679 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.679 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.679 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:39.679 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822920+0000", 2024-09-04T17:47:39.679 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.679 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.680 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.680 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:39.680 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822920+0000", 2024-09-04T17:47:39.680 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4999999999999999e-07 2024-09-04T17:47:39.680 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.680 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.680 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:39.680 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822921+0000", 2024-09-04T17:47:39.680 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.113e-06 2024-09-04T17:47:39.680 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.680 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.680 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:39.680 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822924+0000", 2024-09-04T17:47:39.681 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.354e-06 2024-09-04T17:47:39.681 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.681 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.681 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:39.681 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822927+0000", 2024-09-04T17:47:39.681 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6199999999999999e-06 2024-09-04T17:47:39.681 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.681 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.681 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:39.681 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896586+0000", 2024-09-04T17:47:39.681 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073658976000000001 2024-09-04T17:47:39.681 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.681 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.682 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:39.682 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896626+0000", 2024-09-04T17:47:39.682 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0484000000000001e-05 2024-09-04T17:47:39.682 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.682 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.682 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:39.682 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896824+0000", 2024-09-04T17:47:39.682 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000197598 2024-09-04T17:47:39.682 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.682 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:39.682 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.682 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.682 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.683 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60922 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:39.683 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822930+0000", 2024-09-04T17:47:39.683 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 25.852921352999999, 2024-09-04T17:47:39.683 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074178872000000007, 2024-09-04T17:47:39.683 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:39.683 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:39.683 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:39.683 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:39.683 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:39.683 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:39.683 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60922 2024-09-04T17:47:39.683 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.683 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:39.683 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.684 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:39.684 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822930+0000", 2024-09-04T17:47:39.684 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.684 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.684 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.684 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:39.684 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822930+0000", 2024-09-04T17:47:39.684 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.684 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.684 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.684 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:39.684 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822930+0000", 2024-09-04T17:47:39.684 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.36e-07 2024-09-04T17:47:39.685 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.685 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.685 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:39.685 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822932+0000", 2024-09-04T17:47:39.685 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.567e-06 2024-09-04T17:47:39.685 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.685 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.685 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:39.685 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822938+0000", 2024-09-04T17:47:39.685 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4840000000000003e-06 2024-09-04T17:47:39.685 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.685 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.686 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:39.686 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822942+0000", 2024-09-04T17:47:39.686 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2039999999999999e-06 2024-09-04T17:47:39.686 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.686 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.686 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:39.686 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896850+0000", 2024-09-04T17:47:39.686 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073907941000000005 2024-09-04T17:47:39.687 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.687 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.687 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:39.687 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896891+0000", 2024-09-04T17:47:39.687 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0964999999999998e-05 2024-09-04T17:47:39.687 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.687 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.687 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:39.687 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897109+0000", 2024-09-04T17:47:39.688 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021847500000000001 2024-09-04T17:47:39.688 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.688 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:39.688 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.688 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.688 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.688 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60923 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 475136~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:39.688 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822949+0000", 2024-09-04T17:47:39.689 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 25.852903062999999, 2024-09-04T17:47:39.689 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074426928000000003, 2024-09-04T17:47:39.689 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:39.689 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:39.689 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:39.689 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:39.689 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:39.689 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:39.689 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60923 2024-09-04T17:47:39.689 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.689 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:39.690 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.690 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:39.690 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822949+0000", 2024-09-04T17:47:39.690 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.690 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.690 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.690 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:39.690 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822949+0000", 2024-09-04T17:47:39.690 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.690 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.690 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.690 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:39.691 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822949+0000", 2024-09-04T17:47:39.691 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7399999999999999e-07 2024-09-04T17:47:39.691 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.691 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.691 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:39.691 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822951+0000", 2024-09-04T17:47:39.691 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1940000000000001e-06 2024-09-04T17:47:39.691 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.691 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.691 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:39.691 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822957+0000", 2024-09-04T17:47:39.691 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.0650000000000004e-06 2024-09-04T17:47:39.691 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.692 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.692 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:39.692 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822960+0000", 2024-09-04T17:47:39.692 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.8009999999999999e-06 2024-09-04T17:47:39.692 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.692 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.692 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:39.692 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897138+0000", 2024-09-04T17:47:39.692 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074177990999999999 2024-09-04T17:47:39.692 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.692 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.692 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:39.692 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897179+0000", 2024-09-04T17:47:39.693 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.1057999999999999e-05 2024-09-04T17:47:39.693 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.693 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.693 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:39.693 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897375+0000", 2024-09-04T17:47:39.693 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019644500000000001 2024-09-04T17:47:39.693 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.693 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:39.693 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.693 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.693 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.693 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60924 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:39.693 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822964+0000", 2024-09-04T17:47:39.694 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 25.852887878000001, 2024-09-04T17:47:39.694 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074531902999999997, 2024-09-04T17:47:39.694 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:39.694 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:39.694 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:39.694 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:39.694 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:39.695 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:39.695 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60924 2024-09-04T17:47:39.695 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.695 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:39.695 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.695 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:39.695 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822964+0000", 2024-09-04T17:47:39.695 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.695 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.695 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.695 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:39.695 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822964+0000", 2024-09-04T17:47:39.695 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.696 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.696 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.696 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:39.696 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822964+0000", 2024-09-04T17:47:39.696 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4999999999999999e-07 2024-09-04T17:47:39.696 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.696 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.696 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:39.696 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822966+0000", 2024-09-04T17:47:39.697 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.917e-06 2024-09-04T17:47:39.697 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.697 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.697 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:39.697 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822969+0000", 2024-09-04T17:47:39.697 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.315e-06 2024-09-04T17:47:39.697 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.697 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.697 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:39.697 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822972+0000", 2024-09-04T17:47:39.697 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.599e-06 2024-09-04T17:47:39.697 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.697 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.698 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:39.698 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897398+0000", 2024-09-04T17:47:39.698 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074426168000000001 2024-09-04T17:47:39.698 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.698 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.698 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:39.698 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897441+0000", 2024-09-04T17:47:39.698 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.3158000000000003e-05 2024-09-04T17:47:39.698 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.698 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.698 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:39.698 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897496+0000", 2024-09-04T17:47:39.698 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4496000000000001e-05 2024-09-04T17:47:39.699 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.699 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:39.699 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.699 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.699 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.699 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60925 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 720896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:39.699 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822975+0000", 2024-09-04T17:47:39.699 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 25.852876443, 2024-09-04T17:47:39.699 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074635272000000003, 2024-09-04T17:47:39.700 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:39.700 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:39.700 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:39.700 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:39.700 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:39.700 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:39.700 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60925 2024-09-04T17:47:39.700 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.700 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:39.700 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.700 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:39.700 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822975+0000", 2024-09-04T17:47:39.701 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.701 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.701 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.701 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:39.701 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822975+0000", 2024-09-04T17:47:39.701 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.701 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.701 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.701 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:39.701 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822975+0000", 2024-09-04T17:47:39.701 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.34e-07 2024-09-04T17:47:39.701 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.701 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.702 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:39.702 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822976+0000", 2024-09-04T17:47:39.702 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0359999999999999e-06 2024-09-04T17:47:39.702 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.702 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.702 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:39.702 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822980+0000", 2024-09-04T17:47:39.702 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.242e-06 2024-09-04T17:47:39.702 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.702 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.702 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:39.702 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822983+0000", 2024-09-04T17:47:39.702 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.8830000000000002e-06 2024-09-04T17:47:39.703 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.703 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.703 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:39.703 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897519+0000", 2024-09-04T17:47:39.703 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074536366000000007 2024-09-04T17:47:39.703 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.703 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.703 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:39.703 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897560+0000", 2024-09-04T17:47:39.703 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0677999999999999e-05 2024-09-04T17:47:39.703 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.703 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.703 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:39.703 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897610+0000", 2024-09-04T17:47:39.703 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0832999999999999e-05 2024-09-04T17:47:39.704 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.704 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:39.704 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.704 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.704 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.704 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60926 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 860160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:39.704 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822986+0000", 2024-09-04T17:47:39.704 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 25.852865845, 2024-09-04T17:47:39.704 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074736413000000002, 2024-09-04T17:47:39.704 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:39.704 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:39.704 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:39.704 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:39.705 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:39.705 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:39.705 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60926 2024-09-04T17:47:39.705 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.705 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:39.705 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.705 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:39.705 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822986+0000", 2024-09-04T17:47:39.705 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.705 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.705 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.705 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:39.705 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822986+0000", 2024-09-04T17:47:39.706 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.706 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.706 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.706 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:39.706 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822986+0000", 2024-09-04T17:47:39.706 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5600000000000002e-07 2024-09-04T17:47:39.706 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.706 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.706 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:39.706 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822987+0000", 2024-09-04T17:47:39.706 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.082e-06 2024-09-04T17:47:39.706 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.706 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.706 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:39.707 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822990+0000", 2024-09-04T17:47:39.707 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.2219999999999998e-06 2024-09-04T17:47:39.707 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.707 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.707 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:39.707 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822994+0000", 2024-09-04T17:47:39.707 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.19e-06 2024-09-04T17:47:39.707 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.707 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.707 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:39.707 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897632+0000", 2024-09-04T17:47:39.707 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074638205999999999 2024-09-04T17:47:39.707 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.707 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.708 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:39.708 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897672+0000", 2024-09-04T17:47:39.708 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.9873999999999998e-05 2024-09-04T17:47:39.708 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.708 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.708 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:39.708 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897722+0000", 2024-09-04T17:47:39.708 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0583e-05 2024-09-04T17:47:39.708 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.708 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:39.708 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.708 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.708 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.708 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60927 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 966656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:39.709 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823000+0000", 2024-09-04T17:47:39.709 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 25.852851662999999, 2024-09-04T17:47:39.709 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074834036000000007, 2024-09-04T17:47:39.709 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:39.709 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:39.709 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:39.709 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:39.709 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:39.709 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:39.709 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60927 2024-09-04T17:47:39.709 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.709 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:39.709 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.710 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:39.710 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823000+0000", 2024-09-04T17:47:39.710 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.710 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.710 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.710 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:39.710 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823000+0000", 2024-09-04T17:47:39.710 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.710 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.710 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.710 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:39.710 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823000+0000", 2024-09-04T17:47:39.710 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4099999999999999e-07 2024-09-04T17:47:39.711 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.711 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.711 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:39.711 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823003+0000", 2024-09-04T17:47:39.711 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0810000000000001e-06 2024-09-04T17:47:39.711 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.711 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.711 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:39.711 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823011+0000", 2024-09-04T17:47:39.711 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7090000000000008e-06 2024-09-04T17:47:39.711 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.711 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.711 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:39.712 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823017+0000", 2024-09-04T17:47:39.712 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.3890000000000004e-06 2024-09-04T17:47:39.712 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.712 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.712 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:39.712 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897745+0000", 2024-09-04T17:47:39.712 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074728252999999994 2024-09-04T17:47:39.712 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.712 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.712 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:39.712 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897785+0000", 2024-09-04T17:47:39.712 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.9830999999999999e-05 2024-09-04T17:47:39.712 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.712 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.713 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:39.713 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897834+0000", 2024-09-04T17:47:39.713 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9231999999999999e-05 2024-09-04T17:47:39.713 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.713 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:39.713 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.713 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.713 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.713 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60928 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:39.713 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823024+0000", 2024-09-04T17:47:39.713 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 25.852827732000002, 2024-09-04T17:47:39.713 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075067479000000006, 2024-09-04T17:47:39.713 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:39.713 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:39.714 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:39.714 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:39.714 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:39.714 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:39.714 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60928 2024-09-04T17:47:39.714 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.714 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:39.714 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.714 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:39.714 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823024+0000", 2024-09-04T17:47:39.714 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.714 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.714 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.714 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:39.714 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823024+0000", 2024-09-04T17:47:39.715 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.715 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.715 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.715 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:39.715 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823024+0000", 2024-09-04T17:47:39.715 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6499999999999999e-07 2024-09-04T17:47:39.715 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.715 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.715 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:39.715 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823027+0000", 2024-09-04T17:47:39.715 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.3099999999999999e-06 2024-09-04T17:47:39.715 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.715 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.715 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:39.716 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823033+0000", 2024-09-04T17:47:39.716 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7429999999999996e-06 2024-09-04T17:47:39.716 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.716 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.716 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:39.716 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823038+0000", 2024-09-04T17:47:39.716 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.561e-06 2024-09-04T17:47:39.716 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.716 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.716 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:39.716 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897855+0000", 2024-09-04T17:47:39.716 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074816740000000007 2024-09-04T17:47:39.716 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.717 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.717 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:39.717 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897894+0000", 2024-09-04T17:47:39.717 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.9085999999999998e-05 2024-09-04T17:47:39.717 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.717 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.717 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:39.717 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898091+0000", 2024-09-04T17:47:39.717 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000197574 2024-09-04T17:47:39.717 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.717 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:39.717 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.717 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.717 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.718 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60929 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1212416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:39.718 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823042+0000", 2024-09-04T17:47:39.718 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 25.85280929, 2024-09-04T17:47:39.718 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075301251, 2024-09-04T17:47:39.718 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:39.718 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:39.718 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:39.718 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:39.718 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:39.718 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:39.718 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60929 2024-09-04T17:47:39.718 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.718 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:39.719 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.719 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:39.719 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823042+0000", 2024-09-04T17:47:39.719 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.719 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.719 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.719 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:39.719 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823042+0000", 2024-09-04T17:47:39.719 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.719 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.719 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.719 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:39.719 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823043+0000", 2024-09-04T17:47:39.720 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.1100000000000002e-07 2024-09-04T17:47:39.720 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.720 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.720 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:39.720 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823045+0000", 2024-09-04T17:47:39.720 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.933e-06 2024-09-04T17:47:39.720 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.720 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.720 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:39.720 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823051+0000", 2024-09-04T17:47:39.720 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.9569999999999999e-06 2024-09-04T17:47:39.720 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.720 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.721 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:39.721 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823055+0000", 2024-09-04T17:47:39.721 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4479999999999996e-06 2024-09-04T17:47:39.721 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.721 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.721 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:39.721 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898117+0000", 2024-09-04T17:47:39.721 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075062344000000003 2024-09-04T17:47:39.721 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.721 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.721 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:39.721 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898157+0000", 2024-09-04T17:47:39.721 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0095999999999997e-05 2024-09-04T17:47:39.722 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.722 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.722 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:39.722 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898344+0000", 2024-09-04T17:47:39.722 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000186162 2024-09-04T17:47:39.722 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.722 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:39.722 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.722 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.722 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.722 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60930 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1351680~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:39.722 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823061+0000", 2024-09-04T17:47:39.722 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 25.852790411000001, 2024-09-04T17:47:39.722 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075545507999999997, 2024-09-04T17:47:39.723 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:39.723 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:39.723 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:39.723 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:39.723 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:39.723 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:39.723 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60930 2024-09-04T17:47:39.723 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.723 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:39.723 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.723 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:39.723 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823061+0000", 2024-09-04T17:47:39.723 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.723 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.724 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.724 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:39.724 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823061+0000", 2024-09-04T17:47:39.724 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.724 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.724 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.724 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:39.724 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823378+0000", 2024-09-04T17:47:39.725 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00031631299999999998 2024-09-04T17:47:39.725 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.725 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.725 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:39.725 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823381+0000", 2024-09-04T17:47:39.725 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4020000000000002e-06 2024-09-04T17:47:39.725 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.725 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.725 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:39.725 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823389+0000", 2024-09-04T17:47:39.725 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.1429999999999998e-06 2024-09-04T17:47:39.725 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.725 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.726 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:39.726 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823395+0000", 2024-09-04T17:47:39.726 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.0059999999999998e-06 2024-09-04T17:47:39.726 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.726 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.726 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:39.726 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898368+0000", 2024-09-04T17:47:39.726 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074973032999999994 2024-09-04T17:47:39.726 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.726 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.726 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:39.726 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898410+0000", 2024-09-04T17:47:39.726 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2243000000000002e-05 2024-09-04T17:47:39.726 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.727 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.727 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:39.727 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898607+0000", 2024-09-04T17:47:39.727 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019636799999999999 2024-09-04T17:47:39.727 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.727 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:39.727 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.727 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.727 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.727 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60931 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1458176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:39.727 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823407+0000", 2024-09-04T17:47:39.727 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 25.852444337000001, 2024-09-04T17:47:39.727 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075460334000000004, 2024-09-04T17:47:39.728 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:39.728 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:39.728 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:39.728 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:39.728 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:39.728 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:39.728 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60931 2024-09-04T17:47:39.728 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.728 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:39.728 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.728 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:39.728 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823407+0000", 2024-09-04T17:47:39.728 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.728 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.729 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.729 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:39.729 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823407+0000", 2024-09-04T17:47:39.729 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.729 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.729 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.729 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:39.729 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823408+0000", 2024-09-04T17:47:39.729 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9999999999999998e-07 2024-09-04T17:47:39.729 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.729 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.729 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:39.729 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823415+0000", 2024-09-04T17:47:39.729 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.5510000000000002e-06 2024-09-04T17:47:39.730 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.730 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.730 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:39.730 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823423+0000", 2024-09-04T17:47:39.730 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2760000000000003e-06 2024-09-04T17:47:39.730 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.730 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.730 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:39.730 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823426+0000", 2024-09-04T17:47:39.730 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.2839999999999998e-06 2024-09-04T17:47:39.730 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.730 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.730 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:39.730 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898631+0000", 2024-09-04T17:47:39.731 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075205045999999998 2024-09-04T17:47:39.731 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.731 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.731 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:39.731 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898672+0000", 2024-09-04T17:47:39.731 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0614000000000001e-05 2024-09-04T17:47:39.731 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.731 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.731 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:39.731 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898868+0000", 2024-09-04T17:47:39.731 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019606300000000001 2024-09-04T17:47:39.731 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.731 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:39.731 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.732 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.732 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.732 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60932 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1564672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:39.732 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823434+0000", 2024-09-04T17:47:39.732 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 25.852418038, 2024-09-04T17:47:39.732 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075693393999999997, 2024-09-04T17:47:39.732 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:39.732 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:39.732 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:39.732 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:39.732 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:39.732 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:39.732 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60932 2024-09-04T17:47:39.733 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.733 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:39.733 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.733 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:39.733 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823434+0000", 2024-09-04T17:47:39.733 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.733 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.733 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.733 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:39.733 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823434+0000", 2024-09-04T17:47:39.733 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.733 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.733 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.733 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:39.734 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.830290+0000", 2024-09-04T17:47:39.734 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0068563039999999997 2024-09-04T17:47:39.734 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.734 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.734 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:39.734 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.830297+0000", 2024-09-04T17:47:39.734 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4760000000000001e-06 2024-09-04T17:47:39.734 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.734 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.734 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:39.734 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.830308+0000", 2024-09-04T17:47:39.734 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0927999999999999e-05 2024-09-04T17:47:39.734 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.734 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.735 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:39.735 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.830317+0000", 2024-09-04T17:47:39.735 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7280000000000001e-06 2024-09-04T17:47:39.735 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.735 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.735 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:39.735 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898892+0000", 2024-09-04T17:47:39.735 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.068574722000000005 2024-09-04T17:47:39.735 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.735 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.735 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:39.735 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898934+0000", 2024-09-04T17:47:39.735 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2249000000000002e-05 2024-09-04T17:47:39.736 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.736 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.736 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:39.736 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.899127+0000", 2024-09-04T17:47:39.736 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019298699999999999 2024-09-04T17:47:39.736 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.736 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:39.736 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.736 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.736 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.736 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61182 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 868352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:39.736 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839249+0000", 2024-09-04T17:47:39.736 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 25.836602146000001, 2024-09-04T17:47:39.736 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073680151999999999, 2024-09-04T17:47:39.737 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:39.737 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:39.737 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:39.737 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:39.737 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:39.737 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:39.737 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61182 2024-09-04T17:47:39.737 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.737 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:39.737 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.737 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:39.737 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839249+0000", 2024-09-04T17:47:39.737 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.737 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.738 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.738 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:39.738 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839249+0000", 2024-09-04T17:47:39.738 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.738 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.738 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.738 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:39.738 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839250+0000", 2024-09-04T17:47:39.738 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4999999999999998e-07 2024-09-04T17:47:39.738 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.738 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.738 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:39.738 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839257+0000", 2024-09-04T17:47:39.738 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7700000000000004e-06 2024-09-04T17:47:39.739 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.739 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.739 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:39.739 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839263+0000", 2024-09-04T17:47:39.739 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.246e-06 2024-09-04T17:47:39.739 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.739 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.739 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:39.739 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839268+0000", 2024-09-04T17:47:39.739 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.814e-06 2024-09-04T17:47:39.739 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.739 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.739 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:39.739 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.912831+0000", 2024-09-04T17:47:39.740 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073563294000000001 2024-09-04T17:47:39.740 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.740 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.740 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:39.740 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.912876+0000", 2024-09-04T17:47:39.740 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4742e-05 2024-09-04T17:47:39.740 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.740 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.740 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:39.740 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.912930+0000", 2024-09-04T17:47:39.740 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.3936e-05 2024-09-04T17:47:39.740 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.740 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:39.741 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.741 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.741 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.741 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61183 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:39.741 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839275+0000", 2024-09-04T17:47:39.741 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 25.836576953000002, 2024-09-04T17:47:39.741 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073781886000000005, 2024-09-04T17:47:39.741 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:39.741 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:39.741 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:39.741 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:39.741 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:39.741 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:39.741 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61183 2024-09-04T17:47:39.742 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.742 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:39.742 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.742 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:39.742 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839275+0000", 2024-09-04T17:47:39.742 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.742 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.742 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.742 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:39.742 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839275+0000", 2024-09-04T17:47:39.742 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.742 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.742 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.742 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:39.743 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839275+0000", 2024-09-04T17:47:39.743 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6899999999999997e-07 2024-09-04T17:47:39.743 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.743 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.743 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:39.743 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839277+0000", 2024-09-04T17:47:39.743 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9530000000000002e-06 2024-09-04T17:47:39.743 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.743 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.743 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:39.743 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839283+0000", 2024-09-04T17:47:39.743 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.9329999999999998e-06 2024-09-04T17:47:39.743 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.744 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.744 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:39.744 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839288+0000", 2024-09-04T17:47:39.744 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4100000000000001e-06 2024-09-04T17:47:39.744 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.744 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.744 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:39.744 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.912959+0000", 2024-09-04T17:47:39.744 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073671511999999995 2024-09-04T17:47:39.744 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.744 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.744 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:39.744 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913001+0000", 2024-09-04T17:47:39.744 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.1984999999999999e-05 2024-09-04T17:47:39.745 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.745 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.745 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:39.745 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913057+0000", 2024-09-04T17:47:39.745 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.5523999999999999e-05 2024-09-04T17:47:39.745 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.745 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:39.745 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.745 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.745 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.745 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61184 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1114112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:39.745 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839294+0000", 2024-09-04T17:47:39.745 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 25.836557741, 2024-09-04T17:47:39.746 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073995263000000006, 2024-09-04T17:47:39.746 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:39.746 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:39.746 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:39.746 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:39.746 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:39.746 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:39.746 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61184 2024-09-04T17:47:39.746 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.746 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:39.746 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.746 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:39.746 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839294+0000", 2024-09-04T17:47:39.746 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.747 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.747 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.747 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:39.747 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839294+0000", 2024-09-04T17:47:39.747 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.747 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.747 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.747 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:39.747 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839294+0000", 2024-09-04T17:47:39.747 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.77e-07 2024-09-04T17:47:39.747 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.747 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.747 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:39.747 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839296+0000", 2024-09-04T17:47:39.748 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9999999999999999e-06 2024-09-04T17:47:39.748 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.748 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.748 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:39.748 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839302+0000", 2024-09-04T17:47:39.748 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7230000000000001e-06 2024-09-04T17:47:39.748 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.748 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.748 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:39.748 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839314+0000", 2024-09-04T17:47:39.748 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1826e-05 2024-09-04T17:47:39.748 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.748 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.749 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:39.749 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913084+0000", 2024-09-04T17:47:39.749 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073770215 2024-09-04T17:47:39.749 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.749 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.749 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:39.749 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913127+0000", 2024-09-04T17:47:39.749 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2982999999999998e-05 2024-09-04T17:47:39.749 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.749 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.749 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:39.749 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913289+0000", 2024-09-04T17:47:39.749 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000162139 2024-09-04T17:47:39.749 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.750 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:39.750 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.750 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.750 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.750 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61185 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1220608~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:39.750 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839321+0000", 2024-09-04T17:47:39.750 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 25.836530274000001, 2024-09-04T17:47:39.750 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074206062000000003, 2024-09-04T17:47:39.750 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:39.750 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:39.751 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:39.751 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:39.751 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:39.751 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:39.751 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61185 2024-09-04T17:47:39.751 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.751 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:39.751 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.751 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:39.751 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839321+0000", 2024-09-04T17:47:39.751 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.751 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.751 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.752 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:39.752 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839321+0000", 2024-09-04T17:47:39.752 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.752 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.752 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.752 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:39.752 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839322+0000", 2024-09-04T17:47:39.752 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.5400000000000001e-07 2024-09-04T17:47:39.752 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.752 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.752 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:39.752 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839324+0000", 2024-09-04T17:47:39.752 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2539999999999999e-06 2024-09-04T17:47:39.752 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.753 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.753 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:39.753 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839331+0000", 2024-09-04T17:47:39.753 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2139999999999999e-06 2024-09-04T17:47:39.753 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.753 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.753 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:39.753 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839336+0000", 2024-09-04T17:47:39.753 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7029999999999997e-06 2024-09-04T17:47:39.753 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.753 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.753 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:39.753 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913320+0000", 2024-09-04T17:47:39.754 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073984016 2024-09-04T17:47:39.754 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.754 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.754 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:39.754 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913366+0000", 2024-09-04T17:47:39.754 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.5704000000000002e-05 2024-09-04T17:47:39.754 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.754 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.754 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:39.754 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913527+0000", 2024-09-04T17:47:39.754 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00016161699999999999 2024-09-04T17:47:39.754 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.754 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:39.754 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.755 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.755 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.755 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61186 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1359872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:39.755 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839344+0000", 2024-09-04T17:47:39.755 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 25.836507932, 2024-09-04T17:47:39.755 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074450673999999994, 2024-09-04T17:47:39.755 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:39.755 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:39.755 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:39.755 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:39.755 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:39.755 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:39.755 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61186 2024-09-04T17:47:39.755 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.756 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:39.756 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.756 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:39.756 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839344+0000", 2024-09-04T17:47:39.756 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.756 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.756 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.756 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:39.756 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839344+0000", 2024-09-04T17:47:39.756 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.756 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.756 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.756 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:39.757 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839344+0000", 2024-09-04T17:47:39.757 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4799999999999999e-07 2024-09-04T17:47:39.757 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.757 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.757 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:39.757 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839346+0000", 2024-09-04T17:47:39.757 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0889999999999998e-06 2024-09-04T17:47:39.757 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.757 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.757 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:39.757 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839352+0000", 2024-09-04T17:47:39.757 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6389999999999997e-06 2024-09-04T17:47:39.757 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.757 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.758 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:39.758 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839356+0000", 2024-09-04T17:47:39.758 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.233e-06 2024-09-04T17:47:39.758 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.758 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.758 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:39.758 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913557+0000", 2024-09-04T17:47:39.758 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074200944000000005 2024-09-04T17:47:39.758 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.758 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.758 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:39.759 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913601+0000", 2024-09-04T17:47:39.759 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4350000000000001e-05 2024-09-04T17:47:39.759 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.759 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.759 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:39.759 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913794+0000", 2024-09-04T17:47:39.759 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019297100000000001 2024-09-04T17:47:39.759 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.759 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:39.759 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.759 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.760 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.760 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61187 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1466368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:39.760 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839364+0000", 2024-09-04T17:47:39.760 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 25.836488082999999, 2024-09-04T17:47:39.760 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074692334999999999, 2024-09-04T17:47:39.760 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:39.760 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:39.760 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:39.760 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:39.760 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:39.760 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:39.760 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61187 2024-09-04T17:47:39.760 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.761 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:39.761 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.761 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:39.761 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839364+0000", 2024-09-04T17:47:39.761 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.761 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.761 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.761 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:39.761 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839364+0000", 2024-09-04T17:47:39.761 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.761 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.761 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.761 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:39.762 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839364+0000", 2024-09-04T17:47:39.762 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.75e-07 2024-09-04T17:47:39.762 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.762 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.762 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:39.762 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839366+0000", 2024-09-04T17:47:39.762 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0990000000000001e-06 2024-09-04T17:47:39.762 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.762 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.762 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:39.762 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839372+0000", 2024-09-04T17:47:39.762 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.2609999999999999e-06 2024-09-04T17:47:39.762 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.763 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.763 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:39.763 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839389+0000", 2024-09-04T17:47:39.763 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6489000000000001e-05 2024-09-04T17:47:39.763 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.763 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.763 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:39.763 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913824+0000", 2024-09-04T17:47:39.763 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074434811000000004 2024-09-04T17:47:39.763 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.763 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.763 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:39.763 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913868+0000", 2024-09-04T17:47:39.763 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4768000000000002e-05 2024-09-04T17:47:39.764 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.764 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.764 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:39.764 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914056+0000", 2024-09-04T17:47:39.764 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00018743199999999999 2024-09-04T17:47:39.764 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.764 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:39.764 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.764 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.764 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.764 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61188 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1605632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:39.764 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839399+0000", 2024-09-04T17:47:39.764 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 25.836452186999999, 2024-09-04T17:47:39.764 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074975095000000005, 2024-09-04T17:47:39.765 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:39.765 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:39.765 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:39.765 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:39.765 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:39.765 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:39.765 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61188 2024-09-04T17:47:39.765 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.765 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:39.765 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.765 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:39.765 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839399+0000", 2024-09-04T17:47:39.765 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.765 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.766 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.766 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:39.766 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839399+0000", 2024-09-04T17:47:39.766 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.766 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.766 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.766 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:39.766 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839400+0000", 2024-09-04T17:47:39.766 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0100000000000005e-07 2024-09-04T17:47:39.766 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.766 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.766 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:39.766 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839402+0000", 2024-09-04T17:47:39.766 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.491e-06 2024-09-04T17:47:39.766 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.767 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.767 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:39.767 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839409+0000", 2024-09-04T17:47:39.767 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.0450000000000003e-06 2024-09-04T17:47:39.767 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.767 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.767 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:39.767 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839414+0000", 2024-09-04T17:47:39.767 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.87e-06 2024-09-04T17:47:39.767 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.767 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.767 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:39.767 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914086+0000", 2024-09-04T17:47:39.767 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074671596000000007 2024-09-04T17:47:39.768 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.768 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.768 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:39.768 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914130+0000", 2024-09-04T17:47:39.768 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4579999999999997e-05 2024-09-04T17:47:39.768 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.768 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.768 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:39.768 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914375+0000", 2024-09-04T17:47:39.768 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024401199999999999 2024-09-04T17:47:39.768 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.768 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:39.768 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.769 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.769 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.769 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61189 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1712128~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:39.769 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839424+0000", 2024-09-04T17:47:39.769 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 25.836427986, 2024-09-04T17:47:39.769 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075247680999999997, 2024-09-04T17:47:39.769 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:39.769 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:39.769 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:39.769 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:39.769 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:39.769 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:39.769 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61189 2024-09-04T17:47:39.770 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.770 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:39.770 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.770 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:39.770 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839424+0000", 2024-09-04T17:47:39.770 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.770 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.770 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.770 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:39.770 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839424+0000", 2024-09-04T17:47:39.770 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:39.770 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.770 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.770 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:39.771 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.846320+0000", 2024-09-04T17:47:39.771 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0068965210000000001 2024-09-04T17:47:39.771 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.771 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.771 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:39.771 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.846328+0000", 2024-09-04T17:47:39.771 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.8189999999999992e-06 2024-09-04T17:47:39.771 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.771 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.771 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:39.772 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.846341+0000", 2024-09-04T17:47:39.772 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3338e-05 2024-09-04T17:47:39.772 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.772 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.772 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:39.772 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.846352+0000", 2024-09-04T17:47:39.772 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0479000000000001e-05 2024-09-04T17:47:39.772 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.772 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.772 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:39.772 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914405+0000", 2024-09-04T17:47:39.772 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.068053049000000004 2024-09-04T17:47:39.773 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.773 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.773 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:39.773 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914451+0000", 2024-09-04T17:47:39.773 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6545999999999998e-05 2024-09-04T17:47:39.773 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:39.773 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:39.773 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:39.773 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914671+0000", 2024-09-04T17:47:39.773 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000219929 2024-09-04T17:47:39.773 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.773 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:39.773 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.774 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:39.774 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:39.774 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:39.775 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-04T17:47:39.776 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:39.771+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:39.877 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:39.875+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:39.913 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:39.913 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:47:39.913 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:47:39.913 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:39.915 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-04T17:47:39.928 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-04T17:47:39.933 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:47:39.933+0000 7fdf43fed640 -1 osd.2 83 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-04T17:47:39.978 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:39.975+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:40.053 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:40.053 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:47:40.053 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:47:40.053 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:47:40.053 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:40.070 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-04T17:47:40.079 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:40.075+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:40.180 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:47:40.175+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:47:40.189 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:40.189 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:47:40.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:47:40.189 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:47:40.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.189 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:27242 3.b 3.34414deb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-04T17:47:40.189 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:28.402347+0000", 2024-09-04T17:47:40.190 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 71.784944800999995, 2024-09-04T17:47:40.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.93265329299999999, 2024-09-04T17:47:40.190 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:40.190 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:40.190 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-04T17:47:40.190 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:40.190 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:40.190 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:40.190 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 27242 2024-09-04T17:47:40.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.190 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:40.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:40.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402347+0000", 2024-09-04T17:47:40.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:40.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402347+0000", 2024-09-04T17:47:40.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:40.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402349+0000", 2024-09-04T17:47:40.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.471e-06 2024-09-04T17:47:40.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.192 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:40.192 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402355+0000", 2024-09-04T17:47:40.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8399999999999997e-06 2024-09-04T17:47:40.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.192 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:40.192 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402364+0000", 2024-09-04T17:47:40.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1269999999999992e-06 2024-09-04T17:47:40.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:40.193 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402373+0000", 2024-09-04T17:47:40.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7909999999999993e-06 2024-09-04T17:47:40.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:40.193 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402460+0000", 2024-09-04T17:47:40.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6686000000000003e-05 2024-09-04T17:47:40.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-04T17:47:40.193 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402473+0000", 2024-09-04T17:47:40.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3281e-05 2024-09-04T17:47:40.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:40.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:29.334989+0000", 2024-09-04T17:47:40.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.93251549099999997 2024-09-04T17:47:40.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:40.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:29.335000+0000", 2024-09-04T17:47:40.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1606e-05 2024-09-04T17:47:40.195 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.195 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:40.195 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.195 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:33552 3.7 3:e79ad1db:::rbd_data.11342e8f3384.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-04T17:47:40.195 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:35.999748+0000", 2024-09-04T17:47:40.195 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 64.187543751000007, 2024-09-04T17:47:40.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045683823999999998, 2024-09-04T17:47:40.195 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:40.196 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:40.196 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:40.196 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:40.196 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:40.196 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:40.196 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 33552 2024-09-04T17:47:40.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.196 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:40.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.196 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:40.196 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999748+0000", 2024-09-04T17:47:40.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:40.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999748+0000", 2024-09-04T17:47:40.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:40.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999750+0000", 2024-09-04T17:47:40.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1299999999999999e-06 2024-09-04T17:47:40.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:40.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999759+0000", 2024-09-04T17:47:40.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4649999999999994e-06 2024-09-04T17:47:40.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:40.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999778+0000", 2024-09-04T17:47:40.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8833000000000001e-05 2024-09-04T17:47:40.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:40.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999786+0000", 2024-09-04T17:47:40.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7369999999999995e-06 2024-09-04T17:47:40.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:40.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999856+0000", 2024-09-04T17:47:40.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9247999999999998e-05 2024-09-04T17:47:40.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:40.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999973+0000", 2024-09-04T17:47:40.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000117921 2024-09-04T17:47:40.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 7", 2024-09-04T17:47:40.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.005955+0000", 2024-09-04T17:47:40.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059820560000000004 2024-09-04T17:47:40.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:40.200 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.045297+0000", 2024-09-04T17:47:40.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.039341650999999998 2024-09-04T17:47:40.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.200 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:40.200 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.045343+0000", 2024-09-04T17:47:40.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5899000000000002e-05 2024-09-04T17:47:40.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.200 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:40.200 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.045388+0000", 2024-09-04T17:47:40.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4601999999999997e-05 2024-09-04T17:47:40.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:40.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.045432+0000", 2024-09-04T17:47:40.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4282000000000001e-05 2024-09-04T17:47:40.201 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.201 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:40.201 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.201 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:37357 3.15 3:a897312a:::rbd_data.1134c5a43465.000000000000001f:head [stat,write 62464~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-04T17:47:40.201 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:39.616724+0000", 2024-09-04T17:47:40.201 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 60.570568342999998, 2024-09-04T17:47:40.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054965854000000001, 2024-09-04T17:47:40.201 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:40.202 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:40.202 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:40.202 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:40.202 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:40.202 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:40.202 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 37357 2024-09-04T17:47:40.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.202 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:40.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.202 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:40.202 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616724+0000", 2024-09-04T17:47:40.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:40.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616724+0000", 2024-09-04T17:47:40.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:40.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616725+0000", 2024-09-04T17:47:40.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.815e-06 2024-09-04T17:47:40.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:40.204 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616739+0000", 2024-09-04T17:47:40.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3723000000000001e-05 2024-09-04T17:47:40.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.204 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:40.204 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616755+0000", 2024-09-04T17:47:40.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5964000000000002e-05 2024-09-04T17:47:40.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.205 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:40.205 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616763+0000", 2024-09-04T17:47:40.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3869999999999995e-06 2024-09-04T17:47:40.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:40.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616825+0000", 2024-09-04T17:47:40.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1854999999999996e-05 2024-09-04T17:47:40.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.207 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:40.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616895+0000", 2024-09-04T17:47:40.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9597999999999993e-05 2024-09-04T17:47:40.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.208 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-04T17:47:40.208 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616991+0000", 2024-09-04T17:47:40.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5828000000000005e-05 2024-09-04T17:47:40.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.208 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:40.209 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.617659+0000", 2024-09-04T17:47:40.209 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00066821600000000001 2024-09-04T17:47:40.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.209 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.209 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:40.210 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.671482+0000", 2024-09-04T17:47:40.210 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053823533999999999 2024-09-04T17:47:40.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.210 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:40.210 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.671579+0000", 2024-09-04T17:47:40.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6657999999999995e-05 2024-09-04T17:47:40.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:40.211 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.671689+0000", 2024-09-04T17:47:40.212 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000110276 2024-09-04T17:47:40.212 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.212 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:40.212 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.213 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:43873 3.15 3:abd207ee:::rbd_data.113410ddd206.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:40.213 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:49.932940+0000", 2024-09-04T17:47:40.213 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.254351790000001, 2024-09-04T17:47:40.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042740514, 2024-09-04T17:47:40.213 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:40.213 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:40.214 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:40.214 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:40.214 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:40.214 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:40.214 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 43873 2024-09-04T17:47:40.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.215 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:40.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:40.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.932940+0000", 2024-09-04T17:47:40.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:40.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.932940+0000", 2024-09-04T17:47:40.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.217 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:40.217 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.932941+0000", 2024-09-04T17:47:40.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.404e-06 2024-09-04T17:47:40.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:40.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.933065+0000", 2024-09-04T17:47:40.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012361199999999999 2024-09-04T17:47:40.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:40.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.933078+0000", 2024-09-04T17:47:40.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3182e-05 2024-09-04T17:47:40.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:40.220 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.933085+0000", 2024-09-04T17:47:40.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4500000000000001e-06 2024-09-04T17:47:40.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.221 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.221 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:40.221 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.933134+0000", 2024-09-04T17:47:40.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9150999999999997e-05 2024-09-04T17:47:40.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:40.222 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.933242+0000", 2024-09-04T17:47:40.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010781 2024-09-04T17:47:40.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-04T17:47:40.222 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.938358+0000", 2024-09-04T17:47:40.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051160349999999997 2024-09-04T17:47:40.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.223 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:40.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.975343+0000", 2024-09-04T17:47:40.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036985663000000002 2024-09-04T17:47:40.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:40.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.975392+0000", 2024-09-04T17:47:40.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9044e-05 2024-09-04T17:47:40.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:40.225 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.975548+0000", 2024-09-04T17:47:40.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015572900000000001 2024-09-04T17:47:40.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.225 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:40.225 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.975681+0000", 2024-09-04T17:47:40.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013243400000000001 2024-09-04T17:47:40.225 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.226 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:40.226 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.226 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:46649 3.7 3:e0749ed5:::rbd_data.1134326b938e.000000000000014f:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-04T17:47:40.226 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:54.114730+0000", 2024-09-04T17:47:40.226 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 46.072561657999998, 2024-09-04T17:47:40.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032972972000000003, 2024-09-04T17:47:40.227 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:40.227 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:40.227 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:40.227 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:40.227 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:40.227 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:40.227 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 46649 2024-09-04T17:47:40.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.228 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:40.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.228 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:40.228 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114730+0000", 2024-09-04T17:47:40.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.228 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:40.228 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114730+0000", 2024-09-04T17:47:40.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.229 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:40.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114731+0000", 2024-09-04T17:47:40.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.147e-06 2024-09-04T17:47:40.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.229 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:40.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114737+0000", 2024-09-04T17:47:40.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6380000000000001e-06 2024-09-04T17:47:40.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.229 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:40.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114751+0000", 2024-09-04T17:47:40.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4012999999999999e-05 2024-09-04T17:47:40.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:40.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114758+0000", 2024-09-04T17:47:40.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1539999999999996e-06 2024-09-04T17:47:40.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:40.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114876+0000", 2024-09-04T17:47:40.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000117421 2024-09-04T17:47:40.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:40.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.115173+0000", 2024-09-04T17:47:40.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00029750799999999998 2024-09-04T17:47:40.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 7", 2024-09-04T17:47:40.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.120273+0000", 2024-09-04T17:47:40.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051002900000000004 2024-09-04T17:47:40.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:40.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.147571+0000", 2024-09-04T17:47:40.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027297906 2024-09-04T17:47:40.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:40.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.147588+0000", 2024-09-04T17:47:40.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6807999999999999e-05 2024-09-04T17:47:40.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:40.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.147637+0000", 2024-09-04T17:47:40.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8466e-05 2024-09-04T17:47:40.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:40.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.147703+0000", 2024-09-04T17:47:40.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6620999999999998e-05 2024-09-04T17:47:40.233 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.233 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:40.233 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.233 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49801 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 720896~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:40.233 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.075094+0000", 2024-09-04T17:47:40.233 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.112197670999997, 2024-09-04T17:47:40.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017824974, 2024-09-04T17:47:40.234 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:40.234 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:40.234 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:40.234 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:40.234 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:40.234 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:40.234 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49801 2024-09-04T17:47:40.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.234 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:40.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:40.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075094+0000", 2024-09-04T17:47:40.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:40.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075094+0000", 2024-09-04T17:47:40.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:40.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075095+0000", 2024-09-04T17:47:40.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.017e-06 2024-09-04T17:47:40.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:40.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075101+0000", 2024-09-04T17:47:40.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5029999999999997e-06 2024-09-04T17:47:40.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:40.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075119+0000", 2024-09-04T17:47:40.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8133000000000001e-05 2024-09-04T17:47:40.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:40.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075132+0000", 2024-09-04T17:47:40.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3305999999999999e-05 2024-09-04T17:47:40.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:40.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.092677+0000", 2024-09-04T17:47:40.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017544466000000002 2024-09-04T17:47:40.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:40.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.092782+0000", 2024-09-04T17:47:40.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000105284 2024-09-04T17:47:40.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.239 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:40.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.092919+0000", 2024-09-04T17:47:40.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013726499999999999 2024-09-04T17:47:40.239 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.240 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:40.240 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.240 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49809 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 163840~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:40.240 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077146+0000", 2024-09-04T17:47:40.240 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.110146227999998, 2024-09-04T17:47:40.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017984541999999999, 2024-09-04T17:47:40.240 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:40.240 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:40.240 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:40.240 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:40.241 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:40.241 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:40.241 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49809 2024-09-04T17:47:40.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.241 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:40.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:40.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077146+0000", 2024-09-04T17:47:40.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:40.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077146+0000", 2024-09-04T17:47:40.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:40.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077147+0000", 2024-09-04T17:47:40.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0330000000000001e-06 2024-09-04T17:47:40.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:40.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077164+0000", 2024-09-04T17:47:40.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7810000000000001e-05 2024-09-04T17:47:40.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:40.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077187+0000", 2024-09-04T17:47:40.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2246e-05 2024-09-04T17:47:40.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:40.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077203+0000", 2024-09-04T17:47:40.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6392000000000002e-05 2024-09-04T17:47:40.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:40.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.094885+0000", 2024-09-04T17:47:40.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017681589000000001 2024-09-04T17:47:40.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.244 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:40.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.094991+0000", 2024-09-04T17:47:40.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010668899999999999 2024-09-04T17:47:40.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.244 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:40.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095130+0000", 2024-09-04T17:47:40.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013878299999999999 2024-09-04T17:47:40.244 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.244 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:40.244 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.245 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49810 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 229376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:40.245 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077225+0000", 2024-09-04T17:47:40.245 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.110066387000003, 2024-09-04T17:47:40.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018218793, 2024-09-04T17:47:40.245 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:40.245 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:40.245 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:40.245 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:40.245 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:40.245 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:40.245 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49810 2024-09-04T17:47:40.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.246 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:40.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:40.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077225+0000", 2024-09-04T17:47:40.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:40.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077225+0000", 2024-09-04T17:47:40.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:40.247 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077227+0000", 2024-09-04T17:47:40.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1829999999999999e-06 2024-09-04T17:47:40.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:40.247 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077233+0000", 2024-09-04T17:47:40.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3899999999999998e-06 2024-09-04T17:47:40.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:40.247 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077253+0000", 2024-09-04T17:47:40.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0009000000000001e-05 2024-09-04T17:47:40.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:40.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077270+0000", 2024-09-04T17:47:40.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7336000000000001e-05 2024-09-04T17:47:40.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:40.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095177+0000", 2024-09-04T17:47:40.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017907101000000002 2024-09-04T17:47:40.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:40.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095299+0000", 2024-09-04T17:47:40.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000121749 2024-09-04T17:47:40.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:40.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095444+0000", 2024-09-04T17:47:40.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014502499999999999 2024-09-04T17:47:40.249 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.249 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:40.249 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.250 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49811 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 294912~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:40.250 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077288+0000", 2024-09-04T17:47:40.250 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.110004072999999, 2024-09-04T17:47:40.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018443620000000001, 2024-09-04T17:47:40.250 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:40.250 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:40.250 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:40.250 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:40.250 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:40.250 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:40.250 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49811 2024-09-04T17:47:40.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.250 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:40.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:40.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077288+0000", 2024-09-04T17:47:40.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:40.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077288+0000", 2024-09-04T17:47:40.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:40.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077289+0000", 2024-09-04T17:47:40.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6199999999999999e-06 2024-09-04T17:47:40.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:40.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077295+0000", 2024-09-04T17:47:40.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1499999999999998e-06 2024-09-04T17:47:40.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:40.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077312+0000", 2024-09-04T17:47:40.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7353000000000001e-05 2024-09-04T17:47:40.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:40.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077324+0000", 2024-09-04T17:47:40.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2593e-05 2024-09-04T17:47:40.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:40.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095490+0000", 2024-09-04T17:47:40.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018165976 2024-09-04T17:47:40.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:40.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095597+0000", 2024-09-04T17:47:40.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010629900000000001 2024-09-04T17:47:40.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:40.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095731+0000", 2024-09-04T17:47:40.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013462899999999999 2024-09-04T17:47:40.255 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.255 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:40.255 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.255 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49813 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 360448~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:40.255 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077349+0000", 2024-09-04T17:47:40.255 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.109942871000001, 2024-09-04T17:47:40.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018780305000000001, 2024-09-04T17:47:40.255 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:40.256 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:40.256 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:40.256 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:40.256 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:40.256 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:40.256 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49813 2024-09-04T17:47:40.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.256 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:40.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:40.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077349+0000", 2024-09-04T17:47:40.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:40.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077349+0000", 2024-09-04T17:47:40.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:40.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077350+0000", 2024-09-04T17:47:40.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0330000000000001e-06 2024-09-04T17:47:40.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:40.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077355+0000", 2024-09-04T17:47:40.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0699999999999997e-06 2024-09-04T17:47:40.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:40.258 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077376+0000", 2024-09-04T17:47:40.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0846e-05 2024-09-04T17:47:40.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:40.258 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077388+0000", 2024-09-04T17:47:40.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2353e-05 2024-09-04T17:47:40.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:40.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095819+0000", 2024-09-04T17:47:40.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01843057 2024-09-04T17:47:40.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:40.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095960+0000", 2024-09-04T17:47:40.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000140965 2024-09-04T17:47:40.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:40.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096129+0000", 2024-09-04T17:47:40.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016946800000000001 2024-09-04T17:47:40.259 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.260 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:40.260 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.260 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49814 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 425984~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:40.260 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077407+0000", 2024-09-04T17:47:40.260 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.109884370000003, 2024-09-04T17:47:40.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019041563000000001, 2024-09-04T17:47:40.260 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:40.260 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:40.260 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:40.260 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:40.260 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:40.260 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:40.261 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49814 2024-09-04T17:47:40.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.261 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:40.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:40.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077407+0000", 2024-09-04T17:47:40.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:40.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077407+0000", 2024-09-04T17:47:40.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:40.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077409+0000", 2024-09-04T17:47:40.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2330000000000001e-06 2024-09-04T17:47:40.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:40.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077415+0000", 2024-09-04T17:47:40.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2729999999999999e-06 2024-09-04T17:47:40.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:40.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077436+0000", 2024-09-04T17:47:40.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1209999999999999e-05 2024-09-04T17:47:40.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:40.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077448+0000", 2024-09-04T17:47:40.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2225999999999999e-05 2024-09-04T17:47:40.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:40.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096183+0000", 2024-09-04T17:47:40.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018735004 2024-09-04T17:47:40.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:40.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096297+0000", 2024-09-04T17:47:40.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000113962 2024-09-04T17:47:40.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:40.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096449+0000", 2024-09-04T17:47:40.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000151655 2024-09-04T17:47:40.264 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.264 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:40.264 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.264 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49815 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 491520~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:40.265 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077471+0000", 2024-09-04T17:47:40.265 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.109821132, 2024-09-04T17:47:40.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019266536000000001, 2024-09-04T17:47:40.265 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:40.265 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:40.265 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:40.265 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:40.265 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:40.265 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:40.265 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49815 2024-09-04T17:47:40.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.265 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:40.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:40.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077471+0000", 2024-09-04T17:47:40.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:40.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077471+0000", 2024-09-04T17:47:40.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:40.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077472+0000", 2024-09-04T17:47:40.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.17e-06 2024-09-04T17:47:40.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.267 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:40.267 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077478+0000", 2024-09-04T17:47:40.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0399999999999998e-06 2024-09-04T17:47:40.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.267 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:40.267 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077500+0000", 2024-09-04T17:47:40.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1959000000000001e-05 2024-09-04T17:47:40.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.267 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:40.267 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077513+0000", 2024-09-04T17:47:40.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2823000000000001e-05 2024-09-04T17:47:40.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:40.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096493+0000", 2024-09-04T17:47:40.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018980701999999999 2024-09-04T17:47:40.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:40.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096603+0000", 2024-09-04T17:47:40.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010942000000000001 2024-09-04T17:47:40.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:40.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096737+0000", 2024-09-04T17:47:40.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013442200000000001 2024-09-04T17:47:40.269 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.269 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:40.269 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.269 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49816 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 557056~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:40.269 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077532+0000", 2024-09-04T17:47:40.269 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.109759390999997, 2024-09-04T17:47:40.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019604331999999999, 2024-09-04T17:47:40.269 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:40.270 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:40.270 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:40.270 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:40.270 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:40.270 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:40.270 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49816 2024-09-04T17:47:40.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.270 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:40.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:40.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077532+0000", 2024-09-04T17:47:40.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:40.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077532+0000", 2024-09-04T17:47:40.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:40.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077534+0000", 2024-09-04T17:47:40.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1200000000000001e-06 2024-09-04T17:47:40.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:40.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077545+0000", 2024-09-04T17:47:40.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1673e-05 2024-09-04T17:47:40.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:40.272 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077569+0000", 2024-09-04T17:47:40.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3555999999999999e-05 2024-09-04T17:47:40.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:40.272 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077584+0000", 2024-09-04T17:47:40.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4883e-05 2024-09-04T17:47:40.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:40.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096825+0000", 2024-09-04T17:47:40.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019241734 2024-09-04T17:47:40.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:40.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096964+0000", 2024-09-04T17:47:40.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000138085 2024-09-04T17:47:40.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:40.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097137+0000", 2024-09-04T17:47:40.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017328100000000001 2024-09-04T17:47:40.273 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.274 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:40.274 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.274 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49817 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 622592~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:40.274 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077603+0000", 2024-09-04T17:47:40.274 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.109688833, 2024-09-04T17:47:40.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01986928, 2024-09-04T17:47:40.274 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:40.274 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:40.274 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:40.274 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:40.274 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:40.275 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:40.275 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49817 2024-09-04T17:47:40.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.275 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:40.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:40.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077603+0000", 2024-09-04T17:47:40.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:40.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077603+0000", 2024-09-04T17:47:40.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:40.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077604+0000", 2024-09-04T17:47:40.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7699999999999992e-07 2024-09-04T17:47:40.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:40.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077609+0000", 2024-09-04T17:47:40.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0830000000000003e-06 2024-09-04T17:47:40.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:40.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077630+0000", 2024-09-04T17:47:40.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0438999999999999e-05 2024-09-04T17:47:40.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:40.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077664+0000", 2024-09-04T17:47:40.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4579000000000002e-05 2024-09-04T17:47:40.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:40.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097193+0000", 2024-09-04T17:47:40.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019529174 2024-09-04T17:47:40.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:40.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097323+0000", 2024-09-04T17:47:40.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000129696 2024-09-04T17:47:40.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:40.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097472+0000", 2024-09-04T17:47:40.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014933200000000001 2024-09-04T17:47:40.278 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.278 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:40.278 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.278 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49818 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 688128~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:40.279 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077683+0000", 2024-09-04T17:47:40.279 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.109608469000001, 2024-09-04T17:47:40.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020086333000000001, 2024-09-04T17:47:40.279 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:40.279 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:40.279 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:40.279 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:40.279 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:40.279 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:40.279 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49818 2024-09-04T17:47:40.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.279 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:40.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:40.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077683+0000", 2024-09-04T17:47:40.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:40.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077683+0000", 2024-09-04T17:47:40.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:40.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077684+0000", 2024-09-04T17:47:40.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4e-07 2024-09-04T17:47:40.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:40.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077690+0000", 2024-09-04T17:47:40.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4729999999999999e-06 2024-09-04T17:47:40.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:40.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077710+0000", 2024-09-04T17:47:40.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0149999999999999e-05 2024-09-04T17:47:40.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:40.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077723+0000", 2024-09-04T17:47:40.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3271999999999999e-05 2024-09-04T17:47:40.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:40.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097521+0000", 2024-09-04T17:47:40.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019797981999999999 2024-09-04T17:47:40.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:40.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097628+0000", 2024-09-04T17:47:40.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010665700000000001 2024-09-04T17:47:40.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:40.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097770+0000", 2024-09-04T17:47:40.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014185900000000001 2024-09-04T17:47:40.283 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.283 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:40.283 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.283 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49820 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 753664~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:40.283 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077744+0000", 2024-09-04T17:47:40.283 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.109547538000001, 2024-09-04T17:47:40.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020364786999999999, 2024-09-04T17:47:40.284 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:40.284 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:40.284 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:40.284 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:40.284 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:40.284 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:40.284 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49820 2024-09-04T17:47:40.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.284 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:40.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:40.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077744+0000", 2024-09-04T17:47:40.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:40.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077744+0000", 2024-09-04T17:47:40.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:40.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077745+0000", 2024-09-04T17:47:40.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0270000000000001e-06 2024-09-04T17:47:40.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:40.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077751+0000", 2024-09-04T17:47:40.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.626e-06 2024-09-04T17:47:40.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:40.286 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077771+0000", 2024-09-04T17:47:40.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0469999999999999e-05 2024-09-04T17:47:40.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:40.286 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077785+0000", 2024-09-04T17:47:40.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3129000000000001e-05 2024-09-04T17:47:40.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:40.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097816+0000", 2024-09-04T17:47:40.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020031917999999999 2024-09-04T17:47:40.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:40.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097939+0000", 2024-09-04T17:47:40.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000122533 2024-09-04T17:47:40.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:40.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098109+0000", 2024-09-04T17:47:40.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017008400000000001 2024-09-04T17:47:40.288 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.288 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:40.288 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.288 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49821 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 819200~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:40.288 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077807+0000", 2024-09-04T17:47:40.288 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.109485325999998, 2024-09-04T17:47:40.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020702928999999998, 2024-09-04T17:47:40.288 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:40.288 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:40.288 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:40.289 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:40.289 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:40.289 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:40.289 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49821 2024-09-04T17:47:40.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.289 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:40.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:40.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077807+0000", 2024-09-04T17:47:40.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:40.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077807+0000", 2024-09-04T17:47:40.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:40.290 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077808+0000", 2024-09-04T17:47:40.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.463e-06 2024-09-04T17:47:40.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:40.290 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077813+0000", 2024-09-04T17:47:40.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7530000000000001e-06 2024-09-04T17:47:40.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:40.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077832+0000", 2024-09-04T17:47:40.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.946e-05 2024-09-04T17:47:40.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:40.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077846+0000", 2024-09-04T17:47:40.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3786e-05 2024-09-04T17:47:40.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:40.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098201+0000", 2024-09-04T17:47:40.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020354797000000001 2024-09-04T17:47:40.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:40.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098341+0000", 2024-09-04T17:47:40.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000140042 2024-09-04T17:47:40.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:40.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098509+0000", 2024-09-04T17:47:40.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016862800000000001 2024-09-04T17:47:40.292 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.292 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:40.292 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.293 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49823 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 884736~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:40.293 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077865+0000", 2024-09-04T17:47:40.293 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.109426882000001, 2024-09-04T17:47:40.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021008723, 2024-09-04T17:47:40.293 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:40.293 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:40.293 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:40.293 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:40.293 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:40.293 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:40.293 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49823 2024-09-04T17:47:40.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.293 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:40.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:40.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077865+0000", 2024-09-04T17:47:40.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:40.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077865+0000", 2024-09-04T17:47:40.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:40.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077866+0000", 2024-09-04T17:47:40.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.164e-06 2024-09-04T17:47:40.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:40.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077872+0000", 2024-09-04T17:47:40.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3959999999999997e-06 2024-09-04T17:47:40.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:40.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077892+0000", 2024-09-04T17:47:40.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0970000000000001e-05 2024-09-04T17:47:40.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.296 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:40.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077905+0000", 2024-09-04T17:47:40.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2312e-05 2024-09-04T17:47:40.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.296 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:40.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098579+0000", 2024-09-04T17:47:40.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020674486999999998 2024-09-04T17:47:40.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.296 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:40.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098705+0000", 2024-09-04T17:47:40.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012541900000000001 2024-09-04T17:47:40.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:40.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098874+0000", 2024-09-04T17:47:40.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000168975 2024-09-04T17:47:40.297 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.297 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:40.297 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.297 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49824 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 950272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:40.297 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077923+0000", 2024-09-04T17:47:40.297 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.109369190000002, 2024-09-04T17:47:40.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021246287999999999, 2024-09-04T17:47:40.298 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:40.298 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:40.298 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:40.298 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:40.298 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:40.298 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:40.298 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49824 2024-09-04T17:47:40.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.298 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:40.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:40.298 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077923+0000", 2024-09-04T17:47:40.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:40.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077923+0000", 2024-09-04T17:47:40.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:40.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077924+0000", 2024-09-04T17:47:40.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0100000000000001e-06 2024-09-04T17:47:40.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:40.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077944+0000", 2024-09-04T17:47:40.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0449e-05 2024-09-04T17:47:40.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.300 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:40.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077965+0000", 2024-09-04T17:47:40.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0479000000000002e-05 2024-09-04T17:47:40.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.300 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:40.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077990+0000", 2024-09-04T17:47:40.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5582999999999999e-05 2024-09-04T17:47:40.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:40.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098924+0000", 2024-09-04T17:47:40.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020933395 2024-09-04T17:47:40.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:40.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.099032+0000", 2024-09-04T17:47:40.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010846299999999999 2024-09-04T17:47:40.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:40.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.099169+0000", 2024-09-04T17:47:40.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000136909 2024-09-04T17:47:40.302 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.302 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:40.302 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.302 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49826 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 1015808~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:40.302 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.078010+0000", 2024-09-04T17:47:40.302 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.109281582999998, 2024-09-04T17:47:40.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021584564000000001, 2024-09-04T17:47:40.302 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:40.302 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:40.302 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:40.302 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:40.303 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:40.303 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:40.303 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49826 2024-09-04T17:47:40.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.303 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:40.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.303 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:40.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078010+0000", 2024-09-04T17:47:40.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.303 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:40.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078010+0000", 2024-09-04T17:47:40.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:40.304 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078011+0000", 2024-09-04T17:47:40.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1340000000000001e-06 2024-09-04T17:47:40.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:40.304 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078017+0000", 2024-09-04T17:47:40.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.682e-06 2024-09-04T17:47:40.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:40.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078036+0000", 2024-09-04T17:47:40.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9357000000000001e-05 2024-09-04T17:47:40.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:40.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078050+0000", 2024-09-04T17:47:40.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3122000000000001e-05 2024-09-04T17:47:40.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:40.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.099271+0000", 2024-09-04T17:47:40.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021221772999999999 2024-09-04T17:47:40.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:40.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.099410+0000", 2024-09-04T17:47:40.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000138415 2024-09-04T17:47:40.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:40.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.099595+0000", 2024-09-04T17:47:40.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000185081 2024-09-04T17:47:40.306 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.306 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:40.306 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.306 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.306 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:40.307 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:40.308 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-04T17:47:40.308 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:40.303+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:40.409 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:47:40.407+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:47:40.432 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:40.432 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:47:40.432 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:47:40.432 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:40.446 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-04T17:47:40.510 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:40.509+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:40.571 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:40.572 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:47:40.572 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:47:40.572 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:47:40.572 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:40.586 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-04T17:47:40.611 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:47:40.607+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:47:40.712 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:40.707+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:40.754 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:40.754 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:47:40.754 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:47:40.754 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:47:40.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.755 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:48444 3.12 3:4ff80fb5:::rbd_data.11346a2b0bc9.000000000000001d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-04T17:47:40.755 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:58.122601+0000", 2024-09-04T17:47:40.755 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.628659177999999, 2024-09-04T17:47:40.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057065713999999997, 2024-09-04T17:47:40.755 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:40.756 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:40.756 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:40.756 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:40.756 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:40.756 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:40.756 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 48444 2024-09-04T17:47:40.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.756 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:40.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.757 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:40.757 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122601+0000", 2024-09-04T17:47:40.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.757 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.757 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:40.757 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122601+0000", 2024-09-04T17:47:40.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.758 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:40.758 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122603+0000", 2024-09-04T17:47:40.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3499999999999999e-06 2024-09-04T17:47:40.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.758 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:40.758 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122614+0000", 2024-09-04T17:47:40.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.063e-05 2024-09-04T17:47:40.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:40.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122643+0000", 2024-09-04T17:47:40.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9434999999999999e-05 2024-09-04T17:47:40.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:40.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122657+0000", 2024-09-04T17:47:40.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3920000000000001e-05 2024-09-04T17:47:40.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:40.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122767+0000", 2024-09-04T17:47:40.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010981599999999999 2024-09-04T17:47:40.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:40.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122989+0000", 2024-09-04T17:47:40.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022158200000000001 2024-09-04T17:47:40.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 3", 2024-09-04T17:47:40.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.133349+0000", 2024-09-04T17:47:40.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010360729000000001 2024-09-04T17:47:40.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:40.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.179423+0000", 2024-09-04T17:47:40.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046073408000000003 2024-09-04T17:47:40.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:40.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.179453+0000", 2024-09-04T17:47:40.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0229000000000001e-05 2024-09-04T17:47:40.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:40.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.179545+0000", 2024-09-04T17:47:40.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1580000000000001e-05 2024-09-04T17:47:40.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.762 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:40.762 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.179667+0000", 2024-09-04T17:47:40.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012203500000000001 2024-09-04T17:47:40.762 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.762 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:40.762 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.762 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:55376 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-04T17:47:40.762 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:11.974053+0000", 2024-09-04T17:47:40.762 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.777207394000001, 2024-09-04T17:47:40.763 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060514722, 2024-09-04T17:47:40.763 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:40.763 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:40.763 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:40.763 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:40.763 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:40.763 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:40.763 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 55376 2024-09-04T17:47:40.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.763 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:40.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.763 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:40.763 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974053+0000", 2024-09-04T17:47:40.763 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:40.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974053+0000", 2024-09-04T17:47:40.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:40.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974054+0000", 2024-09-04T17:47:40.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.841e-06 2024-09-04T17:47:40.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:40.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974062+0000", 2024-09-04T17:47:40.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5460000000000003e-06 2024-09-04T17:47:40.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:40.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974077+0000", 2024-09-04T17:47:40.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5389000000000001e-05 2024-09-04T17:47:40.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:40.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974085+0000", 2024-09-04T17:47:40.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7659999999999998e-06 2024-09-04T17:47:40.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.766 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:40.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974188+0000", 2024-09-04T17:47:40.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000102521 2024-09-04T17:47:40.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.766 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:40.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974321+0000", 2024-09-04T17:47:40.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013324699999999999 2024-09-04T17:47:40.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.767 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-04T17:47:40.767 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.986481+0000", 2024-09-04T17:47:40.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012160450999999999 2024-09-04T17:47:40.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.767 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:40.767 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:12.034415+0000", 2024-09-04T17:47:40.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.047934065999999997 2024-09-04T17:47:40.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.767 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:40.767 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:12.034438+0000", 2024-09-04T17:47:40.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2923999999999999e-05 2024-09-04T17:47:40.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.768 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:40.768 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:12.034505+0000", 2024-09-04T17:47:40.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6711000000000006e-05 2024-09-04T17:47:40.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.768 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:40.768 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:12.034567+0000", 2024-09-04T17:47:40.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2260000000000004e-05 2024-09-04T17:47:40.768 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.768 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:40.769 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.769 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61128 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 1458176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:40.769 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819489+0000", 2024-09-04T17:47:40.769 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.931771058999999, 2024-09-04T17:47:40.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.038380478000000003, 2024-09-04T17:47:40.769 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:40.769 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:40.770 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:40.770 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:40.770 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:40.770 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:40.770 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61128 2024-09-04T17:47:40.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.770 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:40.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:40.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819489+0000", 2024-09-04T17:47:40.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:40.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819489+0000", 2024-09-04T17:47:40.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:40.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819490+0000", 2024-09-04T17:47:40.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6599999999999996e-07 2024-09-04T17:47:40.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:40.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819493+0000", 2024-09-04T17:47:40.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2279999999999998e-06 2024-09-04T17:47:40.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.772 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:40.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819501+0000", 2024-09-04T17:47:40.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4039999999999995e-06 2024-09-04T17:47:40.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.772 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:40.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819508+0000", 2024-09-04T17:47:40.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2400000000000004e-06 2024-09-04T17:47:40.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.773 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:40.773 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.857534+0000", 2024-09-04T17:47:40.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.038026272 2024-09-04T17:47:40.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.773 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:40.773 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.857629+0000", 2024-09-04T17:47:40.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5397999999999997e-05 2024-09-04T17:47:40.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:40.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.857869+0000", 2024-09-04T17:47:40.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024027 2024-09-04T17:47:40.774 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.774 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:40.774 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.774 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61129 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 1564672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:40.775 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819517+0000", 2024-09-04T17:47:40.775 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.931743237999999, 2024-09-04T17:47:40.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.038772745999999997, 2024-09-04T17:47:40.775 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:40.775 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:40.775 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:40.775 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:40.775 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:40.775 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:40.775 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61129 2024-09-04T17:47:40.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.776 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:40.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:40.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819517+0000", 2024-09-04T17:47:40.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:40.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819517+0000", 2024-09-04T17:47:40.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.777 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:40.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819517+0000", 2024-09-04T17:47:40.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5999999999999999e-07 2024-09-04T17:47:40.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:40.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819520+0000", 2024-09-04T17:47:40.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2709999999999999e-06 2024-09-04T17:47:40.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:40.779 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819526+0000", 2024-09-04T17:47:40.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7059999999999998e-06 2024-09-04T17:47:40.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:40.779 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819531+0000", 2024-09-04T17:47:40.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0549999999999999e-06 2024-09-04T17:47:40.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:40.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.857913+0000", 2024-09-04T17:47:40.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.038381954000000003 2024-09-04T17:47:40.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.780 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:40.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.858000+0000", 2024-09-04T17:47:40.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6963000000000001e-05 2024-09-04T17:47:40.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:40.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.858290+0000", 2024-09-04T17:47:40.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028933700000000003 2024-09-04T17:47:40.782 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.782 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:40.782 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.782 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61130 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 1597440~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:40.782 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819539+0000", 2024-09-04T17:47:40.782 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.931721411000002, 2024-09-04T17:47:40.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.039166595999999998, 2024-09-04T17:47:40.783 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:40.783 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:40.783 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:40.783 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:40.783 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:40.783 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:40.783 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61130 2024-09-04T17:47:40.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.783 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:40.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:40.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819539+0000", 2024-09-04T17:47:40.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:40.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819539+0000", 2024-09-04T17:47:40.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:40.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819539+0000", 2024-09-04T17:47:40.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9400000000000005e-07 2024-09-04T17:47:40.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.785 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:40.785 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819542+0000", 2024-09-04T17:47:40.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2630000000000002e-06 2024-09-04T17:47:40.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.785 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:40.785 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819548+0000", 2024-09-04T17:47:40.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3269999999999997e-06 2024-09-04T17:47:40.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:40.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819553+0000", 2024-09-04T17:47:40.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9849999999999997e-06 2024-09-04T17:47:40.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:40.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.858321+0000", 2024-09-04T17:47:40.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.038768395999999997 2024-09-04T17:47:40.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:40.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.858406+0000", 2024-09-04T17:47:40.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4456999999999995e-05 2024-09-04T17:47:40.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:40.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.858705+0000", 2024-09-04T17:47:40.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00029947399999999999 2024-09-04T17:47:40.788 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.788 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:40.788 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.788 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61131 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 1703936~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:40.788 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819560+0000", 2024-09-04T17:47:40.788 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.931700059000001, 2024-09-04T17:47:40.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.039522082, 2024-09-04T17:47:40.788 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:40.788 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:40.788 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:40.788 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:40.789 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:40.789 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:40.789 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61131 2024-09-04T17:47:40.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.789 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:40.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:40.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819560+0000", 2024-09-04T17:47:40.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:40.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819560+0000", 2024-09-04T17:47:40.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.790 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:40.790 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819561+0000", 2024-09-04T17:47:40.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5300000000000004e-07 2024-09-04T17:47:40.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.790 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:40.790 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819563+0000", 2024-09-04T17:47:40.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.187e-06 2024-09-04T17:47:40.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:40.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819569+0000", 2024-09-04T17:47:40.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6170000000000003e-06 2024-09-04T17:47:40.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:40.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819574+0000", 2024-09-04T17:47:40.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9640000000000002e-06 2024-09-04T17:47:40.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:40.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.858754+0000", 2024-09-04T17:47:40.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.039179295000000003 2024-09-04T17:47:40.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:40.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.858800+0000", 2024-09-04T17:47:40.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.587e-05 2024-09-04T17:47:40.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:40.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.859082+0000", 2024-09-04T17:47:40.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028249599999999997 2024-09-04T17:47:40.792 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.792 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:40.793 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.793 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61132 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 1810432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:40.793 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819581+0000", 2024-09-04T17:47:40.793 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.931678616999999, 2024-09-04T17:47:40.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.039953120000000002, 2024-09-04T17:47:40.793 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:40.793 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:40.793 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:40.793 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:40.794 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:40.794 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:40.794 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61132 2024-09-04T17:47:40.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.794 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:40.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:40.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819581+0000", 2024-09-04T17:47:40.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:40.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819581+0000", 2024-09-04T17:47:40.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:40.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819582+0000", 2024-09-04T17:47:40.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3399999999999999e-07 2024-09-04T17:47:40.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:40.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819584+0000", 2024-09-04T17:47:40.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1320000000000001e-06 2024-09-04T17:47:40.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:40.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819591+0000", 2024-09-04T17:47:40.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3759999999999996e-06 2024-09-04T17:47:40.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:40.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819595+0000", 2024-09-04T17:47:40.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7339999999999999e-06 2024-09-04T17:47:40.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:40.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.859119+0000", 2024-09-04T17:47:40.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.039523677 2024-09-04T17:47:40.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:40.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.859232+0000", 2024-09-04T17:47:40.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000113001 2024-09-04T17:47:40.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:40.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.859535+0000", 2024-09-04T17:47:40.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030256599999999999 2024-09-04T17:47:40.798 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.798 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:40.798 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.799 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61133 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 1843200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:40.799 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819602+0000", 2024-09-04T17:47:40.799 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.931658308999999, 2024-09-04T17:47:40.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.040226171999999998, 2024-09-04T17:47:40.799 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:40.799 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:40.799 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:40.799 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:40.799 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:40.799 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:40.799 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61133 2024-09-04T17:47:40.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.800 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:40.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:40.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819602+0000", 2024-09-04T17:47:40.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:40.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819602+0000", 2024-09-04T17:47:40.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:40.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819602+0000", 2024-09-04T17:47:40.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.75e-07 2024-09-04T17:47:40.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:40.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819605+0000", 2024-09-04T17:47:40.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.52e-06 2024-09-04T17:47:40.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:40.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819611+0000", 2024-09-04T17:47:40.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5200000000000003e-06 2024-09-04T17:47:40.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:40.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819616+0000", 2024-09-04T17:47:40.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0939999999999998e-06 2024-09-04T17:47:40.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:40.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.859538+0000", 2024-09-04T17:47:40.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.039921660999999997 2024-09-04T17:47:40.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:40.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.859594+0000", 2024-09-04T17:47:40.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5671999999999998e-05 2024-09-04T17:47:40.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:40.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.859828+0000", 2024-09-04T17:47:40.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023423000000000001 2024-09-04T17:47:40.804 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.804 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:40.804 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.804 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61134 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 1949696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:40.804 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819623+0000", 2024-09-04T17:47:40.804 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.931636831999999, 2024-09-04T17:47:40.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.040606862000000001, 2024-09-04T17:47:40.804 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:40.804 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:40.805 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:40.805 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:40.805 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:40.805 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:40.805 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61134 2024-09-04T17:47:40.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.805 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:40.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:40.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819623+0000", 2024-09-04T17:47:40.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:40.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819623+0000", 2024-09-04T17:47:40.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:40.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819624+0000", 2024-09-04T17:47:40.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2100000000000002e-07 2024-09-04T17:47:40.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:40.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819626+0000", 2024-09-04T17:47:40.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2460000000000002e-06 2024-09-04T17:47:40.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:40.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819632+0000", 2024-09-04T17:47:40.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1659999999999999e-06 2024-09-04T17:47:40.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:40.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819637+0000", 2024-09-04T17:47:40.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2680000000000002e-06 2024-09-04T17:47:40.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:40.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.859872+0000", 2024-09-04T17:47:40.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.040234377000000002 2024-09-04T17:47:40.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:40.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.859950+0000", 2024-09-04T17:47:40.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8128999999999997e-05 2024-09-04T17:47:40.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:40.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.860230+0000", 2024-09-04T17:47:40.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028025499999999998 2024-09-04T17:47:40.808 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.808 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:40.809 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.809 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61135 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 2056192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:40.809 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819644+0000", 2024-09-04T17:47:40.809 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.931615791999999, 2024-09-04T17:47:40.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.040925930999999999, 2024-09-04T17:47:40.809 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:40.809 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:40.809 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:40.809 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:40.809 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:40.809 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:40.809 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61135 2024-09-04T17:47:40.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.810 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:40.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:40.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819644+0000", 2024-09-04T17:47:40.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:40.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819644+0000", 2024-09-04T17:47:40.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:40.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819645+0000", 2024-09-04T17:47:40.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9399999999999995e-07 2024-09-04T17:47:40.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:40.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819647+0000", 2024-09-04T17:47:40.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.311e-06 2024-09-04T17:47:40.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:40.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819653+0000", 2024-09-04T17:47:40.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3940000000000001e-06 2024-09-04T17:47:40.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:40.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819658+0000", 2024-09-04T17:47:40.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8420000000000004e-06 2024-09-04T17:47:40.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:40.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.860263+0000", 2024-09-04T17:47:40.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.040605109 2024-09-04T17:47:40.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:40.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.860339+0000", 2024-09-04T17:47:40.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5612999999999996e-05 2024-09-04T17:47:40.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:40.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.860570+0000", 2024-09-04T17:47:40.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023116799999999999 2024-09-04T17:47:40.815 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.815 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:40.815 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.815 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61136 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 2195456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:40.815 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819665+0000", 2024-09-04T17:47:40.815 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.931595338000001, 2024-09-04T17:47:40.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.041275408999999999, 2024-09-04T17:47:40.815 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:40.815 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:40.816 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:40.816 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:40.816 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:40.816 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:40.816 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61136 2024-09-04T17:47:40.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.816 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:40.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:40.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819665+0000", 2024-09-04T17:47:40.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:40.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819665+0000", 2024-09-04T17:47:40.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:40.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819665+0000", 2024-09-04T17:47:40.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8599999999999999e-07 2024-09-04T17:47:40.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:40.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819667+0000", 2024-09-04T17:47:40.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9439999999999999e-06 2024-09-04T17:47:40.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:40.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819673+0000", 2024-09-04T17:47:40.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0220000000000001e-06 2024-09-04T17:47:40.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:40.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819677+0000", 2024-09-04T17:47:40.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4139999999999996e-06 2024-09-04T17:47:40.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:40.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.860603+0000", 2024-09-04T17:47:40.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.040925504000000001 2024-09-04T17:47:40.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:40.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.860673+0000", 2024-09-04T17:47:40.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9707000000000002e-05 2024-09-04T17:47:40.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:40.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.860940+0000", 2024-09-04T17:47:40.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026743200000000001 2024-09-04T17:47:40.819 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.819 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:40.820 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.820 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61137 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 2301952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:40.820 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819685+0000", 2024-09-04T17:47:40.820 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.931574545, 2024-09-04T17:47:40.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.041579208999999999, 2024-09-04T17:47:40.820 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:40.820 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:40.820 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:40.820 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:40.820 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:40.820 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:40.821 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61137 2024-09-04T17:47:40.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.821 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:40.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:40.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819685+0000", 2024-09-04T17:47:40.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:40.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819685+0000", 2024-09-04T17:47:40.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:40.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819686+0000", 2024-09-04T17:47:40.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3000000000000001e-07 2024-09-04T17:47:40.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:40.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819688+0000", 2024-09-04T17:47:40.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1150000000000001e-06 2024-09-04T17:47:40.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:40.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819694+0000", 2024-09-04T17:47:40.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.75e-06 2024-09-04T17:47:40.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:40.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819699+0000", 2024-09-04T17:47:40.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8110000000000002e-06 2024-09-04T17:47:40.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:40.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.860977+0000", 2024-09-04T17:47:40.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.041277945000000003 2024-09-04T17:47:40.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:40.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.861071+0000", 2024-09-04T17:47:40.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4252000000000005e-05 2024-09-04T17:47:40.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:40.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.861265+0000", 2024-09-04T17:47:40.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019390600000000001 2024-09-04T17:47:40.824 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.824 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:40.824 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.825 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61138 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 2441216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:40.825 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819706+0000", 2024-09-04T17:47:40.825 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.931554144, 2024-09-04T17:47:40.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.041876780000000002, 2024-09-04T17:47:40.825 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:40.825 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:40.825 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:40.825 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:40.825 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:40.825 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:40.825 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61138 2024-09-04T17:47:40.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.826 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:40.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:40.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819706+0000", 2024-09-04T17:47:40.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:40.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819706+0000", 2024-09-04T17:47:40.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:40.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819707+0000", 2024-09-04T17:47:40.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7299999999999995e-07 2024-09-04T17:47:40.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:40.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819727+0000", 2024-09-04T17:47:40.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0610000000000001e-05 2024-09-04T17:47:40.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:40.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819735+0000", 2024-09-04T17:47:40.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.123e-06 2024-09-04T17:47:40.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:40.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819741+0000", 2024-09-04T17:47:40.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4009999999999996e-06 2024-09-04T17:47:40.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:40.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.861298+0000", 2024-09-04T17:47:40.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.041557135000000002 2024-09-04T17:47:40.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:40.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.861369+0000", 2024-09-04T17:47:40.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0794e-05 2024-09-04T17:47:40.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:40.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.861583+0000", 2024-09-04T17:47:40.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021404400000000001 2024-09-04T17:47:40.829 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.829 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:40.830 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.830 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61139 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 2547712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:40.830 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819750+0000", 2024-09-04T17:47:40.830 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.931509680000001, 2024-09-04T17:47:40.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042185699, 2024-09-04T17:47:40.830 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:40.830 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:40.830 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:40.830 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:40.830 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:40.831 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:40.831 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61139 2024-09-04T17:47:40.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.831 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:40.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:40.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819750+0000", 2024-09-04T17:47:40.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:40.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819750+0000", 2024-09-04T17:47:40.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:40.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819751+0000", 2024-09-04T17:47:40.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4000000000000002e-07 2024-09-04T17:47:40.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:40.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819753+0000", 2024-09-04T17:47:40.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5179999999999999e-06 2024-09-04T17:47:40.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:40.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819760+0000", 2024-09-04T17:47:40.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5440000000000004e-06 2024-09-04T17:47:40.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:40.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819765+0000", 2024-09-04T17:47:40.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8760000000000004e-06 2024-09-04T17:47:40.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:40.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.861622+0000", 2024-09-04T17:47:40.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.041857093999999997 2024-09-04T17:47:40.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:40.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.861698+0000", 2024-09-04T17:47:40.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6080000000000003e-05 2024-09-04T17:47:40.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:40.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.861936+0000", 2024-09-04T17:47:40.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023804699999999999 2024-09-04T17:47:40.835 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.835 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:40.835 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.835 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61140 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 2686976~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:40.835 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819772+0000", 2024-09-04T17:47:40.835 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.931487546, 2024-09-04T17:47:40.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042515052999999997, 2024-09-04T17:47:40.835 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:40.835 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:40.835 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:40.835 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:40.836 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:40.836 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:40.836 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61140 2024-09-04T17:47:40.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.836 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:40.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:40.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819772+0000", 2024-09-04T17:47:40.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:40.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819772+0000", 2024-09-04T17:47:40.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.837 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:40.837 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819773+0000", 2024-09-04T17:47:40.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2900000000000003e-07 2024-09-04T17:47:40.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.837 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:40.837 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819775+0000", 2024-09-04T17:47:40.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3309999999999998e-06 2024-09-04T17:47:40.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.837 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:40.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819781+0000", 2024-09-04T17:47:40.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.981e-06 2024-09-04T17:47:40.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:40.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819786+0000", 2024-09-04T17:47:40.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5719999999999996e-06 2024-09-04T17:47:40.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:40.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.861966+0000", 2024-09-04T17:47:40.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042179541000000001 2024-09-04T17:47:40.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:40.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862038+0000", 2024-09-04T17:47:40.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2719000000000006e-05 2024-09-04T17:47:40.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:40.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862288+0000", 2024-09-04T17:47:40.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024928 2024-09-04T17:47:40.839 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.839 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:40.839 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.840 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61141 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 2793472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:40.840 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819909+0000", 2024-09-04T17:47:40.840 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.931351402000001, 2024-09-04T17:47:40.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042786419999999999, 2024-09-04T17:47:40.840 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:40.840 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:40.840 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:40.840 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:40.840 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:40.840 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:40.840 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61141 2024-09-04T17:47:40.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.841 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:40.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:40.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819909+0000", 2024-09-04T17:47:40.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:40.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819909+0000", 2024-09-04T17:47:40.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:40.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819909+0000", 2024-09-04T17:47:40.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6300000000000005e-07 2024-09-04T17:47:40.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:40.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819913+0000", 2024-09-04T17:47:40.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4189999999999998e-06 2024-09-04T17:47:40.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:40.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819920+0000", 2024-09-04T17:47:40.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8619999999999997e-06 2024-09-04T17:47:40.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:40.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819926+0000", 2024-09-04T17:47:40.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9399999999999999e-06 2024-09-04T17:47:40.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:40.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862331+0000", 2024-09-04T17:47:40.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042405348000000002 2024-09-04T17:47:40.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:40.844 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862410+0000", 2024-09-04T17:47:40.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9078000000000004e-05 2024-09-04T17:47:40.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:40.844 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862695+0000", 2024-09-04T17:47:40.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028510999999999999 2024-09-04T17:47:40.844 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.844 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:40.844 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.845 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61142 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 2932736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:40.845 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819934+0000", 2024-09-04T17:47:40.845 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.931325783999998, 2024-09-04T17:47:40.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043101985000000002, 2024-09-04T17:47:40.845 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:40.845 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:40.845 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:40.845 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:40.845 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:40.845 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:40.845 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61142 2024-09-04T17:47:40.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.845 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:40.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:40.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819934+0000", 2024-09-04T17:47:40.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:40.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819934+0000", 2024-09-04T17:47:40.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:40.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819935+0000", 2024-09-04T17:47:40.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2024-09-04T17:47:40.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:40.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819937+0000", 2024-09-04T17:47:40.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5770000000000001e-06 2024-09-04T17:47:40.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:40.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819943+0000", 2024-09-04T17:47:40.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7950000000000004e-06 2024-09-04T17:47:40.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.848 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:40.848 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819948+0000", 2024-09-04T17:47:40.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6779999999999999e-06 2024-09-04T17:47:40.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.848 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:40.848 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862739+0000", 2024-09-04T17:47:40.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042791454999999999 2024-09-04T17:47:40.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.848 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:40.848 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862824+0000", 2024-09-04T17:47:40.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4790999999999995e-05 2024-09-04T17:47:40.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:40.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.863036+0000", 2024-09-04T17:47:40.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021224899999999999 2024-09-04T17:47:40.849 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.849 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:40.849 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.849 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61143 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 3039232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:40.849 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819954+0000", 2024-09-04T17:47:40.849 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 26.931305666, 2024-09-04T17:47:40.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043485134000000002, 2024-09-04T17:47:40.850 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:40.850 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:40.850 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:40.850 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:40.850 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:40.850 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:40.851 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61143 2024-09-04T17:47:40.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.851 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:40.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:40.851 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819954+0000", 2024-09-04T17:47:40.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:40.851 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819954+0000", 2024-09-04T17:47:40.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:40.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819955+0000", 2024-09-04T17:47:40.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7300000000000002e-07 2024-09-04T17:47:40.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:40.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819957+0000", 2024-09-04T17:47:40.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.024e-06 2024-09-04T17:47:40.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:40.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819963+0000", 2024-09-04T17:47:40.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8109999999999999e-06 2024-09-04T17:47:40.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:40.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819967+0000", 2024-09-04T17:47:40.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6149999999999999e-06 2024-09-04T17:47:40.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:40.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.863082+0000", 2024-09-04T17:47:40.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043115078000000001 2024-09-04T17:47:40.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.854 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:40.854 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.863172+0000", 2024-09-04T17:47:40.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0181999999999998e-05 2024-09-04T17:47:40.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.854 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:40.854 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.863440+0000", 2024-09-04T17:47:40.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026705099999999997 2024-09-04T17:47:40.854 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.854 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:40.854 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.854 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:70015 3.6 3:6012e780:::rbd_data.11341aa245a3.000000000000005b:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-04T17:47:40.855 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:26.155497+0000", 2024-09-04T17:47:40.855 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.595762608999999, 2024-09-04T17:47:40.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.049007124999999999, 2024-09-04T17:47:40.855 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:40.855 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:40.855 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:40.855 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:40.855 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:40.855 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:40.855 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 70015 2024-09-04T17:47:40.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.855 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:40.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:40.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155497+0000", 2024-09-04T17:47:40.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:40.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155497+0000", 2024-09-04T17:47:40.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:40.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155500+0000", 2024-09-04T17:47:40.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7070000000000001e-06 2024-09-04T17:47:40.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.857 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:40.857 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155513+0000", 2024-09-04T17:47:40.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2989e-05 2024-09-04T17:47:40.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.857 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:40.857 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155544+0000", 2024-09-04T17:47:40.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0747000000000002e-05 2024-09-04T17:47:40.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.857 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:40.858 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155560+0000", 2024-09-04T17:47:40.858 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5866e-05 2024-09-04T17:47:40.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.858 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.858 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:40.858 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155675+0000", 2024-09-04T17:47:40.858 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000114838 2024-09-04T17:47:40.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.858 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.858 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:40.858 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155803+0000", 2024-09-04T17:47:40.859 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012851799999999999 2024-09-04T17:47:40.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.859 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.859 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 1", 2024-09-04T17:47:40.859 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.163582+0000", 2024-09-04T17:47:40.859 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0077790960000000001 2024-09-04T17:47:40.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.859 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.859 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:40.859 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.204055+0000", 2024-09-04T17:47:40.859 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.040472517 2024-09-04T17:47:40.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.859 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.860 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:40.860 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.204195+0000", 2024-09-04T17:47:40.860 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014055200000000001 2024-09-04T17:47:40.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.860 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:40.860 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.204373+0000", 2024-09-04T17:47:40.860 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000177784 2024-09-04T17:47:40.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.860 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:40.860 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.204505+0000", 2024-09-04T17:47:40.860 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013151100000000001 2024-09-04T17:47:40.861 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.861 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:40.861 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.861 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.861 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.861 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:70434 3.12 3:4cd2342b:::rbd_data.11341aa245a3.0000000000000091:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-04T17:47:40.861 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:26.943124+0000", 2024-09-04T17:47:40.861 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.808135826999999, 2024-09-04T17:47:40.861 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055011831999999997, 2024-09-04T17:47:40.861 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:40.861 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:40.861 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:40.861 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:40.861 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:40.862 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:40.862 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 70434 2024-09-04T17:47:40.862 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.862 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:40.862 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.862 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:40.862 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943124+0000", 2024-09-04T17:47:40.862 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.862 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.862 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.862 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:40.862 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943124+0000", 2024-09-04T17:47:40.862 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:40.862 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.863 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:40.863 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943128+0000", 2024-09-04T17:47:40.863 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8460000000000001e-06 2024-09-04T17:47:40.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.863 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:40.863 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943142+0000", 2024-09-04T17:47:40.863 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4089e-05 2024-09-04T17:47:40.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.863 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:40.864 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943177+0000", 2024-09-04T17:47:40.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5167999999999999e-05 2024-09-04T17:47:40.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.864 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:40.864 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943194+0000", 2024-09-04T17:47:40.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6235e-05 2024-09-04T17:47:40.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.864 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:40.864 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943286+0000", 2024-09-04T17:47:40.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2793000000000004e-05 2024-09-04T17:47:40.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.865 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.865 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:40.865 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943526+0000", 2024-09-04T17:47:40.865 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023950999999999999 2024-09-04T17:47:40.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.865 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.865 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 1", 2024-09-04T17:47:40.865 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.952871+0000", 2024-09-04T17:47:40.865 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0093453070000000006 2024-09-04T17:47:40.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.865 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.865 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:40.866 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.997968+0000", 2024-09-04T17:47:40.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045096613000000001 2024-09-04T17:47:40.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.866 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.866 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:40.866 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.998006+0000", 2024-09-04T17:47:40.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8257000000000003e-05 2024-09-04T17:47:40.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.868 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.868 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:40.868 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.998073+0000", 2024-09-04T17:47:40.868 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6803000000000004e-05 2024-09-04T17:47:40.868 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:40.868 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:40.868 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:40.868 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.998136+0000", 2024-09-04T17:47:40.868 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3211000000000002e-05 2024-09-04T17:47:40.868 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.869 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:40.869 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.869 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:40.869 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:40.869 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:40.871 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-04T17:47:40.871 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:47:40.867+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:47:40.893 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is unset 2024-09-04T17:47:40.971 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:40.969+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:41.006 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:41.006 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:47:41.006 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:47:41.006 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:41.020 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-04T17:47:41.109 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:41.109+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:41.174 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:41.173+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:41.183 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:41.183 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:47:41.183 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:47:41.183 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:47:41.183 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:41.197 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-04T17:47:41.201 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-04T17:47:41.286 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:41.285+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:41.366 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:41.366 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:47:41.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:47:41.366 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:47:41.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.366 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59950 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5365760~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:47:41.367 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654917+0000", 2024-09-04T17:47:41.367 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.710429933, 2024-09-04T17:47:41.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066822450000000005, 2024-09-04T17:47:41.367 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:41.367 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:41.367 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:41.367 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:41.367 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:41.367 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:41.367 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59950 2024-09-04T17:47:41.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.368 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:41.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:41.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654917+0000", 2024-09-04T17:47:41.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:41.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:41.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654917+0000", 2024-09-04T17:47:41.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:41.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:41.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654918+0000", 2024-09-04T17:47:41.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.29e-07 2024-09-04T17:47:41.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:41.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654919+0000", 2024-09-04T17:47:41.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2589999999999999e-06 2024-09-04T17:47:41.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:41.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654922+0000", 2024-09-04T17:47:41.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1350000000000001e-06 2024-09-04T17:47:41.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:41.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654925+0000", 2024-09-04T17:47:41.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1250000000000001e-06 2024-09-04T17:47:41.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:41.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721466+0000", 2024-09-04T17:47:41.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066540959999999996 2024-09-04T17:47:41.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:41.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721574+0000", 2024-09-04T17:47:41.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010747399999999999 2024-09-04T17:47:41.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:41.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721740+0000", 2024-09-04T17:47:41.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016626799999999999 2024-09-04T17:47:41.372 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:41.372 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:41.372 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:41.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.372 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59951 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5472256~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:47:41.372 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654929+0000", 2024-09-04T17:47:41.372 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.710418747999999, 2024-09-04T17:47:41.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067110534999999999, 2024-09-04T17:47:41.372 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:41.372 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:41.373 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:41.373 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:41.373 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:41.373 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:41.373 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59951 2024-09-04T17:47:41.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.373 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:41.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:41.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654929+0000", 2024-09-04T17:47:41.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:41.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:41.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654929+0000", 2024-09-04T17:47:41.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:41.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:41.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654929+0000", 2024-09-04T17:47:41.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5100000000000001e-07 2024-09-04T17:47:41.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:41.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654930+0000", 2024-09-04T17:47:41.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2440000000000001e-06 2024-09-04T17:47:41.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:41.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654933+0000", 2024-09-04T17:47:41.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1599999999999998e-06 2024-09-04T17:47:41.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:41.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654936+0000", 2024-09-04T17:47:41.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6560000000000001e-06 2024-09-04T17:47:41.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:41.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721789+0000", 2024-09-04T17:47:41.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066853250000000003 2024-09-04T17:47:41.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:41.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721885+0000", 2024-09-04T17:47:41.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5616000000000001e-05 2024-09-04T17:47:41.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:41.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722039+0000", 2024-09-04T17:47:41.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000154258 2024-09-04T17:47:41.377 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:41.377 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:41.377 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:41.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.377 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59952 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5505024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:47:41.378 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654939+0000", 2024-09-04T17:47:41.378 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.710407871000001, 2024-09-04T17:47:41.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067433742000000005, 2024-09-04T17:47:41.378 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:41.378 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:41.378 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:41.378 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:41.378 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:41.378 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:41.378 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59952 2024-09-04T17:47:41.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.378 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:41.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:41.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654939+0000", 2024-09-04T17:47:41.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:41.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:41.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654939+0000", 2024-09-04T17:47:41.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:41.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:41.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654940+0000", 2024-09-04T17:47:41.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3300000000000001e-07 2024-09-04T17:47:41.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:41.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654941+0000", 2024-09-04T17:47:41.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1570000000000001e-06 2024-09-04T17:47:41.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:41.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654944+0000", 2024-09-04T17:47:41.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1319999999999998e-06 2024-09-04T17:47:41.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:41.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654947+0000", 2024-09-04T17:47:41.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5969999999999999e-06 2024-09-04T17:47:41.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:41.382 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722089+0000", 2024-09-04T17:47:41.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067142347000000005 2024-09-04T17:47:41.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:41.382 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722201+0000", 2024-09-04T17:47:41.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011235600000000001 2024-09-04T17:47:41.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:41.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722373+0000", 2024-09-04T17:47:41.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017191999999999999 2024-09-04T17:47:41.383 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:41.383 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:41.384 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:41.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.384 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59953 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5611520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:47:41.384 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654950+0000", 2024-09-04T17:47:41.384 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.710397231000002, 2024-09-04T17:47:41.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067601852000000004, 2024-09-04T17:47:41.384 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:41.384 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:41.384 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:41.384 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:41.385 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:41.385 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:41.385 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59953 2024-09-04T17:47:41.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.385 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:41.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:41.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654950+0000", 2024-09-04T17:47:41.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:41.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:41.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654950+0000", 2024-09-04T17:47:41.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:41.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:41.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654950+0000", 2024-09-04T17:47:41.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2499999999999999e-07 2024-09-04T17:47:41.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:41.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654954+0000", 2024-09-04T17:47:41.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.287e-06 2024-09-04T17:47:41.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:41.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654962+0000", 2024-09-04T17:47:41.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4230000000000005e-06 2024-09-04T17:47:41.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:41.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654966+0000", 2024-09-04T17:47:41.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6449999999999998e-06 2024-09-04T17:47:41.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:41.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722417+0000", 2024-09-04T17:47:41.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067451188999999995 2024-09-04T17:47:41.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:41.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722467+0000", 2024-09-04T17:47:41.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0556999999999997e-05 2024-09-04T17:47:41.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:41.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722552+0000", 2024-09-04T17:47:41.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4525999999999997e-05 2024-09-04T17:47:41.389 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:41.389 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:41.389 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:41.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.390 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59954 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5718016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:47:41.390 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654970+0000", 2024-09-04T17:47:41.390 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.710377470000001, 2024-09-04T17:47:41.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067874285000000006, 2024-09-04T17:47:41.390 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:41.390 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:41.390 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:41.390 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:41.390 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:41.390 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:41.391 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59954 2024-09-04T17:47:41.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.391 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:41.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:41.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654970+0000", 2024-09-04T17:47:41.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:41.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:41.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654970+0000", 2024-09-04T17:47:41.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:41.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:41.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654970+0000", 2024-09-04T17:47:41.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5899999999999998e-07 2024-09-04T17:47:41.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:41.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654971+0000", 2024-09-04T17:47:41.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2929999999999999e-06 2024-09-04T17:47:41.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:41.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654975+0000", 2024-09-04T17:47:41.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4699999999999998e-06 2024-09-04T17:47:41.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:41.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654978+0000", 2024-09-04T17:47:41.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7930000000000002e-06 2024-09-04T17:47:41.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:41.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722600+0000", 2024-09-04T17:47:41.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067622229000000006 2024-09-04T17:47:41.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:41.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722689+0000", 2024-09-04T17:47:41.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9511000000000004e-05 2024-09-04T17:47:41.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:41.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722844+0000", 2024-09-04T17:47:41.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015473 2024-09-04T17:47:41.395 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:41.395 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:41.395 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:41.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.396 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59955 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5857280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:47:41.396 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654981+0000", 2024-09-04T17:47:41.396 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.71036612, 2024-09-04T17:47:41.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068052360000000006, 2024-09-04T17:47:41.396 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:41.396 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:41.396 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:41.396 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:41.396 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:41.396 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:41.396 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59955 2024-09-04T17:47:41.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.397 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:41.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:41.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654981+0000", 2024-09-04T17:47:41.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:41.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:41.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654981+0000", 2024-09-04T17:47:41.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:41.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:41.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654981+0000", 2024-09-04T17:47:41.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2399999999999999e-07 2024-09-04T17:47:41.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:41.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654985+0000", 2024-09-04T17:47:41.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0980000000000004e-06 2024-09-04T17:47:41.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:41.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654990+0000", 2024-09-04T17:47:41.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3429999999999998e-06 2024-09-04T17:47:41.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:41.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654993+0000", 2024-09-04T17:47:41.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8700000000000001e-06 2024-09-04T17:47:41.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:41.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722888+0000", 2024-09-04T17:47:41.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067895239999999996 2024-09-04T17:47:41.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:41.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722948+0000", 2024-09-04T17:47:41.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9750999999999998e-05 2024-09-04T17:47:41.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:41.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.723034+0000", 2024-09-04T17:47:41.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5834000000000005e-05 2024-09-04T17:47:41.401 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:41.401 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:41.401 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:41.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.401 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59956 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5963776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:47:41.401 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654998+0000", 2024-09-04T17:47:41.401 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.710349591, 2024-09-04T17:47:41.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068359518999999994, 2024-09-04T17:47:41.401 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:41.402 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:41.402 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:41.402 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:41.402 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:41.402 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:41.402 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59956 2024-09-04T17:47:41.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.402 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:41.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:41.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654998+0000", 2024-09-04T17:47:41.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:41.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:41.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654998+0000", 2024-09-04T17:47:41.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:41.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:41.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654998+0000", 2024-09-04T17:47:41.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4299999999999999e-07 2024-09-04T17:47:41.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:41.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654999+0000", 2024-09-04T17:47:41.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1799999999999999e-06 2024-09-04T17:47:41.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:41.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.655002+0000", 2024-09-04T17:47:41.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3639999999999999e-06 2024-09-04T17:47:41.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:41.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.655005+0000", 2024-09-04T17:47:41.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7970000000000001e-06 2024-09-04T17:47:41.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:41.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.723085+0000", 2024-09-04T17:47:41.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068080229000000006 2024-09-04T17:47:41.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:41.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.723197+0000", 2024-09-04T17:47:41.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000111344 2024-09-04T17:47:41.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:41.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.723357+0000", 2024-09-04T17:47:41.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016036199999999999 2024-09-04T17:47:41.406 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:41.406 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:41.406 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:41.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.406 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61082 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 2654208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:41.407 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.822664+0000", 2024-09-04T17:47:41.407 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.542683203999999, 2024-09-04T17:47:41.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067072242000000004, 2024-09-04T17:47:41.407 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:41.407 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:41.407 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:41.407 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:41.407 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:41.407 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:41.407 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61082 2024-09-04T17:47:41.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.408 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:41.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:41.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822664+0000", 2024-09-04T17:47:41.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:41.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:41.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822664+0000", 2024-09-04T17:47:41.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:41.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:41.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822664+0000", 2024-09-04T17:47:41.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6600000000000003e-07 2024-09-04T17:47:41.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:41.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822666+0000", 2024-09-04T17:47:41.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.536e-06 2024-09-04T17:47:41.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:41.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822669+0000", 2024-09-04T17:47:41.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3759999999999999e-06 2024-09-04T17:47:41.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:41.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822672+0000", 2024-09-04T17:47:41.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8109999999999999e-06 2024-09-04T17:47:41.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.411 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:41.411 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889263+0000", 2024-09-04T17:47:41.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066591259999999999 2024-09-04T17:47:41.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.411 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:41.411 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889369+0000", 2024-09-04T17:47:41.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000105362 2024-09-04T17:47:41.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.411 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:41.412 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889736+0000", 2024-09-04T17:47:41.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00036763099999999999 2024-09-04T17:47:41.412 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:41.412 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:41.412 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:41.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.412 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61083 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 2760704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:41.412 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.822678+0000", 2024-09-04T17:47:41.412 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.54266947, 2024-09-04T17:47:41.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067475996999999996, 2024-09-04T17:47:41.413 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:41.413 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:41.413 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:41.413 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:41.413 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:41.413 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:41.413 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61083 2024-09-04T17:47:41.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.413 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:41.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:41.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822678+0000", 2024-09-04T17:47:41.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:41.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:41.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822678+0000", 2024-09-04T17:47:41.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:41.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:41.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822725+0000", 2024-09-04T17:47:41.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7429000000000003e-05 2024-09-04T17:47:41.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:41.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822728+0000", 2024-09-04T17:47:41.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2140000000000001e-06 2024-09-04T17:47:41.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:41.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822737+0000", 2024-09-04T17:47:41.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4370000000000007e-06 2024-09-04T17:47:41.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:41.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822743+0000", 2024-09-04T17:47:41.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7520000000000002e-06 2024-09-04T17:47:41.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:41.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889785+0000", 2024-09-04T17:47:41.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067042265000000004 2024-09-04T17:47:41.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:41.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889840+0000", 2024-09-04T17:47:41.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5254000000000003e-05 2024-09-04T17:47:41.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:41.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.890154+0000", 2024-09-04T17:47:41.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000313646 2024-09-04T17:47:41.417 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:41.417 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:41.418 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:41.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.418 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61084 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 2867200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:41.418 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.822753+0000", 2024-09-04T17:47:41.418 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.542594533999999, 2024-09-04T17:47:41.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068497161000000001, 2024-09-04T17:47:41.418 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:41.418 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:41.418 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:41.418 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:41.419 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:41.419 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:41.419 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61084 2024-09-04T17:47:41.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.419 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:41.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:41.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822753+0000", 2024-09-04T17:47:41.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:41.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:41.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822753+0000", 2024-09-04T17:47:41.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:41.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:41.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822753+0000", 2024-09-04T17:47:41.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9100000000000003e-07 2024-09-04T17:47:41.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:41.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822756+0000", 2024-09-04T17:47:41.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.678e-06 2024-09-04T17:47:41.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:41.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822763+0000", 2024-09-04T17:47:41.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8220000000000001e-06 2024-09-04T17:47:41.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:41.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822768+0000", 2024-09-04T17:47:41.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1930000000000001e-06 2024-09-04T17:47:41.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:41.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.890246+0000", 2024-09-04T17:47:41.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067477996999999998 2024-09-04T17:47:41.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:41.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.890555+0000", 2024-09-04T17:47:41.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000309143 2024-09-04T17:47:41.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:41.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.891250+0000", 2024-09-04T17:47:41.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00069463700000000001 2024-09-04T17:47:41.424 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:41.424 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:41.424 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:41.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.425 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61285 3.8 3:117cfeb9:::rbd_data.11341dfde0c5.0000000000000006:1f [read 2760704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:41.425 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835200+0000", 2024-09-04T17:47:41.425 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.530147466999999, 2024-09-04T17:47:41.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066899803999999993, 2024-09-04T17:47:41.425 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:41.425 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:41.425 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:41.425 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:41.425 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:41.425 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:41.426 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61285 2024-09-04T17:47:41.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.426 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:41.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:41.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835200+0000", 2024-09-04T17:47:41.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:41.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:41.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835200+0000", 2024-09-04T17:47:41.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:41.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:41.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835200+0000", 2024-09-04T17:47:41.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0899999999999997e-07 2024-09-04T17:47:41.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:41.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835206+0000", 2024-09-04T17:47:41.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3280000000000002e-06 2024-09-04T17:47:41.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:41.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835211+0000", 2024-09-04T17:47:41.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9230000000000001e-06 2024-09-04T17:47:41.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:41.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835216+0000", 2024-09-04T17:47:41.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9019999999999998e-06 2024-09-04T17:47:41.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:41.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.901719+0000", 2024-09-04T17:47:41.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066502981000000003 2024-09-04T17:47:41.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:41.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.901813+0000", 2024-09-04T17:47:41.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3905000000000003e-05 2024-09-04T17:47:41.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.430 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:41.430 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902100+0000", 2024-09-04T17:47:41.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028645599999999998 2024-09-04T17:47:41.430 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:41.430 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:41.430 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:41.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.430 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61286 3.8 3:117cfeb9:::rbd_data.11341dfde0c5.0000000000000006:1f [read 2867200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:41.430 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835244+0000", 2024-09-04T17:47:41.430 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.530102847999999, 2024-09-04T17:47:41.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067143215000000006, 2024-09-04T17:47:41.431 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:41.431 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:41.431 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:41.431 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:41.431 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:41.431 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:41.431 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61286 2024-09-04T17:47:41.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.431 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:41.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:41.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835244+0000", 2024-09-04T17:47:41.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:41.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:41.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835244+0000", 2024-09-04T17:47:41.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:41.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:41.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835245+0000", 2024-09-04T17:47:41.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8400000000000004e-07 2024-09-04T17:47:41.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:41.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835248+0000", 2024-09-04T17:47:41.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6850000000000001e-06 2024-09-04T17:47:41.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:41.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835254+0000", 2024-09-04T17:47:41.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.494e-06 2024-09-04T17:47:41.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.434 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:41.434 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835260+0000", 2024-09-04T17:47:41.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0580000000000003e-06 2024-09-04T17:47:41.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.434 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:41.434 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902119+0000", 2024-09-04T17:47:41.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066858318 2024-09-04T17:47:41.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:41.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902163+0000", 2024-09-04T17:47:41.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4753000000000003e-05 2024-09-04T17:47:41.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:41.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902388+0000", 2024-09-04T17:47:41.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022422300000000001 2024-09-04T17:47:41.435 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:41.435 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:41.436 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:41.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.436 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61315 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 98304~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:41.436 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835269+0000", 2024-09-04T17:47:41.436 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.530078056000001, 2024-09-04T17:47:41.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067329396, 2024-09-04T17:47:41.436 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:41.436 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:41.436 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:41.436 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:41.437 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:41.437 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:41.437 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61315 2024-09-04T17:47:41.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.437 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:41.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:41.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835269+0000", 2024-09-04T17:47:41.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:41.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:41.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835269+0000", 2024-09-04T17:47:41.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:41.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:41.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835270+0000", 2024-09-04T17:47:41.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6599999999999996e-07 2024-09-04T17:47:41.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:41.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835272+0000", 2024-09-04T17:47:41.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.261e-06 2024-09-04T17:47:41.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.439 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:41.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835278+0000", 2024-09-04T17:47:41.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.443e-06 2024-09-04T17:47:41.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.439 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:41.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835284+0000", 2024-09-04T17:47:41.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3290000000000001e-06 2024-09-04T17:47:41.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:41.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902147+0000", 2024-09-04T17:47:41.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066862695999999999 2024-09-04T17:47:41.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:41.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902248+0000", 2024-09-04T17:47:41.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000101501 2024-09-04T17:47:41.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:41.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902599+0000", 2024-09-04T17:47:41.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00035060000000000001 2024-09-04T17:47:41.441 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:41.441 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:41.441 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:41.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.441 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61316 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 204800~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:41.442 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835291+0000", 2024-09-04T17:47:41.442 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.530056037000001, 2024-09-04T17:47:41.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067632052999999998, 2024-09-04T17:47:41.442 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:41.442 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:41.442 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:41.442 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:41.442 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:41.442 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:41.442 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61316 2024-09-04T17:47:41.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.442 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:41.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:41.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835291+0000", 2024-09-04T17:47:41.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:41.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:41.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835291+0000", 2024-09-04T17:47:41.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:41.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:41.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835292+0000", 2024-09-04T17:47:41.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1899999999999998e-07 2024-09-04T17:47:41.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:41.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835294+0000", 2024-09-04T17:47:41.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0880000000000002e-06 2024-09-04T17:47:41.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:41.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835300+0000", 2024-09-04T17:47:41.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.224e-06 2024-09-04T17:47:41.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:41.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835305+0000", 2024-09-04T17:47:41.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0730000000000004e-06 2024-09-04T17:47:41.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:41.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902632+0000", 2024-09-04T17:47:41.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067327322999999994 2024-09-04T17:47:41.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:41.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902688+0000", 2024-09-04T17:47:41.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5177999999999999e-05 2024-09-04T17:47:41.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:41.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902923+0000", 2024-09-04T17:47:41.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023574800000000001 2024-09-04T17:47:41.447 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:41.447 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:41.447 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:41.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.447 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61317 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 344064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:41.447 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835312+0000", 2024-09-04T17:47:41.447 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.530035033000001, 2024-09-04T17:47:41.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067918511000000001, 2024-09-04T17:47:41.447 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:41.447 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:41.447 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:41.448 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:41.448 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:41.448 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:41.448 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61317 2024-09-04T17:47:41.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.448 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:41.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:41.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835312+0000", 2024-09-04T17:47:41.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:41.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:41.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835312+0000", 2024-09-04T17:47:41.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:41.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:41.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835313+0000", 2024-09-04T17:47:41.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9000000000000002e-07 2024-09-04T17:47:41.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:41.450 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835315+0000", 2024-09-04T17:47:41.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1689999999999999e-06 2024-09-04T17:47:41.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.450 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:41.450 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835321+0000", 2024-09-04T17:47:41.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0619999999999997e-06 2024-09-04T17:47:41.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.450 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:41.451 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835326+0000", 2024-09-04T17:47:41.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0390000000000004e-06 2024-09-04T17:47:41.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.451 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:41.451 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902967+0000", 2024-09-04T17:47:41.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067641520999999996 2024-09-04T17:47:41.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.451 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:41.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903067+0000", 2024-09-04T17:47:41.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9426999999999995e-05 2024-09-04T17:47:41.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:41.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903231+0000", 2024-09-04T17:47:41.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016390299999999999 2024-09-04T17:47:41.452 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:41.452 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:41.452 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:41.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.453 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61318 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 450560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:41.453 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835334+0000", 2024-09-04T17:47:41.453 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.530013516, 2024-09-04T17:47:41.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068091771999999995, 2024-09-04T17:47:41.453 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:41.453 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:41.453 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:41.453 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:41.453 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:41.453 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:41.454 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61318 2024-09-04T17:47:41.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.454 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:41.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.454 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:41.454 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835334+0000", 2024-09-04T17:47:41.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:41.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.454 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:41.454 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835334+0000", 2024-09-04T17:47:41.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:41.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:41.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835334+0000", 2024-09-04T17:47:41.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5699999999999998e-07 2024-09-04T17:47:41.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:41.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835337+0000", 2024-09-04T17:47:41.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3889999999999999e-06 2024-09-04T17:47:41.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:41.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835343+0000", 2024-09-04T17:47:41.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1809999999999998e-06 2024-09-04T17:47:41.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:41.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835348+0000", 2024-09-04T17:47:41.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3170000000000001e-06 2024-09-04T17:47:41.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:41.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903270+0000", 2024-09-04T17:47:41.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067922191000000007 2024-09-04T17:47:41.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:41.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903321+0000", 2024-09-04T17:47:41.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0889e-05 2024-09-04T17:47:41.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:41.458 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903426+0000", 2024-09-04T17:47:41.458 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104348 2024-09-04T17:47:41.458 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:41.458 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:41.458 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:41.458 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.458 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.458 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61319 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 589824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:41.458 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835357+0000", 2024-09-04T17:47:41.458 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.529990698999999, 2024-09-04T17:47:41.458 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068372535999999998, 2024-09-04T17:47:41.458 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:41.459 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:41.459 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:41.459 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:41.459 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:41.459 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:41.459 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61319 2024-09-04T17:47:41.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.459 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:41.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:41.459 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835357+0000", 2024-09-04T17:47:41.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:41.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:41.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835357+0000", 2024-09-04T17:47:41.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:41.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:41.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835357+0000", 2024-09-04T17:47:41.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.32e-07 2024-09-04T17:47:41.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:41.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835359+0000", 2024-09-04T17:47:41.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1660000000000001e-06 2024-09-04T17:47:41.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:41.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835366+0000", 2024-09-04T17:47:41.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3550000000000001e-06 2024-09-04T17:47:41.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.462 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:41.462 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835371+0000", 2024-09-04T17:47:41.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1050000000000003e-06 2024-09-04T17:47:41.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.462 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:41.462 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903471+0000", 2024-09-04T17:47:41.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068100451000000006 2024-09-04T17:47:41.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.462 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:41.463 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903573+0000", 2024-09-04T17:47:41.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000101691 2024-09-04T17:47:41.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.463 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:41.463 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903729+0000", 2024-09-04T17:47:41.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000156336 2024-09-04T17:47:41.463 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:41.463 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:41.463 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:41.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.464 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61320 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:41.464 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835380+0000", 2024-09-04T17:47:41.464 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.529966934000001, 2024-09-04T17:47:41.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068543058000000004, 2024-09-04T17:47:41.464 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:41.464 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:41.464 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:41.464 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:41.464 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:41.464 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:41.464 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61320 2024-09-04T17:47:41.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.465 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:41.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:41.465 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835380+0000", 2024-09-04T17:47:41.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:41.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:41.465 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835380+0000", 2024-09-04T17:47:41.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:41.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:41.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835381+0000", 2024-09-04T17:47:41.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9999999999999998e-07 2024-09-04T17:47:41.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:41.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835383+0000", 2024-09-04T17:47:41.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2230000000000001e-06 2024-09-04T17:47:41.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.467 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:41.467 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835390+0000", 2024-09-04T17:47:41.467 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.34e-06 2024-09-04T17:47:41.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.467 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.467 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:41.467 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835396+0000", 2024-09-04T17:47:41.467 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9830000000000001e-06 2024-09-04T17:47:41.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.467 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.467 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:41.467 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903765+0000", 2024-09-04T17:47:41.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068368441000000002 2024-09-04T17:47:41.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.468 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:41.468 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903835+0000", 2024-09-04T17:47:41.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0631999999999997e-05 2024-09-04T17:47:41.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.468 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:41.468 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903923+0000", 2024-09-04T17:47:41.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8039000000000002e-05 2024-09-04T17:47:41.469 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:41.469 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:41.469 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:41.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.469 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61321 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 835584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:41.469 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835404+0000", 2024-09-04T17:47:41.469 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.529942875, 2024-09-04T17:47:41.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068911043000000005, 2024-09-04T17:47:41.469 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:41.469 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:41.470 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:41.470 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:41.470 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:41.470 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:41.470 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61321 2024-09-04T17:47:41.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.470 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:41.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.470 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:41.470 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835404+0000", 2024-09-04T17:47:41.470 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:41.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:41.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835404+0000", 2024-09-04T17:47:41.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:41.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:41.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835405+0000", 2024-09-04T17:47:41.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.89e-07 2024-09-04T17:47:41.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.472 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:41.472 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835407+0000", 2024-09-04T17:47:41.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4439999999999998e-06 2024-09-04T17:47:41.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.472 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:41.472 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835414+0000", 2024-09-04T17:47:41.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4930000000000003e-06 2024-09-04T17:47:41.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.473 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:41.473 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835419+0000", 2024-09-04T17:47:41.473 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8919999999999999e-06 2024-09-04T17:47:41.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.473 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:41.473 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903957+0000", 2024-09-04T17:47:41.473 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068538500000000002 2024-09-04T17:47:41.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.474 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:41.474 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904019+0000", 2024-09-04T17:47:41.474 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1419000000000003e-05 2024-09-04T17:47:41.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.474 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:41.474 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904315+0000", 2024-09-04T17:47:41.474 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00029680599999999999 2024-09-04T17:47:41.474 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:41.474 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:41.474 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:41.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.475 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61322 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:41.475 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835427+0000", 2024-09-04T17:47:41.475 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 27.529920488999998, 2024-09-04T17:47:41.475 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.069550138999999997, 2024-09-04T17:47:41.475 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:41.475 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:41.475 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:41.475 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:41.475 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:41.476 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:41.476 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61322 2024-09-04T17:47:41.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.476 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:41.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:41.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835427+0000", 2024-09-04T17:47:41.476 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:41.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:41.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835427+0000", 2024-09-04T17:47:41.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:41.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.477 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:41.477 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.842157+0000", 2024-09-04T17:47:41.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067298419999999998 2024-09-04T17:47:41.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.477 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:41.477 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.842163+0000", 2024-09-04T17:47:41.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2029999999999997e-06 2024-09-04T17:47:41.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.478 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.478 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:41.478 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.842175+0000", 2024-09-04T17:47:41.478 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2072000000000001e-05 2024-09-04T17:47:41.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.478 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.478 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:41.478 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.842183+0000", 2024-09-04T17:47:41.478 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3850000000000002e-06 2024-09-04T17:47:41.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.478 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.478 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:41.479 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904371+0000", 2024-09-04T17:47:41.479 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.062187754999999997 2024-09-04T17:47:41.479 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.479 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:41.479 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904467+0000", 2024-09-04T17:47:41.479 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6117000000000001e-05 2024-09-04T17:47:41.479 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:41.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:41.479 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:41.479 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904977+0000", 2024-09-04T17:47:41.480 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00050976500000000002 2024-09-04T17:47:41.480 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:41.480 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:41.480 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:41.480 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:41.480 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:41.480 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:41.482 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-04T17:47:41.484 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:47:41.484 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:47:41.485 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:47:41.485 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:47:41.486 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-04T17:47:41.486 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_enable_op_tracker = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' 2024-09-04T17:47:41.486 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-04T17:47:41.486 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-04T17:47:41.487 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:47:41.487 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:47:41.487 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:47:41.488 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:47:41.488 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-04T17:47:41.488 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:41.479+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:41.503 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:47:41.503 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:47:41.507 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-04T17:47:41.507 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-04T17:47:41.583 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:47:41.579+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:47:41.617 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:41.617 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:41.631 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-04T17:47:41.717 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:41.715+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:41.785 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:41.781+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:41.798 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:41.798 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:41.812 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-04T17:47:41.897 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:47:41.895+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:47:41.987 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:41.983+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:41.988 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:41.988 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:42.003 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-04T17:47:42.089 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:42.087+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:42.168 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:42.168 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:42.182 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-04T17:47:42.188 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:42.185+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:42.289 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:42.289+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:42.350 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:42.351 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:42.365 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-04T17:47:42.390 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:42.389+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:42.487 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:42.488 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:42.491 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:42.489+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:42.503 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-04T17:47:42.592 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:42.587+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:42.626 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:42.626 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:42.642 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-04T17:47:42.729 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:42.727+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:42.794 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:42.793+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:42.808 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:42.808 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:42.822 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-04T17:47:42.895 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:42.893+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:42.927 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-04T17:47:42.990 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:42.990 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:42.995 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:47:42.991+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:47:43.005 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-04T17:47:43.096 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:47:43.091+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:47:43.152 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:43.153 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:43.167 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-04T17:47:43.196 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:47:43.191+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:47:43.290 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:43.290 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:43.297 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:43.296+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:43.305 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-04T17:47:43.399 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:43.396+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:43.472 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:43.473 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:43.486 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-04T17:47:43.508 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-04T17:47:43.529 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:43.528+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:43.601 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:43.599+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:43.646 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:43.646 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:43.662 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-04T17:47:43.676 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:47:43.676 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:47:43.693 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:47:43.693 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:47:43.702 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:43.699+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:43.709 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-04T17:47:43.710 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-04T17:47:43.726 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-04T17:47:43.727 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-04T17:47:43.747 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:47:43.747 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:47:43.768 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:47:43.768 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:47:43.768 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-04T17:47:43.776 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:43.776 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:47:43.776 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:47:43.776 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:47:43.776 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:43.789 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:47:43.789 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:47:43.790 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-04T17:47:43.792 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-04T17:47:43.793 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-04T17:47:43.803 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:43.799+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:43.904 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:43.903+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:43.944 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is set 2024-09-04T17:47:43.969 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:43.969 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:47:43.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:47:43.969 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:47:43.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.969 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:48444 3.12 3:4ff80fb5:::rbd_data.11346a2b0bc9.000000000000001d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-04T17:47:43.970 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:58.122601+0000", 2024-09-04T17:47:43.970 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 45.845760902000002, 2024-09-04T17:47:43.970 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057065713999999997, 2024-09-04T17:47:43.970 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:43.970 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:43.970 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:43.970 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:43.970 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:43.970 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:43.970 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 48444 2024-09-04T17:47:43.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.970 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:43.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.970 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:43.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122601+0000", 2024-09-04T17:47:43.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:43.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.971 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:43.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122601+0000", 2024-09-04T17:47:43.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:43.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.971 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:43.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122603+0000", 2024-09-04T17:47:43.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3499999999999999e-06 2024-09-04T17:47:43.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:43.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122614+0000", 2024-09-04T17:47:43.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.063e-05 2024-09-04T17:47:43.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:43.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122643+0000", 2024-09-04T17:47:43.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9434999999999999e-05 2024-09-04T17:47:43.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:43.973 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122657+0000", 2024-09-04T17:47:43.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3920000000000001e-05 2024-09-04T17:47:43.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.973 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.973 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:43.973 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122767+0000", 2024-09-04T17:47:43.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010981599999999999 2024-09-04T17:47:43.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.973 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.973 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:43.973 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122989+0000", 2024-09-04T17:47:43.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022158200000000001 2024-09-04T17:47:43.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.974 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 3", 2024-09-04T17:47:43.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.133349+0000", 2024-09-04T17:47:43.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010360729000000001 2024-09-04T17:47:43.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.974 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:43.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.179423+0000", 2024-09-04T17:47:43.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046073408000000003 2024-09-04T17:47:43.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.974 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:43.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.179453+0000", 2024-09-04T17:47:43.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0229000000000001e-05 2024-09-04T17:47:43.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:43.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.179545+0000", 2024-09-04T17:47:43.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1580000000000001e-05 2024-09-04T17:47:43.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:43.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.179667+0000", 2024-09-04T17:47:43.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012203500000000001 2024-09-04T17:47:43.975 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:43.975 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:43.976 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:43.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.976 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:55376 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-04T17:47:43.976 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:11.974053+0000", 2024-09-04T17:47:43.976 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 31.994309118, 2024-09-04T17:47:43.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060514722, 2024-09-04T17:47:43.976 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:43.976 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:43.976 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:43.976 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:43.976 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:43.976 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:43.976 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 55376 2024-09-04T17:47:43.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.977 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:43.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.977 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:43.977 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974053+0000", 2024-09-04T17:47:43.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:43.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.977 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:43.977 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974053+0000", 2024-09-04T17:47:43.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:43.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.978 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:43.978 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974054+0000", 2024-09-04T17:47:43.978 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.841e-06 2024-09-04T17:47:43.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.978 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.978 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:43.978 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974062+0000", 2024-09-04T17:47:43.978 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5460000000000003e-06 2024-09-04T17:47:43.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.978 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.978 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:43.978 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974077+0000", 2024-09-04T17:47:43.978 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5389000000000001e-05 2024-09-04T17:47:43.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:43.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974085+0000", 2024-09-04T17:47:43.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7659999999999998e-06 2024-09-04T17:47:43.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:43.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974188+0000", 2024-09-04T17:47:43.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000102521 2024-09-04T17:47:43.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:43.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974321+0000", 2024-09-04T17:47:43.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013324699999999999 2024-09-04T17:47:43.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-04T17:47:43.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.986481+0000", 2024-09-04T17:47:43.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012160450999999999 2024-09-04T17:47:43.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:43.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:12.034415+0000", 2024-09-04T17:47:43.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.047934065999999997 2024-09-04T17:47:43.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:43.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:12.034438+0000", 2024-09-04T17:47:43.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2923999999999999e-05 2024-09-04T17:47:43.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:43.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:12.034505+0000", 2024-09-04T17:47:43.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6711000000000006e-05 2024-09-04T17:47:43.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:43.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:12.034567+0000", 2024-09-04T17:47:43.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2260000000000004e-05 2024-09-04T17:47:43.982 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:43.982 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:43.982 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:43.982 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.982 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.982 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61128 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 1458176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:43.982 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819489+0000", 2024-09-04T17:47:43.982 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.148872783000002, 2024-09-04T17:47:43.982 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.038380478000000003, 2024-09-04T17:47:43.982 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:43.982 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:43.982 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:43.982 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:43.982 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:43.983 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:43.983 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61128 2024-09-04T17:47:43.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.983 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:43.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.983 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:43.983 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819489+0000", 2024-09-04T17:47:43.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:43.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.983 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:43.983 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819489+0000", 2024-09-04T17:47:43.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:43.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:43.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819490+0000", 2024-09-04T17:47:43.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6599999999999996e-07 2024-09-04T17:47:43.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:43.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819493+0000", 2024-09-04T17:47:43.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2279999999999998e-06 2024-09-04T17:47:43.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:43.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819501+0000", 2024-09-04T17:47:43.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4039999999999995e-06 2024-09-04T17:47:43.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:43.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819508+0000", 2024-09-04T17:47:43.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2400000000000004e-06 2024-09-04T17:47:43.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:43.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.857534+0000", 2024-09-04T17:47:43.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.038026272 2024-09-04T17:47:43.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.986 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.986 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:43.986 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.857629+0000", 2024-09-04T17:47:43.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5397999999999997e-05 2024-09-04T17:47:43.986 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.986 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.986 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:43.986 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.857869+0000", 2024-09-04T17:47:43.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024027 2024-09-04T17:47:43.986 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:43.986 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:43.986 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:43.986 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.987 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.987 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61129 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 1564672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:43.987 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819517+0000", 2024-09-04T17:47:43.987 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.148844961999998, 2024-09-04T17:47:43.987 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.038772745999999997, 2024-09-04T17:47:43.987 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:43.987 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:43.987 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:43.987 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:43.987 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:43.987 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:43.987 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61129 2024-09-04T17:47:43.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.988 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:43.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.988 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:43.988 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819517+0000", 2024-09-04T17:47:43.988 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:43.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.988 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:43.988 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819517+0000", 2024-09-04T17:47:43.988 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:43.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.988 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:43.989 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819517+0000", 2024-09-04T17:47:43.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5999999999999999e-07 2024-09-04T17:47:43.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.989 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.989 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:43.989 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819520+0000", 2024-09-04T17:47:43.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2709999999999999e-06 2024-09-04T17:47:43.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.989 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.989 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:43.989 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819526+0000", 2024-09-04T17:47:43.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7059999999999998e-06 2024-09-04T17:47:43.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.990 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.990 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:43.990 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819531+0000", 2024-09-04T17:47:43.990 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0549999999999999e-06 2024-09-04T17:47:43.990 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.990 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.990 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:43.990 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.857913+0000", 2024-09-04T17:47:43.990 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.038381954000000003 2024-09-04T17:47:43.990 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.990 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.990 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:43.990 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.858000+0000", 2024-09-04T17:47:43.991 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6963000000000001e-05 2024-09-04T17:47:43.991 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.991 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.991 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:43.991 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.858290+0000", 2024-09-04T17:47:43.991 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028933700000000003 2024-09-04T17:47:43.991 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:43.991 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:43.991 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:43.991 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.991 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.991 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61130 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 1597440~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:43.991 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819539+0000", 2024-09-04T17:47:43.991 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.148823135000001, 2024-09-04T17:47:43.992 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.039166595999999998, 2024-09-04T17:47:43.992 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:43.992 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:43.992 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:43.992 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:43.992 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:43.992 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:43.992 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61130 2024-09-04T17:47:43.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.992 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:43.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.992 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:43.992 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819539+0000", 2024-09-04T17:47:43.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:43.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:43.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819539+0000", 2024-09-04T17:47:43.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:43.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:43.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819539+0000", 2024-09-04T17:47:43.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9400000000000005e-07 2024-09-04T17:47:43.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:43.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819542+0000", 2024-09-04T17:47:43.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2630000000000002e-06 2024-09-04T17:47:43.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:43.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819548+0000", 2024-09-04T17:47:43.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3269999999999997e-06 2024-09-04T17:47:43.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:43.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819553+0000", 2024-09-04T17:47:43.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9849999999999997e-06 2024-09-04T17:47:43.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.995 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:43.995 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.858321+0000", 2024-09-04T17:47:43.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.038768395999999997 2024-09-04T17:47:43.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.995 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:43.995 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.858406+0000", 2024-09-04T17:47:43.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4456999999999995e-05 2024-09-04T17:47:43.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.996 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:43.996 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.858705+0000", 2024-09-04T17:47:43.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00029947399999999999 2024-09-04T17:47:43.996 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:43.996 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:43.996 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:43.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.996 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61131 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 1703936~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:43.996 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819560+0000", 2024-09-04T17:47:43.996 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.148801783, 2024-09-04T17:47:43.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.039522082, 2024-09-04T17:47:43.996 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:43.996 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:43.997 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:43.997 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:43.997 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:43.997 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:43.997 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61131 2024-09-04T17:47:43.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.997 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:43.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:43.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819560+0000", 2024-09-04T17:47:43.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:43.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:43.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819560+0000", 2024-09-04T17:47:43.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:43.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:43.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819561+0000", 2024-09-04T17:47:43.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5300000000000004e-07 2024-09-04T17:47:43.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:43.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819563+0000", 2024-09-04T17:47:43.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.187e-06 2024-09-04T17:47:43.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:43.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819569+0000", 2024-09-04T17:47:43.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6170000000000003e-06 2024-09-04T17:47:43.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:43.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:43.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819574+0000", 2024-09-04T17:47:43.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9640000000000002e-06 2024-09-04T17:47:43.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:43.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:44.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.858754+0000", 2024-09-04T17:47:44.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.039179295000000003 2024-09-04T17:47:44.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:44.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.858800+0000", 2024-09-04T17:47:44.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.587e-05 2024-09-04T17:47:44.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:44.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.859082+0000", 2024-09-04T17:47:44.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028249599999999997 2024-09-04T17:47:44.001 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.001 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:44.001 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.001 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61132 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 1810432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:44.001 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819581+0000", 2024-09-04T17:47:44.001 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.148780340999998, 2024-09-04T17:47:44.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.039953120000000002, 2024-09-04T17:47:44.001 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:44.001 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:44.001 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:44.001 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:44.002 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:44.002 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:44.002 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61132 2024-09-04T17:47:44.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.002 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:44.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:44.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819581+0000", 2024-09-04T17:47:44.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:44.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819581+0000", 2024-09-04T17:47:44.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:44.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819582+0000", 2024-09-04T17:47:44.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3399999999999999e-07 2024-09-04T17:47:44.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:44.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819584+0000", 2024-09-04T17:47:44.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1320000000000001e-06 2024-09-04T17:47:44.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:44.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819591+0000", 2024-09-04T17:47:44.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3759999999999996e-06 2024-09-04T17:47:44.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:44.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819595+0000", 2024-09-04T17:47:44.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7339999999999999e-06 2024-09-04T17:47:44.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:44.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.859119+0000", 2024-09-04T17:47:44.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.039523677 2024-09-04T17:47:44.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:44.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.859232+0000", 2024-09-04T17:47:44.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000113001 2024-09-04T17:47:44.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:44.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.859535+0000", 2024-09-04T17:47:44.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030256599999999999 2024-09-04T17:47:44.005 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.005 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:44.006 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.006 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61133 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 1843200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:44.006 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819602+0000", 2024-09-04T17:47:44.006 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.148760032999999, 2024-09-04T17:47:44.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.040226171999999998, 2024-09-04T17:47:44.006 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:44.006 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:44.006 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:44.006 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:44.006 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:44.006 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:44.006 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61133 2024-09-04T17:47:44.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.007 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:44.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:44.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819602+0000", 2024-09-04T17:47:44.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:44.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819602+0000", 2024-09-04T17:47:44.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:44.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819602+0000", 2024-09-04T17:47:44.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.75e-07 2024-09-04T17:47:44.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:44.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819605+0000", 2024-09-04T17:47:44.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.52e-06 2024-09-04T17:47:44.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:44.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819611+0000", 2024-09-04T17:47:44.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5200000000000003e-06 2024-09-04T17:47:44.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:44.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819616+0000", 2024-09-04T17:47:44.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0939999999999998e-06 2024-09-04T17:47:44.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:44.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.859538+0000", 2024-09-04T17:47:44.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.039921660999999997 2024-09-04T17:47:44.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:44.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.859594+0000", 2024-09-04T17:47:44.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5671999999999998e-05 2024-09-04T17:47:44.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:44.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.859828+0000", 2024-09-04T17:47:44.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023423000000000001 2024-09-04T17:47:44.010 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.010 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:44.010 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.010 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61134 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 1949696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:44.010 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819623+0000", 2024-09-04T17:47:44.011 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.148738556000001, 2024-09-04T17:47:44.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.040606862000000001, 2024-09-04T17:47:44.011 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:44.011 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:44.011 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:44.011 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:44.011 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:44.011 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:44.011 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61134 2024-09-04T17:47:44.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.011 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:44.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:44.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819623+0000", 2024-09-04T17:47:44.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:44.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819623+0000", 2024-09-04T17:47:44.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:44.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819624+0000", 2024-09-04T17:47:44.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2100000000000002e-07 2024-09-04T17:47:44.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:44.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819626+0000", 2024-09-04T17:47:44.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2460000000000002e-06 2024-09-04T17:47:44.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:44.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819632+0000", 2024-09-04T17:47:44.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1659999999999999e-06 2024-09-04T17:47:44.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:44.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819637+0000", 2024-09-04T17:47:44.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2680000000000002e-06 2024-09-04T17:47:44.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:44.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.859872+0000", 2024-09-04T17:47:44.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.040234377000000002 2024-09-04T17:47:44.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:44.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.859950+0000", 2024-09-04T17:47:44.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8128999999999997e-05 2024-09-04T17:47:44.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:44.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.860230+0000", 2024-09-04T17:47:44.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028025499999999998 2024-09-04T17:47:44.015 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.015 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:44.015 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.015 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61135 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 2056192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:44.015 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819644+0000", 2024-09-04T17:47:44.015 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.148717516000001, 2024-09-04T17:47:44.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.040925930999999999, 2024-09-04T17:47:44.015 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:44.015 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:44.015 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:44.015 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:44.016 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:44.016 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:44.016 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61135 2024-09-04T17:47:44.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.016 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:44.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.016 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:44.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819644+0000", 2024-09-04T17:47:44.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.016 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:44.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819644+0000", 2024-09-04T17:47:44.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:44.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819645+0000", 2024-09-04T17:47:44.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9399999999999995e-07 2024-09-04T17:47:44.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:44.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819647+0000", 2024-09-04T17:47:44.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.311e-06 2024-09-04T17:47:44.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:44.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819653+0000", 2024-09-04T17:47:44.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3940000000000001e-06 2024-09-04T17:47:44.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:44.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819658+0000", 2024-09-04T17:47:44.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8420000000000004e-06 2024-09-04T17:47:44.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:44.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.860263+0000", 2024-09-04T17:47:44.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.040605109 2024-09-04T17:47:44.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:44.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.860339+0000", 2024-09-04T17:47:44.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5612999999999996e-05 2024-09-04T17:47:44.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:44.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.860570+0000", 2024-09-04T17:47:44.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023116799999999999 2024-09-04T17:47:44.019 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.019 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:44.019 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.020 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61136 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 2195456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:44.020 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819665+0000", 2024-09-04T17:47:44.020 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.148697062, 2024-09-04T17:47:44.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.041275408999999999, 2024-09-04T17:47:44.020 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:44.020 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:44.020 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:44.020 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:44.020 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:44.020 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:44.020 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61136 2024-09-04T17:47:44.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.021 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:44.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.021 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:44.021 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819665+0000", 2024-09-04T17:47:44.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.021 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:44.021 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819665+0000", 2024-09-04T17:47:44.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.021 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:44.021 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819665+0000", 2024-09-04T17:47:44.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8599999999999999e-07 2024-09-04T17:47:44.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:44.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819667+0000", 2024-09-04T17:47:44.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9439999999999999e-06 2024-09-04T17:47:44.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:44.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819673+0000", 2024-09-04T17:47:44.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0220000000000001e-06 2024-09-04T17:47:44.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:44.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819677+0000", 2024-09-04T17:47:44.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4139999999999996e-06 2024-09-04T17:47:44.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:44.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.860603+0000", 2024-09-04T17:47:44.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.040925504000000001 2024-09-04T17:47:44.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:44.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.860673+0000", 2024-09-04T17:47:44.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9707000000000002e-05 2024-09-04T17:47:44.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:44.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.860940+0000", 2024-09-04T17:47:44.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026743200000000001 2024-09-04T17:47:44.025 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.025 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:44.025 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.025 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61137 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 2301952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:44.025 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819685+0000", 2024-09-04T17:47:44.025 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.148676268999999, 2024-09-04T17:47:44.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.041579208999999999, 2024-09-04T17:47:44.025 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:44.026 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:44.026 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:44.026 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:44.026 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:44.026 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:44.026 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61137 2024-09-04T17:47:44.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.026 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:44.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:44.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819685+0000", 2024-09-04T17:47:44.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:44.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819685+0000", 2024-09-04T17:47:44.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:44.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819686+0000", 2024-09-04T17:47:44.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3000000000000001e-07 2024-09-04T17:47:44.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:44.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819688+0000", 2024-09-04T17:47:44.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1150000000000001e-06 2024-09-04T17:47:44.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:44.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819694+0000", 2024-09-04T17:47:44.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.75e-06 2024-09-04T17:47:44.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:44.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819699+0000", 2024-09-04T17:47:44.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8110000000000002e-06 2024-09-04T17:47:44.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:44.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.860977+0000", 2024-09-04T17:47:44.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.041277945000000003 2024-09-04T17:47:44.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:44.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.861071+0000", 2024-09-04T17:47:44.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4252000000000005e-05 2024-09-04T17:47:44.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:44.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.861265+0000", 2024-09-04T17:47:44.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019390600000000001 2024-09-04T17:47:44.030 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.030 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:44.030 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.030 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61138 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 2441216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:44.030 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819706+0000", 2024-09-04T17:47:44.030 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.148655867999999, 2024-09-04T17:47:44.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.041876780000000002, 2024-09-04T17:47:44.030 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:44.030 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:44.030 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:44.030 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:44.030 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:44.031 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:44.031 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61138 2024-09-04T17:47:44.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.031 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:44.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:44.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819706+0000", 2024-09-04T17:47:44.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:44.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819706+0000", 2024-09-04T17:47:44.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:44.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819707+0000", 2024-09-04T17:47:44.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7299999999999995e-07 2024-09-04T17:47:44.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:44.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819727+0000", 2024-09-04T17:47:44.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0610000000000001e-05 2024-09-04T17:47:44.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:44.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819735+0000", 2024-09-04T17:47:44.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.123e-06 2024-09-04T17:47:44.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:44.033 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819741+0000", 2024-09-04T17:47:44.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4009999999999996e-06 2024-09-04T17:47:44.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:44.033 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.861298+0000", 2024-09-04T17:47:44.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.041557135000000002 2024-09-04T17:47:44.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:44.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.861369+0000", 2024-09-04T17:47:44.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0794e-05 2024-09-04T17:47:44.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:44.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.861583+0000", 2024-09-04T17:47:44.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021404400000000001 2024-09-04T17:47:44.034 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.034 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:44.034 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.035 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61139 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 2547712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:44.035 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819750+0000", 2024-09-04T17:47:44.035 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.148611404, 2024-09-04T17:47:44.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042185699, 2024-09-04T17:47:44.035 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:44.035 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:44.035 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:44.035 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:44.035 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:44.035 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:44.035 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61139 2024-09-04T17:47:44.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.035 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:44.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:44.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819750+0000", 2024-09-04T17:47:44.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:44.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819750+0000", 2024-09-04T17:47:44.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:44.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819751+0000", 2024-09-04T17:47:44.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4000000000000002e-07 2024-09-04T17:47:44.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.037 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:44.037 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819753+0000", 2024-09-04T17:47:44.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5179999999999999e-06 2024-09-04T17:47:44.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.037 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:44.037 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819760+0000", 2024-09-04T17:47:44.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5440000000000004e-06 2024-09-04T17:47:44.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.037 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:44.037 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819765+0000", 2024-09-04T17:47:44.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8760000000000004e-06 2024-09-04T17:47:44.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:44.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.861622+0000", 2024-09-04T17:47:44.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.041857093999999997 2024-09-04T17:47:44.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:44.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.861698+0000", 2024-09-04T17:47:44.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6080000000000003e-05 2024-09-04T17:47:44.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:44.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.861936+0000", 2024-09-04T17:47:44.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023804699999999999 2024-09-04T17:47:44.039 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.039 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:44.039 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.039 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61140 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 2686976~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:44.039 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819772+0000", 2024-09-04T17:47:44.039 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.148589269999999, 2024-09-04T17:47:44.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042515052999999997, 2024-09-04T17:47:44.039 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:44.039 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:44.040 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:44.040 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:44.040 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:44.040 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:44.040 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61140 2024-09-04T17:47:44.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.040 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:44.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:44.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819772+0000", 2024-09-04T17:47:44.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:44.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819772+0000", 2024-09-04T17:47:44.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:44.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819773+0000", 2024-09-04T17:47:44.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2900000000000003e-07 2024-09-04T17:47:44.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:44.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819775+0000", 2024-09-04T17:47:44.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3309999999999998e-06 2024-09-04T17:47:44.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:44.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819781+0000", 2024-09-04T17:47:44.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.981e-06 2024-09-04T17:47:44.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:44.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819786+0000", 2024-09-04T17:47:44.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5719999999999996e-06 2024-09-04T17:47:44.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:44.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.861966+0000", 2024-09-04T17:47:44.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042179541000000001 2024-09-04T17:47:44.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:44.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862038+0000", 2024-09-04T17:47:44.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2719000000000006e-05 2024-09-04T17:47:44.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:44.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862288+0000", 2024-09-04T17:47:44.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024928 2024-09-04T17:47:44.043 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.043 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:44.043 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.044 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61141 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 2793472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:44.044 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819909+0000", 2024-09-04T17:47:44.044 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.148453126, 2024-09-04T17:47:44.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042786419999999999, 2024-09-04T17:47:44.044 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:44.044 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:44.044 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:44.044 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:44.044 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:44.044 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:44.044 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61141 2024-09-04T17:47:44.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.045 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:44.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:44.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819909+0000", 2024-09-04T17:47:44.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:44.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819909+0000", 2024-09-04T17:47:44.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:44.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819909+0000", 2024-09-04T17:47:44.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6300000000000005e-07 2024-09-04T17:47:44.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:44.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819913+0000", 2024-09-04T17:47:44.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4189999999999998e-06 2024-09-04T17:47:44.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:44.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819920+0000", 2024-09-04T17:47:44.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8619999999999997e-06 2024-09-04T17:47:44.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:44.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819926+0000", 2024-09-04T17:47:44.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9399999999999999e-06 2024-09-04T17:47:44.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:44.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862331+0000", 2024-09-04T17:47:44.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042405348000000002 2024-09-04T17:47:44.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:44.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862410+0000", 2024-09-04T17:47:44.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9078000000000004e-05 2024-09-04T17:47:44.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:44.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862695+0000", 2024-09-04T17:47:44.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028510999999999999 2024-09-04T17:47:44.048 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.048 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:44.048 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.048 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61142 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 2932736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:44.048 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819934+0000", 2024-09-04T17:47:44.048 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.148427508000001, 2024-09-04T17:47:44.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043101985000000002, 2024-09-04T17:47:44.049 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:44.049 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:44.049 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:44.049 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:44.049 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:44.049 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:44.049 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61142 2024-09-04T17:47:44.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.049 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:44.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:44.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819934+0000", 2024-09-04T17:47:44.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:44.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819934+0000", 2024-09-04T17:47:44.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:44.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819935+0000", 2024-09-04T17:47:44.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2024-09-04T17:47:44.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:44.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819937+0000", 2024-09-04T17:47:44.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5770000000000001e-06 2024-09-04T17:47:44.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:44.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819943+0000", 2024-09-04T17:47:44.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7950000000000004e-06 2024-09-04T17:47:44.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:44.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819948+0000", 2024-09-04T17:47:44.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6779999999999999e-06 2024-09-04T17:47:44.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:44.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862739+0000", 2024-09-04T17:47:44.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042791454999999999 2024-09-04T17:47:44.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:44.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862824+0000", 2024-09-04T17:47:44.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4790999999999995e-05 2024-09-04T17:47:44.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:44.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.863036+0000", 2024-09-04T17:47:44.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021224899999999999 2024-09-04T17:47:44.053 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.053 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:44.053 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.053 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61143 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 3039232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:44.053 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819954+0000", 2024-09-04T17:47:44.053 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.148407389999999, 2024-09-04T17:47:44.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043485134000000002, 2024-09-04T17:47:44.053 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:44.054 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:44.054 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:44.054 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:44.054 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:44.054 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:44.054 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61143 2024-09-04T17:47:44.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.054 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:44.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:44.054 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819954+0000", 2024-09-04T17:47:44.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:44.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819954+0000", 2024-09-04T17:47:44.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:44.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819955+0000", 2024-09-04T17:47:44.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7300000000000002e-07 2024-09-04T17:47:44.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:44.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819957+0000", 2024-09-04T17:47:44.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.024e-06 2024-09-04T17:47:44.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:44.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819963+0000", 2024-09-04T17:47:44.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8109999999999999e-06 2024-09-04T17:47:44.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:44.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819967+0000", 2024-09-04T17:47:44.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6149999999999999e-06 2024-09-04T17:47:44.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:44.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.863082+0000", 2024-09-04T17:47:44.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043115078000000001 2024-09-04T17:47:44.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:44.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.863172+0000", 2024-09-04T17:47:44.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0181999999999998e-05 2024-09-04T17:47:44.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:44.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.863440+0000", 2024-09-04T17:47:44.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026705099999999997 2024-09-04T17:47:44.057 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.057 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:44.058 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.058 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:70015 3.6 3:6012e780:::rbd_data.11341aa245a3.000000000000005b:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-04T17:47:44.058 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:26.155497+0000", 2024-09-04T17:47:44.058 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.812864333, 2024-09-04T17:47:44.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.049007124999999999, 2024-09-04T17:47:44.058 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:44.058 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:44.058 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:44.058 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:44.058 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:44.058 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:44.059 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 70015 2024-09-04T17:47:44.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.059 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:44.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:44.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155497+0000", 2024-09-04T17:47:44.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:44.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155497+0000", 2024-09-04T17:47:44.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.060 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:44.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155500+0000", 2024-09-04T17:47:44.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7070000000000001e-06 2024-09-04T17:47:44.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.060 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:44.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155513+0000", 2024-09-04T17:47:44.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2989e-05 2024-09-04T17:47:44.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:44.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155544+0000", 2024-09-04T17:47:44.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0747000000000002e-05 2024-09-04T17:47:44.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:44.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155560+0000", 2024-09-04T17:47:44.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5866e-05 2024-09-04T17:47:44.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:44.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155675+0000", 2024-09-04T17:47:44.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000114838 2024-09-04T17:47:44.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:44.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155803+0000", 2024-09-04T17:47:44.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012851799999999999 2024-09-04T17:47:44.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 1", 2024-09-04T17:47:44.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.163582+0000", 2024-09-04T17:47:44.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0077790960000000001 2024-09-04T17:47:44.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:44.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.204055+0000", 2024-09-04T17:47:44.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.040472517 2024-09-04T17:47:44.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:44.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.204195+0000", 2024-09-04T17:47:44.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014055200000000001 2024-09-04T17:47:44.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:44.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.204373+0000", 2024-09-04T17:47:44.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000177784 2024-09-04T17:47:44.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:44.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.204505+0000", 2024-09-04T17:47:44.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013151100000000001 2024-09-04T17:47:44.064 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.064 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:44.064 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.064 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:70434 3.12 3:4cd2342b:::rbd_data.11341aa245a3.0000000000000091:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-04T17:47:44.064 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:26.943124+0000", 2024-09-04T17:47:44.064 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.025237551, 2024-09-04T17:47:44.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055011831999999997, 2024-09-04T17:47:44.065 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:44.065 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:44.065 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:44.065 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:44.065 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:44.065 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:44.065 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 70434 2024-09-04T17:47:44.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.065 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:44.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:44.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943124+0000", 2024-09-04T17:47:44.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:44.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943124+0000", 2024-09-04T17:47:44.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:44.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943128+0000", 2024-09-04T17:47:44.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8460000000000001e-06 2024-09-04T17:47:44.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:44.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943142+0000", 2024-09-04T17:47:44.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4089e-05 2024-09-04T17:47:44.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:44.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943177+0000", 2024-09-04T17:47:44.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5167999999999999e-05 2024-09-04T17:47:44.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:44.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943194+0000", 2024-09-04T17:47:44.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6235e-05 2024-09-04T17:47:44.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:44.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943286+0000", 2024-09-04T17:47:44.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2793000000000004e-05 2024-09-04T17:47:44.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:44.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943526+0000", 2024-09-04T17:47:44.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023950999999999999 2024-09-04T17:47:44.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 1", 2024-09-04T17:47:44.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.952871+0000", 2024-09-04T17:47:44.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0093453070000000006 2024-09-04T17:47:44.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:44.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.997968+0000", 2024-09-04T17:47:44.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045096613000000001 2024-09-04T17:47:44.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:44.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.998006+0000", 2024-09-04T17:47:44.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8257000000000003e-05 2024-09-04T17:47:44.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:44.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.998073+0000", 2024-09-04T17:47:44.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6803000000000004e-05 2024-09-04T17:47:44.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:44.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.998136+0000", 2024-09-04T17:47:44.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3211000000000002e-05 2024-09-04T17:47:44.070 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.070 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:44.070 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.070 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.070 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:44.070 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:44.072 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-04T17:47:44.072 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:44.068+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:44.081 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-04T17:47:44.172 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:44.167+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:44.236 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:44.236 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:47:44.236 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:47:44.236 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:44.248 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:44.248 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:47:44.248 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:47:44.248 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:44.263 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-04T17:47:44.305 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:47:44.304+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:47:44.374 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:47:44.372+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:47:44.426 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:44.426 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:47:44.426 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:47:44.426 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:47:44.426 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:44.440 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-04T17:47:44.485 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:47:44.484+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:47:44.575 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:44.571+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:44.611 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:44.612 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:47:44.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:47:44.612 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:47:44.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.612 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59950 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5365760~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:47:44.612 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654917+0000", 2024-09-04T17:47:44.612 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.954451116000001, 2024-09-04T17:47:44.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066822450000000005, 2024-09-04T17:47:44.612 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:44.612 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:44.612 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:44.612 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:44.613 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:44.613 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:44.613 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59950 2024-09-04T17:47:44.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.613 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:44.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.613 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:44.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654917+0000", 2024-09-04T17:47:44.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.613 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:44.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654917+0000", 2024-09-04T17:47:44.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:44.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654918+0000", 2024-09-04T17:47:44.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.29e-07 2024-09-04T17:47:44.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:44.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654919+0000", 2024-09-04T17:47:44.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2589999999999999e-06 2024-09-04T17:47:44.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:44.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654922+0000", 2024-09-04T17:47:44.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1350000000000001e-06 2024-09-04T17:47:44.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:44.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654925+0000", 2024-09-04T17:47:44.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1250000000000001e-06 2024-09-04T17:47:44.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:44.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721466+0000", 2024-09-04T17:47:44.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066540959999999996 2024-09-04T17:47:44.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:44.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721574+0000", 2024-09-04T17:47:44.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010747399999999999 2024-09-04T17:47:44.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:44.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721740+0000", 2024-09-04T17:47:44.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016626799999999999 2024-09-04T17:47:44.616 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.616 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:44.616 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.617 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59951 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5472256~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:47:44.617 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654929+0000", 2024-09-04T17:47:44.617 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.954439931, 2024-09-04T17:47:44.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067110534999999999, 2024-09-04T17:47:44.617 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:44.617 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:44.617 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:44.617 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:44.617 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:44.617 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:44.617 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59951 2024-09-04T17:47:44.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.618 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:44.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.618 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:44.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654929+0000", 2024-09-04T17:47:44.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.618 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:44.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654929+0000", 2024-09-04T17:47:44.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.618 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:44.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654929+0000", 2024-09-04T17:47:44.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5100000000000001e-07 2024-09-04T17:47:44.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:44.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654930+0000", 2024-09-04T17:47:44.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2440000000000001e-06 2024-09-04T17:47:44.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:44.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654933+0000", 2024-09-04T17:47:44.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1599999999999998e-06 2024-09-04T17:47:44.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:44.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654936+0000", 2024-09-04T17:47:44.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6560000000000001e-06 2024-09-04T17:47:44.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:44.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721789+0000", 2024-09-04T17:47:44.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066853250000000003 2024-09-04T17:47:44.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:44.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721885+0000", 2024-09-04T17:47:44.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5616000000000001e-05 2024-09-04T17:47:44.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:44.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722039+0000", 2024-09-04T17:47:44.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000154258 2024-09-04T17:47:44.621 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.621 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:44.621 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.621 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59952 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5505024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:47:44.621 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654939+0000", 2024-09-04T17:47:44.621 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.954429053999998, 2024-09-04T17:47:44.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067433742000000005, 2024-09-04T17:47:44.621 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:44.621 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:44.622 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:44.622 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:44.622 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:44.622 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:44.622 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59952 2024-09-04T17:47:44.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.622 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:44.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.622 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:44.622 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654939+0000", 2024-09-04T17:47:44.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.622 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:44.623 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654939+0000", 2024-09-04T17:47:44.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.623 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:44.623 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654940+0000", 2024-09-04T17:47:44.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3300000000000001e-07 2024-09-04T17:47:44.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.623 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:44.623 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654941+0000", 2024-09-04T17:47:44.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1570000000000001e-06 2024-09-04T17:47:44.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:44.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654944+0000", 2024-09-04T17:47:44.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1319999999999998e-06 2024-09-04T17:47:44.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:44.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654947+0000", 2024-09-04T17:47:44.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5969999999999999e-06 2024-09-04T17:47:44.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:44.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722089+0000", 2024-09-04T17:47:44.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067142347000000005 2024-09-04T17:47:44.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:44.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722201+0000", 2024-09-04T17:47:44.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011235600000000001 2024-09-04T17:47:44.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:44.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722373+0000", 2024-09-04T17:47:44.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017191999999999999 2024-09-04T17:47:44.625 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.625 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:44.625 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.626 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59953 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5611520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:47:44.626 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654950+0000", 2024-09-04T17:47:44.626 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.954418413999999, 2024-09-04T17:47:44.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067601852000000004, 2024-09-04T17:47:44.626 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:44.626 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:44.626 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:44.626 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:44.626 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:44.626 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:44.626 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59953 2024-09-04T17:47:44.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.627 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:44.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:44.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654950+0000", 2024-09-04T17:47:44.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:44.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654950+0000", 2024-09-04T17:47:44.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:44.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654950+0000", 2024-09-04T17:47:44.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2499999999999999e-07 2024-09-04T17:47:44.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:44.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654954+0000", 2024-09-04T17:47:44.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.287e-06 2024-09-04T17:47:44.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:44.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654962+0000", 2024-09-04T17:47:44.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4230000000000005e-06 2024-09-04T17:47:44.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:44.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654966+0000", 2024-09-04T17:47:44.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6449999999999998e-06 2024-09-04T17:47:44.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:44.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722417+0000", 2024-09-04T17:47:44.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067451188999999995 2024-09-04T17:47:44.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:44.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722467+0000", 2024-09-04T17:47:44.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0556999999999997e-05 2024-09-04T17:47:44.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:44.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722552+0000", 2024-09-04T17:47:44.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4525999999999997e-05 2024-09-04T17:47:44.630 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.630 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:44.630 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.630 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59954 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5718016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:47:44.630 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654970+0000", 2024-09-04T17:47:44.630 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.954398652999998, 2024-09-04T17:47:44.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067874285000000006, 2024-09-04T17:47:44.631 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:44.631 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:44.631 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:44.631 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:44.631 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:44.631 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:44.631 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59954 2024-09-04T17:47:44.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.631 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:44.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:44.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654970+0000", 2024-09-04T17:47:44.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:44.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654970+0000", 2024-09-04T17:47:44.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:44.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654970+0000", 2024-09-04T17:47:44.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5899999999999998e-07 2024-09-04T17:47:44.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:44.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654971+0000", 2024-09-04T17:47:44.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2929999999999999e-06 2024-09-04T17:47:44.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:44.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654975+0000", 2024-09-04T17:47:44.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4699999999999998e-06 2024-09-04T17:47:44.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:44.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654978+0000", 2024-09-04T17:47:44.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7930000000000002e-06 2024-09-04T17:47:44.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:44.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722600+0000", 2024-09-04T17:47:44.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067622229000000006 2024-09-04T17:47:44.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:44.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722689+0000", 2024-09-04T17:47:44.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9511000000000004e-05 2024-09-04T17:47:44.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:44.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722844+0000", 2024-09-04T17:47:44.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015473 2024-09-04T17:47:44.634 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.635 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:44.635 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.635 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59955 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5857280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:47:44.635 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654981+0000", 2024-09-04T17:47:44.635 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.954387303000001, 2024-09-04T17:47:44.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068052360000000006, 2024-09-04T17:47:44.635 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:44.635 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:44.635 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:44.635 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:44.635 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:44.635 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:44.636 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59955 2024-09-04T17:47:44.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.636 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:44.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:44.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654981+0000", 2024-09-04T17:47:44.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:44.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654981+0000", 2024-09-04T17:47:44.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:44.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654981+0000", 2024-09-04T17:47:44.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2399999999999999e-07 2024-09-04T17:47:44.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:44.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654985+0000", 2024-09-04T17:47:44.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0980000000000004e-06 2024-09-04T17:47:44.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:44.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654990+0000", 2024-09-04T17:47:44.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3429999999999998e-06 2024-09-04T17:47:44.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:44.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654993+0000", 2024-09-04T17:47:44.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8700000000000001e-06 2024-09-04T17:47:44.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:44.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722888+0000", 2024-09-04T17:47:44.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067895239999999996 2024-09-04T17:47:44.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:44.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722948+0000", 2024-09-04T17:47:44.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9750999999999998e-05 2024-09-04T17:47:44.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:44.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.723034+0000", 2024-09-04T17:47:44.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5834000000000005e-05 2024-09-04T17:47:44.639 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.639 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:44.639 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.639 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59956 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5963776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:47:44.639 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654998+0000", 2024-09-04T17:47:44.640 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.954370774000001, 2024-09-04T17:47:44.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068359518999999994, 2024-09-04T17:47:44.640 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:44.640 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:44.640 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:44.640 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:44.640 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:44.640 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:44.640 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59956 2024-09-04T17:47:44.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.640 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:44.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:44.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654998+0000", 2024-09-04T17:47:44.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:44.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654998+0000", 2024-09-04T17:47:44.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:44.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654998+0000", 2024-09-04T17:47:44.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4299999999999999e-07 2024-09-04T17:47:44.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:44.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654999+0000", 2024-09-04T17:47:44.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1799999999999999e-06 2024-09-04T17:47:44.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:44.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.655002+0000", 2024-09-04T17:47:44.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3639999999999999e-06 2024-09-04T17:47:44.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:44.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.655005+0000", 2024-09-04T17:47:44.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7970000000000001e-06 2024-09-04T17:47:44.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:44.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.723085+0000", 2024-09-04T17:47:44.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068080229000000006 2024-09-04T17:47:44.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:44.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.723197+0000", 2024-09-04T17:47:44.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000111344 2024-09-04T17:47:44.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:44.644 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.723357+0000", 2024-09-04T17:47:44.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016036199999999999 2024-09-04T17:47:44.644 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.644 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:44.644 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.644 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61082 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 2654208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:44.644 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.822664+0000", 2024-09-04T17:47:44.644 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.786704387, 2024-09-04T17:47:44.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067072242000000004, 2024-09-04T17:47:44.644 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:44.644 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:44.644 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:44.645 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:44.645 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:44.645 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:44.645 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61082 2024-09-04T17:47:44.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.645 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:44.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:44.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822664+0000", 2024-09-04T17:47:44.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:44.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822664+0000", 2024-09-04T17:47:44.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:44.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822664+0000", 2024-09-04T17:47:44.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6600000000000003e-07 2024-09-04T17:47:44.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:44.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822666+0000", 2024-09-04T17:47:44.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.536e-06 2024-09-04T17:47:44.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.647 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:44.647 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822669+0000", 2024-09-04T17:47:44.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3759999999999999e-06 2024-09-04T17:47:44.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.647 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:44.647 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822672+0000", 2024-09-04T17:47:44.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8109999999999999e-06 2024-09-04T17:47:44.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.647 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:44.647 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889263+0000", 2024-09-04T17:47:44.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066591259999999999 2024-09-04T17:47:44.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.648 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:44.648 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889369+0000", 2024-09-04T17:47:44.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000105362 2024-09-04T17:47:44.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.648 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:44.648 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889736+0000", 2024-09-04T17:47:44.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00036763099999999999 2024-09-04T17:47:44.648 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.648 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:44.648 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.649 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61083 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 2760704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:44.649 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.822678+0000", 2024-09-04T17:47:44.649 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.786690653000001, 2024-09-04T17:47:44.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067475996999999996, 2024-09-04T17:47:44.649 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:44.649 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:44.649 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:44.649 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:44.649 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:44.649 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:44.649 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61083 2024-09-04T17:47:44.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.649 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:44.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:44.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822678+0000", 2024-09-04T17:47:44.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:44.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822678+0000", 2024-09-04T17:47:44.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:44.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822725+0000", 2024-09-04T17:47:44.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7429000000000003e-05 2024-09-04T17:47:44.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:44.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822728+0000", 2024-09-04T17:47:44.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2140000000000001e-06 2024-09-04T17:47:44.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:44.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822737+0000", 2024-09-04T17:47:44.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4370000000000007e-06 2024-09-04T17:47:44.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:44.652 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822743+0000", 2024-09-04T17:47:44.652 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7520000000000002e-06 2024-09-04T17:47:44.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.652 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:44.652 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889785+0000", 2024-09-04T17:47:44.652 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067042265000000004 2024-09-04T17:47:44.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.652 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:44.652 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889840+0000", 2024-09-04T17:47:44.652 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5254000000000003e-05 2024-09-04T17:47:44.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:44.653 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.890154+0000", 2024-09-04T17:47:44.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000313646 2024-09-04T17:47:44.653 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.653 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:44.653 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.653 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61084 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 2867200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:44.653 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.822753+0000", 2024-09-04T17:47:44.653 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.786615717, 2024-09-04T17:47:44.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068497161000000001, 2024-09-04T17:47:44.653 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:44.653 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:44.654 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:44.654 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:44.654 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:44.654 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:44.654 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61084 2024-09-04T17:47:44.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.654 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:44.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:44.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822753+0000", 2024-09-04T17:47:44.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:44.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822753+0000", 2024-09-04T17:47:44.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:44.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822753+0000", 2024-09-04T17:47:44.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9100000000000003e-07 2024-09-04T17:47:44.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:44.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822756+0000", 2024-09-04T17:47:44.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.678e-06 2024-09-04T17:47:44.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:44.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822763+0000", 2024-09-04T17:47:44.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8220000000000001e-06 2024-09-04T17:47:44.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:44.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822768+0000", 2024-09-04T17:47:44.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1930000000000001e-06 2024-09-04T17:47:44.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:44.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.890246+0000", 2024-09-04T17:47:44.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067477996999999998 2024-09-04T17:47:44.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.657 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:44.657 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.890555+0000", 2024-09-04T17:47:44.657 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000309143 2024-09-04T17:47:44.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.657 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:44.657 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.891250+0000", 2024-09-04T17:47:44.657 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00069463700000000001 2024-09-04T17:47:44.657 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.657 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:44.657 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.658 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.658 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.658 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61285 3.8 3:117cfeb9:::rbd_data.11341dfde0c5.0000000000000006:1f [read 2760704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:44.658 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835200+0000", 2024-09-04T17:47:44.658 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.77416865, 2024-09-04T17:47:44.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066899803999999993, 2024-09-04T17:47:44.658 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:44.658 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:44.658 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:44.658 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:44.658 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:44.658 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:44.658 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61285 2024-09-04T17:47:44.658 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.659 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:44.659 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.659 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:44.659 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835200+0000", 2024-09-04T17:47:44.659 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.659 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.659 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:44.659 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835200+0000", 2024-09-04T17:47:44.659 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.659 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.659 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:44.660 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835200+0000", 2024-09-04T17:47:44.660 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0899999999999997e-07 2024-09-04T17:47:44.660 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.660 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:44.660 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835206+0000", 2024-09-04T17:47:44.660 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3280000000000002e-06 2024-09-04T17:47:44.660 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.660 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:44.660 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835211+0000", 2024-09-04T17:47:44.660 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9230000000000001e-06 2024-09-04T17:47:44.660 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.661 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:44.661 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835216+0000", 2024-09-04T17:47:44.661 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9019999999999998e-06 2024-09-04T17:47:44.661 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.661 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.661 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:44.661 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.901719+0000", 2024-09-04T17:47:44.661 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066502981000000003 2024-09-04T17:47:44.661 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.661 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.661 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:44.661 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.901813+0000", 2024-09-04T17:47:44.661 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3905000000000003e-05 2024-09-04T17:47:44.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.662 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:44.662 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902100+0000", 2024-09-04T17:47:44.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028645599999999998 2024-09-04T17:47:44.662 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.662 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:44.662 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.662 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61286 3.8 3:117cfeb9:::rbd_data.11341dfde0c5.0000000000000006:1f [read 2867200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:44.662 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835244+0000", 2024-09-04T17:47:44.662 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.774124030999999, 2024-09-04T17:47:44.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067143215000000006, 2024-09-04T17:47:44.663 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:44.663 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:44.663 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:44.663 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:44.663 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:44.663 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:44.663 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61286 2024-09-04T17:47:44.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.663 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:44.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:44.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835244+0000", 2024-09-04T17:47:44.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:44.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835244+0000", 2024-09-04T17:47:44.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:44.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835245+0000", 2024-09-04T17:47:44.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8400000000000004e-07 2024-09-04T17:47:44.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:44.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835248+0000", 2024-09-04T17:47:44.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6850000000000001e-06 2024-09-04T17:47:44.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.665 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:44.665 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835254+0000", 2024-09-04T17:47:44.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.494e-06 2024-09-04T17:47:44.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.665 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:44.665 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835260+0000", 2024-09-04T17:47:44.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0580000000000003e-06 2024-09-04T17:47:44.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.666 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:44.666 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902119+0000", 2024-09-04T17:47:44.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066858318 2024-09-04T17:47:44.666 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.666 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:44.666 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902163+0000", 2024-09-04T17:47:44.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4753000000000003e-05 2024-09-04T17:47:44.666 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.666 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:44.666 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902388+0000", 2024-09-04T17:47:44.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022422300000000001 2024-09-04T17:47:44.666 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.666 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:44.667 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.667 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61315 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 98304~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:44.667 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835269+0000", 2024-09-04T17:47:44.667 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.774099239000002, 2024-09-04T17:47:44.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067329396, 2024-09-04T17:47:44.667 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:44.667 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:44.667 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:44.667 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:44.667 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:44.667 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:44.667 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61315 2024-09-04T17:47:44.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.668 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:44.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.668 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:44.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835269+0000", 2024-09-04T17:47:44.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.668 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:44.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835269+0000", 2024-09-04T17:47:44.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.669 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:44.669 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835270+0000", 2024-09-04T17:47:44.669 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6599999999999996e-07 2024-09-04T17:47:44.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.670 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:44.670 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835272+0000", 2024-09-04T17:47:44.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.261e-06 2024-09-04T17:47:44.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.670 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:44.670 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835278+0000", 2024-09-04T17:47:44.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.443e-06 2024-09-04T17:47:44.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.670 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:44.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835284+0000", 2024-09-04T17:47:44.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3290000000000001e-06 2024-09-04T17:47:44.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:44.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902147+0000", 2024-09-04T17:47:44.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066862695999999999 2024-09-04T17:47:44.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:44.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902248+0000", 2024-09-04T17:47:44.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000101501 2024-09-04T17:47:44.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.672 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:44.672 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902599+0000", 2024-09-04T17:47:44.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00035060000000000001 2024-09-04T17:47:44.672 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.672 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:44.672 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.672 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61316 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 204800~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:44.672 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835291+0000", 2024-09-04T17:47:44.672 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.774077219999999, 2024-09-04T17:47:44.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067632052999999998, 2024-09-04T17:47:44.672 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:44.673 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:44.673 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:44.673 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:44.673 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:44.673 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:44.673 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61316 2024-09-04T17:47:44.673 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.673 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:44.673 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.673 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:44.673 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835291+0000", 2024-09-04T17:47:44.673 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.673 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.674 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.674 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:44.674 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835291+0000", 2024-09-04T17:47:44.674 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.674 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.674 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:44.674 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835292+0000", 2024-09-04T17:47:44.674 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1899999999999998e-07 2024-09-04T17:47:44.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.674 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.674 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:44.674 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835294+0000", 2024-09-04T17:47:44.674 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0880000000000002e-06 2024-09-04T17:47:44.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.675 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.675 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:44.675 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835300+0000", 2024-09-04T17:47:44.675 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.224e-06 2024-09-04T17:47:44.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.675 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.675 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:44.675 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835305+0000", 2024-09-04T17:47:44.675 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0730000000000004e-06 2024-09-04T17:47:44.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.675 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.675 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:44.676 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902632+0000", 2024-09-04T17:47:44.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067327322999999994 2024-09-04T17:47:44.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.676 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:44.676 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902688+0000", 2024-09-04T17:47:44.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5177999999999999e-05 2024-09-04T17:47:44.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.676 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:44.676 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902923+0000", 2024-09-04T17:47:44.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023574800000000001 2024-09-04T17:47:44.676 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.677 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:44.677 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.677 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61317 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 344064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:44.677 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835312+0000", 2024-09-04T17:47:44.677 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.774056215999998, 2024-09-04T17:47:44.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067918511000000001, 2024-09-04T17:47:44.677 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:44.677 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:44.677 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:44.677 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:44.677 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:44.677 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:44.678 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61317 2024-09-04T17:47:44.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.678 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:44.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.678 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:44.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835312+0000", 2024-09-04T17:47:44.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.678 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:44.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835312+0000", 2024-09-04T17:47:44.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.679 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:44.679 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835313+0000", 2024-09-04T17:47:44.679 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9000000000000002e-07 2024-09-04T17:47:44.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.679 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:44.679 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835315+0000", 2024-09-04T17:47:44.679 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1689999999999999e-06 2024-09-04T17:47:44.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.679 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:44.679 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835321+0000", 2024-09-04T17:47:44.680 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0619999999999997e-06 2024-09-04T17:47:44.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.680 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:44.680 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835326+0000", 2024-09-04T17:47:44.680 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0390000000000004e-06 2024-09-04T17:47:44.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.680 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:44.680 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902967+0000", 2024-09-04T17:47:44.680 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067641520999999996 2024-09-04T17:47:44.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.680 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:44.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903067+0000", 2024-09-04T17:47:44.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9426999999999995e-05 2024-09-04T17:47:44.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:44.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903231+0000", 2024-09-04T17:47:44.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016390299999999999 2024-09-04T17:47:44.681 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.681 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:44.681 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.681 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61318 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 450560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:44.682 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835334+0000", 2024-09-04T17:47:44.682 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.774034699000001, 2024-09-04T17:47:44.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068091771999999995, 2024-09-04T17:47:44.682 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:44.682 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:44.682 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:44.682 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:44.682 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:44.682 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:44.682 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61318 2024-09-04T17:47:44.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.682 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:44.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:44.683 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835334+0000", 2024-09-04T17:47:44.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.683 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:44.683 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835334+0000", 2024-09-04T17:47:44.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.683 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:44.683 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835334+0000", 2024-09-04T17:47:44.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5699999999999998e-07 2024-09-04T17:47:44.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.684 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:44.684 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835337+0000", 2024-09-04T17:47:44.684 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3889999999999999e-06 2024-09-04T17:47:44.684 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.684 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.684 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:44.684 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835343+0000", 2024-09-04T17:47:44.684 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1809999999999998e-06 2024-09-04T17:47:44.684 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.684 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.684 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:44.684 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835348+0000", 2024-09-04T17:47:44.684 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3170000000000001e-06 2024-09-04T17:47:44.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.685 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:44.685 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903270+0000", 2024-09-04T17:47:44.685 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067922191000000007 2024-09-04T17:47:44.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.685 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:44.685 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903321+0000", 2024-09-04T17:47:44.685 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0889e-05 2024-09-04T17:47:44.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.685 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:44.685 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903426+0000", 2024-09-04T17:47:44.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104348 2024-09-04T17:47:44.686 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.686 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:44.686 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.686 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61319 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 589824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:44.686 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835357+0000", 2024-09-04T17:47:44.686 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.774011882, 2024-09-04T17:47:44.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068372535999999998, 2024-09-04T17:47:44.686 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:44.686 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:44.686 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:44.686 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:44.687 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:44.687 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:44.687 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61319 2024-09-04T17:47:44.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.687 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:44.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:44.687 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835357+0000", 2024-09-04T17:47:44.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:44.687 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835357+0000", 2024-09-04T17:47:44.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.688 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.688 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:44.688 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835357+0000", 2024-09-04T17:47:44.688 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.32e-07 2024-09-04T17:47:44.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.688 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.688 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:44.688 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835359+0000", 2024-09-04T17:47:44.688 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1660000000000001e-06 2024-09-04T17:47:44.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.688 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.688 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:44.689 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835366+0000", 2024-09-04T17:47:44.689 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3550000000000001e-06 2024-09-04T17:47:44.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.689 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:44.689 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835371+0000", 2024-09-04T17:47:44.689 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1050000000000003e-06 2024-09-04T17:47:44.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.689 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:44.689 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903471+0000", 2024-09-04T17:47:44.689 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068100451000000006 2024-09-04T17:47:44.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.690 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:44.690 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903573+0000", 2024-09-04T17:47:44.690 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000101691 2024-09-04T17:47:44.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.690 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:44.690 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903729+0000", 2024-09-04T17:47:44.690 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000156336 2024-09-04T17:47:44.690 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.690 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:44.690 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.691 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61320 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:44.691 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835380+0000", 2024-09-04T17:47:44.691 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.773988117000002, 2024-09-04T17:47:44.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068543058000000004, 2024-09-04T17:47:44.691 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:44.691 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:44.691 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:44.691 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:44.691 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:44.691 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:44.691 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61320 2024-09-04T17:47:44.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.691 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:44.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.692 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:44.692 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835380+0000", 2024-09-04T17:47:44.692 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.692 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.692 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.692 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:44.692 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835380+0000", 2024-09-04T17:47:44.692 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.692 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.692 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.692 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:44.692 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835381+0000", 2024-09-04T17:47:44.692 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9999999999999998e-07 2024-09-04T17:47:44.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.693 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:44.693 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835383+0000", 2024-09-04T17:47:44.693 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2230000000000001e-06 2024-09-04T17:47:44.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.693 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:44.693 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835390+0000", 2024-09-04T17:47:44.693 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.34e-06 2024-09-04T17:47:44.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.693 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:44.693 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835396+0000", 2024-09-04T17:47:44.694 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9830000000000001e-06 2024-09-04T17:47:44.694 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.694 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.694 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:44.694 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903765+0000", 2024-09-04T17:47:44.694 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068368441000000002 2024-09-04T17:47:44.694 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.694 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.695 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:44.695 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903835+0000", 2024-09-04T17:47:44.695 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0631999999999997e-05 2024-09-04T17:47:44.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.695 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.695 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:44.695 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903923+0000", 2024-09-04T17:47:44.695 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8039000000000002e-05 2024-09-04T17:47:44.695 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.695 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:44.695 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.695 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.696 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61321 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 835584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:44.696 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835404+0000", 2024-09-04T17:47:44.696 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.773964058000001, 2024-09-04T17:47:44.696 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068911043000000005, 2024-09-04T17:47:44.696 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:44.696 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:44.696 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:44.696 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:44.696 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:44.696 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:44.696 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61321 2024-09-04T17:47:44.696 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.696 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:44.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.697 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:44.697 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835404+0000", 2024-09-04T17:47:44.697 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.697 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:44.697 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835404+0000", 2024-09-04T17:47:44.697 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.697 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:44.697 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835405+0000", 2024-09-04T17:47:44.697 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.89e-07 2024-09-04T17:47:44.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.698 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.698 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:44.698 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835407+0000", 2024-09-04T17:47:44.698 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4439999999999998e-06 2024-09-04T17:47:44.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.698 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.698 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:44.698 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835414+0000", 2024-09-04T17:47:44.698 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4930000000000003e-06 2024-09-04T17:47:44.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.698 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.698 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:44.698 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835419+0000", 2024-09-04T17:47:44.699 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8919999999999999e-06 2024-09-04T17:47:44.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.699 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.699 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:44.699 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903957+0000", 2024-09-04T17:47:44.699 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068538500000000002 2024-09-04T17:47:44.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.699 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.699 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:44.699 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904019+0000", 2024-09-04T17:47:44.699 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1419000000000003e-05 2024-09-04T17:47:44.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.699 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.700 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:44.700 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904315+0000", 2024-09-04T17:47:44.700 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00029680599999999999 2024-09-04T17:47:44.700 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.700 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:44.700 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.700 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.700 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.700 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61322 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:44.700 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835427+0000", 2024-09-04T17:47:44.700 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.773941671999999, 2024-09-04T17:47:44.700 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.069550138999999997, 2024-09-04T17:47:44.700 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:44.700 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:44.701 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:44.701 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:44.701 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:44.701 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:44.701 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61322 2024-09-04T17:47:44.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.701 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:44.701 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.701 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:44.701 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835427+0000", 2024-09-04T17:47:44.701 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.701 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:44.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835427+0000", 2024-09-04T17:47:44.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:44.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:44.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.842157+0000", 2024-09-04T17:47:44.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067298419999999998 2024-09-04T17:47:44.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:44.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.842163+0000", 2024-09-04T17:47:44.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2029999999999997e-06 2024-09-04T17:47:44.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.703 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:44.703 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.842175+0000", 2024-09-04T17:47:44.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2072000000000001e-05 2024-09-04T17:47:44.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.703 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:44.703 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.842183+0000", 2024-09-04T17:47:44.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3850000000000002e-06 2024-09-04T17:47:44.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.703 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:44.703 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904371+0000", 2024-09-04T17:47:44.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.062187754999999997 2024-09-04T17:47:44.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.704 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:44.704 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904467+0000", 2024-09-04T17:47:44.704 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6117000000000001e-05 2024-09-04T17:47:44.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:44.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:44.704 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:44.704 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904977+0000", 2024-09-04T17:47:44.704 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00050976500000000002 2024-09-04T17:47:44.704 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.704 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:44.704 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.705 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:44.705 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:44.705 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:44.706 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-04T17:47:44.706 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:44.704+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:44.807 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:47:44.803+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:47:44.838 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:44.838 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:47:44.838 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:47:44.838 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:44.852 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-04T17:47:44.937 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:44.935+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:45.010 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:45.007+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:45.024 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:45.025 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:47:45.025 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:47:45.025 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:47:45.025 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:45.040 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-04T17:47:45.111 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:45.108+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:45.208 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:45.208 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:47:45.208 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:47:45.208 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:47:45.208 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.208 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:12722 3.1b 3:d8152af1:::rbd_data.1134ef3a5a7d.00000000000000bd:head [delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-04T17:47:45.208 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:16.421965+0000", 2024-09-04T17:47:45.208 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 88.784121059, 2024-09-04T17:47:45.208 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.890002811, 2024-09-04T17:47:45.208 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:45.209 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:45.209 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:45.209 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:45.209 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:45.209 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:45.209 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 12722 2024-09-04T17:47:45.209 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.209 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:45.209 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.209 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:45.209 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421965+0000", 2024-09-04T17:47:45.209 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.209 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.209 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.210 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:45.210 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421965+0000", 2024-09-04T17:47:45.210 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.210 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.210 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.210 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:45.210 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421966+0000", 2024-09-04T17:47:45.210 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.102e-06 2024-09-04T17:47:45.210 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.210 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.210 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:45.210 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421971+0000", 2024-09-04T17:47:45.210 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9989999999999999e-06 2024-09-04T17:47:45.211 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.211 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.211 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:45.211 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421981+0000", 2024-09-04T17:47:45.211 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.7380000000000006e-06 2024-09-04T17:47:45.211 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.211 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.211 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:45.211 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421989+0000", 2024-09-04T17:47:45.211 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.1890000000000007e-06 2024-09-04T17:47:45.211 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.211 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.211 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:45.211 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.422050+0000", 2024-09-04T17:47:45.212 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.0702000000000003e-05 2024-09-04T17:47:45.212 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.212 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.212 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for peered", 2024-09-04T17:47:45.212 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.422059+0000", 2024-09-04T17:47:45.212 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5440000000000004e-06 2024-09-04T17:47:45.212 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.212 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.212 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:45.212 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.308156+0000", 2024-09-04T17:47:45.212 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.88609676199999998 2024-09-04T17:47:45.212 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.212 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.213 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:45.213 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.308398+0000", 2024-09-04T17:47:45.213 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024215400000000001 2024-09-04T17:47:45.213 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.213 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.213 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 7", 2024-09-04T17:47:45.213 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.308795+0000", 2024-09-04T17:47:45.213 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000396704 2024-09-04T17:47:45.213 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.213 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.213 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:47:45.213 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.310831+0000", 2024-09-04T17:47:45.213 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0020364200000000002 2024-09-04T17:47:45.214 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.214 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.214 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:45.214 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.311818+0000", 2024-09-04T17:47:45.214 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00098649799999999993 2024-09-04T17:47:45.214 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.214 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.214 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:47:45.214 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.311905+0000", 2024-09-04T17:47:45.214 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7070999999999994e-05 2024-09-04T17:47:45.214 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.214 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.214 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:45.215 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.311968+0000", 2024-09-04T17:47:45.215 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.2928000000000005e-05 2024-09-04T17:47:45.215 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.215 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:45.215 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.215 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.215 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.215 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37359 3.f e58/24 3:f29e50a5:::rbd_data.1134c5a43465.0000000000000021:head v 58'135, mlcod=58'133)", 2024-09-04T17:47:45.215 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:47:45.215 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 65.588790454999994, 2024-09-04T17:47:45.215 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053483197000000003, 2024-09-04T17:47:45.215 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:45.215 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:45.215 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:45.216 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:45.216 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.216 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:45.216 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:47:45.216 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.216 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.216 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.216 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:45.216 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:47:45.216 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.216 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.216 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.216 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:45.217 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:47:45.217 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1699999999999997e-07 2024-09-04T17:47:45.217 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.217 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.217 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:45.217 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617338+0000", 2024-09-04T17:47:45.217 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.1168999999999998e-05 2024-09-04T17:47:45.217 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.217 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.217 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:45.217 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617351+0000", 2024-09-04T17:47:45.217 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3094e-05 2024-09-04T17:47:45.217 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.217 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.218 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:45.218 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617359+0000", 2024-09-04T17:47:45.218 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7970000000000006e-06 2024-09-04T17:47:45.218 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.218 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.218 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:45.218 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617417+0000", 2024-09-04T17:47:45.218 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7079e-05 2024-09-04T17:47:45.218 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.218 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.218 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:45.218 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617438+0000", 2024-09-04T17:47:45.218 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1654999999999998e-05 2024-09-04T17:47:45.218 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.219 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.219 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:47:45.219 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670738+0000", 2024-09-04T17:47:45.219 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053300142000000002 2024-09-04T17:47:45.219 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.219 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.219 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:45.219 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670779+0000", 2024-09-04T17:47:45.219 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0343999999999999e-05 2024-09-04T17:47:45.219 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.219 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:45.219 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.219 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.220 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.220 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37358 3.1d e58/24 3:bd3215f3:::rbd_data.1134c5a43465.0000000000000020:head v 58'91, mlcod=58'89)", 2024-09-04T17:47:45.220 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617502+0000", 2024-09-04T17:47:45.220 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 65.588583736999993, 2024-09-04T17:47:45.220 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053505641999999999, 2024-09-04T17:47:45.220 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:45.220 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:45.220 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:45.220 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:45.220 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.220 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:45.220 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617502+0000", 2024-09-04T17:47:45.220 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.221 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.221 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.221 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:45.221 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617502+0000", 2024-09-04T17:47:45.221 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.221 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.221 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.221 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:45.221 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617503+0000", 2024-09-04T17:47:45.221 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.184e-06 2024-09-04T17:47:45.221 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.221 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.221 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:45.222 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617526+0000", 2024-09-04T17:47:45.222 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2099999999999998e-05 2024-09-04T17:47:45.222 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.222 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.222 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:45.222 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617540+0000", 2024-09-04T17:47:45.222 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4741e-05 2024-09-04T17:47:45.222 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.222 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.222 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:45.222 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617550+0000", 2024-09-04T17:47:45.222 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4259999999999992e-06 2024-09-04T17:47:45.222 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.223 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.223 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:45.223 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617613+0000", 2024-09-04T17:47:45.223 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.3625000000000002e-05 2024-09-04T17:47:45.223 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.223 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.223 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:45.223 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617637+0000", 2024-09-04T17:47:45.223 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.3604999999999998e-05 2024-09-04T17:47:45.223 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.223 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.223 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:47:45.223 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670967+0000", 2024-09-04T17:47:45.223 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053329881000000003 2024-09-04T17:47:45.224 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.224 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.224 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:45.224 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671008+0000", 2024-09-04T17:47:45.224 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.108e-05 2024-09-04T17:47:45.224 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.224 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:45.224 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.224 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.224 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.224 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37361 3.1c e58/24 3:3c7f8976:::rbd_data.1134c5a43465.0000000000000023:head v 58'159, mlcod=58'157)", 2024-09-04T17:47:45.224 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617578+0000", 2024-09-04T17:47:45.224 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 65.588507887000006, 2024-09-04T17:47:45.225 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053990367999999997, 2024-09-04T17:47:45.225 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:45.225 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:45.225 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:45.225 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:45.225 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.225 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:45.225 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617578+0000", 2024-09-04T17:47:45.225 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.225 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.225 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.225 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:45.225 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617578+0000", 2024-09-04T17:47:45.226 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.226 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.226 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.226 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:45.226 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617579+0000", 2024-09-04T17:47:45.226 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7299999999999996e-07 2024-09-04T17:47:45.226 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.226 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.226 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:45.226 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617610+0000", 2024-09-04T17:47:45.226 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0951000000000002e-05 2024-09-04T17:47:45.226 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.226 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.227 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:45.227 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:47:45.227 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0002e-05 2024-09-04T17:47:45.227 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.227 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.227 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:45.227 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617626+0000", 2024-09-04T17:47:45.227 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6660000000000002e-06 2024-09-04T17:47:45.227 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.227 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.227 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:45.227 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670548+0000", 2024-09-04T17:47:45.227 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.052921943999999999 2024-09-04T17:47:45.228 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.228 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.228 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:45.228 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670574+0000", 2024-09-04T17:47:45.228 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5405000000000002e-05 2024-09-04T17:47:45.228 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.228 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.228 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:47:45.228 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671522+0000", 2024-09-04T17:47:45.228 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00094815299999999997 2024-09-04T17:47:45.228 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.228 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.228 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:45.228 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671568+0000", 2024-09-04T17:47:45.229 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6674000000000001e-05 2024-09-04T17:47:45.229 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.229 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:45.229 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.229 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.229 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.229 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37357 3.15 e58/49 3:a897312a:::rbd_data.1134c5a43465.000000000000001f:head v 58'117, mlcod=58'115)", 2024-09-04T17:47:45.229 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:47:45.229 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 65.588465549000006, 2024-09-04T17:47:45.229 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053734297, 2024-09-04T17:47:45.229 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:45.229 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:45.229 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:45.230 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:45.230 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.230 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:45.230 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:47:45.230 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.230 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.230 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.230 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:45.230 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:47:45.230 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.230 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.230 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.230 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:45.231 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617621+0000", 2024-09-04T17:47:45.231 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.7300000000000004e-07 2024-09-04T17:47:45.231 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.231 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.231 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:45.231 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617642+0000", 2024-09-04T17:47:45.231 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0185000000000001e-05 2024-09-04T17:47:45.231 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.231 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.231 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:45.231 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617654+0000", 2024-09-04T17:47:45.231 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2745000000000001e-05 2024-09-04T17:47:45.231 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.231 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.232 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:45.232 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617662+0000", 2024-09-04T17:47:45.232 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.0499999999999992e-06 2024-09-04T17:47:45.232 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.232 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.232 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:45.232 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617727+0000", 2024-09-04T17:47:45.232 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.4923000000000002e-05 2024-09-04T17:47:45.232 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.232 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.232 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:45.232 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617753+0000", 2024-09-04T17:47:45.232 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5899999999999999e-05 2024-09-04T17:47:45.233 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.233 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.233 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:47:45.233 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671298+0000", 2024-09-04T17:47:45.233 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053545276000000003 2024-09-04T17:47:45.233 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.233 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.233 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:45.233 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671355+0000", 2024-09-04T17:47:45.233 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6245000000000001e-05 2024-09-04T17:47:45.233 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.233 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:45.233 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.233 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.234 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.234 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44687 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:45.234 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253012+0000", 2024-09-04T17:47:45.234 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 54.953073482000001, 2024-09-04T17:47:45.234 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051128777, 2024-09-04T17:47:45.234 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:45.234 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:45.234 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:45.234 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:45.234 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:45.234 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:45.234 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44687 2024-09-04T17:47:45.234 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.234 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:45.235 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.235 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:45.235 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253012+0000", 2024-09-04T17:47:45.235 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.235 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.235 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.235 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:45.235 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253012+0000", 2024-09-04T17:47:45.235 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.235 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.235 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.235 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:45.235 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253014+0000", 2024-09-04T17:47:45.235 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.24e-06 2024-09-04T17:47:45.236 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.236 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.236 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:45.236 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253019+0000", 2024-09-04T17:47:45.236 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4569999999999996e-06 2024-09-04T17:47:45.236 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.236 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.236 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:45.236 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253035+0000", 2024-09-04T17:47:45.236 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6291999999999999e-05 2024-09-04T17:47:45.236 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.236 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.237 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:45.237 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253050+0000", 2024-09-04T17:47:45.237 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4423000000000001e-05 2024-09-04T17:47:45.237 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.237 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.237 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:45.237 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304115+0000", 2024-09-04T17:47:45.237 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051064778999999998 2024-09-04T17:47:45.237 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.237 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.237 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:45.237 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304141+0000", 2024-09-04T17:47:45.237 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6585999999999999e-05 2024-09-04T17:47:45.238 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.238 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:45.238 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.238 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.238 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.238 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44688 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:45.238 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253067+0000", 2024-09-04T17:47:45.238 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 54.953018929999999, 2024-09-04T17:47:45.238 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051116979, 2024-09-04T17:47:45.238 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:45.238 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:45.238 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:45.238 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:45.238 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:45.239 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:45.239 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44688 2024-09-04T17:47:45.239 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.239 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:45.239 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.239 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:45.239 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253067+0000", 2024-09-04T17:47:45.239 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.239 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.239 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.239 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:45.239 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253067+0000", 2024-09-04T17:47:45.239 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.240 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.240 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.240 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:45.240 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253068+0000", 2024-09-04T17:47:45.240 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0729999999999999e-06 2024-09-04T17:47:45.240 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.240 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.240 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:45.240 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253073+0000", 2024-09-04T17:47:45.240 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2259999999999996e-06 2024-09-04T17:47:45.240 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.240 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.240 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:45.241 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253091+0000", 2024-09-04T17:47:45.241 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7255999999999999e-05 2024-09-04T17:47:45.241 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.241 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.241 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:45.241 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253103+0000", 2024-09-04T17:47:45.241 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2632999999999999e-05 2024-09-04T17:47:45.241 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.241 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.241 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:45.241 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304173+0000", 2024-09-04T17:47:45.241 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051069537999999998 2024-09-04T17:47:45.241 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.241 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.242 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:45.242 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304184+0000", 2024-09-04T17:47:45.242 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1253000000000001e-05 2024-09-04T17:47:45.242 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.242 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:45.242 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.242 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.242 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.242 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44689 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:45.242 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253121+0000", 2024-09-04T17:47:45.242 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 54.952964793, 2024-09-04T17:47:45.242 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051192595, 2024-09-04T17:47:45.242 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:45.243 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:45.243 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:45.243 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:45.243 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:45.243 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:45.243 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44689 2024-09-04T17:47:45.243 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.243 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:45.243 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.243 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:45.243 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253121+0000", 2024-09-04T17:47:45.243 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.243 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.243 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.244 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:45.244 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253121+0000", 2024-09-04T17:47:45.244 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.244 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.244 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.244 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:45.244 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253122+0000", 2024-09-04T17:47:45.244 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.15e-06 2024-09-04T17:47:45.244 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.244 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.244 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:45.244 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253128+0000", 2024-09-04T17:47:45.244 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2569999999999998e-06 2024-09-04T17:47:45.244 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.245 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.245 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:45.245 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253145+0000", 2024-09-04T17:47:45.245 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7176e-05 2024-09-04T17:47:45.245 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.245 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.245 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:45.245 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253158+0000", 2024-09-04T17:47:45.245 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3139000000000001e-05 2024-09-04T17:47:45.245 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.245 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.245 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:45.245 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304287+0000", 2024-09-04T17:47:45.245 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051129149999999998 2024-09-04T17:47:45.246 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.246 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.246 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:45.246 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304314+0000", 2024-09-04T17:47:45.246 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6723000000000001e-05 2024-09-04T17:47:45.246 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.246 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:45.246 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.246 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.246 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.246 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44690 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:45.246 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253177+0000", 2024-09-04T17:47:45.246 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 54.952908667999999, 2024-09-04T17:47:45.246 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051169263, 2024-09-04T17:47:45.247 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:45.247 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:45.247 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:45.247 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:45.247 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:45.247 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:45.247 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44690 2024-09-04T17:47:45.247 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.247 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:45.247 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.247 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:45.247 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253177+0000", 2024-09-04T17:47:45.247 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.247 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.248 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.248 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:45.248 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253177+0000", 2024-09-04T17:47:45.248 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.248 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.248 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.248 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:45.248 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253178+0000", 2024-09-04T17:47:45.248 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1000000000000001e-06 2024-09-04T17:47:45.248 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.248 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.248 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:45.248 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253184+0000", 2024-09-04T17:47:45.249 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2089999999999996e-06 2024-09-04T17:47:45.249 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.249 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.249 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:45.249 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253200+0000", 2024-09-04T17:47:45.249 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6087e-05 2024-09-04T17:47:45.249 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.249 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.249 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:45.249 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253231+0000", 2024-09-04T17:47:45.249 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0874999999999998e-05 2024-09-04T17:47:45.249 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.249 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.250 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:45.250 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304337+0000", 2024-09-04T17:47:45.250 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051106430000000001 2024-09-04T17:47:45.250 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.250 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.250 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:45.250 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304347+0000", 2024-09-04T17:47:45.250 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5619999999999993e-06 2024-09-04T17:47:45.250 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.250 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:45.250 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.250 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.250 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.250 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44691 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:45.251 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253249+0000", 2024-09-04T17:47:45.251 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 54.952836634000001, 2024-09-04T17:47:45.251 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051195630999999998, 2024-09-04T17:47:45.251 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:45.251 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:45.251 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:45.251 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:45.251 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:45.251 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:45.251 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44691 2024-09-04T17:47:45.251 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.251 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:45.251 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.251 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:45.252 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253249+0000", 2024-09-04T17:47:45.252 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.252 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.252 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.252 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:45.252 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253249+0000", 2024-09-04T17:47:45.252 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.252 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.252 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.252 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:45.252 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253251+0000", 2024-09-04T17:47:45.252 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2330000000000001e-06 2024-09-04T17:47:45.253 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.253 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.253 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:45.253 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253256+0000", 2024-09-04T17:47:45.253 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6099999999999997e-06 2024-09-04T17:47:45.253 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.253 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.253 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:45.253 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253274+0000", 2024-09-04T17:47:45.253 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7952999999999999e-05 2024-09-04T17:47:45.253 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.253 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.253 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:45.254 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253287+0000", 2024-09-04T17:47:45.254 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2839e-05 2024-09-04T17:47:45.254 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.254 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.254 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:45.254 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304396+0000", 2024-09-04T17:47:45.254 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051108990999999999 2024-09-04T17:47:45.254 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.254 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.254 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:45.254 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304445+0000", 2024-09-04T17:47:45.254 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9005000000000003e-05 2024-09-04T17:47:45.254 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.254 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:45.255 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.255 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.255 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.255 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44692 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:45.255 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253337+0000", 2024-09-04T17:47:45.255 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 54.952749060999999, 2024-09-04T17:47:45.255 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051138909000000003, 2024-09-04T17:47:45.255 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:45.255 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:45.255 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:45.255 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:45.255 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:45.255 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:45.255 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44692 2024-09-04T17:47:45.256 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.256 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:45.256 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.256 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:45.256 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253337+0000", 2024-09-04T17:47:45.256 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.256 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.256 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.256 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:45.256 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253337+0000", 2024-09-04T17:47:45.256 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.256 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.256 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.257 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:45.257 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253338+0000", 2024-09-04T17:47:45.257 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.463e-06 2024-09-04T17:47:45.257 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.257 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.257 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:45.257 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253357+0000", 2024-09-04T17:47:45.257 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9043e-05 2024-09-04T17:47:45.257 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.257 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.257 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:45.257 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253379+0000", 2024-09-04T17:47:45.257 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1243e-05 2024-09-04T17:47:45.258 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.258 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.258 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:45.258 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253396+0000", 2024-09-04T17:47:45.258 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7059e-05 2024-09-04T17:47:45.258 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.258 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.258 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:45.258 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304466+0000", 2024-09-04T17:47:45.258 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051070624000000002 2024-09-04T17:47:45.258 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.258 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.258 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:45.258 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304476+0000", 2024-09-04T17:47:45.259 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4769999999999992e-06 2024-09-04T17:47:45.259 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.259 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:45.259 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.259 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.259 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.259 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44693 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:45.259 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253415+0000", 2024-09-04T17:47:45.259 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 54.952670658000002, 2024-09-04T17:47:45.259 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051141949999999999, 2024-09-04T17:47:45.259 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:45.259 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:45.259 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:45.260 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:45.260 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:45.260 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:45.260 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44693 2024-09-04T17:47:45.260 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.260 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:45.260 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.260 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:45.260 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253415+0000", 2024-09-04T17:47:45.260 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.260 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.260 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.260 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:45.260 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253415+0000", 2024-09-04T17:47:45.261 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.261 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.261 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.261 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:45.261 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253416+0000", 2024-09-04T17:47:45.261 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1799999999999999e-06 2024-09-04T17:47:45.261 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.261 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.261 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:45.261 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253422+0000", 2024-09-04T17:47:45.261 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.5940000000000001e-06 2024-09-04T17:47:45.261 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.261 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.262 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:45.262 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253438+0000", 2024-09-04T17:47:45.262 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6008999999999999e-05 2024-09-04T17:47:45.262 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.262 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.262 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:45.262 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253452+0000", 2024-09-04T17:47:45.262 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3543000000000001e-05 2024-09-04T17:47:45.262 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.262 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.262 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:45.262 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304533+0000", 2024-09-04T17:47:45.262 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051081609 2024-09-04T17:47:45.262 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.263 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.263 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:45.263 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304557+0000", 2024-09-04T17:47:45.263 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4015e-05 2024-09-04T17:47:45.263 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.263 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:45.263 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.263 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.263 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.263 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44694 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:45.263 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253469+0000", 2024-09-04T17:47:45.263 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 54.952616532999997, 2024-09-04T17:47:45.263 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051121486000000001, 2024-09-04T17:47:45.263 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:45.264 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:45.264 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:45.264 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:45.264 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:45.264 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:45.264 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44694 2024-09-04T17:47:45.264 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.264 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:45.264 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.264 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:45.264 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253469+0000", 2024-09-04T17:47:45.264 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.264 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.265 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.265 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:45.265 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253469+0000", 2024-09-04T17:47:45.265 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.265 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.265 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.265 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:45.265 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253471+0000", 2024-09-04T17:47:45.265 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1400000000000001e-06 2024-09-04T17:47:45.265 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.265 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.265 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:45.265 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253476+0000", 2024-09-04T17:47:45.266 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7370000000000003e-06 2024-09-04T17:47:45.266 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.266 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.266 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:45.266 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253493+0000", 2024-09-04T17:47:45.266 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6549e-05 2024-09-04T17:47:45.266 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.266 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.266 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:45.266 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253506+0000", 2024-09-04T17:47:45.266 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3406e-05 2024-09-04T17:47:45.266 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.266 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.267 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:45.267 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304582+0000", 2024-09-04T17:47:45.267 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051075534999999998 2024-09-04T17:47:45.267 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.267 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.267 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:45.267 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304591+0000", 2024-09-04T17:47:45.267 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1190000000000003e-06 2024-09-04T17:47:45.267 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.267 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:45.267 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.267 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.267 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.267 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44695 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:45.268 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253552+0000", 2024-09-04T17:47:45.268 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 54.952533778999999, 2024-09-04T17:47:45.268 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051068872000000001, 2024-09-04T17:47:45.268 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:45.268 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:45.268 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:45.268 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:45.268 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:45.268 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:45.268 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44695 2024-09-04T17:47:45.268 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.268 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:45.268 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.269 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:45.269 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253552+0000", 2024-09-04T17:47:45.269 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.269 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.269 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.269 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:45.269 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253552+0000", 2024-09-04T17:47:45.269 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.269 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.269 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.269 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:45.269 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253553+0000", 2024-09-04T17:47:45.269 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1960000000000001e-06 2024-09-04T17:47:45.270 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.270 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.270 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:45.270 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253560+0000", 2024-09-04T17:47:45.270 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.2040000000000002e-06 2024-09-04T17:47:45.270 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.270 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.270 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:45.270 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253578+0000", 2024-09-04T17:47:45.270 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8241999999999999e-05 2024-09-04T17:47:45.270 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.270 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.270 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:45.270 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253591+0000", 2024-09-04T17:47:45.271 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3453e-05 2024-09-04T17:47:45.271 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.271 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.271 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:45.271 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304614+0000", 2024-09-04T17:47:45.271 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051022456000000001 2024-09-04T17:47:45.271 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.271 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.271 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:45.271 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304621+0000", 2024-09-04T17:47:45.271 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3209999999999999e-06 2024-09-04T17:47:45.271 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.271 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:45.272 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.272 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.272 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.272 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44696 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:45.272 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253609+0000", 2024-09-04T17:47:45.272 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 54.952477278000003, 2024-09-04T17:47:45.272 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051115351000000003, 2024-09-04T17:47:45.272 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:45.272 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:45.272 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:45.272 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:45.272 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:45.272 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:45.273 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44696 2024-09-04T17:47:45.273 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.273 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:45.273 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.273 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:45.273 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253609+0000", 2024-09-04T17:47:45.273 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.273 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.273 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.273 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:45.273 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253609+0000", 2024-09-04T17:47:45.273 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.273 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.273 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.274 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:45.274 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253610+0000", 2024-09-04T17:47:45.274 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2300000000000001e-06 2024-09-04T17:47:45.274 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.274 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.274 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:45.274 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253615+0000", 2024-09-04T17:47:45.274 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7260000000000002e-06 2024-09-04T17:47:45.274 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.274 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.274 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:45.274 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253632+0000", 2024-09-04T17:47:45.275 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7039999999999999e-05 2024-09-04T17:47:45.275 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.275 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.275 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:45.275 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253666+0000", 2024-09-04T17:47:45.275 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4662000000000001e-05 2024-09-04T17:47:45.275 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.275 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.275 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:45.275 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304677+0000", 2024-09-04T17:47:45.275 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051011017999999998 2024-09-04T17:47:45.275 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.275 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.275 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:45.276 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304724+0000", 2024-09-04T17:47:45.276 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6675000000000003e-05 2024-09-04T17:47:45.276 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.276 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:45.276 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.276 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.276 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.276 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44697 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:45.276 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253684+0000", 2024-09-04T17:47:45.276 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 54.952401784000003, 2024-09-04T17:47:45.276 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051073107, 2024-09-04T17:47:45.276 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:45.276 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:45.277 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:45.277 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:45.277 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:45.277 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:45.277 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44697 2024-09-04T17:47:45.277 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.277 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:45.277 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.277 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:45.277 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253684+0000", 2024-09-04T17:47:45.277 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.277 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.278 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.278 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:45.278 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253684+0000", 2024-09-04T17:47:45.278 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.278 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.278 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.278 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:45.278 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253685+0000", 2024-09-04T17:47:45.278 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0699999999999999e-06 2024-09-04T17:47:45.278 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.278 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.278 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:45.278 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253690+0000", 2024-09-04T17:47:45.278 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1959999999999999e-06 2024-09-04T17:47:45.279 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.279 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.279 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:45.279 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253707+0000", 2024-09-04T17:47:45.279 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6946e-05 2024-09-04T17:47:45.279 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.279 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.279 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:45.279 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253720+0000", 2024-09-04T17:47:45.279 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3076e-05 2024-09-04T17:47:45.279 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.279 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.279 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:45.279 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304745+0000", 2024-09-04T17:47:45.280 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051024760000000002 2024-09-04T17:47:45.280 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.280 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.280 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:45.280 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304757+0000", 2024-09-04T17:47:45.280 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2058999999999999e-05 2024-09-04T17:47:45.280 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.280 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:45.280 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.280 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.280 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.280 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44698 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:45.280 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253737+0000", 2024-09-04T17:47:45.281 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 54.952349433000002, 2024-09-04T17:47:45.281 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051176078, 2024-09-04T17:47:45.281 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:45.281 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:45.281 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:45.281 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:45.281 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:45.281 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:45.281 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44698 2024-09-04T17:47:45.281 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.281 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:45.281 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.281 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:45.281 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253737+0000", 2024-09-04T17:47:45.282 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.282 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.282 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.282 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:45.282 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253737+0000", 2024-09-04T17:47:45.282 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.282 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.282 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.282 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:45.282 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253738+0000", 2024-09-04T17:47:45.282 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0300000000000001e-06 2024-09-04T17:47:45.282 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.282 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.282 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:45.283 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253752+0000", 2024-09-04T17:47:45.283 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4358999999999999e-05 2024-09-04T17:47:45.283 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.283 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.283 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:45.283 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253771+0000", 2024-09-04T17:47:45.283 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8879999999999999e-05 2024-09-04T17:47:45.283 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.283 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.283 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:45.283 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253785+0000", 2024-09-04T17:47:45.283 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3852000000000001e-05 2024-09-04T17:47:45.283 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.284 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.284 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:45.284 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304876+0000", 2024-09-04T17:47:45.284 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051091105999999997 2024-09-04T17:47:45.284 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.284 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.284 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:45.284 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304913+0000", 2024-09-04T17:47:45.284 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.6850999999999997e-05 2024-09-04T17:47:45.284 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.284 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:45.284 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.284 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.284 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.285 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44699 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:45.285 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253804+0000", 2024-09-04T17:47:45.285 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 54.952281909, 2024-09-04T17:47:45.285 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051172250000000002, 2024-09-04T17:47:45.285 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:45.285 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:45.285 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:45.285 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:45.285 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:45.285 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:45.285 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44699 2024-09-04T17:47:45.285 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.285 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:45.286 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.286 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:45.286 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253804+0000", 2024-09-04T17:47:45.286 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.286 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.286 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.286 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:45.286 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253804+0000", 2024-09-04T17:47:45.286 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.286 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.286 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.286 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:45.286 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253805+0000", 2024-09-04T17:47:45.287 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.08e-06 2024-09-04T17:47:45.287 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.287 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.287 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:45.287 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253810+0000", 2024-09-04T17:47:45.287 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1830000000000002e-06 2024-09-04T17:47:45.287 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.287 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.287 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:45.287 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253828+0000", 2024-09-04T17:47:45.287 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7662000000000001e-05 2024-09-04T17:47:45.287 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.287 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.288 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:45.288 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253841+0000", 2024-09-04T17:47:45.288 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3203999999999999e-05 2024-09-04T17:47:45.288 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.288 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.288 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:45.288 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304966+0000", 2024-09-04T17:47:45.288 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051125273999999998 2024-09-04T17:47:45.288 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.288 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.288 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:45.288 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304976+0000", 2024-09-04T17:47:45.288 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.8470000000000008e-06 2024-09-04T17:47:45.288 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.289 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:45.289 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.289 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.289 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.289 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44700 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:45.289 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253858+0000", 2024-09-04T17:47:45.289 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 54.952228255000001, 2024-09-04T17:47:45.289 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051218957000000002, 2024-09-04T17:47:45.289 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:45.289 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:45.289 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:45.289 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:45.289 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:45.289 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:45.290 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44700 2024-09-04T17:47:45.290 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.290 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:45.290 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.290 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:45.290 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253858+0000", 2024-09-04T17:47:45.290 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.290 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.290 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.290 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:45.290 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253858+0000", 2024-09-04T17:47:45.290 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.290 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.290 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.290 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:45.290 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253859+0000", 2024-09-04T17:47:45.291 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.217e-06 2024-09-04T17:47:45.291 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.291 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.291 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:45.291 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253864+0000", 2024-09-04T17:47:45.291 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7600000000000002e-06 2024-09-04T17:47:45.291 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.291 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.291 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:45.291 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253880+0000", 2024-09-04T17:47:45.291 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6501999999999999e-05 2024-09-04T17:47:45.291 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.291 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.291 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:45.291 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253899+0000", 2024-09-04T17:47:45.292 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8907e-05 2024-09-04T17:47:45.292 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.292 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.292 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:45.292 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305052+0000", 2024-09-04T17:47:45.292 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051152989000000003 2024-09-04T17:47:45.292 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.292 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.292 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:45.292 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305077+0000", 2024-09-04T17:47:45.292 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4581999999999999e-05 2024-09-04T17:47:45.292 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.292 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:45.292 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.293 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.293 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.293 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44701 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:45.293 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253917+0000", 2024-09-04T17:47:45.293 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 54.952168903, 2024-09-04T17:47:45.293 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051231113000000002, 2024-09-04T17:47:45.293 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:45.293 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:45.293 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:45.293 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:45.293 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:45.293 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:45.293 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44701 2024-09-04T17:47:45.293 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.294 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:45.294 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.294 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:45.294 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253917+0000", 2024-09-04T17:47:45.294 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.294 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.294 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.294 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:45.294 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253917+0000", 2024-09-04T17:47:45.294 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.294 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.294 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.294 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:45.295 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253918+0000", 2024-09-04T17:47:45.295 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.136e-06 2024-09-04T17:47:45.295 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.295 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.295 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:45.295 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253924+0000", 2024-09-04T17:47:45.295 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.3070000000000002e-06 2024-09-04T17:47:45.295 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.295 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.295 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:45.295 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253940+0000", 2024-09-04T17:47:45.295 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6612e-05 2024-09-04T17:47:45.295 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.296 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.296 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:45.296 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253954+0000", 2024-09-04T17:47:45.296 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3709999999999999e-05 2024-09-04T17:47:45.296 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.296 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.296 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:45.296 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305136+0000", 2024-09-04T17:47:45.296 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051181935999999997 2024-09-04T17:47:45.296 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.296 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.297 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:45.297 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305148+0000", 2024-09-04T17:47:45.297 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2412000000000001e-05 2024-09-04T17:47:45.297 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.297 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:45.297 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.297 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.297 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:45.297 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:45.299 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-04T17:47:45.299 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:45.296+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:45.391 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #34, image image_client.0-clone34 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent35 2024-09-04T17:47:45.400 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:45.396+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:45.400 INFO:teuthology.orchestra.run.smithi092.stdout:5361 writesame 0xda05f17 thru 0xda101d0 data_size 0x6(0xa2ba bytes) 2024-09-04T17:47:45.428 INFO:teuthology.orchestra.run.smithi092.stdout:5362 write 0x2637847 thru 0x263f363 (0x7b1d bytes) 2024-09-04T17:47:45.432 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:45.432 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:47:45.432 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:47:45.432 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:45.448 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-04T17:47:45.453 INFO:teuthology.orchestra.run.smithi092.stdout:5363 write 0xedc13ef thru 0xedd08d5 (0xf4e7 bytes) 2024-09-04T17:47:45.459 INFO:teuthology.orchestra.run.smithi092.stdout:5364 read 0xc3925d7 thru 0xc39e21f (0xbc49 bytes) 2024-09-04T17:47:45.460 INFO:teuthology.orchestra.run.smithi092.stdout:5367 write 0x3c2c426 thru 0x3c32ea1 (0x6a7c bytes) 2024-09-04T17:47:45.470 INFO:teuthology.orchestra.run.smithi092.stdout:5369 write 0x684043b thru 0x6840b60 (0x726 bytes) 2024-09-04T17:47:45.478 INFO:teuthology.orchestra.run.smithi092.stdout:5370 write 0x5c7cab8 thru 0x5c83d76 (0x72bf bytes) 2024-09-04T17:47:45.492 INFO:teuthology.orchestra.run.smithi092.stdout:5371 read 0x312211a thru 0x3123bd1 (0x1ab8 bytes) 2024-09-04T17:47:45.492 INFO:teuthology.orchestra.run.smithi092.stdout:5374 read 0x4560a50 thru 0x45635f8 (0x2ba9 bytes) 2024-09-04T17:47:45.493 INFO:teuthology.orchestra.run.smithi092.stdout:5375 write 0x28d58d9 thru 0x28e5164 (0xf88c bytes) 2024-09-04T17:47:45.501 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:45.500+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:45.507 INFO:teuthology.orchestra.run.smithi092.stdout:5379 write 0xaa286f3 thru 0xaa30e85 (0x8793 bytes) 2024-09-04T17:47:45.524 INFO:teuthology.orchestra.run.smithi092.stdout:5380 trunc from 0xedd08d6 to 0xc1051ee 2024-09-04T17:47:45.575 INFO:teuthology.orchestra.run.smithi092.stdout:5381 read 0x3323e83 thru 0x33285b9 (0x4737 bytes) 2024-09-04T17:47:45.576 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:45.576 INFO:teuthology.orchestra.run.smithi092.stdout:5383 punch from 0xb230b42 to 0xb236f97, (0x6455 bytes) 2024-09-04T17:47:45.576 INFO:teuthology.orchestra.run.smithi092.stdout:5385 writesame 0xc0fbbfe thru 0xc1008cb data_size 0x6(0x4cce bytes) 2024-09-04T17:47:45.581 INFO:teuthology.orchestra.run.smithi092.stdout:5387 read 0x83126b2 thru 0x8312f55 (0x8a4 bytes) 2024-09-04T17:47:45.582 INFO:teuthology.orchestra.run.smithi092.stdout:5388 punch from 0x1364cba to 0x1368416, (0x375c bytes) 2024-09-04T17:47:45.582 INFO:teuthology.orchestra.run.smithi092.stdout:5389 writesame 0x562515 thru 0x56e00e data_size 0x5b(0xbafa bytes) 2024-09-04T17:47:45.583 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:45.583 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:47:45.583 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:47:45.583 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:47:45.583 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:45.599 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-04T17:47:45.602 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:45.600+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:45.602 INFO:teuthology.orchestra.run.smithi092.stdout:5391 trunc from 0xc1051ee to 0x8edbbb1 2024-09-04T17:47:45.652 INFO:teuthology.orchestra.run.smithi092.stdout:5393 read 0x65fcab1 thru 0x65ff32b (0x287b bytes) 2024-09-04T17:47:45.653 INFO:teuthology.orchestra.run.smithi092.stdout:5394 punch from 0x8010a94 to 0x8010bf0, (0x15c bytes) 2024-09-04T17:47:45.653 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:45.653 INFO:teuthology.orchestra.run.smithi092.stdout:5396 punch from 0x505e20e to 0x5067758, (0x954a bytes) 2024-09-04T17:47:45.653 INFO:teuthology.orchestra.run.smithi092.stdout:5398 writesame 0x349a5fd thru 0x34a6fc0 data_size 0x4a(0xc9c4 bytes) 2024-09-04T17:47:45.665 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:45.665 INFO:teuthology.orchestra.run.smithi092.stdout:5400 read 0x6bdfa67 thru 0x6be8cca (0x9264 bytes) 2024-09-04T17:47:45.666 INFO:teuthology.orchestra.run.smithi092.stdout:5401 writesame 0x21c76aa thru 0x21cd7fe data_size 0x1(0x6155 bytes) 2024-09-04T17:47:45.703 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:45.700+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:45.706 INFO:teuthology.orchestra.run.smithi092.stdout:5402 write 0x4860afb thru 0x486b3c1 (0xa8c7 bytes) 2024-09-04T17:47:45.718 INFO:teuthology.orchestra.run.smithi092.stdout:5403 read 0x212935e thru 0x212d279 (0x3f1c bytes) 2024-09-04T17:47:45.719 INFO:teuthology.orchestra.run.smithi092.stdout:5404 punch from 0x635e817 to 0x635edbd, (0x5a6 bytes) 2024-09-04T17:47:45.719 INFO:teuthology.orchestra.run.smithi092.stdout:5406 punch from 0x828e47e to 0x8292900, (0x4482 bytes) 2024-09-04T17:47:45.719 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:45.719 INFO:teuthology.orchestra.run.smithi092.stdout:5409 clone 37 order 21 su 32768 sc 15 2024-09-04T17:47:45.734 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:45.734 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:47:45.734 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:47:45.734 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:47:45.735 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.735 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8831 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2453504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:45.735 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:47:45.735 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 92.675091292999994, 2024-09-04T17:47:45.735 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14731111799999999, 2024-09-04T17:47:45.735 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:45.735 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:45.735 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:45.735 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:45.735 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:45.735 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:45.735 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8831 2024-09-04T17:47:45.735 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.735 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:45.736 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.736 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:45.736 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:47:45.736 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.736 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.736 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.736 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:45.736 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:47:45.736 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.736 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.736 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.736 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:45.736 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057811+0000", 2024-09-04T17:47:45.737 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.083e-06 2024-09-04T17:47:45.737 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.737 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.737 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:45.737 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057814+0000", 2024-09-04T17:47:45.737 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.407e-06 2024-09-04T17:47:45.737 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.737 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.737 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:45.737 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057825+0000", 2024-09-04T17:47:45.737 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.17e-05 2024-09-04T17:47:45.737 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.737 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.738 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:45.738 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057836+0000", 2024-09-04T17:47:45.738 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0283e-05 2024-09-04T17:47:45.738 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.738 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.738 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:45.738 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.204762+0000", 2024-09-04T17:47:45.738 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.146926205 2024-09-04T17:47:45.738 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.738 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.738 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:45.738 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.204895+0000", 2024-09-04T17:47:45.738 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013328999999999999 2024-09-04T17:47:45.739 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.739 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.739 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:45.739 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205121+0000", 2024-09-04T17:47:45.739 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00022614999999999999 2024-09-04T17:47:45.739 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.739 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:45.739 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.739 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.739 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.739 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8832 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2490368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:45.739 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:47:45.739 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 92.675053259999999, 2024-09-04T17:47:45.739 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14770012299999999, 2024-09-04T17:47:45.740 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:45.740 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:45.740 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:45.740 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:45.740 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:45.740 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:45.740 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8832 2024-09-04T17:47:45.740 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.740 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:45.740 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.740 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:45.740 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:47:45.741 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.741 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.741 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.741 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:45.741 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:47:45.741 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.741 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.741 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.741 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:45.741 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057849+0000", 2024-09-04T17:47:45.741 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6700000000000003e-07 2024-09-04T17:47:45.741 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.741 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.742 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:45.742 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057852+0000", 2024-09-04T17:47:45.742 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6359999999999998e-06 2024-09-04T17:47:45.742 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.742 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.742 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:45.742 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057860+0000", 2024-09-04T17:47:45.742 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8140000000000007e-06 2024-09-04T17:47:45.742 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.742 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.742 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:45.742 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057868+0000", 2024-09-04T17:47:45.743 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1999999999999997e-06 2024-09-04T17:47:45.743 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.743 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.743 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:45.743 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205208+0000", 2024-09-04T17:47:45.743 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14734067300000001 2024-09-04T17:47:45.743 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.743 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.743 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:45.743 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205347+0000", 2024-09-04T17:47:45.743 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013914900000000001 2024-09-04T17:47:45.743 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.743 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.744 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:45.744 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205548+0000", 2024-09-04T17:47:45.744 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000200984 2024-09-04T17:47:45.744 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.744 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:45.744 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.744 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.744 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.744 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8833 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2527232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:45.744 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:47:45.744 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 92.675021389999998, 2024-09-04T17:47:45.744 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148062572, 2024-09-04T17:47:45.744 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:45.744 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:45.745 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:45.745 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:45.745 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:45.745 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:45.745 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8833 2024-09-04T17:47:45.745 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.745 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:45.745 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.745 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:45.745 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:47:45.745 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.745 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.745 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.746 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:45.746 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:47:45.746 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.746 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.746 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.746 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:45.746 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057881+0000", 2024-09-04T17:47:45.746 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4300000000000002e-07 2024-09-04T17:47:45.746 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.746 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.746 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:45.746 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057893+0000", 2024-09-04T17:47:45.746 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1973999999999999e-05 2024-09-04T17:47:45.746 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.747 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.747 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:45.747 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057906+0000", 2024-09-04T17:47:45.747 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2716e-05 2024-09-04T17:47:45.747 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.747 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.747 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:45.747 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057913+0000", 2024-09-04T17:47:45.747 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3930000000000002e-06 2024-09-04T17:47:45.747 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.747 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.747 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:45.747 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205621+0000", 2024-09-04T17:47:45.747 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14770788800000001 2024-09-04T17:47:45.748 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.748 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.748 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:45.748 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205740+0000", 2024-09-04T17:47:45.748 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000118936 2024-09-04T17:47:45.748 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.748 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.748 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:45.748 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205943+0000", 2024-09-04T17:47:45.748 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020292199999999999 2024-09-04T17:47:45.748 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.748 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:45.748 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.748 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.749 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.749 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8834 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2564096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:45.749 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:47:45.749 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 92.674978111000001, 2024-09-04T17:47:45.749 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14846753200000001, 2024-09-04T17:47:45.749 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:45.749 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:45.749 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:45.749 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:45.749 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:45.749 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:45.749 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8834 2024-09-04T17:47:45.750 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.750 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:45.750 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.750 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:45.750 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:47:45.750 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.750 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.750 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.750 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:45.750 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:47:45.750 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.750 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.750 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.750 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:45.751 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057925+0000", 2024-09-04T17:47:45.751 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0270000000000001e-06 2024-09-04T17:47:45.751 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.751 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.751 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:45.751 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057936+0000", 2024-09-04T17:47:45.751 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.112e-05 2024-09-04T17:47:45.751 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.751 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.751 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:45.751 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057946+0000", 2024-09-04T17:47:45.751 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.009e-05 2024-09-04T17:47:45.751 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.752 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.752 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:45.752 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057953+0000", 2024-09-04T17:47:45.752 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.6229999999999997e-06 2024-09-04T17:47:45.752 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.752 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.752 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:45.752 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206027+0000", 2024-09-04T17:47:45.752 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148073591 2024-09-04T17:47:45.752 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.752 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.752 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:45.753 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206161+0000", 2024-09-04T17:47:45.753 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013411599999999999 2024-09-04T17:47:45.753 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.753 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.753 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:45.753 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206391+0000", 2024-09-04T17:47:45.753 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000229965 2024-09-04T17:47:45.753 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.753 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:45.753 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.753 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.753 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.753 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8835 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2600960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:45.754 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:47:45.754 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 92.674938460999996, 2024-09-04T17:47:45.754 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14882941599999999, 2024-09-04T17:47:45.754 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:45.754 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:45.754 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:45.754 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:45.754 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:45.754 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:45.754 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8835 2024-09-04T17:47:45.754 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.754 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:45.754 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.755 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:45.755 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:47:45.755 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.755 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.755 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.755 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:45.755 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:47:45.755 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.755 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.755 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.755 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:45.755 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058544+0000", 2024-09-04T17:47:45.755 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00058061899999999999 2024-09-04T17:47:45.755 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.756 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.756 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:45.756 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058550+0000", 2024-09-04T17:47:45.756 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.1129999999999997e-06 2024-09-04T17:47:45.756 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.756 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.756 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:45.756 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058566+0000", 2024-09-04T17:47:45.756 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5820000000000001e-05 2024-09-04T17:47:45.756 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.756 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.756 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:45.756 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058578+0000", 2024-09-04T17:47:45.757 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2537e-05 2024-09-04T17:47:45.757 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.757 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.757 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:45.757 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206470+0000", 2024-09-04T17:47:45.757 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14789153799999999 2024-09-04T17:47:45.757 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.757 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.757 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:45.757 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206589+0000", 2024-09-04T17:47:45.757 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000118837 2024-09-04T17:47:45.757 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.757 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.757 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:45.758 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206793+0000", 2024-09-04T17:47:45.758 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000203952 2024-09-04T17:47:45.758 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.758 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:45.758 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.758 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.758 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.758 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8836 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2654208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:45.758 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:47:45.758 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 92.674305773, 2024-09-04T17:47:45.758 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14862433999999999, 2024-09-04T17:47:45.758 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:45.758 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:45.759 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:45.759 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:45.759 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:45.759 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:45.759 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8836 2024-09-04T17:47:45.759 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.759 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:45.759 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.759 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:45.759 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:47:45.759 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.759 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.759 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.759 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:45.760 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:47:45.760 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.760 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.760 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.760 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:45.760 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062663+0000", 2024-09-04T17:47:45.760 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0040670619999999998 2024-09-04T17:47:45.760 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.760 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.760 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:45.760 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062668+0000", 2024-09-04T17:47:45.760 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4199999999999998e-06 2024-09-04T17:47:45.760 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.760 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.761 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:45.761 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062689+0000", 2024-09-04T17:47:45.761 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0356999999999998e-05 2024-09-04T17:47:45.761 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.761 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.761 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:45.761 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062726+0000", 2024-09-04T17:47:45.761 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7302999999999998e-05 2024-09-04T17:47:45.761 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.761 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.761 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:45.761 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206875+0000", 2024-09-04T17:47:45.761 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14414883100000001 2024-09-04T17:47:45.761 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.762 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.762 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:45.762 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.207016+0000", 2024-09-04T17:47:45.762 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000140822 2024-09-04T17:47:45.762 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.762 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.762 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:45.762 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.207220+0000", 2024-09-04T17:47:45.762 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020454499999999999 2024-09-04T17:47:45.762 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.762 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:45.762 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.762 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.763 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.763 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8844 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2928640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:45.763 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:47:45.763 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 92.669941690000002, 2024-09-04T17:47:45.763 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14752153100000001, 2024-09-04T17:47:45.763 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:45.763 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:45.763 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:45.763 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:45.763 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:45.763 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:45.763 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8844 2024-09-04T17:47:45.763 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.764 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:45.764 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.764 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:45.764 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:47:45.764 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.764 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.764 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.764 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:45.764 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:47:45.764 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.764 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.764 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.764 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:45.765 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062961+0000", 2024-09-04T17:47:45.765 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6700000000000003e-07 2024-09-04T17:47:45.765 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.765 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.765 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:45.765 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062973+0000", 2024-09-04T17:47:45.765 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2109999999999999e-05 2024-09-04T17:47:45.765 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.765 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.765 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:45.765 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062982+0000", 2024-09-04T17:47:45.765 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.6900000000000004e-06 2024-09-04T17:47:45.765 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.765 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.766 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:45.766 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062990+0000", 2024-09-04T17:47:45.766 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1899999999999998e-06 2024-09-04T17:47:45.766 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.766 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.766 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:45.766 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210090+0000", 2024-09-04T17:47:45.766 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14710026300000001 2024-09-04T17:47:45.766 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.766 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.766 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:45.766 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210225+0000", 2024-09-04T17:47:45.766 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013532400000000001 2024-09-04T17:47:45.766 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.767 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.767 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:45.767 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210481+0000", 2024-09-04T17:47:45.767 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00025628700000000001 2024-09-04T17:47:45.767 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.767 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:45.767 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.767 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.767 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.767 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8845 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2965504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:45.767 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:47:45.767 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 92.669901967000001, 2024-09-04T17:47:45.767 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147916819, 2024-09-04T17:47:45.767 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:45.768 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:45.768 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:45.768 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:45.768 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:45.768 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:45.768 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8845 2024-09-04T17:47:45.768 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.768 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:45.768 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.768 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:45.768 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:47:45.768 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.768 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.768 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.769 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:45.769 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:47:45.769 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.769 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.769 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.769 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:45.769 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:47:45.769 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.0399999999999995e-07 2024-09-04T17:47:45.769 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.769 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.769 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:45.769 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063003+0000", 2024-09-04T17:47:45.770 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6529999999999998e-06 2024-09-04T17:47:45.770 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.770 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.770 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:45.770 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063012+0000", 2024-09-04T17:47:45.770 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9500000000000007e-06 2024-09-04T17:47:45.770 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.770 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.770 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:45.770 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063027+0000", 2024-09-04T17:47:45.770 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5443000000000001e-05 2024-09-04T17:47:45.770 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.770 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.771 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:45.771 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210571+0000", 2024-09-04T17:47:45.771 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14754353100000001 2024-09-04T17:47:45.771 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.771 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.771 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:45.771 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210714+0000", 2024-09-04T17:47:45.771 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014295599999999999 2024-09-04T17:47:45.771 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.771 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.771 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:45.771 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210916+0000", 2024-09-04T17:47:45.771 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020258200000000001 2024-09-04T17:47:45.772 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.772 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:45.772 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.772 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.772 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.772 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8846 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3002368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:45.772 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:47:45.772 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 92.669864430000004, 2024-09-04T17:47:45.772 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14828011599999999, 2024-09-04T17:47:45.772 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:45.772 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:45.772 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:45.772 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:45.773 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:45.773 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:45.773 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8846 2024-09-04T17:47:45.773 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.773 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:45.773 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.773 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:45.773 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:47:45.773 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.773 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.773 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.773 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:45.773 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:47:45.774 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.774 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.774 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.774 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:45.774 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063038+0000", 2024-09-04T17:47:45.774 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.23e-07 2024-09-04T17:47:45.774 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.774 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.774 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:45.774 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063051+0000", 2024-09-04T17:47:45.774 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2757e-05 2024-09-04T17:47:45.774 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.774 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.775 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:45.775 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063070+0000", 2024-09-04T17:47:45.775 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9663000000000001e-05 2024-09-04T17:47:45.775 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.775 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.775 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:45.775 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063079+0000", 2024-09-04T17:47:45.775 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.0929999999999992e-06 2024-09-04T17:47:45.775 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.775 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.775 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:45.775 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210989+0000", 2024-09-04T17:47:45.775 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14790984500000001 2024-09-04T17:47:45.775 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.776 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.776 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:45.776 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211109+0000", 2024-09-04T17:47:45.776 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000120056 2024-09-04T17:47:45.776 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.776 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.776 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:45.776 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211317+0000", 2024-09-04T17:47:45.776 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020797900000000001 2024-09-04T17:47:45.776 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.776 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:45.776 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.776 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.777 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.777 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8847 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3039232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:45.777 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:47:45.777 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 92.669789997999999, 2024-09-04T17:47:45.777 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14859408900000001, 2024-09-04T17:47:45.777 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:45.777 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:45.777 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:45.777 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:45.777 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:45.777 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:45.777 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8847 2024-09-04T17:47:45.777 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.778 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:45.778 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.778 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:45.778 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:47:45.778 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.778 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.778 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.778 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:45.778 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:47:45.778 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.778 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.778 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.778 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:45.779 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063113+0000", 2024-09-04T17:47:45.779 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.204e-06 2024-09-04T17:47:45.779 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.779 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.779 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:45.779 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063116+0000", 2024-09-04T17:47:45.779 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.3330000000000001e-06 2024-09-04T17:47:45.779 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.779 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.779 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:45.779 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063126+0000", 2024-09-04T17:47:45.779 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0166999999999999e-05 2024-09-04T17:47:45.779 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.780 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.780 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:45.780 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063134+0000", 2024-09-04T17:47:45.780 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4329999999999998e-06 2024-09-04T17:47:45.780 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.780 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.780 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:45.780 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211392+0000", 2024-09-04T17:47:45.780 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148258109 2024-09-04T17:47:45.780 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.780 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.780 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:45.780 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211506+0000", 2024-09-04T17:47:45.781 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000113813 2024-09-04T17:47:45.781 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.781 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.781 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:45.781 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211706+0000", 2024-09-04T17:47:45.781 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020002999999999999 2024-09-04T17:47:45.781 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.781 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:45.781 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.781 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.781 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.781 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8848 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:45.781 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:47:45.782 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 92.669757661000006, 2024-09-04T17:47:45.782 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14907967599999999, 2024-09-04T17:47:45.782 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:45.782 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:45.782 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:45.782 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:45.782 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:45.782 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:45.782 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8848 2024-09-04T17:47:45.782 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.782 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:45.782 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.782 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:45.782 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:47:45.783 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.783 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.783 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.783 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:45.783 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:47:45.783 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.783 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.783 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.783 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:45.783 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070589+0000", 2024-09-04T17:47:45.783 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.007445386 2024-09-04T17:47:45.783 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.783 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.784 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:45.784 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070600+0000", 2024-09-04T17:47:45.784 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0346999999999999e-05 2024-09-04T17:47:45.784 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.784 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.784 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:45.784 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070622+0000", 2024-09-04T17:47:45.784 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2289999999999998e-05 2024-09-04T17:47:45.784 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.784 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.784 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:45.784 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070636+0000", 2024-09-04T17:47:45.784 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4093e-05 2024-09-04T17:47:45.785 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.785 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.785 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:45.785 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211815+0000", 2024-09-04T17:47:45.785 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14117874699999999 2024-09-04T17:47:45.785 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.785 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.785 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:45.785 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211983+0000", 2024-09-04T17:47:45.785 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00016852800000000001 2024-09-04T17:47:45.785 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.785 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.785 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:45.785 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.212224+0000", 2024-09-04T17:47:45.786 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024028499999999999 2024-09-04T17:47:45.786 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.786 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:45.786 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.786 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.786 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.786 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8862 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2019328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:45.786 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:47:45.786 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 92.661708192999996, 2024-09-04T17:47:45.786 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14755236399999999, 2024-09-04T17:47:45.786 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:45.786 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:45.786 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:45.787 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:45.787 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:45.787 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:45.787 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8862 2024-09-04T17:47:45.787 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.787 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:45.787 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.787 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:45.787 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:47:45.787 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.787 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.787 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.787 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:45.787 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:47:45.788 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.788 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.788 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.788 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:45.788 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071194+0000", 2024-09-04T17:47:45.788 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.5000000000000002e-07 2024-09-04T17:47:45.788 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.788 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.788 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:45.788 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071197+0000", 2024-09-04T17:47:45.788 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5900000000000002e-06 2024-09-04T17:47:45.788 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.788 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.789 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:45.789 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071206+0000", 2024-09-04T17:47:45.789 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9700000000000005e-06 2024-09-04T17:47:45.789 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.789 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.789 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:45.789 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071213+0000", 2024-09-04T17:47:45.789 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.9070000000000001e-06 2024-09-04T17:47:45.789 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.789 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.789 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:45.789 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218396+0000", 2024-09-04T17:47:45.789 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147183273 2024-09-04T17:47:45.789 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.790 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.790 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:45.790 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218538+0000", 2024-09-04T17:47:45.790 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014194199999999999 2024-09-04T17:47:45.790 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.790 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.790 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:45.790 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218746+0000", 2024-09-04T17:47:45.790 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000208032 2024-09-04T17:47:45.790 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.790 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:45.790 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.790 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.790 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.791 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8863 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2056192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:45.791 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:47:45.791 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 92.661679883000005, 2024-09-04T17:47:45.791 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14792281099999999, 2024-09-04T17:47:45.791 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:45.791 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:45.791 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:45.791 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:45.791 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:45.791 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:45.791 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8863 2024-09-04T17:47:45.791 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.791 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:45.792 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.792 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:45.792 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:47:45.792 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.792 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.792 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.792 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:45.792 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:47:45.792 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.792 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.792 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.792 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:45.792 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:47:45.793 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1299999999999999e-07 2024-09-04T17:47:45.793 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.793 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.793 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:45.793 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071225+0000", 2024-09-04T17:47:45.793 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6199999999999999e-06 2024-09-04T17:47:45.793 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.793 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.793 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:45.793 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071234+0000", 2024-09-04T17:47:45.793 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8769999999999999e-06 2024-09-04T17:47:45.793 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.793 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.793 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:45.794 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071251+0000", 2024-09-04T17:47:45.794 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.738e-05 2024-09-04T17:47:45.794 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.794 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.794 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:45.794 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218798+0000", 2024-09-04T17:47:45.794 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14754645 2024-09-04T17:47:45.794 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.794 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.794 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:45.794 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218926+0000", 2024-09-04T17:47:45.794 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000128446 2024-09-04T17:47:45.794 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.795 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.795 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:45.795 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219145+0000", 2024-09-04T17:47:45.795 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021832500000000001 2024-09-04T17:47:45.795 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.795 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:45.795 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.795 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.795 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.795 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8864 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2093056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:45.795 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:47:45.795 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 92.661638307000004, 2024-09-04T17:47:45.795 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148277309, 2024-09-04T17:47:45.795 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:45.796 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:45.796 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:45.796 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:45.796 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:45.796 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:45.796 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8864 2024-09-04T17:47:45.796 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.796 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:45.796 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.796 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:45.796 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:47:45.796 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.796 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.797 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.797 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:45.797 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:47:45.797 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.797 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.797 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.797 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:45.797 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071264+0000", 2024-09-04T17:47:45.797 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3e-07 2024-09-04T17:47:45.797 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.797 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.797 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:45.797 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071267+0000", 2024-09-04T17:47:45.798 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0139999999999999e-06 2024-09-04T17:47:45.798 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.798 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.798 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:45.798 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071276+0000", 2024-09-04T17:47:45.798 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1689999999999998e-06 2024-09-04T17:47:45.798 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.798 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.798 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:45.798 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071283+0000", 2024-09-04T17:47:45.798 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2269999999999996e-06 2024-09-04T17:47:45.798 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.799 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.799 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:45.799 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219195+0000", 2024-09-04T17:47:45.799 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147911868 2024-09-04T17:47:45.799 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.799 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.799 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:45.799 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219324+0000", 2024-09-04T17:47:45.799 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00012835900000000001 2024-09-04T17:47:45.799 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.799 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.799 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:45.799 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219541+0000", 2024-09-04T17:47:45.799 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000216942 2024-09-04T17:47:45.800 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.800 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:45.800 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.800 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.800 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.800 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8865 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2109440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:45.800 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:47:45.800 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 92.661608846999997, 2024-09-04T17:47:45.800 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14865747300000001, 2024-09-04T17:47:45.800 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:45.800 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:45.800 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:45.800 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:45.801 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:45.801 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:45.801 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8865 2024-09-04T17:47:45.801 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.801 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:45.801 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.801 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:45.801 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:47:45.801 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.801 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.801 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.801 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:45.801 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:47:45.802 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.802 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.802 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.802 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:45.802 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:47:45.802 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7999999999999995e-07 2024-09-04T17:47:45.802 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.802 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.802 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:45.802 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071296+0000", 2024-09-04T17:47:45.802 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6900000000000001e-06 2024-09-04T17:47:45.802 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.802 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.802 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:45.803 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071305+0000", 2024-09-04T17:47:45.803 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.5499999999999995e-06 2024-09-04T17:47:45.803 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.803 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.803 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:45.803 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071312+0000", 2024-09-04T17:47:45.803 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1559999999999998e-06 2024-09-04T17:47:45.803 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.803 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.803 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:45.803 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219591+0000", 2024-09-04T17:47:45.803 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14827944900000001 2024-09-04T17:47:45.803 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.803 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.804 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:45.804 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219731+0000", 2024-09-04T17:47:45.804 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013931600000000001 2024-09-04T17:47:45.804 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.804 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.804 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:45.804 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219950+0000", 2024-09-04T17:47:45.804 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021963199999999999 2024-09-04T17:47:45.804 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.804 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:45.804 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.804 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.805 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.805 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8866 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2146304~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:45.805 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:47:45.805 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 92.661580614000002, 2024-09-04T17:47:45.805 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14906117899999999, 2024-09-04T17:47:45.805 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:45.805 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:45.805 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:45.805 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:45.805 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:45.805 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:45.805 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8866 2024-09-04T17:47:45.805 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.805 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:45.806 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.806 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:45.806 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:47:45.806 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.806 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.806 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.806 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:45.806 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:47:45.806 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.806 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.806 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.806 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:45.806 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071322+0000", 2024-09-04T17:47:45.807 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7999999999999995e-07 2024-09-04T17:47:45.807 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.807 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.807 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:45.807 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071338+0000", 2024-09-04T17:47:45.807 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5872999999999999e-05 2024-09-04T17:47:45.807 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.807 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.807 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:45.807 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071347+0000", 2024-09-04T17:47:45.807 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4669999999999993e-06 2024-09-04T17:47:45.807 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.807 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.807 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:45.808 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071354+0000", 2024-09-04T17:47:45.808 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2099999999999996e-06 2024-09-04T17:47:45.808 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.808 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.808 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:45.808 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220008+0000", 2024-09-04T17:47:45.808 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148653642 2024-09-04T17:47:45.808 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.808 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.808 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:45.808 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220155+0000", 2024-09-04T17:47:45.808 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014726600000000001 2024-09-04T17:47:45.809 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.809 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.809 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:45.809 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220382+0000", 2024-09-04T17:47:45.809 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00022704100000000001 2024-09-04T17:47:45.809 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.809 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:45.809 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.809 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.809 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.809 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8867 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2183168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:45.809 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:47:45.809 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 92.661537261000007, 2024-09-04T17:47:45.810 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.149445831, 2024-09-04T17:47:45.810 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:45.810 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:45.810 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:45.810 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:45.810 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:45.810 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:45.810 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8867 2024-09-04T17:47:45.810 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.810 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:45.810 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.810 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:45.810 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:47:45.810 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.811 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.811 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.811 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:45.811 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:47:45.811 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.811 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.811 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.811 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:45.811 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078653+0000", 2024-09-04T17:47:45.811 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0072890430000000003 2024-09-04T17:47:45.811 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.811 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.811 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:45.812 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078664+0000", 2024-09-04T17:47:45.812 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0307e-05 2024-09-04T17:47:45.812 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.812 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.812 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:45.812 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078690+0000", 2024-09-04T17:47:45.812 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6780000000000001e-05 2024-09-04T17:47:45.812 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.812 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.812 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:45.812 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078706+0000", 2024-09-04T17:47:45.812 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5676999999999999e-05 2024-09-04T17:47:45.812 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.813 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.813 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:45.813 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220467+0000", 2024-09-04T17:47:45.813 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14176103500000001 2024-09-04T17:47:45.813 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.813 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.813 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:45.813 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220595+0000", 2024-09-04T17:47:45.813 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000127427 2024-09-04T17:47:45.813 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.813 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.813 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:45.813 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220810+0000", 2024-09-04T17:47:45.813 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021556200000000001 2024-09-04T17:47:45.814 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.814 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:45.814 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.814 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.814 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.814 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8880 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2658304~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:45.814 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.079158+0000", 2024-09-04T17:47:45.814 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 92.653743981999995, 2024-09-04T17:47:45.814 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14718968499999999, 2024-09-04T17:47:45.814 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:45.814 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:45.814 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:45.814 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:45.815 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:45.815 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:45.815 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8880 2024-09-04T17:47:45.815 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.815 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:45.815 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.815 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:45.815 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079158+0000", 2024-09-04T17:47:45.815 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.815 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.815 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.815 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:45.815 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079158+0000", 2024-09-04T17:47:45.815 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.816 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.816 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.816 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:45.816 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079158+0000", 2024-09-04T17:47:45.816 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3e-07 2024-09-04T17:47:45.816 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.816 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.816 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:45.816 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079161+0000", 2024-09-04T17:47:45.816 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.57e-06 2024-09-04T17:47:45.816 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.816 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.816 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:45.817 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079170+0000", 2024-09-04T17:47:45.817 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.0170000000000002e-06 2024-09-04T17:47:45.817 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.817 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.817 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:45.817 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079177+0000", 2024-09-04T17:47:45.817 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.0530000000000001e-06 2024-09-04T17:47:45.817 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.817 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.817 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:45.817 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.225946+0000", 2024-09-04T17:47:45.817 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14676936500000001 2024-09-04T17:47:45.817 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.817 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.818 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:45.818 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226098+0000", 2024-09-04T17:47:45.818 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000151805 2024-09-04T17:47:45.818 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.818 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.818 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:45.818 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226347+0000", 2024-09-04T17:47:45.818 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024914499999999999 2024-09-04T17:47:45.818 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.818 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:45.818 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.818 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.818 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.819 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8881 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2695168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:45.819 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:47:45.819 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 92.653715191999993, 2024-09-04T17:47:45.819 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147543547, 2024-09-04T17:47:45.819 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:45.819 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:45.819 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:45.819 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:45.819 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:45.819 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:45.819 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8881 2024-09-04T17:47:45.819 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.819 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:45.819 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.820 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:45.820 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:47:45.820 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.820 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.820 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.820 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:45.820 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:47:45.820 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.820 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.820 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.820 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:45.820 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079187+0000", 2024-09-04T17:47:45.821 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7700000000000004e-07 2024-09-04T17:47:45.821 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.821 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.821 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:45.821 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079198+0000", 2024-09-04T17:47:45.821 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0849999999999999e-05 2024-09-04T17:47:45.821 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.821 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.821 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:45.821 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079207+0000", 2024-09-04T17:47:45.821 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5599999999999999e-06 2024-09-04T17:47:45.821 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.821 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.821 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:45.822 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079224+0000", 2024-09-04T17:47:45.822 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6387000000000001e-05 2024-09-04T17:47:45.822 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.822 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.822 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:45.822 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226414+0000", 2024-09-04T17:47:45.822 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14718975000000001 2024-09-04T17:47:45.822 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.822 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.822 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:45.822 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226534+0000", 2024-09-04T17:47:45.822 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000120288 2024-09-04T17:47:45.822 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.823 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.823 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:45.823 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226730+0000", 2024-09-04T17:47:45.823 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000196035 2024-09-04T17:47:45.823 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.823 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:45.823 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.823 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.823 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.823 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8882 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2732032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:45.823 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:47:45.823 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 92.653666978000004, 2024-09-04T17:47:45.823 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14795022399999999, 2024-09-04T17:47:45.823 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:45.824 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:45.824 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:45.824 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:45.824 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:45.824 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:45.824 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8882 2024-09-04T17:47:45.824 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.824 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:45.824 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.824 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:45.824 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:47:45.824 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.824 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.824 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.825 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:45.825 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:47:45.825 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:45.825 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.825 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.825 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:45.825 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086588+0000", 2024-09-04T17:47:45.825 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0073530959999999999 2024-09-04T17:47:45.825 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.825 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.825 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:45.825 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086598+0000", 2024-09-04T17:47:45.825 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0122999999999999e-05 2024-09-04T17:47:45.826 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.826 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.826 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:45.826 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086626+0000", 2024-09-04T17:47:45.826 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.7914000000000001e-05 2024-09-04T17:47:45.826 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.826 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.826 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:45.826 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086641+0000", 2024-09-04T17:47:45.826 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5270000000000001e-05 2024-09-04T17:47:45.826 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.826 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.826 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:45.826 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226801+0000", 2024-09-04T17:47:45.827 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.140160321 2024-09-04T17:47:45.827 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.827 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.827 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:45.827 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226951+0000", 2024-09-04T17:47:45.827 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014965200000000001 2024-09-04T17:47:45.827 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:45.827 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:45.827 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:45.827 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.227185+0000", 2024-09-04T17:47:45.827 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00023384799999999999 2024-09-04T17:47:45.827 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.827 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:45.827 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.828 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:45.828 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:45.828 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:45.828 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-04T17:47:45.830 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-04T17:47:45.830 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:45.827+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:45.930 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:45.927+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:45.967 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-04T17:47:45.994 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:47:45.994 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:47:46.005 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:46.005 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:47:46.005 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:47:46.005 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:46.013 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:47:46.014 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:47:46.021 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-04T17:47:46.032 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-04T17:47:46.033 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-04T17:47:46.050 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-04T17:47:46.050 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-04T17:47:46.058 INFO:teuthology.orchestra.run.smithi092.stdout:leaving image image_client.0-clone36 intact 2024-09-04T17:47:46.065 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:46.063+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:46.076 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:47:46.077 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:47:46.098 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:47:46.099 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:47:46.103 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-04T17:47:46.131 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:46.128+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:46.138 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:47:46.138 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:47:46.142 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-04T17:47:46.143 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-04T17:47:46.193 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:46.193 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:46.209 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-04T17:47:46.221 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #35, image image_client.0-clone35 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent36 2024-09-04T17:47:46.232 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:46.228+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:46.332 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:46.333 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:46.333 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:46.327+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:46.347 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-04T17:47:46.433 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:47:46.431+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:47:46.473 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:46.473 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:46.489 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-04T17:47:46.537 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:46.536+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:46.615 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:46.615 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:46.619 INFO:teuthology.orchestra.run.smithi092.stdout:5411 writesame 0x498796d thru 0x498f59b data_size 0x1(0x7c2f bytes) 2024-09-04T17:47:46.631 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-04T17:47:46.677 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:46.676+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:46.684 INFO:teuthology.orchestra.run.smithi092.stdout:5412 punch from 0x5d0da33 to 0x5d16451, (0x8a1e bytes) 2024-09-04T17:47:46.684 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:46.685 INFO:teuthology.orchestra.run.smithi092.stdout:5415 writesame 0x59d7295 thru 0x59d970e data_size 0x7(0x247a bytes) 2024-09-04T17:47:46.712 INFO:teuthology.orchestra.run.smithi092.stdout:5416 read 0x726690c thru 0x7274971 (0xe066 bytes) 2024-09-04T17:47:46.712 INFO:teuthology.orchestra.run.smithi092.stdout:5417 write 0x6df4fc2 thru 0x6df78f3 (0x2932 bytes) 2024-09-04T17:47:46.727 INFO:teuthology.orchestra.run.smithi092.stdout:5418 write 0x1c81b0e thru 0x1c9165c (0xfb4f bytes) 2024-09-04T17:47:46.735 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:46.731+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:46.782 INFO:teuthology.orchestra.run.smithi092.stdout:5419 read 0x6788195 thru 0x678864a (0x4b6 bytes) 2024-09-04T17:47:46.782 INFO:teuthology.orchestra.run.smithi092.stdout:5420 writesame 0x17adb49 thru 0x17b5880 data_size 0x4(0x7d38 bytes) 2024-09-04T17:47:46.816 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:46.816 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:46.824 INFO:teuthology.orchestra.run.smithi092.stdout:5421 read 0x312cb05 thru 0x313808b (0xb587 bytes) 2024-09-04T17:47:46.830 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-04T17:47:46.836 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:47:46.831+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:47:46.841 INFO:teuthology.orchestra.run.smithi092.stdout:5424 writesame 0xa3aca70 thru 0xa3af1bf data_size 0x11(0x2750 bytes) 2024-09-04T17:47:46.853 INFO:teuthology.orchestra.run.smithi092.stdout:5425 write 0xb18b3a0 thru 0xb193385 (0x7fe6 bytes) 2024-09-04T17:47:46.863 INFO:teuthology.orchestra.run.smithi092.stdout:5428 punch from 0x764322c to 0x7651891, (0xe665 bytes) 2024-09-04T17:47:46.863 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:46.863 INFO:teuthology.orchestra.run.smithi092.stdout:5431 read 0x563a70 thru 0x56e69c (0xac2d bytes) 2024-09-04T17:47:46.864 INFO:teuthology.orchestra.run.smithi092.stdout:5433 read 0x95f8b7 thru 0x96a9b2 (0xb0fc bytes) 2024-09-04T17:47:46.866 INFO:teuthology.orchestra.run.smithi092.stdout:5436 write 0xabb5628 thru 0xabc26c4 (0xd09d bytes) 2024-09-04T17:47:46.884 INFO:teuthology.orchestra.run.smithi092.stdout:5438 trunc from 0xb193386 to 0x7393ac9 2024-09-04T17:47:46.937 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:47:46.936+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:47:46.964 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is set 2024-09-04T17:47:46.981 INFO:teuthology.orchestra.run.smithi092.stdout:5439 read 0x34839dc thru 0x348a5a7 (0x6bcc bytes) 2024-09-04T17:47:46.990 INFO:teuthology.orchestra.run.smithi092.stdout:5440 write 0x8c4bb2f thru 0x8c59366 (0xd838 bytes) 2024-09-04T17:47:46.993 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:46.993 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:46.995 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:46.995 INFO:teuthology.orchestra.run.smithi092.stdout:5442 read 0x18d54b1 thru 0x18dc45a (0x6faa bytes) 2024-09-04T17:47:47.008 INFO:teuthology.orchestra.run.smithi092.stdout:5443 writesame 0xb77ec74 thru 0xb785dbc data_size 0x7(0x7149 bytes) 2024-09-04T17:47:47.008 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:47.021 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-04T17:47:47.022 INFO:teuthology.orchestra.run.smithi092.stdout:5446 writesame 0xea6920f thru 0xea70e4f data_size 0x17(0x7c41 bytes) 2024-09-04T17:47:47.022 INFO:teuthology.orchestra.run.smithi092.stdout:5448 read 0x349990c thru 0x34a5d58 (0xc44d bytes) 2024-09-04T17:47:47.022 INFO:teuthology.orchestra.run.smithi092.stdout:5449 write 0xe267bd9 thru 0xe26829e (0x6c6 bytes) 2024-09-04T17:47:47.026 INFO:teuthology.orchestra.run.smithi092.stdout:5451 read 0x5b18ede thru 0x5b1c791 (0x38b4 bytes) 2024-09-04T17:47:47.027 INFO:teuthology.orchestra.run.smithi092.stdout:5453 read 0x783affd thru 0x783d8fc (0x2900 bytes) 2024-09-04T17:47:47.027 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:47.027 INFO:teuthology.orchestra.run.smithi092.stdout:5455 write 0xb5f34ee thru 0xb5fb3ef (0x7f02 bytes) 2024-09-04T17:47:47.033 INFO:teuthology.orchestra.run.smithi092.stdout:5456 write 0x84d7a2c thru 0x84decd9 (0x72ae bytes) 2024-09-04T17:47:47.035 INFO:teuthology.orchestra.run.smithi092.stdout:5457 write 0x2f134ec thru 0x2f161ff (0x2d14 bytes) 2024-09-04T17:47:47.038 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:47.036+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:47.045 INFO:teuthology.orchestra.run.smithi092.stdout:5458 writesame 0xe6400a1 thru 0xe64e72c data_size 0x82(0xe68c bytes) 2024-09-04T17:47:47.054 INFO:teuthology.orchestra.run.smithi092.stdout:5459 read 0x4ceb388 thru 0x4cf6230 (0xaea9 bytes) 2024-09-04T17:47:47.070 INFO:teuthology.orchestra.run.smithi092.stdout:5460 writesame 0xeca9945 thru 0xecb2701 data_size 0x7b(0x8dbd bytes) 2024-09-04T17:47:47.089 INFO:teuthology.orchestra.run.smithi092.stdout:5465 trunc from 0xecb2702 to 0x28c1328 2024-09-04T17:47:47.140 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:47.136+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:47.168 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:47.168 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:47.188 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-04T17:47:47.220 INFO:teuthology.orchestra.run.smithi092.stdout:5467 write 0xb04c562 thru 0xb053da0 (0x783f bytes) 2024-09-04T17:47:47.227 INFO:teuthology.orchestra.run.smithi092.stdout:5468 punch from 0xa00b7c2 to 0xa00e647, (0x2e85 bytes) 2024-09-04T17:47:47.231 INFO:teuthology.orchestra.run.smithi092.stdout:5469 trunc from 0xb053da1 to 0xe20bef3 2024-09-04T17:47:47.238 INFO:teuthology.orchestra.run.smithi092.stdout:5470 read 0xbc02bc1 thru 0xbc0afda (0x841a bytes) 2024-09-04T17:47:47.238 INFO:teuthology.orchestra.run.smithi092.stdout:5471 writesame 0xb586f91 thru 0xb58b1db data_size 0x1(0x424b bytes) 2024-09-04T17:47:47.241 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:47.239+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:47.272 INFO:teuthology.orchestra.run.smithi092.stdout:5473 trunc from 0xe20bef3 to 0x229270d 2024-09-04T17:47:47.298 INFO:teuthology.orchestra.run.smithi092.stdout:5474 punch from 0x19db64b to 0x19e9bc1, (0xe576 bytes) 2024-09-04T17:47:47.316 INFO:teuthology.orchestra.run.smithi092.stdout:5475 writesame 0x5ed0731 thru 0x5ed4d8a data_size 0x5(0x465a bytes) 2024-09-04T17:47:47.317 INFO:teuthology.orchestra.run.smithi092.stdout:5476 read 0x9e59e9 thru 0x9ef54e (0x9b66 bytes) 2024-09-04T17:47:47.317 INFO:teuthology.orchestra.run.smithi092.stdout:5477 read 0x1902244 thru 0x190e37c (0xc139 bytes) 2024-09-04T17:47:47.333 INFO:teuthology.orchestra.run.smithi092.stdout:5479 write 0x78a22e9 thru 0x78aa2cf (0x7fe7 bytes) 2024-09-04T17:47:47.341 INFO:teuthology.orchestra.run.smithi092.stdout:5480 read 0x2affaf6 thru 0x2b06a9b (0x6fa6 bytes) 2024-09-04T17:47:47.342 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:47.339+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:47.342 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:47.342 INFO:teuthology.orchestra.run.smithi092.stdout:5482 punch from 0x1896492 to 0x1898b40, (0x26ae bytes) 2024-09-04T17:47:47.342 INFO:teuthology.orchestra.run.smithi092.stdout:5483 read 0x12d17ae thru 0x12df5ab (0xddfe bytes) 2024-09-04T17:47:47.342 INFO:teuthology.orchestra.run.smithi092.stdout:5484 write 0x52e0293 thru 0x52ed5ff (0xd36d bytes) 2024-09-04T17:47:47.347 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:47.347 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:47.366 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-04T17:47:47.373 INFO:teuthology.orchestra.run.smithi092.stdout:5485 write 0xbb13c51 thru 0xbb204e5 (0xc895 bytes) 2024-09-04T17:47:47.398 INFO:teuthology.orchestra.run.smithi092.stdout:5486 trunc from 0xbb204e6 to 0x421f5e9 2024-09-04T17:47:47.422 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:47.428 INFO:teuthology.orchestra.run.smithi092.stdout:5488 trunc from 0x421f5e9 to 0x102cf2 2024-09-04T17:47:47.442 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:47.439+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:47.518 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:47.518 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:47.533 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-04T17:47:47.543 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:47.539+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:47.547 INFO:teuthology.orchestra.run.smithi092.stdout:5489 read 0xc69bb thru 0xcc787 (0x5dcd bytes) 2024-09-04T17:47:47.549 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:47.549 INFO:teuthology.orchestra.run.smithi092.stdout:5494 read 0x7f10d thru 0x8b514 (0xc408 bytes) 2024-09-04T17:47:47.550 INFO:teuthology.orchestra.run.smithi092.stdout:5495 trunc from 0x102cf2 to 0x78df93 2024-09-04T17:47:47.561 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:47.561 INFO:teuthology.orchestra.run.smithi092.stdout:5498 read 0x556079 thru 0x55ebfd (0x8b85 bytes) 2024-09-04T17:47:47.572 INFO:teuthology.orchestra.run.smithi092.stdout:5502 writesame 0xf3650d thru 0xf3ad4f data_size 0x1(0x4843 bytes) 2024-09-04T17:47:47.574 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:47.574 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:47.596 INFO:teuthology.orchestra.run.smithi092.stdout:5506 writesame 0x624d028 thru 0x6255f1a data_size 0xd(0x8ef3 bytes) 2024-09-04T17:47:47.596 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:47.596 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:47.596 INFO:teuthology.orchestra.run.smithi092.stdout:5513 punch from 0x129a97f to 0x12a6701, (0xbd82 bytes) 2024-09-04T17:47:47.605 INFO:teuthology.orchestra.run.smithi092.stdout:5514 trunc from 0x6255f1b to 0xbd332 2024-09-04T17:47:47.625 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:47.644 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:47.640+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:47.645 INFO:teuthology.orchestra.run.smithi092.stdout:5517 trunc from 0xbd332 to 0xd75fef3 2024-09-04T17:47:47.666 INFO:teuthology.orchestra.run.smithi092.stdout:5518 trunc from 0xd75fef3 to 0x8b2c1a4 2024-09-04T17:47:47.672 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:47.672 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:47.688 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-04T17:47:47.691 INFO:teuthology.orchestra.run.smithi092.stdout:5519 trunc from 0x8b2c1a4 to 0xd5e421f 2024-09-04T17:47:47.705 INFO:teuthology.orchestra.run.smithi092.stdout:5520 read 0x26996c0 thru 0x269a74d (0x108e bytes) 2024-09-04T17:47:47.717 INFO:teuthology.orchestra.run.smithi092.stdout:5521 trunc from 0xd5e421f to 0x5db07eb 2024-09-04T17:47:47.733 INFO:teuthology.orchestra.run.smithi092.stdout:5523 read 0x22c5b78 thru 0x22c7e6b (0x22f4 bytes) 2024-09-04T17:47:47.733 INFO:teuthology.orchestra.run.smithi092.stdout:5524 read 0x34d140 thru 0x350879 (0x373a bytes) 2024-09-04T17:47:47.733 INFO:teuthology.orchestra.run.smithi092.stdout:5526 writesame 0x33b4ece thru 0x33bac9f data_size 0x3(0x5dd2 bytes) 2024-09-04T17:47:47.745 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:47.743+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:47.748 INFO:teuthology.orchestra.run.smithi092.stdout:5527 punch from 0x352dbb5 to 0x353a7ae, (0xcbf9 bytes) 2024-09-04T17:47:47.748 INFO:teuthology.orchestra.run.smithi092.stdout:5528 punch from 0x34faa0b to 0x34fbe6d, (0x1462 bytes) 2024-09-04T17:47:47.774 INFO:teuthology.orchestra.run.smithi092.stdout:5531 write 0xd60791f thru 0xd6119da (0xa0bc bytes) 2024-09-04T17:47:47.785 INFO:teuthology.orchestra.run.smithi092.stdout:5532 writesame 0x4a908c4 thru 0x4a9fad4 data_size 0x1(0xf211 bytes) 2024-09-04T17:47:47.832 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:47.832 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:47.836 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:47.836 INFO:teuthology.orchestra.run.smithi092.stdout:5535 writesame 0x2f00fab thru 0x2f07333 data_size 0x1(0x6389 bytes) 2024-09-04T17:47:47.847 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-04T17:47:47.847 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:47:47.844+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:47:47.871 INFO:teuthology.orchestra.run.smithi092.stdout:5536 read 0x715349b thru 0x71546b1 (0x1217 bytes) 2024-09-04T17:47:47.871 INFO:teuthology.orchestra.run.smithi092.stdout:5537 punch from 0x995e39d to 0x9965a56, (0x76b9 bytes) 2024-09-04T17:47:47.871 INFO:teuthology.orchestra.run.smithi092.stdout:5538 writesame 0x571898d thru 0x57269e6 data_size 0x2f(0xe05a bytes) 2024-09-04T17:47:47.881 INFO:teuthology.orchestra.run.smithi092.stdout:5539 read 0x1220c95 thru 0x122f863 (0xebcf bytes) 2024-09-04T17:47:47.882 INFO:teuthology.orchestra.run.smithi092.stdout:5540 writesame 0xbbcc237 thru 0xbbce79b data_size 0x1(0x2565 bytes) 2024-09-04T17:47:47.907 INFO:teuthology.orchestra.run.smithi092.stdout:5541 write 0xeb47ea9 thru 0xeb4f4c6 (0x761e bytes) 2024-09-04T17:47:47.917 INFO:teuthology.orchestra.run.smithi092.stdout:5542 read 0x4ede2fb thru 0x4ee3473 (0x5179 bytes) 2024-09-04T17:47:47.917 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:47.917 INFO:teuthology.orchestra.run.smithi092.stdout:5545 read 0xbf33a10 thru 0xbf3ccb4 (0x92a5 bytes) 2024-09-04T17:47:47.917 INFO:teuthology.orchestra.run.smithi092.stdout:5546 write 0x3537200 thru 0x3547047 (0xfe48 bytes) 2024-09-04T17:47:47.930 INFO:teuthology.orchestra.run.smithi092.stdout:5547 read 0x3d33c4 thru 0x3d4087 (0xcc4 bytes) 2024-09-04T17:47:47.930 INFO:teuthology.orchestra.run.smithi092.stdout:5548 writesame 0xd7f378b thru 0xd7f7355 data_size 0x1(0x3bcb bytes) 2024-09-04T17:47:47.943 INFO:teuthology.orchestra.run.smithi092.stdout:5549 write 0xe055559 thru 0xe05d6cd (0x8175 bytes) 2024-09-04T17:47:47.947 INFO:teuthology.orchestra.run.smithi092.stdout:5550 read 0x68f65c9 thru 0x68fca46 (0x647e bytes) 2024-09-04T17:47:47.947 INFO:teuthology.orchestra.run.smithi092.stdout:5551 write 0x198156c thru 0x198baf8 (0xa58d bytes) 2024-09-04T17:47:47.947 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:47.944+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:47.951 INFO:teuthology.orchestra.run.smithi092.stdout:5552 write 0xa2f8d4c thru 0xa308c09 (0xfebe bytes) 2024-09-04T17:47:47.964 INFO:teuthology.orchestra.run.smithi092.stdout:5553 write 0x1ee960b thru 0x1eeede8 (0x57de bytes) 2024-09-04T17:47:47.994 INFO:teuthology.orchestra.run.smithi092.stdout:5555 trunc from 0xeb4f4c7 to 0x17f57c0 2024-09-04T17:47:47.994 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:47.994 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:48.014 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-04T17:47:48.049 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:48.047+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:48.065 INFO:teuthology.orchestra.run.smithi092.stdout:5556 writesame 0xea88f3b thru 0xea95350 data_size 0x13(0xc416 bytes) 2024-09-04T17:47:48.065 INFO:teuthology.orchestra.run.smithi092.stdout:5557 punch from 0x86e4d5b to 0x86edcb8, (0x8f5d bytes) 2024-09-04T17:47:48.065 INFO:teuthology.orchestra.run.smithi092.stdout:5558 trunc from 0xea95351 to 0xe7b00fa 2024-09-04T17:47:48.078 INFO:teuthology.orchestra.run.smithi092.stdout:5559 write 0x8684ba0 thru 0x8690f5f (0xc3c0 bytes) 2024-09-04T17:47:48.091 INFO:teuthology.orchestra.run.smithi092.stdout:5560 punch from 0xdd70713 to 0xdd7eb4c, (0xe439 bytes) 2024-09-04T17:47:48.091 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:48.091 INFO:teuthology.orchestra.run.smithi092.stdout:5562 punch from 0xa0d9124 to 0xa0e5b48, (0xca24 bytes) 2024-09-04T17:47:48.091 INFO:teuthology.orchestra.run.smithi092.stdout:5563 read 0x5d989d4 thru 0x5da077d (0x7daa bytes) 2024-09-04T17:47:48.091 INFO:teuthology.orchestra.run.smithi092.stdout:5565 write 0x526a972 thru 0x527096f (0x5ffe bytes) 2024-09-04T17:47:48.099 INFO:teuthology.orchestra.run.smithi092.stdout:5566 punch from 0xd0b0474 to 0xd0b1077, (0xc03 bytes) 2024-09-04T17:47:48.099 INFO:teuthology.orchestra.run.smithi092.stdout:5567 write 0x6d75b09 thru 0x6d840ef (0xe5e7 bytes) 2024-09-04T17:47:48.110 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:48.110 INFO:teuthology.orchestra.run.smithi092.stdout:5569 write 0x200c9f6 thru 0x201c7c0 (0xfdcb bytes) 2024-09-04T17:47:48.122 INFO:teuthology.orchestra.run.smithi092.stdout:5570 writesame 0x367e2b6 thru 0x3683ae0 data_size 0x1(0x582b bytes) 2024-09-04T17:47:48.144 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-04T17:47:48.152 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:48.152 INFO:teuthology.orchestra.run.smithi092.stdout:5574 clone 38 order 25 su 16777216 sc 3 2024-09-04T17:47:48.152 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:48.148+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:48.183 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:48.183 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:48.198 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-04T17:47:48.251 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:48.247+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:48.282 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:47:48.282 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:47:48.299 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:47:48.299 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:47:48.315 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-04T17:47:48.316 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-04T17:47:48.332 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-04T17:47:48.333 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-04T17:47:48.351 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:48.347+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:48.353 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:47:48.353 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:47:48.364 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:48.364 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:47:48.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:47:48.365 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:47:48.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.365 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59950 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5365760~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:47:48.365 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654917+0000", 2024-09-04T17:47:48.365 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 34.708404330999997, 2024-09-04T17:47:48.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066822450000000005, 2024-09-04T17:47:48.365 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:48.365 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:48.365 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:48.365 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:48.366 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:48.366 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:48.366 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59950 2024-09-04T17:47:48.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.366 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:48.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:48.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654917+0000", 2024-09-04T17:47:48.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:48.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:48.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654917+0000", 2024-09-04T17:47:48.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:48.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:48.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654918+0000", 2024-09-04T17:47:48.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.29e-07 2024-09-04T17:47:48.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:48.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654919+0000", 2024-09-04T17:47:48.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2589999999999999e-06 2024-09-04T17:47:48.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:48.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654922+0000", 2024-09-04T17:47:48.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1350000000000001e-06 2024-09-04T17:47:48.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:48.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654925+0000", 2024-09-04T17:47:48.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1250000000000001e-06 2024-09-04T17:47:48.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:48.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721466+0000", 2024-09-04T17:47:48.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066540959999999996 2024-09-04T17:47:48.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:48.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721574+0000", 2024-09-04T17:47:48.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010747399999999999 2024-09-04T17:47:48.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:48.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721740+0000", 2024-09-04T17:47:48.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016626799999999999 2024-09-04T17:47:48.369 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:48.370 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:48.370 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:48.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.370 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59951 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5472256~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:47:48.370 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654929+0000", 2024-09-04T17:47:48.370 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 34.708393145999999, 2024-09-04T17:47:48.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067110534999999999, 2024-09-04T17:47:48.370 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:48.370 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:48.370 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:48.370 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:48.370 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:48.370 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:48.371 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59951 2024-09-04T17:47:48.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.371 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:48.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:48.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654929+0000", 2024-09-04T17:47:48.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:48.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:48.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654929+0000", 2024-09-04T17:47:48.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:48.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:48.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654929+0000", 2024-09-04T17:47:48.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5100000000000001e-07 2024-09-04T17:47:48.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:48.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654930+0000", 2024-09-04T17:47:48.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2440000000000001e-06 2024-09-04T17:47:48.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:48.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654933+0000", 2024-09-04T17:47:48.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1599999999999998e-06 2024-09-04T17:47:48.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:48.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654936+0000", 2024-09-04T17:47:48.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6560000000000001e-06 2024-09-04T17:47:48.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:48.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721789+0000", 2024-09-04T17:47:48.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066853250000000003 2024-09-04T17:47:48.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:48.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721885+0000", 2024-09-04T17:47:48.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5616000000000001e-05 2024-09-04T17:47:48.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:48.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722039+0000", 2024-09-04T17:47:48.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000154258 2024-09-04T17:47:48.374 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:48.374 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:48.374 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:48.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.375 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59952 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5505024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:47:48.375 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654939+0000", 2024-09-04T17:47:48.375 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 34.708382268999998, 2024-09-04T17:47:48.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067433742000000005, 2024-09-04T17:47:48.375 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:48.375 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:48.375 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:48.375 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:48.375 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:48.375 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:48.376 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59952 2024-09-04T17:47:48.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.376 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:48.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:48.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654939+0000", 2024-09-04T17:47:48.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:48.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:48.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654939+0000", 2024-09-04T17:47:48.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:48.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:48.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654940+0000", 2024-09-04T17:47:48.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3300000000000001e-07 2024-09-04T17:47:48.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:48.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654941+0000", 2024-09-04T17:47:48.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1570000000000001e-06 2024-09-04T17:47:48.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:48.378 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654944+0000", 2024-09-04T17:47:48.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1319999999999998e-06 2024-09-04T17:47:48.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.378 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:48.378 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654947+0000", 2024-09-04T17:47:48.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5969999999999999e-06 2024-09-04T17:47:48.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.378 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:48.378 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722089+0000", 2024-09-04T17:47:48.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067142347000000005 2024-09-04T17:47:48.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:48.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722201+0000", 2024-09-04T17:47:48.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011235600000000001 2024-09-04T17:47:48.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:48.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722373+0000", 2024-09-04T17:47:48.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017191999999999999 2024-09-04T17:47:48.380 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:48.380 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:48.380 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:48.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.380 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59953 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5611520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:47:48.380 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654950+0000", 2024-09-04T17:47:48.380 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 34.708371628999998, 2024-09-04T17:47:48.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067601852000000004, 2024-09-04T17:47:48.381 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:48.381 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:48.381 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:48.381 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:48.381 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:48.381 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:48.381 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59953 2024-09-04T17:47:48.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.382 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:48.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:48.382 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654950+0000", 2024-09-04T17:47:48.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:48.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:48.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654950+0000", 2024-09-04T17:47:48.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:48.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:48.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654950+0000", 2024-09-04T17:47:48.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2499999999999999e-07 2024-09-04T17:47:48.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:48.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654954+0000", 2024-09-04T17:47:48.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.287e-06 2024-09-04T17:47:48.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:48.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654962+0000", 2024-09-04T17:47:48.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4230000000000005e-06 2024-09-04T17:47:48.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:48.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654966+0000", 2024-09-04T17:47:48.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6449999999999998e-06 2024-09-04T17:47:48.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:48.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722417+0000", 2024-09-04T17:47:48.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067451188999999995 2024-09-04T17:47:48.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:48.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722467+0000", 2024-09-04T17:47:48.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0556999999999997e-05 2024-09-04T17:47:48.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:48.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722552+0000", 2024-09-04T17:47:48.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4525999999999997e-05 2024-09-04T17:47:48.387 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:48.387 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:48.388 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:48.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.388 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59954 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5718016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:47:48.388 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654970+0000", 2024-09-04T17:47:48.388 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 34.708351868000001, 2024-09-04T17:47:48.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067874285000000006, 2024-09-04T17:47:48.389 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:48.389 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:48.389 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:48.389 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:48.389 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:48.389 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:48.389 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59954 2024-09-04T17:47:48.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.390 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:48.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:48.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654970+0000", 2024-09-04T17:47:48.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:48.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:48.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654970+0000", 2024-09-04T17:47:48.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:48.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:48.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654970+0000", 2024-09-04T17:47:48.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5899999999999998e-07 2024-09-04T17:47:48.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:48.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654971+0000", 2024-09-04T17:47:48.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2929999999999999e-06 2024-09-04T17:47:48.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:48.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654975+0000", 2024-09-04T17:47:48.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4699999999999998e-06 2024-09-04T17:47:48.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:48.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654978+0000", 2024-09-04T17:47:48.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7930000000000002e-06 2024-09-04T17:47:48.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:48.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722600+0000", 2024-09-04T17:47:48.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067622229000000006 2024-09-04T17:47:48.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:48.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722689+0000", 2024-09-04T17:47:48.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9511000000000004e-05 2024-09-04T17:47:48.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:48.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722844+0000", 2024-09-04T17:47:48.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015473 2024-09-04T17:47:48.395 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:48.395 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:48.395 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:48.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.396 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59955 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5857280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:47:48.396 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654981+0000", 2024-09-04T17:47:48.396 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 34.708340518, 2024-09-04T17:47:48.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068052360000000006, 2024-09-04T17:47:48.396 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:48.396 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:48.396 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:48.396 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:48.396 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:48.396 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:48.396 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59955 2024-09-04T17:47:48.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.397 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:48.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:48.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654981+0000", 2024-09-04T17:47:48.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:48.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:48.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654981+0000", 2024-09-04T17:47:48.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:48.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:48.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654981+0000", 2024-09-04T17:47:48.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2399999999999999e-07 2024-09-04T17:47:48.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:48.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654985+0000", 2024-09-04T17:47:48.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0980000000000004e-06 2024-09-04T17:47:48.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:48.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654990+0000", 2024-09-04T17:47:48.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3429999999999998e-06 2024-09-04T17:47:48.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:48.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654993+0000", 2024-09-04T17:47:48.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8700000000000001e-06 2024-09-04T17:47:48.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:48.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722888+0000", 2024-09-04T17:47:48.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067895239999999996 2024-09-04T17:47:48.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:48.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722948+0000", 2024-09-04T17:47:48.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9750999999999998e-05 2024-09-04T17:47:48.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:48.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.723034+0000", 2024-09-04T17:47:48.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5834000000000005e-05 2024-09-04T17:47:48.400 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:48.400 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:48.400 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:48.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.400 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59956 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5963776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:47:48.400 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654998+0000", 2024-09-04T17:47:48.400 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 34.708323989, 2024-09-04T17:47:48.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068359518999999994, 2024-09-04T17:47:48.400 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:48.400 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:48.400 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:48.401 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:48.401 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:48.401 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:48.401 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59956 2024-09-04T17:47:48.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.401 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:48.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:48.401 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654998+0000", 2024-09-04T17:47:48.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:48.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:48.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654998+0000", 2024-09-04T17:47:48.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:48.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:48.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654998+0000", 2024-09-04T17:47:48.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4299999999999999e-07 2024-09-04T17:47:48.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:48.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654999+0000", 2024-09-04T17:47:48.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1799999999999999e-06 2024-09-04T17:47:48.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:48.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.655002+0000", 2024-09-04T17:47:48.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3639999999999999e-06 2024-09-04T17:47:48.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:48.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.655005+0000", 2024-09-04T17:47:48.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7970000000000001e-06 2024-09-04T17:47:48.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:48.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.723085+0000", 2024-09-04T17:47:48.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068080229000000006 2024-09-04T17:47:48.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:48.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.723197+0000", 2024-09-04T17:47:48.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000111344 2024-09-04T17:47:48.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:48.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.723357+0000", 2024-09-04T17:47:48.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016036199999999999 2024-09-04T17:47:48.405 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:48.405 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:48.405 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:48.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.405 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61082 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 2654208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:48.405 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.822664+0000", 2024-09-04T17:47:48.405 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 34.540657602000003, 2024-09-04T17:47:48.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067072242000000004, 2024-09-04T17:47:48.405 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:48.405 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:48.405 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:48.405 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:48.406 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:48.406 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:48.406 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61082 2024-09-04T17:47:48.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.406 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:48.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:48.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822664+0000", 2024-09-04T17:47:48.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:48.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:48.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822664+0000", 2024-09-04T17:47:48.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:48.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:48.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822664+0000", 2024-09-04T17:47:48.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6600000000000003e-07 2024-09-04T17:47:48.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:48.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822666+0000", 2024-09-04T17:47:48.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.536e-06 2024-09-04T17:47:48.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:48.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822669+0000", 2024-09-04T17:47:48.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3759999999999999e-06 2024-09-04T17:47:48.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:48.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822672+0000", 2024-09-04T17:47:48.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8109999999999999e-06 2024-09-04T17:47:48.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:48.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889263+0000", 2024-09-04T17:47:48.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066591259999999999 2024-09-04T17:47:48.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:48.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889369+0000", 2024-09-04T17:47:48.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000105362 2024-09-04T17:47:48.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:48.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889736+0000", 2024-09-04T17:47:48.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00036763099999999999 2024-09-04T17:47:48.410 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:48.410 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:48.410 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:48.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.410 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61083 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 2760704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:48.410 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.822678+0000", 2024-09-04T17:47:48.410 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 34.540643867999997, 2024-09-04T17:47:48.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067475996999999996, 2024-09-04T17:47:48.410 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:48.411 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:48.411 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:48.411 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:48.411 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:48.411 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:48.411 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61083 2024-09-04T17:47:48.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.411 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:48.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.411 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:48.411 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822678+0000", 2024-09-04T17:47:48.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:48.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:48.412 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822678+0000", 2024-09-04T17:47:48.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:48.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:48.412 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822725+0000", 2024-09-04T17:47:48.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7429000000000003e-05 2024-09-04T17:47:48.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:48.412 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822728+0000", 2024-09-04T17:47:48.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2140000000000001e-06 2024-09-04T17:47:48.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:48.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822737+0000", 2024-09-04T17:47:48.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4370000000000007e-06 2024-09-04T17:47:48.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:48.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822743+0000", 2024-09-04T17:47:48.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7520000000000002e-06 2024-09-04T17:47:48.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:48.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889785+0000", 2024-09-04T17:47:48.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067042265000000004 2024-09-04T17:47:48.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:48.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889840+0000", 2024-09-04T17:47:48.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5254000000000003e-05 2024-09-04T17:47:48.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:48.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.890154+0000", 2024-09-04T17:47:48.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000313646 2024-09-04T17:47:48.414 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:48.414 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:48.414 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:48.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.415 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61084 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 2867200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:48.415 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.822753+0000", 2024-09-04T17:47:48.415 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 34.540568931999999, 2024-09-04T17:47:48.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068497161000000001, 2024-09-04T17:47:48.415 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:48.415 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:48.415 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:48.415 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:48.415 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:48.415 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:48.415 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61084 2024-09-04T17:47:48.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.416 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:48.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:48.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822753+0000", 2024-09-04T17:47:48.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:48.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:48.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822753+0000", 2024-09-04T17:47:48.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:48.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:48.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822753+0000", 2024-09-04T17:47:48.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9100000000000003e-07 2024-09-04T17:47:48.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:48.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822756+0000", 2024-09-04T17:47:48.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.678e-06 2024-09-04T17:47:48.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:48.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822763+0000", 2024-09-04T17:47:48.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8220000000000001e-06 2024-09-04T17:47:48.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:48.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822768+0000", 2024-09-04T17:47:48.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1930000000000001e-06 2024-09-04T17:47:48.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:48.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.890246+0000", 2024-09-04T17:47:48.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067477996999999998 2024-09-04T17:47:48.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:48.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.890555+0000", 2024-09-04T17:47:48.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000309143 2024-09-04T17:47:48.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:48.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.891250+0000", 2024-09-04T17:47:48.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00069463700000000001 2024-09-04T17:47:48.419 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:48.419 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:48.419 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:48.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.419 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61285 3.8 3:117cfeb9:::rbd_data.11341dfde0c5.0000000000000006:1f [read 2760704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:48.419 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835200+0000", 2024-09-04T17:47:48.419 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 34.528121865000003, 2024-09-04T17:47:48.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066899803999999993, 2024-09-04T17:47:48.420 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:48.420 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:48.420 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:48.420 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:48.420 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:48.420 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:48.420 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61285 2024-09-04T17:47:48.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.420 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:48.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:48.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835200+0000", 2024-09-04T17:47:48.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:48.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:48.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835200+0000", 2024-09-04T17:47:48.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:48.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:48.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835200+0000", 2024-09-04T17:47:48.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0899999999999997e-07 2024-09-04T17:47:48.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:48.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835206+0000", 2024-09-04T17:47:48.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3280000000000002e-06 2024-09-04T17:47:48.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:48.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835211+0000", 2024-09-04T17:47:48.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9230000000000001e-06 2024-09-04T17:47:48.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:48.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835216+0000", 2024-09-04T17:47:48.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9019999999999998e-06 2024-09-04T17:47:48.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:48.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.901719+0000", 2024-09-04T17:47:48.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066502981000000003 2024-09-04T17:47:48.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:48.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.901813+0000", 2024-09-04T17:47:48.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3905000000000003e-05 2024-09-04T17:47:48.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:48.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902100+0000", 2024-09-04T17:47:48.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028645599999999998 2024-09-04T17:47:48.423 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:48.423 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:48.423 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:48.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.424 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61286 3.8 3:117cfeb9:::rbd_data.11341dfde0c5.0000000000000006:1f [read 2867200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:48.424 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835244+0000", 2024-09-04T17:47:48.424 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 34.528077246000002, 2024-09-04T17:47:48.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067143215000000006, 2024-09-04T17:47:48.424 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:48.424 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:48.424 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:48.424 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:48.424 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:48.424 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:48.424 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61286 2024-09-04T17:47:48.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.424 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:48.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:48.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835244+0000", 2024-09-04T17:47:48.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:48.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:48.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835244+0000", 2024-09-04T17:47:48.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:48.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:48.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835245+0000", 2024-09-04T17:47:48.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8400000000000004e-07 2024-09-04T17:47:48.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:48.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835248+0000", 2024-09-04T17:47:48.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6850000000000001e-06 2024-09-04T17:47:48.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:48.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835254+0000", 2024-09-04T17:47:48.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.494e-06 2024-09-04T17:47:48.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:48.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835260+0000", 2024-09-04T17:47:48.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0580000000000003e-06 2024-09-04T17:47:48.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:48.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902119+0000", 2024-09-04T17:47:48.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066858318 2024-09-04T17:47:48.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:48.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902163+0000", 2024-09-04T17:47:48.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4753000000000003e-05 2024-09-04T17:47:48.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:48.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902388+0000", 2024-09-04T17:47:48.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022422300000000001 2024-09-04T17:47:48.428 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:48.428 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:48.428 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:48.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.428 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61315 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 98304~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:48.428 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835269+0000", 2024-09-04T17:47:48.428 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 34.528052453999997, 2024-09-04T17:47:48.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067329396, 2024-09-04T17:47:48.428 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:48.428 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:48.428 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:48.429 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:48.429 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:48.429 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:48.429 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61315 2024-09-04T17:47:48.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.429 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:48.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:48.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835269+0000", 2024-09-04T17:47:48.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:48.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:48.430 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835269+0000", 2024-09-04T17:47:48.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:48.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.430 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:48.430 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835270+0000", 2024-09-04T17:47:48.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6599999999999996e-07 2024-09-04T17:47:48.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:48.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835272+0000", 2024-09-04T17:47:48.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.261e-06 2024-09-04T17:47:48.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:48.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835278+0000", 2024-09-04T17:47:48.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.443e-06 2024-09-04T17:47:48.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:48.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835284+0000", 2024-09-04T17:47:48.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3290000000000001e-06 2024-09-04T17:47:48.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:48.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902147+0000", 2024-09-04T17:47:48.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066862695999999999 2024-09-04T17:47:48.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:48.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902248+0000", 2024-09-04T17:47:48.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000101501 2024-09-04T17:47:48.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:48.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902599+0000", 2024-09-04T17:47:48.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00035060000000000001 2024-09-04T17:47:48.433 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:48.433 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:48.433 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:48.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.433 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61316 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 204800~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:48.433 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835291+0000", 2024-09-04T17:47:48.433 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 34.528030434999998, 2024-09-04T17:47:48.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067632052999999998, 2024-09-04T17:47:48.433 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:48.433 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:48.434 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:48.434 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:48.434 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:48.434 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:48.434 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61316 2024-09-04T17:47:48.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.434 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:48.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.434 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:48.434 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835291+0000", 2024-09-04T17:47:48.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:48.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.434 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:48.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835291+0000", 2024-09-04T17:47:48.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:48.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:48.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835292+0000", 2024-09-04T17:47:48.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1899999999999998e-07 2024-09-04T17:47:48.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:48.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835294+0000", 2024-09-04T17:47:48.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0880000000000002e-06 2024-09-04T17:47:48.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:48.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835300+0000", 2024-09-04T17:47:48.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.224e-06 2024-09-04T17:47:48.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:48.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835305+0000", 2024-09-04T17:47:48.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0730000000000004e-06 2024-09-04T17:47:48.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:48.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902632+0000", 2024-09-04T17:47:48.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067327322999999994 2024-09-04T17:47:48.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:48.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902688+0000", 2024-09-04T17:47:48.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5177999999999999e-05 2024-09-04T17:47:48.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:48.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902923+0000", 2024-09-04T17:47:48.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023574800000000001 2024-09-04T17:47:48.438 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:48.438 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:48.438 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:48.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.438 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61317 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 344064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:48.438 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835312+0000", 2024-09-04T17:47:48.438 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 34.528009431000001, 2024-09-04T17:47:48.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067918511000000001, 2024-09-04T17:47:48.438 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:48.439 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:48.439 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:48.439 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:48.439 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:48.439 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:48.440 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61317 2024-09-04T17:47:48.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.440 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:48.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:48.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835312+0000", 2024-09-04T17:47:48.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:48.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:48.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835312+0000", 2024-09-04T17:47:48.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:48.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:48.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835313+0000", 2024-09-04T17:47:48.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9000000000000002e-07 2024-09-04T17:47:48.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:48.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835315+0000", 2024-09-04T17:47:48.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1689999999999999e-06 2024-09-04T17:47:48.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:48.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835321+0000", 2024-09-04T17:47:48.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0619999999999997e-06 2024-09-04T17:47:48.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:48.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835326+0000", 2024-09-04T17:47:48.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0390000000000004e-06 2024-09-04T17:47:48.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:48.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902967+0000", 2024-09-04T17:47:48.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067641520999999996 2024-09-04T17:47:48.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:48.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903067+0000", 2024-09-04T17:47:48.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9426999999999995e-05 2024-09-04T17:47:48.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:48.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903231+0000", 2024-09-04T17:47:48.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016390299999999999 2024-09-04T17:47:48.443 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:48.443 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:48.443 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:48.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.444 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61318 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 450560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:48.444 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835334+0000", 2024-09-04T17:47:48.444 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 34.527987914000001, 2024-09-04T17:47:48.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068091771999999995, 2024-09-04T17:47:48.444 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:48.444 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:48.444 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:48.444 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:48.444 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:48.444 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:48.444 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61318 2024-09-04T17:47:48.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.445 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:48.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:48.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835334+0000", 2024-09-04T17:47:48.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:48.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:48.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835334+0000", 2024-09-04T17:47:48.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:48.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:48.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835334+0000", 2024-09-04T17:47:48.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5699999999999998e-07 2024-09-04T17:47:48.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:48.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835337+0000", 2024-09-04T17:47:48.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3889999999999999e-06 2024-09-04T17:47:48.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:48.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835343+0000", 2024-09-04T17:47:48.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1809999999999998e-06 2024-09-04T17:47:48.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:48.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835348+0000", 2024-09-04T17:47:48.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3170000000000001e-06 2024-09-04T17:47:48.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:48.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903270+0000", 2024-09-04T17:47:48.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067922191000000007 2024-09-04T17:47:48.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:48.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903321+0000", 2024-09-04T17:47:48.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0889e-05 2024-09-04T17:47:48.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:48.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903426+0000", 2024-09-04T17:47:48.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104348 2024-09-04T17:47:48.448 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:48.448 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:48.448 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:48.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.448 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61319 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 589824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:48.449 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835357+0000", 2024-09-04T17:47:48.449 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 34.527965096999999, 2024-09-04T17:47:48.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068372535999999998, 2024-09-04T17:47:48.449 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:48.449 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:48.449 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:48.449 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:48.449 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:48.449 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:48.449 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61319 2024-09-04T17:47:48.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.449 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:48.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.450 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:48.450 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835357+0000", 2024-09-04T17:47:48.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:48.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.450 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:48.450 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835357+0000", 2024-09-04T17:47:48.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:48.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.450 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:48.450 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835357+0000", 2024-09-04T17:47:48.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.32e-07 2024-09-04T17:47:48.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.451 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:48.451 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835359+0000", 2024-09-04T17:47:48.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1660000000000001e-06 2024-09-04T17:47:48.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.451 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:48.451 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835366+0000", 2024-09-04T17:47:48.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3550000000000001e-06 2024-09-04T17:47:48.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.451 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:48.451 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835371+0000", 2024-09-04T17:47:48.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1050000000000003e-06 2024-09-04T17:47:48.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:48.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903471+0000", 2024-09-04T17:47:48.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068100451000000006 2024-09-04T17:47:48.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:48.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903573+0000", 2024-09-04T17:47:48.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000101691 2024-09-04T17:47:48.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.453 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:48.453 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903729+0000", 2024-09-04T17:47:48.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000156336 2024-09-04T17:47:48.453 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:48.453 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:48.453 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:48.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.453 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61320 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:48.453 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835380+0000", 2024-09-04T17:47:48.453 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 34.527941331999997, 2024-09-04T17:47:48.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068543058000000004, 2024-09-04T17:47:48.453 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:48.453 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:48.453 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:48.454 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:48.454 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:48.454 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:48.454 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61320 2024-09-04T17:47:48.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.454 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:48.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.454 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:48.454 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835380+0000", 2024-09-04T17:47:48.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:48.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.454 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:48.454 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835380+0000", 2024-09-04T17:47:48.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:48.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:48.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835381+0000", 2024-09-04T17:47:48.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9999999999999998e-07 2024-09-04T17:47:48.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:48.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835383+0000", 2024-09-04T17:47:48.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2230000000000001e-06 2024-09-04T17:47:48.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:48.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835390+0000", 2024-09-04T17:47:48.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.34e-06 2024-09-04T17:47:48.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:48.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835396+0000", 2024-09-04T17:47:48.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9830000000000001e-06 2024-09-04T17:47:48.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:48.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903765+0000", 2024-09-04T17:47:48.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068368441000000002 2024-09-04T17:47:48.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:48.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903835+0000", 2024-09-04T17:47:48.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0631999999999997e-05 2024-09-04T17:47:48.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:48.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903923+0000", 2024-09-04T17:47:48.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8039000000000002e-05 2024-09-04T17:47:48.457 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:48.457 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:48.457 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:48.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.457 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61321 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 835584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:48.457 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835404+0000", 2024-09-04T17:47:48.457 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 34.527917273, 2024-09-04T17:47:48.458 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068911043000000005, 2024-09-04T17:47:48.458 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:48.458 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:48.458 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:48.458 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:48.458 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:48.458 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:48.458 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61321 2024-09-04T17:47:48.458 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.458 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:48.458 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.458 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:48.458 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835404+0000", 2024-09-04T17:47:48.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:48.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:48.459 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835404+0000", 2024-09-04T17:47:48.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:48.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:48.459 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835405+0000", 2024-09-04T17:47:48.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.89e-07 2024-09-04T17:47:48.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:48.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835407+0000", 2024-09-04T17:47:48.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4439999999999998e-06 2024-09-04T17:47:48.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:48.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835414+0000", 2024-09-04T17:47:48.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4930000000000003e-06 2024-09-04T17:47:48.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:48.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835419+0000", 2024-09-04T17:47:48.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8919999999999999e-06 2024-09-04T17:47:48.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:48.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903957+0000", 2024-09-04T17:47:48.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068538500000000002 2024-09-04T17:47:48.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:48.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904019+0000", 2024-09-04T17:47:48.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1419000000000003e-05 2024-09-04T17:47:48.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:48.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904315+0000", 2024-09-04T17:47:48.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00029680599999999999 2024-09-04T17:47:48.461 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:48.462 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:48.462 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:48.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.462 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61322 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:48.462 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835427+0000", 2024-09-04T17:47:48.462 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 34.527894887000002, 2024-09-04T17:47:48.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.069550138999999997, 2024-09-04T17:47:48.462 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:48.462 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:48.462 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:48.462 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:48.462 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:48.463 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:48.463 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61322 2024-09-04T17:47:48.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.463 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:48.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.463 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:48.463 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835427+0000", 2024-09-04T17:47:48.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:48.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.463 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:48.463 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835427+0000", 2024-09-04T17:47:48.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:48.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.464 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:48.464 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.842157+0000", 2024-09-04T17:47:48.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067298419999999998 2024-09-04T17:47:48.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.464 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:48.464 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.842163+0000", 2024-09-04T17:47:48.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2029999999999997e-06 2024-09-04T17:47:48.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.464 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:48.464 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.842175+0000", 2024-09-04T17:47:48.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2072000000000001e-05 2024-09-04T17:47:48.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:48.465 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.842183+0000", 2024-09-04T17:47:48.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3850000000000002e-06 2024-09-04T17:47:48.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:48.465 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904371+0000", 2024-09-04T17:47:48.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.062187754999999997 2024-09-04T17:47:48.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:48.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904467+0000", 2024-09-04T17:47:48.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6117000000000001e-05 2024-09-04T17:47:48.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:48.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:48.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:48.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904977+0000", 2024-09-04T17:47:48.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00050976500000000002 2024-09-04T17:47:48.466 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:48.466 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:48.466 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:48.466 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:48.466 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:48.466 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:48.467 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:47:48.468 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:47:48.468 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-04T17:47:48.469 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-04T17:47:48.469 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:47:48.470 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-04T17:47:48.470 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:47:48.470 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-04T17:47:48.470 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:48.463+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:48.569 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:48.567+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:48.599 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:48.599 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:47:48.599 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:47:48.599 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:48.612 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-04T17:47:48.670 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:48.668+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:48.771 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:48.768+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:48.778 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:48.778 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:47:48.778 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:47:48.778 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:47:48.778 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:48.792 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-04T17:47:48.871 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:47:48.867+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:47:48.913 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:48.913 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:47:48.913 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:47:48.913 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:47:48.914 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.914 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:12722 3.1b 3:d8152af1:::rbd_data.1134ef3a5a7d.00000000000000bd:head [delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-04T17:47:48.914 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:16.421965+0000", 2024-09-04T17:47:48.914 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 92.489600972999995, 2024-09-04T17:47:48.914 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.890002811, 2024-09-04T17:47:48.914 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:48.914 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:48.914 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:48.914 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:48.914 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:48.914 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:48.914 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 12722 2024-09-04T17:47:48.914 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.914 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:48.914 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.915 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:48.915 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421965+0000", 2024-09-04T17:47:48.915 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:48.915 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.915 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.915 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:48.915 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421965+0000", 2024-09-04T17:47:48.915 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:48.915 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.915 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.915 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:48.915 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421966+0000", 2024-09-04T17:47:48.915 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.102e-06 2024-09-04T17:47:48.916 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.916 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.916 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:48.916 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421971+0000", 2024-09-04T17:47:48.916 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9989999999999999e-06 2024-09-04T17:47:48.916 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.916 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.916 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:48.916 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421981+0000", 2024-09-04T17:47:48.916 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.7380000000000006e-06 2024-09-04T17:47:48.916 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.916 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.916 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:48.917 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421989+0000", 2024-09-04T17:47:48.917 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.1890000000000007e-06 2024-09-04T17:47:48.917 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.917 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.917 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:48.917 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.422050+0000", 2024-09-04T17:47:48.917 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.0702000000000003e-05 2024-09-04T17:47:48.917 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.917 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.917 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for peered", 2024-09-04T17:47:48.917 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.422059+0000", 2024-09-04T17:47:48.917 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5440000000000004e-06 2024-09-04T17:47:48.917 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.918 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.918 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:48.918 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.308156+0000", 2024-09-04T17:47:48.918 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.88609676199999998 2024-09-04T17:47:48.918 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.918 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.918 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:48.918 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.308398+0000", 2024-09-04T17:47:48.918 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024215400000000001 2024-09-04T17:47:48.918 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.918 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.918 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 7", 2024-09-04T17:47:48.918 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.308795+0000", 2024-09-04T17:47:48.918 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000396704 2024-09-04T17:47:48.919 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.919 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.919 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:47:48.919 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.310831+0000", 2024-09-04T17:47:48.919 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0020364200000000002 2024-09-04T17:47:48.919 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.919 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.919 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:48.919 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.311818+0000", 2024-09-04T17:47:48.919 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00098649799999999993 2024-09-04T17:47:48.919 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.919 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.919 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:47:48.919 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.311905+0000", 2024-09-04T17:47:48.920 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7070999999999994e-05 2024-09-04T17:47:48.920 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.920 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.920 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:48.920 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.311968+0000", 2024-09-04T17:47:48.920 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.2928000000000005e-05 2024-09-04T17:47:48.920 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:48.920 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:48.920 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:48.920 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.920 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.920 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37359 3.f e58/24 3:f29e50a5:::rbd_data.1134c5a43465.0000000000000021:head v 58'135, mlcod=58'133)", 2024-09-04T17:47:48.920 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:47:48.921 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 69.294270369000003, 2024-09-04T17:47:48.921 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053483197000000003, 2024-09-04T17:47:48.921 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:48.921 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:48.921 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:48.921 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:48.921 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.921 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:48.921 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:47:48.921 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:48.921 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.921 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.921 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:48.921 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:47:48.922 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:48.922 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.922 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.922 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:48.922 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:47:48.922 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1699999999999997e-07 2024-09-04T17:47:48.922 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.922 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.922 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:48.922 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617338+0000", 2024-09-04T17:47:48.922 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.1168999999999998e-05 2024-09-04T17:47:48.922 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.922 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.922 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:48.923 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617351+0000", 2024-09-04T17:47:48.923 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3094e-05 2024-09-04T17:47:48.923 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.923 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.923 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:48.923 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617359+0000", 2024-09-04T17:47:48.923 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7970000000000006e-06 2024-09-04T17:47:48.923 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.923 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.923 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:48.923 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617417+0000", 2024-09-04T17:47:48.923 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7079e-05 2024-09-04T17:47:48.923 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.924 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.924 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:48.924 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617438+0000", 2024-09-04T17:47:48.924 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1654999999999998e-05 2024-09-04T17:47:48.924 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.924 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.924 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:47:48.924 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670738+0000", 2024-09-04T17:47:48.924 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053300142000000002 2024-09-04T17:47:48.924 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.924 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.924 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:48.925 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670779+0000", 2024-09-04T17:47:48.925 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0343999999999999e-05 2024-09-04T17:47:48.925 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:48.925 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:48.925 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:48.925 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.925 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.925 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37358 3.1d e58/24 3:bd3215f3:::rbd_data.1134c5a43465.0000000000000020:head v 58'91, mlcod=58'89)", 2024-09-04T17:47:48.925 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617502+0000", 2024-09-04T17:47:48.925 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 69.294063651000002, 2024-09-04T17:47:48.925 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053505641999999999, 2024-09-04T17:47:48.925 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:48.925 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:48.926 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:48.926 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:48.926 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.926 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:48.926 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617502+0000", 2024-09-04T17:47:48.926 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:48.926 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.926 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.926 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:48.926 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617502+0000", 2024-09-04T17:47:48.926 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:48.926 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.926 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.927 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:48.927 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617503+0000", 2024-09-04T17:47:48.927 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.184e-06 2024-09-04T17:47:48.927 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.927 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.927 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:48.927 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617526+0000", 2024-09-04T17:47:48.927 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2099999999999998e-05 2024-09-04T17:47:48.927 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.927 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.927 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:48.927 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617540+0000", 2024-09-04T17:47:48.927 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4741e-05 2024-09-04T17:47:48.927 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.928 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.928 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:48.928 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617550+0000", 2024-09-04T17:47:48.928 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4259999999999992e-06 2024-09-04T17:47:48.928 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.928 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.928 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:48.928 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617613+0000", 2024-09-04T17:47:48.928 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.3625000000000002e-05 2024-09-04T17:47:48.928 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.928 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.928 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:48.929 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617637+0000", 2024-09-04T17:47:48.929 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.3604999999999998e-05 2024-09-04T17:47:48.929 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.929 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.929 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:47:48.929 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670967+0000", 2024-09-04T17:47:48.929 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053329881000000003 2024-09-04T17:47:48.929 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.929 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.929 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:48.929 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671008+0000", 2024-09-04T17:47:48.929 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.108e-05 2024-09-04T17:47:48.930 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:48.930 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:48.930 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:48.930 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.930 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.930 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37361 3.1c e58/24 3:3c7f8976:::rbd_data.1134c5a43465.0000000000000023:head v 58'159, mlcod=58'157)", 2024-09-04T17:47:48.930 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617578+0000", 2024-09-04T17:47:48.930 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 69.293987801, 2024-09-04T17:47:48.930 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053990367999999997, 2024-09-04T17:47:48.930 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:48.930 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:48.930 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:48.930 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:48.930 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.931 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:48.931 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617578+0000", 2024-09-04T17:47:48.931 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:48.931 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.931 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.931 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:48.931 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617578+0000", 2024-09-04T17:47:48.931 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:48.931 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.931 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.931 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:48.931 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617579+0000", 2024-09-04T17:47:48.931 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7299999999999996e-07 2024-09-04T17:47:48.932 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.932 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.932 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:48.932 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617610+0000", 2024-09-04T17:47:48.932 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0951000000000002e-05 2024-09-04T17:47:48.932 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.932 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.932 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:48.932 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:47:48.932 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0002e-05 2024-09-04T17:47:48.932 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.932 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.933 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:48.933 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617626+0000", 2024-09-04T17:47:48.933 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6660000000000002e-06 2024-09-04T17:47:48.933 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.933 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.933 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:48.933 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670548+0000", 2024-09-04T17:47:48.933 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.052921943999999999 2024-09-04T17:47:48.933 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.933 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.933 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:48.933 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670574+0000", 2024-09-04T17:47:48.933 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5405000000000002e-05 2024-09-04T17:47:48.934 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.934 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.934 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:47:48.934 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671522+0000", 2024-09-04T17:47:48.934 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00094815299999999997 2024-09-04T17:47:48.934 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.934 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.934 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:48.934 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671568+0000", 2024-09-04T17:47:48.934 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6674000000000001e-05 2024-09-04T17:47:48.934 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:48.934 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:48.934 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:48.935 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.935 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.935 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37357 3.15 e58/49 3:a897312a:::rbd_data.1134c5a43465.000000000000001f:head v 58'117, mlcod=58'115)", 2024-09-04T17:47:48.935 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:47:48.935 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 69.293945463, 2024-09-04T17:47:48.935 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053734297, 2024-09-04T17:47:48.935 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:48.935 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:48.935 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:48.935 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:48.935 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.935 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:48.935 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:47:48.936 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:48.936 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.936 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.936 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:48.936 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:47:48.936 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:48.936 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.936 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.936 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:48.936 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617621+0000", 2024-09-04T17:47:48.936 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.7300000000000004e-07 2024-09-04T17:47:48.936 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.937 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.937 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:48.937 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617642+0000", 2024-09-04T17:47:48.937 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0185000000000001e-05 2024-09-04T17:47:48.937 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.937 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.937 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:48.937 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617654+0000", 2024-09-04T17:47:48.937 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2745000000000001e-05 2024-09-04T17:47:48.937 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.937 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.937 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:48.937 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617662+0000", 2024-09-04T17:47:48.937 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.0499999999999992e-06 2024-09-04T17:47:48.938 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.938 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.938 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:48.938 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617727+0000", 2024-09-04T17:47:48.938 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.4923000000000002e-05 2024-09-04T17:47:48.938 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.938 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.938 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:48.938 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617753+0000", 2024-09-04T17:47:48.938 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5899999999999999e-05 2024-09-04T17:47:48.938 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.938 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.938 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:47:48.939 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671298+0000", 2024-09-04T17:47:48.939 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053545276000000003 2024-09-04T17:47:48.939 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.939 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.939 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:48.939 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671355+0000", 2024-09-04T17:47:48.939 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6245000000000001e-05 2024-09-04T17:47:48.939 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:48.939 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:48.939 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:48.939 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.939 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.939 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44687 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:48.940 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253012+0000", 2024-09-04T17:47:48.940 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 58.658553396000002, 2024-09-04T17:47:48.940 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051128777, 2024-09-04T17:47:48.940 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:48.940 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:48.940 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:48.940 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:48.940 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:48.940 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:48.940 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44687 2024-09-04T17:47:48.940 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.940 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:48.940 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.941 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:48.941 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253012+0000", 2024-09-04T17:47:48.941 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:48.941 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.941 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.941 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:48.941 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253012+0000", 2024-09-04T17:47:48.941 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:48.941 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.941 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.941 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:48.941 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253014+0000", 2024-09-04T17:47:48.941 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.24e-06 2024-09-04T17:47:48.942 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.942 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.942 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:48.942 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253019+0000", 2024-09-04T17:47:48.942 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4569999999999996e-06 2024-09-04T17:47:48.942 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.942 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.942 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:48.942 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253035+0000", 2024-09-04T17:47:48.942 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6291999999999999e-05 2024-09-04T17:47:48.942 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.942 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.942 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:48.943 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253050+0000", 2024-09-04T17:47:48.943 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4423000000000001e-05 2024-09-04T17:47:48.943 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.943 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.943 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:48.943 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304115+0000", 2024-09-04T17:47:48.943 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051064778999999998 2024-09-04T17:47:48.943 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.943 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.943 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:48.943 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304141+0000", 2024-09-04T17:47:48.943 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6585999999999999e-05 2024-09-04T17:47:48.943 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:48.944 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:48.944 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:48.944 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.944 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.944 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44688 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:48.944 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253067+0000", 2024-09-04T17:47:48.944 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 58.658498844, 2024-09-04T17:47:48.944 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051116979, 2024-09-04T17:47:48.944 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:48.944 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:48.944 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:48.944 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:48.944 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:48.945 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:48.945 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44688 2024-09-04T17:47:48.945 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.945 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:48.945 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.945 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:48.945 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253067+0000", 2024-09-04T17:47:48.945 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:48.945 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.945 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.945 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:48.945 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253067+0000", 2024-09-04T17:47:48.945 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:48.946 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.946 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.946 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:48.946 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253068+0000", 2024-09-04T17:47:48.946 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0729999999999999e-06 2024-09-04T17:47:48.946 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.946 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.946 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:48.946 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253073+0000", 2024-09-04T17:47:48.946 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2259999999999996e-06 2024-09-04T17:47:48.946 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.946 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.946 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:48.947 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253091+0000", 2024-09-04T17:47:48.947 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7255999999999999e-05 2024-09-04T17:47:48.947 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.947 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.947 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:48.947 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253103+0000", 2024-09-04T17:47:48.947 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2632999999999999e-05 2024-09-04T17:47:48.947 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.947 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.947 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:48.947 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304173+0000", 2024-09-04T17:47:48.947 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051069537999999998 2024-09-04T17:47:48.947 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.948 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.948 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:48.948 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304184+0000", 2024-09-04T17:47:48.948 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1253000000000001e-05 2024-09-04T17:47:48.948 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:48.948 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:48.948 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:48.948 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.948 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.948 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44689 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:48.948 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253121+0000", 2024-09-04T17:47:48.948 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 58.658444707000001, 2024-09-04T17:47:48.948 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051192595, 2024-09-04T17:47:48.949 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:48.949 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:48.949 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:48.949 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:48.949 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:48.949 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:48.949 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44689 2024-09-04T17:47:48.949 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.949 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:48.949 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.949 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:48.949 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253121+0000", 2024-09-04T17:47:48.949 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:48.950 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.950 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.950 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:48.950 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253121+0000", 2024-09-04T17:47:48.950 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:48.950 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.950 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.950 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:48.950 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253122+0000", 2024-09-04T17:47:48.950 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.15e-06 2024-09-04T17:47:48.950 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.950 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.950 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:48.951 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253128+0000", 2024-09-04T17:47:48.951 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2569999999999998e-06 2024-09-04T17:47:48.951 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.951 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.951 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:48.951 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253145+0000", 2024-09-04T17:47:48.951 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7176e-05 2024-09-04T17:47:48.951 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.951 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.951 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:48.951 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253158+0000", 2024-09-04T17:47:48.951 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3139000000000001e-05 2024-09-04T17:47:48.951 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.951 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.952 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:48.952 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304287+0000", 2024-09-04T17:47:48.952 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051129149999999998 2024-09-04T17:47:48.952 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.952 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.952 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:48.952 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304314+0000", 2024-09-04T17:47:48.952 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6723000000000001e-05 2024-09-04T17:47:48.952 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:48.952 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:48.952 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:48.952 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.952 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.952 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44690 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:48.953 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253177+0000", 2024-09-04T17:47:48.953 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 58.658388582000001, 2024-09-04T17:47:48.953 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051169263, 2024-09-04T17:47:48.953 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:48.953 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:48.953 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:48.953 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:48.953 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:48.953 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:48.953 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44690 2024-09-04T17:47:48.953 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.953 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:48.953 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.954 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:48.954 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253177+0000", 2024-09-04T17:47:48.954 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:48.954 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.954 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.954 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:48.954 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253177+0000", 2024-09-04T17:47:48.954 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:48.954 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.954 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.954 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:48.954 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253178+0000", 2024-09-04T17:47:48.954 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1000000000000001e-06 2024-09-04T17:47:48.955 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.955 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.955 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:48.955 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253184+0000", 2024-09-04T17:47:48.955 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2089999999999996e-06 2024-09-04T17:47:48.955 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.955 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.955 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:48.955 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253200+0000", 2024-09-04T17:47:48.955 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6087e-05 2024-09-04T17:47:48.955 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.955 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.955 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:48.955 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253231+0000", 2024-09-04T17:47:48.956 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0874999999999998e-05 2024-09-04T17:47:48.956 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.956 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.956 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:48.956 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304337+0000", 2024-09-04T17:47:48.956 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051106430000000001 2024-09-04T17:47:48.956 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.956 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.956 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:48.956 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304347+0000", 2024-09-04T17:47:48.956 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5619999999999993e-06 2024-09-04T17:47:48.956 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:48.956 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:48.957 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:48.957 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.957 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.957 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44691 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:48.957 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253249+0000", 2024-09-04T17:47:48.957 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 58.658316548000002, 2024-09-04T17:47:48.957 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051195630999999998, 2024-09-04T17:47:48.957 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:48.957 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:48.957 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:48.957 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:48.957 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:48.957 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:48.958 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44691 2024-09-04T17:47:48.958 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.958 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:48.958 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.958 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:48.958 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253249+0000", 2024-09-04T17:47:48.958 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:48.958 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.958 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.958 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:48.958 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253249+0000", 2024-09-04T17:47:48.958 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:48.958 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.959 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.959 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:48.959 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253251+0000", 2024-09-04T17:47:48.959 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2330000000000001e-06 2024-09-04T17:47:48.959 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.959 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.959 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:48.959 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253256+0000", 2024-09-04T17:47:48.959 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6099999999999997e-06 2024-09-04T17:47:48.959 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.959 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.959 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:48.959 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253274+0000", 2024-09-04T17:47:48.959 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7952999999999999e-05 2024-09-04T17:47:48.960 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.960 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.960 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:48.960 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253287+0000", 2024-09-04T17:47:48.960 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2839e-05 2024-09-04T17:47:48.960 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.960 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.960 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:48.960 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304396+0000", 2024-09-04T17:47:48.960 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051108990999999999 2024-09-04T17:47:48.960 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.960 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.961 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:48.961 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304445+0000", 2024-09-04T17:47:48.961 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9005000000000003e-05 2024-09-04T17:47:48.961 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:48.961 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:48.961 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:48.961 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.961 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.961 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44692 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:48.961 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253337+0000", 2024-09-04T17:47:48.961 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 58.658228975, 2024-09-04T17:47:48.961 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051138909000000003, 2024-09-04T17:47:48.961 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:48.961 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:48.962 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:48.962 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:48.962 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:48.962 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:48.962 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44692 2024-09-04T17:47:48.962 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.962 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:48.962 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.962 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:48.962 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253337+0000", 2024-09-04T17:47:48.962 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:48.962 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.962 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.963 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:48.963 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253337+0000", 2024-09-04T17:47:48.963 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:48.963 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.963 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.963 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:48.963 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253338+0000", 2024-09-04T17:47:48.963 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.463e-06 2024-09-04T17:47:48.963 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.963 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.963 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:48.963 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253357+0000", 2024-09-04T17:47:48.963 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9043e-05 2024-09-04T17:47:48.963 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.964 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.964 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:48.964 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253379+0000", 2024-09-04T17:47:48.964 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1243e-05 2024-09-04T17:47:48.964 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.964 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.964 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:48.964 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253396+0000", 2024-09-04T17:47:48.964 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7059e-05 2024-09-04T17:47:48.964 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.964 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.964 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:48.964 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304466+0000", 2024-09-04T17:47:48.965 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051070624000000002 2024-09-04T17:47:48.965 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.965 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.965 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:48.965 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304476+0000", 2024-09-04T17:47:48.965 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4769999999999992e-06 2024-09-04T17:47:48.965 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:48.965 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:48.965 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:48.965 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.965 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.965 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44693 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:48.965 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253415+0000", 2024-09-04T17:47:48.966 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 58.658150571999997, 2024-09-04T17:47:48.966 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051141949999999999, 2024-09-04T17:47:48.966 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:48.966 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:48.966 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:48.966 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:48.966 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:48.966 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:48.966 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44693 2024-09-04T17:47:48.966 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.966 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:48.966 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.966 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:48.966 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253415+0000", 2024-09-04T17:47:48.967 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:48.967 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.967 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.967 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:48.967 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253415+0000", 2024-09-04T17:47:48.967 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:48.967 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.967 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.967 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:48.967 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253416+0000", 2024-09-04T17:47:48.967 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1799999999999999e-06 2024-09-04T17:47:48.967 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.967 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.967 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:48.968 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253422+0000", 2024-09-04T17:47:48.968 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.5940000000000001e-06 2024-09-04T17:47:48.968 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.968 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.968 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:48.968 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253438+0000", 2024-09-04T17:47:48.968 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6008999999999999e-05 2024-09-04T17:47:48.968 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.968 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.968 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:48.968 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253452+0000", 2024-09-04T17:47:48.968 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3543000000000001e-05 2024-09-04T17:47:48.968 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.969 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.969 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:48.969 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304533+0000", 2024-09-04T17:47:48.969 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051081609 2024-09-04T17:47:48.969 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.969 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.969 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:48.969 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304557+0000", 2024-09-04T17:47:48.969 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4015e-05 2024-09-04T17:47:48.969 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:48.969 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:48.969 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:48.969 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.969 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.970 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44694 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:48.970 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253469+0000", 2024-09-04T17:47:48.970 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 58.658096446999998, 2024-09-04T17:47:48.970 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051121486000000001, 2024-09-04T17:47:48.970 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:48.970 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:48.970 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:48.970 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:48.970 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:48.970 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:48.970 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44694 2024-09-04T17:47:48.970 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.970 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:48.971 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.971 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:48.971 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253469+0000", 2024-09-04T17:47:48.971 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:48.971 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.971 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.971 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:48.971 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253469+0000", 2024-09-04T17:47:48.971 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:48.971 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.971 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.971 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:48.971 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253471+0000", 2024-09-04T17:47:48.971 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1400000000000001e-06 2024-09-04T17:47:48.972 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.972 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.972 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:48.972 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253476+0000", 2024-09-04T17:47:48.972 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7370000000000003e-06 2024-09-04T17:47:48.972 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.972 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.972 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:48.972 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253493+0000", 2024-09-04T17:47:48.972 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6549e-05 2024-09-04T17:47:48.972 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.972 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.972 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:48.973 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253506+0000", 2024-09-04T17:47:48.973 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3406e-05 2024-09-04T17:47:48.973 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.973 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.973 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:48.973 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304582+0000", 2024-09-04T17:47:48.973 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051075534999999998 2024-09-04T17:47:48.973 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.973 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.973 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:48.973 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304591+0000", 2024-09-04T17:47:48.973 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1190000000000003e-06 2024-09-04T17:47:48.973 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:48.974 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:48.974 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:48.974 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.974 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.974 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44695 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:48.974 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253552+0000", 2024-09-04T17:47:48.974 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 58.658013693000001, 2024-09-04T17:47:48.974 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051068872000000001, 2024-09-04T17:47:48.974 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:48.974 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:48.974 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:48.974 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:48.974 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:48.974 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:48.975 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44695 2024-09-04T17:47:48.975 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.975 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:48.975 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.975 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:48.975 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253552+0000", 2024-09-04T17:47:48.975 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:48.975 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.975 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.975 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:48.975 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253552+0000", 2024-09-04T17:47:48.975 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:48.975 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.976 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.976 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:48.976 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253553+0000", 2024-09-04T17:47:48.976 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1960000000000001e-06 2024-09-04T17:47:48.976 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.976 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.976 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:48.976 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253560+0000", 2024-09-04T17:47:48.976 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.2040000000000002e-06 2024-09-04T17:47:48.976 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.976 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.976 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:48.976 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253578+0000", 2024-09-04T17:47:48.977 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8241999999999999e-05 2024-09-04T17:47:48.977 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.977 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.977 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:48.977 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253591+0000", 2024-09-04T17:47:48.977 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3453e-05 2024-09-04T17:47:48.977 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.977 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.977 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:48.977 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304614+0000", 2024-09-04T17:47:48.977 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051022456000000001 2024-09-04T17:47:48.977 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.977 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.978 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:48.978 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304621+0000", 2024-09-04T17:47:48.978 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3209999999999999e-06 2024-09-04T17:47:48.978 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:48.978 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:48.978 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:48.978 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.978 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.978 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44696 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:48.978 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253609+0000", 2024-09-04T17:47:48.978 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 58.657957191999998, 2024-09-04T17:47:48.978 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051115351000000003, 2024-09-04T17:47:48.978 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:48.979 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:48.979 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:48.979 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:48.979 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:48.979 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:48.979 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44696 2024-09-04T17:47:48.979 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.979 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:48.979 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.979 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:48.979 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253609+0000", 2024-09-04T17:47:48.979 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:48.979 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.980 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.980 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:48.980 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253609+0000", 2024-09-04T17:47:48.980 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:48.980 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.980 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.980 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:48.980 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253610+0000", 2024-09-04T17:47:48.980 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2300000000000001e-06 2024-09-04T17:47:48.980 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.980 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.980 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:48.980 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253615+0000", 2024-09-04T17:47:48.980 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7260000000000002e-06 2024-09-04T17:47:48.981 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.981 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.981 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:48.981 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253632+0000", 2024-09-04T17:47:48.981 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7039999999999999e-05 2024-09-04T17:47:48.981 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.981 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.981 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:48.981 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253666+0000", 2024-09-04T17:47:48.981 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4662000000000001e-05 2024-09-04T17:47:48.981 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.981 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.981 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:48.982 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304677+0000", 2024-09-04T17:47:48.982 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051011017999999998 2024-09-04T17:47:48.982 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.982 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.982 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:48.982 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304724+0000", 2024-09-04T17:47:48.982 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6675000000000003e-05 2024-09-04T17:47:48.982 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:48.982 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:48.982 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:48.982 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.982 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.982 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44697 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:48.982 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253684+0000", 2024-09-04T17:47:48.982 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 58.657881697999997, 2024-09-04T17:47:48.983 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051073107, 2024-09-04T17:47:48.983 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:48.983 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:48.983 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:48.983 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:48.983 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:48.983 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:48.983 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44697 2024-09-04T17:47:48.983 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.983 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:48.983 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.983 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:48.983 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253684+0000", 2024-09-04T17:47:48.984 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:48.984 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.984 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.984 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:48.984 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253684+0000", 2024-09-04T17:47:48.984 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:48.984 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.984 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.984 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:48.984 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253685+0000", 2024-09-04T17:47:48.984 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0699999999999999e-06 2024-09-04T17:47:48.984 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.985 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.985 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:48.985 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253690+0000", 2024-09-04T17:47:48.985 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1959999999999999e-06 2024-09-04T17:47:48.985 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.985 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.985 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:48.985 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253707+0000", 2024-09-04T17:47:48.985 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6946e-05 2024-09-04T17:47:48.985 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.985 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.985 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:48.985 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253720+0000", 2024-09-04T17:47:48.985 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3076e-05 2024-09-04T17:47:48.986 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.986 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.986 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:48.986 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304745+0000", 2024-09-04T17:47:48.986 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051024760000000002 2024-09-04T17:47:48.986 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.986 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.986 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:48.986 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304757+0000", 2024-09-04T17:47:48.986 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2058999999999999e-05 2024-09-04T17:47:48.986 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:48.986 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:48.986 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:48.987 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.987 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.987 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44698 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:48.987 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253737+0000", 2024-09-04T17:47:48.987 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 58.657829347000003, 2024-09-04T17:47:48.987 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051176078, 2024-09-04T17:47:48.987 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:48.987 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:48.987 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:48.987 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:48.987 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:48.987 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:48.987 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44698 2024-09-04T17:47:48.987 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.988 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:48.988 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.988 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:48.988 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253737+0000", 2024-09-04T17:47:48.988 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:48.988 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.988 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.988 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:48.988 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253737+0000", 2024-09-04T17:47:48.988 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:48.988 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.988 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.988 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:48.989 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253738+0000", 2024-09-04T17:47:48.989 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0300000000000001e-06 2024-09-04T17:47:48.989 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.989 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.989 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:48.989 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253752+0000", 2024-09-04T17:47:48.989 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4358999999999999e-05 2024-09-04T17:47:48.989 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.989 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.989 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:48.989 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253771+0000", 2024-09-04T17:47:48.989 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8879999999999999e-05 2024-09-04T17:47:48.989 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.989 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.990 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:48.990 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253785+0000", 2024-09-04T17:47:48.990 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3852000000000001e-05 2024-09-04T17:47:48.990 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.990 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.990 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:48.990 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304876+0000", 2024-09-04T17:47:48.990 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051091105999999997 2024-09-04T17:47:48.990 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.990 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.990 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:48.990 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304913+0000", 2024-09-04T17:47:48.990 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.6850999999999997e-05 2024-09-04T17:47:48.991 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:48.991 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:48.991 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:48.991 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.991 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.991 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44699 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:48.991 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253804+0000", 2024-09-04T17:47:48.991 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 58.657761823000001, 2024-09-04T17:47:48.991 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051172250000000002, 2024-09-04T17:47:48.991 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:48.991 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:48.991 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:48.991 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:48.992 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:48.992 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:48.992 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44699 2024-09-04T17:47:48.992 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.992 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:48.992 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.992 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:48.992 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253804+0000", 2024-09-04T17:47:48.992 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:48.992 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.992 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.992 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:48.992 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253804+0000", 2024-09-04T17:47:48.993 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:48.993 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.993 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.993 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:48.993 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253805+0000", 2024-09-04T17:47:48.993 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.08e-06 2024-09-04T17:47:48.993 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.993 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.993 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:48.993 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253810+0000", 2024-09-04T17:47:48.993 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1830000000000002e-06 2024-09-04T17:47:48.993 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.993 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.993 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:48.994 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253828+0000", 2024-09-04T17:47:48.994 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7662000000000001e-05 2024-09-04T17:47:48.994 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.994 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.994 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:48.994 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253841+0000", 2024-09-04T17:47:48.994 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3203999999999999e-05 2024-09-04T17:47:48.994 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.994 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.994 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:48.994 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304966+0000", 2024-09-04T17:47:48.994 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051125273999999998 2024-09-04T17:47:48.994 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.995 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.995 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:48.995 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304976+0000", 2024-09-04T17:47:48.995 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.8470000000000008e-06 2024-09-04T17:47:48.995 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:48.995 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:48.995 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:48.995 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.995 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.995 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44700 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:48.995 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253858+0000", 2024-09-04T17:47:48.995 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 58.657708169000003, 2024-09-04T17:47:48.995 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051218957000000002, 2024-09-04T17:47:48.996 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:48.996 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:48.996 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:48.996 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:48.996 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:48.996 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:48.996 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44700 2024-09-04T17:47:48.996 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.996 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:48.996 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.996 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:48.996 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253858+0000", 2024-09-04T17:47:48.996 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:48.996 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.996 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.997 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:48.997 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253858+0000", 2024-09-04T17:47:48.997 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:48.997 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.997 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.997 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:48.997 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253859+0000", 2024-09-04T17:47:48.997 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.217e-06 2024-09-04T17:47:48.997 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.997 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.997 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:48.997 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253864+0000", 2024-09-04T17:47:48.997 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7600000000000002e-06 2024-09-04T17:47:48.997 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.998 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.998 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:48.998 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253880+0000", 2024-09-04T17:47:48.998 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6501999999999999e-05 2024-09-04T17:47:48.998 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.998 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.998 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:48.998 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253899+0000", 2024-09-04T17:47:48.998 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8907e-05 2024-09-04T17:47:48.998 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.998 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.998 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:48.998 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305052+0000", 2024-09-04T17:47:48.998 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051152989000000003 2024-09-04T17:47:48.999 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.999 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.999 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:48.999 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305077+0000", 2024-09-04T17:47:48.999 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4581999999999999e-05 2024-09-04T17:47:48.999 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:48.999 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:48.999 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:48.999 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:48.999 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:48.999 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44701 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:48.999 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253917+0000", 2024-09-04T17:47:48.999 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 58.657648817000002, 2024-09-04T17:47:48.999 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051231113000000002, 2024-09-04T17:47:49.000 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:49.000 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:49.000 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:47:49.000 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:49.000 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:49.000 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:49.000 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44701 2024-09-04T17:47:49.000 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.000 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:49.000 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.000 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:49.000 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253917+0000", 2024-09-04T17:47:49.000 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:49.001 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.001 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.001 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:49.001 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253917+0000", 2024-09-04T17:47:49.001 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:49.001 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.001 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.001 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:49.001 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253918+0000", 2024-09-04T17:47:49.001 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.136e-06 2024-09-04T17:47:49.001 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.001 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.001 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:49.002 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253924+0000", 2024-09-04T17:47:49.002 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.3070000000000002e-06 2024-09-04T17:47:49.002 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.002 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.002 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:49.002 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253940+0000", 2024-09-04T17:47:49.002 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6612e-05 2024-09-04T17:47:49.002 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.002 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.002 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:49.002 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253954+0000", 2024-09-04T17:47:49.002 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3709999999999999e-05 2024-09-04T17:47:49.002 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.002 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.002 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:49.003 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305136+0000", 2024-09-04T17:47:49.003 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051181935999999997 2024-09-04T17:47:49.003 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.003 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.003 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:49.003 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305148+0000", 2024-09-04T17:47:49.003 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2412000000000001e-05 2024-09-04T17:47:49.003 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:49.003 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:49.003 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:49.003 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:49.003 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:49.003 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:49.004 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-04T17:47:49.006 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-04T17:47:49.006 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:49.003+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:49.106 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:49.104+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:49.116 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone37 from 0xe7b00fa (overlap 0xbd332) to 0x5e999 2024-09-04T17:47:49.193 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:49.194 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:47:49.194 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:47:49.194 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:49.207 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:49.204+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:49.209 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-04T17:47:49.252 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 4, 5, 6, 7] out_osds: [3, 1] dead_osds: [6] live_osds: [0, 1, 4, 5, 7, 3, 2] 2024-09-04T17:47:49.252 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-04T17:47:49.253 INFO:tasks.thrashosds.thrasher:Reviving osd 6 2024-09-04T17:47:49.253 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-04T17:47:49.253 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-04T17:47:49.255 INFO:tasks.ceph.osd.6:Started 2024-09-04T17:47:49.255 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-04T17:47:49.257 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #36, image image_client.0-clone36 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent37 2024-09-04T17:47:49.308 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:49.304+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:49.335 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:49.335 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:47:49.335 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:47:49.335 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:47:49.336 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:49.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-04T17:47:49.352+0000 7f9c0d2ef640 0 --2- 172.21.15.92:0/2777611349 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9c08142da0 0x7f9bec0082c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-04T17:47:49.357 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-04T17:47:49.393 INFO:teuthology.orchestra.run.smithi190.stderr:no valid command found; 10 closest matches: 2024-09-04T17:47:49.393 INFO:teuthology.orchestra.run.smithi190.stderr:0 2024-09-04T17:47:49.393 INFO:teuthology.orchestra.run.smithi190.stderr:1 2024-09-04T17:47:49.393 INFO:teuthology.orchestra.run.smithi190.stderr:2 2024-09-04T17:47:49.393 INFO:teuthology.orchestra.run.smithi190.stderr:abort 2024-09-04T17:47:49.393 INFO:teuthology.orchestra.run.smithi190.stderr:assert 2024-09-04T17:47:49.393 INFO:teuthology.orchestra.run.smithi190.stderr:config diff 2024-09-04T17:47:49.393 INFO:teuthology.orchestra.run.smithi190.stderr:config diff get 2024-09-04T17:47:49.393 INFO:teuthology.orchestra.run.smithi190.stderr:config get 2024-09-04T17:47:49.393 INFO:teuthology.orchestra.run.smithi190.stderr:config help [] 2024-09-04T17:47:49.393 INFO:teuthology.orchestra.run.smithi190.stderr:config set ... 2024-09-04T17:47:49.393 INFO:teuthology.orchestra.run.smithi190.stderr:admin_socket: invalid command 2024-09-04T17:47:49.395 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-04T17:47:49.395 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-6, ['dump_ops_in_flight'] 2024-09-04T17:47:49.441 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:49.439+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:49.488 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:49.488 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:47:49.488 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:47:49.488 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:47:49.488 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.488 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8831 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2453504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:49.488 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:47:49.488 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 96.430206858999995, 2024-09-04T17:47:49.488 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14731111799999999, 2024-09-04T17:47:49.488 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:49.489 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:49.489 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:49.489 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:49.489 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:49.489 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:49.489 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8831 2024-09-04T17:47:49.489 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.489 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:49.489 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.489 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:49.489 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:47:49.489 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:49.489 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.489 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.490 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:49.490 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:47:49.490 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:49.490 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.490 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.490 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:49.490 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057811+0000", 2024-09-04T17:47:49.490 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.083e-06 2024-09-04T17:47:49.490 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.490 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.490 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:49.490 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057814+0000", 2024-09-04T17:47:49.490 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.407e-06 2024-09-04T17:47:49.490 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.491 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.491 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:49.491 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057825+0000", 2024-09-04T17:47:49.491 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.17e-05 2024-09-04T17:47:49.491 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.491 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.491 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:49.491 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057836+0000", 2024-09-04T17:47:49.491 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0283e-05 2024-09-04T17:47:49.491 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.491 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.491 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:49.491 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.204762+0000", 2024-09-04T17:47:49.491 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.146926205 2024-09-04T17:47:49.492 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.492 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.492 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:49.492 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.204895+0000", 2024-09-04T17:47:49.492 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013328999999999999 2024-09-04T17:47:49.492 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.492 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.492 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:49.492 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205121+0000", 2024-09-04T17:47:49.492 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00022614999999999999 2024-09-04T17:47:49.492 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:49.492 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:49.492 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:49.492 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.492 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.493 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8832 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2490368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:49.493 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:47:49.493 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 96.430168825999999, 2024-09-04T17:47:49.493 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14770012299999999, 2024-09-04T17:47:49.493 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:49.493 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:49.493 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:49.493 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:49.493 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:49.493 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:49.493 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8832 2024-09-04T17:47:49.493 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.493 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:49.493 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.493 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:49.494 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:47:49.494 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:49.494 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.494 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.494 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:49.494 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:47:49.494 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:49.494 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.494 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.494 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:49.494 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057849+0000", 2024-09-04T17:47:49.494 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6700000000000003e-07 2024-09-04T17:47:49.494 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.494 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.495 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:49.495 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057852+0000", 2024-09-04T17:47:49.495 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6359999999999998e-06 2024-09-04T17:47:49.495 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.495 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.495 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:49.495 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057860+0000", 2024-09-04T17:47:49.495 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8140000000000007e-06 2024-09-04T17:47:49.495 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.495 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.495 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:49.495 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057868+0000", 2024-09-04T17:47:49.495 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1999999999999997e-06 2024-09-04T17:47:49.495 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.496 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.496 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:49.496 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205208+0000", 2024-09-04T17:47:49.496 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14734067300000001 2024-09-04T17:47:49.496 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.496 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.496 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:49.496 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205347+0000", 2024-09-04T17:47:49.496 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013914900000000001 2024-09-04T17:47:49.496 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.496 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.496 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:49.496 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205548+0000", 2024-09-04T17:47:49.496 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000200984 2024-09-04T17:47:49.496 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:49.497 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:49.497 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:49.497 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.497 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.497 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8833 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2527232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:49.497 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:47:49.497 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 96.430136955999998, 2024-09-04T17:47:49.497 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148062572, 2024-09-04T17:47:49.497 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:49.497 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:49.497 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:49.497 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:49.497 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:49.497 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:49.497 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8833 2024-09-04T17:47:49.498 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.498 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:49.498 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.498 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:49.498 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:47:49.498 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:49.498 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.498 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.498 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:49.498 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:47:49.498 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:49.498 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.498 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.498 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:49.498 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057881+0000", 2024-09-04T17:47:49.499 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4300000000000002e-07 2024-09-04T17:47:49.499 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.499 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.499 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:49.499 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057893+0000", 2024-09-04T17:47:49.499 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1973999999999999e-05 2024-09-04T17:47:49.499 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.499 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.499 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:49.499 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057906+0000", 2024-09-04T17:47:49.499 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2716e-05 2024-09-04T17:47:49.499 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.499 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.499 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:49.499 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057913+0000", 2024-09-04T17:47:49.499 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3930000000000002e-06 2024-09-04T17:47:49.500 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.500 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.500 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:49.500 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205621+0000", 2024-09-04T17:47:49.500 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14770788800000001 2024-09-04T17:47:49.500 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.500 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.500 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:49.500 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205740+0000", 2024-09-04T17:47:49.500 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000118936 2024-09-04T17:47:49.500 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.500 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.501 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:49.501 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205943+0000", 2024-09-04T17:47:49.501 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020292199999999999 2024-09-04T17:47:49.501 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:49.501 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:49.501 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:49.501 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.501 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.501 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8834 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2564096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:49.501 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:47:49.501 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 96.430093677000002, 2024-09-04T17:47:49.501 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14846753200000001, 2024-09-04T17:47:49.501 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:49.501 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:49.501 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:49.502 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:49.502 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:49.502 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:49.502 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8834 2024-09-04T17:47:49.502 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.502 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:49.502 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.502 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:49.502 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:47:49.502 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:49.502 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.502 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.502 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:49.502 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:47:49.502 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:49.503 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.503 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.503 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:49.503 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057925+0000", 2024-09-04T17:47:49.503 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0270000000000001e-06 2024-09-04T17:47:49.503 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.503 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.503 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:49.503 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057936+0000", 2024-09-04T17:47:49.503 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.112e-05 2024-09-04T17:47:49.503 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.503 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.503 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:49.503 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057946+0000", 2024-09-04T17:47:49.503 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.009e-05 2024-09-04T17:47:49.504 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.504 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.504 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:49.504 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057953+0000", 2024-09-04T17:47:49.504 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.6229999999999997e-06 2024-09-04T17:47:49.504 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.504 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.504 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:49.504 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206027+0000", 2024-09-04T17:47:49.504 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148073591 2024-09-04T17:47:49.504 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.504 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.504 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:49.504 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206161+0000", 2024-09-04T17:47:49.505 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013411599999999999 2024-09-04T17:47:49.505 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.505 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.505 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:49.505 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206391+0000", 2024-09-04T17:47:49.505 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000229965 2024-09-04T17:47:49.505 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:49.505 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:49.505 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:49.505 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.505 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.505 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8835 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2600960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:49.505 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:47:49.505 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 96.430054026999997, 2024-09-04T17:47:49.505 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14882941599999999, 2024-09-04T17:47:49.506 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:49.506 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:49.506 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:49.506 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:49.506 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:49.506 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:49.506 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8835 2024-09-04T17:47:49.506 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.506 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:49.506 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.506 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:49.506 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:47:49.506 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:49.506 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.506 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.507 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:49.507 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:47:49.507 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:49.507 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.507 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.507 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:49.507 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058544+0000", 2024-09-04T17:47:49.507 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00058061899999999999 2024-09-04T17:47:49.507 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.507 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.507 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:49.507 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058550+0000", 2024-09-04T17:47:49.507 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.1129999999999997e-06 2024-09-04T17:47:49.507 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.507 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.507 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:49.508 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058566+0000", 2024-09-04T17:47:49.508 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5820000000000001e-05 2024-09-04T17:47:49.508 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.508 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.508 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:49.508 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058578+0000", 2024-09-04T17:47:49.508 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2537e-05 2024-09-04T17:47:49.508 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.508 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.508 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:49.508 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206470+0000", 2024-09-04T17:47:49.508 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14789153799999999 2024-09-04T17:47:49.508 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.508 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.509 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:49.509 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206589+0000", 2024-09-04T17:47:49.509 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000118837 2024-09-04T17:47:49.509 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.509 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.509 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:49.509 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206793+0000", 2024-09-04T17:47:49.509 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000203952 2024-09-04T17:47:49.509 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:49.509 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:49.509 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:49.509 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.509 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.509 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8836 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2654208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:49.509 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:47:49.509 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 96.429421339000001, 2024-09-04T17:47:49.510 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14862433999999999, 2024-09-04T17:47:49.510 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:49.510 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:49.510 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:49.510 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:49.510 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:49.510 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:49.510 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8836 2024-09-04T17:47:49.510 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.510 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:49.510 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.510 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:49.510 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:47:49.510 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:49.511 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.511 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.511 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:49.511 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:47:49.511 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:49.511 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.511 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.511 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:49.511 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062663+0000", 2024-09-04T17:47:49.511 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0040670619999999998 2024-09-04T17:47:49.511 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.511 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.511 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:49.511 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062668+0000", 2024-09-04T17:47:49.511 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4199999999999998e-06 2024-09-04T17:47:49.512 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.512 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.512 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:49.512 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062689+0000", 2024-09-04T17:47:49.512 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0356999999999998e-05 2024-09-04T17:47:49.512 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.512 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.512 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:49.512 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062726+0000", 2024-09-04T17:47:49.512 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7302999999999998e-05 2024-09-04T17:47:49.512 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.512 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.512 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:49.512 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206875+0000", 2024-09-04T17:47:49.513 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14414883100000001 2024-09-04T17:47:49.513 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.513 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.513 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:49.513 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.207016+0000", 2024-09-04T17:47:49.513 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000140822 2024-09-04T17:47:49.513 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.513 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.513 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:49.513 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.207220+0000", 2024-09-04T17:47:49.513 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020454499999999999 2024-09-04T17:47:49.513 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:49.513 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:49.513 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:49.514 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.514 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.514 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8844 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2928640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:49.514 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:47:49.514 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 96.425057256000002, 2024-09-04T17:47:49.514 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14752153100000001, 2024-09-04T17:47:49.514 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:49.514 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:49.514 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:49.514 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:49.514 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:49.514 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:49.514 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8844 2024-09-04T17:47:49.514 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.514 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:49.514 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.515 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:49.515 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:47:49.515 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:49.515 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.515 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.515 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:49.515 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:47:49.515 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:49.515 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.515 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.515 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:49.515 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062961+0000", 2024-09-04T17:47:49.515 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6700000000000003e-07 2024-09-04T17:47:49.515 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.515 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.516 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:49.516 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062973+0000", 2024-09-04T17:47:49.516 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2109999999999999e-05 2024-09-04T17:47:49.516 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.516 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.516 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:49.516 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062982+0000", 2024-09-04T17:47:49.516 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.6900000000000004e-06 2024-09-04T17:47:49.516 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.516 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.516 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:49.516 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062990+0000", 2024-09-04T17:47:49.516 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1899999999999998e-06 2024-09-04T17:47:49.516 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.517 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.517 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:49.517 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210090+0000", 2024-09-04T17:47:49.517 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14710026300000001 2024-09-04T17:47:49.517 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.517 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.517 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:49.517 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210225+0000", 2024-09-04T17:47:49.517 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013532400000000001 2024-09-04T17:47:49.517 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.517 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.517 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:49.517 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210481+0000", 2024-09-04T17:47:49.517 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00025628700000000001 2024-09-04T17:47:49.517 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:49.518 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:49.518 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:49.518 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.518 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.518 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8845 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2965504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:49.518 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:47:49.518 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 96.425017533000002, 2024-09-04T17:47:49.518 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147916819, 2024-09-04T17:47:49.518 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:49.518 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:49.518 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:49.518 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:49.518 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:49.518 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:49.518 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8845 2024-09-04T17:47:49.519 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.519 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:49.519 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.519 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:49.519 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:47:49.519 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:49.519 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.519 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.519 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:49.519 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:47:49.519 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:49.519 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.519 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.519 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:49.519 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:47:49.519 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.0399999999999995e-07 2024-09-04T17:47:49.520 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.520 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.520 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:49.520 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063003+0000", 2024-09-04T17:47:49.520 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6529999999999998e-06 2024-09-04T17:47:49.520 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.520 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.520 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:49.520 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063012+0000", 2024-09-04T17:47:49.520 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9500000000000007e-06 2024-09-04T17:47:49.520 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.520 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.520 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:49.520 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063027+0000", 2024-09-04T17:47:49.521 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5443000000000001e-05 2024-09-04T17:47:49.521 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.521 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.521 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:49.521 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210571+0000", 2024-09-04T17:47:49.521 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14754353100000001 2024-09-04T17:47:49.521 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.521 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.521 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:49.521 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210714+0000", 2024-09-04T17:47:49.521 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014295599999999999 2024-09-04T17:47:49.521 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.521 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.521 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:49.522 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210916+0000", 2024-09-04T17:47:49.522 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020258200000000001 2024-09-04T17:47:49.522 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:49.522 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:49.522 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:49.522 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.522 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.522 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8846 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3002368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:49.522 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:47:49.522 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 96.424979996000005, 2024-09-04T17:47:49.522 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14828011599999999, 2024-09-04T17:47:49.522 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:49.522 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:49.522 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:49.522 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:49.523 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:49.523 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:49.523 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8846 2024-09-04T17:47:49.523 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.523 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:49.523 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.523 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:49.523 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:47:49.523 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:49.523 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.523 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.523 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:49.523 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:47:49.523 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:49.523 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.524 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.524 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:49.524 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063038+0000", 2024-09-04T17:47:49.524 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.23e-07 2024-09-04T17:47:49.524 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.524 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.524 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:49.524 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063051+0000", 2024-09-04T17:47:49.524 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2757e-05 2024-09-04T17:47:49.524 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.524 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.524 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:49.524 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063070+0000", 2024-09-04T17:47:49.524 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9663000000000001e-05 2024-09-04T17:47:49.524 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.525 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.525 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:49.525 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063079+0000", 2024-09-04T17:47:49.525 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.0929999999999992e-06 2024-09-04T17:47:49.525 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.525 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.525 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:49.525 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210989+0000", 2024-09-04T17:47:49.525 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14790984500000001 2024-09-04T17:47:49.525 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.525 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.525 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:49.525 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211109+0000", 2024-09-04T17:47:49.525 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000120056 2024-09-04T17:47:49.526 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.526 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.526 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:49.526 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211317+0000", 2024-09-04T17:47:49.526 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020797900000000001 2024-09-04T17:47:49.526 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:49.526 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:49.526 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:49.526 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.526 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.526 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8847 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3039232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:49.526 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:47:49.526 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 96.424905563999999, 2024-09-04T17:47:49.526 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14859408900000001, 2024-09-04T17:47:49.527 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:49.527 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:49.527 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:49.527 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:49.527 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:49.527 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:49.527 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8847 2024-09-04T17:47:49.527 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.527 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:49.527 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.527 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:49.527 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:47:49.527 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:49.527 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.527 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.527 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:49.528 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:47:49.528 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:49.528 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.528 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.528 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:49.528 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063113+0000", 2024-09-04T17:47:49.528 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.204e-06 2024-09-04T17:47:49.528 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.528 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.528 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:49.528 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063116+0000", 2024-09-04T17:47:49.528 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.3330000000000001e-06 2024-09-04T17:47:49.528 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.528 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.529 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:49.529 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063126+0000", 2024-09-04T17:47:49.529 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0166999999999999e-05 2024-09-04T17:47:49.529 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.529 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.529 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:49.529 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063134+0000", 2024-09-04T17:47:49.529 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4329999999999998e-06 2024-09-04T17:47:49.529 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.529 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.529 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:49.529 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211392+0000", 2024-09-04T17:47:49.529 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148258109 2024-09-04T17:47:49.529 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.529 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.530 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:49.530 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211506+0000", 2024-09-04T17:47:49.530 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000113813 2024-09-04T17:47:49.530 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.530 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.530 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:49.530 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211706+0000", 2024-09-04T17:47:49.530 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020002999999999999 2024-09-04T17:47:49.530 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:49.530 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:49.530 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:49.530 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.530 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.530 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8848 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:49.530 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:47:49.531 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 96.424873227000006, 2024-09-04T17:47:49.531 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14907967599999999, 2024-09-04T17:47:49.531 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:49.531 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:49.531 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:49.531 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:49.531 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:49.531 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:49.531 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8848 2024-09-04T17:47:49.531 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.531 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:49.531 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.531 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:49.531 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:47:49.531 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:49.532 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.532 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.532 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:49.532 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:47:49.532 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:49.532 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.532 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.532 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:49.532 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070589+0000", 2024-09-04T17:47:49.532 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.007445386 2024-09-04T17:47:49.532 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.532 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.532 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:49.532 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070600+0000", 2024-09-04T17:47:49.532 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0346999999999999e-05 2024-09-04T17:47:49.533 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.533 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.533 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:49.533 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070622+0000", 2024-09-04T17:47:49.533 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2289999999999998e-05 2024-09-04T17:47:49.533 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.533 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.533 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:49.533 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070636+0000", 2024-09-04T17:47:49.533 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4093e-05 2024-09-04T17:47:49.533 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.533 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.533 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:49.533 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211815+0000", 2024-09-04T17:47:49.533 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14117874699999999 2024-09-04T17:47:49.533 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.534 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.534 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:49.534 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211983+0000", 2024-09-04T17:47:49.534 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00016852800000000001 2024-09-04T17:47:49.534 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.534 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.534 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:49.534 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.212224+0000", 2024-09-04T17:47:49.534 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024028499999999999 2024-09-04T17:47:49.534 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:49.534 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:49.534 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:49.534 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.534 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.534 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8862 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2019328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:49.534 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:47:49.535 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 96.416823758999996, 2024-09-04T17:47:49.535 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14755236399999999, 2024-09-04T17:47:49.535 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:49.535 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:49.535 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:49.535 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:49.535 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:49.535 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:49.535 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8862 2024-09-04T17:47:49.535 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.535 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:49.535 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.535 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:49.535 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:47:49.535 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:49.536 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.536 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.536 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:49.536 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:47:49.536 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:49.536 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.536 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.536 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:49.536 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071194+0000", 2024-09-04T17:47:49.536 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.5000000000000002e-07 2024-09-04T17:47:49.536 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.536 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.536 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:49.536 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071197+0000", 2024-09-04T17:47:49.536 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5900000000000002e-06 2024-09-04T17:47:49.536 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.537 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.537 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:49.537 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071206+0000", 2024-09-04T17:47:49.537 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9700000000000005e-06 2024-09-04T17:47:49.537 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.537 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.537 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:49.537 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071213+0000", 2024-09-04T17:47:49.537 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.9070000000000001e-06 2024-09-04T17:47:49.537 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.537 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.537 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:49.538 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218396+0000", 2024-09-04T17:47:49.538 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147183273 2024-09-04T17:47:49.538 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.538 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.538 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:49.538 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218538+0000", 2024-09-04T17:47:49.538 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014194199999999999 2024-09-04T17:47:49.538 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.538 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.538 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:49.538 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218746+0000", 2024-09-04T17:47:49.538 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000208032 2024-09-04T17:47:49.538 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:49.538 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:49.539 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:49.539 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.539 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.539 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8863 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2056192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:49.539 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:47:49.539 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 96.416795449000006, 2024-09-04T17:47:49.539 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14792281099999999, 2024-09-04T17:47:49.539 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:49.539 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:49.539 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:49.539 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:49.539 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:49.539 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:49.540 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8863 2024-09-04T17:47:49.540 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.540 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:49.540 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.540 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:49.540 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:47:49.540 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:49.540 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.540 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.540 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:49.540 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:47:49.540 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:49.540 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.540 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.540 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:49.541 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:47:49.541 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1299999999999999e-07 2024-09-04T17:47:49.541 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.541 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.541 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:49.541 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071225+0000", 2024-09-04T17:47:49.541 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6199999999999999e-06 2024-09-04T17:47:49.541 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.541 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.541 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:49.541 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071234+0000", 2024-09-04T17:47:49.541 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8769999999999999e-06 2024-09-04T17:47:49.541 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.541 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.541 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:49.542 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071251+0000", 2024-09-04T17:47:49.542 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.738e-05 2024-09-04T17:47:49.542 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.542 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.542 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:49.542 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218798+0000", 2024-09-04T17:47:49.542 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14754645 2024-09-04T17:47:49.542 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.542 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.542 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:49.542 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218926+0000", 2024-09-04T17:47:49.542 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000128446 2024-09-04T17:47:49.542 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.543 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.543 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:49.543 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219145+0000", 2024-09-04T17:47:49.543 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021832500000000001 2024-09-04T17:47:49.543 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:49.543 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:49.543 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:49.543 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.543 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.543 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8864 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2093056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:49.543 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:47:49.543 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 96.416753873000005, 2024-09-04T17:47:49.543 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148277309, 2024-09-04T17:47:49.543 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:49.543 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:49.544 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:49.544 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:49.544 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:49.544 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:49.544 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8864 2024-09-04T17:47:49.544 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.544 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:49.544 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.544 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:49.544 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:47:49.544 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:49.544 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.544 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.544 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:49.544 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:47:49.545 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:49.545 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.545 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.545 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:49.545 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071264+0000", 2024-09-04T17:47:49.545 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3e-07 2024-09-04T17:47:49.545 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.545 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.545 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:49.545 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071267+0000", 2024-09-04T17:47:49.545 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0139999999999999e-06 2024-09-04T17:47:49.545 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.545 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.546 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:49.546 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071276+0000", 2024-09-04T17:47:49.546 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1689999999999998e-06 2024-09-04T17:47:49.546 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.546 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.546 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:49.546 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071283+0000", 2024-09-04T17:47:49.546 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2269999999999996e-06 2024-09-04T17:47:49.546 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.546 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.546 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:49.546 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219195+0000", 2024-09-04T17:47:49.546 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147911868 2024-09-04T17:47:49.546 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.547 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.547 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:49.547 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219324+0000", 2024-09-04T17:47:49.547 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00012835900000000001 2024-09-04T17:47:49.547 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.547 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.547 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:49.547 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219541+0000", 2024-09-04T17:47:49.547 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000216942 2024-09-04T17:47:49.547 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:49.547 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:49.547 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:49.547 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.548 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.548 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8865 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2109440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:49.548 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:47:49.548 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 96.416724412999997, 2024-09-04T17:47:49.548 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14865747300000001, 2024-09-04T17:47:49.548 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:49.548 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:49.548 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:49.548 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:49.548 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:49.548 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:49.548 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8865 2024-09-04T17:47:49.548 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.548 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:49.549 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.549 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:49.549 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:47:49.549 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:49.549 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.549 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.549 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:49.549 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:47:49.549 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:49.549 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.549 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.549 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:49.549 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:47:49.549 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7999999999999995e-07 2024-09-04T17:47:49.550 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.550 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.550 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:49.550 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071296+0000", 2024-09-04T17:47:49.550 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6900000000000001e-06 2024-09-04T17:47:49.550 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.550 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.550 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:49.550 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071305+0000", 2024-09-04T17:47:49.550 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.5499999999999995e-06 2024-09-04T17:47:49.550 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.550 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.550 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:49.550 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071312+0000", 2024-09-04T17:47:49.551 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1559999999999998e-06 2024-09-04T17:47:49.551 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.551 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.551 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:49.551 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219591+0000", 2024-09-04T17:47:49.551 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14827944900000001 2024-09-04T17:47:49.551 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.551 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.551 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:49.551 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219731+0000", 2024-09-04T17:47:49.551 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013931600000000001 2024-09-04T17:47:49.551 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.551 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.551 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:49.551 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219950+0000", 2024-09-04T17:47:49.552 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021963199999999999 2024-09-04T17:47:49.552 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:49.552 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:49.552 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:49.552 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.552 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.552 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8866 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2146304~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:49.552 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:47:49.552 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 96.416696180000002, 2024-09-04T17:47:49.552 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14906117899999999, 2024-09-04T17:47:49.552 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:49.552 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:49.553 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:49.553 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:49.553 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:49.553 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:49.553 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8866 2024-09-04T17:47:49.553 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.553 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:49.553 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.553 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:49.553 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:47:49.553 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:49.553 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.553 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.553 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:49.554 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:47:49.554 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:49.554 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.554 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.554 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:49.554 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071322+0000", 2024-09-04T17:47:49.554 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7999999999999995e-07 2024-09-04T17:47:49.554 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.554 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.554 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:49.554 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071338+0000", 2024-09-04T17:47:49.554 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5872999999999999e-05 2024-09-04T17:47:49.554 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.554 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.554 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:49.555 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071347+0000", 2024-09-04T17:47:49.555 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4669999999999993e-06 2024-09-04T17:47:49.555 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.555 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.555 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:49.555 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071354+0000", 2024-09-04T17:47:49.555 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2099999999999996e-06 2024-09-04T17:47:49.555 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.555 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.555 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:49.555 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220008+0000", 2024-09-04T17:47:49.555 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148653642 2024-09-04T17:47:49.555 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.555 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.556 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:49.556 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220155+0000", 2024-09-04T17:47:49.556 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014726600000000001 2024-09-04T17:47:49.556 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.556 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.556 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:49.556 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220382+0000", 2024-09-04T17:47:49.556 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00022704100000000001 2024-09-04T17:47:49.556 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:49.556 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:49.556 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:49.556 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.556 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.556 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8867 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2183168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:49.557 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:47:49.557 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 96.416652826999993, 2024-09-04T17:47:49.557 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.149445831, 2024-09-04T17:47:49.557 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:49.557 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:49.557 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:49.557 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:49.557 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:49.557 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:49.557 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8867 2024-09-04T17:47:49.557 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.557 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:49.557 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.557 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:49.558 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:47:49.558 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:49.558 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.558 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.558 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:49.558 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:47:49.558 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:49.558 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.558 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.558 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:49.558 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078653+0000", 2024-09-04T17:47:49.558 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0072890430000000003 2024-09-04T17:47:49.558 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.558 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.558 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:49.559 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078664+0000", 2024-09-04T17:47:49.559 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0307e-05 2024-09-04T17:47:49.559 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.559 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.559 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:49.559 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078690+0000", 2024-09-04T17:47:49.559 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6780000000000001e-05 2024-09-04T17:47:49.559 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.559 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.559 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:49.559 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078706+0000", 2024-09-04T17:47:49.559 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5676999999999999e-05 2024-09-04T17:47:49.559 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.560 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.560 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:49.560 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220467+0000", 2024-09-04T17:47:49.560 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14176103500000001 2024-09-04T17:47:49.560 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.560 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.560 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:49.560 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220595+0000", 2024-09-04T17:47:49.560 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000127427 2024-09-04T17:47:49.560 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.560 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.560 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:49.561 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220810+0000", 2024-09-04T17:47:49.561 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021556200000000001 2024-09-04T17:47:49.561 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:49.561 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:49.561 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:49.561 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.561 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.561 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8880 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2658304~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:49.561 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.079158+0000", 2024-09-04T17:47:49.561 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 96.408859547999995, 2024-09-04T17:47:49.561 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14718968499999999, 2024-09-04T17:47:49.561 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:49.561 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:49.561 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:49.561 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:49.562 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:49.562 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:49.562 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8880 2024-09-04T17:47:49.562 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.562 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:49.562 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.562 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:49.562 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079158+0000", 2024-09-04T17:47:49.562 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:49.562 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.562 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.562 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:49.562 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079158+0000", 2024-09-04T17:47:49.562 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:49.563 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.563 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.563 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:49.563 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079158+0000", 2024-09-04T17:47:49.563 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3e-07 2024-09-04T17:47:49.563 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.563 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.563 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:49.563 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079161+0000", 2024-09-04T17:47:49.563 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.57e-06 2024-09-04T17:47:49.563 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.563 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.563 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:49.563 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079170+0000", 2024-09-04T17:47:49.564 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.0170000000000002e-06 2024-09-04T17:47:49.564 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.564 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.564 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:49.564 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079177+0000", 2024-09-04T17:47:49.564 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.0530000000000001e-06 2024-09-04T17:47:49.564 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.564 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.564 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:49.564 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.225946+0000", 2024-09-04T17:47:49.564 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14676936500000001 2024-09-04T17:47:49.565 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.565 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.565 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:49.565 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226098+0000", 2024-09-04T17:47:49.565 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000151805 2024-09-04T17:47:49.565 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.565 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.565 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:49.565 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226347+0000", 2024-09-04T17:47:49.565 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024914499999999999 2024-09-04T17:47:49.565 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:49.565 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:49.565 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:49.565 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.566 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.566 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8881 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2695168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:49.566 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:47:49.566 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 96.408830757999993, 2024-09-04T17:47:49.566 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147543547, 2024-09-04T17:47:49.566 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:49.566 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:49.566 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:49.566 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:49.566 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:49.566 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:49.566 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8881 2024-09-04T17:47:49.566 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.566 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:49.566 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.567 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:49.567 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:47:49.567 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:49.567 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.567 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.567 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:49.567 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:47:49.567 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:49.567 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.567 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.567 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:49.567 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079187+0000", 2024-09-04T17:47:49.568 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7700000000000004e-07 2024-09-04T17:47:49.568 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.568 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.568 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:49.568 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079198+0000", 2024-09-04T17:47:49.568 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0849999999999999e-05 2024-09-04T17:47:49.568 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.568 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.568 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:49.568 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079207+0000", 2024-09-04T17:47:49.568 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5599999999999999e-06 2024-09-04T17:47:49.568 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.568 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.568 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:49.569 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079224+0000", 2024-09-04T17:47:49.569 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6387000000000001e-05 2024-09-04T17:47:49.569 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.569 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.569 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:49.569 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226414+0000", 2024-09-04T17:47:49.569 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14718975000000001 2024-09-04T17:47:49.569 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.569 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.569 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:49.569 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226534+0000", 2024-09-04T17:47:49.569 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000120288 2024-09-04T17:47:49.569 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.569 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.570 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:49.570 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226730+0000", 2024-09-04T17:47:49.570 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000196035 2024-09-04T17:47:49.570 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:49.570 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:49.570 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:49.570 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.570 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.570 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8882 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2732032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:49.570 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:47:49.570 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 96.408782544000005, 2024-09-04T17:47:49.570 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14795022399999999, 2024-09-04T17:47:49.571 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:49.571 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:49.571 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:49.571 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:49.571 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:49.571 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:49.571 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8882 2024-09-04T17:47:49.571 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.571 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:49.571 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.571 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:49.571 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:47:49.571 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:49.571 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.571 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.572 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:49.572 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:47:49.572 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:49.572 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.572 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.572 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:49.572 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086588+0000", 2024-09-04T17:47:49.572 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0073530959999999999 2024-09-04T17:47:49.572 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.572 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.572 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:49.572 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086598+0000", 2024-09-04T17:47:49.572 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0122999999999999e-05 2024-09-04T17:47:49.572 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.572 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.573 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:49.573 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086626+0000", 2024-09-04T17:47:49.573 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.7914000000000001e-05 2024-09-04T17:47:49.573 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.573 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.573 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:49.573 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086641+0000", 2024-09-04T17:47:49.573 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5270000000000001e-05 2024-09-04T17:47:49.573 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.573 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.573 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:49.573 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226801+0000", 2024-09-04T17:47:49.573 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.140160321 2024-09-04T17:47:49.573 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.573 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.573 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:49.574 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226951+0000", 2024-09-04T17:47:49.574 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014965200000000001 2024-09-04T17:47:49.574 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:49.574 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:49.574 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:49.574 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.227185+0000", 2024-09-04T17:47:49.574 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00023384799999999999 2024-09-04T17:47:49.574 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:49.574 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:49.574 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:49.574 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:49.574 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:49.574 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:49.576 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-04T17:47:49.576 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:49.571+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:49.629 INFO:teuthology.orchestra.run.smithi092.stdout:5575 trunc from 0xe7b00fa to 0x4d0001b 2024-09-04T17:47:49.676 INFO:teuthology.orchestra.run.smithi092.stdout:5576 writesame 0x9919d49 thru 0x992521e data_size 0x4f(0xb4d6 bytes) 2024-09-04T17:47:49.677 INFO:teuthology.orchestra.run.smithi092.stdout:5577 trunc from 0x992521f to 0x9ac5105 2024-09-04T17:47:49.677 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:49.675+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:49.686 INFO:teuthology.orchestra.run.smithi092.stdout:5584 read 0x7a641d6 thru 0x7a65465 (0x1290 bytes) 2024-09-04T17:47:49.687 INFO:teuthology.orchestra.run.smithi092.stdout:5586 read 0xee7432 thru 0xeee8b8 (0x7487 bytes) 2024-09-04T17:47:49.705 INFO:teuthology.orchestra.run.smithi092.stdout:5587 write 0xd5e2a10 thru 0xd5f1a2c (0xf01d bytes) 2024-09-04T17:47:49.712 INFO:teuthology.orchestra.run.smithi092.stdout:5588 read 0xad8bc8e thru 0xad96cf6 (0xb069 bytes) 2024-09-04T17:47:49.712 INFO:teuthology.orchestra.run.smithi092.stdout:5589 write 0x5223880 thru 0x52318e0 (0xe061 bytes) 2024-09-04T17:47:49.716 INFO:teuthology.orchestra.run.smithi092.stdout:5591 punch from 0x7b1421c to 0x7b1a77b, (0x655f bytes) 2024-09-04T17:47:49.716 INFO:teuthology.orchestra.run.smithi092.stdout:5592 read 0xbfa24d4 thru 0xbfa629e (0x3dcb bytes) 2024-09-04T17:47:49.716 INFO:teuthology.orchestra.run.smithi092.stdout:5593 read 0x60d37cd thru 0x60db248 (0x7a7c bytes) 2024-09-04T17:47:49.739 INFO:teuthology.orchestra.run.smithi092.stdout:5594 trunc from 0xd5f1a2d to 0x26d48b5 2024-09-04T17:47:49.755 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:49.755 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:47:49.755 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:47:49.755 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:49.770 INFO:teuthology.orchestra.run.smithi092.stdout:5597 write 0x1f1199e thru 0x1f18ce6 (0x7349 bytes) 2024-09-04T17:47:49.771 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-04T17:47:49.778 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:49.776+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:49.801 INFO:teuthology.orchestra.run.smithi092.stdout:5598 writesame 0xc1225fc thru 0xc129a58 data_size 0x1(0x745d bytes) 2024-09-04T17:47:49.801 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:49.801 INFO:teuthology.orchestra.run.smithi092.stdout:5600 read 0x22a7485 thru 0x22adedf (0x6a5b bytes) 2024-09-04T17:47:49.802 INFO:teuthology.orchestra.run.smithi092.stdout:5601 read 0x67b9115 thru 0x67c5820 (0xc70c bytes) 2024-09-04T17:47:49.802 INFO:teuthology.orchestra.run.smithi092.stdout:5602 punch from 0x2a48abd to 0x2a497b0, (0xcf3 bytes) 2024-09-04T17:47:49.802 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:49.802 INFO:teuthology.orchestra.run.smithi092.stdout:5604 write 0x73563a6 thru 0x7359925 (0x3580 bytes) 2024-09-04T17:47:49.807 INFO:teuthology.orchestra.run.smithi092.stdout:5605 read 0x6d61739 thru 0x6d6d64e (0xbf16 bytes) 2024-09-04T17:47:49.819 INFO:teuthology.orchestra.run.smithi092.stdout:5609 writesame 0xce6f408 thru 0xce7d733 data_size 0x86(0xe32c bytes) 2024-09-04T17:47:49.819 INFO:teuthology.orchestra.run.smithi092.stdout:5610 write 0x608cd58 thru 0x608db72 (0xe1b bytes) 2024-09-04T17:47:49.825 INFO:teuthology.orchestra.run.smithi092.stdout:5611 write 0x56759b0 thru 0x5680c2f (0xb280 bytes) 2024-09-04T17:47:49.827 INFO:teuthology.orchestra.run.smithi092.stdout:5612 read 0x187c8e7 thru 0x187fabe (0x31d8 bytes) 2024-09-04T17:47:49.828 INFO:teuthology.orchestra.run.smithi092.stdout:5613 writesame 0xc46edf1 thru 0xc47c79c data_size 0x2(0xd9ac bytes) 2024-09-04T17:47:49.852 INFO:teuthology.orchestra.run.smithi092.stdout:5617 clone 39 order 24 su 2097152 sc 4 2024-09-04T17:47:49.881 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:47:49.880+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:47:49.937 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:49.937 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:47:49.937 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:47:49.937 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:47:49.937 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:49.951 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-04T17:47:49.982 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:47:49.980+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:47:49.988 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is unset 2024-09-04T17:47:50.083 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:50.079+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:50.085 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:50.085 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:47:50.085 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:47:50.086 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:47:50.086 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.086 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60921 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 262144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:50.086 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822920+0000", 2024-09-04T17:47:50.086 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 36.261713268000001, 2024-09-04T17:47:50.086 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073904394999999998, 2024-09-04T17:47:50.086 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:50.086 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:50.086 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:50.086 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:50.086 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:50.086 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:50.086 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60921 2024-09-04T17:47:50.087 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.087 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:50.087 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.087 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:50.087 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822920+0000", 2024-09-04T17:47:50.087 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:50.087 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.087 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.087 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:50.087 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822920+0000", 2024-09-04T17:47:50.087 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:50.087 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.087 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.088 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:50.088 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822920+0000", 2024-09-04T17:47:50.088 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4999999999999999e-07 2024-09-04T17:47:50.088 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.088 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.088 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:50.088 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822921+0000", 2024-09-04T17:47:50.088 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.113e-06 2024-09-04T17:47:50.088 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.088 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.088 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:50.089 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822924+0000", 2024-09-04T17:47:50.089 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.354e-06 2024-09-04T17:47:50.089 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.089 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.089 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:50.089 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822927+0000", 2024-09-04T17:47:50.089 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6199999999999999e-06 2024-09-04T17:47:50.089 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.089 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.089 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:50.089 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896586+0000", 2024-09-04T17:47:50.089 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073658976000000001 2024-09-04T17:47:50.090 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.090 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.090 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:50.090 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896626+0000", 2024-09-04T17:47:50.090 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0484000000000001e-05 2024-09-04T17:47:50.090 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.090 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.090 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:50.090 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896824+0000", 2024-09-04T17:47:50.090 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000197598 2024-09-04T17:47:50.090 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:50.090 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:50.090 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:50.090 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.091 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.091 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60922 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:50.091 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822930+0000", 2024-09-04T17:47:50.091 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 36.261702692999997, 2024-09-04T17:47:50.091 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074178872000000007, 2024-09-04T17:47:50.091 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:50.091 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:50.091 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:50.091 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:50.091 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:50.091 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:50.091 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60922 2024-09-04T17:47:50.091 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.091 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:50.092 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.092 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:50.092 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822930+0000", 2024-09-04T17:47:50.092 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:50.092 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.092 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.092 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:50.092 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822930+0000", 2024-09-04T17:47:50.092 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:50.092 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.093 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.093 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:50.093 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822930+0000", 2024-09-04T17:47:50.093 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.36e-07 2024-09-04T17:47:50.093 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.093 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.093 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:50.093 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822932+0000", 2024-09-04T17:47:50.093 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.567e-06 2024-09-04T17:47:50.093 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.093 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.093 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:50.094 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822938+0000", 2024-09-04T17:47:50.094 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4840000000000003e-06 2024-09-04T17:47:50.094 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.094 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.094 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:50.094 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822942+0000", 2024-09-04T17:47:50.094 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2039999999999999e-06 2024-09-04T17:47:50.094 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.094 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.094 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:50.094 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896850+0000", 2024-09-04T17:47:50.094 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073907941000000005 2024-09-04T17:47:50.094 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.095 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.095 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:50.095 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896891+0000", 2024-09-04T17:47:50.095 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0964999999999998e-05 2024-09-04T17:47:50.095 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.095 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.095 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:50.095 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897109+0000", 2024-09-04T17:47:50.095 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021847500000000001 2024-09-04T17:47:50.095 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:50.095 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:50.095 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:50.095 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.096 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.096 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60923 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 475136~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:50.096 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822949+0000", 2024-09-04T17:47:50.096 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 36.261684402999997, 2024-09-04T17:47:50.096 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074426928000000003, 2024-09-04T17:47:50.096 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:50.096 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:50.096 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:50.096 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:50.096 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:50.096 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:50.096 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60923 2024-09-04T17:47:50.097 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.097 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:50.097 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.097 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:50.097 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822949+0000", 2024-09-04T17:47:50.097 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:50.097 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.097 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.097 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:50.097 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822949+0000", 2024-09-04T17:47:50.097 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:50.097 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.098 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.098 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:50.098 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822949+0000", 2024-09-04T17:47:50.098 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7399999999999999e-07 2024-09-04T17:47:50.098 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.098 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.098 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:50.098 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822951+0000", 2024-09-04T17:47:50.098 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1940000000000001e-06 2024-09-04T17:47:50.098 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.098 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.098 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:50.098 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822957+0000", 2024-09-04T17:47:50.098 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.0650000000000004e-06 2024-09-04T17:47:50.098 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.099 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.099 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:50.099 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822960+0000", 2024-09-04T17:47:50.099 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.8009999999999999e-06 2024-09-04T17:47:50.099 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.099 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.099 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:50.099 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897138+0000", 2024-09-04T17:47:50.099 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074177990999999999 2024-09-04T17:47:50.099 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.099 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.099 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:50.099 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897179+0000", 2024-09-04T17:47:50.100 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.1057999999999999e-05 2024-09-04T17:47:50.100 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.100 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.100 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:50.100 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897375+0000", 2024-09-04T17:47:50.100 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019644500000000001 2024-09-04T17:47:50.100 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:50.100 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:50.100 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:50.100 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.100 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.100 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60924 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:50.100 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822964+0000", 2024-09-04T17:47:50.101 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 36.261669218000002, 2024-09-04T17:47:50.101 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074531902999999997, 2024-09-04T17:47:50.101 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:50.101 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:50.101 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:50.101 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:50.101 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:50.101 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:50.101 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60924 2024-09-04T17:47:50.101 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.101 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:50.102 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.102 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:50.102 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822964+0000", 2024-09-04T17:47:50.102 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:50.102 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.102 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.102 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:50.102 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822964+0000", 2024-09-04T17:47:50.102 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:50.103 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.103 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.103 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:50.103 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822964+0000", 2024-09-04T17:47:50.103 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4999999999999999e-07 2024-09-04T17:47:50.103 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.103 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.103 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:50.103 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822966+0000", 2024-09-04T17:47:50.103 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.917e-06 2024-09-04T17:47:50.103 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.104 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.104 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:50.104 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822969+0000", 2024-09-04T17:47:50.104 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.315e-06 2024-09-04T17:47:50.104 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.104 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.104 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:50.104 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822972+0000", 2024-09-04T17:47:50.104 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.599e-06 2024-09-04T17:47:50.104 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.104 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.104 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:50.105 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897398+0000", 2024-09-04T17:47:50.105 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074426168000000001 2024-09-04T17:47:50.105 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.105 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.105 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:50.105 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897441+0000", 2024-09-04T17:47:50.105 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.3158000000000003e-05 2024-09-04T17:47:50.105 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.105 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.105 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:50.105 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897496+0000", 2024-09-04T17:47:50.105 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4496000000000001e-05 2024-09-04T17:47:50.106 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:50.106 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:50.106 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:50.106 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.106 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.106 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60925 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 720896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:50.106 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822975+0000", 2024-09-04T17:47:50.106 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 36.261657782999997, 2024-09-04T17:47:50.106 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074635272000000003, 2024-09-04T17:47:50.106 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:50.106 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:50.106 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:50.106 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:50.106 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:50.107 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:50.107 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60925 2024-09-04T17:47:50.107 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.107 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:50.107 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.107 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:50.107 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822975+0000", 2024-09-04T17:47:50.107 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:50.107 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.107 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.107 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:50.107 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822975+0000", 2024-09-04T17:47:50.108 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:50.108 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.108 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.108 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:50.108 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822975+0000", 2024-09-04T17:47:50.108 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.34e-07 2024-09-04T17:47:50.108 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.108 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.108 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:50.108 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822976+0000", 2024-09-04T17:47:50.108 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0359999999999999e-06 2024-09-04T17:47:50.108 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.108 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.109 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:50.109 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822980+0000", 2024-09-04T17:47:50.109 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.242e-06 2024-09-04T17:47:50.109 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.109 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.109 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:50.109 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822983+0000", 2024-09-04T17:47:50.109 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.8830000000000002e-06 2024-09-04T17:47:50.109 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.109 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.109 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:50.109 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897519+0000", 2024-09-04T17:47:50.110 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074536366000000007 2024-09-04T17:47:50.110 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.110 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.110 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:50.110 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897560+0000", 2024-09-04T17:47:50.110 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0677999999999999e-05 2024-09-04T17:47:50.110 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.110 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.110 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:50.110 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897610+0000", 2024-09-04T17:47:50.110 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0832999999999999e-05 2024-09-04T17:47:50.110 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:50.110 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:50.110 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:50.111 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.111 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.111 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60926 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 860160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:50.111 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822986+0000", 2024-09-04T17:47:50.111 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 36.261647185000001, 2024-09-04T17:47:50.111 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074736413000000002, 2024-09-04T17:47:50.111 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:50.111 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:50.111 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:50.111 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:50.111 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:50.111 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:50.111 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60926 2024-09-04T17:47:50.111 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.112 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:50.112 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.112 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:50.112 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822986+0000", 2024-09-04T17:47:50.112 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:50.112 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.112 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.112 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:50.112 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822986+0000", 2024-09-04T17:47:50.112 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:50.112 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.113 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.113 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:50.113 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822986+0000", 2024-09-04T17:47:50.113 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5600000000000002e-07 2024-09-04T17:47:50.113 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.113 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.113 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:50.113 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822987+0000", 2024-09-04T17:47:50.114 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.082e-06 2024-09-04T17:47:50.114 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.114 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.114 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:50.114 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822990+0000", 2024-09-04T17:47:50.114 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.2219999999999998e-06 2024-09-04T17:47:50.114 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.114 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.114 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:50.114 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822994+0000", 2024-09-04T17:47:50.114 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.19e-06 2024-09-04T17:47:50.115 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.115 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.115 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:50.115 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897632+0000", 2024-09-04T17:47:50.115 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074638205999999999 2024-09-04T17:47:50.115 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.116 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.116 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:50.116 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897672+0000", 2024-09-04T17:47:50.116 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.9873999999999998e-05 2024-09-04T17:47:50.116 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.116 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.116 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:50.116 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897722+0000", 2024-09-04T17:47:50.116 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0583e-05 2024-09-04T17:47:50.116 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:50.117 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:50.117 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:50.117 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.117 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.117 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60927 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 966656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:50.117 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823000+0000", 2024-09-04T17:47:50.117 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 36.261633003, 2024-09-04T17:47:50.117 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074834036000000007, 2024-09-04T17:47:50.117 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:50.117 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:50.118 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:50.118 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:50.118 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:50.118 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:50.118 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60927 2024-09-04T17:47:50.118 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.118 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:50.118 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.118 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:50.118 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823000+0000", 2024-09-04T17:47:50.118 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:50.118 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.118 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.119 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:50.119 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823000+0000", 2024-09-04T17:47:50.119 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:50.119 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.119 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.119 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:50.119 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823000+0000", 2024-09-04T17:47:50.119 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4099999999999999e-07 2024-09-04T17:47:50.119 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.119 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.119 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:50.119 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823003+0000", 2024-09-04T17:47:50.119 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0810000000000001e-06 2024-09-04T17:47:50.120 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.120 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.120 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:50.120 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823011+0000", 2024-09-04T17:47:50.120 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7090000000000008e-06 2024-09-04T17:47:50.120 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.120 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.120 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:50.120 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823017+0000", 2024-09-04T17:47:50.120 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.3890000000000004e-06 2024-09-04T17:47:50.120 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.120 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.120 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:50.121 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897745+0000", 2024-09-04T17:47:50.121 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074728252999999994 2024-09-04T17:47:50.121 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.121 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.121 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:50.121 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897785+0000", 2024-09-04T17:47:50.121 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.9830999999999999e-05 2024-09-04T17:47:50.121 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.121 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.121 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:50.121 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897834+0000", 2024-09-04T17:47:50.121 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9231999999999999e-05 2024-09-04T17:47:50.122 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:50.122 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:50.122 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:50.122 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.122 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.122 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60928 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:50.122 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823024+0000", 2024-09-04T17:47:50.122 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 36.261609071999999, 2024-09-04T17:47:50.122 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075067479000000006, 2024-09-04T17:47:50.122 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:50.122 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:50.122 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:50.122 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:50.123 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:50.123 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:50.123 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60928 2024-09-04T17:47:50.123 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.123 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:50.123 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.123 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:50.123 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823024+0000", 2024-09-04T17:47:50.124 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:50.124 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.124 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.124 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:50.124 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823024+0000", 2024-09-04T17:47:50.124 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:50.124 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.124 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.125 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:50.125 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823024+0000", 2024-09-04T17:47:50.125 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6499999999999999e-07 2024-09-04T17:47:50.125 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.125 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.125 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:50.125 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823027+0000", 2024-09-04T17:47:50.125 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.3099999999999999e-06 2024-09-04T17:47:50.125 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.125 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.125 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:50.126 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823033+0000", 2024-09-04T17:47:50.126 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7429999999999996e-06 2024-09-04T17:47:50.126 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.126 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.126 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:50.126 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823038+0000", 2024-09-04T17:47:50.126 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.561e-06 2024-09-04T17:47:50.126 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.126 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.126 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:50.126 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897855+0000", 2024-09-04T17:47:50.126 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074816740000000007 2024-09-04T17:47:50.127 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.127 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.127 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:50.127 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897894+0000", 2024-09-04T17:47:50.127 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.9085999999999998e-05 2024-09-04T17:47:50.127 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.127 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.127 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:50.127 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898091+0000", 2024-09-04T17:47:50.127 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000197574 2024-09-04T17:47:50.127 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:50.127 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:50.127 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:50.127 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.127 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.128 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60929 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1212416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:50.128 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823042+0000", 2024-09-04T17:47:50.128 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 36.261590630000001, 2024-09-04T17:47:50.128 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075301251, 2024-09-04T17:47:50.128 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:50.128 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:50.128 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:50.128 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:50.128 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:50.128 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:50.128 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60929 2024-09-04T17:47:50.128 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.128 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:50.129 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.129 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:50.129 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823042+0000", 2024-09-04T17:47:50.129 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:50.129 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.129 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.130 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:50.130 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823042+0000", 2024-09-04T17:47:50.130 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:50.130 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.130 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.130 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:50.130 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823043+0000", 2024-09-04T17:47:50.130 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.1100000000000002e-07 2024-09-04T17:47:50.130 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.130 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.130 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:50.131 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823045+0000", 2024-09-04T17:47:50.131 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.933e-06 2024-09-04T17:47:50.131 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.131 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.131 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:50.131 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823051+0000", 2024-09-04T17:47:50.131 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.9569999999999999e-06 2024-09-04T17:47:50.131 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.131 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.131 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:50.131 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823055+0000", 2024-09-04T17:47:50.131 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4479999999999996e-06 2024-09-04T17:47:50.131 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.131 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.132 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:50.132 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898117+0000", 2024-09-04T17:47:50.132 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075062344000000003 2024-09-04T17:47:50.132 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.132 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.132 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:50.132 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898157+0000", 2024-09-04T17:47:50.132 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0095999999999997e-05 2024-09-04T17:47:50.132 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.132 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.132 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:50.132 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898344+0000", 2024-09-04T17:47:50.132 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000186162 2024-09-04T17:47:50.132 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:50.132 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:50.133 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:50.133 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.133 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.133 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60930 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1351680~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:50.133 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823061+0000", 2024-09-04T17:47:50.133 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 36.261571750999998, 2024-09-04T17:47:50.133 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075545507999999997, 2024-09-04T17:47:50.133 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:50.133 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:50.133 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:50.133 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:50.134 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:50.134 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:50.134 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60930 2024-09-04T17:47:50.134 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.134 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:50.134 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.134 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:50.134 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823061+0000", 2024-09-04T17:47:50.134 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:50.134 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.135 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.135 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:50.135 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823061+0000", 2024-09-04T17:47:50.135 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:50.135 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.135 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.135 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:50.135 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823378+0000", 2024-09-04T17:47:50.135 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00031631299999999998 2024-09-04T17:47:50.135 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.135 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.135 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:50.135 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823381+0000", 2024-09-04T17:47:50.136 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4020000000000002e-06 2024-09-04T17:47:50.136 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.136 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.136 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:50.136 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823389+0000", 2024-09-04T17:47:50.136 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.1429999999999998e-06 2024-09-04T17:47:50.136 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.136 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.136 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:50.136 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823395+0000", 2024-09-04T17:47:50.136 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.0059999999999998e-06 2024-09-04T17:47:50.136 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.137 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.137 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:50.137 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898368+0000", 2024-09-04T17:47:50.137 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074973032999999994 2024-09-04T17:47:50.137 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.137 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.137 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:50.137 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898410+0000", 2024-09-04T17:47:50.137 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2243000000000002e-05 2024-09-04T17:47:50.137 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.137 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.137 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:50.137 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898607+0000", 2024-09-04T17:47:50.138 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019636799999999999 2024-09-04T17:47:50.138 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:50.138 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:50.138 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:50.138 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.138 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.138 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60931 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1458176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:50.138 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823407+0000", 2024-09-04T17:47:50.138 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 36.261225676999999, 2024-09-04T17:47:50.138 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075460334000000004, 2024-09-04T17:47:50.138 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:50.138 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:50.138 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:50.139 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:50.139 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:50.139 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:50.139 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60931 2024-09-04T17:47:50.139 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.139 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:50.139 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.139 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:50.139 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823407+0000", 2024-09-04T17:47:50.139 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:50.139 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.139 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.140 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:50.140 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823407+0000", 2024-09-04T17:47:50.140 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:50.140 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.140 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.140 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:50.140 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823408+0000", 2024-09-04T17:47:50.140 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9999999999999998e-07 2024-09-04T17:47:50.140 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.140 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.140 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:50.140 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823415+0000", 2024-09-04T17:47:50.140 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.5510000000000002e-06 2024-09-04T17:47:50.141 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.141 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.141 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:50.141 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823423+0000", 2024-09-04T17:47:50.141 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2760000000000003e-06 2024-09-04T17:47:50.141 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.141 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.141 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:50.141 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823426+0000", 2024-09-04T17:47:50.141 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.2839999999999998e-06 2024-09-04T17:47:50.141 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.141 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.142 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:50.142 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898631+0000", 2024-09-04T17:47:50.142 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075205045999999998 2024-09-04T17:47:50.142 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.142 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.142 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:50.142 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898672+0000", 2024-09-04T17:47:50.142 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0614000000000001e-05 2024-09-04T17:47:50.142 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.142 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.142 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:50.142 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898868+0000", 2024-09-04T17:47:50.142 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019606300000000001 2024-09-04T17:47:50.142 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:50.143 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:50.143 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:50.143 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.143 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.143 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60932 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1564672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:50.143 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823434+0000", 2024-09-04T17:47:50.143 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 36.261199378000001, 2024-09-04T17:47:50.143 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075693393999999997, 2024-09-04T17:47:50.143 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:50.143 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:50.143 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:50.143 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:50.143 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:50.144 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:50.144 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60932 2024-09-04T17:47:50.144 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.144 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:50.144 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.144 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:50.144 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823434+0000", 2024-09-04T17:47:50.144 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:50.145 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.145 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.145 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:50.145 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823434+0000", 2024-09-04T17:47:50.145 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:50.145 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.145 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.145 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:50.145 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.830290+0000", 2024-09-04T17:47:50.145 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0068563039999999997 2024-09-04T17:47:50.145 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.146 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.146 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:50.146 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.830297+0000", 2024-09-04T17:47:50.146 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4760000000000001e-06 2024-09-04T17:47:50.146 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.146 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.146 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:50.146 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.830308+0000", 2024-09-04T17:47:50.146 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0927999999999999e-05 2024-09-04T17:47:50.146 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.146 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.146 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:50.146 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.830317+0000", 2024-09-04T17:47:50.147 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7280000000000001e-06 2024-09-04T17:47:50.147 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.147 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.147 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:50.147 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898892+0000", 2024-09-04T17:47:50.147 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.068574722000000005 2024-09-04T17:47:50.147 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.147 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.147 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:50.147 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898934+0000", 2024-09-04T17:47:50.147 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2249000000000002e-05 2024-09-04T17:47:50.147 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.147 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.147 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:50.147 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.899127+0000", 2024-09-04T17:47:50.148 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019298699999999999 2024-09-04T17:47:50.148 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:50.148 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:50.148 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:50.148 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.148 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.148 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61182 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 868352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:50.148 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839249+0000", 2024-09-04T17:47:50.148 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 36.245383486000001, 2024-09-04T17:47:50.148 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073680151999999999, 2024-09-04T17:47:50.148 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:50.148 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:50.149 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:50.149 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:50.149 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:50.149 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:50.149 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61182 2024-09-04T17:47:50.149 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.149 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:50.149 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.149 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:50.149 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839249+0000", 2024-09-04T17:47:50.149 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:50.150 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.150 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.150 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:50.150 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839249+0000", 2024-09-04T17:47:50.150 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:50.150 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.150 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.150 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:50.150 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839250+0000", 2024-09-04T17:47:50.150 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4999999999999998e-07 2024-09-04T17:47:50.150 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.150 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.151 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:50.151 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839257+0000", 2024-09-04T17:47:50.151 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7700000000000004e-06 2024-09-04T17:47:50.151 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.151 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.151 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:50.151 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839263+0000", 2024-09-04T17:47:50.151 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.246e-06 2024-09-04T17:47:50.151 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.151 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.151 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:50.151 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839268+0000", 2024-09-04T17:47:50.151 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.814e-06 2024-09-04T17:47:50.151 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.152 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.152 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:50.152 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.912831+0000", 2024-09-04T17:47:50.152 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073563294000000001 2024-09-04T17:47:50.152 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.152 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.152 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:50.152 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.912876+0000", 2024-09-04T17:47:50.152 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4742e-05 2024-09-04T17:47:50.152 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.152 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.152 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:50.153 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.912930+0000", 2024-09-04T17:47:50.153 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.3936e-05 2024-09-04T17:47:50.153 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:50.153 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:50.153 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:50.153 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.153 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.153 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61183 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:50.153 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839275+0000", 2024-09-04T17:47:50.153 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 36.245358293000002, 2024-09-04T17:47:50.153 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073781886000000005, 2024-09-04T17:47:50.153 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:50.154 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:50.154 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:50.154 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:50.154 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:50.154 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:50.154 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61183 2024-09-04T17:47:50.154 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.154 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:50.154 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.154 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:50.154 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839275+0000", 2024-09-04T17:47:50.155 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:50.155 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.155 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.155 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:50.155 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839275+0000", 2024-09-04T17:47:50.155 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:50.155 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.155 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.155 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:50.155 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839275+0000", 2024-09-04T17:47:50.155 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6899999999999997e-07 2024-09-04T17:47:50.156 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.156 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.156 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:50.156 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839277+0000", 2024-09-04T17:47:50.156 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9530000000000002e-06 2024-09-04T17:47:50.156 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.156 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.156 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:50.156 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839283+0000", 2024-09-04T17:47:50.156 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.9329999999999998e-06 2024-09-04T17:47:50.157 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.157 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.157 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:50.157 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839288+0000", 2024-09-04T17:47:50.157 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4100000000000001e-06 2024-09-04T17:47:50.157 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.157 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.157 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:50.157 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.912959+0000", 2024-09-04T17:47:50.157 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073671511999999995 2024-09-04T17:47:50.157 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.157 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.157 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:50.158 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913001+0000", 2024-09-04T17:47:50.158 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.1984999999999999e-05 2024-09-04T17:47:50.158 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.158 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.158 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:50.158 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913057+0000", 2024-09-04T17:47:50.158 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.5523999999999999e-05 2024-09-04T17:47:50.158 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:50.158 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:50.158 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:50.158 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.158 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.158 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61184 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1114112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:50.159 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839294+0000", 2024-09-04T17:47:50.159 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 36.245339080999997, 2024-09-04T17:47:50.159 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073995263000000006, 2024-09-04T17:47:50.159 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:50.159 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:50.159 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:50.159 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:50.159 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:50.159 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:50.159 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61184 2024-09-04T17:47:50.159 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.159 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:50.159 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.160 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:50.160 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839294+0000", 2024-09-04T17:47:50.160 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:50.160 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.160 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.160 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:50.160 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839294+0000", 2024-09-04T17:47:50.160 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:50.160 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.160 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.160 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:50.160 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839294+0000", 2024-09-04T17:47:50.160 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.77e-07 2024-09-04T17:47:50.161 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.161 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.161 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:50.161 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839296+0000", 2024-09-04T17:47:50.161 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9999999999999999e-06 2024-09-04T17:47:50.161 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.161 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.161 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:50.161 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839302+0000", 2024-09-04T17:47:50.161 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7230000000000001e-06 2024-09-04T17:47:50.161 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.161 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.161 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:50.161 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839314+0000", 2024-09-04T17:47:50.162 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1826e-05 2024-09-04T17:47:50.162 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.162 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.162 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:50.162 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913084+0000", 2024-09-04T17:47:50.162 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073770215 2024-09-04T17:47:50.162 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.162 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.162 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:50.162 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913127+0000", 2024-09-04T17:47:50.162 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2982999999999998e-05 2024-09-04T17:47:50.162 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.162 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.162 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:50.163 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913289+0000", 2024-09-04T17:47:50.163 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000162139 2024-09-04T17:47:50.163 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:50.163 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:50.163 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:50.163 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.163 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.163 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61185 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1220608~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:50.163 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839321+0000", 2024-09-04T17:47:50.163 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 36.245311614000002, 2024-09-04T17:47:50.163 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074206062000000003, 2024-09-04T17:47:50.163 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:50.163 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:50.163 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:50.164 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:50.164 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:50.164 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:50.164 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61185 2024-09-04T17:47:50.164 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.164 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:50.164 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.164 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:50.164 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839321+0000", 2024-09-04T17:47:50.164 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:50.164 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.164 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.164 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:50.164 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839321+0000", 2024-09-04T17:47:50.164 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:50.165 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.165 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.165 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:50.165 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839322+0000", 2024-09-04T17:47:50.165 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.5400000000000001e-07 2024-09-04T17:47:50.165 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.165 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.165 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:50.165 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839324+0000", 2024-09-04T17:47:50.165 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2539999999999999e-06 2024-09-04T17:47:50.166 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.166 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.166 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:50.166 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839331+0000", 2024-09-04T17:47:50.166 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2139999999999999e-06 2024-09-04T17:47:50.166 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.166 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.166 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:50.166 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839336+0000", 2024-09-04T17:47:50.166 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7029999999999997e-06 2024-09-04T17:47:50.166 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.166 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.166 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:50.166 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913320+0000", 2024-09-04T17:47:50.167 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073984016 2024-09-04T17:47:50.167 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.167 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.167 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:50.167 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913366+0000", 2024-09-04T17:47:50.167 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.5704000000000002e-05 2024-09-04T17:47:50.167 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.167 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.167 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:50.167 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913527+0000", 2024-09-04T17:47:50.167 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00016161699999999999 2024-09-04T17:47:50.167 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:50.167 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:50.167 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:50.168 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.168 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.168 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61186 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1359872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:50.168 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839344+0000", 2024-09-04T17:47:50.168 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 36.245289272000001, 2024-09-04T17:47:50.168 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074450673999999994, 2024-09-04T17:47:50.168 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:50.168 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:50.168 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:50.168 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:50.168 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:50.168 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:50.168 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61186 2024-09-04T17:47:50.169 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.169 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:50.169 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.169 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:50.169 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839344+0000", 2024-09-04T17:47:50.169 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:50.169 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.169 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.169 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:50.169 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839344+0000", 2024-09-04T17:47:50.169 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:50.170 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.170 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.170 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:50.170 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839344+0000", 2024-09-04T17:47:50.170 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4799999999999999e-07 2024-09-04T17:47:50.170 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.170 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.170 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:50.170 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839346+0000", 2024-09-04T17:47:50.170 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0889999999999998e-06 2024-09-04T17:47:50.170 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.171 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.171 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:50.171 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839352+0000", 2024-09-04T17:47:50.171 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6389999999999997e-06 2024-09-04T17:47:50.171 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.171 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.171 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:50.171 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839356+0000", 2024-09-04T17:47:50.171 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.233e-06 2024-09-04T17:47:50.171 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.171 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.171 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:50.171 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913557+0000", 2024-09-04T17:47:50.171 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074200944000000005 2024-09-04T17:47:50.171 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.172 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.172 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:50.172 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913601+0000", 2024-09-04T17:47:50.172 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4350000000000001e-05 2024-09-04T17:47:50.172 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.172 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.172 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:50.172 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913794+0000", 2024-09-04T17:47:50.172 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019297100000000001 2024-09-04T17:47:50.172 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:50.172 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:50.172 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:50.172 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.172 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.173 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61187 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1466368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:50.173 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839364+0000", 2024-09-04T17:47:50.173 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 36.245269423000003, 2024-09-04T17:47:50.173 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074692334999999999, 2024-09-04T17:47:50.173 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:50.173 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:50.173 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:50.173 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:50.173 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:50.173 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:50.173 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61187 2024-09-04T17:47:50.173 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.173 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:50.173 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.173 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:50.174 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839364+0000", 2024-09-04T17:47:50.174 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:50.174 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.174 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.174 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:50.174 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839364+0000", 2024-09-04T17:47:50.174 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:50.174 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.174 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.174 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:50.174 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839364+0000", 2024-09-04T17:47:50.174 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.75e-07 2024-09-04T17:47:50.174 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.174 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.175 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:50.175 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839366+0000", 2024-09-04T17:47:50.175 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0990000000000001e-06 2024-09-04T17:47:50.175 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.175 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.175 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:50.175 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839372+0000", 2024-09-04T17:47:50.175 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.2609999999999999e-06 2024-09-04T17:47:50.175 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.175 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.175 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:50.176 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839389+0000", 2024-09-04T17:47:50.176 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6489000000000001e-05 2024-09-04T17:47:50.176 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.176 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.176 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:50.176 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913824+0000", 2024-09-04T17:47:50.176 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074434811000000004 2024-09-04T17:47:50.176 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.176 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.176 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:50.176 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913868+0000", 2024-09-04T17:47:50.176 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4768000000000002e-05 2024-09-04T17:47:50.176 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.177 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.177 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:50.177 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914056+0000", 2024-09-04T17:47:50.177 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00018743199999999999 2024-09-04T17:47:50.177 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:50.177 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:50.177 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:50.177 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.177 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.177 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61188 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1605632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:50.177 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839399+0000", 2024-09-04T17:47:50.178 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 36.245233527000003, 2024-09-04T17:47:50.178 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074975095000000005, 2024-09-04T17:47:50.178 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:50.178 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:50.178 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:50.178 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:50.178 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:50.178 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:50.178 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61188 2024-09-04T17:47:50.178 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.178 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:50.179 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.179 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:50.179 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839399+0000", 2024-09-04T17:47:50.179 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:50.179 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.179 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.179 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:50.179 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839399+0000", 2024-09-04T17:47:50.179 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:50.179 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.179 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.179 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:50.179 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839400+0000", 2024-09-04T17:47:50.180 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0100000000000005e-07 2024-09-04T17:47:50.180 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.180 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.180 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:50.180 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839402+0000", 2024-09-04T17:47:50.180 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.491e-06 2024-09-04T17:47:50.180 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.180 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.180 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:50.180 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839409+0000", 2024-09-04T17:47:50.180 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.0450000000000003e-06 2024-09-04T17:47:50.180 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.181 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.181 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:50.181 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839414+0000", 2024-09-04T17:47:50.181 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.87e-06 2024-09-04T17:47:50.181 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.181 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.181 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:50.181 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914086+0000", 2024-09-04T17:47:50.181 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074671596000000007 2024-09-04T17:47:50.181 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.181 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.181 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:50.181 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914130+0000", 2024-09-04T17:47:50.182 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4579999999999997e-05 2024-09-04T17:47:50.182 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.182 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.182 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:50.182 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914375+0000", 2024-09-04T17:47:50.182 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024401199999999999 2024-09-04T17:47:50.182 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:50.182 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:50.182 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:50.182 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.182 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.182 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61189 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1712128~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:50.182 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839424+0000", 2024-09-04T17:47:50.183 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 36.245209326000001, 2024-09-04T17:47:50.183 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075247680999999997, 2024-09-04T17:47:50.183 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:50.183 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:50.183 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:50.183 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:50.183 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:50.183 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:50.183 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61189 2024-09-04T17:47:50.183 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.183 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:50.183 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.183 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:50.183 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839424+0000", 2024-09-04T17:47:50.184 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:50.184 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.184 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.184 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:50.184 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839424+0000", 2024-09-04T17:47:50.184 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:50.184 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.184 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.184 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:50.184 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.846320+0000", 2024-09-04T17:47:50.184 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0068965210000000001 2024-09-04T17:47:50.185 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.185 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.185 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:50.185 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.846328+0000", 2024-09-04T17:47:50.185 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.8189999999999992e-06 2024-09-04T17:47:50.185 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.185 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.185 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:50.185 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.846341+0000", 2024-09-04T17:47:50.185 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3338e-05 2024-09-04T17:47:50.185 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.185 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.185 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:50.186 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.846352+0000", 2024-09-04T17:47:50.186 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0479000000000001e-05 2024-09-04T17:47:50.186 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.186 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.186 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:50.186 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914405+0000", 2024-09-04T17:47:50.186 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.068053049000000004 2024-09-04T17:47:50.186 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.186 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.186 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:50.186 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914451+0000", 2024-09-04T17:47:50.187 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6545999999999998e-05 2024-09-04T17:47:50.187 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:50.187 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:50.187 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:50.187 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914671+0000", 2024-09-04T17:47:50.187 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000219929 2024-09-04T17:47:50.187 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:50.187 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:50.187 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:50.187 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:50.187 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:50.187 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:50.188 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone38 from 0xce7d734 (overlap 0x26d48b5) to 0x1b2e618 2024-09-04T17:47:50.189 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-04T17:47:50.190 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:50.184+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:50.290 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:50.288+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:50.327 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:50.327 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:47:50.327 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:47:50.327 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:50.336 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #37, image image_client.0-clone37 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent38 2024-09-04T17:47:50.343 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-04T17:47:50.355 INFO:teuthology.orchestra.run.smithi092.stdout:5618 trunc from 0xce7d734 to 0x7b45563 2024-09-04T17:47:50.399 INFO:teuthology.orchestra.run.smithi092.stdout:5619 writesame 0xc7a78b5 thru 0xc7ab455 data_size 0x5(0x3ba1 bytes) 2024-09-04T17:47:50.399 INFO:teuthology.orchestra.run.smithi092.stdout:5621 read 0x3c0fb38 thru 0x3c0fbed (0xb6 bytes) 2024-09-04T17:47:50.399 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:50.396+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:50.400 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:50.400 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:50.400 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:50.400 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:50.406 INFO:teuthology.orchestra.run.smithi092.stdout:5629 write 0xcac00cd thru 0xcac3b45 (0x3a79 bytes) 2024-09-04T17:47:50.412 INFO:teuthology.orchestra.run.smithi092.stdout:5630 writesame 0xc77fe52 thru 0xc782c70 data_size 0x1(0x2e1f bytes) 2024-09-04T17:47:50.422 INFO:teuthology.orchestra.run.smithi092.stdout:5632 writesame 0x8c73de7 thru 0x8c80ad0 data_size 0x15(0xccea bytes) 2024-09-04T17:47:50.429 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:50.429 INFO:teuthology.orchestra.run.smithi092.stdout:5634 read 0x92e7d8f thru 0x92f6ddd (0xf04f bytes) 2024-09-04T17:47:50.429 INFO:teuthology.orchestra.run.smithi092.stdout:5635 write 0xfcebfc thru 0xfdac5d (0xc062 bytes) 2024-09-04T17:47:50.436 INFO:teuthology.orchestra.run.smithi092.stdout:5636 write 0xc3e0d85 thru 0xc3eb9ab (0xac27 bytes) 2024-09-04T17:47:50.439 INFO:teuthology.orchestra.run.smithi092.stdout:5639 punch from 0x868272e to 0x8686ad0, (0x43a2 bytes) 2024-09-04T17:47:50.440 INFO:teuthology.orchestra.run.smithi092.stdout:5640 punch from 0x4e3c454 to 0x4e3d304, (0xeb0 bytes) 2024-09-04T17:47:50.440 INFO:teuthology.orchestra.run.smithi092.stdout:5642 write 0x3483f27 thru 0x348be25 (0x7eff bytes) 2024-09-04T17:47:50.446 INFO:teuthology.orchestra.run.smithi092.stdout:5644 punch from 0x275ffa1 to 0x276e271, (0xe2d0 bytes) 2024-09-04T17:47:50.447 INFO:teuthology.orchestra.run.smithi092.stdout:5648 read 0xb3133d thru 0xb36263 (0x4f27 bytes) 2024-09-04T17:47:50.447 INFO:teuthology.orchestra.run.smithi092.stdout:5649 write 0x708572c thru 0x7087fac (0x2881 bytes) 2024-09-04T17:47:50.453 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:50.453 INFO:teuthology.orchestra.run.smithi092.stdout:5651 punch from 0xc48cfa4 to 0xc49b23c, (0xe298 bytes) 2024-09-04T17:47:50.453 INFO:teuthology.orchestra.run.smithi092.stdout:5652 write 0x15abda9 thru 0x15b53e9 (0x9641 bytes) 2024-09-04T17:47:50.456 INFO:teuthology.orchestra.run.smithi092.stdout:5653 punch from 0xa6a6b2f to 0xa6ada5b, (0x6f2c bytes) 2024-09-04T17:47:50.456 INFO:teuthology.orchestra.run.smithi092.stdout:5655 punch from 0x791a68c to 0x7924ea2, (0xa816 bytes) 2024-09-04T17:47:50.456 INFO:teuthology.orchestra.run.smithi092.stdout:5658 read 0x2864903 thru 0x28739fd (0xf0fb bytes) 2024-09-04T17:47:50.456 INFO:teuthology.orchestra.run.smithi092.stdout:5659 read 0x2332cb5 thru 0x2335bd6 (0x2f22 bytes) 2024-09-04T17:47:50.457 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:50.457 INFO:teuthology.orchestra.run.smithi092.stdout:5661 write 0x1ce660a thru 0x1ce81b1 (0x1ba8 bytes) 2024-09-04T17:47:50.459 INFO:teuthology.orchestra.run.smithi092.stdout:5662 punch from 0x4130521 to 0x4134d24, (0x4803 bytes) 2024-09-04T17:47:50.459 INFO:teuthology.orchestra.run.smithi092.stdout:5664 read 0x5516e2 thru 0x55d4cd (0xbdec bytes) 2024-09-04T17:47:50.467 INFO:teuthology.orchestra.run.smithi092.stdout:5668 trunc from 0xcac3b46 to 0x76d7149 2024-09-04T17:47:50.471 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-04T17:47:50.486 INFO:teuthology.orchestra.run.smithi092.stdout:5669 read 0x4e34095 thru 0x4e39bc0 (0x5b2c bytes) 2024-09-04T17:47:50.492 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:50.492 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:47:50.492 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:47:50.492 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:47:50.492 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:50.492 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:50.488+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:50.498 INFO:teuthology.orchestra.run.smithi092.stdout:5670 trunc from 0x76d7149 to 0xdf4d6f5 2024-09-04T17:47:50.507 INFO:teuthology.orchestra.run.smithi092.stdout:5671 write 0x912e532 thru 0x9135f06 (0x79d5 bytes) 2024-09-04T17:47:50.512 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-04T17:47:50.513 INFO:teuthology.orchestra.run.smithi092.stdout:5672 read 0x438f0e5 thru 0x439415d (0x5079 bytes) 2024-09-04T17:47:50.513 INFO:teuthology.orchestra.run.smithi092.stdout:5673 clone 40 order 18 su 16384 sc 14 2024-09-04T17:47:50.593 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:47:50.592+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:47:50.622 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:47:50.622 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:47:50.630 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:50.630 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:47:50.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:47:50.630 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:47:50.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.631 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:27242 3.b 3.34414deb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-04T17:47:50.631 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:28.402347+0000", 2024-09-04T17:47:50.631 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 82.227241864000007, 2024-09-04T17:47:50.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.93265329299999999, 2024-09-04T17:47:50.631 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:50.631 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:50.631 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-04T17:47:50.631 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:50.631 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:50.631 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:50.631 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 27242 2024-09-04T17:47:50.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.631 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:50.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:50.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402347+0000", 2024-09-04T17:47:50.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:50.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:50.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402347+0000", 2024-09-04T17:47:50.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:50.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:50.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402349+0000", 2024-09-04T17:47:50.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.471e-06 2024-09-04T17:47:50.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:50.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402355+0000", 2024-09-04T17:47:50.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8399999999999997e-06 2024-09-04T17:47:50.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:50.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402364+0000", 2024-09-04T17:47:50.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1269999999999992e-06 2024-09-04T17:47:50.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:50.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402373+0000", 2024-09-04T17:47:50.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7909999999999993e-06 2024-09-04T17:47:50.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:50.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402460+0000", 2024-09-04T17:47:50.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6686000000000003e-05 2024-09-04T17:47:50.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-04T17:47:50.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402473+0000", 2024-09-04T17:47:50.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3281e-05 2024-09-04T17:47:50.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:50.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:29.334989+0000", 2024-09-04T17:47:50.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.93251549099999997 2024-09-04T17:47:50.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:50.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:29.335000+0000", 2024-09-04T17:47:50.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1606e-05 2024-09-04T17:47:50.635 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:50.635 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:50.635 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:50.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.635 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:33552 3.7 3:e79ad1db:::rbd_data.11342e8f3384.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-04T17:47:50.635 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:35.999748+0000", 2024-09-04T17:47:50.636 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 74.629840814000005, 2024-09-04T17:47:50.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045683823999999998, 2024-09-04T17:47:50.636 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:50.636 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:50.636 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:50.636 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:50.636 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:50.636 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:50.636 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 33552 2024-09-04T17:47:50.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.636 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:50.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:50.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999748+0000", 2024-09-04T17:47:50.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:50.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:50.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999748+0000", 2024-09-04T17:47:50.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:50.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:50.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999750+0000", 2024-09-04T17:47:50.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1299999999999999e-06 2024-09-04T17:47:50.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:50.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999759+0000", 2024-09-04T17:47:50.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4649999999999994e-06 2024-09-04T17:47:50.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:50.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999778+0000", 2024-09-04T17:47:50.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8833000000000001e-05 2024-09-04T17:47:50.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:50.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999786+0000", 2024-09-04T17:47:50.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7369999999999995e-06 2024-09-04T17:47:50.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:50.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999856+0000", 2024-09-04T17:47:50.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9247999999999998e-05 2024-09-04T17:47:50.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:50.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999973+0000", 2024-09-04T17:47:50.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000117921 2024-09-04T17:47:50.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 7", 2024-09-04T17:47:50.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.005955+0000", 2024-09-04T17:47:50.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059820560000000004 2024-09-04T17:47:50.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:50.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.045297+0000", 2024-09-04T17:47:50.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.039341650999999998 2024-09-04T17:47:50.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:50.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.045343+0000", 2024-09-04T17:47:50.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5899000000000002e-05 2024-09-04T17:47:50.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:50.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.045388+0000", 2024-09-04T17:47:50.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4601999999999997e-05 2024-09-04T17:47:50.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:50.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.045432+0000", 2024-09-04T17:47:50.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4282000000000001e-05 2024-09-04T17:47:50.641 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:50.641 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:50.641 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:50.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.642 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:37357 3.15 3:a897312a:::rbd_data.1134c5a43465.000000000000001f:head [stat,write 62464~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-04T17:47:50.642 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:39.616724+0000", 2024-09-04T17:47:50.642 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 71.012865406000003, 2024-09-04T17:47:50.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054965854000000001, 2024-09-04T17:47:50.642 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:50.642 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:50.642 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:50.642 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:50.642 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:50.642 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:50.642 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 37357 2024-09-04T17:47:50.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.643 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:50.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:50.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616724+0000", 2024-09-04T17:47:50.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:50.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:50.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616724+0000", 2024-09-04T17:47:50.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:50.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.644 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:50.644 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616725+0000", 2024-09-04T17:47:50.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.815e-06 2024-09-04T17:47:50.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.644 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:50.644 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616739+0000", 2024-09-04T17:47:50.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3723000000000001e-05 2024-09-04T17:47:50.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.644 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:50.644 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616755+0000", 2024-09-04T17:47:50.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5964000000000002e-05 2024-09-04T17:47:50.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:50.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616763+0000", 2024-09-04T17:47:50.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3869999999999995e-06 2024-09-04T17:47:50.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:50.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616825+0000", 2024-09-04T17:47:50.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1854999999999996e-05 2024-09-04T17:47:50.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:50.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616895+0000", 2024-09-04T17:47:50.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9597999999999993e-05 2024-09-04T17:47:50.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-04T17:47:50.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616991+0000", 2024-09-04T17:47:50.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5828000000000005e-05 2024-09-04T17:47:50.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:50.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.617659+0000", 2024-09-04T17:47:50.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00066821600000000001 2024-09-04T17:47:50.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.647 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:50.647 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.671482+0000", 2024-09-04T17:47:50.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053823533999999999 2024-09-04T17:47:50.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.647 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:50.648 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.671579+0000", 2024-09-04T17:47:50.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6657999999999995e-05 2024-09-04T17:47:50.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.648 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:50.648 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.671689+0000", 2024-09-04T17:47:50.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000110276 2024-09-04T17:47:50.648 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:50.648 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:50.648 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:50.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.648 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:43873 3.15 3:abd207ee:::rbd_data.113410ddd206.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:50.649 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:49.932940+0000", 2024-09-04T17:47:50.649 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 60.696648852999999, 2024-09-04T17:47:50.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042740514, 2024-09-04T17:47:50.649 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:50.649 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:50.649 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:50.649 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:50.649 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:50.649 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:50.649 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 43873 2024-09-04T17:47:50.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.649 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:50.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.649 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:50.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.932940+0000", 2024-09-04T17:47:50.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:50.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:50.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.932940+0000", 2024-09-04T17:47:50.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:50.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:50.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.932941+0000", 2024-09-04T17:47:50.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.404e-06 2024-09-04T17:47:50.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:50.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.933065+0000", 2024-09-04T17:47:50.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012361199999999999 2024-09-04T17:47:50.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:50.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.933078+0000", 2024-09-04T17:47:50.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3182e-05 2024-09-04T17:47:50.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:50.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.933085+0000", 2024-09-04T17:47:50.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4500000000000001e-06 2024-09-04T17:47:50.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.652 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:50.652 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.933134+0000", 2024-09-04T17:47:50.652 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9150999999999997e-05 2024-09-04T17:47:50.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.652 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:50.652 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.933242+0000", 2024-09-04T17:47:50.652 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010781 2024-09-04T17:47:50.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.652 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-04T17:47:50.652 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.938358+0000", 2024-09-04T17:47:50.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051160349999999997 2024-09-04T17:47:50.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:50.653 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.975343+0000", 2024-09-04T17:47:50.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036985663000000002 2024-09-04T17:47:50.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:50.653 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.975392+0000", 2024-09-04T17:47:50.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9044e-05 2024-09-04T17:47:50.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:50.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.975548+0000", 2024-09-04T17:47:50.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015572900000000001 2024-09-04T17:47:50.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:50.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.975681+0000", 2024-09-04T17:47:50.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013243400000000001 2024-09-04T17:47:50.654 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:50.654 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:50.654 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:50.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.654 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:46649 3.7 3:e0749ed5:::rbd_data.1134326b938e.000000000000014f:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-04T17:47:50.655 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:54.114730+0000", 2024-09-04T17:47:50.655 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 56.514858721000003, 2024-09-04T17:47:50.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032972972000000003, 2024-09-04T17:47:50.655 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:50.655 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:50.655 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:50.655 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:50.655 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:50.655 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:50.655 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 46649 2024-09-04T17:47:50.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.655 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:50.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:50.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114730+0000", 2024-09-04T17:47:50.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:50.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:50.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114730+0000", 2024-09-04T17:47:50.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:50.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:50.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114731+0000", 2024-09-04T17:47:50.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.147e-06 2024-09-04T17:47:50.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.657 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:50.657 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114737+0000", 2024-09-04T17:47:50.657 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6380000000000001e-06 2024-09-04T17:47:50.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.657 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:50.657 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114751+0000", 2024-09-04T17:47:50.657 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4012999999999999e-05 2024-09-04T17:47:50.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.657 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:50.657 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114758+0000", 2024-09-04T17:47:50.657 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1539999999999996e-06 2024-09-04T17:47:50.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.658 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.658 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:50.658 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114876+0000", 2024-09-04T17:47:50.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000117421 2024-09-04T17:47:50.658 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.658 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.658 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:50.658 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.115173+0000", 2024-09-04T17:47:50.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00029750799999999998 2024-09-04T17:47:50.658 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.658 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.658 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 7", 2024-09-04T17:47:50.658 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.120273+0000", 2024-09-04T17:47:50.659 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051002900000000004 2024-09-04T17:47:50.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.659 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.659 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:50.659 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.147571+0000", 2024-09-04T17:47:50.659 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027297906 2024-09-04T17:47:50.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.659 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.659 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:50.659 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.147588+0000", 2024-09-04T17:47:50.659 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6807999999999999e-05 2024-09-04T17:47:50.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.659 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.659 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:50.659 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.147637+0000", 2024-09-04T17:47:50.660 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8466e-05 2024-09-04T17:47:50.660 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.660 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:50.660 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.147703+0000", 2024-09-04T17:47:50.660 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6620999999999998e-05 2024-09-04T17:47:50.660 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:50.660 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:50.660 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:50.660 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.660 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49801 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 720896~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:50.660 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.075094+0000", 2024-09-04T17:47:50.660 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.554494734000002, 2024-09-04T17:47:50.661 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017824974, 2024-09-04T17:47:50.661 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:50.661 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:50.661 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:50.661 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:50.661 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:50.661 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:50.661 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49801 2024-09-04T17:47:50.661 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.661 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:50.661 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.661 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:50.661 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075094+0000", 2024-09-04T17:47:50.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:50.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.662 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:50.662 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075094+0000", 2024-09-04T17:47:50.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:50.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.662 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:50.662 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075095+0000", 2024-09-04T17:47:50.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.017e-06 2024-09-04T17:47:50.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.662 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:50.662 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075101+0000", 2024-09-04T17:47:50.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5029999999999997e-06 2024-09-04T17:47:50.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:50.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075119+0000", 2024-09-04T17:47:50.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8133000000000001e-05 2024-09-04T17:47:50.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:50.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075132+0000", 2024-09-04T17:47:50.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3305999999999999e-05 2024-09-04T17:47:50.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:50.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.092677+0000", 2024-09-04T17:47:50.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017544466000000002 2024-09-04T17:47:50.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:50.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.092782+0000", 2024-09-04T17:47:50.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000105284 2024-09-04T17:47:50.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:50.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.092919+0000", 2024-09-04T17:47:50.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013726499999999999 2024-09-04T17:47:50.664 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:50.664 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:50.664 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:50.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.665 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49809 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 163840~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:50.665 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077146+0000", 2024-09-04T17:47:50.665 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.552443291000003, 2024-09-04T17:47:50.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017984541999999999, 2024-09-04T17:47:50.665 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:50.665 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:50.665 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:50.665 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:50.665 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:50.665 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:50.665 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49809 2024-09-04T17:47:50.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.665 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:50.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.666 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:50.666 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077146+0000", 2024-09-04T17:47:50.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:50.666 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.666 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:50.666 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077146+0000", 2024-09-04T17:47:50.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:50.666 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.666 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:50.666 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077147+0000", 2024-09-04T17:47:50.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0330000000000001e-06 2024-09-04T17:47:50.666 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.667 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:50.667 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077164+0000", 2024-09-04T17:47:50.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7810000000000001e-05 2024-09-04T17:47:50.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.667 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:50.667 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077187+0000", 2024-09-04T17:47:50.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2246e-05 2024-09-04T17:47:50.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.667 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:50.667 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077203+0000", 2024-09-04T17:47:50.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6392000000000002e-05 2024-09-04T17:47:50.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.668 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:50.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.094885+0000", 2024-09-04T17:47:50.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017681589000000001 2024-09-04T17:47:50.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.668 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:50.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.094991+0000", 2024-09-04T17:47:50.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010668899999999999 2024-09-04T17:47:50.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.668 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:50.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095130+0000", 2024-09-04T17:47:50.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013878299999999999 2024-09-04T17:47:50.669 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:50.669 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:50.669 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:50.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.669 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49810 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 229376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:50.669 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077225+0000", 2024-09-04T17:47:50.669 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.552363450000001, 2024-09-04T17:47:50.669 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018218793, 2024-09-04T17:47:50.669 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:50.669 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:50.669 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:50.669 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:50.670 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:50.670 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:50.670 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49810 2024-09-04T17:47:50.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.670 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:50.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.670 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:50.670 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077225+0000", 2024-09-04T17:47:50.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:50.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.670 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:50.670 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077225+0000", 2024-09-04T17:47:50.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:50.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:50.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077227+0000", 2024-09-04T17:47:50.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1829999999999999e-06 2024-09-04T17:47:50.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:50.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077233+0000", 2024-09-04T17:47:50.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3899999999999998e-06 2024-09-04T17:47:50.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:50.672 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077253+0000", 2024-09-04T17:47:50.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0009000000000001e-05 2024-09-04T17:47:50.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.672 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:50.672 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077270+0000", 2024-09-04T17:47:50.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7336000000000001e-05 2024-09-04T17:47:50.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.672 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:50.672 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095177+0000", 2024-09-04T17:47:50.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017907101000000002 2024-09-04T17:47:50.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.673 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.673 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:50.673 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095299+0000", 2024-09-04T17:47:50.673 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000121749 2024-09-04T17:47:50.673 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.673 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.673 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:50.673 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095444+0000", 2024-09-04T17:47:50.673 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014502499999999999 2024-09-04T17:47:50.673 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:50.673 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:50.673 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:50.673 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.674 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.674 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49811 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 294912~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:50.674 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077288+0000", 2024-09-04T17:47:50.674 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.552301135999997, 2024-09-04T17:47:50.674 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018443620000000001, 2024-09-04T17:47:50.674 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:50.674 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:50.674 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:50.674 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:50.674 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:50.674 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:50.674 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49811 2024-09-04T17:47:50.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.674 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:50.675 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.675 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:50.675 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077288+0000", 2024-09-04T17:47:50.675 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:50.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.675 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.675 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:50.675 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077288+0000", 2024-09-04T17:47:50.675 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:50.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.675 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.675 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:50.675 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077289+0000", 2024-09-04T17:47:50.675 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6199999999999999e-06 2024-09-04T17:47:50.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.676 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:50.676 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077295+0000", 2024-09-04T17:47:50.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1499999999999998e-06 2024-09-04T17:47:50.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.676 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:50.676 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077312+0000", 2024-09-04T17:47:50.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7353000000000001e-05 2024-09-04T17:47:50.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.676 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:50.676 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077324+0000", 2024-09-04T17:47:50.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2593e-05 2024-09-04T17:47:50.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:50.677 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095490+0000", 2024-09-04T17:47:50.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018165976 2024-09-04T17:47:50.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:50.677 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095597+0000", 2024-09-04T17:47:50.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010629900000000001 2024-09-04T17:47:50.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.678 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:50.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095731+0000", 2024-09-04T17:47:50.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013462899999999999 2024-09-04T17:47:50.678 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:50.678 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:50.678 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:50.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.678 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49813 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 360448~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:50.678 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077349+0000", 2024-09-04T17:47:50.678 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.552239933999999, 2024-09-04T17:47:50.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018780305000000001, 2024-09-04T17:47:50.679 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:50.679 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:50.679 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:50.679 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:50.679 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:50.679 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:50.679 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49813 2024-09-04T17:47:50.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.679 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:50.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.679 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:50.679 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077349+0000", 2024-09-04T17:47:50.679 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:50.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.680 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:50.680 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077349+0000", 2024-09-04T17:47:50.680 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:50.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.680 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:50.680 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077350+0000", 2024-09-04T17:47:50.680 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0330000000000001e-06 2024-09-04T17:47:50.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.680 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:50.680 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077355+0000", 2024-09-04T17:47:50.680 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0699999999999997e-06 2024-09-04T17:47:50.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:50.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077376+0000", 2024-09-04T17:47:50.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0846e-05 2024-09-04T17:47:50.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:50.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077388+0000", 2024-09-04T17:47:50.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2353e-05 2024-09-04T17:47:50.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:50.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095819+0000", 2024-09-04T17:47:50.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01843057 2024-09-04T17:47:50.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:50.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095960+0000", 2024-09-04T17:47:50.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000140965 2024-09-04T17:47:50.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:50.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096129+0000", 2024-09-04T17:47:50.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016946800000000001 2024-09-04T17:47:50.682 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:50.682 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:50.683 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:50.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.683 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49814 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 425984~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:50.683 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077407+0000", 2024-09-04T17:47:50.683 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.552181433000001, 2024-09-04T17:47:50.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019041563000000001, 2024-09-04T17:47:50.683 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:50.683 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:50.683 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:50.683 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:50.683 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:50.683 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:50.683 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49814 2024-09-04T17:47:50.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.684 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:50.684 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.684 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:50.684 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077407+0000", 2024-09-04T17:47:50.684 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:50.684 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.684 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.684 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:50.684 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077407+0000", 2024-09-04T17:47:50.684 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:50.684 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.684 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.684 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:50.684 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077409+0000", 2024-09-04T17:47:50.685 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2330000000000001e-06 2024-09-04T17:47:50.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.685 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:50.685 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077415+0000", 2024-09-04T17:47:50.685 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2729999999999999e-06 2024-09-04T17:47:50.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.685 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:50.685 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077436+0000", 2024-09-04T17:47:50.685 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1209999999999999e-05 2024-09-04T17:47:50.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:50.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077448+0000", 2024-09-04T17:47:50.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2225999999999999e-05 2024-09-04T17:47:50.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:50.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096183+0000", 2024-09-04T17:47:50.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018735004 2024-09-04T17:47:50.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:50.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096297+0000", 2024-09-04T17:47:50.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000113962 2024-09-04T17:47:50.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:50.687 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096449+0000", 2024-09-04T17:47:50.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000151655 2024-09-04T17:47:50.687 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:50.687 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:50.687 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:50.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.687 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49815 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 491520~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:50.687 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077471+0000", 2024-09-04T17:47:50.687 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.552118194999998, 2024-09-04T17:47:50.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019266536000000001, 2024-09-04T17:47:50.687 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:50.687 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:50.688 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:50.688 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:50.688 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:50.688 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:50.688 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49815 2024-09-04T17:47:50.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.688 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:50.688 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.688 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:50.688 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077471+0000", 2024-09-04T17:47:50.688 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:50.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.688 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.689 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:50.689 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077471+0000", 2024-09-04T17:47:50.689 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:50.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.689 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:50.689 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077472+0000", 2024-09-04T17:47:50.689 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.17e-06 2024-09-04T17:47:50.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.689 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:50.689 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077478+0000", 2024-09-04T17:47:50.689 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0399999999999998e-06 2024-09-04T17:47:50.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.690 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:50.690 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077500+0000", 2024-09-04T17:47:50.690 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1959000000000001e-05 2024-09-04T17:47:50.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.690 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:50.690 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077513+0000", 2024-09-04T17:47:50.690 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2823000000000001e-05 2024-09-04T17:47:50.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.690 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:50.690 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096493+0000", 2024-09-04T17:47:50.690 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018980701999999999 2024-09-04T17:47:50.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.691 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:50.691 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096603+0000", 2024-09-04T17:47:50.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010942000000000001 2024-09-04T17:47:50.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.691 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:50.691 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096737+0000", 2024-09-04T17:47:50.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013442200000000001 2024-09-04T17:47:50.691 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:50.691 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:50.691 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:50.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.692 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.692 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49816 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 557056~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:50.692 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077532+0000", 2024-09-04T17:47:50.692 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.552056454000002, 2024-09-04T17:47:50.692 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019604331999999999, 2024-09-04T17:47:50.692 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:50.692 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:50.692 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:50.692 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:50.692 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:50.692 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:50.692 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49816 2024-09-04T17:47:50.692 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.693 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:50.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.693 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:50.693 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077532+0000", 2024-09-04T17:47:50.693 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:50.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.693 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:50.693 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077532+0000", 2024-09-04T17:47:50.693 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:50.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.693 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:50.693 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077534+0000", 2024-09-04T17:47:50.694 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1200000000000001e-06 2024-09-04T17:47:50.694 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.694 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.694 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:50.694 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077545+0000", 2024-09-04T17:47:50.694 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1673e-05 2024-09-04T17:47:50.694 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.694 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.694 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:50.694 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077569+0000", 2024-09-04T17:47:50.694 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3555999999999999e-05 2024-09-04T17:47:50.694 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.694 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.694 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:50.694 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077584+0000", 2024-09-04T17:47:50.695 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4883e-05 2024-09-04T17:47:50.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.695 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.695 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:50.695 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096825+0000", 2024-09-04T17:47:50.695 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019241734 2024-09-04T17:47:50.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.695 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.695 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:50.695 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096964+0000", 2024-09-04T17:47:50.695 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000138085 2024-09-04T17:47:50.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.695 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.695 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:50.695 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097137+0000", 2024-09-04T17:47:50.696 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017328100000000001 2024-09-04T17:47:50.696 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:50.696 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:50.696 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:50.696 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.696 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.696 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49817 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 622592~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:50.696 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077603+0000", 2024-09-04T17:47:50.696 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.551985895999998, 2024-09-04T17:47:50.696 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01986928, 2024-09-04T17:47:50.696 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:50.696 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:50.696 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:50.696 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:50.696 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:50.697 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:50.697 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49817 2024-09-04T17:47:50.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.697 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:50.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.697 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:50.697 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077603+0000", 2024-09-04T17:47:50.697 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:50.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.697 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:50.697 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077603+0000", 2024-09-04T17:47:50.697 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:50.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.698 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.698 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:50.698 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077604+0000", 2024-09-04T17:47:50.698 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7699999999999992e-07 2024-09-04T17:47:50.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.698 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.698 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:50.698 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077609+0000", 2024-09-04T17:47:50.698 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0830000000000003e-06 2024-09-04T17:47:50.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.698 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.698 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:50.698 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077630+0000", 2024-09-04T17:47:50.698 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0438999999999999e-05 2024-09-04T17:47:50.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.699 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.699 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:50.699 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077664+0000", 2024-09-04T17:47:50.699 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4579000000000002e-05 2024-09-04T17:47:50.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.699 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.699 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:50.699 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097193+0000", 2024-09-04T17:47:50.699 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019529174 2024-09-04T17:47:50.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.699 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.699 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:50.699 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097323+0000", 2024-09-04T17:47:50.700 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000129696 2024-09-04T17:47:50.700 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.700 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.700 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:50.700 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097472+0000", 2024-09-04T17:47:50.700 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014933200000000001 2024-09-04T17:47:50.700 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:50.700 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:50.700 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:50.700 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.700 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.700 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49818 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 688128~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:50.701 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077683+0000", 2024-09-04T17:47:50.701 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.551905531999999, 2024-09-04T17:47:50.701 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020086333000000001, 2024-09-04T17:47:50.701 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:50.701 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:50.701 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:50.701 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:50.701 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:50.701 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:50.701 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49818 2024-09-04T17:47:50.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.701 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:50.701 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:50.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077683+0000", 2024-09-04T17:47:50.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:50.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:50.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077683+0000", 2024-09-04T17:47:50.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:50.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:50.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077684+0000", 2024-09-04T17:47:50.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4e-07 2024-09-04T17:47:50.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.703 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:50.703 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077690+0000", 2024-09-04T17:47:50.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4729999999999999e-06 2024-09-04T17:47:50.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.703 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:50.703 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077710+0000", 2024-09-04T17:47:50.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0149999999999999e-05 2024-09-04T17:47:50.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.703 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:50.703 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077723+0000", 2024-09-04T17:47:50.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3271999999999999e-05 2024-09-04T17:47:50.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.704 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:50.704 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097521+0000", 2024-09-04T17:47:50.704 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019797981999999999 2024-09-04T17:47:50.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.704 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:50.704 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097628+0000", 2024-09-04T17:47:50.704 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010665700000000001 2024-09-04T17:47:50.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.704 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:50.704 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097770+0000", 2024-09-04T17:47:50.705 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014185900000000001 2024-09-04T17:47:50.705 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:50.705 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:50.705 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:50.705 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.705 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.705 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49820 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 753664~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:50.705 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077744+0000", 2024-09-04T17:47:50.705 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.551844600999999, 2024-09-04T17:47:50.705 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020364786999999999, 2024-09-04T17:47:50.705 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:50.705 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:50.705 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:50.706 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:50.706 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:50.706 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:50.706 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49820 2024-09-04T17:47:50.706 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.706 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:50.706 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.706 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:50.706 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077744+0000", 2024-09-04T17:47:50.706 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:50.706 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.706 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.706 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:50.706 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077744+0000", 2024-09-04T17:47:50.706 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:50.707 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.707 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:50.707 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077745+0000", 2024-09-04T17:47:50.707 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0270000000000001e-06 2024-09-04T17:47:50.707 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.707 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:50.707 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077751+0000", 2024-09-04T17:47:50.707 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.626e-06 2024-09-04T17:47:50.707 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.707 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:50.708 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077771+0000", 2024-09-04T17:47:50.708 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0469999999999999e-05 2024-09-04T17:47:50.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.708 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.708 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:50.708 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077785+0000", 2024-09-04T17:47:50.708 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3129000000000001e-05 2024-09-04T17:47:50.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.708 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.708 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:50.708 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097816+0000", 2024-09-04T17:47:50.708 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020031917999999999 2024-09-04T17:47:50.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.708 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.708 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:50.709 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097939+0000", 2024-09-04T17:47:50.709 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000122533 2024-09-04T17:47:50.709 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.709 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.709 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:50.709 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098109+0000", 2024-09-04T17:47:50.709 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017008400000000001 2024-09-04T17:47:50.709 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:50.709 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:50.709 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:50.709 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.709 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.709 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49821 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 819200~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:50.710 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077807+0000", 2024-09-04T17:47:50.710 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.551782389000003, 2024-09-04T17:47:50.710 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020702928999999998, 2024-09-04T17:47:50.710 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:50.710 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:50.710 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:50.710 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:50.710 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:50.710 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:50.710 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49821 2024-09-04T17:47:50.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.710 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:50.710 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.710 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:50.711 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077807+0000", 2024-09-04T17:47:50.711 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:50.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.711 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:50.711 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077807+0000", 2024-09-04T17:47:50.711 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:50.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.711 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:50.711 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077808+0000", 2024-09-04T17:47:50.711 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.463e-06 2024-09-04T17:47:50.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.712 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:50.712 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077813+0000", 2024-09-04T17:47:50.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7530000000000001e-06 2024-09-04T17:47:50.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.712 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:50.712 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077832+0000", 2024-09-04T17:47:50.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.946e-05 2024-09-04T17:47:50.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.713 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:50.713 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077846+0000", 2024-09-04T17:47:50.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3786e-05 2024-09-04T17:47:50.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.713 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:50.713 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098201+0000", 2024-09-04T17:47:50.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020354797000000001 2024-09-04T17:47:50.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.713 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:50.713 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098341+0000", 2024-09-04T17:47:50.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000140042 2024-09-04T17:47:50.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.714 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.714 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:50.714 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098509+0000", 2024-09-04T17:47:50.714 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016862800000000001 2024-09-04T17:47:50.714 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:50.714 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:50.714 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:50.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.714 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.714 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49823 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 884736~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:50.714 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077865+0000", 2024-09-04T17:47:50.714 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.551723944999999, 2024-09-04T17:47:50.714 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021008723, 2024-09-04T17:47:50.714 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:50.715 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:50.715 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:50.715 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:50.715 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:50.715 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:50.715 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49823 2024-09-04T17:47:50.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.715 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:50.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.715 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:50.715 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077865+0000", 2024-09-04T17:47:50.715 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:50.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.716 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.716 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:50.716 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077865+0000", 2024-09-04T17:47:50.716 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:50.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.716 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.716 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:50.716 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077866+0000", 2024-09-04T17:47:50.716 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.164e-06 2024-09-04T17:47:50.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.716 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.716 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:50.716 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077872+0000", 2024-09-04T17:47:50.717 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3959999999999997e-06 2024-09-04T17:47:50.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.717 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:50.717 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077892+0000", 2024-09-04T17:47:50.717 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0970000000000001e-05 2024-09-04T17:47:50.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.717 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:50.717 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077905+0000", 2024-09-04T17:47:50.717 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2312e-05 2024-09-04T17:47:50.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.717 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:50.718 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098579+0000", 2024-09-04T17:47:50.718 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020674486999999998 2024-09-04T17:47:50.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.718 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:50.718 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098705+0000", 2024-09-04T17:47:50.718 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012541900000000001 2024-09-04T17:47:50.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.718 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:50.718 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098874+0000", 2024-09-04T17:47:50.718 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000168975 2024-09-04T17:47:50.718 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:50.718 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:50.719 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:50.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.719 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.719 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49824 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 950272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:50.719 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077923+0000", 2024-09-04T17:47:50.719 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.551666253, 2024-09-04T17:47:50.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021246287999999999, 2024-09-04T17:47:50.719 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:50.719 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:50.719 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:50.719 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:50.719 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:50.719 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:50.719 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49824 2024-09-04T17:47:50.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.720 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:50.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.720 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:50.720 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077923+0000", 2024-09-04T17:47:50.720 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:50.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.720 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:50.720 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077923+0000", 2024-09-04T17:47:50.720 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:50.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.721 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:50.721 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077924+0000", 2024-09-04T17:47:50.721 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0100000000000001e-06 2024-09-04T17:47:50.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.721 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.721 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:50.721 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077944+0000", 2024-09-04T17:47:50.721 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0449e-05 2024-09-04T17:47:50.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.721 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.721 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:50.721 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077965+0000", 2024-09-04T17:47:50.721 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0479000000000002e-05 2024-09-04T17:47:50.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.722 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:50.722 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077990+0000", 2024-09-04T17:47:50.722 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5582999999999999e-05 2024-09-04T17:47:50.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.722 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:50.722 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098924+0000", 2024-09-04T17:47:50.722 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020933395 2024-09-04T17:47:50.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.722 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:50.722 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.099032+0000", 2024-09-04T17:47:50.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010846299999999999 2024-09-04T17:47:50.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:50.723 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.099169+0000", 2024-09-04T17:47:50.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000136909 2024-09-04T17:47:50.723 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:50.723 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:50.723 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:50.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.723 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49826 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 1015808~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:50.723 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.078010+0000", 2024-09-04T17:47:50.723 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.551578646000003, 2024-09-04T17:47:50.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021584564000000001, 2024-09-04T17:47:50.724 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:50.724 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:50.724 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:50.724 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:50.724 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:50.724 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:50.724 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49826 2024-09-04T17:47:50.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.724 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:50.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.724 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:50.724 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078010+0000", 2024-09-04T17:47:50.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:50.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:50.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078010+0000", 2024-09-04T17:47:50.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:50.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:50.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078011+0000", 2024-09-04T17:47:50.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1340000000000001e-06 2024-09-04T17:47:50.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:50.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078017+0000", 2024-09-04T17:47:50.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.682e-06 2024-09-04T17:47:50.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.726 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:50.726 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078036+0000", 2024-09-04T17:47:50.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9357000000000001e-05 2024-09-04T17:47:50.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.726 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:50.726 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078050+0000", 2024-09-04T17:47:50.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3122000000000001e-05 2024-09-04T17:47:50.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.726 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:50.727 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.099271+0000", 2024-09-04T17:47:50.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021221772999999999 2024-09-04T17:47:50.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.727 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.727 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:50.727 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.099410+0000", 2024-09-04T17:47:50.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000138415 2024-09-04T17:47:50.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:50.727 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:50.727 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:50.727 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.099595+0000", 2024-09-04T17:47:50.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000185081 2024-09-04T17:47:50.727 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:50.727 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:50.728 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:50.728 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:50.728 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:50.728 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:50.729 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:47:50.729 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:47:50.730 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-04T17:47:50.730 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-04T17:47:50.730 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-04T17:47:50.731 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-04T17:47:50.731 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-04T17:47:50.731 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:47:50.731 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:47:50.732 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:47:50.732 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:47:50.732 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-04T17:47:50.732 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:50.728+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:50.735 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:47:50.736 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:47:50.739 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-04T17:47:50.739 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-04T17:47:50.830 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:50.827+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:50.853 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:50.853 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:50.867 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-04T17:47:50.883 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:47:50.879+0000 7f579621d7c0 -1 Falling back to public interface 2024-09-04T17:47:50.931 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:47:50.928+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:47:50.990 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:50.990 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:51.010 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-04T17:47:51.057 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:51.056+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:51.134 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:51.132+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:51.180 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:51.180 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:51.192 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone39 from 0xdf4d6f5 (overlap 0x76d7149) to 0x6af4c5b 2024-09-04T17:47:51.196 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-04T17:47:51.237 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:51.236+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:51.335 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:51.331+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:51.359 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:51.360 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:51.374 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-04T17:47:51.436 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:47:51.431+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:47:51.481 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #38, image image_client.0-clone38 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent39 2024-09-04T17:47:51.501 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:51.501 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:51.516 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-04T17:47:51.547 INFO:teuthology.orchestra.run.smithi092.stdout:5674 clone 41 order 20 su 65536 sc 9 2024-09-04T17:47:51.547 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:51.544+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:51.638 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:47:51.636+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:47:51.640 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:51.640 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:51.654 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-04T17:47:51.741 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:47:51.740+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:47:51.820 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:51.820 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:51.836 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-04T17:47:51.878 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:51.876+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:51.941 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:51.935+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:52.003 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:52.003 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:52.016 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-04T17:47:52.021 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-04T17:47:52.041 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:52.039+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:52.142 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:52.139+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:52.186 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone40 from 0xdf4d6f5 (overlap 0xdf4d6f5) to 0x59522c8 2024-09-04T17:47:52.197 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:52.197 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:52.215 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-04T17:47:52.242 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:47:52.240+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:47:52.343 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:52.340+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:52.386 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:52.386 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:52.406 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-04T17:47:52.457 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:52.455+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:52.545 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:52.544+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:52.583 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:52.583 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:52.605 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-04T17:47:52.646 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:52.644+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:52.741 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-04T17:47:52.785 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:52.784+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:52.792 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:52.792 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:52.806 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-04T17:47:52.849 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:52.847+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:52.879 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:47:52.880 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:47:52.896 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:47:52.896 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:47:52.916 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-04T17:47:52.916 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-04T17:47:52.934 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-04T17:47:52.934 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-04T17:47:52.936 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #39, image image_client.0-clone39 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent40 2024-09-04T17:47:52.939 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:52.939 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:52.948 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:52.944+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:52.954 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-04T17:47:52.954 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:47:52.955 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:47:52.975 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:47:52.975 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:47:52.975 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-04T17:47:52.994 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:47:52.995 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:47:52.998 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-04T17:47:52.998 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-04T17:47:53.007 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is unset 2024-09-04T17:47:53.049 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:47:53.047+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:47:53.119 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:53.119 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:47:53.119 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:47:53.119 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:47:53.120 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:53.133 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-04T17:47:53.134 INFO:teuthology.orchestra.run.smithi092.stdout:5677 punch from 0x2111ce1 to 0x2118cb1, (0x6fd0 bytes) 2024-09-04T17:47:53.142 INFO:teuthology.orchestra.run.smithi092.stdout:5678 read 0x63353a8 thru 0x63423b1 (0xd00a bytes) 2024-09-04T17:47:53.144 INFO:teuthology.orchestra.run.smithi092.stdout:5680 punch from 0x2913b98 to 0x2914484, (0x8ec bytes) 2024-09-04T17:47:53.144 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:53.144 INFO:teuthology.orchestra.run.smithi092.stdout:5682 write 0x1910122 thru 0x1916182 (0x6061 bytes) 2024-09-04T17:47:53.161 INFO:teuthology.orchestra.run.smithi092.stdout:5684 read 0x6101646 thru 0x6107573 (0x5f2e bytes) 2024-09-04T17:47:53.162 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:53.163 INFO:teuthology.orchestra.run.smithi092.stdout:5686 write 0xd104817 thru 0xd10f646 (0xae30 bytes) 2024-09-04T17:47:53.167 INFO:teuthology.orchestra.run.smithi092.stdout:5687 read 0x8328c8d thru 0x832f865 (0x6bd9 bytes) 2024-09-04T17:47:53.167 INFO:teuthology.orchestra.run.smithi092.stdout:5688 punch from 0xb11ac9a to 0xb127fd2, (0xd338 bytes) 2024-09-04T17:47:53.167 INFO:teuthology.orchestra.run.smithi092.stdout:5689 read 0xdd14f38 thru 0xdd24c34 (0xfcfd bytes) 2024-09-04T17:47:53.167 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:53.168 INFO:teuthology.orchestra.run.smithi092.stdout:5691 read 0x9ae6cb0 thru 0x9af19e6 (0xad37 bytes) 2024-09-04T17:47:53.168 INFO:teuthology.orchestra.run.smithi092.stdout:5694 read 0x1bfe8a thru 0x1cb914 (0xba8b bytes) 2024-09-04T17:47:53.177 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:53.175+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:53.185 INFO:teuthology.orchestra.run.smithi092.stdout:5695 trunc from 0xdf4d6f5 to 0x1818142 2024-09-04T17:47:53.251 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:47:53.247+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:47:53.266 INFO:teuthology.orchestra.run.smithi092.stdout:5697 read 0xcfd9f4 thru 0xd037a5 (0x5db2 bytes) 2024-09-04T17:47:53.280 INFO:teuthology.orchestra.run.smithi092.stdout:5698 write 0x59a5df6 thru 0x59b553a (0xf745 bytes) 2024-09-04T17:47:53.285 INFO:teuthology.orchestra.run.smithi092.stdout:5700 read 0x49a81ae thru 0x49b2ecb (0xad1e bytes) 2024-09-04T17:47:53.286 INFO:teuthology.orchestra.run.smithi092.stdout:5701 read 0x236e098 thru 0x2370912 (0x287b bytes) 2024-09-04T17:47:53.286 INFO:teuthology.orchestra.run.smithi092.stdout:5703 read 0x25ad8e8 thru 0x25b56f3 (0x7e0c bytes) 2024-09-04T17:47:53.286 INFO:teuthology.orchestra.run.smithi092.stdout:5704 read 0x4fea6e6 thru 0x4ff20b2 (0x79cd bytes) 2024-09-04T17:47:53.286 INFO:teuthology.orchestra.run.smithi092.stdout:5705 write 0x364e98 thru 0x36f2c5 (0xa42e bytes) 2024-09-04T17:47:53.301 INFO:teuthology.orchestra.run.smithi092.stdout:5706 write 0xedc748 thru 0xee14b2 (0x4d6b bytes) 2024-09-04T17:47:53.323 INFO:teuthology.orchestra.run.smithi092.stdout:5707 read 0x23bae5c thru 0x23bf88e (0x4a33 bytes) 2024-09-04T17:47:53.323 INFO:teuthology.orchestra.run.smithi092.stdout:5708 read 0x437923d thru 0x43860a9 (0xce6d bytes) 2024-09-04T17:47:53.323 INFO:teuthology.orchestra.run.smithi092.stdout:5710 writesame 0x414dfd2 thru 0x414fe62 data_size 0x5(0x1e91 bytes) 2024-09-04T17:47:53.328 INFO:teuthology.orchestra.run.smithi092.stdout:5711 read 0x3773c8a thru 0x377f48e (0xb805 bytes) 2024-09-04T17:47:53.329 INFO:teuthology.orchestra.run.smithi092.stdout:5713 punch from 0x544b97 to 0x5485f8, (0x3a61 bytes) 2024-09-04T17:47:53.329 INFO:teuthology.orchestra.run.smithi092.stdout:5714 read 0x1afa48b thru 0x1b0942b (0xefa1 bytes) 2024-09-04T17:47:53.329 INFO:teuthology.orchestra.run.smithi092.stdout:5715 read 0x51f420 thru 0x525bde (0x67bf bytes) 2024-09-04T17:47:53.330 INFO:teuthology.orchestra.run.smithi092.stdout:5716 read 0x1c5864c thru 0x1c5e0df (0x5a94 bytes) 2024-09-04T17:47:53.330 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:53.330 INFO:teuthology.orchestra.run.smithi092.stdout:5718 read 0x185223 thru 0x185cc7 (0xaa5 bytes) 2024-09-04T17:47:53.336 INFO:teuthology.orchestra.run.smithi092.stdout:5719 trunc from 0x59b553b to 0x3edbfe2 2024-09-04T17:47:53.336 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:53.337 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:47:53.337 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:47:53.337 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:47:53.337 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.337 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8831 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2453504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:53.337 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:47:53.337 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 100.278232731, 2024-09-04T17:47:53.337 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14731111799999999, 2024-09-04T17:47:53.337 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:53.337 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:53.337 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:53.337 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:53.337 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:53.338 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:53.338 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8831 2024-09-04T17:47:53.338 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.338 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:53.338 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.338 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:53.338 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:47:53.338 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.338 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.338 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.338 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:53.338 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:47:53.338 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.338 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.338 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.339 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:53.339 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057811+0000", 2024-09-04T17:47:53.339 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.083e-06 2024-09-04T17:47:53.339 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.339 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.339 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:53.339 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057814+0000", 2024-09-04T17:47:53.339 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.407e-06 2024-09-04T17:47:53.339 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.339 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.339 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:53.339 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057825+0000", 2024-09-04T17:47:53.339 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.17e-05 2024-09-04T17:47:53.339 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.339 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.340 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:53.340 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057836+0000", 2024-09-04T17:47:53.340 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0283e-05 2024-09-04T17:47:53.340 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.340 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.340 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:53.340 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.204762+0000", 2024-09-04T17:47:53.340 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.146926205 2024-09-04T17:47:53.340 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.340 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.340 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:53.340 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.204895+0000", 2024-09-04T17:47:53.340 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013328999999999999 2024-09-04T17:47:53.340 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.341 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.341 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:53.341 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205121+0000", 2024-09-04T17:47:53.341 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00022614999999999999 2024-09-04T17:47:53.341 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.341 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:53.341 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.341 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.341 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.341 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8832 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2490368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:53.341 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:47:53.341 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 100.27819469799999, 2024-09-04T17:47:53.341 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14770012299999999, 2024-09-04T17:47:53.342 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:53.342 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:53.342 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:53.342 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:53.342 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:53.342 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:53.342 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8832 2024-09-04T17:47:53.342 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.342 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:53.342 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.342 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:53.342 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:47:53.342 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.342 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.342 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.343 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:53.343 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:47:53.343 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.343 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.343 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.343 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:53.343 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057849+0000", 2024-09-04T17:47:53.343 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6700000000000003e-07 2024-09-04T17:47:53.343 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.343 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.343 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:53.343 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057852+0000", 2024-09-04T17:47:53.343 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6359999999999998e-06 2024-09-04T17:47:53.343 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.343 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.344 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:53.344 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057860+0000", 2024-09-04T17:47:53.344 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8140000000000007e-06 2024-09-04T17:47:53.344 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.344 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.344 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:53.344 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057868+0000", 2024-09-04T17:47:53.344 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1999999999999997e-06 2024-09-04T17:47:53.344 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.344 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.344 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:53.344 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205208+0000", 2024-09-04T17:47:53.344 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14734067300000001 2024-09-04T17:47:53.344 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.345 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.345 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:53.345 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205347+0000", 2024-09-04T17:47:53.345 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013914900000000001 2024-09-04T17:47:53.345 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.345 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.345 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:53.345 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205548+0000", 2024-09-04T17:47:53.345 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000200984 2024-09-04T17:47:53.345 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.345 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:53.345 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.345 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.345 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.346 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8833 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2527232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:53.346 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:47:53.346 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 100.27816282800001, 2024-09-04T17:47:53.346 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148062572, 2024-09-04T17:47:53.346 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:53.346 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:53.346 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:53.346 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:53.346 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:53.346 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:53.346 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8833 2024-09-04T17:47:53.346 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.346 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:53.346 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.347 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:53.347 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:47:53.347 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.347 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.347 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.347 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:53.347 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:47:53.347 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.347 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.347 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.347 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:53.347 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057881+0000", 2024-09-04T17:47:53.348 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4300000000000002e-07 2024-09-04T17:47:53.348 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.348 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.348 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:53.348 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057893+0000", 2024-09-04T17:47:53.348 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1973999999999999e-05 2024-09-04T17:47:53.348 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.348 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.348 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:53.348 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057906+0000", 2024-09-04T17:47:53.348 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2716e-05 2024-09-04T17:47:53.348 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.348 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.349 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:53.349 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057913+0000", 2024-09-04T17:47:53.349 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3930000000000002e-06 2024-09-04T17:47:53.349 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.349 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.349 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:53.349 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205621+0000", 2024-09-04T17:47:53.349 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14770788800000001 2024-09-04T17:47:53.349 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.349 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.349 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:53.349 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205740+0000", 2024-09-04T17:47:53.349 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000118936 2024-09-04T17:47:53.350 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.350 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.350 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:53.350 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205943+0000", 2024-09-04T17:47:53.350 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020292199999999999 2024-09-04T17:47:53.350 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.350 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:53.350 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.350 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.350 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.350 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8834 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2564096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:53.350 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:47:53.350 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 100.278119549, 2024-09-04T17:47:53.351 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14846753200000001, 2024-09-04T17:47:53.351 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:53.351 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:53.351 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:53.351 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:53.351 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:53.351 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:53.351 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8834 2024-09-04T17:47:53.351 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.351 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:53.351 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.351 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:53.352 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:47:53.352 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.352 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.352 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.352 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:53.352 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:47:53.352 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.352 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.352 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.352 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:53.352 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057925+0000", 2024-09-04T17:47:53.352 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0270000000000001e-06 2024-09-04T17:47:53.352 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.352 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.353 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:53.353 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057936+0000", 2024-09-04T17:47:53.353 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.112e-05 2024-09-04T17:47:53.353 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.353 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.353 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:53.353 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057946+0000", 2024-09-04T17:47:53.353 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.009e-05 2024-09-04T17:47:53.353 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.353 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.353 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:53.353 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057953+0000", 2024-09-04T17:47:53.353 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.6229999999999997e-06 2024-09-04T17:47:53.353 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.354 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.354 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:53.354 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206027+0000", 2024-09-04T17:47:53.354 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148073591 2024-09-04T17:47:53.354 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.354 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.354 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:53.354 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206161+0000", 2024-09-04T17:47:53.354 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013411599999999999 2024-09-04T17:47:53.354 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.354 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.354 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:53.354 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206391+0000", 2024-09-04T17:47:53.354 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000229965 2024-09-04T17:47:53.354 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.355 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:53.355 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.355 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.355 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.355 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8835 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2600960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:53.355 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:47:53.355 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 100.27807989900001, 2024-09-04T17:47:53.355 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14882941599999999, 2024-09-04T17:47:53.355 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:53.355 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:53.355 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:53.355 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:53.355 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:53.355 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:53.356 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8835 2024-09-04T17:47:53.356 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.356 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:53.356 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.356 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:53.356 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:47:53.356 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.356 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.356 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.356 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:53.356 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:47:53.356 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.356 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.356 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.357 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:53.357 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058544+0000", 2024-09-04T17:47:53.357 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00058061899999999999 2024-09-04T17:47:53.357 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.357 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.357 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:53.357 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058550+0000", 2024-09-04T17:47:53.357 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.1129999999999997e-06 2024-09-04T17:47:53.357 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.357 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.357 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:53.357 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058566+0000", 2024-09-04T17:47:53.358 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5820000000000001e-05 2024-09-04T17:47:53.358 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.358 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.358 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:53.358 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058578+0000", 2024-09-04T17:47:53.358 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2537e-05 2024-09-04T17:47:53.358 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.358 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.358 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:53.358 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206470+0000", 2024-09-04T17:47:53.358 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14789153799999999 2024-09-04T17:47:53.358 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.358 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.358 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:53.358 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206589+0000", 2024-09-04T17:47:53.359 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000118837 2024-09-04T17:47:53.359 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.359 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.359 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:53.359 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206793+0000", 2024-09-04T17:47:53.359 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000203952 2024-09-04T17:47:53.359 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.359 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:53.359 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.359 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.359 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.359 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8836 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2654208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:53.360 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:47:53.360 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 100.27744721099999, 2024-09-04T17:47:53.360 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14862433999999999, 2024-09-04T17:47:53.360 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:53.360 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:53.360 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:53.360 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:53.360 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:53.360 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:53.360 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8836 2024-09-04T17:47:53.360 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.360 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:53.360 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.361 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:53.361 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:47:53.361 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.361 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.361 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.361 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:53.361 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:47:53.361 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.361 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.361 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.361 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:53.362 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062663+0000", 2024-09-04T17:47:53.362 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0040670619999999998 2024-09-04T17:47:53.362 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.362 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.362 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:53.362 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062668+0000", 2024-09-04T17:47:53.362 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4199999999999998e-06 2024-09-04T17:47:53.362 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.362 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.362 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:53.362 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062689+0000", 2024-09-04T17:47:53.362 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0356999999999998e-05 2024-09-04T17:47:53.362 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.362 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.363 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:53.363 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062726+0000", 2024-09-04T17:47:53.363 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7302999999999998e-05 2024-09-04T17:47:53.363 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.363 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.363 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:53.363 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206875+0000", 2024-09-04T17:47:53.363 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14414883100000001 2024-09-04T17:47:53.363 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.363 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.363 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:53.363 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.207016+0000", 2024-09-04T17:47:53.363 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000140822 2024-09-04T17:47:53.363 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.364 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.364 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:53.364 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.207220+0000", 2024-09-04T17:47:53.364 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020454499999999999 2024-09-04T17:47:53.364 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.364 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:53.364 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.364 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.364 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.364 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8844 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2928640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:53.364 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:47:53.365 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 100.273083128, 2024-09-04T17:47:53.365 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14752153100000001, 2024-09-04T17:47:53.365 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:53.365 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:53.365 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:53.365 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:53.365 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:53.365 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:53.365 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8844 2024-09-04T17:47:53.365 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.365 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:53.365 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.365 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:53.366 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:47:53.366 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.366 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.366 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.366 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:53.366 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:47:53.366 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.366 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.366 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.366 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:53.366 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062961+0000", 2024-09-04T17:47:53.366 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6700000000000003e-07 2024-09-04T17:47:53.366 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.367 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.367 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:53.367 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062973+0000", 2024-09-04T17:47:53.367 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2109999999999999e-05 2024-09-04T17:47:53.367 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.367 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.367 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:53.367 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062982+0000", 2024-09-04T17:47:53.367 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.6900000000000004e-06 2024-09-04T17:47:53.367 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.367 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.367 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:53.367 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062990+0000", 2024-09-04T17:47:53.367 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1899999999999998e-06 2024-09-04T17:47:53.368 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.368 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.368 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:53.368 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210090+0000", 2024-09-04T17:47:53.368 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14710026300000001 2024-09-04T17:47:53.368 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.368 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.368 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:53.368 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210225+0000", 2024-09-04T17:47:53.368 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013532400000000001 2024-09-04T17:47:53.368 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.368 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.368 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:53.368 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210481+0000", 2024-09-04T17:47:53.368 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00025628700000000001 2024-09-04T17:47:53.369 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.369 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:53.369 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.369 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.369 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.369 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8845 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2965504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:53.369 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:47:53.369 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 100.273043405, 2024-09-04T17:47:53.369 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147916819, 2024-09-04T17:47:53.369 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:53.369 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:53.369 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:53.369 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:53.370 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:53.370 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:53.370 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8845 2024-09-04T17:47:53.370 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.370 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:53.370 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.370 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:53.370 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:47:53.370 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.370 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.370 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.370 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:53.370 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:47:53.370 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.370 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.371 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.371 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:53.371 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:47:53.371 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.0399999999999995e-07 2024-09-04T17:47:53.371 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.371 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.371 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:53.371 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063003+0000", 2024-09-04T17:47:53.371 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6529999999999998e-06 2024-09-04T17:47:53.371 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.371 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.371 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:53.371 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063012+0000", 2024-09-04T17:47:53.371 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9500000000000007e-06 2024-09-04T17:47:53.371 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.371 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.372 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:53.372 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063027+0000", 2024-09-04T17:47:53.372 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5443000000000001e-05 2024-09-04T17:47:53.372 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.372 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.372 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:53.372 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210571+0000", 2024-09-04T17:47:53.372 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14754353100000001 2024-09-04T17:47:53.372 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.372 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.372 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:53.372 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210714+0000", 2024-09-04T17:47:53.372 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014295599999999999 2024-09-04T17:47:53.372 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.372 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.373 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:53.373 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210916+0000", 2024-09-04T17:47:53.373 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020258200000000001 2024-09-04T17:47:53.373 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.373 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:53.373 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.373 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.373 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.373 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8846 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3002368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:53.373 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:47:53.373 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 100.273005868, 2024-09-04T17:47:53.373 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14828011599999999, 2024-09-04T17:47:53.373 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:53.373 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:53.374 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:53.374 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:53.374 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:53.374 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:53.374 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8846 2024-09-04T17:47:53.374 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.374 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:53.374 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.375 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:53.375 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:47:53.375 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.376 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.376 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.376 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:53.376 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:47:53.376 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.376 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.376 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.376 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:53.376 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063038+0000", 2024-09-04T17:47:53.376 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.23e-07 2024-09-04T17:47:53.376 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.376 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.376 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:53.376 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063051+0000", 2024-09-04T17:47:53.377 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2757e-05 2024-09-04T17:47:53.377 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.377 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.377 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:53.377 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063070+0000", 2024-09-04T17:47:53.377 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9663000000000001e-05 2024-09-04T17:47:53.377 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.377 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.377 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:53.377 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063079+0000", 2024-09-04T17:47:53.377 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.0929999999999992e-06 2024-09-04T17:47:53.377 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.377 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.377 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:53.378 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210989+0000", 2024-09-04T17:47:53.378 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14790984500000001 2024-09-04T17:47:53.378 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.378 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.378 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:53.378 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211109+0000", 2024-09-04T17:47:53.378 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000120056 2024-09-04T17:47:53.378 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.378 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.378 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:53.378 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211317+0000", 2024-09-04T17:47:53.378 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020797900000000001 2024-09-04T17:47:53.378 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.378 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:53.379 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.379 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.379 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.379 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8847 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3039232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:53.379 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:47:53.379 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 100.27293143599999, 2024-09-04T17:47:53.379 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14859408900000001, 2024-09-04T17:47:53.379 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:53.379 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:53.379 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:53.379 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:53.379 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:53.379 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:53.379 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8847 2024-09-04T17:47:53.380 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.380 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:53.380 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.380 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:53.380 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:47:53.380 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.380 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.380 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.380 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:53.380 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:47:53.380 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.380 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.380 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.380 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:53.380 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063113+0000", 2024-09-04T17:47:53.381 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.204e-06 2024-09-04T17:47:53.381 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.381 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.381 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:53.381 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063116+0000", 2024-09-04T17:47:53.381 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.3330000000000001e-06 2024-09-04T17:47:53.381 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.381 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.381 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:53.381 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063126+0000", 2024-09-04T17:47:53.381 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0166999999999999e-05 2024-09-04T17:47:53.381 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.381 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.382 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:53.382 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063134+0000", 2024-09-04T17:47:53.382 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4329999999999998e-06 2024-09-04T17:47:53.382 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.382 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.382 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:53.382 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211392+0000", 2024-09-04T17:47:53.382 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148258109 2024-09-04T17:47:53.382 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.382 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.382 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:53.382 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211506+0000", 2024-09-04T17:47:53.382 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000113813 2024-09-04T17:47:53.382 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.383 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.383 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:53.383 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211706+0000", 2024-09-04T17:47:53.383 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020002999999999999 2024-09-04T17:47:53.383 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.383 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:53.383 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.383 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.383 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.383 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8848 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:53.383 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:47:53.383 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 100.272899099, 2024-09-04T17:47:53.383 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14907967599999999, 2024-09-04T17:47:53.383 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:53.383 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:53.384 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:53.384 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:53.384 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:53.384 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:53.384 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8848 2024-09-04T17:47:53.384 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.384 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:53.384 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.384 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:53.384 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:47:53.384 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.384 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.385 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.385 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:53.385 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:47:53.385 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.385 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.385 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.385 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:53.385 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070589+0000", 2024-09-04T17:47:53.385 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.007445386 2024-09-04T17:47:53.385 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.385 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.385 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:53.385 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070600+0000", 2024-09-04T17:47:53.385 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0346999999999999e-05 2024-09-04T17:47:53.385 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.386 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.386 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:53.386 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070622+0000", 2024-09-04T17:47:53.386 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2289999999999998e-05 2024-09-04T17:47:53.386 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.386 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.386 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:53.386 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070636+0000", 2024-09-04T17:47:53.386 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4093e-05 2024-09-04T17:47:53.386 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.386 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.386 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:53.386 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211815+0000", 2024-09-04T17:47:53.386 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14117874699999999 2024-09-04T17:47:53.387 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.387 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.387 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:53.387 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211983+0000", 2024-09-04T17:47:53.387 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00016852800000000001 2024-09-04T17:47:53.387 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.387 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.387 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:53.387 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.212224+0000", 2024-09-04T17:47:53.387 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024028499999999999 2024-09-04T17:47:53.387 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.387 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:53.387 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.388 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.388 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.388 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8862 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2019328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:53.388 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:47:53.388 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 100.264849631, 2024-09-04T17:47:53.388 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14755236399999999, 2024-09-04T17:47:53.388 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:53.388 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:53.388 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:53.388 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:53.388 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:53.388 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:53.388 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8862 2024-09-04T17:47:53.388 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.389 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:53.389 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.389 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:53.389 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:47:53.389 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.389 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.389 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.389 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:53.389 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:47:53.389 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.389 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.389 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.389 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:53.389 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071194+0000", 2024-09-04T17:47:53.390 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.5000000000000002e-07 2024-09-04T17:47:53.390 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.390 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.390 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:53.390 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071197+0000", 2024-09-04T17:47:53.390 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5900000000000002e-06 2024-09-04T17:47:53.390 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.390 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.390 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:53.390 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071206+0000", 2024-09-04T17:47:53.390 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9700000000000005e-06 2024-09-04T17:47:53.390 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.390 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.390 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:53.390 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071213+0000", 2024-09-04T17:47:53.391 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.9070000000000001e-06 2024-09-04T17:47:53.391 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.391 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.391 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:53.391 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218396+0000", 2024-09-04T17:47:53.391 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147183273 2024-09-04T17:47:53.391 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.391 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.391 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:53.391 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218538+0000", 2024-09-04T17:47:53.391 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014194199999999999 2024-09-04T17:47:53.391 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.391 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.391 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:53.392 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218746+0000", 2024-09-04T17:47:53.392 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000208032 2024-09-04T17:47:53.392 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.392 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:53.392 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.392 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.392 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.392 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8863 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2056192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:53.392 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:47:53.392 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 100.264821321, 2024-09-04T17:47:53.392 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14792281099999999, 2024-09-04T17:47:53.392 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:53.392 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:53.392 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:53.393 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:53.393 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:53.393 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:53.393 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8863 2024-09-04T17:47:53.393 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.393 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:53.393 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.393 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:53.393 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:47:53.393 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.393 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.393 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.393 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:53.394 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:47:53.394 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.394 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.394 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.394 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:53.394 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:47:53.394 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1299999999999999e-07 2024-09-04T17:47:53.394 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.394 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.394 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:53.394 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071225+0000", 2024-09-04T17:47:53.394 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6199999999999999e-06 2024-09-04T17:47:53.394 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.394 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.394 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:53.395 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071234+0000", 2024-09-04T17:47:53.395 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8769999999999999e-06 2024-09-04T17:47:53.395 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.395 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.395 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:53.395 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071251+0000", 2024-09-04T17:47:53.395 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.738e-05 2024-09-04T17:47:53.395 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.395 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.395 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:53.395 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218798+0000", 2024-09-04T17:47:53.395 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14754645 2024-09-04T17:47:53.395 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.395 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.396 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:53.396 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218926+0000", 2024-09-04T17:47:53.396 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000128446 2024-09-04T17:47:53.396 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.396 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.396 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:53.396 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219145+0000", 2024-09-04T17:47:53.396 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021832500000000001 2024-09-04T17:47:53.396 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.396 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:53.396 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.396 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.396 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.396 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8864 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2093056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:53.396 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:47:53.397 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 100.264779745, 2024-09-04T17:47:53.397 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148277309, 2024-09-04T17:47:53.397 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:53.397 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:53.397 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:53.397 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:53.397 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:53.397 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:53.397 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8864 2024-09-04T17:47:53.397 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.397 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:53.397 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.397 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:53.397 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:47:53.398 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.398 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.398 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.398 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:53.398 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:47:53.398 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.398 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.398 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.398 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:53.398 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071264+0000", 2024-09-04T17:47:53.398 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3e-07 2024-09-04T17:47:53.398 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.398 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.398 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:53.399 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071267+0000", 2024-09-04T17:47:53.399 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0139999999999999e-06 2024-09-04T17:47:53.399 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.399 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.399 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:53.399 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071276+0000", 2024-09-04T17:47:53.399 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1689999999999998e-06 2024-09-04T17:47:53.399 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.399 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.399 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:53.399 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071283+0000", 2024-09-04T17:47:53.399 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2269999999999996e-06 2024-09-04T17:47:53.399 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.400 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.400 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:53.400 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219195+0000", 2024-09-04T17:47:53.400 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147911868 2024-09-04T17:47:53.400 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.400 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.400 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:53.400 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219324+0000", 2024-09-04T17:47:53.400 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00012835900000000001 2024-09-04T17:47:53.400 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.400 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.400 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:53.400 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219541+0000", 2024-09-04T17:47:53.401 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000216942 2024-09-04T17:47:53.401 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.401 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:53.401 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.401 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.401 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.401 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8865 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2109440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:53.401 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:47:53.401 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 100.26475028500001, 2024-09-04T17:47:53.401 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14865747300000001, 2024-09-04T17:47:53.401 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:53.401 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:53.401 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:53.401 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:53.401 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:53.402 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:53.402 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8865 2024-09-04T17:47:53.402 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.402 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:53.402 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.402 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:53.402 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:47:53.402 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.402 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.402 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.402 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:53.402 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:47:53.402 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.403 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.403 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.403 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:53.403 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:47:53.403 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7999999999999995e-07 2024-09-04T17:47:53.403 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.403 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.403 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:53.403 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071296+0000", 2024-09-04T17:47:53.403 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6900000000000001e-06 2024-09-04T17:47:53.403 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.403 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.403 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:53.403 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071305+0000", 2024-09-04T17:47:53.404 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.5499999999999995e-06 2024-09-04T17:47:53.404 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.404 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.404 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:53.404 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071312+0000", 2024-09-04T17:47:53.404 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1559999999999998e-06 2024-09-04T17:47:53.404 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.404 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.404 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:53.404 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219591+0000", 2024-09-04T17:47:53.404 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14827944900000001 2024-09-04T17:47:53.404 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.404 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.404 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:53.404 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219731+0000", 2024-09-04T17:47:53.405 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013931600000000001 2024-09-04T17:47:53.405 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.405 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.405 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:53.405 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219950+0000", 2024-09-04T17:47:53.405 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021963199999999999 2024-09-04T17:47:53.405 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.405 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:53.405 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.405 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.405 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.405 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8866 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2146304~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:53.405 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:47:53.405 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 100.264722052, 2024-09-04T17:47:53.406 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14906117899999999, 2024-09-04T17:47:53.406 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:53.406 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:53.406 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:53.406 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:53.406 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:53.406 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:53.406 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8866 2024-09-04T17:47:53.406 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.406 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:53.406 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.406 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:53.406 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:47:53.406 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.406 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.406 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.407 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:53.407 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:47:53.407 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.407 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.407 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.407 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:53.407 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071322+0000", 2024-09-04T17:47:53.407 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7999999999999995e-07 2024-09-04T17:47:53.407 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.407 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.407 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:53.407 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071338+0000", 2024-09-04T17:47:53.407 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5872999999999999e-05 2024-09-04T17:47:53.407 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.407 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.408 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:53.408 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071347+0000", 2024-09-04T17:47:53.408 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4669999999999993e-06 2024-09-04T17:47:53.408 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.408 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.408 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:53.408 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071354+0000", 2024-09-04T17:47:53.408 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2099999999999996e-06 2024-09-04T17:47:53.408 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.408 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.408 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:53.408 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220008+0000", 2024-09-04T17:47:53.408 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148653642 2024-09-04T17:47:53.408 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.409 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.409 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:53.409 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220155+0000", 2024-09-04T17:47:53.409 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014726600000000001 2024-09-04T17:47:53.409 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.409 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.409 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:53.409 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220382+0000", 2024-09-04T17:47:53.409 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00022704100000000001 2024-09-04T17:47:53.409 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.409 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:53.409 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.409 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.410 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.410 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8867 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2183168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:53.410 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:47:53.410 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 100.264678699, 2024-09-04T17:47:53.410 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.149445831, 2024-09-04T17:47:53.410 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:53.410 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:53.410 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:53.410 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:53.410 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:53.410 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:53.410 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8867 2024-09-04T17:47:53.410 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.410 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:53.410 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.411 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:53.411 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:47:53.411 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.411 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.411 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.411 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:53.411 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:47:53.411 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.411 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.411 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.411 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:53.411 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078653+0000", 2024-09-04T17:47:53.411 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0072890430000000003 2024-09-04T17:47:53.411 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.411 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.412 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:53.412 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078664+0000", 2024-09-04T17:47:53.412 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0307e-05 2024-09-04T17:47:53.412 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.412 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.412 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:53.412 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078690+0000", 2024-09-04T17:47:53.412 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6780000000000001e-05 2024-09-04T17:47:53.412 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.412 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.412 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:53.412 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078706+0000", 2024-09-04T17:47:53.412 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5676999999999999e-05 2024-09-04T17:47:53.412 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.413 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.413 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:53.413 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220467+0000", 2024-09-04T17:47:53.413 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14176103500000001 2024-09-04T17:47:53.413 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.413 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.413 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:53.413 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220595+0000", 2024-09-04T17:47:53.413 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000127427 2024-09-04T17:47:53.413 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.413 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.413 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:53.413 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220810+0000", 2024-09-04T17:47:53.414 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021556200000000001 2024-09-04T17:47:53.414 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.414 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:53.414 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.414 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.414 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.414 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8880 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2658304~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:53.414 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.079158+0000", 2024-09-04T17:47:53.414 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 100.25688542, 2024-09-04T17:47:53.414 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14718968499999999, 2024-09-04T17:47:53.414 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:53.414 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:53.414 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:53.415 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:53.415 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:53.415 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:53.415 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8880 2024-09-04T17:47:53.415 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.415 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:53.415 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.415 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:53.415 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079158+0000", 2024-09-04T17:47:53.415 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.415 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.415 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.415 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:53.415 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079158+0000", 2024-09-04T17:47:53.415 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.416 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.416 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.416 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:53.416 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079158+0000", 2024-09-04T17:47:53.416 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3e-07 2024-09-04T17:47:53.416 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.416 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.416 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:53.416 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079161+0000", 2024-09-04T17:47:53.416 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.57e-06 2024-09-04T17:47:53.416 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.416 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.416 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:53.416 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079170+0000", 2024-09-04T17:47:53.417 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.0170000000000002e-06 2024-09-04T17:47:53.417 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.417 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.417 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:53.417 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079177+0000", 2024-09-04T17:47:53.417 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.0530000000000001e-06 2024-09-04T17:47:53.417 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.417 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.417 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:53.417 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.225946+0000", 2024-09-04T17:47:53.417 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14676936500000001 2024-09-04T17:47:53.417 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.417 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.418 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:53.418 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226098+0000", 2024-09-04T17:47:53.418 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000151805 2024-09-04T17:47:53.418 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.418 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.418 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:53.418 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226347+0000", 2024-09-04T17:47:53.418 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024914499999999999 2024-09-04T17:47:53.418 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.418 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:53.418 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.418 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.418 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.418 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8881 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2695168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:53.419 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:47:53.419 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 100.25685663, 2024-09-04T17:47:53.419 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147543547, 2024-09-04T17:47:53.419 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:53.419 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:53.419 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:53.419 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:53.419 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:53.419 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:53.419 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8881 2024-09-04T17:47:53.419 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.419 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:53.419 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.420 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:53.420 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:47:53.420 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.420 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.420 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.420 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:53.420 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:47:53.420 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.420 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.420 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.420 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:53.420 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079187+0000", 2024-09-04T17:47:53.420 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7700000000000004e-07 2024-09-04T17:47:53.420 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.421 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.421 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:53.421 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079198+0000", 2024-09-04T17:47:53.421 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0849999999999999e-05 2024-09-04T17:47:53.421 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.421 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.421 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:53.421 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079207+0000", 2024-09-04T17:47:53.421 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5599999999999999e-06 2024-09-04T17:47:53.421 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.421 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.421 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:53.421 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079224+0000", 2024-09-04T17:47:53.422 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6387000000000001e-05 2024-09-04T17:47:53.422 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.422 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.422 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:53.422 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226414+0000", 2024-09-04T17:47:53.422 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14718975000000001 2024-09-04T17:47:53.422 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.422 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.422 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:53.422 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226534+0000", 2024-09-04T17:47:53.422 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000120288 2024-09-04T17:47:53.422 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.422 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.423 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:53.423 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226730+0000", 2024-09-04T17:47:53.423 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000196035 2024-09-04T17:47:53.423 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.423 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:53.423 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.423 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.423 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.423 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8882 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2732032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:47:53.423 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:47:53.423 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 100.256808416, 2024-09-04T17:47:53.423 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14795022399999999, 2024-09-04T17:47:53.423 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:53.424 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:53.424 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:53.424 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:53.424 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:53.424 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:53.424 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8882 2024-09-04T17:47:53.424 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.424 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:53.424 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.424 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:53.424 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:47:53.424 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.424 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.425 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.425 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:53.425 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:47:53.425 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.425 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.425 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.425 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:53.425 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086588+0000", 2024-09-04T17:47:53.425 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0073530959999999999 2024-09-04T17:47:53.426 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.426 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.426 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:53.426 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086598+0000", 2024-09-04T17:47:53.426 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0122999999999999e-05 2024-09-04T17:47:53.426 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.426 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.426 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:53.426 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086626+0000", 2024-09-04T17:47:53.426 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.7914000000000001e-05 2024-09-04T17:47:53.426 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.426 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.426 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:53.427 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086641+0000", 2024-09-04T17:47:53.427 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5270000000000001e-05 2024-09-04T17:47:53.427 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.427 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.427 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:53.427 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226801+0000", 2024-09-04T17:47:53.427 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.140160321 2024-09-04T17:47:53.427 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.427 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.427 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:53.427 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226951+0000", 2024-09-04T17:47:53.427 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014965200000000001 2024-09-04T17:47:53.427 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.427 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.427 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:53.428 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.227185+0000", 2024-09-04T17:47:53.428 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00023384799999999999 2024-09-04T17:47:53.428 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.428 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:53.428 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.428 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.428 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:53.428 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:53.430 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-04T17:47:53.431 INFO:teuthology.orchestra.run.smithi092.stdout:5720 trunc from 0x3edbfe2 to 0x9d97aca 2024-09-04T17:47:53.431 INFO:teuthology.orchestra.run.smithi092.stdout:5721 punch from 0x9550434 to 0x9559b31, (0x96fd bytes) 2024-09-04T17:47:53.431 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:53.431 INFO:teuthology.orchestra.run.smithi092.stdout:5724 write 0x3fafdd6 thru 0x3fba646 (0xa871 bytes) 2024-09-04T17:47:53.431 INFO:teuthology.orchestra.run.smithi092.stdout:5725 writesame 0x29a7a4b thru 0x29a85a7 data_size 0x1(0xb5d bytes) 2024-09-04T17:47:53.431 INFO:teuthology.orchestra.run.smithi092.stdout:5726 punch from 0x389f195 to 0x38ac974, (0xd7df bytes) 2024-09-04T17:47:53.431 INFO:teuthology.orchestra.run.smithi092.stdout:5728 write 0xe579167 thru 0xe57bbd8 (0x2a72 bytes) 2024-09-04T17:47:53.431 INFO:teuthology.orchestra.run.smithi092.stdout:5730 read 0xbac5b7e thru 0xbac7dac (0x222f bytes) 2024-09-04T17:47:53.431 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:53.431 INFO:teuthology.orchestra.run.smithi092.stdout:5733 writesame 0xb961329 thru 0xb96d818 data_size 0x89(0xc4f0 bytes) 2024-09-04T17:47:53.431 INFO:teuthology.orchestra.run.smithi092.stdout:5734 punch from 0x102a21b to 0x10310ed, (0x6ed2 bytes) 2024-09-04T17:47:53.432 INFO:teuthology.orchestra.run.smithi092.stdout:5735 writesame 0x516e321 thru 0x517ca91 data_size 0x1(0xe771 bytes) 2024-09-04T17:47:53.432 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:53.428+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:53.468 INFO:teuthology.orchestra.run.smithi092.stdout:5736 write 0x9f3e1e3 thru 0x9f4d431 (0xf24f bytes) 2024-09-04T17:47:53.478 INFO:teuthology.orchestra.run.smithi092.stdout:5737 punch from 0xe21b1d5 to 0xe223c97, (0x8ac2 bytes) 2024-09-04T17:47:53.478 INFO:teuthology.orchestra.run.smithi092.stdout:5738 write 0x621552e thru 0x621b58b (0x605e bytes) 2024-09-04T17:47:53.484 INFO:teuthology.orchestra.run.smithi092.stdout:5739 write 0xa6dbe2d thru 0xa6e72f4 (0xb4c8 bytes) 2024-09-04T17:47:53.517 INFO:teuthology.orchestra.run.smithi092.stdout:5742 trunc from 0xe57bbd9 to 0x586fb3b 2024-09-04T17:47:53.530 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:53.527+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:53.538 INFO:teuthology.orchestra.run.smithi092.stdout:5743 write 0x1ce68fc thru 0x1cf2c66 (0xc36b bytes) 2024-09-04T17:47:53.545 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:53.546 INFO:teuthology.orchestra.run.smithi092.stdout:5747 read 0x3f6e827 thru 0x3f76a00 (0x81da bytes) 2024-09-04T17:47:53.565 INFO:teuthology.orchestra.run.smithi092.stdout:5748 write 0x9a836de thru 0x9a8f9f8 (0xc31b bytes) 2024-09-04T17:47:53.571 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:53.571 INFO:teuthology.orchestra.run.smithi092.stdout:5750 read 0x45ba2f6 thru 0x45bda4c (0x3757 bytes) 2024-09-04T17:47:53.571 INFO:teuthology.orchestra.run.smithi092.stdout:5751 write 0x47b137f thru 0x47bd5e6 (0xc268 bytes) 2024-09-04T17:47:53.576 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:53.576 INFO:teuthology.orchestra.run.smithi092.stdout:5753 clone 42 order 25 su 8388608 sc 8 2024-09-04T17:47:53.604 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:53.604 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:47:53.604 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:47:53.604 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:53.619 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-04T17:47:53.661 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:47:53.659+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:47:53.732 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:47:53.728+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:47:53.778 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:53.778 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:47:53.778 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:47:53.779 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:47:53.779 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:53.794 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-04T17:47:53.833 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:53.828+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:53.914 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:53.914 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:47:53.914 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:47:53.915 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:47:53.915 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.915 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60921 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 262144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:53.915 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822920+0000", 2024-09-04T17:47:53.915 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 40.089919725000001, 2024-09-04T17:47:53.915 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073904394999999998, 2024-09-04T17:47:53.915 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:53.915 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:53.915 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:53.915 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:53.915 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:53.915 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:53.915 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60921 2024-09-04T17:47:53.916 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.916 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:53.916 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.916 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:53.916 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822920+0000", 2024-09-04T17:47:53.916 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.916 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.916 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.916 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:53.916 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822920+0000", 2024-09-04T17:47:53.916 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.916 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.916 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.917 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:53.917 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822920+0000", 2024-09-04T17:47:53.917 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4999999999999999e-07 2024-09-04T17:47:53.917 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.917 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.917 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:53.917 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822921+0000", 2024-09-04T17:47:53.917 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.113e-06 2024-09-04T17:47:53.917 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.917 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.917 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:53.917 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822924+0000", 2024-09-04T17:47:53.917 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.354e-06 2024-09-04T17:47:53.918 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.918 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.918 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:53.918 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822927+0000", 2024-09-04T17:47:53.918 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6199999999999999e-06 2024-09-04T17:47:53.918 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.918 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.918 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:53.918 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896586+0000", 2024-09-04T17:47:53.918 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073658976000000001 2024-09-04T17:47:53.918 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.918 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.918 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:53.919 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896626+0000", 2024-09-04T17:47:53.919 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0484000000000001e-05 2024-09-04T17:47:53.919 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.919 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.919 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:53.919 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896824+0000", 2024-09-04T17:47:53.919 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000197598 2024-09-04T17:47:53.919 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.919 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:53.919 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.919 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.919 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.919 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60922 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:53.920 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822930+0000", 2024-09-04T17:47:53.920 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 40.089909149999997, 2024-09-04T17:47:53.920 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074178872000000007, 2024-09-04T17:47:53.920 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:53.920 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:53.920 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:53.920 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:53.920 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:53.920 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:53.920 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60922 2024-09-04T17:47:53.920 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.920 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:53.920 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.920 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:53.921 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822930+0000", 2024-09-04T17:47:53.921 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.921 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.921 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.921 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:53.921 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822930+0000", 2024-09-04T17:47:53.921 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.921 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.921 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.921 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:53.921 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822930+0000", 2024-09-04T17:47:53.921 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.36e-07 2024-09-04T17:47:53.921 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.922 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.922 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:53.922 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822932+0000", 2024-09-04T17:47:53.922 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.567e-06 2024-09-04T17:47:53.922 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.922 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.922 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:53.922 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822938+0000", 2024-09-04T17:47:53.922 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4840000000000003e-06 2024-09-04T17:47:53.922 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.922 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.922 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:53.922 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822942+0000", 2024-09-04T17:47:53.922 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2039999999999999e-06 2024-09-04T17:47:53.923 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.923 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.923 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:53.923 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896850+0000", 2024-09-04T17:47:53.923 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073907941000000005 2024-09-04T17:47:53.923 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.923 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.923 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:53.923 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896891+0000", 2024-09-04T17:47:53.923 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0964999999999998e-05 2024-09-04T17:47:53.923 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.923 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.923 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:53.923 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897109+0000", 2024-09-04T17:47:53.923 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021847500000000001 2024-09-04T17:47:53.924 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.924 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:53.924 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.924 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.924 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.924 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60923 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 475136~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:53.924 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822949+0000", 2024-09-04T17:47:53.924 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 40.089890859999997, 2024-09-04T17:47:53.924 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074426928000000003, 2024-09-04T17:47:53.924 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:53.924 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:53.924 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:53.924 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:53.924 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:53.925 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:53.925 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60923 2024-09-04T17:47:53.925 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.925 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:53.925 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.925 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:53.925 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822949+0000", 2024-09-04T17:47:53.925 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.925 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.925 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.925 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:53.925 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822949+0000", 2024-09-04T17:47:53.925 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.926 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.926 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.926 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:53.926 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822949+0000", 2024-09-04T17:47:53.926 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7399999999999999e-07 2024-09-04T17:47:53.926 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.926 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.926 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:53.926 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822951+0000", 2024-09-04T17:47:53.926 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1940000000000001e-06 2024-09-04T17:47:53.926 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.926 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.926 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:53.926 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822957+0000", 2024-09-04T17:47:53.926 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.0650000000000004e-06 2024-09-04T17:47:53.927 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.927 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.927 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:53.927 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822960+0000", 2024-09-04T17:47:53.927 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.8009999999999999e-06 2024-09-04T17:47:53.927 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.927 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.927 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:53.927 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897138+0000", 2024-09-04T17:47:53.927 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074177990999999999 2024-09-04T17:47:53.927 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.927 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.928 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:53.928 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897179+0000", 2024-09-04T17:47:53.928 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.1057999999999999e-05 2024-09-04T17:47:53.928 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.928 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.928 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:53.928 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897375+0000", 2024-09-04T17:47:53.928 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019644500000000001 2024-09-04T17:47:53.928 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.928 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:53.928 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.928 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.929 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.929 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60924 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:53.929 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822964+0000", 2024-09-04T17:47:53.929 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 40.089875675000002, 2024-09-04T17:47:53.929 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074531902999999997, 2024-09-04T17:47:53.929 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:53.929 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:53.929 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:53.929 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:53.929 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:53.929 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:53.929 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60924 2024-09-04T17:47:53.930 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.930 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:53.930 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.930 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:53.930 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822964+0000", 2024-09-04T17:47:53.930 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.930 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.930 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.930 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:53.930 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822964+0000", 2024-09-04T17:47:53.930 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.930 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.930 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.930 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:53.930 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822964+0000", 2024-09-04T17:47:53.931 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4999999999999999e-07 2024-09-04T17:47:53.931 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.931 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.931 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:53.931 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822966+0000", 2024-09-04T17:47:53.931 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.917e-06 2024-09-04T17:47:53.931 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.931 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.931 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:53.931 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822969+0000", 2024-09-04T17:47:53.931 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.315e-06 2024-09-04T17:47:53.931 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.931 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.931 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:53.931 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822972+0000", 2024-09-04T17:47:53.932 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.599e-06 2024-09-04T17:47:53.932 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.932 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.932 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:53.932 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897398+0000", 2024-09-04T17:47:53.932 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074426168000000001 2024-09-04T17:47:53.932 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.932 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.932 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:53.932 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897441+0000", 2024-09-04T17:47:53.932 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.3158000000000003e-05 2024-09-04T17:47:53.932 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.932 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.932 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:53.933 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897496+0000", 2024-09-04T17:47:53.933 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4496000000000001e-05 2024-09-04T17:47:53.933 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.933 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:53.933 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.933 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.933 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.933 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60925 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 720896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:53.933 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822975+0000", 2024-09-04T17:47:53.933 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 40.089864239999997, 2024-09-04T17:47:53.933 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074635272000000003, 2024-09-04T17:47:53.933 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:53.933 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:53.933 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:53.933 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:53.934 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:53.934 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:53.934 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60925 2024-09-04T17:47:53.934 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.934 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:53.934 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.934 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:53.934 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822975+0000", 2024-09-04T17:47:53.934 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.934 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.934 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.934 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:53.934 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822975+0000", 2024-09-04T17:47:53.934 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.935 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.935 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.935 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:53.935 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822975+0000", 2024-09-04T17:47:53.935 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.34e-07 2024-09-04T17:47:53.935 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.935 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.935 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:53.935 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822976+0000", 2024-09-04T17:47:53.935 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0359999999999999e-06 2024-09-04T17:47:53.935 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.935 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.935 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:53.935 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822980+0000", 2024-09-04T17:47:53.936 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.242e-06 2024-09-04T17:47:53.936 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.936 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.936 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:53.936 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822983+0000", 2024-09-04T17:47:53.936 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.8830000000000002e-06 2024-09-04T17:47:53.936 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.936 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.936 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:53.936 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897519+0000", 2024-09-04T17:47:53.936 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074536366000000007 2024-09-04T17:47:53.936 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.936 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.936 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:53.937 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897560+0000", 2024-09-04T17:47:53.937 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0677999999999999e-05 2024-09-04T17:47:53.937 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.937 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.937 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:53.937 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897610+0000", 2024-09-04T17:47:53.937 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0832999999999999e-05 2024-09-04T17:47:53.937 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.937 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:53.937 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.937 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.937 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.937 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60926 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 860160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:53.938 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822986+0000", 2024-09-04T17:47:53.938 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 40.089853642000001, 2024-09-04T17:47:53.938 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074736413000000002, 2024-09-04T17:47:53.938 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:53.938 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:53.938 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:53.938 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:53.938 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:53.938 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:53.938 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60926 2024-09-04T17:47:53.938 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.938 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:53.938 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.939 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:53.939 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822986+0000", 2024-09-04T17:47:53.939 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.939 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.939 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.939 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:53.939 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822986+0000", 2024-09-04T17:47:53.939 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.939 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.939 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.939 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:53.939 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822986+0000", 2024-09-04T17:47:53.939 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5600000000000002e-07 2024-09-04T17:47:53.939 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.940 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.940 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:53.940 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822987+0000", 2024-09-04T17:47:53.940 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.082e-06 2024-09-04T17:47:53.940 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.940 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.940 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:53.940 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822990+0000", 2024-09-04T17:47:53.940 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.2219999999999998e-06 2024-09-04T17:47:53.940 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.940 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.940 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:53.940 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822994+0000", 2024-09-04T17:47:53.941 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.19e-06 2024-09-04T17:47:53.941 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.941 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.941 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:53.941 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897632+0000", 2024-09-04T17:47:53.941 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074638205999999999 2024-09-04T17:47:53.941 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.941 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.941 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:53.941 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897672+0000", 2024-09-04T17:47:53.941 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.9873999999999998e-05 2024-09-04T17:47:53.941 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.941 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.941 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:53.941 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897722+0000", 2024-09-04T17:47:53.942 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0583e-05 2024-09-04T17:47:53.942 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.942 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:53.942 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.942 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.942 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.942 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60927 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 966656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:53.942 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823000+0000", 2024-09-04T17:47:53.942 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 40.08983946, 2024-09-04T17:47:53.942 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074834036000000007, 2024-09-04T17:47:53.942 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:53.942 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:53.942 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:53.942 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:53.943 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:53.943 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:53.943 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60927 2024-09-04T17:47:53.943 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.943 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:53.943 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.943 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:53.943 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823000+0000", 2024-09-04T17:47:53.943 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.943 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.943 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.943 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:53.943 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823000+0000", 2024-09-04T17:47:53.943 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.943 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.944 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.944 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:53.944 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823000+0000", 2024-09-04T17:47:53.944 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4099999999999999e-07 2024-09-04T17:47:53.944 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.944 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.944 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:53.944 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823003+0000", 2024-09-04T17:47:53.944 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0810000000000001e-06 2024-09-04T17:47:53.944 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.944 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.944 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:53.944 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823011+0000", 2024-09-04T17:47:53.944 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7090000000000008e-06 2024-09-04T17:47:53.945 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.945 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.945 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:53.945 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823017+0000", 2024-09-04T17:47:53.945 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.3890000000000004e-06 2024-09-04T17:47:53.945 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.945 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.945 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:53.945 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897745+0000", 2024-09-04T17:47:53.945 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074728252999999994 2024-09-04T17:47:53.945 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.945 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.945 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:53.946 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897785+0000", 2024-09-04T17:47:53.946 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.9830999999999999e-05 2024-09-04T17:47:53.946 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.946 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.946 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:53.946 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897834+0000", 2024-09-04T17:47:53.946 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9231999999999999e-05 2024-09-04T17:47:53.946 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.946 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:53.946 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.946 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.946 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.946 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60928 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:53.947 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823024+0000", 2024-09-04T17:47:53.947 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 40.089815528999999, 2024-09-04T17:47:53.947 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075067479000000006, 2024-09-04T17:47:53.947 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:53.947 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:53.947 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:53.947 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:53.947 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:53.947 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:53.947 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60928 2024-09-04T17:47:53.947 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.947 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:53.947 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.947 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:53.948 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823024+0000", 2024-09-04T17:47:53.948 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.948 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.948 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.948 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:53.948 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823024+0000", 2024-09-04T17:47:53.948 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.948 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.948 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.948 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:53.948 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823024+0000", 2024-09-04T17:47:53.948 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6499999999999999e-07 2024-09-04T17:47:53.948 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.948 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.948 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:53.949 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823027+0000", 2024-09-04T17:47:53.949 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.3099999999999999e-06 2024-09-04T17:47:53.949 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.949 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.949 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:53.949 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823033+0000", 2024-09-04T17:47:53.949 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7429999999999996e-06 2024-09-04T17:47:53.949 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.949 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.949 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:53.949 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823038+0000", 2024-09-04T17:47:53.949 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.561e-06 2024-09-04T17:47:53.950 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.950 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.950 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:53.950 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897855+0000", 2024-09-04T17:47:53.950 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074816740000000007 2024-09-04T17:47:53.950 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.950 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.950 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:53.950 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897894+0000", 2024-09-04T17:47:53.950 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.9085999999999998e-05 2024-09-04T17:47:53.950 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.950 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.950 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:53.951 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898091+0000", 2024-09-04T17:47:53.951 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000197574 2024-09-04T17:47:53.951 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.951 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:53.951 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.951 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.951 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.951 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60929 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1212416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:53.951 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823042+0000", 2024-09-04T17:47:53.951 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 40.089797087000001, 2024-09-04T17:47:53.951 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075301251, 2024-09-04T17:47:53.951 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:53.951 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:53.951 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:53.952 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:53.952 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:53.952 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:53.952 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60929 2024-09-04T17:47:53.952 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.952 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:53.952 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.952 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:53.952 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823042+0000", 2024-09-04T17:47:53.952 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.952 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.952 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.952 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:53.953 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823042+0000", 2024-09-04T17:47:53.953 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.953 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.953 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.953 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:53.953 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823043+0000", 2024-09-04T17:47:53.953 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.1100000000000002e-07 2024-09-04T17:47:53.953 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.953 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.953 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:53.953 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823045+0000", 2024-09-04T17:47:53.953 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.933e-06 2024-09-04T17:47:53.953 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.954 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.954 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:53.954 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823051+0000", 2024-09-04T17:47:53.954 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.9569999999999999e-06 2024-09-04T17:47:53.954 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.954 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.954 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:53.954 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823055+0000", 2024-09-04T17:47:53.954 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4479999999999996e-06 2024-09-04T17:47:53.954 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.954 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.954 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:53.954 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898117+0000", 2024-09-04T17:47:53.954 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075062344000000003 2024-09-04T17:47:53.955 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.955 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.955 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:53.955 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898157+0000", 2024-09-04T17:47:53.955 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0095999999999997e-05 2024-09-04T17:47:53.955 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.955 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.955 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:53.955 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898344+0000", 2024-09-04T17:47:53.955 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000186162 2024-09-04T17:47:53.955 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.955 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:53.955 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.955 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.956 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.956 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60930 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1351680~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:53.956 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823061+0000", 2024-09-04T17:47:53.956 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 40.089778207999998, 2024-09-04T17:47:53.956 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075545507999999997, 2024-09-04T17:47:53.956 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:53.956 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:53.956 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:53.956 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:53.956 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:53.956 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:53.956 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60930 2024-09-04T17:47:53.956 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.957 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:53.957 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.957 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:53.957 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823061+0000", 2024-09-04T17:47:53.957 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.957 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.957 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.957 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:53.957 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823061+0000", 2024-09-04T17:47:53.957 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.957 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.957 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.957 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:53.958 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823378+0000", 2024-09-04T17:47:53.958 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00031631299999999998 2024-09-04T17:47:53.958 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.958 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.958 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:53.958 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823381+0000", 2024-09-04T17:47:53.958 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4020000000000002e-06 2024-09-04T17:47:53.958 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.958 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.958 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:53.958 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823389+0000", 2024-09-04T17:47:53.958 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.1429999999999998e-06 2024-09-04T17:47:53.958 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.958 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.958 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:53.959 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823395+0000", 2024-09-04T17:47:53.959 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.0059999999999998e-06 2024-09-04T17:47:53.959 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.959 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.959 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:53.959 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898368+0000", 2024-09-04T17:47:53.959 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074973032999999994 2024-09-04T17:47:53.959 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.959 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.959 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:53.959 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898410+0000", 2024-09-04T17:47:53.959 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2243000000000002e-05 2024-09-04T17:47:53.959 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.959 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.960 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:53.960 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898607+0000", 2024-09-04T17:47:53.960 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019636799999999999 2024-09-04T17:47:53.960 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.960 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:53.960 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.960 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.960 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.960 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60931 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1458176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:53.960 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823407+0000", 2024-09-04T17:47:53.960 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 40.089432133999999, 2024-09-04T17:47:53.960 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075460334000000004, 2024-09-04T17:47:53.960 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:53.961 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:53.961 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:53.961 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:53.961 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:53.961 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:53.961 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60931 2024-09-04T17:47:53.961 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.961 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:53.961 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.961 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:53.961 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823407+0000", 2024-09-04T17:47:53.961 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.961 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.961 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.962 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:53.962 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823407+0000", 2024-09-04T17:47:53.962 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.962 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.962 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.962 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:53.962 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823408+0000", 2024-09-04T17:47:53.962 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9999999999999998e-07 2024-09-04T17:47:53.962 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.962 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.962 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:53.962 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823415+0000", 2024-09-04T17:47:53.963 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.5510000000000002e-06 2024-09-04T17:47:53.963 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.963 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.963 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:53.963 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823423+0000", 2024-09-04T17:47:53.963 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2760000000000003e-06 2024-09-04T17:47:53.963 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.963 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.963 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:53.963 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823426+0000", 2024-09-04T17:47:53.963 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.2839999999999998e-06 2024-09-04T17:47:53.963 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.963 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.964 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:53.964 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898631+0000", 2024-09-04T17:47:53.964 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075205045999999998 2024-09-04T17:47:53.964 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.964 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.964 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:53.964 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898672+0000", 2024-09-04T17:47:53.964 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0614000000000001e-05 2024-09-04T17:47:53.964 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.964 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.964 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:53.964 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898868+0000", 2024-09-04T17:47:53.964 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019606300000000001 2024-09-04T17:47:53.964 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.965 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:53.965 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.965 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.965 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.965 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60932 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1564672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:53.965 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823434+0000", 2024-09-04T17:47:53.965 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 40.089405835000001, 2024-09-04T17:47:53.965 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075693393999999997, 2024-09-04T17:47:53.965 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:53.965 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:53.965 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:53.965 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:53.965 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:53.965 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:53.966 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60932 2024-09-04T17:47:53.966 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.966 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:53.966 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.966 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:53.966 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823434+0000", 2024-09-04T17:47:53.966 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.966 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.966 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.966 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:53.966 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823434+0000", 2024-09-04T17:47:53.966 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.966 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.967 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.967 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:53.967 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.830290+0000", 2024-09-04T17:47:53.967 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0068563039999999997 2024-09-04T17:47:53.967 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.967 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.967 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:53.967 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.830297+0000", 2024-09-04T17:47:53.967 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4760000000000001e-06 2024-09-04T17:47:53.967 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.967 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.967 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:53.967 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.830308+0000", 2024-09-04T17:47:53.968 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0927999999999999e-05 2024-09-04T17:47:53.968 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.968 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.968 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:53.968 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.830317+0000", 2024-09-04T17:47:53.968 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7280000000000001e-06 2024-09-04T17:47:53.968 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.968 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.968 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:53.968 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898892+0000", 2024-09-04T17:47:53.968 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.068574722000000005 2024-09-04T17:47:53.968 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.968 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.969 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:53.969 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898934+0000", 2024-09-04T17:47:53.969 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2249000000000002e-05 2024-09-04T17:47:53.969 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.969 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.969 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:53.969 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.899127+0000", 2024-09-04T17:47:53.969 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019298699999999999 2024-09-04T17:47:53.969 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.969 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:53.969 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.969 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.970 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.970 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61182 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 868352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:53.970 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839249+0000", 2024-09-04T17:47:53.970 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 40.073589943000002, 2024-09-04T17:47:53.970 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073680151999999999, 2024-09-04T17:47:53.970 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:53.970 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:53.970 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:53.970 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:53.970 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:53.970 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:53.970 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61182 2024-09-04T17:47:53.970 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.971 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:53.971 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.971 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:53.971 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839249+0000", 2024-09-04T17:47:53.971 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.971 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.971 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.971 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:53.971 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839249+0000", 2024-09-04T17:47:53.971 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.971 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.971 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.971 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:53.971 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839250+0000", 2024-09-04T17:47:53.972 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4999999999999998e-07 2024-09-04T17:47:53.972 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.972 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.972 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:53.972 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839257+0000", 2024-09-04T17:47:53.972 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7700000000000004e-06 2024-09-04T17:47:53.972 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.972 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.972 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:53.972 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839263+0000", 2024-09-04T17:47:53.972 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.246e-06 2024-09-04T17:47:53.972 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.973 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.973 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:53.973 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839268+0000", 2024-09-04T17:47:53.973 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.814e-06 2024-09-04T17:47:53.973 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.973 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.973 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:53.973 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.912831+0000", 2024-09-04T17:47:53.973 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073563294000000001 2024-09-04T17:47:53.973 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.973 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.973 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:53.973 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.912876+0000", 2024-09-04T17:47:53.974 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4742e-05 2024-09-04T17:47:53.974 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.974 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.974 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:53.974 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.912930+0000", 2024-09-04T17:47:53.974 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.3936e-05 2024-09-04T17:47:53.974 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.974 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:53.974 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.974 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.974 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.974 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61183 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:53.974 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839275+0000", 2024-09-04T17:47:53.975 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 40.073564750000003, 2024-09-04T17:47:53.975 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073781886000000005, 2024-09-04T17:47:53.975 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:53.975 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:53.975 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:53.975 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:53.975 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:53.975 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:53.975 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61183 2024-09-04T17:47:53.975 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.975 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:53.975 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.975 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:53.976 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839275+0000", 2024-09-04T17:47:53.976 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.976 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.976 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.976 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:53.976 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839275+0000", 2024-09-04T17:47:53.976 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.976 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.976 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.976 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:53.976 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839275+0000", 2024-09-04T17:47:53.976 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6899999999999997e-07 2024-09-04T17:47:53.977 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.977 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.977 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:53.977 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839277+0000", 2024-09-04T17:47:53.977 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9530000000000002e-06 2024-09-04T17:47:53.977 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.977 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.977 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:53.977 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839283+0000", 2024-09-04T17:47:53.977 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.9329999999999998e-06 2024-09-04T17:47:53.977 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.977 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.977 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:53.978 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839288+0000", 2024-09-04T17:47:53.978 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4100000000000001e-06 2024-09-04T17:47:53.978 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.978 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.978 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:53.978 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.912959+0000", 2024-09-04T17:47:53.978 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073671511999999995 2024-09-04T17:47:53.978 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.978 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.978 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:53.978 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913001+0000", 2024-09-04T17:47:53.978 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.1984999999999999e-05 2024-09-04T17:47:53.979 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.979 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.979 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:53.979 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913057+0000", 2024-09-04T17:47:53.979 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.5523999999999999e-05 2024-09-04T17:47:53.979 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.979 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:53.979 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.979 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.979 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.979 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61184 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1114112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:53.979 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839294+0000", 2024-09-04T17:47:53.979 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 40.073545537999998, 2024-09-04T17:47:53.979 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073995263000000006, 2024-09-04T17:47:53.980 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:53.980 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:53.980 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:53.980 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:53.980 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:53.980 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:53.980 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61184 2024-09-04T17:47:53.980 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.980 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:53.980 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.980 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:53.980 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839294+0000", 2024-09-04T17:47:53.980 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.981 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.981 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.981 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:53.981 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839294+0000", 2024-09-04T17:47:53.981 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.981 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.981 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.981 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:53.981 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839294+0000", 2024-09-04T17:47:53.981 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.77e-07 2024-09-04T17:47:53.981 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.981 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.981 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:53.981 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839296+0000", 2024-09-04T17:47:53.982 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9999999999999999e-06 2024-09-04T17:47:53.982 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.982 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.982 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:53.982 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839302+0000", 2024-09-04T17:47:53.982 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7230000000000001e-06 2024-09-04T17:47:53.982 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.982 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.982 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:53.982 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839314+0000", 2024-09-04T17:47:53.982 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1826e-05 2024-09-04T17:47:53.982 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.982 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.982 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:53.982 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913084+0000", 2024-09-04T17:47:53.983 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073770215 2024-09-04T17:47:53.983 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.983 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.983 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:53.983 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913127+0000", 2024-09-04T17:47:53.983 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2982999999999998e-05 2024-09-04T17:47:53.983 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.983 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.983 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:53.983 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913289+0000", 2024-09-04T17:47:53.983 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000162139 2024-09-04T17:47:53.983 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.983 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:53.983 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.983 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.984 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.984 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61185 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1220608~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:53.984 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839321+0000", 2024-09-04T17:47:53.984 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 40.073518071000002, 2024-09-04T17:47:53.984 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074206062000000003, 2024-09-04T17:47:53.984 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:53.984 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:53.984 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:53.984 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:53.984 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:53.984 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:53.984 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61185 2024-09-04T17:47:53.984 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.984 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:53.984 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.985 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:53.985 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839321+0000", 2024-09-04T17:47:53.985 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.985 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.985 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.987 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:53.988 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839321+0000", 2024-09-04T17:47:53.988 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.988 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.988 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.988 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:53.988 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839322+0000", 2024-09-04T17:47:53.988 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.5400000000000001e-07 2024-09-04T17:47:53.988 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.988 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.988 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:53.988 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839324+0000", 2024-09-04T17:47:53.988 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2539999999999999e-06 2024-09-04T17:47:53.988 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.989 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.989 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:53.989 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839331+0000", 2024-09-04T17:47:53.989 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2139999999999999e-06 2024-09-04T17:47:53.989 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.989 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.989 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:53.989 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839336+0000", 2024-09-04T17:47:53.989 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7029999999999997e-06 2024-09-04T17:47:53.989 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.989 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.989 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:53.989 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913320+0000", 2024-09-04T17:47:53.990 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073984016 2024-09-04T17:47:53.990 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.990 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.990 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:53.990 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913366+0000", 2024-09-04T17:47:53.990 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.5704000000000002e-05 2024-09-04T17:47:53.990 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.990 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.990 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:53.990 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913527+0000", 2024-09-04T17:47:53.990 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00016161699999999999 2024-09-04T17:47:53.990 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.990 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:53.990 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.990 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.990 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.991 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61186 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1359872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:53.991 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839344+0000", 2024-09-04T17:47:53.991 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 40.073495729000001, 2024-09-04T17:47:53.991 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074450673999999994, 2024-09-04T17:47:53.991 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:53.991 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:53.991 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:53.991 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:53.991 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:53.991 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:53.991 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61186 2024-09-04T17:47:53.991 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.991 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:53.991 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.991 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:53.992 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839344+0000", 2024-09-04T17:47:53.992 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.992 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.992 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.992 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:53.992 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839344+0000", 2024-09-04T17:47:53.992 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.992 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.992 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.992 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:53.992 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839344+0000", 2024-09-04T17:47:53.992 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4799999999999999e-07 2024-09-04T17:47:53.992 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.993 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.993 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:53.993 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839346+0000", 2024-09-04T17:47:53.993 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0889999999999998e-06 2024-09-04T17:47:53.993 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.993 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.993 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:53.993 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839352+0000", 2024-09-04T17:47:53.993 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6389999999999997e-06 2024-09-04T17:47:53.993 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.993 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.993 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:53.993 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839356+0000", 2024-09-04T17:47:53.994 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.233e-06 2024-09-04T17:47:53.994 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.994 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.994 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:53.994 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913557+0000", 2024-09-04T17:47:53.994 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074200944000000005 2024-09-04T17:47:53.994 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.994 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.994 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:53.994 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913601+0000", 2024-09-04T17:47:53.994 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4350000000000001e-05 2024-09-04T17:47:53.994 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.994 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.994 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:53.994 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913794+0000", 2024-09-04T17:47:53.995 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019297100000000001 2024-09-04T17:47:53.995 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.995 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:53.995 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.995 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.995 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.995 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61187 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1466368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:53.995 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839364+0000", 2024-09-04T17:47:53.995 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 40.073475879999997, 2024-09-04T17:47:53.995 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074692334999999999, 2024-09-04T17:47:53.995 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:53.995 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:53.995 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:53.995 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:53.995 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:53.996 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:53.996 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61187 2024-09-04T17:47:53.996 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.996 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:53.996 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.996 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:53.996 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839364+0000", 2024-09-04T17:47:53.996 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.996 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.996 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.996 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:53.996 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839364+0000", 2024-09-04T17:47:53.996 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:53.996 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.997 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.997 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:53.997 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839364+0000", 2024-09-04T17:47:53.997 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.75e-07 2024-09-04T17:47:53.997 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.997 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.997 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:53.997 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839366+0000", 2024-09-04T17:47:53.997 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0990000000000001e-06 2024-09-04T17:47:53.997 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.997 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.997 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:53.997 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839372+0000", 2024-09-04T17:47:53.997 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.2609999999999999e-06 2024-09-04T17:47:53.997 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.998 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.998 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:53.998 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839389+0000", 2024-09-04T17:47:53.998 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6489000000000001e-05 2024-09-04T17:47:53.998 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.998 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.998 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:53.998 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913824+0000", 2024-09-04T17:47:53.998 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074434811000000004 2024-09-04T17:47:53.998 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.998 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.998 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:53.998 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913868+0000", 2024-09-04T17:47:53.998 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4768000000000002e-05 2024-09-04T17:47:53.999 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.999 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.999 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:53.999 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914056+0000", 2024-09-04T17:47:53.999 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00018743199999999999 2024-09-04T17:47:53.999 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.999 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:53.999 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:53.999 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:53.999 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:53.999 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61188 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1605632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:53.999 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839399+0000", 2024-09-04T17:47:53.999 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 40.073439983999997, 2024-09-04T17:47:53.999 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074975095000000005, 2024-09-04T17:47:54.000 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:54.000 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:54.000 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:54.000 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:54.000 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:54.000 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:54.000 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61188 2024-09-04T17:47:54.000 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:54.000 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:54.000 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:54.000 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:54.001 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839399+0000", 2024-09-04T17:47:54.001 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:54.001 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:54.001 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:54.001 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:54.001 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839399+0000", 2024-09-04T17:47:54.001 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:54.001 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:54.001 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:54.001 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:54.001 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839400+0000", 2024-09-04T17:47:54.001 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0100000000000005e-07 2024-09-04T17:47:54.001 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:54.001 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:54.001 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:54.002 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839402+0000", 2024-09-04T17:47:54.002 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.491e-06 2024-09-04T17:47:54.002 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:54.002 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:54.002 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:54.002 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839409+0000", 2024-09-04T17:47:54.002 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.0450000000000003e-06 2024-09-04T17:47:54.002 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:54.002 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:54.002 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:54.002 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839414+0000", 2024-09-04T17:47:54.002 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.87e-06 2024-09-04T17:47:54.002 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:54.002 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:54.003 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:54.003 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914086+0000", 2024-09-04T17:47:54.003 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074671596000000007 2024-09-04T17:47:54.003 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:54.003 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:54.003 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:54.003 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914130+0000", 2024-09-04T17:47:54.003 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4579999999999997e-05 2024-09-04T17:47:54.003 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:54.003 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:54.003 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:54.003 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914375+0000", 2024-09-04T17:47:54.003 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024401199999999999 2024-09-04T17:47:54.003 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:54.004 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:54.004 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:54.004 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:54.004 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:54.004 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61189 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1712128~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:54.004 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839424+0000", 2024-09-04T17:47:54.004 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 40.073415783000002, 2024-09-04T17:47:54.004 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075247680999999997, 2024-09-04T17:47:54.004 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:54.004 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:54.004 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:54.004 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:54.004 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:54.004 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:54.005 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61189 2024-09-04T17:47:54.005 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:54.005 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:54.005 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:54.005 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:54.005 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839424+0000", 2024-09-04T17:47:54.005 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:54.005 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:54.005 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:54.005 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:54.005 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839424+0000", 2024-09-04T17:47:54.005 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:54.005 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:54.005 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:54.006 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:54.006 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.846320+0000", 2024-09-04T17:47:54.006 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0068965210000000001 2024-09-04T17:47:54.006 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:54.006 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:54.006 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:54.006 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.846328+0000", 2024-09-04T17:47:54.006 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.8189999999999992e-06 2024-09-04T17:47:54.006 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:54.006 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:54.006 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:54.006 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.846341+0000", 2024-09-04T17:47:54.006 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3338e-05 2024-09-04T17:47:54.006 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:54.006 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:54.007 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:54.007 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.846352+0000", 2024-09-04T17:47:54.007 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0479000000000001e-05 2024-09-04T17:47:54.007 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:54.007 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:54.007 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:54.007 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914405+0000", 2024-09-04T17:47:54.007 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.068053049000000004 2024-09-04T17:47:54.007 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:54.007 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:54.007 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:54.007 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914451+0000", 2024-09-04T17:47:54.007 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6545999999999998e-05 2024-09-04T17:47:54.007 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:54.007 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:54.008 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:54.008 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914671+0000", 2024-09-04T17:47:54.008 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000219929 2024-09-04T17:47:54.008 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:54.008 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:54.008 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:54.008 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:54.008 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:54.008 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:54.009 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-04T17:47:54.010 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:54.008+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:54.111 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:47:54.108+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:47:54.148 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:54.148 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:47:54.148 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:47:54.148 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:54.164 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-04T17:47:54.212 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:47:54.207+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:47:54.217 INFO:teuthology.orchestra.run.smithi092.stdout:truncating image image_client.0-clone41 from 0x9a8f9f9 (overlap 0x1818142) to 0x73a6c7 2024-09-04T17:47:54.313 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:54.311+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:54.346 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:54.346 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:47:54.346 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:47:54.346 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:47:54.347 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:54.362 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-04T17:47:54.396 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-04T17:47:54.453 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:54.452+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:54.515 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:54.511+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:54.517 INFO:teuthology.orchestra.run.smithi190.stderr:no valid command found; 10 closest matches: 2024-09-04T17:47:54.517 INFO:teuthology.orchestra.run.smithi190.stderr:0 2024-09-04T17:47:54.518 INFO:teuthology.orchestra.run.smithi190.stderr:1 2024-09-04T17:47:54.518 INFO:teuthology.orchestra.run.smithi190.stderr:2 2024-09-04T17:47:54.518 INFO:teuthology.orchestra.run.smithi190.stderr:abort 2024-09-04T17:47:54.518 INFO:teuthology.orchestra.run.smithi190.stderr:assert 2024-09-04T17:47:54.518 INFO:teuthology.orchestra.run.smithi190.stderr:bluestore allocator dump block 2024-09-04T17:47:54.518 INFO:teuthology.orchestra.run.smithi190.stderr:bluestore allocator fragmentation block 2024-09-04T17:47:54.518 INFO:teuthology.orchestra.run.smithi190.stderr:bluestore allocator fragmentation histogram block [] [] 2024-09-04T17:47:54.518 INFO:teuthology.orchestra.run.smithi190.stderr:bluestore allocator score block 2024-09-04T17:47:54.518 INFO:teuthology.orchestra.run.smithi190.stderr:config diff 2024-09-04T17:47:54.518 INFO:teuthology.orchestra.run.smithi190.stderr:admin_socket: invalid command 2024-09-04T17:47:54.520 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-04T17:47:54.520 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-6, ['dump_ops_in_flight'] 2024-09-04T17:47:54.538 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:54.538 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:47:54.539 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:47:54.539 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:47:54.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.539 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:27242 3.b 3.34414deb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-04T17:47:54.539 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:28.402347+0000", 2024-09-04T17:47:54.539 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.133038880000001, 2024-09-04T17:47:54.539 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.93265329299999999, 2024-09-04T17:47:54.539 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:54.539 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:54.539 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-04T17:47:54.539 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:54.539 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:54.539 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:54.540 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 27242 2024-09-04T17:47:54.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.540 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:54.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:54.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402347+0000", 2024-09-04T17:47:54.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:54.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:54.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402347+0000", 2024-09-04T17:47:54.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:54.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.541 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:54.541 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402349+0000", 2024-09-04T17:47:54.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.471e-06 2024-09-04T17:47:54.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.541 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:54.541 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402355+0000", 2024-09-04T17:47:54.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8399999999999997e-06 2024-09-04T17:47:54.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.541 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:54.542 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402364+0000", 2024-09-04T17:47:54.542 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1269999999999992e-06 2024-09-04T17:47:54.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.542 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.542 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:54.542 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402373+0000", 2024-09-04T17:47:54.542 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7909999999999993e-06 2024-09-04T17:47:54.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.542 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.542 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:54.542 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402460+0000", 2024-09-04T17:47:54.542 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6686000000000003e-05 2024-09-04T17:47:54.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.542 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.543 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-04T17:47:54.543 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402473+0000", 2024-09-04T17:47:54.543 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3281e-05 2024-09-04T17:47:54.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.543 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.543 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:54.543 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:29.334989+0000", 2024-09-04T17:47:54.543 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.93251549099999997 2024-09-04T17:47:54.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.543 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.543 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:54.543 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:29.335000+0000", 2024-09-04T17:47:54.544 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1606e-05 2024-09-04T17:47:54.544 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:54.544 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:54.544 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:54.544 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.544 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.544 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:33552 3.7 3:e79ad1db:::rbd_data.11342e8f3384.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-04T17:47:54.544 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:35.999748+0000", 2024-09-04T17:47:54.544 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 78.535637829999999, 2024-09-04T17:47:54.544 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045683823999999998, 2024-09-04T17:47:54.544 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:54.544 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:54.544 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:54.545 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:54.545 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:54.545 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:54.545 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 33552 2024-09-04T17:47:54.545 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.545 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:54.545 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.545 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:54.545 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999748+0000", 2024-09-04T17:47:54.545 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:54.545 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.545 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.545 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:54.546 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999748+0000", 2024-09-04T17:47:54.546 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:54.546 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.546 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.546 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:54.546 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999750+0000", 2024-09-04T17:47:54.546 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1299999999999999e-06 2024-09-04T17:47:54.546 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.546 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.546 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:54.546 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999759+0000", 2024-09-04T17:47:54.546 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4649999999999994e-06 2024-09-04T17:47:54.546 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.546 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.547 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:54.547 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999778+0000", 2024-09-04T17:47:54.547 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8833000000000001e-05 2024-09-04T17:47:54.547 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.547 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.547 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:54.547 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999786+0000", 2024-09-04T17:47:54.547 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7369999999999995e-06 2024-09-04T17:47:54.547 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.547 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.547 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:54.547 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999856+0000", 2024-09-04T17:47:54.547 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9247999999999998e-05 2024-09-04T17:47:54.548 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.548 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.548 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:54.548 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999973+0000", 2024-09-04T17:47:54.548 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000117921 2024-09-04T17:47:54.548 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.548 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.548 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 7", 2024-09-04T17:47:54.548 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.005955+0000", 2024-09-04T17:47:54.548 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059820560000000004 2024-09-04T17:47:54.548 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.548 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.548 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:54.549 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.045297+0000", 2024-09-04T17:47:54.549 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.039341650999999998 2024-09-04T17:47:54.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.549 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.549 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:54.549 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.045343+0000", 2024-09-04T17:47:54.549 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5899000000000002e-05 2024-09-04T17:47:54.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.549 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.549 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:54.549 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.045388+0000", 2024-09-04T17:47:54.549 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4601999999999997e-05 2024-09-04T17:47:54.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.550 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.550 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:54.550 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.045432+0000", 2024-09-04T17:47:54.550 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4282000000000001e-05 2024-09-04T17:47:54.550 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:54.550 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:54.550 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:54.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.550 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.550 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:37357 3.15 3:a897312a:::rbd_data.1134c5a43465.000000000000001f:head [stat,write 62464~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-04T17:47:54.551 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:39.616724+0000", 2024-09-04T17:47:54.551 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 74.918662421999997, 2024-09-04T17:47:54.551 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054965854000000001, 2024-09-04T17:47:54.551 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:54.551 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:54.551 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:54.551 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:54.551 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:54.551 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:54.551 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 37357 2024-09-04T17:47:54.551 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.551 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:54.551 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.552 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:54.552 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616724+0000", 2024-09-04T17:47:54.552 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:54.552 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.552 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.552 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:54.552 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616724+0000", 2024-09-04T17:47:54.552 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:54.552 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.552 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.552 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:54.552 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616725+0000", 2024-09-04T17:47:54.552 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.815e-06 2024-09-04T17:47:54.552 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.553 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.553 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:54.553 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616739+0000", 2024-09-04T17:47:54.553 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3723000000000001e-05 2024-09-04T17:47:54.553 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.553 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.553 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:54.553 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616755+0000", 2024-09-04T17:47:54.553 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5964000000000002e-05 2024-09-04T17:47:54.553 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.553 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.553 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:54.553 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616763+0000", 2024-09-04T17:47:54.553 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3869999999999995e-06 2024-09-04T17:47:54.554 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.554 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.554 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:54.554 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616825+0000", 2024-09-04T17:47:54.554 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1854999999999996e-05 2024-09-04T17:47:54.554 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.554 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.554 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:54.554 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616895+0000", 2024-09-04T17:47:54.554 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9597999999999993e-05 2024-09-04T17:47:54.554 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.554 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.555 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-04T17:47:54.555 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616991+0000", 2024-09-04T17:47:54.555 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5828000000000005e-05 2024-09-04T17:47:54.555 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.555 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.555 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:54.555 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.617659+0000", 2024-09-04T17:47:54.555 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00066821600000000001 2024-09-04T17:47:54.555 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.555 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.555 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:54.555 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.671482+0000", 2024-09-04T17:47:54.556 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053823533999999999 2024-09-04T17:47:54.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.556 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.556 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:54.556 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.671579+0000", 2024-09-04T17:47:54.556 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6657999999999995e-05 2024-09-04T17:47:54.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.556 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.556 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:54.556 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.671689+0000", 2024-09-04T17:47:54.556 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000110276 2024-09-04T17:47:54.556 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:54.556 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:54.557 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:54.557 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.557 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.557 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:43873 3.15 3:abd207ee:::rbd_data.113410ddd206.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:54.557 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:49.932940+0000", 2024-09-04T17:47:54.557 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 64.602445868999993, 2024-09-04T17:47:54.557 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042740514, 2024-09-04T17:47:54.557 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:54.557 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:54.557 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:54.557 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:54.557 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:54.557 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:54.557 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 43873 2024-09-04T17:47:54.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.558 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:54.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.558 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:54.558 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.932940+0000", 2024-09-04T17:47:54.558 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:54.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.558 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:54.558 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.932940+0000", 2024-09-04T17:47:54.558 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:54.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.558 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:54.559 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.932941+0000", 2024-09-04T17:47:54.559 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.404e-06 2024-09-04T17:47:54.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.559 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.559 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:54.559 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.933065+0000", 2024-09-04T17:47:54.559 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012361199999999999 2024-09-04T17:47:54.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.559 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.559 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:54.559 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.933078+0000", 2024-09-04T17:47:54.559 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3182e-05 2024-09-04T17:47:54.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.560 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:54.560 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.933085+0000", 2024-09-04T17:47:54.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4500000000000001e-06 2024-09-04T17:47:54.560 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.560 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:54.560 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.933134+0000", 2024-09-04T17:47:54.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9150999999999997e-05 2024-09-04T17:47:54.560 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.560 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:54.560 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.933242+0000", 2024-09-04T17:47:54.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010781 2024-09-04T17:47:54.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.561 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-04T17:47:54.561 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.938358+0000", 2024-09-04T17:47:54.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051160349999999997 2024-09-04T17:47:54.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.561 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:54.561 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.975343+0000", 2024-09-04T17:47:54.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036985663000000002 2024-09-04T17:47:54.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.561 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:54.561 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.975392+0000", 2024-09-04T17:47:54.562 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9044e-05 2024-09-04T17:47:54.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.562 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.562 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:54.562 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.975548+0000", 2024-09-04T17:47:54.562 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015572900000000001 2024-09-04T17:47:54.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.562 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.562 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:54.562 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.975681+0000", 2024-09-04T17:47:54.562 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013243400000000001 2024-09-04T17:47:54.562 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:54.562 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:54.562 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:54.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.563 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:46649 3.7 3:e0749ed5:::rbd_data.1134326b938e.000000000000014f:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-04T17:47:54.563 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:54.114730+0000", 2024-09-04T17:47:54.563 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 60.420655736999997, 2024-09-04T17:47:54.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032972972000000003, 2024-09-04T17:47:54.563 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:54.563 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:54.563 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:54.563 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:54.563 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:54.563 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:54.563 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 46649 2024-09-04T17:47:54.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.563 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:54.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:54.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114730+0000", 2024-09-04T17:47:54.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:54.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:54.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114730+0000", 2024-09-04T17:47:54.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:54.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:54.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114731+0000", 2024-09-04T17:47:54.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.147e-06 2024-09-04T17:47:54.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.565 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:54.565 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114737+0000", 2024-09-04T17:47:54.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6380000000000001e-06 2024-09-04T17:47:54.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.565 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:54.565 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114751+0000", 2024-09-04T17:47:54.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4012999999999999e-05 2024-09-04T17:47:54.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.566 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:54.566 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114758+0000", 2024-09-04T17:47:54.566 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1539999999999996e-06 2024-09-04T17:47:54.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.566 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:54.566 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114876+0000", 2024-09-04T17:47:54.566 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000117421 2024-09-04T17:47:54.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.566 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:54.566 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.115173+0000", 2024-09-04T17:47:54.566 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00029750799999999998 2024-09-04T17:47:54.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 7", 2024-09-04T17:47:54.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.120273+0000", 2024-09-04T17:47:54.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051002900000000004 2024-09-04T17:47:54.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:54.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.147571+0000", 2024-09-04T17:47:54.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027297906 2024-09-04T17:47:54.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:54.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.147588+0000", 2024-09-04T17:47:54.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6807999999999999e-05 2024-09-04T17:47:54.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:54.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.147637+0000", 2024-09-04T17:47:54.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8466e-05 2024-09-04T17:47:54.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:54.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.147703+0000", 2024-09-04T17:47:54.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6620999999999998e-05 2024-09-04T17:47:54.568 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:54.569 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:54.569 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:54.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.569 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49801 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 720896~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:54.569 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.075094+0000", 2024-09-04T17:47:54.569 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.460291750000003, 2024-09-04T17:47:54.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017824974, 2024-09-04T17:47:54.569 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:54.569 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:54.569 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:54.569 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:54.569 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:54.570 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:54.570 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49801 2024-09-04T17:47:54.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.570 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:54.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:54.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075094+0000", 2024-09-04T17:47:54.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:54.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:54.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075094+0000", 2024-09-04T17:47:54.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:54.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:54.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075095+0000", 2024-09-04T17:47:54.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.017e-06 2024-09-04T17:47:54.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:54.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075101+0000", 2024-09-04T17:47:54.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5029999999999997e-06 2024-09-04T17:47:54.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:54.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075119+0000", 2024-09-04T17:47:54.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8133000000000001e-05 2024-09-04T17:47:54.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.572 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.572 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:54.572 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075132+0000", 2024-09-04T17:47:54.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3305999999999999e-05 2024-09-04T17:47:54.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.572 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.572 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:54.572 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.092677+0000", 2024-09-04T17:47:54.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017544466000000002 2024-09-04T17:47:54.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.572 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.572 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:54.572 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.092782+0000", 2024-09-04T17:47:54.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000105284 2024-09-04T17:47:54.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.573 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:54.573 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.092919+0000", 2024-09-04T17:47:54.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013726499999999999 2024-09-04T17:47:54.573 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:54.573 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:54.573 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:54.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.573 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49809 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 163840~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:54.573 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077146+0000", 2024-09-04T17:47:54.573 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.458240306999997, 2024-09-04T17:47:54.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017984541999999999, 2024-09-04T17:47:54.574 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:54.574 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:54.574 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:54.574 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:54.574 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:54.574 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:54.574 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49809 2024-09-04T17:47:54.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.574 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:54.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:54.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077146+0000", 2024-09-04T17:47:54.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:54.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:54.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077146+0000", 2024-09-04T17:47:54.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:54.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:54.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077147+0000", 2024-09-04T17:47:54.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0330000000000001e-06 2024-09-04T17:47:54.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:54.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077164+0000", 2024-09-04T17:47:54.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7810000000000001e-05 2024-09-04T17:47:54.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.576 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:54.576 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077187+0000", 2024-09-04T17:47:54.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2246e-05 2024-09-04T17:47:54.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.576 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:54.576 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077203+0000", 2024-09-04T17:47:54.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6392000000000002e-05 2024-09-04T17:47:54.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.576 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:54.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.094885+0000", 2024-09-04T17:47:54.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017681589000000001 2024-09-04T17:47:54.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:54.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.094991+0000", 2024-09-04T17:47:54.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010668899999999999 2024-09-04T17:47:54.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:54.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095130+0000", 2024-09-04T17:47:54.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013878299999999999 2024-09-04T17:47:54.577 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:54.578 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:54.578 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:54.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.578 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49810 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 229376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:54.578 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077225+0000", 2024-09-04T17:47:54.578 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.458160466000002, 2024-09-04T17:47:54.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018218793, 2024-09-04T17:47:54.578 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:54.578 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:54.578 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:54.578 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:54.578 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:54.578 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:54.579 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49810 2024-09-04T17:47:54.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.579 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:54.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:54.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077225+0000", 2024-09-04T17:47:54.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:54.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:54.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077225+0000", 2024-09-04T17:47:54.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:54.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:54.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077227+0000", 2024-09-04T17:47:54.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1829999999999999e-06 2024-09-04T17:47:54.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:54.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077233+0000", 2024-09-04T17:47:54.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3899999999999998e-06 2024-09-04T17:47:54.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:54.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077253+0000", 2024-09-04T17:47:54.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0009000000000001e-05 2024-09-04T17:47:54.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.581 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:54.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077270+0000", 2024-09-04T17:47:54.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7336000000000001e-05 2024-09-04T17:47:54.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.581 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:54.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095177+0000", 2024-09-04T17:47:54.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017907101000000002 2024-09-04T17:47:54.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.581 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:54.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095299+0000", 2024-09-04T17:47:54.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000121749 2024-09-04T17:47:54.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:54.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095444+0000", 2024-09-04T17:47:54.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014502499999999999 2024-09-04T17:47:54.582 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:54.582 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:54.582 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:54.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.582 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49811 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 294912~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:54.582 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077288+0000", 2024-09-04T17:47:54.582 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.458098151999998, 2024-09-04T17:47:54.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018443620000000001, 2024-09-04T17:47:54.583 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:54.583 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:54.583 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:54.583 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:54.583 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:54.583 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:54.583 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49811 2024-09-04T17:47:54.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.583 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:54.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.583 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:54.583 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077288+0000", 2024-09-04T17:47:54.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:54.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:54.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077288+0000", 2024-09-04T17:47:54.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:54.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:54.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077289+0000", 2024-09-04T17:47:54.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6199999999999999e-06 2024-09-04T17:47:54.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:54.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077295+0000", 2024-09-04T17:47:54.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1499999999999998e-06 2024-09-04T17:47:54.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:54.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077312+0000", 2024-09-04T17:47:54.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7353000000000001e-05 2024-09-04T17:47:54.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:54.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077324+0000", 2024-09-04T17:47:54.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2593e-05 2024-09-04T17:47:54.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:54.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095490+0000", 2024-09-04T17:47:54.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018165976 2024-09-04T17:47:54.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:54.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095597+0000", 2024-09-04T17:47:54.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010629900000000001 2024-09-04T17:47:54.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:54.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095731+0000", 2024-09-04T17:47:54.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013462899999999999 2024-09-04T17:47:54.586 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:54.587 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:54.587 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:54.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.587 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49813 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 360448~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:54.587 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077349+0000", 2024-09-04T17:47:54.587 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.45803695, 2024-09-04T17:47:54.587 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018780305000000001, 2024-09-04T17:47:54.587 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:54.587 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:54.587 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:54.587 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:54.587 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:54.587 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:54.588 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49813 2024-09-04T17:47:54.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.588 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:54.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:54.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077349+0000", 2024-09-04T17:47:54.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:54.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:54.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077349+0000", 2024-09-04T17:47:54.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:54.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:54.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077350+0000", 2024-09-04T17:47:54.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0330000000000001e-06 2024-09-04T17:47:54.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:54.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077355+0000", 2024-09-04T17:47:54.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0699999999999997e-06 2024-09-04T17:47:54.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:54.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077376+0000", 2024-09-04T17:47:54.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0846e-05 2024-09-04T17:47:54.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:54.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077388+0000", 2024-09-04T17:47:54.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2353e-05 2024-09-04T17:47:54.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:54.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095819+0000", 2024-09-04T17:47:54.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01843057 2024-09-04T17:47:54.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.591 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:54.591 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095960+0000", 2024-09-04T17:47:54.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000140965 2024-09-04T17:47:54.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.591 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:54.591 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096129+0000", 2024-09-04T17:47:54.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016946800000000001 2024-09-04T17:47:54.591 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:54.591 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:54.591 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:54.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.592 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49814 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 425984~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:54.592 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077407+0000", 2024-09-04T17:47:54.592 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.457978449000002, 2024-09-04T17:47:54.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019041563000000001, 2024-09-04T17:47:54.592 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:54.592 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:54.592 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:54.592 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:54.592 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:54.592 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:54.592 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49814 2024-09-04T17:47:54.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.592 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:54.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:54.593 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077407+0000", 2024-09-04T17:47:54.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:54.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:54.593 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077407+0000", 2024-09-04T17:47:54.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:54.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:54.593 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077409+0000", 2024-09-04T17:47:54.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2330000000000001e-06 2024-09-04T17:47:54.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.594 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:54.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077415+0000", 2024-09-04T17:47:54.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2729999999999999e-06 2024-09-04T17:47:54.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.596 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:54.596 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077436+0000", 2024-09-04T17:47:54.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1209999999999999e-05 2024-09-04T17:47:54.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.597 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:54.597 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077448+0000", 2024-09-04T17:47:54.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2225999999999999e-05 2024-09-04T17:47:54.597 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.597 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:54.597 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096183+0000", 2024-09-04T17:47:54.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018735004 2024-09-04T17:47:54.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:54.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096297+0000", 2024-09-04T17:47:54.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000113962 2024-09-04T17:47:54.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:54.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096449+0000", 2024-09-04T17:47:54.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000151655 2024-09-04T17:47:54.599 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:54.599 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:54.599 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:54.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.599 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49815 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 491520~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:54.600 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077471+0000", 2024-09-04T17:47:54.600 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.457915211, 2024-09-04T17:47:54.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019266536000000001, 2024-09-04T17:47:54.600 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:54.600 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:54.600 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:54.600 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:54.601 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:54.601 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:54.601 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49815 2024-09-04T17:47:54.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.601 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:54.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:54.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077471+0000", 2024-09-04T17:47:54.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:54.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:54.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077471+0000", 2024-09-04T17:47:54.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:54.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:54.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077472+0000", 2024-09-04T17:47:54.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.17e-06 2024-09-04T17:47:54.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:54.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077478+0000", 2024-09-04T17:47:54.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0399999999999998e-06 2024-09-04T17:47:54.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:54.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077500+0000", 2024-09-04T17:47:54.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1959000000000001e-05 2024-09-04T17:47:54.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:54.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077513+0000", 2024-09-04T17:47:54.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2823000000000001e-05 2024-09-04T17:47:54.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:54.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096493+0000", 2024-09-04T17:47:54.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018980701999999999 2024-09-04T17:47:54.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:54.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096603+0000", 2024-09-04T17:47:54.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010942000000000001 2024-09-04T17:47:54.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:54.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096737+0000", 2024-09-04T17:47:54.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013442200000000001 2024-09-04T17:47:54.607 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:54.608 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:54.608 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:54.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.608 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49816 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 557056~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:54.608 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077532+0000", 2024-09-04T17:47:54.608 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.457853470000003, 2024-09-04T17:47:54.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019604331999999999, 2024-09-04T17:47:54.609 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:54.609 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:54.609 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:54.609 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:54.609 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:54.609 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:54.609 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49816 2024-09-04T17:47:54.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.610 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:54.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:54.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077532+0000", 2024-09-04T17:47:54.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:54.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:54.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077532+0000", 2024-09-04T17:47:54.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:54.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:54.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077534+0000", 2024-09-04T17:47:54.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1200000000000001e-06 2024-09-04T17:47:54.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:54.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077545+0000", 2024-09-04T17:47:54.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1673e-05 2024-09-04T17:47:54.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:54.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077569+0000", 2024-09-04T17:47:54.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3555999999999999e-05 2024-09-04T17:47:54.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.613 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:54.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077584+0000", 2024-09-04T17:47:54.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4883e-05 2024-09-04T17:47:54.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.613 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:54.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096825+0000", 2024-09-04T17:47:54.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019241734 2024-09-04T17:47:54.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:54.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096964+0000", 2024-09-04T17:47:54.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000138085 2024-09-04T17:47:54.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:54.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097137+0000", 2024-09-04T17:47:54.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017328100000000001 2024-09-04T17:47:54.614 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:54.614 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:54.614 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:54.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.615 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49817 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 622592~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:54.615 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077603+0000", 2024-09-04T17:47:54.615 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.457782911999999, 2024-09-04T17:47:54.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01986928, 2024-09-04T17:47:54.615 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:54.615 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:54.615 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:54.615 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:54.615 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:54.615 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:54.615 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49817 2024-09-04T17:47:54.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.616 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:54.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:54.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077603+0000", 2024-09-04T17:47:54.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:54.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:54.617 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077603+0000", 2024-09-04T17:47:54.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:54.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.617 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:54.617 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077604+0000", 2024-09-04T17:47:54.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7699999999999992e-07 2024-09-04T17:47:54.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.617 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:54.617 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077609+0000", 2024-09-04T17:47:54.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0830000000000003e-06 2024-09-04T17:47:54.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.618 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:54.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077630+0000", 2024-09-04T17:47:54.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0438999999999999e-05 2024-09-04T17:47:54.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.618 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:54.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077664+0000", 2024-09-04T17:47:54.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4579000000000002e-05 2024-09-04T17:47:54.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:54.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097193+0000", 2024-09-04T17:47:54.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019529174 2024-09-04T17:47:54.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:54.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097323+0000", 2024-09-04T17:47:54.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000129696 2024-09-04T17:47:54.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:54.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097472+0000", 2024-09-04T17:47:54.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014933200000000001 2024-09-04T17:47:54.619 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:54.620 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:54.620 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:54.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.620 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49818 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 688128~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:54.620 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077683+0000", 2024-09-04T17:47:54.620 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.457702548, 2024-09-04T17:47:54.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020086333000000001, 2024-09-04T17:47:54.620 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:54.620 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:54.620 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:54.620 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:54.621 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:54.621 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:54.621 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49818 2024-09-04T17:47:54.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.621 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:54.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:54.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077683+0000", 2024-09-04T17:47:54.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:54.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:54.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077683+0000", 2024-09-04T17:47:54.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:54.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.622 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:54.622 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077684+0000", 2024-09-04T17:47:54.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4e-07 2024-09-04T17:47:54.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.622 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:54.622 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077690+0000", 2024-09-04T17:47:54.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4729999999999999e-06 2024-09-04T17:47:54.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.623 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:54.623 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077710+0000", 2024-09-04T17:47:54.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0149999999999999e-05 2024-09-04T17:47:54.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.623 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:54.623 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077723+0000", 2024-09-04T17:47:54.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3271999999999999e-05 2024-09-04T17:47:54.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.623 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:54.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097521+0000", 2024-09-04T17:47:54.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019797981999999999 2024-09-04T17:47:54.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:54.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097628+0000", 2024-09-04T17:47:54.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010665700000000001 2024-09-04T17:47:54.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:54.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097770+0000", 2024-09-04T17:47:54.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014185900000000001 2024-09-04T17:47:54.624 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:54.625 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:54.625 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:54.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.625 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49820 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 753664~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:54.625 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077744+0000", 2024-09-04T17:47:54.625 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.457641617, 2024-09-04T17:47:54.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020364786999999999, 2024-09-04T17:47:54.626 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:54.626 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:54.626 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:54.626 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:54.626 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:54.626 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:54.626 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49820 2024-09-04T17:47:54.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.626 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:54.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:54.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077744+0000", 2024-09-04T17:47:54.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:54.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:54.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077744+0000", 2024-09-04T17:47:54.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:54.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:54.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077745+0000", 2024-09-04T17:47:54.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0270000000000001e-06 2024-09-04T17:47:54.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:54.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077751+0000", 2024-09-04T17:47:54.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.626e-06 2024-09-04T17:47:54.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:54.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077771+0000", 2024-09-04T17:47:54.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0469999999999999e-05 2024-09-04T17:47:54.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:54.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077785+0000", 2024-09-04T17:47:54.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3129000000000001e-05 2024-09-04T17:47:54.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:54.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097816+0000", 2024-09-04T17:47:54.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020031917999999999 2024-09-04T17:47:54.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:54.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097939+0000", 2024-09-04T17:47:54.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000122533 2024-09-04T17:47:54.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:54.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098109+0000", 2024-09-04T17:47:54.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017008400000000001 2024-09-04T17:47:54.631 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:54.631 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:54.631 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:54.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.631 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49821 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 819200~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:54.631 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077807+0000", 2024-09-04T17:47:54.631 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.457579404999997, 2024-09-04T17:47:54.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020702928999999998, 2024-09-04T17:47:54.631 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:54.631 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:54.631 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:54.631 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:54.632 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:54.632 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:54.632 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49821 2024-09-04T17:47:54.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.632 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:54.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:54.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077807+0000", 2024-09-04T17:47:54.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:54.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:54.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077807+0000", 2024-09-04T17:47:54.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:54.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:54.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077808+0000", 2024-09-04T17:47:54.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.463e-06 2024-09-04T17:47:54.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:54.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077813+0000", 2024-09-04T17:47:54.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7530000000000001e-06 2024-09-04T17:47:54.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:54.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077832+0000", 2024-09-04T17:47:54.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.946e-05 2024-09-04T17:47:54.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:54.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077846+0000", 2024-09-04T17:47:54.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3786e-05 2024-09-04T17:47:54.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:54.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098201+0000", 2024-09-04T17:47:54.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020354797000000001 2024-09-04T17:47:54.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:54.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098341+0000", 2024-09-04T17:47:54.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000140042 2024-09-04T17:47:54.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:54.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098509+0000", 2024-09-04T17:47:54.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016862800000000001 2024-09-04T17:47:54.635 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:54.635 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:54.636 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:54.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.636 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49823 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 884736~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:54.636 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077865+0000", 2024-09-04T17:47:54.636 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.457520961, 2024-09-04T17:47:54.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021008723, 2024-09-04T17:47:54.636 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:54.636 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:54.636 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:54.636 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:54.636 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:54.636 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:54.637 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49823 2024-09-04T17:47:54.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.637 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:54.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:54.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077865+0000", 2024-09-04T17:47:54.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:54.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:54.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077865+0000", 2024-09-04T17:47:54.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:54.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:54.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077866+0000", 2024-09-04T17:47:54.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.164e-06 2024-09-04T17:47:54.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:54.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077872+0000", 2024-09-04T17:47:54.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3959999999999997e-06 2024-09-04T17:47:54.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:54.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077892+0000", 2024-09-04T17:47:54.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0970000000000001e-05 2024-09-04T17:47:54.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:54.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077905+0000", 2024-09-04T17:47:54.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2312e-05 2024-09-04T17:47:54.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:54.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098579+0000", 2024-09-04T17:47:54.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020674486999999998 2024-09-04T17:47:54.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:54.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098705+0000", 2024-09-04T17:47:54.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012541900000000001 2024-09-04T17:47:54.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:54.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098874+0000", 2024-09-04T17:47:54.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000168975 2024-09-04T17:47:54.641 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:54.641 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:54.641 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:54.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.642 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49824 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 950272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:54.642 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077923+0000", 2024-09-04T17:47:54.642 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.457463269000002, 2024-09-04T17:47:54.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021246287999999999, 2024-09-04T17:47:54.642 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:54.642 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:54.642 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:54.642 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:54.642 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:54.642 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:54.642 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49824 2024-09-04T17:47:54.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.643 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:54.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:54.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077923+0000", 2024-09-04T17:47:54.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:54.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:54.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077923+0000", 2024-09-04T17:47:54.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:54.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:54.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077924+0000", 2024-09-04T17:47:54.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0100000000000001e-06 2024-09-04T17:47:54.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.644 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:54.644 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077944+0000", 2024-09-04T17:47:54.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0449e-05 2024-09-04T17:47:54.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.644 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:54.644 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077965+0000", 2024-09-04T17:47:54.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0479000000000002e-05 2024-09-04T17:47:54.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:54.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077990+0000", 2024-09-04T17:47:54.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5582999999999999e-05 2024-09-04T17:47:54.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:54.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098924+0000", 2024-09-04T17:47:54.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020933395 2024-09-04T17:47:54.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:54.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.099032+0000", 2024-09-04T17:47:54.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010846299999999999 2024-09-04T17:47:54.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:54.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.099169+0000", 2024-09-04T17:47:54.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000136909 2024-09-04T17:47:54.646 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:54.646 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:54.646 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:54.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.646 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49826 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 1015808~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:54.646 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.078010+0000", 2024-09-04T17:47:54.647 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.457375661999997, 2024-09-04T17:47:54.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021584564000000001, 2024-09-04T17:47:54.647 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:54.647 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:54.647 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:54.647 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:54.647 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:54.647 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:54.647 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49826 2024-09-04T17:47:54.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.647 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:54.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.648 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:54.648 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078010+0000", 2024-09-04T17:47:54.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:54.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.648 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:54.648 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078010+0000", 2024-09-04T17:47:54.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:54.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.649 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:54.649 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078011+0000", 2024-09-04T17:47:54.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1340000000000001e-06 2024-09-04T17:47:54.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.649 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:54.649 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078017+0000", 2024-09-04T17:47:54.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.682e-06 2024-09-04T17:47:54.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:54.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078036+0000", 2024-09-04T17:47:54.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9357000000000001e-05 2024-09-04T17:47:54.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:54.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078050+0000", 2024-09-04T17:47:54.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3122000000000001e-05 2024-09-04T17:47:54.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:54.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.099271+0000", 2024-09-04T17:47:54.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021221772999999999 2024-09-04T17:47:54.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:54.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.099410+0000", 2024-09-04T17:47:54.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000138415 2024-09-04T17:47:54.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:54.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:54.652 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:54.652 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.099595+0000", 2024-09-04T17:47:54.652 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000185081 2024-09-04T17:47:54.652 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:54.652 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:54.652 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:54.652 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:54.652 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:54.652 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:54.654 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-04T17:47:54.654 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #40, image image_client.0-clone40 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent41 2024-09-04T17:47:54.655 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:54.651+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:54.754 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:47:54.752+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:47:54.851 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:54.851 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:47:54.851 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:47:54.851 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:54.855 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:47:54.852+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:47:54.867 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-04T17:47:54.957 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:47:54.956+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:47:54.999 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-04T17:47:55.030 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-04T17:47:55.031 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:55.032 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:47:55.032 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:47:55.032 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:47:55.032 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:55.048 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-04T17:47:55.093 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:55.092+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:55.141 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:47:55.141 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:47:55.159 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:47:55.159 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:47:55.160 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:55.155+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:55.176 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-04T17:47:55.176 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-04T17:47:55.193 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-04T17:47:55.193 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-04T17:47:55.216 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:55.217 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:55.218 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:47:55.218 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:47:55.240 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-04T17:47:55.250 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:47:55.250 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:47:55.250 INFO:teuthology.orchestra.run.smithi092.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-04T17:47:55.260 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:55.256+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:55.305 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:47:55.305 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:47:55.309 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-04T17:47:55.309 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-04T17:47:55.362 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:55.359+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:55.373 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:55.373 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:55.395 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-04T17:47:55.463 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:55.459+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:55.473 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:47:55.471+0000 7f579621d7c0 -1 osd.6 58 log_to_monitors true 2024-09-04T17:47:55.517 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:55.517 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:55.531 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-04T17:47:55.563 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:47:55.559+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:47:55.664 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:55.660+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:55.684 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:55.684 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:55.698 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-04T17:47:55.704 INFO:teuthology.orchestra.run.smithi092.stdout:5754 punch from 0x4d7a5a1 to 0x4d84bfc, (0xa65b bytes) 2024-09-04T17:47:55.711 INFO:teuthology.orchestra.run.smithi092.stdout:5755 read 0x4391314 thru 0x4394c07 (0x38f4 bytes) 2024-09-04T17:47:55.712 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:55.715 INFO:teuthology.orchestra.run.smithi092.stdout:5757 trunc from 0x9a8f9f9 to 0x7a84186 2024-09-04T17:47:55.764 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:55.760+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:55.844 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:55.845 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:55.861 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-04T17:47:55.882 INFO:teuthology.orchestra.run.smithi092.stdout:5758 write 0x6fd14bc thru 0x6fdb326 (0x9e6b bytes) 2024-09-04T17:47:55.883 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:55.880+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:55.890 INFO:teuthology.orchestra.run.smithi092.stdout:5762 trunc from 0x7a84186 to 0x5a5964d 2024-09-04T17:47:55.904 INFO:teuthology.orchestra.run.smithi092.stdout:5764 read 0x1c6d6f4 thru 0x1c6f97b (0x2288 bytes) 2024-09-04T17:47:55.904 INFO:teuthology.orchestra.run.smithi092.stdout:5765 read 0xc8e653 thru 0xc918bb (0x3269 bytes) 2024-09-04T17:47:55.909 INFO:teuthology.orchestra.run.smithi092.stdout:5768 trunc from 0x5a5964d to 0x9182566 2024-09-04T17:47:55.919 INFO:teuthology.orchestra.run.smithi092.stdout:5771 write 0x2fa7bb7 thru 0x2fb67e9 (0xec33 bytes) 2024-09-04T17:47:55.927 INFO:teuthology.orchestra.run.smithi092.stdout:5772 trunc from 0x9182566 to 0xd3616a4 2024-09-04T17:47:55.936 INFO:teuthology.orchestra.run.smithi092.stdout:5773 read 0x81c9e2b thru 0x81d19ed (0x7bc3 bytes) 2024-09-04T17:47:55.948 INFO:teuthology.orchestra.run.smithi092.stdout:5775 trunc from 0xd3616a4 to 0x3b8e45e 2024-09-04T17:47:55.966 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:47:55.963+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:47:55.980 INFO:teuthology.orchestra.run.smithi092.stdout:5776 write 0x9338042 thru 0x933e18b (0x614a bytes) 2024-09-04T17:47:55.998 INFO:teuthology.orchestra.run.smithi092.stdout:5777 writesame 0xaa95077 thru 0xaaa4fa5 data_size 0x1(0xff2f bytes) 2024-09-04T17:47:56.000 INFO:teuthology.orchestra.run.smithi092.stdout:5779 punch from 0x3a473b3 to 0x3a4a80c, (0x3459 bytes) 2024-09-04T17:47:56.003 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:56.003 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:56.010 INFO:teuthology.orchestra.run.smithi092.stdout:5780 trunc from 0xaaa4fa6 to 0x2d6369a 2024-09-04T17:47:56.024 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-04T17:47:56.031 INFO:teuthology.orchestra.run.smithi092.stdout:5781 read 0x1d9c463 thru 0x1da8bcd (0xc76b bytes) 2024-09-04T17:47:56.031 INFO:teuthology.orchestra.run.smithi092.stdout:5782 read 0x15b54fa thru 0x15bf4ba (0x9fc1 bytes) 2024-09-04T17:47:56.043 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is set 2024-09-04T17:47:56.065 INFO:teuthology.orchestra.run.smithi092.stdout:5783 write 0xe81d90e thru 0xe81e520 (0xc13 bytes) 2024-09-04T17:47:56.067 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:56.064+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:56.068 INFO:teuthology.orchestra.run.smithi092.stdout:5784 write 0x7be5790 thru 0x7bf3de6 (0xe657 bytes) 2024-09-04T17:47:56.091 INFO:teuthology.orchestra.run.smithi092.stdout:5785 trunc from 0xe81e521 to 0x39a0c09 2024-09-04T17:47:56.096 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:47:56.091+0000 7f5789ff8640 -1 osd.6 116 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-04T17:47:56.116 INFO:teuthology.orchestra.run.smithi092.stdout:5786 write 0x4a1e1fa thru 0x4a2859d (0xa3a4 bytes) 2024-09-04T17:47:56.136 INFO:teuthology.orchestra.run.smithi092.stdout:5787 write 0x9be2c54 thru 0x9bf238a (0xf737 bytes) 2024-09-04T17:47:56.139 INFO:teuthology.orchestra.run.smithi092.stdout:5789 punch from 0x71c84de to 0x71cbd73, (0x3895 bytes) 2024-09-04T17:47:56.139 INFO:teuthology.orchestra.run.smithi092.stdout:5790 punch from 0x430cec5 to 0x430efbb, (0x20f6 bytes) 2024-09-04T17:47:56.161 INFO:teuthology.orchestra.run.smithi092.stdout:5791 writesame 0xed48e81 thru 0xed5609b data_size 0x1(0xd21b bytes) 2024-09-04T17:47:56.161 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:56.161 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:56.163 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:56.163 INFO:teuthology.orchestra.run.smithi092.stdout:5793 writesame 0x1e26bdb thru 0x1e337e0 data_size 0xf(0xcc06 bytes) 2024-09-04T17:47:56.167 INFO:teuthology.orchestra.run.smithi092.stdout:5795 read 0xec68bcf thru 0xec74b46 (0xbf78 bytes) 2024-09-04T17:47:56.167 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:47:56.164+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:47:56.167 INFO:teuthology.orchestra.run.smithi092.stdout:5796 writesame 0x7c85e76 thru 0x7c95dca data_size 0x11(0xff55 bytes) 2024-09-04T17:47:56.177 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-04T17:47:56.192 INFO:teuthology.orchestra.run.smithi092.stdout:5798 trunc from 0xed5609c to 0x6fbbcef 2024-09-04T17:47:56.217 INFO:teuthology.orchestra.run.smithi092.stdout:5799 trunc from 0x6fbbcef to 0x128ecb0 2024-09-04T17:47:56.269 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:56.263+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:56.279 INFO:teuthology.orchestra.run.smithi092.stdout:5801 write 0xd04e2eb thru 0xd056c44 (0x895a bytes) 2024-09-04T17:47:56.282 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:56.282 INFO:teuthology.orchestra.run.smithi092.stdout:5804 punch from 0x7820b0f to 0x782ba4d, (0xaf3e bytes) 2024-09-04T17:47:56.296 INFO:teuthology.orchestra.run.smithi092.stdout:5805 trunc from 0xd056c45 to 0x7afe2be 2024-09-04T17:47:56.312 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:56.313 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:56.326 INFO:teuthology.orchestra.run.smithi092.stdout:5806 writesame 0x7c0cc73 thru 0x7c0d5bb data_size 0x1(0x949 bytes) 2024-09-04T17:47:56.328 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-04T17:47:56.348 INFO:teuthology.orchestra.run.smithi092.stdout:5807 write 0xaa2424b thru 0xaa324eb (0xe2a1 bytes) 2024-09-04T17:47:56.354 INFO:teuthology.orchestra.run.smithi092.stdout:5808 punch from 0x1e193fb to 0x1e1e598, (0x519d bytes) 2024-09-04T17:47:56.370 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:56.367+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:56.370 INFO:teuthology.orchestra.run.smithi092.stdout:5809 trunc from 0xaa324ec to 0x4de9085 2024-09-04T17:47:56.398 INFO:teuthology.orchestra.run.smithi092.stdout:5810 writesame 0x65f52e6 thru 0x65f67c8 data_size 0x1(0x14e3 bytes) 2024-09-04T17:47:56.398 INFO:teuthology.orchestra.run.smithi092.stdout:5811 writesame 0x126f6f3 thru 0x126fe5e data_size 0x19(0x76c bytes) 2024-09-04T17:47:56.401 INFO:teuthology.orchestra.run.smithi092.stdout:5812 read 0xbc108a thru 0xbc9c04 (0x8b7b bytes) 2024-09-04T17:47:56.402 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:56.414 INFO:teuthology.orchestra.run.smithi092.stdout:5814 writesame 0x72469f2 thru 0x724eec3 data_size 0x9(0x84d2 bytes) 2024-09-04T17:47:56.440 INFO:teuthology.orchestra.run.smithi092.stdout:5815 writesame 0xb445982 thru 0xb44d19a data_size 0x37(0x7819 bytes) 2024-09-04T17:47:56.440 INFO:teuthology.orchestra.run.smithi092.stdout:5816 read 0x674e438 thru 0x675e22b (0xfdf4 bytes) 2024-09-04T17:47:56.455 INFO:teuthology.orchestra.run.smithi092.stdout:5817 trunc from 0xb44d19b to 0x572960f 2024-09-04T17:47:56.470 INFO:teuthology.orchestra.run.smithi092.stdout:5818 read 0x40206 thru 0x4953f (0x933a bytes) 2024-09-04T17:47:56.470 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:56.467+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:56.471 INFO:teuthology.orchestra.run.smithi092.stdout:5821 read 0x32339f7 thru 0x3234e0b (0x1415 bytes) 2024-09-04T17:47:56.471 INFO:teuthology.orchestra.run.smithi092.stdout:5822 trunc from 0x572960f to 0x561c507 2024-09-04T17:47:56.489 INFO:teuthology.orchestra.run.smithi092.stdout:5825 trunc from 0x561c507 to 0x644a19d 2024-09-04T17:47:56.501 INFO:teuthology.orchestra.run.smithi092.stdout:5826 read 0x9da380 thru 0x9e158a (0x720b bytes) 2024-09-04T17:47:56.507 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:56.507 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:56.522 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-04T17:47:56.522 INFO:teuthology.orchestra.run.smithi092.stdout:5827 trunc from 0x644a19d to 0xd43ba52 2024-09-04T17:47:56.533 INFO:teuthology.orchestra.run.smithi092.stdout:5828 writesame 0x4677bc3 thru 0x46811ce data_size 0x2c(0x960c bytes) 2024-09-04T17:47:56.537 INFO:teuthology.orchestra.run.smithi092.stdout:5829 read 0x601351b thru 0x60178ce (0x43b4 bytes) 2024-09-04T17:47:56.537 INFO:teuthology.orchestra.run.smithi092.stdout:5830 write 0x98e51f5 thru 0x98ea3f5 (0x5201 bytes) 2024-09-04T17:47:56.543 INFO:teuthology.orchestra.run.smithi092.stdout:5834 write 0x1e66415 thru 0x1e6bdc9 (0x59b5 bytes) 2024-09-04T17:47:56.546 INFO:teuthology.orchestra.run.smithi092.stdout:5835 trunc from 0xd43ba52 to 0xd74b201 2024-09-04T17:47:56.559 INFO:teuthology.orchestra.run.smithi092.stdout:5836 write 0x99adb8d thru 0x99b0a1e (0x2e92 bytes) 2024-09-04T17:47:56.561 INFO:teuthology.orchestra.run.smithi092.stdout:5837 writesame 0xad5511 thru 0xae05f0 data_size 0x50(0xb0e0 bytes) 2024-09-04T17:47:56.566 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:56.566 INFO:teuthology.orchestra.run.smithi092.stdout:5839 read 0x5e6d111 thru 0x5e761f3 (0x90e3 bytes) 2024-09-04T17:47:56.567 INFO:teuthology.orchestra.run.smithi092.stdout:5841 read 0x9c15933 thru 0x9c21a52 (0xc120 bytes) 2024-09-04T17:47:56.568 INFO:teuthology.orchestra.run.smithi092.stdout:5843 read 0xc244172 thru 0xc252d0e (0xeb9d bytes) 2024-09-04T17:47:56.568 INFO:teuthology.orchestra.run.smithi092.stdout:5844 writesame 0x767db41 thru 0x768a46c data_size 0xa(0xc92c bytes) 2024-09-04T17:47:56.570 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:47:56.568+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:47:56.579 INFO:teuthology.orchestra.run.smithi092.stdout:5845 punch from 0xd30b11 to 0xd368fe, (0x5ded bytes) 2024-09-04T17:47:56.580 INFO:teuthology.orchestra.run.smithi092.stdout:5846 punch from 0x3bb7f1 to 0x3bb94f, (0x15e bytes) 2024-09-04T17:47:56.580 INFO:teuthology.orchestra.run.smithi092.stdout:5847 writesame 0x5a3f028 thru 0x5a48c3c data_size 0x13(0x9c15 bytes) 2024-09-04T17:47:56.584 INFO:teuthology.orchestra.run.smithi092.stdout:5848 writesame 0x5ae81aa thru 0x5af695f data_size 0x26(0xe7b6 bytes) 2024-09-04T17:47:56.588 INFO:teuthology.orchestra.run.smithi092.stdout:5849 read 0xb0ad0c3 thru 0xb0bb9ac (0xe8ea bytes) 2024-09-04T17:47:56.589 INFO:teuthology.orchestra.run.smithi092.stdout:5850 write 0x619a2f5 thru 0x619fb7b (0x5887 bytes) 2024-09-04T17:47:56.595 INFO:teuthology.orchestra.run.smithi092.stdout:5851 write 0x19499e1 thru 0x1949bd8 (0x1f8 bytes) 2024-09-04T17:47:56.603 INFO:teuthology.orchestra.run.smithi092.stdout:5852 trunc from 0xd74b201 to 0xb70dc12 2024-09-04T17:47:56.618 INFO:teuthology.orchestra.run.smithi092.stdout:5854 read 0x2a649f7 thru 0x2a67d92 (0x339c bytes) 2024-09-04T17:47:56.619 INFO:teuthology.orchestra.run.smithi092.stdout:5855 read 0x7b4137c thru 0x7b4e5c8 (0xd24d bytes) 2024-09-04T17:47:56.622 INFO:teuthology.orchestra.run.smithi092.stdout:5856 trunc from 0xb70dc12 to 0xa6a26fd 2024-09-04T17:47:56.638 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:56.638 INFO:teuthology.orchestra.run.smithi092.stdout:5859 read 0x1dacea5 thru 0x1dba376 (0xd4d2 bytes) 2024-09-04T17:47:56.639 INFO:teuthology.orchestra.run.smithi092.stdout:5860 punch from 0x37d9ef1 to 0x37e5b6a, (0xbc79 bytes) 2024-09-04T17:47:56.639 INFO:teuthology.orchestra.run.smithi092.stdout:5861 read 0x2c2b315 thru 0x2c32bf9 (0x78e5 bytes) 2024-09-04T17:47:56.640 INFO:teuthology.orchestra.run.smithi092.stdout:5862 read 0x61a20d5 thru 0x61ac252 (0xa17e bytes) 2024-09-04T17:47:56.641 INFO:teuthology.orchestra.run.smithi092.stdout:5863 write 0x5c3d954 thru 0x5c49f9f (0xc64c bytes) 2024-09-04T17:47:56.666 INFO:teuthology.orchestra.run.smithi092.stdout:5864 write 0xdc0ede7 thru 0xdc18024 (0x923e bytes) 2024-09-04T17:47:56.671 INFO:teuthology.orchestra.run.smithi092.stdout:5865 flatten 2024-09-04T17:47:56.671 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:47:56.668+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:47:56.689 INFO:teuthology.orchestra.run.smithi092.stdout:5866 read 0xd1d0bda thru 0xd1d7162 (0x6589 bytes) 2024-09-04T17:47:56.694 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:56.694 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:56.708 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-04T17:47:56.710 INFO:teuthology.orchestra.run.smithi092.stdout:5869 trunc from 0xdc18025 to 0x663995b 2024-09-04T17:47:56.728 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:56.728 INFO:teuthology.orchestra.run.smithi092.stdout:5871 read 0x37fc554 thru 0x38046f4 (0x81a1 bytes) 2024-09-04T17:47:56.754 INFO:teuthology.orchestra.run.smithi092.stdout:5872 write 0xbafa58c thru 0xbb04edf (0xa954 bytes) 2024-09-04T17:47:56.759 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:56.759 INFO:teuthology.orchestra.run.smithi092.stdout:5876 write 0x6d056a5 thru 0x6d11ac8 (0xc424 bytes) 2024-09-04T17:47:56.762 INFO:teuthology.orchestra.run.smithi092.stdout:5877 writesame 0xacdfe04 thru 0xace43b6 data_size 0x1(0x45b3 bytes) 2024-09-04T17:47:56.772 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:56.767+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:56.788 INFO:teuthology.orchestra.run.smithi092.stdout:5878 read 0x6a9cfca thru 0x6a9fb32 (0x2b69 bytes) 2024-09-04T17:47:56.808 INFO:teuthology.orchestra.run.smithi092.stdout:5879 write 0xc977f99 thru 0xc97b80d (0x3875 bytes) 2024-09-04T17:47:56.811 INFO:teuthology.orchestra.run.smithi092.stdout:5882 read 0x46a7deb thru 0x46b186d (0x9a83 bytes) 2024-09-04T17:47:56.812 INFO:teuthology.orchestra.run.smithi092.stdout:5884 read 0x3f7b2e6 thru 0x3f8834d (0xd068 bytes) 2024-09-04T17:47:56.814 INFO:teuthology.orchestra.run.smithi092.stdout:5886 read 0xd96467 thru 0xda2e08 (0xc9a2 bytes) 2024-09-04T17:47:56.815 INFO:teuthology.orchestra.run.smithi092.stdout:5888 read 0x71e81cc thru 0x71f3487 (0xb2bc bytes) 2024-09-04T17:47:56.817 INFO:teuthology.orchestra.run.smithi092.stdout:5889 read 0xc04ce4 thru 0xc14b4b (0xfe68 bytes) 2024-09-04T17:47:56.829 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:56.829 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:56.837 INFO:teuthology.orchestra.run.smithi092.stdout:5890 write 0xe7dd312 thru 0xe7e1264 (0x3f53 bytes) 2024-09-04T17:47:56.841 INFO:teuthology.orchestra.run.smithi092.stdout:5891 punch from 0xd68eeb8 to 0xd69ea8c, (0xfbd4 bytes) 2024-09-04T17:47:56.842 INFO:teuthology.orchestra.run.smithi092.stdout:5892 write 0x9a61abb thru 0x9a6cf5c (0xb4a2 bytes) 2024-09-04T17:47:56.844 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-04T17:47:56.845 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:56.845 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:56.845 INFO:teuthology.orchestra.run.smithi092.stdout:5897 read 0x2deea8f thru 0x2df0761 (0x1cd3 bytes) 2024-09-04T17:47:56.846 INFO:teuthology.orchestra.run.smithi092.stdout:5898 read 0xab2088d thru 0xab238b8 (0x302c bytes) 2024-09-04T17:47:56.847 INFO:teuthology.orchestra.run.smithi092.stdout:5899 punch from 0x8731edf to 0x8734ae6, (0x2c07 bytes) 2024-09-04T17:47:56.848 INFO:teuthology.orchestra.run.smithi092.stdout:5900 trunc from 0xe7e1265 to 0xebfb213 2024-09-04T17:47:56.860 INFO:teuthology.orchestra.run.smithi092.stdout:5901 read 0xebe60d1 thru 0xebe9b72 (0x3aa2 bytes) 2024-09-04T17:47:56.861 INFO:teuthology.orchestra.run.smithi092.stdout:5902 writesame 0x3a45b3b thru 0x3a4f75b data_size 0x3(0x9c21 bytes) 2024-09-04T17:47:56.885 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:56.883+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:56.886 INFO:teuthology.orchestra.run.smithi092.stdout:5903 trunc from 0xebfb213 to 0x8ed4303 2024-09-04T17:47:56.900 INFO:teuthology.orchestra.run.smithi092.stdout:5904 clone 43 order 19 su 65536 sc 2 2024-09-04T17:47:56.974 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:47:56.972+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:47:57.002 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:57.003 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:57.016 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-04T17:47:57.075 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:57.072+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:57.144 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:57.145 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:57.158 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-04T17:47:57.205 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:57.203+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:57.277 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:57.272+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:57.280 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:57.281 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:57.295 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-04T17:47:57.311 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-04T17:47:57.377 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:57.376+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:57.465 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:57.465 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:57.469 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:47:57.469 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:47:57.477 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:57.476+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:57.479 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-04T17:47:57.485 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:47:57.485 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:47:57.501 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-04T17:47:57.501 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-04T17:47:57.518 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-04T17:47:57.518 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-04T17:47:57.535 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:47:57.535 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:47:57.553 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:47:57.553 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:47:57.573 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-04T17:47:57.573 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_enable_op_tracker = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' 2024-09-04T17:47:57.578 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:47:57.576+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:47:57.593 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:47:57.593 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:47:57.600 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:57.600 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:47:57.600 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:47:57.600 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:47:57.600 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.600 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60921 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 262144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:57.600 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822920+0000", 2024-09-04T17:47:57.600 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 43.775524218999998, 2024-09-04T17:47:57.600 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073904394999999998, 2024-09-04T17:47:57.600 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:57.601 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:57.601 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:57.601 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:57.601 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:57.601 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:57.601 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60921 2024-09-04T17:47:57.601 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.601 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:57.601 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.601 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:57.601 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822920+0000", 2024-09-04T17:47:57.601 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:57.601 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.602 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.602 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:57.602 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822920+0000", 2024-09-04T17:47:57.602 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:57.602 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.602 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.602 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:57.602 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822920+0000", 2024-09-04T17:47:57.602 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4999999999999999e-07 2024-09-04T17:47:57.602 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.602 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.602 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:57.602 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822921+0000", 2024-09-04T17:47:57.602 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.113e-06 2024-09-04T17:47:57.603 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.603 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.603 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:57.603 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822924+0000", 2024-09-04T17:47:57.603 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.354e-06 2024-09-04T17:47:57.603 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.603 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.603 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:57.603 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822927+0000", 2024-09-04T17:47:57.603 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6199999999999999e-06 2024-09-04T17:47:57.603 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.603 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.603 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:57.603 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896586+0000", 2024-09-04T17:47:57.604 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073658976000000001 2024-09-04T17:47:57.604 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.604 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.604 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:57.604 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896626+0000", 2024-09-04T17:47:57.604 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0484000000000001e-05 2024-09-04T17:47:57.604 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.604 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.604 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:57.604 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896824+0000", 2024-09-04T17:47:57.604 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000197598 2024-09-04T17:47:57.604 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:57.604 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:57.604 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:57.605 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.605 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.605 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60922 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:57.605 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822930+0000", 2024-09-04T17:47:57.605 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 43.775513644, 2024-09-04T17:47:57.605 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074178872000000007, 2024-09-04T17:47:57.605 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:57.605 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:57.605 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:57.605 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:57.605 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:57.605 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:57.605 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60922 2024-09-04T17:47:57.606 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.606 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:57.606 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.606 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:57.606 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822930+0000", 2024-09-04T17:47:57.606 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:57.606 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.606 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.606 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:57.606 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822930+0000", 2024-09-04T17:47:57.606 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:57.606 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.606 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.606 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:57.606 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822930+0000", 2024-09-04T17:47:57.607 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.36e-07 2024-09-04T17:47:57.607 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.607 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.607 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:57.607 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822932+0000", 2024-09-04T17:47:57.607 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.567e-06 2024-09-04T17:47:57.607 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.607 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.607 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:57.607 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822938+0000", 2024-09-04T17:47:57.607 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4840000000000003e-06 2024-09-04T17:47:57.607 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.608 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.608 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:57.608 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822942+0000", 2024-09-04T17:47:57.608 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2039999999999999e-06 2024-09-04T17:47:57.608 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.608 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.608 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:57.608 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896850+0000", 2024-09-04T17:47:57.608 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073907941000000005 2024-09-04T17:47:57.608 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.609 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.609 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:57.609 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896891+0000", 2024-09-04T17:47:57.609 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0964999999999998e-05 2024-09-04T17:47:57.609 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.609 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.609 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:57.609 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897109+0000", 2024-09-04T17:47:57.609 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021847500000000001 2024-09-04T17:47:57.609 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:57.610 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:57.610 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:57.610 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.610 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.610 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60923 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 475136~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:57.610 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822949+0000", 2024-09-04T17:47:57.610 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 43.775495354, 2024-09-04T17:47:57.610 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074426928000000003, 2024-09-04T17:47:57.610 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:57.610 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:57.610 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:57.611 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:57.611 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:57.611 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:57.611 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60923 2024-09-04T17:47:57.611 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.611 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:57.611 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.611 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:57.611 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822949+0000", 2024-09-04T17:47:57.611 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:57.611 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.612 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.612 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:57.612 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822949+0000", 2024-09-04T17:47:57.612 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:57.612 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.612 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.612 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:57.612 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822949+0000", 2024-09-04T17:47:57.612 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7399999999999999e-07 2024-09-04T17:47:57.612 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.612 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.613 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:57.613 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822951+0000", 2024-09-04T17:47:57.613 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1940000000000001e-06 2024-09-04T17:47:57.613 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.613 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.613 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:57.613 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822957+0000", 2024-09-04T17:47:57.613 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.0650000000000004e-06 2024-09-04T17:47:57.613 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.613 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.614 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:57.614 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822960+0000", 2024-09-04T17:47:57.614 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.8009999999999999e-06 2024-09-04T17:47:57.614 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.614 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.614 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:57.614 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897138+0000", 2024-09-04T17:47:57.614 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074177990999999999 2024-09-04T17:47:57.614 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.614 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.614 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:57.614 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897179+0000", 2024-09-04T17:47:57.615 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.1057999999999999e-05 2024-09-04T17:47:57.615 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.615 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.615 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:57.615 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897375+0000", 2024-09-04T17:47:57.615 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019644500000000001 2024-09-04T17:47:57.615 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:57.615 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:57.615 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:57.615 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.615 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.615 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60924 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:57.615 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822964+0000", 2024-09-04T17:47:57.616 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 43.775480168999998, 2024-09-04T17:47:57.616 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074531902999999997, 2024-09-04T17:47:57.616 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:57.616 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:57.616 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:57.616 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:57.616 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:57.616 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:57.616 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60924 2024-09-04T17:47:57.616 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.616 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:57.616 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.617 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:57.617 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822964+0000", 2024-09-04T17:47:57.617 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:57.617 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.617 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.617 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:57.617 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822964+0000", 2024-09-04T17:47:57.617 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:57.617 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.617 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.617 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:57.617 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822964+0000", 2024-09-04T17:47:57.618 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4999999999999999e-07 2024-09-04T17:47:57.618 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.618 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.618 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:57.618 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822966+0000", 2024-09-04T17:47:57.618 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.917e-06 2024-09-04T17:47:57.618 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.618 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.618 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:57.618 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822969+0000", 2024-09-04T17:47:57.618 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.315e-06 2024-09-04T17:47:57.618 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.619 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.619 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:57.619 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822972+0000", 2024-09-04T17:47:57.619 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.599e-06 2024-09-04T17:47:57.619 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.619 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.619 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:57.619 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897398+0000", 2024-09-04T17:47:57.619 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074426168000000001 2024-09-04T17:47:57.619 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.619 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.619 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:57.619 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897441+0000", 2024-09-04T17:47:57.620 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.3158000000000003e-05 2024-09-04T17:47:57.620 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.620 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.620 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:57.620 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897496+0000", 2024-09-04T17:47:57.620 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4496000000000001e-05 2024-09-04T17:47:57.620 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:57.620 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:57.620 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:57.620 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.620 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.620 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60925 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 720896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:57.621 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822975+0000", 2024-09-04T17:47:57.621 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 43.775468734, 2024-09-04T17:47:57.621 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074635272000000003, 2024-09-04T17:47:57.621 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:57.621 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:57.621 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:57.621 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:57.621 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:57.621 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:57.621 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60925 2024-09-04T17:47:57.621 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.622 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:57.622 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.622 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:57.622 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822975+0000", 2024-09-04T17:47:57.622 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:57.622 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.622 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.622 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:57.622 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822975+0000", 2024-09-04T17:47:57.622 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:57.622 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.622 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.622 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:57.623 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822975+0000", 2024-09-04T17:47:57.623 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.34e-07 2024-09-04T17:47:57.623 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.623 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.623 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:57.623 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822976+0000", 2024-09-04T17:47:57.623 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0359999999999999e-06 2024-09-04T17:47:57.623 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.623 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.623 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:57.623 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822980+0000", 2024-09-04T17:47:57.623 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.242e-06 2024-09-04T17:47:57.624 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.624 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.624 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:57.624 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822983+0000", 2024-09-04T17:47:57.624 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.8830000000000002e-06 2024-09-04T17:47:57.624 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.624 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.624 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:57.624 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897519+0000", 2024-09-04T17:47:57.624 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074536366000000007 2024-09-04T17:47:57.624 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.625 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.625 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:57.625 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897560+0000", 2024-09-04T17:47:57.625 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0677999999999999e-05 2024-09-04T17:47:57.625 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.625 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.625 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:57.625 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897610+0000", 2024-09-04T17:47:57.625 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0832999999999999e-05 2024-09-04T17:47:57.625 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:57.625 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:57.625 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:57.626 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.626 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.626 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60926 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 860160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:57.626 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822986+0000", 2024-09-04T17:47:57.626 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 43.775458135999997, 2024-09-04T17:47:57.626 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074736413000000002, 2024-09-04T17:47:57.626 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:57.626 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:57.626 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:57.626 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:57.626 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:57.626 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:57.627 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60926 2024-09-04T17:47:57.627 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.627 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:57.627 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.627 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:57.627 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822986+0000", 2024-09-04T17:47:57.627 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:57.627 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.627 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.627 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:57.627 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822986+0000", 2024-09-04T17:47:57.627 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:57.628 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.628 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.628 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:57.628 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822986+0000", 2024-09-04T17:47:57.628 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5600000000000002e-07 2024-09-04T17:47:57.628 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.628 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.628 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:57.628 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822987+0000", 2024-09-04T17:47:57.628 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.082e-06 2024-09-04T17:47:57.628 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.628 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.629 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:57.629 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822990+0000", 2024-09-04T17:47:57.629 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.2219999999999998e-06 2024-09-04T17:47:57.629 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.629 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.629 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:57.629 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822994+0000", 2024-09-04T17:47:57.629 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.19e-06 2024-09-04T17:47:57.629 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.629 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.630 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:57.630 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897632+0000", 2024-09-04T17:47:57.630 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074638205999999999 2024-09-04T17:47:57.630 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.630 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.630 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:57.630 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897672+0000", 2024-09-04T17:47:57.630 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.9873999999999998e-05 2024-09-04T17:47:57.630 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.630 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.630 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:57.630 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897722+0000", 2024-09-04T17:47:57.631 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0583e-05 2024-09-04T17:47:57.631 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:57.631 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:57.631 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:57.631 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.631 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.631 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60927 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 966656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:57.631 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823000+0000", 2024-09-04T17:47:57.631 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 43.775443953999996, 2024-09-04T17:47:57.631 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074834036000000007, 2024-09-04T17:47:57.631 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:57.632 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:57.632 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:57.632 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:57.632 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:57.632 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:57.632 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60927 2024-09-04T17:47:57.632 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.632 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:57.632 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.632 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:57.632 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823000+0000", 2024-09-04T17:47:57.632 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:57.633 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.633 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.633 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:57.633 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823000+0000", 2024-09-04T17:47:57.633 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:57.633 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.633 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.633 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:57.633 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823000+0000", 2024-09-04T17:47:57.633 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4099999999999999e-07 2024-09-04T17:47:57.634 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.634 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.634 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:57.634 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823003+0000", 2024-09-04T17:47:57.634 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0810000000000001e-06 2024-09-04T17:47:57.634 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.634 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.634 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:57.634 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823011+0000", 2024-09-04T17:47:57.634 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7090000000000008e-06 2024-09-04T17:47:57.634 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.634 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.635 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:57.635 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823017+0000", 2024-09-04T17:47:57.635 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.3890000000000004e-06 2024-09-04T17:47:57.635 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.635 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.635 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:57.635 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897745+0000", 2024-09-04T17:47:57.635 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074728252999999994 2024-09-04T17:47:57.635 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.635 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.635 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:57.636 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897785+0000", 2024-09-04T17:47:57.636 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.9830999999999999e-05 2024-09-04T17:47:57.636 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.636 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.636 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:57.636 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897834+0000", 2024-09-04T17:47:57.636 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9231999999999999e-05 2024-09-04T17:47:57.636 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:57.636 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:57.636 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:57.636 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.637 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.637 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60928 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:57.637 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823024+0000", 2024-09-04T17:47:57.637 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 43.775420023000002, 2024-09-04T17:47:57.637 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075067479000000006, 2024-09-04T17:47:57.637 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:57.637 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:57.637 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:57.637 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:57.637 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:57.637 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:57.638 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60928 2024-09-04T17:47:57.638 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.638 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:57.638 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.638 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:57.638 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823024+0000", 2024-09-04T17:47:57.638 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:57.638 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.638 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.638 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:57.638 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823024+0000", 2024-09-04T17:47:57.638 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:57.639 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.639 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.639 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:57.639 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823024+0000", 2024-09-04T17:47:57.639 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6499999999999999e-07 2024-09-04T17:47:57.639 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.639 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.639 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:57.639 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823027+0000", 2024-09-04T17:47:57.639 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.3099999999999999e-06 2024-09-04T17:47:57.639 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.640 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.640 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:57.640 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823033+0000", 2024-09-04T17:47:57.640 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7429999999999996e-06 2024-09-04T17:47:57.640 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.640 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.640 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:57.640 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823038+0000", 2024-09-04T17:47:57.640 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.561e-06 2024-09-04T17:47:57.640 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.641 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.641 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:57.641 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897855+0000", 2024-09-04T17:47:57.641 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074816740000000007 2024-09-04T17:47:57.641 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.641 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.641 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:57.641 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897894+0000", 2024-09-04T17:47:57.641 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.9085999999999998e-05 2024-09-04T17:47:57.641 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.641 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.642 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:57.642 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898091+0000", 2024-09-04T17:47:57.642 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000197574 2024-09-04T17:47:57.642 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:57.642 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:57.642 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:57.642 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.642 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.642 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60929 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1212416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:57.642 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823042+0000", 2024-09-04T17:47:57.642 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 43.775401580999997, 2024-09-04T17:47:57.643 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075301251, 2024-09-04T17:47:57.643 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:57.643 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:57.643 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:57.643 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:57.643 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:57.643 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:57.643 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60929 2024-09-04T17:47:57.643 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.643 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:57.644 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.644 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:57.644 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823042+0000", 2024-09-04T17:47:57.644 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:57.644 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.644 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.644 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:57.644 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823042+0000", 2024-09-04T17:47:57.644 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:57.644 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.644 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.645 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:57.645 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823043+0000", 2024-09-04T17:47:57.645 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.1100000000000002e-07 2024-09-04T17:47:57.645 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.645 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.645 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:57.645 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823045+0000", 2024-09-04T17:47:57.645 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.933e-06 2024-09-04T17:47:57.645 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.645 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.645 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:57.646 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823051+0000", 2024-09-04T17:47:57.646 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.9569999999999999e-06 2024-09-04T17:47:57.646 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.646 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.646 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:57.646 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823055+0000", 2024-09-04T17:47:57.646 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4479999999999996e-06 2024-09-04T17:47:57.646 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.646 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.646 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:57.646 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898117+0000", 2024-09-04T17:47:57.647 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075062344000000003 2024-09-04T17:47:57.647 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.647 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.647 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:57.647 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898157+0000", 2024-09-04T17:47:57.647 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0095999999999997e-05 2024-09-04T17:47:57.647 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.647 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.647 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:57.647 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898344+0000", 2024-09-04T17:47:57.648 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000186162 2024-09-04T17:47:57.648 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:57.648 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:57.648 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:57.648 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.648 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.648 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60930 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1351680~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:57.648 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823061+0000", 2024-09-04T17:47:57.648 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 43.775382702000002, 2024-09-04T17:47:57.649 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075545507999999997, 2024-09-04T17:47:57.649 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:57.649 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:57.649 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:57.649 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:57.649 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:57.649 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:57.649 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60930 2024-09-04T17:47:57.649 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.649 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:57.649 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.650 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:57.650 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823061+0000", 2024-09-04T17:47:57.650 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:57.650 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.650 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.650 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:57.650 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823061+0000", 2024-09-04T17:47:57.650 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:57.650 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.650 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.651 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:57.651 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823378+0000", 2024-09-04T17:47:57.651 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00031631299999999998 2024-09-04T17:47:57.651 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.651 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.651 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:57.651 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823381+0000", 2024-09-04T17:47:57.651 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4020000000000002e-06 2024-09-04T17:47:57.651 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.651 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.651 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:57.652 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823389+0000", 2024-09-04T17:47:57.652 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.1429999999999998e-06 2024-09-04T17:47:57.652 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.652 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.652 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:57.652 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823395+0000", 2024-09-04T17:47:57.652 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.0059999999999998e-06 2024-09-04T17:47:57.652 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.652 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.652 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:57.653 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898368+0000", 2024-09-04T17:47:57.653 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074973032999999994 2024-09-04T17:47:57.653 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.653 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.653 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:57.653 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898410+0000", 2024-09-04T17:47:57.653 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2243000000000002e-05 2024-09-04T17:47:57.653 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.653 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.653 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:57.653 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898607+0000", 2024-09-04T17:47:57.654 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019636799999999999 2024-09-04T17:47:57.654 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:57.654 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:57.654 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:57.654 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.654 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.654 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60931 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1458176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:57.654 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823407+0000", 2024-09-04T17:47:57.654 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 43.775036628000002, 2024-09-04T17:47:57.654 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075460334000000004, 2024-09-04T17:47:57.654 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:57.655 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:57.655 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:57.655 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:57.655 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:57.655 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:57.655 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60931 2024-09-04T17:47:57.655 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.655 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:57.655 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.655 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:57.655 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823407+0000", 2024-09-04T17:47:57.656 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:57.656 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.656 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.656 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:57.656 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823407+0000", 2024-09-04T17:47:57.656 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:57.656 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.656 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.656 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:57.656 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823408+0000", 2024-09-04T17:47:57.657 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9999999999999998e-07 2024-09-04T17:47:57.657 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.657 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.657 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:57.657 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823415+0000", 2024-09-04T17:47:57.657 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.5510000000000002e-06 2024-09-04T17:47:57.657 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.657 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.657 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:57.657 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823423+0000", 2024-09-04T17:47:57.657 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2760000000000003e-06 2024-09-04T17:47:57.657 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.658 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.658 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:57.658 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823426+0000", 2024-09-04T17:47:57.658 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.2839999999999998e-06 2024-09-04T17:47:57.658 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.658 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.658 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:57.658 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898631+0000", 2024-09-04T17:47:57.658 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075205045999999998 2024-09-04T17:47:57.658 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.658 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.658 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:57.659 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898672+0000", 2024-09-04T17:47:57.659 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0614000000000001e-05 2024-09-04T17:47:57.659 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.659 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.659 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:57.659 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898868+0000", 2024-09-04T17:47:57.659 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019606300000000001 2024-09-04T17:47:57.659 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:57.659 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:57.659 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:57.660 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.660 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.660 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60932 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1564672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:57.660 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823434+0000", 2024-09-04T17:47:57.660 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 43.775010328999997, 2024-09-04T17:47:57.660 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075693393999999997, 2024-09-04T17:47:57.660 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:57.660 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:57.660 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:57.660 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:57.660 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:57.660 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:57.661 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60932 2024-09-04T17:47:57.661 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.661 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:57.661 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.661 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:57.661 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823434+0000", 2024-09-04T17:47:57.661 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:57.661 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.661 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.661 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:57.661 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823434+0000", 2024-09-04T17:47:57.662 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:57.662 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.662 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.662 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:57.662 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.830290+0000", 2024-09-04T17:47:57.662 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0068563039999999997 2024-09-04T17:47:57.662 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.662 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.662 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:57.662 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.830297+0000", 2024-09-04T17:47:57.662 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4760000000000001e-06 2024-09-04T17:47:57.662 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.663 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.663 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:57.663 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.830308+0000", 2024-09-04T17:47:57.663 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0927999999999999e-05 2024-09-04T17:47:57.663 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.663 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.663 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:57.663 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.830317+0000", 2024-09-04T17:47:57.663 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7280000000000001e-06 2024-09-04T17:47:57.663 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.663 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.664 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:57.664 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898892+0000", 2024-09-04T17:47:57.664 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.068574722000000005 2024-09-04T17:47:57.664 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.664 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.664 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:57.664 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898934+0000", 2024-09-04T17:47:57.664 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2249000000000002e-05 2024-09-04T17:47:57.664 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.664 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.664 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:57.665 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.899127+0000", 2024-09-04T17:47:57.665 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019298699999999999 2024-09-04T17:47:57.665 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:57.665 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:57.665 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:57.665 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.665 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.665 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61182 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 868352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:57.665 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839249+0000", 2024-09-04T17:47:57.665 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 43.759194436999998, 2024-09-04T17:47:57.665 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073680151999999999, 2024-09-04T17:47:57.666 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:57.666 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:57.666 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:57.666 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:57.666 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:57.666 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:57.666 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61182 2024-09-04T17:47:57.666 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.666 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:57.666 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.666 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:57.666 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839249+0000", 2024-09-04T17:47:57.667 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:57.667 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.667 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.667 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:57.667 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839249+0000", 2024-09-04T17:47:57.667 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:57.667 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.667 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.667 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:57.667 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839250+0000", 2024-09-04T17:47:57.667 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4999999999999998e-07 2024-09-04T17:47:57.667 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.667 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.668 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:57.668 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839257+0000", 2024-09-04T17:47:57.668 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7700000000000004e-06 2024-09-04T17:47:57.668 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.668 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.668 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:57.668 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839263+0000", 2024-09-04T17:47:57.668 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.246e-06 2024-09-04T17:47:57.668 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.668 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.668 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:57.669 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839268+0000", 2024-09-04T17:47:57.669 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.814e-06 2024-09-04T17:47:57.669 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.669 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.669 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:57.669 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.912831+0000", 2024-09-04T17:47:57.669 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073563294000000001 2024-09-04T17:47:57.669 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.669 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.669 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:57.669 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.912876+0000", 2024-09-04T17:47:57.670 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4742e-05 2024-09-04T17:47:57.670 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.670 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.670 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:57.670 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.912930+0000", 2024-09-04T17:47:57.670 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.3936e-05 2024-09-04T17:47:57.670 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:57.670 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:57.670 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:57.670 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.670 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.671 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61183 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:57.671 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839275+0000", 2024-09-04T17:47:57.671 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 43.759169243999999, 2024-09-04T17:47:57.671 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073781886000000005, 2024-09-04T17:47:57.671 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:57.671 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:57.671 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:57.671 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:57.671 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:57.671 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:57.671 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61183 2024-09-04T17:47:57.672 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.672 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:57.672 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.672 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:57.672 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839275+0000", 2024-09-04T17:47:57.672 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:57.672 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.672 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.672 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:57.672 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839275+0000", 2024-09-04T17:47:57.672 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:57.673 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.673 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.673 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:57.673 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839275+0000", 2024-09-04T17:47:57.673 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6899999999999997e-07 2024-09-04T17:47:57.673 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.673 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.673 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:57.673 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839277+0000", 2024-09-04T17:47:57.673 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9530000000000002e-06 2024-09-04T17:47:57.674 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.674 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.674 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:57.674 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839283+0000", 2024-09-04T17:47:57.674 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.9329999999999998e-06 2024-09-04T17:47:57.674 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.674 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.674 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:57.674 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839288+0000", 2024-09-04T17:47:57.674 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4100000000000001e-06 2024-09-04T17:47:57.674 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.674 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.675 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:57.675 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.912959+0000", 2024-09-04T17:47:57.675 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073671511999999995 2024-09-04T17:47:57.675 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.675 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.675 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:57.675 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913001+0000", 2024-09-04T17:47:57.675 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.1984999999999999e-05 2024-09-04T17:47:57.675 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.675 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.675 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:57.676 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913057+0000", 2024-09-04T17:47:57.676 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.5523999999999999e-05 2024-09-04T17:47:57.676 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:57.676 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:57.676 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:57.676 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.676 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.676 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61184 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1114112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:57.676 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839294+0000", 2024-09-04T17:47:57.676 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 43.759150032000001, 2024-09-04T17:47:57.676 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073995263000000006, 2024-09-04T17:47:57.677 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:57.677 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:57.677 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:57.677 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:57.677 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:57.677 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:57.677 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61184 2024-09-04T17:47:57.677 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.677 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:57.677 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.677 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:57.678 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839294+0000", 2024-09-04T17:47:57.678 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:57.678 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.678 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.678 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:57.678 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839294+0000", 2024-09-04T17:47:57.678 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:57.678 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.678 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.678 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:57.678 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839294+0000", 2024-09-04T17:47:57.678 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.77e-07 2024-09-04T17:47:57.679 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.679 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.679 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:57.679 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839296+0000", 2024-09-04T17:47:57.679 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9999999999999999e-06 2024-09-04T17:47:57.679 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.679 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.679 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:57.679 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839302+0000", 2024-09-04T17:47:57.679 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7230000000000001e-06 2024-09-04T17:47:57.679 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.680 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.680 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:57.680 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839314+0000", 2024-09-04T17:47:57.680 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1826e-05 2024-09-04T17:47:57.680 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.680 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.680 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:57.680 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913084+0000", 2024-09-04T17:47:57.680 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073770215 2024-09-04T17:47:57.680 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.681 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.681 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:57.681 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913127+0000", 2024-09-04T17:47:57.681 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2982999999999998e-05 2024-09-04T17:47:57.681 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.681 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.681 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:57.681 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913289+0000", 2024-09-04T17:47:57.681 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000162139 2024-09-04T17:47:57.681 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:57.681 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:57.681 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:57.682 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.682 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.682 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61185 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1220608~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:57.682 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839321+0000", 2024-09-04T17:47:57.682 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 43.759122564999998, 2024-09-04T17:47:57.682 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074206062000000003, 2024-09-04T17:47:57.682 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:57.682 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:57.682 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:57.682 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:57.683 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:57.683 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:57.683 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61185 2024-09-04T17:47:57.683 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.683 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:57.683 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.683 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:57.683 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839321+0000", 2024-09-04T17:47:57.683 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:57.683 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.683 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.683 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:57.684 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839321+0000", 2024-09-04T17:47:57.684 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:57.684 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.684 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.684 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:57.684 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839322+0000", 2024-09-04T17:47:57.684 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.5400000000000001e-07 2024-09-04T17:47:57.684 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.684 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.684 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:57.684 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839324+0000", 2024-09-04T17:47:57.684 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2539999999999999e-06 2024-09-04T17:47:57.685 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.685 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.685 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:57.685 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839331+0000", 2024-09-04T17:47:57.685 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2139999999999999e-06 2024-09-04T17:47:57.685 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.685 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.685 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:57.685 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839336+0000", 2024-09-04T17:47:57.685 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7029999999999997e-06 2024-09-04T17:47:57.685 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.686 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.686 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:57.686 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913320+0000", 2024-09-04T17:47:57.686 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073984016 2024-09-04T17:47:57.686 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.686 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.686 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:57.686 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913366+0000", 2024-09-04T17:47:57.686 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.5704000000000002e-05 2024-09-04T17:47:57.686 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.686 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.686 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:57.687 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913527+0000", 2024-09-04T17:47:57.687 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00016161699999999999 2024-09-04T17:47:57.687 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:57.687 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:57.687 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:57.687 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.687 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.687 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61186 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1359872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:57.687 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839344+0000", 2024-09-04T17:47:57.687 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 43.759100222999997, 2024-09-04T17:47:57.687 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074450673999999994, 2024-09-04T17:47:57.688 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:57.688 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:57.688 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:57.688 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:57.688 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:57.688 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:57.688 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61186 2024-09-04T17:47:57.688 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.688 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:57.688 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.688 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:57.689 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839344+0000", 2024-09-04T17:47:57.689 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:57.689 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.689 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.689 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:57.689 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839344+0000", 2024-09-04T17:47:57.689 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:57.689 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.689 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.689 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:57.689 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839344+0000", 2024-09-04T17:47:57.690 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4799999999999999e-07 2024-09-04T17:47:57.690 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.690 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.690 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:57.690 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839346+0000", 2024-09-04T17:47:57.690 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0889999999999998e-06 2024-09-04T17:47:57.690 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.690 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.690 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:57.690 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839352+0000", 2024-09-04T17:47:57.690 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6389999999999997e-06 2024-09-04T17:47:57.690 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.691 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.691 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:57.691 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839356+0000", 2024-09-04T17:47:57.691 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.233e-06 2024-09-04T17:47:57.691 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.691 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.691 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:57.691 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913557+0000", 2024-09-04T17:47:57.691 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074200944000000005 2024-09-04T17:47:57.691 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.691 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.691 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:57.692 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913601+0000", 2024-09-04T17:47:57.692 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4350000000000001e-05 2024-09-04T17:47:57.692 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.692 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.692 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:57.692 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913794+0000", 2024-09-04T17:47:57.692 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019297100000000001 2024-09-04T17:47:57.692 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:57.692 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:57.692 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:57.693 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.693 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.693 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61187 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1466368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:57.693 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839364+0000", 2024-09-04T17:47:57.693 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 43.759080374, 2024-09-04T17:47:57.693 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074692334999999999, 2024-09-04T17:47:57.693 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:57.693 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:57.693 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:57.693 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:57.693 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:57.694 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:57.694 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61187 2024-09-04T17:47:57.694 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.694 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:57.694 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.694 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:57.694 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839364+0000", 2024-09-04T17:47:57.694 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:57.694 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.694 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.694 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:57.695 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839364+0000", 2024-09-04T17:47:57.695 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:57.695 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.695 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.695 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:57.695 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839364+0000", 2024-09-04T17:47:57.695 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.75e-07 2024-09-04T17:47:57.695 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.695 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.695 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:57.695 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839366+0000", 2024-09-04T17:47:57.696 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0990000000000001e-06 2024-09-04T17:47:57.696 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.696 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.696 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:57.696 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839372+0000", 2024-09-04T17:47:57.696 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.2609999999999999e-06 2024-09-04T17:47:57.696 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.696 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.696 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:57.696 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839389+0000", 2024-09-04T17:47:57.696 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6489000000000001e-05 2024-09-04T17:47:57.697 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.697 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.697 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:57.697 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913824+0000", 2024-09-04T17:47:57.697 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074434811000000004 2024-09-04T17:47:57.697 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.697 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.697 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:57.697 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913868+0000", 2024-09-04T17:47:57.697 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4768000000000002e-05 2024-09-04T17:47:57.698 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.698 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.698 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:57.698 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914056+0000", 2024-09-04T17:47:57.698 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00018743199999999999 2024-09-04T17:47:57.698 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:57.698 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:57.698 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:57.698 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.698 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.698 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61188 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1605632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:57.699 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839399+0000", 2024-09-04T17:47:57.699 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 43.759044478, 2024-09-04T17:47:57.699 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074975095000000005, 2024-09-04T17:47:57.699 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:57.699 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:57.699 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:57.699 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:57.699 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:57.699 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:57.699 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61188 2024-09-04T17:47:57.700 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.700 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:57.700 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.700 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:57.700 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839399+0000", 2024-09-04T17:47:57.700 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:57.700 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.700 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.700 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:57.700 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839399+0000", 2024-09-04T17:47:57.700 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:57.701 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.701 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.701 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:57.701 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839400+0000", 2024-09-04T17:47:57.701 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0100000000000005e-07 2024-09-04T17:47:57.701 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.701 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.701 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:57.701 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839402+0000", 2024-09-04T17:47:57.701 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.491e-06 2024-09-04T17:47:57.702 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.702 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.702 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:57.702 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839409+0000", 2024-09-04T17:47:57.702 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.0450000000000003e-06 2024-09-04T17:47:57.702 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.702 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.702 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:57.702 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839414+0000", 2024-09-04T17:47:57.702 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.87e-06 2024-09-04T17:47:57.702 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.703 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.703 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:57.703 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914086+0000", 2024-09-04T17:47:57.703 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074671596000000007 2024-09-04T17:47:57.703 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.703 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.703 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:57.703 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914130+0000", 2024-09-04T17:47:57.704 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4579999999999997e-05 2024-09-04T17:47:57.704 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.704 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.704 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:57.704 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914375+0000", 2024-09-04T17:47:57.704 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024401199999999999 2024-09-04T17:47:57.704 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:57.704 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:57.704 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:57.704 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.704 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.705 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61189 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1712128~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:57.705 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839424+0000", 2024-09-04T17:47:57.705 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 43.759020276999998, 2024-09-04T17:47:57.705 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075247680999999997, 2024-09-04T17:47:57.705 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:47:57.705 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:47:57.705 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:47:57.705 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:47:57.705 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:47:57.705 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:57.705 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61189 2024-09-04T17:47:57.706 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.706 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:47:57.706 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.706 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:47:57.706 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839424+0000", 2024-09-04T17:47:57.706 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:57.706 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.706 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.706 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:47:57.706 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839424+0000", 2024-09-04T17:47:57.706 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:47:57.707 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.707 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.707 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:47:57.707 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.846320+0000", 2024-09-04T17:47:57.707 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0068965210000000001 2024-09-04T17:47:57.707 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.707 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.707 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:47:57.707 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.846328+0000", 2024-09-04T17:47:57.707 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.8189999999999992e-06 2024-09-04T17:47:57.707 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.708 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.708 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:47:57.708 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.846341+0000", 2024-09-04T17:47:57.708 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3338e-05 2024-09-04T17:47:57.708 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.708 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.708 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:47:57.708 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.846352+0000", 2024-09-04T17:47:57.708 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0479000000000001e-05 2024-09-04T17:47:57.708 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.708 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.708 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:47:57.709 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914405+0000", 2024-09-04T17:47:57.709 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.068053049000000004 2024-09-04T17:47:57.709 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.709 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.709 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:47:57.709 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914451+0000", 2024-09-04T17:47:57.709 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6545999999999998e-05 2024-09-04T17:47:57.709 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:47:57.709 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:47:57.709 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:47:57.709 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914671+0000", 2024-09-04T17:47:57.710 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000219929 2024-09-04T17:47:57.710 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:57.710 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:57.710 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:57.710 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:47:57.710 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:47:57.710 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:57.712 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-04T17:47:57.713 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:57.711+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:57.813 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:57.808+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:57.887 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:57.887 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:47:57.887 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:47:57.887 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:57.901 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-04T17:47:57.945 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:57.944+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:47:58.014 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:58.012+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:58.064 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-04T17:47:58.065 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:58.065 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:47:58.065 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:47:58.065 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:47:58.065 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:58.081 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-04T17:47:58.115 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:47:58.111+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:47:58.215 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:47:58.211+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:47:58.247 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:58.247 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:47:58.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:47:58.247 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:47:58.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.247 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:27242 3.b 3.34414deb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-04T17:47:58.248 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:28.402347+0000", 2024-09-04T17:47:58.248 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 89.843658019000003, 2024-09-04T17:47:58.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.93265329299999999, 2024-09-04T17:47:58.248 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:58.248 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:58.248 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-04T17:47:58.248 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:58.248 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:58.248 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:58.248 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 27242 2024-09-04T17:47:58.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.248 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:58.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:58.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402347+0000", 2024-09-04T17:47:58.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:58.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402347+0000", 2024-09-04T17:47:58.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:58.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402349+0000", 2024-09-04T17:47:58.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.471e-06 2024-09-04T17:47:58.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:58.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402355+0000", 2024-09-04T17:47:58.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8399999999999997e-06 2024-09-04T17:47:58.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:58.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402364+0000", 2024-09-04T17:47:58.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1269999999999992e-06 2024-09-04T17:47:58.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:58.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402373+0000", 2024-09-04T17:47:58.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7909999999999993e-06 2024-09-04T17:47:58.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:58.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402460+0000", 2024-09-04T17:47:58.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6686000000000003e-05 2024-09-04T17:47:58.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-04T17:47:58.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402473+0000", 2024-09-04T17:47:58.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3281e-05 2024-09-04T17:47:58.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:58.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:29.334989+0000", 2024-09-04T17:47:58.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.93251549099999997 2024-09-04T17:47:58.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:58.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:29.335000+0000", 2024-09-04T17:47:58.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1606e-05 2024-09-04T17:47:58.253 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.253 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:58.253 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.254 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:33552 3.7 3:e79ad1db:::rbd_data.11342e8f3384.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-04T17:47:58.254 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:35.999748+0000", 2024-09-04T17:47:58.254 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 82.246256969000001, 2024-09-04T17:47:58.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045683823999999998, 2024-09-04T17:47:58.254 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:58.254 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:58.254 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:58.254 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:58.254 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:58.254 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:58.254 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 33552 2024-09-04T17:47:58.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.255 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:58.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:58.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999748+0000", 2024-09-04T17:47:58.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:58.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999748+0000", 2024-09-04T17:47:58.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:58.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999750+0000", 2024-09-04T17:47:58.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1299999999999999e-06 2024-09-04T17:47:58.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:58.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999759+0000", 2024-09-04T17:47:58.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4649999999999994e-06 2024-09-04T17:47:58.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:58.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999778+0000", 2024-09-04T17:47:58.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8833000000000001e-05 2024-09-04T17:47:58.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:58.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999786+0000", 2024-09-04T17:47:58.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7369999999999995e-06 2024-09-04T17:47:58.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:58.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999856+0000", 2024-09-04T17:47:58.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9247999999999998e-05 2024-09-04T17:47:58.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:58.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999973+0000", 2024-09-04T17:47:58.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000117921 2024-09-04T17:47:58.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 7", 2024-09-04T17:47:58.258 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.005955+0000", 2024-09-04T17:47:58.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059820560000000004 2024-09-04T17:47:58.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:58.258 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.045297+0000", 2024-09-04T17:47:58.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.039341650999999998 2024-09-04T17:47:58.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:58.258 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.045343+0000", 2024-09-04T17:47:58.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5899000000000002e-05 2024-09-04T17:47:58.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:58.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.045388+0000", 2024-09-04T17:47:58.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4601999999999997e-05 2024-09-04T17:47:58.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:58.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.045432+0000", 2024-09-04T17:47:58.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4282000000000001e-05 2024-09-04T17:47:58.260 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.260 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:58.260 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.260 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:37357 3.15 3:a897312a:::rbd_data.1134c5a43465.000000000000001f:head [stat,write 62464~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-04T17:47:58.260 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:39.616724+0000", 2024-09-04T17:47:58.260 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 78.629281560999999, 2024-09-04T17:47:58.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054965854000000001, 2024-09-04T17:47:58.260 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:58.261 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:58.261 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:58.261 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:58.261 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:58.261 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:58.261 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 37357 2024-09-04T17:47:58.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.261 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:58.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:58.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616724+0000", 2024-09-04T17:47:58.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:58.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616724+0000", 2024-09-04T17:47:58.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:58.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616725+0000", 2024-09-04T17:47:58.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.815e-06 2024-09-04T17:47:58.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:58.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616739+0000", 2024-09-04T17:47:58.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3723000000000001e-05 2024-09-04T17:47:58.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:58.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616755+0000", 2024-09-04T17:47:58.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5964000000000002e-05 2024-09-04T17:47:58.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:58.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616763+0000", 2024-09-04T17:47:58.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3869999999999995e-06 2024-09-04T17:47:58.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:58.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616825+0000", 2024-09-04T17:47:58.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1854999999999996e-05 2024-09-04T17:47:58.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:58.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616895+0000", 2024-09-04T17:47:58.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9597999999999993e-05 2024-09-04T17:47:58.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-04T17:47:58.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616991+0000", 2024-09-04T17:47:58.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5828000000000005e-05 2024-09-04T17:47:58.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:58.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.617659+0000", 2024-09-04T17:47:58.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00066821600000000001 2024-09-04T17:47:58.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:58.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.671482+0000", 2024-09-04T17:47:58.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053823533999999999 2024-09-04T17:47:58.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:58.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.671579+0000", 2024-09-04T17:47:58.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6657999999999995e-05 2024-09-04T17:47:58.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:58.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.671689+0000", 2024-09-04T17:47:58.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000110276 2024-09-04T17:47:58.266 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.266 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:58.266 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.266 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:43873 3.15 3:abd207ee:::rbd_data.113410ddd206.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:47:58.266 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:49.932940+0000", 2024-09-04T17:47:58.266 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 68.313065007999995, 2024-09-04T17:47:58.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042740514, 2024-09-04T17:47:58.267 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:58.267 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:58.267 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:58.267 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:58.267 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:58.267 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:58.267 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 43873 2024-09-04T17:47:58.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.267 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:58.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.267 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:58.267 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.932940+0000", 2024-09-04T17:47:58.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:58.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.932940+0000", 2024-09-04T17:47:58.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:58.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.932941+0000", 2024-09-04T17:47:58.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.404e-06 2024-09-04T17:47:58.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:58.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.933065+0000", 2024-09-04T17:47:58.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012361199999999999 2024-09-04T17:47:58.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:58.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.933078+0000", 2024-09-04T17:47:58.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3182e-05 2024-09-04T17:47:58.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:58.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.933085+0000", 2024-09-04T17:47:58.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4500000000000001e-06 2024-09-04T17:47:58.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:58.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.933134+0000", 2024-09-04T17:47:58.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9150999999999997e-05 2024-09-04T17:47:58.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:58.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.933242+0000", 2024-09-04T17:47:58.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010781 2024-09-04T17:47:58.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-04T17:47:58.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.938358+0000", 2024-09-04T17:47:58.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051160349999999997 2024-09-04T17:47:58.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:58.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.975343+0000", 2024-09-04T17:47:58.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036985663000000002 2024-09-04T17:47:58.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:58.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.975392+0000", 2024-09-04T17:47:58.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9044e-05 2024-09-04T17:47:58.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:58.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.975548+0000", 2024-09-04T17:47:58.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015572900000000001 2024-09-04T17:47:58.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:58.272 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.975681+0000", 2024-09-04T17:47:58.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013243400000000001 2024-09-04T17:47:58.272 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.272 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:58.272 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.272 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:46649 3.7 3:e0749ed5:::rbd_data.1134326b938e.000000000000014f:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-04T17:47:58.272 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:54.114730+0000", 2024-09-04T17:47:58.273 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 64.131274876000006, 2024-09-04T17:47:58.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032972972000000003, 2024-09-04T17:47:58.273 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:58.273 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:58.273 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:58.273 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:58.273 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:58.273 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:58.273 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 46649 2024-09-04T17:47:58.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.273 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:58.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:58.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114730+0000", 2024-09-04T17:47:58.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:58.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114730+0000", 2024-09-04T17:47:58.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:58.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114731+0000", 2024-09-04T17:47:58.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.147e-06 2024-09-04T17:47:58.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:58.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114737+0000", 2024-09-04T17:47:58.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6380000000000001e-06 2024-09-04T17:47:58.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:58.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114751+0000", 2024-09-04T17:47:58.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4012999999999999e-05 2024-09-04T17:47:58.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:58.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114758+0000", 2024-09-04T17:47:58.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1539999999999996e-06 2024-09-04T17:47:58.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:58.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114876+0000", 2024-09-04T17:47:58.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000117421 2024-09-04T17:47:58.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:58.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.115173+0000", 2024-09-04T17:47:58.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00029750799999999998 2024-09-04T17:47:58.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 7", 2024-09-04T17:47:58.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.120273+0000", 2024-09-04T17:47:58.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051002900000000004 2024-09-04T17:47:58.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:58.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.147571+0000", 2024-09-04T17:47:58.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027297906 2024-09-04T17:47:58.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:58.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.147588+0000", 2024-09-04T17:47:58.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6807999999999999e-05 2024-09-04T17:47:58.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:58.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.147637+0000", 2024-09-04T17:47:58.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8466e-05 2024-09-04T17:47:58.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:58.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.147703+0000", 2024-09-04T17:47:58.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6620999999999998e-05 2024-09-04T17:47:58.278 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.279 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:58.279 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.279 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49801 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 720896~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:58.279 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.075094+0000", 2024-09-04T17:47:58.279 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.170910888999998, 2024-09-04T17:47:58.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017824974, 2024-09-04T17:47:58.279 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:58.279 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:58.279 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:58.279 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:58.279 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:58.279 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:58.280 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49801 2024-09-04T17:47:58.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.280 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:58.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:58.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075094+0000", 2024-09-04T17:47:58.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:58.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075094+0000", 2024-09-04T17:47:58.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:58.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075095+0000", 2024-09-04T17:47:58.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.017e-06 2024-09-04T17:47:58.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:58.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075101+0000", 2024-09-04T17:47:58.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5029999999999997e-06 2024-09-04T17:47:58.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:58.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075119+0000", 2024-09-04T17:47:58.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8133000000000001e-05 2024-09-04T17:47:58.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:58.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075132+0000", 2024-09-04T17:47:58.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3305999999999999e-05 2024-09-04T17:47:58.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:58.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.092677+0000", 2024-09-04T17:47:58.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017544466000000002 2024-09-04T17:47:58.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:58.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.092782+0000", 2024-09-04T17:47:58.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000105284 2024-09-04T17:47:58.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:58.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.092919+0000", 2024-09-04T17:47:58.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013726499999999999 2024-09-04T17:47:58.283 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.283 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:58.283 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.283 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49809 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 163840~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:58.283 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077146+0000", 2024-09-04T17:47:58.284 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.168859445999999, 2024-09-04T17:47:58.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017984541999999999, 2024-09-04T17:47:58.284 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:58.284 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:58.284 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:58.284 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:58.284 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:58.284 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:58.284 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49809 2024-09-04T17:47:58.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.284 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:58.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:58.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077146+0000", 2024-09-04T17:47:58.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:58.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077146+0000", 2024-09-04T17:47:58.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:58.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077147+0000", 2024-09-04T17:47:58.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0330000000000001e-06 2024-09-04T17:47:58.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:58.286 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077164+0000", 2024-09-04T17:47:58.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7810000000000001e-05 2024-09-04T17:47:58.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:58.286 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077187+0000", 2024-09-04T17:47:58.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2246e-05 2024-09-04T17:47:58.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:58.286 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077203+0000", 2024-09-04T17:47:58.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6392000000000002e-05 2024-09-04T17:47:58.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:58.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.094885+0000", 2024-09-04T17:47:58.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017681589000000001 2024-09-04T17:47:58.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:58.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.094991+0000", 2024-09-04T17:47:58.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010668899999999999 2024-09-04T17:47:58.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:58.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095130+0000", 2024-09-04T17:47:58.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013878299999999999 2024-09-04T17:47:58.288 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.288 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:58.288 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.288 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49810 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 229376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:58.288 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077225+0000", 2024-09-04T17:47:58.288 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.168779604999997, 2024-09-04T17:47:58.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018218793, 2024-09-04T17:47:58.288 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:58.288 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:58.288 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:58.288 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:58.289 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:58.289 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:58.289 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49810 2024-09-04T17:47:58.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.289 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:58.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:58.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077225+0000", 2024-09-04T17:47:58.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:58.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077225+0000", 2024-09-04T17:47:58.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:58.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077227+0000", 2024-09-04T17:47:58.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1829999999999999e-06 2024-09-04T17:47:58.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:58.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077233+0000", 2024-09-04T17:47:58.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3899999999999998e-06 2024-09-04T17:47:58.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:58.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077253+0000", 2024-09-04T17:47:58.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0009000000000001e-05 2024-09-04T17:47:58.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:58.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077270+0000", 2024-09-04T17:47:58.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7336000000000001e-05 2024-09-04T17:47:58.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:58.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095177+0000", 2024-09-04T17:47:58.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017907101000000002 2024-09-04T17:47:58.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:58.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095299+0000", 2024-09-04T17:47:58.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000121749 2024-09-04T17:47:58.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:58.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095444+0000", 2024-09-04T17:47:58.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014502499999999999 2024-09-04T17:47:58.295 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.295 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:58.295 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.295 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49811 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 294912~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:58.295 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077288+0000", 2024-09-04T17:47:58.295 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.168717291, 2024-09-04T17:47:58.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018443620000000001, 2024-09-04T17:47:58.295 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:58.295 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:58.295 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:58.295 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:58.296 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:58.296 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:58.296 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49811 2024-09-04T17:47:58.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.296 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:58.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.296 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:58.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077288+0000", 2024-09-04T17:47:58.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.296 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:58.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077288+0000", 2024-09-04T17:47:58.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:58.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077289+0000", 2024-09-04T17:47:58.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6199999999999999e-06 2024-09-04T17:47:58.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:58.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077295+0000", 2024-09-04T17:47:58.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1499999999999998e-06 2024-09-04T17:47:58.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:58.298 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077312+0000", 2024-09-04T17:47:58.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7353000000000001e-05 2024-09-04T17:47:58.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:58.298 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077324+0000", 2024-09-04T17:47:58.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2593e-05 2024-09-04T17:47:58.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:58.298 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095490+0000", 2024-09-04T17:47:58.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018165976 2024-09-04T17:47:58.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:58.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095597+0000", 2024-09-04T17:47:58.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010629900000000001 2024-09-04T17:47:58.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:58.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095731+0000", 2024-09-04T17:47:58.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013462899999999999 2024-09-04T17:47:58.299 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.299 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:58.300 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.300 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49813 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 360448~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:58.300 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077349+0000", 2024-09-04T17:47:58.300 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.168656089000002, 2024-09-04T17:47:58.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018780305000000001, 2024-09-04T17:47:58.300 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:58.300 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:58.300 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:58.300 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:58.300 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:58.300 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:58.301 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49813 2024-09-04T17:47:58.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.301 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:58.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:58.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077349+0000", 2024-09-04T17:47:58.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:58.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077349+0000", 2024-09-04T17:47:58.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:58.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077350+0000", 2024-09-04T17:47:58.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0330000000000001e-06 2024-09-04T17:47:58.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:58.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077355+0000", 2024-09-04T17:47:58.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0699999999999997e-06 2024-09-04T17:47:58.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.303 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:58.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077376+0000", 2024-09-04T17:47:58.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0846e-05 2024-09-04T17:47:58.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.303 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:58.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077388+0000", 2024-09-04T17:47:58.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2353e-05 2024-09-04T17:47:58.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.303 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:58.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095819+0000", 2024-09-04T17:47:58.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01843057 2024-09-04T17:47:58.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:58.304 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095960+0000", 2024-09-04T17:47:58.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000140965 2024-09-04T17:47:58.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:58.304 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096129+0000", 2024-09-04T17:47:58.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016946800000000001 2024-09-04T17:47:58.304 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.304 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:58.304 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.305 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49814 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 425984~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:58.305 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077407+0000", 2024-09-04T17:47:58.305 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.168597587999997, 2024-09-04T17:47:58.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019041563000000001, 2024-09-04T17:47:58.305 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:58.305 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:58.305 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:58.306 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:58.306 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:58.306 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:58.306 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49814 2024-09-04T17:47:58.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.306 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:58.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:58.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077407+0000", 2024-09-04T17:47:58.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:58.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077407+0000", 2024-09-04T17:47:58.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:58.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077409+0000", 2024-09-04T17:47:58.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2330000000000001e-06 2024-09-04T17:47:58.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:58.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077415+0000", 2024-09-04T17:47:58.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2729999999999999e-06 2024-09-04T17:47:58.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:58.308 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077436+0000", 2024-09-04T17:47:58.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1209999999999999e-05 2024-09-04T17:47:58.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.308 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:58.308 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077448+0000", 2024-09-04T17:47:58.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2225999999999999e-05 2024-09-04T17:47:58.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.308 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:58.308 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096183+0000", 2024-09-04T17:47:58.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018735004 2024-09-04T17:47:58.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.309 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:58.309 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096297+0000", 2024-09-04T17:47:58.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000113962 2024-09-04T17:47:58.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.309 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:58.309 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096449+0000", 2024-09-04T17:47:58.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000151655 2024-09-04T17:47:58.309 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.309 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:58.309 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.310 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49815 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 491520~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:58.310 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077471+0000", 2024-09-04T17:47:58.310 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.168534350000002, 2024-09-04T17:47:58.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019266536000000001, 2024-09-04T17:47:58.310 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:58.310 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:58.310 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:58.310 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:58.310 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:58.310 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:58.310 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49815 2024-09-04T17:47:58.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.310 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:58.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:58.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077471+0000", 2024-09-04T17:47:58.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:58.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077471+0000", 2024-09-04T17:47:58.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:58.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077472+0000", 2024-09-04T17:47:58.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.17e-06 2024-09-04T17:47:58.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.312 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:58.312 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077478+0000", 2024-09-04T17:47:58.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0399999999999998e-06 2024-09-04T17:47:58.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.312 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:58.312 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077500+0000", 2024-09-04T17:47:58.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1959000000000001e-05 2024-09-04T17:47:58.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.312 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:58.312 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077513+0000", 2024-09-04T17:47:58.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2823000000000001e-05 2024-09-04T17:47:58.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:58.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096493+0000", 2024-09-04T17:47:58.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018980701999999999 2024-09-04T17:47:58.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:58.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096603+0000", 2024-09-04T17:47:58.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010942000000000001 2024-09-04T17:47:58.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:58.314 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096737+0000", 2024-09-04T17:47:58.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013442200000000001 2024-09-04T17:47:58.314 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.314 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:58.314 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.314 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49816 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 557056~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:58.314 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077532+0000", 2024-09-04T17:47:58.314 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.168472608999998, 2024-09-04T17:47:58.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019604331999999999, 2024-09-04T17:47:58.314 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:58.314 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:58.315 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:58.315 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:58.315 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:58.315 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:58.315 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49816 2024-09-04T17:47:58.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.315 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:58.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.315 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:58.315 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077532+0000", 2024-09-04T17:47:58.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.315 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:58.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077532+0000", 2024-09-04T17:47:58.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:58.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077534+0000", 2024-09-04T17:47:58.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1200000000000001e-06 2024-09-04T17:47:58.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:58.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077545+0000", 2024-09-04T17:47:58.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1673e-05 2024-09-04T17:47:58.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:58.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077569+0000", 2024-09-04T17:47:58.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3555999999999999e-05 2024-09-04T17:47:58.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:58.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077584+0000", 2024-09-04T17:47:58.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4883e-05 2024-09-04T17:47:58.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.318 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:58.318 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096825+0000", 2024-09-04T17:47:58.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019241734 2024-09-04T17:47:58.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.318 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:58.318 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096964+0000", 2024-09-04T17:47:58.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000138085 2024-09-04T17:47:58.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.318 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:58.318 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097137+0000", 2024-09-04T17:47:58.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017328100000000001 2024-09-04T17:47:58.319 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.319 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:58.319 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.319 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49817 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 622592~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:58.319 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077603+0000", 2024-09-04T17:47:58.319 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.168402051000001, 2024-09-04T17:47:58.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01986928, 2024-09-04T17:47:58.319 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:58.319 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:58.319 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:58.319 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:58.320 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:58.320 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:58.320 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49817 2024-09-04T17:47:58.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.320 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:58.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:58.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077603+0000", 2024-09-04T17:47:58.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:58.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077603+0000", 2024-09-04T17:47:58.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:58.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077604+0000", 2024-09-04T17:47:58.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7699999999999992e-07 2024-09-04T17:47:58.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:58.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077609+0000", 2024-09-04T17:47:58.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0830000000000003e-06 2024-09-04T17:47:58.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:58.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077630+0000", 2024-09-04T17:47:58.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0438999999999999e-05 2024-09-04T17:47:58.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:58.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077664+0000", 2024-09-04T17:47:58.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4579000000000002e-05 2024-09-04T17:47:58.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:58.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097193+0000", 2024-09-04T17:47:58.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019529174 2024-09-04T17:47:58.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.323 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:58.323 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097323+0000", 2024-09-04T17:47:58.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000129696 2024-09-04T17:47:58.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.323 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:58.323 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097472+0000", 2024-09-04T17:47:58.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014933200000000001 2024-09-04T17:47:58.323 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.323 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:58.323 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.324 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.324 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49818 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 688128~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:58.324 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077683+0000", 2024-09-04T17:47:58.324 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.168321687000002, 2024-09-04T17:47:58.324 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020086333000000001, 2024-09-04T17:47:58.324 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:58.324 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:58.324 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:58.324 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:58.324 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:58.324 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:58.325 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49818 2024-09-04T17:47:58.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.325 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:58.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:58.325 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077683+0000", 2024-09-04T17:47:58.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:58.325 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077683+0000", 2024-09-04T17:47:58.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:58.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077684+0000", 2024-09-04T17:47:58.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4e-07 2024-09-04T17:47:58.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:58.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077690+0000", 2024-09-04T17:47:58.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4729999999999999e-06 2024-09-04T17:47:58.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:58.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077710+0000", 2024-09-04T17:47:58.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0149999999999999e-05 2024-09-04T17:47:58.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:58.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077723+0000", 2024-09-04T17:47:58.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3271999999999999e-05 2024-09-04T17:47:58.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:58.328 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097521+0000", 2024-09-04T17:47:58.328 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019797981999999999 2024-09-04T17:47:58.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.328 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.328 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:58.328 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097628+0000", 2024-09-04T17:47:58.328 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010665700000000001 2024-09-04T17:47:58.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.328 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.328 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:58.328 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097770+0000", 2024-09-04T17:47:58.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014185900000000001 2024-09-04T17:47:58.329 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.329 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:58.329 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.329 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49820 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 753664~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:58.329 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077744+0000", 2024-09-04T17:47:58.329 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.168260756000002, 2024-09-04T17:47:58.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020364786999999999, 2024-09-04T17:47:58.329 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:58.329 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:58.329 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:58.330 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:58.330 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:58.330 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:58.330 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49820 2024-09-04T17:47:58.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.330 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:58.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.330 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:58.330 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077744+0000", 2024-09-04T17:47:58.330 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.330 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:58.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077744+0000", 2024-09-04T17:47:58.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.331 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:58.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077745+0000", 2024-09-04T17:47:58.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0270000000000001e-06 2024-09-04T17:47:58.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.331 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:58.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077751+0000", 2024-09-04T17:47:58.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.626e-06 2024-09-04T17:47:58.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.332 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.332 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:58.332 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077771+0000", 2024-09-04T17:47:58.332 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0469999999999999e-05 2024-09-04T17:47:58.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.332 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.332 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:58.332 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077785+0000", 2024-09-04T17:47:58.332 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3129000000000001e-05 2024-09-04T17:47:58.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.332 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.332 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:58.332 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097816+0000", 2024-09-04T17:47:58.332 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020031917999999999 2024-09-04T17:47:58.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.333 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.333 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:58.333 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097939+0000", 2024-09-04T17:47:58.333 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000122533 2024-09-04T17:47:58.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.333 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.333 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:58.333 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098109+0000", 2024-09-04T17:47:58.333 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017008400000000001 2024-09-04T17:47:58.333 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.333 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:58.333 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.334 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49821 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 819200~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:58.334 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077807+0000", 2024-09-04T17:47:58.334 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.168198543999999, 2024-09-04T17:47:58.334 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020702928999999998, 2024-09-04T17:47:58.334 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:58.334 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:58.334 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:58.334 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:58.334 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:58.334 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:58.334 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49821 2024-09-04T17:47:58.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.334 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:58.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.335 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:58.335 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077807+0000", 2024-09-04T17:47:58.335 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.335 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:58.335 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077807+0000", 2024-09-04T17:47:58.335 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.335 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:58.335 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077808+0000", 2024-09-04T17:47:58.335 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.463e-06 2024-09-04T17:47:58.336 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.336 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.336 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:58.336 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077813+0000", 2024-09-04T17:47:58.336 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7530000000000001e-06 2024-09-04T17:47:58.336 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.336 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.336 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:58.336 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077832+0000", 2024-09-04T17:47:58.336 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.946e-05 2024-09-04T17:47:58.336 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.336 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.336 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:58.337 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077846+0000", 2024-09-04T17:47:58.337 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3786e-05 2024-09-04T17:47:58.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.337 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:58.337 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098201+0000", 2024-09-04T17:47:58.337 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020354797000000001 2024-09-04T17:47:58.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.337 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:58.337 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098341+0000", 2024-09-04T17:47:58.337 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000140042 2024-09-04T17:47:58.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.338 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:58.338 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098509+0000", 2024-09-04T17:47:58.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016862800000000001 2024-09-04T17:47:58.338 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.338 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:58.338 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.338 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49823 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 884736~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:58.338 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077865+0000", 2024-09-04T17:47:58.338 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.168140100000002, 2024-09-04T17:47:58.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021008723, 2024-09-04T17:47:58.338 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:58.338 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:58.339 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:58.339 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:58.339 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:58.339 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:58.339 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49823 2024-09-04T17:47:58.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.339 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:58.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.339 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:58.339 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077865+0000", 2024-09-04T17:47:58.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.339 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:58.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077865+0000", 2024-09-04T17:47:58.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.340 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:58.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077866+0000", 2024-09-04T17:47:58.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.164e-06 2024-09-04T17:47:58.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.340 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:58.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077872+0000", 2024-09-04T17:47:58.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3959999999999997e-06 2024-09-04T17:47:58.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.341 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:58.341 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077892+0000", 2024-09-04T17:47:58.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0970000000000001e-05 2024-09-04T17:47:58.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.341 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:58.341 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077905+0000", 2024-09-04T17:47:58.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2312e-05 2024-09-04T17:47:58.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.341 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:58.341 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098579+0000", 2024-09-04T17:47:58.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020674486999999998 2024-09-04T17:47:58.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.342 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:58.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098705+0000", 2024-09-04T17:47:58.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012541900000000001 2024-09-04T17:47:58.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.342 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:58.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098874+0000", 2024-09-04T17:47:58.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000168975 2024-09-04T17:47:58.342 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.342 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:58.342 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.343 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49824 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 950272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:58.343 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077923+0000", 2024-09-04T17:47:58.343 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.168082407999997, 2024-09-04T17:47:58.343 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021246287999999999, 2024-09-04T17:47:58.343 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:58.343 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:58.343 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:58.343 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:58.343 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:58.343 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:58.343 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49824 2024-09-04T17:47:58.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.344 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:58.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.344 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:58.344 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077923+0000", 2024-09-04T17:47:58.344 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.344 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:58.344 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077923+0000", 2024-09-04T17:47:58.344 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.344 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:58.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077924+0000", 2024-09-04T17:47:58.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0100000000000001e-06 2024-09-04T17:47:58.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.345 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:58.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077944+0000", 2024-09-04T17:47:58.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0449e-05 2024-09-04T17:47:58.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.345 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:58.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077965+0000", 2024-09-04T17:47:58.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0479000000000002e-05 2024-09-04T17:47:58.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:58.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077990+0000", 2024-09-04T17:47:58.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5582999999999999e-05 2024-09-04T17:47:58.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:58.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098924+0000", 2024-09-04T17:47:58.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020933395 2024-09-04T17:47:58.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:58.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.099032+0000", 2024-09-04T17:47:58.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010846299999999999 2024-09-04T17:47:58.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:58.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.099169+0000", 2024-09-04T17:47:58.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000136909 2024-09-04T17:47:58.347 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.347 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:58.347 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.347 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49826 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 1015808~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:47:58.347 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.078010+0000", 2024-09-04T17:47:58.347 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.167994800999999, 2024-09-04T17:47:58.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021584564000000001, 2024-09-04T17:47:58.347 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:58.348 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:58.348 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:58.348 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:58.348 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:58.348 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:58.348 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49826 2024-09-04T17:47:58.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.348 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:58.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:58.348 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078010+0000", 2024-09-04T17:47:58.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:58.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078010+0000", 2024-09-04T17:47:58.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:58.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078011+0000", 2024-09-04T17:47:58.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1340000000000001e-06 2024-09-04T17:47:58.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:58.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078017+0000", 2024-09-04T17:47:58.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.682e-06 2024-09-04T17:47:58.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:58.350 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078036+0000", 2024-09-04T17:47:58.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9357000000000001e-05 2024-09-04T17:47:58.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:58.350 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078050+0000", 2024-09-04T17:47:58.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3122000000000001e-05 2024-09-04T17:47:58.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:58.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.099271+0000", 2024-09-04T17:47:58.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021221772999999999 2024-09-04T17:47:58.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:58.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.099410+0000", 2024-09-04T17:47:58.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000138415 2024-09-04T17:47:58.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:58.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.099595+0000", 2024-09-04T17:47:58.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000185081 2024-09-04T17:47:58.351 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.351 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:58.352 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.352 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.352 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:58.352 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:58.353 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-04T17:47:58.354 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:47:58.351+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:47:58.454 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:58.451+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:58.471 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:58.471 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:47:58.471 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:47:58.471 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:58.485 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-04T17:47:58.556 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:47:58.552+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:47:58.610 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:58.610 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:47:58.610 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:47:58.610 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:47:58.611 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:58.624 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-04T17:47:58.666 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:47:58.664+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:47:58.757 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:58.754+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:58.792 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:58.793 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:47:58.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:47:58.793 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:47:58.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.793 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:71638 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-04T17:47:58.793 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:44.479908+0000", 2024-09-04T17:47:58.793 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.309915254, 2024-09-04T17:47:58.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.469396488, 2024-09-04T17:47:58.793 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:58.793 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:58.793 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:58.794 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:58.794 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:58.794 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:58.794 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 71638 2024-09-04T17:47:58.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.794 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:58.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:58.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.479908+0000", 2024-09-04T17:47:58.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:58.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.479908+0000", 2024-09-04T17:47:58.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:58.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.479911+0000", 2024-09-04T17:47:58.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9160000000000001e-06 2024-09-04T17:47:58.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:58.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.479925+0000", 2024-09-04T17:47:58.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3896e-05 2024-09-04T17:47:58.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:58.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.479954+0000", 2024-09-04T17:47:58.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9249000000000001e-05 2024-09-04T17:47:58.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:58.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.479969+0000", 2024-09-04T17:47:58.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4683e-05 2024-09-04T17:47:58.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:58.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.480109+0000", 2024-09-04T17:47:58.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013995699999999999 2024-09-04T17:47:58.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-04T17:47:58.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.480125+0000", 2024-09-04T17:47:58.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6651999999999999e-05 2024-09-04T17:47:58.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:58.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.948598+0000", 2024-09-04T17:47:58.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.46847249600000002 2024-09-04T17:47:58.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:58.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.948889+0000", 2024-09-04T17:47:58.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000290947 2024-09-04T17:47:58.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:58.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.949304+0000", 2024-09-04T17:47:58.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000415692 2024-09-04T17:47:58.798 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.798 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:58.798 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.798 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:72025 3.9 3:95e86ba4:::rbd_object_map.11342ece613c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-09-04T17:47:58.799 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:45.537012+0000", 2024-09-04T17:47:58.799 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.252811368, 2024-09-04T17:47:58.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015242879000000001, 2024-09-04T17:47:58.799 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:58.799 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:58.799 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:58.799 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:58.799 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:58.799 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:58.799 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 72025 2024-09-04T17:47:58.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.799 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:58.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:58.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.537012+0000", 2024-09-04T17:47:58.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:58.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.537012+0000", 2024-09-04T17:47:58.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:58.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.537014+0000", 2024-09-04T17:47:58.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4700000000000001e-06 2024-09-04T17:47:58.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:58.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.537027+0000", 2024-09-04T17:47:58.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2292e-05 2024-09-04T17:47:58.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:58.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.537058+0000", 2024-09-04T17:47:58.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1819000000000001e-05 2024-09-04T17:47:58.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:58.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.537078+0000", 2024-09-04T17:47:58.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9966000000000002e-05 2024-09-04T17:47:58.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:58.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.544756+0000", 2024-09-04T17:47:58.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0076779049999999996 2024-09-04T17:47:58.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:58.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.544971+0000", 2024-09-04T17:47:58.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021493799999999999 2024-09-04T17:47:58.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-04T17:47:58.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.545629+0000", 2024-09-04T17:47:58.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00065790900000000003 2024-09-04T17:47:58.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:58.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.551964+0000", 2024-09-04T17:47:58.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0063348120000000004 2024-09-04T17:47:58.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:58.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.552116+0000", 2024-09-04T17:47:58.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015253700000000001 2024-09-04T17:47:58.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:58.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.552190+0000", 2024-09-04T17:47:58.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3980000000000007e-05 2024-09-04T17:47:58.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:58.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.552255+0000", 2024-09-04T17:47:58.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4251000000000005e-05 2024-09-04T17:47:58.804 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.804 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:58.804 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.805 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:72187 3.9 3:95e86ba4:::rbd_object_map.11342ece613c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-09-04T17:47:58.805 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:45.621539+0000", 2024-09-04T17:47:58.805 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.168284335999999, 2024-09-04T17:47:58.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014964125, 2024-09-04T17:47:58.805 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:58.805 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:58.805 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:58.805 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:58.805 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:58.805 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:58.805 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 72187 2024-09-04T17:47:58.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.806 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:58.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:58.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.621539+0000", 2024-09-04T17:47:58.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:58.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.621539+0000", 2024-09-04T17:47:58.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:58.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.621540+0000", 2024-09-04T17:47:58.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1739999999999999e-06 2024-09-04T17:47:58.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:58.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.621548+0000", 2024-09-04T17:47:58.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9829999999999996e-06 2024-09-04T17:47:58.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:58.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.621581+0000", 2024-09-04T17:47:58.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2768000000000002e-05 2024-09-04T17:47:58.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:58.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.621597+0000", 2024-09-04T17:47:58.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6629999999999998e-05 2024-09-04T17:47:58.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:58.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.628859+0000", 2024-09-04T17:47:58.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0072614860000000002 2024-09-04T17:47:58.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:58.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.629009+0000", 2024-09-04T17:47:58.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015048399999999999 2024-09-04T17:47:58.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-04T17:47:58.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.629568+0000", 2024-09-04T17:47:58.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00055856199999999997 2024-09-04T17:47:58.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:58.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.631466+0000", 2024-09-04T17:47:58.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001898233 2024-09-04T17:47:58.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:58.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.636326+0000", 2024-09-04T17:47:58.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004859749 2024-09-04T17:47:58.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:58.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.636419+0000", 2024-09-04T17:47:58.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3308999999999997e-05 2024-09-04T17:47:58.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:58.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.636503+0000", 2024-09-04T17:47:58.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3746999999999996e-05 2024-09-04T17:47:58.811 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.811 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:58.811 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.811 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:72189 3.9 3:95e86ba4:::rbd_object_map.11342ece613c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-09-04T17:47:58.811 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:45.623660+0000", 2024-09-04T17:47:58.811 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.166163349, 2024-09-04T17:47:58.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017628613000000001, 2024-09-04T17:47:58.811 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:58.811 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:58.811 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:58.811 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:58.812 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:58.812 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:58.812 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 72189 2024-09-04T17:47:58.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.812 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:58.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:58.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.623660+0000", 2024-09-04T17:47:58.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:58.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.623660+0000", 2024-09-04T17:47:58.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:58.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.623662+0000", 2024-09-04T17:47:58.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.866e-06 2024-09-04T17:47:58.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:58.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.623671+0000", 2024-09-04T17:47:58.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7200000000000001e-06 2024-09-04T17:47:58.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:58.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.623692+0000", 2024-09-04T17:47:58.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0574999999999999e-05 2024-09-04T17:47:58.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:58.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.623707+0000", 2024-09-04T17:47:58.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5437000000000001e-05 2024-09-04T17:47:58.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:58.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.634284+0000", 2024-09-04T17:47:58.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010577058 2024-09-04T17:47:58.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:58.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.634490+0000", 2024-09-04T17:47:58.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000205164 2024-09-04T17:47:58.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-04T17:47:58.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.635355+0000", 2024-09-04T17:47:58.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00086585299999999998 2024-09-04T17:47:58.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:58.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.636706+0000", 2024-09-04T17:47:58.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001351002 2024-09-04T17:47:58.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:58.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.641142+0000", 2024-09-04T17:47:58.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0044352910000000001 2024-09-04T17:47:58.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:58.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.641220+0000", 2024-09-04T17:47:58.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8733000000000007e-05 2024-09-04T17:47:58.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:58.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.641288+0000", 2024-09-04T17:47:58.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7914000000000001e-05 2024-09-04T17:47:58.817 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.817 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:58.817 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.817 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:72192 3.9 3:95e86ba4:::rbd_object_map.11342ece613c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-09-04T17:47:58.817 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:45.625578+0000", 2024-09-04T17:47:58.817 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.164244974000001, 2024-09-04T17:47:58.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016153130000000002, 2024-09-04T17:47:58.817 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:58.817 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:58.817 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:58.818 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:58.818 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:58.818 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:58.818 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 72192 2024-09-04T17:47:58.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.818 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:58.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:58.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.625578+0000", 2024-09-04T17:47:58.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:58.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.625578+0000", 2024-09-04T17:47:58.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:58.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.625580+0000", 2024-09-04T17:47:58.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.886e-06 2024-09-04T17:47:58.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:58.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.625589+0000", 2024-09-04T17:47:58.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.329e-06 2024-09-04T17:47:58.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:58.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.625615+0000", 2024-09-04T17:47:58.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5562999999999999e-05 2024-09-04T17:47:58.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:58.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.625628+0000", 2024-09-04T17:47:58.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2726e-05 2024-09-04T17:47:58.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:58.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.636934+0000", 2024-09-04T17:47:58.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01130605 2024-09-04T17:47:58.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:58.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.637112+0000", 2024-09-04T17:47:58.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000177789 2024-09-04T17:47:58.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-04T17:47:58.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.637723+0000", 2024-09-04T17:47:58.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00061164399999999995 2024-09-04T17:47:58.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:58.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.641306+0000", 2024-09-04T17:47:58.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003583227 2024-09-04T17:47:58.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:58.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.641424+0000", 2024-09-04T17:47:58.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000118035 2024-09-04T17:47:58.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:58.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.641596+0000", 2024-09-04T17:47:58.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017191900000000001 2024-09-04T17:47:58.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:58.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.641731+0000", 2024-09-04T17:47:58.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000134962 2024-09-04T17:47:58.823 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.823 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:58.823 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.823 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:72267 3.1a 3:58dbd2e1:::rbd_data.11342ece613c.000000000000001e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-09-04T17:47:58.823 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:45.674662+0000", 2024-09-04T17:47:58.823 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.115161445, 2024-09-04T17:47:58.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032248339000000001, 2024-09-04T17:47:58.823 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:58.823 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:58.823 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:58.824 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:58.824 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:58.824 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:58.824 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 72267 2024-09-04T17:47:58.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.824 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:58.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:58.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.674662+0000", 2024-09-04T17:47:58.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:58.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.674662+0000", 2024-09-04T17:47:58.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:58.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.674664+0000", 2024-09-04T17:47:58.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4870000000000001e-06 2024-09-04T17:47:58.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:58.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.674689+0000", 2024-09-04T17:47:58.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4805000000000001e-05 2024-09-04T17:47:58.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:58.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.674715+0000", 2024-09-04T17:47:58.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5548999999999999e-05 2024-09-04T17:47:58.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:58.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.674731+0000", 2024-09-04T17:47:58.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6549999999999999e-05 2024-09-04T17:47:58.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:58.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.674843+0000", 2024-09-04T17:47:58.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000112001 2024-09-04T17:47:58.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:58.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.675176+0000", 2024-09-04T17:47:58.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00033337200000000002 2024-09-04T17:47:58.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-04T17:47:58.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.680121+0000", 2024-09-04T17:47:58.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0049445540000000003 2024-09-04T17:47:58.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:58.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.706702+0000", 2024-09-04T17:47:58.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026580840000000001 2024-09-04T17:47:58.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:58.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.706738+0000", 2024-09-04T17:47:58.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6158999999999997e-05 2024-09-04T17:47:58.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:58.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.706835+0000", 2024-09-04T17:47:58.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6909000000000003e-05 2024-09-04T17:47:58.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:58.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.706910+0000", 2024-09-04T17:47:58.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5112999999999997e-05 2024-09-04T17:47:58.829 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.829 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:58.829 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.829 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:76779 3.1a 3:5e29c416:::rbd_data.11346ecfd364.0000000000000001:26 [sparse-read 4980736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-04T17:47:58.829 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:49.350892+0000", 2024-09-04T17:47:58.830 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.4389306669999993, 2024-09-04T17:47:58.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015234678, 2024-09-04T17:47:58.830 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:58.830 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:58.830 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:58.830 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:58.830 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:58.830 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:58.830 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 76779 2024-09-04T17:47:58.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.830 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:58.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:58.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.350892+0000", 2024-09-04T17:47:58.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:58.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.350892+0000", 2024-09-04T17:47:58.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:58.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.350893+0000", 2024-09-04T17:47:58.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2599999999999998e-07 2024-09-04T17:47:58.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:58.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.350895+0000", 2024-09-04T17:47:58.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8950000000000001e-06 2024-09-04T17:47:58.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:58.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.350901+0000", 2024-09-04T17:47:58.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2859999999999996e-06 2024-09-04T17:47:58.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:58.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.350906+0000", 2024-09-04T17:47:58.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.006e-06 2024-09-04T17:47:58.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:58.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.365798+0000", 2024-09-04T17:47:58.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014891547999999999 2024-09-04T17:47:58.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:58.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.365929+0000", 2024-09-04T17:47:58.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013089899999999999 2024-09-04T17:47:58.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:58.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.366127+0000", 2024-09-04T17:47:58.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019861799999999999 2024-09-04T17:47:58.833 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.834 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:58.834 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.834 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:76780 3.1a 3:5e29c416:::rbd_data.11346ecfd364.0000000000000001:26 [sparse-read 5373952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-04T17:47:58.834 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:49.350913+0000", 2024-09-04T17:47:58.834 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.4389100419999998, 2024-09-04T17:47:58.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015574363000000001, 2024-09-04T17:47:58.834 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:58.834 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:58.834 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:58.834 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:58.835 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:58.835 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:58.835 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 76780 2024-09-04T17:47:58.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.835 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:58.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.835 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:58.835 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.350913+0000", 2024-09-04T17:47:58.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.835 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:58.835 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.350913+0000", 2024-09-04T17:47:58.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:58.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.350913+0000", 2024-09-04T17:47:58.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6800000000000001e-07 2024-09-04T17:47:58.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:58.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.350921+0000", 2024-09-04T17:47:58.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1489999999999997e-06 2024-09-04T17:47:58.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:58.837 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.350928+0000", 2024-09-04T17:47:58.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9190000000000002e-06 2024-09-04T17:47:58.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.837 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:58.837 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.350933+0000", 2024-09-04T17:47:58.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4260000000000002e-06 2024-09-04T17:47:58.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.837 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:58.837 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.366174+0000", 2024-09-04T17:47:58.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015240965 2024-09-04T17:47:58.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:58.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.366312+0000", 2024-09-04T17:47:58.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013797500000000001 2024-09-04T17:47:58.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:58.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.366487+0000", 2024-09-04T17:47:58.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000175561 2024-09-04T17:47:58.838 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.838 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:58.838 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.839 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:76781 3.1a 3:5e29c416:::rbd_data.11346ecfd364.0000000000000001:26 [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-04T17:47:58.839 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:49.350941+0000", 2024-09-04T17:47:58.839 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.4388823730000002, 2024-09-04T17:47:58.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015907287999999999, 2024-09-04T17:47:58.839 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:58.839 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:58.839 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:58.839 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:58.839 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:58.839 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:58.839 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 76781 2024-09-04T17:47:58.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.839 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:58.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:58.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.350941+0000", 2024-09-04T17:47:58.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:58.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.350941+0000", 2024-09-04T17:47:58.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:58.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.350941+0000", 2024-09-04T17:47:58.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5600000000000001e-07 2024-09-04T17:47:58.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:58.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.350943+0000", 2024-09-04T17:47:58.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1950000000000002e-06 2024-09-04T17:47:58.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:58.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.350950+0000", 2024-09-04T17:47:58.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3609999999999997e-06 2024-09-04T17:47:58.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:58.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.350955+0000", 2024-09-04T17:47:58.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0629999999999996e-06 2024-09-04T17:47:58.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:58.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.366537+0000", 2024-09-04T17:47:58.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015582097 2024-09-04T17:47:58.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:58.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.366678+0000", 2024-09-04T17:47:58.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014130499999999999 2024-09-04T17:47:58.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:58.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.366848+0000", 2024-09-04T17:47:58.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016981099999999999 2024-09-04T17:47:58.843 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.843 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:58.843 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.843 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:76782 3.1a 3:5e29c416:::rbd_data.11346ecfd364.0000000000000001:26 [sparse-read 6160384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-04T17:47:58.843 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:49.350963+0000", 2024-09-04T17:47:58.843 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.4388598189999993, 2024-09-04T17:47:58.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016229093, 2024-09-04T17:47:58.843 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:58.843 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:58.843 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:58.844 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:58.844 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:58.844 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:58.844 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 76782 2024-09-04T17:47:58.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.844 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:58.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:58.844 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.350963+0000", 2024-09-04T17:47:58.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:58.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.350963+0000", 2024-09-04T17:47:58.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:58.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.350964+0000", 2024-09-04T17:47:58.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0900000000000002e-07 2024-09-04T17:47:58.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:58.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.350966+0000", 2024-09-04T17:47:58.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1129999999999999e-06 2024-09-04T17:47:58.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:58.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.350973+0000", 2024-09-04T17:47:58.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6359999999999997e-06 2024-09-04T17:47:58.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:58.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.350978+0000", 2024-09-04T17:47:58.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2499999999999997e-06 2024-09-04T17:47:58.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:58.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.366899+0000", 2024-09-04T17:47:58.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015920937999999999 2024-09-04T17:47:58.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:58.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.367041+0000", 2024-09-04T17:47:58.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014241500000000001 2024-09-04T17:47:58.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:58.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.367192+0000", 2024-09-04T17:47:58.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000151232 2024-09-04T17:47:58.847 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.847 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:58.847 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.848 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:77691 3.1e 3:7bc91465:::rbd_data.11346ecfd364.000000000000000e:26 [sparse-read 6553600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-04T17:47:58.848 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:49.386783+0000", 2024-09-04T17:47:58.848 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.4030405209999994, 2024-09-04T17:47:58.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015016036999999999, 2024-09-04T17:47:58.848 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:58.848 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:58.848 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:58.848 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:58.848 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:58.848 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:58.848 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77691 2024-09-04T17:47:58.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.849 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:58.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:58.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.386783+0000", 2024-09-04T17:47:58.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:58.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.386783+0000", 2024-09-04T17:47:58.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:58.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.386783+0000", 2024-09-04T17:47:58.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3000000000000001e-07 2024-09-04T17:47:58.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:58.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.386786+0000", 2024-09-04T17:47:58.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4490000000000002e-06 2024-09-04T17:47:58.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:58.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.386793+0000", 2024-09-04T17:47:58.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8569999999999995e-06 2024-09-04T17:47:58.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:58.851 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.386800+0000", 2024-09-04T17:47:58.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3849999999999999e-06 2024-09-04T17:47:58.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:58.851 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.401592+0000", 2024-09-04T17:47:58.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014792299 2024-09-04T17:47:58.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:58.851 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.401665+0000", 2024-09-04T17:47:58.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3344999999999997e-05 2024-09-04T17:47:58.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:58.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.401799+0000", 2024-09-04T17:47:58.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000133172 2024-09-04T17:47:58.852 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.852 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:58.852 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.853 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:77693 3.1e 3:7bc91465:::rbd_data.11346ecfd364.000000000000000e:26 [sparse-read 6946816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-04T17:47:58.853 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:49.386808+0000", 2024-09-04T17:47:58.853 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.4030148990000004, 2024-09-04T17:47:58.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015244825, 2024-09-04T17:47:58.853 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:58.853 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:58.853 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:58.854 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:58.854 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:58.854 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:58.854 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77693 2024-09-04T17:47:58.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.854 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:58.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.854 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:58.854 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.386808+0000", 2024-09-04T17:47:58.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.854 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:58.855 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.386808+0000", 2024-09-04T17:47:58.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.855 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:58.855 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.386809+0000", 2024-09-04T17:47:58.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0900000000000002e-07 2024-09-04T17:47:58.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.855 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:58.855 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.386811+0000", 2024-09-04T17:47:58.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2919999999999998e-06 2024-09-04T17:47:58.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:58.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.386818+0000", 2024-09-04T17:47:58.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1740000000000003e-06 2024-09-04T17:47:58.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:58.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.386824+0000", 2024-09-04T17:47:58.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1770000000000003e-06 2024-09-04T17:47:58.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:58.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.401826+0000", 2024-09-04T17:47:58.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015001552 2024-09-04T17:47:58.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.857 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:58.857 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.401900+0000", 2024-09-04T17:47:58.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3594e-05 2024-09-04T17:47:58.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.857 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:58.857 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.402053+0000", 2024-09-04T17:47:58.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015352700000000001 2024-09-04T17:47:58.857 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.857 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:58.858 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.858 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.858 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:77695 3.1e 3:7bc91465:::rbd_data.11346ecfd364.000000000000000e:26 [sparse-read 7340032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-04T17:47:58.858 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:49.386834+0000", 2024-09-04T17:47:58.858 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.4029892670000006, 2024-09-04T17:47:58.858 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015481039, 2024-09-04T17:47:58.858 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:58.858 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:58.858 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:58.858 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:58.858 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:58.858 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:58.859 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77695 2024-09-04T17:47:58.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.859 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:58.859 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.859 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:58.859 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.386834+0000", 2024-09-04T17:47:58.859 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.859 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.859 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:58.859 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.386834+0000", 2024-09-04T17:47:58.859 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.860 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:58.860 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.386835+0000", 2024-09-04T17:47:58.860 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0999999999999998e-07 2024-09-04T17:47:58.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.860 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:58.860 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.386842+0000", 2024-09-04T17:47:58.860 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5009999999999998e-06 2024-09-04T17:47:58.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.860 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:58.860 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.386851+0000", 2024-09-04T17:47:58.860 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4950000000000008e-06 2024-09-04T17:47:58.861 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.861 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.861 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:58.861 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.386857+0000", 2024-09-04T17:47:58.861 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3330000000000002e-06 2024-09-04T17:47:58.861 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.861 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.861 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:58.861 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.402082+0000", 2024-09-04T17:47:58.861 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015225308999999999 2024-09-04T17:47:58.861 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.861 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.861 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:58.861 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.402162+0000", 2024-09-04T17:47:58.862 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9757000000000002e-05 2024-09-04T17:47:58.862 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.862 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.862 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:58.862 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.402315+0000", 2024-09-04T17:47:58.862 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000152934 2024-09-04T17:47:58.862 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.862 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:58.863 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.863 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:77697 3.1e 3:7bc91465:::rbd_data.11346ecfd364.000000000000000e:26 [sparse-read 7733248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-04T17:47:58.863 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:49.386866+0000", 2024-09-04T17:47:58.863 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.4029576329999998, 2024-09-04T17:47:58.863 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015708755000000001, 2024-09-04T17:47:58.863 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:58.863 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:58.863 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:58.863 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:58.863 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:58.863 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:58.864 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77697 2024-09-04T17:47:58.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.864 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:58.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.864 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:58.864 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.386866+0000", 2024-09-04T17:47:58.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.864 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:58.864 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.386866+0000", 2024-09-04T17:47:58.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.865 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.865 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:58.865 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.386866+0000", 2024-09-04T17:47:58.865 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.82e-07 2024-09-04T17:47:58.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.865 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.865 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:58.865 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.386869+0000", 2024-09-04T17:47:58.865 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6469999999999998e-06 2024-09-04T17:47:58.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.865 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.865 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:58.865 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.386876+0000", 2024-09-04T17:47:58.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5900000000000002e-06 2024-09-04T17:47:58.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.866 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.866 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:58.866 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.386882+0000", 2024-09-04T17:47:58.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0569999999999998e-06 2024-09-04T17:47:58.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.866 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.866 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:58.866 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.402346+0000", 2024-09-04T17:47:58.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015463764 2024-09-04T17:47:58.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.867 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.867 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:58.867 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.402420+0000", 2024-09-04T17:47:58.867 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3730000000000001e-05 2024-09-04T17:47:58.867 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.867 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.867 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:58.867 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.402574+0000", 2024-09-04T17:47:58.867 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015438499999999999 2024-09-04T17:47:58.867 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.867 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:58.867 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.868 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.868 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.868 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:77699 3.1e 3:7bc91465:::rbd_data.11346ecfd364.000000000000000e:26 [sparse-read 8126464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-04T17:47:58.868 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:49.386891+0000", 2024-09-04T17:47:58.868 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.4029316420000004, 2024-09-04T17:47:58.868 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015921101, 2024-09-04T17:47:58.868 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:58.868 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:58.868 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:58.868 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:58.868 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:58.868 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:58.869 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77699 2024-09-04T17:47:58.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.869 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:58.869 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.869 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:58.869 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.386891+0000", 2024-09-04T17:47:58.869 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.869 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.869 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:58.869 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.386891+0000", 2024-09-04T17:47:58.869 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.870 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.870 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:58.870 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.386892+0000", 2024-09-04T17:47:58.870 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7e-07 2024-09-04T17:47:58.870 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.870 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.870 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:58.870 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.386894+0000", 2024-09-04T17:47:58.870 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3070000000000001e-06 2024-09-04T17:47:58.870 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.870 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.870 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:58.870 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.386902+0000", 2024-09-04T17:47:58.870 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7500000000000003e-06 2024-09-04T17:47:58.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.871 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.871 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:58.871 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.386908+0000", 2024-09-04T17:47:58.871 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8819999999999998e-06 2024-09-04T17:47:58.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.871 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.871 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:58.871 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.402605+0000", 2024-09-04T17:47:58.871 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015696882999999998 2024-09-04T17:47:58.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.871 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.871 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:58.871 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.402679+0000", 2024-09-04T17:47:58.871 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3715999999999997e-05 2024-09-04T17:47:58.872 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.872 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.872 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:58.872 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.402813+0000", 2024-09-04T17:47:58.872 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013409300000000001 2024-09-04T17:47:58.872 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.872 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:58.872 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.872 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.872 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.872 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:77753 3.1e 3:7bc91465:::rbd_data.11346ecfd364.000000000000000e:26 [sparse-read 6291456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-04T17:47:58.872 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:49.387321+0000", 2024-09-04T17:47:58.872 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.4025020999999995, 2024-09-04T17:47:58.873 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015754196000000002, 2024-09-04T17:47:58.873 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:58.873 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:58.873 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:58.873 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:58.873 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:58.873 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:58.873 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77753 2024-09-04T17:47:58.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.873 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:58.873 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.873 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:58.873 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.387321+0000", 2024-09-04T17:47:58.873 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.874 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:58.874 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.387321+0000", 2024-09-04T17:47:58.874 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.874 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:58.874 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.387322+0000", 2024-09-04T17:47:58.874 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9100000000000004e-07 2024-09-04T17:47:58.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.874 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:58.875 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.387324+0000", 2024-09-04T17:47:58.875 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3819999999999998e-06 2024-09-04T17:47:58.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.875 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.875 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:58.875 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.387332+0000", 2024-09-04T17:47:58.875 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7239999999999992e-06 2024-09-04T17:47:58.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.875 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.875 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:58.875 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.387337+0000", 2024-09-04T17:47:58.875 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7989999999999999e-06 2024-09-04T17:47:58.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.875 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.876 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:58.876 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.402845+0000", 2024-09-04T17:47:58.876 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015507811999999999 2024-09-04T17:47:58.876 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.876 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.876 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:58.876 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.402925+0000", 2024-09-04T17:47:58.876 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9973000000000002e-05 2024-09-04T17:47:58.876 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.876 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.876 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:58.876 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.403075+0000", 2024-09-04T17:47:58.876 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015001500000000001 2024-09-04T17:47:58.876 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.877 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:58.877 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.877 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.877 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.877 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:77754 3.1e 3:7bc91465:::rbd_data.11346ecfd364.000000000000000e:26 [sparse-read 6684672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-04T17:47:58.877 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:49.387346+0000", 2024-09-04T17:47:58.877 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.4024775349999992, 2024-09-04T17:47:58.877 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015977949000000002, 2024-09-04T17:47:58.877 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:58.877 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:58.877 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:58.877 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:58.877 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:58.878 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:58.878 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77754 2024-09-04T17:47:58.878 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.878 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:58.878 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.878 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:58.878 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.387346+0000", 2024-09-04T17:47:58.878 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.878 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.878 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.878 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:58.878 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.387346+0000", 2024-09-04T17:47:58.878 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.878 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.879 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.879 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:58.879 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.387346+0000", 2024-09-04T17:47:58.879 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5499999999999998e-07 2024-09-04T17:47:58.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.879 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.879 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:58.879 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.387349+0000", 2024-09-04T17:47:58.879 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.717e-06 2024-09-04T17:47:58.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.879 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.879 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:58.879 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.387356+0000", 2024-09-04T17:47:58.879 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4739999999999999e-06 2024-09-04T17:47:58.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.880 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.880 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:58.880 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.387362+0000", 2024-09-04T17:47:58.880 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0589999999999999e-06 2024-09-04T17:47:58.880 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.880 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.880 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:58.880 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.403110+0000", 2024-09-04T17:47:58.880 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015747275000000002 2024-09-04T17:47:58.880 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.880 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.880 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:58.880 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.403187+0000", 2024-09-04T17:47:58.881 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7304000000000004e-05 2024-09-04T17:47:58.881 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.881 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.881 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:58.881 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.403324+0000", 2024-09-04T17:47:58.881 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000136565 2024-09-04T17:47:58.881 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.881 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:58.881 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.881 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.881 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.881 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:77891 3.1e 3:7bc91465:::rbd_data.11346ecfd364.000000000000000e:26 [sparse-read 7208960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-04T17:47:58.881 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:49.392196+0000", 2024-09-04T17:47:58.882 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.3976274429999993, 2024-09-04T17:47:58.882 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014895824, 2024-09-04T17:47:58.882 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:58.882 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:58.882 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:58.882 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:58.882 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:58.882 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:58.882 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77891 2024-09-04T17:47:58.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.882 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:58.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.882 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:58.882 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.392196+0000", 2024-09-04T17:47:58.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.883 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.883 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:58.883 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.392196+0000", 2024-09-04T17:47:58.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.883 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.883 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:58.883 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.392196+0000", 2024-09-04T17:47:58.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.68e-07 2024-09-04T17:47:58.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.883 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.883 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:58.884 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.392199+0000", 2024-09-04T17:47:58.884 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6769999999999999e-06 2024-09-04T17:47:58.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.884 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:58.884 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.392207+0000", 2024-09-04T17:47:58.884 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2970000000000004e-06 2024-09-04T17:47:58.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.884 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:58.884 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.392213+0000", 2024-09-04T17:47:58.884 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1879999999999999e-06 2024-09-04T17:47:58.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.885 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.885 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:58.885 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.406872+0000", 2024-09-04T17:47:58.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014658768000000001 2024-09-04T17:47:58.885 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.885 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.885 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:58.885 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.406950+0000", 2024-09-04T17:47:58.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7688999999999994e-05 2024-09-04T17:47:58.885 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.885 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.885 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:58.885 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.407092+0000", 2024-09-04T17:47:58.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014163700000000001 2024-09-04T17:47:58.886 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.886 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:58.886 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.886 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.886 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.886 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:77893 3.1e 3:7bc91465:::rbd_data.11346ecfd364.000000000000000e:26 [sparse-read 7602176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-04T17:47:58.886 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:49.392221+0000", 2024-09-04T17:47:58.886 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.3976016799999993, 2024-09-04T17:47:58.886 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015109442000000001, 2024-09-04T17:47:58.886 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:58.886 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:58.886 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:58.886 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:58.887 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:58.887 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:58.887 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77893 2024-09-04T17:47:58.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.887 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:58.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.887 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:58.887 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.392221+0000", 2024-09-04T17:47:58.887 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.887 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:58.887 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.392221+0000", 2024-09-04T17:47:58.887 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.888 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.888 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.888 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:58.888 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.392222+0000", 2024-09-04T17:47:58.888 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9100000000000004e-07 2024-09-04T17:47:58.888 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.888 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.888 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:58.888 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.392239+0000", 2024-09-04T17:47:58.888 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6888000000000002e-05 2024-09-04T17:47:58.888 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.888 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.888 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:58.888 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.392248+0000", 2024-09-04T17:47:58.889 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8869999999999998e-06 2024-09-04T17:47:58.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.889 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.889 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:58.889 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.392254+0000", 2024-09-04T17:47:58.889 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5760000000000002e-06 2024-09-04T17:47:58.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.889 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.889 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:58.889 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.407120+0000", 2024-09-04T17:47:58.889 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014865991 2024-09-04T17:47:58.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.889 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.889 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:58.890 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.407195+0000", 2024-09-04T17:47:58.890 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4314999999999996e-05 2024-09-04T17:47:58.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.890 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.890 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:58.890 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.407331+0000", 2024-09-04T17:47:58.890 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013629399999999999 2024-09-04T17:47:58.890 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.890 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:58.890 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.890 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.890 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:77894 3.1e 3:7bc91465:::rbd_data.11346ecfd364.000000000000000e:26 [sparse-read 6619136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-04T17:47:58.891 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:49.392264+0000", 2024-09-04T17:47:58.891 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.3975588529999996, 2024-09-04T17:47:58.891 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015316363, 2024-09-04T17:47:58.891 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:58.891 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:58.891 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:58.891 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:58.891 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:58.891 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:58.891 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 77894 2024-09-04T17:47:58.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.891 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:58.891 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.892 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:58.892 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.392264+0000", 2024-09-04T17:47:58.892 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.892 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.892 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.892 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:58.892 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.392264+0000", 2024-09-04T17:47:58.892 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:58.892 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.892 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.892 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:58.892 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.392265+0000", 2024-09-04T17:47:58.892 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8599999999999998e-07 2024-09-04T17:47:58.892 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.893 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.893 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:58.893 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.392267+0000", 2024-09-04T17:47:58.893 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.582e-06 2024-09-04T17:47:58.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.893 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.893 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:58.893 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.392276+0000", 2024-09-04T17:47:58.893 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1100000000000003e-06 2024-09-04T17:47:58.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.893 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.893 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:58.893 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.392288+0000", 2024-09-04T17:47:58.893 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2330999999999999e-05 2024-09-04T17:47:58.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.894 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:58.894 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.407360+0000", 2024-09-04T17:47:58.894 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015071635 2024-09-04T17:47:58.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.894 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:58.894 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.407434+0000", 2024-09-04T17:47:58.894 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4547000000000003e-05 2024-09-04T17:47:58.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:58.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:58.894 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:58.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:49.407581+0000", 2024-09-04T17:47:58.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014657200000000001 2024-09-04T17:47:58.895 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.895 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:58.895 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.895 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:58.895 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:58.895 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:58.897 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-04T17:47:58.897 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is set 2024-09-04T17:47:58.898 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:58.890+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:58.997 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:47:58.996+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:47:59.082 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:59.082 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:47:59.082 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:47:59.082 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:59.099 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:59.094+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:59.100 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-04T17:47:59.200 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:59.196+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:59.258 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #41, image image_client.0-clone41 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent42 2024-09-04T17:47:59.269 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:59.270 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:47:59.270 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:47:59.270 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:47:59.270 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:59.287 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-04T17:47:59.297 INFO:teuthology.orchestra.run.smithi092.stdout:5905 writesame 0x9afe3cf thru 0x9b08078 data_size 0xb(0x9caa bytes) 2024-09-04T17:47:59.297 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:59.297 INFO:teuthology.orchestra.run.smithi092.stdout:5910 punch from 0x2874c05 to 0x287f502, (0xa8fd bytes) 2024-09-04T17:47:59.300 INFO:teuthology.orchestra.run.smithi092.stdout:5911 trunc from 0x9b08079 to 0xb8e915e 2024-09-04T17:47:59.300 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:47:59.296+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:47:59.319 INFO:teuthology.orchestra.run.smithi092.stdout:5912 write 0xc7e8af6 thru 0xc7edc08 (0x5113 bytes) 2024-09-04T17:47:59.326 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:59.326 INFO:teuthology.orchestra.run.smithi092.stdout:5916 read 0x4d668d8 thru 0x4d687ed (0x1f16 bytes) 2024-09-04T17:47:59.328 INFO:teuthology.orchestra.run.smithi092.stdout:5917 trunc from 0xc7edc09 to 0xcdd98a4 2024-09-04T17:47:59.337 INFO:teuthology.orchestra.run.smithi092.stdout:5919 write 0x14ff660 thru 0x15036d4 (0x4075 bytes) 2024-09-04T17:47:59.357 INFO:teuthology.orchestra.run.smithi092.stdout:5920 punch from 0x327dea8 to 0x3286a8c, (0x8be4 bytes) 2024-09-04T17:47:59.357 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:47:59.357 INFO:teuthology.orchestra.run.smithi092.stdout:5923 punch from 0x16f161f to 0x16fca5e, (0xb43f bytes) 2024-09-04T17:47:59.357 INFO:teuthology.orchestra.run.smithi092.stdout:5924 read 0xb487f35 thru 0xb489aca (0x1b96 bytes) 2024-09-04T17:47:59.357 INFO:teuthology.orchestra.run.smithi092.stdout:5925 read 0x845d78a thru 0x8466325 (0x8b9c bytes) 2024-09-04T17:47:59.358 INFO:teuthology.orchestra.run.smithi092.stdout:5926 write 0x74ed1b6 thru 0x74f536e (0x81b9 bytes) 2024-09-04T17:47:59.369 INFO:teuthology.orchestra.run.smithi092.stdout:5927 writesame 0x39f2511 thru 0x39f9200 data_size 0x54(0x6cf0 bytes) 2024-09-04T17:47:59.383 INFO:teuthology.orchestra.run.smithi092.stdout:5928 flatten 2024-09-04T17:47:59.402 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:59.398+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:59.452 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:59.452 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:47:59.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:47:59.452 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:47:59.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.452 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:48444 3.12 3:4ff80fb5:::rbd_data.11346a2b0bc9.000000000000001d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-04T17:47:59.452 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:58.122601+0000", 2024-09-04T17:47:59.452 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 61.327850278, 2024-09-04T17:47:59.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057065713999999997, 2024-09-04T17:47:59.452 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:59.453 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:59.453 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:59.453 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:59.453 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:59.453 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:59.453 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 48444 2024-09-04T17:47:59.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.453 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:59.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.453 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:59.453 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122601+0000", 2024-09-04T17:47:59.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:59.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.454 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:59.454 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122601+0000", 2024-09-04T17:47:59.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:59.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.454 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:59.454 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122603+0000", 2024-09-04T17:47:59.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3499999999999999e-06 2024-09-04T17:47:59.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.454 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:59.454 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122614+0000", 2024-09-04T17:47:59.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.063e-05 2024-09-04T17:47:59.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:59.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122643+0000", 2024-09-04T17:47:59.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9434999999999999e-05 2024-09-04T17:47:59.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:59.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122657+0000", 2024-09-04T17:47:59.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3920000000000001e-05 2024-09-04T17:47:59.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:59.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122767+0000", 2024-09-04T17:47:59.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010981599999999999 2024-09-04T17:47:59.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:59.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122989+0000", 2024-09-04T17:47:59.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022158200000000001 2024-09-04T17:47:59.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 3", 2024-09-04T17:47:59.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.133349+0000", 2024-09-04T17:47:59.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010360729000000001 2024-09-04T17:47:59.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:59.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.179423+0000", 2024-09-04T17:47:59.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046073408000000003 2024-09-04T17:47:59.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:59.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.179453+0000", 2024-09-04T17:47:59.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0229000000000001e-05 2024-09-04T17:47:59.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:59.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.179545+0000", 2024-09-04T17:47:59.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1580000000000001e-05 2024-09-04T17:47:59.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.458 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:59.458 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.179667+0000", 2024-09-04T17:47:59.458 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012203500000000001 2024-09-04T17:47:59.458 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:59.458 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:59.458 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:59.458 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.458 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.458 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:55376 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-04T17:47:59.458 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:11.974053+0000", 2024-09-04T17:47:59.458 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.476398494000001, 2024-09-04T17:47:59.458 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060514722, 2024-09-04T17:47:59.458 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:59.459 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:59.459 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:59.459 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:59.459 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:59.459 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:59.459 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 55376 2024-09-04T17:47:59.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.459 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:59.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:59.459 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974053+0000", 2024-09-04T17:47:59.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:59.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:59.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974053+0000", 2024-09-04T17:47:59.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:59.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:59.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974054+0000", 2024-09-04T17:47:59.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.841e-06 2024-09-04T17:47:59.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:59.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974062+0000", 2024-09-04T17:47:59.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5460000000000003e-06 2024-09-04T17:47:59.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:59.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974077+0000", 2024-09-04T17:47:59.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5389000000000001e-05 2024-09-04T17:47:59.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:59.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974085+0000", 2024-09-04T17:47:59.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7659999999999998e-06 2024-09-04T17:47:59.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.462 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:59.462 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974188+0000", 2024-09-04T17:47:59.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000102521 2024-09-04T17:47:59.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.462 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:59.462 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974321+0000", 2024-09-04T17:47:59.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013324699999999999 2024-09-04T17:47:59.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.462 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-04T17:47:59.462 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.986481+0000", 2024-09-04T17:47:59.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012160450999999999 2024-09-04T17:47:59.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.463 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:59.463 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:12.034415+0000", 2024-09-04T17:47:59.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.047934065999999997 2024-09-04T17:47:59.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.463 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:59.463 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:12.034438+0000", 2024-09-04T17:47:59.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2923999999999999e-05 2024-09-04T17:47:59.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.463 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:59.463 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:12.034505+0000", 2024-09-04T17:47:59.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6711000000000006e-05 2024-09-04T17:47:59.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.464 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:59.464 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:12.034567+0000", 2024-09-04T17:47:59.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2260000000000004e-05 2024-09-04T17:47:59.464 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:59.464 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:59.464 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:59.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.464 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61139 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 2547712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:59.464 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819750+0000", 2024-09-04T17:47:59.464 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 45.630700779999998, 2024-09-04T17:47:59.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042185699, 2024-09-04T17:47:59.465 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:59.465 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:59.465 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:59.465 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:59.465 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:59.465 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:59.465 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61139 2024-09-04T17:47:59.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.465 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:59.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:59.465 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819750+0000", 2024-09-04T17:47:59.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:59.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:59.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819750+0000", 2024-09-04T17:47:59.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:59.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:59.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819751+0000", 2024-09-04T17:47:59.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4000000000000002e-07 2024-09-04T17:47:59.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:59.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819753+0000", 2024-09-04T17:47:59.467 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5179999999999999e-06 2024-09-04T17:47:59.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.467 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.467 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:59.467 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819760+0000", 2024-09-04T17:47:59.467 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5440000000000004e-06 2024-09-04T17:47:59.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.467 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.467 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:59.467 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819765+0000", 2024-09-04T17:47:59.467 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8760000000000004e-06 2024-09-04T17:47:59.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.467 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.468 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:59.468 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.861622+0000", 2024-09-04T17:47:59.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.041857093999999997 2024-09-04T17:47:59.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.468 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:59.468 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.861698+0000", 2024-09-04T17:47:59.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6080000000000003e-05 2024-09-04T17:47:59.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.468 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:59.468 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.861936+0000", 2024-09-04T17:47:59.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023804699999999999 2024-09-04T17:47:59.469 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:59.469 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:59.469 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:59.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.469 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61140 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 2686976~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:59.469 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819772+0000", 2024-09-04T17:47:59.469 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 45.630678646, 2024-09-04T17:47:59.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042515052999999997, 2024-09-04T17:47:59.469 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:59.469 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:59.469 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:59.469 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:59.470 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:59.470 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:59.470 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61140 2024-09-04T17:47:59.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.470 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:59.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.470 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:59.470 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819772+0000", 2024-09-04T17:47:59.470 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:59.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.470 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:59.470 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819772+0000", 2024-09-04T17:47:59.470 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:59.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:59.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819773+0000", 2024-09-04T17:47:59.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2900000000000003e-07 2024-09-04T17:47:59.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:59.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819775+0000", 2024-09-04T17:47:59.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3309999999999998e-06 2024-09-04T17:47:59.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:59.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819781+0000", 2024-09-04T17:47:59.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.981e-06 2024-09-04T17:47:59.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.472 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:59.472 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819786+0000", 2024-09-04T17:47:59.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5719999999999996e-06 2024-09-04T17:47:59.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.472 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:59.472 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.861966+0000", 2024-09-04T17:47:59.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042179541000000001 2024-09-04T17:47:59.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.473 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:59.473 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862038+0000", 2024-09-04T17:47:59.473 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2719000000000006e-05 2024-09-04T17:47:59.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.473 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:59.473 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862288+0000", 2024-09-04T17:47:59.473 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024928 2024-09-04T17:47:59.473 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:59.473 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:59.473 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:59.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.473 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61141 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 2793472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:59.474 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819909+0000", 2024-09-04T17:47:59.474 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 45.630542501999997, 2024-09-04T17:47:59.474 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042786419999999999, 2024-09-04T17:47:59.474 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:59.474 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:59.474 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:59.474 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:59.474 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:59.474 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:59.474 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61141 2024-09-04T17:47:59.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.475 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:59.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:59.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819909+0000", 2024-09-04T17:47:59.476 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:59.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.477 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:59.479 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819909+0000", 2024-09-04T17:47:59.479 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:59.479 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.479 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:59.479 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819909+0000", 2024-09-04T17:47:59.480 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6300000000000005e-07 2024-09-04T17:47:59.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.480 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.480 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:59.480 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819913+0000", 2024-09-04T17:47:59.480 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4189999999999998e-06 2024-09-04T17:47:59.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.480 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.480 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:59.480 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819920+0000", 2024-09-04T17:47:59.480 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8619999999999997e-06 2024-09-04T17:47:59.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.480 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.480 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:59.481 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819926+0000", 2024-09-04T17:47:59.481 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9399999999999999e-06 2024-09-04T17:47:59.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.481 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:59.481 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862331+0000", 2024-09-04T17:47:59.481 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042405348000000002 2024-09-04T17:47:59.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.481 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:59.481 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862410+0000", 2024-09-04T17:47:59.481 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9078000000000004e-05 2024-09-04T17:47:59.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.481 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:59.482 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862695+0000", 2024-09-04T17:47:59.482 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028510999999999999 2024-09-04T17:47:59.482 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:59.482 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:59.482 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:59.482 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.482 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.482 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61142 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 2932736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:59.482 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819934+0000", 2024-09-04T17:47:59.482 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 45.630516884000002, 2024-09-04T17:47:59.483 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043101985000000002, 2024-09-04T17:47:59.483 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:59.483 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:59.483 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:59.483 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:59.483 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:59.483 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:59.483 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61142 2024-09-04T17:47:59.483 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.483 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:59.483 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.483 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:59.484 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819934+0000", 2024-09-04T17:47:59.484 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:59.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.484 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.484 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:59.484 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819934+0000", 2024-09-04T17:47:59.484 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:59.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.484 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.484 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:59.484 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819935+0000", 2024-09-04T17:47:59.484 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2024-09-04T17:47:59.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.484 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.485 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:59.485 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819937+0000", 2024-09-04T17:47:59.485 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5770000000000001e-06 2024-09-04T17:47:59.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.485 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.485 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:59.485 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819943+0000", 2024-09-04T17:47:59.485 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7950000000000004e-06 2024-09-04T17:47:59.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.485 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.485 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:59.485 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819948+0000", 2024-09-04T17:47:59.485 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6779999999999999e-06 2024-09-04T17:47:59.486 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.486 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.486 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:59.486 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862739+0000", 2024-09-04T17:47:59.486 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042791454999999999 2024-09-04T17:47:59.486 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.486 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.486 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:59.486 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862824+0000", 2024-09-04T17:47:59.486 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4790999999999995e-05 2024-09-04T17:47:59.486 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.486 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.487 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:59.487 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.863036+0000", 2024-09-04T17:47:59.487 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021224899999999999 2024-09-04T17:47:59.487 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:59.487 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:59.487 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:59.487 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.488 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.488 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61143 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 3039232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:47:59.488 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819954+0000", 2024-09-04T17:47:59.488 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 45.630496766, 2024-09-04T17:47:59.488 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043485134000000002, 2024-09-04T17:47:59.488 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:59.488 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:59.488 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:59.488 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:59.489 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:59.489 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:59.489 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61143 2024-09-04T17:47:59.489 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.489 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:59.489 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.489 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:59.489 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819954+0000", 2024-09-04T17:47:59.490 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:59.490 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.490 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.490 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:59.490 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819954+0000", 2024-09-04T17:47:59.490 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:59.490 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.490 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.490 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:59.490 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819955+0000", 2024-09-04T17:47:59.490 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7300000000000002e-07 2024-09-04T17:47:59.490 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.490 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.491 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:59.491 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819957+0000", 2024-09-04T17:47:59.491 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.024e-06 2024-09-04T17:47:59.491 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.491 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.491 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:59.491 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819963+0000", 2024-09-04T17:47:59.491 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8109999999999999e-06 2024-09-04T17:47:59.491 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.491 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.491 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:59.491 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819967+0000", 2024-09-04T17:47:59.491 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6149999999999999e-06 2024-09-04T17:47:59.491 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.492 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.492 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:59.492 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.863082+0000", 2024-09-04T17:47:59.492 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043115078000000001 2024-09-04T17:47:59.492 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.492 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.492 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:59.492 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.863172+0000", 2024-09-04T17:47:59.492 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0181999999999998e-05 2024-09-04T17:47:59.492 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.492 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.492 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:59.492 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.863440+0000", 2024-09-04T17:47:59.493 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026705099999999997 2024-09-04T17:47:59.493 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:59.493 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:59.493 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:59.493 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.493 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.493 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:70015 3.6 3:6012e780:::rbd_data.11341aa245a3.000000000000005b:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-04T17:47:59.493 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:26.155497+0000", 2024-09-04T17:47:59.493 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 33.294953708999998, 2024-09-04T17:47:59.493 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.049007124999999999, 2024-09-04T17:47:59.493 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:59.493 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:59.493 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:59.493 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:59.494 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:59.494 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:59.494 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 70015 2024-09-04T17:47:59.494 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.494 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:59.494 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.494 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:59.494 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155497+0000", 2024-09-04T17:47:59.494 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:59.494 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.494 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.494 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:59.494 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155497+0000", 2024-09-04T17:47:59.495 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:59.495 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.495 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.495 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:59.495 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155500+0000", 2024-09-04T17:47:59.495 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7070000000000001e-06 2024-09-04T17:47:59.495 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.495 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.495 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:59.495 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155513+0000", 2024-09-04T17:47:59.495 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2989e-05 2024-09-04T17:47:59.495 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.495 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.496 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:59.496 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155544+0000", 2024-09-04T17:47:59.496 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0747000000000002e-05 2024-09-04T17:47:59.496 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.496 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.496 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:59.496 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155560+0000", 2024-09-04T17:47:59.496 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5866e-05 2024-09-04T17:47:59.496 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.496 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.496 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:59.496 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155675+0000", 2024-09-04T17:47:59.496 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000114838 2024-09-04T17:47:59.497 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.497 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.497 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:59.497 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155803+0000", 2024-09-04T17:47:59.497 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012851799999999999 2024-09-04T17:47:59.497 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.497 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.497 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 1", 2024-09-04T17:47:59.497 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.163582+0000", 2024-09-04T17:47:59.497 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0077790960000000001 2024-09-04T17:47:59.497 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.497 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.498 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:59.498 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.204055+0000", 2024-09-04T17:47:59.498 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.040472517 2024-09-04T17:47:59.498 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.498 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.498 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:59.498 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.204195+0000", 2024-09-04T17:47:59.498 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014055200000000001 2024-09-04T17:47:59.498 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.498 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.498 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:59.498 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.204373+0000", 2024-09-04T17:47:59.499 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000177784 2024-09-04T17:47:59.499 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.499 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.499 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:59.499 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.204505+0000", 2024-09-04T17:47:59.499 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013151100000000001 2024-09-04T17:47:59.499 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:59.499 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:59.499 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:59.499 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.499 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.500 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:70434 3.12 3:4cd2342b:::rbd_data.11341aa245a3.0000000000000091:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-04T17:47:59.500 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:26.943124+0000", 2024-09-04T17:47:59.500 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.507326927000001, 2024-09-04T17:47:59.500 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055011831999999997, 2024-09-04T17:47:59.500 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:59.500 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:59.500 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:47:59.500 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:59.500 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:59.500 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:59.500 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 70434 2024-09-04T17:47:59.500 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.500 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:59.501 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.501 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:59.501 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943124+0000", 2024-09-04T17:47:59.501 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:59.501 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.501 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.501 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:59.501 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943124+0000", 2024-09-04T17:47:59.501 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:59.501 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.501 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.501 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:59.501 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943128+0000", 2024-09-04T17:47:59.502 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8460000000000001e-06 2024-09-04T17:47:59.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.502 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:59.502 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943142+0000", 2024-09-04T17:47:59.502 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4089e-05 2024-09-04T17:47:59.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.502 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:59.502 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943177+0000", 2024-09-04T17:47:59.503 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5167999999999999e-05 2024-09-04T17:47:59.503 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.503 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.503 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:59.503 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943194+0000", 2024-09-04T17:47:59.503 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6235e-05 2024-09-04T17:47:59.503 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.503 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.503 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:59.503 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943286+0000", 2024-09-04T17:47:59.503 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2793000000000004e-05 2024-09-04T17:47:59.503 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.503 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.504 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:59.504 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943526+0000", 2024-09-04T17:47:59.504 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023950999999999999 2024-09-04T17:47:59.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.504 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.504 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 1", 2024-09-04T17:47:59.504 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.952871+0000", 2024-09-04T17:47:59.504 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0093453070000000006 2024-09-04T17:47:59.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.504 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.504 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:47:59.504 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.997968+0000", 2024-09-04T17:47:59.504 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045096613000000001 2024-09-04T17:47:59.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.505 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:47:59.505 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.998006+0000", 2024-09-04T17:47:59.505 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8257000000000003e-05 2024-09-04T17:47:59.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.505 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:47:59.505 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.998073+0000", 2024-09-04T17:47:59.505 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6803000000000004e-05 2024-09-04T17:47:59.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.505 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:59.506 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.998136+0000", 2024-09-04T17:47:59.506 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3211000000000002e-05 2024-09-04T17:47:59.506 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:59.506 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:59.506 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:59.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.506 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.506 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93839 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 4849664~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:47:59.506 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386439+0000", 2024-09-04T17:47:59.506 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.064012150000000004, 2024-09-04T17:47:59.506 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042005084999999998, 2024-09-04T17:47:59.507 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:59.507 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:59.507 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:59.507 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:59.507 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:59.507 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:59.507 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93839 2024-09-04T17:47:59.507 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.507 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:59.507 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.507 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:59.507 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386439+0000", 2024-09-04T17:47:59.507 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:59.507 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.508 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.508 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:59.508 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386439+0000", 2024-09-04T17:47:59.508 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:59.508 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.508 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.508 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:59.508 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386440+0000", 2024-09-04T17:47:59.508 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9200000000000001e-07 2024-09-04T17:47:59.508 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.508 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.508 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:59.508 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386442+0000", 2024-09-04T17:47:59.508 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4449999999999999e-06 2024-09-04T17:47:59.509 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.509 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.509 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:59.509 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386452+0000", 2024-09-04T17:47:59.509 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9070000000000002e-06 2024-09-04T17:47:59.509 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.509 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.509 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:59.509 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386458+0000", 2024-09-04T17:47:59.509 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8850000000000004e-06 2024-09-04T17:47:59.509 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.509 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.509 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:59.509 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.427528+0000", 2024-09-04T17:47:59.510 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.041070137 2024-09-04T17:47:59.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.510 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:59.510 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.427571+0000", 2024-09-04T17:47:59.510 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2728999999999997e-05 2024-09-04T17:47:59.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.510 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:59.510 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.428444+0000", 2024-09-04T17:47:59.510 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00087339000000000004 2024-09-04T17:47:59.510 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:59.510 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:59.510 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:59.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.511 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.511 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93840 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 4980736~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:47:59.511 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386465+0000", 2024-09-04T17:47:59.511 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.063985863000000004, 2024-09-04T17:47:59.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042435764000000001, 2024-09-04T17:47:59.511 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:59.511 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:59.511 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:59.511 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:59.511 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:59.511 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:59.511 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93840 2024-09-04T17:47:59.511 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.511 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:59.511 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.512 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:59.512 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386465+0000", 2024-09-04T17:47:59.512 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:59.512 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.512 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.512 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:59.512 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386465+0000", 2024-09-04T17:47:59.512 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:59.512 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.512 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.512 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:59.512 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386466+0000", 2024-09-04T17:47:59.513 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4700000000000001e-07 2024-09-04T17:47:59.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.513 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.513 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:59.513 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386468+0000", 2024-09-04T17:47:59.513 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1600000000000001e-06 2024-09-04T17:47:59.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.513 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.513 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:59.513 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386477+0000", 2024-09-04T17:47:59.513 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7839999999999992e-06 2024-09-04T17:47:59.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.513 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.513 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:59.513 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386482+0000", 2024-09-04T17:47:59.514 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3970000000000002e-06 2024-09-04T17:47:59.514 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.514 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.514 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:59.514 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.428466+0000", 2024-09-04T17:47:59.514 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.041984279999999999 2024-09-04T17:47:59.514 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.514 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.514 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:59.514 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.428520+0000", 2024-09-04T17:47:59.514 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3328000000000002e-05 2024-09-04T17:47:59.514 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.514 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.514 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:59.514 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.428901+0000", 2024-09-04T17:47:59.515 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00038126799999999998 2024-09-04T17:47:59.515 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:59.515 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:59.515 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:59.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.515 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93841 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 5111808~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:47:59.515 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386491+0000", 2024-09-04T17:47:59.515 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.063959730000000006, 2024-09-04T17:47:59.515 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043069578999999997, 2024-09-04T17:47:59.515 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:59.515 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:59.516 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:59.516 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:59.516 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:59.516 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:59.516 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93841 2024-09-04T17:47:59.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.516 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:59.516 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.517 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:59.517 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386491+0000", 2024-09-04T17:47:59.517 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:59.517 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.517 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.517 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:59.517 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386491+0000", 2024-09-04T17:47:59.517 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:59.517 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.517 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.517 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:59.517 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386492+0000", 2024-09-04T17:47:59.517 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0299999999999999e-07 2024-09-04T17:47:59.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.518 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.518 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:59.518 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386499+0000", 2024-09-04T17:47:59.518 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6089999999999997e-06 2024-09-04T17:47:59.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.518 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.518 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:59.519 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386506+0000", 2024-09-04T17:47:59.519 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3599999999999998e-06 2024-09-04T17:47:59.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.520 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.520 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:59.520 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386514+0000", 2024-09-04T17:47:59.520 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9379999999999992e-06 2024-09-04T17:47:59.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.520 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.520 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:59.520 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.428957+0000", 2024-09-04T17:47:59.520 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042442831 2024-09-04T17:47:59.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.520 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.520 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:59.520 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.429080+0000", 2024-09-04T17:47:59.521 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012359299999999999 2024-09-04T17:47:59.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.521 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.521 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:59.521 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.429561+0000", 2024-09-04T17:47:59.521 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00048074499999999998 2024-09-04T17:47:59.521 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:59.521 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:59.521 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:59.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.521 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.521 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93842 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 4259840~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:47:59.521 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386521+0000", 2024-09-04T17:47:59.522 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.063929797999999996, 2024-09-04T17:47:59.522 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043661933999999999, 2024-09-04T17:47:59.522 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:59.522 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:59.522 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:59.522 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:59.522 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:59.522 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:59.522 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93842 2024-09-04T17:47:59.522 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.522 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:59.523 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.523 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:59.523 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386521+0000", 2024-09-04T17:47:59.523 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:59.523 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.523 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.523 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:59.523 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386521+0000", 2024-09-04T17:47:59.523 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:59.523 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.523 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.523 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:59.524 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386522+0000", 2024-09-04T17:47:59.524 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4700000000000001e-07 2024-09-04T17:47:59.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.524 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.524 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:59.524 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386524+0000", 2024-09-04T17:47:59.524 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4559999999999999e-06 2024-09-04T17:47:59.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.524 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.524 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:59.524 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386534+0000", 2024-09-04T17:47:59.524 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2639999999999997e-06 2024-09-04T17:47:59.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.525 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.525 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:59.525 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386539+0000", 2024-09-04T17:47:59.525 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6130000000000003e-06 2024-09-04T17:47:59.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.525 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.525 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:59.525 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.429577+0000", 2024-09-04T17:47:59.526 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043037757000000003 2024-09-04T17:47:59.526 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.526 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.526 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:59.526 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.429638+0000", 2024-09-04T17:47:59.526 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1235999999999994e-05 2024-09-04T17:47:59.526 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.526 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.526 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:59.526 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.430183+0000", 2024-09-04T17:47:59.526 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000545061 2024-09-04T17:47:59.526 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:59.526 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:59.526 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:59.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.527 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.527 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93843 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 4390912~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:47:59.527 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386547+0000", 2024-09-04T17:47:59.527 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.063904560999999999, 2024-09-04T17:47:59.527 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044260668000000003, 2024-09-04T17:47:59.527 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:59.527 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:59.527 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:59.527 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:59.527 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:59.527 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:59.527 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93843 2024-09-04T17:47:59.528 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.528 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:59.528 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.528 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:59.528 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386547+0000", 2024-09-04T17:47:59.528 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:59.528 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.528 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.528 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:59.528 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386547+0000", 2024-09-04T17:47:59.528 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:59.528 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.528 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.529 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:59.529 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386547+0000", 2024-09-04T17:47:59.529 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4199999999999996e-07 2024-09-04T17:47:59.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.529 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.529 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:59.529 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386550+0000", 2024-09-04T17:47:59.529 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6199999999999999e-06 2024-09-04T17:47:59.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.529 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.529 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:59.529 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386557+0000", 2024-09-04T17:47:59.529 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1919999999999999e-06 2024-09-04T17:47:59.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.530 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.530 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:59.530 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386562+0000", 2024-09-04T17:47:59.530 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2580000000000003e-06 2024-09-04T17:47:59.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.530 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.530 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:59.530 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.430226+0000", 2024-09-04T17:47:59.530 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043663484000000002 2024-09-04T17:47:59.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.531 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.531 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:59.531 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.430286+0000", 2024-09-04T17:47:59.531 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0455999999999999e-05 2024-09-04T17:47:59.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.531 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.531 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:59.531 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.430807+0000", 2024-09-04T17:47:59.531 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00052111599999999996 2024-09-04T17:47:59.531 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:59.531 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:59.531 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:59.532 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.532 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.532 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93844 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 4521984~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:47:59.532 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386603+0000", 2024-09-04T17:47:59.532 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.063848509999999997, 2024-09-04T17:47:59.532 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044705350999999997, 2024-09-04T17:47:59.532 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:59.532 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:59.532 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:59.532 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:59.532 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:59.532 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:59.532 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93844 2024-09-04T17:47:59.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.533 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:59.533 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.533 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:59.533 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386603+0000", 2024-09-04T17:47:59.533 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:59.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.533 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.533 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:59.533 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386603+0000", 2024-09-04T17:47:59.533 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:59.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.533 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.534 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:59.534 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386603+0000", 2024-09-04T17:47:59.534 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.92e-07 2024-09-04T17:47:59.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.534 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.534 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:59.534 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386606+0000", 2024-09-04T17:47:59.534 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5500000000000001e-06 2024-09-04T17:47:59.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.534 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.534 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:59.534 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386614+0000", 2024-09-04T17:47:59.534 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9570000000000002e-06 2024-09-04T17:47:59.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:59.535 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386619+0000", 2024-09-04T17:47:59.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5119999999999999e-06 2024-09-04T17:47:59.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:59.535 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.430828+0000", 2024-09-04T17:47:59.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044208654 2024-09-04T17:47:59.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:59.535 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.430903+0000", 2024-09-04T17:47:59.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4629999999999995e-05 2024-09-04T17:47:59.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.536 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.536 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:59.536 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.431308+0000", 2024-09-04T17:47:59.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000405256 2024-09-04T17:47:59.536 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:59.536 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:59.536 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:59.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.536 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.536 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93845 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 4653056~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:47:59.536 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386627+0000", 2024-09-04T17:47:59.536 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.063823921000000006, 2024-09-04T17:47:59.537 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045262927000000001, 2024-09-04T17:47:59.537 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:59.537 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:59.537 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:59.537 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:59.537 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:59.537 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:59.537 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93845 2024-09-04T17:47:59.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.537 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:59.537 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.538 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:59.538 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386627+0000", 2024-09-04T17:47:59.538 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:59.538 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.538 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.538 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:59.538 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386627+0000", 2024-09-04T17:47:59.538 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:59.538 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.539 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:59.539 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386628+0000", 2024-09-04T17:47:59.539 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0999999999999998e-07 2024-09-04T17:47:59.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.539 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:59.539 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386630+0000", 2024-09-04T17:47:59.539 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3300000000000001e-06 2024-09-04T17:47:59.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:59.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386645+0000", 2024-09-04T17:47:59.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5186e-05 2024-09-04T17:47:59.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:59.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386651+0000", 2024-09-04T17:47:59.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7710000000000003e-06 2024-09-04T17:47:59.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:59.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.431349+0000", 2024-09-04T17:47:59.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044697723000000002 2024-09-04T17:47:59.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.541 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:59.541 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.431410+0000", 2024-09-04T17:47:59.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0677999999999997e-05 2024-09-04T17:47:59.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.541 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:59.541 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.431890+0000", 2024-09-04T17:47:59.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00048052899999999999 2024-09-04T17:47:59.541 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:59.541 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:59.542 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:59.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.542 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.542 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93846 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 4784128~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:47:59.542 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386659+0000", 2024-09-04T17:47:59.542 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.063792251999999994, 2024-09-04T17:47:59.542 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045784490999999997, 2024-09-04T17:47:59.542 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:59.542 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:59.542 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:59.542 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:59.542 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:59.542 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:59.542 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93846 2024-09-04T17:47:59.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.543 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:59.543 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.543 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:59.543 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386659+0000", 2024-09-04T17:47:59.543 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:59.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.543 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.543 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:59.543 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386659+0000", 2024-09-04T17:47:59.543 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:59.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.544 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.544 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:59.544 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386659+0000", 2024-09-04T17:47:59.544 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7000000000000005e-07 2024-09-04T17:47:59.544 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.544 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.544 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:59.544 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386662+0000", 2024-09-04T17:47:59.544 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3300000000000001e-06 2024-09-04T17:47:59.544 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.544 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.544 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:59.545 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386669+0000", 2024-09-04T17:47:59.545 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0589999999999997e-06 2024-09-04T17:47:59.545 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.545 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.545 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:59.545 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386674+0000", 2024-09-04T17:47:59.545 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3009999999999997e-06 2024-09-04T17:47:59.545 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.545 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.545 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:59.545 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.431934+0000", 2024-09-04T17:47:59.546 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045259570999999998 2024-09-04T17:47:59.546 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.546 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.546 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:59.546 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.431998+0000", 2024-09-04T17:47:59.546 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4775000000000002e-05 2024-09-04T17:47:59.546 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.546 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.547 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:59.547 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.432443+0000", 2024-09-04T17:47:59.547 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00044488500000000001 2024-09-04T17:47:59.547 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:59.547 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:59.547 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:59.547 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.547 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.547 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93847 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 4915200~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:47:59.547 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386681+0000", 2024-09-04T17:47:59.548 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.063769994999999996, 2024-09-04T17:47:59.548 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046305101000000001, 2024-09-04T17:47:59.548 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:59.548 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:59.548 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:59.548 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:59.548 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:59.548 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:59.549 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93847 2024-09-04T17:47:59.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.549 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:59.549 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.549 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:59.549 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386681+0000", 2024-09-04T17:47:59.549 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:59.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.549 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.549 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:59.549 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386681+0000", 2024-09-04T17:47:59.549 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:59.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.550 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.550 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:59.550 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386682+0000", 2024-09-04T17:47:59.550 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1699999999999998e-07 2024-09-04T17:47:59.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.550 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.550 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:59.550 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386684+0000", 2024-09-04T17:47:59.550 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.153e-06 2024-09-04T17:47:59.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.550 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.550 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:59.551 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386693+0000", 2024-09-04T17:47:59.551 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9639999999999992e-06 2024-09-04T17:47:59.551 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.551 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.551 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:59.551 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386698+0000", 2024-09-04T17:47:59.551 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3709999999999999e-06 2024-09-04T17:47:59.551 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.551 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.551 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:59.551 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.432476+0000", 2024-09-04T17:47:59.551 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045778147999999998 2024-09-04T17:47:59.551 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.551 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.551 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:59.552 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.432529+0000", 2024-09-04T17:47:59.552 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3165999999999999e-05 2024-09-04T17:47:59.552 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.552 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.552 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:59.552 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.432986+0000", 2024-09-04T17:47:59.552 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00045678199999999998 2024-09-04T17:47:59.552 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:59.552 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:59.553 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:59.553 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.553 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.553 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93848 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 5046272~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:47:59.553 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386705+0000", 2024-09-04T17:47:59.553 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.063745893999999997, 2024-09-04T17:47:59.553 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046825259000000001, 2024-09-04T17:47:59.553 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:59.554 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:59.554 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:59.554 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:59.554 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:59.554 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:59.554 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93848 2024-09-04T17:47:59.554 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.554 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:59.554 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.554 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:59.554 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386705+0000", 2024-09-04T17:47:59.554 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:59.554 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.555 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.555 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:59.555 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386705+0000", 2024-09-04T17:47:59.555 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:59.555 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.555 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.555 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:59.555 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386706+0000", 2024-09-04T17:47:59.555 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1500000000000005e-07 2024-09-04T17:47:59.555 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.555 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.555 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:59.555 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386708+0000", 2024-09-04T17:47:59.556 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2979999999999999e-06 2024-09-04T17:47:59.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.556 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.556 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:59.556 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386715+0000", 2024-09-04T17:47:59.556 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8669999999999996e-06 2024-09-04T17:47:59.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.556 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.557 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:59.557 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386720+0000", 2024-09-04T17:47:59.557 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.242e-06 2024-09-04T17:47:59.557 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.557 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.557 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:59.557 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.433019+0000", 2024-09-04T17:47:59.557 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046298359999999997 2024-09-04T17:47:59.557 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.557 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.558 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:59.558 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.433084+0000", 2024-09-04T17:47:59.558 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5395999999999995e-05 2024-09-04T17:47:59.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.558 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:59.558 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.433531+0000", 2024-09-04T17:47:59.558 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00044658099999999999 2024-09-04T17:47:59.558 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:59.558 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:59.559 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:59.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.559 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.559 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93849 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 5177344~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:47:59.559 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386727+0000", 2024-09-04T17:47:59.559 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.063723996000000005, 2024-09-04T17:47:59.559 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.047267796000000001, 2024-09-04T17:47:59.560 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:47:59.560 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:47:59.560 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:47:59.560 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:47:59.560 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:47:59.561 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:47:59.561 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93849 2024-09-04T17:47:59.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.561 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:47:59.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.561 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:47:59.561 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386727+0000", 2024-09-04T17:47:59.562 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:59.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.562 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.562 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:47:59.562 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386727+0000", 2024-09-04T17:47:59.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:47:59.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.563 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:47:59.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386728+0000", 2024-09-04T17:47:59.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1699999999999998e-07 2024-09-04T17:47:59.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:47:59.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386730+0000", 2024-09-04T17:47:59.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2589999999999999e-06 2024-09-04T17:47:59.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.565 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:47:59.565 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386751+0000", 2024-09-04T17:47:59.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1424999999999999e-05 2024-09-04T17:47:59.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.566 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:47:59.566 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386759+0000", 2024-09-04T17:47:59.566 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9400000000000002e-06 2024-09-04T17:47:59.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.566 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:47:59.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.433559+0000", 2024-09-04T17:47:59.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046799573999999997 2024-09-04T17:47:59.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:47:59.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.433610+0000", 2024-09-04T17:47:59.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1134999999999997e-05 2024-09-04T17:47:59.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:47:59.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:47:59.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:47:59.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.433995+0000", 2024-09-04T17:47:59.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000384946 2024-09-04T17:47:59.569 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:59.569 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:59.569 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:59.569 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:47:59.569 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:47:59.570 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:59.570 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-04T17:47:59.572 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:47:59.566+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:47:59.573 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-04T17:47:59.671 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:47:59.668+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:47:59.713 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-04T17:47:59.727 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:47:59.727 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:47:59.727 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:47:59.727 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:47:59.773 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:47:59.770+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:47:59.776 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:47:59.776 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:47:59.777 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:47:59.777 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:47:59.794 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-04T17:47:59.873 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:47:59.870+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:47:59.898 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:47:59.899 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:47:59.916 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:47:59.917 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:47:59.933 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-04T17:47:59.933 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-04T17:47:59.943 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:47:59.943 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:47:59.963 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-04T17:47:59.964 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-04T17:47:59.964 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-04T17:47:59.983 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:47:59.984 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:47:59.984 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:47:59.980+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:48:00.002 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:48:00.003 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:48:00.025 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-04T17:48:00.025 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-04T17:48:00.044 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:48:00.045 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:48:00.083 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:00.083 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:00.084 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:00.080+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:00.097 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-04T17:48:00.176 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:48:00.170+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:48:00.239 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:00.239 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:00.255 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-04T17:48:00.277 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:48:00.276+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:48:00.378 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:00.376+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:00.437 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:00.437 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:00.452 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-04T17:48:00.479 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:48:00.476+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:48:00.580 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:00.576+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:00.597 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:00.597 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:00.615 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-04T17:48:00.681 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:48:00.678+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:48:00.742 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:00.742 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:00.758 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-04T17:48:00.783 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:48:00.780+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:48:00.855 INFO:teuthology.orchestra.run.smithi092.stdout:5929 writesame 0x213d1c0 thru 0x2141ecb data_size 0x2(0x4d0c bytes) 2024-09-04T17:48:00.867 INFO:teuthology.orchestra.run.smithi092.stdout:5930 writesame 0xa4ef190 thru 0xa4fca32 data_size 0x1(0xd8a3 bytes) 2024-09-04T17:48:00.884 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:48:00.878+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:48:00.898 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-04T17:48:00.902 INFO:teuthology.orchestra.run.smithi092.stdout:5931 read 0x25afe7e thru 0x25be7ea (0xe96d bytes) 2024-09-04T17:48:00.902 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:48:00.902 INFO:teuthology.orchestra.run.smithi092.stdout:5933 write 0x6092ca1 thru 0x60995f8 (0x6958 bytes) 2024-09-04T17:48:00.909 INFO:teuthology.orchestra.run.smithi092.stdout:5935 punch from 0x529e951 to 0x52adc79, (0xf328 bytes) 2024-09-04T17:48:00.921 INFO:teuthology.orchestra.run.smithi092.stdout:5936 write 0xde34dfe thru 0xde34e91 (0x94 bytes) 2024-09-04T17:48:00.925 INFO:teuthology.orchestra.run.smithi092.stdout:5937 punch from 0x7380e97 to 0x738da88, (0xcbf1 bytes) 2024-09-04T17:48:00.925 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:48:00.935 INFO:teuthology.orchestra.run.smithi092.stdout:5940 writesame 0xe62990e thru 0xe631e4a data_size 0x1(0x853d bytes) 2024-09-04T17:48:00.935 INFO:teuthology.orchestra.run.smithi092.stdout:5941 write 0x1e6d03f thru 0x1e78fec (0xbfae bytes) 2024-09-04T17:48:00.939 INFO:teuthology.orchestra.run.smithi092.stdout:5942 punch from 0xb37c6ba to 0xb380718, (0x405e bytes) 2024-09-04T17:48:00.939 INFO:teuthology.orchestra.run.smithi092.stdout:5944 write 0xa03fea3 thru 0xa041457 (0x15b5 bytes) 2024-09-04T17:48:00.943 INFO:teuthology.orchestra.run.smithi092.stdout:5946 read 0x9c4d8b thru 0x9d47ce (0xfa44 bytes) 2024-09-04T17:48:00.945 INFO:teuthology.orchestra.run.smithi092.stdout:5947 trunc from 0xe631e4b to 0xd999bfe 2024-09-04T17:48:00.956 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:00.956 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:00.959 INFO:teuthology.orchestra.run.smithi092.stdout:5949 read 0x1d548dc thru 0x1d5c592 (0x7cb7 bytes) 2024-09-04T17:48:00.959 INFO:teuthology.orchestra.run.smithi092.stdout:5950 writesame 0x57db217 thru 0x57dde52 data_size 0x26(0x2c3c bytes) 2024-09-04T17:48:00.973 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-04T17:48:00.978 INFO:teuthology.orchestra.run.smithi092.stdout:5951 trunc from 0xd999bfe to 0x4e39082 2024-09-04T17:48:00.985 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:48:00.980+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:48:01.086 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:48:01.082+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:48:01.135 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:01.136 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:01.152 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-04T17:48:01.193 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:48:01.190+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:48:01.288 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:01.284+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:01.318 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:01.318 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:01.332 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-04T17:48:01.389 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:48:01.386+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:48:01.456 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:01.456 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:01.470 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-04T17:48:01.513 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:01.510+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:01.591 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:48:01.588+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:48:01.637 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:01.638 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:01.653 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-04T17:48:01.692 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:48:01.686+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:48:01.783 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:01.784 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:01.792 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:01.786+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:01.797 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-04T17:48:01.836 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is unset 2024-09-04T17:48:01.894 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:01.890+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:01.896 INFO:teuthology.orchestra.run.smithi092.stdout:5953 write 0xe8693e7 thru 0xe873f6f (0xab89 bytes) 2024-09-04T17:48:01.927 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:01.927 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:01.941 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-04T17:48:01.995 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:01.990+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:02.064 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-04T17:48:02.108 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:02.108 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:02.108 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:02.104+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:02.122 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-04T17:48:02.196 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:48:02.190+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:48:02.203 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:48:02.203 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:48:02.220 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:48:02.220 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:48:02.237 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-04T17:48:02.237 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-04T17:48:02.254 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-04T17:48:02.254 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-04T17:48:02.283 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:02.283 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:48:02.283 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:48:02.283 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:02.297 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-04T17:48:02.297 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:48:02.292+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:48:02.398 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:48:02.398+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:48:02.462 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:02.462 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:48:02.462 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:48:02.463 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:48:02.463 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:02.476 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-04T17:48:02.479 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:48:02.479 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:48:02.506 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:48:02.506 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:48:02.506 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:48:02.504+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:48:02.535 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-04T17:48:02.536 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-04T17:48:02.566 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:48:02.566 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:48:02.595 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:02.595 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:48:02.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:48:02.595 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:48:02.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.595 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:71638 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-04T17:48:02.596 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:44.479908+0000", 2024-09-04T17:48:02.596 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.114130213999999, 2024-09-04T17:48:02.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.469396488, 2024-09-04T17:48:02.596 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:02.596 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:02.596 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:02.596 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:02.596 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:02.596 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:02.596 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 71638 2024-09-04T17:48:02.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.596 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:02.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.597 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:02.597 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.479908+0000", 2024-09-04T17:48:02.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:02.597 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.597 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:02.597 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.479908+0000", 2024-09-04T17:48:02.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:02.597 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.597 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:02.597 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.479911+0000", 2024-09-04T17:48:02.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9160000000000001e-06 2024-09-04T17:48:02.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:02.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.479925+0000", 2024-09-04T17:48:02.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3896e-05 2024-09-04T17:48:02.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:02.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.479954+0000", 2024-09-04T17:48:02.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9249000000000001e-05 2024-09-04T17:48:02.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:02.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.479969+0000", 2024-09-04T17:48:02.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4683e-05 2024-09-04T17:48:02.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:02.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.480109+0000", 2024-09-04T17:48:02.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013995699999999999 2024-09-04T17:48:02.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-04T17:48:02.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.480125+0000", 2024-09-04T17:48:02.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6651999999999999e-05 2024-09-04T17:48:02.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:02.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.948598+0000", 2024-09-04T17:48:02.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.46847249600000002 2024-09-04T17:48:02.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:02.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.948889+0000", 2024-09-04T17:48:02.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000290947 2024-09-04T17:48:02.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:02.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.949304+0000", 2024-09-04T17:48:02.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000415692 2024-09-04T17:48:02.602 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:02.602 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:02.602 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:02.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.602 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:72267 3.1a 3:58dbd2e1:::rbd_data.11342ece613c.000000000000001e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-09-04T17:48:02.602 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:45.674662+0000", 2024-09-04T17:48:02.602 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 16.919376405000001, 2024-09-04T17:48:02.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032248339000000001, 2024-09-04T17:48:02.602 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:02.602 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:02.602 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:02.602 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:02.603 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:02.603 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:02.603 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 72267 2024-09-04T17:48:02.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.603 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:02.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:02.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.674662+0000", 2024-09-04T17:48:02.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:02.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:02.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.674662+0000", 2024-09-04T17:48:02.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:02.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:02.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.674664+0000", 2024-09-04T17:48:02.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4870000000000001e-06 2024-09-04T17:48:02.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:02.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.674689+0000", 2024-09-04T17:48:02.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4805000000000001e-05 2024-09-04T17:48:02.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:02.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.674715+0000", 2024-09-04T17:48:02.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5548999999999999e-05 2024-09-04T17:48:02.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:02.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.674731+0000", 2024-09-04T17:48:02.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6549999999999999e-05 2024-09-04T17:48:02.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:02.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.674843+0000", 2024-09-04T17:48:02.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000112001 2024-09-04T17:48:02.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:02.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.675176+0000", 2024-09-04T17:48:02.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00033337200000000002 2024-09-04T17:48:02.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-04T17:48:02.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.680121+0000", 2024-09-04T17:48:02.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0049445540000000003 2024-09-04T17:48:02.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:02.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.706702+0000", 2024-09-04T17:48:02.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026580840000000001 2024-09-04T17:48:02.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:02.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.706738+0000", 2024-09-04T17:48:02.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6158999999999997e-05 2024-09-04T17:48:02.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:02.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.706835+0000", 2024-09-04T17:48:02.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6909000000000003e-05 2024-09-04T17:48:02.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:02.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.706910+0000", 2024-09-04T17:48:02.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5112999999999997e-05 2024-09-04T17:48:02.608 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:02.608 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:02.608 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:02.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.609 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94568 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 1703936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:02.609 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.605863+0000", 2024-09-04T17:48:02.609 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.98817559, 2024-09-04T17:48:02.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027249301, 2024-09-04T17:48:02.610 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:02.610 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:02.610 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:02.610 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:02.610 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:02.610 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:02.610 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94568 2024-09-04T17:48:02.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.611 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:02.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:02.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605863+0000", 2024-09-04T17:48:02.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:02.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:02.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605863+0000", 2024-09-04T17:48:02.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:02.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:02.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605863+0000", 2024-09-04T17:48:02.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7999999999999996e-07 2024-09-04T17:48:02.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.613 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:02.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605866+0000", 2024-09-04T17:48:02.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6199999999999999e-06 2024-09-04T17:48:02.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:02.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605875+0000", 2024-09-04T17:48:02.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2499999999999995e-06 2024-09-04T17:48:02.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:02.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605881+0000", 2024-09-04T17:48:02.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.19e-06 2024-09-04T17:48:02.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:02.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.632629+0000", 2024-09-04T17:48:02.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026747567 2024-09-04T17:48:02.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.617 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:02.617 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.632874+0000", 2024-09-04T17:48:02.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024493600000000001 2024-09-04T17:48:02.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.617 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:02.617 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.633112+0000", 2024-09-04T17:48:02.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023825800000000001 2024-09-04T17:48:02.617 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:02.617 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:02.617 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:02.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.618 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94569 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:02.618 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.605888+0000", 2024-09-04T17:48:02.618 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.9881497609999998, 2024-09-04T17:48:02.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027675620000000001, 2024-09-04T17:48:02.618 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:02.618 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:02.618 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:02.618 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:02.618 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:02.618 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:02.619 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94569 2024-09-04T17:48:02.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.619 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:02.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:02.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605888+0000", 2024-09-04T17:48:02.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:02.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:02.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605888+0000", 2024-09-04T17:48:02.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:02.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:02.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605889+0000", 2024-09-04T17:48:02.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4799999999999998e-07 2024-09-04T17:48:02.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:02.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605891+0000", 2024-09-04T17:48:02.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4880000000000001e-06 2024-09-04T17:48:02.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:02.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605901+0000", 2024-09-04T17:48:02.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4760000000000004e-06 2024-09-04T17:48:02.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:02.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605912+0000", 2024-09-04T17:48:02.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0698e-05 2024-09-04T17:48:02.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:02.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.633184+0000", 2024-09-04T17:48:02.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.02727253 2024-09-04T17:48:02.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.622 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:02.622 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.633330+0000", 2024-09-04T17:48:02.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014543800000000001 2024-09-04T17:48:02.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.622 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:02.622 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.633564+0000", 2024-09-04T17:48:02.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023444199999999999 2024-09-04T17:48:02.623 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:02.623 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:02.623 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:02.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.623 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94570 3.0 3.3f52c000 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:02.623 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.605932+0000", 2024-09-04T17:48:02.623 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.9881061940000002, 2024-09-04T17:48:02.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027687021999999999, 2024-09-04T17:48:02.623 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:02.623 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:02.624 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-04T17:48:02.624 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:02.624 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:02.624 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:02.624 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94570 2024-09-04T17:48:02.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.624 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:02.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:02.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605932+0000", 2024-09-04T17:48:02.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:02.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:02.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605932+0000", 2024-09-04T17:48:02.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:02.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:02.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605933+0000", 2024-09-04T17:48:02.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5000000000000001e-07 2024-09-04T17:48:02.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:02.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605936+0000", 2024-09-04T17:48:02.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7769999999999998e-06 2024-09-04T17:48:02.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:02.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605943+0000", 2024-09-04T17:48:02.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7000000000000008e-06 2024-09-04T17:48:02.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:02.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605949+0000", 2024-09-04T17:48:02.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0170000000000002e-06 2024-09-04T17:48:02.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:02.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.633604+0000", 2024-09-04T17:48:02.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027654875999999998 2024-09-04T17:48:02.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:02.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.633619+0000", 2024-09-04T17:48:02.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4802e-05 2024-09-04T17:48:02.627 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:02.627 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:02.628 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:02.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.628 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94641 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 6094848~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:02.628 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.637288+0000", 2024-09-04T17:48:02.628 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.956749743, 2024-09-04T17:48:02.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026966783000000001, 2024-09-04T17:48:02.628 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:02.628 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:02.628 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:02.628 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:02.628 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:02.629 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:02.629 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94641 2024-09-04T17:48:02.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.629 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:02.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:02.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.637288+0000", 2024-09-04T17:48:02.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:02.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:02.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.637288+0000", 2024-09-04T17:48:02.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:02.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:02.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.637289+0000", 2024-09-04T17:48:02.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4600000000000005e-07 2024-09-04T17:48:02.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:02.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.637292+0000", 2024-09-04T17:48:02.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1049999999999999e-06 2024-09-04T17:48:02.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:02.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.637301+0000", 2024-09-04T17:48:02.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7509999999999997e-06 2024-09-04T17:48:02.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:02.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.637309+0000", 2024-09-04T17:48:02.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.67e-06 2024-09-04T17:48:02.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:02.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.663971+0000", 2024-09-04T17:48:02.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026661926999999998 2024-09-04T17:48:02.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:02.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.664062+0000", 2024-09-04T17:48:02.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0673000000000004e-05 2024-09-04T17:48:02.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:02.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.664255+0000", 2024-09-04T17:48:02.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019311099999999999 2024-09-04T17:48:02.633 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:02.633 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:02.633 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:02.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.633 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94642 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 6225920~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:02.633 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.637321+0000", 2024-09-04T17:48:02.633 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.9567167960000003, 2024-09-04T17:48:02.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027131534999999998, 2024-09-04T17:48:02.633 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:02.634 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:02.634 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:02.634 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:02.634 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:02.634 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:02.634 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94642 2024-09-04T17:48:02.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.634 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:02.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:02.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.637321+0000", 2024-09-04T17:48:02.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:02.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:02.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.637321+0000", 2024-09-04T17:48:02.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:02.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:02.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.637322+0000", 2024-09-04T17:48:02.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6599999999999996e-07 2024-09-04T17:48:02.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:02.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.637325+0000", 2024-09-04T17:48:02.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1070000000000001e-06 2024-09-04T17:48:02.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:02.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.637337+0000", 2024-09-04T17:48:02.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1766000000000001e-05 2024-09-04T17:48:02.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:02.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.637350+0000", 2024-09-04T17:48:02.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3515e-05 2024-09-04T17:48:02.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:02.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.664273+0000", 2024-09-04T17:48:02.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026922643 2024-09-04T17:48:02.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:02.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.664337+0000", 2024-09-04T17:48:02.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4492999999999994e-05 2024-09-04T17:48:02.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:02.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.664453+0000", 2024-09-04T17:48:02.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000115345 2024-09-04T17:48:02.638 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:02.638 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:02.638 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:02.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.638 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94647 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 6684672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:02.638 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.638572+0000", 2024-09-04T17:48:02.638 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.95546591, 2024-09-04T17:48:02.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026910377999999999, 2024-09-04T17:48:02.638 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:02.638 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:02.639 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:02.639 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:02.639 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:02.639 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:02.639 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94647 2024-09-04T17:48:02.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.639 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:02.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:02.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638572+0000", 2024-09-04T17:48:02.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:02.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:02.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638572+0000", 2024-09-04T17:48:02.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:02.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:02.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638573+0000", 2024-09-04T17:48:02.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7599999999999997e-07 2024-09-04T17:48:02.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:02.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638575+0000", 2024-09-04T17:48:02.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2010000000000002e-06 2024-09-04T17:48:02.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:02.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638582+0000", 2024-09-04T17:48:02.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7399999999999998e-06 2024-09-04T17:48:02.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:02.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638588+0000", 2024-09-04T17:48:02.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8560000000000003e-06 2024-09-04T17:48:02.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:02.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.665294+0000", 2024-09-04T17:48:02.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026705893000000001 2024-09-04T17:48:02.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:02.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.665355+0000", 2024-09-04T17:48:02.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1253000000000005e-05 2024-09-04T17:48:02.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:02.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.665483+0000", 2024-09-04T17:48:02.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012775899999999999 2024-09-04T17:48:02.642 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:02.643 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:02.643 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:02.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.643 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94648 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 6815744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:02.643 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.638593+0000", 2024-09-04T17:48:02.643 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.955444822, 2024-09-04T17:48:02.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027163676000000001, 2024-09-04T17:48:02.643 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:02.643 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:02.643 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:02.643 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:02.643 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:02.644 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:02.644 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94648 2024-09-04T17:48:02.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.644 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:02.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.644 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:02.644 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638593+0000", 2024-09-04T17:48:02.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:02.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.644 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:02.644 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638593+0000", 2024-09-04T17:48:02.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:02.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:02.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638594+0000", 2024-09-04T17:48:02.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.53e-07 2024-09-04T17:48:02.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:02.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638596+0000", 2024-09-04T17:48:02.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8870000000000001e-06 2024-09-04T17:48:02.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:02.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638600+0000", 2024-09-04T17:48:02.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3290000000000004e-06 2024-09-04T17:48:02.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:02.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638603+0000", 2024-09-04T17:48:02.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4680000000000001e-06 2024-09-04T17:48:02.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:02.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.665524+0000", 2024-09-04T17:48:02.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.02692052 2024-09-04T17:48:02.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.647 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:02.647 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.665600+0000", 2024-09-04T17:48:02.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6569000000000005e-05 2024-09-04T17:48:02.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.647 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:02.647 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.665757+0000", 2024-09-04T17:48:02.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015655000000000001 2024-09-04T17:48:02.647 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:02.647 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:02.647 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:02.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.648 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94649 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 6946816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:02.648 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.638608+0000", 2024-09-04T17:48:02.648 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.9554303920000002, 2024-09-04T17:48:02.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027355753999999999, 2024-09-04T17:48:02.648 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:02.648 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:02.648 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:02.648 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:02.648 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:02.648 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:02.648 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94649 2024-09-04T17:48:02.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.649 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:02.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.649 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:02.649 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638608+0000", 2024-09-04T17:48:02.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:02.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.649 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:02.649 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638608+0000", 2024-09-04T17:48:02.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:02.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.649 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:02.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638608+0000", 2024-09-04T17:48:02.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3099999999999998e-07 2024-09-04T17:48:02.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:02.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638610+0000", 2024-09-04T17:48:02.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.716e-06 2024-09-04T17:48:02.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:02.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638614+0000", 2024-09-04T17:48:02.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.941e-06 2024-09-04T17:48:02.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:02.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638618+0000", 2024-09-04T17:48:02.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5009999999999998e-06 2024-09-04T17:48:02.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:02.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.665795+0000", 2024-09-04T17:48:02.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027176404000000001 2024-09-04T17:48:02.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:02.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.665851+0000", 2024-09-04T17:48:02.652 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6530999999999998e-05 2024-09-04T17:48:02.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.652 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:02.652 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.665963+0000", 2024-09-04T17:48:02.652 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011213 2024-09-04T17:48:02.652 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:02.652 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:02.652 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:02.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.652 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94650 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 7077888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:02.653 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.638623+0000", 2024-09-04T17:48:02.653 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.9554150959999999, 2024-09-04T17:48:02.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027665261999999999, 2024-09-04T17:48:02.653 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:02.653 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:02.653 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:02.653 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:02.653 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:02.653 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:02.653 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94650 2024-09-04T17:48:02.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.653 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:02.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:02.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638623+0000", 2024-09-04T17:48:02.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:02.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:02.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638623+0000", 2024-09-04T17:48:02.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:02.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:02.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638623+0000", 2024-09-04T17:48:02.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6300000000000001e-07 2024-09-04T17:48:02.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:02.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638625+0000", 2024-09-04T17:48:02.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7260000000000001e-06 2024-09-04T17:48:02.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:02.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638629+0000", 2024-09-04T17:48:02.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1309999999999999e-06 2024-09-04T17:48:02.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:02.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638634+0000", 2024-09-04T17:48:02.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2700000000000004e-06 2024-09-04T17:48:02.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:02.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.666028+0000", 2024-09-04T17:48:02.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027393303000000001 2024-09-04T17:48:02.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:02.657 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.666112+0000", 2024-09-04T17:48:02.657 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3844000000000006e-05 2024-09-04T17:48:02.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.657 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:02.657 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.666288+0000", 2024-09-04T17:48:02.657 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000176725 2024-09-04T17:48:02.657 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:02.657 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:02.657 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:02.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.657 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94651 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 7208960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:02.657 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.638639+0000", 2024-09-04T17:48:02.658 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.9553987020000001, 2024-09-04T17:48:02.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027839592999999999, 2024-09-04T17:48:02.658 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:02.658 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:02.658 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:02.661 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:02.661 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:02.661 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:02.661 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94651 2024-09-04T17:48:02.661 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.661 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:02.661 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.661 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:02.662 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638639+0000", 2024-09-04T17:48:02.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:02.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.662 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:02.662 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638639+0000", 2024-09-04T17:48:02.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:02.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.662 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:02.662 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638640+0000", 2024-09-04T17:48:02.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8000000000000002e-07 2024-09-04T17:48:02.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:02.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638641+0000", 2024-09-04T17:48:02.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.77e-06 2024-09-04T17:48:02.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:02.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638646+0000", 2024-09-04T17:48:02.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1729999999999997e-06 2024-09-04T17:48:02.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:02.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638649+0000", 2024-09-04T17:48:02.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5260000000000002e-06 2024-09-04T17:48:02.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:02.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.666309+0000", 2024-09-04T17:48:02.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027660179 2024-09-04T17:48:02.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:02.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.666373+0000", 2024-09-04T17:48:02.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4152999999999994e-05 2024-09-04T17:48:02.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:02.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.666479+0000", 2024-09-04T17:48:02.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000105512 2024-09-04T17:48:02.665 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:02.665 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:02.665 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:02.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.665 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94657 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 7012352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:02.665 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.641793+0000", 2024-09-04T17:48:02.665 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.9522452179999998, 2024-09-04T17:48:02.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026893672, 2024-09-04T17:48:02.665 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:02.665 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:02.665 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:02.666 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:02.666 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:02.666 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:02.666 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94657 2024-09-04T17:48:02.666 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.666 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:02.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.666 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:02.666 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641793+0000", 2024-09-04T17:48:02.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:02.666 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.666 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:02.667 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641793+0000", 2024-09-04T17:48:02.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:02.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.667 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:02.667 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641793+0000", 2024-09-04T17:48:02.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5000000000000003e-07 2024-09-04T17:48:02.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.667 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:02.667 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641797+0000", 2024-09-04T17:48:02.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2909999999999999e-06 2024-09-04T17:48:02.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.668 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:02.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641806+0000", 2024-09-04T17:48:02.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3270000000000007e-06 2024-09-04T17:48:02.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.668 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:02.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641812+0000", 2024-09-04T17:48:02.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2310000000000001e-06 2024-09-04T17:48:02.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.668 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:02.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.668412+0000", 2024-09-04T17:48:02.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026600104999999999 2024-09-04T17:48:02.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.669 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:02.669 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.668534+0000", 2024-09-04T17:48:02.669 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012115100000000001 2024-09-04T17:48:02.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.669 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:02.669 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.668687+0000", 2024-09-04T17:48:02.669 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000153017 2024-09-04T17:48:02.669 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:02.669 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:02.669 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:02.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.670 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94658 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 7143424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:02.670 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.641822+0000", 2024-09-04T17:48:02.670 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.9522162870000002, 2024-09-04T17:48:02.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027395880000000001, 2024-09-04T17:48:02.670 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:02.670 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:02.670 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:02.670 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:02.670 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:02.670 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:02.670 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94658 2024-09-04T17:48:02.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.671 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:02.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:02.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641822+0000", 2024-09-04T17:48:02.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:02.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:02.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641822+0000", 2024-09-04T17:48:02.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:02.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.672 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:02.672 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641822+0000", 2024-09-04T17:48:02.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.58e-07 2024-09-04T17:48:02.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.672 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:02.672 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641825+0000", 2024-09-04T17:48:02.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5909999999999998e-06 2024-09-04T17:48:02.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.672 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:02.672 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641832+0000", 2024-09-04T17:48:02.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1799999999999999e-06 2024-09-04T17:48:02.673 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.673 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.673 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:02.673 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641838+0000", 2024-09-04T17:48:02.673 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5040000000000002e-06 2024-09-04T17:48:02.673 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.673 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.673 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:02.673 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.668748+0000", 2024-09-04T17:48:02.673 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026910119999999999 2024-09-04T17:48:02.673 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.673 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.673 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:02.673 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.668896+0000", 2024-09-04T17:48:02.674 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014804299999999999 2024-09-04T17:48:02.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.674 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.674 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:02.674 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.669218+0000", 2024-09-04T17:48:02.674 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00032198399999999999 2024-09-04T17:48:02.674 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:02.674 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:02.674 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:02.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.674 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.674 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94659 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 7274496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:02.674 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.641846+0000", 2024-09-04T17:48:02.675 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.9521922169999999, 2024-09-04T17:48:02.675 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027757314000000002, 2024-09-04T17:48:02.675 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:02.675 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:02.675 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:02.675 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:02.675 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:02.675 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:02.675 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94659 2024-09-04T17:48:02.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.675 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:02.675 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.676 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:02.676 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641846+0000", 2024-09-04T17:48:02.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:02.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.676 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:02.676 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641846+0000", 2024-09-04T17:48:02.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:02.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.676 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:02.676 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641846+0000", 2024-09-04T17:48:02.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5400000000000002e-07 2024-09-04T17:48:02.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:02.677 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641849+0000", 2024-09-04T17:48:02.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0170000000000001e-06 2024-09-04T17:48:02.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:02.677 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641858+0000", 2024-09-04T17:48:02.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1499999999999999e-06 2024-09-04T17:48:02.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:02.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641864+0000", 2024-09-04T17:48:02.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2280000000000003e-06 2024-09-04T17:48:02.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.678 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:02.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.669278+0000", 2024-09-04T17:48:02.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027413843 2024-09-04T17:48:02.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.678 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:02.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.669392+0000", 2024-09-04T17:48:02.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000114654 2024-09-04T17:48:02.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.679 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:02.679 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.669603+0000", 2024-09-04T17:48:02.679 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021096799999999999 2024-09-04T17:48:02.679 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:02.679 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:02.679 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:02.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.679 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94664 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 7864320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:02.679 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.644474+0000", 2024-09-04T17:48:02.679 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.9495645370000001, 2024-09-04T17:48:02.679 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026965042000000002, 2024-09-04T17:48:02.680 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:02.680 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:02.680 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:02.680 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:02.680 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:02.680 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:02.680 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94664 2024-09-04T17:48:02.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.680 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:02.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.680 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:02.680 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644474+0000", 2024-09-04T17:48:02.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:02.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:02.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644474+0000", 2024-09-04T17:48:02.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:02.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:02.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644474+0000", 2024-09-04T17:48:02.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.16e-07 2024-09-04T17:48:02.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:02.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644478+0000", 2024-09-04T17:48:02.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1389999999999999e-06 2024-09-04T17:48:02.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:02.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644494+0000", 2024-09-04T17:48:02.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6977e-05 2024-09-04T17:48:02.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:02.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644501+0000", 2024-09-04T17:48:02.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.438e-06 2024-09-04T17:48:02.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.683 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:02.683 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.671241+0000", 2024-09-04T17:48:02.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026740525000000001 2024-09-04T17:48:02.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.683 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:02.683 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.671325+0000", 2024-09-04T17:48:02.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3987999999999996e-05 2024-09-04T17:48:02.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.683 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:02.684 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.671439+0000", 2024-09-04T17:48:02.684 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011315900000000001 2024-09-04T17:48:02.684 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:02.684 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:02.684 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:02.684 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.684 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.684 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94665 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 7995392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:02.684 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.644511+0000", 2024-09-04T17:48:02.684 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.9495275049999998, 2024-09-04T17:48:02.684 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027156390999999998, 2024-09-04T17:48:02.684 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:02.684 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:02.685 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:02.685 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:02.685 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:02.685 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:02.685 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94665 2024-09-04T17:48:02.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.685 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:02.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.685 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:02.685 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644511+0000", 2024-09-04T17:48:02.685 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:02.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:02.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644511+0000", 2024-09-04T17:48:02.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:02.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:02.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644511+0000", 2024-09-04T17:48:02.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2699999999999999e-07 2024-09-04T17:48:02.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:02.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644514+0000", 2024-09-04T17:48:02.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9890000000000001e-06 2024-09-04T17:48:02.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:02.687 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644521+0000", 2024-09-04T17:48:02.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5409999999999997e-06 2024-09-04T17:48:02.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:02.687 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644527+0000", 2024-09-04T17:48:02.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0340000000000002e-06 2024-09-04T17:48:02.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:02.687 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.671474+0000", 2024-09-04T17:48:02.688 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.02694767 2024-09-04T17:48:02.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.688 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.688 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:02.688 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.671529+0000", 2024-09-04T17:48:02.688 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4373000000000003e-05 2024-09-04T17:48:02.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.688 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.688 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:02.688 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.671667+0000", 2024-09-04T17:48:02.688 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000138257 2024-09-04T17:48:02.688 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:02.688 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:02.689 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:02.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.689 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94666 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 8126464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:02.689 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.644536+0000", 2024-09-04T17:48:02.689 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.949502367, 2024-09-04T17:48:02.689 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027360524000000001, 2024-09-04T17:48:02.689 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:02.689 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:02.689 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:02.689 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:02.689 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:02.689 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:02.690 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94666 2024-09-04T17:48:02.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.690 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:02.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.690 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:02.690 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644536+0000", 2024-09-04T17:48:02.690 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:02.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.690 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:02.690 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644536+0000", 2024-09-04T17:48:02.690 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:02.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.691 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:02.691 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644536+0000", 2024-09-04T17:48:02.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7000000000000004e-07 2024-09-04T17:48:02.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.691 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:02.691 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644540+0000", 2024-09-04T17:48:02.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1609999999999999e-06 2024-09-04T17:48:02.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.691 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:02.691 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644552+0000", 2024-09-04T17:48:02.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2794e-05 2024-09-04T17:48:02.692 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.692 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.692 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:02.692 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644559+0000", 2024-09-04T17:48:02.692 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5010000000000001e-06 2024-09-04T17:48:02.692 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.692 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.692 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:02.692 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.671703+0000", 2024-09-04T17:48:02.692 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027144492999999999 2024-09-04T17:48:02.692 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.692 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.692 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:02.692 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.671779+0000", 2024-09-04T17:48:02.693 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5578e-05 2024-09-04T17:48:02.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.693 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:02.693 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.671896+0000", 2024-09-04T17:48:02.693 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000117327 2024-09-04T17:48:02.693 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:02.693 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:02.693 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:02.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.693 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94667 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 8257536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:02.693 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.644570+0000", 2024-09-04T17:48:02.694 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.9494677029999998, 2024-09-04T17:48:02.694 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027556085000000001, 2024-09-04T17:48:02.694 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:02.694 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:02.694 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:02.694 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:02.694 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:02.694 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:02.694 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94667 2024-09-04T17:48:02.694 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.694 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:02.694 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.694 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:02.694 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644570+0000", 2024-09-04T17:48:02.695 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:02.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.695 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.695 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:02.695 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644570+0000", 2024-09-04T17:48:02.695 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:02.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.695 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.695 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:02.695 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644571+0000", 2024-09-04T17:48:02.695 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5199999999999997e-07 2024-09-04T17:48:02.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.695 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.696 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:02.696 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644574+0000", 2024-09-04T17:48:02.696 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3340000000000002e-06 2024-09-04T17:48:02.696 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.696 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.696 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:02.696 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644581+0000", 2024-09-04T17:48:02.696 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6579999999999996e-06 2024-09-04T17:48:02.696 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.696 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.696 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:02.696 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644589+0000", 2024-09-04T17:48:02.696 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9699999999999999e-06 2024-09-04T17:48:02.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.697 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:02.697 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.671925+0000", 2024-09-04T17:48:02.697 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027335911000000001 2024-09-04T17:48:02.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.697 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:02.697 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.671987+0000", 2024-09-04T17:48:02.697 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2198000000000003e-05 2024-09-04T17:48:02.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.697 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:02.698 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.672126+0000", 2024-09-04T17:48:02.698 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000139462 2024-09-04T17:48:02.698 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:02.698 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:02.698 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:02.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.698 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.698 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94675 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 8323072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:02.698 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.647655+0000", 2024-09-04T17:48:02.698 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.9463831300000001, 2024-09-04T17:48:02.698 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026819046999999999, 2024-09-04T17:48:02.698 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:02.698 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:02.698 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:02.699 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:02.699 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:02.699 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:02.699 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94675 2024-09-04T17:48:02.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.699 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:02.699 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.699 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:02.699 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.647655+0000", 2024-09-04T17:48:02.699 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:02.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.699 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.699 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:02.700 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.647655+0000", 2024-09-04T17:48:02.700 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:02.700 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.700 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.700 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:02.700 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.647656+0000", 2024-09-04T17:48:02.700 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3500000000000005e-07 2024-09-04T17:48:02.700 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.700 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.700 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:02.700 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.647659+0000", 2024-09-04T17:48:02.700 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0900000000000001e-06 2024-09-04T17:48:02.700 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.700 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.701 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:02.701 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.647666+0000", 2024-09-04T17:48:02.701 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2509999999999997e-06 2024-09-04T17:48:02.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.701 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.701 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:02.701 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.647672+0000", 2024-09-04T17:48:02.701 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8549999999999998e-06 2024-09-04T17:48:02.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.701 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.701 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:02.701 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.674197+0000", 2024-09-04T17:48:02.701 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026524466 2024-09-04T17:48:02.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:02.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.674310+0000", 2024-09-04T17:48:02.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011333299999999999 2024-09-04T17:48:02.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:02.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:02.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:02.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.674474+0000", 2024-09-04T17:48:02.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000164117 2024-09-04T17:48:02.702 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:02.702 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:02.702 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:02.703 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:02.703 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:02.703 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:02.704 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-04T17:48:02.705 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:02.698+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:02.805 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:48:02.804+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:48:02.834 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:02.835 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:48:02.835 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:48:02.835 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:02.849 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-04T17:48:02.857 INFO:teuthology.orchestra.run.smithi092.stdout:5954 read 0x552bad1 thru 0x55307e0 (0x4d10 bytes) 2024-09-04T17:48:02.857 INFO:teuthology.orchestra.run.smithi092.stdout:5955 writesame 0x3161ce2 thru 0x31679ac data_size 0x1(0x5ccb bytes) 2024-09-04T17:48:02.896 INFO:teuthology.orchestra.run.smithi092.stdout:5956 punch from 0xba459b to 0xba4d63, (0x7c8 bytes) 2024-09-04T17:48:02.896 INFO:teuthology.orchestra.run.smithi092.stdout:5957 read 0x6de92be thru 0x6deabbe (0x1901 bytes) 2024-09-04T17:48:02.896 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:48:02.897 INFO:teuthology.orchestra.run.smithi092.stdout:5960 read 0x8214f9 thru 0x822d81 (0x1889 bytes) 2024-09-04T17:48:02.897 INFO:teuthology.orchestra.run.smithi092.stdout:5961 write 0x4241ac6 thru 0x424abdb (0x9116 bytes) 2024-09-04T17:48:02.902 INFO:teuthology.orchestra.run.smithi092.stdout:5962 write 0x505f13e thru 0x506ae27 (0xbcea bytes) 2024-09-04T17:48:02.906 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:02.904+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:02.909 INFO:teuthology.orchestra.run.smithi092.stdout:5963 read 0xc90d20 thru 0xc96c85 (0x5f66 bytes) 2024-09-04T17:48:02.909 INFO:teuthology.orchestra.run.smithi092.stdout:5966 write 0x7350b04 thru 0x735ebda (0xe0d7 bytes) 2024-09-04T17:48:02.915 INFO:teuthology.orchestra.run.smithi092.stdout:5968 writesame 0x3785dc0 thru 0x3794558 data_size 0x1(0xe799 bytes) 2024-09-04T17:48:02.978 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:02.978 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:48:02.978 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:48:02.978 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:48:02.978 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:02.979 INFO:teuthology.orchestra.run.smithi092.stdout:5972 punch from 0x2042b5b to 0x20472bb, (0x4760 bytes) 2024-09-04T17:48:02.979 INFO:teuthology.orchestra.run.smithi092.stdout:5974 read 0x54677da thru 0x547771d (0xff44 bytes) 2024-09-04T17:48:02.979 INFO:teuthology.orchestra.run.smithi092.stdout:5975 read 0xc85dfc2 thru 0xc86db0f (0xfb4e bytes) 2024-09-04T17:48:02.980 INFO:teuthology.orchestra.run.smithi092.stdout:5977 writesame 0x1320b1f thru 0x1325b2c data_size 0x1(0x500e bytes) 2024-09-04T17:48:02.994 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-04T17:48:03.005 INFO:teuthology.orchestra.run.smithi092.stdout:5978 writesame 0xd7c1234 thru 0xd7c4ed6 data_size 0x13(0x3ca3 bytes) 2024-09-04T17:48:03.006 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:03.004+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:03.012 INFO:teuthology.orchestra.run.smithi092.stdout:5979 write 0xaa88bd8 thru 0xaa95eec (0xd315 bytes) 2024-09-04T17:48:03.019 INFO:teuthology.orchestra.run.smithi092.stdout:5980 write 0x87b32ca thru 0x87be55f (0xb296 bytes) 2024-09-04T17:48:03.025 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:48:03.025 INFO:teuthology.orchestra.run.smithi092.stdout:5984 write 0x931177c thru 0x93138b3 (0x2138 bytes) 2024-09-04T17:48:03.030 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:48:03.031 INFO:teuthology.orchestra.run.smithi092.stdout:5986 write 0x427ed65 thru 0x42852a3 (0x653f bytes) 2024-09-04T17:48:03.036 INFO:teuthology.orchestra.run.smithi092.stdout:5987 writesame 0xe6169d6 thru 0xe61e4db data_size 0x6(0x7b06 bytes) 2024-09-04T17:48:03.043 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:48:03.043 INFO:teuthology.orchestra.run.smithi092.stdout:5989 read 0xbf8a64e thru 0xbf90180 (0x5b33 bytes) 2024-09-04T17:48:03.043 INFO:teuthology.orchestra.run.smithi092.stdout:5991 read 0x801da90 thru 0x80228aa (0x4e1b bytes) 2024-09-04T17:48:03.043 INFO:teuthology.orchestra.run.smithi092.stdout:5992 read 0x861109f thru 0x861f37c (0xe2de bytes) 2024-09-04T17:48:03.043 INFO:teuthology.orchestra.run.smithi092.stdout:compare and write disabled 2024-09-04T17:48:03.044 INFO:teuthology.orchestra.run.smithi092.stdout:5995 punch from 0x984a434 to 0x9854399, (0x9f65 bytes) 2024-09-04T17:48:03.044 INFO:teuthology.orchestra.run.smithi092.stdout:5996 read 0x4c274cd thru 0x4c2e94c (0x7480 bytes) 2024-09-04T17:48:03.044 INFO:teuthology.orchestra.run.smithi092.stdout:5997 writesame 0x610c091 thru 0x6111bf9 data_size 0x1(0x5b69 bytes) 2024-09-04T17:48:03.057 INFO:teuthology.orchestra.run.smithi092.stdout:5998 read 0x70de4e thru 0x71bacc (0xdc7f bytes) 2024-09-04T17:48:03.086 INFO:teuthology.orchestra.run.smithi092.stdout:6000 trunc from 0xe873f70 to 0x3f3d121 2024-09-04T17:48:03.107 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:48:03.102+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:48:03.133 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:03.133 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:48:03.133 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:48:03.134 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:48:03.134 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.134 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "pg_update_log_missing(3.1f epoch 119/118 rep_tid 2838 entries 119'424 (0'0) error 3:f95bc674:::rbd_header.1134878eadad:head by client.4404.0:93468 0.000000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-04T17:48:03.134 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:58.898727+0000", 2024-09-04T17:48:03.134 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.2314735739999998, 2024-09-04T17:48:03.134 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0072113999999999998, 2024-09-04T17:48:03.134 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:03.134 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:03.134 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:03.134 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:03.134 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.135 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:03.135 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.898727+0000", 2024-09-04T17:48:03.135 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:03.135 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.135 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.135 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:03.135 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.898727+0000", 2024-09-04T17:48:03.135 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:03.135 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.135 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.135 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:03.136 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.898728+0000", 2024-09-04T17:48:03.136 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.8200000000000001e-07 2024-09-04T17:48:03.136 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.136 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.136 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:03.136 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.898740+0000", 2024-09-04T17:48:03.136 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2186e-05 2024-09-04T17:48:03.136 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.136 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.136 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:03.136 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.898772+0000", 2024-09-04T17:48:03.136 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.2156000000000002e-05 2024-09-04T17:48:03.137 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.137 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.137 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:03.137 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.898796+0000", 2024-09-04T17:48:03.137 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4017999999999999e-05 2024-09-04T17:48:03.137 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.137 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.137 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:03.137 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.898966+0000", 2024-09-04T17:48:03.137 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000170185 2024-09-04T17:48:03.137 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.138 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.138 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:03.138 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.905939+0000", 2024-09-04T17:48:03.138 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0069724729999999999 2024-09-04T17:48:03.138 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:03.138 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:03.138 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:03.138 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.138 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.138 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:93469 3.1f e119/118 3:f95bc674:::rbd_header.1134878eadad:head v 119'425, mlcod=0'0)", 2024-09-04T17:48:03.138 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:58.900173+0000", 2024-09-04T17:48:03.138 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.230027786, 2024-09-04T17:48:03.139 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0059053930000000001, 2024-09-04T17:48:03.139 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:03.139 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:03.139 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:03.139 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:03.139 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.139 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:03.139 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.900173+0000", 2024-09-04T17:48:03.139 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:03.139 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.139 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.139 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:03.139 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.900173+0000", 2024-09-04T17:48:03.140 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:03.140 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.140 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.140 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:03.140 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.900174+0000", 2024-09-04T17:48:03.140 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.127e-06 2024-09-04T17:48:03.140 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.140 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.140 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:03.140 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.900181+0000", 2024-09-04T17:48:03.140 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.0099999999999998e-06 2024-09-04T17:48:03.140 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.141 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.141 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:03.141 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.900199+0000", 2024-09-04T17:48:03.141 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7876e-05 2024-09-04T17:48:03.141 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.141 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.141 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:03.141 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.900208+0000", 2024-09-04T17:48:03.141 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.4999999999999999e-06 2024-09-04T17:48:03.141 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.141 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.142 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:03.142 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.900310+0000", 2024-09-04T17:48:03.142 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00010202800000000001 2024-09-04T17:48:03.142 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.142 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.142 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:03.142 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.900353+0000", 2024-09-04T17:48:03.142 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.3213000000000001e-05 2024-09-04T17:48:03.142 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.142 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.142 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:03.142 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.905944+0000", 2024-09-04T17:48:03.143 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0055914900000000002 2024-09-04T17:48:03.143 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.143 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.143 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:03.143 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.906078+0000", 2024-09-04T17:48:03.143 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013414899999999999 2024-09-04T17:48:03.143 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:03.143 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:03.143 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:03.144 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.144 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.144 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "MOSDPGPush(3.19 119/118 [PushOp(3:9e238ce7:::rbd_data.1134b705d747.000000000000006b:11, version: 53'167, data_included: [0~65536,172896~10695,196608~65536], data_size: 141767, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:9e238ce7:::rbd_data.1134b705d747.000000000000006b:11@53'167, size: 262144, copy_subset: [(0, 262144)], clone_subset: {}, snapset: 11=[]:{11: [11]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 262144, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-04T17:48:03.144 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:59.130405+0000", 2024-09-04T17:48:03.144 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 3.999795373, 2024-09-04T17:48:03.144 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0026941209999999998, 2024-09-04T17:48:03.145 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:03.145 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:03.145 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:03.145 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:03.145 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.146 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:03.146 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.130405+0000", 2024-09-04T17:48:03.146 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:03.146 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.146 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.146 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:03.147 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.130405+0000", 2024-09-04T17:48:03.147 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:03.147 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.147 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.147 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:03.147 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.130407+0000", 2024-09-04T17:48:03.147 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5069999999999999e-06 2024-09-04T17:48:03.148 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.148 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.148 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:03.148 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.130644+0000", 2024-09-04T17:48:03.148 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000236574 2024-09-04T17:48:03.148 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.149 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.149 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:03.149 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.130684+0000", 2024-09-04T17:48:03.149 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0535999999999999e-05 2024-09-04T17:48:03.149 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.149 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.150 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:03.150 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.130717+0000", 2024-09-04T17:48:03.150 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.2755000000000001e-05 2024-09-04T17:48:03.150 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.150 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.150 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:03.150 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.130948+0000", 2024-09-04T17:48:03.151 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00023101400000000001 2024-09-04T17:48:03.151 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.151 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.151 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:03.151 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.130987+0000", 2024-09-04T17:48:03.151 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.9329000000000002e-05 2024-09-04T17:48:03.152 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.152 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.152 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:03.152 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.133100+0000", 2024-09-04T17:48:03.152 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0021124059999999998 2024-09-04T17:48:03.152 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:03.153 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:03.153 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:03.153 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.153 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.153 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "MOSDPGPush(3.19 119/118 [PushOp(3:98c2eddd:::rbd_data.11342e8f3384.0000000000000008:13, version: 56'177, data_included: [0~524288,1310720~6278,2101248~16384,2121728~16384,2142208~16384,2162688~13095], data_size: 592813, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:98c2eddd:::rbd_data.11342e8f3384.0000000000000008:13@56'177, size: 2175783, copy_subset: [(0, 2175783)], clone_subset: {}, snapset: 13=[]:{13: [13]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2175783, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-04T17:48:03.153 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:59.147984+0000", 2024-09-04T17:48:03.154 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 3.9822168050000002, 2024-09-04T17:48:03.154 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0043689150000000001, 2024-09-04T17:48:03.154 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:03.154 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:03.154 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:03.154 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:03.154 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.154 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:03.155 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.147984+0000", 2024-09-04T17:48:03.155 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:03.155 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.155 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.155 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:03.155 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.147984+0000", 2024-09-04T17:48:03.155 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:03.156 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.156 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.156 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:03.156 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.147986+0000", 2024-09-04T17:48:03.157 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.714e-06 2024-09-04T17:48:03.157 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.157 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.157 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:03.157 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.148457+0000", 2024-09-04T17:48:03.157 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00047091100000000002 2024-09-04T17:48:03.158 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.158 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.158 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:03.158 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.148504+0000", 2024-09-04T17:48:03.158 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7265999999999998e-05 2024-09-04T17:48:03.159 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.159 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.159 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:03.159 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.148542+0000", 2024-09-04T17:48:03.159 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7792e-05 2024-09-04T17:48:03.160 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.160 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.160 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:03.160 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.148797+0000", 2024-09-04T17:48:03.160 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00025577999999999999 2024-09-04T17:48:03.160 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.161 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.161 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:03.161 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.148840+0000", 2024-09-04T17:48:03.161 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2599000000000003e-05 2024-09-04T17:48:03.161 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.161 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.162 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:03.162 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.152353+0000", 2024-09-04T17:48:03.162 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0035128529999999998 2024-09-04T17:48:03.162 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:03.162 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:03.162 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:03.162 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.163 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.163 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "MOSDPGPush(3.19 119/118 [PushOp(3:9db25b11:::rbd_data.11341db94a06.0000000000000001:head, version: 78'254, data_included: [0~294912,458752~32768,1802240~65536,1966080~98304,3014656~294912], data_size: 786432, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:9db25b11:::rbd_data.11341db94a06.0000000000000001:head@78'254, size: 4096000, copy_subset: [(0, 4096000)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096000, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-04T17:48:03.163 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:59.229007+0000", 2024-09-04T17:48:03.163 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 3.901194292, 2024-09-04T17:48:03.163 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0049484100000000003, 2024-09-04T17:48:03.163 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:03.163 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:03.163 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:03.163 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:03.164 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.164 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:03.164 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.229007+0000", 2024-09-04T17:48:03.164 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:03.164 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.164 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.164 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:03.164 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.229007+0000", 2024-09-04T17:48:03.165 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:03.165 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.165 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.165 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:03.165 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.229008+0000", 2024-09-04T17:48:03.165 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.827e-06 2024-09-04T17:48:03.166 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.166 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.166 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:03.166 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.230334+0000", 2024-09-04T17:48:03.166 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0013259089999999999 2024-09-04T17:48:03.166 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.167 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.167 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:03.167 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.230382+0000", 2024-09-04T17:48:03.167 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.8168999999999999e-05 2024-09-04T17:48:03.167 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.167 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.168 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:03.168 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.230415+0000", 2024-09-04T17:48:03.168 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.2916000000000001e-05 2024-09-04T17:48:03.168 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.168 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.169 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:03.169 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.230676+0000", 2024-09-04T17:48:03.169 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00026107600000000001 2024-09-04T17:48:03.169 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.169 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.169 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:03.169 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.230719+0000", 2024-09-04T17:48:03.169 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2571999999999998e-05 2024-09-04T17:48:03.169 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.169 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.170 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:03.170 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.233955+0000", 2024-09-04T17:48:03.170 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.003235941 2024-09-04T17:48:03.170 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:03.170 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:03.170 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:03.170 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.170 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.170 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "MOSDPGPush(3.19 119/118 [PushOp(3:9f46236a:::rbd_data.113481c24fbf.0000000000000002:23, version: 91'285, data_included: [292708~22999,1504212~42332], data_size: 65331, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:9f46236a:::rbd_data.113481c24fbf.0000000000000002:23@91'285, size: 16777216, copy_subset: [(0, 16777216)], clone_subset: {}, snapset: 23=[]:{23: [23]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 16777216, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-04T17:48:03.170 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:59.246120+0000", 2024-09-04T17:48:03.170 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 3.884080859, 2024-09-04T17:48:03.170 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0025576779999999999, 2024-09-04T17:48:03.171 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:03.171 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:03.171 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:03.171 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:03.171 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.171 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:03.171 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.246120+0000", 2024-09-04T17:48:03.171 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:03.171 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.171 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.171 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:03.171 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.246120+0000", 2024-09-04T17:48:03.171 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:03.172 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.172 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.172 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:03.172 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.246121+0000", 2024-09-04T17:48:03.172 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.207e-06 2024-09-04T17:48:03.172 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.172 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.172 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:03.172 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.246247+0000", 2024-09-04T17:48:03.172 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000125387 2024-09-04T17:48:03.172 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.172 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.172 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:03.173 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.246311+0000", 2024-09-04T17:48:03.173 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.4565000000000003e-05 2024-09-04T17:48:03.173 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.173 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.173 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:03.173 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.246366+0000", 2024-09-04T17:48:03.173 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4595000000000002e-05 2024-09-04T17:48:03.173 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.173 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.173 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:03.173 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.246544+0000", 2024-09-04T17:48:03.173 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000178689 2024-09-04T17:48:03.174 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.174 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.174 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:03.174 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.246567+0000", 2024-09-04T17:48:03.174 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2739e-05 2024-09-04T17:48:03.174 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.174 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.174 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:03.174 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.248678+0000", 2024-09-04T17:48:03.174 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0021104959999999999 2024-09-04T17:48:03.174 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:03.174 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:03.174 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:03.174 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.175 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.175 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "MOSDPGPush(3.19 119/118 [PushOp(3:9e581cd8:::rbd_data.11346ecfd364.0000000000000000:head, version: 98'308, data_included: [0~65536,196608~65536,422582~17651,3518558~569,3801088~20417], data_size: 169709, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:9e581cd8:::rbd_data.11346ecfd364.0000000000000000:head@98'308, size: 3821505, copy_subset: [(0, 3821505)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3821505, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-04T17:48:03.175 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:59.272553+0000", 2024-09-04T17:48:03.175 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 3.8576481249999999, 2024-09-04T17:48:03.175 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0028161240000000001, 2024-09-04T17:48:03.175 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:03.175 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:03.175 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:03.175 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:03.175 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.175 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:03.175 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.272553+0000", 2024-09-04T17:48:03.176 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:03.176 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.176 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.176 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:03.176 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.272553+0000", 2024-09-04T17:48:03.176 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:03.176 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.176 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.176 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:03.176 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.272554+0000", 2024-09-04T17:48:03.176 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7129999999999999e-06 2024-09-04T17:48:03.176 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.176 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.176 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:03.177 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.272731+0000", 2024-09-04T17:48:03.177 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00017618899999999999 2024-09-04T17:48:03.177 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.177 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.177 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:03.177 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.272778+0000", 2024-09-04T17:48:03.177 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7584999999999999e-05 2024-09-04T17:48:03.177 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.177 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.177 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:03.177 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.272811+0000", 2024-09-04T17:48:03.177 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.2733e-05 2024-09-04T17:48:03.177 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.178 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.178 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:03.178 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.273093+0000", 2024-09-04T17:48:03.178 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00028174899999999998 2024-09-04T17:48:03.178 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.178 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.178 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:03.178 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.273138+0000", 2024-09-04T17:48:03.178 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.5565999999999997e-05 2024-09-04T17:48:03.178 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.178 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.178 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:03.179 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.275369+0000", 2024-09-04T17:48:03.179 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0022305889999999998 2024-09-04T17:48:03.179 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:03.179 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:03.179 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:03.179 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.179 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.179 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:93768 3.d e119/118 3:b012e48a:::rbd_data.1134c4e2d26.0000000000000073:head v 119'1272, mlcod=0'0)", 2024-09-04T17:48:03.179 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:59.380804+0000", 2024-09-04T17:48:03.179 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 3.7493972680000001, 2024-09-04T17:48:03.180 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.002404158, 2024-09-04T17:48:03.180 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:03.180 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:03.180 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:03.180 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:03.180 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.180 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:03.180 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.380804+0000", 2024-09-04T17:48:03.180 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:03.180 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.180 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.181 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:03.181 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.380804+0000", 2024-09-04T17:48:03.181 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:03.181 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.181 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.181 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:03.181 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.380806+0000", 2024-09-04T17:48:03.181 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0600000000000002e-06 2024-09-04T17:48:03.181 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.181 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.181 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:03.181 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.380863+0000", 2024-09-04T17:48:03.182 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6918e-05 2024-09-04T17:48:03.182 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.182 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.182 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:03.182 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.380890+0000", 2024-09-04T17:48:03.182 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.7195999999999998e-05 2024-09-04T17:48:03.182 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.182 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.182 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:03.182 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.380904+0000", 2024-09-04T17:48:03.182 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4659e-05 2024-09-04T17:48:03.182 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.182 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.183 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:03.183 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.381004+0000", 2024-09-04T17:48:03.183 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.9642999999999994e-05 2024-09-04T17:48:03.183 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.183 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.183 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:03.183 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.381053+0000", 2024-09-04T17:48:03.183 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9330999999999999e-05 2024-09-04T17:48:03.183 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.183 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.183 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:03.183 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.383037+0000", 2024-09-04T17:48:03.183 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0019841110000000002 2024-09-04T17:48:03.184 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.184 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.184 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:03.184 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.383208+0000", 2024-09-04T17:48:03.184 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00017024 2024-09-04T17:48:03.184 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:03.184 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:03.184 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:03.184 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.184 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.184 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "MOSDPGPush(3.10 119/118 [PushOp(3:0c2d06e5:::rbd_data.113410ddd206.0000000000000000:18, version: 68'308, data_included: [0~884736,950272~65536,1081344~65536,1212416~32768], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:0c2d06e5:::rbd_data.113410ddd206.0000000000000000:18@68'308, size: 7471104, copy_subset: [(0, 7471104)], clone_subset: {}, snapset: 18=[]:{18: [18]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1245184, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-04T17:48:03.184 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:59.426517+0000", 2024-09-04T17:48:03.185 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 3.7036837079999998, 2024-09-04T17:48:03.185 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0064985670000000002, 2024-09-04T17:48:03.185 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:03.185 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:03.185 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:03.185 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:03.185 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.185 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:03.185 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.426517+0000", 2024-09-04T17:48:03.185 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:03.185 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.185 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.186 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:03.186 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.426517+0000", 2024-09-04T17:48:03.186 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:03.186 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.186 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.186 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:03.186 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.426754+0000", 2024-09-04T17:48:03.186 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00023681799999999999 2024-09-04T17:48:03.186 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.186 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.186 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:03.186 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.429886+0000", 2024-09-04T17:48:03.187 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0031323789999999998 2024-09-04T17:48:03.187 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.187 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.187 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:03.187 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.429940+0000", 2024-09-04T17:48:03.187 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4104999999999998e-05 2024-09-04T17:48:03.187 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.187 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.187 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:03.187 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.429975+0000", 2024-09-04T17:48:03.187 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4289000000000003e-05 2024-09-04T17:48:03.187 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.187 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.188 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:03.188 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.430229+0000", 2024-09-04T17:48:03.188 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00025465900000000002 2024-09-04T17:48:03.188 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.188 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.188 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:03.188 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.430350+0000", 2024-09-04T17:48:03.188 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000120992 2024-09-04T17:48:03.188 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.188 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.188 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:03.188 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.433016+0000", 2024-09-04T17:48:03.188 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0026653250000000001 2024-09-04T17:48:03.189 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:03.189 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:03.189 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:03.189 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.189 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.189 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "MOSDPGPush(3.10 119/118 [PushOp(3:0c2d06e5:::rbd_data.113410ddd206.0000000000000000:18, version: 68'308, data_included: [1245184~32768,1343488~65536,1474560~65536,1605632~65536,1736704~65536,1867776~65536,1998848~65536,2129920~65536,2260992~65536,2392064~65536,2523136~65536,2654208~65536,2785280~65536,2916352~65536,3047424~65536,3154405~89627,3309568~8677], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:0c2d06e5:::rbd_data.113410ddd206.0000000000000000:18@68'308, size: 7471104, copy_subset: [(0, 7471104)], clone_subset: {}, snapset: 18=[]:{18: [18]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3318245, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1245184, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-04T17:48:03.189 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:59.434957+0000", 2024-09-04T17:48:03.189 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 3.6952440850000001, 2024-09-04T17:48:03.189 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0061603969999999997, 2024-09-04T17:48:03.189 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:03.189 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:03.189 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:03.189 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:03.190 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.190 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:03.190 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.434957+0000", 2024-09-04T17:48:03.190 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:03.190 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.190 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.191 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:03.191 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.434957+0000", 2024-09-04T17:48:03.191 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:03.191 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.191 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.192 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:03.192 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.434958+0000", 2024-09-04T17:48:03.192 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4339999999999999e-06 2024-09-04T17:48:03.192 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.192 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.192 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:03.192 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.436921+0000", 2024-09-04T17:48:03.193 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0019629460000000001 2024-09-04T17:48:03.193 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.193 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.193 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:03.193 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.436973+0000", 2024-09-04T17:48:03.193 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1492000000000001e-05 2024-09-04T17:48:03.193 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.193 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.193 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:03.193 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.437041+0000", 2024-09-04T17:48:03.193 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.8275000000000006e-05 2024-09-04T17:48:03.193 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.193 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.194 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:03.194 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.437312+0000", 2024-09-04T17:48:03.194 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00027156800000000003 2024-09-04T17:48:03.194 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.194 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.194 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:03.194 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.437357+0000", 2024-09-04T17:48:03.194 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4079000000000003e-05 2024-09-04T17:48:03.194 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.195 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.195 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:03.195 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.441117+0000", 2024-09-04T17:48:03.195 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0037606029999999999 2024-09-04T17:48:03.195 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:03.195 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:03.195 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:03.195 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.195 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.195 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "MOSDPGPush(3.10 119/118 [PushOp(3:0c2d06e5:::rbd_data.113410ddd206.0000000000000000:18, version: 68'308, data_included: [3318245~24091,3440640~65536,3604480~32768,3665463~4553,3833856~32768,4043811~23,4931930~46881,7437058~34046], data_size: 240666, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:0c2d06e5:::rbd_data.113410ddd206.0000000000000000:18@68'308, size: 7471104, copy_subset: [(0, 7471104)], clone_subset: {}, snapset: 18=[]:{18: [18]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 7471104, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3318245, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-04T17:48:03.195 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:59.442797+0000", 2024-09-04T17:48:03.195 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 3.6874035890000001, 2024-09-04T17:48:03.195 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.004018743, 2024-09-04T17:48:03.196 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:03.196 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:03.196 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:03.196 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:03.196 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.196 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:03.196 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.442797+0000", 2024-09-04T17:48:03.196 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:03.196 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.196 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.196 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:03.196 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.442797+0000", 2024-09-04T17:48:03.197 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:03.197 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.197 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.197 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:03.197 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.442799+0000", 2024-09-04T17:48:03.197 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6279999999999999e-06 2024-09-04T17:48:03.197 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.197 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.197 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:03.197 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.443240+0000", 2024-09-04T17:48:03.197 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00044157799999999998 2024-09-04T17:48:03.197 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.198 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.198 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:03.198 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.443274+0000", 2024-09-04T17:48:03.198 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.3519999999999998e-05 2024-09-04T17:48:03.198 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.198 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.198 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:03.198 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.443300+0000", 2024-09-04T17:48:03.198 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6029000000000001e-05 2024-09-04T17:48:03.198 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.198 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.198 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:03.198 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.443497+0000", 2024-09-04T17:48:03.199 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019724200000000001 2024-09-04T17:48:03.199 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.199 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.199 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:03.199 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.443518+0000", 2024-09-04T17:48:03.199 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0633999999999999e-05 2024-09-04T17:48:03.199 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.199 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.199 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:03.199 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.446816+0000", 2024-09-04T17:48:03.199 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0032981120000000002 2024-09-04T17:48:03.199 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:03.200 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:03.200 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:03.200 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.200 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.200 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "MOSDPGPush(3.10 119/118 [PushOp(3:0f0c5418:::rbd_data.1134911bffa0.0000000000000005:head, version: 72'337, data_included: [458752~2381], data_size: 2381, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:0f0c5418:::rbd_data.1134911bffa0.0000000000000005:head@72'337, size: 461133, copy_subset: [(0, 461133)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 461133, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-04T17:48:03.200 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:59.464033+0000", 2024-09-04T17:48:03.200 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 3.666167373, 2024-09-04T17:48:03.200 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0022763729999999999, 2024-09-04T17:48:03.200 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:03.200 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:03.200 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:03.201 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:03.201 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.201 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:03.201 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.464033+0000", 2024-09-04T17:48:03.201 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:03.201 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.201 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.201 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:03.202 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.464033+0000", 2024-09-04T17:48:03.202 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:03.202 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.202 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.202 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:03.202 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.464036+0000", 2024-09-04T17:48:03.202 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2800000000000002e-06 2024-09-04T17:48:03.202 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.202 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.202 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:03.203 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.464103+0000", 2024-09-04T17:48:03.203 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7218000000000006e-05 2024-09-04T17:48:03.203 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.203 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.203 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:03.205 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.464227+0000", 2024-09-04T17:48:03.205 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00012380099999999999 2024-09-04T17:48:03.205 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.205 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.205 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:03.205 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.464369+0000", 2024-09-04T17:48:03.206 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014252899999999999 2024-09-04T17:48:03.206 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.206 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.206 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:03.206 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.464740+0000", 2024-09-04T17:48:03.206 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00037052299999999999 2024-09-04T17:48:03.206 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.206 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.206 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:03.206 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.464788+0000", 2024-09-04T17:48:03.206 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7852999999999997e-05 2024-09-04T17:48:03.206 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.206 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.207 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:03.207 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.466310+0000", 2024-09-04T17:48:03.207 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0015221690000000001 2024-09-04T17:48:03.207 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:03.207 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:03.207 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:03.207 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.207 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.207 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "MOSDPGPush(3.10 119/118 [PushOp(3:08a6a427:::rbd_data.11341db94a06.0000000000000002:head, version: 78'342, data_included: [0~271649,458752~32768,1802240~65536,1966080~98304,3014656~294912], data_size: 763169, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:08a6a427:::rbd_data.11341db94a06.0000000000000002:head@78'342, size: 4096000, copy_subset: [(0, 4096000)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096000, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-04T17:48:03.207 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:59.471247+0000", 2024-09-04T17:48:03.207 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 3.6589538409999998, 2024-09-04T17:48:03.207 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0048034530000000001, 2024-09-04T17:48:03.207 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:03.208 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:03.208 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:03.208 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:03.208 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.208 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:03.208 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.471247+0000", 2024-09-04T17:48:03.208 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:03.208 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.208 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.208 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:03.208 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.471247+0000", 2024-09-04T17:48:03.208 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:03.208 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.209 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.209 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:03.209 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.471249+0000", 2024-09-04T17:48:03.209 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9e-06 2024-09-04T17:48:03.209 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.209 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.209 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:03.209 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.472560+0000", 2024-09-04T17:48:03.209 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0013114190000000001 2024-09-04T17:48:03.209 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.209 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.209 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:03.210 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.472610+0000", 2024-09-04T17:48:03.210 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9916000000000001e-05 2024-09-04T17:48:03.210 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.210 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.210 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:03.210 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.472692+0000", 2024-09-04T17:48:03.210 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.1384000000000005e-05 2024-09-04T17:48:03.210 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.210 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.210 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:03.210 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.472912+0000", 2024-09-04T17:48:03.210 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000220201 2024-09-04T17:48:03.211 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.211 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.211 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:03.211 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.472983+0000", 2024-09-04T17:48:03.211 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1137999999999996e-05 2024-09-04T17:48:03.211 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.211 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.211 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:03.211 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.476050+0000", 2024-09-04T17:48:03.211 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0030674949999999999 2024-09-04T17:48:03.211 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:03.211 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:03.211 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:03.212 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.212 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.212 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "MOSDPGPush(3.10 119/118 [PushOp(3:0b7b6f02:::rbd_id.image_client.0-clone30:head, version: 88'344, data_included: [0~16], data_size: 16, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:0b7b6f02:::rbd_id.image_client.0-clone30:head@88'344, size: 16, copy_subset: [(0, 16)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 16, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-04T17:48:03.212 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:59.478015+0000", 2024-09-04T17:48:03.212 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 3.6521854519999999, 2024-09-04T17:48:03.212 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0022468219999999999, 2024-09-04T17:48:03.212 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:03.212 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:03.212 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:03.212 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:03.212 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.212 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:03.213 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.478015+0000", 2024-09-04T17:48:03.213 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:03.213 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.213 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.213 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:03.213 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.478015+0000", 2024-09-04T17:48:03.213 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:03.213 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.213 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.213 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:03.213 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.478024+0000", 2024-09-04T17:48:03.213 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.5290000000000008e-06 2024-09-04T17:48:03.213 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.214 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.214 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:03.214 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.478056+0000", 2024-09-04T17:48:03.214 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.1665999999999998e-05 2024-09-04T17:48:03.214 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.214 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.214 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:03.214 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.478101+0000", 2024-09-04T17:48:03.214 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.5252e-05 2024-09-04T17:48:03.215 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.215 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.215 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:03.215 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.478130+0000", 2024-09-04T17:48:03.215 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.9082999999999999e-05 2024-09-04T17:48:03.215 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.215 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.215 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:03.215 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.478387+0000", 2024-09-04T17:48:03.215 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00025696999999999997 2024-09-04T17:48:03.215 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.215 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.216 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:03.216 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.478427+0000", 2024-09-04T17:48:03.216 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.9975999999999998e-05 2024-09-04T17:48:03.216 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.216 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.216 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:03.216 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.480262+0000", 2024-09-04T17:48:03.216 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0018353460000000001 2024-09-04T17:48:03.216 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:03.216 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:03.216 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:03.216 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.217 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.217 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:94268 3.2 e119/118 3:44c82781:::rbd_data.1134c4e2d26.000000000000001d:head v 119'504, mlcod=0'0)", 2024-09-04T17:48:03.217 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:59.569460+0000", 2024-09-04T17:48:03.217 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 3.5607404310000002, 2024-09-04T17:48:03.217 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0025042300000000001, 2024-09-04T17:48:03.217 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:03.217 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:03.217 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:03.217 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:03.217 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.217 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:03.217 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.569460+0000", 2024-09-04T17:48:03.218 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:03.218 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.218 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.218 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:03.218 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.569460+0000", 2024-09-04T17:48:03.218 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:03.218 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.218 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.218 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:03.218 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.569462+0000", 2024-09-04T17:48:03.219 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6649999999999999e-06 2024-09-04T17:48:03.219 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.219 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.219 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:03.219 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.569790+0000", 2024-09-04T17:48:03.219 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00032760599999999998 2024-09-04T17:48:03.219 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.220 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.220 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:03.220 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.569806+0000", 2024-09-04T17:48:03.220 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6770999999999999e-05 2024-09-04T17:48:03.220 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.220 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.220 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:03.220 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.569814+0000", 2024-09-04T17:48:03.220 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.413e-06 2024-09-04T17:48:03.220 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.220 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.220 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:03.221 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.569889+0000", 2024-09-04T17:48:03.221 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4857999999999994e-05 2024-09-04T17:48:03.221 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.221 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.221 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:03.221 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.569921+0000", 2024-09-04T17:48:03.221 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.2505000000000001e-05 2024-09-04T17:48:03.221 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.221 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.221 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:03.221 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.571923+0000", 2024-09-04T17:48:03.221 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0020019769999999998 2024-09-04T17:48:03.221 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.222 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.222 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:03.222 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.571965+0000", 2024-09-04T17:48:03.222 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.1434999999999999e-05 2024-09-04T17:48:03.222 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:03.222 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:03.222 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:03.222 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.222 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.222 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:95613 3.d e119/118 3:b2adbabf:::rbd_data.1134c4e2d26.00000000000000da:head v 119'1274, mlcod=62'747)", 2024-09-04T17:48:03.222 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:59.768682+0000", 2024-09-04T17:48:03.222 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 3.3615187629999999, 2024-09-04T17:48:03.222 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0023070809999999999, 2024-09-04T17:48:03.222 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:03.223 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:03.223 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:03.223 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:03.223 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.223 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:03.223 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.768682+0000", 2024-09-04T17:48:03.223 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:03.223 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.223 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.223 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:03.223 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.768682+0000", 2024-09-04T17:48:03.223 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:03.224 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.224 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.224 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:03.224 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.768683+0000", 2024-09-04T17:48:03.224 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.5799999999999998e-07 2024-09-04T17:48:03.224 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.224 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.224 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:03.224 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.768724+0000", 2024-09-04T17:48:03.224 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.1329999999999999e-05 2024-09-04T17:48:03.224 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.224 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.224 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:03.225 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.768735+0000", 2024-09-04T17:48:03.225 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1083999999999999e-05 2024-09-04T17:48:03.225 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.225 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.225 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:03.225 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.768742+0000", 2024-09-04T17:48:03.225 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1130000000000003e-06 2024-09-04T17:48:03.225 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.225 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.225 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:03.225 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.769880+0000", 2024-09-04T17:48:03.225 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0011376170000000001 2024-09-04T17:48:03.226 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.226 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.226 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:03.226 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.769903+0000", 2024-09-04T17:48:03.226 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2515e-05 2024-09-04T17:48:03.226 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.226 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.226 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:03.226 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.770952+0000", 2024-09-04T17:48:03.226 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.001049201 2024-09-04T17:48:03.226 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.226 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.226 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:03.227 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.770989+0000", 2024-09-04T17:48:03.227 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7363000000000001e-05 2024-09-04T17:48:03.227 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:03.227 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:03.227 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:03.227 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.227 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.227 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "MOSDPGPush(3.0 120/118 [PushOp(3:00fa19d2:::rbd_data.113460fdfacd.0000000000000007:head, version: 87'271, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:00fa19d2:::rbd_data.113460fdfacd.0000000000000007:head@87'271, size: 2097152, copy_subset: [(0, 2097152)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-04T17:48:03.227 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:59.962140+0000", 2024-09-04T17:48:03.228 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 3.168060788, 2024-09-04T17:48:03.228 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0023609310000000001, 2024-09-04T17:48:03.228 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:03.228 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:03.228 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:03.228 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:03.228 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.228 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:03.228 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.962140+0000", 2024-09-04T17:48:03.228 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:03.228 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.228 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.228 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:03.229 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.962140+0000", 2024-09-04T17:48:03.229 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:03.229 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.229 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.229 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:03.229 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.962141+0000", 2024-09-04T17:48:03.229 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3689999999999999e-06 2024-09-04T17:48:03.229 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.229 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.229 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:03.229 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.963500+0000", 2024-09-04T17:48:03.230 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.001358122 2024-09-04T17:48:03.230 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.230 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.230 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:03.230 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.963536+0000", 2024-09-04T17:48:03.230 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.6828e-05 2024-09-04T17:48:03.230 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.230 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.230 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:03.230 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.963563+0000", 2024-09-04T17:48:03.230 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6387999999999999e-05 2024-09-04T17:48:03.231 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.231 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.231 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:03.231 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.963673+0000", 2024-09-04T17:48:03.231 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000110338 2024-09-04T17:48:03.231 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.231 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.232 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:03.232 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.963682+0000", 2024-09-04T17:48:03.232 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8130000000000002e-06 2024-09-04T17:48:03.232 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.232 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.232 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:03.233 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.964501+0000", 2024-09-04T17:48:03.233 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00081907300000000002 2024-09-04T17:48:03.233 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:03.233 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:03.233 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:03.234 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.234 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.234 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:96178 3.1e 3:7c9d4526:::rbd_data.1134c4e2d26.0000000000000026:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123)", 2024-09-04T17:48:03.234 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:02.985763+0000", 2024-09-04T17:48:03.234 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.14443793699999999, 2024-09-04T17:48:03.234 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.020686758999999999, 2024-09-04T17:48:03.234 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:03.234 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:03.234 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:03.234 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:03.235 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:03.235 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:03.235 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 96178 2024-09-04T17:48:03.235 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.235 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:03.235 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.236 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:03.236 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:02.985763+0000", 2024-09-04T17:48:03.236 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:03.236 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.236 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.236 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:03.236 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:02.985763+0000", 2024-09-04T17:48:03.236 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:03.236 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.236 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.236 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:03.237 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:02.985766+0000", 2024-09-04T17:48:03.237 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.2990000000000001e-06 2024-09-04T17:48:03.237 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.237 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.237 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:03.237 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:02.985783+0000", 2024-09-04T17:48:03.237 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.645e-05 2024-09-04T17:48:03.237 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.237 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.237 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:03.237 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:02.985818+0000", 2024-09-04T17:48:03.237 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.5459000000000001e-05 2024-09-04T17:48:03.237 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.238 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.238 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:03.238 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:02.985835+0000", 2024-09-04T17:48:03.238 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.715e-05 2024-09-04T17:48:03.238 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.238 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.238 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:03.238 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:02.985953+0000", 2024-09-04T17:48:03.238 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000117351 2024-09-04T17:48:03.238 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.238 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.238 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:03.238 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:02.986207+0000", 2024-09-04T17:48:03.238 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00025483 2024-09-04T17:48:03.239 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.239 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.239 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 2", 2024-09-04T17:48:03.239 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:02.990784+0000", 2024-09-04T17:48:03.239 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0045763219999999999 2024-09-04T17:48:03.239 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.239 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.239 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:48:03.239 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.006139+0000", 2024-09-04T17:48:03.239 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.015355526 2024-09-04T17:48:03.239 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.239 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.239 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:03.240 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.006225+0000", 2024-09-04T17:48:03.240 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.6003999999999999e-05 2024-09-04T17:48:03.240 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.240 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.240 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:03.240 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.006371+0000", 2024-09-04T17:48:03.240 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000146052 2024-09-04T17:48:03.240 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.240 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.240 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:03.240 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.006450+0000", 2024-09-04T17:48:03.241 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.8316e-05 2024-09-04T17:48:03.241 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:03.241 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:03.241 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:03.241 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.241 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.241 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop_reply(client.4404.0:96178 3.1e e123/120 ondisk, result = 0)", 2024-09-04T17:48:03.241 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:03.003795+0000", 2024-09-04T17:48:03.241 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.126405881, 2024-09-04T17:48:03.241 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.002664165, 2024-09-04T17:48:03.241 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:03.242 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:03.242 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:03.242 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:03.242 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.242 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:03.242 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.003795+0000", 2024-09-04T17:48:03.242 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:03.242 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.242 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.242 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:03.242 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.003795+0000", 2024-09-04T17:48:03.243 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:03.243 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.243 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.243 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:03.243 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.003798+0000", 2024-09-04T17:48:03.243 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.2629999999999999e-06 2024-09-04T17:48:03.243 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.243 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.243 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:03.243 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.003809+0000", 2024-09-04T17:48:03.243 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1083e-05 2024-09-04T17:48:03.244 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.244 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.244 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:03.244 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.003844+0000", 2024-09-04T17:48:03.244 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4378999999999997e-05 2024-09-04T17:48:03.244 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.244 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.244 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:03.244 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.003861+0000", 2024-09-04T17:48:03.244 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7316000000000002e-05 2024-09-04T17:48:03.245 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.245 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.245 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:03.245 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.006196+0000", 2024-09-04T17:48:03.245 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0023352450000000001 2024-09-04T17:48:03.245 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.245 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.245 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:03.245 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.006214+0000", 2024-09-04T17:48:03.245 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7331999999999999e-05 2024-09-04T17:48:03.246 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.246 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.246 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:03.246 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.006459+0000", 2024-09-04T17:48:03.246 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024554699999999999 2024-09-04T17:48:03.246 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:03.246 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:03.246 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:03.246 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.246 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.246 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:96197 3.10 3:08a64171:::rbd_data.1134c4e2d26.00000000000000c3:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123)", 2024-09-04T17:48:03.246 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:03.049064+0000", 2024-09-04T17:48:03.246 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.081136966000000005, 2024-09-04T17:48:03.247 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0053841139999999997, 2024-09-04T17:48:03.247 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:03.247 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:03.247 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:03.247 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:03.247 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:03.247 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:03.247 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 96197 2024-09-04T17:48:03.247 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.247 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:03.247 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.247 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:03.247 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.049064+0000", 2024-09-04T17:48:03.248 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:03.248 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.248 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.248 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:03.248 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.049064+0000", 2024-09-04T17:48:03.248 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:03.248 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.248 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.248 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:03.248 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.049066+0000", 2024-09-04T17:48:03.248 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.3360000000000002e-06 2024-09-04T17:48:03.248 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.248 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.249 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:03.249 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.049075+0000", 2024-09-04T17:48:03.249 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.2210000000000003e-06 2024-09-04T17:48:03.249 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.249 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.249 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:03.249 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.049097+0000", 2024-09-04T17:48:03.249 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1798e-05 2024-09-04T17:48:03.249 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.249 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.249 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:03.249 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.049107+0000", 2024-09-04T17:48:03.249 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0200999999999999e-05 2024-09-04T17:48:03.250 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.250 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.250 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:03.250 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.049183+0000", 2024-09-04T17:48:03.250 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.6079000000000001e-05 2024-09-04T17:48:03.250 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.250 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.250 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:03.250 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.049276+0000", 2024-09-04T17:48:03.250 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.2911000000000005e-05 2024-09-04T17:48:03.250 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.250 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.250 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 0", 2024-09-04T17:48:03.251 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.050313+0000", 2024-09-04T17:48:03.251 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.001036705 2024-09-04T17:48:03.251 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.251 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.251 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:48:03.251 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.054220+0000", 2024-09-04T17:48:03.251 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.003906779 2024-09-04T17:48:03.251 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.251 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.251 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:03.251 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.054297+0000", 2024-09-04T17:48:03.251 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.6816000000000005e-05 2024-09-04T17:48:03.251 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.251 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.252 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:03.252 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.054374+0000", 2024-09-04T17:48:03.252 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.7490000000000005e-05 2024-09-04T17:48:03.252 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:03.252 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:03.252 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:03.252 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.054448+0000", 2024-09-04T17:48:03.252 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3777999999999997e-05 2024-09-04T17:48:03.252 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:03.252 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:03.252 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:03.252 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:03.252 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:03.253 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:03.253 INFO:teuthology.orchestra.run.smithi092.stdout:checking clone #42, image image_client.0-clone42 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent43 2024-09-04T17:48:03.254 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-04T17:48:03.255 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:03.252+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:03.355 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:03.350+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:03.377 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:03.377 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:48:03.377 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:48:03.377 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:03.392 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-04T17:48:03.456 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:48:03.450+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:48:03.521 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:03.521 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:48:03.522 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:48:03.522 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:48:03.522 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:03.535 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-04T17:48:03.557 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:48:03.550+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:48:03.657 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:03.654+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:03.659 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:03.660 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:48:03.660 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:48:03.660 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:48:03.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.660 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:48444 3.12 3:4ff80fb5:::rbd_data.11346a2b0bc9.000000000000001d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-04T17:48:03.660 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:58.122601+0000", 2024-09-04T17:48:03.660 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 65.535810710999996, 2024-09-04T17:48:03.660 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057065713999999997, 2024-09-04T17:48:03.660 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:03.660 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:03.660 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:03.660 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:03.661 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:03.661 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:03.661 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 48444 2024-09-04T17:48:03.661 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.661 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:03.661 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.661 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:03.661 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122601+0000", 2024-09-04T17:48:03.661 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:03.661 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.661 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.661 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:03.662 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122601+0000", 2024-09-04T17:48:03.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:03.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.662 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:03.662 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122603+0000", 2024-09-04T17:48:03.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3499999999999999e-06 2024-09-04T17:48:03.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.662 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:03.662 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122614+0000", 2024-09-04T17:48:03.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.063e-05 2024-09-04T17:48:03.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:03.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122643+0000", 2024-09-04T17:48:03.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9434999999999999e-05 2024-09-04T17:48:03.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:03.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122657+0000", 2024-09-04T17:48:03.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3920000000000001e-05 2024-09-04T17:48:03.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:03.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122767+0000", 2024-09-04T17:48:03.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010981599999999999 2024-09-04T17:48:03.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:03.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122989+0000", 2024-09-04T17:48:03.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022158200000000001 2024-09-04T17:48:03.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 3", 2024-09-04T17:48:03.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.133349+0000", 2024-09-04T17:48:03.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010360729000000001 2024-09-04T17:48:03.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.665 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:03.665 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.179423+0000", 2024-09-04T17:48:03.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046073408000000003 2024-09-04T17:48:03.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.665 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:03.665 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.179453+0000", 2024-09-04T17:48:03.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0229000000000001e-05 2024-09-04T17:48:03.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.665 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:03.665 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.179545+0000", 2024-09-04T17:48:03.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1580000000000001e-05 2024-09-04T17:48:03.666 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.666 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:03.666 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.179667+0000", 2024-09-04T17:48:03.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012203500000000001 2024-09-04T17:48:03.666 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:03.666 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:03.666 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:03.666 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.666 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:55376 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-04T17:48:03.666 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:11.974053+0000", 2024-09-04T17:48:03.666 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.684358926999998, 2024-09-04T17:48:03.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060514722, 2024-09-04T17:48:03.667 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:03.667 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:03.667 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:03.667 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:03.667 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:03.667 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:03.667 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 55376 2024-09-04T17:48:03.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.667 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:03.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.667 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:03.667 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974053+0000", 2024-09-04T17:48:03.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:03.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.668 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:03.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974053+0000", 2024-09-04T17:48:03.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:03.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.668 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:03.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974054+0000", 2024-09-04T17:48:03.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.841e-06 2024-09-04T17:48:03.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.668 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:03.669 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974062+0000", 2024-09-04T17:48:03.669 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5460000000000003e-06 2024-09-04T17:48:03.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.669 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:03.669 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974077+0000", 2024-09-04T17:48:03.669 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5389000000000001e-05 2024-09-04T17:48:03.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.669 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:03.669 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974085+0000", 2024-09-04T17:48:03.669 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7659999999999998e-06 2024-09-04T17:48:03.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.670 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:03.670 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974188+0000", 2024-09-04T17:48:03.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000102521 2024-09-04T17:48:03.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.670 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:03.670 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974321+0000", 2024-09-04T17:48:03.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013324699999999999 2024-09-04T17:48:03.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.670 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-04T17:48:03.670 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.986481+0000", 2024-09-04T17:48:03.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012160450999999999 2024-09-04T17:48:03.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:03.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:12.034415+0000", 2024-09-04T17:48:03.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.047934065999999997 2024-09-04T17:48:03.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:03.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:12.034438+0000", 2024-09-04T17:48:03.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2923999999999999e-05 2024-09-04T17:48:03.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.672 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:03.672 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:12.034505+0000", 2024-09-04T17:48:03.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6711000000000006e-05 2024-09-04T17:48:03.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.672 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:03.672 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:12.034567+0000", 2024-09-04T17:48:03.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2260000000000004e-05 2024-09-04T17:48:03.672 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:03.672 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:03.672 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:03.673 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.673 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.673 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61139 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 2547712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:03.673 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819750+0000", 2024-09-04T17:48:03.673 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 49.838661213000002, 2024-09-04T17:48:03.673 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042185699, 2024-09-04T17:48:03.673 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:03.673 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:03.673 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:03.673 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:03.673 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:03.673 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:03.673 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61139 2024-09-04T17:48:03.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.674 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:03.674 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.674 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:03.674 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819750+0000", 2024-09-04T17:48:03.674 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:03.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.674 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.674 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:03.674 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819750+0000", 2024-09-04T17:48:03.674 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:03.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.674 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.675 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:03.675 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819751+0000", 2024-09-04T17:48:03.675 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4000000000000002e-07 2024-09-04T17:48:03.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.675 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.675 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:03.675 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819753+0000", 2024-09-04T17:48:03.675 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5179999999999999e-06 2024-09-04T17:48:03.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.675 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.675 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:03.675 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819760+0000", 2024-09-04T17:48:03.675 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5440000000000004e-06 2024-09-04T17:48:03.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.676 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:03.676 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819765+0000", 2024-09-04T17:48:03.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8760000000000004e-06 2024-09-04T17:48:03.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.676 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:03.676 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.861622+0000", 2024-09-04T17:48:03.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.041857093999999997 2024-09-04T17:48:03.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:03.677 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.861698+0000", 2024-09-04T17:48:03.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6080000000000003e-05 2024-09-04T17:48:03.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:03.677 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.861936+0000", 2024-09-04T17:48:03.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023804699999999999 2024-09-04T17:48:03.677 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:03.677 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:03.677 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:03.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.677 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61140 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 2686976~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:03.678 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819772+0000", 2024-09-04T17:48:03.678 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 49.838639079000004, 2024-09-04T17:48:03.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042515052999999997, 2024-09-04T17:48:03.678 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:03.678 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:03.678 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:03.678 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:03.678 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:03.678 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:03.678 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61140 2024-09-04T17:48:03.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.678 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:03.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.679 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:03.679 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819772+0000", 2024-09-04T17:48:03.679 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:03.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.679 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:03.679 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819772+0000", 2024-09-04T17:48:03.679 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:03.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.679 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:03.679 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819773+0000", 2024-09-04T17:48:03.680 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2900000000000003e-07 2024-09-04T17:48:03.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.680 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:03.680 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819775+0000", 2024-09-04T17:48:03.680 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3309999999999998e-06 2024-09-04T17:48:03.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.680 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:03.680 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819781+0000", 2024-09-04T17:48:03.680 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.981e-06 2024-09-04T17:48:03.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:03.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819786+0000", 2024-09-04T17:48:03.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5719999999999996e-06 2024-09-04T17:48:03.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:03.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.861966+0000", 2024-09-04T17:48:03.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042179541000000001 2024-09-04T17:48:03.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:03.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862038+0000", 2024-09-04T17:48:03.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2719000000000006e-05 2024-09-04T17:48:03.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.683 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:03.683 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862288+0000", 2024-09-04T17:48:03.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024928 2024-09-04T17:48:03.683 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:03.683 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:03.683 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:03.684 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.684 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.684 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61141 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 2793472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:03.684 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819909+0000", 2024-09-04T17:48:03.684 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 49.838502935000001, 2024-09-04T17:48:03.685 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042786419999999999, 2024-09-04T17:48:03.685 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:03.685 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:03.685 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:03.685 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:03.685 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:03.686 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:03.686 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61141 2024-09-04T17:48:03.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.686 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:03.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:03.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819909+0000", 2024-09-04T17:48:03.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:03.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:03.687 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819909+0000", 2024-09-04T17:48:03.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:03.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.688 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.688 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:03.688 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819909+0000", 2024-09-04T17:48:03.688 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6300000000000005e-07 2024-09-04T17:48:03.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.689 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:03.689 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819913+0000", 2024-09-04T17:48:03.689 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4189999999999998e-06 2024-09-04T17:48:03.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.690 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:03.690 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819920+0000", 2024-09-04T17:48:03.690 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8619999999999997e-06 2024-09-04T17:48:03.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.690 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:03.691 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819926+0000", 2024-09-04T17:48:03.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9399999999999999e-06 2024-09-04T17:48:03.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.691 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:03.691 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862331+0000", 2024-09-04T17:48:03.692 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042405348000000002 2024-09-04T17:48:03.692 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.692 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.692 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:03.692 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862410+0000", 2024-09-04T17:48:03.693 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9078000000000004e-05 2024-09-04T17:48:03.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.693 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:03.693 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862695+0000", 2024-09-04T17:48:03.693 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028510999999999999 2024-09-04T17:48:03.694 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:03.694 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:03.694 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:03.694 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.694 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.694 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61142 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 2932736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:03.695 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819934+0000", 2024-09-04T17:48:03.695 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 49.838477316999999, 2024-09-04T17:48:03.695 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043101985000000002, 2024-09-04T17:48:03.695 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:03.695 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:03.695 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:03.695 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:03.696 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:03.696 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:03.696 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61142 2024-09-04T17:48:03.696 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.696 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:03.696 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.697 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:03.697 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819934+0000", 2024-09-04T17:48:03.697 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:03.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.698 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:03.698 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819934+0000", 2024-09-04T17:48:03.698 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:03.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.698 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.698 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:03.699 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819935+0000", 2024-09-04T17:48:03.699 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2024-09-04T17:48:03.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.699 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.699 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:03.699 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819937+0000", 2024-09-04T17:48:03.700 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5770000000000001e-06 2024-09-04T17:48:03.700 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.700 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.700 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:03.700 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819943+0000", 2024-09-04T17:48:03.701 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7950000000000004e-06 2024-09-04T17:48:03.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.701 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.701 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:03.701 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819948+0000", 2024-09-04T17:48:03.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6779999999999999e-06 2024-09-04T17:48:03.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:03.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862739+0000", 2024-09-04T17:48:03.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042791454999999999 2024-09-04T17:48:03.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.703 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:03.703 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862824+0000", 2024-09-04T17:48:03.704 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4790999999999995e-05 2024-09-04T17:48:03.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.704 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:03.704 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.863036+0000", 2024-09-04T17:48:03.705 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021224899999999999 2024-09-04T17:48:03.705 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:03.705 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:03.705 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:03.705 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.705 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.706 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61143 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 3039232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:03.706 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819954+0000", 2024-09-04T17:48:03.706 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 49.838457198999997, 2024-09-04T17:48:03.706 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043485134000000002, 2024-09-04T17:48:03.706 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:03.706 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:03.707 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:03.707 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:03.707 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:03.707 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:03.707 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61143 2024-09-04T17:48:03.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.708 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:03.708 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.708 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:03.708 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819954+0000", 2024-09-04T17:48:03.708 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:03.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.710 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.710 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:03.710 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819954+0000", 2024-09-04T17:48:03.710 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:03.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.710 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.710 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:03.710 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819955+0000", 2024-09-04T17:48:03.710 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7300000000000002e-07 2024-09-04T17:48:03.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.711 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:03.711 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819957+0000", 2024-09-04T17:48:03.711 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.024e-06 2024-09-04T17:48:03.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.711 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:03.711 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819963+0000", 2024-09-04T17:48:03.711 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8109999999999999e-06 2024-09-04T17:48:03.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.711 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:03.712 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819967+0000", 2024-09-04T17:48:03.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6149999999999999e-06 2024-09-04T17:48:03.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.712 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:03.712 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.863082+0000", 2024-09-04T17:48:03.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043115078000000001 2024-09-04T17:48:03.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.712 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:03.712 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.863172+0000", 2024-09-04T17:48:03.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0181999999999998e-05 2024-09-04T17:48:03.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.713 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:03.713 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.863440+0000", 2024-09-04T17:48:03.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026705099999999997 2024-09-04T17:48:03.713 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:03.713 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:03.713 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:03.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.713 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:70015 3.6 3:6012e780:::rbd_data.11341aa245a3.000000000000005b:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-04T17:48:03.714 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:26.155497+0000", 2024-09-04T17:48:03.714 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.502914142000002, 2024-09-04T17:48:03.714 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.049007124999999999, 2024-09-04T17:48:03.714 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:03.714 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:03.714 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:03.714 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:03.714 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:03.714 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:03.714 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 70015 2024-09-04T17:48:03.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.714 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:03.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.715 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:03.715 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155497+0000", 2024-09-04T17:48:03.715 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:03.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.715 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:03.715 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155497+0000", 2024-09-04T17:48:03.715 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:03.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.715 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:03.716 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155500+0000", 2024-09-04T17:48:03.716 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7070000000000001e-06 2024-09-04T17:48:03.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.716 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.716 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:03.716 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155513+0000", 2024-09-04T17:48:03.716 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2989e-05 2024-09-04T17:48:03.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.716 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.716 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:03.716 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155544+0000", 2024-09-04T17:48:03.716 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0747000000000002e-05 2024-09-04T17:48:03.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.717 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:03.717 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155560+0000", 2024-09-04T17:48:03.717 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5866e-05 2024-09-04T17:48:03.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.717 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:03.717 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155675+0000", 2024-09-04T17:48:03.717 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000114838 2024-09-04T17:48:03.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.717 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:03.717 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155803+0000", 2024-09-04T17:48:03.718 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012851799999999999 2024-09-04T17:48:03.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.718 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 1", 2024-09-04T17:48:03.718 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.163582+0000", 2024-09-04T17:48:03.718 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0077790960000000001 2024-09-04T17:48:03.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.718 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:03.718 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.204055+0000", 2024-09-04T17:48:03.718 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.040472517 2024-09-04T17:48:03.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.718 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:03.719 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.204195+0000", 2024-09-04T17:48:03.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014055200000000001 2024-09-04T17:48:03.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.719 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.719 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:03.719 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.204373+0000", 2024-09-04T17:48:03.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000177784 2024-09-04T17:48:03.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.719 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.719 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:03.719 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.204505+0000", 2024-09-04T17:48:03.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013151100000000001 2024-09-04T17:48:03.720 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:03.720 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:03.720 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:03.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.720 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:70434 3.12 3:4cd2342b:::rbd_data.11341aa245a3.0000000000000091:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-04T17:48:03.720 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:26.943124+0000", 2024-09-04T17:48:03.720 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.715287359999998, 2024-09-04T17:48:03.720 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055011831999999997, 2024-09-04T17:48:03.720 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:03.720 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:03.720 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:03.721 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:03.721 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:03.721 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:03.721 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 70434 2024-09-04T17:48:03.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.721 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:03.721 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.721 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:03.721 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943124+0000", 2024-09-04T17:48:03.721 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:03.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.722 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:03.722 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943124+0000", 2024-09-04T17:48:03.722 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:03.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.722 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:03.722 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943128+0000", 2024-09-04T17:48:03.722 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8460000000000001e-06 2024-09-04T17:48:03.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.722 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:03.722 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943142+0000", 2024-09-04T17:48:03.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4089e-05 2024-09-04T17:48:03.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:03.723 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943177+0000", 2024-09-04T17:48:03.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5167999999999999e-05 2024-09-04T17:48:03.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:03.723 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943194+0000", 2024-09-04T17:48:03.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6235e-05 2024-09-04T17:48:03.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:03.724 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943286+0000", 2024-09-04T17:48:03.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2793000000000004e-05 2024-09-04T17:48:03.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.724 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:03.724 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943526+0000", 2024-09-04T17:48:03.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023950999999999999 2024-09-04T17:48:03.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.724 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 1", 2024-09-04T17:48:03.724 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.952871+0000", 2024-09-04T17:48:03.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0093453070000000006 2024-09-04T17:48:03.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:03.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.997968+0000", 2024-09-04T17:48:03.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045096613000000001 2024-09-04T17:48:03.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:03.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.998006+0000", 2024-09-04T17:48:03.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8257000000000003e-05 2024-09-04T17:48:03.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.726 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:03.726 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.998073+0000", 2024-09-04T17:48:03.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6803000000000004e-05 2024-09-04T17:48:03.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.726 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:03.726 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.998136+0000", 2024-09-04T17:48:03.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3211000000000002e-05 2024-09-04T17:48:03.726 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:03.726 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:03.726 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:03.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.727 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93840 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 4980736~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:03.727 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386465+0000", 2024-09-04T17:48:03.727 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.2719462960000003, 2024-09-04T17:48:03.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042435764000000001, 2024-09-04T17:48:03.727 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:03.727 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:03.727 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:03.727 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:03.727 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:03.727 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:03.727 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93840 2024-09-04T17:48:03.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.728 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:03.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:03.728 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386465+0000", 2024-09-04T17:48:03.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:03.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:03.728 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386465+0000", 2024-09-04T17:48:03.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:03.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.729 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:03.729 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386466+0000", 2024-09-04T17:48:03.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4700000000000001e-07 2024-09-04T17:48:03.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.729 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:03.729 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386468+0000", 2024-09-04T17:48:03.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1600000000000001e-06 2024-09-04T17:48:03.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.729 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:03.730 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386477+0000", 2024-09-04T17:48:03.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7839999999999992e-06 2024-09-04T17:48:03.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.730 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:03.730 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386482+0000", 2024-09-04T17:48:03.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3970000000000002e-06 2024-09-04T17:48:03.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.730 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:03.730 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.428466+0000", 2024-09-04T17:48:03.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.041984279999999999 2024-09-04T17:48:03.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.731 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:03.731 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.428520+0000", 2024-09-04T17:48:03.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3328000000000002e-05 2024-09-04T17:48:03.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.731 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:03.731 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.428901+0000", 2024-09-04T17:48:03.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00038126799999999998 2024-09-04T17:48:03.731 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:03.732 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:03.732 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:03.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.732 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93841 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 5111808~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:03.732 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386491+0000", 2024-09-04T17:48:03.732 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.2719201629999999, 2024-09-04T17:48:03.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043069578999999997, 2024-09-04T17:48:03.732 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:03.732 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:03.732 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:03.732 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:03.732 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:03.733 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:03.733 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93841 2024-09-04T17:48:03.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.733 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:03.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:03.733 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386491+0000", 2024-09-04T17:48:03.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:03.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:03.733 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386491+0000", 2024-09-04T17:48:03.734 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:03.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.734 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:03.734 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386492+0000", 2024-09-04T17:48:03.734 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0299999999999999e-07 2024-09-04T17:48:03.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.734 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:03.734 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386499+0000", 2024-09-04T17:48:03.734 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6089999999999997e-06 2024-09-04T17:48:03.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.735 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:03.735 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386506+0000", 2024-09-04T17:48:03.735 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3599999999999998e-06 2024-09-04T17:48:03.735 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.735 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.735 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:03.735 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386514+0000", 2024-09-04T17:48:03.735 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9379999999999992e-06 2024-09-04T17:48:03.735 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.735 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.735 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:03.735 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.428957+0000", 2024-09-04T17:48:03.735 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042442831 2024-09-04T17:48:03.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.736 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:03.736 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.429080+0000", 2024-09-04T17:48:03.736 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012359299999999999 2024-09-04T17:48:03.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.736 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:03.736 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.429561+0000", 2024-09-04T17:48:03.736 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00048074499999999998 2024-09-04T17:48:03.736 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:03.736 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:03.736 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:03.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.737 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93842 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 4259840~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:03.737 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386521+0000", 2024-09-04T17:48:03.737 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.2718902310000004, 2024-09-04T17:48:03.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043661933999999999, 2024-09-04T17:48:03.737 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:03.737 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:03.737 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:03.737 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:03.737 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:03.737 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:03.737 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93842 2024-09-04T17:48:03.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.738 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:03.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:03.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386521+0000", 2024-09-04T17:48:03.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:03.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:03.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386521+0000", 2024-09-04T17:48:03.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:03.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:03.739 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386522+0000", 2024-09-04T17:48:03.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4700000000000001e-07 2024-09-04T17:48:03.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.739 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.739 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:03.739 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386524+0000", 2024-09-04T17:48:03.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4559999999999999e-06 2024-09-04T17:48:03.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.739 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.739 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:03.739 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386534+0000", 2024-09-04T17:48:03.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2639999999999997e-06 2024-09-04T17:48:03.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.740 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.740 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:03.740 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386539+0000", 2024-09-04T17:48:03.740 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6130000000000003e-06 2024-09-04T17:48:03.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.740 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.740 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:03.740 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.429577+0000", 2024-09-04T17:48:03.740 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043037757000000003 2024-09-04T17:48:03.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.740 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.740 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:03.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.429638+0000", 2024-09-04T17:48:03.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1235999999999994e-05 2024-09-04T17:48:03.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:03.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.430183+0000", 2024-09-04T17:48:03.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000545061 2024-09-04T17:48:03.741 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:03.741 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:03.741 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:03.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.742 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93843 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 4390912~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:03.742 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386547+0000", 2024-09-04T17:48:03.742 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.2718649940000004, 2024-09-04T17:48:03.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044260668000000003, 2024-09-04T17:48:03.742 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:03.742 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:03.742 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:03.742 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:03.742 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:03.742 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:03.742 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93843 2024-09-04T17:48:03.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.743 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:03.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.743 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:03.743 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386547+0000", 2024-09-04T17:48:03.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:03.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.743 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:03.743 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386547+0000", 2024-09-04T17:48:03.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:03.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.743 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:03.744 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386547+0000", 2024-09-04T17:48:03.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4199999999999996e-07 2024-09-04T17:48:03.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.744 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:03.744 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386550+0000", 2024-09-04T17:48:03.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6199999999999999e-06 2024-09-04T17:48:03.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.744 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:03.744 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386557+0000", 2024-09-04T17:48:03.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1919999999999999e-06 2024-09-04T17:48:03.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:03.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386562+0000", 2024-09-04T17:48:03.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2580000000000003e-06 2024-09-04T17:48:03.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:03.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.430226+0000", 2024-09-04T17:48:03.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043663484000000002 2024-09-04T17:48:03.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:03.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.430286+0000", 2024-09-04T17:48:03.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0455999999999999e-05 2024-09-04T17:48:03.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:03.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.430807+0000", 2024-09-04T17:48:03.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00052111599999999996 2024-09-04T17:48:03.746 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:03.746 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:03.746 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:03.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.746 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93844 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 4521984~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:03.746 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386603+0000", 2024-09-04T17:48:03.746 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.2718089429999999, 2024-09-04T17:48:03.747 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044705350999999997, 2024-09-04T17:48:03.747 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:03.747 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:03.747 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:03.747 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:03.747 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:03.747 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:03.747 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93844 2024-09-04T17:48:03.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.747 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:03.747 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.747 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:03.748 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386603+0000", 2024-09-04T17:48:03.748 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:03.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.748 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.748 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:03.748 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386603+0000", 2024-09-04T17:48:03.748 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:03.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.748 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.748 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:03.748 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386603+0000", 2024-09-04T17:48:03.748 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.92e-07 2024-09-04T17:48:03.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.749 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:03.749 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386606+0000", 2024-09-04T17:48:03.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5500000000000001e-06 2024-09-04T17:48:03.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.749 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:03.749 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386614+0000", 2024-09-04T17:48:03.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9570000000000002e-06 2024-09-04T17:48:03.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:03.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386619+0000", 2024-09-04T17:48:03.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5119999999999999e-06 2024-09-04T17:48:03.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:03.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.430828+0000", 2024-09-04T17:48:03.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044208654 2024-09-04T17:48:03.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:03.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.430903+0000", 2024-09-04T17:48:03.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4629999999999995e-05 2024-09-04T17:48:03.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:03.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.431308+0000", 2024-09-04T17:48:03.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000405256 2024-09-04T17:48:03.751 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:03.751 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:03.751 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:03.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.751 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93845 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 4653056~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:03.751 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386627+0000", 2024-09-04T17:48:03.751 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.2717843540000002, 2024-09-04T17:48:03.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045262927000000001, 2024-09-04T17:48:03.752 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:03.752 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:03.752 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:03.752 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:03.752 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:03.752 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:03.752 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93845 2024-09-04T17:48:03.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.752 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:03.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.752 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:03.752 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386627+0000", 2024-09-04T17:48:03.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:03.753 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.753 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:03.753 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386627+0000", 2024-09-04T17:48:03.753 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:03.753 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.753 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:03.753 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386628+0000", 2024-09-04T17:48:03.753 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0999999999999998e-07 2024-09-04T17:48:03.753 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:03.754 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386630+0000", 2024-09-04T17:48:03.754 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3300000000000001e-06 2024-09-04T17:48:03.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:03.754 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386645+0000", 2024-09-04T17:48:03.754 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5186e-05 2024-09-04T17:48:03.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:03.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386651+0000", 2024-09-04T17:48:03.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7710000000000003e-06 2024-09-04T17:48:03.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:03.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.431349+0000", 2024-09-04T17:48:03.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044697723000000002 2024-09-04T17:48:03.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:03.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.431410+0000", 2024-09-04T17:48:03.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0677999999999997e-05 2024-09-04T17:48:03.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.756 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:03.756 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.431890+0000", 2024-09-04T17:48:03.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00048052899999999999 2024-09-04T17:48:03.756 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:03.756 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:03.756 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:03.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.756 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93846 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 4784128~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:03.756 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386659+0000", 2024-09-04T17:48:03.757 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.271752685, 2024-09-04T17:48:03.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045784490999999997, 2024-09-04T17:48:03.757 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:03.757 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:03.757 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:03.757 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:03.757 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:03.757 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:03.757 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93846 2024-09-04T17:48:03.757 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.757 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:03.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.757 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:03.757 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386659+0000", 2024-09-04T17:48:03.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:03.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.758 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:03.758 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386659+0000", 2024-09-04T17:48:03.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:03.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.758 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:03.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386659+0000", 2024-09-04T17:48:03.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7000000000000005e-07 2024-09-04T17:48:03.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:03.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386662+0000", 2024-09-04T17:48:03.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3300000000000001e-06 2024-09-04T17:48:03.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:03.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386669+0000", 2024-09-04T17:48:03.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0589999999999997e-06 2024-09-04T17:48:03.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:03.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386674+0000", 2024-09-04T17:48:03.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3009999999999997e-06 2024-09-04T17:48:03.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:03.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.431934+0000", 2024-09-04T17:48:03.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045259570999999998 2024-09-04T17:48:03.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:03.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.431998+0000", 2024-09-04T17:48:03.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4775000000000002e-05 2024-09-04T17:48:03.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:03.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.432443+0000", 2024-09-04T17:48:03.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00044488500000000001 2024-09-04T17:48:03.761 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:03.761 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:03.761 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:03.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.761 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93847 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 4915200~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:03.761 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386681+0000", 2024-09-04T17:48:03.762 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.2717304279999997, 2024-09-04T17:48:03.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046305101000000001, 2024-09-04T17:48:03.762 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:03.762 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:03.762 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:03.762 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:03.762 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:03.762 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:03.762 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93847 2024-09-04T17:48:03.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.763 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:03.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.763 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:03.763 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386681+0000", 2024-09-04T17:48:03.763 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:03.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.763 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:03.763 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386681+0000", 2024-09-04T17:48:03.763 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:03.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.763 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:03.763 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386682+0000", 2024-09-04T17:48:03.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1699999999999998e-07 2024-09-04T17:48:03.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:03.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386684+0000", 2024-09-04T17:48:03.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.153e-06 2024-09-04T17:48:03.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:03.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386693+0000", 2024-09-04T17:48:03.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9639999999999992e-06 2024-09-04T17:48:03.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.766 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:03.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386698+0000", 2024-09-04T17:48:03.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3709999999999999e-06 2024-09-04T17:48:03.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.766 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:03.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.432476+0000", 2024-09-04T17:48:03.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045778147999999998 2024-09-04T17:48:03.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.766 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:03.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.432529+0000", 2024-09-04T17:48:03.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3165999999999999e-05 2024-09-04T17:48:03.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.767 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:03.767 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.432986+0000", 2024-09-04T17:48:03.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00045678199999999998 2024-09-04T17:48:03.767 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:03.767 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:03.767 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:03.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.767 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93848 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 5046272~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:03.767 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386705+0000", 2024-09-04T17:48:03.768 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.2717063270000004, 2024-09-04T17:48:03.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046825259000000001, 2024-09-04T17:48:03.768 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:03.768 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:03.768 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:03.768 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:03.768 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:03.768 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:03.768 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93848 2024-09-04T17:48:03.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.769 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:03.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:03.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386705+0000", 2024-09-04T17:48:03.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:03.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:03.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386705+0000", 2024-09-04T17:48:03.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:03.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:03.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386706+0000", 2024-09-04T17:48:03.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1500000000000005e-07 2024-09-04T17:48:03.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:03.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386708+0000", 2024-09-04T17:48:03.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2979999999999999e-06 2024-09-04T17:48:03.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:03.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386715+0000", 2024-09-04T17:48:03.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8669999999999996e-06 2024-09-04T17:48:03.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:03.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386720+0000", 2024-09-04T17:48:03.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.242e-06 2024-09-04T17:48:03.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:03.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.433019+0000", 2024-09-04T17:48:03.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046298359999999997 2024-09-04T17:48:03.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.772 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:03.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.433084+0000", 2024-09-04T17:48:03.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5395999999999995e-05 2024-09-04T17:48:03.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.772 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:03.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.433531+0000", 2024-09-04T17:48:03.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00044658099999999999 2024-09-04T17:48:03.772 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:03.772 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:03.772 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:03.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.773 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93849 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 5177344~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:03.773 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386727+0000", 2024-09-04T17:48:03.773 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.2716844290000004, 2024-09-04T17:48:03.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.047267796000000001, 2024-09-04T17:48:03.773 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:03.773 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:03.773 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:03.773 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:03.773 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:03.773 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:03.773 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93849 2024-09-04T17:48:03.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.774 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:03.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:03.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386727+0000", 2024-09-04T17:48:03.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:03.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:03.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386727+0000", 2024-09-04T17:48:03.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:03.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:03.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386728+0000", 2024-09-04T17:48:03.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1699999999999998e-07 2024-09-04T17:48:03.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:03.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386730+0000", 2024-09-04T17:48:03.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2589999999999999e-06 2024-09-04T17:48:03.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:03.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386751+0000", 2024-09-04T17:48:03.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1424999999999999e-05 2024-09-04T17:48:03.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:03.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386759+0000", 2024-09-04T17:48:03.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9400000000000002e-06 2024-09-04T17:48:03.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.777 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:03.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.433559+0000", 2024-09-04T17:48:03.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046799573999999997 2024-09-04T17:48:03.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.777 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:03.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.433610+0000", 2024-09-04T17:48:03.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1134999999999997e-05 2024-09-04T17:48:03.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:03.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.433995+0000", 2024-09-04T17:48:03.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000384946 2024-09-04T17:48:03.778 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:03.778 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:03.778 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:03.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.778 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:96175 3.12 3:4d4471be:::rbd_data.1134c4e2d26.000000000000006e:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123)", 2024-09-04T17:48:03.778 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:48:02.919542+0000", 2024-09-04T17:48:03.778 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.73886918499999998, 2024-09-04T17:48:03.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060180626000000001, 2024-09-04T17:48:03.778 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:03.779 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:03.779 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:03.779 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:03.779 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:03.779 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:03.779 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 96175 2024-09-04T17:48:03.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.779 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:03.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:03.779 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.919542+0000", 2024-09-04T17:48:03.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:03.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.780 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:03.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.919542+0000", 2024-09-04T17:48:03.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:03.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.780 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:03.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.919545+0000", 2024-09-04T17:48:03.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0199999999999999e-06 2024-09-04T17:48:03.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:03.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.919559+0000", 2024-09-04T17:48:03.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3293e-05 2024-09-04T17:48:03.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:03.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.919587+0000", 2024-09-04T17:48:03.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7922000000000001e-05 2024-09-04T17:48:03.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:03.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.919601+0000", 2024-09-04T17:48:03.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4657e-05 2024-09-04T17:48:03.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:03.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.919708+0000", 2024-09-04T17:48:03.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010624999999999999 2024-09-04T17:48:03.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:03.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.919929+0000", 2024-09-04T17:48:03.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022147399999999999 2024-09-04T17:48:03.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2024-09-04T17:48:03.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.929774+0000", 2024-09-04T17:48:03.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0098449839999999993 2024-09-04T17:48:03.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:03.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.979511+0000", 2024-09-04T17:48:03.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.049736537999999997 2024-09-04T17:48:03.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:03.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.979547+0000", 2024-09-04T17:48:03.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6069000000000003e-05 2024-09-04T17:48:03.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:03.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.979642+0000", 2024-09-04T17:48:03.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5526999999999995e-05 2024-09-04T17:48:03.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:03.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:03.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:03.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.979723+0000", 2024-09-04T17:48:03.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0891999999999997e-05 2024-09-04T17:48:03.784 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:03.784 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:03.784 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:03.785 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:03.785 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:03.785 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:03.786 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-04T17:48:03.787 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:03.786+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:03.863 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-04T17:48:03.909 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:03.908+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:03.917 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:03.917 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:48:03.917 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:48:03.917 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:03.933 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-04T17:48:03.988 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:48:03.982+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:48:04.089 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:04.082+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:04.100 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:04.100 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:48:04.100 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:48:04.100 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:48:04.100 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:04.115 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-04T17:48:04.189 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:48:04.186+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:48:04.237 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:04.237 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:48:04.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:48:04.238 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:48:04.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.238 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59950 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5365760~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:48:04.238 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654917+0000", 2024-09-04T17:48:04.238 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.580324378999997, 2024-09-04T17:48:04.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066822450000000005, 2024-09-04T17:48:04.238 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:04.238 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:04.238 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:04.238 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:04.238 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:04.239 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:04.239 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59950 2024-09-04T17:48:04.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.239 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:04.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.239 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:04.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654917+0000", 2024-09-04T17:48:04.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:04.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.239 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:04.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654917+0000", 2024-09-04T17:48:04.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:04.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:04.240 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654918+0000", 2024-09-04T17:48:04.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.29e-07 2024-09-04T17:48:04.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:04.240 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654919+0000", 2024-09-04T17:48:04.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2589999999999999e-06 2024-09-04T17:48:04.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:04.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654922+0000", 2024-09-04T17:48:04.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1350000000000001e-06 2024-09-04T17:48:04.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:04.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654925+0000", 2024-09-04T17:48:04.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1250000000000001e-06 2024-09-04T17:48:04.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:04.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721466+0000", 2024-09-04T17:48:04.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066540959999999996 2024-09-04T17:48:04.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:04.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721574+0000", 2024-09-04T17:48:04.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010747399999999999 2024-09-04T17:48:04.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:04.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721740+0000", 2024-09-04T17:48:04.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016626799999999999 2024-09-04T17:48:04.242 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:04.242 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:04.242 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:04.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.243 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59951 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5472256~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:48:04.243 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654929+0000", 2024-09-04T17:48:04.243 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.580313193999999, 2024-09-04T17:48:04.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067110534999999999, 2024-09-04T17:48:04.243 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:04.243 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:04.243 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:04.243 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:04.243 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:04.243 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:04.243 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59951 2024-09-04T17:48:04.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.244 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:04.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.244 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:04.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654929+0000", 2024-09-04T17:48:04.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:04.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.244 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:04.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654929+0000", 2024-09-04T17:48:04.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:04.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.245 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:04.245 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654929+0000", 2024-09-04T17:48:04.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5100000000000001e-07 2024-09-04T17:48:04.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.245 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:04.245 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654930+0000", 2024-09-04T17:48:04.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2440000000000001e-06 2024-09-04T17:48:04.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.245 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:04.245 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654933+0000", 2024-09-04T17:48:04.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1599999999999998e-06 2024-09-04T17:48:04.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:04.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654936+0000", 2024-09-04T17:48:04.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6560000000000001e-06 2024-09-04T17:48:04.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:04.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721789+0000", 2024-09-04T17:48:04.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066853250000000003 2024-09-04T17:48:04.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:04.247 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721885+0000", 2024-09-04T17:48:04.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5616000000000001e-05 2024-09-04T17:48:04.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:04.247 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722039+0000", 2024-09-04T17:48:04.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000154258 2024-09-04T17:48:04.247 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:04.247 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:04.247 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:04.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.247 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59952 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5505024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:48:04.247 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654939+0000", 2024-09-04T17:48:04.248 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.580302316999997, 2024-09-04T17:48:04.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067433742000000005, 2024-09-04T17:48:04.248 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:04.248 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:04.248 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:04.248 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:04.248 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:04.248 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:04.248 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59952 2024-09-04T17:48:04.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.248 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:04.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:04.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654939+0000", 2024-09-04T17:48:04.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:04.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:04.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654939+0000", 2024-09-04T17:48:04.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:04.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:04.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654940+0000", 2024-09-04T17:48:04.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3300000000000001e-07 2024-09-04T17:48:04.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:04.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654941+0000", 2024-09-04T17:48:04.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1570000000000001e-06 2024-09-04T17:48:04.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:04.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654944+0000", 2024-09-04T17:48:04.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1319999999999998e-06 2024-09-04T17:48:04.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:04.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654947+0000", 2024-09-04T17:48:04.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5969999999999999e-06 2024-09-04T17:48:04.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:04.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722089+0000", 2024-09-04T17:48:04.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067142347000000005 2024-09-04T17:48:04.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:04.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722201+0000", 2024-09-04T17:48:04.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011235600000000001 2024-09-04T17:48:04.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:04.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722373+0000", 2024-09-04T17:48:04.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017191999999999999 2024-09-04T17:48:04.252 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:04.252 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:04.252 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:04.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.252 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59953 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5611520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:48:04.252 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654950+0000", 2024-09-04T17:48:04.252 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.580291676999998, 2024-09-04T17:48:04.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067601852000000004, 2024-09-04T17:48:04.252 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:04.253 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:04.253 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:04.253 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:04.253 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:04.253 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:04.253 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59953 2024-09-04T17:48:04.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.253 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:04.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:04.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654950+0000", 2024-09-04T17:48:04.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:04.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:04.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654950+0000", 2024-09-04T17:48:04.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:04.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:04.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654950+0000", 2024-09-04T17:48:04.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2499999999999999e-07 2024-09-04T17:48:04.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:04.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654954+0000", 2024-09-04T17:48:04.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.287e-06 2024-09-04T17:48:04.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:04.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654962+0000", 2024-09-04T17:48:04.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4230000000000005e-06 2024-09-04T17:48:04.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:04.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654966+0000", 2024-09-04T17:48:04.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6449999999999998e-06 2024-09-04T17:48:04.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:04.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722417+0000", 2024-09-04T17:48:04.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067451188999999995 2024-09-04T17:48:04.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:04.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722467+0000", 2024-09-04T17:48:04.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0556999999999997e-05 2024-09-04T17:48:04.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:04.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722552+0000", 2024-09-04T17:48:04.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4525999999999997e-05 2024-09-04T17:48:04.257 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:04.257 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:04.257 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:04.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.257 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59954 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5718016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:48:04.257 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654970+0000", 2024-09-04T17:48:04.257 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.580271916000001, 2024-09-04T17:48:04.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067874285000000006, 2024-09-04T17:48:04.257 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:04.257 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:04.257 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:04.258 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:04.258 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:04.258 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:04.258 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59954 2024-09-04T17:48:04.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.258 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:04.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:04.258 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654970+0000", 2024-09-04T17:48:04.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:04.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:04.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654970+0000", 2024-09-04T17:48:04.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:04.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:04.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654970+0000", 2024-09-04T17:48:04.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5899999999999998e-07 2024-09-04T17:48:04.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:04.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654971+0000", 2024-09-04T17:48:04.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2929999999999999e-06 2024-09-04T17:48:04.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:04.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654975+0000", 2024-09-04T17:48:04.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4699999999999998e-06 2024-09-04T17:48:04.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:04.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654978+0000", 2024-09-04T17:48:04.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7930000000000002e-06 2024-09-04T17:48:04.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:04.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722600+0000", 2024-09-04T17:48:04.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067622229000000006 2024-09-04T17:48:04.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:04.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722689+0000", 2024-09-04T17:48:04.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9511000000000004e-05 2024-09-04T17:48:04.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:04.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722844+0000", 2024-09-04T17:48:04.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015473 2024-09-04T17:48:04.261 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:04.262 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:04.262 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:04.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.262 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59955 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5857280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:48:04.262 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654981+0000", 2024-09-04T17:48:04.262 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.580260566, 2024-09-04T17:48:04.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068052360000000006, 2024-09-04T17:48:04.262 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:04.262 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:04.262 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:04.262 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:04.262 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:04.262 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:04.263 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59955 2024-09-04T17:48:04.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.263 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:04.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:04.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654981+0000", 2024-09-04T17:48:04.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:04.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:04.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654981+0000", 2024-09-04T17:48:04.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:04.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:04.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654981+0000", 2024-09-04T17:48:04.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2399999999999999e-07 2024-09-04T17:48:04.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:04.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654985+0000", 2024-09-04T17:48:04.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0980000000000004e-06 2024-09-04T17:48:04.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:04.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654990+0000", 2024-09-04T17:48:04.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3429999999999998e-06 2024-09-04T17:48:04.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:04.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654993+0000", 2024-09-04T17:48:04.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8700000000000001e-06 2024-09-04T17:48:04.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:04.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722888+0000", 2024-09-04T17:48:04.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067895239999999996 2024-09-04T17:48:04.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:04.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722948+0000", 2024-09-04T17:48:04.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9750999999999998e-05 2024-09-04T17:48:04.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:04.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.723034+0000", 2024-09-04T17:48:04.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5834000000000005e-05 2024-09-04T17:48:04.267 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:04.267 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:04.267 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:04.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.267 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59956 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5963776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:48:04.267 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654998+0000", 2024-09-04T17:48:04.267 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.580244037, 2024-09-04T17:48:04.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068359518999999994, 2024-09-04T17:48:04.267 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:04.267 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:04.267 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:04.268 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:04.268 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:04.268 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:04.268 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59956 2024-09-04T17:48:04.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.268 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:04.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:04.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654998+0000", 2024-09-04T17:48:04.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:04.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:04.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654998+0000", 2024-09-04T17:48:04.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:04.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:04.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654998+0000", 2024-09-04T17:48:04.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4299999999999999e-07 2024-09-04T17:48:04.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:04.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654999+0000", 2024-09-04T17:48:04.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1799999999999999e-06 2024-09-04T17:48:04.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:04.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.655002+0000", 2024-09-04T17:48:04.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3639999999999999e-06 2024-09-04T17:48:04.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:04.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.655005+0000", 2024-09-04T17:48:04.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7970000000000001e-06 2024-09-04T17:48:04.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:04.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.723085+0000", 2024-09-04T17:48:04.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068080229000000006 2024-09-04T17:48:04.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:04.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.723197+0000", 2024-09-04T17:48:04.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000111344 2024-09-04T17:48:04.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:04.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.723357+0000", 2024-09-04T17:48:04.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016036199999999999 2024-09-04T17:48:04.272 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:04.272 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:04.272 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:04.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.272 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61082 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 2654208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:04.272 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.822664+0000", 2024-09-04T17:48:04.272 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.412577650000003, 2024-09-04T17:48:04.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067072242000000004, 2024-09-04T17:48:04.272 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:04.272 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:04.272 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:04.273 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:04.273 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:04.273 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:04.273 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61082 2024-09-04T17:48:04.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.273 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:04.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:04.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822664+0000", 2024-09-04T17:48:04.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:04.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:04.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822664+0000", 2024-09-04T17:48:04.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:04.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:04.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822664+0000", 2024-09-04T17:48:04.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6600000000000003e-07 2024-09-04T17:48:04.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:04.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822666+0000", 2024-09-04T17:48:04.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.536e-06 2024-09-04T17:48:04.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:04.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822669+0000", 2024-09-04T17:48:04.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3759999999999999e-06 2024-09-04T17:48:04.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:04.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822672+0000", 2024-09-04T17:48:04.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8109999999999999e-06 2024-09-04T17:48:04.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:04.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889263+0000", 2024-09-04T17:48:04.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066591259999999999 2024-09-04T17:48:04.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:04.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889369+0000", 2024-09-04T17:48:04.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000105362 2024-09-04T17:48:04.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:04.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889736+0000", 2024-09-04T17:48:04.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00036763099999999999 2024-09-04T17:48:04.277 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:04.277 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:04.277 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:04.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.277 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61083 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 2760704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:04.277 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.822678+0000", 2024-09-04T17:48:04.277 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.412563916000003, 2024-09-04T17:48:04.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067475996999999996, 2024-09-04T17:48:04.277 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:04.277 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:04.277 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:04.278 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:04.278 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:04.278 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:04.278 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61083 2024-09-04T17:48:04.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.278 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:04.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:04.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822678+0000", 2024-09-04T17:48:04.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:04.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:04.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822678+0000", 2024-09-04T17:48:04.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:04.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:04.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822725+0000", 2024-09-04T17:48:04.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7429000000000003e-05 2024-09-04T17:48:04.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:04.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822728+0000", 2024-09-04T17:48:04.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2140000000000001e-06 2024-09-04T17:48:04.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:04.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822737+0000", 2024-09-04T17:48:04.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4370000000000007e-06 2024-09-04T17:48:04.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:04.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822743+0000", 2024-09-04T17:48:04.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7520000000000002e-06 2024-09-04T17:48:04.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:04.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889785+0000", 2024-09-04T17:48:04.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067042265000000004 2024-09-04T17:48:04.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:04.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889840+0000", 2024-09-04T17:48:04.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5254000000000003e-05 2024-09-04T17:48:04.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:04.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.890154+0000", 2024-09-04T17:48:04.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000313646 2024-09-04T17:48:04.281 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:04.282 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:04.282 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:04.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.282 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61084 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 2867200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:04.282 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.822753+0000", 2024-09-04T17:48:04.282 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.412488979999999, 2024-09-04T17:48:04.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068497161000000001, 2024-09-04T17:48:04.282 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:04.282 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:04.282 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:04.282 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:04.282 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:04.283 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:04.283 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61084 2024-09-04T17:48:04.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.283 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:04.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:04.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822753+0000", 2024-09-04T17:48:04.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:04.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:04.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822753+0000", 2024-09-04T17:48:04.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:04.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:04.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822753+0000", 2024-09-04T17:48:04.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9100000000000003e-07 2024-09-04T17:48:04.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:04.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822756+0000", 2024-09-04T17:48:04.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.678e-06 2024-09-04T17:48:04.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:04.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822763+0000", 2024-09-04T17:48:04.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8220000000000001e-06 2024-09-04T17:48:04.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:04.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822768+0000", 2024-09-04T17:48:04.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1930000000000001e-06 2024-09-04T17:48:04.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:04.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.890246+0000", 2024-09-04T17:48:04.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067477996999999998 2024-09-04T17:48:04.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:04.286 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.890555+0000", 2024-09-04T17:48:04.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000309143 2024-09-04T17:48:04.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:04.286 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.891250+0000", 2024-09-04T17:48:04.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00069463700000000001 2024-09-04T17:48:04.286 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:04.287 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:04.287 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:04.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.287 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61285 3.8 3:117cfeb9:::rbd_data.11341dfde0c5.0000000000000006:1f [read 2760704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:04.287 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835200+0000", 2024-09-04T17:48:04.287 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.400041913000003, 2024-09-04T17:48:04.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066899803999999993, 2024-09-04T17:48:04.287 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:04.287 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:04.287 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:04.287 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:04.287 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:04.287 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:04.288 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61285 2024-09-04T17:48:04.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.288 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:04.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:04.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835200+0000", 2024-09-04T17:48:04.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:04.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:04.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835200+0000", 2024-09-04T17:48:04.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:04.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:04.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835200+0000", 2024-09-04T17:48:04.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0899999999999997e-07 2024-09-04T17:48:04.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:04.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835206+0000", 2024-09-04T17:48:04.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3280000000000002e-06 2024-09-04T17:48:04.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:04.290 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835211+0000", 2024-09-04T17:48:04.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9230000000000001e-06 2024-09-04T17:48:04.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:04.290 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835216+0000", 2024-09-04T17:48:04.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9019999999999998e-06 2024-09-04T17:48:04.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:04.290 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.901719+0000", 2024-09-04T17:48:04.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066502981000000003 2024-09-04T17:48:04.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:04.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.901813+0000", 2024-09-04T17:48:04.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3905000000000003e-05 2024-09-04T17:48:04.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:04.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902100+0000", 2024-09-04T17:48:04.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028645599999999998 2024-09-04T17:48:04.291 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:04.291 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:04.291 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:04.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.292 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61286 3.8 3:117cfeb9:::rbd_data.11341dfde0c5.0000000000000006:1f [read 2867200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:04.292 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835244+0000", 2024-09-04T17:48:04.292 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.399997294000002, 2024-09-04T17:48:04.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067143215000000006, 2024-09-04T17:48:04.292 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:04.292 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:04.292 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:04.292 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:04.292 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:04.292 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:04.292 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61286 2024-09-04T17:48:04.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.292 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:04.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:04.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835244+0000", 2024-09-04T17:48:04.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:04.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:04.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835244+0000", 2024-09-04T17:48:04.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:04.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:04.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835245+0000", 2024-09-04T17:48:04.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8400000000000004e-07 2024-09-04T17:48:04.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:04.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835248+0000", 2024-09-04T17:48:04.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6850000000000001e-06 2024-09-04T17:48:04.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:04.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835254+0000", 2024-09-04T17:48:04.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.494e-06 2024-09-04T17:48:04.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:04.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835260+0000", 2024-09-04T17:48:04.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0580000000000003e-06 2024-09-04T17:48:04.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:04.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902119+0000", 2024-09-04T17:48:04.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066858318 2024-09-04T17:48:04.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:04.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902163+0000", 2024-09-04T17:48:04.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4753000000000003e-05 2024-09-04T17:48:04.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.296 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:04.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902388+0000", 2024-09-04T17:48:04.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022422300000000001 2024-09-04T17:48:04.296 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:04.296 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:04.296 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:04.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.296 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61315 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 98304~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:04.296 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835269+0000", 2024-09-04T17:48:04.296 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.399972501999997, 2024-09-04T17:48:04.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067329396, 2024-09-04T17:48:04.297 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:04.297 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:04.297 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:04.297 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:04.297 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:04.297 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:04.297 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61315 2024-09-04T17:48:04.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.297 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:04.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:04.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835269+0000", 2024-09-04T17:48:04.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:04.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:04.298 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835269+0000", 2024-09-04T17:48:04.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:04.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:04.298 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835270+0000", 2024-09-04T17:48:04.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6599999999999996e-07 2024-09-04T17:48:04.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:04.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835272+0000", 2024-09-04T17:48:04.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.261e-06 2024-09-04T17:48:04.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:04.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835278+0000", 2024-09-04T17:48:04.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.443e-06 2024-09-04T17:48:04.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:04.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835284+0000", 2024-09-04T17:48:04.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3290000000000001e-06 2024-09-04T17:48:04.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.300 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:04.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902147+0000", 2024-09-04T17:48:04.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066862695999999999 2024-09-04T17:48:04.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.300 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:04.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902248+0000", 2024-09-04T17:48:04.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000101501 2024-09-04T17:48:04.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.300 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:04.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902599+0000", 2024-09-04T17:48:04.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00035060000000000001 2024-09-04T17:48:04.301 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:04.301 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:04.301 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:04.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.301 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61316 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 204800~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:04.301 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835291+0000", 2024-09-04T17:48:04.301 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.399950482999998, 2024-09-04T17:48:04.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067632052999999998, 2024-09-04T17:48:04.301 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:04.301 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:04.301 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:04.302 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:04.302 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:04.302 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:04.302 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61316 2024-09-04T17:48:04.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.302 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:04.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:04.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835291+0000", 2024-09-04T17:48:04.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:04.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:04.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835291+0000", 2024-09-04T17:48:04.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:04.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.303 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:04.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835292+0000", 2024-09-04T17:48:04.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1899999999999998e-07 2024-09-04T17:48:04.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.303 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:04.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835294+0000", 2024-09-04T17:48:04.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0880000000000002e-06 2024-09-04T17:48:04.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:04.304 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835300+0000", 2024-09-04T17:48:04.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.224e-06 2024-09-04T17:48:04.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:04.304 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835305+0000", 2024-09-04T17:48:04.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0730000000000004e-06 2024-09-04T17:48:04.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:04.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902632+0000", 2024-09-04T17:48:04.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067327322999999994 2024-09-04T17:48:04.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:04.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902688+0000", 2024-09-04T17:48:04.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5177999999999999e-05 2024-09-04T17:48:04.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:04.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902923+0000", 2024-09-04T17:48:04.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023574800000000001 2024-09-04T17:48:04.305 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:04.306 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:04.306 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:04.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.306 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61317 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 344064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:04.306 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835312+0000", 2024-09-04T17:48:04.306 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.399929479000001, 2024-09-04T17:48:04.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067918511000000001, 2024-09-04T17:48:04.306 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:04.306 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:04.306 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:04.306 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:04.306 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:04.307 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:04.307 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61317 2024-09-04T17:48:04.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.307 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:04.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:04.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835312+0000", 2024-09-04T17:48:04.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:04.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:04.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835312+0000", 2024-09-04T17:48:04.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:04.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.308 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:04.308 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835313+0000", 2024-09-04T17:48:04.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9000000000000002e-07 2024-09-04T17:48:04.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.308 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:04.308 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835315+0000", 2024-09-04T17:48:04.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1689999999999999e-06 2024-09-04T17:48:04.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.308 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:04.308 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835321+0000", 2024-09-04T17:48:04.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0619999999999997e-06 2024-09-04T17:48:04.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.309 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:04.309 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835326+0000", 2024-09-04T17:48:04.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0390000000000004e-06 2024-09-04T17:48:04.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.309 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:04.309 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902967+0000", 2024-09-04T17:48:04.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067641520999999996 2024-09-04T17:48:04.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:04.310 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903067+0000", 2024-09-04T17:48:04.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9426999999999995e-05 2024-09-04T17:48:04.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:04.310 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903231+0000", 2024-09-04T17:48:04.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016390299999999999 2024-09-04T17:48:04.310 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:04.310 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:04.310 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:04.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.311 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61318 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 450560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:04.311 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835334+0000", 2024-09-04T17:48:04.311 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.399907962, 2024-09-04T17:48:04.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068091771999999995, 2024-09-04T17:48:04.311 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:04.311 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:04.311 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:04.311 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:04.311 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:04.311 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:04.311 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61318 2024-09-04T17:48:04.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.311 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:04.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.312 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:04.312 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835334+0000", 2024-09-04T17:48:04.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:04.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.312 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:04.312 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835334+0000", 2024-09-04T17:48:04.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:04.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.312 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:04.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835334+0000", 2024-09-04T17:48:04.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5699999999999998e-07 2024-09-04T17:48:04.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:04.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835337+0000", 2024-09-04T17:48:04.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3889999999999999e-06 2024-09-04T17:48:04.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:04.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835343+0000", 2024-09-04T17:48:04.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1809999999999998e-06 2024-09-04T17:48:04.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.314 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:04.314 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835348+0000", 2024-09-04T17:48:04.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3170000000000001e-06 2024-09-04T17:48:04.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.314 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:04.314 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903270+0000", 2024-09-04T17:48:04.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067922191000000007 2024-09-04T17:48:04.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.314 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:04.314 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903321+0000", 2024-09-04T17:48:04.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0889e-05 2024-09-04T17:48:04.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.315 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:04.315 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903426+0000", 2024-09-04T17:48:04.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104348 2024-09-04T17:48:04.315 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:04.315 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:04.315 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:04.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.315 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61319 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 589824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:04.315 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835357+0000", 2024-09-04T17:48:04.315 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.399885144999999, 2024-09-04T17:48:04.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068372535999999998, 2024-09-04T17:48:04.316 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:04.316 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:04.316 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:04.316 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:04.316 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:04.316 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:04.316 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61319 2024-09-04T17:48:04.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.316 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:04.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:04.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835357+0000", 2024-09-04T17:48:04.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:04.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:04.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835357+0000", 2024-09-04T17:48:04.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:04.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:04.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835357+0000", 2024-09-04T17:48:04.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.32e-07 2024-09-04T17:48:04.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:04.318 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835359+0000", 2024-09-04T17:48:04.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1660000000000001e-06 2024-09-04T17:48:04.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.318 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:04.318 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835366+0000", 2024-09-04T17:48:04.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3550000000000001e-06 2024-09-04T17:48:04.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.318 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:04.318 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835371+0000", 2024-09-04T17:48:04.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1050000000000003e-06 2024-09-04T17:48:04.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:04.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903471+0000", 2024-09-04T17:48:04.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068100451000000006 2024-09-04T17:48:04.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:04.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903573+0000", 2024-09-04T17:48:04.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000101691 2024-09-04T17:48:04.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:04.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903729+0000", 2024-09-04T17:48:04.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000156336 2024-09-04T17:48:04.320 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:04.320 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:04.320 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:04.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.320 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61320 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:04.320 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835380+0000", 2024-09-04T17:48:04.320 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.399861379999997, 2024-09-04T17:48:04.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068543058000000004, 2024-09-04T17:48:04.320 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:04.321 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:04.321 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:04.321 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:04.321 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:04.321 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:04.321 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61320 2024-09-04T17:48:04.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.321 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:04.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:04.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835380+0000", 2024-09-04T17:48:04.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:04.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:04.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835380+0000", 2024-09-04T17:48:04.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:04.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:04.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835381+0000", 2024-09-04T17:48:04.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9999999999999998e-07 2024-09-04T17:48:04.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:04.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835383+0000", 2024-09-04T17:48:04.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2230000000000001e-06 2024-09-04T17:48:04.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.323 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:04.323 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835390+0000", 2024-09-04T17:48:04.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.34e-06 2024-09-04T17:48:04.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.323 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:04.323 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835396+0000", 2024-09-04T17:48:04.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9830000000000001e-06 2024-09-04T17:48:04.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.323 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:04.324 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903765+0000", 2024-09-04T17:48:04.324 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068368441000000002 2024-09-04T17:48:04.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.324 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.324 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:04.324 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903835+0000", 2024-09-04T17:48:04.324 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0631999999999997e-05 2024-09-04T17:48:04.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.324 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.324 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:04.324 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903923+0000", 2024-09-04T17:48:04.324 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8039000000000002e-05 2024-09-04T17:48:04.325 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:04.325 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:04.325 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:04.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.325 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61321 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 835584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:04.325 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835404+0000", 2024-09-04T17:48:04.325 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.399837321, 2024-09-04T17:48:04.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068911043000000005, 2024-09-04T17:48:04.325 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:04.325 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:04.325 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:04.325 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:04.325 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:04.326 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:04.326 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61321 2024-09-04T17:48:04.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.326 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:04.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:04.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835404+0000", 2024-09-04T17:48:04.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:04.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:04.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835404+0000", 2024-09-04T17:48:04.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:04.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:04.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835405+0000", 2024-09-04T17:48:04.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.89e-07 2024-09-04T17:48:04.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:04.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835407+0000", 2024-09-04T17:48:04.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4439999999999998e-06 2024-09-04T17:48:04.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:04.328 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835414+0000", 2024-09-04T17:48:04.328 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4930000000000003e-06 2024-09-04T17:48:04.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.328 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.328 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:04.328 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835419+0000", 2024-09-04T17:48:04.328 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8919999999999999e-06 2024-09-04T17:48:04.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.328 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.328 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:04.328 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903957+0000", 2024-09-04T17:48:04.328 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068538500000000002 2024-09-04T17:48:04.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.329 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:04.329 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904019+0000", 2024-09-04T17:48:04.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1419000000000003e-05 2024-09-04T17:48:04.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.329 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:04.329 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904315+0000", 2024-09-04T17:48:04.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00029680599999999999 2024-09-04T17:48:04.329 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:04.329 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:04.329 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:04.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.330 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61322 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:04.330 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835427+0000", 2024-09-04T17:48:04.330 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.399814935000002, 2024-09-04T17:48:04.330 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.069550138999999997, 2024-09-04T17:48:04.330 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:04.330 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:04.331 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:04.331 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:04.331 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:04.331 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:04.331 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61322 2024-09-04T17:48:04.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.331 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:04.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.331 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:04.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835427+0000", 2024-09-04T17:48:04.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:04.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.331 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:04.332 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835427+0000", 2024-09-04T17:48:04.332 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:04.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.332 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.332 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:04.332 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.842157+0000", 2024-09-04T17:48:04.332 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067298419999999998 2024-09-04T17:48:04.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.332 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.332 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:04.332 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.842163+0000", 2024-09-04T17:48:04.332 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2029999999999997e-06 2024-09-04T17:48:04.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.333 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.333 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:04.333 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.842175+0000", 2024-09-04T17:48:04.333 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2072000000000001e-05 2024-09-04T17:48:04.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.333 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.333 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:04.333 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.842183+0000", 2024-09-04T17:48:04.333 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3850000000000002e-06 2024-09-04T17:48:04.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.333 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.333 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:04.333 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904371+0000", 2024-09-04T17:48:04.334 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.062187754999999997 2024-09-04T17:48:04.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.334 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:04.334 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904467+0000", 2024-09-04T17:48:04.334 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6117000000000001e-05 2024-09-04T17:48:04.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:04.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:04.334 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:04.334 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904977+0000", 2024-09-04T17:48:04.334 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00050976500000000002 2024-09-04T17:48:04.334 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:04.334 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:04.335 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:04.335 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:04.335 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:04.335 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:04.336 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-04T17:48:04.337 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:04.334+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:04.437 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:04.436+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:04.467 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:04.467 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:48:04.467 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:48:04.467 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:04.481 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-04T17:48:04.538 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:48:04.536+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:48:04.589 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-04T17:48:04.600 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:04.600 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:48:04.600 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:48:04.600 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:48:04.600 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:04.614 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-04T17:48:04.661 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:04.658+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:04.739 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:04.739 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:48:04.739 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:48:04.739 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:48:04.739 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.740 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:12722 3.1b 3:d8152af1:::rbd_data.1134ef3a5a7d.00000000000000bd:head [delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-04T17:48:04.740 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:16.421965+0000", 2024-09-04T17:48:04.740 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 108.316831732, 2024-09-04T17:48:04.740 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.890002811, 2024-09-04T17:48:04.740 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:04.740 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:04.740 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:04.740 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:04.740 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:04.741 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:04.741 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 12722 2024-09-04T17:48:04.741 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.741 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:04.741 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.741 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:04.741 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421965+0000", 2024-09-04T17:48:04.741 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:04.741 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.741 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.741 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:04.741 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421965+0000", 2024-09-04T17:48:04.742 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:04.742 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.742 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.742 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:04.742 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421966+0000", 2024-09-04T17:48:04.742 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.102e-06 2024-09-04T17:48:04.742 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.742 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.742 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:04.742 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421971+0000", 2024-09-04T17:48:04.742 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9989999999999999e-06 2024-09-04T17:48:04.742 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.742 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.743 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:04.743 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421981+0000", 2024-09-04T17:48:04.743 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.7380000000000006e-06 2024-09-04T17:48:04.743 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.743 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.743 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:04.743 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421989+0000", 2024-09-04T17:48:04.743 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.1890000000000007e-06 2024-09-04T17:48:04.743 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.743 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.743 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:04.743 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.422050+0000", 2024-09-04T17:48:04.743 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.0702000000000003e-05 2024-09-04T17:48:04.744 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.744 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.744 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for peered", 2024-09-04T17:48:04.744 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.422059+0000", 2024-09-04T17:48:04.744 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5440000000000004e-06 2024-09-04T17:48:04.744 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.744 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.744 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:04.744 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.308156+0000", 2024-09-04T17:48:04.744 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.88609676199999998 2024-09-04T17:48:04.744 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.744 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.744 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:04.745 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.308398+0000", 2024-09-04T17:48:04.745 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024215400000000001 2024-09-04T17:48:04.745 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.745 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.745 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 7", 2024-09-04T17:48:04.745 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.308795+0000", 2024-09-04T17:48:04.745 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000396704 2024-09-04T17:48:04.745 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.745 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.745 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:48:04.745 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.310831+0000", 2024-09-04T17:48:04.745 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0020364200000000002 2024-09-04T17:48:04.745 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.746 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.746 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:04.746 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.311818+0000", 2024-09-04T17:48:04.746 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00098649799999999993 2024-09-04T17:48:04.746 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.746 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.746 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:04.746 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.311905+0000", 2024-09-04T17:48:04.746 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7070999999999994e-05 2024-09-04T17:48:04.746 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.746 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.746 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:04.746 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.311968+0000", 2024-09-04T17:48:04.746 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.2928000000000005e-05 2024-09-04T17:48:04.747 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:04.747 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:04.747 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:04.747 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.747 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.747 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37359 3.f e58/24 3:f29e50a5:::rbd_data.1134c5a43465.0000000000000021:head v 58'135, mlcod=58'133)", 2024-09-04T17:48:04.747 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:48:04.747 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 85.121501128000006, 2024-09-04T17:48:04.747 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053483197000000003, 2024-09-04T17:48:04.747 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:04.747 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:04.747 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:04.747 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:04.747 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.748 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:04.748 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:48:04.748 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:04.748 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.748 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.748 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:04.748 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:48:04.748 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:04.748 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.748 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.748 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:04.748 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:48:04.748 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1699999999999997e-07 2024-09-04T17:48:04.749 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.749 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.749 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:04.749 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617338+0000", 2024-09-04T17:48:04.749 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.1168999999999998e-05 2024-09-04T17:48:04.749 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.749 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.749 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:04.749 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617351+0000", 2024-09-04T17:48:04.749 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3094e-05 2024-09-04T17:48:04.749 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.749 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.749 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:04.749 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617359+0000", 2024-09-04T17:48:04.750 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7970000000000006e-06 2024-09-04T17:48:04.750 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.750 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.750 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:04.750 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617417+0000", 2024-09-04T17:48:04.750 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7079e-05 2024-09-04T17:48:04.750 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.750 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.750 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:04.750 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617438+0000", 2024-09-04T17:48:04.750 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1654999999999998e-05 2024-09-04T17:48:04.750 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.750 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.751 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:04.751 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670738+0000", 2024-09-04T17:48:04.751 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053300142000000002 2024-09-04T17:48:04.751 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.751 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.751 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:04.751 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670779+0000", 2024-09-04T17:48:04.751 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0343999999999999e-05 2024-09-04T17:48:04.752 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:04.752 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:04.752 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:04.752 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.752 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.752 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37358 3.1d e58/24 3:bd3215f3:::rbd_data.1134c5a43465.0000000000000020:head v 58'91, mlcod=58'89)", 2024-09-04T17:48:04.752 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617502+0000", 2024-09-04T17:48:04.752 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 85.121294410000004, 2024-09-04T17:48:04.752 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053505641999999999, 2024-09-04T17:48:04.752 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:04.752 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:04.752 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:04.752 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:04.752 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.753 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:04.753 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617502+0000", 2024-09-04T17:48:04.753 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:04.753 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.753 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.753 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:04.753 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617502+0000", 2024-09-04T17:48:04.753 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:04.753 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.753 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.753 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:04.753 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617503+0000", 2024-09-04T17:48:04.753 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.184e-06 2024-09-04T17:48:04.754 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.754 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.754 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:04.754 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617526+0000", 2024-09-04T17:48:04.754 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2099999999999998e-05 2024-09-04T17:48:04.754 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.754 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.754 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:04.754 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617540+0000", 2024-09-04T17:48:04.754 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4741e-05 2024-09-04T17:48:04.754 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.754 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.754 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:04.755 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617550+0000", 2024-09-04T17:48:04.755 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4259999999999992e-06 2024-09-04T17:48:04.755 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.755 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.755 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:04.755 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617613+0000", 2024-09-04T17:48:04.755 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.3625000000000002e-05 2024-09-04T17:48:04.755 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.755 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.755 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:04.755 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617637+0000", 2024-09-04T17:48:04.755 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.3604999999999998e-05 2024-09-04T17:48:04.755 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.755 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.756 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:04.756 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670967+0000", 2024-09-04T17:48:04.756 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053329881000000003 2024-09-04T17:48:04.756 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.756 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.756 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:04.756 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671008+0000", 2024-09-04T17:48:04.756 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.108e-05 2024-09-04T17:48:04.756 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:04.756 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:04.756 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:04.756 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.757 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.757 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37361 3.1c e58/24 3:3c7f8976:::rbd_data.1134c5a43465.0000000000000023:head v 58'159, mlcod=58'157)", 2024-09-04T17:48:04.757 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617578+0000", 2024-09-04T17:48:04.757 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 85.121218560000003, 2024-09-04T17:48:04.757 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053990367999999997, 2024-09-04T17:48:04.757 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:04.757 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:04.757 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:04.757 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:04.757 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.757 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:04.757 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617578+0000", 2024-09-04T17:48:04.757 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:04.757 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.758 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.758 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:04.758 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617578+0000", 2024-09-04T17:48:04.758 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:04.758 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.758 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.758 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:04.758 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617579+0000", 2024-09-04T17:48:04.758 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7299999999999996e-07 2024-09-04T17:48:04.758 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.758 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.758 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:04.758 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617610+0000", 2024-09-04T17:48:04.759 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0951000000000002e-05 2024-09-04T17:48:04.759 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.759 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.759 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:04.759 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:48:04.759 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0002e-05 2024-09-04T17:48:04.759 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.759 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.759 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:04.759 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617626+0000", 2024-09-04T17:48:04.759 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6660000000000002e-06 2024-09-04T17:48:04.759 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.759 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.759 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:04.760 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670548+0000", 2024-09-04T17:48:04.760 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.052921943999999999 2024-09-04T17:48:04.760 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.760 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.760 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:04.760 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670574+0000", 2024-09-04T17:48:04.760 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5405000000000002e-05 2024-09-04T17:48:04.760 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.760 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.760 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:04.760 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671522+0000", 2024-09-04T17:48:04.760 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00094815299999999997 2024-09-04T17:48:04.760 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.761 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.761 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:04.761 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671568+0000", 2024-09-04T17:48:04.761 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6674000000000001e-05 2024-09-04T17:48:04.761 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:04.761 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:04.761 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:04.761 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.761 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.761 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37357 3.15 e58/49 3:a897312a:::rbd_data.1134c5a43465.000000000000001f:head v 58'117, mlcod=58'115)", 2024-09-04T17:48:04.761 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:48:04.761 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 85.121176222000003, 2024-09-04T17:48:04.762 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053734297, 2024-09-04T17:48:04.762 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:04.762 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:04.762 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:04.762 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:04.762 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.762 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:04.762 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:48:04.762 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:04.762 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.762 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.762 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:04.762 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:48:04.762 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:04.763 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.763 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.763 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:04.763 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617621+0000", 2024-09-04T17:48:04.763 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.7300000000000004e-07 2024-09-04T17:48:04.763 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.763 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.763 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:04.763 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617642+0000", 2024-09-04T17:48:04.763 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0185000000000001e-05 2024-09-04T17:48:04.763 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.763 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.764 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:04.764 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617654+0000", 2024-09-04T17:48:04.764 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2745000000000001e-05 2024-09-04T17:48:04.764 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.764 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.764 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:04.764 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617662+0000", 2024-09-04T17:48:04.764 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.0499999999999992e-06 2024-09-04T17:48:04.764 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.764 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.764 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:04.764 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617727+0000", 2024-09-04T17:48:04.765 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.4923000000000002e-05 2024-09-04T17:48:04.765 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.765 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.765 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:04.765 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617753+0000", 2024-09-04T17:48:04.765 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5899999999999999e-05 2024-09-04T17:48:04.765 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.765 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.765 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:04.765 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671298+0000", 2024-09-04T17:48:04.766 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053545276000000003 2024-09-04T17:48:04.766 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.766 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.766 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:04.766 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671355+0000", 2024-09-04T17:48:04.766 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6245000000000001e-05 2024-09-04T17:48:04.766 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:04.766 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:04.766 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:04.766 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.766 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.766 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44687 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:04.766 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253012+0000", 2024-09-04T17:48:04.766 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 74.485784155000005, 2024-09-04T17:48:04.767 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051128777, 2024-09-04T17:48:04.767 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:04.767 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:04.767 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:04.767 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:04.767 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:04.767 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:04.767 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44687 2024-09-04T17:48:04.767 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.767 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:04.767 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.767 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:04.767 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253012+0000", 2024-09-04T17:48:04.767 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:04.768 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.768 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.768 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:04.768 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253012+0000", 2024-09-04T17:48:04.768 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:04.768 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.768 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.768 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:04.768 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253014+0000", 2024-09-04T17:48:04.768 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.24e-06 2024-09-04T17:48:04.768 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.768 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.769 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:04.769 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253019+0000", 2024-09-04T17:48:04.769 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4569999999999996e-06 2024-09-04T17:48:04.769 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.769 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.769 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:04.770 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253035+0000", 2024-09-04T17:48:04.770 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6291999999999999e-05 2024-09-04T17:48:04.770 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.770 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.770 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:04.770 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253050+0000", 2024-09-04T17:48:04.770 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4423000000000001e-05 2024-09-04T17:48:04.770 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.770 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.770 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:04.770 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304115+0000", 2024-09-04T17:48:04.770 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051064778999999998 2024-09-04T17:48:04.770 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.771 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.771 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:04.771 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304141+0000", 2024-09-04T17:48:04.771 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6585999999999999e-05 2024-09-04T17:48:04.771 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:04.771 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:04.771 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:04.771 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.771 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.771 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44688 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:04.771 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253067+0000", 2024-09-04T17:48:04.771 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 74.485729602999996, 2024-09-04T17:48:04.771 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051116979, 2024-09-04T17:48:04.771 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:04.772 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:04.772 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:04.772 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:04.772 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:04.772 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:04.772 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44688 2024-09-04T17:48:04.772 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.772 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:04.772 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.772 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:04.772 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253067+0000", 2024-09-04T17:48:04.772 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:04.772 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.773 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.773 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:04.773 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253067+0000", 2024-09-04T17:48:04.773 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:04.773 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.773 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.773 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:04.773 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253068+0000", 2024-09-04T17:48:04.773 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0729999999999999e-06 2024-09-04T17:48:04.773 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.773 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.773 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:04.773 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253073+0000", 2024-09-04T17:48:04.774 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2259999999999996e-06 2024-09-04T17:48:04.774 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.774 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.774 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:04.774 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253091+0000", 2024-09-04T17:48:04.774 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7255999999999999e-05 2024-09-04T17:48:04.774 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.774 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.774 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:04.774 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253103+0000", 2024-09-04T17:48:04.774 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2632999999999999e-05 2024-09-04T17:48:04.774 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.775 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.775 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:04.775 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304173+0000", 2024-09-04T17:48:04.775 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051069537999999998 2024-09-04T17:48:04.775 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.775 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.775 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:04.775 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304184+0000", 2024-09-04T17:48:04.775 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1253000000000001e-05 2024-09-04T17:48:04.775 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:04.775 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:04.775 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:04.775 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.775 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.776 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44689 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:04.776 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253121+0000", 2024-09-04T17:48:04.776 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 74.485675466000004, 2024-09-04T17:48:04.776 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051192595, 2024-09-04T17:48:04.776 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:04.776 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:04.776 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:04.776 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:04.776 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:04.776 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:04.776 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44689 2024-09-04T17:48:04.776 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.777 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:04.777 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.777 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:04.777 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253121+0000", 2024-09-04T17:48:04.777 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:04.777 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.777 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.777 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:04.777 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253121+0000", 2024-09-04T17:48:04.777 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:04.777 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.777 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.777 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:04.778 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253122+0000", 2024-09-04T17:48:04.778 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.15e-06 2024-09-04T17:48:04.778 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.778 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.778 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:04.778 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253128+0000", 2024-09-04T17:48:04.778 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2569999999999998e-06 2024-09-04T17:48:04.778 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.778 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.778 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:04.778 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253145+0000", 2024-09-04T17:48:04.778 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7176e-05 2024-09-04T17:48:04.779 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.779 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.779 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:04.779 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253158+0000", 2024-09-04T17:48:04.779 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3139000000000001e-05 2024-09-04T17:48:04.779 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.779 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.779 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:04.779 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304287+0000", 2024-09-04T17:48:04.779 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051129149999999998 2024-09-04T17:48:04.779 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.779 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.779 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:04.780 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304314+0000", 2024-09-04T17:48:04.780 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6723000000000001e-05 2024-09-04T17:48:04.780 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:04.780 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:04.780 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:04.780 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.780 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.780 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44690 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:04.780 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253177+0000", 2024-09-04T17:48:04.780 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 74.485619341000003, 2024-09-04T17:48:04.780 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051169263, 2024-09-04T17:48:04.780 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:04.780 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:04.781 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:04.781 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:04.781 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:04.781 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:04.781 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44690 2024-09-04T17:48:04.781 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.781 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:04.781 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.781 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:04.781 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253177+0000", 2024-09-04T17:48:04.781 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:04.781 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.781 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.782 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:04.782 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253177+0000", 2024-09-04T17:48:04.782 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:04.782 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.782 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.782 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:04.782 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253178+0000", 2024-09-04T17:48:04.782 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1000000000000001e-06 2024-09-04T17:48:04.782 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.782 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.782 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:04.782 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253184+0000", 2024-09-04T17:48:04.782 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2089999999999996e-06 2024-09-04T17:48:04.782 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.783 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.783 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:04.783 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253200+0000", 2024-09-04T17:48:04.783 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6087e-05 2024-09-04T17:48:04.783 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.783 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.783 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:04.783 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253231+0000", 2024-09-04T17:48:04.783 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0874999999999998e-05 2024-09-04T17:48:04.783 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.783 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.783 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:04.783 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304337+0000", 2024-09-04T17:48:04.784 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051106430000000001 2024-09-04T17:48:04.784 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.784 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.784 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:04.784 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304347+0000", 2024-09-04T17:48:04.784 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5619999999999993e-06 2024-09-04T17:48:04.784 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:04.784 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:04.784 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:04.784 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.784 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.784 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44691 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:04.784 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253249+0000", 2024-09-04T17:48:04.785 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 74.485547307000004, 2024-09-04T17:48:04.785 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051195630999999998, 2024-09-04T17:48:04.785 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:04.785 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:04.785 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:04.785 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:04.785 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:04.785 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:04.785 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44691 2024-09-04T17:48:04.785 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.785 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:04.785 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.785 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:04.785 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253249+0000", 2024-09-04T17:48:04.786 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:04.786 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.786 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.786 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:04.786 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253249+0000", 2024-09-04T17:48:04.786 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:04.786 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.786 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.786 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:04.786 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253251+0000", 2024-09-04T17:48:04.786 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2330000000000001e-06 2024-09-04T17:48:04.786 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.787 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.787 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:04.787 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253256+0000", 2024-09-04T17:48:04.787 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6099999999999997e-06 2024-09-04T17:48:04.787 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.787 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.787 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:04.787 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253274+0000", 2024-09-04T17:48:04.787 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7952999999999999e-05 2024-09-04T17:48:04.787 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.787 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.787 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:04.787 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253287+0000", 2024-09-04T17:48:04.787 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2839e-05 2024-09-04T17:48:04.788 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.788 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.788 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:04.788 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304396+0000", 2024-09-04T17:48:04.788 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051108990999999999 2024-09-04T17:48:04.788 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.788 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.788 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:04.788 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304445+0000", 2024-09-04T17:48:04.788 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9005000000000003e-05 2024-09-04T17:48:04.788 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:04.788 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:04.788 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:04.789 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.789 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.789 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44692 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:04.789 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253337+0000", 2024-09-04T17:48:04.789 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 74.485459734000003, 2024-09-04T17:48:04.789 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051138909000000003, 2024-09-04T17:48:04.789 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:04.789 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:04.789 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:04.789 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:04.789 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:04.789 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:04.789 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44692 2024-09-04T17:48:04.790 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.790 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:04.790 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.790 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:04.790 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253337+0000", 2024-09-04T17:48:04.790 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:04.790 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.790 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.790 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:04.790 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253337+0000", 2024-09-04T17:48:04.790 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:04.790 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.790 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.790 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:04.791 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253338+0000", 2024-09-04T17:48:04.791 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.463e-06 2024-09-04T17:48:04.791 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.791 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.791 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:04.791 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253357+0000", 2024-09-04T17:48:04.791 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9043e-05 2024-09-04T17:48:04.791 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.791 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.791 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:04.791 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253379+0000", 2024-09-04T17:48:04.791 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1243e-05 2024-09-04T17:48:04.791 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.792 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.792 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:04.792 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253396+0000", 2024-09-04T17:48:04.792 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7059e-05 2024-09-04T17:48:04.792 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.792 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.792 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:04.792 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304466+0000", 2024-09-04T17:48:04.792 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051070624000000002 2024-09-04T17:48:04.792 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.792 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.792 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:04.792 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304476+0000", 2024-09-04T17:48:04.793 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4769999999999992e-06 2024-09-04T17:48:04.793 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:04.793 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:04.793 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:04.793 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.793 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.793 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44693 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:04.793 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253415+0000", 2024-09-04T17:48:04.793 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 74.485381330999999, 2024-09-04T17:48:04.793 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051141949999999999, 2024-09-04T17:48:04.793 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:04.793 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:04.793 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:04.793 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:04.794 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:04.794 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:04.794 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44693 2024-09-04T17:48:04.794 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.794 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:04.794 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.794 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:04.794 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253415+0000", 2024-09-04T17:48:04.794 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:04.794 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.794 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.794 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:04.794 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253415+0000", 2024-09-04T17:48:04.795 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:04.795 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.795 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.795 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:04.795 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253416+0000", 2024-09-04T17:48:04.795 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1799999999999999e-06 2024-09-04T17:48:04.795 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.795 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.795 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:04.795 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253422+0000", 2024-09-04T17:48:04.795 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.5940000000000001e-06 2024-09-04T17:48:04.795 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.795 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.796 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:04.796 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253438+0000", 2024-09-04T17:48:04.796 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6008999999999999e-05 2024-09-04T17:48:04.796 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.796 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.796 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:04.796 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253452+0000", 2024-09-04T17:48:04.796 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3543000000000001e-05 2024-09-04T17:48:04.796 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.796 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.796 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:04.796 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304533+0000", 2024-09-04T17:48:04.796 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051081609 2024-09-04T17:48:04.796 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.797 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.797 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:04.797 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304557+0000", 2024-09-04T17:48:04.797 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4015e-05 2024-09-04T17:48:04.797 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:04.797 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:04.797 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:04.797 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.797 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.797 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44694 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:04.797 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253469+0000", 2024-09-04T17:48:04.797 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 74.485327205999994, 2024-09-04T17:48:04.797 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051121486000000001, 2024-09-04T17:48:04.798 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:04.798 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:04.798 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:04.798 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:04.798 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:04.798 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:04.798 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44694 2024-09-04T17:48:04.798 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.798 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:04.798 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.798 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:04.798 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253469+0000", 2024-09-04T17:48:04.798 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:04.798 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.799 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.799 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:04.799 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253469+0000", 2024-09-04T17:48:04.799 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:04.799 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.799 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.799 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:04.799 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253471+0000", 2024-09-04T17:48:04.799 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1400000000000001e-06 2024-09-04T17:48:04.799 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.799 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.799 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:04.799 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253476+0000", 2024-09-04T17:48:04.800 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7370000000000003e-06 2024-09-04T17:48:04.800 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.800 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.800 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:04.800 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253493+0000", 2024-09-04T17:48:04.800 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6549e-05 2024-09-04T17:48:04.800 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.800 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.800 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:04.800 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253506+0000", 2024-09-04T17:48:04.800 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3406e-05 2024-09-04T17:48:04.800 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.800 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.801 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:04.801 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304582+0000", 2024-09-04T17:48:04.801 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051075534999999998 2024-09-04T17:48:04.801 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.801 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.801 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:04.801 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304591+0000", 2024-09-04T17:48:04.801 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1190000000000003e-06 2024-09-04T17:48:04.801 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:04.801 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:04.801 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:04.801 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.801 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.802 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44695 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:04.802 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253552+0000", 2024-09-04T17:48:04.802 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 74.485244452000003, 2024-09-04T17:48:04.802 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051068872000000001, 2024-09-04T17:48:04.802 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:04.802 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:04.802 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:04.802 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:04.802 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:04.802 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:04.802 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44695 2024-09-04T17:48:04.802 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.802 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:04.803 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.803 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:04.803 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253552+0000", 2024-09-04T17:48:04.803 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:04.803 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.803 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.803 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:04.803 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253552+0000", 2024-09-04T17:48:04.803 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:04.803 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.803 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.803 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:04.803 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253553+0000", 2024-09-04T17:48:04.804 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1960000000000001e-06 2024-09-04T17:48:04.804 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.804 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.804 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:04.804 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253560+0000", 2024-09-04T17:48:04.804 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.2040000000000002e-06 2024-09-04T17:48:04.804 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.804 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.804 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:04.804 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253578+0000", 2024-09-04T17:48:04.804 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8241999999999999e-05 2024-09-04T17:48:04.804 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.804 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.805 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:04.805 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253591+0000", 2024-09-04T17:48:04.805 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3453e-05 2024-09-04T17:48:04.805 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.805 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.805 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:04.805 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304614+0000", 2024-09-04T17:48:04.805 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051022456000000001 2024-09-04T17:48:04.805 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.805 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.805 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:04.805 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304621+0000", 2024-09-04T17:48:04.805 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3209999999999999e-06 2024-09-04T17:48:04.806 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:04.806 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:04.806 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:04.806 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.806 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.806 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44696 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:04.806 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253609+0000", 2024-09-04T17:48:04.806 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 74.485187951, 2024-09-04T17:48:04.806 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051115351000000003, 2024-09-04T17:48:04.806 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:04.806 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:04.806 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:04.807 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:04.807 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:04.807 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:04.807 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44696 2024-09-04T17:48:04.807 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.807 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:04.807 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.807 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:04.807 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253609+0000", 2024-09-04T17:48:04.807 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:04.807 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.807 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.807 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:04.807 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253609+0000", 2024-09-04T17:48:04.808 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:04.808 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.808 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.808 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:04.808 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253610+0000", 2024-09-04T17:48:04.808 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2300000000000001e-06 2024-09-04T17:48:04.808 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.808 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.808 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:04.808 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253615+0000", 2024-09-04T17:48:04.808 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7260000000000002e-06 2024-09-04T17:48:04.808 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.809 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.809 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:04.809 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253632+0000", 2024-09-04T17:48:04.809 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7039999999999999e-05 2024-09-04T17:48:04.809 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.809 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.809 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:04.809 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253666+0000", 2024-09-04T17:48:04.809 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4662000000000001e-05 2024-09-04T17:48:04.809 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.809 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.809 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:04.809 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304677+0000", 2024-09-04T17:48:04.810 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051011017999999998 2024-09-04T17:48:04.810 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.810 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.810 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:04.810 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304724+0000", 2024-09-04T17:48:04.810 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6675000000000003e-05 2024-09-04T17:48:04.810 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:04.810 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:04.810 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:04.810 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.810 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.810 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44697 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:04.810 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253684+0000", 2024-09-04T17:48:04.811 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 74.485112457, 2024-09-04T17:48:04.811 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051073107, 2024-09-04T17:48:04.811 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:04.811 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:04.811 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:04.811 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:04.811 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:04.811 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:04.811 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44697 2024-09-04T17:48:04.811 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.811 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:04.811 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.811 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:04.811 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253684+0000", 2024-09-04T17:48:04.812 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:04.812 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.812 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.812 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:04.812 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253684+0000", 2024-09-04T17:48:04.812 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:04.812 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.812 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.812 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:04.812 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253685+0000", 2024-09-04T17:48:04.812 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0699999999999999e-06 2024-09-04T17:48:04.812 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.812 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.813 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:04.813 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253690+0000", 2024-09-04T17:48:04.813 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1959999999999999e-06 2024-09-04T17:48:04.813 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.813 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.813 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:04.813 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253707+0000", 2024-09-04T17:48:04.813 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6946e-05 2024-09-04T17:48:04.813 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.813 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.813 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:04.813 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253720+0000", 2024-09-04T17:48:04.813 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3076e-05 2024-09-04T17:48:04.814 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.814 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.814 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:04.814 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304745+0000", 2024-09-04T17:48:04.814 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051024760000000002 2024-09-04T17:48:04.814 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.814 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.814 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:04.814 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304757+0000", 2024-09-04T17:48:04.814 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2058999999999999e-05 2024-09-04T17:48:04.814 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:04.814 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:04.814 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:04.815 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.815 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.815 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44698 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:04.815 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253737+0000", 2024-09-04T17:48:04.815 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 74.485060106000006, 2024-09-04T17:48:04.815 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051176078, 2024-09-04T17:48:04.815 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:04.815 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:04.815 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:04.815 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:04.815 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:04.815 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:04.815 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44698 2024-09-04T17:48:04.815 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.816 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:04.816 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.816 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:04.816 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253737+0000", 2024-09-04T17:48:04.816 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:04.816 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.816 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.816 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:04.816 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253737+0000", 2024-09-04T17:48:04.816 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:04.816 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.816 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.817 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:04.817 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253738+0000", 2024-09-04T17:48:04.817 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0300000000000001e-06 2024-09-04T17:48:04.817 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.817 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.817 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:04.817 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253752+0000", 2024-09-04T17:48:04.817 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4358999999999999e-05 2024-09-04T17:48:04.817 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.817 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.817 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:04.817 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253771+0000", 2024-09-04T17:48:04.817 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8879999999999999e-05 2024-09-04T17:48:04.818 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.818 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.818 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:04.818 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253785+0000", 2024-09-04T17:48:04.818 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3852000000000001e-05 2024-09-04T17:48:04.818 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.818 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.818 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:04.818 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304876+0000", 2024-09-04T17:48:04.818 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051091105999999997 2024-09-04T17:48:04.818 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.818 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.818 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:04.819 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304913+0000", 2024-09-04T17:48:04.819 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.6850999999999997e-05 2024-09-04T17:48:04.819 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:04.819 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:04.819 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:04.819 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.819 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.819 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44699 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:04.819 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253804+0000", 2024-09-04T17:48:04.819 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 74.484992582000004, 2024-09-04T17:48:04.819 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051172250000000002, 2024-09-04T17:48:04.819 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:04.820 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:04.820 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:04.820 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:04.820 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:04.820 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:04.820 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44699 2024-09-04T17:48:04.820 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.820 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:04.820 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.820 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:04.820 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253804+0000", 2024-09-04T17:48:04.820 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:04.820 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.821 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.821 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:04.821 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253804+0000", 2024-09-04T17:48:04.821 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:04.821 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.821 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.821 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:04.821 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253805+0000", 2024-09-04T17:48:04.821 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.08e-06 2024-09-04T17:48:04.821 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.821 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.821 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:04.821 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253810+0000", 2024-09-04T17:48:04.822 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1830000000000002e-06 2024-09-04T17:48:04.822 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.822 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.822 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:04.822 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253828+0000", 2024-09-04T17:48:04.822 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7662000000000001e-05 2024-09-04T17:48:04.822 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.822 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.822 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:04.822 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253841+0000", 2024-09-04T17:48:04.822 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3203999999999999e-05 2024-09-04T17:48:04.822 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.822 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.823 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:04.823 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304966+0000", 2024-09-04T17:48:04.823 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051125273999999998 2024-09-04T17:48:04.823 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.823 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.823 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:04.823 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304976+0000", 2024-09-04T17:48:04.823 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.8470000000000008e-06 2024-09-04T17:48:04.823 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:04.823 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:04.823 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:04.823 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.823 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.823 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44700 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:04.824 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253858+0000", 2024-09-04T17:48:04.824 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 74.484938928000005, 2024-09-04T17:48:04.824 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051218957000000002, 2024-09-04T17:48:04.824 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:04.824 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:04.824 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:04.824 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:04.824 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:04.824 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:04.824 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44700 2024-09-04T17:48:04.824 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.824 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:04.824 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.825 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:04.825 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253858+0000", 2024-09-04T17:48:04.825 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:04.825 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.825 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.825 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:04.825 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253858+0000", 2024-09-04T17:48:04.825 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:04.825 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.825 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.825 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:04.825 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253859+0000", 2024-09-04T17:48:04.825 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.217e-06 2024-09-04T17:48:04.826 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.826 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.826 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:04.826 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253864+0000", 2024-09-04T17:48:04.826 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7600000000000002e-06 2024-09-04T17:48:04.826 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.826 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.826 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:04.826 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253880+0000", 2024-09-04T17:48:04.826 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6501999999999999e-05 2024-09-04T17:48:04.826 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.826 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.826 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:04.827 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253899+0000", 2024-09-04T17:48:04.827 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8907e-05 2024-09-04T17:48:04.827 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.827 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.827 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:04.827 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305052+0000", 2024-09-04T17:48:04.827 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051152989000000003 2024-09-04T17:48:04.827 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.827 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.827 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:04.827 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305077+0000", 2024-09-04T17:48:04.827 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4581999999999999e-05 2024-09-04T17:48:04.827 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:04.828 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:04.828 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:04.828 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.828 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.828 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44701 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:04.828 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253917+0000", 2024-09-04T17:48:04.828 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 74.484879575999997, 2024-09-04T17:48:04.828 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051231113000000002, 2024-09-04T17:48:04.828 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:04.828 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:04.828 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:04.828 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:04.828 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:04.828 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:04.829 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44701 2024-09-04T17:48:04.829 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.829 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:04.829 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.829 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:04.829 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253917+0000", 2024-09-04T17:48:04.829 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:04.829 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.829 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.829 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:04.829 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253917+0000", 2024-09-04T17:48:04.829 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:04.829 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.830 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.830 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:04.830 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253918+0000", 2024-09-04T17:48:04.830 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.136e-06 2024-09-04T17:48:04.830 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.830 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.830 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:04.830 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253924+0000", 2024-09-04T17:48:04.830 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.3070000000000002e-06 2024-09-04T17:48:04.830 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.830 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.830 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:04.830 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253940+0000", 2024-09-04T17:48:04.830 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6612e-05 2024-09-04T17:48:04.831 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.831 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.831 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:04.831 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253954+0000", 2024-09-04T17:48:04.831 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3709999999999999e-05 2024-09-04T17:48:04.831 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.831 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.831 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:04.831 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305136+0000", 2024-09-04T17:48:04.831 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051181935999999997 2024-09-04T17:48:04.831 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:04.831 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:04.831 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:04.832 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305148+0000", 2024-09-04T17:48:04.832 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2412000000000001e-05 2024-09-04T17:48:04.832 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:04.832 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:04.832 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:04.832 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:04.832 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:04.832 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:04.833 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 4, 5, 6, 7] out_osds: [3, 1] dead_osds: [] live_osds: [0, 1, 4, 5, 7, 3, 2, 6] 2024-09-04T17:48:04.833 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-04T17:48:04.833 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:48:04.834 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:48:04.835 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:48:04.835 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:48:04.835 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-04T17:48:04.836 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:48:04.830+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:48:04.836 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-04T17:48:04.836 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-04T17:48:04.837 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-04T17:48:04.837 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-04T17:48:04.837 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:48:04.838 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:48:04.838 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:48:04.838 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:48:04.855 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-04T17:48:04.855 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-04T17:48:04.861 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is unset 2024-09-04T17:48:04.894 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:48:04.894 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:48:04.957 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:48:04.956+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:48:05.030 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:05.030 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:05.034 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:05.032+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:05.045 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-04T17:48:05.135 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:05.130+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:05.170 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:05.170 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:05.184 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-04T17:48:05.273 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:05.270+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:05.337 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:05.330+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:05.355 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:05.355 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:05.369 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-04T17:48:05.457 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:05.454+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:05.537 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:05.537 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:05.538 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:48:05.536+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:48:05.552 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-04T17:48:05.639 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:48:05.634+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:48:05.715 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:05.715 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:05.730 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-04T17:48:05.773 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:05.770+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:05.841 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:48:05.836+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:48:05.879 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.0 to 1 (10000) 2024-09-04T17:48:05.908 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-04T17:48:05.908 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:05.908 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:05.925 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-04T17:48:05.942 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:05.938+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:06.042 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:06.038+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:06.076 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:06.076 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:06.093 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-04T17:48:06.144 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:06.138+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:06.207 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:06.207 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:06.221 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-04T17:48:06.245 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:06.242+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:06.343 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:06.343 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:06.345 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:48:06.342+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:48:06.355 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-04T17:48:06.446 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:48:06.444+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:48:06.476 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:06.477 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:06.491 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-04T17:48:06.577 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:48:06.576+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:48:06.648 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:06.644+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:06.659 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:06.659 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:06.673 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-04T17:48:06.765 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:48:06.764+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:48:06.847 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:06.847 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:06.850 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:06.848+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:06.862 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-04T17:48:06.875 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.2 to 1 (10000) 2024-09-04T17:48:06.891 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-04T17:48:06.904 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-04T17:48:06.914 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-04T17:48:06.993 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:48:06.992+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:48:06.999 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:06.999 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:07.013 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-04T17:48:07.057 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:07.054+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:07.086 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:48:07.086 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:48:07.106 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:48:07.132 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:07.132 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:07.146 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-04T17:48:07.153 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:48:07.148+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:48:07.253 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:48:07.250+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:48:07.312 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:07.313 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:07.327 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-04T17:48:07.330 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-04T17:48:07.330 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-04T17:48:07.354 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-04T17:48:07.354 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-04T17:48:07.369 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:48:07.366+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:48:07.379 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:48:07.379 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:48:07.402 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:48:07.402 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:48:07.427 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-04T17:48:07.427 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-04T17:48:07.452 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:48:07.452 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:48:07.452 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:07.452 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:48:07.452 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:48:07.453 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:07.454 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:07.452+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:07.468 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-04T17:48:07.555 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:07.550+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:07.638 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:07.638 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:48:07.638 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:48:07.638 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:48:07.638 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:07.652 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-04T17:48:07.697 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:48:07.696+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:48:07.757 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:07.754+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:07.773 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:07.773 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:48:07.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:48:07.774 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:48:07.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.774 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:48444 3.12 3:4ff80fb5:::rbd_data.11346a2b0bc9.000000000000001d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-04T17:48:07.774 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:58.122601+0000", 2024-09-04T17:48:07.774 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 69.648192961000007, 2024-09-04T17:48:07.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057065713999999997, 2024-09-04T17:48:07.774 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:07.774 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:07.774 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:07.774 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:07.774 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:07.774 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:07.774 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 48444 2024-09-04T17:48:07.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.775 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:07.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:07.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122601+0000", 2024-09-04T17:48:07.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:07.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:07.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122601+0000", 2024-09-04T17:48:07.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:07.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:07.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122603+0000", 2024-09-04T17:48:07.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3499999999999999e-06 2024-09-04T17:48:07.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:07.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122614+0000", 2024-09-04T17:48:07.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.063e-05 2024-09-04T17:48:07.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:07.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122643+0000", 2024-09-04T17:48:07.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9434999999999999e-05 2024-09-04T17:48:07.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.777 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:07.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122657+0000", 2024-09-04T17:48:07.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3920000000000001e-05 2024-09-04T17:48:07.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.777 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:07.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122767+0000", 2024-09-04T17:48:07.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010981599999999999 2024-09-04T17:48:07.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:07.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122989+0000", 2024-09-04T17:48:07.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022158200000000001 2024-09-04T17:48:07.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 3", 2024-09-04T17:48:07.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.133349+0000", 2024-09-04T17:48:07.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010360729000000001 2024-09-04T17:48:07.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:07.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.179423+0000", 2024-09-04T17:48:07.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046073408000000003 2024-09-04T17:48:07.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:07.779 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.179453+0000", 2024-09-04T17:48:07.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0229000000000001e-05 2024-09-04T17:48:07.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:07.779 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.179545+0000", 2024-09-04T17:48:07.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1580000000000001e-05 2024-09-04T17:48:07.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:07.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.179667+0000", 2024-09-04T17:48:07.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012203500000000001 2024-09-04T17:48:07.780 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:07.780 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:07.780 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:07.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.780 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:55376 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-04T17:48:07.780 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:11.974053+0000", 2024-09-04T17:48:07.780 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.796741177000001, 2024-09-04T17:48:07.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060514722, 2024-09-04T17:48:07.780 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:07.781 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:07.781 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:07.781 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:07.781 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:07.781 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:07.781 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 55376 2024-09-04T17:48:07.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.781 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:07.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:07.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974053+0000", 2024-09-04T17:48:07.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:07.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:07.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974053+0000", 2024-09-04T17:48:07.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:07.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:07.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974054+0000", 2024-09-04T17:48:07.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.841e-06 2024-09-04T17:48:07.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:07.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974062+0000", 2024-09-04T17:48:07.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5460000000000003e-06 2024-09-04T17:48:07.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:07.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974077+0000", 2024-09-04T17:48:07.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5389000000000001e-05 2024-09-04T17:48:07.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:07.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974085+0000", 2024-09-04T17:48:07.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7659999999999998e-06 2024-09-04T17:48:07.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:07.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974188+0000", 2024-09-04T17:48:07.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000102521 2024-09-04T17:48:07.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:07.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974321+0000", 2024-09-04T17:48:07.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013324699999999999 2024-09-04T17:48:07.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-04T17:48:07.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.986481+0000", 2024-09-04T17:48:07.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012160450999999999 2024-09-04T17:48:07.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.785 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:07.785 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:12.034415+0000", 2024-09-04T17:48:07.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.047934065999999997 2024-09-04T17:48:07.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.785 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:07.785 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:12.034438+0000", 2024-09-04T17:48:07.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2923999999999999e-05 2024-09-04T17:48:07.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:07.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:12.034505+0000", 2024-09-04T17:48:07.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6711000000000006e-05 2024-09-04T17:48:07.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:07.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:12.034567+0000", 2024-09-04T17:48:07.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2260000000000004e-05 2024-09-04T17:48:07.786 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:07.786 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:07.786 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:07.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.787 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61139 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 2547712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:07.787 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819750+0000", 2024-09-04T17:48:07.787 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 53.951043462999998, 2024-09-04T17:48:07.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042185699, 2024-09-04T17:48:07.787 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:07.787 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:07.787 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:07.787 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:07.787 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:07.787 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:07.787 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61139 2024-09-04T17:48:07.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.787 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:07.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:07.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819750+0000", 2024-09-04T17:48:07.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:07.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:07.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819750+0000", 2024-09-04T17:48:07.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:07.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:07.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819751+0000", 2024-09-04T17:48:07.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4000000000000002e-07 2024-09-04T17:48:07.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:07.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819753+0000", 2024-09-04T17:48:07.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5179999999999999e-06 2024-09-04T17:48:07.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:07.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819760+0000", 2024-09-04T17:48:07.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5440000000000004e-06 2024-09-04T17:48:07.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:07.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819765+0000", 2024-09-04T17:48:07.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8760000000000004e-06 2024-09-04T17:48:07.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.790 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:07.790 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.861622+0000", 2024-09-04T17:48:07.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.041857093999999997 2024-09-04T17:48:07.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.790 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:07.790 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.861698+0000", 2024-09-04T17:48:07.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6080000000000003e-05 2024-09-04T17:48:07.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:07.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.861936+0000", 2024-09-04T17:48:07.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023804699999999999 2024-09-04T17:48:07.791 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:07.791 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:07.791 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:07.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.791 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61140 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 2686976~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:07.791 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819772+0000", 2024-09-04T17:48:07.791 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 53.951021329, 2024-09-04T17:48:07.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042515052999999997, 2024-09-04T17:48:07.791 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:07.791 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:07.791 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:07.791 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:07.792 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:07.792 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:07.792 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61140 2024-09-04T17:48:07.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.792 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:07.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:07.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819772+0000", 2024-09-04T17:48:07.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:07.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:07.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819772+0000", 2024-09-04T17:48:07.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:07.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:07.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819773+0000", 2024-09-04T17:48:07.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2900000000000003e-07 2024-09-04T17:48:07.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:07.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819775+0000", 2024-09-04T17:48:07.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3309999999999998e-06 2024-09-04T17:48:07.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:07.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819781+0000", 2024-09-04T17:48:07.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.981e-06 2024-09-04T17:48:07.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:07.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819786+0000", 2024-09-04T17:48:07.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5719999999999996e-06 2024-09-04T17:48:07.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:07.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.861966+0000", 2024-09-04T17:48:07.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042179541000000001 2024-09-04T17:48:07.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:07.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862038+0000", 2024-09-04T17:48:07.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2719000000000006e-05 2024-09-04T17:48:07.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:07.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862288+0000", 2024-09-04T17:48:07.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024928 2024-09-04T17:48:07.795 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:07.795 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:07.795 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:07.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.795 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61141 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 2793472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:07.795 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819909+0000", 2024-09-04T17:48:07.796 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 53.950885184999997, 2024-09-04T17:48:07.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042786419999999999, 2024-09-04T17:48:07.796 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:07.796 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:07.796 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:07.796 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:07.796 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:07.796 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:07.796 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61141 2024-09-04T17:48:07.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.796 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:07.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:07.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819909+0000", 2024-09-04T17:48:07.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:07.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:07.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819909+0000", 2024-09-04T17:48:07.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:07.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:07.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819909+0000", 2024-09-04T17:48:07.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6300000000000005e-07 2024-09-04T17:48:07.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:07.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819913+0000", 2024-09-04T17:48:07.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4189999999999998e-06 2024-09-04T17:48:07.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:07.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819920+0000", 2024-09-04T17:48:07.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8619999999999997e-06 2024-09-04T17:48:07.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:07.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819926+0000", 2024-09-04T17:48:07.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9399999999999999e-06 2024-09-04T17:48:07.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:07.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862331+0000", 2024-09-04T17:48:07.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042405348000000002 2024-09-04T17:48:07.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:07.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862410+0000", 2024-09-04T17:48:07.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9078000000000004e-05 2024-09-04T17:48:07.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:07.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862695+0000", 2024-09-04T17:48:07.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028510999999999999 2024-09-04T17:48:07.800 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:07.800 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:07.800 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:07.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.800 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61142 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 2932736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:07.800 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819934+0000", 2024-09-04T17:48:07.800 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 53.950859567000002, 2024-09-04T17:48:07.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043101985000000002, 2024-09-04T17:48:07.800 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:07.800 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:07.800 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:07.801 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:07.801 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:07.801 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:07.801 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61142 2024-09-04T17:48:07.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.801 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:07.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:07.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819934+0000", 2024-09-04T17:48:07.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:07.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:07.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819934+0000", 2024-09-04T17:48:07.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:07.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:07.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819935+0000", 2024-09-04T17:48:07.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2024-09-04T17:48:07.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:07.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819937+0000", 2024-09-04T17:48:07.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5770000000000001e-06 2024-09-04T17:48:07.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:07.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819943+0000", 2024-09-04T17:48:07.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7950000000000004e-06 2024-09-04T17:48:07.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:07.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819948+0000", 2024-09-04T17:48:07.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6779999999999999e-06 2024-09-04T17:48:07.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:07.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862739+0000", 2024-09-04T17:48:07.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042791454999999999 2024-09-04T17:48:07.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:07.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862824+0000", 2024-09-04T17:48:07.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4790999999999995e-05 2024-09-04T17:48:07.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:07.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.863036+0000", 2024-09-04T17:48:07.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021224899999999999 2024-09-04T17:48:07.805 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:07.805 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:07.805 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:07.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.805 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61143 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 3039232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:07.805 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819954+0000", 2024-09-04T17:48:07.805 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 53.950839449, 2024-09-04T17:48:07.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043485134000000002, 2024-09-04T17:48:07.805 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:07.805 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:07.805 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:07.805 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:07.805 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:07.805 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:07.806 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61143 2024-09-04T17:48:07.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.806 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:07.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:07.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819954+0000", 2024-09-04T17:48:07.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:07.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:07.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819954+0000", 2024-09-04T17:48:07.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:07.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:07.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819955+0000", 2024-09-04T17:48:07.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7300000000000002e-07 2024-09-04T17:48:07.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:07.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819957+0000", 2024-09-04T17:48:07.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.024e-06 2024-09-04T17:48:07.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:07.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819963+0000", 2024-09-04T17:48:07.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8109999999999999e-06 2024-09-04T17:48:07.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:07.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819967+0000", 2024-09-04T17:48:07.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6149999999999999e-06 2024-09-04T17:48:07.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:07.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.863082+0000", 2024-09-04T17:48:07.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043115078000000001 2024-09-04T17:48:07.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:07.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.863172+0000", 2024-09-04T17:48:07.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0181999999999998e-05 2024-09-04T17:48:07.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:07.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.863440+0000", 2024-09-04T17:48:07.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026705099999999997 2024-09-04T17:48:07.809 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:07.809 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:07.809 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:07.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.809 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:70015 3.6 3:6012e780:::rbd_data.11341aa245a3.000000000000005b:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-04T17:48:07.809 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:26.155497+0000", 2024-09-04T17:48:07.809 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 41.615296391999998, 2024-09-04T17:48:07.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.049007124999999999, 2024-09-04T17:48:07.810 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:07.810 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:07.810 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:07.810 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:07.810 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:07.810 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:07.810 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 70015 2024-09-04T17:48:07.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.810 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:07.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:07.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155497+0000", 2024-09-04T17:48:07.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:07.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:07.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155497+0000", 2024-09-04T17:48:07.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:07.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:07.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155500+0000", 2024-09-04T17:48:07.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7070000000000001e-06 2024-09-04T17:48:07.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:07.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155513+0000", 2024-09-04T17:48:07.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2989e-05 2024-09-04T17:48:07.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:07.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155544+0000", 2024-09-04T17:48:07.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0747000000000002e-05 2024-09-04T17:48:07.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:07.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155560+0000", 2024-09-04T17:48:07.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5866e-05 2024-09-04T17:48:07.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:07.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155675+0000", 2024-09-04T17:48:07.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000114838 2024-09-04T17:48:07.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:07.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155803+0000", 2024-09-04T17:48:07.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012851799999999999 2024-09-04T17:48:07.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 1", 2024-09-04T17:48:07.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.163582+0000", 2024-09-04T17:48:07.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0077790960000000001 2024-09-04T17:48:07.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:07.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.204055+0000", 2024-09-04T17:48:07.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.040472517 2024-09-04T17:48:07.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:07.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.204195+0000", 2024-09-04T17:48:07.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014055200000000001 2024-09-04T17:48:07.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:07.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.204373+0000", 2024-09-04T17:48:07.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000177784 2024-09-04T17:48:07.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:07.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.204505+0000", 2024-09-04T17:48:07.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013151100000000001 2024-09-04T17:48:07.815 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:07.815 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:07.815 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:07.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.815 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:70434 3.12 3:4cd2342b:::rbd_data.11341aa245a3.0000000000000091:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-04T17:48:07.815 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:26.943124+0000", 2024-09-04T17:48:07.815 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.827669610000001, 2024-09-04T17:48:07.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055011831999999997, 2024-09-04T17:48:07.816 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:07.816 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:07.816 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:07.816 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:07.816 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:07.816 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:07.816 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 70434 2024-09-04T17:48:07.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.816 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:07.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:07.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943124+0000", 2024-09-04T17:48:07.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:07.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:07.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943124+0000", 2024-09-04T17:48:07.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:07.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:07.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943128+0000", 2024-09-04T17:48:07.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8460000000000001e-06 2024-09-04T17:48:07.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:07.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943142+0000", 2024-09-04T17:48:07.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4089e-05 2024-09-04T17:48:07.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:07.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943177+0000", 2024-09-04T17:48:07.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5167999999999999e-05 2024-09-04T17:48:07.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:07.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943194+0000", 2024-09-04T17:48:07.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6235e-05 2024-09-04T17:48:07.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:07.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943286+0000", 2024-09-04T17:48:07.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2793000000000004e-05 2024-09-04T17:48:07.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:07.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943526+0000", 2024-09-04T17:48:07.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023950999999999999 2024-09-04T17:48:07.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 1", 2024-09-04T17:48:07.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.952871+0000", 2024-09-04T17:48:07.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0093453070000000006 2024-09-04T17:48:07.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:07.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.997968+0000", 2024-09-04T17:48:07.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045096613000000001 2024-09-04T17:48:07.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:07.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.998006+0000", 2024-09-04T17:48:07.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8257000000000003e-05 2024-09-04T17:48:07.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:07.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.998073+0000", 2024-09-04T17:48:07.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6803000000000004e-05 2024-09-04T17:48:07.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:07.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.998136+0000", 2024-09-04T17:48:07.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3211000000000002e-05 2024-09-04T17:48:07.821 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:07.821 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:07.821 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:07.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.821 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93840 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 4980736~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:07.821 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386465+0000", 2024-09-04T17:48:07.822 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.3843285460000008, 2024-09-04T17:48:07.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042435764000000001, 2024-09-04T17:48:07.822 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:07.822 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:07.822 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:07.822 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:07.822 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:07.822 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:07.822 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93840 2024-09-04T17:48:07.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.822 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:07.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:07.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386465+0000", 2024-09-04T17:48:07.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:07.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:07.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386465+0000", 2024-09-04T17:48:07.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:07.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:07.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386466+0000", 2024-09-04T17:48:07.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4700000000000001e-07 2024-09-04T17:48:07.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:07.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386468+0000", 2024-09-04T17:48:07.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1600000000000001e-06 2024-09-04T17:48:07.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:07.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386477+0000", 2024-09-04T17:48:07.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7839999999999992e-06 2024-09-04T17:48:07.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:07.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386482+0000", 2024-09-04T17:48:07.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3970000000000002e-06 2024-09-04T17:48:07.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:07.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.428466+0000", 2024-09-04T17:48:07.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.041984279999999999 2024-09-04T17:48:07.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:07.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.428520+0000", 2024-09-04T17:48:07.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3328000000000002e-05 2024-09-04T17:48:07.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:07.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.428901+0000", 2024-09-04T17:48:07.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00038126799999999998 2024-09-04T17:48:07.826 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:07.826 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:07.826 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:07.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.826 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93841 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 5111808~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:07.826 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386491+0000", 2024-09-04T17:48:07.827 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.3843024130000003, 2024-09-04T17:48:07.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043069578999999997, 2024-09-04T17:48:07.827 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:07.827 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:07.827 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:07.827 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:07.827 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:07.827 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:07.827 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93841 2024-09-04T17:48:07.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.827 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:07.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:07.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386491+0000", 2024-09-04T17:48:07.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:07.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:07.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386491+0000", 2024-09-04T17:48:07.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:07.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:07.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386492+0000", 2024-09-04T17:48:07.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0299999999999999e-07 2024-09-04T17:48:07.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:07.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386499+0000", 2024-09-04T17:48:07.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6089999999999997e-06 2024-09-04T17:48:07.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:07.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386506+0000", 2024-09-04T17:48:07.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3599999999999998e-06 2024-09-04T17:48:07.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:07.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386514+0000", 2024-09-04T17:48:07.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9379999999999992e-06 2024-09-04T17:48:07.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:07.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.428957+0000", 2024-09-04T17:48:07.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042442831 2024-09-04T17:48:07.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:07.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.429080+0000", 2024-09-04T17:48:07.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012359299999999999 2024-09-04T17:48:07.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:07.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.429561+0000", 2024-09-04T17:48:07.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00048074499999999998 2024-09-04T17:48:07.830 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:07.831 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:07.831 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:07.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.831 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93842 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 4259840~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:07.831 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386521+0000", 2024-09-04T17:48:07.831 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.384272481, 2024-09-04T17:48:07.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043661933999999999, 2024-09-04T17:48:07.831 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:07.831 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:07.831 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:07.831 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:07.831 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:07.831 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:07.831 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93842 2024-09-04T17:48:07.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.832 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:07.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:07.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386521+0000", 2024-09-04T17:48:07.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:07.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:07.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386521+0000", 2024-09-04T17:48:07.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:07.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:07.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386522+0000", 2024-09-04T17:48:07.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4700000000000001e-07 2024-09-04T17:48:07.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:07.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386524+0000", 2024-09-04T17:48:07.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4559999999999999e-06 2024-09-04T17:48:07.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:07.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386534+0000", 2024-09-04T17:48:07.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2639999999999997e-06 2024-09-04T17:48:07.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:07.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386539+0000", 2024-09-04T17:48:07.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6130000000000003e-06 2024-09-04T17:48:07.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:07.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.429577+0000", 2024-09-04T17:48:07.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043037757000000003 2024-09-04T17:48:07.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:07.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.429638+0000", 2024-09-04T17:48:07.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1235999999999994e-05 2024-09-04T17:48:07.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.835 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:07.835 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.430183+0000", 2024-09-04T17:48:07.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000545061 2024-09-04T17:48:07.835 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:07.835 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:07.835 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:07.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.835 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93843 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 4390912~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:07.835 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386547+0000", 2024-09-04T17:48:07.835 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.3842472440000009, 2024-09-04T17:48:07.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044260668000000003, 2024-09-04T17:48:07.836 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:07.836 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:07.836 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:07.836 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:07.836 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:07.836 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:07.836 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93843 2024-09-04T17:48:07.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.836 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:07.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:07.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386547+0000", 2024-09-04T17:48:07.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:07.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.837 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:07.837 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386547+0000", 2024-09-04T17:48:07.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:07.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.837 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:07.837 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386547+0000", 2024-09-04T17:48:07.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4199999999999996e-07 2024-09-04T17:48:07.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.837 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:07.837 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386550+0000", 2024-09-04T17:48:07.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6199999999999999e-06 2024-09-04T17:48:07.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:07.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386557+0000", 2024-09-04T17:48:07.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1919999999999999e-06 2024-09-04T17:48:07.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:07.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386562+0000", 2024-09-04T17:48:07.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2580000000000003e-06 2024-09-04T17:48:07.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:07.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.430226+0000", 2024-09-04T17:48:07.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043663484000000002 2024-09-04T17:48:07.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:07.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.430286+0000", 2024-09-04T17:48:07.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0455999999999999e-05 2024-09-04T17:48:07.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:07.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.430807+0000", 2024-09-04T17:48:07.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00052111599999999996 2024-09-04T17:48:07.839 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:07.839 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:07.839 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:07.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.840 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93844 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 4521984~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:07.840 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386603+0000", 2024-09-04T17:48:07.840 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.3841911929999995, 2024-09-04T17:48:07.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044705350999999997, 2024-09-04T17:48:07.840 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:07.840 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:07.840 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:07.840 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:07.840 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:07.840 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:07.840 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93844 2024-09-04T17:48:07.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.841 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:07.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:07.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386603+0000", 2024-09-04T17:48:07.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:07.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:07.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386603+0000", 2024-09-04T17:48:07.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:07.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:07.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386603+0000", 2024-09-04T17:48:07.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.92e-07 2024-09-04T17:48:07.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:07.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386606+0000", 2024-09-04T17:48:07.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5500000000000001e-06 2024-09-04T17:48:07.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:07.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386614+0000", 2024-09-04T17:48:07.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9570000000000002e-06 2024-09-04T17:48:07.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:07.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386619+0000", 2024-09-04T17:48:07.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5119999999999999e-06 2024-09-04T17:48:07.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:07.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.430828+0000", 2024-09-04T17:48:07.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044208654 2024-09-04T17:48:07.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:07.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.430903+0000", 2024-09-04T17:48:07.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4629999999999995e-05 2024-09-04T17:48:07.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:07.844 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.431308+0000", 2024-09-04T17:48:07.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000405256 2024-09-04T17:48:07.844 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:07.844 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:07.844 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:07.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.844 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93845 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 4653056~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:07.844 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386627+0000", 2024-09-04T17:48:07.844 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.3841666040000007, 2024-09-04T17:48:07.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045262927000000001, 2024-09-04T17:48:07.844 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:07.844 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:07.844 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:07.845 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:07.845 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:07.845 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:07.845 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93845 2024-09-04T17:48:07.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.845 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:07.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:07.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386627+0000", 2024-09-04T17:48:07.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:07.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:07.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386627+0000", 2024-09-04T17:48:07.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:07.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:07.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386628+0000", 2024-09-04T17:48:07.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0999999999999998e-07 2024-09-04T17:48:07.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:07.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386630+0000", 2024-09-04T17:48:07.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3300000000000001e-06 2024-09-04T17:48:07.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:07.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386645+0000", 2024-09-04T17:48:07.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5186e-05 2024-09-04T17:48:07.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:07.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386651+0000", 2024-09-04T17:48:07.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7710000000000003e-06 2024-09-04T17:48:07.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:07.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.431349+0000", 2024-09-04T17:48:07.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044697723000000002 2024-09-04T17:48:07.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.848 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:07.848 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.431410+0000", 2024-09-04T17:48:07.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0677999999999997e-05 2024-09-04T17:48:07.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.848 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:07.848 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.431890+0000", 2024-09-04T17:48:07.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00048052899999999999 2024-09-04T17:48:07.848 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:07.848 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:07.848 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:07.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.849 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93846 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 4784128~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:07.849 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386659+0000", 2024-09-04T17:48:07.849 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.3841349350000005, 2024-09-04T17:48:07.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045784490999999997, 2024-09-04T17:48:07.849 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:07.849 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:07.849 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:07.849 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:07.849 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:07.849 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:07.849 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93846 2024-09-04T17:48:07.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.849 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:07.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:07.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386659+0000", 2024-09-04T17:48:07.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:07.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:07.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386659+0000", 2024-09-04T17:48:07.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:07.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:07.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386659+0000", 2024-09-04T17:48:07.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7000000000000005e-07 2024-09-04T17:48:07.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:07.851 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386662+0000", 2024-09-04T17:48:07.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3300000000000001e-06 2024-09-04T17:48:07.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:07.851 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386669+0000", 2024-09-04T17:48:07.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0589999999999997e-06 2024-09-04T17:48:07.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:07.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386674+0000", 2024-09-04T17:48:07.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3009999999999997e-06 2024-09-04T17:48:07.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:07.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.431934+0000", 2024-09-04T17:48:07.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045259570999999998 2024-09-04T17:48:07.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:07.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.431998+0000", 2024-09-04T17:48:07.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4775000000000002e-05 2024-09-04T17:48:07.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:07.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.432443+0000", 2024-09-04T17:48:07.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00044488500000000001 2024-09-04T17:48:07.853 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:07.853 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:07.853 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:07.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.854 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93847 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 4915200~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:07.854 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386681+0000", 2024-09-04T17:48:07.854 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.3841126779999993, 2024-09-04T17:48:07.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046305101000000001, 2024-09-04T17:48:07.854 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:07.854 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:07.854 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:07.854 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:07.854 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:07.854 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:07.854 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93847 2024-09-04T17:48:07.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.855 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:07.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.855 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:07.855 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386681+0000", 2024-09-04T17:48:07.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:07.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.855 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:07.855 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386681+0000", 2024-09-04T17:48:07.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:07.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:07.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386682+0000", 2024-09-04T17:48:07.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1699999999999998e-07 2024-09-04T17:48:07.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:07.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386684+0000", 2024-09-04T17:48:07.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.153e-06 2024-09-04T17:48:07.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:07.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386693+0000", 2024-09-04T17:48:07.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9639999999999992e-06 2024-09-04T17:48:07.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.857 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:07.857 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386698+0000", 2024-09-04T17:48:07.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3709999999999999e-06 2024-09-04T17:48:07.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.857 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:07.857 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.432476+0000", 2024-09-04T17:48:07.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045778147999999998 2024-09-04T17:48:07.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.858 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:07.858 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.432529+0000", 2024-09-04T17:48:07.858 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3165999999999999e-05 2024-09-04T17:48:07.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.858 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.858 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:07.858 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.432986+0000", 2024-09-04T17:48:07.858 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00045678199999999998 2024-09-04T17:48:07.858 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:07.858 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:07.858 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:07.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.858 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.858 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93848 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 5046272~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:07.859 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386705+0000", 2024-09-04T17:48:07.859 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.384088577, 2024-09-04T17:48:07.859 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046825259000000001, 2024-09-04T17:48:07.859 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:07.859 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:07.859 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:07.859 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:07.859 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:07.859 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:07.859 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93848 2024-09-04T17:48:07.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.860 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:07.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.860 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:07.860 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386705+0000", 2024-09-04T17:48:07.860 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:07.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.860 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:07.860 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386705+0000", 2024-09-04T17:48:07.860 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:07.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.860 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:07.860 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386706+0000", 2024-09-04T17:48:07.861 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1500000000000005e-07 2024-09-04T17:48:07.861 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.861 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.861 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:07.861 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386708+0000", 2024-09-04T17:48:07.861 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2979999999999999e-06 2024-09-04T17:48:07.861 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.861 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.861 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:07.861 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386715+0000", 2024-09-04T17:48:07.861 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8669999999999996e-06 2024-09-04T17:48:07.861 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.861 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.862 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:07.862 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386720+0000", 2024-09-04T17:48:07.862 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.242e-06 2024-09-04T17:48:07.862 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.862 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.862 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:07.862 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.433019+0000", 2024-09-04T17:48:07.862 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046298359999999997 2024-09-04T17:48:07.862 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.862 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.862 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:07.862 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.433084+0000", 2024-09-04T17:48:07.862 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5395999999999995e-05 2024-09-04T17:48:07.862 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.863 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:07.863 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.433531+0000", 2024-09-04T17:48:07.863 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00044658099999999999 2024-09-04T17:48:07.863 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:07.863 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:07.863 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:07.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.863 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93849 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 5177344~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:07.863 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386727+0000", 2024-09-04T17:48:07.863 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.384066679, 2024-09-04T17:48:07.863 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.047267796000000001, 2024-09-04T17:48:07.864 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:07.864 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:07.864 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:07.864 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:07.864 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:07.864 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:07.864 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93849 2024-09-04T17:48:07.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.864 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:07.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.864 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:07.864 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386727+0000", 2024-09-04T17:48:07.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:07.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.865 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.865 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:07.865 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386727+0000", 2024-09-04T17:48:07.865 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:07.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.865 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.865 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:07.865 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386728+0000", 2024-09-04T17:48:07.865 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1699999999999998e-07 2024-09-04T17:48:07.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.865 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.865 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:07.865 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386730+0000", 2024-09-04T17:48:07.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2589999999999999e-06 2024-09-04T17:48:07.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.866 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.866 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:07.866 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386751+0000", 2024-09-04T17:48:07.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1424999999999999e-05 2024-09-04T17:48:07.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.866 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.866 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:07.866 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386759+0000", 2024-09-04T17:48:07.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9400000000000002e-06 2024-09-04T17:48:07.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.867 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.867 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:07.867 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.433559+0000", 2024-09-04T17:48:07.867 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046799573999999997 2024-09-04T17:48:07.867 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.867 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.867 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:07.867 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.433610+0000", 2024-09-04T17:48:07.867 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1134999999999997e-05 2024-09-04T17:48:07.867 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.867 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.867 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:07.868 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.433995+0000", 2024-09-04T17:48:07.868 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000384946 2024-09-04T17:48:07.868 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:07.868 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:07.868 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:07.868 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.868 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.868 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:96175 3.12 3:4d4471be:::rbd_data.1134c4e2d26.000000000000006e:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123)", 2024-09-04T17:48:07.868 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:48:02.919542+0000", 2024-09-04T17:48:07.868 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.851251435, 2024-09-04T17:48:07.868 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060180626000000001, 2024-09-04T17:48:07.868 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:07.868 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:07.869 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:07.869 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:07.869 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:07.869 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:07.869 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 96175 2024-09-04T17:48:07.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.869 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:07.869 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.869 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:07.869 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.919542+0000", 2024-09-04T17:48:07.869 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:07.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.869 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.870 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:07.870 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.919542+0000", 2024-09-04T17:48:07.870 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:07.870 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.870 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.870 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:07.870 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.919545+0000", 2024-09-04T17:48:07.870 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0199999999999999e-06 2024-09-04T17:48:07.870 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.870 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.870 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:07.871 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.919559+0000", 2024-09-04T17:48:07.871 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3293e-05 2024-09-04T17:48:07.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.871 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.871 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:07.871 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.919587+0000", 2024-09-04T17:48:07.871 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7922000000000001e-05 2024-09-04T17:48:07.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.871 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.871 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:07.871 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.919601+0000", 2024-09-04T17:48:07.871 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4657e-05 2024-09-04T17:48:07.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.871 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.871 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:07.872 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.919708+0000", 2024-09-04T17:48:07.872 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010624999999999999 2024-09-04T17:48:07.872 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.872 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.872 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:07.872 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.919929+0000", 2024-09-04T17:48:07.872 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022147399999999999 2024-09-04T17:48:07.872 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.872 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.872 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2024-09-04T17:48:07.872 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.929774+0000", 2024-09-04T17:48:07.872 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0098449839999999993 2024-09-04T17:48:07.872 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.872 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.873 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:07.873 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.979511+0000", 2024-09-04T17:48:07.873 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.049736537999999997 2024-09-04T17:48:07.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.873 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.873 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:07.873 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.979547+0000", 2024-09-04T17:48:07.873 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6069000000000003e-05 2024-09-04T17:48:07.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.873 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.873 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:07.873 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.979642+0000", 2024-09-04T17:48:07.873 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5526999999999995e-05 2024-09-04T17:48:07.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:07.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:07.874 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:07.874 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.979723+0000", 2024-09-04T17:48:07.874 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0891999999999997e-05 2024-09-04T17:48:07.874 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:07.874 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:07.874 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:07.874 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:07.874 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:07.874 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:07.876 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-04T17:48:07.876 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is set 2024-09-04T17:48:07.876 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.4 to 1 (10000) 2024-09-04T17:48:07.878 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-04T17:48:07.878 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:48:07.872+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:48:07.977 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:48:07.974+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:48:08.063 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:08.063 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:48:08.063 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:48:08.063 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:08.077 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:08.076+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:08.078 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-04T17:48:08.178 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:48:08.174+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:48:08.236 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:08.236 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:48:08.236 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:48:08.237 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:48:08.237 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:08.250 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-04T17:48:08.279 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:08.274+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:08.370 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:08.370 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:48:08.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:48:08.371 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:48:08.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.371 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59950 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5365760~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:48:08.371 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654917+0000", 2024-09-04T17:48:08.371 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 54.714148299999998, 2024-09-04T17:48:08.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066822450000000005, 2024-09-04T17:48:08.371 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:08.371 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:08.371 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:08.371 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:08.371 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:08.371 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:08.372 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59950 2024-09-04T17:48:08.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.372 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:08.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:08.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654917+0000", 2024-09-04T17:48:08.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:08.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:08.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654917+0000", 2024-09-04T17:48:08.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:08.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:08.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654918+0000", 2024-09-04T17:48:08.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.29e-07 2024-09-04T17:48:08.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:08.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654919+0000", 2024-09-04T17:48:08.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2589999999999999e-06 2024-09-04T17:48:08.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:08.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654922+0000", 2024-09-04T17:48:08.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1350000000000001e-06 2024-09-04T17:48:08.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:08.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654925+0000", 2024-09-04T17:48:08.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1250000000000001e-06 2024-09-04T17:48:08.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:08.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721466+0000", 2024-09-04T17:48:08.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066540959999999996 2024-09-04T17:48:08.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:08.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721574+0000", 2024-09-04T17:48:08.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010747399999999999 2024-09-04T17:48:08.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:08.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721740+0000", 2024-09-04T17:48:08.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016626799999999999 2024-09-04T17:48:08.375 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:08.375 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:08.375 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:08.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.376 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59951 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5472256~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:48:08.376 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654929+0000", 2024-09-04T17:48:08.376 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 54.714137115, 2024-09-04T17:48:08.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067110534999999999, 2024-09-04T17:48:08.376 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:08.376 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:08.376 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:08.376 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:08.376 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:08.376 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:08.376 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59951 2024-09-04T17:48:08.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.377 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:08.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:08.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654929+0000", 2024-09-04T17:48:08.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:08.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:08.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654929+0000", 2024-09-04T17:48:08.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:08.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:08.378 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654929+0000", 2024-09-04T17:48:08.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5100000000000001e-07 2024-09-04T17:48:08.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.378 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:08.378 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654930+0000", 2024-09-04T17:48:08.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2440000000000001e-06 2024-09-04T17:48:08.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.378 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:08.378 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654933+0000", 2024-09-04T17:48:08.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1599999999999998e-06 2024-09-04T17:48:08.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:08.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654936+0000", 2024-09-04T17:48:08.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6560000000000001e-06 2024-09-04T17:48:08.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:08.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721789+0000", 2024-09-04T17:48:08.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066853250000000003 2024-09-04T17:48:08.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:08.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721885+0000", 2024-09-04T17:48:08.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5616000000000001e-05 2024-09-04T17:48:08.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:08.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722039+0000", 2024-09-04T17:48:08.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000154258 2024-09-04T17:48:08.380 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:08.380 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:08.380 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:08.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.380 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59952 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5505024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:48:08.380 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654939+0000", 2024-09-04T17:48:08.381 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 54.714126237999999, 2024-09-04T17:48:08.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067433742000000005, 2024-09-04T17:48:08.381 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:08.381 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:08.381 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:08.381 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:08.381 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:08.381 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:08.381 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59952 2024-09-04T17:48:08.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.381 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:08.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:08.382 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654939+0000", 2024-09-04T17:48:08.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:08.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:08.382 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654939+0000", 2024-09-04T17:48:08.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:08.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:08.382 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654940+0000", 2024-09-04T17:48:08.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3300000000000001e-07 2024-09-04T17:48:08.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:08.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654941+0000", 2024-09-04T17:48:08.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1570000000000001e-06 2024-09-04T17:48:08.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:08.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654944+0000", 2024-09-04T17:48:08.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1319999999999998e-06 2024-09-04T17:48:08.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:08.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654947+0000", 2024-09-04T17:48:08.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5969999999999999e-06 2024-09-04T17:48:08.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:08.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722089+0000", 2024-09-04T17:48:08.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067142347000000005 2024-09-04T17:48:08.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:08.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722201+0000", 2024-09-04T17:48:08.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011235600000000001 2024-09-04T17:48:08.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:08.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722373+0000", 2024-09-04T17:48:08.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017191999999999999 2024-09-04T17:48:08.385 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:08.385 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:08.385 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:08.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.385 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59953 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5611520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:48:08.385 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654950+0000", 2024-09-04T17:48:08.385 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 54.714115597999999, 2024-09-04T17:48:08.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067601852000000004, 2024-09-04T17:48:08.386 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:08.386 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:08.386 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:08.386 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:08.386 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:08.386 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:08.386 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59953 2024-09-04T17:48:08.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.386 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:08.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:08.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654950+0000", 2024-09-04T17:48:08.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:08.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:08.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654950+0000", 2024-09-04T17:48:08.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:08.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:08.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654950+0000", 2024-09-04T17:48:08.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2499999999999999e-07 2024-09-04T17:48:08.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:08.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654954+0000", 2024-09-04T17:48:08.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.287e-06 2024-09-04T17:48:08.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:08.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654962+0000", 2024-09-04T17:48:08.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4230000000000005e-06 2024-09-04T17:48:08.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:08.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654966+0000", 2024-09-04T17:48:08.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6449999999999998e-06 2024-09-04T17:48:08.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:08.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722417+0000", 2024-09-04T17:48:08.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067451188999999995 2024-09-04T17:48:08.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:08.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722467+0000", 2024-09-04T17:48:08.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0556999999999997e-05 2024-09-04T17:48:08.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:08.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722552+0000", 2024-09-04T17:48:08.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4525999999999997e-05 2024-09-04T17:48:08.390 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:08.390 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:08.390 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:08.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.390 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59954 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5718016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:48:08.390 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654970+0000", 2024-09-04T17:48:08.390 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 54.714095837000002, 2024-09-04T17:48:08.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067874285000000006, 2024-09-04T17:48:08.390 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:08.390 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:08.391 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:08.391 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:08.391 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:08.391 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:08.391 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59954 2024-09-04T17:48:08.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.391 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:08.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:08.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654970+0000", 2024-09-04T17:48:08.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:08.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:08.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654970+0000", 2024-09-04T17:48:08.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:08.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:08.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654970+0000", 2024-09-04T17:48:08.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5899999999999998e-07 2024-09-04T17:48:08.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:08.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654971+0000", 2024-09-04T17:48:08.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2929999999999999e-06 2024-09-04T17:48:08.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:08.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654975+0000", 2024-09-04T17:48:08.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4699999999999998e-06 2024-09-04T17:48:08.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:08.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654978+0000", 2024-09-04T17:48:08.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7930000000000002e-06 2024-09-04T17:48:08.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:08.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722600+0000", 2024-09-04T17:48:08.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067622229000000006 2024-09-04T17:48:08.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:08.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722689+0000", 2024-09-04T17:48:08.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9511000000000004e-05 2024-09-04T17:48:08.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:08.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722844+0000", 2024-09-04T17:48:08.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015473 2024-09-04T17:48:08.394 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:08.394 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:08.395 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:08.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.395 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59955 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5857280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:48:08.395 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654981+0000", 2024-09-04T17:48:08.395 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 54.714084487000001, 2024-09-04T17:48:08.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068052360000000006, 2024-09-04T17:48:08.395 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:08.395 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:08.395 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:08.395 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:08.395 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:08.395 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:08.396 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59955 2024-09-04T17:48:08.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.396 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:08.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:08.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654981+0000", 2024-09-04T17:48:08.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:08.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:08.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654981+0000", 2024-09-04T17:48:08.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:08.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:08.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654981+0000", 2024-09-04T17:48:08.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2399999999999999e-07 2024-09-04T17:48:08.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:08.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654985+0000", 2024-09-04T17:48:08.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0980000000000004e-06 2024-09-04T17:48:08.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:08.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654990+0000", 2024-09-04T17:48:08.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3429999999999998e-06 2024-09-04T17:48:08.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:08.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654993+0000", 2024-09-04T17:48:08.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8700000000000001e-06 2024-09-04T17:48:08.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:08.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722888+0000", 2024-09-04T17:48:08.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067895239999999996 2024-09-04T17:48:08.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:08.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722948+0000", 2024-09-04T17:48:08.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9750999999999998e-05 2024-09-04T17:48:08.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:08.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.723034+0000", 2024-09-04T17:48:08.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5834000000000005e-05 2024-09-04T17:48:08.399 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:08.399 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:08.399 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:08.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.399 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59956 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5963776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:48:08.399 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654998+0000", 2024-09-04T17:48:08.400 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 54.714067958000001, 2024-09-04T17:48:08.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068359518999999994, 2024-09-04T17:48:08.400 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:08.400 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:08.400 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:08.400 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:08.400 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:08.400 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:08.400 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59956 2024-09-04T17:48:08.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.400 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:08.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:08.401 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654998+0000", 2024-09-04T17:48:08.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:08.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:08.401 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654998+0000", 2024-09-04T17:48:08.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:08.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:08.401 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654998+0000", 2024-09-04T17:48:08.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4299999999999999e-07 2024-09-04T17:48:08.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:08.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654999+0000", 2024-09-04T17:48:08.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1799999999999999e-06 2024-09-04T17:48:08.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:08.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.655002+0000", 2024-09-04T17:48:08.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3639999999999999e-06 2024-09-04T17:48:08.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:08.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.655005+0000", 2024-09-04T17:48:08.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7970000000000001e-06 2024-09-04T17:48:08.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:08.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.723085+0000", 2024-09-04T17:48:08.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068080229000000006 2024-09-04T17:48:08.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:08.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.723197+0000", 2024-09-04T17:48:08.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000111344 2024-09-04T17:48:08.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:08.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.723357+0000", 2024-09-04T17:48:08.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016036199999999999 2024-09-04T17:48:08.404 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:08.404 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:08.404 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:08.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.404 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61082 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 2654208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:08.404 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.822664+0000", 2024-09-04T17:48:08.405 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 54.546401570999997, 2024-09-04T17:48:08.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067072242000000004, 2024-09-04T17:48:08.405 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:08.405 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:08.405 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:08.405 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:08.405 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:08.405 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:08.405 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61082 2024-09-04T17:48:08.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.405 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:08.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:08.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822664+0000", 2024-09-04T17:48:08.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:08.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:08.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822664+0000", 2024-09-04T17:48:08.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:08.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:08.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822664+0000", 2024-09-04T17:48:08.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6600000000000003e-07 2024-09-04T17:48:08.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:08.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822666+0000", 2024-09-04T17:48:08.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.536e-06 2024-09-04T17:48:08.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:08.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822669+0000", 2024-09-04T17:48:08.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3759999999999999e-06 2024-09-04T17:48:08.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:08.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822672+0000", 2024-09-04T17:48:08.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8109999999999999e-06 2024-09-04T17:48:08.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:08.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889263+0000", 2024-09-04T17:48:08.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066591259999999999 2024-09-04T17:48:08.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:08.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889369+0000", 2024-09-04T17:48:08.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000105362 2024-09-04T17:48:08.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:08.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889736+0000", 2024-09-04T17:48:08.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00036763099999999999 2024-09-04T17:48:08.409 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:08.409 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:08.409 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:08.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.409 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61083 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 2760704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:08.409 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.822678+0000", 2024-09-04T17:48:08.409 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 54.546387836999997, 2024-09-04T17:48:08.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067475996999999996, 2024-09-04T17:48:08.410 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:08.410 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:08.410 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:08.410 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:08.410 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:08.410 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:08.410 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61083 2024-09-04T17:48:08.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.410 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:08.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:08.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822678+0000", 2024-09-04T17:48:08.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:08.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.411 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:08.411 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822678+0000", 2024-09-04T17:48:08.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:08.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.411 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:08.411 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822725+0000", 2024-09-04T17:48:08.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7429000000000003e-05 2024-09-04T17:48:08.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.411 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:08.411 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822728+0000", 2024-09-04T17:48:08.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2140000000000001e-06 2024-09-04T17:48:08.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:08.412 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822737+0000", 2024-09-04T17:48:08.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4370000000000007e-06 2024-09-04T17:48:08.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:08.412 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822743+0000", 2024-09-04T17:48:08.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7520000000000002e-06 2024-09-04T17:48:08.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:08.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889785+0000", 2024-09-04T17:48:08.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067042265000000004 2024-09-04T17:48:08.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:08.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889840+0000", 2024-09-04T17:48:08.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5254000000000003e-05 2024-09-04T17:48:08.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:08.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.890154+0000", 2024-09-04T17:48:08.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000313646 2024-09-04T17:48:08.414 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:08.414 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:08.414 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:08.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.414 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61084 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 2867200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:08.414 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.822753+0000", 2024-09-04T17:48:08.414 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 54.546312901, 2024-09-04T17:48:08.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068497161000000001, 2024-09-04T17:48:08.414 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:08.414 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:08.414 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:08.414 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:08.414 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:08.415 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:08.415 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61084 2024-09-04T17:48:08.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.415 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:08.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:08.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822753+0000", 2024-09-04T17:48:08.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:08.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:08.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822753+0000", 2024-09-04T17:48:08.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:08.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:08.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822753+0000", 2024-09-04T17:48:08.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9100000000000003e-07 2024-09-04T17:48:08.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:08.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822756+0000", 2024-09-04T17:48:08.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.678e-06 2024-09-04T17:48:08.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:08.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822763+0000", 2024-09-04T17:48:08.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8220000000000001e-06 2024-09-04T17:48:08.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:08.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822768+0000", 2024-09-04T17:48:08.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1930000000000001e-06 2024-09-04T17:48:08.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:08.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.890246+0000", 2024-09-04T17:48:08.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067477996999999998 2024-09-04T17:48:08.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:08.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.890555+0000", 2024-09-04T17:48:08.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000309143 2024-09-04T17:48:08.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:08.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.891250+0000", 2024-09-04T17:48:08.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00069463700000000001 2024-09-04T17:48:08.418 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:08.418 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:08.418 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:08.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.419 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61285 3.8 3:117cfeb9:::rbd_data.11341dfde0c5.0000000000000006:1f [read 2760704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:08.419 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835200+0000", 2024-09-04T17:48:08.419 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 54.533865833999997, 2024-09-04T17:48:08.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066899803999999993, 2024-09-04T17:48:08.419 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:08.419 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:08.419 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:08.419 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:08.419 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:08.419 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:08.419 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61285 2024-09-04T17:48:08.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.419 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:08.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:08.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835200+0000", 2024-09-04T17:48:08.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:08.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:08.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835200+0000", 2024-09-04T17:48:08.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:08.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:08.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835200+0000", 2024-09-04T17:48:08.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0899999999999997e-07 2024-09-04T17:48:08.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:08.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835206+0000", 2024-09-04T17:48:08.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3280000000000002e-06 2024-09-04T17:48:08.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:08.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835211+0000", 2024-09-04T17:48:08.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9230000000000001e-06 2024-09-04T17:48:08.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:08.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835216+0000", 2024-09-04T17:48:08.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9019999999999998e-06 2024-09-04T17:48:08.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:08.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.901719+0000", 2024-09-04T17:48:08.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066502981000000003 2024-09-04T17:48:08.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:08.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.901813+0000", 2024-09-04T17:48:08.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3905000000000003e-05 2024-09-04T17:48:08.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:08.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902100+0000", 2024-09-04T17:48:08.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028645599999999998 2024-09-04T17:48:08.423 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:08.423 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:08.423 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:08.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.423 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61286 3.8 3:117cfeb9:::rbd_data.11341dfde0c5.0000000000000006:1f [read 2867200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:08.423 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835244+0000", 2024-09-04T17:48:08.423 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 54.533821215000003, 2024-09-04T17:48:08.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067143215000000006, 2024-09-04T17:48:08.424 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:08.424 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:08.424 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:08.424 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:08.424 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:08.424 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:08.424 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61286 2024-09-04T17:48:08.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.424 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:08.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:08.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835244+0000", 2024-09-04T17:48:08.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:08.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:08.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835244+0000", 2024-09-04T17:48:08.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:08.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:08.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835245+0000", 2024-09-04T17:48:08.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8400000000000004e-07 2024-09-04T17:48:08.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:08.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835248+0000", 2024-09-04T17:48:08.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6850000000000001e-06 2024-09-04T17:48:08.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:08.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835254+0000", 2024-09-04T17:48:08.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.494e-06 2024-09-04T17:48:08.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:08.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835260+0000", 2024-09-04T17:48:08.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0580000000000003e-06 2024-09-04T17:48:08.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:08.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902119+0000", 2024-09-04T17:48:08.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066858318 2024-09-04T17:48:08.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:08.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902163+0000", 2024-09-04T17:48:08.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4753000000000003e-05 2024-09-04T17:48:08.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:08.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902388+0000", 2024-09-04T17:48:08.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022422300000000001 2024-09-04T17:48:08.428 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:08.428 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:08.428 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:08.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.428 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61315 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 98304~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:08.428 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835269+0000", 2024-09-04T17:48:08.428 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 54.533796422999998, 2024-09-04T17:48:08.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067329396, 2024-09-04T17:48:08.428 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:08.428 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:08.429 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:08.429 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:08.429 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:08.429 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:08.429 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61315 2024-09-04T17:48:08.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.429 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:08.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:08.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835269+0000", 2024-09-04T17:48:08.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:08.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:08.430 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835269+0000", 2024-09-04T17:48:08.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:08.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.430 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:08.430 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835270+0000", 2024-09-04T17:48:08.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6599999999999996e-07 2024-09-04T17:48:08.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.430 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:08.430 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835272+0000", 2024-09-04T17:48:08.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.261e-06 2024-09-04T17:48:08.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:08.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835278+0000", 2024-09-04T17:48:08.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.443e-06 2024-09-04T17:48:08.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:08.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835284+0000", 2024-09-04T17:48:08.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3290000000000001e-06 2024-09-04T17:48:08.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:08.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902147+0000", 2024-09-04T17:48:08.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066862695999999999 2024-09-04T17:48:08.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:08.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902248+0000", 2024-09-04T17:48:08.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000101501 2024-09-04T17:48:08.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:08.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902599+0000", 2024-09-04T17:48:08.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00035060000000000001 2024-09-04T17:48:08.432 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:08.432 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:08.432 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:08.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.433 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61316 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 204800~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:08.433 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835291+0000", 2024-09-04T17:48:08.433 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 54.533774403999999, 2024-09-04T17:48:08.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067632052999999998, 2024-09-04T17:48:08.433 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:08.433 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:08.433 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:08.433 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:08.433 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:08.433 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:08.433 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61316 2024-09-04T17:48:08.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.434 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:08.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.434 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:08.434 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835291+0000", 2024-09-04T17:48:08.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:08.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.434 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:08.434 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835291+0000", 2024-09-04T17:48:08.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:08.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:08.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835292+0000", 2024-09-04T17:48:08.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1899999999999998e-07 2024-09-04T17:48:08.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:08.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835294+0000", 2024-09-04T17:48:08.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0880000000000002e-06 2024-09-04T17:48:08.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:08.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835300+0000", 2024-09-04T17:48:08.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.224e-06 2024-09-04T17:48:08.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:08.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835305+0000", 2024-09-04T17:48:08.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0730000000000004e-06 2024-09-04T17:48:08.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:08.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902632+0000", 2024-09-04T17:48:08.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067327322999999994 2024-09-04T17:48:08.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:08.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902688+0000", 2024-09-04T17:48:08.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5177999999999999e-05 2024-09-04T17:48:08.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:08.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902923+0000", 2024-09-04T17:48:08.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023574800000000001 2024-09-04T17:48:08.437 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:08.437 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:08.437 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:08.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.437 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61317 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 344064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:08.437 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835312+0000", 2024-09-04T17:48:08.438 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 54.533753400000002, 2024-09-04T17:48:08.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067918511000000001, 2024-09-04T17:48:08.438 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:08.438 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:08.438 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:08.438 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:08.438 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:08.438 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:08.438 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61317 2024-09-04T17:48:08.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.438 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:08.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:08.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835312+0000", 2024-09-04T17:48:08.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:08.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.439 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:08.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835312+0000", 2024-09-04T17:48:08.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:08.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.439 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:08.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835313+0000", 2024-09-04T17:48:08.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9000000000000002e-07 2024-09-04T17:48:08.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:08.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835315+0000", 2024-09-04T17:48:08.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1689999999999999e-06 2024-09-04T17:48:08.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:08.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835321+0000", 2024-09-04T17:48:08.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0619999999999997e-06 2024-09-04T17:48:08.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:08.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835326+0000", 2024-09-04T17:48:08.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0390000000000004e-06 2024-09-04T17:48:08.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:08.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902967+0000", 2024-09-04T17:48:08.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067641520999999996 2024-09-04T17:48:08.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:08.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903067+0000", 2024-09-04T17:48:08.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9426999999999995e-05 2024-09-04T17:48:08.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:08.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903231+0000", 2024-09-04T17:48:08.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016390299999999999 2024-09-04T17:48:08.442 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:08.442 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:08.442 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:08.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.442 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61318 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 450560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:08.442 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835334+0000", 2024-09-04T17:48:08.442 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 54.533731883000002, 2024-09-04T17:48:08.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068091771999999995, 2024-09-04T17:48:08.442 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:08.442 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:08.442 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:08.443 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:08.443 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:08.443 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:08.443 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61318 2024-09-04T17:48:08.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.443 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:08.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:08.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835334+0000", 2024-09-04T17:48:08.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:08.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:08.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835334+0000", 2024-09-04T17:48:08.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:08.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:08.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835334+0000", 2024-09-04T17:48:08.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5699999999999998e-07 2024-09-04T17:48:08.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:08.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835337+0000", 2024-09-04T17:48:08.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3889999999999999e-06 2024-09-04T17:48:08.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:08.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835343+0000", 2024-09-04T17:48:08.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1809999999999998e-06 2024-09-04T17:48:08.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:08.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835348+0000", 2024-09-04T17:48:08.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3170000000000001e-06 2024-09-04T17:48:08.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:08.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903270+0000", 2024-09-04T17:48:08.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067922191000000007 2024-09-04T17:48:08.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:08.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903321+0000", 2024-09-04T17:48:08.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0889e-05 2024-09-04T17:48:08.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:08.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903426+0000", 2024-09-04T17:48:08.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104348 2024-09-04T17:48:08.446 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:08.446 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:08.446 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:08.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.447 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61319 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 589824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:08.447 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835357+0000", 2024-09-04T17:48:08.447 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 54.533709066, 2024-09-04T17:48:08.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068372535999999998, 2024-09-04T17:48:08.447 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:08.447 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:08.447 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:08.447 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:08.447 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:08.447 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:08.447 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61319 2024-09-04T17:48:08.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.447 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:08.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:08.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835357+0000", 2024-09-04T17:48:08.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:08.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:08.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835357+0000", 2024-09-04T17:48:08.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:08.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:08.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835357+0000", 2024-09-04T17:48:08.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.32e-07 2024-09-04T17:48:08.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:08.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835359+0000", 2024-09-04T17:48:08.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1660000000000001e-06 2024-09-04T17:48:08.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:08.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835366+0000", 2024-09-04T17:48:08.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3550000000000001e-06 2024-09-04T17:48:08.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.450 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:08.450 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835371+0000", 2024-09-04T17:48:08.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1050000000000003e-06 2024-09-04T17:48:08.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.450 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:08.450 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903471+0000", 2024-09-04T17:48:08.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068100451000000006 2024-09-04T17:48:08.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.450 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:08.450 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903573+0000", 2024-09-04T17:48:08.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000101691 2024-09-04T17:48:08.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.451 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:08.451 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903729+0000", 2024-09-04T17:48:08.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000156336 2024-09-04T17:48:08.451 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:08.451 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:08.451 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:08.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.451 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61320 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:08.451 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835380+0000", 2024-09-04T17:48:08.452 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 54.533685300999998, 2024-09-04T17:48:08.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068543058000000004, 2024-09-04T17:48:08.452 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:08.452 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:08.452 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:08.452 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:08.452 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:08.452 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:08.452 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61320 2024-09-04T17:48:08.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.452 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:08.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:08.453 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835380+0000", 2024-09-04T17:48:08.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:08.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.453 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:08.453 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835380+0000", 2024-09-04T17:48:08.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:08.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.453 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:08.453 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835381+0000", 2024-09-04T17:48:08.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9999999999999998e-07 2024-09-04T17:48:08.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.454 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:08.454 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835383+0000", 2024-09-04T17:48:08.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2230000000000001e-06 2024-09-04T17:48:08.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.454 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:08.454 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835390+0000", 2024-09-04T17:48:08.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.34e-06 2024-09-04T17:48:08.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.454 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:08.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835396+0000", 2024-09-04T17:48:08.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9830000000000001e-06 2024-09-04T17:48:08.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:08.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903765+0000", 2024-09-04T17:48:08.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068368441000000002 2024-09-04T17:48:08.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:08.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903835+0000", 2024-09-04T17:48:08.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0631999999999997e-05 2024-09-04T17:48:08.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:08.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903923+0000", 2024-09-04T17:48:08.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8039000000000002e-05 2024-09-04T17:48:08.456 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:08.456 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:08.456 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:08.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.456 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61321 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 835584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:08.456 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835404+0000", 2024-09-04T17:48:08.456 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 54.533661242000001, 2024-09-04T17:48:08.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068911043000000005, 2024-09-04T17:48:08.456 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:08.456 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:08.457 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:08.457 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:08.457 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:08.457 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:08.457 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61321 2024-09-04T17:48:08.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.457 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:08.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:08.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835404+0000", 2024-09-04T17:48:08.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:08.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:08.458 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835404+0000", 2024-09-04T17:48:08.458 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:08.458 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.458 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.458 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:08.458 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835405+0000", 2024-09-04T17:48:08.458 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.89e-07 2024-09-04T17:48:08.458 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.458 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.458 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:08.458 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835407+0000", 2024-09-04T17:48:08.458 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4439999999999998e-06 2024-09-04T17:48:08.458 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:08.459 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835414+0000", 2024-09-04T17:48:08.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4930000000000003e-06 2024-09-04T17:48:08.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:08.459 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835419+0000", 2024-09-04T17:48:08.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8919999999999999e-06 2024-09-04T17:48:08.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:08.459 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903957+0000", 2024-09-04T17:48:08.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068538500000000002 2024-09-04T17:48:08.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:08.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904019+0000", 2024-09-04T17:48:08.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1419000000000003e-05 2024-09-04T17:48:08.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:08.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904315+0000", 2024-09-04T17:48:08.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00029680599999999999 2024-09-04T17:48:08.460 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:08.460 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:08.460 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:08.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.461 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61322 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:08.461 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835427+0000", 2024-09-04T17:48:08.461 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 54.533638856000003, 2024-09-04T17:48:08.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.069550138999999997, 2024-09-04T17:48:08.461 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:08.461 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:08.461 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:08.461 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:08.461 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:08.461 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:08.461 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61322 2024-09-04T17:48:08.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.462 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:08.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.462 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:08.462 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835427+0000", 2024-09-04T17:48:08.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:08.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.462 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:08.462 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835427+0000", 2024-09-04T17:48:08.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:08.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.462 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:08.463 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.842157+0000", 2024-09-04T17:48:08.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067298419999999998 2024-09-04T17:48:08.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.463 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:08.463 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.842163+0000", 2024-09-04T17:48:08.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2029999999999997e-06 2024-09-04T17:48:08.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.463 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:08.463 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.842175+0000", 2024-09-04T17:48:08.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2072000000000001e-05 2024-09-04T17:48:08.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.464 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:08.464 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.842183+0000", 2024-09-04T17:48:08.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3850000000000002e-06 2024-09-04T17:48:08.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.464 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:08.464 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904371+0000", 2024-09-04T17:48:08.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.062187754999999997 2024-09-04T17:48:08.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.464 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:08.464 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904467+0000", 2024-09-04T17:48:08.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6117000000000001e-05 2024-09-04T17:48:08.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:08.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:08.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:08.465 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904977+0000", 2024-09-04T17:48:08.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00050976500000000002 2024-09-04T17:48:08.465 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:08.465 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:08.465 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:08.465 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:08.465 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:08.465 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:08.466 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-04T17:48:08.467 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:08.462+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:08.567 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:48:08.564+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:48:08.607 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:08.607 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:48:08.607 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:48:08.607 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:08.621 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-04T17:48:08.668 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:08.664+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:08.749 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:08.749 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:48:08.749 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:48:08.749 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:48:08.749 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:08.763 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-04T17:48:08.809 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:08.806+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:08.829 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.5 to 1 (10000) 2024-09-04T17:48:08.860 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-04T17:48:08.870 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:08.866+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:08.906 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:08.906 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:48:08.906 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:48:08.906 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:48:08.906 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.906 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:12722 3.1b 3:d8152af1:::rbd_data.1134ef3a5a7d.00000000000000bd:head [delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-04T17:48:08.906 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:16.421965+0000", 2024-09-04T17:48:08.907 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 112.48334040100001, 2024-09-04T17:48:08.907 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.890002811, 2024-09-04T17:48:08.907 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:08.907 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:08.907 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:08.907 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:08.907 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:08.907 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:08.907 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 12722 2024-09-04T17:48:08.907 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.907 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:08.907 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.907 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:08.908 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421965+0000", 2024-09-04T17:48:08.908 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:08.908 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.908 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.908 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:08.908 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421965+0000", 2024-09-04T17:48:08.908 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:08.908 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.908 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.908 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:08.908 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421966+0000", 2024-09-04T17:48:08.908 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.102e-06 2024-09-04T17:48:08.908 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.908 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.909 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:08.909 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421971+0000", 2024-09-04T17:48:08.909 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9989999999999999e-06 2024-09-04T17:48:08.909 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.909 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.909 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:08.909 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421981+0000", 2024-09-04T17:48:08.909 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.7380000000000006e-06 2024-09-04T17:48:08.909 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.909 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.909 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:08.909 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421989+0000", 2024-09-04T17:48:08.909 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.1890000000000007e-06 2024-09-04T17:48:08.910 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.910 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.910 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:08.910 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.422050+0000", 2024-09-04T17:48:08.910 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.0702000000000003e-05 2024-09-04T17:48:08.910 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.910 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.910 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for peered", 2024-09-04T17:48:08.910 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.422059+0000", 2024-09-04T17:48:08.910 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5440000000000004e-06 2024-09-04T17:48:08.910 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.910 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.910 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:08.911 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.308156+0000", 2024-09-04T17:48:08.911 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.88609676199999998 2024-09-04T17:48:08.911 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.911 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.911 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:08.911 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.308398+0000", 2024-09-04T17:48:08.911 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024215400000000001 2024-09-04T17:48:08.911 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.911 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.911 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 7", 2024-09-04T17:48:08.911 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.308795+0000", 2024-09-04T17:48:08.911 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000396704 2024-09-04T17:48:08.911 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.911 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.912 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:48:08.912 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.310831+0000", 2024-09-04T17:48:08.912 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0020364200000000002 2024-09-04T17:48:08.912 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.912 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.912 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:08.912 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.311818+0000", 2024-09-04T17:48:08.912 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00098649799999999993 2024-09-04T17:48:08.912 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.912 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.912 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:08.912 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.311905+0000", 2024-09-04T17:48:08.913 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7070999999999994e-05 2024-09-04T17:48:08.913 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.913 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.913 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:08.913 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.311968+0000", 2024-09-04T17:48:08.913 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.2928000000000005e-05 2024-09-04T17:48:08.913 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:08.913 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:08.913 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:08.913 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.913 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.913 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37359 3.f e58/24 3:f29e50a5:::rbd_data.1134c5a43465.0000000000000021:head v 58'135, mlcod=58'133)", 2024-09-04T17:48:08.913 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:48:08.914 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 89.288009797000001, 2024-09-04T17:48:08.914 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053483197000000003, 2024-09-04T17:48:08.914 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:08.914 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:08.914 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:08.914 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:08.914 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.914 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:08.914 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:48:08.914 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:08.914 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.914 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.914 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:08.915 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:48:08.915 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:08.915 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.915 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.915 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:08.915 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:48:08.915 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1699999999999997e-07 2024-09-04T17:48:08.915 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.915 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.915 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:08.915 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617338+0000", 2024-09-04T17:48:08.915 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.1168999999999998e-05 2024-09-04T17:48:08.915 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.915 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.915 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:08.916 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617351+0000", 2024-09-04T17:48:08.916 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3094e-05 2024-09-04T17:48:08.916 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.916 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.916 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:08.916 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617359+0000", 2024-09-04T17:48:08.916 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7970000000000006e-06 2024-09-04T17:48:08.916 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.916 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.916 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:08.916 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617417+0000", 2024-09-04T17:48:08.916 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7079e-05 2024-09-04T17:48:08.916 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.916 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.917 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:08.917 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617438+0000", 2024-09-04T17:48:08.917 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1654999999999998e-05 2024-09-04T17:48:08.917 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.917 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.917 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:08.917 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670738+0000", 2024-09-04T17:48:08.917 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053300142000000002 2024-09-04T17:48:08.917 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.917 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.917 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:08.917 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670779+0000", 2024-09-04T17:48:08.917 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0343999999999999e-05 2024-09-04T17:48:08.917 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:08.917 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:08.918 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:08.918 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.918 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.918 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37358 3.1d e58/24 3:bd3215f3:::rbd_data.1134c5a43465.0000000000000020:head v 58'91, mlcod=58'89)", 2024-09-04T17:48:08.918 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617502+0000", 2024-09-04T17:48:08.918 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 89.287803079, 2024-09-04T17:48:08.918 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053505641999999999, 2024-09-04T17:48:08.918 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:08.918 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:08.918 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:08.918 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:08.918 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.919 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:08.919 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617502+0000", 2024-09-04T17:48:08.919 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:08.919 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.919 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.919 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:08.919 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617502+0000", 2024-09-04T17:48:08.919 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:08.919 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.919 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.919 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:08.919 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617503+0000", 2024-09-04T17:48:08.920 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.184e-06 2024-09-04T17:48:08.920 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.920 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.920 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:08.920 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617526+0000", 2024-09-04T17:48:08.920 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2099999999999998e-05 2024-09-04T17:48:08.920 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.920 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.920 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:08.920 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617540+0000", 2024-09-04T17:48:08.920 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4741e-05 2024-09-04T17:48:08.920 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.920 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.920 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:08.920 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617550+0000", 2024-09-04T17:48:08.921 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4259999999999992e-06 2024-09-04T17:48:08.921 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.921 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.921 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:08.921 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617613+0000", 2024-09-04T17:48:08.921 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.3625000000000002e-05 2024-09-04T17:48:08.921 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.921 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.921 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:08.921 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617637+0000", 2024-09-04T17:48:08.921 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.3604999999999998e-05 2024-09-04T17:48:08.921 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.921 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.922 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:08.922 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670967+0000", 2024-09-04T17:48:08.922 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053329881000000003 2024-09-04T17:48:08.922 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.922 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.922 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:08.922 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671008+0000", 2024-09-04T17:48:08.922 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.108e-05 2024-09-04T17:48:08.922 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:08.922 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:08.922 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:08.922 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.922 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.922 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37361 3.1c e58/24 3:3c7f8976:::rbd_data.1134c5a43465.0000000000000023:head v 58'159, mlcod=58'157)", 2024-09-04T17:48:08.923 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617578+0000", 2024-09-04T17:48:08.923 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 89.287727228999998, 2024-09-04T17:48:08.923 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053990367999999997, 2024-09-04T17:48:08.923 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:08.923 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:08.923 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:08.923 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:08.923 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.923 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:08.923 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617578+0000", 2024-09-04T17:48:08.923 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:08.923 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.923 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.924 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:08.924 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617578+0000", 2024-09-04T17:48:08.924 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:08.924 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.924 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.924 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:08.924 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617579+0000", 2024-09-04T17:48:08.924 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7299999999999996e-07 2024-09-04T17:48:08.924 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.924 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.924 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:08.924 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617610+0000", 2024-09-04T17:48:08.924 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0951000000000002e-05 2024-09-04T17:48:08.924 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.925 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.925 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:08.925 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:48:08.925 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0002e-05 2024-09-04T17:48:08.925 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.925 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.925 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:08.925 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617626+0000", 2024-09-04T17:48:08.925 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6660000000000002e-06 2024-09-04T17:48:08.925 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.925 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.925 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:08.926 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670548+0000", 2024-09-04T17:48:08.926 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.052921943999999999 2024-09-04T17:48:08.926 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.926 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.926 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:08.926 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670574+0000", 2024-09-04T17:48:08.926 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5405000000000002e-05 2024-09-04T17:48:08.926 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.926 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.926 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:08.926 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671522+0000", 2024-09-04T17:48:08.926 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00094815299999999997 2024-09-04T17:48:08.926 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.927 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.927 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:08.927 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671568+0000", 2024-09-04T17:48:08.927 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6674000000000001e-05 2024-09-04T17:48:08.927 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:08.927 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:08.927 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:08.927 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.927 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.927 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37357 3.15 e58/49 3:a897312a:::rbd_data.1134c5a43465.000000000000001f:head v 58'117, mlcod=58'115)", 2024-09-04T17:48:08.927 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:48:08.927 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 89.287684890999998, 2024-09-04T17:48:08.927 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053734297, 2024-09-04T17:48:08.928 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:08.928 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:08.928 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:08.928 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:08.928 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.928 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:08.928 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:48:08.928 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:08.928 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.928 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.928 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:08.928 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:48:08.928 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:08.929 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.929 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.929 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:08.929 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617621+0000", 2024-09-04T17:48:08.929 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.7300000000000004e-07 2024-09-04T17:48:08.929 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.929 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.929 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:08.929 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617642+0000", 2024-09-04T17:48:08.929 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0185000000000001e-05 2024-09-04T17:48:08.929 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.929 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.929 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:08.929 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617654+0000", 2024-09-04T17:48:08.930 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2745000000000001e-05 2024-09-04T17:48:08.930 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.930 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.930 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:08.930 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617662+0000", 2024-09-04T17:48:08.930 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.0499999999999992e-06 2024-09-04T17:48:08.930 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.930 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.930 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:08.930 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617727+0000", 2024-09-04T17:48:08.930 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.4923000000000002e-05 2024-09-04T17:48:08.930 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.930 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.931 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:08.931 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617753+0000", 2024-09-04T17:48:08.931 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5899999999999999e-05 2024-09-04T17:48:08.931 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.931 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.931 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:08.931 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671298+0000", 2024-09-04T17:48:08.931 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053545276000000003 2024-09-04T17:48:08.931 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.931 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.931 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:08.931 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671355+0000", 2024-09-04T17:48:08.932 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6245000000000001e-05 2024-09-04T17:48:08.932 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:08.932 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:08.932 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:08.932 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.932 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.932 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44687 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:08.932 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253012+0000", 2024-09-04T17:48:08.932 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 78.652292824, 2024-09-04T17:48:08.932 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051128777, 2024-09-04T17:48:08.932 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:08.932 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:08.932 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:08.932 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:08.933 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:08.933 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:08.933 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44687 2024-09-04T17:48:08.933 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.933 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:08.933 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.933 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:08.933 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253012+0000", 2024-09-04T17:48:08.933 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:08.933 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.933 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.933 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:08.934 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253012+0000", 2024-09-04T17:48:08.934 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:08.934 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.934 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.934 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:08.934 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253014+0000", 2024-09-04T17:48:08.934 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.24e-06 2024-09-04T17:48:08.934 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.934 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.934 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:08.934 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253019+0000", 2024-09-04T17:48:08.934 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4569999999999996e-06 2024-09-04T17:48:08.934 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.934 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.935 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:08.935 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253035+0000", 2024-09-04T17:48:08.935 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6291999999999999e-05 2024-09-04T17:48:08.935 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.935 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.935 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:08.935 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253050+0000", 2024-09-04T17:48:08.935 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4423000000000001e-05 2024-09-04T17:48:08.935 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.935 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.935 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:08.935 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304115+0000", 2024-09-04T17:48:08.936 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051064778999999998 2024-09-04T17:48:08.936 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.936 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.936 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:08.936 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304141+0000", 2024-09-04T17:48:08.936 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6585999999999999e-05 2024-09-04T17:48:08.936 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:08.936 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:08.936 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:08.936 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.936 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.936 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44688 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:08.936 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253067+0000", 2024-09-04T17:48:08.937 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 78.652238272000005, 2024-09-04T17:48:08.937 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051116979, 2024-09-04T17:48:08.937 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:08.937 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:08.937 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:08.937 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:08.937 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:08.937 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:08.937 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44688 2024-09-04T17:48:08.937 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.937 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:08.937 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.937 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:08.937 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253067+0000", 2024-09-04T17:48:08.938 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:08.938 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.938 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.938 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:08.938 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253067+0000", 2024-09-04T17:48:08.938 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:08.938 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.938 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.938 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:08.938 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253068+0000", 2024-09-04T17:48:08.938 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0729999999999999e-06 2024-09-04T17:48:08.938 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.938 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.939 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:08.939 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253073+0000", 2024-09-04T17:48:08.939 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2259999999999996e-06 2024-09-04T17:48:08.939 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.939 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.939 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:08.939 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253091+0000", 2024-09-04T17:48:08.939 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7255999999999999e-05 2024-09-04T17:48:08.939 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.939 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.939 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:08.939 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253103+0000", 2024-09-04T17:48:08.939 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2632999999999999e-05 2024-09-04T17:48:08.940 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.940 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.940 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:08.940 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304173+0000", 2024-09-04T17:48:08.940 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051069537999999998 2024-09-04T17:48:08.940 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.940 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.940 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:08.940 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304184+0000", 2024-09-04T17:48:08.940 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1253000000000001e-05 2024-09-04T17:48:08.940 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:08.940 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:08.940 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:08.941 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.941 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.941 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44689 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:08.941 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253121+0000", 2024-09-04T17:48:08.941 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 78.652184134999999, 2024-09-04T17:48:08.941 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051192595, 2024-09-04T17:48:08.941 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:08.941 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:08.941 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:08.941 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:08.941 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:08.941 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:08.942 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44689 2024-09-04T17:48:08.942 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.942 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:08.942 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.942 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:08.942 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253121+0000", 2024-09-04T17:48:08.942 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:08.942 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.942 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.942 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:08.942 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253121+0000", 2024-09-04T17:48:08.942 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:08.942 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.943 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.943 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:08.943 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253122+0000", 2024-09-04T17:48:08.943 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.15e-06 2024-09-04T17:48:08.943 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.943 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.943 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:08.943 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253128+0000", 2024-09-04T17:48:08.943 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2569999999999998e-06 2024-09-04T17:48:08.943 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.943 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.943 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:08.943 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253145+0000", 2024-09-04T17:48:08.944 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7176e-05 2024-09-04T17:48:08.944 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.944 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.944 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:08.944 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253158+0000", 2024-09-04T17:48:08.944 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3139000000000001e-05 2024-09-04T17:48:08.944 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.944 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.944 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:08.944 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304287+0000", 2024-09-04T17:48:08.944 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051129149999999998 2024-09-04T17:48:08.944 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.944 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.945 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:08.945 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304314+0000", 2024-09-04T17:48:08.945 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6723000000000001e-05 2024-09-04T17:48:08.945 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:08.945 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:08.945 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:08.945 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.945 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.945 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44690 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:08.945 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253177+0000", 2024-09-04T17:48:08.945 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 78.652128009999998, 2024-09-04T17:48:08.945 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051169263, 2024-09-04T17:48:08.945 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:08.946 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:08.946 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:08.946 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:08.946 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:08.946 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:08.946 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44690 2024-09-04T17:48:08.946 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.946 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:08.946 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.946 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:08.946 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253177+0000", 2024-09-04T17:48:08.946 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:08.946 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.946 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.947 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:08.947 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253177+0000", 2024-09-04T17:48:08.947 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:08.947 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.947 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.947 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:08.947 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253178+0000", 2024-09-04T17:48:08.947 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1000000000000001e-06 2024-09-04T17:48:08.947 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.947 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.947 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:08.947 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253184+0000", 2024-09-04T17:48:08.947 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2089999999999996e-06 2024-09-04T17:48:08.947 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.948 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.948 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:08.948 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253200+0000", 2024-09-04T17:48:08.948 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6087e-05 2024-09-04T17:48:08.948 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.948 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.948 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:08.948 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253231+0000", 2024-09-04T17:48:08.948 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0874999999999998e-05 2024-09-04T17:48:08.948 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.948 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.948 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:08.948 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304337+0000", 2024-09-04T17:48:08.949 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051106430000000001 2024-09-04T17:48:08.949 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.949 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.949 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:08.949 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304347+0000", 2024-09-04T17:48:08.949 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5619999999999993e-06 2024-09-04T17:48:08.949 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:08.949 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:08.949 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:08.949 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.949 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.949 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44691 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:08.949 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253249+0000", 2024-09-04T17:48:08.950 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 78.652055976, 2024-09-04T17:48:08.950 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051195630999999998, 2024-09-04T17:48:08.950 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:08.950 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:08.950 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:08.950 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:08.950 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:08.950 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:08.950 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44691 2024-09-04T17:48:08.950 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.950 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:08.950 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.950 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:08.951 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253249+0000", 2024-09-04T17:48:08.951 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:08.951 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.951 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.951 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:08.951 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253249+0000", 2024-09-04T17:48:08.951 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:08.951 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.951 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.951 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:08.951 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253251+0000", 2024-09-04T17:48:08.951 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2330000000000001e-06 2024-09-04T17:48:08.951 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.952 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.952 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:08.952 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253256+0000", 2024-09-04T17:48:08.952 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6099999999999997e-06 2024-09-04T17:48:08.952 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.952 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.952 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:08.952 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253274+0000", 2024-09-04T17:48:08.952 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7952999999999999e-05 2024-09-04T17:48:08.952 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.952 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.952 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:08.952 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253287+0000", 2024-09-04T17:48:08.953 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2839e-05 2024-09-04T17:48:08.953 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.953 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.953 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:08.953 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304396+0000", 2024-09-04T17:48:08.953 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051108990999999999 2024-09-04T17:48:08.953 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.953 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.953 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:08.953 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304445+0000", 2024-09-04T17:48:08.953 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9005000000000003e-05 2024-09-04T17:48:08.953 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:08.953 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:08.954 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:08.954 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.954 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.954 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44692 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:08.954 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253337+0000", 2024-09-04T17:48:08.954 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 78.651968402999998, 2024-09-04T17:48:08.954 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051138909000000003, 2024-09-04T17:48:08.954 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:08.954 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:08.954 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:08.954 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:08.954 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:08.954 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:08.955 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44692 2024-09-04T17:48:08.955 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.955 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:08.955 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.955 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:08.955 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253337+0000", 2024-09-04T17:48:08.955 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:08.955 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.955 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.955 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:08.955 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253337+0000", 2024-09-04T17:48:08.955 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:08.956 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.956 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.956 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:08.956 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253338+0000", 2024-09-04T17:48:08.956 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.463e-06 2024-09-04T17:48:08.956 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.956 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.956 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:08.957 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253357+0000", 2024-09-04T17:48:08.957 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9043e-05 2024-09-04T17:48:08.957 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.957 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.957 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:08.957 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253379+0000", 2024-09-04T17:48:08.957 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1243e-05 2024-09-04T17:48:08.957 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.957 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.957 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:08.957 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253396+0000", 2024-09-04T17:48:08.957 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7059e-05 2024-09-04T17:48:08.957 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.958 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.958 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:08.958 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304466+0000", 2024-09-04T17:48:08.958 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051070624000000002 2024-09-04T17:48:08.958 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.958 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.958 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:08.958 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304476+0000", 2024-09-04T17:48:08.958 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4769999999999992e-06 2024-09-04T17:48:08.958 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:08.958 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:08.958 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:08.959 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.959 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.959 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44693 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:08.959 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253415+0000", 2024-09-04T17:48:08.959 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 78.651889999999995, 2024-09-04T17:48:08.959 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051141949999999999, 2024-09-04T17:48:08.959 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:08.959 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:08.959 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:08.959 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:08.959 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:08.959 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:08.959 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44693 2024-09-04T17:48:08.959 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.960 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:08.960 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.960 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:08.960 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253415+0000", 2024-09-04T17:48:08.960 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:08.960 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.960 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.960 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:08.960 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253415+0000", 2024-09-04T17:48:08.960 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:08.960 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.960 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.960 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:08.961 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253416+0000", 2024-09-04T17:48:08.961 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1799999999999999e-06 2024-09-04T17:48:08.961 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.961 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.961 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:08.961 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253422+0000", 2024-09-04T17:48:08.961 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.5940000000000001e-06 2024-09-04T17:48:08.961 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.961 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.961 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:08.961 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253438+0000", 2024-09-04T17:48:08.961 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6008999999999999e-05 2024-09-04T17:48:08.962 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.962 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.962 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:08.962 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253452+0000", 2024-09-04T17:48:08.962 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3543000000000001e-05 2024-09-04T17:48:08.962 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.962 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.962 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:08.962 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304533+0000", 2024-09-04T17:48:08.962 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051081609 2024-09-04T17:48:08.962 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.962 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.962 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:08.963 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304557+0000", 2024-09-04T17:48:08.963 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4015e-05 2024-09-04T17:48:08.963 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:08.963 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:08.963 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:08.963 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.963 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.963 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44694 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:08.963 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253469+0000", 2024-09-04T17:48:08.963 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 78.651835875000003, 2024-09-04T17:48:08.963 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051121486000000001, 2024-09-04T17:48:08.963 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:08.963 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:08.963 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:08.964 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:08.964 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:08.964 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:08.964 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44694 2024-09-04T17:48:08.964 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.964 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:08.964 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.964 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:08.964 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253469+0000", 2024-09-04T17:48:08.964 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:08.964 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.964 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.964 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:08.964 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253469+0000", 2024-09-04T17:48:08.965 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:08.965 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.965 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.965 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:08.965 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253471+0000", 2024-09-04T17:48:08.965 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1400000000000001e-06 2024-09-04T17:48:08.965 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.965 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.965 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:08.965 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253476+0000", 2024-09-04T17:48:08.965 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7370000000000003e-06 2024-09-04T17:48:08.965 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.965 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.966 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:08.966 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253493+0000", 2024-09-04T17:48:08.966 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6549e-05 2024-09-04T17:48:08.966 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.966 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.966 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:08.966 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253506+0000", 2024-09-04T17:48:08.966 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3406e-05 2024-09-04T17:48:08.966 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.966 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.966 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:08.966 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304582+0000", 2024-09-04T17:48:08.967 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051075534999999998 2024-09-04T17:48:08.967 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.967 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.967 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:08.967 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304591+0000", 2024-09-04T17:48:08.967 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1190000000000003e-06 2024-09-04T17:48:08.967 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:08.967 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:08.967 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:08.967 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.967 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.967 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44695 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:08.967 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253552+0000", 2024-09-04T17:48:08.967 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 78.651753120999999, 2024-09-04T17:48:08.968 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051068872000000001, 2024-09-04T17:48:08.968 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:08.968 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:08.968 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:08.968 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:08.968 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:08.968 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:08.968 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44695 2024-09-04T17:48:08.968 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.968 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:08.968 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.968 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:08.968 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253552+0000", 2024-09-04T17:48:08.968 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:08.969 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.969 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.969 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:08.969 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253552+0000", 2024-09-04T17:48:08.969 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:08.969 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.969 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.969 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:08.969 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253553+0000", 2024-09-04T17:48:08.969 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1960000000000001e-06 2024-09-04T17:48:08.969 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.969 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.969 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:08.970 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253560+0000", 2024-09-04T17:48:08.970 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.2040000000000002e-06 2024-09-04T17:48:08.970 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.970 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.970 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:08.970 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253578+0000", 2024-09-04T17:48:08.970 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8241999999999999e-05 2024-09-04T17:48:08.970 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.970 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.970 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:08.970 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253591+0000", 2024-09-04T17:48:08.970 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3453e-05 2024-09-04T17:48:08.971 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.971 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.971 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:08.971 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304614+0000", 2024-09-04T17:48:08.971 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051022456000000001 2024-09-04T17:48:08.971 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.971 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.971 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:08.971 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304621+0000", 2024-09-04T17:48:08.971 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3209999999999999e-06 2024-09-04T17:48:08.971 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:08.971 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:08.971 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:08.971 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.972 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.972 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44696 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:08.972 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253609+0000", 2024-09-04T17:48:08.972 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 78.651696619999996, 2024-09-04T17:48:08.972 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051115351000000003, 2024-09-04T17:48:08.972 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:08.972 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:08.972 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:08.972 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:08.972 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:08.972 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:08.972 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44696 2024-09-04T17:48:08.972 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.973 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:08.973 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.973 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:08.973 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253609+0000", 2024-09-04T17:48:08.973 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:08.973 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.973 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.973 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:08.973 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253609+0000", 2024-09-04T17:48:08.973 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:08.973 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.973 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.973 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:08.974 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253610+0000", 2024-09-04T17:48:08.974 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2300000000000001e-06 2024-09-04T17:48:08.974 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.974 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.974 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:08.974 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253615+0000", 2024-09-04T17:48:08.974 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7260000000000002e-06 2024-09-04T17:48:08.974 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.974 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.974 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:08.974 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253632+0000", 2024-09-04T17:48:08.974 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7039999999999999e-05 2024-09-04T17:48:08.974 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.974 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.975 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:08.975 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253666+0000", 2024-09-04T17:48:08.975 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4662000000000001e-05 2024-09-04T17:48:08.975 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.975 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.975 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:08.975 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304677+0000", 2024-09-04T17:48:08.975 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051011017999999998 2024-09-04T17:48:08.975 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.975 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.975 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:08.975 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304724+0000", 2024-09-04T17:48:08.975 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6675000000000003e-05 2024-09-04T17:48:08.976 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:08.976 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:08.976 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:08.976 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.976 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.976 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44697 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:08.976 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253684+0000", 2024-09-04T17:48:08.976 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 78.651621125999995, 2024-09-04T17:48:08.976 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051073107, 2024-09-04T17:48:08.976 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:08.976 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:08.976 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:08.976 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:08.976 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:08.977 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:08.977 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44697 2024-09-04T17:48:08.977 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.977 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:08.977 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.977 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:08.977 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253684+0000", 2024-09-04T17:48:08.977 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:08.977 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.977 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.977 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:08.977 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253684+0000", 2024-09-04T17:48:08.977 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:08.978 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.978 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.978 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:08.978 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253685+0000", 2024-09-04T17:48:08.978 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0699999999999999e-06 2024-09-04T17:48:08.978 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.978 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.978 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:08.978 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253690+0000", 2024-09-04T17:48:08.978 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1959999999999999e-06 2024-09-04T17:48:08.978 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.978 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.978 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:08.979 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253707+0000", 2024-09-04T17:48:08.979 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6946e-05 2024-09-04T17:48:08.979 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.979 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.979 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:08.979 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253720+0000", 2024-09-04T17:48:08.979 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3076e-05 2024-09-04T17:48:08.979 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.979 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.979 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:08.979 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304745+0000", 2024-09-04T17:48:08.979 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051024760000000002 2024-09-04T17:48:08.979 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.980 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.980 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:08.980 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304757+0000", 2024-09-04T17:48:08.980 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2058999999999999e-05 2024-09-04T17:48:08.980 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:08.980 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:08.980 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:08.980 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.980 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.980 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44698 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:08.980 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253737+0000", 2024-09-04T17:48:08.980 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 78.651568775000001, 2024-09-04T17:48:08.981 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051176078, 2024-09-04T17:48:08.981 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:08.981 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:08.981 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:08.981 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:08.981 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:08.981 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:08.981 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44698 2024-09-04T17:48:08.981 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.981 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:08.981 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.981 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:08.981 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253737+0000", 2024-09-04T17:48:08.981 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:08.982 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.982 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.982 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:08.982 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253737+0000", 2024-09-04T17:48:08.982 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:08.982 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.982 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.982 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:08.982 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253738+0000", 2024-09-04T17:48:08.982 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0300000000000001e-06 2024-09-04T17:48:08.982 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.982 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.983 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:08.983 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253752+0000", 2024-09-04T17:48:08.983 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4358999999999999e-05 2024-09-04T17:48:08.983 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.983 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.983 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:08.983 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253771+0000", 2024-09-04T17:48:08.983 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8879999999999999e-05 2024-09-04T17:48:08.983 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.983 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.983 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:08.983 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253785+0000", 2024-09-04T17:48:08.983 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3852000000000001e-05 2024-09-04T17:48:08.984 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.984 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.984 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:08.984 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304876+0000", 2024-09-04T17:48:08.984 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051091105999999997 2024-09-04T17:48:08.984 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.984 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.984 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:08.984 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304913+0000", 2024-09-04T17:48:08.984 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.6850999999999997e-05 2024-09-04T17:48:08.984 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:08.985 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:08.985 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:08.985 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.985 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.985 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44699 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:08.985 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253804+0000", 2024-09-04T17:48:08.985 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 78.651501250999999, 2024-09-04T17:48:08.985 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051172250000000002, 2024-09-04T17:48:08.985 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:08.985 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:08.985 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:08.986 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:08.986 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:08.986 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:08.986 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44699 2024-09-04T17:48:08.986 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.986 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:08.986 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.986 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:08.986 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253804+0000", 2024-09-04T17:48:08.986 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:08.986 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.986 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.987 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:08.987 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253804+0000", 2024-09-04T17:48:08.987 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:08.987 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.987 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.987 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:08.987 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253805+0000", 2024-09-04T17:48:08.987 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.08e-06 2024-09-04T17:48:08.987 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.987 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.987 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:08.987 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253810+0000", 2024-09-04T17:48:08.987 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1830000000000002e-06 2024-09-04T17:48:08.987 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.988 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.988 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:08.988 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253828+0000", 2024-09-04T17:48:08.988 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7662000000000001e-05 2024-09-04T17:48:08.988 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.988 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.988 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:08.988 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253841+0000", 2024-09-04T17:48:08.988 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3203999999999999e-05 2024-09-04T17:48:08.988 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.988 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.988 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:08.988 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304966+0000", 2024-09-04T17:48:08.989 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051125273999999998 2024-09-04T17:48:08.989 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.989 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.989 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:08.989 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304976+0000", 2024-09-04T17:48:08.989 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.8470000000000008e-06 2024-09-04T17:48:08.989 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:08.989 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:08.989 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:08.989 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.989 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.989 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44700 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:08.989 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253858+0000", 2024-09-04T17:48:08.990 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 78.651447597000001, 2024-09-04T17:48:08.990 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051218957000000002, 2024-09-04T17:48:08.990 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:08.990 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:08.990 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:08.990 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:08.990 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:08.990 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:08.990 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44700 2024-09-04T17:48:08.990 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.990 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:08.990 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.991 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:08.991 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253858+0000", 2024-09-04T17:48:08.991 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:08.991 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.991 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.991 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:08.991 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253858+0000", 2024-09-04T17:48:08.991 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:08.991 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.991 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.991 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:08.991 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253859+0000", 2024-09-04T17:48:08.992 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.217e-06 2024-09-04T17:48:08.992 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.992 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.992 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:08.992 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253864+0000", 2024-09-04T17:48:08.992 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7600000000000002e-06 2024-09-04T17:48:08.992 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.992 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.992 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:08.992 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253880+0000", 2024-09-04T17:48:08.992 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6501999999999999e-05 2024-09-04T17:48:08.992 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.992 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.993 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:08.993 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253899+0000", 2024-09-04T17:48:08.993 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8907e-05 2024-09-04T17:48:08.993 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.993 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.993 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:08.993 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305052+0000", 2024-09-04T17:48:08.993 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051152989000000003 2024-09-04T17:48:08.993 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.993 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.993 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:08.993 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305077+0000", 2024-09-04T17:48:08.993 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4581999999999999e-05 2024-09-04T17:48:08.994 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:08.994 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:08.994 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:08.994 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.994 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.994 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44701 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:08.994 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253917+0000", 2024-09-04T17:48:08.994 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 78.651388245000007, 2024-09-04T17:48:08.994 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051231113000000002, 2024-09-04T17:48:08.994 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:08.994 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:08.994 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:08.994 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:08.994 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:08.995 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:08.995 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44701 2024-09-04T17:48:08.995 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.995 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:08.995 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.995 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:08.995 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253917+0000", 2024-09-04T17:48:08.995 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:08.995 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.995 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.995 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:08.995 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253917+0000", 2024-09-04T17:48:08.995 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:08.996 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.996 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.996 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:08.996 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253918+0000", 2024-09-04T17:48:08.996 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.136e-06 2024-09-04T17:48:08.996 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.996 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.996 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:08.996 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253924+0000", 2024-09-04T17:48:08.996 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.3070000000000002e-06 2024-09-04T17:48:08.996 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.996 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.996 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:08.996 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253940+0000", 2024-09-04T17:48:08.997 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6612e-05 2024-09-04T17:48:08.997 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.997 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.997 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:08.997 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253954+0000", 2024-09-04T17:48:08.997 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3709999999999999e-05 2024-09-04T17:48:08.997 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.997 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.997 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:08.997 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305136+0000", 2024-09-04T17:48:08.997 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051181935999999997 2024-09-04T17:48:08.997 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:08.997 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:08.998 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:08.998 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305148+0000", 2024-09-04T17:48:08.998 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2412000000000001e-05 2024-09-04T17:48:08.998 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:08.998 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:08.998 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:08.998 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:08.998 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:08.998 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:09.000 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-04T17:48:09.000 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:48:08.994+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:48:09.100 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:09.096+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:09.164 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:09.164 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:48:09.164 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:48:09.164 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:09.179 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-04T17:48:09.201 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:09.200+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:09.302 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:09.302 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:48:09.302 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:48:09.302 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:48:09.302 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:09.302 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:09.300+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:09.315 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-04T17:48:09.403 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:09.400+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:09.439 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:09.439 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:48:09.439 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:48:09.440 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:48:09.440 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.440 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8831 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2453504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:09.440 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:48:09.440 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 116.381395356, 2024-09-04T17:48:09.440 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14731111799999999, 2024-09-04T17:48:09.440 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:09.440 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:09.440 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:09.440 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:09.440 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:09.440 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:09.440 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8831 2024-09-04T17:48:09.441 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.441 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:09.441 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.441 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:09.441 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:48:09.441 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:09.441 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.441 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.441 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:09.441 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:48:09.441 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:09.441 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.442 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.442 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:09.442 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057811+0000", 2024-09-04T17:48:09.442 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.083e-06 2024-09-04T17:48:09.442 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.442 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.442 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:09.442 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057814+0000", 2024-09-04T17:48:09.442 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.407e-06 2024-09-04T17:48:09.442 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.442 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.442 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:09.442 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057825+0000", 2024-09-04T17:48:09.442 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.17e-05 2024-09-04T17:48:09.443 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.443 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.443 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:09.443 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057836+0000", 2024-09-04T17:48:09.443 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0283e-05 2024-09-04T17:48:09.443 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.443 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.443 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:09.443 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.204762+0000", 2024-09-04T17:48:09.443 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.146926205 2024-09-04T17:48:09.443 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.443 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.443 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:09.444 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.204895+0000", 2024-09-04T17:48:09.444 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013328999999999999 2024-09-04T17:48:09.444 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.444 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.444 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:09.444 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205121+0000", 2024-09-04T17:48:09.444 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00022614999999999999 2024-09-04T17:48:09.444 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:09.444 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:09.444 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:09.444 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.444 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.444 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8832 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2490368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:09.445 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:48:09.445 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 116.381357323, 2024-09-04T17:48:09.445 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14770012299999999, 2024-09-04T17:48:09.445 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:09.445 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:09.445 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:09.445 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:09.445 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:09.445 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:09.445 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8832 2024-09-04T17:48:09.445 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.445 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:09.445 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.446 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:09.446 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:48:09.446 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:09.446 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.446 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.446 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:09.446 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:48:09.446 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:09.446 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.446 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.446 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:09.446 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057849+0000", 2024-09-04T17:48:09.446 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6700000000000003e-07 2024-09-04T17:48:09.447 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.447 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.447 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:09.447 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057852+0000", 2024-09-04T17:48:09.447 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6359999999999998e-06 2024-09-04T17:48:09.447 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.447 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.447 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:09.447 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057860+0000", 2024-09-04T17:48:09.447 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8140000000000007e-06 2024-09-04T17:48:09.447 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.447 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.447 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:09.447 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057868+0000", 2024-09-04T17:48:09.448 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1999999999999997e-06 2024-09-04T17:48:09.448 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.448 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.448 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:09.448 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205208+0000", 2024-09-04T17:48:09.448 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14734067300000001 2024-09-04T17:48:09.448 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.448 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.448 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:09.448 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205347+0000", 2024-09-04T17:48:09.448 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013914900000000001 2024-09-04T17:48:09.448 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.449 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.449 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:09.449 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205548+0000", 2024-09-04T17:48:09.449 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000200984 2024-09-04T17:48:09.449 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:09.449 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:09.449 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:09.449 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.449 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.449 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8833 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2527232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:09.449 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:48:09.449 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 116.381325453, 2024-09-04T17:48:09.449 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148062572, 2024-09-04T17:48:09.450 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:09.450 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:09.450 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:09.450 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:09.450 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:09.450 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:09.450 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8833 2024-09-04T17:48:09.450 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.450 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:09.450 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.450 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:09.450 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:48:09.450 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:09.450 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.451 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.451 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:09.451 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:48:09.451 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:09.451 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.451 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.451 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:09.451 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057881+0000", 2024-09-04T17:48:09.451 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4300000000000002e-07 2024-09-04T17:48:09.451 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.451 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.451 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:09.451 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057893+0000", 2024-09-04T17:48:09.451 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1973999999999999e-05 2024-09-04T17:48:09.452 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.452 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.452 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:09.452 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057906+0000", 2024-09-04T17:48:09.452 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2716e-05 2024-09-04T17:48:09.452 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.452 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.452 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:09.452 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057913+0000", 2024-09-04T17:48:09.452 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3930000000000002e-06 2024-09-04T17:48:09.452 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.452 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.452 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:09.452 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205621+0000", 2024-09-04T17:48:09.453 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14770788800000001 2024-09-04T17:48:09.453 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.453 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.453 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:09.453 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205740+0000", 2024-09-04T17:48:09.453 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000118936 2024-09-04T17:48:09.453 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.453 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.453 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:09.453 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205943+0000", 2024-09-04T17:48:09.453 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020292199999999999 2024-09-04T17:48:09.453 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:09.453 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:09.454 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:09.454 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.454 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.454 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8834 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2564096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:09.454 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:48:09.454 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 116.38128217400001, 2024-09-04T17:48:09.454 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14846753200000001, 2024-09-04T17:48:09.454 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:09.454 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:09.454 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:09.454 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:09.454 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:09.454 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:09.455 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8834 2024-09-04T17:48:09.455 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.455 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:09.455 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.455 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:09.455 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:48:09.455 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:09.455 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.455 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.455 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:09.455 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:48:09.455 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:09.455 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.455 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.456 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:09.456 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057925+0000", 2024-09-04T17:48:09.456 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0270000000000001e-06 2024-09-04T17:48:09.456 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.456 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.456 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:09.456 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057936+0000", 2024-09-04T17:48:09.456 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.112e-05 2024-09-04T17:48:09.456 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.456 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.456 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:09.456 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057946+0000", 2024-09-04T17:48:09.457 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.009e-05 2024-09-04T17:48:09.457 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.457 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.457 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:09.457 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057953+0000", 2024-09-04T17:48:09.457 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.6229999999999997e-06 2024-09-04T17:48:09.457 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.457 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.457 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:09.457 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206027+0000", 2024-09-04T17:48:09.457 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148073591 2024-09-04T17:48:09.457 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.458 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.458 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:09.458 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206161+0000", 2024-09-04T17:48:09.458 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013411599999999999 2024-09-04T17:48:09.458 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.458 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.458 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:09.458 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206391+0000", 2024-09-04T17:48:09.458 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000229965 2024-09-04T17:48:09.458 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:09.458 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:09.458 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:09.458 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.459 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.459 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8835 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2600960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:09.459 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:48:09.459 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 116.381242524, 2024-09-04T17:48:09.459 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14882941599999999, 2024-09-04T17:48:09.459 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:09.459 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:09.459 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:09.459 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:09.459 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:09.459 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:09.459 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8835 2024-09-04T17:48:09.459 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.460 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:09.460 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.460 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:09.460 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:48:09.460 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:09.460 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.460 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.460 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:09.460 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:48:09.460 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:09.460 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.460 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.460 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:09.460 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058544+0000", 2024-09-04T17:48:09.461 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00058061899999999999 2024-09-04T17:48:09.461 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.461 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.461 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:09.461 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058550+0000", 2024-09-04T17:48:09.461 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.1129999999999997e-06 2024-09-04T17:48:09.461 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.461 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.461 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:09.461 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058566+0000", 2024-09-04T17:48:09.461 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5820000000000001e-05 2024-09-04T17:48:09.461 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.461 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.462 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:09.462 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058578+0000", 2024-09-04T17:48:09.462 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2537e-05 2024-09-04T17:48:09.462 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.462 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.462 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:09.462 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206470+0000", 2024-09-04T17:48:09.462 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14789153799999999 2024-09-04T17:48:09.462 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.462 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.462 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:09.462 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206589+0000", 2024-09-04T17:48:09.462 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000118837 2024-09-04T17:48:09.462 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.463 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.463 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:09.463 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206793+0000", 2024-09-04T17:48:09.463 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000203952 2024-09-04T17:48:09.463 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:09.463 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:09.463 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:09.463 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.463 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.463 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8836 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2654208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:09.463 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:48:09.463 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 116.380609836, 2024-09-04T17:48:09.463 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14862433999999999, 2024-09-04T17:48:09.464 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:09.464 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:09.464 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:09.464 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:09.464 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:09.464 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:09.464 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8836 2024-09-04T17:48:09.464 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.464 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:09.464 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.464 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:09.464 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:48:09.465 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:09.465 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.465 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.465 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:09.465 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:48:09.465 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:09.465 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.465 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.465 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:09.465 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062663+0000", 2024-09-04T17:48:09.465 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0040670619999999998 2024-09-04T17:48:09.465 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.465 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.466 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:09.466 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062668+0000", 2024-09-04T17:48:09.466 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4199999999999998e-06 2024-09-04T17:48:09.466 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.466 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.466 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:09.466 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062689+0000", 2024-09-04T17:48:09.466 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0356999999999998e-05 2024-09-04T17:48:09.466 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.466 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.466 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:09.466 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062726+0000", 2024-09-04T17:48:09.466 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7302999999999998e-05 2024-09-04T17:48:09.467 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.467 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.467 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:09.467 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206875+0000", 2024-09-04T17:48:09.467 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14414883100000001 2024-09-04T17:48:09.467 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.467 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.467 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:09.467 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.207016+0000", 2024-09-04T17:48:09.467 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000140822 2024-09-04T17:48:09.467 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.467 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.467 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:09.467 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.207220+0000", 2024-09-04T17:48:09.468 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020454499999999999 2024-09-04T17:48:09.468 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:09.468 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:09.468 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:09.468 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.468 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.468 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8844 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2928640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:09.468 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:48:09.468 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 116.37624575300001, 2024-09-04T17:48:09.468 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14752153100000001, 2024-09-04T17:48:09.468 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:09.468 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:09.468 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:09.469 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:09.469 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:09.469 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:09.469 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8844 2024-09-04T17:48:09.469 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.469 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:09.469 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.469 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:09.469 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:48:09.469 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:09.469 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.469 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.469 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:09.470 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:48:09.470 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:09.470 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.470 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.470 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:09.470 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062961+0000", 2024-09-04T17:48:09.470 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6700000000000003e-07 2024-09-04T17:48:09.470 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.470 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.470 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:09.470 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062973+0000", 2024-09-04T17:48:09.470 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2109999999999999e-05 2024-09-04T17:48:09.470 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.471 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.471 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:09.471 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062982+0000", 2024-09-04T17:48:09.471 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.6900000000000004e-06 2024-09-04T17:48:09.471 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.471 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.471 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:09.471 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062990+0000", 2024-09-04T17:48:09.471 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1899999999999998e-06 2024-09-04T17:48:09.471 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.471 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.471 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:09.471 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210090+0000", 2024-09-04T17:48:09.471 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14710026300000001 2024-09-04T17:48:09.471 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.472 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.472 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:09.472 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210225+0000", 2024-09-04T17:48:09.472 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013532400000000001 2024-09-04T17:48:09.472 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.472 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.472 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:09.472 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210481+0000", 2024-09-04T17:48:09.472 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00025628700000000001 2024-09-04T17:48:09.472 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:09.472 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:09.472 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:09.472 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.473 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.473 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8845 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2965504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:09.473 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:48:09.473 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 116.37620603000001, 2024-09-04T17:48:09.473 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147916819, 2024-09-04T17:48:09.473 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:09.473 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:09.473 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:09.473 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:09.473 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:09.473 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:09.473 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8845 2024-09-04T17:48:09.473 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.474 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:09.474 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.474 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:09.474 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:48:09.474 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:09.474 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.474 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.474 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:09.474 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:48:09.474 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:09.474 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.474 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.474 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:09.474 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:48:09.475 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.0399999999999995e-07 2024-09-04T17:48:09.475 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.475 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.475 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:09.475 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063003+0000", 2024-09-04T17:48:09.475 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6529999999999998e-06 2024-09-04T17:48:09.475 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.475 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.475 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:09.475 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063012+0000", 2024-09-04T17:48:09.475 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9500000000000007e-06 2024-09-04T17:48:09.475 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.475 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.475 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:09.476 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063027+0000", 2024-09-04T17:48:09.476 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5443000000000001e-05 2024-09-04T17:48:09.476 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.476 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.476 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:09.476 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210571+0000", 2024-09-04T17:48:09.476 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14754353100000001 2024-09-04T17:48:09.476 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.476 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.476 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:09.476 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210714+0000", 2024-09-04T17:48:09.476 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014295599999999999 2024-09-04T17:48:09.476 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.477 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.477 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:09.477 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210916+0000", 2024-09-04T17:48:09.477 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020258200000000001 2024-09-04T17:48:09.477 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:09.477 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:09.477 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:09.477 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.477 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.477 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8846 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3002368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:09.477 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:48:09.477 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 116.37616849299999, 2024-09-04T17:48:09.477 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14828011599999999, 2024-09-04T17:48:09.478 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:09.478 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:09.478 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:09.478 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:09.478 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:09.478 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:09.478 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8846 2024-09-04T17:48:09.478 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.478 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:09.478 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.478 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:09.478 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:48:09.478 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:09.479 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.479 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.479 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:09.479 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:48:09.479 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:09.479 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.479 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.479 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:09.479 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063038+0000", 2024-09-04T17:48:09.479 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.23e-07 2024-09-04T17:48:09.479 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.479 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.479 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:09.480 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063051+0000", 2024-09-04T17:48:09.480 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2757e-05 2024-09-04T17:48:09.480 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.480 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.480 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:09.480 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063070+0000", 2024-09-04T17:48:09.480 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9663000000000001e-05 2024-09-04T17:48:09.480 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.480 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.480 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:09.480 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063079+0000", 2024-09-04T17:48:09.480 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.0929999999999992e-06 2024-09-04T17:48:09.480 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.480 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.481 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:09.481 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210989+0000", 2024-09-04T17:48:09.481 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14790984500000001 2024-09-04T17:48:09.481 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.481 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.481 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:09.481 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211109+0000", 2024-09-04T17:48:09.481 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000120056 2024-09-04T17:48:09.481 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.481 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.481 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:09.481 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211317+0000", 2024-09-04T17:48:09.481 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020797900000000001 2024-09-04T17:48:09.482 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:09.482 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:09.482 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:09.482 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.482 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.482 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8847 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3039232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:09.482 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:48:09.482 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 116.376094061, 2024-09-04T17:48:09.482 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14859408900000001, 2024-09-04T17:48:09.482 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:09.482 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:09.482 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:09.482 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:09.483 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:09.483 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:09.483 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8847 2024-09-04T17:48:09.483 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.483 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:09.483 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.483 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:09.483 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:48:09.483 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:09.483 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.483 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.483 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:09.483 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:48:09.484 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:09.484 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.484 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.484 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:09.484 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063113+0000", 2024-09-04T17:48:09.484 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.204e-06 2024-09-04T17:48:09.484 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.484 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.484 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:09.484 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063116+0000", 2024-09-04T17:48:09.484 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.3330000000000001e-06 2024-09-04T17:48:09.484 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.484 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.485 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:09.485 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063126+0000", 2024-09-04T17:48:09.485 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0166999999999999e-05 2024-09-04T17:48:09.485 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.485 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.485 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:09.485 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063134+0000", 2024-09-04T17:48:09.485 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4329999999999998e-06 2024-09-04T17:48:09.485 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.485 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.485 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:09.485 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211392+0000", 2024-09-04T17:48:09.485 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148258109 2024-09-04T17:48:09.486 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.486 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.486 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:09.486 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211506+0000", 2024-09-04T17:48:09.486 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000113813 2024-09-04T17:48:09.486 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.486 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.486 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:09.486 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211706+0000", 2024-09-04T17:48:09.486 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020002999999999999 2024-09-04T17:48:09.487 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:09.487 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:09.487 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:09.487 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.487 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.487 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8848 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:09.487 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:48:09.487 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 116.376061724, 2024-09-04T17:48:09.487 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14907967599999999, 2024-09-04T17:48:09.487 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:09.487 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:09.487 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:09.488 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:09.488 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:09.488 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:09.488 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8848 2024-09-04T17:48:09.488 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.488 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:09.488 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.488 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:09.488 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:48:09.488 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:09.488 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.488 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.488 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:09.489 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:48:09.489 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:09.489 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.489 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.489 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:09.489 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070589+0000", 2024-09-04T17:48:09.489 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.007445386 2024-09-04T17:48:09.489 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.489 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.489 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:09.489 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070600+0000", 2024-09-04T17:48:09.489 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0346999999999999e-05 2024-09-04T17:48:09.489 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.489 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.490 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:09.490 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070622+0000", 2024-09-04T17:48:09.490 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2289999999999998e-05 2024-09-04T17:48:09.490 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.490 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.490 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:09.490 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070636+0000", 2024-09-04T17:48:09.490 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4093e-05 2024-09-04T17:48:09.490 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.490 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.490 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:09.490 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211815+0000", 2024-09-04T17:48:09.490 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14117874699999999 2024-09-04T17:48:09.491 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.491 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.491 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:09.491 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211983+0000", 2024-09-04T17:48:09.491 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00016852800000000001 2024-09-04T17:48:09.491 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.491 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.491 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:09.491 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.212224+0000", 2024-09-04T17:48:09.491 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024028499999999999 2024-09-04T17:48:09.491 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:09.491 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:09.491 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:09.491 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.492 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.492 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8862 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2019328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:09.492 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:48:09.492 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 116.368012256, 2024-09-04T17:48:09.492 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14755236399999999, 2024-09-04T17:48:09.492 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:09.492 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:09.492 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:09.492 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:09.492 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:09.492 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:09.492 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8862 2024-09-04T17:48:09.493 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.493 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:09.493 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.493 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:09.493 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:48:09.493 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:09.493 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.493 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.493 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:09.493 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:48:09.493 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:09.493 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.493 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.494 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:09.494 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071194+0000", 2024-09-04T17:48:09.494 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.5000000000000002e-07 2024-09-04T17:48:09.494 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.494 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.494 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:09.494 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071197+0000", 2024-09-04T17:48:09.494 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5900000000000002e-06 2024-09-04T17:48:09.494 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.494 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.494 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:09.495 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071206+0000", 2024-09-04T17:48:09.495 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9700000000000005e-06 2024-09-04T17:48:09.495 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.495 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.495 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:09.495 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071213+0000", 2024-09-04T17:48:09.495 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.9070000000000001e-06 2024-09-04T17:48:09.495 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.495 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.495 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:09.495 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218396+0000", 2024-09-04T17:48:09.495 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147183273 2024-09-04T17:48:09.495 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.496 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.496 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:09.496 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218538+0000", 2024-09-04T17:48:09.496 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014194199999999999 2024-09-04T17:48:09.496 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.496 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.496 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:09.496 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218746+0000", 2024-09-04T17:48:09.496 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000208032 2024-09-04T17:48:09.496 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:09.496 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:09.496 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:09.496 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.497 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.497 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8863 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2056192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:09.497 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:48:09.497 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 116.367983946, 2024-09-04T17:48:09.497 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14792281099999999, 2024-09-04T17:48:09.497 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:09.497 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:09.497 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:09.497 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:09.497 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:09.497 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:09.497 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8863 2024-09-04T17:48:09.497 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.498 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:09.498 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.498 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:09.498 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:48:09.498 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:09.498 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.498 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.498 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:09.498 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:48:09.498 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:09.498 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.498 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.498 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:09.498 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:48:09.499 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1299999999999999e-07 2024-09-04T17:48:09.499 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.499 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.499 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:09.499 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071225+0000", 2024-09-04T17:48:09.499 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6199999999999999e-06 2024-09-04T17:48:09.499 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.499 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.499 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:09.499 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071234+0000", 2024-09-04T17:48:09.499 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8769999999999999e-06 2024-09-04T17:48:09.499 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.500 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.500 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:09.500 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071251+0000", 2024-09-04T17:48:09.500 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.738e-05 2024-09-04T17:48:09.500 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.500 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.500 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:09.500 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218798+0000", 2024-09-04T17:48:09.500 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14754645 2024-09-04T17:48:09.500 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.500 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.500 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:09.500 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218926+0000", 2024-09-04T17:48:09.501 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000128446 2024-09-04T17:48:09.501 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.501 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.501 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:09.501 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219145+0000", 2024-09-04T17:48:09.501 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021832500000000001 2024-09-04T17:48:09.501 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:09.501 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:09.501 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:09.501 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.501 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.501 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8864 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2093056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:09.501 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:48:09.501 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 116.36794236999999, 2024-09-04T17:48:09.502 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148277309, 2024-09-04T17:48:09.502 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:09.502 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:09.502 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:09.502 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:09.502 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:09.502 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:09.502 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8864 2024-09-04T17:48:09.502 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.502 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:09.502 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.502 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:09.502 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:48:09.503 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:09.503 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.503 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.503 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:09.503 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:48:09.503 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:09.503 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.503 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.503 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:09.503 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071264+0000", 2024-09-04T17:48:09.503 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3e-07 2024-09-04T17:48:09.503 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.503 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.504 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:09.504 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071267+0000", 2024-09-04T17:48:09.504 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0139999999999999e-06 2024-09-04T17:48:09.504 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.504 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.504 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:09.504 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071276+0000", 2024-09-04T17:48:09.504 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1689999999999998e-06 2024-09-04T17:48:09.504 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.504 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.504 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:09.504 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071283+0000", 2024-09-04T17:48:09.505 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2269999999999996e-06 2024-09-04T17:48:09.505 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.505 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.505 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:09.505 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219195+0000", 2024-09-04T17:48:09.505 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147911868 2024-09-04T17:48:09.505 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.505 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.505 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:09.505 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219324+0000", 2024-09-04T17:48:09.505 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00012835900000000001 2024-09-04T17:48:09.505 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.506 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.506 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:09.506 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219541+0000", 2024-09-04T17:48:09.506 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000216942 2024-09-04T17:48:09.506 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:09.506 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:09.506 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:09.506 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.506 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.506 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8865 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2109440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:09.506 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:48:09.506 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 116.36791291, 2024-09-04T17:48:09.506 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14865747300000001, 2024-09-04T17:48:09.506 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:09.507 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:09.507 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:09.507 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:09.507 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:09.507 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:09.507 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8865 2024-09-04T17:48:09.507 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.507 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:09.507 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.507 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:09.507 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:48:09.507 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:09.507 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.508 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.508 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:09.508 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:48:09.508 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:09.508 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.508 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.508 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:09.508 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:48:09.508 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7999999999999995e-07 2024-09-04T17:48:09.508 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.508 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.508 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:09.508 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071296+0000", 2024-09-04T17:48:09.508 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6900000000000001e-06 2024-09-04T17:48:09.509 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.509 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.509 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:09.509 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071305+0000", 2024-09-04T17:48:09.509 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.5499999999999995e-06 2024-09-04T17:48:09.509 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.509 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.509 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:09.509 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071312+0000", 2024-09-04T17:48:09.509 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1559999999999998e-06 2024-09-04T17:48:09.509 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.509 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.509 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:09.509 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219591+0000", 2024-09-04T17:48:09.510 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14827944900000001 2024-09-04T17:48:09.510 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.510 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.510 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:09.510 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219731+0000", 2024-09-04T17:48:09.510 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013931600000000001 2024-09-04T17:48:09.510 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.510 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.510 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:09.510 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219950+0000", 2024-09-04T17:48:09.510 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021963199999999999 2024-09-04T17:48:09.510 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:09.510 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:09.510 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:09.511 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.511 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.511 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8866 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2146304~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:09.511 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:48:09.511 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 116.36788467700001, 2024-09-04T17:48:09.511 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14906117899999999, 2024-09-04T17:48:09.511 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:09.511 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:09.511 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:09.511 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:09.511 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:09.511 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:09.511 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8866 2024-09-04T17:48:09.512 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.512 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:09.512 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.512 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:09.512 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:48:09.512 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:09.512 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.512 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.512 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:09.512 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:48:09.512 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:09.512 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.512 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.513 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:09.513 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071322+0000", 2024-09-04T17:48:09.513 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7999999999999995e-07 2024-09-04T17:48:09.513 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.513 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.513 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:09.513 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071338+0000", 2024-09-04T17:48:09.513 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5872999999999999e-05 2024-09-04T17:48:09.513 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.513 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.513 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:09.513 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071347+0000", 2024-09-04T17:48:09.513 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4669999999999993e-06 2024-09-04T17:48:09.514 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.514 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.514 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:09.514 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071354+0000", 2024-09-04T17:48:09.514 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2099999999999996e-06 2024-09-04T17:48:09.514 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.514 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.514 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:09.514 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220008+0000", 2024-09-04T17:48:09.514 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148653642 2024-09-04T17:48:09.514 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.514 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.514 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:09.515 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220155+0000", 2024-09-04T17:48:09.515 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014726600000000001 2024-09-04T17:48:09.515 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.515 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.515 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:09.515 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220382+0000", 2024-09-04T17:48:09.515 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00022704100000000001 2024-09-04T17:48:09.515 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:09.515 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:09.515 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:09.515 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.515 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.515 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8867 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2183168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:09.516 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:48:09.516 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 116.367841324, 2024-09-04T17:48:09.516 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.149445831, 2024-09-04T17:48:09.516 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:09.516 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:09.516 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:09.516 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:09.516 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:09.516 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:09.516 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8867 2024-09-04T17:48:09.516 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.516 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:09.516 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.517 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:09.517 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:48:09.517 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:09.517 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.517 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.517 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:09.517 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:48:09.517 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:09.517 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.517 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.517 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:09.517 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078653+0000", 2024-09-04T17:48:09.517 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0072890430000000003 2024-09-04T17:48:09.518 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.518 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.518 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:09.518 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078664+0000", 2024-09-04T17:48:09.518 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0307e-05 2024-09-04T17:48:09.518 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.518 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.518 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:09.518 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078690+0000", 2024-09-04T17:48:09.518 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6780000000000001e-05 2024-09-04T17:48:09.518 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.518 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.518 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:09.518 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078706+0000", 2024-09-04T17:48:09.519 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5676999999999999e-05 2024-09-04T17:48:09.519 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.519 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.519 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:09.519 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220467+0000", 2024-09-04T17:48:09.519 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14176103500000001 2024-09-04T17:48:09.519 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.519 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.519 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:09.519 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220595+0000", 2024-09-04T17:48:09.519 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000127427 2024-09-04T17:48:09.519 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.519 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.519 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:09.520 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220810+0000", 2024-09-04T17:48:09.520 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021556200000000001 2024-09-04T17:48:09.520 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:09.520 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:09.520 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:09.520 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.520 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.520 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8881 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2695168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:09.520 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:48:09.520 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 116.360019255, 2024-09-04T17:48:09.520 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147543547, 2024-09-04T17:48:09.520 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:09.520 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:09.521 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:09.521 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:09.521 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:09.521 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:09.521 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8881 2024-09-04T17:48:09.521 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.521 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:09.521 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.521 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:09.521 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:48:09.521 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:09.522 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.522 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.522 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:09.522 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:48:09.522 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:09.522 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.522 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.522 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:09.522 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079187+0000", 2024-09-04T17:48:09.522 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7700000000000004e-07 2024-09-04T17:48:09.522 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.522 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.522 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:09.522 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079198+0000", 2024-09-04T17:48:09.523 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0849999999999999e-05 2024-09-04T17:48:09.523 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.523 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.523 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:09.523 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079207+0000", 2024-09-04T17:48:09.523 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5599999999999999e-06 2024-09-04T17:48:09.523 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.523 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.523 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:09.523 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079224+0000", 2024-09-04T17:48:09.523 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6387000000000001e-05 2024-09-04T17:48:09.523 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.523 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.523 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:09.524 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226414+0000", 2024-09-04T17:48:09.524 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14718975000000001 2024-09-04T17:48:09.524 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.524 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.524 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:09.524 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226534+0000", 2024-09-04T17:48:09.524 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000120288 2024-09-04T17:48:09.524 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.524 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.524 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:09.524 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226730+0000", 2024-09-04T17:48:09.524 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000196035 2024-09-04T17:48:09.525 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:09.525 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:09.525 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:09.525 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.525 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.525 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8882 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2732032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:09.525 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:48:09.525 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 116.35997104099999, 2024-09-04T17:48:09.525 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14795022399999999, 2024-09-04T17:48:09.525 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:09.525 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:09.525 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:09.525 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:09.526 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:09.526 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:09.526 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8882 2024-09-04T17:48:09.526 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.526 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:09.526 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.526 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:09.526 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:48:09.526 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:09.526 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.526 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.526 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:09.526 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:48:09.527 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:09.527 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.527 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.527 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:09.527 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086588+0000", 2024-09-04T17:48:09.527 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0073530959999999999 2024-09-04T17:48:09.527 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.527 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.527 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:09.527 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086598+0000", 2024-09-04T17:48:09.527 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0122999999999999e-05 2024-09-04T17:48:09.527 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.527 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.528 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:09.528 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086626+0000", 2024-09-04T17:48:09.528 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.7914000000000001e-05 2024-09-04T17:48:09.528 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.528 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.528 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:09.528 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086641+0000", 2024-09-04T17:48:09.528 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5270000000000001e-05 2024-09-04T17:48:09.528 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.528 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.528 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:09.528 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226801+0000", 2024-09-04T17:48:09.529 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.140160321 2024-09-04T17:48:09.529 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.529 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.529 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:09.529 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226951+0000", 2024-09-04T17:48:09.529 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014965200000000001 2024-09-04T17:48:09.529 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.529 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.529 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:09.529 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.227185+0000", 2024-09-04T17:48:09.529 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00023384799999999999 2024-09-04T17:48:09.529 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:09.529 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:09.530 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:09.530 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.530 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.530 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:93470 3.1f 3:f95bc674:::rbd_header.1134878eadad:head [watch reconnect cookie 140018821420464 gen 16] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-04T17:48:09.530 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:57.837081+0000", 2024-09-04T17:48:09.530 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 11.602125158, 2024-09-04T17:48:09.530 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.072023486, 2024-09-04T17:48:09.530 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:09.530 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:09.530 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:09.530 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:09.530 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:09.530 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:09.530 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 93470 2024-09-04T17:48:09.531 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.531 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:09.531 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.531 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:09.531 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:57.837081+0000", 2024-09-04T17:48:09.531 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:09.531 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.531 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.531 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:09.531 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:57.837081+0000", 2024-09-04T17:48:09.531 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:09.531 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.531 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.531 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:09.532 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:57.837084+0000", 2024-09-04T17:48:09.532 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.8229999999999997e-06 2024-09-04T17:48:09.532 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.532 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.532 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:09.532 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:57.837098+0000", 2024-09-04T17:48:09.532 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3436e-05 2024-09-04T17:48:09.532 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.532 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.532 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:09.532 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:57.837123+0000", 2024-09-04T17:48:09.532 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5079000000000002e-05 2024-09-04T17:48:09.533 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.533 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.533 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:09.533 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:57.837145+0000", 2024-09-04T17:48:09.533 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.228e-05 2024-09-04T17:48:09.533 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.533 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.533 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:09.533 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:57.837360+0000", 2024-09-04T17:48:09.533 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021464100000000001 2024-09-04T17:48:09.533 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.533 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.533 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for peered", 2024-09-04T17:48:09.534 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:57.837410+0000", 2024-09-04T17:48:09.534 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0235000000000003e-05 2024-09-04T17:48:09.534 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.534 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.534 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:09.534 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.900175+0000", 2024-09-04T17:48:09.534 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0627652830000001 2024-09-04T17:48:09.534 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.534 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.534 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for rw locks", 2024-09-04T17:48:09.534 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.900370+0000", 2024-09-04T17:48:09.534 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019497199999999999 2024-09-04T17:48:09.534 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.535 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.535 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:09.535 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.906957+0000", 2024-09-04T17:48:09.535 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0065866889999999997 2024-09-04T17:48:09.535 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.535 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.535 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:09.535 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.907088+0000", 2024-09-04T17:48:09.535 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013056100000000001 2024-09-04T17:48:09.535 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:09.535 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:09.535 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:09.535 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.909104+0000", 2024-09-04T17:48:09.536 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.002016487 2024-09-04T17:48:09.536 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:09.536 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:09.536 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:09.536 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:09.536 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:09.536 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:09.536 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-04T17:48:09.538 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-04T17:48:09.538 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:48:09.536+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:48:09.639 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:09.634+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:09.664 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:09.664 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:48:09.664 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:48:09.664 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:09.678 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-04T17:48:09.693 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:48:09.693 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:48:09.710 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:48:09.710 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:48:09.727 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-04T17:48:09.727 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-04T17:48:09.743 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-04T17:48:09.743 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-04T17:48:09.761 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:48:09.761 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:48:09.773 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:48:09.770+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:48:09.777 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:48:09.777 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:48:09.796 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-04T17:48:09.796 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-04T17:48:09.817 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:48:09.817 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:48:09.831 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.6 to 1 (10000) 2024-09-04T17:48:09.840 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:09.836+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:09.858 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-04T17:48:09.861 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:09.861 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:09.877 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-04T17:48:09.883 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-04T17:48:09.969 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:09.966+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:10.042 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:10.040+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:10.052 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:10.052 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:10.067 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-04T17:48:10.144 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:48:10.140+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:48:10.194 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:10.195 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:10.210 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-04T17:48:10.257 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:10.256+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:10.345 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:48:10.342+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:48:10.372 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:10.372 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:10.386 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-04T17:48:10.445 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:48:10.442+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:48:10.509 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:10.509 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:10.523 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-04T17:48:10.546 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:48:10.542+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:48:10.645 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:10.645 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:10.648 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:10.642+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:10.659 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-04T17:48:10.748 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:10.742+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:10.775 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:10.775 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:10.791 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-04T17:48:10.833 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is set 2024-09-04T17:48:10.833 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.7 to 1 (10000) 2024-09-04T17:48:10.848 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:48:10.844+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:48:10.864 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-04T17:48:10.864 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-04T17:48:10.917 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:10.917 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:10.931 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-04T17:48:10.977 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:10.976+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:11.049 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:11.046+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:11.073 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:11.073 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:11.087 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-04T17:48:11.151 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:48:11.148+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:48:11.183 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:48:11.184 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-09-04T17:48:11.200 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":200,"stamp":"2024-09-04T17:48:11.110187+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1466761465,"num_objects":1926,"num_object_clones":265,"num_object_copies":3852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":118,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1926,"num_whiteouts":1030,"num_read":117345,"num_read_kb":797933,"num_write":16923,"num_write_kb":375963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2371,"num_bytes_recovered":131763676,"num_keys_recovered":419,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14157,"ondisk_log_size":14157,"up":82,"acting":80,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":81,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":722356,"kb_used_data":461204,"kb_used_omap":40,"kb_used_meta":256791,"kb_avail":565508684,"statfs":{"total":579820584960,"available":579080892416,"internally_reserved":0,"allocated":472272896,"data_stored":571044003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41921,"internal_metadata":262954047},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-79098444,"num_objects":-33,"num_object_clones":-13,"num_object_copies":-66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":118,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-33,"num_whiteouts":-8,"num_read":278,"num_read_kb":237,"num_write":41,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":6771070,"num_keys_recovered":21,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000910"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":1,"reported_epoch":126,"state":"unknown","last_fresh":"2024-09-04T17:48:05.889951+0000","last_change":"2024-09-04T17:48:06.154637+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-04T17:48:05.889951+0000","last_undegraded":"2024-09-04T17:48:05.889951+0000","last_fullsized":"2024-09-04T17:48:05.889951+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"123'580","reported_seq":3319,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.843555+0000","last_change":"2024-09-04T17:48:09.855336+0000","last_active":"2024-09-04T17:48:10.843555+0000","last_peered":"2024-09-04T17:48:10.843555+0000","last_clean":"2024-09-04T17:48:10.843555+0000","last_became_active":"2024-09-04T17:48:00.841031+0000","last_became_peered":"2024-09-04T17:48:00.841031+0000","last_unstale":"2024-09-04T17:48:10.843555+0000","last_undegraded":"2024-09-04T17:48:10.843555+0000","last_fullsized":"2024-09-04T17:48:10.843555+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T19:09:57.450916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001300933,"stat_sum":{"num_bytes":56202357,"num_objects":54,"num_object_clones":6,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":26,"num_read":4487,"num_read_kb":30277,"num_write":751,"num_write_kb":10557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":10538943,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.1d","version":"126'500","reported_seq":2156,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-04T17:48:06.105094+0000","last_change":"2024-09-04T17:48:05.895726+0000","last_active":"2024-09-04T17:48:06.105094+0000","last_peered":"2024-09-04T17:48:06.105094+0000","last_clean":"2024-09-04T17:48:06.105094+0000","last_became_active":"2024-09-04T17:45:55.955213+0000","last_became_peered":"2024-09-04T17:45:55.955213+0000","last_unstale":"2024-09-04T17:48:06.105094+0000","last_undegraded":"2024-09-04T17:48:06.105094+0000","last_fullsized":"2024-09-04T17:48:06.105094+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:35:41.643022+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0097749550000000001,"stat_sum":{"num_bytes":40967343,"num_objects":63,"num_object_clones":7,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":33,"num_read":2785,"num_read_kb":20258,"num_write":587,"num_write_kb":9396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2b","length":"3"}]},{"pgid":"3.1c","version":"126'640","reported_seq":6088,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-04T17:48:06.065129+0000","last_change":"2024-09-04T17:48:05.890887+0000","last_active":"2024-09-04T17:48:06.065129+0000","last_peered":"2024-09-04T17:48:06.065129+0000","last_clean":"2024-09-04T17:48:06.065129+0000","last_became_active":"2024-09-04T17:45:55.951880+0000","last_became_peered":"2024-09-04T17:45:55.951880+0000","last_unstale":"2024-09-04T17:48:06.065129+0000","last_undegraded":"2024-09-04T17:48:06.065129+0000","last_fullsized":"2024-09-04T17:48:06.065129+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":640,"log_dups_size":0,"ondisk_log_size":640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T19:09:09.894385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006468606,"stat_sum":{"num_bytes":92513810,"num_objects":79,"num_object_clones":11,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":43,"num_read":6939,"num_read_kb":47530,"num_write":741,"num_write_kb":11545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2b","length":"3"}]},{"pgid":"3.1b","version":"0'0","reported_seq":1,"reported_epoch":126,"state":"unknown","last_fresh":"2024-09-04T17:48:05.882902+0000","last_change":"2024-09-04T17:48:06.154609+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-04T17:48:05.882902+0000","last_undegraded":"2024-09-04T17:48:05.882902+0000","last_fullsized":"2024-09-04T17:48:05.882902+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"130'362","reported_seq":5739,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-04T17:48:09.894153+0000","last_change":"2024-09-04T17:48:09.853880+0000","last_active":"2024-09-04T17:48:09.894153+0000","last_peered":"2024-09-04T17:48:09.894153+0000","last_clean":"2024-09-04T17:48:09.894153+0000","last_became_active":"2024-09-04T17:47:44.946366+0000","last_became_peered":"2024-09-04T17:47:44.946366+0000","last_unstale":"2024-09-04T17:48:09.894153+0000","last_undegraded":"2024-09-04T17:48:09.894153+0000","last_fullsized":"2024-09-04T17:48:09.894153+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":362,"log_dups_size":0,"ondisk_log_size":362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:38:44.964441+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.01933491,"stat_sum":{"num_bytes":72634271,"num_objects":65,"num_object_clones":8,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":31,"num_read":5538,"num_read_kb":45426,"num_write":335,"num_write_kb":15441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":5992020,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.19","version":"131'361","reported_seq":2126,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.866058+0000","last_change":"2024-09-04T17:48:10.866726+0000","last_active":"2024-09-04T17:48:10.866058+0000","last_peered":"2024-09-04T17:48:10.866058+0000","last_clean":"2024-09-04T17:48:10.866058+0000","last_became_active":"2024-09-04T17:48:10.851145+0000","last_became_peered":"2024-09-04T17:48:10.851145+0000","last_unstale":"2024-09-04T17:48:10.866058+0000","last_undegraded":"2024-09-04T17:48:10.866058+0000","last_fullsized":"2024-09-04T17:48:10.866058+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":361,"log_dups_size":0,"ondisk_log_size":361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:29:15.116357+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0093494070000000006,"stat_sum":{"num_bytes":36387745,"num_objects":54,"num_object_clones":7,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":24,"num_read":2015,"num_read_kb":16004,"num_write":395,"num_write_kb":4601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":4015757,"num_keys_recovered":27,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.18","version":"0'0","reported_seq":1,"reported_epoch":126,"state":"unknown","last_fresh":"2024-09-04T17:48:05.882697+0000","last_change":"2024-09-04T17:48:06.154582+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-04T17:48:05.882697+0000","last_undegraded":"2024-09-04T17:48:05.882697+0000","last_fullsized":"2024-09-04T17:48:05.882697+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"125'283","reported_seq":1996,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-04T17:48:05.886189+0000","last_change":"2024-09-04T17:48:05.886597+0000","last_active":"2024-09-04T17:48:05.886189+0000","last_peered":"2024-09-04T17:48:05.886189+0000","last_clean":"2024-09-04T17:48:05.886189+0000","last_became_active":"2024-09-04T17:45:55.958625+0000","last_became_peered":"2024-09-04T17:45:55.958625+0000","last_unstale":"2024-09-04T17:48:05.886189+0000","last_undegraded":"2024-09-04T17:48:05.886189+0000","last_fullsized":"2024-09-04T17:48:05.886189+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:42:21.486986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019604779999999999,"stat_sum":{"num_bytes":82172026,"num_objects":63,"num_object_clones":12,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":27,"num_read":1606,"num_read_kb":20936,"num_write":292,"num_write_kb":18406,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2b","length":"3"}]},{"pgid":"3.16","version":"126'324","reported_seq":1662,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-04T17:48:05.994313+0000","last_change":"2024-09-04T17:48:05.889294+0000","last_active":"2024-09-04T17:48:05.994313+0000","last_peered":"2024-09-04T17:48:05.994313+0000","last_clean":"2024-09-04T17:48:05.994313+0000","last_became_active":"2024-09-04T17:45:55.953254+0000","last_became_peered":"2024-09-04T17:45:55.953254+0000","last_unstale":"2024-09-04T17:48:05.994313+0000","last_undegraded":"2024-09-04T17:48:05.994313+0000","last_fullsized":"2024-09-04T17:48:05.994313+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T22:42:49.754648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0042812759999999997,"stat_sum":{"num_bytes":37296010,"num_objects":51,"num_object_clones":7,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":29,"num_read":1692,"num_read_kb":10758,"num_write":369,"num_write_kb":7387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2b","length":"3"}]},{"pgid":"3.15","version":"125'455","reported_seq":1982,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-04T17:48:05.886108+0000","last_change":"2024-09-04T17:48:05.886521+0000","last_active":"2024-09-04T17:48:05.886108+0000","last_peered":"2024-09-04T17:48:05.886108+0000","last_clean":"2024-09-04T17:48:05.886108+0000","last_became_active":"2024-09-04T17:47:08.657112+0000","last_became_peered":"2024-09-04T17:47:08.657112+0000","last_unstale":"2024-09-04T17:48:05.886108+0000","last_undegraded":"2024-09-04T17:48:05.886108+0000","last_fullsized":"2024-09-04T17:48:05.886108+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:28:53.473428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021417509999999999,"stat_sum":{"num_bytes":17977259,"num_objects":46,"num_object_clones":4,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":27,"num_read":3707,"num_read_kb":16708,"num_write":482,"num_write_kb":4429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":7294656,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2b","length":"3"}]},{"pgid":"3.14","version":"125'691","reported_seq":2049,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-04T17:48:09.860236+0000","last_change":"2024-09-04T17:48:09.860717+0000","last_active":"2024-09-04T17:48:09.860236+0000","last_peered":"2024-09-04T17:48:09.860236+0000","last_clean":"2024-09-04T17:48:09.860236+0000","last_became_active":"2024-09-04T17:46:53.539886+0000","last_became_peered":"2024-09-04T17:46:53.539886+0000","last_unstale":"2024-09-04T17:48:09.860236+0000","last_undegraded":"2024-09-04T17:48:09.860236+0000","last_fullsized":"2024-09-04T17:48:09.860236+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":691,"log_dups_size":0,"ondisk_log_size":691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:39:44.106583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0051295710000000003,"stat_sum":{"num_bytes":54551302,"num_objects":66,"num_object_clones":10,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":35,"num_read":3772,"num_read_kb":29665,"num_write":939,"num_write_kb":35894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"2.7","version":"0'0","reported_seq":95,"reported_epoch":130,"state":"peering","last_fresh":"2024-09-04T17:48:09.858550+0000","last_change":"2024-09-04T17:48:10.840997+0000","last_active":"2024-09-04T17:48:10.840997+0000","last_peered":"2024-09-04T17:48:09.858550+0000","last_clean":"2024-09-04T17:48:09.858550+0000","last_became_active":"2024-09-04T17:47:57.840926+0000","last_became_peered":"2024-09-04T17:47:57.840926+0000","last_unstale":"2024-09-04T17:48:09.858550+0000","last_undegraded":"2024-09-04T17:48:09.858550+0000","last_fullsized":"2024-09-04T17:48:09.858550+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001226074,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"126'319","reported_seq":2771,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-04T17:48:06.021270+0000","last_change":"2024-09-04T17:48:05.886025+0000","last_active":"2024-09-04T17:48:06.021270+0000","last_peered":"2024-09-04T17:48:06.021270+0000","last_clean":"2024-09-04T17:48:06.021270+0000","last_became_active":"2024-09-04T17:47:32.848310+0000","last_became_peered":"2024-09-04T17:47:32.848310+0000","last_unstale":"2024-09-04T17:48:06.021270+0000","last_undegraded":"2024-09-04T17:48:06.021270+0000","last_fullsized":"2024-09-04T17:48:06.021270+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:15:06.255989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001696892,"stat_sum":{"num_bytes":40074006,"num_objects":54,"num_object_clones":4,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":28,"num_read":2418,"num_read_kb":14504,"num_write":368,"num_write_kb":5524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":919873,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2b","length":"3"}]},{"pgid":"2.6","version":"0'0","reported_seq":37,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.843292+0000","last_change":"2024-09-04T17:47:58.862844+0000","last_active":"2024-09-04T17:48:10.843292+0000","last_peered":"2024-09-04T17:48:10.843292+0000","last_clean":"2024-09-04T17:48:10.843292+0000","last_became_active":"2024-09-04T17:47:58.859941+0000","last_became_peered":"2024-09-04T17:47:58.859941+0000","last_unstale":"2024-09-04T17:48:10.843292+0000","last_undegraded":"2024-09-04T17:48:10.843292+0000","last_fullsized":"2024-09-04T17:48:10.843292+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:08:53.409036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011364179999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"129'412","reported_seq":2639,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-04T17:48:09.871164+0000","last_change":"2024-09-04T17:48:09.871644+0000","last_active":"2024-09-04T17:48:09.871164+0000","last_peered":"2024-09-04T17:48:09.871164+0000","last_clean":"2024-09-04T17:48:09.871164+0000","last_became_active":"2024-09-04T17:47:08.653159+0000","last_became_peered":"2024-09-04T17:47:08.653159+0000","last_unstale":"2024-09-04T17:48:09.871164+0000","last_undegraded":"2024-09-04T17:48:09.871164+0000","last_fullsized":"2024-09-04T17:48:09.871164+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":412,"log_dups_size":0,"ondisk_log_size":412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T21:59:05.657735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01497677,"stat_sum":{"num_bytes":43421480,"num_objects":63,"num_object_clones":8,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":36,"num_read":2882,"num_read_kb":23337,"num_write":531,"num_write_kb":21560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":952382,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"2.5","version":"0'0","reported_seq":236,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-04T17:48:09.857533+0000","last_change":"2024-09-04T17:45:49.476607+0000","last_active":"2024-09-04T17:48:09.857533+0000","last_peered":"2024-09-04T17:48:09.857533+0000","last_clean":"2024-09-04T17:48:09.857533+0000","last_became_active":"2024-09-04T17:45:47.369308+0000","last_became_peered":"2024-09-04T17:45:47.369308+0000","last_unstale":"2024-09-04T17:48:09.857533+0000","last_undegraded":"2024-09-04T17:48:09.857533+0000","last_fullsized":"2024-09-04T17:48:09.857533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:17:13.378553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001029826,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"129'355","reported_seq":2393,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-04T17:48:09.094306+0000","last_change":"2024-09-04T17:48:09.940167+0000","last_active":"2024-09-04T17:48:09.940167+0000","last_peered":"2024-09-04T17:48:09.094306+0000","last_clean":"2024-09-04T17:48:05.885073+0000","last_became_active":"2024-09-04T17:48:08.854562+0000","last_became_peered":"2024-09-04T17:48:08.854562+0000","last_unstale":"2024-09-04T17:48:09.094306+0000","last_undegraded":"2024-09-04T17:48:09.094306+0000","last_fullsized":"2024-09-04T17:48:09.094306+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":355,"log_dups_size":0,"ondisk_log_size":355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.019597868000000001,"stat_sum":{"num_bytes":39637658,"num_objects":63,"num_object_clones":5,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":35,"num_read":2424,"num_read_kb":18572,"num_write":397,"num_write_kb":9850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":237,"num_bytes_recovered":13375622,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2b","length":"3"}]},{"pgid":"2.4","version":"0'0","reported_seq":72,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-04T17:48:05.884096+0000","last_change":"2024-09-04T17:47:29.803682+0000","last_active":"2024-09-04T17:48:05.884096+0000","last_peered":"2024-09-04T17:48:05.884096+0000","last_clean":"2024-09-04T17:48:05.884096+0000","last_became_active":"2024-09-04T17:47:29.803256+0000","last_became_peered":"2024-09-04T17:47:29.803256+0000","last_unstale":"2024-09-04T17:48:05.884096+0000","last_undegraded":"2024-09-04T17:48:05.884096+0000","last_fullsized":"2024-09-04T17:48:05.884096+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:32:26.523162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"125'273","reported_seq":1783,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-04T17:48:05.891886+0000","last_change":"2024-09-04T17:48:05.891886+0000","last_active":"2024-09-04T17:48:05.891886+0000","last_peered":"2024-09-04T17:48:05.891886+0000","last_clean":"2024-09-04T17:48:05.891886+0000","last_became_active":"2024-09-04T17:47:41.918947+0000","last_became_peered":"2024-09-04T17:47:41.918947+0000","last_unstale":"2024-09-04T17:48:05.891886+0000","last_undegraded":"2024-09-04T17:48:05.891886+0000","last_fullsized":"2024-09-04T17:48:05.891886+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":273,"log_dups_size":0,"ondisk_log_size":273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T19:53:30.404276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0060645610000000004,"stat_sum":{"num_bytes":26359679,"num_objects":68,"num_object_clones":10,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":36,"num_read":2054,"num_read_kb":7214,"num_write":251,"num_write_kb":6999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":10024551,"num_keys_recovered":46,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2b","length":"3"}]},{"pgid":"3.3","version":"126'634","reported_seq":2745,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-04T17:48:06.103617+0000","last_change":"2024-09-04T17:48:05.885357+0000","last_active":"2024-09-04T17:48:06.103617+0000","last_peered":"2024-09-04T17:48:06.103617+0000","last_clean":"2024-09-04T17:48:06.103617+0000","last_became_active":"2024-09-04T17:46:53.542889+0000","last_became_peered":"2024-09-04T17:46:53.542889+0000","last_unstale":"2024-09-04T17:48:06.103617+0000","last_undegraded":"2024-09-04T17:48:06.103617+0000","last_fullsized":"2024-09-04T17:48:06.103617+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":634,"log_dups_size":0,"ondisk_log_size":634,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:08:46.739406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00134242,"stat_sum":{"num_bytes":66410315,"num_objects":71,"num_object_clones":12,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":38,"num_read":4231,"num_read_kb":26156,"num_write":871,"num_write_kb":14086,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2b","length":"3"}]},{"pgid":"2.2","version":"22'2","reported_seq":245,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-04T17:48:05.884649+0000","last_change":"2024-09-04T17:47:59.598241+0000","last_active":"2024-09-04T17:48:05.884649+0000","last_peered":"2024-09-04T17:48:05.884649+0000","last_clean":"2024-09-04T17:48:05.884649+0000","last_became_active":"2024-09-04T17:47:58.884634+0000","last_became_peered":"2024-09-04T17:47:58.884634+0000","last_unstale":"2024-09-04T17:48:05.884649+0000","last_undegraded":"2024-09-04T17:48:05.884649+0000","last_fullsized":"2024-09-04T17:48:05.884649+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T22:02:31.692385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026316069999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"129'373","reported_seq":2865,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-04T17:48:09.835070+0000","last_change":"2024-09-04T17:48:09.835714+0000","last_active":"2024-09-04T17:48:09.835070+0000","last_peered":"2024-09-04T17:48:09.835070+0000","last_clean":"2024-09-04T17:48:09.835070+0000","last_became_active":"2024-09-04T17:48:01.849363+0000","last_became_peered":"2024-09-04T17:48:01.849363+0000","last_unstale":"2024-09-04T17:48:09.835070+0000","last_undegraded":"2024-09-04T17:48:09.835070+0000","last_fullsized":"2024-09-04T17:48:09.835070+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":373,"log_dups_size":0,"ondisk_log_size":373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T22:04:09.650735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013141769999999999,"stat_sum":{"num_bytes":42792223,"num_objects":71,"num_object_clones":9,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":42,"num_read":3143,"num_read_kb":22159,"num_write":313,"num_write_kb":4881,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":5692590,"num_keys_recovered":35,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"2.1","version":"0'0","reported_seq":216,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-04T17:48:09.834804+0000","last_change":"2024-09-04T17:47:41.913896+0000","last_active":"2024-09-04T17:48:09.834804+0000","last_peered":"2024-09-04T17:48:09.834804+0000","last_clean":"2024-09-04T17:48:09.834804+0000","last_became_active":"2024-09-04T17:47:41.913057+0000","last_became_peered":"2024-09-04T17:47:41.913057+0000","last_unstale":"2024-09-04T17:48:09.834804+0000","last_undegraded":"2024-09-04T17:48:09.834804+0000","last_fullsized":"2024-09-04T17:48:09.834804+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:49:25.382532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001025195,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"126'527","reported_seq":2536,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-04T17:48:05.885237+0000","last_change":"2024-09-04T17:48:05.885237+0000","last_active":"2024-09-04T17:48:05.885237+0000","last_peered":"2024-09-04T17:48:05.885237+0000","last_clean":"2024-09-04T17:48:05.885237+0000","last_became_active":"2024-09-04T17:47:08.655358+0000","last_became_peered":"2024-09-04T17:47:08.655358+0000","last_unstale":"2024-09-04T17:48:05.885237+0000","last_undegraded":"2024-09-04T17:48:05.885237+0000","last_fullsized":"2024-09-04T17:48:05.885237+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T22:56:57.415892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001259982,"stat_sum":{"num_bytes":87377265,"num_objects":81,"num_object_clones":18,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":41,"num_read":2920,"num_read_kb":40212,"num_write":602,"num_write_kb":23345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":112,"num_bytes_recovered":8952402,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2b","length":"3"}]},{"pgid":"2.0","version":"0'0","reported_seq":242,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-04T17:48:09.854561+0000","last_change":"2024-09-04T17:47:29.804429+0000","last_active":"2024-09-04T17:48:09.854561+0000","last_peered":"2024-09-04T17:48:09.854561+0000","last_clean":"2024-09-04T17:48:09.854561+0000","last_became_active":"2024-09-04T17:47:29.803992+0000","last_became_peered":"2024-09-04T17:47:29.803992+0000","last_unstale":"2024-09-04T17:48:09.854561+0000","last_undegraded":"2024-09-04T17:48:09.854561+0000","last_fullsized":"2024-09-04T17:48:09.854561+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:50:32.563345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010553590000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"126'525","reported_seq":2225,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-04T17:48:06.054465+0000","last_change":"2024-09-04T17:48:05.886573+0000","last_active":"2024-09-04T17:48:06.054465+0000","last_peered":"2024-09-04T17:48:06.054465+0000","last_clean":"2024-09-04T17:48:06.054465+0000","last_became_active":"2024-09-04T17:48:01.850400+0000","last_became_peered":"2024-09-04T17:48:01.850400+0000","last_unstale":"2024-09-04T17:48:06.054465+0000","last_undegraded":"2024-09-04T17:48:06.054465+0000","last_fullsized":"2024-09-04T17:48:06.054465+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T22:55:10.338360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024575920000000002,"stat_sum":{"num_bytes":50198348,"num_objects":64,"num_object_clones":9,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":33,"num_read":5199,"num_read_kb":29723,"num_write":673,"num_write_kb":13801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":13799490,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2b","length":"3"}]},{"pgid":"2.3","version":"20'1","reported_seq":239,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-04T17:48:05.883926+0000","last_change":"2024-09-04T17:47:41.919544+0000","last_active":"2024-09-04T17:48:05.883926+0000","last_peered":"2024-09-04T17:48:05.883926+0000","last_clean":"2024-09-04T17:48:05.883926+0000","last_became_active":"2024-09-04T17:47:41.919027+0000","last_became_peered":"2024-09-04T17:47:41.919027+0000","last_unstale":"2024-09-04T17:48:05.883926+0000","last_undegraded":"2024-09-04T17:48:05.883926+0000","last_fullsized":"2024-09-04T17:48:05.883926+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:42:55.688562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086618299999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":327,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-04T17:48:09.855144+0000","last_change":"2024-09-04T17:45:42.341149+0000","last_active":"2024-09-04T17:48:09.855144+0000","last_peered":"2024-09-04T17:48:09.855144+0000","last_clean":"2024-09-04T17:48:09.855144+0000","last_became_active":"2024-09-04T17:45:42.339286+0000","last_became_peered":"2024-09-04T17:45:42.339286+0000","last_unstale":"2024-09-04T17:48:09.855144+0000","last_undegraded":"2024-09-04T17:48:09.855144+0000","last_fullsized":"2024-09-04T17:48:09.855144+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:41.305128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:41.305128+0000","last_clean_scrub_stamp":"2024-09-04T17:45:41.305128+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:52:24.860049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"126'483","reported_seq":2668,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-04T17:48:09.855504+0000","last_change":"2024-09-04T17:48:09.857214+0000","last_active":"2024-09-04T17:48:09.855504+0000","last_peered":"2024-09-04T17:48:09.855504+0000","last_clean":"2024-09-04T17:48:09.855504+0000","last_became_active":"2024-09-04T17:47:32.847682+0000","last_became_peered":"2024-09-04T17:47:32.847682+0000","last_unstale":"2024-09-04T17:48:09.855504+0000","last_undegraded":"2024-09-04T17:48:09.855504+0000","last_fullsized":"2024-09-04T17:48:09.855504+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":483,"log_dups_size":0,"ondisk_log_size":483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:07:21.110914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002703474,"stat_sum":{"num_bytes":45080057,"num_objects":72,"num_object_clones":11,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":40,"num_read":2705,"num_read_kb":17207,"num_write":531,"num_write_kb":29364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":14692620,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.9","version":"129'591","reported_seq":4115,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-04T17:48:09.835133+0000","last_change":"2024-09-04T17:48:09.836181+0000","last_active":"2024-09-04T17:48:09.835133+0000","last_peered":"2024-09-04T17:48:09.835133+0000","last_clean":"2024-09-04T17:48:09.835133+0000","last_became_active":"2024-09-04T17:47:44.949630+0000","last_became_peered":"2024-09-04T17:47:44.949630+0000","last_unstale":"2024-09-04T17:48:09.835133+0000","last_undegraded":"2024-09-04T17:48:09.835133+0000","last_fullsized":"2024-09-04T17:48:09.835133+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:20:22.482600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001705781,"stat_sum":{"num_bytes":55724561,"num_objects":78,"num_object_clones":14,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":42,"num_read":5692,"num_read_kb":37073,"num_write":680,"num_write_kb":18627,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":4534479,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.a","version":"131'533","reported_seq":3019,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.890923+0000","last_change":"2024-09-04T17:48:09.853469+0000","last_active":"2024-09-04T17:48:10.890923+0000","last_peered":"2024-09-04T17:48:10.890923+0000","last_clean":"2024-09-04T17:48:10.890923+0000","last_became_active":"2024-09-04T17:48:00.841070+0000","last_became_peered":"2024-09-04T17:48:00.841070+0000","last_unstale":"2024-09-04T17:48:10.890923+0000","last_undegraded":"2024-09-04T17:48:10.890923+0000","last_fullsized":"2024-09-04T17:48:10.890923+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:47:07.864899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038466329999999999,"stat_sum":{"num_bytes":36655948,"num_objects":72,"num_object_clones":11,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":40,"num_read":5202,"num_read_kb":16044,"num_write":529,"num_write_kb":2741,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":1572433,"num_keys_recovered":66,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.b","version":"125'677","reported_seq":2186,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-04T17:48:05.885296+0000","last_change":"2024-09-04T17:48:05.886018+0000","last_active":"2024-09-04T17:48:05.885296+0000","last_peered":"2024-09-04T17:48:05.885296+0000","last_clean":"2024-09-04T17:48:05.885296+0000","last_became_active":"2024-09-04T17:47:08.657286+0000","last_became_peered":"2024-09-04T17:47:08.657286+0000","last_unstale":"2024-09-04T17:48:05.885296+0000","last_undegraded":"2024-09-04T17:48:05.885296+0000","last_fullsized":"2024-09-04T17:48:05.885296+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":677,"log_dups_size":0,"ondisk_log_size":677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:39:42.239271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018167669999999999,"stat_sum":{"num_bytes":31620378,"num_objects":71,"num_object_clones":10,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":33,"num_read":3911,"num_read_kb":14517,"num_write":845,"num_write_kb":7911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":3759450,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2b","length":"3"}]},{"pgid":"3.c","version":"125'550","reported_seq":7774,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-04T17:48:05.888379+0000","last_change":"2024-09-04T17:48:05.888779+0000","last_active":"2024-09-04T17:48:05.888379+0000","last_peered":"2024-09-04T17:48:05.888379+0000","last_clean":"2024-09-04T17:48:05.888379+0000","last_became_active":"2024-09-04T17:48:02.855127+0000","last_became_peered":"2024-09-04T17:48:02.855127+0000","last_unstale":"2024-09-04T17:48:05.888379+0000","last_undegraded":"2024-09-04T17:48:05.888379+0000","last_fullsized":"2024-09-04T17:48:05.888379+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:40:26.367843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004127805,"stat_sum":{"num_bytes":66893850,"num_objects":76,"num_object_clones":11,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":43,"num_read":8646,"num_read_kb":71089,"num_write":688,"num_write_kb":16847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":153,"num_bytes_recovered":7666524,"num_keys_recovered":34,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2b","length":"3"}]},{"pgid":"3.d","version":"126'1282","reported_seq":3848,"reported_epoch":130,"state":"remapped+peering","last_fresh":"2024-09-04T17:48:09.855528+0000","last_change":"2024-09-04T17:48:09.855528+0000","last_active":"2024-09-04T17:48:08.833037+0000","last_peered":"2024-09-04T17:48:07.832005+0000","last_clean":"2024-09-04T17:48:07.832005+0000","last_became_active":"2024-09-04T17:48:00.840986+0000","last_became_peered":"2024-09-04T17:48:00.840986+0000","last_unstale":"2024-09-04T17:48:09.855528+0000","last_undegraded":"2024-09-04T17:48:09.855528+0000","last_fullsized":"2024-09-04T17:48:09.855528+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":1282,"log_dups_size":0,"ondisk_log_size":1282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018889099999999999,"stat_sum":{"num_bytes":49889564,"num_objects":60,"num_object_clones":6,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":34,"num_read":8526,"num_read_kb":26249,"num_write":2192,"num_write_kb":13182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":8914239,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2a","length":"4"}]},{"pgid":"3.e","version":"127'440","reported_seq":5206,"reported_epoch":130,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-04T17:48:09.886153+0000","last_change":"2024-09-04T17:48:09.886153+0000","last_active":"2024-09-04T17:48:09.886153+0000","last_peered":"2024-09-04T17:48:09.886153+0000","last_clean":"2024-09-04T17:48:06.890201+0000","last_became_active":"2024-09-04T17:48:09.884964+0000","last_became_peered":"2024-09-04T17:48:09.884964+0000","last_unstale":"2024-09-04T17:48:09.886153+0000","last_undegraded":"2024-09-04T17:48:09.857903+0000","last_fullsized":"2024-09-04T17:48:09.854505+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.010017414000000001,"stat_sum":{"num_bytes":63389770,"num_objects":66,"num_object_clones":13,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":118,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":32,"num_read":6129,"num_read_kb":28040,"num_write":457,"num_write_kb":11211,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":3926000,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7],"avail_no_missing":["7","0"],"object_location_counts":[{"shards":"0,7","objects":66}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2a","length":"4"}]},{"pgid":"3.f","version":"130'292","reported_seq":2844,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-04T17:48:09.870980+0000","last_change":"2024-09-04T17:48:09.870980+0000","last_active":"2024-09-04T17:48:09.870980+0000","last_peered":"2024-09-04T17:48:09.870980+0000","last_clean":"2024-09-04T17:48:09.870980+0000","last_became_active":"2024-09-04T17:45:55.956734+0000","last_became_peered":"2024-09-04T17:45:55.956734+0000","last_unstale":"2024-09-04T17:48:09.870980+0000","last_undegraded":"2024-09-04T17:48:09.870980+0000","last_fullsized":"2024-09-04T17:48:09.870980+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":292,"log_dups_size":0,"ondisk_log_size":292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T23:59:11.397371+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.01496131,"stat_sum":{"num_bytes":49550709,"num_objects":69,"num_object_clones":12,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":36,"num_read":2563,"num_read_kb":44941,"num_write":297,"num_write_kb":13423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.10","version":"126'514","reported_seq":3148,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.842877+0000","last_change":"2024-09-04T17:48:09.851750+0000","last_active":"2024-09-04T17:48:10.842877+0000","last_peered":"2024-09-04T17:48:10.842877+0000","last_clean":"2024-09-04T17:48:10.842877+0000","last_became_active":"2024-09-04T17:48:00.843136+0000","last_became_peered":"2024-09-04T17:48:00.843136+0000","last_unstale":"2024-09-04T17:48:10.842877+0000","last_undegraded":"2024-09-04T17:48:10.842877+0000","last_fullsized":"2024-09-04T17:48:10.842877+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":514,"log_dups_size":0,"ondisk_log_size":514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T23:07:02.271876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049468649999999999,"stat_sum":{"num_bytes":51575959,"num_objects":78,"num_object_clones":10,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":46,"num_read":4069,"num_read_kb":18348,"num_write":591,"num_write_kb":12832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":3377599,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.11","version":"130'500","reported_seq":5739,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-04T17:48:09.877129+0000","last_change":"2024-09-04T17:48:09.879368+0000","last_active":"2024-09-04T17:48:09.877129+0000","last_peered":"2024-09-04T17:48:09.877129+0000","last_clean":"2024-09-04T17:48:09.877129+0000","last_became_active":"2024-09-04T17:45:55.951870+0000","last_became_peered":"2024-09-04T17:45:55.951870+0000","last_unstale":"2024-09-04T17:48:09.877129+0000","last_undegraded":"2024-09-04T17:48:09.877129+0000","last_fullsized":"2024-09-04T17:48:09.877129+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T22:56:47.076439+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.024746269000000001,"stat_sum":{"num_bytes":74079184,"num_objects":82,"num_object_clones":10,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":46,"num_read":6681,"num_read_kb":61026,"num_write":514,"num_write_kb":17560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.12","version":"126'342","reported_seq":3401,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-04T17:48:05.884718+0000","last_change":"2024-09-04T17:48:05.884718+0000","last_active":"2024-09-04T17:48:05.884718+0000","last_peered":"2024-09-04T17:48:05.884718+0000","last_clean":"2024-09-04T17:48:05.884718+0000","last_became_active":"2024-09-04T17:47:44.952109+0000","last_became_peered":"2024-09-04T17:47:44.952109+0000","last_unstale":"2024-09-04T17:48:05.884718+0000","last_undegraded":"2024-09-04T17:48:05.884718+0000","last_fullsized":"2024-09-04T17:48:05.884718+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T22:00:53.272098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097872999999999992,"stat_sum":{"num_bytes":29554039,"num_objects":60,"num_object_clones":5,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":35,"num_read":3162,"num_read_kb":21367,"num_write":383,"num_write_kb":8559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":1762027,"num_keys_recovered":48,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2b","length":"3"}]},{"pgid":"3.13","version":"126'264","reported_seq":2227,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-04T17:48:09.857330+0000","last_change":"2024-09-04T17:48:09.860061+0000","last_active":"2024-09-04T17:48:09.857330+0000","last_peered":"2024-09-04T17:48:09.857330+0000","last_clean":"2024-09-04T17:48:09.857330+0000","last_became_active":"2024-09-04T17:45:55.954544+0000","last_became_peered":"2024-09-04T17:45:55.954544+0000","last_unstale":"2024-09-04T17:48:09.857330+0000","last_undegraded":"2024-09-04T17:48:09.857330+0000","last_fullsized":"2024-09-04T17:48:09.857330+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":264,"log_dups_size":0,"ondisk_log_size":264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T23:14:42.541437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0051521240000000001,"stat_sum":{"num_bytes":25183962,"num_objects":62,"num_object_clones":5,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":39,"num_read":2181,"num_read_kb":22538,"num_write":216,"num_write_kb":4374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"29","length":"5"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1466171078,"num_objects":1922,"num_object_clones":265,"num_object_copies":3844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":118,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1922,"num_whiteouts":1030,"num_read":117279,"num_read_kb":797878,"num_write":16820,"num_write_kb":374333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2370,"num_bytes_recovered":131763657,"num_keys_recovered":419,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":463052800,"data_stored":566059244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21395,"internal_metadata":0},"log_size":14082,"ondisk_log_size":14082,"up":64,"acting":62,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":117,"seq":502511173636,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":132212,"kb_used_data":98456,"kb_used_omap":22,"kb_used_meta":33705,"kb_avail":94239628,"statfs":{"total":96636764160,"available":96501379072,"internally_reserved":0,"allocated":100818944,"data_stored":103901506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22742,"internal_metadata":34514730},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509472,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":100900,"kb_used_data":71972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94270940,"statfs":{"total":96636764160,"available":96533442560,"internally_reserved":0,"allocated":73699328,"data_stored":89732950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509473,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":150248,"kb_used_data":96852,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94221592,"statfs":{"total":96636764160,"available":96482910208,"internally_reserved":0,"allocated":99176448,"data_stored":125977686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":0,"seq":206158430229,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":64424509472,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509472,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":115640,"kb_used_data":64120,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94256200,"statfs":{"total":96636764160,"available":96518348800,"internally_reserved":0,"allocated":65658880,"data_stored":82900159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542177,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":137620,"kb_used_data":84216,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94234220,"statfs":{"total":96636764160,"available":96495841280,"internally_reserved":0,"allocated":86237184,"data_stored":111513171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":103,"seq":442381631495,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":85736,"kb_used_data":45588,"kb_used_omap":12,"kb_used_meta":40115,"kb_avail":94286104,"statfs":{"total":96636764160,"available":96548970496,"internally_reserved":0,"allocated":46682112,"data_stored":57018531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12804,"internal_metadata":41078268},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63565824,"data_stored":81595126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2035712,"data_stored":2023494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45027328,"data_stored":56174527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6121,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":7245824,"data_stored":9302332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":97673216,"data_stored":125262983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":72192000,"data_stored":89018228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":99254272,"data_stored":103158122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15274,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":76058624,"data_stored":99524432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-04T17:48:11.203 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-04T17:48:11.252 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:11.246+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:11.254 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:11.254 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:11.268 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-04T17:48:11.353 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:11.350+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:11.430 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:11.430 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:11.444 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-04T17:48:11.485 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:11.482+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:11.540 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:48:11.541 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-09-04T17:48:11.555 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:11.550+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:11.558 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":200,"stamp":"2024-09-04T17:48:11.110187+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1466761465,"num_objects":1926,"num_object_clones":265,"num_object_copies":3852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":118,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1926,"num_whiteouts":1030,"num_read":117345,"num_read_kb":797933,"num_write":16923,"num_write_kb":375963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2371,"num_bytes_recovered":131763676,"num_keys_recovered":419,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14157,"ondisk_log_size":14157,"up":82,"acting":80,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":81,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":722356,"kb_used_data":461204,"kb_used_omap":40,"kb_used_meta":256791,"kb_avail":565508684,"statfs":{"total":579820584960,"available":579080892416,"internally_reserved":0,"allocated":472272896,"data_stored":571044003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41921,"internal_metadata":262954047},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-79098444,"num_objects":-33,"num_object_clones":-13,"num_object_copies":-66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":118,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-33,"num_whiteouts":-8,"num_read":278,"num_read_kb":237,"num_write":41,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":6771070,"num_keys_recovered":21,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000910"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":1,"reported_epoch":126,"state":"unknown","last_fresh":"2024-09-04T17:48:05.889951+0000","last_change":"2024-09-04T17:48:06.154637+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-04T17:48:05.889951+0000","last_undegraded":"2024-09-04T17:48:05.889951+0000","last_fullsized":"2024-09-04T17:48:05.889951+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"123'580","reported_seq":3319,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.843555+0000","last_change":"2024-09-04T17:48:09.855336+0000","last_active":"2024-09-04T17:48:10.843555+0000","last_peered":"2024-09-04T17:48:10.843555+0000","last_clean":"2024-09-04T17:48:10.843555+0000","last_became_active":"2024-09-04T17:48:00.841031+0000","last_became_peered":"2024-09-04T17:48:00.841031+0000","last_unstale":"2024-09-04T17:48:10.843555+0000","last_undegraded":"2024-09-04T17:48:10.843555+0000","last_fullsized":"2024-09-04T17:48:10.843555+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T19:09:57.450916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001300933,"stat_sum":{"num_bytes":56202357,"num_objects":54,"num_object_clones":6,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":26,"num_read":4487,"num_read_kb":30277,"num_write":751,"num_write_kb":10557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":10538943,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.1d","version":"126'500","reported_seq":2156,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-04T17:48:06.105094+0000","last_change":"2024-09-04T17:48:05.895726+0000","last_active":"2024-09-04T17:48:06.105094+0000","last_peered":"2024-09-04T17:48:06.105094+0000","last_clean":"2024-09-04T17:48:06.105094+0000","last_became_active":"2024-09-04T17:45:55.955213+0000","last_became_peered":"2024-09-04T17:45:55.955213+0000","last_unstale":"2024-09-04T17:48:06.105094+0000","last_undegraded":"2024-09-04T17:48:06.105094+0000","last_fullsized":"2024-09-04T17:48:06.105094+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:35:41.643022+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0097749550000000001,"stat_sum":{"num_bytes":40967343,"num_objects":63,"num_object_clones":7,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":33,"num_read":2785,"num_read_kb":20258,"num_write":587,"num_write_kb":9396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2b","length":"3"}]},{"pgid":"3.1c","version":"126'640","reported_seq":6088,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-04T17:48:06.065129+0000","last_change":"2024-09-04T17:48:05.890887+0000","last_active":"2024-09-04T17:48:06.065129+0000","last_peered":"2024-09-04T17:48:06.065129+0000","last_clean":"2024-09-04T17:48:06.065129+0000","last_became_active":"2024-09-04T17:45:55.951880+0000","last_became_peered":"2024-09-04T17:45:55.951880+0000","last_unstale":"2024-09-04T17:48:06.065129+0000","last_undegraded":"2024-09-04T17:48:06.065129+0000","last_fullsized":"2024-09-04T17:48:06.065129+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":640,"log_dups_size":0,"ondisk_log_size":640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T19:09:09.894385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006468606,"stat_sum":{"num_bytes":92513810,"num_objects":79,"num_object_clones":11,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":43,"num_read":6939,"num_read_kb":47530,"num_write":741,"num_write_kb":11545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2b","length":"3"}]},{"pgid":"3.1b","version":"0'0","reported_seq":1,"reported_epoch":126,"state":"unknown","last_fresh":"2024-09-04T17:48:05.882902+0000","last_change":"2024-09-04T17:48:06.154609+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-04T17:48:05.882902+0000","last_undegraded":"2024-09-04T17:48:05.882902+0000","last_fullsized":"2024-09-04T17:48:05.882902+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"130'362","reported_seq":5739,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-04T17:48:09.894153+0000","last_change":"2024-09-04T17:48:09.853880+0000","last_active":"2024-09-04T17:48:09.894153+0000","last_peered":"2024-09-04T17:48:09.894153+0000","last_clean":"2024-09-04T17:48:09.894153+0000","last_became_active":"2024-09-04T17:47:44.946366+0000","last_became_peered":"2024-09-04T17:47:44.946366+0000","last_unstale":"2024-09-04T17:48:09.894153+0000","last_undegraded":"2024-09-04T17:48:09.894153+0000","last_fullsized":"2024-09-04T17:48:09.894153+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":362,"log_dups_size":0,"ondisk_log_size":362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:38:44.964441+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.01933491,"stat_sum":{"num_bytes":72634271,"num_objects":65,"num_object_clones":8,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":31,"num_read":5538,"num_read_kb":45426,"num_write":335,"num_write_kb":15441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":5992020,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.19","version":"131'361","reported_seq":2126,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.866058+0000","last_change":"2024-09-04T17:48:10.866726+0000","last_active":"2024-09-04T17:48:10.866058+0000","last_peered":"2024-09-04T17:48:10.866058+0000","last_clean":"2024-09-04T17:48:10.866058+0000","last_became_active":"2024-09-04T17:48:10.851145+0000","last_became_peered":"2024-09-04T17:48:10.851145+0000","last_unstale":"2024-09-04T17:48:10.866058+0000","last_undegraded":"2024-09-04T17:48:10.866058+0000","last_fullsized":"2024-09-04T17:48:10.866058+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":361,"log_dups_size":0,"ondisk_log_size":361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:29:15.116357+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0093494070000000006,"stat_sum":{"num_bytes":36387745,"num_objects":54,"num_object_clones":7,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":24,"num_read":2015,"num_read_kb":16004,"num_write":395,"num_write_kb":4601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":4015757,"num_keys_recovered":27,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.18","version":"0'0","reported_seq":1,"reported_epoch":126,"state":"unknown","last_fresh":"2024-09-04T17:48:05.882697+0000","last_change":"2024-09-04T17:48:06.154582+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-04T17:48:05.882697+0000","last_undegraded":"2024-09-04T17:48:05.882697+0000","last_fullsized":"2024-09-04T17:48:05.882697+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"125'283","reported_seq":1996,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-04T17:48:05.886189+0000","last_change":"2024-09-04T17:48:05.886597+0000","last_active":"2024-09-04T17:48:05.886189+0000","last_peered":"2024-09-04T17:48:05.886189+0000","last_clean":"2024-09-04T17:48:05.886189+0000","last_became_active":"2024-09-04T17:45:55.958625+0000","last_became_peered":"2024-09-04T17:45:55.958625+0000","last_unstale":"2024-09-04T17:48:05.886189+0000","last_undegraded":"2024-09-04T17:48:05.886189+0000","last_fullsized":"2024-09-04T17:48:05.886189+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:42:21.486986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019604779999999999,"stat_sum":{"num_bytes":82172026,"num_objects":63,"num_object_clones":12,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":27,"num_read":1606,"num_read_kb":20936,"num_write":292,"num_write_kb":18406,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2b","length":"3"}]},{"pgid":"3.16","version":"126'324","reported_seq":1662,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-04T17:48:05.994313+0000","last_change":"2024-09-04T17:48:05.889294+0000","last_active":"2024-09-04T17:48:05.994313+0000","last_peered":"2024-09-04T17:48:05.994313+0000","last_clean":"2024-09-04T17:48:05.994313+0000","last_became_active":"2024-09-04T17:45:55.953254+0000","last_became_peered":"2024-09-04T17:45:55.953254+0000","last_unstale":"2024-09-04T17:48:05.994313+0000","last_undegraded":"2024-09-04T17:48:05.994313+0000","last_fullsized":"2024-09-04T17:48:05.994313+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T22:42:49.754648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0042812759999999997,"stat_sum":{"num_bytes":37296010,"num_objects":51,"num_object_clones":7,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":29,"num_read":1692,"num_read_kb":10758,"num_write":369,"num_write_kb":7387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2b","length":"3"}]},{"pgid":"3.15","version":"125'455","reported_seq":1982,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-04T17:48:05.886108+0000","last_change":"2024-09-04T17:48:05.886521+0000","last_active":"2024-09-04T17:48:05.886108+0000","last_peered":"2024-09-04T17:48:05.886108+0000","last_clean":"2024-09-04T17:48:05.886108+0000","last_became_active":"2024-09-04T17:47:08.657112+0000","last_became_peered":"2024-09-04T17:47:08.657112+0000","last_unstale":"2024-09-04T17:48:05.886108+0000","last_undegraded":"2024-09-04T17:48:05.886108+0000","last_fullsized":"2024-09-04T17:48:05.886108+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:28:53.473428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021417509999999999,"stat_sum":{"num_bytes":17977259,"num_objects":46,"num_object_clones":4,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":27,"num_read":3707,"num_read_kb":16708,"num_write":482,"num_write_kb":4429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":7294656,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2b","length":"3"}]},{"pgid":"3.14","version":"125'691","reported_seq":2049,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-04T17:48:09.860236+0000","last_change":"2024-09-04T17:48:09.860717+0000","last_active":"2024-09-04T17:48:09.860236+0000","last_peered":"2024-09-04T17:48:09.860236+0000","last_clean":"2024-09-04T17:48:09.860236+0000","last_became_active":"2024-09-04T17:46:53.539886+0000","last_became_peered":"2024-09-04T17:46:53.539886+0000","last_unstale":"2024-09-04T17:48:09.860236+0000","last_undegraded":"2024-09-04T17:48:09.860236+0000","last_fullsized":"2024-09-04T17:48:09.860236+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":691,"log_dups_size":0,"ondisk_log_size":691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:39:44.106583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0051295710000000003,"stat_sum":{"num_bytes":54551302,"num_objects":66,"num_object_clones":10,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":35,"num_read":3772,"num_read_kb":29665,"num_write":939,"num_write_kb":35894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"2.7","version":"0'0","reported_seq":95,"reported_epoch":130,"state":"peering","last_fresh":"2024-09-04T17:48:09.858550+0000","last_change":"2024-09-04T17:48:10.840997+0000","last_active":"2024-09-04T17:48:10.840997+0000","last_peered":"2024-09-04T17:48:09.858550+0000","last_clean":"2024-09-04T17:48:09.858550+0000","last_became_active":"2024-09-04T17:47:57.840926+0000","last_became_peered":"2024-09-04T17:47:57.840926+0000","last_unstale":"2024-09-04T17:48:09.858550+0000","last_undegraded":"2024-09-04T17:48:09.858550+0000","last_fullsized":"2024-09-04T17:48:09.858550+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001226074,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"126'319","reported_seq":2771,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-04T17:48:06.021270+0000","last_change":"2024-09-04T17:48:05.886025+0000","last_active":"2024-09-04T17:48:06.021270+0000","last_peered":"2024-09-04T17:48:06.021270+0000","last_clean":"2024-09-04T17:48:06.021270+0000","last_became_active":"2024-09-04T17:47:32.848310+0000","last_became_peered":"2024-09-04T17:47:32.848310+0000","last_unstale":"2024-09-04T17:48:06.021270+0000","last_undegraded":"2024-09-04T17:48:06.021270+0000","last_fullsized":"2024-09-04T17:48:06.021270+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:15:06.255989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001696892,"stat_sum":{"num_bytes":40074006,"num_objects":54,"num_object_clones":4,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":28,"num_read":2418,"num_read_kb":14504,"num_write":368,"num_write_kb":5524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":919873,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2b","length":"3"}]},{"pgid":"2.6","version":"0'0","reported_seq":37,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.843292+0000","last_change":"2024-09-04T17:47:58.862844+0000","last_active":"2024-09-04T17:48:10.843292+0000","last_peered":"2024-09-04T17:48:10.843292+0000","last_clean":"2024-09-04T17:48:10.843292+0000","last_became_active":"2024-09-04T17:47:58.859941+0000","last_became_peered":"2024-09-04T17:47:58.859941+0000","last_unstale":"2024-09-04T17:48:10.843292+0000","last_undegraded":"2024-09-04T17:48:10.843292+0000","last_fullsized":"2024-09-04T17:48:10.843292+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:08:53.409036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011364179999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"129'412","reported_seq":2639,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-04T17:48:09.871164+0000","last_change":"2024-09-04T17:48:09.871644+0000","last_active":"2024-09-04T17:48:09.871164+0000","last_peered":"2024-09-04T17:48:09.871164+0000","last_clean":"2024-09-04T17:48:09.871164+0000","last_became_active":"2024-09-04T17:47:08.653159+0000","last_became_peered":"2024-09-04T17:47:08.653159+0000","last_unstale":"2024-09-04T17:48:09.871164+0000","last_undegraded":"2024-09-04T17:48:09.871164+0000","last_fullsized":"2024-09-04T17:48:09.871164+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":412,"log_dups_size":0,"ondisk_log_size":412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T21:59:05.657735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01497677,"stat_sum":{"num_bytes":43421480,"num_objects":63,"num_object_clones":8,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":36,"num_read":2882,"num_read_kb":23337,"num_write":531,"num_write_kb":21560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":952382,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"2.5","version":"0'0","reported_seq":236,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-04T17:48:09.857533+0000","last_change":"2024-09-04T17:45:49.476607+0000","last_active":"2024-09-04T17:48:09.857533+0000","last_peered":"2024-09-04T17:48:09.857533+0000","last_clean":"2024-09-04T17:48:09.857533+0000","last_became_active":"2024-09-04T17:45:47.369308+0000","last_became_peered":"2024-09-04T17:45:47.369308+0000","last_unstale":"2024-09-04T17:48:09.857533+0000","last_undegraded":"2024-09-04T17:48:09.857533+0000","last_fullsized":"2024-09-04T17:48:09.857533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:17:13.378553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001029826,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"129'355","reported_seq":2393,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-04T17:48:09.094306+0000","last_change":"2024-09-04T17:48:09.940167+0000","last_active":"2024-09-04T17:48:09.940167+0000","last_peered":"2024-09-04T17:48:09.094306+0000","last_clean":"2024-09-04T17:48:05.885073+0000","last_became_active":"2024-09-04T17:48:08.854562+0000","last_became_peered":"2024-09-04T17:48:08.854562+0000","last_unstale":"2024-09-04T17:48:09.094306+0000","last_undegraded":"2024-09-04T17:48:09.094306+0000","last_fullsized":"2024-09-04T17:48:09.094306+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":355,"log_dups_size":0,"ondisk_log_size":355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.019597868000000001,"stat_sum":{"num_bytes":39637658,"num_objects":63,"num_object_clones":5,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":35,"num_read":2424,"num_read_kb":18572,"num_write":397,"num_write_kb":9850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":237,"num_bytes_recovered":13375622,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2b","length":"3"}]},{"pgid":"2.4","version":"0'0","reported_seq":72,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-04T17:48:05.884096+0000","last_change":"2024-09-04T17:47:29.803682+0000","last_active":"2024-09-04T17:48:05.884096+0000","last_peered":"2024-09-04T17:48:05.884096+0000","last_clean":"2024-09-04T17:48:05.884096+0000","last_became_active":"2024-09-04T17:47:29.803256+0000","last_became_peered":"2024-09-04T17:47:29.803256+0000","last_unstale":"2024-09-04T17:48:05.884096+0000","last_undegraded":"2024-09-04T17:48:05.884096+0000","last_fullsized":"2024-09-04T17:48:05.884096+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:32:26.523162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"125'273","reported_seq":1783,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-04T17:48:05.891886+0000","last_change":"2024-09-04T17:48:05.891886+0000","last_active":"2024-09-04T17:48:05.891886+0000","last_peered":"2024-09-04T17:48:05.891886+0000","last_clean":"2024-09-04T17:48:05.891886+0000","last_became_active":"2024-09-04T17:47:41.918947+0000","last_became_peered":"2024-09-04T17:47:41.918947+0000","last_unstale":"2024-09-04T17:48:05.891886+0000","last_undegraded":"2024-09-04T17:48:05.891886+0000","last_fullsized":"2024-09-04T17:48:05.891886+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":273,"log_dups_size":0,"ondisk_log_size":273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T19:53:30.404276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0060645610000000004,"stat_sum":{"num_bytes":26359679,"num_objects":68,"num_object_clones":10,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":36,"num_read":2054,"num_read_kb":7214,"num_write":251,"num_write_kb":6999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":10024551,"num_keys_recovered":46,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2b","length":"3"}]},{"pgid":"3.3","version":"126'634","reported_seq":2745,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-04T17:48:06.103617+0000","last_change":"2024-09-04T17:48:05.885357+0000","last_active":"2024-09-04T17:48:06.103617+0000","last_peered":"2024-09-04T17:48:06.103617+0000","last_clean":"2024-09-04T17:48:06.103617+0000","last_became_active":"2024-09-04T17:46:53.542889+0000","last_became_peered":"2024-09-04T17:46:53.542889+0000","last_unstale":"2024-09-04T17:48:06.103617+0000","last_undegraded":"2024-09-04T17:48:06.103617+0000","last_fullsized":"2024-09-04T17:48:06.103617+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":634,"log_dups_size":0,"ondisk_log_size":634,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:08:46.739406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00134242,"stat_sum":{"num_bytes":66410315,"num_objects":71,"num_object_clones":12,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":38,"num_read":4231,"num_read_kb":26156,"num_write":871,"num_write_kb":14086,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2b","length":"3"}]},{"pgid":"2.2","version":"22'2","reported_seq":245,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-04T17:48:05.884649+0000","last_change":"2024-09-04T17:47:59.598241+0000","last_active":"2024-09-04T17:48:05.884649+0000","last_peered":"2024-09-04T17:48:05.884649+0000","last_clean":"2024-09-04T17:48:05.884649+0000","last_became_active":"2024-09-04T17:47:58.884634+0000","last_became_peered":"2024-09-04T17:47:58.884634+0000","last_unstale":"2024-09-04T17:48:05.884649+0000","last_undegraded":"2024-09-04T17:48:05.884649+0000","last_fullsized":"2024-09-04T17:48:05.884649+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T22:02:31.692385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026316069999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"129'373","reported_seq":2865,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-04T17:48:09.835070+0000","last_change":"2024-09-04T17:48:09.835714+0000","last_active":"2024-09-04T17:48:09.835070+0000","last_peered":"2024-09-04T17:48:09.835070+0000","last_clean":"2024-09-04T17:48:09.835070+0000","last_became_active":"2024-09-04T17:48:01.849363+0000","last_became_peered":"2024-09-04T17:48:01.849363+0000","last_unstale":"2024-09-04T17:48:09.835070+0000","last_undegraded":"2024-09-04T17:48:09.835070+0000","last_fullsized":"2024-09-04T17:48:09.835070+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":373,"log_dups_size":0,"ondisk_log_size":373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T22:04:09.650735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013141769999999999,"stat_sum":{"num_bytes":42792223,"num_objects":71,"num_object_clones":9,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":42,"num_read":3143,"num_read_kb":22159,"num_write":313,"num_write_kb":4881,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":5692590,"num_keys_recovered":35,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"2.1","version":"0'0","reported_seq":216,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-04T17:48:09.834804+0000","last_change":"2024-09-04T17:47:41.913896+0000","last_active":"2024-09-04T17:48:09.834804+0000","last_peered":"2024-09-04T17:48:09.834804+0000","last_clean":"2024-09-04T17:48:09.834804+0000","last_became_active":"2024-09-04T17:47:41.913057+0000","last_became_peered":"2024-09-04T17:47:41.913057+0000","last_unstale":"2024-09-04T17:48:09.834804+0000","last_undegraded":"2024-09-04T17:48:09.834804+0000","last_fullsized":"2024-09-04T17:48:09.834804+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:49:25.382532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001025195,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"126'527","reported_seq":2536,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-04T17:48:05.885237+0000","last_change":"2024-09-04T17:48:05.885237+0000","last_active":"2024-09-04T17:48:05.885237+0000","last_peered":"2024-09-04T17:48:05.885237+0000","last_clean":"2024-09-04T17:48:05.885237+0000","last_became_active":"2024-09-04T17:47:08.655358+0000","last_became_peered":"2024-09-04T17:47:08.655358+0000","last_unstale":"2024-09-04T17:48:05.885237+0000","last_undegraded":"2024-09-04T17:48:05.885237+0000","last_fullsized":"2024-09-04T17:48:05.885237+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T22:56:57.415892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001259982,"stat_sum":{"num_bytes":87377265,"num_objects":81,"num_object_clones":18,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":41,"num_read":2920,"num_read_kb":40212,"num_write":602,"num_write_kb":23345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":112,"num_bytes_recovered":8952402,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2b","length":"3"}]},{"pgid":"2.0","version":"0'0","reported_seq":242,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-04T17:48:09.854561+0000","last_change":"2024-09-04T17:47:29.804429+0000","last_active":"2024-09-04T17:48:09.854561+0000","last_peered":"2024-09-04T17:48:09.854561+0000","last_clean":"2024-09-04T17:48:09.854561+0000","last_became_active":"2024-09-04T17:47:29.803992+0000","last_became_peered":"2024-09-04T17:47:29.803992+0000","last_unstale":"2024-09-04T17:48:09.854561+0000","last_undegraded":"2024-09-04T17:48:09.854561+0000","last_fullsized":"2024-09-04T17:48:09.854561+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:50:32.563345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010553590000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"126'525","reported_seq":2225,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-04T17:48:06.054465+0000","last_change":"2024-09-04T17:48:05.886573+0000","last_active":"2024-09-04T17:48:06.054465+0000","last_peered":"2024-09-04T17:48:06.054465+0000","last_clean":"2024-09-04T17:48:06.054465+0000","last_became_active":"2024-09-04T17:48:01.850400+0000","last_became_peered":"2024-09-04T17:48:01.850400+0000","last_unstale":"2024-09-04T17:48:06.054465+0000","last_undegraded":"2024-09-04T17:48:06.054465+0000","last_fullsized":"2024-09-04T17:48:06.054465+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T22:55:10.338360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024575920000000002,"stat_sum":{"num_bytes":50198348,"num_objects":64,"num_object_clones":9,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":33,"num_read":5199,"num_read_kb":29723,"num_write":673,"num_write_kb":13801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":13799490,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2b","length":"3"}]},{"pgid":"2.3","version":"20'1","reported_seq":239,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-04T17:48:05.883926+0000","last_change":"2024-09-04T17:47:41.919544+0000","last_active":"2024-09-04T17:48:05.883926+0000","last_peered":"2024-09-04T17:48:05.883926+0000","last_clean":"2024-09-04T17:48:05.883926+0000","last_became_active":"2024-09-04T17:47:41.919027+0000","last_became_peered":"2024-09-04T17:47:41.919027+0000","last_unstale":"2024-09-04T17:48:05.883926+0000","last_undegraded":"2024-09-04T17:48:05.883926+0000","last_fullsized":"2024-09-04T17:48:05.883926+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:42:55.688562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086618299999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":327,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-04T17:48:09.855144+0000","last_change":"2024-09-04T17:45:42.341149+0000","last_active":"2024-09-04T17:48:09.855144+0000","last_peered":"2024-09-04T17:48:09.855144+0000","last_clean":"2024-09-04T17:48:09.855144+0000","last_became_active":"2024-09-04T17:45:42.339286+0000","last_became_peered":"2024-09-04T17:45:42.339286+0000","last_unstale":"2024-09-04T17:48:09.855144+0000","last_undegraded":"2024-09-04T17:48:09.855144+0000","last_fullsized":"2024-09-04T17:48:09.855144+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:41.305128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:41.305128+0000","last_clean_scrub_stamp":"2024-09-04T17:45:41.305128+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:52:24.860049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"126'483","reported_seq":2668,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-04T17:48:09.855504+0000","last_change":"2024-09-04T17:48:09.857214+0000","last_active":"2024-09-04T17:48:09.855504+0000","last_peered":"2024-09-04T17:48:09.855504+0000","last_clean":"2024-09-04T17:48:09.855504+0000","last_became_active":"2024-09-04T17:47:32.847682+0000","last_became_peered":"2024-09-04T17:47:32.847682+0000","last_unstale":"2024-09-04T17:48:09.855504+0000","last_undegraded":"2024-09-04T17:48:09.855504+0000","last_fullsized":"2024-09-04T17:48:09.855504+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":483,"log_dups_size":0,"ondisk_log_size":483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:07:21.110914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002703474,"stat_sum":{"num_bytes":45080057,"num_objects":72,"num_object_clones":11,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":40,"num_read":2705,"num_read_kb":17207,"num_write":531,"num_write_kb":29364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":14692620,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.9","version":"129'591","reported_seq":4115,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-04T17:48:09.835133+0000","last_change":"2024-09-04T17:48:09.836181+0000","last_active":"2024-09-04T17:48:09.835133+0000","last_peered":"2024-09-04T17:48:09.835133+0000","last_clean":"2024-09-04T17:48:09.835133+0000","last_became_active":"2024-09-04T17:47:44.949630+0000","last_became_peered":"2024-09-04T17:47:44.949630+0000","last_unstale":"2024-09-04T17:48:09.835133+0000","last_undegraded":"2024-09-04T17:48:09.835133+0000","last_fullsized":"2024-09-04T17:48:09.835133+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:20:22.482600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001705781,"stat_sum":{"num_bytes":55724561,"num_objects":78,"num_object_clones":14,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":42,"num_read":5692,"num_read_kb":37073,"num_write":680,"num_write_kb":18627,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":4534479,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.a","version":"131'533","reported_seq":3019,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.890923+0000","last_change":"2024-09-04T17:48:09.853469+0000","last_active":"2024-09-04T17:48:10.890923+0000","last_peered":"2024-09-04T17:48:10.890923+0000","last_clean":"2024-09-04T17:48:10.890923+0000","last_became_active":"2024-09-04T17:48:00.841070+0000","last_became_peered":"2024-09-04T17:48:00.841070+0000","last_unstale":"2024-09-04T17:48:10.890923+0000","last_undegraded":"2024-09-04T17:48:10.890923+0000","last_fullsized":"2024-09-04T17:48:10.890923+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:47:07.864899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038466329999999999,"stat_sum":{"num_bytes":36655948,"num_objects":72,"num_object_clones":11,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":40,"num_read":5202,"num_read_kb":16044,"num_write":529,"num_write_kb":2741,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":1572433,"num_keys_recovered":66,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.b","version":"125'677","reported_seq":2186,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-04T17:48:05.885296+0000","last_change":"2024-09-04T17:48:05.886018+0000","last_active":"2024-09-04T17:48:05.885296+0000","last_peered":"2024-09-04T17:48:05.885296+0000","last_clean":"2024-09-04T17:48:05.885296+0000","last_became_active":"2024-09-04T17:47:08.657286+0000","last_became_peered":"2024-09-04T17:47:08.657286+0000","last_unstale":"2024-09-04T17:48:05.885296+0000","last_undegraded":"2024-09-04T17:48:05.885296+0000","last_fullsized":"2024-09-04T17:48:05.885296+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":677,"log_dups_size":0,"ondisk_log_size":677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:39:42.239271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018167669999999999,"stat_sum":{"num_bytes":31620378,"num_objects":71,"num_object_clones":10,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":33,"num_read":3911,"num_read_kb":14517,"num_write":845,"num_write_kb":7911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":3759450,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2b","length":"3"}]},{"pgid":"3.c","version":"125'550","reported_seq":7774,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-04T17:48:05.888379+0000","last_change":"2024-09-04T17:48:05.888779+0000","last_active":"2024-09-04T17:48:05.888379+0000","last_peered":"2024-09-04T17:48:05.888379+0000","last_clean":"2024-09-04T17:48:05.888379+0000","last_became_active":"2024-09-04T17:48:02.855127+0000","last_became_peered":"2024-09-04T17:48:02.855127+0000","last_unstale":"2024-09-04T17:48:05.888379+0000","last_undegraded":"2024-09-04T17:48:05.888379+0000","last_fullsized":"2024-09-04T17:48:05.888379+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:40:26.367843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004127805,"stat_sum":{"num_bytes":66893850,"num_objects":76,"num_object_clones":11,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":43,"num_read":8646,"num_read_kb":71089,"num_write":688,"num_write_kb":16847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":153,"num_bytes_recovered":7666524,"num_keys_recovered":34,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2b","length":"3"}]},{"pgid":"3.d","version":"126'1282","reported_seq":3848,"reported_epoch":130,"state":"remapped+peering","last_fresh":"2024-09-04T17:48:09.855528+0000","last_change":"2024-09-04T17:48:09.855528+0000","last_active":"2024-09-04T17:48:08.833037+0000","last_peered":"2024-09-04T17:48:07.832005+0000","last_clean":"2024-09-04T17:48:07.832005+0000","last_became_active":"2024-09-04T17:48:00.840986+0000","last_became_peered":"2024-09-04T17:48:00.840986+0000","last_unstale":"2024-09-04T17:48:09.855528+0000","last_undegraded":"2024-09-04T17:48:09.855528+0000","last_fullsized":"2024-09-04T17:48:09.855528+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":1282,"log_dups_size":0,"ondisk_log_size":1282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018889099999999999,"stat_sum":{"num_bytes":49889564,"num_objects":60,"num_object_clones":6,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":34,"num_read":8526,"num_read_kb":26249,"num_write":2192,"num_write_kb":13182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":8914239,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2a","length":"4"}]},{"pgid":"3.e","version":"127'440","reported_seq":5206,"reported_epoch":130,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-04T17:48:09.886153+0000","last_change":"2024-09-04T17:48:09.886153+0000","last_active":"2024-09-04T17:48:09.886153+0000","last_peered":"2024-09-04T17:48:09.886153+0000","last_clean":"2024-09-04T17:48:06.890201+0000","last_became_active":"2024-09-04T17:48:09.884964+0000","last_became_peered":"2024-09-04T17:48:09.884964+0000","last_unstale":"2024-09-04T17:48:09.886153+0000","last_undegraded":"2024-09-04T17:48:09.857903+0000","last_fullsized":"2024-09-04T17:48:09.854505+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.010017414000000001,"stat_sum":{"num_bytes":63389770,"num_objects":66,"num_object_clones":13,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":118,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":32,"num_read":6129,"num_read_kb":28040,"num_write":457,"num_write_kb":11211,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":3926000,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7],"avail_no_missing":["7","0"],"object_location_counts":[{"shards":"0,7","objects":66}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2a","length":"4"}]},{"pgid":"3.f","version":"130'292","reported_seq":2844,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-04T17:48:09.870980+0000","last_change":"2024-09-04T17:48:09.870980+0000","last_active":"2024-09-04T17:48:09.870980+0000","last_peered":"2024-09-04T17:48:09.870980+0000","last_clean":"2024-09-04T17:48:09.870980+0000","last_became_active":"2024-09-04T17:45:55.956734+0000","last_became_peered":"2024-09-04T17:45:55.956734+0000","last_unstale":"2024-09-04T17:48:09.870980+0000","last_undegraded":"2024-09-04T17:48:09.870980+0000","last_fullsized":"2024-09-04T17:48:09.870980+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":292,"log_dups_size":0,"ondisk_log_size":292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T23:59:11.397371+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.01496131,"stat_sum":{"num_bytes":49550709,"num_objects":69,"num_object_clones":12,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":36,"num_read":2563,"num_read_kb":44941,"num_write":297,"num_write_kb":13423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.10","version":"126'514","reported_seq":3148,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.842877+0000","last_change":"2024-09-04T17:48:09.851750+0000","last_active":"2024-09-04T17:48:10.842877+0000","last_peered":"2024-09-04T17:48:10.842877+0000","last_clean":"2024-09-04T17:48:10.842877+0000","last_became_active":"2024-09-04T17:48:00.843136+0000","last_became_peered":"2024-09-04T17:48:00.843136+0000","last_unstale":"2024-09-04T17:48:10.842877+0000","last_undegraded":"2024-09-04T17:48:10.842877+0000","last_fullsized":"2024-09-04T17:48:10.842877+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":514,"log_dups_size":0,"ondisk_log_size":514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T23:07:02.271876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049468649999999999,"stat_sum":{"num_bytes":51575959,"num_objects":78,"num_object_clones":10,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":46,"num_read":4069,"num_read_kb":18348,"num_write":591,"num_write_kb":12832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":3377599,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.11","version":"130'500","reported_seq":5739,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-04T17:48:09.877129+0000","last_change":"2024-09-04T17:48:09.879368+0000","last_active":"2024-09-04T17:48:09.877129+0000","last_peered":"2024-09-04T17:48:09.877129+0000","last_clean":"2024-09-04T17:48:09.877129+0000","last_became_active":"2024-09-04T17:45:55.951870+0000","last_became_peered":"2024-09-04T17:45:55.951870+0000","last_unstale":"2024-09-04T17:48:09.877129+0000","last_undegraded":"2024-09-04T17:48:09.877129+0000","last_fullsized":"2024-09-04T17:48:09.877129+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T22:56:47.076439+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.024746269000000001,"stat_sum":{"num_bytes":74079184,"num_objects":82,"num_object_clones":10,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":46,"num_read":6681,"num_read_kb":61026,"num_write":514,"num_write_kb":17560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.12","version":"126'342","reported_seq":3401,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-04T17:48:05.884718+0000","last_change":"2024-09-04T17:48:05.884718+0000","last_active":"2024-09-04T17:48:05.884718+0000","last_peered":"2024-09-04T17:48:05.884718+0000","last_clean":"2024-09-04T17:48:05.884718+0000","last_became_active":"2024-09-04T17:47:44.952109+0000","last_became_peered":"2024-09-04T17:47:44.952109+0000","last_unstale":"2024-09-04T17:48:05.884718+0000","last_undegraded":"2024-09-04T17:48:05.884718+0000","last_fullsized":"2024-09-04T17:48:05.884718+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T22:00:53.272098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097872999999999992,"stat_sum":{"num_bytes":29554039,"num_objects":60,"num_object_clones":5,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":35,"num_read":3162,"num_read_kb":21367,"num_write":383,"num_write_kb":8559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":1762027,"num_keys_recovered":48,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2b","length":"3"}]},{"pgid":"3.13","version":"126'264","reported_seq":2227,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-04T17:48:09.857330+0000","last_change":"2024-09-04T17:48:09.860061+0000","last_active":"2024-09-04T17:48:09.857330+0000","last_peered":"2024-09-04T17:48:09.857330+0000","last_clean":"2024-09-04T17:48:09.857330+0000","last_became_active":"2024-09-04T17:45:55.954544+0000","last_became_peered":"2024-09-04T17:45:55.954544+0000","last_unstale":"2024-09-04T17:48:09.857330+0000","last_undegraded":"2024-09-04T17:48:09.857330+0000","last_fullsized":"2024-09-04T17:48:09.857330+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":264,"log_dups_size":0,"ondisk_log_size":264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T23:14:42.541437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0051521240000000001,"stat_sum":{"num_bytes":25183962,"num_objects":62,"num_object_clones":5,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":39,"num_read":2181,"num_read_kb":22538,"num_write":216,"num_write_kb":4374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"29","length":"5"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1466171078,"num_objects":1922,"num_object_clones":265,"num_object_copies":3844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":118,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1922,"num_whiteouts":1030,"num_read":117279,"num_read_kb":797878,"num_write":16820,"num_write_kb":374333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2370,"num_bytes_recovered":131763657,"num_keys_recovered":419,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":463052800,"data_stored":566059244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21395,"internal_metadata":0},"log_size":14082,"ondisk_log_size":14082,"up":64,"acting":62,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":117,"seq":502511173636,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":132212,"kb_used_data":98456,"kb_used_omap":22,"kb_used_meta":33705,"kb_avail":94239628,"statfs":{"total":96636764160,"available":96501379072,"internally_reserved":0,"allocated":100818944,"data_stored":103901506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22742,"internal_metadata":34514730},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509472,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":100900,"kb_used_data":71972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94270940,"statfs":{"total":96636764160,"available":96533442560,"internally_reserved":0,"allocated":73699328,"data_stored":89732950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509473,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":150248,"kb_used_data":96852,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94221592,"statfs":{"total":96636764160,"available":96482910208,"internally_reserved":0,"allocated":99176448,"data_stored":125977686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":0,"seq":206158430229,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":64424509472,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509472,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":115640,"kb_used_data":64120,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94256200,"statfs":{"total":96636764160,"available":96518348800,"internally_reserved":0,"allocated":65658880,"data_stored":82900159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542177,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":137620,"kb_used_data":84216,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94234220,"statfs":{"total":96636764160,"available":96495841280,"internally_reserved":0,"allocated":86237184,"data_stored":111513171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":103,"seq":442381631495,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":85736,"kb_used_data":45588,"kb_used_omap":12,"kb_used_meta":40115,"kb_avail":94286104,"statfs":{"total":96636764160,"available":96548970496,"internally_reserved":0,"allocated":46682112,"data_stored":57018531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12804,"internal_metadata":41078268},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63565824,"data_stored":81595126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2035712,"data_stored":2023494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45027328,"data_stored":56174527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6121,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":7245824,"data_stored":9302332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":97673216,"data_stored":125262983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":72192000,"data_stored":89018228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":99254272,"data_stored":103158122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15274,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":76058624,"data_stored":99524432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-04T17:48:11.560 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-04T17:48:11.562 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:11.562 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:11.575 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-04T17:48:11.657 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:11.654+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:11.738 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:11.738 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:11.752 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-04T17:48:11.797 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:48:11.796+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:48:11.835 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-04T17:48:11.858 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:11.854+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:11.874 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:11.874 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:11.888 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-04T17:48:11.909 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:48:11.909 INFO:teuthology.orchestra.run.smithi092.stdout:{"fsid":"f36b6ce2-8c72-4c99-8607-5c53c38ef67b","health":{"status":"HEALTH_WARN","checks":{"OSDMAP_FLAGS":{"severity":"HEALTH_WARN","summary":{"message":"noscrub,nodeep-scrub flag(s) set","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":163,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":132,"num_osds":8,"num_up_osds":8,"osd_up_since":1725472077,"num_in_osds":6,"osd_in_since":1725472048,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":37},{"state_name":"unknown","count":3},{"state_name":"remapped+peering","count":1}],"num_pgs":41,"num_pools":3,"num_objects":1959,"data_bytes":1545859909,"bytes_used":749375488,"bytes_avail":579071209472,"bytes_total":579820584960,"unknown_pgs_ratio":0.073170728981494904,"inactive_pgs_ratio":0.024390242993831635},"fsmap":{"epoch":1,"btime":"2024-09-04T17:45:28:071986+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-04T17:47:29.098058+0000","services":{}},"progress_events":{}} 2024-09-04T17:48:11.937 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-04T17:48:11.938 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-04T17:48:11.981 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:48:11.981 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:48:11.981 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:11.976+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:11.997 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:48:11.997 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:48:12.013 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-04T17:48:12.013 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-04T17:48:12.025 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:12.025 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:48:12.025 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:48:12.025 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:12.029 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-04T17:48:12.029 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-04T17:48:12.039 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-04T17:48:12.049 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:48:12.050 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:48:12.061 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:12.054+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:12.069 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:48:12.070 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:48:12.090 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-04T17:48:12.090 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-04T17:48:12.125 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:48:12.125 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:48:12.161 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:12.158+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:12.173 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:12.173 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:48:12.173 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:48:12.173 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:48:12.173 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:12.186 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-04T17:48:12.269 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:48:12.270 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-09-04T17:48:12.270 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:48:12.268+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:48:12.286 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":201,"stamp":"2024-09-04T17:48:11.836777+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1555731356,"num_objects":2114,"num_object_clones":283,"num_object_copies":4228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":118,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2114,"num_whiteouts":1142,"num_read":131937,"num_read_kb":867912,"num_write":18385,"num_write_kb":405399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3177,"num_bytes_recovered":196192261,"num_keys_recovered":606,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15552,"ondisk_log_size":15552,"up":82,"acting":80,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":758500,"kb_used_data":497356,"kb_used_omap":40,"kb_used_meta":260951,"kb_avail":565472540,"statfs":{"total":579820584960,"available":579043880960,"internally_reserved":0,"allocated":509292544,"data_stored":614321985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41921,"internal_metadata":267213887},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9871447,"num_objects":155,"num_object_clones":5,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":118,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":104,"num_read":14870,"num_read_kb":70216,"num_write":1503,"num_write_kb":29447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1016,"num_bytes_recovered":71199655,"num_keys_recovered":208,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.008301"},"pg_stats":[{"pgid":"3.1f","version":"125'458","reported_seq":2760,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.840024+0000","last_change":"2024-09-04T17:48:09.847643+0000","last_active":"2024-09-04T17:48:10.840024+0000","last_peered":"2024-09-04T17:48:10.840024+0000","last_clean":"2024-09-04T17:48:10.840024+0000","last_became_active":"2024-09-04T17:48:09.845199+0000","last_became_peered":"2024-09-04T17:48:09.845199+0000","last_unstale":"2024-09-04T17:48:10.840024+0000","last_undegraded":"2024-09-04T17:48:10.840024+0000","last_fullsized":"2024-09-04T17:48:10.840024+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":458,"log_dups_size":0,"ondisk_log_size":458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:59:04.183494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012519320000000001,"stat_sum":{"num_bytes":37179569,"num_objects":67,"num_object_clones":7,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":42,"num_read":2947,"num_read_kb":27164,"num_write":494,"num_write_kb":10441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":24433786,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.1e","version":"123'580","reported_seq":3319,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.843555+0000","last_change":"2024-09-04T17:48:09.855336+0000","last_active":"2024-09-04T17:48:10.843555+0000","last_peered":"2024-09-04T17:48:10.843555+0000","last_clean":"2024-09-04T17:48:10.843555+0000","last_became_active":"2024-09-04T17:48:00.841031+0000","last_became_peered":"2024-09-04T17:48:00.841031+0000","last_unstale":"2024-09-04T17:48:10.843555+0000","last_undegraded":"2024-09-04T17:48:10.843555+0000","last_fullsized":"2024-09-04T17:48:10.843555+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T19:09:57.450916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001300933,"stat_sum":{"num_bytes":56202357,"num_objects":54,"num_object_clones":6,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":26,"num_read":4487,"num_read_kb":30277,"num_write":751,"num_write_kb":10557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":10538943,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.1d","version":"126'500","reported_seq":2176,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.845642+0000","last_change":"2024-09-04T17:48:09.849221+0000","last_active":"2024-09-04T17:48:10.845642+0000","last_peered":"2024-09-04T17:48:10.845642+0000","last_clean":"2024-09-04T17:48:10.845642+0000","last_became_active":"2024-09-04T17:45:55.955213+0000","last_became_peered":"2024-09-04T17:45:55.955213+0000","last_unstale":"2024-09-04T17:48:10.845642+0000","last_undegraded":"2024-09-04T17:48:10.845642+0000","last_fullsized":"2024-09-04T17:48:10.845642+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T18:15:40.664134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0067016410000000004,"stat_sum":{"num_bytes":40967343,"num_objects":63,"num_object_clones":7,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":33,"num_read":2785,"num_read_kb":20258,"num_write":587,"num_write_kb":9396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.1c","version":"131'645","reported_seq":6132,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.845503+0000","last_change":"2024-09-04T17:48:09.845659+0000","last_active":"2024-09-04T17:48:10.845503+0000","last_peered":"2024-09-04T17:48:10.845503+0000","last_clean":"2024-09-04T17:48:10.845503+0000","last_became_active":"2024-09-04T17:45:55.951880+0000","last_became_peered":"2024-09-04T17:45:55.951880+0000","last_unstale":"2024-09-04T17:48:10.845503+0000","last_undegraded":"2024-09-04T17:48:10.845503+0000","last_fullsized":"2024-09-04T17:48:10.845503+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":645,"log_dups_size":0,"ondisk_log_size":645,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T18:55:53.482772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0046450099999999998,"stat_sum":{"num_bytes":77833746,"num_objects":77,"num_object_clones":10,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":43,"num_read":6986,"num_read_kb":47572,"num_write":746,"num_write_kb":11545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.1b","version":"125'488","reported_seq":1720,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.839099+0000","last_change":"2024-09-04T17:48:09.856409+0000","last_active":"2024-09-04T17:48:10.839099+0000","last_peered":"2024-09-04T17:48:10.839099+0000","last_clean":"2024-09-04T17:48:10.839099+0000","last_became_active":"2024-09-04T17:48:09.853793+0000","last_became_peered":"2024-09-04T17:48:09.853793+0000","last_unstale":"2024-09-04T17:48:10.839099+0000","last_undegraded":"2024-09-04T17:48:10.839099+0000","last_fullsized":"2024-09-04T17:48:10.839099+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T23:09:36.303529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010814819999999999,"stat_sum":{"num_bytes":19881692,"num_objects":67,"num_object_clones":7,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":39,"num_read":3057,"num_read_kb":7588,"num_write":565,"num_write_kb":4577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":1993914,"num_keys_recovered":48,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.1a","version":"130'362","reported_seq":5739,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-04T17:48:09.894153+0000","last_change":"2024-09-04T17:48:09.853880+0000","last_active":"2024-09-04T17:48:09.894153+0000","last_peered":"2024-09-04T17:48:09.894153+0000","last_clean":"2024-09-04T17:48:09.894153+0000","last_became_active":"2024-09-04T17:47:44.946366+0000","last_became_peered":"2024-09-04T17:47:44.946366+0000","last_unstale":"2024-09-04T17:48:09.894153+0000","last_undegraded":"2024-09-04T17:48:09.894153+0000","last_fullsized":"2024-09-04T17:48:09.894153+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":362,"log_dups_size":0,"ondisk_log_size":362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:38:44.964441+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.01933491,"stat_sum":{"num_bytes":72634271,"num_objects":65,"num_object_clones":8,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":31,"num_read":5538,"num_read_kb":45426,"num_write":335,"num_write_kb":15441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":5992020,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.19","version":"131'361","reported_seq":2126,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.866058+0000","last_change":"2024-09-04T17:48:10.866726+0000","last_active":"2024-09-04T17:48:10.866058+0000","last_peered":"2024-09-04T17:48:10.866058+0000","last_clean":"2024-09-04T17:48:10.866058+0000","last_became_active":"2024-09-04T17:48:10.851145+0000","last_became_peered":"2024-09-04T17:48:10.851145+0000","last_unstale":"2024-09-04T17:48:10.866058+0000","last_undegraded":"2024-09-04T17:48:10.866058+0000","last_fullsized":"2024-09-04T17:48:10.866058+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":361,"log_dups_size":0,"ondisk_log_size":361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:29:15.116357+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0093494070000000006,"stat_sum":{"num_bytes":36387745,"num_objects":54,"num_object_clones":7,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":24,"num_read":2015,"num_read_kb":16004,"num_write":395,"num_write_kb":4601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":4015757,"num_keys_recovered":27,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.18","version":"125'408","reported_seq":1702,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.839094+0000","last_change":"2024-09-04T17:48:09.841174+0000","last_active":"2024-09-04T17:48:10.839094+0000","last_peered":"2024-09-04T17:48:10.839094+0000","last_clean":"2024-09-04T17:48:10.839094+0000","last_became_active":"2024-09-04T17:48:09.838648+0000","last_became_peered":"2024-09-04T17:48:09.838648+0000","last_unstale":"2024-09-04T17:48:10.839094+0000","last_undegraded":"2024-09-04T17:48:10.839094+0000","last_fullsized":"2024-09-04T17:48:10.839094+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:23:30.666247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012459330000000001,"stat_sum":{"num_bytes":70698825,"num_objects":61,"num_object_clones":6,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":32,"num_read":8379,"num_read_kb":35044,"num_write":372,"num_write_kb":14411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":288,"num_bytes_recovered":25655550,"num_keys_recovered":117,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.17","version":"125'283","reported_seq":2016,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.840918+0000","last_change":"2024-09-04T17:48:09.839869+0000","last_active":"2024-09-04T17:48:10.840918+0000","last_peered":"2024-09-04T17:48:10.840918+0000","last_clean":"2024-09-04T17:48:10.840918+0000","last_became_active":"2024-09-04T17:45:55.958625+0000","last_became_peered":"2024-09-04T17:45:55.958625+0000","last_unstale":"2024-09-04T17:48:10.840918+0000","last_undegraded":"2024-09-04T17:48:10.840918+0000","last_fullsized":"2024-09-04T17:48:10.840918+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T22:57:41.662122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021994229999999998,"stat_sum":{"num_bytes":82172026,"num_objects":63,"num_object_clones":12,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":27,"num_read":1606,"num_read_kb":20936,"num_write":292,"num_write_kb":18406,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.16","version":"126'324","reported_seq":1682,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.845454+0000","last_change":"2024-09-04T17:48:09.843759+0000","last_active":"2024-09-04T17:48:10.845454+0000","last_peered":"2024-09-04T17:48:10.845454+0000","last_clean":"2024-09-04T17:48:10.845454+0000","last_became_active":"2024-09-04T17:45:55.953254+0000","last_became_peered":"2024-09-04T17:45:55.953254+0000","last_unstale":"2024-09-04T17:48:10.845454+0000","last_undegraded":"2024-09-04T17:48:10.845454+0000","last_fullsized":"2024-09-04T17:48:10.845454+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:18:29.645248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003761838,"stat_sum":{"num_bytes":37296010,"num_objects":51,"num_object_clones":7,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":29,"num_read":1692,"num_read_kb":10758,"num_write":369,"num_write_kb":7387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.15","version":"125'455","reported_seq":1982,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-04T17:48:05.886108+0000","last_change":"2024-09-04T17:48:05.886521+0000","last_active":"2024-09-04T17:48:05.886108+0000","last_peered":"2024-09-04T17:48:05.886108+0000","last_clean":"2024-09-04T17:48:05.886108+0000","last_became_active":"2024-09-04T17:47:08.657112+0000","last_became_peered":"2024-09-04T17:47:08.657112+0000","last_unstale":"2024-09-04T17:48:05.886108+0000","last_undegraded":"2024-09-04T17:48:05.886108+0000","last_fullsized":"2024-09-04T17:48:05.886108+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:28:53.473428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021417509999999999,"stat_sum":{"num_bytes":17977259,"num_objects":46,"num_object_clones":4,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":27,"num_read":3707,"num_read_kb":16708,"num_write":482,"num_write_kb":4429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":7294656,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2b","length":"3"}]},{"pgid":"3.14","version":"125'691","reported_seq":2049,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-04T17:48:09.860236+0000","last_change":"2024-09-04T17:48:09.860717+0000","last_active":"2024-09-04T17:48:09.860236+0000","last_peered":"2024-09-04T17:48:09.860236+0000","last_clean":"2024-09-04T17:48:09.860236+0000","last_became_active":"2024-09-04T17:46:53.539886+0000","last_became_peered":"2024-09-04T17:46:53.539886+0000","last_unstale":"2024-09-04T17:48:09.860236+0000","last_undegraded":"2024-09-04T17:48:09.860236+0000","last_fullsized":"2024-09-04T17:48:09.860236+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":691,"log_dups_size":0,"ondisk_log_size":691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:39:44.106583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0051295710000000003,"stat_sum":{"num_bytes":54551302,"num_objects":66,"num_object_clones":10,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":35,"num_read":3772,"num_read_kb":29665,"num_write":939,"num_write_kb":35894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"2.7","version":"0'0","reported_seq":95,"reported_epoch":130,"state":"peering","last_fresh":"2024-09-04T17:48:09.858550+0000","last_change":"2024-09-04T17:48:10.840997+0000","last_active":"2024-09-04T17:48:10.840997+0000","last_peered":"2024-09-04T17:48:09.858550+0000","last_clean":"2024-09-04T17:48:09.858550+0000","last_became_active":"2024-09-04T17:47:57.840926+0000","last_became_peered":"2024-09-04T17:47:57.840926+0000","last_unstale":"2024-09-04T17:48:09.858550+0000","last_undegraded":"2024-09-04T17:48:09.858550+0000","last_fullsized":"2024-09-04T17:48:09.858550+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001226074,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"129'320","reported_seq":2792,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.840537+0000","last_change":"2024-09-04T17:48:09.838682+0000","last_active":"2024-09-04T17:48:10.840537+0000","last_peered":"2024-09-04T17:48:10.840537+0000","last_clean":"2024-09-04T17:48:10.840537+0000","last_became_active":"2024-09-04T17:47:32.848310+0000","last_became_peered":"2024-09-04T17:47:32.848310+0000","last_unstale":"2024-09-04T17:48:10.840537+0000","last_undegraded":"2024-09-04T17:48:10.840537+0000","last_fullsized":"2024-09-04T17:48:10.840537+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":320,"log_dups_size":0,"ondisk_log_size":320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:08:10.685688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001293892,"stat_sum":{"num_bytes":40074006,"num_objects":54,"num_object_clones":4,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":28,"num_read":2418,"num_read_kb":14504,"num_write":368,"num_write_kb":5524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":919873,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"2.6","version":"0'0","reported_seq":37,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.843292+0000","last_change":"2024-09-04T17:47:58.862844+0000","last_active":"2024-09-04T17:48:10.843292+0000","last_peered":"2024-09-04T17:48:10.843292+0000","last_clean":"2024-09-04T17:48:10.843292+0000","last_became_active":"2024-09-04T17:47:58.859941+0000","last_became_peered":"2024-09-04T17:47:58.859941+0000","last_unstale":"2024-09-04T17:48:10.843292+0000","last_undegraded":"2024-09-04T17:48:10.843292+0000","last_fullsized":"2024-09-04T17:48:10.843292+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:08:53.409036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011364179999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"129'412","reported_seq":2639,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-04T17:48:09.871164+0000","last_change":"2024-09-04T17:48:09.871644+0000","last_active":"2024-09-04T17:48:09.871164+0000","last_peered":"2024-09-04T17:48:09.871164+0000","last_clean":"2024-09-04T17:48:09.871164+0000","last_became_active":"2024-09-04T17:47:08.653159+0000","last_became_peered":"2024-09-04T17:47:08.653159+0000","last_unstale":"2024-09-04T17:48:09.871164+0000","last_undegraded":"2024-09-04T17:48:09.871164+0000","last_fullsized":"2024-09-04T17:48:09.871164+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":412,"log_dups_size":0,"ondisk_log_size":412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T21:59:05.657735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01497677,"stat_sum":{"num_bytes":43421480,"num_objects":63,"num_object_clones":8,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":36,"num_read":2882,"num_read_kb":23337,"num_write":531,"num_write_kb":21560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":952382,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"2.5","version":"0'0","reported_seq":236,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-04T17:48:09.857533+0000","last_change":"2024-09-04T17:45:49.476607+0000","last_active":"2024-09-04T17:48:09.857533+0000","last_peered":"2024-09-04T17:48:09.857533+0000","last_clean":"2024-09-04T17:48:09.857533+0000","last_became_active":"2024-09-04T17:45:47.369308+0000","last_became_peered":"2024-09-04T17:45:47.369308+0000","last_unstale":"2024-09-04T17:48:09.857533+0000","last_undegraded":"2024-09-04T17:48:09.857533+0000","last_fullsized":"2024-09-04T17:48:09.857533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:17:13.378553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001029826,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"129'355","reported_seq":2393,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-04T17:48:09.094306+0000","last_change":"2024-09-04T17:48:09.940167+0000","last_active":"2024-09-04T17:48:09.940167+0000","last_peered":"2024-09-04T17:48:09.094306+0000","last_clean":"2024-09-04T17:48:05.885073+0000","last_became_active":"2024-09-04T17:48:08.854562+0000","last_became_peered":"2024-09-04T17:48:08.854562+0000","last_unstale":"2024-09-04T17:48:09.094306+0000","last_undegraded":"2024-09-04T17:48:09.094306+0000","last_fullsized":"2024-09-04T17:48:09.094306+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":355,"log_dups_size":0,"ondisk_log_size":355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.019597868000000001,"stat_sum":{"num_bytes":39637658,"num_objects":63,"num_object_clones":5,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":35,"num_read":2424,"num_read_kb":18572,"num_write":397,"num_write_kb":9850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":237,"num_bytes_recovered":13375622,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2b","length":"3"}]},{"pgid":"2.4","version":"0'0","reported_seq":82,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.840019+0000","last_change":"2024-09-04T17:47:29.803682+0000","last_active":"2024-09-04T17:48:10.840019+0000","last_peered":"2024-09-04T17:48:10.840019+0000","last_clean":"2024-09-04T17:48:10.840019+0000","last_became_active":"2024-09-04T17:47:29.803256+0000","last_became_peered":"2024-09-04T17:47:29.803256+0000","last_unstale":"2024-09-04T17:48:10.840019+0000","last_undegraded":"2024-09-04T17:48:10.840019+0000","last_fullsized":"2024-09-04T17:48:10.840019+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:32:26.523162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"125'273","reported_seq":1804,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.845948+0000","last_change":"2024-09-04T17:48:09.849956+0000","last_active":"2024-09-04T17:48:10.845948+0000","last_peered":"2024-09-04T17:48:10.845948+0000","last_clean":"2024-09-04T17:48:10.845948+0000","last_became_active":"2024-09-04T17:47:41.918947+0000","last_became_peered":"2024-09-04T17:47:41.918947+0000","last_unstale":"2024-09-04T17:48:10.845948+0000","last_undegraded":"2024-09-04T17:48:10.845948+0000","last_fullsized":"2024-09-04T17:48:10.845948+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":273,"log_dups_size":0,"ondisk_log_size":273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T23:41:18.557967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069886820000000004,"stat_sum":{"num_bytes":26359679,"num_objects":68,"num_object_clones":10,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":36,"num_read":2056,"num_read_kb":7215,"num_write":251,"num_write_kb":6999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":10024551,"num_keys_recovered":46,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.3","version":"131'660","reported_seq":2848,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.871783+0000","last_change":"2024-09-04T17:48:09.864268+0000","last_active":"2024-09-04T17:48:10.871783+0000","last_peered":"2024-09-04T17:48:10.871783+0000","last_clean":"2024-09-04T17:48:10.871783+0000","last_became_active":"2024-09-04T17:46:53.542889+0000","last_became_peered":"2024-09-04T17:46:53.542889+0000","last_unstale":"2024-09-04T17:48:10.871783+0000","last_undegraded":"2024-09-04T17:48:10.871783+0000","last_fullsized":"2024-09-04T17:48:10.871783+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":660,"log_dups_size":0,"ondisk_log_size":660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T22:07:45.745138+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.027057795999999999,"stat_sum":{"num_bytes":62897701,"num_objects":68,"num_object_clones":11,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":37,"num_read":4342,"num_read_kb":26256,"num_write":889,"num_write_kb":14093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"2.2","version":"22'2","reported_seq":255,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.845294+0000","last_change":"2024-09-04T17:47:59.598241+0000","last_active":"2024-09-04T17:48:10.845294+0000","last_peered":"2024-09-04T17:48:10.845294+0000","last_clean":"2024-09-04T17:48:10.845294+0000","last_became_active":"2024-09-04T17:47:58.884634+0000","last_became_peered":"2024-09-04T17:47:58.884634+0000","last_unstale":"2024-09-04T17:48:10.845294+0000","last_undegraded":"2024-09-04T17:48:10.845294+0000","last_fullsized":"2024-09-04T17:48:10.845294+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T22:02:31.692385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026316069999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"129'373","reported_seq":2865,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-04T17:48:09.835070+0000","last_change":"2024-09-04T17:48:09.835714+0000","last_active":"2024-09-04T17:48:09.835070+0000","last_peered":"2024-09-04T17:48:09.835070+0000","last_clean":"2024-09-04T17:48:09.835070+0000","last_became_active":"2024-09-04T17:48:01.849363+0000","last_became_peered":"2024-09-04T17:48:01.849363+0000","last_unstale":"2024-09-04T17:48:09.835070+0000","last_undegraded":"2024-09-04T17:48:09.835070+0000","last_fullsized":"2024-09-04T17:48:09.835070+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":373,"log_dups_size":0,"ondisk_log_size":373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T22:04:09.650735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013141769999999999,"stat_sum":{"num_bytes":42792223,"num_objects":71,"num_object_clones":9,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":42,"num_read":3143,"num_read_kb":22159,"num_write":313,"num_write_kb":4881,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":5692590,"num_keys_recovered":35,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"2.1","version":"0'0","reported_seq":216,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-04T17:48:09.834804+0000","last_change":"2024-09-04T17:47:41.913896+0000","last_active":"2024-09-04T17:48:09.834804+0000","last_peered":"2024-09-04T17:48:09.834804+0000","last_clean":"2024-09-04T17:48:09.834804+0000","last_became_active":"2024-09-04T17:47:41.913057+0000","last_became_peered":"2024-09-04T17:47:41.913057+0000","last_unstale":"2024-09-04T17:48:09.834804+0000","last_undegraded":"2024-09-04T17:48:09.834804+0000","last_fullsized":"2024-09-04T17:48:09.834804+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:49:25.382532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001025195,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"126'527","reported_seq":2810,"reported_epoch":130,"state":"peering","last_fresh":"2024-09-04T17:48:10.253927+0000","last_change":"2024-09-04T17:48:10.839758+0000","last_active":"2024-09-04T17:48:10.839758+0000","last_peered":"2024-09-04T17:48:10.253927+0000","last_clean":"2024-09-04T17:48:06.888233+0000","last_became_active":"2024-09-04T17:48:09.878998+0000","last_became_peered":"2024-09-04T17:48:09.878998+0000","last_unstale":"2024-09-04T17:48:10.253927+0000","last_undegraded":"2024-09-04T17:48:10.253927+0000","last_fullsized":"2024-09-04T17:48:09.837030+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0052483269999999997,"stat_sum":{"num_bytes":87377265,"num_objects":81,"num_object_clones":18,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":41,"num_read":2920,"num_read_kb":40212,"num_write":602,"num_write_kb":23345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":240,"num_bytes_recovered":21297737,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,5],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2a","length":"4"}]},{"pgid":"2.0","version":"0'0","reported_seq":242,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-04T17:48:09.854561+0000","last_change":"2024-09-04T17:47:29.804429+0000","last_active":"2024-09-04T17:48:09.854561+0000","last_peered":"2024-09-04T17:48:09.854561+0000","last_clean":"2024-09-04T17:48:09.854561+0000","last_became_active":"2024-09-04T17:47:29.803992+0000","last_became_peered":"2024-09-04T17:47:29.803992+0000","last_unstale":"2024-09-04T17:48:09.854561+0000","last_undegraded":"2024-09-04T17:48:09.854561+0000","last_fullsized":"2024-09-04T17:48:09.854561+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:50:32.563345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010553590000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"126'525","reported_seq":2255,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.845194+0000","last_change":"2024-09-04T17:48:09.844640+0000","last_active":"2024-09-04T17:48:10.845194+0000","last_peered":"2024-09-04T17:48:10.845194+0000","last_clean":"2024-09-04T17:48:10.845194+0000","last_became_active":"2024-09-04T17:48:01.850400+0000","last_became_peered":"2024-09-04T17:48:01.850400+0000","last_unstale":"2024-09-04T17:48:10.845194+0000","last_undegraded":"2024-09-04T17:48:10.845194+0000","last_fullsized":"2024-09-04T17:48:10.845194+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T18:01:48.725657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049494070000000003,"stat_sum":{"num_bytes":50198348,"num_objects":64,"num_object_clones":9,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":33,"num_read":5233,"num_read_kb":29752,"num_write":673,"num_write_kb":13801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":13799490,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"2.3","version":"20'1","reported_seq":249,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.845299+0000","last_change":"2024-09-04T17:47:41.919544+0000","last_active":"2024-09-04T17:48:10.845299+0000","last_peered":"2024-09-04T17:48:10.845299+0000","last_clean":"2024-09-04T17:48:10.845299+0000","last_became_active":"2024-09-04T17:47:41.919027+0000","last_became_peered":"2024-09-04T17:47:41.919027+0000","last_unstale":"2024-09-04T17:48:10.845299+0000","last_undegraded":"2024-09-04T17:48:10.845299+0000","last_fullsized":"2024-09-04T17:48:10.845299+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:42:55.688562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086618299999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":327,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-04T17:48:09.855144+0000","last_change":"2024-09-04T17:45:42.341149+0000","last_active":"2024-09-04T17:48:09.855144+0000","last_peered":"2024-09-04T17:48:09.855144+0000","last_clean":"2024-09-04T17:48:09.855144+0000","last_became_active":"2024-09-04T17:45:42.339286+0000","last_became_peered":"2024-09-04T17:45:42.339286+0000","last_unstale":"2024-09-04T17:48:09.855144+0000","last_undegraded":"2024-09-04T17:48:09.855144+0000","last_fullsized":"2024-09-04T17:48:09.855144+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:41.305128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:41.305128+0000","last_clean_scrub_stamp":"2024-09-04T17:45:41.305128+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:52:24.860049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"126'483","reported_seq":2668,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-04T17:48:09.855504+0000","last_change":"2024-09-04T17:48:09.857214+0000","last_active":"2024-09-04T17:48:09.855504+0000","last_peered":"2024-09-04T17:48:09.855504+0000","last_clean":"2024-09-04T17:48:09.855504+0000","last_became_active":"2024-09-04T17:47:32.847682+0000","last_became_peered":"2024-09-04T17:47:32.847682+0000","last_unstale":"2024-09-04T17:48:09.855504+0000","last_undegraded":"2024-09-04T17:48:09.855504+0000","last_fullsized":"2024-09-04T17:48:09.855504+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":483,"log_dups_size":0,"ondisk_log_size":483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:07:21.110914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002703474,"stat_sum":{"num_bytes":45080057,"num_objects":72,"num_object_clones":11,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":40,"num_read":2705,"num_read_kb":17207,"num_write":531,"num_write_kb":29364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":14692620,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.9","version":"129'591","reported_seq":4115,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-04T17:48:09.835133+0000","last_change":"2024-09-04T17:48:09.836181+0000","last_active":"2024-09-04T17:48:09.835133+0000","last_peered":"2024-09-04T17:48:09.835133+0000","last_clean":"2024-09-04T17:48:09.835133+0000","last_became_active":"2024-09-04T17:47:44.949630+0000","last_became_peered":"2024-09-04T17:47:44.949630+0000","last_unstale":"2024-09-04T17:48:09.835133+0000","last_undegraded":"2024-09-04T17:48:09.835133+0000","last_fullsized":"2024-09-04T17:48:09.835133+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:20:22.482600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001705781,"stat_sum":{"num_bytes":55724561,"num_objects":78,"num_object_clones":14,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":42,"num_read":5692,"num_read_kb":37073,"num_write":680,"num_write_kb":18627,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":4534479,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.a","version":"131'533","reported_seq":3019,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.890923+0000","last_change":"2024-09-04T17:48:09.853469+0000","last_active":"2024-09-04T17:48:10.890923+0000","last_peered":"2024-09-04T17:48:10.890923+0000","last_clean":"2024-09-04T17:48:10.890923+0000","last_became_active":"2024-09-04T17:48:00.841070+0000","last_became_peered":"2024-09-04T17:48:00.841070+0000","last_unstale":"2024-09-04T17:48:10.890923+0000","last_undegraded":"2024-09-04T17:48:10.890923+0000","last_fullsized":"2024-09-04T17:48:10.890923+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:47:07.864899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038466329999999999,"stat_sum":{"num_bytes":36655948,"num_objects":72,"num_object_clones":11,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":40,"num_read":5202,"num_read_kb":16044,"num_write":529,"num_write_kb":2741,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":1572433,"num_keys_recovered":66,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.b","version":"125'677","reported_seq":2207,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.840785+0000","last_change":"2024-09-04T17:48:09.840108+0000","last_active":"2024-09-04T17:48:10.840785+0000","last_peered":"2024-09-04T17:48:10.840785+0000","last_clean":"2024-09-04T17:48:10.840785+0000","last_became_active":"2024-09-04T17:47:08.657286+0000","last_became_peered":"2024-09-04T17:47:08.657286+0000","last_unstale":"2024-09-04T17:48:10.840785+0000","last_undegraded":"2024-09-04T17:48:10.840785+0000","last_fullsized":"2024-09-04T17:48:10.840785+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":677,"log_dups_size":0,"ondisk_log_size":677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T19:10:08.386664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023202159999999999,"stat_sum":{"num_bytes":31620378,"num_objects":71,"num_object_clones":10,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":33,"num_read":3911,"num_read_kb":14517,"num_write":845,"num_write_kb":7911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":3759450,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.c","version":"129'552","reported_seq":7797,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.845669+0000","last_change":"2024-09-04T17:48:09.845373+0000","last_active":"2024-09-04T17:48:10.845669+0000","last_peered":"2024-09-04T17:48:10.845669+0000","last_clean":"2024-09-04T17:48:10.845669+0000","last_became_active":"2024-09-04T17:48:02.855127+0000","last_became_peered":"2024-09-04T17:48:02.855127+0000","last_unstale":"2024-09-04T17:48:10.845669+0000","last_undegraded":"2024-09-04T17:48:10.845669+0000","last_fullsized":"2024-09-04T17:48:10.845669+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":552,"log_dups_size":0,"ondisk_log_size":552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:49:26.298655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035566629999999998,"stat_sum":{"num_bytes":46296333,"num_objects":74,"num_object_clones":11,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":43,"num_read":8646,"num_read_kb":71089,"num_write":690,"num_write_kb":16847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":153,"num_bytes_recovered":7666524,"num_keys_recovered":34,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.d","version":"126'1282","reported_seq":3848,"reported_epoch":130,"state":"remapped+peering","last_fresh":"2024-09-04T17:48:09.855528+0000","last_change":"2024-09-04T17:48:09.855528+0000","last_active":"2024-09-04T17:48:08.833037+0000","last_peered":"2024-09-04T17:48:07.832005+0000","last_clean":"2024-09-04T17:48:07.832005+0000","last_became_active":"2024-09-04T17:48:00.840986+0000","last_became_peered":"2024-09-04T17:48:00.840986+0000","last_unstale":"2024-09-04T17:48:09.855528+0000","last_undegraded":"2024-09-04T17:48:09.855528+0000","last_fullsized":"2024-09-04T17:48:09.855528+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":1282,"log_dups_size":0,"ondisk_log_size":1282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018889099999999999,"stat_sum":{"num_bytes":49889564,"num_objects":60,"num_object_clones":6,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":34,"num_read":8526,"num_read_kb":26249,"num_write":2192,"num_write_kb":13182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":8914239,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2a","length":"4"}]},{"pgid":"3.e","version":"127'440","reported_seq":5206,"reported_epoch":130,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-04T17:48:09.886153+0000","last_change":"2024-09-04T17:48:09.886153+0000","last_active":"2024-09-04T17:48:09.886153+0000","last_peered":"2024-09-04T17:48:09.886153+0000","last_clean":"2024-09-04T17:48:06.890201+0000","last_became_active":"2024-09-04T17:48:09.884964+0000","last_became_peered":"2024-09-04T17:48:09.884964+0000","last_unstale":"2024-09-04T17:48:09.886153+0000","last_undegraded":"2024-09-04T17:48:09.857903+0000","last_fullsized":"2024-09-04T17:48:09.854505+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.010017414000000001,"stat_sum":{"num_bytes":63389770,"num_objects":66,"num_object_clones":13,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":118,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":32,"num_read":6129,"num_read_kb":28040,"num_write":457,"num_write_kb":11211,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":3926000,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7],"avail_no_missing":["7","0"],"object_location_counts":[{"shards":"0,7","objects":66}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2a","length":"4"}]},{"pgid":"3.f","version":"130'292","reported_seq":2844,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-04T17:48:09.870980+0000","last_change":"2024-09-04T17:48:09.870980+0000","last_active":"2024-09-04T17:48:09.870980+0000","last_peered":"2024-09-04T17:48:09.870980+0000","last_clean":"2024-09-04T17:48:09.870980+0000","last_became_active":"2024-09-04T17:45:55.956734+0000","last_became_peered":"2024-09-04T17:45:55.956734+0000","last_unstale":"2024-09-04T17:48:09.870980+0000","last_undegraded":"2024-09-04T17:48:09.870980+0000","last_fullsized":"2024-09-04T17:48:09.870980+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":292,"log_dups_size":0,"ondisk_log_size":292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T23:59:11.397371+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.01496131,"stat_sum":{"num_bytes":49550709,"num_objects":69,"num_object_clones":12,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":36,"num_read":2563,"num_read_kb":44941,"num_write":297,"num_write_kb":13423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.10","version":"126'514","reported_seq":3148,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.842877+0000","last_change":"2024-09-04T17:48:09.851750+0000","last_active":"2024-09-04T17:48:10.842877+0000","last_peered":"2024-09-04T17:48:10.842877+0000","last_clean":"2024-09-04T17:48:10.842877+0000","last_became_active":"2024-09-04T17:48:00.843136+0000","last_became_peered":"2024-09-04T17:48:00.843136+0000","last_unstale":"2024-09-04T17:48:10.842877+0000","last_undegraded":"2024-09-04T17:48:10.842877+0000","last_fullsized":"2024-09-04T17:48:10.842877+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":514,"log_dups_size":0,"ondisk_log_size":514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T23:07:02.271876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049468649999999999,"stat_sum":{"num_bytes":51575959,"num_objects":78,"num_object_clones":10,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":46,"num_read":4069,"num_read_kb":18348,"num_write":591,"num_write_kb":12832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":3377599,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.11","version":"130'500","reported_seq":5739,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-04T17:48:09.877129+0000","last_change":"2024-09-04T17:48:09.879368+0000","last_active":"2024-09-04T17:48:09.877129+0000","last_peered":"2024-09-04T17:48:09.877129+0000","last_clean":"2024-09-04T17:48:09.877129+0000","last_became_active":"2024-09-04T17:45:55.951870+0000","last_became_peered":"2024-09-04T17:45:55.951870+0000","last_unstale":"2024-09-04T17:48:09.877129+0000","last_undegraded":"2024-09-04T17:48:09.877129+0000","last_fullsized":"2024-09-04T17:48:09.877129+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T22:56:47.076439+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.024746269000000001,"stat_sum":{"num_bytes":74079184,"num_objects":82,"num_object_clones":10,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":46,"num_read":6681,"num_read_kb":61026,"num_write":514,"num_write_kb":17560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.12","version":"130'349","reported_seq":3430,"reported_epoch":130,"state":"peering","last_fresh":"2024-09-04T17:48:09.930543+0000","last_change":"2024-09-04T17:48:10.839711+0000","last_active":"2024-09-04T17:48:10.839711+0000","last_peered":"2024-09-04T17:48:09.930543+0000","last_clean":"2024-09-04T17:48:09.930543+0000","last_became_active":"2024-09-04T17:47:44.952109+0000","last_became_peered":"2024-09-04T17:47:44.952109+0000","last_unstale":"2024-09-04T17:48:09.930543+0000","last_undegraded":"2024-09-04T17:48:09.930543+0000","last_fullsized":"2024-09-04T17:48:09.930543+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017447070000000001,"stat_sum":{"num_bytes":29554039,"num_objects":60,"num_object_clones":5,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":35,"num_read":3177,"num_read_kb":21378,"num_write":389,"num_write_kb":8559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":1762027,"num_keys_recovered":48,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,7],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.13","version":"126'264","reported_seq":2227,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-04T17:48:09.857330+0000","last_change":"2024-09-04T17:48:09.860061+0000","last_active":"2024-09-04T17:48:09.857330+0000","last_peered":"2024-09-04T17:48:09.857330+0000","last_clean":"2024-09-04T17:48:09.857330+0000","last_became_active":"2024-09-04T17:45:55.954544+0000","last_became_peered":"2024-09-04T17:45:55.954544+0000","last_unstale":"2024-09-04T17:48:09.857330+0000","last_undegraded":"2024-09-04T17:48:09.857330+0000","last_fullsized":"2024-09-04T17:48:09.857330+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":264,"log_dups_size":0,"ondisk_log_size":264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T23:14:42.541437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0051521240000000001,"stat_sum":{"num_bytes":25183962,"num_objects":62,"num_object_clones":5,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":39,"num_read":2181,"num_read_kb":22538,"num_write":216,"num_write_kb":4374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"29","length":"5"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1555140969,"num_objects":2110,"num_object_clones":283,"num_object_copies":4220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":118,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2110,"num_whiteouts":1142,"num_read":131871,"num_read_kb":867857,"num_write":18282,"num_write_kb":403769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3176,"num_bytes_recovered":196192242,"num_keys_recovered":606,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":502431744,"data_stored":611754297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21395,"internal_metadata":0},"log_size":15477,"ondisk_log_size":15477,"up":64,"acting":62,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":117,"seq":502511173636,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":132212,"kb_used_data":98456,"kb_used_omap":22,"kb_used_meta":33705,"kb_avail":94239628,"statfs":{"total":96636764160,"available":96501379072,"internally_reserved":0,"allocated":100818944,"data_stored":103901506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22742,"internal_metadata":34514730},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509473,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":106128,"kb_used_data":79096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94265712,"statfs":{"total":96636764160,"available":96528089088,"internally_reserved":0,"allocated":80994304,"data_stored":99363602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509474,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":172236,"kb_used_data":114672,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94199604,"statfs":{"total":96636764160,"available":96460394496,"internally_reserved":0,"allocated":117424128,"data_stored":147072672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":0,"seq":206158430229,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":64424509473,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509473,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":124568,"kb_used_data":75328,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94247272,"statfs":{"total":96636764160,"available":96509206528,"internally_reserved":0,"allocated":77135872,"data_stored":95452503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542177,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":137620,"kb_used_data":84216,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94234220,"statfs":{"total":96636764160,"available":96495841280,"internally_reserved":0,"allocated":86237184,"data_stored":111513171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":103,"seq":442381631495,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":85736,"kb_used_data":45588,"kb_used_omap":12,"kb_used_meta":40115,"kb_avail":94286104,"statfs":{"total":96636764160,"available":96548970496,"internally_reserved":0,"allocated":46682112,"data_stored":57018531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12804,"internal_metadata":41078268},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":74981376,"data_stored":94116394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2035712,"data_stored":2023494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45027328,"data_stored":56174527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6121,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":7245824,"data_stored":9302332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":115863552,"data_stored":146326931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":81965056,"data_stored":101128065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":99254272,"data_stored":103158122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15274,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":76058624,"data_stored":99524432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-04T17:48:12.355 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:12.355 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:48:12.355 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:48:12.355 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:48:12.355 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.355 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59950 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5365760~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:48:12.355 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654917+0000", 2024-09-04T17:48:12.355 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 58.699141976, 2024-09-04T17:48:12.355 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066822450000000005, 2024-09-04T17:48:12.355 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:12.356 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:12.356 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:12.356 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:12.356 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:12.356 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:12.356 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59950 2024-09-04T17:48:12.356 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.356 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:12.356 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.356 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:12.356 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654917+0000", 2024-09-04T17:48:12.356 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:12.356 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.357 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:12.357 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654917+0000", 2024-09-04T17:48:12.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:12.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.357 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:12.357 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654918+0000", 2024-09-04T17:48:12.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.29e-07 2024-09-04T17:48:12.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.357 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:12.357 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654919+0000", 2024-09-04T17:48:12.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2589999999999999e-06 2024-09-04T17:48:12.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:12.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654922+0000", 2024-09-04T17:48:12.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1350000000000001e-06 2024-09-04T17:48:12.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:12.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654925+0000", 2024-09-04T17:48:12.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1250000000000001e-06 2024-09-04T17:48:12.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:12.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721466+0000", 2024-09-04T17:48:12.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066540959999999996 2024-09-04T17:48:12.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:12.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721574+0000", 2024-09-04T17:48:12.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010747399999999999 2024-09-04T17:48:12.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:12.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721740+0000", 2024-09-04T17:48:12.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016626799999999999 2024-09-04T17:48:12.359 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:12.360 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:12.360 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:12.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.360 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59951 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5472256~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:48:12.360 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654929+0000", 2024-09-04T17:48:12.360 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 58.699130791000002, 2024-09-04T17:48:12.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067110534999999999, 2024-09-04T17:48:12.360 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:12.360 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:12.360 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:12.360 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:12.360 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:12.360 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:12.361 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59951 2024-09-04T17:48:12.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.361 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:12.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:12.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654929+0000", 2024-09-04T17:48:12.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:12.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:12.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654929+0000", 2024-09-04T17:48:12.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:12.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.362 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:12.362 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654929+0000", 2024-09-04T17:48:12.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5100000000000001e-07 2024-09-04T17:48:12.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.362 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:12.362 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654930+0000", 2024-09-04T17:48:12.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2440000000000001e-06 2024-09-04T17:48:12.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.362 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:12.362 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654933+0000", 2024-09-04T17:48:12.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1599999999999998e-06 2024-09-04T17:48:12.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:12.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654936+0000", 2024-09-04T17:48:12.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6560000000000001e-06 2024-09-04T17:48:12.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:12.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721789+0000", 2024-09-04T17:48:12.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066853250000000003 2024-09-04T17:48:12.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:12.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721885+0000", 2024-09-04T17:48:12.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5616000000000001e-05 2024-09-04T17:48:12.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:12.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722039+0000", 2024-09-04T17:48:12.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000154258 2024-09-04T17:48:12.364 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:12.364 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:12.364 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:12.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.364 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59952 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5505024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:48:12.364 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654939+0000", 2024-09-04T17:48:12.365 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 58.699119914000001, 2024-09-04T17:48:12.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067433742000000005, 2024-09-04T17:48:12.365 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:12.365 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:12.365 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:12.365 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:12.365 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:12.365 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:12.365 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59952 2024-09-04T17:48:12.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.365 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:12.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:12.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654939+0000", 2024-09-04T17:48:12.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:12.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:12.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654939+0000", 2024-09-04T17:48:12.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:12.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:12.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654940+0000", 2024-09-04T17:48:12.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3300000000000001e-07 2024-09-04T17:48:12.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:12.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654941+0000", 2024-09-04T17:48:12.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1570000000000001e-06 2024-09-04T17:48:12.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:12.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654944+0000", 2024-09-04T17:48:12.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1319999999999998e-06 2024-09-04T17:48:12.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:12.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654947+0000", 2024-09-04T17:48:12.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5969999999999999e-06 2024-09-04T17:48:12.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:12.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722089+0000", 2024-09-04T17:48:12.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067142347000000005 2024-09-04T17:48:12.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:12.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722201+0000", 2024-09-04T17:48:12.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011235600000000001 2024-09-04T17:48:12.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:12.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722373+0000", 2024-09-04T17:48:12.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017191999999999999 2024-09-04T17:48:12.369 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:12.369 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:12.369 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:12.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.369 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59953 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5611520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:48:12.369 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654950+0000", 2024-09-04T17:48:12.369 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 58.699109274000001, 2024-09-04T17:48:12.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067601852000000004, 2024-09-04T17:48:12.369 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:12.369 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:12.369 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:12.369 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:12.370 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:12.370 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:12.370 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59953 2024-09-04T17:48:12.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.370 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:12.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:12.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654950+0000", 2024-09-04T17:48:12.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:12.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:12.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654950+0000", 2024-09-04T17:48:12.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:12.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:12.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654950+0000", 2024-09-04T17:48:12.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2499999999999999e-07 2024-09-04T17:48:12.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:12.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654954+0000", 2024-09-04T17:48:12.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.287e-06 2024-09-04T17:48:12.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:12.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654962+0000", 2024-09-04T17:48:12.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4230000000000005e-06 2024-09-04T17:48:12.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:12.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654966+0000", 2024-09-04T17:48:12.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6449999999999998e-06 2024-09-04T17:48:12.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:12.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722417+0000", 2024-09-04T17:48:12.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067451188999999995 2024-09-04T17:48:12.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:12.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722467+0000", 2024-09-04T17:48:12.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0556999999999997e-05 2024-09-04T17:48:12.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:12.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722552+0000", 2024-09-04T17:48:12.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4525999999999997e-05 2024-09-04T17:48:12.373 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:12.373 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:12.373 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:12.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.373 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59954 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5718016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:48:12.374 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654970+0000", 2024-09-04T17:48:12.374 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 58.699089512999997, 2024-09-04T17:48:12.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067874285000000006, 2024-09-04T17:48:12.374 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:12.374 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:12.374 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:12.374 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:12.374 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:12.374 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:12.374 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59954 2024-09-04T17:48:12.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.374 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:12.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:12.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654970+0000", 2024-09-04T17:48:12.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:12.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:12.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654970+0000", 2024-09-04T17:48:12.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:12.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:12.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654970+0000", 2024-09-04T17:48:12.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5899999999999998e-07 2024-09-04T17:48:12.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:12.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654971+0000", 2024-09-04T17:48:12.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2929999999999999e-06 2024-09-04T17:48:12.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:12.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654975+0000", 2024-09-04T17:48:12.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4699999999999998e-06 2024-09-04T17:48:12.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:12.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654978+0000", 2024-09-04T17:48:12.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7930000000000002e-06 2024-09-04T17:48:12.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:12.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722600+0000", 2024-09-04T17:48:12.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067622229000000006 2024-09-04T17:48:12.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:12.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722689+0000", 2024-09-04T17:48:12.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9511000000000004e-05 2024-09-04T17:48:12.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:12.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722844+0000", 2024-09-04T17:48:12.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015473 2024-09-04T17:48:12.378 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:12.378 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:12.378 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:12.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.378 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59955 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5857280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:48:12.378 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654981+0000", 2024-09-04T17:48:12.378 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 58.699078163000003, 2024-09-04T17:48:12.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068052360000000006, 2024-09-04T17:48:12.378 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:12.378 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:12.378 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:12.378 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:12.379 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:12.379 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:12.379 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59955 2024-09-04T17:48:12.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.379 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:12.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:12.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654981+0000", 2024-09-04T17:48:12.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:12.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:12.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654981+0000", 2024-09-04T17:48:12.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:12.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:12.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654981+0000", 2024-09-04T17:48:12.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2399999999999999e-07 2024-09-04T17:48:12.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:12.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654985+0000", 2024-09-04T17:48:12.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0980000000000004e-06 2024-09-04T17:48:12.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:12.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654990+0000", 2024-09-04T17:48:12.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3429999999999998e-06 2024-09-04T17:48:12.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:12.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654993+0000", 2024-09-04T17:48:12.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8700000000000001e-06 2024-09-04T17:48:12.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:12.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722888+0000", 2024-09-04T17:48:12.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067895239999999996 2024-09-04T17:48:12.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:12.382 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722948+0000", 2024-09-04T17:48:12.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9750999999999998e-05 2024-09-04T17:48:12.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:12.382 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.723034+0000", 2024-09-04T17:48:12.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5834000000000005e-05 2024-09-04T17:48:12.382 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:12.382 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:12.382 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:12.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.382 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59956 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5963776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:48:12.382 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654998+0000", 2024-09-04T17:48:12.383 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 58.699061634000003, 2024-09-04T17:48:12.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068359518999999994, 2024-09-04T17:48:12.383 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:12.383 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:12.383 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:12.383 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:12.383 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:12.383 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:12.383 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59956 2024-09-04T17:48:12.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.383 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:12.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:12.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654998+0000", 2024-09-04T17:48:12.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:12.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:12.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654998+0000", 2024-09-04T17:48:12.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:12.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:12.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654998+0000", 2024-09-04T17:48:12.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4299999999999999e-07 2024-09-04T17:48:12.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:12.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654999+0000", 2024-09-04T17:48:12.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1799999999999999e-06 2024-09-04T17:48:12.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:12.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.655002+0000", 2024-09-04T17:48:12.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3639999999999999e-06 2024-09-04T17:48:12.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:12.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.655005+0000", 2024-09-04T17:48:12.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7970000000000001e-06 2024-09-04T17:48:12.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:12.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.723085+0000", 2024-09-04T17:48:12.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068080229000000006 2024-09-04T17:48:12.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:12.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.723197+0000", 2024-09-04T17:48:12.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000111344 2024-09-04T17:48:12.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:12.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.723357+0000", 2024-09-04T17:48:12.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016036199999999999 2024-09-04T17:48:12.387 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:12.387 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:12.387 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:12.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.387 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61082 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 2654208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:12.387 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.822664+0000", 2024-09-04T17:48:12.387 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 58.531395246999999, 2024-09-04T17:48:12.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067072242000000004, 2024-09-04T17:48:12.387 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:12.388 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:12.388 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:12.388 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:12.388 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:12.388 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:12.388 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61082 2024-09-04T17:48:12.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.388 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:12.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:12.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822664+0000", 2024-09-04T17:48:12.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:12.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:12.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822664+0000", 2024-09-04T17:48:12.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:12.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:12.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822664+0000", 2024-09-04T17:48:12.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6600000000000003e-07 2024-09-04T17:48:12.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:12.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822666+0000", 2024-09-04T17:48:12.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.536e-06 2024-09-04T17:48:12.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:12.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822669+0000", 2024-09-04T17:48:12.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3759999999999999e-06 2024-09-04T17:48:12.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:12.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822672+0000", 2024-09-04T17:48:12.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8109999999999999e-06 2024-09-04T17:48:12.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:12.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889263+0000", 2024-09-04T17:48:12.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066591259999999999 2024-09-04T17:48:12.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:12.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889369+0000", 2024-09-04T17:48:12.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000105362 2024-09-04T17:48:12.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:12.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889736+0000", 2024-09-04T17:48:12.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00036763099999999999 2024-09-04T17:48:12.391 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:12.392 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:12.392 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:12.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.392 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61083 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 2760704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:12.392 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.822678+0000", 2024-09-04T17:48:12.392 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 58.531381512999999, 2024-09-04T17:48:12.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067475996999999996, 2024-09-04T17:48:12.392 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:12.392 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:12.392 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:12.392 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:12.392 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:12.393 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:12.393 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61083 2024-09-04T17:48:12.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.393 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:12.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:12.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822678+0000", 2024-09-04T17:48:12.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:12.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:12.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822678+0000", 2024-09-04T17:48:12.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:12.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:12.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822725+0000", 2024-09-04T17:48:12.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7429000000000003e-05 2024-09-04T17:48:12.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:12.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822728+0000", 2024-09-04T17:48:12.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2140000000000001e-06 2024-09-04T17:48:12.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:12.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822737+0000", 2024-09-04T17:48:12.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4370000000000007e-06 2024-09-04T17:48:12.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:12.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822743+0000", 2024-09-04T17:48:12.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7520000000000002e-06 2024-09-04T17:48:12.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:12.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889785+0000", 2024-09-04T17:48:12.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067042265000000004 2024-09-04T17:48:12.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:12.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889840+0000", 2024-09-04T17:48:12.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5254000000000003e-05 2024-09-04T17:48:12.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:12.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.890154+0000", 2024-09-04T17:48:12.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000313646 2024-09-04T17:48:12.396 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:12.397 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:12.397 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:12.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.397 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61084 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 2867200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:12.397 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.822753+0000", 2024-09-04T17:48:12.397 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 58.531306577000002, 2024-09-04T17:48:12.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068497161000000001, 2024-09-04T17:48:12.398 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:12.398 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:12.398 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:12.398 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:12.398 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:12.398 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:12.398 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61084 2024-09-04T17:48:12.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.398 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:12.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:12.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822753+0000", 2024-09-04T17:48:12.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:12.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:12.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822753+0000", 2024-09-04T17:48:12.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:12.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:12.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822753+0000", 2024-09-04T17:48:12.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9100000000000003e-07 2024-09-04T17:48:12.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:12.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822756+0000", 2024-09-04T17:48:12.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.678e-06 2024-09-04T17:48:12.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:12.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822763+0000", 2024-09-04T17:48:12.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8220000000000001e-06 2024-09-04T17:48:12.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:12.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822768+0000", 2024-09-04T17:48:12.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1930000000000001e-06 2024-09-04T17:48:12.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:12.401 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.890246+0000", 2024-09-04T17:48:12.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067477996999999998 2024-09-04T17:48:12.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:12.401 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.890555+0000", 2024-09-04T17:48:12.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000309143 2024-09-04T17:48:12.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:12.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.891250+0000", 2024-09-04T17:48:12.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00069463700000000001 2024-09-04T17:48:12.402 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:12.402 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:12.402 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:12.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.402 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61285 3.8 3:117cfeb9:::rbd_data.11341dfde0c5.0000000000000006:1f [read 2760704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:12.402 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835200+0000", 2024-09-04T17:48:12.402 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 58.518859509999999, 2024-09-04T17:48:12.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066899803999999993, 2024-09-04T17:48:12.402 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:12.403 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:12.403 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:12.403 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:12.403 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:12.403 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:12.403 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61285 2024-09-04T17:48:12.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.403 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:12.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:12.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835200+0000", 2024-09-04T17:48:12.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:12.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:12.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835200+0000", 2024-09-04T17:48:12.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:12.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:12.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835200+0000", 2024-09-04T17:48:12.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0899999999999997e-07 2024-09-04T17:48:12.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:12.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835206+0000", 2024-09-04T17:48:12.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3280000000000002e-06 2024-09-04T17:48:12.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:12.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835211+0000", 2024-09-04T17:48:12.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9230000000000001e-06 2024-09-04T17:48:12.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:12.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835216+0000", 2024-09-04T17:48:12.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9019999999999998e-06 2024-09-04T17:48:12.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:12.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.901719+0000", 2024-09-04T17:48:12.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066502981000000003 2024-09-04T17:48:12.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:12.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.901813+0000", 2024-09-04T17:48:12.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3905000000000003e-05 2024-09-04T17:48:12.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:12.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902100+0000", 2024-09-04T17:48:12.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028645599999999998 2024-09-04T17:48:12.407 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:12.407 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:12.407 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:12.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.407 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61286 3.8 3:117cfeb9:::rbd_data.11341dfde0c5.0000000000000006:1f [read 2867200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:12.407 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835244+0000", 2024-09-04T17:48:12.407 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 58.518814890999998, 2024-09-04T17:48:12.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067143215000000006, 2024-09-04T17:48:12.407 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:12.407 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:12.407 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:12.407 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:12.408 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:12.408 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:12.408 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61286 2024-09-04T17:48:12.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.408 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:12.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:12.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835244+0000", 2024-09-04T17:48:12.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:12.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:12.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835244+0000", 2024-09-04T17:48:12.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:12.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:12.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835245+0000", 2024-09-04T17:48:12.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8400000000000004e-07 2024-09-04T17:48:12.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:12.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835248+0000", 2024-09-04T17:48:12.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6850000000000001e-06 2024-09-04T17:48:12.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:12.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835254+0000", 2024-09-04T17:48:12.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.494e-06 2024-09-04T17:48:12.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:12.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835260+0000", 2024-09-04T17:48:12.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0580000000000003e-06 2024-09-04T17:48:12.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:12.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902119+0000", 2024-09-04T17:48:12.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066858318 2024-09-04T17:48:12.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.411 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:12.411 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902163+0000", 2024-09-04T17:48:12.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4753000000000003e-05 2024-09-04T17:48:12.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.411 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:12.411 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902388+0000", 2024-09-04T17:48:12.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022422300000000001 2024-09-04T17:48:12.411 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:12.411 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:12.411 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:12.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.412 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61315 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 98304~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:12.412 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835269+0000", 2024-09-04T17:48:12.412 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 58.518790099, 2024-09-04T17:48:12.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067329396, 2024-09-04T17:48:12.412 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:12.412 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:12.412 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:12.412 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:12.412 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:12.412 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:12.412 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61315 2024-09-04T17:48:12.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.412 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:12.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:12.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835269+0000", 2024-09-04T17:48:12.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:12.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:12.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835269+0000", 2024-09-04T17:48:12.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:12.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:12.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835270+0000", 2024-09-04T17:48:12.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6599999999999996e-07 2024-09-04T17:48:12.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:12.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835272+0000", 2024-09-04T17:48:12.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.261e-06 2024-09-04T17:48:12.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:12.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835278+0000", 2024-09-04T17:48:12.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.443e-06 2024-09-04T17:48:12.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:12.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835284+0000", 2024-09-04T17:48:12.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3290000000000001e-06 2024-09-04T17:48:12.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:12.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902147+0000", 2024-09-04T17:48:12.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066862695999999999 2024-09-04T17:48:12.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:12.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902248+0000", 2024-09-04T17:48:12.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000101501 2024-09-04T17:48:12.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:12.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902599+0000", 2024-09-04T17:48:12.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00035060000000000001 2024-09-04T17:48:12.416 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:12.416 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:12.416 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:12.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.416 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61316 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 204800~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:12.416 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835291+0000", 2024-09-04T17:48:12.417 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 58.518768080000001, 2024-09-04T17:48:12.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067632052999999998, 2024-09-04T17:48:12.417 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:12.417 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:12.417 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:12.417 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:12.417 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:12.417 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:12.417 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61316 2024-09-04T17:48:12.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.417 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:12.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:12.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835291+0000", 2024-09-04T17:48:12.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:12.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:12.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835291+0000", 2024-09-04T17:48:12.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:12.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:12.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835292+0000", 2024-09-04T17:48:12.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1899999999999998e-07 2024-09-04T17:48:12.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:12.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835294+0000", 2024-09-04T17:48:12.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0880000000000002e-06 2024-09-04T17:48:12.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:12.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835300+0000", 2024-09-04T17:48:12.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.224e-06 2024-09-04T17:48:12.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:12.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835305+0000", 2024-09-04T17:48:12.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0730000000000004e-06 2024-09-04T17:48:12.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:12.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902632+0000", 2024-09-04T17:48:12.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067327322999999994 2024-09-04T17:48:12.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:12.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902688+0000", 2024-09-04T17:48:12.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5177999999999999e-05 2024-09-04T17:48:12.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:12.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902923+0000", 2024-09-04T17:48:12.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023574800000000001 2024-09-04T17:48:12.421 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:12.421 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:12.421 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:12.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.421 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61317 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 344064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:12.421 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835312+0000", 2024-09-04T17:48:12.421 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 58.518747075999997, 2024-09-04T17:48:12.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067918511000000001, 2024-09-04T17:48:12.421 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:12.421 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:12.422 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:12.422 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:12.422 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:12.422 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:12.422 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61317 2024-09-04T17:48:12.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.422 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:12.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:12.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835312+0000", 2024-09-04T17:48:12.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:12.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:12.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835312+0000", 2024-09-04T17:48:12.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:12.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:12.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835313+0000", 2024-09-04T17:48:12.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9000000000000002e-07 2024-09-04T17:48:12.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:12.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835315+0000", 2024-09-04T17:48:12.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1689999999999999e-06 2024-09-04T17:48:12.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:12.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835321+0000", 2024-09-04T17:48:12.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0619999999999997e-06 2024-09-04T17:48:12.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:12.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835326+0000", 2024-09-04T17:48:12.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0390000000000004e-06 2024-09-04T17:48:12.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:12.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902967+0000", 2024-09-04T17:48:12.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067641520999999996 2024-09-04T17:48:12.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:12.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903067+0000", 2024-09-04T17:48:12.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9426999999999995e-05 2024-09-04T17:48:12.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:12.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903231+0000", 2024-09-04T17:48:12.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016390299999999999 2024-09-04T17:48:12.425 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:12.425 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:12.426 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:12.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.426 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61318 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 450560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:12.426 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835334+0000", 2024-09-04T17:48:12.426 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 58.518725559000003, 2024-09-04T17:48:12.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068091771999999995, 2024-09-04T17:48:12.426 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:12.426 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:12.426 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:12.426 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:12.426 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:12.426 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:12.427 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61318 2024-09-04T17:48:12.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.427 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:12.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:12.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835334+0000", 2024-09-04T17:48:12.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:12.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:12.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835334+0000", 2024-09-04T17:48:12.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:12.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:12.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835334+0000", 2024-09-04T17:48:12.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5699999999999998e-07 2024-09-04T17:48:12.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:12.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835337+0000", 2024-09-04T17:48:12.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3889999999999999e-06 2024-09-04T17:48:12.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:12.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835343+0000", 2024-09-04T17:48:12.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1809999999999998e-06 2024-09-04T17:48:12.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:12.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835348+0000", 2024-09-04T17:48:12.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3170000000000001e-06 2024-09-04T17:48:12.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:12.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903270+0000", 2024-09-04T17:48:12.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067922191000000007 2024-09-04T17:48:12.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.430 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:12.430 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903321+0000", 2024-09-04T17:48:12.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0889e-05 2024-09-04T17:48:12.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.430 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:12.430 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903426+0000", 2024-09-04T17:48:12.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104348 2024-09-04T17:48:12.430 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:12.430 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:12.430 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:12.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.431 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61319 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 589824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:12.431 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835357+0000", 2024-09-04T17:48:12.431 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 58.518702742000002, 2024-09-04T17:48:12.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068372535999999998, 2024-09-04T17:48:12.431 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:12.431 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:12.431 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:12.431 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:12.431 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:12.431 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:12.431 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61319 2024-09-04T17:48:12.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.431 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:12.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:12.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835357+0000", 2024-09-04T17:48:12.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:12.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:12.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835357+0000", 2024-09-04T17:48:12.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:12.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:12.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835357+0000", 2024-09-04T17:48:12.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.32e-07 2024-09-04T17:48:12.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:12.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835359+0000", 2024-09-04T17:48:12.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1660000000000001e-06 2024-09-04T17:48:12.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:12.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835366+0000", 2024-09-04T17:48:12.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3550000000000001e-06 2024-09-04T17:48:12.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.434 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:12.434 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835371+0000", 2024-09-04T17:48:12.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1050000000000003e-06 2024-09-04T17:48:12.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.434 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:12.434 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903471+0000", 2024-09-04T17:48:12.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068100451000000006 2024-09-04T17:48:12.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.434 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:12.434 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903573+0000", 2024-09-04T17:48:12.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000101691 2024-09-04T17:48:12.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:12.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903729+0000", 2024-09-04T17:48:12.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000156336 2024-09-04T17:48:12.435 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:12.435 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:12.435 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:12.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.435 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61320 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:12.435 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835380+0000", 2024-09-04T17:48:12.436 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 58.518678977, 2024-09-04T17:48:12.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068543058000000004, 2024-09-04T17:48:12.436 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:12.436 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:12.436 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:12.436 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:12.436 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:12.436 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:12.436 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61320 2024-09-04T17:48:12.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.436 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:12.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:12.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835380+0000", 2024-09-04T17:48:12.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:12.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:12.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835380+0000", 2024-09-04T17:48:12.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:12.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:12.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835381+0000", 2024-09-04T17:48:12.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9999999999999998e-07 2024-09-04T17:48:12.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:12.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835383+0000", 2024-09-04T17:48:12.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2230000000000001e-06 2024-09-04T17:48:12.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:12.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835390+0000", 2024-09-04T17:48:12.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.34e-06 2024-09-04T17:48:12.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:12.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835396+0000", 2024-09-04T17:48:12.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9830000000000001e-06 2024-09-04T17:48:12.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.439 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:12.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903765+0000", 2024-09-04T17:48:12.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068368441000000002 2024-09-04T17:48:12.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.439 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:12.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903835+0000", 2024-09-04T17:48:12.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0631999999999997e-05 2024-09-04T17:48:12.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:12.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903923+0000", 2024-09-04T17:48:12.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8039000000000002e-05 2024-09-04T17:48:12.440 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:12.440 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:12.440 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:12.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.440 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61321 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 835584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:12.440 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835404+0000", 2024-09-04T17:48:12.440 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 58.518654918000003, 2024-09-04T17:48:12.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068911043000000005, 2024-09-04T17:48:12.440 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:12.441 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:12.441 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:12.441 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:12.441 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:12.441 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:12.441 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61321 2024-09-04T17:48:12.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.441 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:12.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:12.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835404+0000", 2024-09-04T17:48:12.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:12.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:12.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835404+0000", 2024-09-04T17:48:12.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:12.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:12.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835405+0000", 2024-09-04T17:48:12.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.89e-07 2024-09-04T17:48:12.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:12.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835407+0000", 2024-09-04T17:48:12.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4439999999999998e-06 2024-09-04T17:48:12.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:12.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835414+0000", 2024-09-04T17:48:12.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4930000000000003e-06 2024-09-04T17:48:12.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:12.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835419+0000", 2024-09-04T17:48:12.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8919999999999999e-06 2024-09-04T17:48:12.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:12.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903957+0000", 2024-09-04T17:48:12.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068538500000000002 2024-09-04T17:48:12.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:12.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904019+0000", 2024-09-04T17:48:12.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1419000000000003e-05 2024-09-04T17:48:12.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:12.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904315+0000", 2024-09-04T17:48:12.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00029680599999999999 2024-09-04T17:48:12.445 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:12.445 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:12.445 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:12.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.445 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61322 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:12.445 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835427+0000", 2024-09-04T17:48:12.445 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 58.518632531999998, 2024-09-04T17:48:12.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.069550138999999997, 2024-09-04T17:48:12.445 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:12.445 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:12.445 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:12.445 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:12.446 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:12.446 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:12.446 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61322 2024-09-04T17:48:12.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.446 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:12.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:12.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835427+0000", 2024-09-04T17:48:12.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:12.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:12.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835427+0000", 2024-09-04T17:48:12.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:12.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:12.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.842157+0000", 2024-09-04T17:48:12.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067298419999999998 2024-09-04T17:48:12.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:12.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.842163+0000", 2024-09-04T17:48:12.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2029999999999997e-06 2024-09-04T17:48:12.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:12.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.842175+0000", 2024-09-04T17:48:12.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2072000000000001e-05 2024-09-04T17:48:12.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:12.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.842183+0000", 2024-09-04T17:48:12.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3850000000000002e-06 2024-09-04T17:48:12.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:12.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904371+0000", 2024-09-04T17:48:12.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.062187754999999997 2024-09-04T17:48:12.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:12.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904467+0000", 2024-09-04T17:48:12.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6117000000000001e-05 2024-09-04T17:48:12.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:12.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:12.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:12.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904977+0000", 2024-09-04T17:48:12.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00050976500000000002 2024-09-04T17:48:12.449 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:12.449 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:12.449 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:12.450 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:12.450 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:12.450 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:12.451 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-04T17:48:12.452 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:12.448+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:12.552 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:48:12.548+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:48:12.583 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:12.584 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:48:12.584 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:48:12.584 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:12.598 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-04T17:48:12.653 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:12.650+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:12.718 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:12.718 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:48:12.718 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:48:12.718 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:48:12.718 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:12.732 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-04T17:48:12.754 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:12.752+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:12.855 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:12.850+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:12.860 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-04T17:48:12.915 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:12.915 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:48:12.915 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:48:12.915 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:48:12.915 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.916 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:12722 3.1b 3:d8152af1:::rbd_data.1134ef3a5a7d.00000000000000bd:head [delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-04T17:48:12.916 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:16.421965+0000", 2024-09-04T17:48:12.916 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 116.492634552, 2024-09-04T17:48:12.916 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.890002811, 2024-09-04T17:48:12.916 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:12.916 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:12.916 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:12.916 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:12.916 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:12.916 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:12.916 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 12722 2024-09-04T17:48:12.917 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.917 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:12.917 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.917 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:12.917 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421965+0000", 2024-09-04T17:48:12.917 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:12.917 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.917 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.917 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:12.917 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421965+0000", 2024-09-04T17:48:12.917 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:12.917 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.917 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.918 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:12.918 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421966+0000", 2024-09-04T17:48:12.918 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.102e-06 2024-09-04T17:48:12.918 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.918 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.918 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:12.918 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421971+0000", 2024-09-04T17:48:12.918 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9989999999999999e-06 2024-09-04T17:48:12.918 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.918 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.918 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:12.918 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421981+0000", 2024-09-04T17:48:12.918 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.7380000000000006e-06 2024-09-04T17:48:12.918 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.919 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.919 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:12.919 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421989+0000", 2024-09-04T17:48:12.919 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.1890000000000007e-06 2024-09-04T17:48:12.919 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.919 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.919 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:12.919 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.422050+0000", 2024-09-04T17:48:12.919 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.0702000000000003e-05 2024-09-04T17:48:12.919 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.919 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.919 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for peered", 2024-09-04T17:48:12.919 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.422059+0000", 2024-09-04T17:48:12.919 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5440000000000004e-06 2024-09-04T17:48:12.920 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.920 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.920 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:12.920 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.308156+0000", 2024-09-04T17:48:12.920 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.88609676199999998 2024-09-04T17:48:12.920 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.920 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.920 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:12.920 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.308398+0000", 2024-09-04T17:48:12.920 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024215400000000001 2024-09-04T17:48:12.920 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.920 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.920 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 7", 2024-09-04T17:48:12.921 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.308795+0000", 2024-09-04T17:48:12.921 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000396704 2024-09-04T17:48:12.921 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.921 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.921 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:48:12.921 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.310831+0000", 2024-09-04T17:48:12.921 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0020364200000000002 2024-09-04T17:48:12.921 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.921 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.921 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:12.921 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.311818+0000", 2024-09-04T17:48:12.922 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00098649799999999993 2024-09-04T17:48:12.922 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.922 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.922 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:12.922 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.311905+0000", 2024-09-04T17:48:12.922 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7070999999999994e-05 2024-09-04T17:48:12.922 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.922 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.922 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:12.922 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.311968+0000", 2024-09-04T17:48:12.922 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.2928000000000005e-05 2024-09-04T17:48:12.922 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:12.922 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:12.922 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:12.923 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.923 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.923 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37359 3.f e58/24 3:f29e50a5:::rbd_data.1134c5a43465.0000000000000021:head v 58'135, mlcod=58'133)", 2024-09-04T17:48:12.923 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:48:12.923 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 93.297303948000007, 2024-09-04T17:48:12.923 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053483197000000003, 2024-09-04T17:48:12.923 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:12.923 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:12.923 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:12.923 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:12.923 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.923 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:12.923 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:48:12.923 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:12.924 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.924 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.924 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:12.924 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:48:12.924 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:12.924 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.924 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.924 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:12.924 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:48:12.924 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1699999999999997e-07 2024-09-04T17:48:12.924 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.924 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.924 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:12.925 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617338+0000", 2024-09-04T17:48:12.925 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.1168999999999998e-05 2024-09-04T17:48:12.925 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.925 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.925 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:12.925 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617351+0000", 2024-09-04T17:48:12.925 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3094e-05 2024-09-04T17:48:12.925 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.925 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.925 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:12.925 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617359+0000", 2024-09-04T17:48:12.925 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7970000000000006e-06 2024-09-04T17:48:12.925 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.926 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.926 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:12.926 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617417+0000", 2024-09-04T17:48:12.926 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7079e-05 2024-09-04T17:48:12.926 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.926 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.926 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:12.926 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617438+0000", 2024-09-04T17:48:12.926 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1654999999999998e-05 2024-09-04T17:48:12.926 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.926 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.926 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:12.926 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670738+0000", 2024-09-04T17:48:12.926 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053300142000000002 2024-09-04T17:48:12.927 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.927 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.927 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:12.927 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670779+0000", 2024-09-04T17:48:12.927 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0343999999999999e-05 2024-09-04T17:48:12.927 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:12.927 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:12.927 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:12.927 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.927 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.927 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37358 3.1d e58/24 3:bd3215f3:::rbd_data.1134c5a43465.0000000000000020:head v 58'91, mlcod=58'89)", 2024-09-04T17:48:12.927 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617502+0000", 2024-09-04T17:48:12.927 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 93.297097230000006, 2024-09-04T17:48:12.928 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053505641999999999, 2024-09-04T17:48:12.928 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:12.928 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:12.928 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:12.928 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:12.928 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.928 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:12.928 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617502+0000", 2024-09-04T17:48:12.928 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:12.928 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.928 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.928 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:12.928 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617502+0000", 2024-09-04T17:48:12.928 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:12.929 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.929 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.929 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:12.929 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617503+0000", 2024-09-04T17:48:12.929 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.184e-06 2024-09-04T17:48:12.929 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.929 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.929 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:12.929 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617526+0000", 2024-09-04T17:48:12.929 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2099999999999998e-05 2024-09-04T17:48:12.929 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.929 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.929 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:12.930 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617540+0000", 2024-09-04T17:48:12.930 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4741e-05 2024-09-04T17:48:12.930 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.930 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.930 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:12.930 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617550+0000", 2024-09-04T17:48:12.930 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4259999999999992e-06 2024-09-04T17:48:12.930 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.930 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.930 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:12.930 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617613+0000", 2024-09-04T17:48:12.930 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.3625000000000002e-05 2024-09-04T17:48:12.931 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.931 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.931 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:12.931 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617637+0000", 2024-09-04T17:48:12.931 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.3604999999999998e-05 2024-09-04T17:48:12.931 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.931 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.931 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:12.931 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670967+0000", 2024-09-04T17:48:12.931 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053329881000000003 2024-09-04T17:48:12.931 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.932 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.932 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:12.932 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671008+0000", 2024-09-04T17:48:12.932 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.108e-05 2024-09-04T17:48:12.932 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:12.932 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:12.932 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:12.932 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.932 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.932 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37361 3.1c e58/24 3:3c7f8976:::rbd_data.1134c5a43465.0000000000000023:head v 58'159, mlcod=58'157)", 2024-09-04T17:48:12.932 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617578+0000", 2024-09-04T17:48:12.932 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 93.297021380000004, 2024-09-04T17:48:12.932 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053990367999999997, 2024-09-04T17:48:12.933 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:12.933 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:12.933 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:12.933 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:12.933 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.933 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:12.933 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617578+0000", 2024-09-04T17:48:12.933 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:12.933 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.933 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.933 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:12.933 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617578+0000", 2024-09-04T17:48:12.934 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:12.934 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.934 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.934 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:12.934 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617579+0000", 2024-09-04T17:48:12.934 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7299999999999996e-07 2024-09-04T17:48:12.934 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.934 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.934 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:12.934 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617610+0000", 2024-09-04T17:48:12.934 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0951000000000002e-05 2024-09-04T17:48:12.934 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.934 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.935 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:12.935 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:48:12.935 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0002e-05 2024-09-04T17:48:12.935 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.935 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.935 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:12.935 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617626+0000", 2024-09-04T17:48:12.935 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6660000000000002e-06 2024-09-04T17:48:12.935 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.935 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.935 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:12.935 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670548+0000", 2024-09-04T17:48:12.935 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.052921943999999999 2024-09-04T17:48:12.935 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.936 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.936 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:12.936 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670574+0000", 2024-09-04T17:48:12.936 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5405000000000002e-05 2024-09-04T17:48:12.936 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.936 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.936 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:12.936 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671522+0000", 2024-09-04T17:48:12.936 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00094815299999999997 2024-09-04T17:48:12.936 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.936 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.936 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:12.937 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671568+0000", 2024-09-04T17:48:12.937 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6674000000000001e-05 2024-09-04T17:48:12.937 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:12.937 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:12.937 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:12.937 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.937 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.937 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37357 3.15 e58/49 3:a897312a:::rbd_data.1134c5a43465.000000000000001f:head v 58'117, mlcod=58'115)", 2024-09-04T17:48:12.937 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:48:12.938 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 93.296979042000004, 2024-09-04T17:48:12.938 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053734297, 2024-09-04T17:48:12.938 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:12.938 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:12.938 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:12.938 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:12.938 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.938 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:12.938 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:48:12.938 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:12.938 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.938 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.938 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:12.938 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:48:12.938 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:12.938 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.939 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.939 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:12.939 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617621+0000", 2024-09-04T17:48:12.939 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.7300000000000004e-07 2024-09-04T17:48:12.939 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.939 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.939 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:12.939 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617642+0000", 2024-09-04T17:48:12.939 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0185000000000001e-05 2024-09-04T17:48:12.939 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.939 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.939 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:12.940 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617654+0000", 2024-09-04T17:48:12.940 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2745000000000001e-05 2024-09-04T17:48:12.940 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.940 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.940 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:12.940 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617662+0000", 2024-09-04T17:48:12.940 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.0499999999999992e-06 2024-09-04T17:48:12.940 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.940 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.940 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:12.940 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617727+0000", 2024-09-04T17:48:12.940 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.4923000000000002e-05 2024-09-04T17:48:12.941 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.941 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.941 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:12.941 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617753+0000", 2024-09-04T17:48:12.941 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5899999999999999e-05 2024-09-04T17:48:12.941 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.941 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.941 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:12.941 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671298+0000", 2024-09-04T17:48:12.942 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053545276000000003 2024-09-04T17:48:12.942 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.942 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.942 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:12.942 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671355+0000", 2024-09-04T17:48:12.942 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6245000000000001e-05 2024-09-04T17:48:12.942 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:12.942 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:12.943 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:12.943 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.943 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.943 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44687 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:12.943 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253012+0000", 2024-09-04T17:48:12.943 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 82.661586975000006, 2024-09-04T17:48:12.943 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051128777, 2024-09-04T17:48:12.943 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:12.943 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:12.943 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:12.943 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:12.943 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:12.944 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:12.944 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44687 2024-09-04T17:48:12.944 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.944 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:12.944 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.944 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:12.944 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253012+0000", 2024-09-04T17:48:12.944 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:12.944 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.944 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.944 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:12.944 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253012+0000", 2024-09-04T17:48:12.945 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:12.945 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.945 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.945 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:12.945 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253014+0000", 2024-09-04T17:48:12.945 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.24e-06 2024-09-04T17:48:12.945 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.945 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.945 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:12.945 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253019+0000", 2024-09-04T17:48:12.945 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4569999999999996e-06 2024-09-04T17:48:12.945 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.945 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.946 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:12.946 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253035+0000", 2024-09-04T17:48:12.946 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6291999999999999e-05 2024-09-04T17:48:12.946 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.946 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.946 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:12.946 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253050+0000", 2024-09-04T17:48:12.946 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4423000000000001e-05 2024-09-04T17:48:12.946 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.946 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.946 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:12.946 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304115+0000", 2024-09-04T17:48:12.946 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051064778999999998 2024-09-04T17:48:12.946 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.946 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.947 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:12.947 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304141+0000", 2024-09-04T17:48:12.947 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6585999999999999e-05 2024-09-04T17:48:12.947 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:12.947 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:12.947 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:12.947 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.947 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.947 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44688 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:12.947 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253067+0000", 2024-09-04T17:48:12.947 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 82.661532422999997, 2024-09-04T17:48:12.947 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051116979, 2024-09-04T17:48:12.948 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:12.948 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:12.948 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:12.948 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:12.948 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:12.948 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:12.948 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44688 2024-09-04T17:48:12.948 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.948 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:12.948 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.948 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:12.948 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253067+0000", 2024-09-04T17:48:12.948 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:12.949 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.949 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.949 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:12.949 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253067+0000", 2024-09-04T17:48:12.949 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:12.949 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.949 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.949 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:12.949 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253068+0000", 2024-09-04T17:48:12.949 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0729999999999999e-06 2024-09-04T17:48:12.949 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.949 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.949 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:12.950 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253073+0000", 2024-09-04T17:48:12.950 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2259999999999996e-06 2024-09-04T17:48:12.950 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.950 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.950 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:12.950 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253091+0000", 2024-09-04T17:48:12.950 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7255999999999999e-05 2024-09-04T17:48:12.950 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.950 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.950 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:12.950 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253103+0000", 2024-09-04T17:48:12.950 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2632999999999999e-05 2024-09-04T17:48:12.950 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.950 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.950 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:12.951 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304173+0000", 2024-09-04T17:48:12.951 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051069537999999998 2024-09-04T17:48:12.951 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.951 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.951 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:12.951 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304184+0000", 2024-09-04T17:48:12.951 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1253000000000001e-05 2024-09-04T17:48:12.951 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:12.951 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:12.951 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:12.951 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.951 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.951 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44689 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:12.951 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253121+0000", 2024-09-04T17:48:12.952 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 82.661478286000005, 2024-09-04T17:48:12.952 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051192595, 2024-09-04T17:48:12.952 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:12.952 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:12.952 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:12.952 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:12.952 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:12.952 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:12.952 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44689 2024-09-04T17:48:12.952 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.952 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:12.953 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.953 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:12.953 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253121+0000", 2024-09-04T17:48:12.953 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:12.953 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.953 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.953 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:12.953 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253121+0000", 2024-09-04T17:48:12.953 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:12.953 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.953 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.953 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:12.954 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253122+0000", 2024-09-04T17:48:12.954 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.15e-06 2024-09-04T17:48:12.954 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.954 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.954 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:12.954 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253128+0000", 2024-09-04T17:48:12.954 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2569999999999998e-06 2024-09-04T17:48:12.954 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.954 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.954 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:12.954 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253145+0000", 2024-09-04T17:48:12.954 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7176e-05 2024-09-04T17:48:12.954 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.955 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.955 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:12.955 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253158+0000", 2024-09-04T17:48:12.955 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3139000000000001e-05 2024-09-04T17:48:12.955 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.955 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.955 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:12.955 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304287+0000", 2024-09-04T17:48:12.955 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051129149999999998 2024-09-04T17:48:12.955 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.955 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.955 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:12.955 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304314+0000", 2024-09-04T17:48:12.955 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6723000000000001e-05 2024-09-04T17:48:12.956 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:12.956 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:12.956 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:12.956 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.956 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.956 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44690 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:12.956 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253177+0000", 2024-09-04T17:48:12.956 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 82.661422161000004, 2024-09-04T17:48:12.956 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051169263, 2024-09-04T17:48:12.956 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:12.956 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:12.956 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:12.957 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:12.957 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:12.957 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:12.957 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44690 2024-09-04T17:48:12.957 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.957 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:12.957 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.957 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:12.957 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253177+0000", 2024-09-04T17:48:12.957 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:12.957 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.957 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.957 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:12.958 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253177+0000", 2024-09-04T17:48:12.958 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:12.958 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.958 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.958 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:12.958 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253178+0000", 2024-09-04T17:48:12.958 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1000000000000001e-06 2024-09-04T17:48:12.958 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.958 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.958 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:12.958 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253184+0000", 2024-09-04T17:48:12.958 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2089999999999996e-06 2024-09-04T17:48:12.958 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.959 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.959 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:12.959 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253200+0000", 2024-09-04T17:48:12.959 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6087e-05 2024-09-04T17:48:12.959 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.959 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.959 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:12.959 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253231+0000", 2024-09-04T17:48:12.960 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0874999999999998e-05 2024-09-04T17:48:12.960 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.960 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.960 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:12.960 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304337+0000", 2024-09-04T17:48:12.960 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051106430000000001 2024-09-04T17:48:12.960 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.960 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.960 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:12.960 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304347+0000", 2024-09-04T17:48:12.960 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5619999999999993e-06 2024-09-04T17:48:12.960 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:12.960 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:12.961 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:12.961 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.961 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.961 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44691 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:12.961 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253249+0000", 2024-09-04T17:48:12.961 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 82.661350127000006, 2024-09-04T17:48:12.961 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051195630999999998, 2024-09-04T17:48:12.961 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:12.961 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:12.961 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:12.961 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:12.961 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:12.961 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:12.962 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44691 2024-09-04T17:48:12.962 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.962 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:12.962 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.962 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:12.962 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253249+0000", 2024-09-04T17:48:12.962 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:12.962 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.962 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.962 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:12.962 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253249+0000", 2024-09-04T17:48:12.962 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:12.962 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.963 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.963 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:12.963 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253251+0000", 2024-09-04T17:48:12.963 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2330000000000001e-06 2024-09-04T17:48:12.963 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.963 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.963 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:12.963 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253256+0000", 2024-09-04T17:48:12.963 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6099999999999997e-06 2024-09-04T17:48:12.963 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.964 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.964 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:12.964 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253274+0000", 2024-09-04T17:48:12.964 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7952999999999999e-05 2024-09-04T17:48:12.964 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.964 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.964 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:12.964 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253287+0000", 2024-09-04T17:48:12.964 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2839e-05 2024-09-04T17:48:12.964 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.964 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.964 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:12.965 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304396+0000", 2024-09-04T17:48:12.965 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051108990999999999 2024-09-04T17:48:12.965 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.965 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.965 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:12.965 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304445+0000", 2024-09-04T17:48:12.965 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9005000000000003e-05 2024-09-04T17:48:12.965 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:12.965 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:12.965 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:12.965 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.965 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.966 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44692 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:12.966 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253337+0000", 2024-09-04T17:48:12.966 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 82.661262554000004, 2024-09-04T17:48:12.966 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051138909000000003, 2024-09-04T17:48:12.966 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:12.966 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:12.966 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:12.966 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:12.966 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:12.966 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:12.966 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44692 2024-09-04T17:48:12.966 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.966 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:12.966 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.967 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:12.967 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253337+0000", 2024-09-04T17:48:12.967 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:12.967 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.967 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.967 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:12.967 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253337+0000", 2024-09-04T17:48:12.967 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:12.967 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.967 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.967 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:12.968 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253338+0000", 2024-09-04T17:48:12.968 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.463e-06 2024-09-04T17:48:12.968 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.968 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.968 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:12.968 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253357+0000", 2024-09-04T17:48:12.968 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9043e-05 2024-09-04T17:48:12.968 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.968 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.968 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:12.968 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253379+0000", 2024-09-04T17:48:12.968 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1243e-05 2024-09-04T17:48:12.968 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.969 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.969 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:12.969 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253396+0000", 2024-09-04T17:48:12.969 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7059e-05 2024-09-04T17:48:12.969 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.969 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.969 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:12.969 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304466+0000", 2024-09-04T17:48:12.970 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051070624000000002 2024-09-04T17:48:12.970 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.970 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.970 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:12.970 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304476+0000", 2024-09-04T17:48:12.970 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4769999999999992e-06 2024-09-04T17:48:12.970 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:12.970 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:12.970 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:12.970 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.970 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.970 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44693 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:12.971 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253415+0000", 2024-09-04T17:48:12.971 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 82.661184151, 2024-09-04T17:48:12.971 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051141949999999999, 2024-09-04T17:48:12.971 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:12.971 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:12.971 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:12.971 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:12.971 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:12.971 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:12.971 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44693 2024-09-04T17:48:12.971 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.971 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:12.972 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.972 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:12.972 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253415+0000", 2024-09-04T17:48:12.972 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:12.972 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.972 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.972 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:12.972 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253415+0000", 2024-09-04T17:48:12.972 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:12.972 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.972 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.972 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:12.972 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253416+0000", 2024-09-04T17:48:12.973 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1799999999999999e-06 2024-09-04T17:48:12.973 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.973 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.973 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:12.973 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253422+0000", 2024-09-04T17:48:12.973 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.5940000000000001e-06 2024-09-04T17:48:12.973 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.973 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.973 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:12.973 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253438+0000", 2024-09-04T17:48:12.973 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6008999999999999e-05 2024-09-04T17:48:12.973 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.974 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.974 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:12.974 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253452+0000", 2024-09-04T17:48:12.974 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3543000000000001e-05 2024-09-04T17:48:12.974 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.974 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.974 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:12.974 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304533+0000", 2024-09-04T17:48:12.974 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051081609 2024-09-04T17:48:12.974 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.974 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.974 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:12.975 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304557+0000", 2024-09-04T17:48:12.975 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4015e-05 2024-09-04T17:48:12.975 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:12.975 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:12.975 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:12.975 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.976 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.976 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44694 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:12.976 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253469+0000", 2024-09-04T17:48:12.976 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 82.661130025999995, 2024-09-04T17:48:12.976 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051121486000000001, 2024-09-04T17:48:12.976 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:12.977 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:12.977 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:12.977 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:12.977 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:12.977 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:12.977 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44694 2024-09-04T17:48:12.977 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.977 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:12.977 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.978 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:12.978 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253469+0000", 2024-09-04T17:48:12.978 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:12.978 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.978 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.978 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:12.978 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253469+0000", 2024-09-04T17:48:12.978 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:12.978 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.978 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.978 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:12.978 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253471+0000", 2024-09-04T17:48:12.978 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1400000000000001e-06 2024-09-04T17:48:12.978 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.979 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.979 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:12.979 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253476+0000", 2024-09-04T17:48:12.979 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7370000000000003e-06 2024-09-04T17:48:12.979 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.979 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.979 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:12.979 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253493+0000", 2024-09-04T17:48:12.979 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6549e-05 2024-09-04T17:48:12.979 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.979 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.979 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:12.979 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253506+0000", 2024-09-04T17:48:12.980 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3406e-05 2024-09-04T17:48:12.980 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.980 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.980 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:12.980 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304582+0000", 2024-09-04T17:48:12.980 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051075534999999998 2024-09-04T17:48:12.980 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.981 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.981 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:12.981 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304591+0000", 2024-09-04T17:48:12.981 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1190000000000003e-06 2024-09-04T17:48:12.981 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:12.982 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:12.982 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:12.982 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.982 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.982 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44695 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:12.982 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253552+0000", 2024-09-04T17:48:12.982 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 82.661047272000005, 2024-09-04T17:48:12.982 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051068872000000001, 2024-09-04T17:48:12.982 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:12.982 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:12.982 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:12.982 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:12.983 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:12.983 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:12.983 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44695 2024-09-04T17:48:12.983 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.983 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:12.983 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.983 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:12.983 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253552+0000", 2024-09-04T17:48:12.984 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:12.984 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.984 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.984 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:12.984 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253552+0000", 2024-09-04T17:48:12.984 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:12.984 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.984 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.984 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:12.984 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253553+0000", 2024-09-04T17:48:12.984 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1960000000000001e-06 2024-09-04T17:48:12.984 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.984 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.985 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:12.985 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253560+0000", 2024-09-04T17:48:12.985 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.2040000000000002e-06 2024-09-04T17:48:12.985 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.985 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.985 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:12.985 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253578+0000", 2024-09-04T17:48:12.985 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8241999999999999e-05 2024-09-04T17:48:12.985 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.985 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.985 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:12.985 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253591+0000", 2024-09-04T17:48:12.985 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3453e-05 2024-09-04T17:48:12.986 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.986 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.986 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:12.986 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304614+0000", 2024-09-04T17:48:12.986 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051022456000000001 2024-09-04T17:48:12.986 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.986 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.986 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:12.986 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304621+0000", 2024-09-04T17:48:12.986 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3209999999999999e-06 2024-09-04T17:48:12.986 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:12.986 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:12.986 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:12.986 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.987 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.987 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44696 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:12.987 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253609+0000", 2024-09-04T17:48:12.987 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 82.660990771000002, 2024-09-04T17:48:12.987 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051115351000000003, 2024-09-04T17:48:12.987 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:12.987 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:12.987 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:12.987 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:12.987 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:12.987 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:12.987 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44696 2024-09-04T17:48:12.987 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.987 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:12.987 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.988 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:12.988 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253609+0000", 2024-09-04T17:48:12.988 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:12.988 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.988 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.988 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:12.988 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253609+0000", 2024-09-04T17:48:12.988 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:12.988 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.988 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.988 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:12.988 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253610+0000", 2024-09-04T17:48:12.989 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2300000000000001e-06 2024-09-04T17:48:12.989 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.989 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.989 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:12.989 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253615+0000", 2024-09-04T17:48:12.989 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7260000000000002e-06 2024-09-04T17:48:12.989 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.989 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.989 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:12.989 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253632+0000", 2024-09-04T17:48:12.989 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7039999999999999e-05 2024-09-04T17:48:12.990 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.990 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.990 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:12.990 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253666+0000", 2024-09-04T17:48:12.990 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4662000000000001e-05 2024-09-04T17:48:12.990 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.990 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.990 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:12.990 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304677+0000", 2024-09-04T17:48:12.990 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051011017999999998 2024-09-04T17:48:12.990 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.990 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.990 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:12.990 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304724+0000", 2024-09-04T17:48:12.991 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6675000000000003e-05 2024-09-04T17:48:12.991 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:12.991 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:12.991 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:12.991 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.991 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.991 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44697 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:12.991 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253684+0000", 2024-09-04T17:48:12.991 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 82.660915277000001, 2024-09-04T17:48:12.991 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051073107, 2024-09-04T17:48:12.991 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:12.991 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:12.991 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:12.991 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:12.992 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:12.992 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:12.992 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44697 2024-09-04T17:48:12.992 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.992 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:12.992 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.992 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:12.992 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253684+0000", 2024-09-04T17:48:12.992 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:12.992 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.993 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.993 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:12.993 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253684+0000", 2024-09-04T17:48:12.993 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:12.993 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.993 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.993 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:12.993 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253685+0000", 2024-09-04T17:48:12.993 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0699999999999999e-06 2024-09-04T17:48:12.994 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.994 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.994 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:12.994 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253690+0000", 2024-09-04T17:48:12.994 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1959999999999999e-06 2024-09-04T17:48:12.994 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.995 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.995 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:12.995 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253707+0000", 2024-09-04T17:48:12.995 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6946e-05 2024-09-04T17:48:12.995 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.996 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.996 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:12.996 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253720+0000", 2024-09-04T17:48:12.996 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3076e-05 2024-09-04T17:48:12.996 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.996 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.997 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:12.997 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304745+0000", 2024-09-04T17:48:12.997 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051024760000000002 2024-09-04T17:48:12.997 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.997 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.997 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:12.998 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304757+0000", 2024-09-04T17:48:12.998 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2058999999999999e-05 2024-09-04T17:48:12.998 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:12.998 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:12.998 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:12.998 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:12.999 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:12.999 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44698 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:12.999 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253737+0000", 2024-09-04T17:48:12.999 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 82.660862925999993, 2024-09-04T17:48:12.999 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051176078, 2024-09-04T17:48:12.999 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:13.000 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:13.000 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:13.000 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:13.000 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:13.000 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:13.000 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44698 2024-09-04T17:48:13.001 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.001 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:13.001 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.001 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:13.001 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253737+0000", 2024-09-04T17:48:13.001 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:13.001 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.002 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.002 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:13.002 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253737+0000", 2024-09-04T17:48:13.002 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:13.002 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.002 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.002 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:13.002 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253738+0000", 2024-09-04T17:48:13.002 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0300000000000001e-06 2024-09-04T17:48:13.002 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.002 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.002 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:13.002 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253752+0000", 2024-09-04T17:48:13.003 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4358999999999999e-05 2024-09-04T17:48:13.003 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.003 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.003 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:13.003 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253771+0000", 2024-09-04T17:48:13.003 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8879999999999999e-05 2024-09-04T17:48:13.003 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.003 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.003 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:13.003 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253785+0000", 2024-09-04T17:48:13.003 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3852000000000001e-05 2024-09-04T17:48:13.003 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.003 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.003 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:13.004 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304876+0000", 2024-09-04T17:48:13.004 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051091105999999997 2024-09-04T17:48:13.004 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.004 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.004 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:13.004 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304913+0000", 2024-09-04T17:48:13.004 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.6850999999999997e-05 2024-09-04T17:48:13.004 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:13.004 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:13.004 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:13.004 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.004 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.004 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44699 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:13.005 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253804+0000", 2024-09-04T17:48:13.005 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 82.660795402000005, 2024-09-04T17:48:13.005 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051172250000000002, 2024-09-04T17:48:13.005 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:13.005 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:13.005 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:13.005 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:13.005 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:13.005 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:13.005 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44699 2024-09-04T17:48:13.005 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.005 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:13.005 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.005 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:13.006 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253804+0000", 2024-09-04T17:48:13.006 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:13.006 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.006 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.006 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:13.006 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253804+0000", 2024-09-04T17:48:13.006 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:13.006 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.006 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.006 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:13.006 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253805+0000", 2024-09-04T17:48:13.006 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.08e-06 2024-09-04T17:48:13.006 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.007 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.007 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:13.007 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253810+0000", 2024-09-04T17:48:13.007 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1830000000000002e-06 2024-09-04T17:48:13.007 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.007 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.007 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:13.007 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253828+0000", 2024-09-04T17:48:13.007 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7662000000000001e-05 2024-09-04T17:48:13.007 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.007 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.007 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:13.007 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253841+0000", 2024-09-04T17:48:13.007 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3203999999999999e-05 2024-09-04T17:48:13.008 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.008 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.008 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:13.008 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304966+0000", 2024-09-04T17:48:13.008 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051125273999999998 2024-09-04T17:48:13.008 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.008 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.008 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:13.008 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304976+0000", 2024-09-04T17:48:13.008 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.8470000000000008e-06 2024-09-04T17:48:13.008 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:13.008 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:13.008 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:13.009 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.009 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.009 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44700 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:13.009 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253858+0000", 2024-09-04T17:48:13.009 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 82.660741748000007, 2024-09-04T17:48:13.009 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051218957000000002, 2024-09-04T17:48:13.009 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:13.009 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:13.009 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:13.009 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:13.009 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:13.009 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:13.009 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44700 2024-09-04T17:48:13.009 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.010 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:13.010 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.010 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:13.010 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253858+0000", 2024-09-04T17:48:13.010 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:13.010 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.010 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.010 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:13.010 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253858+0000", 2024-09-04T17:48:13.010 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:13.010 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.010 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.010 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:13.010 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253859+0000", 2024-09-04T17:48:13.011 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.217e-06 2024-09-04T17:48:13.011 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.011 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.011 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:13.011 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253864+0000", 2024-09-04T17:48:13.011 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7600000000000002e-06 2024-09-04T17:48:13.011 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.011 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.011 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:13.011 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253880+0000", 2024-09-04T17:48:13.011 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6501999999999999e-05 2024-09-04T17:48:13.011 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.011 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.011 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:13.012 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253899+0000", 2024-09-04T17:48:13.012 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8907e-05 2024-09-04T17:48:13.012 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.012 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.012 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:13.012 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305052+0000", 2024-09-04T17:48:13.012 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051152989000000003 2024-09-04T17:48:13.012 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.012 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.012 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:13.012 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305077+0000", 2024-09-04T17:48:13.012 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4581999999999999e-05 2024-09-04T17:48:13.012 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:13.012 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:13.013 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:13.013 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.013 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.013 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44701 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:13.013 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253917+0000", 2024-09-04T17:48:13.013 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 82.660682395999999, 2024-09-04T17:48:13.013 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051231113000000002, 2024-09-04T17:48:13.013 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:13.013 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:13.013 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:13.013 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:13.013 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:13.013 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:13.013 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44701 2024-09-04T17:48:13.014 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.014 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:13.014 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.014 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:13.014 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253917+0000", 2024-09-04T17:48:13.014 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:13.014 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.014 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.014 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:13.014 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253917+0000", 2024-09-04T17:48:13.014 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:13.014 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.014 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.014 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:13.015 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253918+0000", 2024-09-04T17:48:13.015 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.136e-06 2024-09-04T17:48:13.015 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.015 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.015 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:13.015 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253924+0000", 2024-09-04T17:48:13.015 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.3070000000000002e-06 2024-09-04T17:48:13.015 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.015 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.015 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:13.015 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253940+0000", 2024-09-04T17:48:13.015 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6612e-05 2024-09-04T17:48:13.015 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.016 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.016 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:13.016 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253954+0000", 2024-09-04T17:48:13.016 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3709999999999999e-05 2024-09-04T17:48:13.016 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.016 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.016 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:13.016 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305136+0000", 2024-09-04T17:48:13.016 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051181935999999997 2024-09-04T17:48:13.016 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.016 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.016 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:13.016 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305148+0000", 2024-09-04T17:48:13.016 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2412000000000001e-05 2024-09-04T17:48:13.017 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:13.017 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:13.017 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:13.017 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:13.017 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:13.017 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:13.018 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-04T17:48:13.019 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:13.015+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:13.119 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:13.114+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:13.199 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:13.200 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:48:13.200 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:48:13.200 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:13.216 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-04T17:48:13.257 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:13.254+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:13.320 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:48:13.315+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:48:13.394 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:13.394 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:48:13.394 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:48:13.394 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:48:13.394 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:13.408 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-04T17:48:13.457 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:48:13.454+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:48:13.522 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:13.519+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:13.533 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:13.534 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:48:13.534 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:48:13.534 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:48:13.534 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.534 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8831 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2453504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:13.534 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:48:13.534 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 120.47430516, 2024-09-04T17:48:13.534 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14731111799999999, 2024-09-04T17:48:13.534 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:13.534 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:13.534 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:13.535 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:13.535 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:13.535 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:13.535 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8831 2024-09-04T17:48:13.535 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.535 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:13.535 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.535 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:13.535 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:48:13.535 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:13.535 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.535 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.536 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:13.536 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:48:13.536 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:13.536 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.536 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.536 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:13.536 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057811+0000", 2024-09-04T17:48:13.536 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.083e-06 2024-09-04T17:48:13.536 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.536 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.536 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:13.536 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057814+0000", 2024-09-04T17:48:13.537 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.407e-06 2024-09-04T17:48:13.537 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.537 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.537 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:13.537 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057825+0000", 2024-09-04T17:48:13.537 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.17e-05 2024-09-04T17:48:13.537 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.537 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.537 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:13.537 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057836+0000", 2024-09-04T17:48:13.537 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0283e-05 2024-09-04T17:48:13.538 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.538 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.538 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:13.538 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.204762+0000", 2024-09-04T17:48:13.538 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.146926205 2024-09-04T17:48:13.538 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.538 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.538 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:13.538 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.204895+0000", 2024-09-04T17:48:13.538 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013328999999999999 2024-09-04T17:48:13.538 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.538 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.538 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:13.539 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205121+0000", 2024-09-04T17:48:13.539 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00022614999999999999 2024-09-04T17:48:13.539 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:13.539 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:13.539 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:13.539 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.539 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.539 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8832 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2490368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:13.539 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:48:13.539 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 120.474267127, 2024-09-04T17:48:13.539 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14770012299999999, 2024-09-04T17:48:13.539 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:13.540 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:13.540 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:13.540 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:13.540 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:13.540 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:13.540 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8832 2024-09-04T17:48:13.540 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.540 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:13.540 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.540 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:13.540 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:48:13.540 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:13.540 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.541 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.541 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:13.541 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:48:13.541 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:13.541 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.541 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.541 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:13.541 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057849+0000", 2024-09-04T17:48:13.541 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6700000000000003e-07 2024-09-04T17:48:13.541 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.541 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.541 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:13.542 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057852+0000", 2024-09-04T17:48:13.542 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6359999999999998e-06 2024-09-04T17:48:13.542 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.542 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.542 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:13.542 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057860+0000", 2024-09-04T17:48:13.542 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8140000000000007e-06 2024-09-04T17:48:13.542 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.542 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.542 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:13.542 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057868+0000", 2024-09-04T17:48:13.542 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1999999999999997e-06 2024-09-04T17:48:13.542 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.542 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.543 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:13.543 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205208+0000", 2024-09-04T17:48:13.543 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14734067300000001 2024-09-04T17:48:13.543 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.543 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.543 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:13.543 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205347+0000", 2024-09-04T17:48:13.543 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013914900000000001 2024-09-04T17:48:13.543 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.543 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.543 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:13.543 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205548+0000", 2024-09-04T17:48:13.543 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000200984 2024-09-04T17:48:13.544 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:13.544 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:13.544 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:13.544 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.544 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.544 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8833 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2527232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:13.544 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:48:13.544 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 120.474235257, 2024-09-04T17:48:13.544 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148062572, 2024-09-04T17:48:13.544 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:13.544 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:13.544 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:13.544 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:13.545 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:13.545 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:13.545 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8833 2024-09-04T17:48:13.545 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.545 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:13.545 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.545 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:13.545 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:48:13.545 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:13.545 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.545 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.545 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:13.545 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:48:13.546 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:13.546 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.546 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.546 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:13.546 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057881+0000", 2024-09-04T17:48:13.546 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4300000000000002e-07 2024-09-04T17:48:13.546 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.546 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.546 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:13.546 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057893+0000", 2024-09-04T17:48:13.546 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1973999999999999e-05 2024-09-04T17:48:13.546 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.546 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.546 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:13.547 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057906+0000", 2024-09-04T17:48:13.547 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2716e-05 2024-09-04T17:48:13.547 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.547 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.547 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:13.547 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057913+0000", 2024-09-04T17:48:13.547 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3930000000000002e-06 2024-09-04T17:48:13.547 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.547 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.547 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:13.547 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205621+0000", 2024-09-04T17:48:13.547 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14770788800000001 2024-09-04T17:48:13.547 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.547 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.548 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:13.548 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205740+0000", 2024-09-04T17:48:13.548 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000118936 2024-09-04T17:48:13.548 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.548 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.548 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:13.548 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205943+0000", 2024-09-04T17:48:13.548 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020292199999999999 2024-09-04T17:48:13.548 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:13.548 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:13.548 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:13.548 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.548 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.548 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8834 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2564096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:13.549 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:48:13.549 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 120.47419197799999, 2024-09-04T17:48:13.549 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14846753200000001, 2024-09-04T17:48:13.549 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:13.549 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:13.549 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:13.549 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:13.549 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:13.549 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:13.549 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8834 2024-09-04T17:48:13.549 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.549 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:13.550 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.550 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:13.550 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:48:13.550 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:13.550 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.550 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.550 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:13.550 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:48:13.550 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:13.550 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.550 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.550 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:13.550 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057925+0000", 2024-09-04T17:48:13.551 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0270000000000001e-06 2024-09-04T17:48:13.551 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.551 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.551 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:13.551 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057936+0000", 2024-09-04T17:48:13.551 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.112e-05 2024-09-04T17:48:13.551 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.551 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.551 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:13.551 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057946+0000", 2024-09-04T17:48:13.551 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.009e-05 2024-09-04T17:48:13.551 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.551 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.551 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:13.551 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057953+0000", 2024-09-04T17:48:13.552 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.6229999999999997e-06 2024-09-04T17:48:13.552 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.552 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.552 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:13.552 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206027+0000", 2024-09-04T17:48:13.552 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148073591 2024-09-04T17:48:13.552 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.552 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.552 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:13.552 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206161+0000", 2024-09-04T17:48:13.552 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013411599999999999 2024-09-04T17:48:13.552 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.552 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.552 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:13.553 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206391+0000", 2024-09-04T17:48:13.553 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000229965 2024-09-04T17:48:13.553 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:13.553 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:13.553 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:13.553 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.553 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.553 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8835 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2600960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:13.553 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:48:13.553 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 120.474152328, 2024-09-04T17:48:13.553 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14882941599999999, 2024-09-04T17:48:13.553 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:13.553 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:13.554 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:13.554 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:13.554 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:13.554 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:13.554 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8835 2024-09-04T17:48:13.554 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.554 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:13.554 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.554 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:13.554 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:48:13.554 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:13.554 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.554 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.555 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:13.555 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:48:13.555 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:13.555 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.555 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.555 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:13.555 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058544+0000", 2024-09-04T17:48:13.555 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00058061899999999999 2024-09-04T17:48:13.555 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.555 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.555 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:13.555 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058550+0000", 2024-09-04T17:48:13.555 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.1129999999999997e-06 2024-09-04T17:48:13.556 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.556 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.556 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:13.556 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058566+0000", 2024-09-04T17:48:13.556 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5820000000000001e-05 2024-09-04T17:48:13.556 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.556 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.556 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:13.556 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058578+0000", 2024-09-04T17:48:13.556 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2537e-05 2024-09-04T17:48:13.556 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.556 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.556 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:13.556 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206470+0000", 2024-09-04T17:48:13.557 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14789153799999999 2024-09-04T17:48:13.557 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.557 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.557 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:13.557 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206589+0000", 2024-09-04T17:48:13.557 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000118837 2024-09-04T17:48:13.557 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.557 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.557 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:13.557 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206793+0000", 2024-09-04T17:48:13.557 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000203952 2024-09-04T17:48:13.557 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:13.557 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:13.557 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:13.558 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.558 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.558 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8836 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2654208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:13.558 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:48:13.558 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 120.47351964000001, 2024-09-04T17:48:13.558 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14862433999999999, 2024-09-04T17:48:13.558 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:13.558 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:13.558 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:13.558 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:13.558 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:13.558 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:13.558 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8836 2024-09-04T17:48:13.559 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.559 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:13.559 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.559 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:13.559 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:48:13.559 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:13.559 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.559 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.559 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:13.559 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:48:13.559 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:13.559 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.560 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.560 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:13.560 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062663+0000", 2024-09-04T17:48:13.560 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0040670619999999998 2024-09-04T17:48:13.560 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.560 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.560 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:13.560 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062668+0000", 2024-09-04T17:48:13.560 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4199999999999998e-06 2024-09-04T17:48:13.560 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.560 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.560 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:13.560 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062689+0000", 2024-09-04T17:48:13.561 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0356999999999998e-05 2024-09-04T17:48:13.561 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.561 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.561 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:13.561 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062726+0000", 2024-09-04T17:48:13.561 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7302999999999998e-05 2024-09-04T17:48:13.561 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.561 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.561 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:13.561 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206875+0000", 2024-09-04T17:48:13.561 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14414883100000001 2024-09-04T17:48:13.561 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.561 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.562 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:13.562 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.207016+0000", 2024-09-04T17:48:13.562 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000140822 2024-09-04T17:48:13.562 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.562 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.562 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:13.562 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.207220+0000", 2024-09-04T17:48:13.562 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020454499999999999 2024-09-04T17:48:13.562 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:13.562 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:13.562 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:13.562 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.562 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.563 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8844 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2928640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:13.563 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:48:13.563 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 120.46915555699999, 2024-09-04T17:48:13.563 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14752153100000001, 2024-09-04T17:48:13.563 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:13.563 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:13.563 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:13.563 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:13.563 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:13.563 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:13.563 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8844 2024-09-04T17:48:13.563 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.563 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:13.563 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.563 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:13.564 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:48:13.564 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:13.564 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.564 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.564 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:13.564 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:48:13.564 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:13.564 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.564 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.564 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:13.564 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062961+0000", 2024-09-04T17:48:13.564 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6700000000000003e-07 2024-09-04T17:48:13.564 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.564 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.565 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:13.565 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062973+0000", 2024-09-04T17:48:13.565 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2109999999999999e-05 2024-09-04T17:48:13.565 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.565 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.565 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:13.565 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062982+0000", 2024-09-04T17:48:13.565 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.6900000000000004e-06 2024-09-04T17:48:13.565 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.565 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.565 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:13.565 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062990+0000", 2024-09-04T17:48:13.565 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1899999999999998e-06 2024-09-04T17:48:13.565 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.566 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.566 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:13.566 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210090+0000", 2024-09-04T17:48:13.566 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14710026300000001 2024-09-04T17:48:13.566 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.566 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.566 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:13.566 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210225+0000", 2024-09-04T17:48:13.566 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013532400000000001 2024-09-04T17:48:13.566 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.566 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.566 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:13.566 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210481+0000", 2024-09-04T17:48:13.566 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00025628700000000001 2024-09-04T17:48:13.567 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:13.567 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:13.567 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:13.567 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.567 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.567 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8845 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2965504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:13.567 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:48:13.567 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 120.46911583399999, 2024-09-04T17:48:13.567 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147916819, 2024-09-04T17:48:13.567 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:13.567 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:13.567 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:13.567 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:13.567 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:13.567 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:13.568 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8845 2024-09-04T17:48:13.568 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.568 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:13.568 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.568 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:13.568 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:48:13.568 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:13.568 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.568 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.568 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:13.568 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:48:13.568 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:13.568 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.569 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.569 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:13.569 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:48:13.569 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.0399999999999995e-07 2024-09-04T17:48:13.569 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.569 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.569 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:13.569 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063003+0000", 2024-09-04T17:48:13.569 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6529999999999998e-06 2024-09-04T17:48:13.569 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.569 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.569 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:13.569 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063012+0000", 2024-09-04T17:48:13.570 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9500000000000007e-06 2024-09-04T17:48:13.570 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.570 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.570 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:13.570 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063027+0000", 2024-09-04T17:48:13.570 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5443000000000001e-05 2024-09-04T17:48:13.570 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.570 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.570 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:13.570 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210571+0000", 2024-09-04T17:48:13.570 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14754353100000001 2024-09-04T17:48:13.570 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.570 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.570 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:13.571 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210714+0000", 2024-09-04T17:48:13.571 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014295599999999999 2024-09-04T17:48:13.571 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.571 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.571 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:13.571 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210916+0000", 2024-09-04T17:48:13.571 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020258200000000001 2024-09-04T17:48:13.571 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:13.571 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:13.571 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:13.571 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.571 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.571 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8846 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3002368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:13.571 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:48:13.571 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 120.469078297, 2024-09-04T17:48:13.572 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14828011599999999, 2024-09-04T17:48:13.572 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:13.572 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:13.572 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:13.572 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:13.572 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:13.572 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:13.572 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8846 2024-09-04T17:48:13.572 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.572 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:13.572 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.572 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:13.572 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:48:13.572 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:13.573 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.573 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.573 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:13.573 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:48:13.573 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:13.573 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.573 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.573 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:13.573 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063038+0000", 2024-09-04T17:48:13.573 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.23e-07 2024-09-04T17:48:13.573 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.573 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.573 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:13.573 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063051+0000", 2024-09-04T17:48:13.574 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2757e-05 2024-09-04T17:48:13.574 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.574 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.574 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:13.574 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063070+0000", 2024-09-04T17:48:13.574 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9663000000000001e-05 2024-09-04T17:48:13.574 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.574 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.574 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:13.574 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063079+0000", 2024-09-04T17:48:13.574 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.0929999999999992e-06 2024-09-04T17:48:13.575 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.575 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.575 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:13.575 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210989+0000", 2024-09-04T17:48:13.575 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14790984500000001 2024-09-04T17:48:13.575 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.575 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.575 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:13.575 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211109+0000", 2024-09-04T17:48:13.575 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000120056 2024-09-04T17:48:13.575 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.575 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.575 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:13.575 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211317+0000", 2024-09-04T17:48:13.576 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020797900000000001 2024-09-04T17:48:13.576 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:13.576 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:13.576 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:13.576 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.576 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.576 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8847 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3039232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:13.576 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:48:13.576 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 120.469003865, 2024-09-04T17:48:13.576 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14859408900000001, 2024-09-04T17:48:13.576 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:13.576 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:13.577 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:13.577 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:13.577 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:13.577 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:13.577 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8847 2024-09-04T17:48:13.577 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.577 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:13.577 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.577 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:13.577 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:48:13.577 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:13.577 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.577 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.578 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:13.578 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:48:13.578 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:13.578 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.578 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.578 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:13.578 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063113+0000", 2024-09-04T17:48:13.578 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.204e-06 2024-09-04T17:48:13.578 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.578 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.578 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:13.578 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063116+0000", 2024-09-04T17:48:13.578 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.3330000000000001e-06 2024-09-04T17:48:13.579 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.579 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.579 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:13.579 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063126+0000", 2024-09-04T17:48:13.579 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0166999999999999e-05 2024-09-04T17:48:13.579 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.579 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.579 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:13.579 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063134+0000", 2024-09-04T17:48:13.579 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4329999999999998e-06 2024-09-04T17:48:13.579 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.579 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.579 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:13.580 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211392+0000", 2024-09-04T17:48:13.580 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148258109 2024-09-04T17:48:13.580 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.580 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.580 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:13.580 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211506+0000", 2024-09-04T17:48:13.580 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000113813 2024-09-04T17:48:13.580 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.580 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.580 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:13.580 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211706+0000", 2024-09-04T17:48:13.580 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020002999999999999 2024-09-04T17:48:13.580 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:13.581 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:13.581 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:13.581 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.581 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.581 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8848 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:13.581 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:48:13.581 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 120.468971528, 2024-09-04T17:48:13.581 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14907967599999999, 2024-09-04T17:48:13.581 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:13.581 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:13.581 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:13.581 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:13.581 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:13.582 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:13.582 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8848 2024-09-04T17:48:13.582 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.582 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:13.582 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.582 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:13.582 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:48:13.582 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:13.582 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.582 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.582 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:13.582 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:48:13.583 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:13.583 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.583 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.583 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:13.583 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070589+0000", 2024-09-04T17:48:13.583 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.007445386 2024-09-04T17:48:13.583 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.583 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.583 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:13.583 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070600+0000", 2024-09-04T17:48:13.583 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0346999999999999e-05 2024-09-04T17:48:13.583 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.583 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.584 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:13.584 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070622+0000", 2024-09-04T17:48:13.584 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2289999999999998e-05 2024-09-04T17:48:13.584 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.584 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.584 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:13.584 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070636+0000", 2024-09-04T17:48:13.584 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4093e-05 2024-09-04T17:48:13.584 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.584 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.584 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:13.585 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211815+0000", 2024-09-04T17:48:13.585 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14117874699999999 2024-09-04T17:48:13.585 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.585 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.585 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:13.585 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211983+0000", 2024-09-04T17:48:13.585 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00016852800000000001 2024-09-04T17:48:13.585 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.585 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.585 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:13.585 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.212224+0000", 2024-09-04T17:48:13.585 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024028499999999999 2024-09-04T17:48:13.585 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:13.586 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:13.586 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:13.586 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.586 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.586 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8862 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2019328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:13.586 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:48:13.586 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 120.46092206, 2024-09-04T17:48:13.586 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14755236399999999, 2024-09-04T17:48:13.586 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:13.586 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:13.586 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:13.586 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:13.586 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:13.587 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:13.587 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8862 2024-09-04T17:48:13.587 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.587 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:13.587 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.587 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:13.587 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:48:13.587 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:13.587 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.587 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.587 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:13.587 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:48:13.587 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:13.588 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.588 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.588 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:13.588 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071194+0000", 2024-09-04T17:48:13.588 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.5000000000000002e-07 2024-09-04T17:48:13.588 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.588 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.588 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:13.588 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071197+0000", 2024-09-04T17:48:13.588 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5900000000000002e-06 2024-09-04T17:48:13.588 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.588 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.588 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:13.589 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071206+0000", 2024-09-04T17:48:13.589 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9700000000000005e-06 2024-09-04T17:48:13.589 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.589 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.589 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:13.589 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071213+0000", 2024-09-04T17:48:13.589 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.9070000000000001e-06 2024-09-04T17:48:13.589 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.589 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.589 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:13.589 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218396+0000", 2024-09-04T17:48:13.589 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147183273 2024-09-04T17:48:13.589 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.589 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.590 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:13.590 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218538+0000", 2024-09-04T17:48:13.590 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014194199999999999 2024-09-04T17:48:13.590 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.590 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.590 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:13.590 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218746+0000", 2024-09-04T17:48:13.590 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000208032 2024-09-04T17:48:13.590 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:13.590 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:13.590 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:13.590 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.590 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.590 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8863 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2056192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:13.591 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:48:13.591 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 120.46089375, 2024-09-04T17:48:13.591 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14792281099999999, 2024-09-04T17:48:13.591 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:13.591 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:13.591 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:13.591 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:13.591 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:13.591 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:13.591 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8863 2024-09-04T17:48:13.591 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.591 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:13.591 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.592 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:13.592 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:48:13.592 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:13.592 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.592 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.592 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:13.592 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:48:13.592 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:13.592 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.592 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.592 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:13.592 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:48:13.592 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1299999999999999e-07 2024-09-04T17:48:13.593 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.593 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.593 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:13.593 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071225+0000", 2024-09-04T17:48:13.593 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6199999999999999e-06 2024-09-04T17:48:13.593 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.593 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.593 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:13.593 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071234+0000", 2024-09-04T17:48:13.593 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8769999999999999e-06 2024-09-04T17:48:13.593 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.593 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.593 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:13.594 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071251+0000", 2024-09-04T17:48:13.594 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.738e-05 2024-09-04T17:48:13.594 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.594 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.594 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:13.594 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218798+0000", 2024-09-04T17:48:13.594 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14754645 2024-09-04T17:48:13.594 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.594 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.594 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:13.594 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218926+0000", 2024-09-04T17:48:13.594 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000128446 2024-09-04T17:48:13.594 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.595 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.595 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:13.595 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219145+0000", 2024-09-04T17:48:13.595 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021832500000000001 2024-09-04T17:48:13.595 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:13.595 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:13.595 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:13.595 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.595 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.595 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8864 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2093056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:13.595 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:48:13.595 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 120.460852174, 2024-09-04T17:48:13.596 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148277309, 2024-09-04T17:48:13.596 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:13.596 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:13.596 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:13.596 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:13.596 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:13.596 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:13.596 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8864 2024-09-04T17:48:13.596 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.596 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:13.596 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.597 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:13.597 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:48:13.597 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:13.597 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.597 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.597 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:13.597 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:48:13.597 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:13.597 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.597 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.598 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:13.598 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071264+0000", 2024-09-04T17:48:13.598 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3e-07 2024-09-04T17:48:13.598 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.598 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.598 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:13.598 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071267+0000", 2024-09-04T17:48:13.598 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0139999999999999e-06 2024-09-04T17:48:13.598 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.598 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.598 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:13.598 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071276+0000", 2024-09-04T17:48:13.599 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1689999999999998e-06 2024-09-04T17:48:13.599 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.599 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.599 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:13.599 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071283+0000", 2024-09-04T17:48:13.599 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2269999999999996e-06 2024-09-04T17:48:13.599 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.599 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.599 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:13.599 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219195+0000", 2024-09-04T17:48:13.599 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147911868 2024-09-04T17:48:13.599 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.599 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.600 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:13.600 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219324+0000", 2024-09-04T17:48:13.600 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00012835900000000001 2024-09-04T17:48:13.600 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.600 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.600 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:13.600 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219541+0000", 2024-09-04T17:48:13.600 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000216942 2024-09-04T17:48:13.600 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:13.600 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:13.600 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:13.600 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.600 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.601 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8865 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2109440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:13.601 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:48:13.601 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 120.460822714, 2024-09-04T17:48:13.601 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14865747300000001, 2024-09-04T17:48:13.601 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:13.601 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:13.601 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:13.601 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:13.601 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:13.601 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:13.601 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8865 2024-09-04T17:48:13.601 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.601 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:13.602 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.602 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:13.602 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:48:13.602 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:13.602 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.602 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.602 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:13.602 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:48:13.602 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:13.602 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.602 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.602 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:13.603 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:48:13.603 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7999999999999995e-07 2024-09-04T17:48:13.603 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.603 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.603 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:13.603 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071296+0000", 2024-09-04T17:48:13.603 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6900000000000001e-06 2024-09-04T17:48:13.603 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.603 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.603 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:13.603 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071305+0000", 2024-09-04T17:48:13.603 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.5499999999999995e-06 2024-09-04T17:48:13.604 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.604 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.604 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:13.604 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071312+0000", 2024-09-04T17:48:13.604 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1559999999999998e-06 2024-09-04T17:48:13.604 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.604 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.604 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:13.604 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219591+0000", 2024-09-04T17:48:13.604 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14827944900000001 2024-09-04T17:48:13.604 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.604 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.604 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:13.605 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219731+0000", 2024-09-04T17:48:13.605 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013931600000000001 2024-09-04T17:48:13.605 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.605 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.605 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:13.605 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219950+0000", 2024-09-04T17:48:13.605 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021963199999999999 2024-09-04T17:48:13.605 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:13.605 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:13.605 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:13.605 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.605 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.605 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8866 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2146304~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:13.605 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:48:13.606 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 120.46079448099999, 2024-09-04T17:48:13.606 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14906117899999999, 2024-09-04T17:48:13.606 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:13.606 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:13.606 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:13.606 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:13.606 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:13.606 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:13.606 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8866 2024-09-04T17:48:13.606 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.606 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:13.606 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.606 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:13.606 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:48:13.607 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:13.607 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.607 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.607 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:13.607 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:48:13.607 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:13.607 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.607 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.607 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:13.607 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071322+0000", 2024-09-04T17:48:13.607 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7999999999999995e-07 2024-09-04T17:48:13.607 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.607 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.607 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:13.607 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071338+0000", 2024-09-04T17:48:13.608 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5872999999999999e-05 2024-09-04T17:48:13.608 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.608 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.608 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:13.608 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071347+0000", 2024-09-04T17:48:13.608 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4669999999999993e-06 2024-09-04T17:48:13.608 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.608 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.608 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:13.608 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071354+0000", 2024-09-04T17:48:13.608 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2099999999999996e-06 2024-09-04T17:48:13.608 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.608 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.609 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:13.609 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220008+0000", 2024-09-04T17:48:13.609 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148653642 2024-09-04T17:48:13.609 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.609 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.609 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:13.609 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220155+0000", 2024-09-04T17:48:13.609 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014726600000000001 2024-09-04T17:48:13.609 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.609 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.609 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:13.609 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220382+0000", 2024-09-04T17:48:13.610 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00022704100000000001 2024-09-04T17:48:13.610 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:13.610 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:13.610 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:13.610 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.610 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.610 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8867 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2183168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:13.610 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:48:13.610 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 120.460751128, 2024-09-04T17:48:13.610 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.149445831, 2024-09-04T17:48:13.610 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:13.610 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:13.610 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:13.610 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:13.611 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:13.611 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:13.611 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8867 2024-09-04T17:48:13.611 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.611 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:13.611 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.611 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:13.611 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:48:13.611 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:13.611 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.611 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.611 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:13.612 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:48:13.612 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:13.612 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.612 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.612 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:13.612 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078653+0000", 2024-09-04T17:48:13.612 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0072890430000000003 2024-09-04T17:48:13.612 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.612 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.612 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:13.612 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078664+0000", 2024-09-04T17:48:13.612 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0307e-05 2024-09-04T17:48:13.613 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.613 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.613 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:13.613 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078690+0000", 2024-09-04T17:48:13.613 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6780000000000001e-05 2024-09-04T17:48:13.613 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.613 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.613 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:13.613 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078706+0000", 2024-09-04T17:48:13.613 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5676999999999999e-05 2024-09-04T17:48:13.613 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.613 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.613 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:13.614 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220467+0000", 2024-09-04T17:48:13.614 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14176103500000001 2024-09-04T17:48:13.614 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.614 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.614 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:13.614 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220595+0000", 2024-09-04T17:48:13.614 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000127427 2024-09-04T17:48:13.614 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.614 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.614 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:13.614 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220810+0000", 2024-09-04T17:48:13.614 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021556200000000001 2024-09-04T17:48:13.614 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:13.614 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:13.615 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:13.615 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.615 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.615 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8881 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2695168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:13.615 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:48:13.615 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 120.452929059, 2024-09-04T17:48:13.615 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147543547, 2024-09-04T17:48:13.615 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:13.615 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:13.615 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:13.615 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:13.615 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:13.615 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:13.615 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8881 2024-09-04T17:48:13.616 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.616 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:13.616 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.616 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:13.616 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:48:13.616 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:13.616 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.616 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.616 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:13.616 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:48:13.616 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:13.616 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.616 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.617 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:13.617 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079187+0000", 2024-09-04T17:48:13.617 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7700000000000004e-07 2024-09-04T17:48:13.617 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.617 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.617 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:13.617 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079198+0000", 2024-09-04T17:48:13.617 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0849999999999999e-05 2024-09-04T17:48:13.617 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.617 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.617 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:13.617 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079207+0000", 2024-09-04T17:48:13.617 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5599999999999999e-06 2024-09-04T17:48:13.618 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.618 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.618 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:13.618 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079224+0000", 2024-09-04T17:48:13.618 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6387000000000001e-05 2024-09-04T17:48:13.618 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.618 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.618 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:13.618 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226414+0000", 2024-09-04T17:48:13.618 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14718975000000001 2024-09-04T17:48:13.618 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.618 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.618 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:13.619 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226534+0000", 2024-09-04T17:48:13.619 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000120288 2024-09-04T17:48:13.619 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.619 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.619 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:13.619 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226730+0000", 2024-09-04T17:48:13.619 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000196035 2024-09-04T17:48:13.619 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:13.619 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:13.619 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:13.619 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.619 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.619 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8882 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2732032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:13.620 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:48:13.620 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 120.452880845, 2024-09-04T17:48:13.620 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14795022399999999, 2024-09-04T17:48:13.620 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:13.620 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:13.620 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:13.620 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:13.620 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:13.620 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:13.620 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8882 2024-09-04T17:48:13.620 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.620 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:13.621 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.621 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:13.621 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:48:13.621 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:13.621 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.621 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.621 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:13.621 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:48:13.621 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:13.621 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.621 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.621 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:13.621 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086588+0000", 2024-09-04T17:48:13.622 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0073530959999999999 2024-09-04T17:48:13.622 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.622 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.622 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:13.622 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086598+0000", 2024-09-04T17:48:13.622 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0122999999999999e-05 2024-09-04T17:48:13.622 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.622 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.622 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:13.622 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086626+0000", 2024-09-04T17:48:13.622 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.7914000000000001e-05 2024-09-04T17:48:13.622 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.623 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.623 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:13.623 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086641+0000", 2024-09-04T17:48:13.623 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5270000000000001e-05 2024-09-04T17:48:13.623 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.623 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.623 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:13.623 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226801+0000", 2024-09-04T17:48:13.623 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.140160321 2024-09-04T17:48:13.623 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.623 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.623 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:13.623 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226951+0000", 2024-09-04T17:48:13.623 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014965200000000001 2024-09-04T17:48:13.624 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.624 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.624 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:13.624 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.227185+0000", 2024-09-04T17:48:13.624 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00023384799999999999 2024-09-04T17:48:13.624 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:13.624 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:13.624 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:13.624 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.624 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.624 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:93470 3.1f 3:f95bc674:::rbd_header.1134878eadad:head [watch reconnect cookie 140018821420464 gen 16] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-04T17:48:13.624 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:57.837081+0000", 2024-09-04T17:48:13.624 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 15.695034961999999, 2024-09-04T17:48:13.625 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.072023486, 2024-09-04T17:48:13.625 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:13.625 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:13.625 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:13.625 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:13.625 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:13.625 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:13.625 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 93470 2024-09-04T17:48:13.625 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.625 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:13.625 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.625 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:13.625 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:57.837081+0000", 2024-09-04T17:48:13.626 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:13.626 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.626 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.626 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:13.626 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:57.837081+0000", 2024-09-04T17:48:13.626 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:13.626 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.626 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.626 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:13.626 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:57.837084+0000", 2024-09-04T17:48:13.626 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.8229999999999997e-06 2024-09-04T17:48:13.626 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.626 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.627 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:13.627 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:57.837098+0000", 2024-09-04T17:48:13.627 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3436e-05 2024-09-04T17:48:13.627 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.627 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.627 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:13.627 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:57.837123+0000", 2024-09-04T17:48:13.627 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5079000000000002e-05 2024-09-04T17:48:13.627 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.627 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.627 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:13.627 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:57.837145+0000", 2024-09-04T17:48:13.627 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.228e-05 2024-09-04T17:48:13.628 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.628 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.628 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:13.628 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:57.837360+0000", 2024-09-04T17:48:13.628 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021464100000000001 2024-09-04T17:48:13.628 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.628 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.628 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for peered", 2024-09-04T17:48:13.628 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:57.837410+0000", 2024-09-04T17:48:13.628 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0235000000000003e-05 2024-09-04T17:48:13.628 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.628 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.629 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:13.629 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.900175+0000", 2024-09-04T17:48:13.629 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0627652830000001 2024-09-04T17:48:13.629 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.629 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.629 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for rw locks", 2024-09-04T17:48:13.629 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.900370+0000", 2024-09-04T17:48:13.629 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019497199999999999 2024-09-04T17:48:13.629 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.629 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.629 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:13.629 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.906957+0000", 2024-09-04T17:48:13.629 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0065866889999999997 2024-09-04T17:48:13.630 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.630 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.630 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:13.630 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.907088+0000", 2024-09-04T17:48:13.630 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013056100000000001 2024-09-04T17:48:13.630 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:13.630 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:13.630 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:13.630 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.909104+0000", 2024-09-04T17:48:13.630 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.002016487 2024-09-04T17:48:13.630 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:13.630 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:13.631 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:13.631 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:13.631 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:13.631 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:13.632 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-04T17:48:13.633 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:48:13.626+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:48:13.733 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:13.730+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:13.806 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:13.806 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:48:13.806 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:48:13.806 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:13.820 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-04T17:48:13.861 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:13.858+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:13.935 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:13.931+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:13.937 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is unset 2024-09-04T17:48:14.003 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:14.003 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:48:14.003 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:48:14.004 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:48:14.004 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:14.020 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-04T17:48:14.065 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:14.062+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:14.136 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:14.130+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:14.148 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-04T17:48:14.155 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:14.155 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:48:14.155 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:48:14.155 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:48:14.155 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.155 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60921 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 262144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:14.155 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822920+0000", 2024-09-04T17:48:14.156 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 60.331286403, 2024-09-04T17:48:14.156 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073904394999999998, 2024-09-04T17:48:14.156 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:14.156 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:14.156 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:14.156 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:14.156 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:14.156 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:14.156 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60921 2024-09-04T17:48:14.156 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.156 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:14.156 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.156 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:14.157 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822920+0000", 2024-09-04T17:48:14.157 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:14.157 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.157 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.157 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:14.157 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822920+0000", 2024-09-04T17:48:14.157 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:14.157 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.157 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.157 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:14.157 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822920+0000", 2024-09-04T17:48:14.157 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4999999999999999e-07 2024-09-04T17:48:14.157 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.158 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.158 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:14.158 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822921+0000", 2024-09-04T17:48:14.158 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.113e-06 2024-09-04T17:48:14.158 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.158 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.158 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:14.158 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822924+0000", 2024-09-04T17:48:14.158 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.354e-06 2024-09-04T17:48:14.158 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.158 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.158 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:14.159 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822927+0000", 2024-09-04T17:48:14.159 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6199999999999999e-06 2024-09-04T17:48:14.159 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.159 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.159 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:14.159 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896586+0000", 2024-09-04T17:48:14.159 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073658976000000001 2024-09-04T17:48:14.159 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.159 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.159 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:14.159 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896626+0000", 2024-09-04T17:48:14.159 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0484000000000001e-05 2024-09-04T17:48:14.159 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.159 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.160 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:14.160 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896824+0000", 2024-09-04T17:48:14.160 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000197598 2024-09-04T17:48:14.160 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:14.160 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:14.160 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:14.160 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.160 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.160 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60922 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:14.160 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822930+0000", 2024-09-04T17:48:14.160 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 60.331275828000003, 2024-09-04T17:48:14.160 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074178872000000007, 2024-09-04T17:48:14.161 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:14.161 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:14.161 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:14.161 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:14.161 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:14.161 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:14.161 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60922 2024-09-04T17:48:14.161 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.161 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:14.161 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.161 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:14.161 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822930+0000", 2024-09-04T17:48:14.161 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:14.162 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.162 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.162 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:14.162 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822930+0000", 2024-09-04T17:48:14.162 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:14.162 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.162 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.162 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:14.162 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822930+0000", 2024-09-04T17:48:14.162 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.36e-07 2024-09-04T17:48:14.162 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.162 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.162 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:14.163 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822932+0000", 2024-09-04T17:48:14.163 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.567e-06 2024-09-04T17:48:14.163 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.163 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.163 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:14.163 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822938+0000", 2024-09-04T17:48:14.163 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4840000000000003e-06 2024-09-04T17:48:14.163 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.163 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.163 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:14.163 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822942+0000", 2024-09-04T17:48:14.163 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2039999999999999e-06 2024-09-04T17:48:14.163 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.163 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.164 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:14.164 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896850+0000", 2024-09-04T17:48:14.164 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073907941000000005 2024-09-04T17:48:14.164 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.164 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.164 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:14.164 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896891+0000", 2024-09-04T17:48:14.164 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0964999999999998e-05 2024-09-04T17:48:14.164 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.164 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.164 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:14.164 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897109+0000", 2024-09-04T17:48:14.164 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021847500000000001 2024-09-04T17:48:14.164 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:14.165 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:14.165 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:14.165 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.165 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.165 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60923 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 475136~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:14.165 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822949+0000", 2024-09-04T17:48:14.165 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 60.331257538000003, 2024-09-04T17:48:14.165 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074426928000000003, 2024-09-04T17:48:14.165 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:14.165 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:14.165 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:14.165 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:14.165 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:14.165 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:14.166 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60923 2024-09-04T17:48:14.166 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.166 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:14.166 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.166 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:14.166 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822949+0000", 2024-09-04T17:48:14.166 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:14.166 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.166 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.166 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:14.166 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822949+0000", 2024-09-04T17:48:14.166 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:14.166 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.166 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.167 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:14.167 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822949+0000", 2024-09-04T17:48:14.167 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7399999999999999e-07 2024-09-04T17:48:14.167 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.167 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.167 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:14.167 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822951+0000", 2024-09-04T17:48:14.167 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1940000000000001e-06 2024-09-04T17:48:14.167 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.167 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.167 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:14.167 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822957+0000", 2024-09-04T17:48:14.167 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.0650000000000004e-06 2024-09-04T17:48:14.167 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.168 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.168 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:14.168 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822960+0000", 2024-09-04T17:48:14.168 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.8009999999999999e-06 2024-09-04T17:48:14.168 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.168 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.168 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:14.168 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897138+0000", 2024-09-04T17:48:14.168 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074177990999999999 2024-09-04T17:48:14.168 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.168 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.168 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:14.168 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897179+0000", 2024-09-04T17:48:14.168 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.1057999999999999e-05 2024-09-04T17:48:14.169 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.169 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.169 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:14.169 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897375+0000", 2024-09-04T17:48:14.169 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019644500000000001 2024-09-04T17:48:14.169 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:14.169 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:14.169 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:14.169 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.169 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.169 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60924 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:14.169 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822964+0000", 2024-09-04T17:48:14.169 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 60.331242353, 2024-09-04T17:48:14.170 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074531902999999997, 2024-09-04T17:48:14.170 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:14.170 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:14.170 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:14.170 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:14.170 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:14.170 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:14.170 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60924 2024-09-04T17:48:14.170 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.170 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:14.170 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.170 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:14.170 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822964+0000", 2024-09-04T17:48:14.171 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:14.171 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.171 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.171 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:14.171 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822964+0000", 2024-09-04T17:48:14.171 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:14.171 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.171 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.171 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:14.171 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822964+0000", 2024-09-04T17:48:14.171 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4999999999999999e-07 2024-09-04T17:48:14.171 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.171 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.171 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:14.171 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822966+0000", 2024-09-04T17:48:14.172 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.917e-06 2024-09-04T17:48:14.172 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.172 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.172 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:14.172 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822969+0000", 2024-09-04T17:48:14.172 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.315e-06 2024-09-04T17:48:14.172 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.172 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.172 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:14.172 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822972+0000", 2024-09-04T17:48:14.172 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.599e-06 2024-09-04T17:48:14.172 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.172 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.173 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:14.173 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897398+0000", 2024-09-04T17:48:14.173 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074426168000000001 2024-09-04T17:48:14.173 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.173 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.173 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:14.173 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897441+0000", 2024-09-04T17:48:14.173 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.3158000000000003e-05 2024-09-04T17:48:14.173 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.173 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.173 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:14.173 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897496+0000", 2024-09-04T17:48:14.173 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4496000000000001e-05 2024-09-04T17:48:14.173 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:14.174 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:14.174 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:14.174 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.174 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.174 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60925 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 720896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:14.174 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822975+0000", 2024-09-04T17:48:14.174 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 60.331230918000003, 2024-09-04T17:48:14.174 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074635272000000003, 2024-09-04T17:48:14.174 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:14.174 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:14.174 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:14.174 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:14.174 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:14.174 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:14.175 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60925 2024-09-04T17:48:14.175 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.175 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:14.175 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.175 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:14.175 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822975+0000", 2024-09-04T17:48:14.175 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:14.175 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.175 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.175 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:14.175 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822975+0000", 2024-09-04T17:48:14.175 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:14.175 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.175 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.176 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:14.176 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822975+0000", 2024-09-04T17:48:14.176 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.34e-07 2024-09-04T17:48:14.176 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.176 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.176 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:14.176 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822976+0000", 2024-09-04T17:48:14.176 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0359999999999999e-06 2024-09-04T17:48:14.176 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.176 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.176 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:14.176 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822980+0000", 2024-09-04T17:48:14.176 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.242e-06 2024-09-04T17:48:14.177 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.177 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.177 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:14.177 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822983+0000", 2024-09-04T17:48:14.177 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.8830000000000002e-06 2024-09-04T17:48:14.177 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.177 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.177 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:14.177 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897519+0000", 2024-09-04T17:48:14.177 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074536366000000007 2024-09-04T17:48:14.177 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.177 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.177 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:14.178 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897560+0000", 2024-09-04T17:48:14.178 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0677999999999999e-05 2024-09-04T17:48:14.178 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.178 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.178 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:14.178 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897610+0000", 2024-09-04T17:48:14.178 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0832999999999999e-05 2024-09-04T17:48:14.178 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:14.178 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:14.178 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:14.178 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.178 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.178 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60926 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 860160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:14.178 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822986+0000", 2024-09-04T17:48:14.179 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 60.33122032, 2024-09-04T17:48:14.179 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074736413000000002, 2024-09-04T17:48:14.179 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:14.179 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:14.179 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:14.179 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:14.179 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:14.179 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:14.179 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60926 2024-09-04T17:48:14.179 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.179 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:14.179 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.179 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:14.180 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822986+0000", 2024-09-04T17:48:14.180 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:14.180 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.180 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.180 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:14.180 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822986+0000", 2024-09-04T17:48:14.180 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:14.180 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.180 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.180 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:14.180 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822986+0000", 2024-09-04T17:48:14.180 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5600000000000002e-07 2024-09-04T17:48:14.180 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.181 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.181 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:14.181 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822987+0000", 2024-09-04T17:48:14.181 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.082e-06 2024-09-04T17:48:14.181 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.181 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.181 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:14.181 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822990+0000", 2024-09-04T17:48:14.181 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.2219999999999998e-06 2024-09-04T17:48:14.182 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.182 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.182 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:14.182 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822994+0000", 2024-09-04T17:48:14.182 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.19e-06 2024-09-04T17:48:14.182 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.182 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.182 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:14.182 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897632+0000", 2024-09-04T17:48:14.182 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074638205999999999 2024-09-04T17:48:14.183 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.183 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.183 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:14.183 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897672+0000", 2024-09-04T17:48:14.183 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.9873999999999998e-05 2024-09-04T17:48:14.183 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.183 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.183 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:14.183 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897722+0000", 2024-09-04T17:48:14.183 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0583e-05 2024-09-04T17:48:14.183 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:14.183 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:14.183 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:14.184 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.184 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.184 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60927 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 966656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:14.184 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823000+0000", 2024-09-04T17:48:14.184 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 60.331206137999999, 2024-09-04T17:48:14.184 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074834036000000007, 2024-09-04T17:48:14.184 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:14.184 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:14.184 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:14.184 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:14.184 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:14.184 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:14.185 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60927 2024-09-04T17:48:14.185 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.185 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:14.185 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.185 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:14.185 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823000+0000", 2024-09-04T17:48:14.185 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:14.185 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.185 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.185 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:14.185 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823000+0000", 2024-09-04T17:48:14.186 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:14.186 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.186 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.186 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:14.186 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823000+0000", 2024-09-04T17:48:14.186 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4099999999999999e-07 2024-09-04T17:48:14.186 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.187 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.187 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:14.187 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823003+0000", 2024-09-04T17:48:14.187 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0810000000000001e-06 2024-09-04T17:48:14.187 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.187 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.187 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:14.187 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823011+0000", 2024-09-04T17:48:14.187 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7090000000000008e-06 2024-09-04T17:48:14.187 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.187 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.187 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:14.187 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823017+0000", 2024-09-04T17:48:14.188 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.3890000000000004e-06 2024-09-04T17:48:14.188 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.188 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.188 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:14.188 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897745+0000", 2024-09-04T17:48:14.188 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074728252999999994 2024-09-04T17:48:14.188 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.188 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.188 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:14.189 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897785+0000", 2024-09-04T17:48:14.189 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.9830999999999999e-05 2024-09-04T17:48:14.189 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.189 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.189 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:14.189 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897834+0000", 2024-09-04T17:48:14.189 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9231999999999999e-05 2024-09-04T17:48:14.189 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:14.190 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:14.190 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:14.190 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.190 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.190 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60928 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:14.190 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823024+0000", 2024-09-04T17:48:14.190 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 60.331182206999998, 2024-09-04T17:48:14.190 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075067479000000006, 2024-09-04T17:48:14.190 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:14.190 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:14.190 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:14.191 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:14.191 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:14.191 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:14.191 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60928 2024-09-04T17:48:14.191 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.191 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:14.191 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.191 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:14.191 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823024+0000", 2024-09-04T17:48:14.191 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:14.191 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.191 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.191 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:14.192 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823024+0000", 2024-09-04T17:48:14.192 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:14.192 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.192 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.192 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:14.192 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823024+0000", 2024-09-04T17:48:14.192 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6499999999999999e-07 2024-09-04T17:48:14.192 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.193 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.193 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:14.193 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823027+0000", 2024-09-04T17:48:14.193 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.3099999999999999e-06 2024-09-04T17:48:14.193 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.193 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.193 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:14.193 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823033+0000", 2024-09-04T17:48:14.194 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7429999999999996e-06 2024-09-04T17:48:14.194 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.194 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.194 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:14.194 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823038+0000", 2024-09-04T17:48:14.194 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.561e-06 2024-09-04T17:48:14.194 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.194 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.194 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:14.194 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897855+0000", 2024-09-04T17:48:14.195 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074816740000000007 2024-09-04T17:48:14.195 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.195 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.195 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:14.195 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897894+0000", 2024-09-04T17:48:14.195 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.9085999999999998e-05 2024-09-04T17:48:14.195 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.195 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.195 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:14.195 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898091+0000", 2024-09-04T17:48:14.196 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000197574 2024-09-04T17:48:14.196 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:14.196 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:14.196 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:14.196 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.196 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.196 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60929 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1212416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:14.196 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823042+0000", 2024-09-04T17:48:14.197 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 60.331163764999999, 2024-09-04T17:48:14.197 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075301251, 2024-09-04T17:48:14.197 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:14.197 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:14.197 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:14.197 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:14.197 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:14.197 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:14.197 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60929 2024-09-04T17:48:14.198 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.198 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:14.198 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.198 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:14.198 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823042+0000", 2024-09-04T17:48:14.198 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:14.198 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.198 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.198 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:14.198 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823042+0000", 2024-09-04T17:48:14.198 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:14.198 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.198 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.198 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:14.199 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823043+0000", 2024-09-04T17:48:14.199 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.1100000000000002e-07 2024-09-04T17:48:14.199 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.199 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.199 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:14.199 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823045+0000", 2024-09-04T17:48:14.199 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.933e-06 2024-09-04T17:48:14.199 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.199 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.200 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:14.200 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823051+0000", 2024-09-04T17:48:14.200 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.9569999999999999e-06 2024-09-04T17:48:14.200 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.200 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.200 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:14.200 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823055+0000", 2024-09-04T17:48:14.200 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4479999999999996e-06 2024-09-04T17:48:14.200 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.200 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.200 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:14.200 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898117+0000", 2024-09-04T17:48:14.201 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075062344000000003 2024-09-04T17:48:14.201 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.201 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.201 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:14.201 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898157+0000", 2024-09-04T17:48:14.201 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0095999999999997e-05 2024-09-04T17:48:14.201 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.201 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.201 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:14.201 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898344+0000", 2024-09-04T17:48:14.201 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000186162 2024-09-04T17:48:14.201 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:14.201 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:14.202 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:14.202 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.202 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.202 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60930 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1351680~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:14.202 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823061+0000", 2024-09-04T17:48:14.202 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 60.331144885999997, 2024-09-04T17:48:14.202 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075545507999999997, 2024-09-04T17:48:14.202 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:14.202 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:14.203 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:14.203 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:14.203 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:14.203 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:14.203 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60930 2024-09-04T17:48:14.203 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.203 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:14.203 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.203 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:14.203 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823061+0000", 2024-09-04T17:48:14.204 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:14.204 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.204 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.204 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:14.204 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823061+0000", 2024-09-04T17:48:14.204 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:14.204 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.205 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.205 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:14.205 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823378+0000", 2024-09-04T17:48:14.205 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00031631299999999998 2024-09-04T17:48:14.205 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.205 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.205 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:14.205 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823381+0000", 2024-09-04T17:48:14.205 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4020000000000002e-06 2024-09-04T17:48:14.205 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.205 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.205 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:14.205 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823389+0000", 2024-09-04T17:48:14.205 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.1429999999999998e-06 2024-09-04T17:48:14.206 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.206 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.206 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:14.206 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823395+0000", 2024-09-04T17:48:14.206 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.0059999999999998e-06 2024-09-04T17:48:14.206 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.206 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.206 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:14.206 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898368+0000", 2024-09-04T17:48:14.206 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074973032999999994 2024-09-04T17:48:14.206 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.207 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.207 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:14.207 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898410+0000", 2024-09-04T17:48:14.207 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2243000000000002e-05 2024-09-04T17:48:14.207 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.207 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.207 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:14.207 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898607+0000", 2024-09-04T17:48:14.207 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019636799999999999 2024-09-04T17:48:14.208 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:14.208 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:14.208 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:14.208 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.208 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.208 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60931 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1458176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:14.209 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823407+0000", 2024-09-04T17:48:14.209 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 60.330798811999998, 2024-09-04T17:48:14.209 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075460334000000004, 2024-09-04T17:48:14.209 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:14.209 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:14.209 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:14.210 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:14.210 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:14.210 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:14.210 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60931 2024-09-04T17:48:14.210 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.210 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:14.210 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.210 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:14.210 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823407+0000", 2024-09-04T17:48:14.210 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:14.211 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.211 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.211 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:14.211 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823407+0000", 2024-09-04T17:48:14.211 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:14.211 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.211 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.211 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:14.211 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823408+0000", 2024-09-04T17:48:14.211 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9999999999999998e-07 2024-09-04T17:48:14.211 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.212 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.212 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:14.212 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823415+0000", 2024-09-04T17:48:14.212 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.5510000000000002e-06 2024-09-04T17:48:14.212 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.212 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.212 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:14.212 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823423+0000", 2024-09-04T17:48:14.213 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2760000000000003e-06 2024-09-04T17:48:14.213 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.213 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.213 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:14.213 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823426+0000", 2024-09-04T17:48:14.213 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.2839999999999998e-06 2024-09-04T17:48:14.213 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.213 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.213 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:14.213 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898631+0000", 2024-09-04T17:48:14.213 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075205045999999998 2024-09-04T17:48:14.213 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.214 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.214 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:14.214 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898672+0000", 2024-09-04T17:48:14.214 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0614000000000001e-05 2024-09-04T17:48:14.214 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.214 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.214 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:14.214 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898868+0000", 2024-09-04T17:48:14.214 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019606300000000001 2024-09-04T17:48:14.214 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:14.214 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:14.214 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:14.215 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.215 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.215 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60932 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1564672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:14.215 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823434+0000", 2024-09-04T17:48:14.215 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 60.330772512999999, 2024-09-04T17:48:14.215 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075693393999999997, 2024-09-04T17:48:14.215 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:14.215 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:14.215 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:14.215 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:14.215 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:14.215 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:14.215 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60932 2024-09-04T17:48:14.215 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.216 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:14.216 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.216 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:14.216 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823434+0000", 2024-09-04T17:48:14.216 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:14.216 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.216 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.216 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:14.216 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823434+0000", 2024-09-04T17:48:14.216 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:14.216 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.216 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.217 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:14.217 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.830290+0000", 2024-09-04T17:48:14.217 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0068563039999999997 2024-09-04T17:48:14.217 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.217 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.217 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:14.217 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.830297+0000", 2024-09-04T17:48:14.217 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4760000000000001e-06 2024-09-04T17:48:14.217 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.217 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.217 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:14.218 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.830308+0000", 2024-09-04T17:48:14.218 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0927999999999999e-05 2024-09-04T17:48:14.218 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.218 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.218 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:14.218 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.830317+0000", 2024-09-04T17:48:14.218 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7280000000000001e-06 2024-09-04T17:48:14.218 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.218 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.218 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:14.218 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898892+0000", 2024-09-04T17:48:14.218 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.068574722000000005 2024-09-04T17:48:14.218 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.219 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.219 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:14.219 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898934+0000", 2024-09-04T17:48:14.219 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2249000000000002e-05 2024-09-04T17:48:14.219 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.219 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.219 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:14.219 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.899127+0000", 2024-09-04T17:48:14.219 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019298699999999999 2024-09-04T17:48:14.219 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:14.219 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:14.220 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:14.220 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.220 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.220 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61182 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 868352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:14.220 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839249+0000", 2024-09-04T17:48:14.221 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 60.314956621, 2024-09-04T17:48:14.221 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073680151999999999, 2024-09-04T17:48:14.222 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:14.222 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:14.222 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:14.222 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:14.222 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:14.222 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:14.223 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61182 2024-09-04T17:48:14.223 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.223 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:14.223 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.223 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:14.223 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839249+0000", 2024-09-04T17:48:14.223 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:14.224 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.224 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.224 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:14.224 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839249+0000", 2024-09-04T17:48:14.224 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:14.224 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.225 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.225 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:14.225 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839250+0000", 2024-09-04T17:48:14.225 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4999999999999998e-07 2024-09-04T17:48:14.225 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.225 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.225 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:14.226 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839257+0000", 2024-09-04T17:48:14.226 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7700000000000004e-06 2024-09-04T17:48:14.226 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.226 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.226 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:14.226 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839263+0000", 2024-09-04T17:48:14.226 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.246e-06 2024-09-04T17:48:14.227 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.227 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.227 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:14.227 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839268+0000", 2024-09-04T17:48:14.227 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.814e-06 2024-09-04T17:48:14.227 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.228 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.228 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:14.228 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.912831+0000", 2024-09-04T17:48:14.228 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073563294000000001 2024-09-04T17:48:14.228 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.228 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.229 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:14.229 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.912876+0000", 2024-09-04T17:48:14.229 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4742e-05 2024-09-04T17:48:14.229 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.229 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.229 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:14.229 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.912930+0000", 2024-09-04T17:48:14.230 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.3936e-05 2024-09-04T17:48:14.230 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:14.230 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:14.230 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:14.230 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.230 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.230 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61183 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:14.230 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839275+0000", 2024-09-04T17:48:14.231 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 60.314931428000001, 2024-09-04T17:48:14.231 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073781886000000005, 2024-09-04T17:48:14.231 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:14.231 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:14.231 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:14.231 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:14.231 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:14.232 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:14.232 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61183 2024-09-04T17:48:14.232 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.232 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:14.232 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.232 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:14.232 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839275+0000", 2024-09-04T17:48:14.233 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:14.233 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.233 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.233 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:14.233 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839275+0000", 2024-09-04T17:48:14.233 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:14.234 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.234 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.234 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:14.234 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839275+0000", 2024-09-04T17:48:14.234 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6899999999999997e-07 2024-09-04T17:48:14.235 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.235 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.235 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:14.235 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839277+0000", 2024-09-04T17:48:14.235 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9530000000000002e-06 2024-09-04T17:48:14.235 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.236 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.236 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:14.236 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839283+0000", 2024-09-04T17:48:14.236 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.9329999999999998e-06 2024-09-04T17:48:14.236 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.236 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.237 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:14.237 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839288+0000", 2024-09-04T17:48:14.237 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4100000000000001e-06 2024-09-04T17:48:14.237 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.237 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.237 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:14.238 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.912959+0000", 2024-09-04T17:48:14.238 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073671511999999995 2024-09-04T17:48:14.238 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.238 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.238 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:14.238 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913001+0000", 2024-09-04T17:48:14.238 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.1984999999999999e-05 2024-09-04T17:48:14.238 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.239 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.239 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:14.239 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913057+0000", 2024-09-04T17:48:14.239 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.5523999999999999e-05 2024-09-04T17:48:14.239 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:14.239 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:14.239 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:14.239 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.239 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.240 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61184 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1114112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:14.240 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839294+0000", 2024-09-04T17:48:14.240 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 60.314912216000003, 2024-09-04T17:48:14.240 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073995263000000006, 2024-09-04T17:48:14.240 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:14.240 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:14.240 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:14.240 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:14.241 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:14.241 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:14.241 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61184 2024-09-04T17:48:14.241 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.241 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:14.241 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.241 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:14.242 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839294+0000", 2024-09-04T17:48:14.242 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:14.242 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.242 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.243 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:14.243 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839294+0000", 2024-09-04T17:48:14.243 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:14.243 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.243 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.243 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:14.243 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839294+0000", 2024-09-04T17:48:14.243 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.77e-07 2024-09-04T17:48:14.244 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.244 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.244 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:14.244 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839296+0000", 2024-09-04T17:48:14.244 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9999999999999999e-06 2024-09-04T17:48:14.244 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.244 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.244 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:14.244 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839302+0000", 2024-09-04T17:48:14.244 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7230000000000001e-06 2024-09-04T17:48:14.244 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.245 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.245 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:14.245 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839314+0000", 2024-09-04T17:48:14.245 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1826e-05 2024-09-04T17:48:14.245 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.245 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.245 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:14.245 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913084+0000", 2024-09-04T17:48:14.245 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073770215 2024-09-04T17:48:14.245 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.245 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.245 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:14.245 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913127+0000", 2024-09-04T17:48:14.246 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2982999999999998e-05 2024-09-04T17:48:14.246 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.246 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.246 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:14.246 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913289+0000", 2024-09-04T17:48:14.246 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000162139 2024-09-04T17:48:14.246 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:14.246 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:14.246 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:14.246 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.246 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.246 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61185 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1220608~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:14.246 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839321+0000", 2024-09-04T17:48:14.247 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 60.314884749000001, 2024-09-04T17:48:14.247 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074206062000000003, 2024-09-04T17:48:14.247 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:14.247 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:14.247 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:14.247 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:14.247 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:14.247 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:14.247 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61185 2024-09-04T17:48:14.247 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.247 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:14.248 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.248 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:14.248 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839321+0000", 2024-09-04T17:48:14.248 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:14.248 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.248 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.248 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:14.248 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839321+0000", 2024-09-04T17:48:14.248 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:14.248 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.249 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.249 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:14.249 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839322+0000", 2024-09-04T17:48:14.249 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.5400000000000001e-07 2024-09-04T17:48:14.249 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.249 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.249 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:14.249 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839324+0000", 2024-09-04T17:48:14.249 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2539999999999999e-06 2024-09-04T17:48:14.249 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.250 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.250 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:14.250 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839331+0000", 2024-09-04T17:48:14.250 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2139999999999999e-06 2024-09-04T17:48:14.250 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.250 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.250 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:14.250 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839336+0000", 2024-09-04T17:48:14.250 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7029999999999997e-06 2024-09-04T17:48:14.250 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.250 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.250 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:14.250 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913320+0000", 2024-09-04T17:48:14.251 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073984016 2024-09-04T17:48:14.251 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.251 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.251 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:14.251 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913366+0000", 2024-09-04T17:48:14.251 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.5704000000000002e-05 2024-09-04T17:48:14.251 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.251 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.251 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:14.251 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913527+0000", 2024-09-04T17:48:14.251 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00016161699999999999 2024-09-04T17:48:14.251 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:14.251 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:14.252 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:14.252 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.252 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.252 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61186 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1359872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:14.252 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839344+0000", 2024-09-04T17:48:14.252 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 60.314862407, 2024-09-04T17:48:14.252 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074450673999999994, 2024-09-04T17:48:14.252 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:14.253 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:14.253 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:14.253 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:14.253 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:14.253 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:14.253 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61186 2024-09-04T17:48:14.254 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.254 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:14.254 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.254 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:14.254 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839344+0000", 2024-09-04T17:48:14.254 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:14.255 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.255 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.255 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:14.255 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839344+0000", 2024-09-04T17:48:14.255 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:14.255 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.255 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.255 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:14.256 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839344+0000", 2024-09-04T17:48:14.256 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4799999999999999e-07 2024-09-04T17:48:14.256 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.256 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.256 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:14.256 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839346+0000", 2024-09-04T17:48:14.256 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0889999999999998e-06 2024-09-04T17:48:14.256 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.256 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.256 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:14.256 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839352+0000", 2024-09-04T17:48:14.256 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6389999999999997e-06 2024-09-04T17:48:14.256 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.257 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.257 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:14.257 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839356+0000", 2024-09-04T17:48:14.257 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.233e-06 2024-09-04T17:48:14.257 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.257 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.257 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:14.257 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913557+0000", 2024-09-04T17:48:14.257 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074200944000000005 2024-09-04T17:48:14.257 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.257 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.257 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:14.258 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913601+0000", 2024-09-04T17:48:14.258 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4350000000000001e-05 2024-09-04T17:48:14.258 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.258 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.258 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:14.258 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913794+0000", 2024-09-04T17:48:14.258 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019297100000000001 2024-09-04T17:48:14.258 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:14.258 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:14.259 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:14.259 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.259 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.259 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61187 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1466368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:14.259 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839364+0000", 2024-09-04T17:48:14.259 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 60.314842558000002, 2024-09-04T17:48:14.259 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074692334999999999, 2024-09-04T17:48:14.259 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:14.259 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:14.260 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:14.260 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:14.260 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:14.260 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:14.260 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61187 2024-09-04T17:48:14.260 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.260 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:14.260 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.260 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:14.261 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839364+0000", 2024-09-04T17:48:14.261 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:14.261 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.261 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.261 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:14.261 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839364+0000", 2024-09-04T17:48:14.261 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:14.261 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.261 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.261 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:14.261 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839364+0000", 2024-09-04T17:48:14.261 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.75e-07 2024-09-04T17:48:14.261 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.262 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.262 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:14.262 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839366+0000", 2024-09-04T17:48:14.262 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0990000000000001e-06 2024-09-04T17:48:14.262 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.262 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.262 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:14.262 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839372+0000", 2024-09-04T17:48:14.262 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.2609999999999999e-06 2024-09-04T17:48:14.262 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.262 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.262 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:14.262 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839389+0000", 2024-09-04T17:48:14.263 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6489000000000001e-05 2024-09-04T17:48:14.263 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.263 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.263 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:14.263 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913824+0000", 2024-09-04T17:48:14.263 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074434811000000004 2024-09-04T17:48:14.263 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.263 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.263 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:14.263 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913868+0000", 2024-09-04T17:48:14.263 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4768000000000002e-05 2024-09-04T17:48:14.263 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.263 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.264 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:14.264 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914056+0000", 2024-09-04T17:48:14.264 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00018743199999999999 2024-09-04T17:48:14.264 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:14.264 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:14.264 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:14.264 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.264 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.264 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61188 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1605632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:14.264 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839399+0000", 2024-09-04T17:48:14.264 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 60.314806662000002, 2024-09-04T17:48:14.264 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074975095000000005, 2024-09-04T17:48:14.264 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:14.264 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:14.265 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:14.265 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:14.265 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:14.265 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:14.265 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61188 2024-09-04T17:48:14.265 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.265 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:14.265 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.265 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:14.266 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839399+0000", 2024-09-04T17:48:14.266 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:14.266 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.266 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.266 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:14.266 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839399+0000", 2024-09-04T17:48:14.266 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:14.266 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.266 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.267 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:14.267 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839400+0000", 2024-09-04T17:48:14.267 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0100000000000005e-07 2024-09-04T17:48:14.267 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.267 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.267 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:14.267 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839402+0000", 2024-09-04T17:48:14.267 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.491e-06 2024-09-04T17:48:14.267 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.267 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.267 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:14.267 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839409+0000", 2024-09-04T17:48:14.267 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.0450000000000003e-06 2024-09-04T17:48:14.268 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.268 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.268 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:14.268 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839414+0000", 2024-09-04T17:48:14.268 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.87e-06 2024-09-04T17:48:14.268 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.268 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.268 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:14.268 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914086+0000", 2024-09-04T17:48:14.268 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074671596000000007 2024-09-04T17:48:14.268 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.268 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.268 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:14.269 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914130+0000", 2024-09-04T17:48:14.269 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4579999999999997e-05 2024-09-04T17:48:14.269 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.269 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.269 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:14.269 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914375+0000", 2024-09-04T17:48:14.269 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024401199999999999 2024-09-04T17:48:14.269 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:14.269 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:14.269 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:14.269 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.269 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.269 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61189 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1712128~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:14.270 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839424+0000", 2024-09-04T17:48:14.270 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 60.314782461, 2024-09-04T17:48:14.270 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075247680999999997, 2024-09-04T17:48:14.270 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:14.270 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:14.270 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:14.270 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:14.270 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:14.270 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:14.270 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61189 2024-09-04T17:48:14.271 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.271 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:14.271 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.271 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:14.271 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839424+0000", 2024-09-04T17:48:14.271 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:14.271 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.271 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.271 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:14.271 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839424+0000", 2024-09-04T17:48:14.271 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:14.271 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.271 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.272 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:14.272 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.846320+0000", 2024-09-04T17:48:14.272 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0068965210000000001 2024-09-04T17:48:14.272 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.272 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.272 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:14.272 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.846328+0000", 2024-09-04T17:48:14.272 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.8189999999999992e-06 2024-09-04T17:48:14.272 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.272 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.272 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:14.273 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.846341+0000", 2024-09-04T17:48:14.273 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3338e-05 2024-09-04T17:48:14.273 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.273 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.273 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:14.273 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.846352+0000", 2024-09-04T17:48:14.273 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0479000000000001e-05 2024-09-04T17:48:14.273 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.273 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.273 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:14.273 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914405+0000", 2024-09-04T17:48:14.273 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.068053049000000004 2024-09-04T17:48:14.273 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.273 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.274 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:14.274 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914451+0000", 2024-09-04T17:48:14.274 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6545999999999998e-05 2024-09-04T17:48:14.274 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:14.274 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:14.274 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:14.274 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914671+0000", 2024-09-04T17:48:14.274 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000219929 2024-09-04T17:48:14.274 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:14.274 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:14.274 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:14.274 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:14.274 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:14.274 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:14.276 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-04T17:48:14.277 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:14.270+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:14.302 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:48:14.303 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:48:14.319 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:48:14.319 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:48:14.336 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-04T17:48:14.336 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-04T17:48:14.353 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-04T17:48:14.353 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-04T17:48:14.374 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:48:14.374 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:48:14.377 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:14.371+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:14.393 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:14.393 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:14.393 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:48:14.394 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:48:14.407 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-04T17:48:14.415 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-04T17:48:14.415 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-04T17:48:14.436 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:48:14.437 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:48:14.497 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:48:14.495+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:48:14.536 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:14.536 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:14.550 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-04T17:48:14.593 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:14.591+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:14.680 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:48:14.674+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:48:14.719 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:14.719 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:14.734 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-04T17:48:14.780 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:48:14.778+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:48:14.858 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:14.858 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:14.873 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-04T17:48:14.881 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:48:14.878+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:48:14.982 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:48:14.979+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:48:14.997 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:14.997 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:15.013 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-04T17:48:15.101 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:48:15.099+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:48:15.184 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:15.178+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:15.185 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:15.186 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:15.200 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-04T17:48:15.285 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:15.282+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:15.289 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-04T17:48:15.332 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:15.333 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:15.346 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-04T17:48:15.386 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:48:15.383+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:48:15.487 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:15.482+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:15.508 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:15.509 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:15.524 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-04T17:48:15.588 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:15.583+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:15.599 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:48:15.599 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-09-04T17:48:15.618 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":206,"stamp":"2024-09-04T17:48:15.111056+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1548317759,"num_objects":2108,"num_object_clones":280,"num_object_copies":4216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":118,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2108,"num_whiteouts":1140,"num_read":131962,"num_read_kb":867931,"num_write":18388,"num_write_kb":405400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3177,"num_bytes_recovered":196192261,"num_keys_recovered":606,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15561,"ondisk_log_size":15561,"up":82,"acting":80,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":84,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":758372,"kb_used_data":497220,"kb_used_omap":40,"kb_used_meta":260951,"kb_avail":565472668,"statfs":{"total":579820584960,"available":579044012032,"internally_reserved":0,"allocated":509153280,"data_stored":614172692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41921,"internal_metadata":267213887},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":81556294,"num_objects":182,"num_object_clones":15,"num_object_copies":364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":182,"num_whiteouts":110,"num_read":14617,"num_read_kb":69998,"num_write":1465,"num_write_kb":29437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":806,"num_bytes_recovered":64428585,"num_keys_recovered":187,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000869"},"pg_stats":[{"pgid":"3.1f","version":"125'458","reported_seq":2760,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.840024+0000","last_change":"2024-09-04T17:48:09.847643+0000","last_active":"2024-09-04T17:48:10.840024+0000","last_peered":"2024-09-04T17:48:10.840024+0000","last_clean":"2024-09-04T17:48:10.840024+0000","last_became_active":"2024-09-04T17:48:09.845199+0000","last_became_peered":"2024-09-04T17:48:09.845199+0000","last_unstale":"2024-09-04T17:48:10.840024+0000","last_undegraded":"2024-09-04T17:48:10.840024+0000","last_fullsized":"2024-09-04T17:48:10.840024+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":458,"log_dups_size":0,"ondisk_log_size":458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:59:04.183494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012519320000000001,"stat_sum":{"num_bytes":37179569,"num_objects":67,"num_object_clones":7,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":42,"num_read":2947,"num_read_kb":27164,"num_write":494,"num_write_kb":10441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":24433786,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.1e","version":"123'580","reported_seq":3319,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.843555+0000","last_change":"2024-09-04T17:48:09.855336+0000","last_active":"2024-09-04T17:48:10.843555+0000","last_peered":"2024-09-04T17:48:10.843555+0000","last_clean":"2024-09-04T17:48:10.843555+0000","last_became_active":"2024-09-04T17:48:00.841031+0000","last_became_peered":"2024-09-04T17:48:00.841031+0000","last_unstale":"2024-09-04T17:48:10.843555+0000","last_undegraded":"2024-09-04T17:48:10.843555+0000","last_fullsized":"2024-09-04T17:48:10.843555+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T19:09:57.450916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001300933,"stat_sum":{"num_bytes":56202357,"num_objects":54,"num_object_clones":6,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":26,"num_read":4487,"num_read_kb":30277,"num_write":751,"num_write_kb":10557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":10538943,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.1d","version":"126'500","reported_seq":2176,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.845642+0000","last_change":"2024-09-04T17:48:09.849221+0000","last_active":"2024-09-04T17:48:10.845642+0000","last_peered":"2024-09-04T17:48:10.845642+0000","last_clean":"2024-09-04T17:48:10.845642+0000","last_became_active":"2024-09-04T17:45:55.955213+0000","last_became_peered":"2024-09-04T17:45:55.955213+0000","last_unstale":"2024-09-04T17:48:10.845642+0000","last_undegraded":"2024-09-04T17:48:10.845642+0000","last_fullsized":"2024-09-04T17:48:10.845642+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T18:15:40.664134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0067016410000000004,"stat_sum":{"num_bytes":40967343,"num_objects":63,"num_object_clones":7,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":33,"num_read":2785,"num_read_kb":20258,"num_write":587,"num_write_kb":9396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.1c","version":"131'645","reported_seq":6132,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.845503+0000","last_change":"2024-09-04T17:48:09.845659+0000","last_active":"2024-09-04T17:48:10.845503+0000","last_peered":"2024-09-04T17:48:10.845503+0000","last_clean":"2024-09-04T17:48:10.845503+0000","last_became_active":"2024-09-04T17:45:55.951880+0000","last_became_peered":"2024-09-04T17:45:55.951880+0000","last_unstale":"2024-09-04T17:48:10.845503+0000","last_undegraded":"2024-09-04T17:48:10.845503+0000","last_fullsized":"2024-09-04T17:48:10.845503+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":645,"log_dups_size":0,"ondisk_log_size":645,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T18:55:53.482772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0046450099999999998,"stat_sum":{"num_bytes":77833746,"num_objects":77,"num_object_clones":10,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":43,"num_read":6986,"num_read_kb":47572,"num_write":746,"num_write_kb":11545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.1b","version":"125'488","reported_seq":1720,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.839099+0000","last_change":"2024-09-04T17:48:09.856409+0000","last_active":"2024-09-04T17:48:10.839099+0000","last_peered":"2024-09-04T17:48:10.839099+0000","last_clean":"2024-09-04T17:48:10.839099+0000","last_became_active":"2024-09-04T17:48:09.853793+0000","last_became_peered":"2024-09-04T17:48:09.853793+0000","last_unstale":"2024-09-04T17:48:10.839099+0000","last_undegraded":"2024-09-04T17:48:10.839099+0000","last_fullsized":"2024-09-04T17:48:10.839099+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T23:09:36.303529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010814819999999999,"stat_sum":{"num_bytes":19881692,"num_objects":67,"num_object_clones":7,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":39,"num_read":3057,"num_read_kb":7588,"num_write":565,"num_write_kb":4577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":1993914,"num_keys_recovered":48,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.1a","version":"130'362","reported_seq":5771,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-04T17:48:14.936230+0000","last_change":"2024-09-04T17:48:14.936726+0000","last_active":"2024-09-04T17:48:14.936230+0000","last_peered":"2024-09-04T17:48:14.936230+0000","last_clean":"2024-09-04T17:48:14.936230+0000","last_became_active":"2024-09-04T17:47:44.946366+0000","last_became_peered":"2024-09-04T17:47:44.946366+0000","last_unstale":"2024-09-04T17:48:14.936230+0000","last_undegraded":"2024-09-04T17:48:14.936230+0000","last_fullsized":"2024-09-04T17:48:14.936230+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":362,"log_dups_size":0,"ondisk_log_size":362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T18:31:13.955149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080444200000000005,"stat_sum":{"num_bytes":72634271,"num_objects":65,"num_object_clones":8,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":31,"num_read":5550,"num_read_kb":45438,"num_write":335,"num_write_kb":15441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":5992020,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"3.19","version":"131'361","reported_seq":2126,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.866058+0000","last_change":"2024-09-04T17:48:10.866726+0000","last_active":"2024-09-04T17:48:10.866058+0000","last_peered":"2024-09-04T17:48:10.866058+0000","last_clean":"2024-09-04T17:48:10.866058+0000","last_became_active":"2024-09-04T17:48:10.851145+0000","last_became_peered":"2024-09-04T17:48:10.851145+0000","last_unstale":"2024-09-04T17:48:10.866058+0000","last_undegraded":"2024-09-04T17:48:10.866058+0000","last_fullsized":"2024-09-04T17:48:10.866058+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":361,"log_dups_size":0,"ondisk_log_size":361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:29:15.116357+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0093494070000000006,"stat_sum":{"num_bytes":36387745,"num_objects":54,"num_object_clones":7,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":24,"num_read":2015,"num_read_kb":16004,"num_write":395,"num_write_kb":4601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":4015757,"num_keys_recovered":27,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.18","version":"125'408","reported_seq":1702,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.839094+0000","last_change":"2024-09-04T17:48:09.841174+0000","last_active":"2024-09-04T17:48:10.839094+0000","last_peered":"2024-09-04T17:48:10.839094+0000","last_clean":"2024-09-04T17:48:10.839094+0000","last_became_active":"2024-09-04T17:48:09.838648+0000","last_became_peered":"2024-09-04T17:48:09.838648+0000","last_unstale":"2024-09-04T17:48:10.839094+0000","last_undegraded":"2024-09-04T17:48:10.839094+0000","last_fullsized":"2024-09-04T17:48:10.839094+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:23:30.666247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012459330000000001,"stat_sum":{"num_bytes":70698825,"num_objects":61,"num_object_clones":6,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":32,"num_read":8379,"num_read_kb":35044,"num_write":372,"num_write_kb":14411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":288,"num_bytes_recovered":25655550,"num_keys_recovered":117,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.17","version":"125'283","reported_seq":2016,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.840918+0000","last_change":"2024-09-04T17:48:09.839869+0000","last_active":"2024-09-04T17:48:10.840918+0000","last_peered":"2024-09-04T17:48:10.840918+0000","last_clean":"2024-09-04T17:48:10.840918+0000","last_became_active":"2024-09-04T17:45:55.958625+0000","last_became_peered":"2024-09-04T17:45:55.958625+0000","last_unstale":"2024-09-04T17:48:10.840918+0000","last_undegraded":"2024-09-04T17:48:10.840918+0000","last_fullsized":"2024-09-04T17:48:10.840918+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T22:57:41.662122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021994229999999998,"stat_sum":{"num_bytes":82172026,"num_objects":63,"num_object_clones":12,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":27,"num_read":1606,"num_read_kb":20936,"num_write":292,"num_write_kb":18406,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.16","version":"126'324","reported_seq":1682,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.845454+0000","last_change":"2024-09-04T17:48:09.843759+0000","last_active":"2024-09-04T17:48:10.845454+0000","last_peered":"2024-09-04T17:48:10.845454+0000","last_clean":"2024-09-04T17:48:10.845454+0000","last_became_active":"2024-09-04T17:45:55.953254+0000","last_became_peered":"2024-09-04T17:45:55.953254+0000","last_unstale":"2024-09-04T17:48:10.845454+0000","last_undegraded":"2024-09-04T17:48:10.845454+0000","last_fullsized":"2024-09-04T17:48:10.845454+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:18:29.645248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003761838,"stat_sum":{"num_bytes":37296010,"num_objects":51,"num_object_clones":7,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":29,"num_read":1692,"num_read_kb":10758,"num_write":369,"num_write_kb":7387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.15","version":"125'455","reported_seq":1982,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-04T17:48:05.886108+0000","last_change":"2024-09-04T17:48:05.886521+0000","last_active":"2024-09-04T17:48:05.886108+0000","last_peered":"2024-09-04T17:48:05.886108+0000","last_clean":"2024-09-04T17:48:05.886108+0000","last_became_active":"2024-09-04T17:47:08.657112+0000","last_became_peered":"2024-09-04T17:47:08.657112+0000","last_unstale":"2024-09-04T17:48:05.886108+0000","last_undegraded":"2024-09-04T17:48:05.886108+0000","last_fullsized":"2024-09-04T17:48:05.886108+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:28:53.473428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021417509999999999,"stat_sum":{"num_bytes":17977259,"num_objects":46,"num_object_clones":4,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":27,"num_read":3707,"num_read_kb":16708,"num_write":482,"num_write_kb":4429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":7294656,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2b","length":"3"}]},{"pgid":"3.14","version":"125'691","reported_seq":2049,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-04T17:48:09.860236+0000","last_change":"2024-09-04T17:48:09.860717+0000","last_active":"2024-09-04T17:48:09.860236+0000","last_peered":"2024-09-04T17:48:09.860236+0000","last_clean":"2024-09-04T17:48:09.860236+0000","last_became_active":"2024-09-04T17:46:53.539886+0000","last_became_peered":"2024-09-04T17:46:53.539886+0000","last_unstale":"2024-09-04T17:48:09.860236+0000","last_undegraded":"2024-09-04T17:48:09.860236+0000","last_fullsized":"2024-09-04T17:48:09.860236+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":691,"log_dups_size":0,"ondisk_log_size":691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:39:44.106583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0051295710000000003,"stat_sum":{"num_bytes":54551302,"num_objects":66,"num_object_clones":10,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":35,"num_read":3772,"num_read_kb":29665,"num_write":939,"num_write_kb":35894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"2.7","version":"0'0","reported_seq":95,"reported_epoch":130,"state":"peering","last_fresh":"2024-09-04T17:48:09.858550+0000","last_change":"2024-09-04T17:48:10.840997+0000","last_active":"2024-09-04T17:48:10.840997+0000","last_peered":"2024-09-04T17:48:09.858550+0000","last_clean":"2024-09-04T17:48:09.858550+0000","last_became_active":"2024-09-04T17:47:57.840926+0000","last_became_peered":"2024-09-04T17:47:57.840926+0000","last_unstale":"2024-09-04T17:48:09.858550+0000","last_undegraded":"2024-09-04T17:48:09.858550+0000","last_fullsized":"2024-09-04T17:48:09.858550+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001226074,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"129'320","reported_seq":2792,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.840537+0000","last_change":"2024-09-04T17:48:09.838682+0000","last_active":"2024-09-04T17:48:10.840537+0000","last_peered":"2024-09-04T17:48:10.840537+0000","last_clean":"2024-09-04T17:48:10.840537+0000","last_became_active":"2024-09-04T17:47:32.848310+0000","last_became_peered":"2024-09-04T17:47:32.848310+0000","last_unstale":"2024-09-04T17:48:10.840537+0000","last_undegraded":"2024-09-04T17:48:10.840537+0000","last_fullsized":"2024-09-04T17:48:10.840537+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":320,"log_dups_size":0,"ondisk_log_size":320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:08:10.685688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001293892,"stat_sum":{"num_bytes":40074006,"num_objects":54,"num_object_clones":4,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":28,"num_read":2418,"num_read_kb":14504,"num_write":368,"num_write_kb":5524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":919873,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"2.6","version":"0'0","reported_seq":37,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.843292+0000","last_change":"2024-09-04T17:47:58.862844+0000","last_active":"2024-09-04T17:48:10.843292+0000","last_peered":"2024-09-04T17:48:10.843292+0000","last_clean":"2024-09-04T17:48:10.843292+0000","last_became_active":"2024-09-04T17:47:58.859941+0000","last_became_peered":"2024-09-04T17:47:58.859941+0000","last_unstale":"2024-09-04T17:48:10.843292+0000","last_undegraded":"2024-09-04T17:48:10.843292+0000","last_fullsized":"2024-09-04T17:48:10.843292+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:08:53.409036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011364179999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"129'412","reported_seq":2639,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-04T17:48:09.871164+0000","last_change":"2024-09-04T17:48:09.871644+0000","last_active":"2024-09-04T17:48:09.871164+0000","last_peered":"2024-09-04T17:48:09.871164+0000","last_clean":"2024-09-04T17:48:09.871164+0000","last_became_active":"2024-09-04T17:47:08.653159+0000","last_became_peered":"2024-09-04T17:47:08.653159+0000","last_unstale":"2024-09-04T17:48:09.871164+0000","last_undegraded":"2024-09-04T17:48:09.871164+0000","last_fullsized":"2024-09-04T17:48:09.871164+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":412,"log_dups_size":0,"ondisk_log_size":412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T21:59:05.657735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01497677,"stat_sum":{"num_bytes":43421480,"num_objects":63,"num_object_clones":8,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":36,"num_read":2882,"num_read_kb":23337,"num_write":531,"num_write_kb":21560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":952382,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"2.5","version":"0'0","reported_seq":236,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-04T17:48:09.857533+0000","last_change":"2024-09-04T17:45:49.476607+0000","last_active":"2024-09-04T17:48:09.857533+0000","last_peered":"2024-09-04T17:48:09.857533+0000","last_clean":"2024-09-04T17:48:09.857533+0000","last_became_active":"2024-09-04T17:45:47.369308+0000","last_became_peered":"2024-09-04T17:45:47.369308+0000","last_unstale":"2024-09-04T17:48:09.857533+0000","last_undegraded":"2024-09-04T17:48:09.857533+0000","last_fullsized":"2024-09-04T17:48:09.857533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:17:13.378553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001029826,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"134'359","reported_seq":2432,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-04T17:48:14.936948+0000","last_change":"2024-09-04T17:48:14.937297+0000","last_active":"2024-09-04T17:48:14.936948+0000","last_peered":"2024-09-04T17:48:14.936948+0000","last_clean":"2024-09-04T17:48:14.936948+0000","last_became_active":"2024-09-04T17:48:10.845756+0000","last_became_peered":"2024-09-04T17:48:10.845756+0000","last_unstale":"2024-09-04T17:48:14.936948+0000","last_undegraded":"2024-09-04T17:48:14.936948+0000","last_fullsized":"2024-09-04T17:48:14.936948+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:48:20.910158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079943099999999997,"stat_sum":{"num_bytes":33917693,"num_objects":59,"num_object_clones":3,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":33,"num_read":2424,"num_read_kb":18572,"num_write":397,"num_write_kb":9850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":237,"num_bytes_recovered":13375622,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"2.4","version":"0'0","reported_seq":82,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.840019+0000","last_change":"2024-09-04T17:47:29.803682+0000","last_active":"2024-09-04T17:48:10.840019+0000","last_peered":"2024-09-04T17:48:10.840019+0000","last_clean":"2024-09-04T17:48:10.840019+0000","last_became_active":"2024-09-04T17:47:29.803256+0000","last_became_peered":"2024-09-04T17:47:29.803256+0000","last_unstale":"2024-09-04T17:48:10.840019+0000","last_undegraded":"2024-09-04T17:48:10.840019+0000","last_fullsized":"2024-09-04T17:48:10.840019+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:32:26.523162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"125'273","reported_seq":1804,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.845948+0000","last_change":"2024-09-04T17:48:09.849956+0000","last_active":"2024-09-04T17:48:10.845948+0000","last_peered":"2024-09-04T17:48:10.845948+0000","last_clean":"2024-09-04T17:48:10.845948+0000","last_became_active":"2024-09-04T17:47:41.918947+0000","last_became_peered":"2024-09-04T17:47:41.918947+0000","last_unstale":"2024-09-04T17:48:10.845948+0000","last_undegraded":"2024-09-04T17:48:10.845948+0000","last_fullsized":"2024-09-04T17:48:10.845948+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":273,"log_dups_size":0,"ondisk_log_size":273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T23:41:18.557967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069886820000000004,"stat_sum":{"num_bytes":26359679,"num_objects":68,"num_object_clones":10,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":36,"num_read":2056,"num_read_kb":7215,"num_write":251,"num_write_kb":6999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":10024551,"num_keys_recovered":46,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.3","version":"131'660","reported_seq":2848,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.871783+0000","last_change":"2024-09-04T17:48:09.864268+0000","last_active":"2024-09-04T17:48:10.871783+0000","last_peered":"2024-09-04T17:48:10.871783+0000","last_clean":"2024-09-04T17:48:10.871783+0000","last_became_active":"2024-09-04T17:46:53.542889+0000","last_became_peered":"2024-09-04T17:46:53.542889+0000","last_unstale":"2024-09-04T17:48:10.871783+0000","last_undegraded":"2024-09-04T17:48:10.871783+0000","last_fullsized":"2024-09-04T17:48:10.871783+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":660,"log_dups_size":0,"ondisk_log_size":660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T22:07:45.745138+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.027057795999999999,"stat_sum":{"num_bytes":62897701,"num_objects":68,"num_object_clones":11,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":37,"num_read":4342,"num_read_kb":26256,"num_write":889,"num_write_kb":14093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"2.2","version":"22'2","reported_seq":255,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.845294+0000","last_change":"2024-09-04T17:47:59.598241+0000","last_active":"2024-09-04T17:48:10.845294+0000","last_peered":"2024-09-04T17:48:10.845294+0000","last_clean":"2024-09-04T17:48:10.845294+0000","last_became_active":"2024-09-04T17:47:58.884634+0000","last_became_peered":"2024-09-04T17:47:58.884634+0000","last_unstale":"2024-09-04T17:48:10.845294+0000","last_undegraded":"2024-09-04T17:48:10.845294+0000","last_fullsized":"2024-09-04T17:48:10.845294+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T22:02:31.692385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026316069999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"134'376","reported_seq":2890,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-04T17:48:14.936100+0000","last_change":"2024-09-04T17:48:14.936506+0000","last_active":"2024-09-04T17:48:14.936100+0000","last_peered":"2024-09-04T17:48:14.936100+0000","last_clean":"2024-09-04T17:48:14.936100+0000","last_became_active":"2024-09-04T17:48:01.849363+0000","last_became_peered":"2024-09-04T17:48:01.849363+0000","last_unstale":"2024-09-04T17:48:14.936100+0000","last_undegraded":"2024-09-04T17:48:14.936100+0000","last_fullsized":"2024-09-04T17:48:14.936100+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:28:48.800572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072198599999999998,"stat_sum":{"num_bytes":41098591,"num_objects":69,"num_object_clones":8,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":42,"num_read":3147,"num_read_kb":22161,"num_write":314,"num_write_kb":4881,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":5692590,"num_keys_recovered":35,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"2.1","version":"0'0","reported_seq":226,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-04T17:48:14.935690+0000","last_change":"2024-09-04T17:47:41.913896+0000","last_active":"2024-09-04T17:48:14.935690+0000","last_peered":"2024-09-04T17:48:14.935690+0000","last_clean":"2024-09-04T17:48:14.935690+0000","last_became_active":"2024-09-04T17:47:41.913057+0000","last_became_peered":"2024-09-04T17:47:41.913057+0000","last_unstale":"2024-09-04T17:48:14.935690+0000","last_undegraded":"2024-09-04T17:48:14.935690+0000","last_fullsized":"2024-09-04T17:48:14.935690+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:49:25.382532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001025195,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"126'527","reported_seq":2810,"reported_epoch":130,"state":"peering","last_fresh":"2024-09-04T17:48:10.253927+0000","last_change":"2024-09-04T17:48:10.839758+0000","last_active":"2024-09-04T17:48:10.839758+0000","last_peered":"2024-09-04T17:48:10.253927+0000","last_clean":"2024-09-04T17:48:06.888233+0000","last_became_active":"2024-09-04T17:48:09.878998+0000","last_became_peered":"2024-09-04T17:48:09.878998+0000","last_unstale":"2024-09-04T17:48:10.253927+0000","last_undegraded":"2024-09-04T17:48:10.253927+0000","last_fullsized":"2024-09-04T17:48:09.837030+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0052483269999999997,"stat_sum":{"num_bytes":87377265,"num_objects":81,"num_object_clones":18,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":41,"num_read":2920,"num_read_kb":40212,"num_write":602,"num_write_kb":23345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":240,"num_bytes_recovered":21297737,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,5],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2a","length":"4"}]},{"pgid":"2.0","version":"0'0","reported_seq":242,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-04T17:48:09.854561+0000","last_change":"2024-09-04T17:47:29.804429+0000","last_active":"2024-09-04T17:48:09.854561+0000","last_peered":"2024-09-04T17:48:09.854561+0000","last_clean":"2024-09-04T17:48:09.854561+0000","last_became_active":"2024-09-04T17:47:29.803992+0000","last_became_peered":"2024-09-04T17:47:29.803992+0000","last_unstale":"2024-09-04T17:48:09.854561+0000","last_undegraded":"2024-09-04T17:48:09.854561+0000","last_fullsized":"2024-09-04T17:48:09.854561+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:50:32.563345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010553590000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"126'525","reported_seq":2255,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.845194+0000","last_change":"2024-09-04T17:48:09.844640+0000","last_active":"2024-09-04T17:48:10.845194+0000","last_peered":"2024-09-04T17:48:10.845194+0000","last_clean":"2024-09-04T17:48:10.845194+0000","last_became_active":"2024-09-04T17:48:01.850400+0000","last_became_peered":"2024-09-04T17:48:01.850400+0000","last_unstale":"2024-09-04T17:48:10.845194+0000","last_undegraded":"2024-09-04T17:48:10.845194+0000","last_fullsized":"2024-09-04T17:48:10.845194+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T18:01:48.725657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049494070000000003,"stat_sum":{"num_bytes":50198348,"num_objects":64,"num_object_clones":9,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":33,"num_read":5233,"num_read_kb":29752,"num_write":673,"num_write_kb":13801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":13799490,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"2.3","version":"20'1","reported_seq":249,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.845299+0000","last_change":"2024-09-04T17:47:41.919544+0000","last_active":"2024-09-04T17:48:10.845299+0000","last_peered":"2024-09-04T17:48:10.845299+0000","last_clean":"2024-09-04T17:48:10.845299+0000","last_became_active":"2024-09-04T17:47:41.919027+0000","last_became_peered":"2024-09-04T17:47:41.919027+0000","last_unstale":"2024-09-04T17:48:10.845299+0000","last_undegraded":"2024-09-04T17:48:10.845299+0000","last_fullsized":"2024-09-04T17:48:10.845299+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:42:55.688562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086618299999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":327,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-04T17:48:09.855144+0000","last_change":"2024-09-04T17:45:42.341149+0000","last_active":"2024-09-04T17:48:09.855144+0000","last_peered":"2024-09-04T17:48:09.855144+0000","last_clean":"2024-09-04T17:48:09.855144+0000","last_became_active":"2024-09-04T17:45:42.339286+0000","last_became_peered":"2024-09-04T17:45:42.339286+0000","last_unstale":"2024-09-04T17:48:09.855144+0000","last_undegraded":"2024-09-04T17:48:09.855144+0000","last_fullsized":"2024-09-04T17:48:09.855144+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:41.305128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:41.305128+0000","last_clean_scrub_stamp":"2024-09-04T17:45:41.305128+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:52:24.860049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"126'483","reported_seq":2668,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-04T17:48:09.855504+0000","last_change":"2024-09-04T17:48:09.857214+0000","last_active":"2024-09-04T17:48:09.855504+0000","last_peered":"2024-09-04T17:48:09.855504+0000","last_clean":"2024-09-04T17:48:09.855504+0000","last_became_active":"2024-09-04T17:47:32.847682+0000","last_became_peered":"2024-09-04T17:47:32.847682+0000","last_unstale":"2024-09-04T17:48:09.855504+0000","last_undegraded":"2024-09-04T17:48:09.855504+0000","last_fullsized":"2024-09-04T17:48:09.855504+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":483,"log_dups_size":0,"ondisk_log_size":483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:07:21.110914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002703474,"stat_sum":{"num_bytes":45080057,"num_objects":72,"num_object_clones":11,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":40,"num_read":2705,"num_read_kb":17207,"num_write":531,"num_write_kb":29364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":14692620,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.9","version":"134'593","reported_seq":4140,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-04T17:48:14.936869+0000","last_change":"2024-09-04T17:48:14.937193+0000","last_active":"2024-09-04T17:48:14.936869+0000","last_peered":"2024-09-04T17:48:14.936869+0000","last_clean":"2024-09-04T17:48:14.936869+0000","last_became_active":"2024-09-04T17:47:44.949630+0000","last_became_peered":"2024-09-04T17:47:44.949630+0000","last_unstale":"2024-09-04T17:48:14.936869+0000","last_undegraded":"2024-09-04T17:48:14.936869+0000","last_fullsized":"2024-09-04T17:48:14.936869+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T21:19:18.496371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012454479999999999,"stat_sum":{"num_bytes":55724561,"num_objects":78,"num_object_clones":14,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":42,"num_read":5701,"num_read_kb":37078,"num_write":682,"num_write_kb":18628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":4534479,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"3.a","version":"131'533","reported_seq":3019,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.890923+0000","last_change":"2024-09-04T17:48:09.853469+0000","last_active":"2024-09-04T17:48:10.890923+0000","last_peered":"2024-09-04T17:48:10.890923+0000","last_clean":"2024-09-04T17:48:10.890923+0000","last_became_active":"2024-09-04T17:48:00.841070+0000","last_became_peered":"2024-09-04T17:48:00.841070+0000","last_unstale":"2024-09-04T17:48:10.890923+0000","last_undegraded":"2024-09-04T17:48:10.890923+0000","last_fullsized":"2024-09-04T17:48:10.890923+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:47:07.864899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038466329999999999,"stat_sum":{"num_bytes":36655948,"num_objects":72,"num_object_clones":11,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":40,"num_read":5202,"num_read_kb":16044,"num_write":529,"num_write_kb":2741,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":1572433,"num_keys_recovered":66,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.b","version":"125'677","reported_seq":2207,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.840785+0000","last_change":"2024-09-04T17:48:09.840108+0000","last_active":"2024-09-04T17:48:10.840785+0000","last_peered":"2024-09-04T17:48:10.840785+0000","last_clean":"2024-09-04T17:48:10.840785+0000","last_became_active":"2024-09-04T17:47:08.657286+0000","last_became_peered":"2024-09-04T17:47:08.657286+0000","last_unstale":"2024-09-04T17:48:10.840785+0000","last_undegraded":"2024-09-04T17:48:10.840785+0000","last_fullsized":"2024-09-04T17:48:10.840785+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":677,"log_dups_size":0,"ondisk_log_size":677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T19:10:08.386664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023202159999999999,"stat_sum":{"num_bytes":31620378,"num_objects":71,"num_object_clones":10,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":33,"num_read":3911,"num_read_kb":14517,"num_write":845,"num_write_kb":7911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":3759450,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.c","version":"129'552","reported_seq":7797,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.845669+0000","last_change":"2024-09-04T17:48:09.845373+0000","last_active":"2024-09-04T17:48:10.845669+0000","last_peered":"2024-09-04T17:48:10.845669+0000","last_clean":"2024-09-04T17:48:10.845669+0000","last_became_active":"2024-09-04T17:48:02.855127+0000","last_became_peered":"2024-09-04T17:48:02.855127+0000","last_unstale":"2024-09-04T17:48:10.845669+0000","last_undegraded":"2024-09-04T17:48:10.845669+0000","last_fullsized":"2024-09-04T17:48:10.845669+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":552,"log_dups_size":0,"ondisk_log_size":552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:49:26.298655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035566629999999998,"stat_sum":{"num_bytes":46296333,"num_objects":74,"num_object_clones":11,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":43,"num_read":8646,"num_read_kb":71089,"num_write":690,"num_write_kb":16847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":153,"num_bytes_recovered":7666524,"num_keys_recovered":34,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.d","version":"126'1282","reported_seq":3848,"reported_epoch":130,"state":"remapped+peering","last_fresh":"2024-09-04T17:48:09.855528+0000","last_change":"2024-09-04T17:48:09.855528+0000","last_active":"2024-09-04T17:48:08.833037+0000","last_peered":"2024-09-04T17:48:07.832005+0000","last_clean":"2024-09-04T17:48:07.832005+0000","last_became_active":"2024-09-04T17:48:00.840986+0000","last_became_peered":"2024-09-04T17:48:00.840986+0000","last_unstale":"2024-09-04T17:48:09.855528+0000","last_undegraded":"2024-09-04T17:48:09.855528+0000","last_fullsized":"2024-09-04T17:48:09.855528+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":1282,"log_dups_size":0,"ondisk_log_size":1282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018889099999999999,"stat_sum":{"num_bytes":49889564,"num_objects":60,"num_object_clones":6,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":34,"num_read":8526,"num_read_kb":26249,"num_write":2192,"num_write_kb":13182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":8914239,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2a","length":"4"}]},{"pgid":"3.e","version":"127'440","reported_seq":5206,"reported_epoch":130,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-04T17:48:09.886153+0000","last_change":"2024-09-04T17:48:09.886153+0000","last_active":"2024-09-04T17:48:09.886153+0000","last_peered":"2024-09-04T17:48:09.886153+0000","last_clean":"2024-09-04T17:48:06.890201+0000","last_became_active":"2024-09-04T17:48:09.884964+0000","last_became_peered":"2024-09-04T17:48:09.884964+0000","last_unstale":"2024-09-04T17:48:09.886153+0000","last_undegraded":"2024-09-04T17:48:09.857903+0000","last_fullsized":"2024-09-04T17:48:09.854505+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.010017414000000001,"stat_sum":{"num_bytes":63389770,"num_objects":66,"num_object_clones":13,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":118,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":32,"num_read":6129,"num_read_kb":28040,"num_write":457,"num_write_kb":11211,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":3926000,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7],"avail_no_missing":["7","0"],"object_location_counts":[{"shards":"0,7","objects":66}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2a","length":"4"}]},{"pgid":"3.f","version":"130'292","reported_seq":2844,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-04T17:48:09.870980+0000","last_change":"2024-09-04T17:48:09.870980+0000","last_active":"2024-09-04T17:48:09.870980+0000","last_peered":"2024-09-04T17:48:09.870980+0000","last_clean":"2024-09-04T17:48:09.870980+0000","last_became_active":"2024-09-04T17:45:55.956734+0000","last_became_peered":"2024-09-04T17:45:55.956734+0000","last_unstale":"2024-09-04T17:48:09.870980+0000","last_undegraded":"2024-09-04T17:48:09.870980+0000","last_fullsized":"2024-09-04T17:48:09.870980+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":292,"log_dups_size":0,"ondisk_log_size":292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T23:59:11.397371+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.01496131,"stat_sum":{"num_bytes":49550709,"num_objects":69,"num_object_clones":12,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":36,"num_read":2563,"num_read_kb":44941,"num_write":297,"num_write_kb":13423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.10","version":"126'514","reported_seq":3148,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.842877+0000","last_change":"2024-09-04T17:48:09.851750+0000","last_active":"2024-09-04T17:48:10.842877+0000","last_peered":"2024-09-04T17:48:10.842877+0000","last_clean":"2024-09-04T17:48:10.842877+0000","last_became_active":"2024-09-04T17:48:00.843136+0000","last_became_peered":"2024-09-04T17:48:00.843136+0000","last_unstale":"2024-09-04T17:48:10.842877+0000","last_undegraded":"2024-09-04T17:48:10.842877+0000","last_fullsized":"2024-09-04T17:48:10.842877+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":514,"log_dups_size":0,"ondisk_log_size":514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T23:07:02.271876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049468649999999999,"stat_sum":{"num_bytes":51575959,"num_objects":78,"num_object_clones":10,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":46,"num_read":4069,"num_read_kb":18348,"num_write":591,"num_write_kb":12832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":3377599,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.11","version":"130'500","reported_seq":5739,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-04T17:48:09.877129+0000","last_change":"2024-09-04T17:48:09.879368+0000","last_active":"2024-09-04T17:48:09.877129+0000","last_peered":"2024-09-04T17:48:09.877129+0000","last_clean":"2024-09-04T17:48:09.877129+0000","last_became_active":"2024-09-04T17:45:55.951870+0000","last_became_peered":"2024-09-04T17:45:55.951870+0000","last_unstale":"2024-09-04T17:48:09.877129+0000","last_undegraded":"2024-09-04T17:48:09.877129+0000","last_fullsized":"2024-09-04T17:48:09.877129+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T22:56:47.076439+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.024746269000000001,"stat_sum":{"num_bytes":74079184,"num_objects":82,"num_object_clones":10,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":46,"num_read":6681,"num_read_kb":61026,"num_write":514,"num_write_kb":17560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.12","version":"130'349","reported_seq":3430,"reported_epoch":130,"state":"peering","last_fresh":"2024-09-04T17:48:09.930543+0000","last_change":"2024-09-04T17:48:10.839711+0000","last_active":"2024-09-04T17:48:10.839711+0000","last_peered":"2024-09-04T17:48:09.930543+0000","last_clean":"2024-09-04T17:48:09.930543+0000","last_became_active":"2024-09-04T17:47:44.952109+0000","last_became_peered":"2024-09-04T17:47:44.952109+0000","last_unstale":"2024-09-04T17:48:09.930543+0000","last_undegraded":"2024-09-04T17:48:09.930543+0000","last_fullsized":"2024-09-04T17:48:09.930543+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017447070000000001,"stat_sum":{"num_bytes":29554039,"num_objects":60,"num_object_clones":5,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":35,"num_read":3177,"num_read_kb":21378,"num_write":389,"num_write_kb":8559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":1762027,"num_keys_recovered":48,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,7],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.13","version":"126'264","reported_seq":2227,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-04T17:48:09.857330+0000","last_change":"2024-09-04T17:48:09.860061+0000","last_active":"2024-09-04T17:48:09.857330+0000","last_peered":"2024-09-04T17:48:09.857330+0000","last_clean":"2024-09-04T17:48:09.857330+0000","last_became_active":"2024-09-04T17:45:55.954544+0000","last_became_peered":"2024-09-04T17:45:55.954544+0000","last_unstale":"2024-09-04T17:48:09.857330+0000","last_undegraded":"2024-09-04T17:48:09.857330+0000","last_fullsized":"2024-09-04T17:48:09.857330+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":264,"log_dups_size":0,"ondisk_log_size":264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T23:14:42.541437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0051521240000000001,"stat_sum":{"num_bytes":25183962,"num_objects":62,"num_object_clones":5,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":39,"num_read":2181,"num_read_kb":22538,"num_write":216,"num_write_kb":4374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"29","length":"5"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1547727372,"num_objects":2104,"num_object_clones":280,"num_object_copies":4208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":118,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2104,"num_whiteouts":1140,"num_read":131896,"num_read_kb":867876,"num_write":18285,"num_write_kb":403770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3176,"num_bytes_recovered":196192242,"num_keys_recovered":606,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":502349824,"data_stored":611685101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21395,"internal_metadata":0},"log_size":15486,"ondisk_log_size":15486,"up":64,"acting":62,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":117,"seq":502511173636,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":132212,"kb_used_data":98456,"kb_used_omap":22,"kb_used_meta":33705,"kb_avail":94239628,"statfs":{"total":96636764160,"available":96501379072,"internally_reserved":0,"allocated":100818944,"data_stored":103901506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22742,"internal_metadata":34514730},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509473,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":106128,"kb_used_data":79096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94265712,"statfs":{"total":96636764160,"available":96528089088,"internally_reserved":0,"allocated":80994304,"data_stored":99363602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509474,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":172236,"kb_used_data":114672,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94199604,"statfs":{"total":96636764160,"available":96460394496,"internally_reserved":0,"allocated":117424128,"data_stored":147072672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":0,"seq":206158430230,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":64424509473,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509473,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":124568,"kb_used_data":75328,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94247272,"statfs":{"total":96636764160,"available":96509206528,"internally_reserved":0,"allocated":77135872,"data_stored":95452503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542177,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":137620,"kb_used_data":84216,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94234220,"statfs":{"total":96636764160,"available":96495841280,"internally_reserved":0,"allocated":86237184,"data_stored":111513171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":103,"seq":442381631496,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":85608,"kb_used_data":45452,"kb_used_omap":12,"kb_used_meta":40115,"kb_avail":94286232,"statfs":{"total":96636764160,"available":96549101568,"internally_reserved":0,"allocated":46542848,"data_stored":56869238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12804,"internal_metadata":41078268},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":74981376,"data_stored":94116394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2035712,"data_stored":2023494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":44945408,"data_stored":56105331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6121,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":7245824,"data_stored":9302332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":115863552,"data_stored":146326931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":81965056,"data_stored":101128065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":99254272,"data_stored":103158122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15274,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":76058624,"data_stored":99524432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-04T17:48:15.620 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-04T17:48:15.689 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:48:15.686+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:48:15.694 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:15.694 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:15.708 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-04T17:48:15.790 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:48:15.787+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:48:15.831 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:15.831 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:15.846 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-04T17:48:15.890 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:15.886+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:15.967 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-04T17:48:16.013 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:48:16.011+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:48:16.023 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:16.023 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:16.033 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:48:16.033 INFO:teuthology.orchestra.run.smithi092.stdout:{"fsid":"f36b6ce2-8c72-4c99-8607-5c53c38ef67b","health":{"status":"HEALTH_WARN","checks":{"OSDMAP_FLAGS":{"severity":"HEALTH_WARN","summary":{"message":"nodeep-scrub flag(s) set","count":12},"muted":false},"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 118/4216 objects degraded (2.799%), 1 pg degraded","count":1},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":167,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":136,"num_osds":8,"num_up_osds":8,"osd_up_since":1725472077,"num_in_osds":6,"osd_in_since":1725472048,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":36},{"state_name":"peering","count":3},{"state_name":"remapped+peering","count":1},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1}],"num_pgs":41,"num_pools":3,"num_objects":2108,"data_bytes":1548317759,"bytes_used":776572928,"bytes_avail":579044012032,"bytes_total":579820584960,"inactive_pgs_ratio":0.097560971975326538,"degraded_objects":118,"degraded_total":4216,"degraded_ratio":0.027988614800759013,"recovering_objects_per_sec":201,"recovering_bytes_per_sec":16103647,"recovering_keys_per_sec":46,"num_objects_recovered":806,"num_bytes_recovered":64428585,"num_keys_recovered":187,"read_bytes_sec":17915595,"write_bytes_sec":7534234,"read_op_per_sec":3653,"write_op_per_sec":366},"fsmap":{"epoch":1,"btime":"2024-09-04T17:45:28:071986+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-04T17:47:29.098058+0000","services":{}},"progress_events":{}} 2024-09-04T17:48:16.039 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-04T17:48:16.053 INFO:tasks.ceph.ceph_manager.ceph:making progress, resetting timeout 2024-09-04T17:48:16.053 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-04T17:48:16.102 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:16.099+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:16.193 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:48:16.191+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:48:16.218 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:16.218 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:16.233 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-04T17:48:16.321 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:48:16.319+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:48:16.388 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:16.389 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:16.395 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:48:16.395 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-09-04T17:48:16.397 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:16.394+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:16.403 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-04T17:48:16.412 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":207,"stamp":"2024-09-04T17:48:15.953829+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1508023166,"num_objects":2088,"num_object_clones":273,"num_object_copies":4176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2088,"num_whiteouts":1135,"num_read":131969,"num_read_kb":867934,"num_write":18396,"num_write_kb":405400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3490,"num_bytes_recovered":212950074,"num_keys_recovered":652,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15583,"ondisk_log_size":15583,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":753824,"kb_used_data":492680,"kb_used_omap":40,"kb_used_meta":260951,"kb_avail":565477216,"statfs":{"total":579820584960,"available":579048669184,"internally_reserved":0,"allocated":504504320,"data_stored":606748292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41921,"internal_metadata":267213887},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-47708190,"num_objects":-26,"num_object_clones":-10,"num_object_copies":-52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-118,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-26,"num_whiteouts":-7,"num_read":32,"num_read_kb":22,"num_write":11,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":313,"num_bytes_recovered":16757813,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.117051"},"pg_stats":[{"pgid":"3.1f","version":"125'458","reported_seq":2760,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.840024+0000","last_change":"2024-09-04T17:48:09.847643+0000","last_active":"2024-09-04T17:48:10.840024+0000","last_peered":"2024-09-04T17:48:10.840024+0000","last_clean":"2024-09-04T17:48:10.840024+0000","last_became_active":"2024-09-04T17:48:09.845199+0000","last_became_peered":"2024-09-04T17:48:09.845199+0000","last_unstale":"2024-09-04T17:48:10.840024+0000","last_undegraded":"2024-09-04T17:48:10.840024+0000","last_fullsized":"2024-09-04T17:48:10.840024+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":458,"log_dups_size":0,"ondisk_log_size":458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:59:04.183494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012519320000000001,"stat_sum":{"num_bytes":37179569,"num_objects":67,"num_object_clones":7,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":42,"num_read":2947,"num_read_kb":27164,"num_write":494,"num_write_kb":10441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":24433786,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.1e","version":"123'580","reported_seq":3319,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.843555+0000","last_change":"2024-09-04T17:48:09.855336+0000","last_active":"2024-09-04T17:48:10.843555+0000","last_peered":"2024-09-04T17:48:10.843555+0000","last_clean":"2024-09-04T17:48:10.843555+0000","last_became_active":"2024-09-04T17:48:00.841031+0000","last_became_peered":"2024-09-04T17:48:00.841031+0000","last_unstale":"2024-09-04T17:48:10.843555+0000","last_undegraded":"2024-09-04T17:48:10.843555+0000","last_fullsized":"2024-09-04T17:48:10.843555+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T19:09:57.450916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001300933,"stat_sum":{"num_bytes":56202357,"num_objects":54,"num_object_clones":6,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":26,"num_read":4487,"num_read_kb":30277,"num_write":751,"num_write_kb":10557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":10538943,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.1d","version":"126'500","reported_seq":2176,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.845642+0000","last_change":"2024-09-04T17:48:09.849221+0000","last_active":"2024-09-04T17:48:10.845642+0000","last_peered":"2024-09-04T17:48:10.845642+0000","last_clean":"2024-09-04T17:48:10.845642+0000","last_became_active":"2024-09-04T17:45:55.955213+0000","last_became_peered":"2024-09-04T17:45:55.955213+0000","last_unstale":"2024-09-04T17:48:10.845642+0000","last_undegraded":"2024-09-04T17:48:10.845642+0000","last_fullsized":"2024-09-04T17:48:10.845642+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T18:15:40.664134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0067016410000000004,"stat_sum":{"num_bytes":40967343,"num_objects":63,"num_object_clones":7,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":33,"num_read":2785,"num_read_kb":20258,"num_write":587,"num_write_kb":9396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.1c","version":"131'645","reported_seq":6132,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.845503+0000","last_change":"2024-09-04T17:48:09.845659+0000","last_active":"2024-09-04T17:48:10.845503+0000","last_peered":"2024-09-04T17:48:10.845503+0000","last_clean":"2024-09-04T17:48:10.845503+0000","last_became_active":"2024-09-04T17:45:55.951880+0000","last_became_peered":"2024-09-04T17:45:55.951880+0000","last_unstale":"2024-09-04T17:48:10.845503+0000","last_undegraded":"2024-09-04T17:48:10.845503+0000","last_fullsized":"2024-09-04T17:48:10.845503+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":645,"log_dups_size":0,"ondisk_log_size":645,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T18:55:53.482772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0046450099999999998,"stat_sum":{"num_bytes":77833746,"num_objects":77,"num_object_clones":10,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":43,"num_read":6986,"num_read_kb":47572,"num_write":746,"num_write_kb":11545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.1b","version":"125'488","reported_seq":1720,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.839099+0000","last_change":"2024-09-04T17:48:09.856409+0000","last_active":"2024-09-04T17:48:10.839099+0000","last_peered":"2024-09-04T17:48:10.839099+0000","last_clean":"2024-09-04T17:48:10.839099+0000","last_became_active":"2024-09-04T17:48:09.853793+0000","last_became_peered":"2024-09-04T17:48:09.853793+0000","last_unstale":"2024-09-04T17:48:10.839099+0000","last_undegraded":"2024-09-04T17:48:10.839099+0000","last_fullsized":"2024-09-04T17:48:10.839099+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T23:09:36.303529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010814819999999999,"stat_sum":{"num_bytes":19881692,"num_objects":67,"num_object_clones":7,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":39,"num_read":3057,"num_read_kb":7588,"num_write":565,"num_write_kb":4577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":1993914,"num_keys_recovered":48,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.1a","version":"130'362","reported_seq":5771,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-04T17:48:14.936230+0000","last_change":"2024-09-04T17:48:14.936726+0000","last_active":"2024-09-04T17:48:14.936230+0000","last_peered":"2024-09-04T17:48:14.936230+0000","last_clean":"2024-09-04T17:48:14.936230+0000","last_became_active":"2024-09-04T17:47:44.946366+0000","last_became_peered":"2024-09-04T17:47:44.946366+0000","last_unstale":"2024-09-04T17:48:14.936230+0000","last_undegraded":"2024-09-04T17:48:14.936230+0000","last_fullsized":"2024-09-04T17:48:14.936230+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":362,"log_dups_size":0,"ondisk_log_size":362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T18:31:13.955149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080444200000000005,"stat_sum":{"num_bytes":72634271,"num_objects":65,"num_object_clones":8,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":31,"num_read":5550,"num_read_kb":45438,"num_write":335,"num_write_kb":15441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":5992020,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"3.19","version":"131'361","reported_seq":2126,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.866058+0000","last_change":"2024-09-04T17:48:10.866726+0000","last_active":"2024-09-04T17:48:10.866058+0000","last_peered":"2024-09-04T17:48:10.866058+0000","last_clean":"2024-09-04T17:48:10.866058+0000","last_became_active":"2024-09-04T17:48:10.851145+0000","last_became_peered":"2024-09-04T17:48:10.851145+0000","last_unstale":"2024-09-04T17:48:10.866058+0000","last_undegraded":"2024-09-04T17:48:10.866058+0000","last_fullsized":"2024-09-04T17:48:10.866058+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":361,"log_dups_size":0,"ondisk_log_size":361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:29:15.116357+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0093494070000000006,"stat_sum":{"num_bytes":36387745,"num_objects":54,"num_object_clones":7,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":24,"num_read":2015,"num_read_kb":16004,"num_write":395,"num_write_kb":4601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":4015757,"num_keys_recovered":27,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.18","version":"125'408","reported_seq":1702,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.839094+0000","last_change":"2024-09-04T17:48:09.841174+0000","last_active":"2024-09-04T17:48:10.839094+0000","last_peered":"2024-09-04T17:48:10.839094+0000","last_clean":"2024-09-04T17:48:10.839094+0000","last_became_active":"2024-09-04T17:48:09.838648+0000","last_became_peered":"2024-09-04T17:48:09.838648+0000","last_unstale":"2024-09-04T17:48:10.839094+0000","last_undegraded":"2024-09-04T17:48:10.839094+0000","last_fullsized":"2024-09-04T17:48:10.839094+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:23:30.666247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012459330000000001,"stat_sum":{"num_bytes":70698825,"num_objects":61,"num_object_clones":6,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":32,"num_read":8379,"num_read_kb":35044,"num_write":372,"num_write_kb":14411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":288,"num_bytes_recovered":25655550,"num_keys_recovered":117,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.17","version":"125'283","reported_seq":2016,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.840918+0000","last_change":"2024-09-04T17:48:09.839869+0000","last_active":"2024-09-04T17:48:10.840918+0000","last_peered":"2024-09-04T17:48:10.840918+0000","last_clean":"2024-09-04T17:48:10.840918+0000","last_became_active":"2024-09-04T17:45:55.958625+0000","last_became_peered":"2024-09-04T17:45:55.958625+0000","last_unstale":"2024-09-04T17:48:10.840918+0000","last_undegraded":"2024-09-04T17:48:10.840918+0000","last_fullsized":"2024-09-04T17:48:10.840918+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T22:57:41.662122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021994229999999998,"stat_sum":{"num_bytes":82172026,"num_objects":63,"num_object_clones":12,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":27,"num_read":1606,"num_read_kb":20936,"num_write":292,"num_write_kb":18406,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.16","version":"126'324","reported_seq":1682,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.845454+0000","last_change":"2024-09-04T17:48:09.843759+0000","last_active":"2024-09-04T17:48:10.845454+0000","last_peered":"2024-09-04T17:48:10.845454+0000","last_clean":"2024-09-04T17:48:10.845454+0000","last_became_active":"2024-09-04T17:45:55.953254+0000","last_became_peered":"2024-09-04T17:45:55.953254+0000","last_unstale":"2024-09-04T17:48:10.845454+0000","last_undegraded":"2024-09-04T17:48:10.845454+0000","last_fullsized":"2024-09-04T17:48:10.845454+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:18:29.645248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003761838,"stat_sum":{"num_bytes":37296010,"num_objects":51,"num_object_clones":7,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":29,"num_read":1692,"num_read_kb":10758,"num_write":369,"num_write_kb":7387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.15","version":"125'455","reported_seq":2022,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-04T17:48:14.955297+0000","last_change":"2024-09-04T17:48:14.955297+0000","last_active":"2024-09-04T17:48:14.955297+0000","last_peered":"2024-09-04T17:48:14.955297+0000","last_clean":"2024-09-04T17:48:14.955297+0000","last_became_active":"2024-09-04T17:48:14.948212+0000","last_became_peered":"2024-09-04T17:48:14.948212+0000","last_unstale":"2024-09-04T17:48:14.955297+0000","last_undegraded":"2024-09-04T17:48:14.955297+0000","last_fullsized":"2024-09-04T17:48:14.955297+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:59:03.009864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002853806,"stat_sum":{"num_bytes":17977259,"num_objects":46,"num_object_clones":4,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":27,"num_read":3707,"num_read_kb":16708,"num_write":482,"num_write_kb":4429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":176,"num_bytes_recovered":10208507,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"3.14","version":"134'694","reported_seq":2072,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-04T17:48:14.944280+0000","last_change":"2024-09-04T17:48:14.945013+0000","last_active":"2024-09-04T17:48:14.944280+0000","last_peered":"2024-09-04T17:48:14.944280+0000","last_clean":"2024-09-04T17:48:14.944280+0000","last_became_active":"2024-09-04T17:46:53.539886+0000","last_became_peered":"2024-09-04T17:46:53.539886+0000","last_unstale":"2024-09-04T17:48:14.944280+0000","last_undegraded":"2024-09-04T17:48:14.944280+0000","last_fullsized":"2024-09-04T17:48:14.944280+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":694,"log_dups_size":0,"ondisk_log_size":694,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:04:08.592215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025017669999999998,"stat_sum":{"num_bytes":52880134,"num_objects":64,"num_object_clones":9,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":35,"num_read":3772,"num_read_kb":29665,"num_write":940,"num_write_kb":35894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"2.7","version":"0'0","reported_seq":95,"reported_epoch":130,"state":"peering","last_fresh":"2024-09-04T17:48:09.858550+0000","last_change":"2024-09-04T17:48:10.840997+0000","last_active":"2024-09-04T17:48:10.840997+0000","last_peered":"2024-09-04T17:48:09.858550+0000","last_clean":"2024-09-04T17:48:09.858550+0000","last_became_active":"2024-09-04T17:47:57.840926+0000","last_became_peered":"2024-09-04T17:47:57.840926+0000","last_unstale":"2024-09-04T17:48:09.858550+0000","last_undegraded":"2024-09-04T17:48:09.858550+0000","last_fullsized":"2024-09-04T17:48:09.858550+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001226074,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"129'320","reported_seq":2792,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.840537+0000","last_change":"2024-09-04T17:48:09.838682+0000","last_active":"2024-09-04T17:48:10.840537+0000","last_peered":"2024-09-04T17:48:10.840537+0000","last_clean":"2024-09-04T17:48:10.840537+0000","last_became_active":"2024-09-04T17:47:32.848310+0000","last_became_peered":"2024-09-04T17:47:32.848310+0000","last_unstale":"2024-09-04T17:48:10.840537+0000","last_undegraded":"2024-09-04T17:48:10.840537+0000","last_fullsized":"2024-09-04T17:48:10.840537+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":320,"log_dups_size":0,"ondisk_log_size":320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:08:10.685688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001293892,"stat_sum":{"num_bytes":40074006,"num_objects":54,"num_object_clones":4,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":28,"num_read":2418,"num_read_kb":14504,"num_write":368,"num_write_kb":5524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":919873,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"2.6","version":"0'0","reported_seq":37,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.843292+0000","last_change":"2024-09-04T17:47:58.862844+0000","last_active":"2024-09-04T17:48:10.843292+0000","last_peered":"2024-09-04T17:48:10.843292+0000","last_clean":"2024-09-04T17:48:10.843292+0000","last_became_active":"2024-09-04T17:47:58.859941+0000","last_became_peered":"2024-09-04T17:47:58.859941+0000","last_unstale":"2024-09-04T17:48:10.843292+0000","last_undegraded":"2024-09-04T17:48:10.843292+0000","last_fullsized":"2024-09-04T17:48:10.843292+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:08:53.409036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011364179999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"134'414","reported_seq":2661,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-04T17:48:14.946390+0000","last_change":"2024-09-04T17:48:14.947003+0000","last_active":"2024-09-04T17:48:14.946390+0000","last_peered":"2024-09-04T17:48:14.946390+0000","last_clean":"2024-09-04T17:48:14.946390+0000","last_became_active":"2024-09-04T17:47:08.653159+0000","last_became_peered":"2024-09-04T17:47:08.653159+0000","last_unstale":"2024-09-04T17:48:14.946390+0000","last_undegraded":"2024-09-04T17:48:14.946390+0000","last_fullsized":"2024-09-04T17:48:14.946390+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":414,"log_dups_size":0,"ondisk_log_size":414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:50:34.140487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030432329999999998,"stat_sum":{"num_bytes":43272039,"num_objects":61,"num_object_clones":7,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":35,"num_read":2882,"num_read_kb":23337,"num_write":531,"num_write_kb":21560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":952382,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"2.5","version":"0'0","reported_seq":246,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-04T17:48:14.945008+0000","last_change":"2024-09-04T17:45:49.476607+0000","last_active":"2024-09-04T17:48:14.945008+0000","last_peered":"2024-09-04T17:48:14.945008+0000","last_clean":"2024-09-04T17:48:14.945008+0000","last_became_active":"2024-09-04T17:45:47.369308+0000","last_became_peered":"2024-09-04T17:45:47.369308+0000","last_unstale":"2024-09-04T17:48:14.945008+0000","last_undegraded":"2024-09-04T17:48:14.945008+0000","last_fullsized":"2024-09-04T17:48:14.945008+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:17:13.378553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001029826,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"134'359","reported_seq":2432,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-04T17:48:14.936948+0000","last_change":"2024-09-04T17:48:14.937297+0000","last_active":"2024-09-04T17:48:14.936948+0000","last_peered":"2024-09-04T17:48:14.936948+0000","last_clean":"2024-09-04T17:48:14.936948+0000","last_became_active":"2024-09-04T17:48:10.845756+0000","last_became_peered":"2024-09-04T17:48:10.845756+0000","last_unstale":"2024-09-04T17:48:14.936948+0000","last_undegraded":"2024-09-04T17:48:14.936948+0000","last_fullsized":"2024-09-04T17:48:14.936948+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:48:20.910158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079943099999999997,"stat_sum":{"num_bytes":33917693,"num_objects":59,"num_object_clones":3,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":33,"num_read":2424,"num_read_kb":18572,"num_write":397,"num_write_kb":9850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":237,"num_bytes_recovered":13375622,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"2.4","version":"0'0","reported_seq":82,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.840019+0000","last_change":"2024-09-04T17:47:29.803682+0000","last_active":"2024-09-04T17:48:10.840019+0000","last_peered":"2024-09-04T17:48:10.840019+0000","last_clean":"2024-09-04T17:48:10.840019+0000","last_became_active":"2024-09-04T17:47:29.803256+0000","last_became_peered":"2024-09-04T17:47:29.803256+0000","last_unstale":"2024-09-04T17:48:10.840019+0000","last_undegraded":"2024-09-04T17:48:10.840019+0000","last_fullsized":"2024-09-04T17:48:10.840019+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:32:26.523162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"125'273","reported_seq":1804,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.845948+0000","last_change":"2024-09-04T17:48:09.849956+0000","last_active":"2024-09-04T17:48:10.845948+0000","last_peered":"2024-09-04T17:48:10.845948+0000","last_clean":"2024-09-04T17:48:10.845948+0000","last_became_active":"2024-09-04T17:47:41.918947+0000","last_became_peered":"2024-09-04T17:47:41.918947+0000","last_unstale":"2024-09-04T17:48:10.845948+0000","last_undegraded":"2024-09-04T17:48:10.845948+0000","last_fullsized":"2024-09-04T17:48:10.845948+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":273,"log_dups_size":0,"ondisk_log_size":273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T23:41:18.557967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069886820000000004,"stat_sum":{"num_bytes":26359679,"num_objects":68,"num_object_clones":10,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":36,"num_read":2056,"num_read_kb":7215,"num_write":251,"num_write_kb":6999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":10024551,"num_keys_recovered":46,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.3","version":"131'660","reported_seq":2848,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.871783+0000","last_change":"2024-09-04T17:48:09.864268+0000","last_active":"2024-09-04T17:48:10.871783+0000","last_peered":"2024-09-04T17:48:10.871783+0000","last_clean":"2024-09-04T17:48:10.871783+0000","last_became_active":"2024-09-04T17:46:53.542889+0000","last_became_peered":"2024-09-04T17:46:53.542889+0000","last_unstale":"2024-09-04T17:48:10.871783+0000","last_undegraded":"2024-09-04T17:48:10.871783+0000","last_fullsized":"2024-09-04T17:48:10.871783+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":660,"log_dups_size":0,"ondisk_log_size":660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T22:07:45.745138+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.027057795999999999,"stat_sum":{"num_bytes":62897701,"num_objects":68,"num_object_clones":11,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":37,"num_read":4342,"num_read_kb":26256,"num_write":889,"num_write_kb":14093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"2.2","version":"22'2","reported_seq":255,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.845294+0000","last_change":"2024-09-04T17:47:59.598241+0000","last_active":"2024-09-04T17:48:10.845294+0000","last_peered":"2024-09-04T17:48:10.845294+0000","last_clean":"2024-09-04T17:48:10.845294+0000","last_became_active":"2024-09-04T17:47:58.884634+0000","last_became_peered":"2024-09-04T17:47:58.884634+0000","last_unstale":"2024-09-04T17:48:10.845294+0000","last_undegraded":"2024-09-04T17:48:10.845294+0000","last_fullsized":"2024-09-04T17:48:10.845294+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T22:02:31.692385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026316069999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"134'376","reported_seq":2890,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-04T17:48:14.936100+0000","last_change":"2024-09-04T17:48:14.936506+0000","last_active":"2024-09-04T17:48:14.936100+0000","last_peered":"2024-09-04T17:48:14.936100+0000","last_clean":"2024-09-04T17:48:14.936100+0000","last_became_active":"2024-09-04T17:48:01.849363+0000","last_became_peered":"2024-09-04T17:48:01.849363+0000","last_unstale":"2024-09-04T17:48:14.936100+0000","last_undegraded":"2024-09-04T17:48:14.936100+0000","last_fullsized":"2024-09-04T17:48:14.936100+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:28:48.800572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072198599999999998,"stat_sum":{"num_bytes":41098591,"num_objects":69,"num_object_clones":8,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":42,"num_read":3147,"num_read_kb":22161,"num_write":314,"num_write_kb":4881,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":5692590,"num_keys_recovered":35,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"2.1","version":"0'0","reported_seq":226,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-04T17:48:14.935690+0000","last_change":"2024-09-04T17:47:41.913896+0000","last_active":"2024-09-04T17:48:14.935690+0000","last_peered":"2024-09-04T17:48:14.935690+0000","last_clean":"2024-09-04T17:48:14.935690+0000","last_became_active":"2024-09-04T17:47:41.913057+0000","last_became_peered":"2024-09-04T17:47:41.913057+0000","last_unstale":"2024-09-04T17:48:14.935690+0000","last_undegraded":"2024-09-04T17:48:14.935690+0000","last_fullsized":"2024-09-04T17:48:14.935690+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:49:25.382532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001025195,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"126'527","reported_seq":2810,"reported_epoch":130,"state":"peering","last_fresh":"2024-09-04T17:48:10.253927+0000","last_change":"2024-09-04T17:48:10.839758+0000","last_active":"2024-09-04T17:48:10.839758+0000","last_peered":"2024-09-04T17:48:10.253927+0000","last_clean":"2024-09-04T17:48:06.888233+0000","last_became_active":"2024-09-04T17:48:09.878998+0000","last_became_peered":"2024-09-04T17:48:09.878998+0000","last_unstale":"2024-09-04T17:48:10.253927+0000","last_undegraded":"2024-09-04T17:48:10.253927+0000","last_fullsized":"2024-09-04T17:48:09.837030+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0052483269999999997,"stat_sum":{"num_bytes":87377265,"num_objects":81,"num_object_clones":18,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":41,"num_read":2920,"num_read_kb":40212,"num_write":602,"num_write_kb":23345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":240,"num_bytes_recovered":21297737,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,5],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2a","length":"4"}]},{"pgid":"2.0","version":"0'0","reported_seq":252,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-04T17:48:14.942504+0000","last_change":"2024-09-04T17:47:29.804429+0000","last_active":"2024-09-04T17:48:14.942504+0000","last_peered":"2024-09-04T17:48:14.942504+0000","last_clean":"2024-09-04T17:48:14.942504+0000","last_became_active":"2024-09-04T17:47:29.803992+0000","last_became_peered":"2024-09-04T17:47:29.803992+0000","last_unstale":"2024-09-04T17:48:14.942504+0000","last_undegraded":"2024-09-04T17:48:14.942504+0000","last_fullsized":"2024-09-04T17:48:14.942504+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:50:32.563345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010553590000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"126'525","reported_seq":2255,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.845194+0000","last_change":"2024-09-04T17:48:09.844640+0000","last_active":"2024-09-04T17:48:10.845194+0000","last_peered":"2024-09-04T17:48:10.845194+0000","last_clean":"2024-09-04T17:48:10.845194+0000","last_became_active":"2024-09-04T17:48:01.850400+0000","last_became_peered":"2024-09-04T17:48:01.850400+0000","last_unstale":"2024-09-04T17:48:10.845194+0000","last_undegraded":"2024-09-04T17:48:10.845194+0000","last_fullsized":"2024-09-04T17:48:10.845194+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T18:01:48.725657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049494070000000003,"stat_sum":{"num_bytes":50198348,"num_objects":64,"num_object_clones":9,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":33,"num_read":5233,"num_read_kb":29752,"num_write":673,"num_write_kb":13801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":13799490,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"2.3","version":"20'1","reported_seq":249,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.845299+0000","last_change":"2024-09-04T17:47:41.919544+0000","last_active":"2024-09-04T17:48:10.845299+0000","last_peered":"2024-09-04T17:48:10.845299+0000","last_clean":"2024-09-04T17:48:10.845299+0000","last_became_active":"2024-09-04T17:47:41.919027+0000","last_became_peered":"2024-09-04T17:47:41.919027+0000","last_unstale":"2024-09-04T17:48:10.845299+0000","last_undegraded":"2024-09-04T17:48:10.845299+0000","last_fullsized":"2024-09-04T17:48:10.845299+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:42:55.688562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086618299999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":337,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-04T17:48:14.942810+0000","last_change":"2024-09-04T17:45:42.341149+0000","last_active":"2024-09-04T17:48:14.942810+0000","last_peered":"2024-09-04T17:48:14.942810+0000","last_clean":"2024-09-04T17:48:14.942810+0000","last_became_active":"2024-09-04T17:45:42.339286+0000","last_became_peered":"2024-09-04T17:45:42.339286+0000","last_unstale":"2024-09-04T17:48:14.942810+0000","last_undegraded":"2024-09-04T17:48:14.942810+0000","last_fullsized":"2024-09-04T17:48:14.942810+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:41.305128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:41.305128+0000","last_clean_scrub_stamp":"2024-09-04T17:45:41.305128+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:52:24.860049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"134'485","reported_seq":2690,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-04T17:48:14.943390+0000","last_change":"2024-09-04T17:48:14.944023+0000","last_active":"2024-09-04T17:48:14.943390+0000","last_peered":"2024-09-04T17:48:14.943390+0000","last_clean":"2024-09-04T17:48:14.943390+0000","last_became_active":"2024-09-04T17:47:32.847682+0000","last_became_peered":"2024-09-04T17:47:32.847682+0000","last_unstale":"2024-09-04T17:48:14.943390+0000","last_undegraded":"2024-09-04T17:48:14.943390+0000","last_fullsized":"2024-09-04T17:48:14.943390+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T21:08:35.630294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016638060000000001,"stat_sum":{"num_bytes":43507193,"num_objects":70,"num_object_clones":10,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":39,"num_read":2705,"num_read_kb":17207,"num_write":531,"num_write_kb":29364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":14692620,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"3.9","version":"134'593","reported_seq":4140,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-04T17:48:14.936869+0000","last_change":"2024-09-04T17:48:14.937193+0000","last_active":"2024-09-04T17:48:14.936869+0000","last_peered":"2024-09-04T17:48:14.936869+0000","last_clean":"2024-09-04T17:48:14.936869+0000","last_became_active":"2024-09-04T17:47:44.949630+0000","last_became_peered":"2024-09-04T17:47:44.949630+0000","last_unstale":"2024-09-04T17:48:14.936869+0000","last_undegraded":"2024-09-04T17:48:14.936869+0000","last_fullsized":"2024-09-04T17:48:14.936869+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T21:19:18.496371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012454479999999999,"stat_sum":{"num_bytes":55724561,"num_objects":78,"num_object_clones":14,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":42,"num_read":5701,"num_read_kb":37078,"num_write":682,"num_write_kb":18628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":4534479,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"3.a","version":"131'533","reported_seq":3019,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.890923+0000","last_change":"2024-09-04T17:48:09.853469+0000","last_active":"2024-09-04T17:48:10.890923+0000","last_peered":"2024-09-04T17:48:10.890923+0000","last_clean":"2024-09-04T17:48:10.890923+0000","last_became_active":"2024-09-04T17:48:00.841070+0000","last_became_peered":"2024-09-04T17:48:00.841070+0000","last_unstale":"2024-09-04T17:48:10.890923+0000","last_undegraded":"2024-09-04T17:48:10.890923+0000","last_fullsized":"2024-09-04T17:48:10.890923+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:47:07.864899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038466329999999999,"stat_sum":{"num_bytes":36655948,"num_objects":72,"num_object_clones":11,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":40,"num_read":5202,"num_read_kb":16044,"num_write":529,"num_write_kb":2741,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":1572433,"num_keys_recovered":66,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.b","version":"125'677","reported_seq":2207,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.840785+0000","last_change":"2024-09-04T17:48:09.840108+0000","last_active":"2024-09-04T17:48:10.840785+0000","last_peered":"2024-09-04T17:48:10.840785+0000","last_clean":"2024-09-04T17:48:10.840785+0000","last_became_active":"2024-09-04T17:47:08.657286+0000","last_became_peered":"2024-09-04T17:47:08.657286+0000","last_unstale":"2024-09-04T17:48:10.840785+0000","last_undegraded":"2024-09-04T17:48:10.840785+0000","last_fullsized":"2024-09-04T17:48:10.840785+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":677,"log_dups_size":0,"ondisk_log_size":677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T19:10:08.386664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023202159999999999,"stat_sum":{"num_bytes":31620378,"num_objects":71,"num_object_clones":10,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":33,"num_read":3911,"num_read_kb":14517,"num_write":845,"num_write_kb":7911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":3759450,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.c","version":"129'552","reported_seq":7797,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.845669+0000","last_change":"2024-09-04T17:48:09.845373+0000","last_active":"2024-09-04T17:48:10.845669+0000","last_peered":"2024-09-04T17:48:10.845669+0000","last_clean":"2024-09-04T17:48:10.845669+0000","last_became_active":"2024-09-04T17:48:02.855127+0000","last_became_peered":"2024-09-04T17:48:02.855127+0000","last_unstale":"2024-09-04T17:48:10.845669+0000","last_undegraded":"2024-09-04T17:48:10.845669+0000","last_fullsized":"2024-09-04T17:48:10.845669+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":552,"log_dups_size":0,"ondisk_log_size":552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:49:26.298655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035566629999999998,"stat_sum":{"num_bytes":46296333,"num_objects":74,"num_object_clones":11,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":43,"num_read":8646,"num_read_kb":71089,"num_write":690,"num_write_kb":16847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":153,"num_bytes_recovered":7666524,"num_keys_recovered":34,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.d","version":"126'1282","reported_seq":4096,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-04T17:48:15.146132+0000","last_change":"2024-09-04T17:48:14.947144+0000","last_active":"2024-09-04T17:48:15.146132+0000","last_peered":"2024-09-04T17:48:15.146132+0000","last_clean":"2024-09-04T17:48:15.146132+0000","last_became_active":"2024-09-04T17:48:12.850381+0000","last_became_peered":"2024-09-04T17:48:12.850381+0000","last_unstale":"2024-09-04T17:48:15.146132+0000","last_undegraded":"2024-09-04T17:48:15.146132+0000","last_fullsized":"2024-09-04T17:48:15.146132+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":1282,"log_dups_size":0,"ondisk_log_size":1282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T19:19:15.721797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026779030000000001,"stat_sum":{"num_bytes":49889564,"num_objects":60,"num_object_clones":6,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":34,"num_read":8529,"num_read_kb":26250,"num_write":2192,"num_write_kb":13182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":179,"num_bytes_recovered":18877339,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"3.e","version":"135'446","reported_seq":5479,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-04T17:48:14.946295+0000","last_change":"2024-09-04T17:48:14.946295+0000","last_active":"2024-09-04T17:48:14.946295+0000","last_peered":"2024-09-04T17:48:14.946295+0000","last_clean":"2024-09-04T17:48:14.946295+0000","last_became_active":"2024-09-04T17:48:11.849605+0000","last_became_peered":"2024-09-04T17:48:11.849605+0000","last_unstale":"2024-09-04T17:48:14.946295+0000","last_undegraded":"2024-09-04T17:48:14.946295+0000","last_fullsized":"2024-09-04T17:48:14.946295+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":446,"log_dups_size":0,"ondisk_log_size":446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:15:11.313510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030014099999999999,"stat_sum":{"num_bytes":59920521,"num_objects":61,"num_object_clones":11,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":31,"num_read":6129,"num_read_kb":28040,"num_write":459,"num_write_kb":11211,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":7806862,"num_keys_recovered":46,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"3.f","version":"134'295","reported_seq":2867,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-04T17:48:14.945597+0000","last_change":"2024-09-04T17:48:14.946214+0000","last_active":"2024-09-04T17:48:14.945597+0000","last_peered":"2024-09-04T17:48:14.945597+0000","last_clean":"2024-09-04T17:48:14.945597+0000","last_became_active":"2024-09-04T17:45:55.956734+0000","last_became_peered":"2024-09-04T17:45:55.956734+0000","last_unstale":"2024-09-04T17:48:14.945597+0000","last_undegraded":"2024-09-04T17:48:14.945597+0000","last_fullsized":"2024-09-04T17:48:14.945597+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":295,"log_dups_size":0,"ondisk_log_size":295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T21:49:16.444214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002751561,"stat_sum":{"num_bytes":36107388,"num_objects":66,"num_object_clones":11,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":35,"num_read":2563,"num_read_kb":44941,"num_write":298,"num_write_kb":13423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"3.10","version":"126'514","reported_seq":3148,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-04T17:48:10.842877+0000","last_change":"2024-09-04T17:48:09.851750+0000","last_active":"2024-09-04T17:48:10.842877+0000","last_peered":"2024-09-04T17:48:10.842877+0000","last_clean":"2024-09-04T17:48:10.842877+0000","last_became_active":"2024-09-04T17:48:00.843136+0000","last_became_peered":"2024-09-04T17:48:00.843136+0000","last_unstale":"2024-09-04T17:48:10.842877+0000","last_undegraded":"2024-09-04T17:48:10.842877+0000","last_fullsized":"2024-09-04T17:48:10.842877+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":514,"log_dups_size":0,"ondisk_log_size":514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T23:07:02.271876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049468649999999999,"stat_sum":{"num_bytes":51575959,"num_objects":78,"num_object_clones":10,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":46,"num_read":4069,"num_read_kb":18348,"num_write":591,"num_write_kb":12832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":3377599,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.11","version":"135'503","reported_seq":5765,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-04T17:48:15.108116+0000","last_change":"2024-09-04T17:48:14.944990+0000","last_active":"2024-09-04T17:48:15.108116+0000","last_peered":"2024-09-04T17:48:15.108116+0000","last_clean":"2024-09-04T17:48:15.108116+0000","last_became_active":"2024-09-04T17:45:55.951870+0000","last_became_peered":"2024-09-04T17:45:55.951870+0000","last_unstale":"2024-09-04T17:48:15.108116+0000","last_undegraded":"2024-09-04T17:48:15.108116+0000","last_fullsized":"2024-09-04T17:48:15.108116+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":503,"log_dups_size":0,"ondisk_log_size":503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T21:51:34.296936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021887949999999999,"stat_sum":{"num_bytes":56384394,"num_objects":79,"num_object_clones":10,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":46,"num_read":6685,"num_read_kb":61028,"num_write":517,"num_write_kb":17560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"3.12","version":"130'349","reported_seq":3430,"reported_epoch":130,"state":"peering","last_fresh":"2024-09-04T17:48:09.930543+0000","last_change":"2024-09-04T17:48:10.839711+0000","last_active":"2024-09-04T17:48:10.839711+0000","last_peered":"2024-09-04T17:48:09.930543+0000","last_clean":"2024-09-04T17:48:09.930543+0000","last_became_active":"2024-09-04T17:47:44.952109+0000","last_became_peered":"2024-09-04T17:47:44.952109+0000","last_unstale":"2024-09-04T17:48:09.930543+0000","last_undegraded":"2024-09-04T17:48:09.930543+0000","last_fullsized":"2024-09-04T17:48:09.930543+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017447070000000001,"stat_sum":{"num_bytes":29554039,"num_objects":60,"num_object_clones":5,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":35,"num_read":3177,"num_read_kb":21378,"num_write":389,"num_write_kb":8559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":1762027,"num_keys_recovered":48,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,7],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"29","length":"5"}]},{"pgid":"3.13","version":"135'267","reported_seq":2251,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-04T17:48:15.108668+0000","last_change":"2024-09-04T17:48:14.943848+0000","last_active":"2024-09-04T17:48:15.108668+0000","last_peered":"2024-09-04T17:48:15.108668+0000","last_clean":"2024-09-04T17:48:15.108668+0000","last_became_active":"2024-09-04T17:45:55.954544+0000","last_became_peered":"2024-09-04T17:45:55.954544+0000","last_unstale":"2024-09-04T17:48:15.108668+0000","last_undegraded":"2024-09-04T17:48:15.108668+0000","last_fullsized":"2024-09-04T17:48:15.108668+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":267,"log_dups_size":0,"ondisk_log_size":267,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T22:59:30.435260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00158848,"stat_sum":{"num_bytes":22890202,"num_objects":59,"num_object_clones":4,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":38,"num_read":2181,"num_read_kb":22538,"num_write":217,"num_write_kb":4374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"27","length":"7"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1507432779,"num_objects":2084,"num_object_clones":273,"num_object_copies":4168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2084,"num_whiteouts":1135,"num_read":131903,"num_read_kb":867879,"num_write":18293,"num_write_kb":403770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3489,"num_bytes_recovered":212950055,"num_keys_recovered":652,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":505659392,"data_stored":614866556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21395,"internal_metadata":0},"log_size":15508,"ondisk_log_size":15508,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":117,"seq":502511173636,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":132212,"kb_used_data":98456,"kb_used_omap":22,"kb_used_meta":33705,"kb_avail":94239628,"statfs":{"total":96636764160,"available":96501379072,"internally_reserved":0,"allocated":100818944,"data_stored":103901506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22742,"internal_metadata":34514730},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509473,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":106128,"kb_used_data":79096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94265712,"statfs":{"total":96636764160,"available":96528089088,"internally_reserved":0,"allocated":80994304,"data_stored":99363602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509474,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":172236,"kb_used_data":114672,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94199604,"statfs":{"total":96636764160,"available":96460394496,"internally_reserved":0,"allocated":117424128,"data_stored":147072672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":0,"seq":206158430230,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":64424509473,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509473,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":124568,"kb_used_data":75328,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94247272,"statfs":{"total":96636764160,"available":96509206528,"internally_reserved":0,"allocated":77135872,"data_stored":95452503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542178,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":133072,"kb_used_data":79676,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94238768,"statfs":{"total":96636764160,"available":96500498432,"internally_reserved":0,"allocated":81588224,"data_stored":104088771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":103,"seq":442381631496,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":85608,"kb_used_data":45452,"kb_used_omap":12,"kb_used_meta":40115,"kb_avail":94286232,"statfs":{"total":96636764160,"available":96549101568,"internally_reserved":0,"allocated":46542848,"data_stored":56869238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12804,"internal_metadata":41078268},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":74981376,"data_stored":94116394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2035712,"data_stored":2023494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":44945408,"data_stored":56105331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6121,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":7245824,"data_stored":9302332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":115863552,"data_stored":146326931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":81965056,"data_stored":101128065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":99254272,"data_stored":103158122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15274,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":79368192,"data_stored":102705887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-04T17:48:16.455 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-04T17:48:16.505 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:16.503+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:16.584 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:16.584 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:16.595 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:48:16.595 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:48:16.598 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-04T17:48:16.612 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:48:16.612 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:48:16.613 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:16.607+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:16.628 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-04T17:48:16.628 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-04T17:48:16.645 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-04T17:48:16.645 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-04T17:48:16.665 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:48:16.666 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:48:16.686 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:48:16.686 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:48:16.700 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:48:16.695+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:48:16.706 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-04T17:48:16.706 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-04T17:48:16.723 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:16.723 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:48:16.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:48:16.723 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:48:16.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.724 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59950 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5365760~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:48:16.724 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654917+0000", 2024-09-04T17:48:16.724 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 63.066993869999997, 2024-09-04T17:48:16.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066822450000000005, 2024-09-04T17:48:16.724 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:16.724 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:16.724 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:16.724 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:16.724 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:16.724 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:16.724 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59950 2024-09-04T17:48:16.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.725 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:16.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:16.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654917+0000", 2024-09-04T17:48:16.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:16.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:16.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654917+0000", 2024-09-04T17:48:16.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:16.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:16.726 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654918+0000", 2024-09-04T17:48:16.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.29e-07 2024-09-04T17:48:16.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.726 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:16.726 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654919+0000", 2024-09-04T17:48:16.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2589999999999999e-06 2024-09-04T17:48:16.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.726 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:16.726 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654922+0000", 2024-09-04T17:48:16.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1350000000000001e-06 2024-09-04T17:48:16.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.727 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.727 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:16.727 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654925+0000", 2024-09-04T17:48:16.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1250000000000001e-06 2024-09-04T17:48:16.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.727 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.727 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:16.727 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721466+0000", 2024-09-04T17:48:16.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066540959999999996 2024-09-04T17:48:16.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:16.728 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721574+0000", 2024-09-04T17:48:16.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010747399999999999 2024-09-04T17:48:16.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:16.728 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721740+0000", 2024-09-04T17:48:16.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016626799999999999 2024-09-04T17:48:16.728 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:16.728 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:16.728 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:16.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.729 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59951 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5472256~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:48:16.729 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654929+0000", 2024-09-04T17:48:16.729 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 63.066982684999999, 2024-09-04T17:48:16.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067110534999999999, 2024-09-04T17:48:16.729 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:16.729 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:16.729 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:16.729 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:16.729 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:16.729 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:16.729 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59951 2024-09-04T17:48:16.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.730 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:16.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.730 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:16.730 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654929+0000", 2024-09-04T17:48:16.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:16.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.730 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:16.730 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654929+0000", 2024-09-04T17:48:16.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:16.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.731 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:16.731 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654929+0000", 2024-09-04T17:48:16.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5100000000000001e-07 2024-09-04T17:48:16.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:16.733 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654930+0000", 2024-09-04T17:48:16.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2440000000000001e-06 2024-09-04T17:48:16.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:16.733 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654933+0000", 2024-09-04T17:48:16.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1599999999999998e-06 2024-09-04T17:48:16.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.734 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:16.734 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654936+0000", 2024-09-04T17:48:16.734 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6560000000000001e-06 2024-09-04T17:48:16.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.734 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:16.734 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721789+0000", 2024-09-04T17:48:16.734 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066853250000000003 2024-09-04T17:48:16.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.734 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:16.735 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721885+0000", 2024-09-04T17:48:16.735 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5616000000000001e-05 2024-09-04T17:48:16.735 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.735 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.735 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:16.735 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722039+0000", 2024-09-04T17:48:16.735 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000154258 2024-09-04T17:48:16.735 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:16.736 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:16.736 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:16.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.736 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59952 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5505024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:48:16.736 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654939+0000", 2024-09-04T17:48:16.736 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 63.066971807999998, 2024-09-04T17:48:16.736 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067433742000000005, 2024-09-04T17:48:16.736 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:16.736 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:16.737 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:16.737 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:16.737 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:16.737 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:16.737 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59952 2024-09-04T17:48:16.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.737 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:16.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:16.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654939+0000", 2024-09-04T17:48:16.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:16.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:16.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654939+0000", 2024-09-04T17:48:16.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:16.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:16.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654940+0000", 2024-09-04T17:48:16.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3300000000000001e-07 2024-09-04T17:48:16.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:16.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654941+0000", 2024-09-04T17:48:16.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1570000000000001e-06 2024-09-04T17:48:16.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.739 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.739 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:16.739 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654944+0000", 2024-09-04T17:48:16.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1319999999999998e-06 2024-09-04T17:48:16.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.739 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.740 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:16.740 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654947+0000", 2024-09-04T17:48:16.740 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5969999999999999e-06 2024-09-04T17:48:16.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.740 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.740 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:16.740 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722089+0000", 2024-09-04T17:48:16.740 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067142347000000005 2024-09-04T17:48:16.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.740 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.740 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:16.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722201+0000", 2024-09-04T17:48:16.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011235600000000001 2024-09-04T17:48:16.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:16.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722373+0000", 2024-09-04T17:48:16.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017191999999999999 2024-09-04T17:48:16.741 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:16.741 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:16.741 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:16.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.741 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59953 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5611520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:48:16.742 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654950+0000", 2024-09-04T17:48:16.742 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 63.066961167999999, 2024-09-04T17:48:16.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067601852000000004, 2024-09-04T17:48:16.742 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:16.742 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:16.742 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:16.742 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:16.742 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:16.742 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:16.742 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59953 2024-09-04T17:48:16.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.742 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:16.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.743 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:16.743 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654950+0000", 2024-09-04T17:48:16.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:16.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.743 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:16.743 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654950+0000", 2024-09-04T17:48:16.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:16.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.743 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:16.743 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654950+0000", 2024-09-04T17:48:16.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2499999999999999e-07 2024-09-04T17:48:16.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.744 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:16.744 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654954+0000", 2024-09-04T17:48:16.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.287e-06 2024-09-04T17:48:16.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.744 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:16.744 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654962+0000", 2024-09-04T17:48:16.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4230000000000005e-06 2024-09-04T17:48:16.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.744 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:16.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654966+0000", 2024-09-04T17:48:16.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6449999999999998e-06 2024-09-04T17:48:16.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:16.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722417+0000", 2024-09-04T17:48:16.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067451188999999995 2024-09-04T17:48:16.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:16.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722467+0000", 2024-09-04T17:48:16.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0556999999999997e-05 2024-09-04T17:48:16.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:16.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722552+0000", 2024-09-04T17:48:16.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4525999999999997e-05 2024-09-04T17:48:16.746 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:16.746 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:16.746 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:16.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.747 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.747 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59954 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5718016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:48:16.747 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654970+0000", 2024-09-04T17:48:16.747 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 63.066941407000002, 2024-09-04T17:48:16.747 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067874285000000006, 2024-09-04T17:48:16.747 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:16.747 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:16.747 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:16.747 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:16.747 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:16.747 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:16.747 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59954 2024-09-04T17:48:16.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.748 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:16.748 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.748 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:16.748 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654970+0000", 2024-09-04T17:48:16.748 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:16.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.748 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.748 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:16.748 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654970+0000", 2024-09-04T17:48:16.748 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:16.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.748 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.749 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:16.749 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654970+0000", 2024-09-04T17:48:16.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5899999999999998e-07 2024-09-04T17:48:16.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.749 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:16.749 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654971+0000", 2024-09-04T17:48:16.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2929999999999999e-06 2024-09-04T17:48:16.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.749 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:16.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654975+0000", 2024-09-04T17:48:16.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4699999999999998e-06 2024-09-04T17:48:16.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:16.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654978+0000", 2024-09-04T17:48:16.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7930000000000002e-06 2024-09-04T17:48:16.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:16.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722600+0000", 2024-09-04T17:48:16.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067622229000000006 2024-09-04T17:48:16.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:16.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722689+0000", 2024-09-04T17:48:16.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9511000000000004e-05 2024-09-04T17:48:16.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:16.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722844+0000", 2024-09-04T17:48:16.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015473 2024-09-04T17:48:16.751 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:16.752 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:16.752 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:16.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.752 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59955 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5857280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:48:16.752 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654981+0000", 2024-09-04T17:48:16.752 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 63.066930057, 2024-09-04T17:48:16.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068052360000000006, 2024-09-04T17:48:16.752 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:16.752 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:16.752 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:16.752 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:16.752 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:16.753 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:16.753 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59955 2024-09-04T17:48:16.753 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.753 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:16.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.753 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:16.753 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654981+0000", 2024-09-04T17:48:16.753 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:16.753 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:16.754 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654981+0000", 2024-09-04T17:48:16.754 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:16.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:16.754 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654981+0000", 2024-09-04T17:48:16.754 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2399999999999999e-07 2024-09-04T17:48:16.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:16.754 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654985+0000", 2024-09-04T17:48:16.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0980000000000004e-06 2024-09-04T17:48:16.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:16.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654990+0000", 2024-09-04T17:48:16.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3429999999999998e-06 2024-09-04T17:48:16.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:16.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654993+0000", 2024-09-04T17:48:16.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8700000000000001e-06 2024-09-04T17:48:16.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.756 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:16.756 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722888+0000", 2024-09-04T17:48:16.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067895239999999996 2024-09-04T17:48:16.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.756 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:16.756 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722948+0000", 2024-09-04T17:48:16.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9750999999999998e-05 2024-09-04T17:48:16.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.757 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:16.757 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.723034+0000", 2024-09-04T17:48:16.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5834000000000005e-05 2024-09-04T17:48:16.757 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:16.757 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:16.757 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:16.757 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.757 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59956 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5963776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:48:16.757 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654998+0000", 2024-09-04T17:48:16.757 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 63.066913528000001, 2024-09-04T17:48:16.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068359518999999994, 2024-09-04T17:48:16.757 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:16.758 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:16.758 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:16.758 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:16.758 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:16.758 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:16.758 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59956 2024-09-04T17:48:16.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.758 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:16.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.758 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:16.758 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654998+0000", 2024-09-04T17:48:16.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:16.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:16.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654998+0000", 2024-09-04T17:48:16.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:16.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:16.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654998+0000", 2024-09-04T17:48:16.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4299999999999999e-07 2024-09-04T17:48:16.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:16.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654999+0000", 2024-09-04T17:48:16.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1799999999999999e-06 2024-09-04T17:48:16.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:16.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.655002+0000", 2024-09-04T17:48:16.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3639999999999999e-06 2024-09-04T17:48:16.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:16.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.655005+0000", 2024-09-04T17:48:16.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7970000000000001e-06 2024-09-04T17:48:16.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:16.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.723085+0000", 2024-09-04T17:48:16.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068080229000000006 2024-09-04T17:48:16.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:16.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.723197+0000", 2024-09-04T17:48:16.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000111344 2024-09-04T17:48:16.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.762 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:16.762 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.723357+0000", 2024-09-04T17:48:16.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016036199999999999 2024-09-04T17:48:16.762 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:16.762 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:16.762 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:16.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.763 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61082 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 2654208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:16.763 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.822664+0000", 2024-09-04T17:48:16.763 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 62.899247141000004, 2024-09-04T17:48:16.763 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067072242000000004, 2024-09-04T17:48:16.763 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:16.763 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:16.764 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:16.764 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:16.764 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:16.764 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:16.764 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61082 2024-09-04T17:48:16.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.764 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:16.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:16.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822664+0000", 2024-09-04T17:48:16.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:16.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:16.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822664+0000", 2024-09-04T17:48:16.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:16.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.768 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:16.768 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822664+0000", 2024-09-04T17:48:16.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6600000000000003e-07 2024-09-04T17:48:16.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:16.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822666+0000", 2024-09-04T17:48:16.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.536e-06 2024-09-04T17:48:16.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:16.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822669+0000", 2024-09-04T17:48:16.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3759999999999999e-06 2024-09-04T17:48:16.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:16.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822672+0000", 2024-09-04T17:48:16.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8109999999999999e-06 2024-09-04T17:48:16.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:16.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889263+0000", 2024-09-04T17:48:16.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066591259999999999 2024-09-04T17:48:16.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:16.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889369+0000", 2024-09-04T17:48:16.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000105362 2024-09-04T17:48:16.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:16.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889736+0000", 2024-09-04T17:48:16.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00036763099999999999 2024-09-04T17:48:16.771 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:16.771 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:16.771 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:16.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.772 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61083 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 2760704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:16.772 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.822678+0000", 2024-09-04T17:48:16.772 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 62.899233406999997, 2024-09-04T17:48:16.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067475996999999996, 2024-09-04T17:48:16.772 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:16.772 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:16.772 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:16.772 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:16.772 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:16.772 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:16.772 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61083 2024-09-04T17:48:16.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.773 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:16.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.773 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:16.773 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822678+0000", 2024-09-04T17:48:16.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:16.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.773 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:16.773 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822678+0000", 2024-09-04T17:48:16.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:16.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:16.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822725+0000", 2024-09-04T17:48:16.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7429000000000003e-05 2024-09-04T17:48:16.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:16.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822728+0000", 2024-09-04T17:48:16.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2140000000000001e-06 2024-09-04T17:48:16.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:16.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822737+0000", 2024-09-04T17:48:16.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4370000000000007e-06 2024-09-04T17:48:16.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:16.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822743+0000", 2024-09-04T17:48:16.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7520000000000002e-06 2024-09-04T17:48:16.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:16.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889785+0000", 2024-09-04T17:48:16.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067042265000000004 2024-09-04T17:48:16.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:16.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889840+0000", 2024-09-04T17:48:16.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5254000000000003e-05 2024-09-04T17:48:16.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:16.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.890154+0000", 2024-09-04T17:48:16.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000313646 2024-09-04T17:48:16.776 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:16.776 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:16.776 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:16.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.777 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61084 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 2867200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:16.777 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.822753+0000", 2024-09-04T17:48:16.777 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 62.899158471, 2024-09-04T17:48:16.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068497161000000001, 2024-09-04T17:48:16.777 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:16.777 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:16.777 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:16.777 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:16.777 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:16.777 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:16.777 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61084 2024-09-04T17:48:16.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.777 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:16.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:16.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822753+0000", 2024-09-04T17:48:16.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:16.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:16.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822753+0000", 2024-09-04T17:48:16.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:16.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:16.779 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822753+0000", 2024-09-04T17:48:16.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9100000000000003e-07 2024-09-04T17:48:16.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:16.779 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822756+0000", 2024-09-04T17:48:16.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.678e-06 2024-09-04T17:48:16.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:16.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822763+0000", 2024-09-04T17:48:16.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8220000000000001e-06 2024-09-04T17:48:16.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.780 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:16.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822768+0000", 2024-09-04T17:48:16.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1930000000000001e-06 2024-09-04T17:48:16.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.780 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:16.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.890246+0000", 2024-09-04T17:48:16.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067477996999999998 2024-09-04T17:48:16.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:16.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.890555+0000", 2024-09-04T17:48:16.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000309143 2024-09-04T17:48:16.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:16.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.891250+0000", 2024-09-04T17:48:16.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00069463700000000001 2024-09-04T17:48:16.781 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:16.781 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:16.781 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:16.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.782 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61285 3.8 3:117cfeb9:::rbd_data.11341dfde0c5.0000000000000006:1f [read 2760704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:16.782 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835200+0000", 2024-09-04T17:48:16.782 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 62.886711404000003, 2024-09-04T17:48:16.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066899803999999993, 2024-09-04T17:48:16.782 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:16.782 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:16.782 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:16.782 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:16.782 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:16.782 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:16.782 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61285 2024-09-04T17:48:16.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.782 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:16.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:16.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835200+0000", 2024-09-04T17:48:16.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:16.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:16.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835200+0000", 2024-09-04T17:48:16.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:16.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:16.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835200+0000", 2024-09-04T17:48:16.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0899999999999997e-07 2024-09-04T17:48:16.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:16.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835206+0000", 2024-09-04T17:48:16.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3280000000000002e-06 2024-09-04T17:48:16.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:16.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835211+0000", 2024-09-04T17:48:16.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9230000000000001e-06 2024-09-04T17:48:16.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:16.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835216+0000", 2024-09-04T17:48:16.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9019999999999998e-06 2024-09-04T17:48:16.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.785 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:16.785 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.901719+0000", 2024-09-04T17:48:16.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066502981000000003 2024-09-04T17:48:16.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.785 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:16.785 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.901813+0000", 2024-09-04T17:48:16.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3905000000000003e-05 2024-09-04T17:48:16.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:16.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902100+0000", 2024-09-04T17:48:16.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028645599999999998 2024-09-04T17:48:16.786 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:16.786 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:16.786 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:16.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.786 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61286 3.8 3:117cfeb9:::rbd_data.11341dfde0c5.0000000000000006:1f [read 2867200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:16.786 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835244+0000", 2024-09-04T17:48:16.786 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 62.886666785000003, 2024-09-04T17:48:16.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067143215000000006, 2024-09-04T17:48:16.787 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:16.787 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:16.787 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:16.787 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:16.787 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:16.787 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:16.787 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61286 2024-09-04T17:48:16.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.787 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:16.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:16.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835244+0000", 2024-09-04T17:48:16.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:16.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:16.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835244+0000", 2024-09-04T17:48:16.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:16.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:16.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835245+0000", 2024-09-04T17:48:16.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8400000000000004e-07 2024-09-04T17:48:16.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:16.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835248+0000", 2024-09-04T17:48:16.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6850000000000001e-06 2024-09-04T17:48:16.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:16.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835254+0000", 2024-09-04T17:48:16.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.494e-06 2024-09-04T17:48:16.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.790 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:16.790 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835260+0000", 2024-09-04T17:48:16.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0580000000000003e-06 2024-09-04T17:48:16.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.790 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:16.790 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902119+0000", 2024-09-04T17:48:16.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066858318 2024-09-04T17:48:16.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.790 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:16.790 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902163+0000", 2024-09-04T17:48:16.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4753000000000003e-05 2024-09-04T17:48:16.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:16.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902388+0000", 2024-09-04T17:48:16.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022422300000000001 2024-09-04T17:48:16.791 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:16.791 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:16.791 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:16.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.791 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61315 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 98304~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:16.791 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835269+0000", 2024-09-04T17:48:16.791 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 62.886641992999998, 2024-09-04T17:48:16.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067329396, 2024-09-04T17:48:16.792 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:16.792 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:16.792 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:16.792 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:16.792 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:16.792 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:16.792 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61315 2024-09-04T17:48:16.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.792 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:16.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:16.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835269+0000", 2024-09-04T17:48:16.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:16.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:16.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835269+0000", 2024-09-04T17:48:16.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:16.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:16.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835270+0000", 2024-09-04T17:48:16.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6599999999999996e-07 2024-09-04T17:48:16.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:16.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835272+0000", 2024-09-04T17:48:16.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.261e-06 2024-09-04T17:48:16.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:16.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835278+0000", 2024-09-04T17:48:16.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.443e-06 2024-09-04T17:48:16.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:16.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835284+0000", 2024-09-04T17:48:16.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3290000000000001e-06 2024-09-04T17:48:16.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:16.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902147+0000", 2024-09-04T17:48:16.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066862695999999999 2024-09-04T17:48:16.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:16.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902248+0000", 2024-09-04T17:48:16.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000101501 2024-09-04T17:48:16.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:16.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902599+0000", 2024-09-04T17:48:16.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00035060000000000001 2024-09-04T17:48:16.796 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:16.796 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:16.796 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:16.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.797 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61316 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 204800~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:16.797 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835291+0000", 2024-09-04T17:48:16.797 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 62.886619973999998, 2024-09-04T17:48:16.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067632052999999998, 2024-09-04T17:48:16.797 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:16.797 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:16.797 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:16.797 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:16.797 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:16.797 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:16.797 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61316 2024-09-04T17:48:16.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.798 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:16.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:16.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835291+0000", 2024-09-04T17:48:16.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:16.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:16.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835291+0000", 2024-09-04T17:48:16.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:16.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:16.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835292+0000", 2024-09-04T17:48:16.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1899999999999998e-07 2024-09-04T17:48:16.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:16.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835294+0000", 2024-09-04T17:48:16.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0880000000000002e-06 2024-09-04T17:48:16.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:16.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835300+0000", 2024-09-04T17:48:16.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.224e-06 2024-09-04T17:48:16.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:16.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835305+0000", 2024-09-04T17:48:16.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0730000000000004e-06 2024-09-04T17:48:16.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:16.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902632+0000", 2024-09-04T17:48:16.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067327322999999994 2024-09-04T17:48:16.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:16.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902688+0000", 2024-09-04T17:48:16.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5177999999999999e-05 2024-09-04T17:48:16.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:16.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902923+0000", 2024-09-04T17:48:16.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023574800000000001 2024-09-04T17:48:16.802 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:16.802 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:16.802 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:16.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.802 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61317 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 344064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:16.802 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835312+0000", 2024-09-04T17:48:16.802 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 62.886598970000001, 2024-09-04T17:48:16.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067918511000000001, 2024-09-04T17:48:16.802 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:16.802 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:16.803 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:16.803 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:16.803 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:16.803 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:16.803 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61317 2024-09-04T17:48:16.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.803 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:16.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:16.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835312+0000", 2024-09-04T17:48:16.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:16.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:16.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835312+0000", 2024-09-04T17:48:16.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:16.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:16.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835313+0000", 2024-09-04T17:48:16.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9000000000000002e-07 2024-09-04T17:48:16.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:16.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835315+0000", 2024-09-04T17:48:16.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1689999999999999e-06 2024-09-04T17:48:16.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:16.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835321+0000", 2024-09-04T17:48:16.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0619999999999997e-06 2024-09-04T17:48:16.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:16.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835326+0000", 2024-09-04T17:48:16.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0390000000000004e-06 2024-09-04T17:48:16.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:16.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902967+0000", 2024-09-04T17:48:16.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067641520999999996 2024-09-04T17:48:16.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:16.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903067+0000", 2024-09-04T17:48:16.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9426999999999995e-05 2024-09-04T17:48:16.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:16.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903231+0000", 2024-09-04T17:48:16.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016390299999999999 2024-09-04T17:48:16.807 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:16.807 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:16.807 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:16.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.807 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61318 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 450560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:16.807 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835334+0000", 2024-09-04T17:48:16.807 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 62.886577453000001, 2024-09-04T17:48:16.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068091771999999995, 2024-09-04T17:48:16.807 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:16.808 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:16.808 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:16.808 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:16.808 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:16.808 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:16.808 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61318 2024-09-04T17:48:16.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.808 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:16.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:16.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835334+0000", 2024-09-04T17:48:16.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:16.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:16.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835334+0000", 2024-09-04T17:48:16.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:16.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:16.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835334+0000", 2024-09-04T17:48:16.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5699999999999998e-07 2024-09-04T17:48:16.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:16.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835337+0000", 2024-09-04T17:48:16.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3889999999999999e-06 2024-09-04T17:48:16.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:16.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835343+0000", 2024-09-04T17:48:16.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1809999999999998e-06 2024-09-04T17:48:16.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:16.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835348+0000", 2024-09-04T17:48:16.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3170000000000001e-06 2024-09-04T17:48:16.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:16.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903270+0000", 2024-09-04T17:48:16.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067922191000000007 2024-09-04T17:48:16.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:16.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903321+0000", 2024-09-04T17:48:16.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0889e-05 2024-09-04T17:48:16.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:16.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903426+0000", 2024-09-04T17:48:16.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104348 2024-09-04T17:48:16.812 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:16.812 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:16.812 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:16.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.812 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61319 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 589824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:16.812 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835357+0000", 2024-09-04T17:48:16.812 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 62.886554636, 2024-09-04T17:48:16.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068372535999999998, 2024-09-04T17:48:16.813 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:16.813 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:16.813 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:16.813 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:16.813 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:16.813 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:16.813 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61319 2024-09-04T17:48:16.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.813 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:16.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:16.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835357+0000", 2024-09-04T17:48:16.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:16.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:16.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835357+0000", 2024-09-04T17:48:16.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:16.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:16.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835357+0000", 2024-09-04T17:48:16.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.32e-07 2024-09-04T17:48:16.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:16.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835359+0000", 2024-09-04T17:48:16.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1660000000000001e-06 2024-09-04T17:48:16.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:16.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835366+0000", 2024-09-04T17:48:16.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3550000000000001e-06 2024-09-04T17:48:16.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:16.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835371+0000", 2024-09-04T17:48:16.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1050000000000003e-06 2024-09-04T17:48:16.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:16.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903471+0000", 2024-09-04T17:48:16.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068100451000000006 2024-09-04T17:48:16.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:16.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903573+0000", 2024-09-04T17:48:16.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000101691 2024-09-04T17:48:16.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:16.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903729+0000", 2024-09-04T17:48:16.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000156336 2024-09-04T17:48:16.817 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:16.817 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:16.817 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:16.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.818 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61320 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:16.818 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835380+0000", 2024-09-04T17:48:16.818 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 62.886530870999998, 2024-09-04T17:48:16.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068543058000000004, 2024-09-04T17:48:16.818 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:16.818 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:16.818 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:16.818 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:16.818 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:16.818 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:16.818 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61320 2024-09-04T17:48:16.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.819 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:16.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:16.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835380+0000", 2024-09-04T17:48:16.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:16.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:16.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835380+0000", 2024-09-04T17:48:16.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:16.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:16.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835381+0000", 2024-09-04T17:48:16.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9999999999999998e-07 2024-09-04T17:48:16.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:16.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835383+0000", 2024-09-04T17:48:16.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2230000000000001e-06 2024-09-04T17:48:16.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:16.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835390+0000", 2024-09-04T17:48:16.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.34e-06 2024-09-04T17:48:16.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:16.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835396+0000", 2024-09-04T17:48:16.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9830000000000001e-06 2024-09-04T17:48:16.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:16.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903765+0000", 2024-09-04T17:48:16.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068368441000000002 2024-09-04T17:48:16.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:16.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903835+0000", 2024-09-04T17:48:16.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0631999999999997e-05 2024-09-04T17:48:16.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:16.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903923+0000", 2024-09-04T17:48:16.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8039000000000002e-05 2024-09-04T17:48:16.822 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:16.822 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:16.822 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:16.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.822 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61321 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 835584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:16.822 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835404+0000", 2024-09-04T17:48:16.823 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 62.886506812, 2024-09-04T17:48:16.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068911043000000005, 2024-09-04T17:48:16.823 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:16.823 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:16.823 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:16.823 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:16.823 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:16.823 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:16.823 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61321 2024-09-04T17:48:16.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.823 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:16.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:16.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835404+0000", 2024-09-04T17:48:16.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:16.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:16.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835404+0000", 2024-09-04T17:48:16.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:16.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:16.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835405+0000", 2024-09-04T17:48:16.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.89e-07 2024-09-04T17:48:16.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:16.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835407+0000", 2024-09-04T17:48:16.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4439999999999998e-06 2024-09-04T17:48:16.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:16.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835414+0000", 2024-09-04T17:48:16.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4930000000000003e-06 2024-09-04T17:48:16.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:16.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835419+0000", 2024-09-04T17:48:16.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8919999999999999e-06 2024-09-04T17:48:16.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:16.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903957+0000", 2024-09-04T17:48:16.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068538500000000002 2024-09-04T17:48:16.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:16.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904019+0000", 2024-09-04T17:48:16.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1419000000000003e-05 2024-09-04T17:48:16.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:16.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904315+0000", 2024-09-04T17:48:16.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00029680599999999999 2024-09-04T17:48:16.827 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:16.827 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:16.827 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:16.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.827 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61322 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:16.827 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835427+0000", 2024-09-04T17:48:16.827 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 62.886484426000003, 2024-09-04T17:48:16.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.069550138999999997, 2024-09-04T17:48:16.828 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:16.828 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:16.828 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:16.828 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:16.828 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:16.828 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:16.828 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61322 2024-09-04T17:48:16.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.828 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:16.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:16.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835427+0000", 2024-09-04T17:48:16.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:16.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:16.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835427+0000", 2024-09-04T17:48:16.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:16.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:16.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.842157+0000", 2024-09-04T17:48:16.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067298419999999998 2024-09-04T17:48:16.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:16.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.842163+0000", 2024-09-04T17:48:16.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2029999999999997e-06 2024-09-04T17:48:16.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:16.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.842175+0000", 2024-09-04T17:48:16.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2072000000000001e-05 2024-09-04T17:48:16.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:16.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.842183+0000", 2024-09-04T17:48:16.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3850000000000002e-06 2024-09-04T17:48:16.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:16.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904371+0000", 2024-09-04T17:48:16.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.062187754999999997 2024-09-04T17:48:16.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:16.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904467+0000", 2024-09-04T17:48:16.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6117000000000001e-05 2024-09-04T17:48:16.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:16.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:16.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:16.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904977+0000", 2024-09-04T17:48:16.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00050976500000000002 2024-09-04T17:48:16.832 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:16.832 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:16.832 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:16.832 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:16.832 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:16.832 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:16.833 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:48:16.833 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:48:16.834 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-04T17:48:16.835 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:48:16.831+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:48:16.934 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:48:16.930+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:48:16.966 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is unset 2024-09-04T17:48:16.970 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:16.970 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:48:16.970 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:16.970 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(osd.2.0:365 3.9 e137/106 3:914df568:::rbd_data.11341aa245a3.00000000000000c0:25 v 137'607, mlcod=135'604)", 2024-09-04T17:48:16.970 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:16.968716+0000", 2024-09-04T17:48:16.970 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.0022125830000000002, 2024-09-04T17:48:16.970 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0022514190000000002, 2024-09-04T17:48:16.971 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:16.971 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:16.971 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:16.971 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:16.971 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:16.971 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:16.971 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:16.968716+0000", 2024-09-04T17:48:16.971 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:16.971 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:16.971 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:16.971 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:16.971 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:16.968716+0000", 2024-09-04T17:48:16.971 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:16.972 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:16.972 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:16.972 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:16.972 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:16.968716+0000", 2024-09-04T17:48:16.972 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.3900000000000001e-07 2024-09-04T17:48:16.972 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:16.972 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:16.972 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:16.972 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:16.968722+0000", 2024-09-04T17:48:16.972 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.5459999999999999e-06 2024-09-04T17:48:16.972 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:16.972 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:16.972 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:16.973 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:16.968736+0000", 2024-09-04T17:48:16.973 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3570000000000001e-05 2024-09-04T17:48:16.973 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:16.973 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:16.973 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:16.973 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:16.968745+0000", 2024-09-04T17:48:16.973 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.7589999999999992e-06 2024-09-04T17:48:16.973 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:16.973 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:16.973 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:16.974 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:16.968865+0000", 2024-09-04T17:48:16.974 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00011959999999999999 2024-09-04T17:48:16.974 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:16.974 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:16.974 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:16.974 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:16.968903+0000", 2024-09-04T17:48:16.974 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7901999999999997e-05 2024-09-04T17:48:16.974 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:16.974 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:16.974 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:16.974 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:16.974 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:16.974 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(osd.2.0:366 3.9 e137/106 3:97b38e21:::rbd_data.11341aa245a3.00000000000000db:25 v 137'609, mlcod=135'604)", 2024-09-04T17:48:16.975 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:16.969270+0000", 2024-09-04T17:48:16.975 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.0016590730000000001, 2024-09-04T17:48:16.975 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.001791731, 2024-09-04T17:48:16.975 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:16.975 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:16.975 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:16.975 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:16.975 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:16.975 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:16.975 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:16.969270+0000", 2024-09-04T17:48:16.975 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:16.975 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:16.976 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:16.976 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:16.976 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:16.969270+0000", 2024-09-04T17:48:16.976 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:16.976 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:16.976 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:16.976 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:16.976 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:16.969271+0000", 2024-09-04T17:48:16.976 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.9800000000000003e-07 2024-09-04T17:48:16.976 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:16.976 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:16.976 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:16.976 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:16.969279+0000", 2024-09-04T17:48:16.977 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.276e-06 2024-09-04T17:48:16.977 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:16.977 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:16.977 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:16.977 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:16.969291+0000", 2024-09-04T17:48:16.977 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2515e-05 2024-09-04T17:48:16.977 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:16.977 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:16.977 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:16.977 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:16.969298+0000", 2024-09-04T17:48:16.977 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.8519999999999998e-06 2024-09-04T17:48:16.977 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:16.977 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:16.978 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:16.978 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:16.970000+0000", 2024-09-04T17:48:16.978 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00070134199999999998 2024-09-04T17:48:16.978 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:16.978 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:16.978 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:16.978 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:16.970093+0000", 2024-09-04T17:48:16.978 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.3844999999999993e-05 2024-09-04T17:48:16.978 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:16.978 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:16.978 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:16.978 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:16.979 INFO:teuthology.orchestra.run.smithi190.stdout: ], 2024-09-04T17:48:16.979 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 2 2024-09-04T17:48:16.979 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:16.989 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-04T17:48:17.035 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:48:17.030+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:48:17.135 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:17.130+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:17.166 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:17.166 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:48:17.166 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:48:17.166 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:48:17.166 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:17.181 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-04T17:48:17.237 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:17.231+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:17.338 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:48:17.335+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:48:17.351 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:17.351 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:48:17.351 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:48:17.351 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:48:17.351 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.352 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:12722 3.1b 3:d8152af1:::rbd_data.1134ef3a5a7d.00000000000000bd:head [delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-04T17:48:17.352 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:16.421965+0000", 2024-09-04T17:48:17.352 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 120.92770190100001, 2024-09-04T17:48:17.352 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.890002811, 2024-09-04T17:48:17.352 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:17.352 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:17.352 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:17.352 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:17.352 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:17.352 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:17.352 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 12722 2024-09-04T17:48:17.352 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.352 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:17.353 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.353 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:17.353 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421965+0000", 2024-09-04T17:48:17.353 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:17.353 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.353 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.353 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:17.353 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421965+0000", 2024-09-04T17:48:17.353 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:17.353 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.353 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.353 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:17.354 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421966+0000", 2024-09-04T17:48:17.354 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.102e-06 2024-09-04T17:48:17.354 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.354 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.354 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:17.354 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421971+0000", 2024-09-04T17:48:17.354 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9989999999999999e-06 2024-09-04T17:48:17.354 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.354 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.354 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:17.354 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421981+0000", 2024-09-04T17:48:17.354 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.7380000000000006e-06 2024-09-04T17:48:17.354 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.355 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.355 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:17.355 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421989+0000", 2024-09-04T17:48:17.355 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.1890000000000007e-06 2024-09-04T17:48:17.355 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.355 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.355 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:17.355 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.422050+0000", 2024-09-04T17:48:17.355 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.0702000000000003e-05 2024-09-04T17:48:17.355 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.355 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.355 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for peered", 2024-09-04T17:48:17.355 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.422059+0000", 2024-09-04T17:48:17.355 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5440000000000004e-06 2024-09-04T17:48:17.356 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.356 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.356 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:17.356 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.308156+0000", 2024-09-04T17:48:17.356 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.88609676199999998 2024-09-04T17:48:17.356 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.356 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.356 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:17.356 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.308398+0000", 2024-09-04T17:48:17.356 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024215400000000001 2024-09-04T17:48:17.356 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.356 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.356 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 7", 2024-09-04T17:48:17.357 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.308795+0000", 2024-09-04T17:48:17.357 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000396704 2024-09-04T17:48:17.357 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.357 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.357 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:48:17.357 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.310831+0000", 2024-09-04T17:48:17.357 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0020364200000000002 2024-09-04T17:48:17.357 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.357 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.357 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:17.357 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.311818+0000", 2024-09-04T17:48:17.357 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00098649799999999993 2024-09-04T17:48:17.357 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.358 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.358 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:17.358 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.311905+0000", 2024-09-04T17:48:17.358 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7070999999999994e-05 2024-09-04T17:48:17.358 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.358 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.358 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:17.358 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.311968+0000", 2024-09-04T17:48:17.358 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.2928000000000005e-05 2024-09-04T17:48:17.358 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:17.358 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:17.358 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:17.358 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.359 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.359 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37359 3.f e58/24 3:f29e50a5:::rbd_data.1134c5a43465.0000000000000021:head v 58'135, mlcod=58'133)", 2024-09-04T17:48:17.359 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:48:17.359 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 97.732371297, 2024-09-04T17:48:17.359 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053483197000000003, 2024-09-04T17:48:17.359 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:17.359 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:17.359 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:17.359 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:17.359 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.359 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:17.359 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:48:17.359 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:17.360 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.360 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.360 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:17.360 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:48:17.360 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:17.360 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.360 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.360 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:17.360 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:48:17.360 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1699999999999997e-07 2024-09-04T17:48:17.360 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.360 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.360 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:17.360 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617338+0000", 2024-09-04T17:48:17.361 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.1168999999999998e-05 2024-09-04T17:48:17.361 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.361 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.361 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:17.361 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617351+0000", 2024-09-04T17:48:17.361 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3094e-05 2024-09-04T17:48:17.361 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.361 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.361 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:17.361 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617359+0000", 2024-09-04T17:48:17.361 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7970000000000006e-06 2024-09-04T17:48:17.361 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.361 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.362 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:17.362 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617417+0000", 2024-09-04T17:48:17.362 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7079e-05 2024-09-04T17:48:17.362 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.362 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.362 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:17.362 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617438+0000", 2024-09-04T17:48:17.362 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1654999999999998e-05 2024-09-04T17:48:17.362 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.362 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.362 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:17.362 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670738+0000", 2024-09-04T17:48:17.362 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053300142000000002 2024-09-04T17:48:17.363 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.363 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.363 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:17.363 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670779+0000", 2024-09-04T17:48:17.363 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0343999999999999e-05 2024-09-04T17:48:17.363 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:17.363 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:17.363 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:17.363 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.363 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.363 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37358 3.1d e58/24 3:bd3215f3:::rbd_data.1134c5a43465.0000000000000020:head v 58'91, mlcod=58'89)", 2024-09-04T17:48:17.363 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617502+0000", 2024-09-04T17:48:17.363 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 97.732164578999999, 2024-09-04T17:48:17.363 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053505641999999999, 2024-09-04T17:48:17.364 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:17.364 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:17.364 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:17.364 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:17.364 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.364 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:17.364 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617502+0000", 2024-09-04T17:48:17.364 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:17.364 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.364 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.364 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:17.364 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617502+0000", 2024-09-04T17:48:17.365 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:17.365 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.365 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.365 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:17.365 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617503+0000", 2024-09-04T17:48:17.365 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.184e-06 2024-09-04T17:48:17.365 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.365 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.365 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:17.365 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617526+0000", 2024-09-04T17:48:17.365 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2099999999999998e-05 2024-09-04T17:48:17.365 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.365 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.365 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:17.366 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617540+0000", 2024-09-04T17:48:17.366 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4741e-05 2024-09-04T17:48:17.366 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.366 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.366 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:17.366 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617550+0000", 2024-09-04T17:48:17.366 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4259999999999992e-06 2024-09-04T17:48:17.368 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.368 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.368 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:17.368 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617613+0000", 2024-09-04T17:48:17.368 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.3625000000000002e-05 2024-09-04T17:48:17.368 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.368 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.369 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:17.369 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617637+0000", 2024-09-04T17:48:17.369 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.3604999999999998e-05 2024-09-04T17:48:17.369 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.369 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.369 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:17.369 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670967+0000", 2024-09-04T17:48:17.369 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053329881000000003 2024-09-04T17:48:17.369 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.369 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.369 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:17.369 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671008+0000", 2024-09-04T17:48:17.369 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.108e-05 2024-09-04T17:48:17.369 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:17.370 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:17.370 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:17.370 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.370 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.370 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37361 3.1c e58/24 3:3c7f8976:::rbd_data.1134c5a43465.0000000000000023:head v 58'159, mlcod=58'157)", 2024-09-04T17:48:17.370 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617578+0000", 2024-09-04T17:48:17.370 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 97.732088728999997, 2024-09-04T17:48:17.370 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053990367999999997, 2024-09-04T17:48:17.370 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:17.370 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:17.370 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:17.370 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:17.370 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.371 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:17.371 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617578+0000", 2024-09-04T17:48:17.371 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:17.371 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.371 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.371 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:17.371 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617578+0000", 2024-09-04T17:48:17.371 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:17.371 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.371 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.371 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:17.371 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617579+0000", 2024-09-04T17:48:17.371 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7299999999999996e-07 2024-09-04T17:48:17.371 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.372 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.372 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:17.372 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617610+0000", 2024-09-04T17:48:17.372 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0951000000000002e-05 2024-09-04T17:48:17.372 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.372 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.372 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:17.372 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:48:17.372 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0002e-05 2024-09-04T17:48:17.372 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.372 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.372 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:17.372 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617626+0000", 2024-09-04T17:48:17.373 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6660000000000002e-06 2024-09-04T17:48:17.373 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.373 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.373 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:17.373 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670548+0000", 2024-09-04T17:48:17.373 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.052921943999999999 2024-09-04T17:48:17.373 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.373 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.373 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:17.373 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670574+0000", 2024-09-04T17:48:17.373 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5405000000000002e-05 2024-09-04T17:48:17.373 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.374 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.374 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:17.374 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671522+0000", 2024-09-04T17:48:17.374 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00094815299999999997 2024-09-04T17:48:17.374 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.374 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.374 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:17.374 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671568+0000", 2024-09-04T17:48:17.374 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6674000000000001e-05 2024-09-04T17:48:17.374 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:17.374 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:17.375 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:17.375 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.375 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.375 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37357 3.15 e58/49 3:a897312a:::rbd_data.1134c5a43465.000000000000001f:head v 58'117, mlcod=58'115)", 2024-09-04T17:48:17.375 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:48:17.375 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 97.732046390999997, 2024-09-04T17:48:17.375 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053734297, 2024-09-04T17:48:17.375 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:17.375 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:17.375 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:17.376 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:17.376 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.376 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:17.376 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:48:17.376 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:17.376 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.376 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.376 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:17.376 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:48:17.376 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:17.376 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.376 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.376 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:17.376 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617621+0000", 2024-09-04T17:48:17.377 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.7300000000000004e-07 2024-09-04T17:48:17.377 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.377 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.377 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:17.377 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617642+0000", 2024-09-04T17:48:17.377 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0185000000000001e-05 2024-09-04T17:48:17.377 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.377 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.377 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:17.377 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617654+0000", 2024-09-04T17:48:17.377 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2745000000000001e-05 2024-09-04T17:48:17.377 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.378 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.378 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:17.378 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617662+0000", 2024-09-04T17:48:17.378 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.0499999999999992e-06 2024-09-04T17:48:17.378 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.378 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.378 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:17.378 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617727+0000", 2024-09-04T17:48:17.378 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.4923000000000002e-05 2024-09-04T17:48:17.378 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.378 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.378 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:17.379 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617753+0000", 2024-09-04T17:48:17.379 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5899999999999999e-05 2024-09-04T17:48:17.379 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.379 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.379 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:17.379 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671298+0000", 2024-09-04T17:48:17.379 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053545276000000003 2024-09-04T17:48:17.379 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.379 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.379 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:17.379 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671355+0000", 2024-09-04T17:48:17.380 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6245000000000001e-05 2024-09-04T17:48:17.380 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:17.380 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:17.380 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:17.380 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.380 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.380 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44687 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:17.380 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253012+0000", 2024-09-04T17:48:17.380 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 87.096654323999999, 2024-09-04T17:48:17.380 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051128777, 2024-09-04T17:48:17.380 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:17.380 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:17.381 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:17.381 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:17.381 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:17.381 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:17.381 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44687 2024-09-04T17:48:17.381 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.381 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:17.381 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.381 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:17.381 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253012+0000", 2024-09-04T17:48:17.381 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:17.381 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.382 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.382 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:17.382 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253012+0000", 2024-09-04T17:48:17.382 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:17.382 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.382 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.382 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:17.382 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253014+0000", 2024-09-04T17:48:17.382 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.24e-06 2024-09-04T17:48:17.382 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.382 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.382 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:17.382 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253019+0000", 2024-09-04T17:48:17.382 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4569999999999996e-06 2024-09-04T17:48:17.383 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.383 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.383 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:17.383 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253035+0000", 2024-09-04T17:48:17.383 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6291999999999999e-05 2024-09-04T17:48:17.383 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.383 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.383 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:17.383 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253050+0000", 2024-09-04T17:48:17.383 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4423000000000001e-05 2024-09-04T17:48:17.383 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.383 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.383 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:17.384 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304115+0000", 2024-09-04T17:48:17.384 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051064778999999998 2024-09-04T17:48:17.384 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.384 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.384 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:17.384 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304141+0000", 2024-09-04T17:48:17.384 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6585999999999999e-05 2024-09-04T17:48:17.384 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:17.384 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:17.384 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:17.384 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.384 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.384 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44688 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:17.385 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253067+0000", 2024-09-04T17:48:17.385 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 87.096599772000005, 2024-09-04T17:48:17.385 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051116979, 2024-09-04T17:48:17.385 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:17.385 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:17.385 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:17.385 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:17.385 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:17.385 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:17.385 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44688 2024-09-04T17:48:17.385 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.385 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:17.385 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.386 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:17.386 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253067+0000", 2024-09-04T17:48:17.386 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:17.386 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.386 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.386 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:17.386 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253067+0000", 2024-09-04T17:48:17.386 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:17.386 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.386 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.386 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:17.386 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253068+0000", 2024-09-04T17:48:17.386 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0729999999999999e-06 2024-09-04T17:48:17.387 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.387 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.387 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:17.387 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253073+0000", 2024-09-04T17:48:17.387 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2259999999999996e-06 2024-09-04T17:48:17.387 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.387 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.387 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:17.387 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253091+0000", 2024-09-04T17:48:17.387 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7255999999999999e-05 2024-09-04T17:48:17.387 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.387 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.387 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:17.387 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253103+0000", 2024-09-04T17:48:17.388 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2632999999999999e-05 2024-09-04T17:48:17.388 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.388 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.388 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:17.388 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304173+0000", 2024-09-04T17:48:17.388 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051069537999999998 2024-09-04T17:48:17.388 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.388 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.388 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:17.389 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304184+0000", 2024-09-04T17:48:17.389 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1253000000000001e-05 2024-09-04T17:48:17.389 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:17.389 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:17.389 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:17.389 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.389 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.389 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44689 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:17.389 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253121+0000", 2024-09-04T17:48:17.389 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 87.096545634999998, 2024-09-04T17:48:17.389 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051192595, 2024-09-04T17:48:17.389 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:17.389 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:17.390 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:17.390 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:17.390 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:17.390 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:17.390 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44689 2024-09-04T17:48:17.390 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.390 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:17.390 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.390 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:17.390 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253121+0000", 2024-09-04T17:48:17.390 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:17.390 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.390 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.391 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:17.391 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253121+0000", 2024-09-04T17:48:17.391 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:17.391 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.391 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.391 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:17.391 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253122+0000", 2024-09-04T17:48:17.391 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.15e-06 2024-09-04T17:48:17.391 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.391 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.391 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:17.391 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253128+0000", 2024-09-04T17:48:17.391 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2569999999999998e-06 2024-09-04T17:48:17.391 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.392 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.392 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:17.392 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253145+0000", 2024-09-04T17:48:17.392 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7176e-05 2024-09-04T17:48:17.392 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.392 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.392 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:17.392 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253158+0000", 2024-09-04T17:48:17.392 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3139000000000001e-05 2024-09-04T17:48:17.392 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.392 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.392 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:17.392 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304287+0000", 2024-09-04T17:48:17.393 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051129149999999998 2024-09-04T17:48:17.393 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.393 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.393 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:17.393 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304314+0000", 2024-09-04T17:48:17.393 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6723000000000001e-05 2024-09-04T17:48:17.393 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:17.393 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:17.393 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:17.393 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.393 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.393 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44690 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:17.393 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253177+0000", 2024-09-04T17:48:17.394 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 87.096489509999998, 2024-09-04T17:48:17.394 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051169263, 2024-09-04T17:48:17.394 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:17.394 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:17.394 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:17.394 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:17.394 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:17.394 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:17.394 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44690 2024-09-04T17:48:17.394 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.394 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:17.394 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.394 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:17.395 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253177+0000", 2024-09-04T17:48:17.395 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:17.395 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.395 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.395 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:17.395 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253177+0000", 2024-09-04T17:48:17.395 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:17.395 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.395 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.395 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:17.395 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253178+0000", 2024-09-04T17:48:17.395 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1000000000000001e-06 2024-09-04T17:48:17.395 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.396 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.396 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:17.396 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253184+0000", 2024-09-04T17:48:17.396 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2089999999999996e-06 2024-09-04T17:48:17.396 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.396 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.396 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:17.396 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253200+0000", 2024-09-04T17:48:17.396 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6087e-05 2024-09-04T17:48:17.396 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.396 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.396 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:17.396 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253231+0000", 2024-09-04T17:48:17.397 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0874999999999998e-05 2024-09-04T17:48:17.397 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.397 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.397 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:17.397 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304337+0000", 2024-09-04T17:48:17.397 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051106430000000001 2024-09-04T17:48:17.397 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.397 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.397 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:17.397 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304347+0000", 2024-09-04T17:48:17.397 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5619999999999993e-06 2024-09-04T17:48:17.397 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:17.397 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:17.398 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:17.398 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.398 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.398 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44691 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:17.398 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253249+0000", 2024-09-04T17:48:17.398 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 87.096417475999999, 2024-09-04T17:48:17.398 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051195630999999998, 2024-09-04T17:48:17.398 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:17.398 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:17.398 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:17.398 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:17.398 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:17.398 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:17.399 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44691 2024-09-04T17:48:17.399 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.399 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:17.399 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.399 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:17.399 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253249+0000", 2024-09-04T17:48:17.399 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:17.399 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.399 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.399 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:17.399 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253249+0000", 2024-09-04T17:48:17.399 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:17.399 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.400 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.400 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:17.400 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253251+0000", 2024-09-04T17:48:17.400 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2330000000000001e-06 2024-09-04T17:48:17.400 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.400 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.400 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:17.400 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253256+0000", 2024-09-04T17:48:17.400 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6099999999999997e-06 2024-09-04T17:48:17.400 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.400 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.400 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:17.400 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253274+0000", 2024-09-04T17:48:17.400 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7952999999999999e-05 2024-09-04T17:48:17.401 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.401 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.401 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:17.401 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253287+0000", 2024-09-04T17:48:17.401 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2839e-05 2024-09-04T17:48:17.401 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.401 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.401 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:17.401 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304396+0000", 2024-09-04T17:48:17.401 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051108990999999999 2024-09-04T17:48:17.401 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.401 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.401 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:17.402 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304445+0000", 2024-09-04T17:48:17.402 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9005000000000003e-05 2024-09-04T17:48:17.402 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:17.402 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:17.402 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:17.402 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.402 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.402 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44692 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:17.402 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253337+0000", 2024-09-04T17:48:17.402 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 87.096329902999997, 2024-09-04T17:48:17.402 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051138909000000003, 2024-09-04T17:48:17.402 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:17.403 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:17.403 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:17.403 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:17.403 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:17.403 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:17.403 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44692 2024-09-04T17:48:17.403 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.403 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:17.403 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.403 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:17.403 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253337+0000", 2024-09-04T17:48:17.403 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:17.403 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.403 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.404 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:17.404 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253337+0000", 2024-09-04T17:48:17.404 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:17.404 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.404 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.404 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:17.404 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253338+0000", 2024-09-04T17:48:17.404 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.463e-06 2024-09-04T17:48:17.404 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.404 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.404 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:17.404 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253357+0000", 2024-09-04T17:48:17.405 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9043e-05 2024-09-04T17:48:17.405 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.405 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.405 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:17.405 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253379+0000", 2024-09-04T17:48:17.405 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1243e-05 2024-09-04T17:48:17.405 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.405 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.405 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:17.405 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253396+0000", 2024-09-04T17:48:17.405 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7059e-05 2024-09-04T17:48:17.405 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.405 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.406 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:17.406 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304466+0000", 2024-09-04T17:48:17.406 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051070624000000002 2024-09-04T17:48:17.406 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.406 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.406 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:17.406 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304476+0000", 2024-09-04T17:48:17.406 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4769999999999992e-06 2024-09-04T17:48:17.406 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:17.406 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:17.406 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:17.406 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.406 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.406 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44693 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:17.407 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253415+0000", 2024-09-04T17:48:17.407 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 87.096251499999994, 2024-09-04T17:48:17.407 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051141949999999999, 2024-09-04T17:48:17.407 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:17.407 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:17.407 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:17.407 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:17.407 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:17.407 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:17.407 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44693 2024-09-04T17:48:17.407 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.407 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:17.407 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.408 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:17.408 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253415+0000", 2024-09-04T17:48:17.408 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:17.408 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.408 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.408 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:17.408 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253415+0000", 2024-09-04T17:48:17.408 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:17.408 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.408 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.408 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:17.408 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253416+0000", 2024-09-04T17:48:17.409 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1799999999999999e-06 2024-09-04T17:48:17.409 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.409 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.409 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:17.409 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253422+0000", 2024-09-04T17:48:17.409 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.5940000000000001e-06 2024-09-04T17:48:17.409 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.409 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.409 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:17.409 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253438+0000", 2024-09-04T17:48:17.409 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6008999999999999e-05 2024-09-04T17:48:17.409 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.410 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.410 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:17.410 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253452+0000", 2024-09-04T17:48:17.410 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3543000000000001e-05 2024-09-04T17:48:17.410 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.410 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.410 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:17.410 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304533+0000", 2024-09-04T17:48:17.410 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051081609 2024-09-04T17:48:17.410 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.410 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.410 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:17.410 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304557+0000", 2024-09-04T17:48:17.411 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4015e-05 2024-09-04T17:48:17.411 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:17.411 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:17.411 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:17.411 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.411 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.411 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44694 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:17.411 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253469+0000", 2024-09-04T17:48:17.411 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 87.096197375000003, 2024-09-04T17:48:17.411 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051121486000000001, 2024-09-04T17:48:17.411 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:17.411 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:17.411 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:17.412 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:17.412 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:17.412 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:17.412 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44694 2024-09-04T17:48:17.412 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.412 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:17.412 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.412 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:17.412 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253469+0000", 2024-09-04T17:48:17.412 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:17.412 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.412 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.413 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:17.413 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253469+0000", 2024-09-04T17:48:17.413 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:17.413 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.413 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.413 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:17.413 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253471+0000", 2024-09-04T17:48:17.413 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1400000000000001e-06 2024-09-04T17:48:17.413 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.413 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.413 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:17.413 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253476+0000", 2024-09-04T17:48:17.413 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7370000000000003e-06 2024-09-04T17:48:17.414 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.414 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.414 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:17.414 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253493+0000", 2024-09-04T17:48:17.414 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6549e-05 2024-09-04T17:48:17.414 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.414 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.414 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:17.414 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253506+0000", 2024-09-04T17:48:17.414 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3406e-05 2024-09-04T17:48:17.414 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.414 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.414 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:17.415 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304582+0000", 2024-09-04T17:48:17.415 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051075534999999998 2024-09-04T17:48:17.415 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.415 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.415 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:17.415 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304591+0000", 2024-09-04T17:48:17.415 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1190000000000003e-06 2024-09-04T17:48:17.415 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:17.415 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:17.415 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:17.415 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.415 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.415 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44695 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:17.416 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253552+0000", 2024-09-04T17:48:17.416 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 87.096114620999998, 2024-09-04T17:48:17.416 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051068872000000001, 2024-09-04T17:48:17.416 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:17.416 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:17.416 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:17.416 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:17.416 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:17.417 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:17.417 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44695 2024-09-04T17:48:17.417 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.417 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:17.417 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.417 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:17.417 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253552+0000", 2024-09-04T17:48:17.417 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:17.417 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.417 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.417 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:17.417 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253552+0000", 2024-09-04T17:48:17.417 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:17.418 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.418 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.418 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:17.418 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253553+0000", 2024-09-04T17:48:17.418 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1960000000000001e-06 2024-09-04T17:48:17.418 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.418 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.418 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:17.418 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253560+0000", 2024-09-04T17:48:17.418 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.2040000000000002e-06 2024-09-04T17:48:17.418 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.418 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.418 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:17.419 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253578+0000", 2024-09-04T17:48:17.419 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8241999999999999e-05 2024-09-04T17:48:17.419 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.419 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.419 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:17.419 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253591+0000", 2024-09-04T17:48:17.419 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3453e-05 2024-09-04T17:48:17.419 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.419 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.419 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:17.419 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304614+0000", 2024-09-04T17:48:17.419 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051022456000000001 2024-09-04T17:48:17.419 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.420 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.420 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:17.420 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304621+0000", 2024-09-04T17:48:17.420 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3209999999999999e-06 2024-09-04T17:48:17.420 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:17.420 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:17.420 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:17.420 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.420 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.420 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44696 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:17.420 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253609+0000", 2024-09-04T17:48:17.420 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 87.096058119999995, 2024-09-04T17:48:17.421 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051115351000000003, 2024-09-04T17:48:17.421 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:17.421 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:17.421 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:17.421 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:17.421 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:17.421 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:17.421 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44696 2024-09-04T17:48:17.421 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.421 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:17.421 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.421 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:17.421 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253609+0000", 2024-09-04T17:48:17.422 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:17.422 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.422 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.422 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:17.422 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253609+0000", 2024-09-04T17:48:17.422 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:17.422 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.422 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.422 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:17.422 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253610+0000", 2024-09-04T17:48:17.422 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2300000000000001e-06 2024-09-04T17:48:17.422 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.422 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.423 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:17.423 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253615+0000", 2024-09-04T17:48:17.423 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7260000000000002e-06 2024-09-04T17:48:17.423 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.423 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.423 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:17.423 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253632+0000", 2024-09-04T17:48:17.423 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7039999999999999e-05 2024-09-04T17:48:17.423 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.423 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.423 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:17.423 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253666+0000", 2024-09-04T17:48:17.423 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4662000000000001e-05 2024-09-04T17:48:17.423 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.424 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.424 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:17.424 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304677+0000", 2024-09-04T17:48:17.424 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051011017999999998 2024-09-04T17:48:17.424 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.424 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.424 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:17.424 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304724+0000", 2024-09-04T17:48:17.424 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6675000000000003e-05 2024-09-04T17:48:17.424 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:17.424 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:17.424 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:17.424 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.425 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.425 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44697 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:17.425 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253684+0000", 2024-09-04T17:48:17.425 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 87.095982625999994, 2024-09-04T17:48:17.425 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051073107, 2024-09-04T17:48:17.425 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:17.425 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:17.425 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:17.425 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:17.425 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:17.425 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:17.425 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44697 2024-09-04T17:48:17.425 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.426 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:17.426 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.426 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:17.426 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253684+0000", 2024-09-04T17:48:17.426 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:17.426 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.426 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.426 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:17.426 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253684+0000", 2024-09-04T17:48:17.426 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:17.426 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.426 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.426 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:17.427 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253685+0000", 2024-09-04T17:48:17.427 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0699999999999999e-06 2024-09-04T17:48:17.427 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.427 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.427 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:17.427 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253690+0000", 2024-09-04T17:48:17.427 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1959999999999999e-06 2024-09-04T17:48:17.427 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.427 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.427 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:17.427 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253707+0000", 2024-09-04T17:48:17.427 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6946e-05 2024-09-04T17:48:17.427 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.427 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.428 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:17.428 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253720+0000", 2024-09-04T17:48:17.428 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3076e-05 2024-09-04T17:48:17.428 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.428 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.428 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:17.428 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304745+0000", 2024-09-04T17:48:17.428 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051024760000000002 2024-09-04T17:48:17.428 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.428 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.428 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:17.428 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304757+0000", 2024-09-04T17:48:17.428 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2058999999999999e-05 2024-09-04T17:48:17.429 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:17.429 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:17.429 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:17.429 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.429 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.429 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44698 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:17.429 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253737+0000", 2024-09-04T17:48:17.429 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 87.095930275000001, 2024-09-04T17:48:17.429 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051176078, 2024-09-04T17:48:17.429 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:17.429 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:17.429 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:17.430 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:17.430 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:17.430 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:17.430 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44698 2024-09-04T17:48:17.430 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.430 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:17.430 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.430 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:17.430 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253737+0000", 2024-09-04T17:48:17.430 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:17.430 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.430 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.430 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:17.431 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253737+0000", 2024-09-04T17:48:17.431 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:17.431 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.431 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.431 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:17.431 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253738+0000", 2024-09-04T17:48:17.431 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0300000000000001e-06 2024-09-04T17:48:17.431 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.431 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.431 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:17.431 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253752+0000", 2024-09-04T17:48:17.431 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4358999999999999e-05 2024-09-04T17:48:17.431 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.432 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.432 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:17.432 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253771+0000", 2024-09-04T17:48:17.432 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8879999999999999e-05 2024-09-04T17:48:17.432 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.432 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.432 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:17.432 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253785+0000", 2024-09-04T17:48:17.432 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3852000000000001e-05 2024-09-04T17:48:17.432 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.432 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.432 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:17.433 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304876+0000", 2024-09-04T17:48:17.433 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051091105999999997 2024-09-04T17:48:17.433 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.433 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.433 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:17.433 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304913+0000", 2024-09-04T17:48:17.433 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.6850999999999997e-05 2024-09-04T17:48:17.433 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:17.433 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:17.433 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:17.433 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.433 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.433 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44699 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:17.434 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253804+0000", 2024-09-04T17:48:17.434 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 87.095862750999999, 2024-09-04T17:48:17.434 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051172250000000002, 2024-09-04T17:48:17.434 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:17.434 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:17.434 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:17.434 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:17.434 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:17.434 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:17.434 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44699 2024-09-04T17:48:17.434 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.434 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:17.435 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.435 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:17.435 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253804+0000", 2024-09-04T17:48:17.435 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:17.435 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.435 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.435 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:17.435 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253804+0000", 2024-09-04T17:48:17.435 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:17.435 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.435 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.435 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:17.435 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253805+0000", 2024-09-04T17:48:17.435 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.08e-06 2024-09-04T17:48:17.436 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.436 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.436 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:17.436 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253810+0000", 2024-09-04T17:48:17.436 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1830000000000002e-06 2024-09-04T17:48:17.436 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.436 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.436 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:17.436 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253828+0000", 2024-09-04T17:48:17.436 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7662000000000001e-05 2024-09-04T17:48:17.436 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.436 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.436 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:17.437 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253841+0000", 2024-09-04T17:48:17.437 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3203999999999999e-05 2024-09-04T17:48:17.437 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.437 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.437 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:17.437 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304966+0000", 2024-09-04T17:48:17.437 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051125273999999998 2024-09-04T17:48:17.437 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.437 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.437 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:17.437 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304976+0000", 2024-09-04T17:48:17.437 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.8470000000000008e-06 2024-09-04T17:48:17.438 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:17.438 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:17.438 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:17.438 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.438 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.438 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44700 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:17.438 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253858+0000", 2024-09-04T17:48:17.438 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 87.095809097, 2024-09-04T17:48:17.438 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051218957000000002, 2024-09-04T17:48:17.438 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:17.438 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:17.438 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:17.438 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:17.438 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:17.439 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:17.439 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44700 2024-09-04T17:48:17.439 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.439 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:17.439 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.439 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:17.439 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253858+0000", 2024-09-04T17:48:17.439 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:17.439 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.439 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.439 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:17.439 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253858+0000", 2024-09-04T17:48:17.439 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:17.440 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.440 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.440 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:17.440 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253859+0000", 2024-09-04T17:48:17.440 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.217e-06 2024-09-04T17:48:17.440 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.440 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.440 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:17.440 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253864+0000", 2024-09-04T17:48:17.440 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7600000000000002e-06 2024-09-04T17:48:17.440 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.440 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.440 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:17.441 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253880+0000", 2024-09-04T17:48:17.441 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6501999999999999e-05 2024-09-04T17:48:17.441 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.441 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.441 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:17.441 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253899+0000", 2024-09-04T17:48:17.441 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8907e-05 2024-09-04T17:48:17.441 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.441 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.441 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:17.441 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305052+0000", 2024-09-04T17:48:17.441 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051152989000000003 2024-09-04T17:48:17.441 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.442 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.442 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:17.442 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305077+0000", 2024-09-04T17:48:17.442 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4581999999999999e-05 2024-09-04T17:48:17.442 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:17.442 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:17.442 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:17.442 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.442 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.442 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44701 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:17.442 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253917+0000", 2024-09-04T17:48:17.442 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 87.095749745000006, 2024-09-04T17:48:17.442 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051231113000000002, 2024-09-04T17:48:17.443 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:17.443 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:17.443 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:17.443 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:17.443 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:17.443 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:17.443 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44701 2024-09-04T17:48:17.443 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.443 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:17.443 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.443 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:17.443 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253917+0000", 2024-09-04T17:48:17.443 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:17.444 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.444 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.444 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:17.444 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253917+0000", 2024-09-04T17:48:17.444 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:17.444 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.444 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.444 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:17.444 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253918+0000", 2024-09-04T17:48:17.444 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.136e-06 2024-09-04T17:48:17.444 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.444 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.444 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:17.444 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253924+0000", 2024-09-04T17:48:17.445 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.3070000000000002e-06 2024-09-04T17:48:17.445 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.445 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.445 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:17.445 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253940+0000", 2024-09-04T17:48:17.445 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6612e-05 2024-09-04T17:48:17.445 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.445 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.445 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:17.445 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253954+0000", 2024-09-04T17:48:17.445 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3709999999999999e-05 2024-09-04T17:48:17.445 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.445 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.446 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:17.446 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305136+0000", 2024-09-04T17:48:17.446 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051181935999999997 2024-09-04T17:48:17.446 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.446 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.446 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:17.446 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305148+0000", 2024-09-04T17:48:17.446 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2412000000000001e-05 2024-09-04T17:48:17.446 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:17.446 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:17.446 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:17.446 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:17.446 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:17.446 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:17.448 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-04T17:48:17.448 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:48:17.442+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:48:17.549 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:17.547+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:17.615 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:17.615 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:48:17.615 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:48:17.615 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:17.629 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-04T17:48:17.673 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:48:17.670+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:48:17.747 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:17.747 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:48:17.747 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:48:17.747 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:48:17.747 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:17.751 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:48:17.746+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:48:17.761 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-04T17:48:17.852 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:17.847+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:17.949 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:17.949 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:48:17.949 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:48:17.949 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:48:17.949 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.949 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8831 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2453504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:17.950 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:48:17.950 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 124.89068012, 2024-09-04T17:48:17.950 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14731111799999999, 2024-09-04T17:48:17.950 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:17.950 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:17.950 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:17.950 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:17.950 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:17.950 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:17.950 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8831 2024-09-04T17:48:17.950 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.950 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:17.950 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.951 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:17.951 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:48:17.951 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:17.951 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.951 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.951 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:17.951 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:48:17.951 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:17.951 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.951 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.951 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:17.951 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057811+0000", 2024-09-04T17:48:17.951 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.083e-06 2024-09-04T17:48:17.951 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.951 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.952 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:17.952 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057814+0000", 2024-09-04T17:48:17.952 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.407e-06 2024-09-04T17:48:17.952 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.952 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.952 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:17.952 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057825+0000", 2024-09-04T17:48:17.952 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.17e-05 2024-09-04T17:48:17.952 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.952 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.952 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:17.952 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057836+0000", 2024-09-04T17:48:17.952 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0283e-05 2024-09-04T17:48:17.953 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.953 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.953 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:17.953 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.204762+0000", 2024-09-04T17:48:17.953 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.146926205 2024-09-04T17:48:17.953 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.953 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.953 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:17.953 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.204895+0000", 2024-09-04T17:48:17.953 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013328999999999999 2024-09-04T17:48:17.953 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.954 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.954 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:17.954 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205121+0000", 2024-09-04T17:48:17.954 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00022614999999999999 2024-09-04T17:48:17.954 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:17.954 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:17.954 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:17.954 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.954 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.954 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8832 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2490368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:17.954 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:48:17.954 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 124.890642087, 2024-09-04T17:48:17.954 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14770012299999999, 2024-09-04T17:48:17.955 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:17.955 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:17.955 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:17.955 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:17.955 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:17.955 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:17.955 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8832 2024-09-04T17:48:17.955 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.955 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:17.955 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.955 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:17.955 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:48:17.955 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:17.956 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.956 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.956 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:17.956 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:48:17.956 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:17.956 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.956 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.956 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:17.956 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057849+0000", 2024-09-04T17:48:17.956 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6700000000000003e-07 2024-09-04T17:48:17.956 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.956 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.956 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:17.956 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057852+0000", 2024-09-04T17:48:17.956 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6359999999999998e-06 2024-09-04T17:48:17.957 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.957 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.957 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:17.957 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057860+0000", 2024-09-04T17:48:17.957 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8140000000000007e-06 2024-09-04T17:48:17.957 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.957 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.957 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:17.957 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057868+0000", 2024-09-04T17:48:17.957 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1999999999999997e-06 2024-09-04T17:48:17.957 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.957 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.957 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:17.958 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205208+0000", 2024-09-04T17:48:17.958 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14734067300000001 2024-09-04T17:48:17.958 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.958 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.958 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:17.958 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205347+0000", 2024-09-04T17:48:17.958 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013914900000000001 2024-09-04T17:48:17.958 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.958 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.958 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:17.958 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205548+0000", 2024-09-04T17:48:17.958 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000200984 2024-09-04T17:48:17.958 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:17.958 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:17.959 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:17.959 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.959 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.959 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8833 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2527232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:17.959 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:48:17.959 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 124.890610217, 2024-09-04T17:48:17.959 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148062572, 2024-09-04T17:48:17.959 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:17.959 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:17.959 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:17.959 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:17.960 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:17.960 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:17.960 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8833 2024-09-04T17:48:17.960 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.960 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:17.960 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.960 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:17.960 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:48:17.960 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:17.960 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.960 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.960 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:17.960 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:48:17.961 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:17.961 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.961 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.961 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:17.961 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057881+0000", 2024-09-04T17:48:17.961 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4300000000000002e-07 2024-09-04T17:48:17.961 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.961 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.961 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:17.961 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057893+0000", 2024-09-04T17:48:17.961 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1973999999999999e-05 2024-09-04T17:48:17.961 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.961 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.961 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:17.962 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057906+0000", 2024-09-04T17:48:17.962 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2716e-05 2024-09-04T17:48:17.962 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.962 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.962 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:17.962 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057913+0000", 2024-09-04T17:48:17.962 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3930000000000002e-06 2024-09-04T17:48:17.962 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.962 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.962 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:17.962 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205621+0000", 2024-09-04T17:48:17.962 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14770788800000001 2024-09-04T17:48:17.963 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.963 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.963 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:17.963 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205740+0000", 2024-09-04T17:48:17.963 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000118936 2024-09-04T17:48:17.963 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.963 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.963 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:17.963 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205943+0000", 2024-09-04T17:48:17.963 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020292199999999999 2024-09-04T17:48:17.964 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:17.964 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:17.964 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:17.964 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.964 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.964 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8834 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2564096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:17.964 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:48:17.964 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 124.89056693800001, 2024-09-04T17:48:17.965 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14846753200000001, 2024-09-04T17:48:17.965 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:17.965 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:17.965 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:17.965 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:17.965 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:17.965 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:17.965 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8834 2024-09-04T17:48:17.965 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.965 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:17.965 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.965 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:17.965 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:48:17.966 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:17.966 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.966 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.966 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:17.966 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:48:17.966 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:17.966 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.966 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.966 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:17.967 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057925+0000", 2024-09-04T17:48:17.967 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0270000000000001e-06 2024-09-04T17:48:17.967 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.967 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.967 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:17.967 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057936+0000", 2024-09-04T17:48:17.967 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.112e-05 2024-09-04T17:48:17.967 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.968 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.968 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:17.968 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057946+0000", 2024-09-04T17:48:17.968 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.009e-05 2024-09-04T17:48:17.968 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.968 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.968 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:17.968 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057953+0000", 2024-09-04T17:48:17.968 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.6229999999999997e-06 2024-09-04T17:48:17.968 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.968 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.968 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:17.969 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206027+0000", 2024-09-04T17:48:17.969 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148073591 2024-09-04T17:48:17.969 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.969 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.969 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:17.969 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206161+0000", 2024-09-04T17:48:17.969 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013411599999999999 2024-09-04T17:48:17.969 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.969 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.969 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:17.969 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206391+0000", 2024-09-04T17:48:17.969 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000229965 2024-09-04T17:48:17.969 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:17.970 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:17.970 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:17.970 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.970 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.970 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8835 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2600960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:17.970 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:48:17.970 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 124.890527288, 2024-09-04T17:48:17.970 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14882941599999999, 2024-09-04T17:48:17.970 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:17.971 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:17.971 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:17.971 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:17.971 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:17.971 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:17.971 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8835 2024-09-04T17:48:17.971 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.971 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:17.971 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.972 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:17.972 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:48:17.972 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:17.972 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.972 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.972 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:17.972 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:48:17.972 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:17.972 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.973 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.973 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:17.973 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058544+0000", 2024-09-04T17:48:17.973 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00058061899999999999 2024-09-04T17:48:17.973 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.973 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.973 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:17.973 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058550+0000", 2024-09-04T17:48:17.973 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.1129999999999997e-06 2024-09-04T17:48:17.973 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.973 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.973 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:17.974 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058566+0000", 2024-09-04T17:48:17.974 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5820000000000001e-05 2024-09-04T17:48:17.974 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.974 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.974 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:17.974 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058578+0000", 2024-09-04T17:48:17.974 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2537e-05 2024-09-04T17:48:17.974 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.974 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.974 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:17.974 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206470+0000", 2024-09-04T17:48:17.974 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14789153799999999 2024-09-04T17:48:17.974 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.975 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.975 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:17.975 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206589+0000", 2024-09-04T17:48:17.975 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000118837 2024-09-04T17:48:17.975 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.975 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.975 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:17.975 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206793+0000", 2024-09-04T17:48:17.975 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000203952 2024-09-04T17:48:17.975 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:17.975 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:17.975 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:17.975 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.976 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.976 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8836 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2654208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:17.976 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:48:17.976 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 124.88989460000001, 2024-09-04T17:48:17.976 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14862433999999999, 2024-09-04T17:48:17.976 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:17.976 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:17.976 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:17.976 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:17.976 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:17.976 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:17.977 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8836 2024-09-04T17:48:17.977 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.977 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:17.977 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.977 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:17.977 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:48:17.977 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:17.978 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.978 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.978 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:17.978 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:48:17.978 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:17.978 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.979 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.979 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:17.979 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062663+0000", 2024-09-04T17:48:17.979 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0040670619999999998 2024-09-04T17:48:17.979 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.979 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.980 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:17.980 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062668+0000", 2024-09-04T17:48:17.980 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4199999999999998e-06 2024-09-04T17:48:17.980 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.980 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.981 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:17.981 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062689+0000", 2024-09-04T17:48:17.981 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0356999999999998e-05 2024-09-04T17:48:17.981 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.981 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.982 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:17.982 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062726+0000", 2024-09-04T17:48:17.982 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7302999999999998e-05 2024-09-04T17:48:17.982 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.982 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.982 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:17.983 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206875+0000", 2024-09-04T17:48:17.983 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14414883100000001 2024-09-04T17:48:17.983 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.983 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.983 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:17.984 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.207016+0000", 2024-09-04T17:48:17.984 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000140822 2024-09-04T17:48:17.984 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.984 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.984 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:17.984 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.207220+0000", 2024-09-04T17:48:17.984 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020454499999999999 2024-09-04T17:48:17.985 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:17.985 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:17.985 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:17.985 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.985 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.989 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8844 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2928640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:17.989 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:48:17.989 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 124.88553051700001, 2024-09-04T17:48:17.989 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14752153100000001, 2024-09-04T17:48:17.989 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:17.990 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:17.990 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:17.990 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:17.990 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:17.990 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:17.990 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8844 2024-09-04T17:48:17.990 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.990 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:17.990 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.990 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:17.990 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:48:17.990 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:17.990 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.991 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.991 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:17.991 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:48:17.991 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:17.991 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.991 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.991 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:17.991 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062961+0000", 2024-09-04T17:48:17.991 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6700000000000003e-07 2024-09-04T17:48:17.991 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.991 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.991 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:17.991 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062973+0000", 2024-09-04T17:48:17.992 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2109999999999999e-05 2024-09-04T17:48:17.992 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.992 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.992 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:17.992 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062982+0000", 2024-09-04T17:48:17.992 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.6900000000000004e-06 2024-09-04T17:48:17.992 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.992 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.992 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:17.992 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062990+0000", 2024-09-04T17:48:17.992 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1899999999999998e-06 2024-09-04T17:48:17.992 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.992 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.993 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:17.993 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210090+0000", 2024-09-04T17:48:17.993 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14710026300000001 2024-09-04T17:48:17.993 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.993 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.993 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:17.993 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210225+0000", 2024-09-04T17:48:17.993 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013532400000000001 2024-09-04T17:48:17.993 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.993 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.993 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:17.993 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210481+0000", 2024-09-04T17:48:17.993 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00025628700000000001 2024-09-04T17:48:17.994 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:17.994 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:17.994 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:17.994 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.994 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.994 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8845 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2965504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:17.994 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:48:17.994 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 124.88549079400001, 2024-09-04T17:48:17.994 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147916819, 2024-09-04T17:48:17.994 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:17.994 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:17.994 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:17.995 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:17.995 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:17.995 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:17.995 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8845 2024-09-04T17:48:17.995 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.995 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:17.995 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.995 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:17.995 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:48:17.995 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:17.995 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.996 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.996 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:17.996 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:48:17.996 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:17.996 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.996 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.996 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:17.996 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:48:17.996 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.0399999999999995e-07 2024-09-04T17:48:17.996 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.996 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.996 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:17.996 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063003+0000", 2024-09-04T17:48:17.997 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6529999999999998e-06 2024-09-04T17:48:17.997 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.997 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.997 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:17.997 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063012+0000", 2024-09-04T17:48:17.997 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9500000000000007e-06 2024-09-04T17:48:17.997 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.997 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.997 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:17.997 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063027+0000", 2024-09-04T17:48:17.997 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5443000000000001e-05 2024-09-04T17:48:17.997 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.997 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.998 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:17.998 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210571+0000", 2024-09-04T17:48:17.998 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14754353100000001 2024-09-04T17:48:17.998 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.998 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.998 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:17.998 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210714+0000", 2024-09-04T17:48:17.998 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014295599999999999 2024-09-04T17:48:17.998 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.998 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.998 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:17.998 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210916+0000", 2024-09-04T17:48:17.998 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020258200000000001 2024-09-04T17:48:17.999 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:17.999 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:17.999 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:17.999 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:17.999 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:17.999 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8846 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3002368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:17.999 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:48:17.999 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 124.88545325699999, 2024-09-04T17:48:17.999 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14828011599999999, 2024-09-04T17:48:17.999 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:17.999 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:17.999 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:17.999 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:17.999 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:18.000 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:18.000 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8846 2024-09-04T17:48:18.000 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.000 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:18.000 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.000 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:18.000 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:48:18.000 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:18.000 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.000 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.000 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:18.000 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:48:18.000 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:18.001 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.001 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.001 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:18.001 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063038+0000", 2024-09-04T17:48:18.001 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.23e-07 2024-09-04T17:48:18.001 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.001 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.001 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:18.001 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063051+0000", 2024-09-04T17:48:18.001 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2757e-05 2024-09-04T17:48:18.001 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.001 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.001 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:18.002 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063070+0000", 2024-09-04T17:48:18.002 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9663000000000001e-05 2024-09-04T17:48:18.002 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.002 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.002 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:18.002 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063079+0000", 2024-09-04T17:48:18.002 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.0929999999999992e-06 2024-09-04T17:48:18.002 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.002 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.002 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:18.002 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210989+0000", 2024-09-04T17:48:18.002 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14790984500000001 2024-09-04T17:48:18.002 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.002 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.003 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:18.003 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211109+0000", 2024-09-04T17:48:18.003 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000120056 2024-09-04T17:48:18.003 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.003 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.003 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:18.003 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211317+0000", 2024-09-04T17:48:18.003 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020797900000000001 2024-09-04T17:48:18.003 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.003 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:18.003 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.003 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.003 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.003 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8847 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3039232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:18.004 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:48:18.004 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 124.885378825, 2024-09-04T17:48:18.004 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14859408900000001, 2024-09-04T17:48:18.004 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:18.004 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:18.004 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:18.004 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:18.004 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:18.004 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:18.004 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8847 2024-09-04T17:48:18.004 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.004 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:18.004 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.004 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:18.005 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:48:18.005 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:18.005 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.005 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.005 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:18.005 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:48:18.005 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:18.005 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.005 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.005 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:18.005 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063113+0000", 2024-09-04T17:48:18.005 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.204e-06 2024-09-04T17:48:18.005 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.006 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.006 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:18.006 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063116+0000", 2024-09-04T17:48:18.006 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.3330000000000001e-06 2024-09-04T17:48:18.006 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.006 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.006 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:18.006 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063126+0000", 2024-09-04T17:48:18.006 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0166999999999999e-05 2024-09-04T17:48:18.006 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.007 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.007 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:18.007 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063134+0000", 2024-09-04T17:48:18.007 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4329999999999998e-06 2024-09-04T17:48:18.007 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.007 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.007 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:18.007 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211392+0000", 2024-09-04T17:48:18.007 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148258109 2024-09-04T17:48:18.007 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.007 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.007 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:18.008 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211506+0000", 2024-09-04T17:48:18.008 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000113813 2024-09-04T17:48:18.008 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.008 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.008 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:18.008 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211706+0000", 2024-09-04T17:48:18.008 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020002999999999999 2024-09-04T17:48:18.008 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.008 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:18.008 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.008 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.008 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.008 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8848 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:18.009 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:48:18.009 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 124.885346488, 2024-09-04T17:48:18.009 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14907967599999999, 2024-09-04T17:48:18.009 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:18.009 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:18.009 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:18.009 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:18.009 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:18.009 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:18.009 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8848 2024-09-04T17:48:18.009 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.009 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:18.009 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.010 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:18.010 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:48:18.010 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:18.010 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.010 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.010 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:18.010 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:48:18.010 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:18.010 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.010 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.010 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:18.010 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070589+0000", 2024-09-04T17:48:18.010 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.007445386 2024-09-04T17:48:18.011 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.011 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.011 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:18.011 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070600+0000", 2024-09-04T17:48:18.011 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0346999999999999e-05 2024-09-04T17:48:18.011 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.011 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.011 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:18.011 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070622+0000", 2024-09-04T17:48:18.011 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2289999999999998e-05 2024-09-04T17:48:18.011 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.011 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.011 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:18.012 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070636+0000", 2024-09-04T17:48:18.012 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4093e-05 2024-09-04T17:48:18.012 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.012 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.012 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:18.012 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211815+0000", 2024-09-04T17:48:18.012 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14117874699999999 2024-09-04T17:48:18.012 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.012 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.012 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:18.012 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211983+0000", 2024-09-04T17:48:18.012 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00016852800000000001 2024-09-04T17:48:18.013 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.013 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.013 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:18.013 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.212224+0000", 2024-09-04T17:48:18.013 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024028499999999999 2024-09-04T17:48:18.013 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.013 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:18.013 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.013 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.013 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.013 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8862 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2019328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:18.013 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:48:18.013 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 124.87729702, 2024-09-04T17:48:18.014 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14755236399999999, 2024-09-04T17:48:18.014 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:18.014 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:18.014 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:18.014 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:18.014 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:18.014 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:18.014 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8862 2024-09-04T17:48:18.014 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.014 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:18.014 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.014 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:18.014 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:48:18.015 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:18.015 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.015 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.015 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:18.015 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:48:18.015 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:18.015 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.015 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.015 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:18.015 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071194+0000", 2024-09-04T17:48:18.015 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.5000000000000002e-07 2024-09-04T17:48:18.015 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.015 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.016 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:18.016 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071197+0000", 2024-09-04T17:48:18.016 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5900000000000002e-06 2024-09-04T17:48:18.016 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.016 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.016 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:18.016 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071206+0000", 2024-09-04T17:48:18.016 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9700000000000005e-06 2024-09-04T17:48:18.017 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.017 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.017 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:18.017 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071213+0000", 2024-09-04T17:48:18.017 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.9070000000000001e-06 2024-09-04T17:48:18.017 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.017 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.017 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:18.017 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218396+0000", 2024-09-04T17:48:18.017 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147183273 2024-09-04T17:48:18.017 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.017 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.017 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:18.017 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218538+0000", 2024-09-04T17:48:18.018 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014194199999999999 2024-09-04T17:48:18.018 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.018 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.018 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:18.018 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218746+0000", 2024-09-04T17:48:18.018 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000208032 2024-09-04T17:48:18.018 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.018 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:18.018 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.018 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.018 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.018 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8863 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2056192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:18.018 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:48:18.019 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 124.87726871, 2024-09-04T17:48:18.019 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14792281099999999, 2024-09-04T17:48:18.019 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:18.019 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:18.019 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:18.019 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:18.019 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:18.019 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:18.019 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8863 2024-09-04T17:48:18.019 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.019 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:18.019 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.019 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:18.020 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:48:18.020 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:18.020 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.020 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.020 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:18.020 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:48:18.020 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:18.020 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.020 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.020 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:18.020 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:48:18.020 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1299999999999999e-07 2024-09-04T17:48:18.020 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.021 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.021 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:18.021 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071225+0000", 2024-09-04T17:48:18.021 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6199999999999999e-06 2024-09-04T17:48:18.021 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.021 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.021 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:18.021 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071234+0000", 2024-09-04T17:48:18.021 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8769999999999999e-06 2024-09-04T17:48:18.021 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.021 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.021 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:18.022 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071251+0000", 2024-09-04T17:48:18.022 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.738e-05 2024-09-04T17:48:18.022 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.022 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.022 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:18.022 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218798+0000", 2024-09-04T17:48:18.022 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14754645 2024-09-04T17:48:18.022 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.022 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.022 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:18.022 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218926+0000", 2024-09-04T17:48:18.022 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000128446 2024-09-04T17:48:18.022 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.023 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.023 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:18.023 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219145+0000", 2024-09-04T17:48:18.023 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021832500000000001 2024-09-04T17:48:18.023 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.023 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:18.023 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.023 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.023 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.023 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8864 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2093056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:18.023 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:48:18.023 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 124.87722713399999, 2024-09-04T17:48:18.023 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148277309, 2024-09-04T17:48:18.023 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:18.024 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:18.024 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:18.024 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:18.024 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:18.024 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:18.024 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8864 2024-09-04T17:48:18.024 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.024 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:18.024 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.024 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:18.024 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:48:18.024 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:18.025 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.025 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.025 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:18.025 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:48:18.025 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:18.025 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.025 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.025 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:18.025 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071264+0000", 2024-09-04T17:48:18.025 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3e-07 2024-09-04T17:48:18.025 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.025 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.025 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:18.026 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071267+0000", 2024-09-04T17:48:18.026 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0139999999999999e-06 2024-09-04T17:48:18.026 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.026 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.026 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:18.026 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071276+0000", 2024-09-04T17:48:18.026 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1689999999999998e-06 2024-09-04T17:48:18.026 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.026 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.026 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:18.026 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071283+0000", 2024-09-04T17:48:18.026 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2269999999999996e-06 2024-09-04T17:48:18.026 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.027 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.027 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:18.027 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219195+0000", 2024-09-04T17:48:18.027 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147911868 2024-09-04T17:48:18.027 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.027 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.027 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:18.027 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219324+0000", 2024-09-04T17:48:18.027 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00012835900000000001 2024-09-04T17:48:18.027 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.027 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.027 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:18.027 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219541+0000", 2024-09-04T17:48:18.027 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000216942 2024-09-04T17:48:18.028 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.028 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:18.028 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.028 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.028 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.028 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8865 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2109440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:18.028 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:48:18.028 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 124.877197674, 2024-09-04T17:48:18.028 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14865747300000001, 2024-09-04T17:48:18.028 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:18.028 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:18.028 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:18.028 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:18.029 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:18.029 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:18.029 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8865 2024-09-04T17:48:18.029 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.029 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:18.029 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.029 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:18.029 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:48:18.029 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:18.029 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.029 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.029 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:18.029 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:48:18.029 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:18.030 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.030 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.030 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:18.030 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:48:18.030 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7999999999999995e-07 2024-09-04T17:48:18.030 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.030 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.030 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:18.030 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071296+0000", 2024-09-04T17:48:18.030 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6900000000000001e-06 2024-09-04T17:48:18.030 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.030 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.030 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:18.031 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071305+0000", 2024-09-04T17:48:18.031 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.5499999999999995e-06 2024-09-04T17:48:18.031 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.031 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.031 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:18.031 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071312+0000", 2024-09-04T17:48:18.031 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1559999999999998e-06 2024-09-04T17:48:18.031 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.031 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.031 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:18.031 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219591+0000", 2024-09-04T17:48:18.031 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14827944900000001 2024-09-04T17:48:18.031 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.031 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.032 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:18.032 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219731+0000", 2024-09-04T17:48:18.032 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013931600000000001 2024-09-04T17:48:18.032 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.032 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.032 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:18.032 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219950+0000", 2024-09-04T17:48:18.032 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021963199999999999 2024-09-04T17:48:18.032 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.032 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:18.032 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.032 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.032 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.033 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8866 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2146304~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:18.033 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:48:18.033 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 124.87716944100001, 2024-09-04T17:48:18.033 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14906117899999999, 2024-09-04T17:48:18.033 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:18.033 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:18.033 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:18.033 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:18.033 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:18.033 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:18.033 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8866 2024-09-04T17:48:18.033 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.034 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:18.034 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.034 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:18.034 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:48:18.034 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:18.034 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.034 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.034 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:18.034 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:48:18.034 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:18.034 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.034 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.034 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:18.035 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071322+0000", 2024-09-04T17:48:18.035 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7999999999999995e-07 2024-09-04T17:48:18.035 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.035 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.035 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:18.035 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071338+0000", 2024-09-04T17:48:18.035 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5872999999999999e-05 2024-09-04T17:48:18.035 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.035 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.035 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:18.035 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071347+0000", 2024-09-04T17:48:18.035 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4669999999999993e-06 2024-09-04T17:48:18.035 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.035 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.036 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:18.036 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071354+0000", 2024-09-04T17:48:18.036 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2099999999999996e-06 2024-09-04T17:48:18.036 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.036 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.036 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:18.036 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220008+0000", 2024-09-04T17:48:18.036 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148653642 2024-09-04T17:48:18.036 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.036 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.036 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:18.036 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220155+0000", 2024-09-04T17:48:18.036 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014726600000000001 2024-09-04T17:48:18.037 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.037 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.037 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:18.037 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220382+0000", 2024-09-04T17:48:18.037 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00022704100000000001 2024-09-04T17:48:18.037 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.037 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:18.037 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.037 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.037 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.037 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8867 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2183168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:18.037 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:48:18.037 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 124.877126088, 2024-09-04T17:48:18.038 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.149445831, 2024-09-04T17:48:18.038 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:18.038 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:18.038 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:18.038 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:18.038 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:18.038 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:18.038 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8867 2024-09-04T17:48:18.038 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.038 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:18.038 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.038 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:18.038 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:48:18.039 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:18.039 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.039 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.039 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:18.039 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:48:18.039 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:18.039 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.039 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.039 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:18.039 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078653+0000", 2024-09-04T17:48:18.039 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0072890430000000003 2024-09-04T17:48:18.039 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.039 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.039 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:18.040 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078664+0000", 2024-09-04T17:48:18.040 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0307e-05 2024-09-04T17:48:18.040 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.040 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.040 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:18.040 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078690+0000", 2024-09-04T17:48:18.040 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6780000000000001e-05 2024-09-04T17:48:18.040 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.040 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.040 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:18.040 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078706+0000", 2024-09-04T17:48:18.040 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5676999999999999e-05 2024-09-04T17:48:18.040 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.041 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.041 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:18.041 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220467+0000", 2024-09-04T17:48:18.041 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14176103500000001 2024-09-04T17:48:18.041 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.041 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.041 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:18.041 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220595+0000", 2024-09-04T17:48:18.041 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000127427 2024-09-04T17:48:18.041 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.041 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.041 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:18.041 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220810+0000", 2024-09-04T17:48:18.042 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021556200000000001 2024-09-04T17:48:18.042 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.042 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:18.042 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.042 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.042 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.042 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8881 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2695168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:18.042 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:48:18.042 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 124.869304019, 2024-09-04T17:48:18.042 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147543547, 2024-09-04T17:48:18.042 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:18.042 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:18.042 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:18.043 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:18.043 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:18.043 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:18.043 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8881 2024-09-04T17:48:18.043 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.043 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:18.043 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.043 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:18.043 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:48:18.043 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:18.043 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.043 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.043 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:18.044 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:48:18.044 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:18.044 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.044 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.044 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:18.044 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079187+0000", 2024-09-04T17:48:18.044 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7700000000000004e-07 2024-09-04T17:48:18.044 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.044 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.044 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:18.044 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079198+0000", 2024-09-04T17:48:18.044 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0849999999999999e-05 2024-09-04T17:48:18.044 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.045 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.045 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:18.045 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079207+0000", 2024-09-04T17:48:18.045 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5599999999999999e-06 2024-09-04T17:48:18.045 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.045 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.045 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:18.045 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079224+0000", 2024-09-04T17:48:18.045 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6387000000000001e-05 2024-09-04T17:48:18.045 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.045 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.045 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:18.045 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226414+0000", 2024-09-04T17:48:18.046 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14718975000000001 2024-09-04T17:48:18.046 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.046 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.046 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:18.046 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226534+0000", 2024-09-04T17:48:18.046 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000120288 2024-09-04T17:48:18.046 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.046 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.046 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:18.046 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226730+0000", 2024-09-04T17:48:18.046 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000196035 2024-09-04T17:48:18.046 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.046 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:18.046 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.047 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.047 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.047 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8882 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2732032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:18.047 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:48:18.047 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 124.86925580499999, 2024-09-04T17:48:18.047 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14795022399999999, 2024-09-04T17:48:18.047 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:18.047 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:18.047 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:18.047 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:18.047 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:18.047 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:18.047 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8882 2024-09-04T17:48:18.048 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.048 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:18.048 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.048 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:18.048 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:48:18.048 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:18.048 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.048 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.048 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:18.048 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:48:18.048 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:18.048 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.048 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.049 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:18.049 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086588+0000", 2024-09-04T17:48:18.049 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0073530959999999999 2024-09-04T17:48:18.049 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.049 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.049 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:18.049 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086598+0000", 2024-09-04T17:48:18.049 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0122999999999999e-05 2024-09-04T17:48:18.049 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.049 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.049 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:18.049 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086626+0000", 2024-09-04T17:48:18.049 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.7914000000000001e-05 2024-09-04T17:48:18.050 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.050 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.050 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:18.050 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086641+0000", 2024-09-04T17:48:18.050 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5270000000000001e-05 2024-09-04T17:48:18.050 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.050 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.050 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:18.050 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226801+0000", 2024-09-04T17:48:18.050 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.140160321 2024-09-04T17:48:18.050 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.050 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.050 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:18.050 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226951+0000", 2024-09-04T17:48:18.051 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014965200000000001 2024-09-04T17:48:18.051 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.051 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.051 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:18.051 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.227185+0000", 2024-09-04T17:48:18.051 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00023384799999999999 2024-09-04T17:48:18.051 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.051 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:18.051 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.051 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.051 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.051 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:93470 3.1f 3:f95bc674:::rbd_header.1134878eadad:head [watch reconnect cookie 140018821420464 gen 16] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-04T17:48:18.051 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:57.837081+0000", 2024-09-04T17:48:18.052 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 20.111409922, 2024-09-04T17:48:18.052 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.072023486, 2024-09-04T17:48:18.052 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:18.052 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:18.052 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:18.052 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:18.052 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:18.052 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:18.052 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 93470 2024-09-04T17:48:18.052 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.052 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:18.052 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.052 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:18.053 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:57.837081+0000", 2024-09-04T17:48:18.053 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:18.053 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.053 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.053 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:18.053 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:57.837081+0000", 2024-09-04T17:48:18.053 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:18.053 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.053 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.053 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:18.053 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:57.837084+0000", 2024-09-04T17:48:18.053 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.8229999999999997e-06 2024-09-04T17:48:18.053 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.054 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.054 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:18.054 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:57.837098+0000", 2024-09-04T17:48:18.054 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3436e-05 2024-09-04T17:48:18.054 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.054 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.054 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:18.054 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:57.837123+0000", 2024-09-04T17:48:18.054 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5079000000000002e-05 2024-09-04T17:48:18.054 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.054 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.054 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:18.054 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:57.837145+0000", 2024-09-04T17:48:18.055 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.228e-05 2024-09-04T17:48:18.055 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.055 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.055 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:18.055 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:57.837360+0000", 2024-09-04T17:48:18.055 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021464100000000001 2024-09-04T17:48:18.055 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.055 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.055 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for peered", 2024-09-04T17:48:18.055 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:57.837410+0000", 2024-09-04T17:48:18.055 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0235000000000003e-05 2024-09-04T17:48:18.055 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.055 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.055 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:18.056 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.900175+0000", 2024-09-04T17:48:18.056 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0627652830000001 2024-09-04T17:48:18.056 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.056 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.056 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for rw locks", 2024-09-04T17:48:18.056 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.900370+0000", 2024-09-04T17:48:18.056 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019497199999999999 2024-09-04T17:48:18.056 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.056 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.056 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:18.056 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.906957+0000", 2024-09-04T17:48:18.056 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0065866889999999997 2024-09-04T17:48:18.056 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.057 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.057 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:18.057 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.907088+0000", 2024-09-04T17:48:18.057 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013056100000000001 2024-09-04T17:48:18.057 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.057 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.057 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:18.057 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.909104+0000", 2024-09-04T17:48:18.057 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.002016487 2024-09-04T17:48:18.057 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.057 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:18.057 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.057 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.058 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:18.058 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:18.059 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-04T17:48:18.060 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:48:18.055+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:48:18.160 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:18.155+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:18.192 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:18.192 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:48:18.193 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:48:18.193 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:18.203 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-04T17:48:18.260 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:48:18.254+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:48:18.324 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:18.325 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:48:18.325 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:48:18.325 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:48:18.325 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:18.339 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-04T17:48:18.381 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:18.378+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:18.462 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:18.458+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:18.510 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:18.510 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:48:18.510 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:48:18.510 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:48:18.510 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.510 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60921 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 262144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:18.510 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822920+0000", 2024-09-04T17:48:18.511 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 64.685558364000002, 2024-09-04T17:48:18.511 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073904394999999998, 2024-09-04T17:48:18.511 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:18.511 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:18.511 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:18.511 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:18.511 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:18.511 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:18.511 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60921 2024-09-04T17:48:18.511 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.511 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:18.511 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.512 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:18.512 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822920+0000", 2024-09-04T17:48:18.512 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:18.512 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.512 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.512 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:18.512 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822920+0000", 2024-09-04T17:48:18.512 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:18.512 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.512 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.512 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:18.512 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822920+0000", 2024-09-04T17:48:18.512 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4999999999999999e-07 2024-09-04T17:48:18.513 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.513 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.513 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:18.513 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822921+0000", 2024-09-04T17:48:18.513 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.113e-06 2024-09-04T17:48:18.513 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.513 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.513 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:18.513 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822924+0000", 2024-09-04T17:48:18.513 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.354e-06 2024-09-04T17:48:18.513 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.513 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.513 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:18.514 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822927+0000", 2024-09-04T17:48:18.514 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6199999999999999e-06 2024-09-04T17:48:18.514 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.514 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.514 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:18.514 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896586+0000", 2024-09-04T17:48:18.514 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073658976000000001 2024-09-04T17:48:18.514 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.514 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.514 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:18.514 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896626+0000", 2024-09-04T17:48:18.514 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0484000000000001e-05 2024-09-04T17:48:18.515 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.515 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.515 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:18.515 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896824+0000", 2024-09-04T17:48:18.515 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000197598 2024-09-04T17:48:18.515 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.515 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:18.515 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.515 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.515 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.515 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60922 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:18.515 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822930+0000", 2024-09-04T17:48:18.516 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 64.685547788999997, 2024-09-04T17:48:18.516 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074178872000000007, 2024-09-04T17:48:18.516 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:18.516 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:18.516 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:18.516 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:18.516 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:18.516 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:18.516 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60922 2024-09-04T17:48:18.516 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.516 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:18.516 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.517 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:18.517 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822930+0000", 2024-09-04T17:48:18.517 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:18.517 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.517 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.517 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:18.517 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822930+0000", 2024-09-04T17:48:18.517 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:18.517 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.517 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.517 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:18.517 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822930+0000", 2024-09-04T17:48:18.517 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.36e-07 2024-09-04T17:48:18.517 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.518 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.518 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:18.518 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822932+0000", 2024-09-04T17:48:18.518 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.567e-06 2024-09-04T17:48:18.518 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.518 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.518 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:18.518 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822938+0000", 2024-09-04T17:48:18.518 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4840000000000003e-06 2024-09-04T17:48:18.518 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.518 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.518 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:18.518 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822942+0000", 2024-09-04T17:48:18.519 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2039999999999999e-06 2024-09-04T17:48:18.519 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.519 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.519 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:18.519 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896850+0000", 2024-09-04T17:48:18.519 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073907941000000005 2024-09-04T17:48:18.519 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.519 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.519 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:18.519 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896891+0000", 2024-09-04T17:48:18.519 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0964999999999998e-05 2024-09-04T17:48:18.519 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.519 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.519 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:18.520 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897109+0000", 2024-09-04T17:48:18.520 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021847500000000001 2024-09-04T17:48:18.520 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.520 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:18.520 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.520 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.520 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.520 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60923 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 475136~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:18.520 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822949+0000", 2024-09-04T17:48:18.520 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 64.685529498999998, 2024-09-04T17:48:18.520 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074426928000000003, 2024-09-04T17:48:18.520 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:18.520 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:18.521 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:18.521 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:18.521 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:18.521 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:18.521 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60923 2024-09-04T17:48:18.521 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.521 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:18.521 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.521 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:18.521 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822949+0000", 2024-09-04T17:48:18.521 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:18.521 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.521 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.522 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:18.522 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822949+0000", 2024-09-04T17:48:18.522 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:18.522 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.522 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.522 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:18.522 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822949+0000", 2024-09-04T17:48:18.522 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7399999999999999e-07 2024-09-04T17:48:18.522 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.523 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.523 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:18.523 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822951+0000", 2024-09-04T17:48:18.523 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1940000000000001e-06 2024-09-04T17:48:18.523 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.523 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.523 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:18.523 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822957+0000", 2024-09-04T17:48:18.523 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.0650000000000004e-06 2024-09-04T17:48:18.523 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.523 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.523 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:18.523 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822960+0000", 2024-09-04T17:48:18.524 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.8009999999999999e-06 2024-09-04T17:48:18.524 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.524 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.524 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:18.524 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897138+0000", 2024-09-04T17:48:18.524 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074177990999999999 2024-09-04T17:48:18.524 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.524 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.524 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:18.524 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897179+0000", 2024-09-04T17:48:18.524 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.1057999999999999e-05 2024-09-04T17:48:18.524 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.525 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.525 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:18.525 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897375+0000", 2024-09-04T17:48:18.525 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019644500000000001 2024-09-04T17:48:18.525 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.525 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:18.525 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.525 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.525 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.525 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60924 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:18.525 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822964+0000", 2024-09-04T17:48:18.525 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 64.685514314000002, 2024-09-04T17:48:18.525 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074531902999999997, 2024-09-04T17:48:18.526 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:18.526 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:18.526 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:18.526 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:18.526 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:18.526 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:18.526 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60924 2024-09-04T17:48:18.526 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.526 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:18.526 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.526 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:18.526 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822964+0000", 2024-09-04T17:48:18.527 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:18.527 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.527 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.527 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:18.527 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822964+0000", 2024-09-04T17:48:18.527 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:18.527 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.527 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.527 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:18.527 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822964+0000", 2024-09-04T17:48:18.527 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4999999999999999e-07 2024-09-04T17:48:18.527 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.527 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.527 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:18.528 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822966+0000", 2024-09-04T17:48:18.528 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.917e-06 2024-09-04T17:48:18.528 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.528 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.528 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:18.528 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822969+0000", 2024-09-04T17:48:18.528 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.315e-06 2024-09-04T17:48:18.528 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.528 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.528 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:18.528 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822972+0000", 2024-09-04T17:48:18.528 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.599e-06 2024-09-04T17:48:18.528 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.528 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.529 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:18.529 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897398+0000", 2024-09-04T17:48:18.529 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074426168000000001 2024-09-04T17:48:18.529 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.529 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.529 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:18.529 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897441+0000", 2024-09-04T17:48:18.529 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.3158000000000003e-05 2024-09-04T17:48:18.529 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.529 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.529 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:18.529 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897496+0000", 2024-09-04T17:48:18.529 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4496000000000001e-05 2024-09-04T17:48:18.530 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.530 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:18.530 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.530 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.530 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.530 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60925 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 720896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:18.530 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822975+0000", 2024-09-04T17:48:18.530 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 64.685502878999998, 2024-09-04T17:48:18.530 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074635272000000003, 2024-09-04T17:48:18.530 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:18.530 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:18.530 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:18.530 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:18.531 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:18.531 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:18.531 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60925 2024-09-04T17:48:18.531 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.531 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:18.531 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.531 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:18.531 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822975+0000", 2024-09-04T17:48:18.531 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:18.531 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.531 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.531 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:18.531 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822975+0000", 2024-09-04T17:48:18.531 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:18.532 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.532 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.532 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:18.532 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822975+0000", 2024-09-04T17:48:18.532 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.34e-07 2024-09-04T17:48:18.532 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.532 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.532 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:18.532 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822976+0000", 2024-09-04T17:48:18.532 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0359999999999999e-06 2024-09-04T17:48:18.532 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.532 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.532 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:18.533 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822980+0000", 2024-09-04T17:48:18.533 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.242e-06 2024-09-04T17:48:18.533 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.533 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.533 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:18.533 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822983+0000", 2024-09-04T17:48:18.533 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.8830000000000002e-06 2024-09-04T17:48:18.533 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.533 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.533 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:18.533 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897519+0000", 2024-09-04T17:48:18.533 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074536366000000007 2024-09-04T17:48:18.533 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.534 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.534 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:18.534 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897560+0000", 2024-09-04T17:48:18.534 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0677999999999999e-05 2024-09-04T17:48:18.534 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.534 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.534 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:18.534 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897610+0000", 2024-09-04T17:48:18.534 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0832999999999999e-05 2024-09-04T17:48:18.534 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.534 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:18.534 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.534 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.534 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.535 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60926 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 860160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:18.535 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822986+0000", 2024-09-04T17:48:18.535 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 64.685492280999995, 2024-09-04T17:48:18.535 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074736413000000002, 2024-09-04T17:48:18.535 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:18.535 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:18.535 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:18.535 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:18.535 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:18.535 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:18.535 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60926 2024-09-04T17:48:18.535 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.535 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:18.536 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.536 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:18.536 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822986+0000", 2024-09-04T17:48:18.536 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:18.536 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.536 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.536 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:18.536 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822986+0000", 2024-09-04T17:48:18.536 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:18.536 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.536 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.536 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:18.536 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822986+0000", 2024-09-04T17:48:18.537 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5600000000000002e-07 2024-09-04T17:48:18.537 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.537 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.537 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:18.537 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822987+0000", 2024-09-04T17:48:18.537 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.082e-06 2024-09-04T17:48:18.537 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.537 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.537 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:18.537 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822990+0000", 2024-09-04T17:48:18.537 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.2219999999999998e-06 2024-09-04T17:48:18.537 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.537 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.537 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:18.538 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822994+0000", 2024-09-04T17:48:18.538 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.19e-06 2024-09-04T17:48:18.538 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.538 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.538 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:18.538 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897632+0000", 2024-09-04T17:48:18.538 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074638205999999999 2024-09-04T17:48:18.538 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.538 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.538 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:18.538 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897672+0000", 2024-09-04T17:48:18.538 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.9873999999999998e-05 2024-09-04T17:48:18.538 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.539 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.539 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:18.539 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897722+0000", 2024-09-04T17:48:18.539 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0583e-05 2024-09-04T17:48:18.539 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.539 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:18.539 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.539 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.539 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.539 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60927 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 966656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:18.539 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823000+0000", 2024-09-04T17:48:18.539 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 64.685478098999994, 2024-09-04T17:48:18.539 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074834036000000007, 2024-09-04T17:48:18.540 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:18.540 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:18.540 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:18.540 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:18.540 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:18.540 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:18.540 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60927 2024-09-04T17:48:18.540 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.540 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:18.540 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.540 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:18.540 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823000+0000", 2024-09-04T17:48:18.540 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:18.541 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.541 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.541 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:18.541 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823000+0000", 2024-09-04T17:48:18.541 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:18.541 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.541 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.541 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:18.541 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823000+0000", 2024-09-04T17:48:18.541 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4099999999999999e-07 2024-09-04T17:48:18.541 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.541 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.541 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:18.542 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823003+0000", 2024-09-04T17:48:18.542 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0810000000000001e-06 2024-09-04T17:48:18.542 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.542 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.542 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:18.542 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823011+0000", 2024-09-04T17:48:18.542 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7090000000000008e-06 2024-09-04T17:48:18.542 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.542 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.542 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:18.542 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823017+0000", 2024-09-04T17:48:18.542 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.3890000000000004e-06 2024-09-04T17:48:18.543 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.543 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.543 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:18.543 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897745+0000", 2024-09-04T17:48:18.543 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074728252999999994 2024-09-04T17:48:18.543 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.543 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.543 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:18.543 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897785+0000", 2024-09-04T17:48:18.543 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.9830999999999999e-05 2024-09-04T17:48:18.543 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.543 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.543 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:18.543 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897834+0000", 2024-09-04T17:48:18.544 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9231999999999999e-05 2024-09-04T17:48:18.544 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.544 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:18.544 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.544 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.544 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.544 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60928 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:18.544 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823024+0000", 2024-09-04T17:48:18.544 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 64.685454168000007, 2024-09-04T17:48:18.544 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075067479000000006, 2024-09-04T17:48:18.544 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:18.544 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:18.544 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:18.545 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:18.545 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:18.545 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:18.545 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60928 2024-09-04T17:48:18.545 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.545 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:18.545 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.545 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:18.545 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823024+0000", 2024-09-04T17:48:18.545 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:18.545 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.545 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.546 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:18.546 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823024+0000", 2024-09-04T17:48:18.546 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:18.546 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.546 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.546 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:18.546 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823024+0000", 2024-09-04T17:48:18.546 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6499999999999999e-07 2024-09-04T17:48:18.546 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.546 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.546 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:18.546 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823027+0000", 2024-09-04T17:48:18.546 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.3099999999999999e-06 2024-09-04T17:48:18.547 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.547 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.547 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:18.547 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823033+0000", 2024-09-04T17:48:18.547 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7429999999999996e-06 2024-09-04T17:48:18.547 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.547 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.547 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:18.547 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823038+0000", 2024-09-04T17:48:18.547 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.561e-06 2024-09-04T17:48:18.547 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.547 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.547 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:18.547 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897855+0000", 2024-09-04T17:48:18.548 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074816740000000007 2024-09-04T17:48:18.548 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.548 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.548 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:18.548 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897894+0000", 2024-09-04T17:48:18.548 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.9085999999999998e-05 2024-09-04T17:48:18.548 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.548 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.548 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:18.548 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898091+0000", 2024-09-04T17:48:18.548 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000197574 2024-09-04T17:48:18.548 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.549 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:18.549 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.549 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.549 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.549 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60929 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1212416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:18.549 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823042+0000", 2024-09-04T17:48:18.549 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 64.685435725999994, 2024-09-04T17:48:18.549 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075301251, 2024-09-04T17:48:18.549 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:18.549 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:18.549 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:18.549 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:18.549 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:18.550 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:18.550 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60929 2024-09-04T17:48:18.550 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.550 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:18.550 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.550 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:18.550 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823042+0000", 2024-09-04T17:48:18.550 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:18.550 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.550 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.550 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:18.550 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823042+0000", 2024-09-04T17:48:18.550 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:18.551 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.551 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.551 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:18.551 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823043+0000", 2024-09-04T17:48:18.551 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.1100000000000002e-07 2024-09-04T17:48:18.551 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.551 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.551 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:18.551 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823045+0000", 2024-09-04T17:48:18.551 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.933e-06 2024-09-04T17:48:18.551 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.551 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.551 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:18.552 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823051+0000", 2024-09-04T17:48:18.552 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.9569999999999999e-06 2024-09-04T17:48:18.552 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.552 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.552 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:18.552 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823055+0000", 2024-09-04T17:48:18.552 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4479999999999996e-06 2024-09-04T17:48:18.552 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.552 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.552 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:18.552 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898117+0000", 2024-09-04T17:48:18.552 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075062344000000003 2024-09-04T17:48:18.552 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.553 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.553 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:18.553 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898157+0000", 2024-09-04T17:48:18.553 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0095999999999997e-05 2024-09-04T17:48:18.553 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.553 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.553 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:18.553 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898344+0000", 2024-09-04T17:48:18.553 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000186162 2024-09-04T17:48:18.553 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.553 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:18.553 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.554 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.554 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.554 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60930 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1351680~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:18.554 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823061+0000", 2024-09-04T17:48:18.554 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 64.685416846999999, 2024-09-04T17:48:18.554 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075545507999999997, 2024-09-04T17:48:18.554 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:18.554 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:18.554 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:18.554 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:18.554 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:18.554 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:18.554 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60930 2024-09-04T17:48:18.555 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.555 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:18.555 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.555 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:18.555 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823061+0000", 2024-09-04T17:48:18.555 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:18.555 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.555 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.555 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:18.555 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823061+0000", 2024-09-04T17:48:18.555 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:18.555 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.555 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.556 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:18.556 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823378+0000", 2024-09-04T17:48:18.556 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00031631299999999998 2024-09-04T17:48:18.556 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.556 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.556 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:18.556 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823381+0000", 2024-09-04T17:48:18.556 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4020000000000002e-06 2024-09-04T17:48:18.556 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.556 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.556 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:18.556 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823389+0000", 2024-09-04T17:48:18.557 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.1429999999999998e-06 2024-09-04T17:48:18.557 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.557 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.557 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:18.557 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823395+0000", 2024-09-04T17:48:18.557 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.0059999999999998e-06 2024-09-04T17:48:18.557 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.557 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.557 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:18.557 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898368+0000", 2024-09-04T17:48:18.557 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074973032999999994 2024-09-04T17:48:18.557 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.557 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.557 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:18.558 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898410+0000", 2024-09-04T17:48:18.558 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2243000000000002e-05 2024-09-04T17:48:18.558 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.558 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.558 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:18.558 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898607+0000", 2024-09-04T17:48:18.558 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019636799999999999 2024-09-04T17:48:18.558 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.558 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:18.558 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.558 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.558 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.558 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60931 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1458176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:18.559 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823407+0000", 2024-09-04T17:48:18.559 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 64.685070773000007, 2024-09-04T17:48:18.559 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075460334000000004, 2024-09-04T17:48:18.559 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:18.559 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:18.559 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:18.559 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:18.559 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:18.559 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:18.559 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60931 2024-09-04T17:48:18.559 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.559 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:18.559 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.560 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:18.560 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823407+0000", 2024-09-04T17:48:18.560 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:18.560 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.560 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.560 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:18.560 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823407+0000", 2024-09-04T17:48:18.560 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:18.560 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.560 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.560 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:18.560 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823408+0000", 2024-09-04T17:48:18.560 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9999999999999998e-07 2024-09-04T17:48:18.561 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.561 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.561 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:18.561 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823415+0000", 2024-09-04T17:48:18.561 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.5510000000000002e-06 2024-09-04T17:48:18.561 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.561 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.561 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:18.561 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823423+0000", 2024-09-04T17:48:18.561 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2760000000000003e-06 2024-09-04T17:48:18.561 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.561 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.561 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:18.562 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823426+0000", 2024-09-04T17:48:18.562 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.2839999999999998e-06 2024-09-04T17:48:18.562 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.562 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.562 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:18.562 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898631+0000", 2024-09-04T17:48:18.562 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075205045999999998 2024-09-04T17:48:18.562 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.562 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.562 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:18.562 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898672+0000", 2024-09-04T17:48:18.562 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0614000000000001e-05 2024-09-04T17:48:18.562 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.563 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.563 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:18.563 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898868+0000", 2024-09-04T17:48:18.563 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019606300000000001 2024-09-04T17:48:18.563 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.563 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:18.563 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.563 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.563 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.563 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60932 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1564672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:18.563 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823434+0000", 2024-09-04T17:48:18.563 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 64.685044473999994, 2024-09-04T17:48:18.563 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075693393999999997, 2024-09-04T17:48:18.564 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:18.564 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:18.564 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:18.564 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:18.564 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:18.564 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:18.564 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60932 2024-09-04T17:48:18.564 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.564 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:18.564 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.564 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:18.564 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823434+0000", 2024-09-04T17:48:18.564 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:18.565 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.565 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.565 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:18.565 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823434+0000", 2024-09-04T17:48:18.565 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:18.565 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.565 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.565 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:18.565 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.830290+0000", 2024-09-04T17:48:18.565 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0068563039999999997 2024-09-04T17:48:18.565 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.565 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.565 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:18.566 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.830297+0000", 2024-09-04T17:48:18.566 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4760000000000001e-06 2024-09-04T17:48:18.566 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.566 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.566 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:18.566 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.830308+0000", 2024-09-04T17:48:18.566 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0927999999999999e-05 2024-09-04T17:48:18.566 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.566 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.566 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:18.566 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.830317+0000", 2024-09-04T17:48:18.566 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7280000000000001e-06 2024-09-04T17:48:18.566 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.566 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.567 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:18.567 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898892+0000", 2024-09-04T17:48:18.567 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.068574722000000005 2024-09-04T17:48:18.567 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.567 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.567 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:18.567 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898934+0000", 2024-09-04T17:48:18.567 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2249000000000002e-05 2024-09-04T17:48:18.567 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.567 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.567 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:18.567 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.899127+0000", 2024-09-04T17:48:18.567 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019298699999999999 2024-09-04T17:48:18.568 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.568 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:18.568 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.568 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.568 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.568 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61182 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 868352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:18.568 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839249+0000", 2024-09-04T17:48:18.568 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 64.669228582000002, 2024-09-04T17:48:18.568 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073680151999999999, 2024-09-04T17:48:18.568 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:18.568 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:18.568 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:18.568 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:18.569 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:18.569 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:18.569 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61182 2024-09-04T17:48:18.569 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.569 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:18.569 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.569 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:18.569 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839249+0000", 2024-09-04T17:48:18.569 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:18.569 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.569 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.569 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:18.569 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839249+0000", 2024-09-04T17:48:18.569 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:18.570 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.570 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.570 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:18.570 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839250+0000", 2024-09-04T17:48:18.570 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4999999999999998e-07 2024-09-04T17:48:18.570 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.570 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.570 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:18.570 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839257+0000", 2024-09-04T17:48:18.570 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7700000000000004e-06 2024-09-04T17:48:18.570 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.570 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.570 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:18.571 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839263+0000", 2024-09-04T17:48:18.571 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.246e-06 2024-09-04T17:48:18.571 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.571 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.571 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:18.571 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839268+0000", 2024-09-04T17:48:18.571 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.814e-06 2024-09-04T17:48:18.571 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.571 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.571 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:18.571 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.912831+0000", 2024-09-04T17:48:18.571 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073563294000000001 2024-09-04T17:48:18.571 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.572 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.572 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:18.572 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.912876+0000", 2024-09-04T17:48:18.572 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4742e-05 2024-09-04T17:48:18.572 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.572 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.572 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:18.572 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.912930+0000", 2024-09-04T17:48:18.572 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.3936e-05 2024-09-04T17:48:18.572 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.572 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:18.572 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.572 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.573 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.573 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61183 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:18.573 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839275+0000", 2024-09-04T17:48:18.573 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 64.669203389000003, 2024-09-04T17:48:18.573 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073781886000000005, 2024-09-04T17:48:18.573 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:18.573 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:18.573 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:18.573 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:18.573 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:18.573 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:18.573 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61183 2024-09-04T17:48:18.574 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.574 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:18.574 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.574 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:18.574 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839275+0000", 2024-09-04T17:48:18.574 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:18.574 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.574 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.574 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:18.574 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839275+0000", 2024-09-04T17:48:18.574 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:18.574 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.574 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.575 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:18.575 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839275+0000", 2024-09-04T17:48:18.575 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6899999999999997e-07 2024-09-04T17:48:18.575 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.575 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.575 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:18.575 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839277+0000", 2024-09-04T17:48:18.575 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9530000000000002e-06 2024-09-04T17:48:18.575 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.575 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.575 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:18.575 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839283+0000", 2024-09-04T17:48:18.575 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.9329999999999998e-06 2024-09-04T17:48:18.576 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.576 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.576 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:18.576 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839288+0000", 2024-09-04T17:48:18.576 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4100000000000001e-06 2024-09-04T17:48:18.576 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.576 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.576 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:18.576 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.912959+0000", 2024-09-04T17:48:18.576 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073671511999999995 2024-09-04T17:48:18.576 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.576 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.576 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:18.577 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913001+0000", 2024-09-04T17:48:18.577 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.1984999999999999e-05 2024-09-04T17:48:18.577 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.577 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.577 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:18.577 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913057+0000", 2024-09-04T17:48:18.577 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.5523999999999999e-05 2024-09-04T17:48:18.577 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.577 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:18.577 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.577 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.577 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.577 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61184 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1114112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:18.578 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839294+0000", 2024-09-04T17:48:18.578 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 64.669184177000005, 2024-09-04T17:48:18.578 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073995263000000006, 2024-09-04T17:48:18.578 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:18.578 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:18.578 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:18.578 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:18.578 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:18.578 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:18.578 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61184 2024-09-04T17:48:18.578 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.578 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:18.578 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.578 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:18.579 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839294+0000", 2024-09-04T17:48:18.579 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:18.579 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.579 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.579 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:18.579 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839294+0000", 2024-09-04T17:48:18.579 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:18.579 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.579 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.579 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:18.579 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839294+0000", 2024-09-04T17:48:18.579 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.77e-07 2024-09-04T17:48:18.579 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.579 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.580 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:18.580 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839296+0000", 2024-09-04T17:48:18.580 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9999999999999999e-06 2024-09-04T17:48:18.580 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.580 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.580 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:18.580 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839302+0000", 2024-09-04T17:48:18.580 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7230000000000001e-06 2024-09-04T17:48:18.580 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.580 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.580 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:18.580 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839314+0000", 2024-09-04T17:48:18.580 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1826e-05 2024-09-04T17:48:18.581 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.581 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.581 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:18.581 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913084+0000", 2024-09-04T17:48:18.581 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073770215 2024-09-04T17:48:18.581 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.581 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.581 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:18.581 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913127+0000", 2024-09-04T17:48:18.581 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2982999999999998e-05 2024-09-04T17:48:18.581 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.581 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.581 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:18.582 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913289+0000", 2024-09-04T17:48:18.582 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000162139 2024-09-04T17:48:18.582 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.582 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:18.582 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.582 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.582 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.582 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61185 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1220608~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:18.582 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839321+0000", 2024-09-04T17:48:18.582 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 64.669156709999996, 2024-09-04T17:48:18.582 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074206062000000003, 2024-09-04T17:48:18.582 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:18.582 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:18.583 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:18.583 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:18.583 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:18.583 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:18.583 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61185 2024-09-04T17:48:18.583 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.583 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:18.583 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.583 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:18.583 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839321+0000", 2024-09-04T17:48:18.583 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:18.583 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.583 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.583 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:18.584 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839321+0000", 2024-09-04T17:48:18.584 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:18.584 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.584 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.584 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:18.584 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839322+0000", 2024-09-04T17:48:18.584 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.5400000000000001e-07 2024-09-04T17:48:18.584 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.584 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.584 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:18.584 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839324+0000", 2024-09-04T17:48:18.584 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2539999999999999e-06 2024-09-04T17:48:18.584 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.585 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.585 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:18.585 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839331+0000", 2024-09-04T17:48:18.585 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2139999999999999e-06 2024-09-04T17:48:18.585 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.585 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.585 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:18.585 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839336+0000", 2024-09-04T17:48:18.585 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7029999999999997e-06 2024-09-04T17:48:18.585 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.585 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.585 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:18.585 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913320+0000", 2024-09-04T17:48:18.586 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073984016 2024-09-04T17:48:18.586 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.586 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.586 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:18.586 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913366+0000", 2024-09-04T17:48:18.586 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.5704000000000002e-05 2024-09-04T17:48:18.586 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.586 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.586 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:18.586 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913527+0000", 2024-09-04T17:48:18.586 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00016161699999999999 2024-09-04T17:48:18.586 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.586 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:18.587 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.587 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.587 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.587 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61186 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1359872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:18.587 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839344+0000", 2024-09-04T17:48:18.587 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 64.669134368000002, 2024-09-04T17:48:18.587 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074450673999999994, 2024-09-04T17:48:18.587 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:18.587 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:18.587 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:18.587 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:18.587 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:18.587 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:18.587 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61186 2024-09-04T17:48:18.588 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.588 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:18.588 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.588 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:18.588 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839344+0000", 2024-09-04T17:48:18.588 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:18.588 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.588 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.588 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:18.588 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839344+0000", 2024-09-04T17:48:18.588 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:18.588 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.588 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.589 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:18.589 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839344+0000", 2024-09-04T17:48:18.589 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4799999999999999e-07 2024-09-04T17:48:18.589 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.589 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.589 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:18.589 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839346+0000", 2024-09-04T17:48:18.589 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0889999999999998e-06 2024-09-04T17:48:18.589 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.589 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.589 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:18.589 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839352+0000", 2024-09-04T17:48:18.589 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6389999999999997e-06 2024-09-04T17:48:18.589 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.590 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.590 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:18.590 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839356+0000", 2024-09-04T17:48:18.590 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.233e-06 2024-09-04T17:48:18.590 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.590 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.590 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:18.590 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913557+0000", 2024-09-04T17:48:18.590 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074200944000000005 2024-09-04T17:48:18.590 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.590 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.590 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:18.590 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913601+0000", 2024-09-04T17:48:18.591 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4350000000000001e-05 2024-09-04T17:48:18.591 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.591 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.591 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:18.591 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913794+0000", 2024-09-04T17:48:18.591 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019297100000000001 2024-09-04T17:48:18.591 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.591 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:18.591 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.591 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.591 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.591 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61187 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1466368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:18.591 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839364+0000", 2024-09-04T17:48:18.591 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 64.669114519000004, 2024-09-04T17:48:18.592 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074692334999999999, 2024-09-04T17:48:18.592 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:18.592 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:18.592 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:18.592 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:18.592 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:18.592 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:18.592 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61187 2024-09-04T17:48:18.592 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.592 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:18.592 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.592 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:18.592 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839364+0000", 2024-09-04T17:48:18.593 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:18.593 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.593 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.593 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:18.593 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839364+0000", 2024-09-04T17:48:18.593 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:18.593 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.593 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.593 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:18.593 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839364+0000", 2024-09-04T17:48:18.593 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.75e-07 2024-09-04T17:48:18.593 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.593 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.593 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:18.594 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839366+0000", 2024-09-04T17:48:18.594 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0990000000000001e-06 2024-09-04T17:48:18.594 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.594 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.594 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:18.594 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839372+0000", 2024-09-04T17:48:18.594 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.2609999999999999e-06 2024-09-04T17:48:18.594 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.594 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.594 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:18.594 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839389+0000", 2024-09-04T17:48:18.594 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6489000000000001e-05 2024-09-04T17:48:18.594 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.595 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.595 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:18.595 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913824+0000", 2024-09-04T17:48:18.595 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074434811000000004 2024-09-04T17:48:18.595 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.595 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.595 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:18.595 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913868+0000", 2024-09-04T17:48:18.595 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4768000000000002e-05 2024-09-04T17:48:18.595 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.595 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.595 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:18.595 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914056+0000", 2024-09-04T17:48:18.596 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00018743199999999999 2024-09-04T17:48:18.596 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.596 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:18.596 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.596 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.596 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.596 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61188 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1605632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:18.596 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839399+0000", 2024-09-04T17:48:18.596 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 64.669078623000004, 2024-09-04T17:48:18.596 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074975095000000005, 2024-09-04T17:48:18.596 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:18.596 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:18.596 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:18.597 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:18.597 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:18.597 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:18.597 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61188 2024-09-04T17:48:18.597 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.597 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:18.597 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.597 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:18.597 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839399+0000", 2024-09-04T17:48:18.597 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:18.597 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.597 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.597 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:18.597 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839399+0000", 2024-09-04T17:48:18.598 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:18.598 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.598 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.598 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:18.598 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839400+0000", 2024-09-04T17:48:18.598 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0100000000000005e-07 2024-09-04T17:48:18.598 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.598 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.598 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:18.598 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839402+0000", 2024-09-04T17:48:18.598 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.491e-06 2024-09-04T17:48:18.598 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.598 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.598 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:18.599 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839409+0000", 2024-09-04T17:48:18.599 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.0450000000000003e-06 2024-09-04T17:48:18.599 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.599 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.599 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:18.599 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839414+0000", 2024-09-04T17:48:18.599 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.87e-06 2024-09-04T17:48:18.599 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.599 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.599 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:18.599 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914086+0000", 2024-09-04T17:48:18.599 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074671596000000007 2024-09-04T17:48:18.599 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.600 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.600 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:18.600 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914130+0000", 2024-09-04T17:48:18.600 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4579999999999997e-05 2024-09-04T17:48:18.600 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.600 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.600 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:18.600 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914375+0000", 2024-09-04T17:48:18.600 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024401199999999999 2024-09-04T17:48:18.600 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.600 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:18.600 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.600 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.601 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.601 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61189 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1712128~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:18.601 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839424+0000", 2024-09-04T17:48:18.601 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 64.669054422000002, 2024-09-04T17:48:18.601 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075247680999999997, 2024-09-04T17:48:18.601 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:18.601 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:18.601 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:18.601 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:18.601 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:18.601 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:18.601 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61189 2024-09-04T17:48:18.601 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.601 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:18.602 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.602 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:18.602 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839424+0000", 2024-09-04T17:48:18.602 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:18.602 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.602 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.602 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:18.602 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839424+0000", 2024-09-04T17:48:18.602 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:18.602 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.602 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.602 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:18.602 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.846320+0000", 2024-09-04T17:48:18.604 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0068965210000000001 2024-09-04T17:48:18.604 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.604 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.604 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:18.604 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.846328+0000", 2024-09-04T17:48:18.604 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.8189999999999992e-06 2024-09-04T17:48:18.604 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.605 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.605 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:18.605 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.846341+0000", 2024-09-04T17:48:18.605 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3338e-05 2024-09-04T17:48:18.605 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.605 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.605 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:18.605 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.846352+0000", 2024-09-04T17:48:18.605 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0479000000000001e-05 2024-09-04T17:48:18.605 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.605 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.605 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:18.605 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914405+0000", 2024-09-04T17:48:18.606 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.068053049000000004 2024-09-04T17:48:18.606 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.606 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.606 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:18.606 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914451+0000", 2024-09-04T17:48:18.606 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6545999999999998e-05 2024-09-04T17:48:18.606 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:18.606 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:18.606 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:18.606 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914671+0000", 2024-09-04T17:48:18.606 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000219929 2024-09-04T17:48:18.606 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.606 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:18.606 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.607 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:18.607 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:18.607 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:18.608 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-04T17:48:18.609 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:48:18.603+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:48:18.709 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:48:18.707+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:48:18.741 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:18.742 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:48:18.742 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:48:18.742 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:18.756 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-04T17:48:18.836 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-04T17:48:18.841 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:18.839+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:18.912 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:18.907+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:18.923 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:18.923 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:48:18.923 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:48:18.923 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:48:18.923 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:18.938 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-04T17:48:18.988 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:48:18.988 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:48:18.992 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-04T17:48:19.009 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:48:19.009 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:48:19.013 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:48:19.007+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:48:19.024 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-04T17:48:19.024 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-04T17:48:19.041 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-04T17:48:19.041 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-04T17:48:19.064 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:48:19.065 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:48:19.086 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:48:19.086 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:48:19.103 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:19.103 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:19.107 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-04T17:48:19.108 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-04T17:48:19.114 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:48:19.110+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:48:19.119 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-04T17:48:19.132 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:48:19.132 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:48:19.217 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:48:19.214+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:48:19.253 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:19.253 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:19.268 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-04T17:48:19.318 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:19.318+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:19.377 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:19.377 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:19.389 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-04T17:48:19.415 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-04T17:48:19.421 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:19.418+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:19.504 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:19.504 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:19.518 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-04T17:48:19.565 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:19.562+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:19.622 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:19.618+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:19.649 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:19.649 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:19.664 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-04T17:48:19.719 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:48:19.719 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-09-04T17:48:19.736 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":212,"stamp":"2024-09-04T17:48:19.112015+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1448466522,"num_objects":2018,"num_object_clones":254,"num_object_copies":4036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2018,"num_whiteouts":1125,"num_read":132695,"num_read_kb":868561,"num_write":18537,"num_write_kb":405440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3602,"num_bytes_recovered":213911072,"num_keys_recovered":676,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15775,"ondisk_log_size":15775,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":81,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":702156,"kb_used_data":436804,"kb_used_omap":40,"kb_used_meta":265111,"kb_avail":565528884,"statfs":{"total":579820584960,"available":579101577216,"internally_reserved":0,"allocated":447287296,"data_stored":545955838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41921,"internal_metadata":271473727},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-99851237,"num_objects":-90,"num_object_clones":-26,"num_object_copies":-180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-118,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-90,"num_whiteouts":-15,"num_read":733,"num_read_kb":630,"num_write":149,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":425,"num_bytes_recovered":17718811,"num_keys_recovered":70,"num_objects_omap":-3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000959"},"pg_stats":[{"pgid":"3.1f","version":"136'462","reported_seq":2789,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-04T17:48:16.121049+0000","last_change":"2024-09-04T17:48:16.020238+0000","last_active":"2024-09-04T17:48:16.121049+0000","last_peered":"2024-09-04T17:48:16.121049+0000","last_clean":"2024-09-04T17:48:16.121049+0000","last_became_active":"2024-09-04T17:48:09.845199+0000","last_became_peered":"2024-09-04T17:48:09.845199+0000","last_unstale":"2024-09-04T17:48:16.121049+0000","last_undegraded":"2024-09-04T17:48:16.121049+0000","last_fullsized":"2024-09-04T17:48:16.121049+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":462,"log_dups_size":0,"ondisk_log_size":462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:15:45.038498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001802622,"stat_sum":{"num_bytes":26673788,"num_objects":63,"num_object_clones":7,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":42,"num_read":2947,"num_read_kb":27164,"num_write":498,"num_write_kb":10441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":24433786,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"26","length":"8"}]},{"pgid":"3.1e","version":"123'580","reported_seq":3344,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-04T17:48:15.963835+0000","last_change":"2024-09-04T17:48:15.964539+0000","last_active":"2024-09-04T17:48:15.963835+0000","last_peered":"2024-09-04T17:48:15.963835+0000","last_clean":"2024-09-04T17:48:15.963835+0000","last_became_active":"2024-09-04T17:48:00.841031+0000","last_became_peered":"2024-09-04T17:48:00.841031+0000","last_unstale":"2024-09-04T17:48:15.963835+0000","last_undegraded":"2024-09-04T17:48:15.963835+0000","last_fullsized":"2024-09-04T17:48:15.963835+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T22:13:01.312024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001939425,"stat_sum":{"num_bytes":56202357,"num_objects":54,"num_object_clones":6,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":26,"num_read":4487,"num_read_kb":30277,"num_write":751,"num_write_kb":10557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":10538943,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"26","length":"8"}]},{"pgid":"3.1d","version":"136'514","reported_seq":2219,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-04T17:48:16.135837+0000","last_change":"2024-09-04T17:48:15.975790+0000","last_active":"2024-09-04T17:48:16.135837+0000","last_peered":"2024-09-04T17:48:16.135837+0000","last_clean":"2024-09-04T17:48:16.135837+0000","last_became_active":"2024-09-04T17:45:55.955213+0000","last_became_peered":"2024-09-04T17:45:55.955213+0000","last_unstale":"2024-09-04T17:48:16.135837+0000","last_undegraded":"2024-09-04T17:48:16.135837+0000","last_fullsized":"2024-09-04T17:48:16.135837+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":514,"log_dups_size":0,"ondisk_log_size":514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T21:52:39.679302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030347830000000001,"stat_sum":{"num_bytes":40803461,"num_objects":60,"num_object_clones":6,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":32,"num_read":2816,"num_read_kb":20282,"num_write":601,"num_write_kb":9401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"26","length":"8"}]},{"pgid":"3.1c","version":"136'673","reported_seq":6280,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-04T17:48:16.081181+0000","last_change":"2024-09-04T17:48:15.976656+0000","last_active":"2024-09-04T17:48:16.081181+0000","last_peered":"2024-09-04T17:48:16.081181+0000","last_clean":"2024-09-04T17:48:16.081181+0000","last_became_active":"2024-09-04T17:45:55.951880+0000","last_became_peered":"2024-09-04T17:45:55.951880+0000","last_unstale":"2024-09-04T17:48:16.081181+0000","last_undegraded":"2024-09-04T17:48:16.081181+0000","last_fullsized":"2024-09-04T17:48:16.081181+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":673,"log_dups_size":0,"ondisk_log_size":673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T18:56:58.164155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036736709999999999,"stat_sum":{"num_bytes":75245074,"num_objects":73,"num_object_clones":9,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":42,"num_read":7207,"num_read_kb":47766,"num_write":770,"num_write_kb":11551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"26","length":"8"}]},{"pgid":"3.1b","version":"134'491","reported_seq":1748,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-04T17:48:16.020392+0000","last_change":"2024-09-04T17:48:16.020658+0000","last_active":"2024-09-04T17:48:16.020392+0000","last_peered":"2024-09-04T17:48:16.020392+0000","last_clean":"2024-09-04T17:48:16.020392+0000","last_became_active":"2024-09-04T17:48:09.853793+0000","last_became_peered":"2024-09-04T17:48:09.853793+0000","last_unstale":"2024-09-04T17:48:16.020392+0000","last_undegraded":"2024-09-04T17:48:16.020392+0000","last_fullsized":"2024-09-04T17:48:16.020392+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":491,"log_dups_size":0,"ondisk_log_size":491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T19:14:49.816158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015887220000000001,"stat_sum":{"num_bytes":18210524,"num_objects":65,"num_object_clones":6,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":39,"num_read":3057,"num_read_kb":7588,"num_write":566,"num_write_kb":4577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":1993914,"num_keys_recovered":48,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"26","length":"8"}]},{"pgid":"3.1a","version":"130'362","reported_seq":5771,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-04T17:48:14.936230+0000","last_change":"2024-09-04T17:48:14.936726+0000","last_active":"2024-09-04T17:48:14.936230+0000","last_peered":"2024-09-04T17:48:14.936230+0000","last_clean":"2024-09-04T17:48:14.936230+0000","last_became_active":"2024-09-04T17:47:44.946366+0000","last_became_peered":"2024-09-04T17:47:44.946366+0000","last_unstale":"2024-09-04T17:48:14.936230+0000","last_undegraded":"2024-09-04T17:48:14.936230+0000","last_fullsized":"2024-09-04T17:48:14.936230+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":362,"log_dups_size":0,"ondisk_log_size":362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T18:31:13.955149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080444200000000005,"stat_sum":{"num_bytes":72634271,"num_objects":65,"num_object_clones":8,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":31,"num_read":5550,"num_read_kb":45438,"num_write":335,"num_write_kb":15441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":5992020,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"3.19","version":"135'362","reported_seq":2152,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-04T17:48:15.963048+0000","last_change":"2024-09-04T17:48:15.963675+0000","last_active":"2024-09-04T17:48:15.963048+0000","last_peered":"2024-09-04T17:48:15.963048+0000","last_clean":"2024-09-04T17:48:15.963048+0000","last_became_active":"2024-09-04T17:48:10.851145+0000","last_became_peered":"2024-09-04T17:48:10.851145+0000","last_unstale":"2024-09-04T17:48:15.963048+0000","last_undegraded":"2024-09-04T17:48:15.963048+0000","last_fullsized":"2024-09-04T17:48:15.963048+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":362,"log_dups_size":0,"ondisk_log_size":362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:10:23.636481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012820279999999999,"stat_sum":{"num_bytes":35470241,"num_objects":53,"num_object_clones":7,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":24,"num_read":2015,"num_read_kb":16004,"num_write":396,"num_write_kb":4601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":4015757,"num_keys_recovered":27,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"26","length":"8"}]},{"pgid":"3.18","version":"135'409","reported_seq":1728,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-04T17:48:16.018774+0000","last_change":"2024-09-04T17:48:16.019088+0000","last_active":"2024-09-04T17:48:16.018774+0000","last_peered":"2024-09-04T17:48:16.018774+0000","last_clean":"2024-09-04T17:48:16.018774+0000","last_became_active":"2024-09-04T17:48:09.838648+0000","last_became_peered":"2024-09-04T17:48:09.838648+0000","last_unstale":"2024-09-04T17:48:16.018774+0000","last_undegraded":"2024-09-04T17:48:16.018774+0000","last_fullsized":"2024-09-04T17:48:16.018774+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":409,"log_dups_size":0,"ondisk_log_size":409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:39:18.649191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088566000000000003,"stat_sum":{"num_bytes":70174537,"num_objects":60,"num_object_clones":6,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":32,"num_read":8379,"num_read_kb":35044,"num_write":373,"num_write_kb":14411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":288,"num_bytes_recovered":25655550,"num_keys_recovered":117,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"26","length":"8"}]},{"pgid":"3.17","version":"135'291","reported_seq":2048,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-04T17:48:16.020756+0000","last_change":"2024-09-04T17:48:16.021039+0000","last_active":"2024-09-04T17:48:16.020756+0000","last_peered":"2024-09-04T17:48:16.020756+0000","last_clean":"2024-09-04T17:48:16.020756+0000","last_became_active":"2024-09-04T17:45:55.958625+0000","last_became_peered":"2024-09-04T17:45:55.958625+0000","last_unstale":"2024-09-04T17:48:16.020756+0000","last_undegraded":"2024-09-04T17:48:16.020756+0000","last_fullsized":"2024-09-04T17:48:16.020756+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":291,"log_dups_size":0,"ondisk_log_size":291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:27:31.490118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023674439999999998,"stat_sum":{"num_bytes":77515034,"num_objects":56,"num_object_clones":9,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":25,"num_read":1606,"num_read_kb":20936,"num_write":294,"num_write_kb":18406,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"26","length":"8"}]},{"pgid":"3.16","version":"134'327","reported_seq":1710,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-04T17:48:15.975793+0000","last_change":"2024-09-04T17:48:15.976444+0000","last_active":"2024-09-04T17:48:15.975793+0000","last_peered":"2024-09-04T17:48:15.975793+0000","last_clean":"2024-09-04T17:48:15.975793+0000","last_became_active":"2024-09-04T17:45:55.953254+0000","last_became_peered":"2024-09-04T17:45:55.953254+0000","last_unstale":"2024-09-04T17:48:15.975793+0000","last_undegraded":"2024-09-04T17:48:15.975793+0000","last_fullsized":"2024-09-04T17:48:15.975793+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:59:13.865184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034833820000000001,"stat_sum":{"num_bytes":35624842,"num_objects":49,"num_object_clones":6,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":29,"num_read":1692,"num_read_kb":10758,"num_write":370,"num_write_kb":7387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"26","length":"8"}]},{"pgid":"3.15","version":"125'455","reported_seq":2022,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-04T17:48:14.955297+0000","last_change":"2024-09-04T17:48:14.955297+0000","last_active":"2024-09-04T17:48:14.955297+0000","last_peered":"2024-09-04T17:48:14.955297+0000","last_clean":"2024-09-04T17:48:14.955297+0000","last_became_active":"2024-09-04T17:48:14.948212+0000","last_became_peered":"2024-09-04T17:48:14.948212+0000","last_unstale":"2024-09-04T17:48:14.955297+0000","last_undegraded":"2024-09-04T17:48:14.955297+0000","last_fullsized":"2024-09-04T17:48:14.955297+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:59:03.009864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002853806,"stat_sum":{"num_bytes":17977259,"num_objects":46,"num_object_clones":4,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":27,"num_read":3707,"num_read_kb":16708,"num_write":482,"num_write_kb":4429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":176,"num_bytes_recovered":10208507,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"3.14","version":"134'694","reported_seq":2072,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-04T17:48:14.944280+0000","last_change":"2024-09-04T17:48:14.945013+0000","last_active":"2024-09-04T17:48:14.944280+0000","last_peered":"2024-09-04T17:48:14.944280+0000","last_clean":"2024-09-04T17:48:14.944280+0000","last_became_active":"2024-09-04T17:46:53.539886+0000","last_became_peered":"2024-09-04T17:46:53.539886+0000","last_unstale":"2024-09-04T17:48:14.944280+0000","last_undegraded":"2024-09-04T17:48:14.944280+0000","last_fullsized":"2024-09-04T17:48:14.944280+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":694,"log_dups_size":0,"ondisk_log_size":694,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:04:08.592215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025017669999999998,"stat_sum":{"num_bytes":52880134,"num_objects":64,"num_object_clones":9,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":35,"num_read":3772,"num_read_kb":29665,"num_write":940,"num_write_kb":35894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"2.7","version":"0'0","reported_seq":112,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-04T17:48:15.962057+0000","last_change":"2024-09-04T17:48:11.850550+0000","last_active":"2024-09-04T17:48:15.962057+0000","last_peered":"2024-09-04T17:48:15.962057+0000","last_clean":"2024-09-04T17:48:15.962057+0000","last_became_active":"2024-09-04T17:48:11.849710+0000","last_became_peered":"2024-09-04T17:48:11.849710+0000","last_unstale":"2024-09-04T17:48:15.962057+0000","last_undegraded":"2024-09-04T17:48:15.962057+0000","last_fullsized":"2024-09-04T17:48:15.962057+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:37:18.791459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001226074,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"136'326","reported_seq":2823,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-04T17:48:16.120067+0000","last_change":"2024-09-04T17:48:16.020616+0000","last_active":"2024-09-04T17:48:16.120067+0000","last_peered":"2024-09-04T17:48:16.120067+0000","last_clean":"2024-09-04T17:48:16.120067+0000","last_became_active":"2024-09-04T17:47:32.848310+0000","last_became_peered":"2024-09-04T17:47:32.848310+0000","last_unstale":"2024-09-04T17:48:16.120067+0000","last_undegraded":"2024-09-04T17:48:16.120067+0000","last_fullsized":"2024-09-04T17:48:16.120067+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T23:08:49.179745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018670360000000001,"stat_sum":{"num_bytes":36538290,"num_objects":52,"num_object_clones":4,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":28,"num_read":2418,"num_read_kb":14504,"num_write":370,"num_write_kb":5524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":919873,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"26","length":"8"}]},{"pgid":"2.6","version":"0'0","reported_seq":47,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-04T17:48:15.961940+0000","last_change":"2024-09-04T17:47:58.862844+0000","last_active":"2024-09-04T17:48:15.961940+0000","last_peered":"2024-09-04T17:48:15.961940+0000","last_clean":"2024-09-04T17:48:15.961940+0000","last_became_active":"2024-09-04T17:47:58.859941+0000","last_became_peered":"2024-09-04T17:47:58.859941+0000","last_unstale":"2024-09-04T17:48:15.961940+0000","last_undegraded":"2024-09-04T17:48:15.961940+0000","last_fullsized":"2024-09-04T17:48:15.961940+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:08:53.409036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011364179999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"134'414","reported_seq":2661,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-04T17:48:14.946390+0000","last_change":"2024-09-04T17:48:14.947003+0000","last_active":"2024-09-04T17:48:14.946390+0000","last_peered":"2024-09-04T17:48:14.946390+0000","last_clean":"2024-09-04T17:48:14.946390+0000","last_became_active":"2024-09-04T17:47:08.653159+0000","last_became_peered":"2024-09-04T17:47:08.653159+0000","last_unstale":"2024-09-04T17:48:14.946390+0000","last_undegraded":"2024-09-04T17:48:14.946390+0000","last_fullsized":"2024-09-04T17:48:14.946390+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":414,"log_dups_size":0,"ondisk_log_size":414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:50:34.140487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030432329999999998,"stat_sum":{"num_bytes":43272039,"num_objects":61,"num_object_clones":7,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":35,"num_read":2882,"num_read_kb":23337,"num_write":531,"num_write_kb":21560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":952382,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"2.5","version":"0'0","reported_seq":246,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-04T17:48:14.945008+0000","last_change":"2024-09-04T17:45:49.476607+0000","last_active":"2024-09-04T17:48:14.945008+0000","last_peered":"2024-09-04T17:48:14.945008+0000","last_clean":"2024-09-04T17:48:14.945008+0000","last_became_active":"2024-09-04T17:45:47.369308+0000","last_became_peered":"2024-09-04T17:45:47.369308+0000","last_unstale":"2024-09-04T17:48:14.945008+0000","last_undegraded":"2024-09-04T17:48:14.945008+0000","last_fullsized":"2024-09-04T17:48:14.945008+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:17:13.378553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001029826,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"134'359","reported_seq":2432,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-04T17:48:14.936948+0000","last_change":"2024-09-04T17:48:14.937297+0000","last_active":"2024-09-04T17:48:14.936948+0000","last_peered":"2024-09-04T17:48:14.936948+0000","last_clean":"2024-09-04T17:48:14.936948+0000","last_became_active":"2024-09-04T17:48:10.845756+0000","last_became_peered":"2024-09-04T17:48:10.845756+0000","last_unstale":"2024-09-04T17:48:14.936948+0000","last_undegraded":"2024-09-04T17:48:14.936948+0000","last_fullsized":"2024-09-04T17:48:14.936948+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:48:20.910158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079943099999999997,"stat_sum":{"num_bytes":33917693,"num_objects":59,"num_object_clones":3,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":33,"num_read":2424,"num_read_kb":18572,"num_write":397,"num_write_kb":9850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":237,"num_bytes_recovered":13375622,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"2.4","version":"0'0","reported_seq":92,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-04T17:48:16.018295+0000","last_change":"2024-09-04T17:47:29.803682+0000","last_active":"2024-09-04T17:48:16.018295+0000","last_peered":"2024-09-04T17:48:16.018295+0000","last_clean":"2024-09-04T17:48:16.018295+0000","last_became_active":"2024-09-04T17:47:29.803256+0000","last_became_peered":"2024-09-04T17:47:29.803256+0000","last_unstale":"2024-09-04T17:48:16.018295+0000","last_undegraded":"2024-09-04T17:48:16.018295+0000","last_fullsized":"2024-09-04T17:48:16.018295+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:32:26.523162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"136'278","reported_seq":1835,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-04T17:48:16.127609+0000","last_change":"2024-09-04T17:48:15.975570+0000","last_active":"2024-09-04T17:48:16.127609+0000","last_peered":"2024-09-04T17:48:16.127609+0000","last_clean":"2024-09-04T17:48:16.127609+0000","last_became_active":"2024-09-04T17:47:41.918947+0000","last_became_peered":"2024-09-04T17:47:41.918947+0000","last_unstale":"2024-09-04T17:48:16.127609+0000","last_undegraded":"2024-09-04T17:48:16.127609+0000","last_fullsized":"2024-09-04T17:48:16.127609+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":278,"log_dups_size":0,"ondisk_log_size":278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T19:02:04.901135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002633678,"stat_sum":{"num_bytes":23152581,"num_objects":63,"num_object_clones":10,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":36,"num_read":2058,"num_read_kb":7216,"num_write":256,"num_write_kb":6999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":10024551,"num_keys_recovered":46,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"26","length":"8"}]},{"pgid":"3.3","version":"136'679","reported_seq":2905,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-04T17:48:16.088887+0000","last_change":"2024-09-04T17:48:16.020193+0000","last_active":"2024-09-04T17:48:16.088887+0000","last_peered":"2024-09-04T17:48:16.088887+0000","last_clean":"2024-09-04T17:48:16.088887+0000","last_became_active":"2024-09-04T17:46:53.542889+0000","last_became_peered":"2024-09-04T17:46:53.542889+0000","last_unstale":"2024-09-04T17:48:16.088887+0000","last_undegraded":"2024-09-04T17:48:16.088887+0000","last_fullsized":"2024-09-04T17:48:16.088887+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":679,"log_dups_size":0,"ondisk_log_size":679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T18:12:40.654690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019551379999999999,"stat_sum":{"num_bytes":59866405,"num_objects":63,"num_object_clones":9,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":36,"num_read":4364,"num_read_kb":26276,"num_write":904,"num_write_kb":14102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"26","length":"8"}]},{"pgid":"2.2","version":"22'2","reported_seq":265,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-04T17:48:15.972364+0000","last_change":"2024-09-04T17:47:59.598241+0000","last_active":"2024-09-04T17:48:15.972364+0000","last_peered":"2024-09-04T17:48:15.972364+0000","last_clean":"2024-09-04T17:48:15.972364+0000","last_became_active":"2024-09-04T17:47:58.884634+0000","last_became_peered":"2024-09-04T17:47:58.884634+0000","last_unstale":"2024-09-04T17:48:15.972364+0000","last_undegraded":"2024-09-04T17:48:15.972364+0000","last_fullsized":"2024-09-04T17:48:15.972364+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T22:02:31.692385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026316069999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"134'376","reported_seq":2890,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-04T17:48:14.936100+0000","last_change":"2024-09-04T17:48:14.936506+0000","last_active":"2024-09-04T17:48:14.936100+0000","last_peered":"2024-09-04T17:48:14.936100+0000","last_clean":"2024-09-04T17:48:14.936100+0000","last_became_active":"2024-09-04T17:48:01.849363+0000","last_became_peered":"2024-09-04T17:48:01.849363+0000","last_unstale":"2024-09-04T17:48:14.936100+0000","last_undegraded":"2024-09-04T17:48:14.936100+0000","last_fullsized":"2024-09-04T17:48:14.936100+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:28:48.800572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072198599999999998,"stat_sum":{"num_bytes":41098591,"num_objects":69,"num_object_clones":8,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":42,"num_read":3147,"num_read_kb":22161,"num_write":314,"num_write_kb":4881,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":5692590,"num_keys_recovered":35,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"2.1","version":"0'0","reported_seq":226,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-04T17:48:14.935690+0000","last_change":"2024-09-04T17:47:41.913896+0000","last_active":"2024-09-04T17:48:14.935690+0000","last_peered":"2024-09-04T17:48:14.935690+0000","last_clean":"2024-09-04T17:48:14.935690+0000","last_became_active":"2024-09-04T17:47:41.913057+0000","last_became_peered":"2024-09-04T17:47:41.913057+0000","last_unstale":"2024-09-04T17:48:14.935690+0000","last_undegraded":"2024-09-04T17:48:14.935690+0000","last_fullsized":"2024-09-04T17:48:14.935690+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:49:25.382532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001025195,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"136'533","reported_seq":2852,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-04T17:48:16.018959+0000","last_change":"2024-09-04T17:48:16.018959+0000","last_active":"2024-09-04T17:48:16.018959+0000","last_peered":"2024-09-04T17:48:16.018959+0000","last_clean":"2024-09-04T17:48:16.018959+0000","last_became_active":"2024-09-04T17:48:11.845510+0000","last_became_peered":"2024-09-04T17:48:11.845510+0000","last_unstale":"2024-09-04T17:48:16.018959+0000","last_undegraded":"2024-09-04T17:48:16.018959+0000","last_fullsized":"2024-09-04T17:48:16.018959+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T19:46:06.093288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000821681,"stat_sum":{"num_bytes":84206946,"num_objects":76,"num_object_clones":17,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":41,"num_read":2920,"num_read_kb":40212,"num_write":606,"num_write_kb":23345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":240,"num_bytes_recovered":21297737,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"26","length":"8"}]},{"pgid":"2.0","version":"0'0","reported_seq":252,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-04T17:48:14.942504+0000","last_change":"2024-09-04T17:47:29.804429+0000","last_active":"2024-09-04T17:48:14.942504+0000","last_peered":"2024-09-04T17:48:14.942504+0000","last_clean":"2024-09-04T17:48:14.942504+0000","last_became_active":"2024-09-04T17:47:29.803992+0000","last_became_peered":"2024-09-04T17:47:29.803992+0000","last_unstale":"2024-09-04T17:48:14.942504+0000","last_undegraded":"2024-09-04T17:48:14.942504+0000","last_fullsized":"2024-09-04T17:48:14.942504+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:50:32.563345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010553590000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"135'557","reported_seq":2382,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-04T17:48:15.973315+0000","last_change":"2024-09-04T17:48:15.973819+0000","last_active":"2024-09-04T17:48:15.973315+0000","last_peered":"2024-09-04T17:48:15.973315+0000","last_clean":"2024-09-04T17:48:15.973315+0000","last_became_active":"2024-09-04T17:48:01.850400+0000","last_became_peered":"2024-09-04T17:48:01.850400+0000","last_unstale":"2024-09-04T17:48:15.973315+0000","last_undegraded":"2024-09-04T17:48:15.973315+0000","last_fullsized":"2024-09-04T17:48:15.973315+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":557,"log_dups_size":0,"ondisk_log_size":557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:34:21.219359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001195214,"stat_sum":{"num_bytes":46184917,"num_objects":58,"num_object_clones":7,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":33,"num_read":5401,"num_read_kb":29898,"num_write":693,"num_write_kb":13807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":13799490,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"26","length":"8"}]},{"pgid":"2.3","version":"20'1","reported_seq":259,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-04T17:48:15.972343+0000","last_change":"2024-09-04T17:47:41.919544+0000","last_active":"2024-09-04T17:48:15.972343+0000","last_peered":"2024-09-04T17:48:15.972343+0000","last_clean":"2024-09-04T17:48:15.972343+0000","last_became_active":"2024-09-04T17:47:41.919027+0000","last_became_peered":"2024-09-04T17:47:41.919027+0000","last_unstale":"2024-09-04T17:48:15.972343+0000","last_undegraded":"2024-09-04T17:48:15.972343+0000","last_fullsized":"2024-09-04T17:48:15.972343+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:42:55.688562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086618299999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":337,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-04T17:48:14.942810+0000","last_change":"2024-09-04T17:45:42.341149+0000","last_active":"2024-09-04T17:48:14.942810+0000","last_peered":"2024-09-04T17:48:14.942810+0000","last_clean":"2024-09-04T17:48:14.942810+0000","last_became_active":"2024-09-04T17:45:42.339286+0000","last_became_peered":"2024-09-04T17:45:42.339286+0000","last_unstale":"2024-09-04T17:48:14.942810+0000","last_undegraded":"2024-09-04T17:48:14.942810+0000","last_fullsized":"2024-09-04T17:48:14.942810+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:41.305128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:41.305128+0000","last_clean_scrub_stamp":"2024-09-04T17:45:41.305128+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:52:24.860049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"134'485","reported_seq":2690,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-04T17:48:14.943390+0000","last_change":"2024-09-04T17:48:14.944023+0000","last_active":"2024-09-04T17:48:14.943390+0000","last_peered":"2024-09-04T17:48:14.943390+0000","last_clean":"2024-09-04T17:48:14.943390+0000","last_became_active":"2024-09-04T17:47:32.847682+0000","last_became_peered":"2024-09-04T17:47:32.847682+0000","last_unstale":"2024-09-04T17:48:14.943390+0000","last_undegraded":"2024-09-04T17:48:14.943390+0000","last_fullsized":"2024-09-04T17:48:14.943390+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T21:08:35.630294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016638060000000001,"stat_sum":{"num_bytes":43507193,"num_objects":70,"num_object_clones":10,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":39,"num_read":2705,"num_read_kb":17207,"num_write":531,"num_write_kb":29364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":14692620,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"3.9","version":"134'593","reported_seq":4140,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-04T17:48:14.936869+0000","last_change":"2024-09-04T17:48:14.937193+0000","last_active":"2024-09-04T17:48:14.936869+0000","last_peered":"2024-09-04T17:48:14.936869+0000","last_clean":"2024-09-04T17:48:14.936869+0000","last_became_active":"2024-09-04T17:47:44.949630+0000","last_became_peered":"2024-09-04T17:47:44.949630+0000","last_unstale":"2024-09-04T17:48:14.936869+0000","last_undegraded":"2024-09-04T17:48:14.936869+0000","last_fullsized":"2024-09-04T17:48:14.936869+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T21:19:18.496371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012454479999999999,"stat_sum":{"num_bytes":55724561,"num_objects":78,"num_object_clones":14,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":42,"num_read":5701,"num_read_kb":37078,"num_write":682,"num_write_kb":18628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":4534479,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"3.a","version":"136'564","reported_seq":3161,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-04T17:48:16.094285+0000","last_change":"2024-09-04T17:48:15.965561+0000","last_active":"2024-09-04T17:48:16.094285+0000","last_peered":"2024-09-04T17:48:16.094285+0000","last_clean":"2024-09-04T17:48:16.094285+0000","last_became_active":"2024-09-04T17:48:00.841070+0000","last_became_peered":"2024-09-04T17:48:00.841070+0000","last_unstale":"2024-09-04T17:48:16.094285+0000","last_undegraded":"2024-09-04T17:48:16.094285+0000","last_fullsized":"2024-09-04T17:48:16.094285+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:20:54.836335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032178160000000001,"stat_sum":{"num_bytes":30919132,"num_objects":64,"num_object_clones":8,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":38,"num_read":5437,"num_read_kb":16249,"num_write":550,"num_write_kb":2747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":1572433,"num_keys_recovered":66,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"26","length":"8"}]},{"pgid":"3.b","version":"135'683","reported_seq":2237,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-04T17:48:16.020689+0000","last_change":"2024-09-04T17:48:16.020943+0000","last_active":"2024-09-04T17:48:16.020689+0000","last_peered":"2024-09-04T17:48:16.020689+0000","last_clean":"2024-09-04T17:48:16.020689+0000","last_became_active":"2024-09-04T17:47:08.657286+0000","last_became_peered":"2024-09-04T17:47:08.657286+0000","last_unstale":"2024-09-04T17:48:16.020689+0000","last_undegraded":"2024-09-04T17:48:16.020689+0000","last_fullsized":"2024-09-04T17:48:16.020689+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":683,"log_dups_size":0,"ondisk_log_size":683,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T19:18:16.608740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015805190000000001,"stat_sum":{"num_bytes":28167800,"num_objects":66,"num_object_clones":8,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":32,"num_read":3911,"num_read_kb":14517,"num_write":847,"num_write_kb":7911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":3759450,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"26","length":"8"}]},{"pgid":"3.c","version":"136'567","reported_seq":7841,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-04T17:48:15.977330+0000","last_change":"2024-09-04T17:48:15.977330+0000","last_active":"2024-09-04T17:48:15.977330+0000","last_peered":"2024-09-04T17:48:15.977330+0000","last_clean":"2024-09-04T17:48:15.977330+0000","last_became_active":"2024-09-04T17:48:02.855127+0000","last_became_peered":"2024-09-04T17:48:02.855127+0000","last_unstale":"2024-09-04T17:48:15.977330+0000","last_undegraded":"2024-09-04T17:48:15.977330+0000","last_fullsized":"2024-09-04T17:48:15.977330+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":567,"log_dups_size":0,"ondisk_log_size":567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:43:01.389348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0042038780000000003,"stat_sum":{"num_bytes":44494056,"num_objects":70,"num_object_clones":10,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":42,"num_read":8677,"num_read_kb":71113,"num_write":704,"num_write_kb":16851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":153,"num_bytes_recovered":7666524,"num_keys_recovered":34,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"26","length":"8"}]},{"pgid":"3.d","version":"126'1282","reported_seq":4096,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-04T17:48:15.146132+0000","last_change":"2024-09-04T17:48:14.947144+0000","last_active":"2024-09-04T17:48:15.146132+0000","last_peered":"2024-09-04T17:48:15.146132+0000","last_clean":"2024-09-04T17:48:15.146132+0000","last_became_active":"2024-09-04T17:48:12.850381+0000","last_became_peered":"2024-09-04T17:48:12.850381+0000","last_unstale":"2024-09-04T17:48:15.146132+0000","last_undegraded":"2024-09-04T17:48:15.146132+0000","last_fullsized":"2024-09-04T17:48:15.146132+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":1282,"log_dups_size":0,"ondisk_log_size":1282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T19:19:15.721797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026779030000000001,"stat_sum":{"num_bytes":49889564,"num_objects":60,"num_object_clones":6,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":34,"num_read":8529,"num_read_kb":26250,"num_write":2192,"num_write_kb":13182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":179,"num_bytes_recovered":18877339,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"3.e","version":"135'446","reported_seq":5479,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-04T17:48:14.946295+0000","last_change":"2024-09-04T17:48:14.946295+0000","last_active":"2024-09-04T17:48:14.946295+0000","last_peered":"2024-09-04T17:48:14.946295+0000","last_clean":"2024-09-04T17:48:14.946295+0000","last_became_active":"2024-09-04T17:48:11.849605+0000","last_became_peered":"2024-09-04T17:48:11.849605+0000","last_unstale":"2024-09-04T17:48:14.946295+0000","last_undegraded":"2024-09-04T17:48:14.946295+0000","last_fullsized":"2024-09-04T17:48:14.946295+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":446,"log_dups_size":0,"ondisk_log_size":446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:15:11.313510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030014099999999999,"stat_sum":{"num_bytes":59920521,"num_objects":61,"num_object_clones":11,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":31,"num_read":6129,"num_read_kb":28040,"num_write":459,"num_write_kb":11211,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":7806862,"num_keys_recovered":46,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"3.f","version":"134'295","reported_seq":2867,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-04T17:48:14.945597+0000","last_change":"2024-09-04T17:48:14.946214+0000","last_active":"2024-09-04T17:48:14.945597+0000","last_peered":"2024-09-04T17:48:14.945597+0000","last_clean":"2024-09-04T17:48:14.945597+0000","last_became_active":"2024-09-04T17:45:55.956734+0000","last_became_peered":"2024-09-04T17:45:55.956734+0000","last_unstale":"2024-09-04T17:48:14.945597+0000","last_undegraded":"2024-09-04T17:48:14.945597+0000","last_fullsized":"2024-09-04T17:48:14.945597+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":295,"log_dups_size":0,"ondisk_log_size":295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T21:49:16.444214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002751561,"stat_sum":{"num_bytes":36107388,"num_objects":66,"num_object_clones":11,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":35,"num_read":2563,"num_read_kb":44941,"num_write":298,"num_write_kb":13423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"3.10","version":"135'517","reported_seq":3176,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-04T17:48:15.964841+0000","last_change":"2024-09-04T17:48:15.965595+0000","last_active":"2024-09-04T17:48:15.964841+0000","last_peered":"2024-09-04T17:48:15.964841+0000","last_clean":"2024-09-04T17:48:15.964841+0000","last_became_active":"2024-09-04T17:48:00.843136+0000","last_became_peered":"2024-09-04T17:48:00.843136+0000","last_unstale":"2024-09-04T17:48:15.964841+0000","last_undegraded":"2024-09-04T17:48:15.964841+0000","last_fullsized":"2024-09-04T17:48:15.964841+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":517,"log_dups_size":0,"ondisk_log_size":517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T18:48:39.927931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028819470000000002,"stat_sum":{"num_bytes":50998066,"num_objects":75,"num_object_clones":9,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":45,"num_read":4069,"num_read_kb":18348,"num_write":592,"num_write_kb":12832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":3377599,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"26","length":"8"}]},{"pgid":"3.11","version":"135'503","reported_seq":5765,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-04T17:48:15.108116+0000","last_change":"2024-09-04T17:48:14.944990+0000","last_active":"2024-09-04T17:48:15.108116+0000","last_peered":"2024-09-04T17:48:15.108116+0000","last_clean":"2024-09-04T17:48:15.108116+0000","last_became_active":"2024-09-04T17:45:55.951870+0000","last_became_peered":"2024-09-04T17:45:55.951870+0000","last_unstale":"2024-09-04T17:48:15.108116+0000","last_undegraded":"2024-09-04T17:48:15.108116+0000","last_fullsized":"2024-09-04T17:48:15.108116+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":503,"log_dups_size":0,"ondisk_log_size":503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T21:51:34.296936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021887949999999999,"stat_sum":{"num_bytes":56384394,"num_objects":79,"num_object_clones":10,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":46,"num_read":6685,"num_read_kb":61028,"num_write":517,"num_write_kb":17560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"3.12","version":"136'356","reported_seq":3697,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-04T17:48:16.123626+0000","last_change":"2024-09-04T17:48:16.018679+0000","last_active":"2024-09-04T17:48:16.123626+0000","last_peered":"2024-09-04T17:48:16.123626+0000","last_clean":"2024-09-04T17:48:16.123626+0000","last_became_active":"2024-09-04T17:48:14.943778+0000","last_became_peered":"2024-09-04T17:48:14.943778+0000","last_unstale":"2024-09-04T17:48:16.123626+0000","last_undegraded":"2024-09-04T17:48:16.123626+0000","last_fullsized":"2024-09-04T17:48:16.123626+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":356,"log_dups_size":0,"ondisk_log_size":356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T19:10:15.445367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061430299999999996,"stat_sum":{"num_bytes":21224274,"num_objects":57,"num_object_clones":5,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":35,"num_read":3193,"num_read_kb":21391,"num_write":398,"num_write_kb":8563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":290,"num_bytes_recovered":2723025,"num_keys_recovered":72,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"26","length":"8"}]},{"pgid":"3.13","version":"135'267","reported_seq":2251,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-04T17:48:15.108668+0000","last_change":"2024-09-04T17:48:14.943848+0000","last_active":"2024-09-04T17:48:15.108668+0000","last_peered":"2024-09-04T17:48:15.108668+0000","last_clean":"2024-09-04T17:48:15.108668+0000","last_became_active":"2024-09-04T17:45:55.954544+0000","last_became_peered":"2024-09-04T17:45:55.954544+0000","last_unstale":"2024-09-04T17:48:15.108668+0000","last_undegraded":"2024-09-04T17:48:15.108668+0000","last_fullsized":"2024-09-04T17:48:15.108668+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":267,"log_dups_size":0,"ondisk_log_size":267,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T22:59:30.435260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00158848,"stat_sum":{"num_bytes":22890202,"num_objects":59,"num_object_clones":4,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":38,"num_read":2181,"num_read_kb":22538,"num_write":217,"num_write_kb":4374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"27","length":"7"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1447876135,"num_objects":2014,"num_object_clones":254,"num_object_copies":4028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2014,"num_whiteouts":1125,"num_read":132629,"num_read_kb":868506,"num_write":18434,"num_write_kb":403810,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3601,"num_bytes_recovered":213911053,"num_keys_recovered":676,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":445329408,"data_stored":551112929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21395,"internal_metadata":0},"log_size":15700,"ondisk_log_size":15700,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":117,"seq":502511173637,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":96468,"kb_used_data":62712,"kb_used_omap":22,"kb_used_meta":33705,"kb_avail":94275372,"statfs":{"total":96636764160,"available":96537980928,"internally_reserved":0,"allocated":64217088,"data_stored":67245296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22742,"internal_metadata":34514730},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509474,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":100228,"kb_used_data":69028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94271612,"statfs":{"total":96636764160,"available":96534130688,"internally_reserved":0,"allocated":70684672,"data_stored":87495400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509475,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":170232,"kb_used_data":112624,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94201608,"statfs":{"total":96636764160,"available":96462446592,"internally_reserved":0,"allocated":115326976,"data_stored":144929182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":0,"seq":206158430231,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":64424509474,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509474,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":116548,"kb_used_data":67312,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94255292,"statfs":{"total":96636764160,"available":96517419008,"internally_reserved":0,"allocated":68927488,"data_stored":85327951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542178,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":133072,"kb_used_data":79676,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94238768,"statfs":{"total":96636764160,"available":96500498432,"internally_reserved":0,"allocated":81588224,"data_stored":104088771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":103,"seq":442381631496,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":85608,"kb_used_data":45452,"kb_used_omap":12,"kb_used_meta":40115,"kb_avail":94286232,"statfs":{"total":96636764160,"available":96549101568,"internally_reserved":0,"allocated":46542848,"data_stored":56869238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12804,"internal_metadata":41078268},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66633728,"data_stored":83884252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2035712,"data_stored":2023494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":44945408,"data_stored":56105331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6121,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":7245824,"data_stored":9302332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":113704960,"data_stored":144152805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":68792320,"data_stored":86463701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":62603264,"data_stored":66475127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15274,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":79368192,"data_stored":102705887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-04T17:48:19.738 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-04T17:48:19.738 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-04T17:48:19.738 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-09-04T17:48:19.738 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-04T17:48:19.738 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-04T17:48:19.738 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set filestore_inject_stall 3 2024-09-04T17:48:19.749 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:48:19.746+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:48:19.824 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:19.818+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:19.830 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:19.830 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:19.844 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-04T17:48:19.855 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:19.855 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-04T17:48:19.855 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:19.929 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:48:19.926+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:48:19.973 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is set 2024-09-04T17:48:20.022 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:20.022 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:20.026 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:48:20.022+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:48:20.037 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-04T17:48:20.128 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:48:20.122+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:48:20.159 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:20.159 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:20.173 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-04T17:48:20.228 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:20.223+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:20.293 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:20.293 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:20.308 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-04T17:48:20.330 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:20.330+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:20.431 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:20.426+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:20.469 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:20.470 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:20.484 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-04T17:48:20.532 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:20.526+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:20.600 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:20.600 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:20.614 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-04T17:48:20.633 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:48:20.630+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:48:20.733 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:20.733 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:20.734 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:20.731+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:20.747 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-04T17:48:20.837 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:20.835+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:20.914 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:20.914 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:20.928 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-04T17:48:20.969 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:48:20.967+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:48:21.036 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:21.031+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:21.073 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:21.074 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:21.091 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-04T17:48:21.137 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:48:21.134+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:48:21.152 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-04T17:48:21.223 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:21.223 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:21.237 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-04T17:48:21.239 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:21.235+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:21.308 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:48:21.309 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:48:21.325 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:48:21.325 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:48:21.340 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:48:21.334+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:48:21.341 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-04T17:48:21.342 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-04T17:48:21.358 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-04T17:48:21.358 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-04T17:48:21.376 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:48:21.376 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:48:21.394 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:48:21.394 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:48:21.405 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:21.405 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:48:21.405 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:48:21.405 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:48:21.405 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.406 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:12722 3.1b 3:d8152af1:::rbd_data.1134ef3a5a7d.00000000000000bd:head [delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-04T17:48:21.406 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:16.421965+0000", 2024-09-04T17:48:21.406 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 124.982522015, 2024-09-04T17:48:21.406 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.890002811, 2024-09-04T17:48:21.406 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:21.406 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:21.406 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:21.406 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:21.406 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:21.406 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:21.407 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 12722 2024-09-04T17:48:21.407 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.407 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:21.407 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.407 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:21.407 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421965+0000", 2024-09-04T17:48:21.407 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:21.407 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.407 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.407 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:21.407 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421965+0000", 2024-09-04T17:48:21.408 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:21.408 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.408 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.408 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:21.408 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421966+0000", 2024-09-04T17:48:21.408 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.102e-06 2024-09-04T17:48:21.408 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.408 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.408 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:21.408 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421971+0000", 2024-09-04T17:48:21.408 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9989999999999999e-06 2024-09-04T17:48:21.408 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.408 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.409 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:21.409 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421981+0000", 2024-09-04T17:48:21.409 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.7380000000000006e-06 2024-09-04T17:48:21.409 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.409 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.409 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:21.409 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421989+0000", 2024-09-04T17:48:21.409 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.1890000000000007e-06 2024-09-04T17:48:21.409 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.409 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.409 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:21.409 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.422050+0000", 2024-09-04T17:48:21.409 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.0702000000000003e-05 2024-09-04T17:48:21.410 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.410 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.410 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for peered", 2024-09-04T17:48:21.410 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.422059+0000", 2024-09-04T17:48:21.410 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5440000000000004e-06 2024-09-04T17:48:21.410 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.410 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.410 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:21.410 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.308156+0000", 2024-09-04T17:48:21.410 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.88609676199999998 2024-09-04T17:48:21.410 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.410 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.410 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:21.411 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.308398+0000", 2024-09-04T17:48:21.411 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024215400000000001 2024-09-04T17:48:21.411 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.411 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.411 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 7", 2024-09-04T17:48:21.411 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.308795+0000", 2024-09-04T17:48:21.411 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000396704 2024-09-04T17:48:21.411 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.411 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.411 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:48:21.411 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.310831+0000", 2024-09-04T17:48:21.411 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0020364200000000002 2024-09-04T17:48:21.412 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.412 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.412 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:21.412 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.311818+0000", 2024-09-04T17:48:21.412 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00098649799999999993 2024-09-04T17:48:21.412 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.412 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.412 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:21.412 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.311905+0000", 2024-09-04T17:48:21.412 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7070999999999994e-05 2024-09-04T17:48:21.412 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.412 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.412 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:21.412 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.311968+0000", 2024-09-04T17:48:21.413 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.2928000000000005e-05 2024-09-04T17:48:21.413 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:21.413 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:21.413 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:21.413 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.413 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.413 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37359 3.f e58/24 3:f29e50a5:::rbd_data.1134c5a43465.0000000000000021:head v 58'135, mlcod=58'133)", 2024-09-04T17:48:21.413 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:48:21.413 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 101.78719141099999, 2024-09-04T17:48:21.413 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053483197000000003, 2024-09-04T17:48:21.413 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:21.413 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:21.413 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:21.414 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:21.414 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.414 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:21.414 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:48:21.414 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:21.414 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.414 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.414 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:21.414 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:48:21.414 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:21.414 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.414 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.414 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:21.414 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:48:21.415 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1699999999999997e-07 2024-09-04T17:48:21.415 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.415 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.415 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:21.415 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617338+0000", 2024-09-04T17:48:21.415 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.1168999999999998e-05 2024-09-04T17:48:21.415 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.415 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.415 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:21.415 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617351+0000", 2024-09-04T17:48:21.415 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3094e-05 2024-09-04T17:48:21.415 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.415 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.415 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:21.416 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617359+0000", 2024-09-04T17:48:21.416 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7970000000000006e-06 2024-09-04T17:48:21.416 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.416 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.416 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:21.416 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617417+0000", 2024-09-04T17:48:21.416 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7079e-05 2024-09-04T17:48:21.416 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.416 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.416 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:21.416 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617438+0000", 2024-09-04T17:48:21.416 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1654999999999998e-05 2024-09-04T17:48:21.416 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.417 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.417 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:21.417 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670738+0000", 2024-09-04T17:48:21.417 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053300142000000002 2024-09-04T17:48:21.417 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.417 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.417 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:21.417 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670779+0000", 2024-09-04T17:48:21.417 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0343999999999999e-05 2024-09-04T17:48:21.417 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:21.417 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:21.417 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:21.417 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.417 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.418 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37358 3.1d e58/24 3:bd3215f3:::rbd_data.1134c5a43465.0000000000000020:head v 58'91, mlcod=58'89)", 2024-09-04T17:48:21.418 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617502+0000", 2024-09-04T17:48:21.418 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 101.78698469299999, 2024-09-04T17:48:21.418 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053505641999999999, 2024-09-04T17:48:21.418 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:21.418 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:21.418 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:21.418 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:21.418 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.418 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:21.418 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617502+0000", 2024-09-04T17:48:21.418 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:21.418 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.419 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.419 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:21.419 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617502+0000", 2024-09-04T17:48:21.419 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:21.419 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.419 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.419 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:21.419 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617503+0000", 2024-09-04T17:48:21.419 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.184e-06 2024-09-04T17:48:21.419 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.419 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.419 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:21.419 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617526+0000", 2024-09-04T17:48:21.419 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2099999999999998e-05 2024-09-04T17:48:21.420 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.420 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.420 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:21.420 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617540+0000", 2024-09-04T17:48:21.420 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4741e-05 2024-09-04T17:48:21.420 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.420 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.420 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:21.420 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617550+0000", 2024-09-04T17:48:21.420 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4259999999999992e-06 2024-09-04T17:48:21.420 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.420 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.421 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:21.421 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617613+0000", 2024-09-04T17:48:21.421 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.3625000000000002e-05 2024-09-04T17:48:21.421 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.421 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.421 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:21.421 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617637+0000", 2024-09-04T17:48:21.421 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.3604999999999998e-05 2024-09-04T17:48:21.421 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.421 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.421 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:21.421 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670967+0000", 2024-09-04T17:48:21.421 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053329881000000003 2024-09-04T17:48:21.421 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.422 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.422 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:21.422 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671008+0000", 2024-09-04T17:48:21.422 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.108e-05 2024-09-04T17:48:21.422 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:21.422 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:21.422 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:21.422 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.422 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.422 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37361 3.1c e58/24 3:3c7f8976:::rbd_data.1134c5a43465.0000000000000023:head v 58'159, mlcod=58'157)", 2024-09-04T17:48:21.422 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617578+0000", 2024-09-04T17:48:21.422 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 101.78690884300001, 2024-09-04T17:48:21.422 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053990367999999997, 2024-09-04T17:48:21.423 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:21.423 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:21.423 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:21.423 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:21.423 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.423 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:21.423 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617578+0000", 2024-09-04T17:48:21.423 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:21.423 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.423 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.423 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:21.423 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617578+0000", 2024-09-04T17:48:21.424 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:21.424 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.424 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.424 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:21.424 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617579+0000", 2024-09-04T17:48:21.424 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7299999999999996e-07 2024-09-04T17:48:21.424 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.424 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.424 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:21.424 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617610+0000", 2024-09-04T17:48:21.424 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0951000000000002e-05 2024-09-04T17:48:21.424 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.424 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.424 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:21.425 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:48:21.425 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0002e-05 2024-09-04T17:48:21.425 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.425 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.425 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:21.425 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617626+0000", 2024-09-04T17:48:21.425 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6660000000000002e-06 2024-09-04T17:48:21.425 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.425 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.425 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:21.425 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670548+0000", 2024-09-04T17:48:21.425 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.052921943999999999 2024-09-04T17:48:21.425 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.426 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.426 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:21.426 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670574+0000", 2024-09-04T17:48:21.426 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5405000000000002e-05 2024-09-04T17:48:21.426 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.426 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.426 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:21.426 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671522+0000", 2024-09-04T17:48:21.426 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00094815299999999997 2024-09-04T17:48:21.426 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.426 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.426 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:21.426 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671568+0000", 2024-09-04T17:48:21.427 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6674000000000001e-05 2024-09-04T17:48:21.427 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:21.427 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:21.427 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:21.427 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.427 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.427 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37357 3.15 e58/49 3:a897312a:::rbd_data.1134c5a43465.000000000000001f:head v 58'117, mlcod=58'115)", 2024-09-04T17:48:21.427 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:48:21.427 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 101.78686650500001, 2024-09-04T17:48:21.427 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053734297, 2024-09-04T17:48:21.427 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:21.427 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:21.427 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:21.428 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:21.428 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.428 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:21.428 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:48:21.428 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:21.428 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.428 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.428 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:21.428 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:48:21.428 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:21.428 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.428 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.429 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:21.429 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617621+0000", 2024-09-04T17:48:21.429 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.7300000000000004e-07 2024-09-04T17:48:21.429 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.429 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.429 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:21.429 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617642+0000", 2024-09-04T17:48:21.429 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0185000000000001e-05 2024-09-04T17:48:21.429 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.429 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.429 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:21.429 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617654+0000", 2024-09-04T17:48:21.429 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2745000000000001e-05 2024-09-04T17:48:21.430 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.430 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.430 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:21.430 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617662+0000", 2024-09-04T17:48:21.430 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.0499999999999992e-06 2024-09-04T17:48:21.430 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.430 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.430 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:21.430 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617727+0000", 2024-09-04T17:48:21.430 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.4923000000000002e-05 2024-09-04T17:48:21.430 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.430 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.430 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:21.431 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617753+0000", 2024-09-04T17:48:21.431 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5899999999999999e-05 2024-09-04T17:48:21.431 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.431 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.431 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:21.431 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671298+0000", 2024-09-04T17:48:21.431 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053545276000000003 2024-09-04T17:48:21.431 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.431 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.431 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:21.431 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671355+0000", 2024-09-04T17:48:21.431 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6245000000000001e-05 2024-09-04T17:48:21.432 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:21.432 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:21.432 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:21.432 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.432 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.432 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44687 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:21.432 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253012+0000", 2024-09-04T17:48:21.432 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 91.151474437999994, 2024-09-04T17:48:21.432 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051128777, 2024-09-04T17:48:21.432 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:21.432 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:21.432 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:21.432 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:21.433 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:21.433 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:21.433 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44687 2024-09-04T17:48:21.433 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.433 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:21.433 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.433 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:21.433 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253012+0000", 2024-09-04T17:48:21.433 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:21.433 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.433 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.433 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:21.434 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253012+0000", 2024-09-04T17:48:21.434 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:21.434 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.434 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.434 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:21.434 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253014+0000", 2024-09-04T17:48:21.434 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.24e-06 2024-09-04T17:48:21.434 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.434 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.434 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:21.434 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253019+0000", 2024-09-04T17:48:21.434 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4569999999999996e-06 2024-09-04T17:48:21.434 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.435 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.435 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:21.435 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253035+0000", 2024-09-04T17:48:21.435 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6291999999999999e-05 2024-09-04T17:48:21.435 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.435 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.435 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:21.435 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253050+0000", 2024-09-04T17:48:21.435 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4423000000000001e-05 2024-09-04T17:48:21.435 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.435 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.435 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:21.436 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304115+0000", 2024-09-04T17:48:21.436 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051064778999999998 2024-09-04T17:48:21.436 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.436 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.436 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:21.436 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304141+0000", 2024-09-04T17:48:21.436 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6585999999999999e-05 2024-09-04T17:48:21.436 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:21.436 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:21.436 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:21.436 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.436 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.436 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44688 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:21.437 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253067+0000", 2024-09-04T17:48:21.437 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 91.151419885999999, 2024-09-04T17:48:21.437 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051116979, 2024-09-04T17:48:21.437 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:21.437 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:21.437 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:21.437 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:21.437 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:21.437 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:21.437 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44688 2024-09-04T17:48:21.437 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.437 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:21.438 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.438 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:21.438 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253067+0000", 2024-09-04T17:48:21.438 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:21.438 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.438 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.438 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:21.438 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253067+0000", 2024-09-04T17:48:21.438 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:21.438 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.438 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.438 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:21.438 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253068+0000", 2024-09-04T17:48:21.439 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0729999999999999e-06 2024-09-04T17:48:21.439 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.439 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.439 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:21.439 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253073+0000", 2024-09-04T17:48:21.439 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2259999999999996e-06 2024-09-04T17:48:21.439 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.439 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.439 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:21.439 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253091+0000", 2024-09-04T17:48:21.439 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7255999999999999e-05 2024-09-04T17:48:21.439 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.440 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.440 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:21.440 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253103+0000", 2024-09-04T17:48:21.440 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2632999999999999e-05 2024-09-04T17:48:21.440 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.440 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.440 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:21.440 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304173+0000", 2024-09-04T17:48:21.440 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051069537999999998 2024-09-04T17:48:21.440 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.440 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.440 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:21.440 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304184+0000", 2024-09-04T17:48:21.440 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1253000000000001e-05 2024-09-04T17:48:21.441 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:21.441 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:21.441 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:21.441 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.441 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.441 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44689 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:21.441 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253121+0000", 2024-09-04T17:48:21.441 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 91.151365749000007, 2024-09-04T17:48:21.441 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051192595, 2024-09-04T17:48:21.441 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:21.441 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:21.441 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:21.442 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:21.442 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:21.442 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:21.442 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44689 2024-09-04T17:48:21.442 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.442 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:21.442 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.442 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:21.442 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253121+0000", 2024-09-04T17:48:21.442 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:21.442 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.442 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.443 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:21.443 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253121+0000", 2024-09-04T17:48:21.443 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:21.443 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.443 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.443 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:21.443 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253122+0000", 2024-09-04T17:48:21.443 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.15e-06 2024-09-04T17:48:21.443 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.443 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.443 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:21.443 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253128+0000", 2024-09-04T17:48:21.443 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2569999999999998e-06 2024-09-04T17:48:21.444 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.444 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.444 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:21.444 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253145+0000", 2024-09-04T17:48:21.444 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7176e-05 2024-09-04T17:48:21.444 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.444 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.444 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:21.444 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253158+0000", 2024-09-04T17:48:21.444 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3139000000000001e-05 2024-09-04T17:48:21.444 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.444 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.444 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:21.445 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304287+0000", 2024-09-04T17:48:21.445 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051129149999999998 2024-09-04T17:48:21.445 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.445 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.445 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:21.445 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304314+0000", 2024-09-04T17:48:21.445 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6723000000000001e-05 2024-09-04T17:48:21.445 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:21.445 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:21.445 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:21.445 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.445 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.445 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44690 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:21.446 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253177+0000", 2024-09-04T17:48:21.446 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 91.151309624000007, 2024-09-04T17:48:21.446 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051169263, 2024-09-04T17:48:21.446 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:21.446 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:21.446 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:21.446 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:21.446 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:21.446 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:21.446 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44690 2024-09-04T17:48:21.446 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.446 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:21.446 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.447 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:21.447 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253177+0000", 2024-09-04T17:48:21.447 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:21.447 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.447 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.447 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:21.447 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253177+0000", 2024-09-04T17:48:21.447 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:21.447 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.447 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.447 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:21.447 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253178+0000", 2024-09-04T17:48:21.447 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1000000000000001e-06 2024-09-04T17:48:21.447 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.448 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.448 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:21.448 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253184+0000", 2024-09-04T17:48:21.448 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2089999999999996e-06 2024-09-04T17:48:21.448 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.448 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.448 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:21.448 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253200+0000", 2024-09-04T17:48:21.448 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6087e-05 2024-09-04T17:48:21.448 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.448 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.448 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:21.448 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253231+0000", 2024-09-04T17:48:21.448 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0874999999999998e-05 2024-09-04T17:48:21.449 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.449 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.449 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:21.449 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304337+0000", 2024-09-04T17:48:21.449 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051106430000000001 2024-09-04T17:48:21.449 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.449 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.449 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:21.449 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304347+0000", 2024-09-04T17:48:21.449 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5619999999999993e-06 2024-09-04T17:48:21.449 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:21.449 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:21.449 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:21.449 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.450 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.450 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44691 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:21.450 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253249+0000", 2024-09-04T17:48:21.450 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 91.151237589999994, 2024-09-04T17:48:21.450 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051195630999999998, 2024-09-04T17:48:21.450 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:21.450 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:21.450 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:21.450 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:21.450 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:21.450 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:21.450 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44691 2024-09-04T17:48:21.450 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.450 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:21.451 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.451 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:21.451 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253249+0000", 2024-09-04T17:48:21.451 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:21.451 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.451 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.451 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:21.451 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253249+0000", 2024-09-04T17:48:21.451 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:21.451 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.451 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.451 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:21.452 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253251+0000", 2024-09-04T17:48:21.452 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2330000000000001e-06 2024-09-04T17:48:21.452 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.452 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.452 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:21.452 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253256+0000", 2024-09-04T17:48:21.452 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6099999999999997e-06 2024-09-04T17:48:21.452 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.452 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.452 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:21.452 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253274+0000", 2024-09-04T17:48:21.452 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7952999999999999e-05 2024-09-04T17:48:21.452 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.453 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.453 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:21.453 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253287+0000", 2024-09-04T17:48:21.453 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2839e-05 2024-09-04T17:48:21.453 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.453 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.453 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:21.453 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304396+0000", 2024-09-04T17:48:21.453 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051108990999999999 2024-09-04T17:48:21.453 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.453 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.453 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:21.453 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304445+0000", 2024-09-04T17:48:21.454 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9005000000000003e-05 2024-09-04T17:48:21.454 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:21.454 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:21.454 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:21.454 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.454 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.454 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44692 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:21.454 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253337+0000", 2024-09-04T17:48:21.454 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 91.151150017000006, 2024-09-04T17:48:21.454 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051138909000000003, 2024-09-04T17:48:21.454 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:21.454 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:21.454 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:21.455 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:21.455 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:21.455 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:21.455 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44692 2024-09-04T17:48:21.455 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.455 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:21.455 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.455 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:21.455 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253337+0000", 2024-09-04T17:48:21.455 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:21.455 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.455 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.455 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:21.455 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253337+0000", 2024-09-04T17:48:21.456 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:21.456 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.456 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.456 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:21.456 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253338+0000", 2024-09-04T17:48:21.456 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.463e-06 2024-09-04T17:48:21.456 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.456 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.456 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:21.456 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253357+0000", 2024-09-04T17:48:21.456 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9043e-05 2024-09-04T17:48:21.456 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.456 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.457 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:21.457 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253379+0000", 2024-09-04T17:48:21.457 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1243e-05 2024-09-04T17:48:21.457 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.457 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.457 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:21.457 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253396+0000", 2024-09-04T17:48:21.457 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7059e-05 2024-09-04T17:48:21.457 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.457 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.457 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:21.457 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304466+0000", 2024-09-04T17:48:21.457 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051070624000000002 2024-09-04T17:48:21.458 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.458 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.458 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:21.458 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304476+0000", 2024-09-04T17:48:21.458 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4769999999999992e-06 2024-09-04T17:48:21.458 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:21.458 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:21.458 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:21.458 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.458 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.458 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44693 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:21.458 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253415+0000", 2024-09-04T17:48:21.458 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 91.151071614000003, 2024-09-04T17:48:21.458 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051141949999999999, 2024-09-04T17:48:21.459 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:21.459 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:21.459 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:21.459 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:21.459 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:21.459 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:21.459 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44693 2024-09-04T17:48:21.459 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.459 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:21.459 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.459 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:21.459 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253415+0000", 2024-09-04T17:48:21.459 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:21.459 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.460 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.460 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:21.460 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253415+0000", 2024-09-04T17:48:21.460 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:21.460 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.460 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.460 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:21.460 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253416+0000", 2024-09-04T17:48:21.460 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1799999999999999e-06 2024-09-04T17:48:21.460 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.460 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.460 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:21.460 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253422+0000", 2024-09-04T17:48:21.461 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.5940000000000001e-06 2024-09-04T17:48:21.461 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.461 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.461 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:21.461 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253438+0000", 2024-09-04T17:48:21.461 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6008999999999999e-05 2024-09-04T17:48:21.461 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.461 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.461 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:21.461 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253452+0000", 2024-09-04T17:48:21.461 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3543000000000001e-05 2024-09-04T17:48:21.461 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.461 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.462 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:21.462 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304533+0000", 2024-09-04T17:48:21.462 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051081609 2024-09-04T17:48:21.462 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.462 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.462 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:21.462 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304557+0000", 2024-09-04T17:48:21.462 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4015e-05 2024-09-04T17:48:21.462 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:21.462 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:21.462 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:21.463 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.463 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.463 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44694 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:21.463 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253469+0000", 2024-09-04T17:48:21.463 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 91.151017488999997, 2024-09-04T17:48:21.463 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051121486000000001, 2024-09-04T17:48:21.463 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:21.463 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:21.463 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:21.463 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:21.463 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:21.463 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:21.463 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44694 2024-09-04T17:48:21.463 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.464 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:21.464 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.464 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:21.464 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253469+0000", 2024-09-04T17:48:21.464 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:21.464 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.464 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.464 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:21.464 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253469+0000", 2024-09-04T17:48:21.464 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:21.464 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.464 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.464 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:21.465 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253471+0000", 2024-09-04T17:48:21.465 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1400000000000001e-06 2024-09-04T17:48:21.465 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.465 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.465 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:21.468 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253476+0000", 2024-09-04T17:48:21.468 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7370000000000003e-06 2024-09-04T17:48:21.468 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.469 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.469 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:21.469 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253493+0000", 2024-09-04T17:48:21.469 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6549e-05 2024-09-04T17:48:21.469 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.469 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.469 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:21.469 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253506+0000", 2024-09-04T17:48:21.469 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3406e-05 2024-09-04T17:48:21.469 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.469 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.469 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:21.470 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304582+0000", 2024-09-04T17:48:21.470 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051075534999999998 2024-09-04T17:48:21.470 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.470 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.470 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:21.470 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304591+0000", 2024-09-04T17:48:21.470 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1190000000000003e-06 2024-09-04T17:48:21.470 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:21.470 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:21.470 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:21.470 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.470 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.470 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44695 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:21.470 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253552+0000", 2024-09-04T17:48:21.470 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 91.150934735000007, 2024-09-04T17:48:21.471 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051068872000000001, 2024-09-04T17:48:21.471 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:21.471 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:21.471 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:21.471 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:21.471 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:21.471 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:21.471 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44695 2024-09-04T17:48:21.471 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.471 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:21.471 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.471 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:21.471 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253552+0000", 2024-09-04T17:48:21.471 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:21.472 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.472 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.472 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:21.472 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253552+0000", 2024-09-04T17:48:21.472 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:21.472 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.472 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.472 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:21.472 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253553+0000", 2024-09-04T17:48:21.472 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1960000000000001e-06 2024-09-04T17:48:21.472 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.473 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.473 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:21.473 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253560+0000", 2024-09-04T17:48:21.473 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.2040000000000002e-06 2024-09-04T17:48:21.473 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.473 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.473 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:21.473 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253578+0000", 2024-09-04T17:48:21.473 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8241999999999999e-05 2024-09-04T17:48:21.473 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.473 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.473 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:21.473 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253591+0000", 2024-09-04T17:48:21.474 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3453e-05 2024-09-04T17:48:21.474 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.474 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.474 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:21.474 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304614+0000", 2024-09-04T17:48:21.474 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051022456000000001 2024-09-04T17:48:21.474 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.474 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.474 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:21.474 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304621+0000", 2024-09-04T17:48:21.474 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3209999999999999e-06 2024-09-04T17:48:21.474 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:21.474 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:21.475 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:21.475 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.475 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.475 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44696 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:21.475 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253609+0000", 2024-09-04T17:48:21.475 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 91.150878234000004, 2024-09-04T17:48:21.475 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051115351000000003, 2024-09-04T17:48:21.475 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:21.475 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:21.475 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:21.475 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:21.475 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:21.475 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:21.475 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44696 2024-09-04T17:48:21.476 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.476 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:21.476 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.476 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:21.476 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253609+0000", 2024-09-04T17:48:21.476 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:21.476 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.476 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.476 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:21.476 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253609+0000", 2024-09-04T17:48:21.476 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:21.476 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.476 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.476 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:21.477 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253610+0000", 2024-09-04T17:48:21.477 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2300000000000001e-06 2024-09-04T17:48:21.477 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.477 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.477 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:21.477 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253615+0000", 2024-09-04T17:48:21.477 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7260000000000002e-06 2024-09-04T17:48:21.477 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.477 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.477 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:21.478 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253632+0000", 2024-09-04T17:48:21.478 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7039999999999999e-05 2024-09-04T17:48:21.478 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.478 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.478 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:21.478 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253666+0000", 2024-09-04T17:48:21.479 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4662000000000001e-05 2024-09-04T17:48:21.479 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.479 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.479 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:21.479 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304677+0000", 2024-09-04T17:48:21.479 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051011017999999998 2024-09-04T17:48:21.479 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.479 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.479 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:21.480 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304724+0000", 2024-09-04T17:48:21.480 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6675000000000003e-05 2024-09-04T17:48:21.480 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:21.480 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:21.480 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:21.480 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.480 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.480 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44697 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:21.480 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253684+0000", 2024-09-04T17:48:21.480 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 91.150802740000003, 2024-09-04T17:48:21.480 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051073107, 2024-09-04T17:48:21.480 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:21.481 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:21.481 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:21.481 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:21.481 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:21.481 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:21.481 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44697 2024-09-04T17:48:21.481 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.481 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:21.481 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.481 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:21.481 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253684+0000", 2024-09-04T17:48:21.481 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:21.481 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.482 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.482 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:21.482 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253684+0000", 2024-09-04T17:48:21.482 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:21.482 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.482 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.482 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:21.482 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253685+0000", 2024-09-04T17:48:21.482 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0699999999999999e-06 2024-09-04T17:48:21.482 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.482 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.482 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:21.482 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253690+0000", 2024-09-04T17:48:21.482 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1959999999999999e-06 2024-09-04T17:48:21.483 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.483 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.483 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:21.483 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253707+0000", 2024-09-04T17:48:21.483 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6946e-05 2024-09-04T17:48:21.483 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.483 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.483 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:21.483 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253720+0000", 2024-09-04T17:48:21.483 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3076e-05 2024-09-04T17:48:21.483 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.483 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.483 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:21.483 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304745+0000", 2024-09-04T17:48:21.484 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051024760000000002 2024-09-04T17:48:21.484 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.484 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.484 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:21.484 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304757+0000", 2024-09-04T17:48:21.484 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2058999999999999e-05 2024-09-04T17:48:21.484 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:21.484 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:21.484 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:21.484 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.484 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.484 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44698 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:21.485 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253737+0000", 2024-09-04T17:48:21.485 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 91.150750388999995, 2024-09-04T17:48:21.485 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051176078, 2024-09-04T17:48:21.485 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:21.485 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:21.485 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:21.485 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:21.485 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:21.485 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:21.485 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44698 2024-09-04T17:48:21.485 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.486 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:21.486 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.486 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:21.486 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253737+0000", 2024-09-04T17:48:21.486 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:21.486 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.486 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.487 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:21.487 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253737+0000", 2024-09-04T17:48:21.487 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:21.487 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.487 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.487 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:21.488 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253738+0000", 2024-09-04T17:48:21.488 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0300000000000001e-06 2024-09-04T17:48:21.488 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.488 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.488 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:21.488 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253752+0000", 2024-09-04T17:48:21.489 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4358999999999999e-05 2024-09-04T17:48:21.489 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.489 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.489 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:21.489 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253771+0000", 2024-09-04T17:48:21.489 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8879999999999999e-05 2024-09-04T17:48:21.490 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.490 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.490 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:21.490 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253785+0000", 2024-09-04T17:48:21.490 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3852000000000001e-05 2024-09-04T17:48:21.490 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.491 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.491 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:21.491 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304876+0000", 2024-09-04T17:48:21.491 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051091105999999997 2024-09-04T17:48:21.491 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.492 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.492 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:21.492 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304913+0000", 2024-09-04T17:48:21.492 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.6850999999999997e-05 2024-09-04T17:48:21.492 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:21.492 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:21.493 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:21.493 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.493 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.493 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44699 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:21.493 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253804+0000", 2024-09-04T17:48:21.494 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 91.150682864999993, 2024-09-04T17:48:21.494 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051172250000000002, 2024-09-04T17:48:21.494 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:21.494 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:21.494 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:21.494 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:21.495 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:21.495 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:21.495 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44699 2024-09-04T17:48:21.495 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.495 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:21.495 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.496 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:21.496 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253804+0000", 2024-09-04T17:48:21.496 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:21.496 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.496 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.496 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:21.497 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253804+0000", 2024-09-04T17:48:21.497 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:21.497 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.497 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.497 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:21.497 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253805+0000", 2024-09-04T17:48:21.498 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.08e-06 2024-09-04T17:48:21.498 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.498 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.498 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:21.498 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253810+0000", 2024-09-04T17:48:21.498 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1830000000000002e-06 2024-09-04T17:48:21.499 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.499 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.499 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:21.499 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253828+0000", 2024-09-04T17:48:21.499 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7662000000000001e-05 2024-09-04T17:48:21.499 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.500 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.500 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:21.500 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253841+0000", 2024-09-04T17:48:21.500 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3203999999999999e-05 2024-09-04T17:48:21.500 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.500 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.501 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:21.501 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304966+0000", 2024-09-04T17:48:21.501 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051125273999999998 2024-09-04T17:48:21.501 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.501 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.501 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:21.502 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304976+0000", 2024-09-04T17:48:21.502 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.8470000000000008e-06 2024-09-04T17:48:21.502 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:21.502 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:21.502 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:21.502 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.503 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.503 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44700 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:21.503 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253858+0000", 2024-09-04T17:48:21.503 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 91.150629210999995, 2024-09-04T17:48:21.503 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051218957000000002, 2024-09-04T17:48:21.503 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:21.504 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:21.504 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:21.504 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:21.504 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:21.504 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:21.505 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44700 2024-09-04T17:48:21.505 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.505 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:21.505 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.505 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:21.505 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253858+0000", 2024-09-04T17:48:21.506 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:21.506 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.506 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.506 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:21.506 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253858+0000", 2024-09-04T17:48:21.507 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:21.507 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.507 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.507 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:21.507 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253859+0000", 2024-09-04T17:48:21.507 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.217e-06 2024-09-04T17:48:21.508 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.508 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.508 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:21.508 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253864+0000", 2024-09-04T17:48:21.508 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7600000000000002e-06 2024-09-04T17:48:21.508 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.509 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.509 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:21.509 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253880+0000", 2024-09-04T17:48:21.509 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6501999999999999e-05 2024-09-04T17:48:21.509 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.510 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.510 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:21.510 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253899+0000", 2024-09-04T17:48:21.510 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8907e-05 2024-09-04T17:48:21.510 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.510 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.511 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:21.511 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305052+0000", 2024-09-04T17:48:21.511 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051152989000000003 2024-09-04T17:48:21.511 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.511 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.511 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:21.512 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305077+0000", 2024-09-04T17:48:21.512 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4581999999999999e-05 2024-09-04T17:48:21.512 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:21.512 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:21.512 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:21.512 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.512 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.513 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44701 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:21.513 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253917+0000", 2024-09-04T17:48:21.513 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 91.150569859000001, 2024-09-04T17:48:21.513 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051231113000000002, 2024-09-04T17:48:21.513 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:21.513 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:21.514 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:21.514 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:21.514 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:21.514 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:21.514 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44701 2024-09-04T17:48:21.514 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.514 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:21.514 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.514 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:21.515 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253917+0000", 2024-09-04T17:48:21.515 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:21.515 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.515 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.515 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:21.515 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253917+0000", 2024-09-04T17:48:21.515 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:21.515 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.515 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.515 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:21.515 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253918+0000", 2024-09-04T17:48:21.515 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.136e-06 2024-09-04T17:48:21.515 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.515 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.516 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:21.516 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253924+0000", 2024-09-04T17:48:21.516 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.3070000000000002e-06 2024-09-04T17:48:21.516 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.516 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.516 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:21.516 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253940+0000", 2024-09-04T17:48:21.516 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6612e-05 2024-09-04T17:48:21.516 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.516 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.516 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:21.516 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253954+0000", 2024-09-04T17:48:21.517 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3709999999999999e-05 2024-09-04T17:48:21.517 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.517 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.517 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:21.517 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305136+0000", 2024-09-04T17:48:21.517 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051181935999999997 2024-09-04T17:48:21.517 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:21.517 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:21.517 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:21.517 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305148+0000", 2024-09-04T17:48:21.517 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2412000000000001e-05 2024-09-04T17:48:21.517 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:21.517 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:21.518 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:21.518 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:21.518 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:21.518 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:21.519 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-04T17:48:21.519 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-04T17:48:21.520 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:48:21.520 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-04T17:48:21.521 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:48:21.514+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:48:21.521 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:48:21.620 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:21.614+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:21.694 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:21.694 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:48:21.694 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:48:21.694 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:21.709 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-04T17:48:21.754 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:21.750+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:21.821 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:21.818+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:21.879 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:21.879 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:48:21.879 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:48:21.879 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:48:21.879 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:21.893 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-04T17:48:21.941 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:48:21.938+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:48:21.997 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-04T17:48:22.041 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:22.039+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:22.081 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:22.082 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:48:22.082 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:48:22.082 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:48:22.082 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.083 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8831 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2453504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:22.083 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:48:22.083 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 129.017135336, 2024-09-04T17:48:22.083 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14731111799999999, 2024-09-04T17:48:22.084 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:22.084 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:22.084 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:22.084 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:22.084 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:22.084 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:22.085 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8831 2024-09-04T17:48:22.085 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.085 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:22.085 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.085 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:22.086 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:48:22.086 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.086 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.086 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.086 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:22.086 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:48:22.087 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.087 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.087 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.087 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:22.087 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057811+0000", 2024-09-04T17:48:22.088 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.083e-06 2024-09-04T17:48:22.088 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.088 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.088 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:22.088 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057814+0000", 2024-09-04T17:48:22.088 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.407e-06 2024-09-04T17:48:22.089 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.089 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.089 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:22.089 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057825+0000", 2024-09-04T17:48:22.089 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.17e-05 2024-09-04T17:48:22.089 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.089 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.090 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:22.090 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057836+0000", 2024-09-04T17:48:22.090 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0283e-05 2024-09-04T17:48:22.090 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.090 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.090 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:22.091 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.204762+0000", 2024-09-04T17:48:22.091 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.146926205 2024-09-04T17:48:22.091 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.091 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.091 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:22.091 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.204895+0000", 2024-09-04T17:48:22.092 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013328999999999999 2024-09-04T17:48:22.092 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.092 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.092 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:22.092 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205121+0000", 2024-09-04T17:48:22.092 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00022614999999999999 2024-09-04T17:48:22.093 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.093 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:22.093 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.093 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.093 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.093 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8832 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2490368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:22.094 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:48:22.094 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 129.01709730300001, 2024-09-04T17:48:22.094 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14770012299999999, 2024-09-04T17:48:22.094 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:22.094 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:22.094 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:22.095 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:22.095 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:22.095 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:22.095 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8832 2024-09-04T17:48:22.095 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.095 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:22.096 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.096 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:22.096 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:48:22.096 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.096 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.096 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.097 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:22.097 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:48:22.097 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.097 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.097 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.097 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:22.097 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057849+0000", 2024-09-04T17:48:22.098 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6700000000000003e-07 2024-09-04T17:48:22.098 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.098 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.098 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:22.098 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057852+0000", 2024-09-04T17:48:22.098 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6359999999999998e-06 2024-09-04T17:48:22.099 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.099 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.099 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:22.099 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057860+0000", 2024-09-04T17:48:22.099 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8140000000000007e-06 2024-09-04T17:48:22.099 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.100 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.100 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:22.100 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057868+0000", 2024-09-04T17:48:22.100 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1999999999999997e-06 2024-09-04T17:48:22.100 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.100 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.100 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:22.101 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205208+0000", 2024-09-04T17:48:22.101 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14734067300000001 2024-09-04T17:48:22.101 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.101 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.101 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:22.101 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205347+0000", 2024-09-04T17:48:22.102 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013914900000000001 2024-09-04T17:48:22.102 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.102 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.102 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:22.102 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205548+0000", 2024-09-04T17:48:22.102 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000200984 2024-09-04T17:48:22.103 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.103 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:22.103 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.103 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.103 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.103 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8833 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2527232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:22.104 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:48:22.104 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 129.017065433, 2024-09-04T17:48:22.104 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148062572, 2024-09-04T17:48:22.104 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:22.104 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:22.104 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:22.105 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:22.105 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:22.105 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:22.105 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8833 2024-09-04T17:48:22.105 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.106 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:22.106 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.106 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:22.106 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:48:22.106 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.106 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.107 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.107 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:22.107 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:48:22.107 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.107 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.108 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.108 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:22.108 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057881+0000", 2024-09-04T17:48:22.108 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4300000000000002e-07 2024-09-04T17:48:22.108 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.108 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.108 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:22.109 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057893+0000", 2024-09-04T17:48:22.109 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1973999999999999e-05 2024-09-04T17:48:22.109 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.109 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.109 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:22.109 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057906+0000", 2024-09-04T17:48:22.109 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2716e-05 2024-09-04T17:48:22.109 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.110 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.110 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:22.110 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057913+0000", 2024-09-04T17:48:22.110 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3930000000000002e-06 2024-09-04T17:48:22.110 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.110 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.110 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:22.110 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205621+0000", 2024-09-04T17:48:22.111 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14770788800000001 2024-09-04T17:48:22.111 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.111 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.111 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:22.111 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205740+0000", 2024-09-04T17:48:22.111 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000118936 2024-09-04T17:48:22.111 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.111 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.112 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:22.112 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205943+0000", 2024-09-04T17:48:22.112 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020292199999999999 2024-09-04T17:48:22.112 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.112 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:22.112 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.112 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.113 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.113 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8834 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2564096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:22.113 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:48:22.113 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 129.01702215399999, 2024-09-04T17:48:22.113 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14846753200000001, 2024-09-04T17:48:22.114 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:22.114 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:22.114 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:22.114 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:22.114 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:22.114 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:22.114 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8834 2024-09-04T17:48:22.114 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.114 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:22.115 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.115 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:22.115 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:48:22.115 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.115 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.115 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.115 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:22.115 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:48:22.115 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.115 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.115 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.115 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:22.116 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057925+0000", 2024-09-04T17:48:22.116 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0270000000000001e-06 2024-09-04T17:48:22.116 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.116 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.116 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:22.116 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057936+0000", 2024-09-04T17:48:22.116 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.112e-05 2024-09-04T17:48:22.116 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.116 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.116 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:22.116 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057946+0000", 2024-09-04T17:48:22.117 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.009e-05 2024-09-04T17:48:22.117 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.117 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.117 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:22.117 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057953+0000", 2024-09-04T17:48:22.117 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.6229999999999997e-06 2024-09-04T17:48:22.117 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.117 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.117 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:22.117 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206027+0000", 2024-09-04T17:48:22.117 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148073591 2024-09-04T17:48:22.117 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.117 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.118 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:22.118 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206161+0000", 2024-09-04T17:48:22.118 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013411599999999999 2024-09-04T17:48:22.118 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.118 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.118 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:22.118 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206391+0000", 2024-09-04T17:48:22.118 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000229965 2024-09-04T17:48:22.118 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.118 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:22.118 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.118 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.118 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.119 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8835 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2600960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:22.119 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:48:22.119 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 129.016982504, 2024-09-04T17:48:22.119 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14882941599999999, 2024-09-04T17:48:22.119 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:22.119 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:22.119 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:22.119 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:22.119 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:22.119 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:22.119 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8835 2024-09-04T17:48:22.119 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.119 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:22.120 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.120 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:22.120 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:48:22.120 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.120 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.120 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.120 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:22.120 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:48:22.120 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.120 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.120 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.120 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:22.120 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058544+0000", 2024-09-04T17:48:22.121 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00058061899999999999 2024-09-04T17:48:22.121 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.121 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.121 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:22.121 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058550+0000", 2024-09-04T17:48:22.121 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.1129999999999997e-06 2024-09-04T17:48:22.121 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.121 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.121 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:22.121 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058566+0000", 2024-09-04T17:48:22.121 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5820000000000001e-05 2024-09-04T17:48:22.121 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.122 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.122 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:22.122 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058578+0000", 2024-09-04T17:48:22.122 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2537e-05 2024-09-04T17:48:22.122 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.122 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.122 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:22.122 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206470+0000", 2024-09-04T17:48:22.122 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14789153799999999 2024-09-04T17:48:22.122 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.122 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.122 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:22.122 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206589+0000", 2024-09-04T17:48:22.122 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000118837 2024-09-04T17:48:22.123 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.123 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.123 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:22.123 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206793+0000", 2024-09-04T17:48:22.123 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000203952 2024-09-04T17:48:22.123 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.123 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:22.123 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.123 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.123 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.123 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8836 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2654208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:22.123 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:48:22.123 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 129.016349816, 2024-09-04T17:48:22.124 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14862433999999999, 2024-09-04T17:48:22.124 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:22.124 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:22.124 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:22.124 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:22.124 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:22.124 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:22.124 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8836 2024-09-04T17:48:22.124 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.124 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:22.125 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.125 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:22.125 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:48:22.125 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.125 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.125 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.125 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:22.125 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:48:22.125 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.125 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.125 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.125 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:22.125 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062663+0000", 2024-09-04T17:48:22.126 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0040670619999999998 2024-09-04T17:48:22.126 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.126 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.126 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:22.126 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062668+0000", 2024-09-04T17:48:22.126 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4199999999999998e-06 2024-09-04T17:48:22.126 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.126 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.126 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:22.126 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062689+0000", 2024-09-04T17:48:22.126 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0356999999999998e-05 2024-09-04T17:48:22.126 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.126 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.127 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:22.127 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062726+0000", 2024-09-04T17:48:22.127 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7302999999999998e-05 2024-09-04T17:48:22.127 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.127 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.127 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:22.127 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206875+0000", 2024-09-04T17:48:22.127 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14414883100000001 2024-09-04T17:48:22.127 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.127 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.127 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:22.127 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.207016+0000", 2024-09-04T17:48:22.127 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000140822 2024-09-04T17:48:22.128 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.128 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.128 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:22.128 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.207220+0000", 2024-09-04T17:48:22.128 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020454499999999999 2024-09-04T17:48:22.128 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.128 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:22.128 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.128 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.128 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.128 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8844 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2928640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:22.128 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:48:22.128 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 129.01198573299999, 2024-09-04T17:48:22.129 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14752153100000001, 2024-09-04T17:48:22.129 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:22.129 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:22.129 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:22.129 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:22.129 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:22.129 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:22.129 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8844 2024-09-04T17:48:22.129 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.129 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:22.130 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.130 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:22.130 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:48:22.130 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.130 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.130 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.130 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:22.130 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:48:22.130 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.130 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.130 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.131 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:22.131 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062961+0000", 2024-09-04T17:48:22.131 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6700000000000003e-07 2024-09-04T17:48:22.131 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.131 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.131 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:22.131 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062973+0000", 2024-09-04T17:48:22.131 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2109999999999999e-05 2024-09-04T17:48:22.131 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.131 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.131 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:22.131 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062982+0000", 2024-09-04T17:48:22.132 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.6900000000000004e-06 2024-09-04T17:48:22.132 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.132 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.132 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:22.132 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062990+0000", 2024-09-04T17:48:22.132 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1899999999999998e-06 2024-09-04T17:48:22.133 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.133 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.133 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:22.133 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210090+0000", 2024-09-04T17:48:22.133 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14710026300000001 2024-09-04T17:48:22.133 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.133 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.133 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:22.133 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210225+0000", 2024-09-04T17:48:22.133 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013532400000000001 2024-09-04T17:48:22.133 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.133 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.134 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:22.134 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210481+0000", 2024-09-04T17:48:22.134 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00025628700000000001 2024-09-04T17:48:22.134 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.134 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:22.134 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.134 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.134 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.134 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8845 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2965504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:22.134 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:48:22.134 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 129.01194601, 2024-09-04T17:48:22.134 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147916819, 2024-09-04T17:48:22.134 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:22.135 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:22.135 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:22.135 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:22.135 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:22.135 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:22.135 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8845 2024-09-04T17:48:22.135 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.135 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:22.135 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.135 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:22.135 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:48:22.135 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.136 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.136 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.136 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:22.136 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:48:22.136 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.136 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.136 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.137 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:22.137 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:48:22.137 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.0399999999999995e-07 2024-09-04T17:48:22.137 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.138 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.138 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:22.138 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063003+0000", 2024-09-04T17:48:22.138 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6529999999999998e-06 2024-09-04T17:48:22.138 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.138 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.139 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:22.139 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063012+0000", 2024-09-04T17:48:22.139 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9500000000000007e-06 2024-09-04T17:48:22.139 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.139 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.139 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:22.139 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063027+0000", 2024-09-04T17:48:22.140 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5443000000000001e-05 2024-09-04T17:48:22.140 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.140 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.140 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:22.140 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210571+0000", 2024-09-04T17:48:22.140 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14754353100000001 2024-09-04T17:48:22.140 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.140 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.141 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:22.141 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210714+0000", 2024-09-04T17:48:22.141 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014295599999999999 2024-09-04T17:48:22.141 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.141 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.141 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:22.142 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210916+0000", 2024-09-04T17:48:22.142 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020258200000000001 2024-09-04T17:48:22.142 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.142 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:22.142 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.142 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.143 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.143 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8846 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3002368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:22.143 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:48:22.143 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 129.01190847300001, 2024-09-04T17:48:22.143 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14828011599999999, 2024-09-04T17:48:22.143 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:22.144 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:22.144 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:22.144 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:22.144 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:22.144 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:22.144 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8846 2024-09-04T17:48:22.144 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.144 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:22.145 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.145 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:22.145 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:48:22.145 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.145 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.145 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.145 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:22.145 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:48:22.145 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.145 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.145 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.145 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:22.145 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063038+0000", 2024-09-04T17:48:22.146 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.23e-07 2024-09-04T17:48:22.146 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.146 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.146 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:22.146 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063051+0000", 2024-09-04T17:48:22.146 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2757e-05 2024-09-04T17:48:22.146 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.146 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.146 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:22.146 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063070+0000", 2024-09-04T17:48:22.146 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9663000000000001e-05 2024-09-04T17:48:22.146 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.146 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.146 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:22.147 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063079+0000", 2024-09-04T17:48:22.147 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.0929999999999992e-06 2024-09-04T17:48:22.147 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.147 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.147 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:22.147 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210989+0000", 2024-09-04T17:48:22.147 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14790984500000001 2024-09-04T17:48:22.147 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.147 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.147 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:22.147 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211109+0000", 2024-09-04T17:48:22.147 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000120056 2024-09-04T17:48:22.147 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.147 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.148 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:22.148 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211317+0000", 2024-09-04T17:48:22.148 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020797900000000001 2024-09-04T17:48:22.148 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.148 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:22.148 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.148 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.148 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.148 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8847 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3039232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:22.148 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:48:22.148 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 129.01183404099999, 2024-09-04T17:48:22.148 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14859408900000001, 2024-09-04T17:48:22.148 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:22.149 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:22.149 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:22.149 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:22.149 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:22.149 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:22.149 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8847 2024-09-04T17:48:22.149 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.149 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:22.149 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.149 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:22.149 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:48:22.149 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.149 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.149 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.150 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:22.150 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:48:22.150 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.150 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.150 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.150 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:22.150 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063113+0000", 2024-09-04T17:48:22.150 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.204e-06 2024-09-04T17:48:22.150 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.150 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.150 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:22.150 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063116+0000", 2024-09-04T17:48:22.150 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.3330000000000001e-06 2024-09-04T17:48:22.150 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.151 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.151 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:22.151 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063126+0000", 2024-09-04T17:48:22.151 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0166999999999999e-05 2024-09-04T17:48:22.151 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.151 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.151 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:22.151 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063134+0000", 2024-09-04T17:48:22.151 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4329999999999998e-06 2024-09-04T17:48:22.152 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.152 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.152 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:22.152 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211392+0000", 2024-09-04T17:48:22.152 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148258109 2024-09-04T17:48:22.152 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.152 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.152 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:22.152 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211506+0000", 2024-09-04T17:48:22.152 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000113813 2024-09-04T17:48:22.152 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.152 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.153 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:22.153 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211706+0000", 2024-09-04T17:48:22.153 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020002999999999999 2024-09-04T17:48:22.153 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.153 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:22.153 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.153 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.153 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.153 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8848 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:22.153 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:48:22.153 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 129.01180170399999, 2024-09-04T17:48:22.153 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14907967599999999, 2024-09-04T17:48:22.153 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:22.154 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:22.154 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:22.154 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:22.154 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:22.154 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:22.154 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8848 2024-09-04T17:48:22.154 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.154 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:22.154 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.154 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:22.154 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:48:22.154 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.154 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.154 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.155 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:22.155 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:48:22.155 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.155 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.155 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.155 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:22.155 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070589+0000", 2024-09-04T17:48:22.155 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.007445386 2024-09-04T17:48:22.155 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.155 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.156 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:22.156 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070600+0000", 2024-09-04T17:48:22.156 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0346999999999999e-05 2024-09-04T17:48:22.156 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.156 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.156 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:22.156 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070622+0000", 2024-09-04T17:48:22.156 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2289999999999998e-05 2024-09-04T17:48:22.156 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.156 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.156 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:22.156 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070636+0000", 2024-09-04T17:48:22.157 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4093e-05 2024-09-04T17:48:22.157 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.157 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.157 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:22.157 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211815+0000", 2024-09-04T17:48:22.157 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14117874699999999 2024-09-04T17:48:22.157 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.157 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.157 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:22.157 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211983+0000", 2024-09-04T17:48:22.157 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00016852800000000001 2024-09-04T17:48:22.157 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.157 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.158 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:22.158 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.212224+0000", 2024-09-04T17:48:22.158 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024028499999999999 2024-09-04T17:48:22.158 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.158 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:22.158 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.158 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.158 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.158 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8862 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2019328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:22.158 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:48:22.158 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 129.003752236, 2024-09-04T17:48:22.158 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14755236399999999, 2024-09-04T17:48:22.159 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:22.159 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:22.159 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:22.159 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:22.159 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:22.159 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:22.159 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8862 2024-09-04T17:48:22.159 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.160 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:22.160 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.160 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:22.160 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:48:22.160 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.160 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.160 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.160 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:22.160 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:48:22.160 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.161 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.161 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.161 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:22.161 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071194+0000", 2024-09-04T17:48:22.161 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.5000000000000002e-07 2024-09-04T17:48:22.161 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.161 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.161 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:22.162 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071197+0000", 2024-09-04T17:48:22.162 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5900000000000002e-06 2024-09-04T17:48:22.162 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.162 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.162 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:22.162 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071206+0000", 2024-09-04T17:48:22.162 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9700000000000005e-06 2024-09-04T17:48:22.162 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.162 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.162 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:22.162 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071213+0000", 2024-09-04T17:48:22.163 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.9070000000000001e-06 2024-09-04T17:48:22.163 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.163 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.163 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:22.163 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218396+0000", 2024-09-04T17:48:22.163 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147183273 2024-09-04T17:48:22.163 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.163 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.163 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:22.163 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218538+0000", 2024-09-04T17:48:22.163 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014194199999999999 2024-09-04T17:48:22.163 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.163 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.164 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:22.164 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218746+0000", 2024-09-04T17:48:22.164 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000208032 2024-09-04T17:48:22.164 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.164 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:22.164 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.164 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.164 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.164 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8863 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2056192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:22.164 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:48:22.164 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 129.00372392599999, 2024-09-04T17:48:22.164 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14792281099999999, 2024-09-04T17:48:22.164 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:22.165 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:22.165 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:22.165 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:22.165 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:22.165 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:22.165 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8863 2024-09-04T17:48:22.165 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.165 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:22.165 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.165 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:22.165 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:48:22.165 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.165 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.166 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.166 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:22.166 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:48:22.166 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.166 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.166 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.166 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:22.166 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:48:22.166 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1299999999999999e-07 2024-09-04T17:48:22.166 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.166 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.166 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:22.166 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071225+0000", 2024-09-04T17:48:22.166 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6199999999999999e-06 2024-09-04T17:48:22.166 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.167 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.167 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:22.167 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071234+0000", 2024-09-04T17:48:22.167 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8769999999999999e-06 2024-09-04T17:48:22.167 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.167 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.167 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:22.167 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071251+0000", 2024-09-04T17:48:22.167 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.738e-05 2024-09-04T17:48:22.167 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.167 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.167 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:22.167 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218798+0000", 2024-09-04T17:48:22.168 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14754645 2024-09-04T17:48:22.168 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.168 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.168 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:22.168 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218926+0000", 2024-09-04T17:48:22.168 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000128446 2024-09-04T17:48:22.168 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.168 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.168 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:22.169 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219145+0000", 2024-09-04T17:48:22.169 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021832500000000001 2024-09-04T17:48:22.169 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.169 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:22.169 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.169 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.169 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.169 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8864 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2093056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:22.169 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:48:22.169 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 129.00368234999999, 2024-09-04T17:48:22.169 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148277309, 2024-09-04T17:48:22.169 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:22.169 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:22.170 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:22.170 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:22.170 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:22.170 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:22.170 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8864 2024-09-04T17:48:22.170 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.170 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:22.170 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.170 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:22.170 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:48:22.170 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.170 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.170 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.170 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:22.170 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:48:22.171 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.171 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.171 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.171 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:22.171 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071264+0000", 2024-09-04T17:48:22.171 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3e-07 2024-09-04T17:48:22.171 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.171 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.171 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:22.171 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071267+0000", 2024-09-04T17:48:22.172 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0139999999999999e-06 2024-09-04T17:48:22.172 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.172 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.172 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:22.172 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071276+0000", 2024-09-04T17:48:22.172 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1689999999999998e-06 2024-09-04T17:48:22.172 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.173 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.173 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:22.173 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071283+0000", 2024-09-04T17:48:22.173 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2269999999999996e-06 2024-09-04T17:48:22.173 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.173 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.173 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:22.173 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219195+0000", 2024-09-04T17:48:22.173 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147911868 2024-09-04T17:48:22.173 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.174 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.174 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:22.174 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219324+0000", 2024-09-04T17:48:22.174 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00012835900000000001 2024-09-04T17:48:22.174 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.174 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.174 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:22.174 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219541+0000", 2024-09-04T17:48:22.174 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000216942 2024-09-04T17:48:22.174 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.174 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:22.174 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.174 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.175 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.175 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8865 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2109440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:22.175 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:48:22.175 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 129.00365289000001, 2024-09-04T17:48:22.175 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14865747300000001, 2024-09-04T17:48:22.175 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:22.175 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:22.175 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:22.175 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:22.175 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:22.175 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:22.175 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8865 2024-09-04T17:48:22.176 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.176 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:22.176 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.176 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:22.176 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:48:22.176 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.176 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.176 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.176 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:22.176 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:48:22.176 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.176 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.176 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.176 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:22.177 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:48:22.177 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7999999999999995e-07 2024-09-04T17:48:22.177 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.177 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.177 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:22.177 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071296+0000", 2024-09-04T17:48:22.177 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6900000000000001e-06 2024-09-04T17:48:22.177 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.177 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.177 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:22.177 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071305+0000", 2024-09-04T17:48:22.177 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.5499999999999995e-06 2024-09-04T17:48:22.177 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.177 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.178 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:22.178 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071312+0000", 2024-09-04T17:48:22.178 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1559999999999998e-06 2024-09-04T17:48:22.178 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.178 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.178 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:22.178 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219591+0000", 2024-09-04T17:48:22.178 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14827944900000001 2024-09-04T17:48:22.178 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.179 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.179 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:22.179 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219731+0000", 2024-09-04T17:48:22.179 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013931600000000001 2024-09-04T17:48:22.179 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.179 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.179 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:22.179 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219950+0000", 2024-09-04T17:48:22.179 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021963199999999999 2024-09-04T17:48:22.180 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.180 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:22.180 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.180 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.180 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.180 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8866 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2146304~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:22.180 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:48:22.180 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 129.00362465699999, 2024-09-04T17:48:22.180 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14906117899999999, 2024-09-04T17:48:22.181 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:22.181 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:22.181 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:22.181 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:22.181 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:22.181 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:22.181 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8866 2024-09-04T17:48:22.181 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.182 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:22.182 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.182 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:22.182 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:48:22.182 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.182 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.182 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.182 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:22.183 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:48:22.183 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.183 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.183 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.183 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:22.183 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071322+0000", 2024-09-04T17:48:22.183 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7999999999999995e-07 2024-09-04T17:48:22.184 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.184 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.184 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:22.184 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071338+0000", 2024-09-04T17:48:22.184 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5872999999999999e-05 2024-09-04T17:48:22.184 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.185 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.185 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:22.185 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071347+0000", 2024-09-04T17:48:22.185 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4669999999999993e-06 2024-09-04T17:48:22.185 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.185 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.185 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:22.185 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071354+0000", 2024-09-04T17:48:22.185 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2099999999999996e-06 2024-09-04T17:48:22.185 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.185 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.185 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:22.186 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220008+0000", 2024-09-04T17:48:22.186 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148653642 2024-09-04T17:48:22.186 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.186 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.186 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:22.186 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220155+0000", 2024-09-04T17:48:22.186 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014726600000000001 2024-09-04T17:48:22.186 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.186 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.186 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:22.186 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220382+0000", 2024-09-04T17:48:22.186 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00022704100000000001 2024-09-04T17:48:22.186 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.186 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:22.186 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.187 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.187 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.187 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8867 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2183168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:22.187 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:48:22.187 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 129.00358130399999, 2024-09-04T17:48:22.187 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.149445831, 2024-09-04T17:48:22.187 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:22.187 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:22.187 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:22.187 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:22.187 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:22.187 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:22.187 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8867 2024-09-04T17:48:22.187 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.188 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:22.188 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.188 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:22.188 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:48:22.188 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.188 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.188 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.188 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:22.188 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:48:22.188 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.189 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.189 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.189 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:22.189 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078653+0000", 2024-09-04T17:48:22.189 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0072890430000000003 2024-09-04T17:48:22.189 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.189 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.189 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:22.189 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078664+0000", 2024-09-04T17:48:22.189 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0307e-05 2024-09-04T17:48:22.189 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.190 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.190 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:22.190 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078690+0000", 2024-09-04T17:48:22.190 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6780000000000001e-05 2024-09-04T17:48:22.190 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.190 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.190 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:22.190 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078706+0000", 2024-09-04T17:48:22.190 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5676999999999999e-05 2024-09-04T17:48:22.190 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.190 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.190 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:22.190 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220467+0000", 2024-09-04T17:48:22.190 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14176103500000001 2024-09-04T17:48:22.191 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.191 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.191 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:22.191 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220595+0000", 2024-09-04T17:48:22.191 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000127427 2024-09-04T17:48:22.191 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.191 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.191 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:22.191 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220810+0000", 2024-09-04T17:48:22.191 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021556200000000001 2024-09-04T17:48:22.191 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.191 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:22.191 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.191 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.191 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.192 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8881 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2695168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:22.192 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:48:22.192 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 128.99575923500001, 2024-09-04T17:48:22.192 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147543547, 2024-09-04T17:48:22.192 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:22.192 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:22.192 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:22.192 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:22.192 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:22.192 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:22.192 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8881 2024-09-04T17:48:22.192 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.192 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:22.192 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.192 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:22.193 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:48:22.193 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.193 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.193 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.193 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:22.193 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:48:22.193 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.193 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.193 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.193 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:22.193 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079187+0000", 2024-09-04T17:48:22.193 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7700000000000004e-07 2024-09-04T17:48:22.193 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.194 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.194 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:22.194 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079198+0000", 2024-09-04T17:48:22.194 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0849999999999999e-05 2024-09-04T17:48:22.194 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.194 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.194 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:22.194 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079207+0000", 2024-09-04T17:48:22.194 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5599999999999999e-06 2024-09-04T17:48:22.194 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.194 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.194 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:22.194 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079224+0000", 2024-09-04T17:48:22.195 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6387000000000001e-05 2024-09-04T17:48:22.195 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.195 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.195 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:22.195 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226414+0000", 2024-09-04T17:48:22.195 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14718975000000001 2024-09-04T17:48:22.195 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.195 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.195 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:22.195 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226534+0000", 2024-09-04T17:48:22.195 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000120288 2024-09-04T17:48:22.195 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.196 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.196 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:22.196 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226730+0000", 2024-09-04T17:48:22.196 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000196035 2024-09-04T17:48:22.196 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.196 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:22.196 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.196 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.196 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.196 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8882 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2732032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:22.196 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:48:22.196 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 128.99571102100001, 2024-09-04T17:48:22.196 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14795022399999999, 2024-09-04T17:48:22.196 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:22.197 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:22.197 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:22.197 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:22.197 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:22.197 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:22.197 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8882 2024-09-04T17:48:22.197 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.197 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:22.197 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.197 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:22.197 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:48:22.197 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.197 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.197 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.198 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:22.198 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:48:22.198 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.198 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.198 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.198 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:22.198 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086588+0000", 2024-09-04T17:48:22.198 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0073530959999999999 2024-09-04T17:48:22.198 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.198 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.198 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:22.198 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086598+0000", 2024-09-04T17:48:22.198 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0122999999999999e-05 2024-09-04T17:48:22.198 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.199 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.199 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:22.199 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086626+0000", 2024-09-04T17:48:22.199 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.7914000000000001e-05 2024-09-04T17:48:22.199 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.199 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.199 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:22.199 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086641+0000", 2024-09-04T17:48:22.199 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5270000000000001e-05 2024-09-04T17:48:22.199 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.199 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.199 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:22.199 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226801+0000", 2024-09-04T17:48:22.199 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.140160321 2024-09-04T17:48:22.199 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.200 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.200 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:22.200 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226951+0000", 2024-09-04T17:48:22.200 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014965200000000001 2024-09-04T17:48:22.200 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.200 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.200 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:22.200 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.227185+0000", 2024-09-04T17:48:22.200 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00023384799999999999 2024-09-04T17:48:22.200 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.200 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:22.200 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.200 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.200 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.201 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:93470 3.1f 3:f95bc674:::rbd_header.1134878eadad:head [watch reconnect cookie 140018821420464 gen 16] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-04T17:48:22.201 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:57.837081+0000", 2024-09-04T17:48:22.201 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 24.237865138, 2024-09-04T17:48:22.201 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.072023486, 2024-09-04T17:48:22.201 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:22.201 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:22.201 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:22.201 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:22.201 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:22.201 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:22.201 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 93470 2024-09-04T17:48:22.201 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.201 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:22.201 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.202 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:22.202 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:57.837081+0000", 2024-09-04T17:48:22.202 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.202 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.202 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.202 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:22.202 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:57.837081+0000", 2024-09-04T17:48:22.202 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.202 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.202 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.202 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:22.202 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:57.837084+0000", 2024-09-04T17:48:22.202 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.8229999999999997e-06 2024-09-04T17:48:22.202 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.203 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.203 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:22.203 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:57.837098+0000", 2024-09-04T17:48:22.203 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3436e-05 2024-09-04T17:48:22.203 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.203 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.203 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:22.203 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:57.837123+0000", 2024-09-04T17:48:22.203 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5079000000000002e-05 2024-09-04T17:48:22.203 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.203 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.203 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:22.203 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:57.837145+0000", 2024-09-04T17:48:22.203 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.228e-05 2024-09-04T17:48:22.204 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.204 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.204 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:22.204 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:57.837360+0000", 2024-09-04T17:48:22.204 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021464100000000001 2024-09-04T17:48:22.204 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.204 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.204 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for peered", 2024-09-04T17:48:22.204 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:57.837410+0000", 2024-09-04T17:48:22.204 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0235000000000003e-05 2024-09-04T17:48:22.204 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.204 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.204 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:22.204 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.900175+0000", 2024-09-04T17:48:22.205 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0627652830000001 2024-09-04T17:48:22.205 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.205 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.205 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for rw locks", 2024-09-04T17:48:22.205 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.900370+0000", 2024-09-04T17:48:22.205 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019497199999999999 2024-09-04T17:48:22.205 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.205 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.205 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:22.205 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.906957+0000", 2024-09-04T17:48:22.205 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0065866889999999997 2024-09-04T17:48:22.205 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.205 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.205 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:22.206 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.907088+0000", 2024-09-04T17:48:22.206 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013056100000000001 2024-09-04T17:48:22.206 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.206 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.206 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:22.206 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.909104+0000", 2024-09-04T17:48:22.206 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.002016487 2024-09-04T17:48:22.206 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.206 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:22.206 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.206 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.206 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:22.207 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:22.208 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-04T17:48:22.208 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:22.203+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:22.308 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:22.302+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:22.336 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:22.336 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:48:22.336 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:48:22.337 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:22.351 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-04T17:48:22.409 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:48:22.407+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:48:22.510 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:22.507+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:22.520 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:22.520 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:48:22.520 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:48:22.520 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:48:22.520 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:22.535 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-04T17:48:22.611 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:22.606+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:22.661 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:22.661 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:48:22.661 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:48:22.661 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:48:22.661 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.661 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60921 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 262144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:22.662 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822920+0000", 2024-09-04T17:48:22.662 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 68.836694612000002, 2024-09-04T17:48:22.662 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073904394999999998, 2024-09-04T17:48:22.662 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:22.662 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:22.662 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:22.662 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:22.662 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:22.662 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:22.662 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60921 2024-09-04T17:48:22.662 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.662 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:22.663 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.663 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:22.663 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822920+0000", 2024-09-04T17:48:22.663 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.663 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.663 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.663 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:22.663 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822920+0000", 2024-09-04T17:48:22.663 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.663 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.663 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.663 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:22.663 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822920+0000", 2024-09-04T17:48:22.663 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4999999999999999e-07 2024-09-04T17:48:22.664 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.664 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.664 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:22.664 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822921+0000", 2024-09-04T17:48:22.664 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.113e-06 2024-09-04T17:48:22.664 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.664 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.664 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:22.664 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822924+0000", 2024-09-04T17:48:22.664 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.354e-06 2024-09-04T17:48:22.664 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.664 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.664 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:22.665 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822927+0000", 2024-09-04T17:48:22.665 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6199999999999999e-06 2024-09-04T17:48:22.665 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.665 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.665 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:22.665 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896586+0000", 2024-09-04T17:48:22.665 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073658976000000001 2024-09-04T17:48:22.665 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.665 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.665 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:22.665 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896626+0000", 2024-09-04T17:48:22.665 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0484000000000001e-05 2024-09-04T17:48:22.665 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.666 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.666 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:22.666 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896824+0000", 2024-09-04T17:48:22.666 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000197598 2024-09-04T17:48:22.666 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.666 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:22.666 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.666 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.666 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.666 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60922 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:22.666 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822930+0000", 2024-09-04T17:48:22.666 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 68.836684036999998, 2024-09-04T17:48:22.666 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074178872000000007, 2024-09-04T17:48:22.667 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:22.667 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:22.667 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:22.667 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:22.667 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:22.667 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:22.667 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60922 2024-09-04T17:48:22.667 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.667 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:22.667 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.667 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:22.667 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822930+0000", 2024-09-04T17:48:22.667 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.668 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.668 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.668 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:22.668 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822930+0000", 2024-09-04T17:48:22.668 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.668 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.668 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.668 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:22.668 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822930+0000", 2024-09-04T17:48:22.668 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.36e-07 2024-09-04T17:48:22.668 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.668 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.668 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:22.669 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822932+0000", 2024-09-04T17:48:22.669 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.567e-06 2024-09-04T17:48:22.669 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.669 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.669 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:22.669 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822938+0000", 2024-09-04T17:48:22.669 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4840000000000003e-06 2024-09-04T17:48:22.669 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.670 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.670 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:22.670 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822942+0000", 2024-09-04T17:48:22.670 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2039999999999999e-06 2024-09-04T17:48:22.670 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.670 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.670 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:22.670 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896850+0000", 2024-09-04T17:48:22.670 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073907941000000005 2024-09-04T17:48:22.670 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.670 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.670 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:22.671 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896891+0000", 2024-09-04T17:48:22.671 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0964999999999998e-05 2024-09-04T17:48:22.671 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.671 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.671 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:22.671 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897109+0000", 2024-09-04T17:48:22.671 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021847500000000001 2024-09-04T17:48:22.671 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.671 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:22.671 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.671 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.671 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.671 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60923 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 475136~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:22.672 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822949+0000", 2024-09-04T17:48:22.672 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 68.836665746999998, 2024-09-04T17:48:22.672 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074426928000000003, 2024-09-04T17:48:22.672 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:22.672 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:22.672 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:22.672 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:22.672 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:22.672 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:22.672 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60923 2024-09-04T17:48:22.673 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.673 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:22.673 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.673 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:22.674 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822949+0000", 2024-09-04T17:48:22.674 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.674 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.674 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.674 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:22.674 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822949+0000", 2024-09-04T17:48:22.674 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.674 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.674 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.674 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:22.674 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822949+0000", 2024-09-04T17:48:22.674 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7399999999999999e-07 2024-09-04T17:48:22.674 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.674 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.674 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:22.675 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822951+0000", 2024-09-04T17:48:22.675 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1940000000000001e-06 2024-09-04T17:48:22.675 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.675 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.675 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:22.675 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822957+0000", 2024-09-04T17:48:22.675 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.0650000000000004e-06 2024-09-04T17:48:22.675 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.675 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.675 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:22.675 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822960+0000", 2024-09-04T17:48:22.675 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.8009999999999999e-06 2024-09-04T17:48:22.675 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.675 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.676 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:22.676 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897138+0000", 2024-09-04T17:48:22.676 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074177990999999999 2024-09-04T17:48:22.676 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.676 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.676 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:22.676 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897179+0000", 2024-09-04T17:48:22.676 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.1057999999999999e-05 2024-09-04T17:48:22.676 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.676 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.676 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:22.676 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897375+0000", 2024-09-04T17:48:22.676 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019644500000000001 2024-09-04T17:48:22.676 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.677 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:22.677 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.677 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.677 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.677 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60924 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:22.677 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822964+0000", 2024-09-04T17:48:22.677 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 68.836650562000003, 2024-09-04T17:48:22.677 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074531902999999997, 2024-09-04T17:48:22.677 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:22.677 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:22.677 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:22.677 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:22.677 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:22.678 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:22.678 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60924 2024-09-04T17:48:22.678 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.678 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:22.678 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.678 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:22.678 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822964+0000", 2024-09-04T17:48:22.678 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.678 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.678 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.678 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:22.678 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822964+0000", 2024-09-04T17:48:22.678 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.678 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.678 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.679 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:22.679 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822964+0000", 2024-09-04T17:48:22.679 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4999999999999999e-07 2024-09-04T17:48:22.679 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.679 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.679 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:22.679 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822966+0000", 2024-09-04T17:48:22.679 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.917e-06 2024-09-04T17:48:22.679 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.679 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.679 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:22.679 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822969+0000", 2024-09-04T17:48:22.679 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.315e-06 2024-09-04T17:48:22.679 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.680 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.680 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:22.680 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822972+0000", 2024-09-04T17:48:22.680 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.599e-06 2024-09-04T17:48:22.680 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.680 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.680 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:22.680 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897398+0000", 2024-09-04T17:48:22.680 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074426168000000001 2024-09-04T17:48:22.680 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.680 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.680 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:22.680 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897441+0000", 2024-09-04T17:48:22.681 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.3158000000000003e-05 2024-09-04T17:48:22.681 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.681 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.681 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:22.681 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897496+0000", 2024-09-04T17:48:22.681 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4496000000000001e-05 2024-09-04T17:48:22.681 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.681 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:22.681 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.681 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.681 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.681 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60925 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 720896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:22.681 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822975+0000", 2024-09-04T17:48:22.681 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 68.836639126999998, 2024-09-04T17:48:22.682 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074635272000000003, 2024-09-04T17:48:22.682 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:22.682 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:22.682 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:22.682 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:22.682 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:22.682 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:22.682 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60925 2024-09-04T17:48:22.682 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.682 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:22.682 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.682 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:22.682 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822975+0000", 2024-09-04T17:48:22.682 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.683 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.683 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.683 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:22.683 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822975+0000", 2024-09-04T17:48:22.683 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.683 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.683 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.683 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:22.683 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822975+0000", 2024-09-04T17:48:22.683 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.34e-07 2024-09-04T17:48:22.683 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.683 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.683 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:22.683 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822976+0000", 2024-09-04T17:48:22.684 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0359999999999999e-06 2024-09-04T17:48:22.684 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.684 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.684 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:22.684 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822980+0000", 2024-09-04T17:48:22.684 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.242e-06 2024-09-04T17:48:22.684 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.684 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.684 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:22.684 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822983+0000", 2024-09-04T17:48:22.684 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.8830000000000002e-06 2024-09-04T17:48:22.684 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.684 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.685 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:22.685 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897519+0000", 2024-09-04T17:48:22.685 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074536366000000007 2024-09-04T17:48:22.685 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.685 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.685 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:22.685 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897560+0000", 2024-09-04T17:48:22.685 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0677999999999999e-05 2024-09-04T17:48:22.685 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.685 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.685 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:22.685 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897610+0000", 2024-09-04T17:48:22.685 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0832999999999999e-05 2024-09-04T17:48:22.686 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.686 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:22.686 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.686 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.686 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.686 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60926 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 860160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:22.686 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822986+0000", 2024-09-04T17:48:22.686 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 68.836628528999995, 2024-09-04T17:48:22.686 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074736413000000002, 2024-09-04T17:48:22.686 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:22.686 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:22.686 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:22.686 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:22.686 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:22.687 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:22.687 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60926 2024-09-04T17:48:22.687 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.687 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:22.687 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.687 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:22.687 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822986+0000", 2024-09-04T17:48:22.687 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.687 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.687 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.687 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:22.687 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822986+0000", 2024-09-04T17:48:22.687 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.687 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.688 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.688 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:22.688 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822986+0000", 2024-09-04T17:48:22.688 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5600000000000002e-07 2024-09-04T17:48:22.688 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.688 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.688 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:22.688 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822987+0000", 2024-09-04T17:48:22.688 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.082e-06 2024-09-04T17:48:22.688 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.688 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.688 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:22.689 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822990+0000", 2024-09-04T17:48:22.689 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.2219999999999998e-06 2024-09-04T17:48:22.689 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.689 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.689 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:22.689 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822994+0000", 2024-09-04T17:48:22.689 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.19e-06 2024-09-04T17:48:22.689 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.689 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.689 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:22.689 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897632+0000", 2024-09-04T17:48:22.689 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074638205999999999 2024-09-04T17:48:22.689 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.690 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.690 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:22.690 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897672+0000", 2024-09-04T17:48:22.690 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.9873999999999998e-05 2024-09-04T17:48:22.690 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.690 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.690 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:22.690 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897722+0000", 2024-09-04T17:48:22.690 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0583e-05 2024-09-04T17:48:22.690 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.690 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:22.690 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.690 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.690 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.691 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60927 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 966656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:22.691 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823000+0000", 2024-09-04T17:48:22.691 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 68.836614346999994, 2024-09-04T17:48:22.691 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074834036000000007, 2024-09-04T17:48:22.691 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:22.691 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:22.691 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:22.691 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:22.691 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:22.691 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:22.691 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60927 2024-09-04T17:48:22.691 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.691 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:22.691 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.692 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:22.692 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823000+0000", 2024-09-04T17:48:22.692 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.692 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.692 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.692 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:22.692 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823000+0000", 2024-09-04T17:48:22.692 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.692 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.692 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.692 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:22.692 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823000+0000", 2024-09-04T17:48:22.692 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4099999999999999e-07 2024-09-04T17:48:22.693 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.693 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.693 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:22.693 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823003+0000", 2024-09-04T17:48:22.693 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0810000000000001e-06 2024-09-04T17:48:22.693 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.693 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.693 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:22.693 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823011+0000", 2024-09-04T17:48:22.693 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7090000000000008e-06 2024-09-04T17:48:22.693 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.693 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.693 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:22.693 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823017+0000", 2024-09-04T17:48:22.694 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.3890000000000004e-06 2024-09-04T17:48:22.694 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.694 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.694 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:22.694 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897745+0000", 2024-09-04T17:48:22.694 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074728252999999994 2024-09-04T17:48:22.694 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.694 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.694 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:22.694 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897785+0000", 2024-09-04T17:48:22.694 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.9830999999999999e-05 2024-09-04T17:48:22.694 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.694 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.694 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:22.695 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897834+0000", 2024-09-04T17:48:22.695 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9231999999999999e-05 2024-09-04T17:48:22.695 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.695 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:22.695 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.695 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.695 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.695 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60928 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:22.695 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823024+0000", 2024-09-04T17:48:22.695 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 68.836590416000007, 2024-09-04T17:48:22.695 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075067479000000006, 2024-09-04T17:48:22.695 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:22.695 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:22.696 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:22.696 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:22.696 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:22.696 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:22.696 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60928 2024-09-04T17:48:22.696 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.696 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:22.696 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.696 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:22.696 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823024+0000", 2024-09-04T17:48:22.696 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.696 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.696 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.697 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:22.697 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823024+0000", 2024-09-04T17:48:22.697 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.697 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.697 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.697 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:22.697 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823024+0000", 2024-09-04T17:48:22.697 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6499999999999999e-07 2024-09-04T17:48:22.697 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.697 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.697 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:22.697 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823027+0000", 2024-09-04T17:48:22.698 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.3099999999999999e-06 2024-09-04T17:48:22.698 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.698 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.698 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:22.698 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823033+0000", 2024-09-04T17:48:22.698 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7429999999999996e-06 2024-09-04T17:48:22.698 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.698 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.698 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:22.698 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823038+0000", 2024-09-04T17:48:22.698 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.561e-06 2024-09-04T17:48:22.698 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.698 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.698 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:22.699 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897855+0000", 2024-09-04T17:48:22.699 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074816740000000007 2024-09-04T17:48:22.699 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.699 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.699 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:22.699 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897894+0000", 2024-09-04T17:48:22.699 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.9085999999999998e-05 2024-09-04T17:48:22.699 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.699 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.699 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:22.699 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898091+0000", 2024-09-04T17:48:22.699 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000197574 2024-09-04T17:48:22.699 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.699 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:22.699 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.700 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.700 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.700 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60929 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1212416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:22.700 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823042+0000", 2024-09-04T17:48:22.700 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 68.836571973999995, 2024-09-04T17:48:22.700 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075301251, 2024-09-04T17:48:22.700 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:22.700 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:22.700 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:22.700 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:22.700 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:22.700 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:22.700 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60929 2024-09-04T17:48:22.700 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.701 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:22.701 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.701 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:22.701 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823042+0000", 2024-09-04T17:48:22.701 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.701 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.701 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.701 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:22.701 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823042+0000", 2024-09-04T17:48:22.701 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.701 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.701 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.701 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:22.701 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823043+0000", 2024-09-04T17:48:22.702 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.1100000000000002e-07 2024-09-04T17:48:22.702 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.702 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.702 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:22.702 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823045+0000", 2024-09-04T17:48:22.702 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.933e-06 2024-09-04T17:48:22.702 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.702 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.702 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:22.702 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823051+0000", 2024-09-04T17:48:22.702 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.9569999999999999e-06 2024-09-04T17:48:22.702 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.702 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.702 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:22.702 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823055+0000", 2024-09-04T17:48:22.703 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4479999999999996e-06 2024-09-04T17:48:22.703 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.703 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.703 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:22.703 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898117+0000", 2024-09-04T17:48:22.703 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075062344000000003 2024-09-04T17:48:22.703 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.703 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.703 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:22.703 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898157+0000", 2024-09-04T17:48:22.703 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0095999999999997e-05 2024-09-04T17:48:22.703 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.703 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.703 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:22.703 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898344+0000", 2024-09-04T17:48:22.704 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000186162 2024-09-04T17:48:22.704 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.704 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:22.704 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.704 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.704 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.704 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60930 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1351680~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:22.704 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823061+0000", 2024-09-04T17:48:22.704 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 68.836553094999999, 2024-09-04T17:48:22.704 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075545507999999997, 2024-09-04T17:48:22.704 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:22.704 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:22.704 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:22.705 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:22.705 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:22.705 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:22.705 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60930 2024-09-04T17:48:22.705 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.705 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:22.705 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.705 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:22.705 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823061+0000", 2024-09-04T17:48:22.705 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.705 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.705 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.705 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:22.705 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823061+0000", 2024-09-04T17:48:22.706 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.706 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.706 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.706 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:22.706 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823378+0000", 2024-09-04T17:48:22.706 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00031631299999999998 2024-09-04T17:48:22.706 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.706 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.706 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:22.706 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823381+0000", 2024-09-04T17:48:22.706 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4020000000000002e-06 2024-09-04T17:48:22.706 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.706 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.707 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:22.707 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823389+0000", 2024-09-04T17:48:22.707 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.1429999999999998e-06 2024-09-04T17:48:22.707 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.707 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.707 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:22.707 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823395+0000", 2024-09-04T17:48:22.707 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.0059999999999998e-06 2024-09-04T17:48:22.707 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.707 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.707 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:22.707 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898368+0000", 2024-09-04T17:48:22.707 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074973032999999994 2024-09-04T17:48:22.707 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.707 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.707 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:22.708 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898410+0000", 2024-09-04T17:48:22.708 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2243000000000002e-05 2024-09-04T17:48:22.708 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.708 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.708 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:22.708 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898607+0000", 2024-09-04T17:48:22.708 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019636799999999999 2024-09-04T17:48:22.708 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.708 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:22.708 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.708 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.709 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.709 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60931 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1458176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:22.709 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823407+0000", 2024-09-04T17:48:22.709 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 68.836207021000007, 2024-09-04T17:48:22.709 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075460334000000004, 2024-09-04T17:48:22.709 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:22.709 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:22.709 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:22.709 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:22.709 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:22.709 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:22.709 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60931 2024-09-04T17:48:22.709 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.710 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:22.710 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.710 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:22.710 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823407+0000", 2024-09-04T17:48:22.710 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.710 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.710 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.710 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:22.710 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823407+0000", 2024-09-04T17:48:22.710 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.710 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.710 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.710 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:22.710 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823408+0000", 2024-09-04T17:48:22.710 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9999999999999998e-07 2024-09-04T17:48:22.711 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.711 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.711 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:22.711 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823415+0000", 2024-09-04T17:48:22.711 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.5510000000000002e-06 2024-09-04T17:48:22.711 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.711 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.711 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:22.711 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823423+0000", 2024-09-04T17:48:22.711 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2760000000000003e-06 2024-09-04T17:48:22.711 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.711 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.711 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:22.711 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823426+0000", 2024-09-04T17:48:22.712 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.2839999999999998e-06 2024-09-04T17:48:22.712 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.712 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.712 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:22.712 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898631+0000", 2024-09-04T17:48:22.712 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075205045999999998 2024-09-04T17:48:22.712 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.712 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.712 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:22.712 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898672+0000", 2024-09-04T17:48:22.712 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0614000000000001e-05 2024-09-04T17:48:22.712 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.712 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.712 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:22.712 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898868+0000", 2024-09-04T17:48:22.713 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019606300000000001 2024-09-04T17:48:22.713 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.713 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:22.713 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.713 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.713 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.713 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60932 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1564672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:22.713 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823434+0000", 2024-09-04T17:48:22.713 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 68.836180721999995, 2024-09-04T17:48:22.713 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075693393999999997, 2024-09-04T17:48:22.713 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:22.713 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:22.713 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:22.713 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:22.714 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:22.714 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:22.714 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60932 2024-09-04T17:48:22.714 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.714 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:22.714 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.714 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:22.714 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823434+0000", 2024-09-04T17:48:22.714 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.714 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.714 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.714 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:22.714 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823434+0000", 2024-09-04T17:48:22.714 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.714 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.715 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.715 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:22.715 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.830290+0000", 2024-09-04T17:48:22.715 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0068563039999999997 2024-09-04T17:48:22.715 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.715 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.715 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:22.715 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.830297+0000", 2024-09-04T17:48:22.715 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4760000000000001e-06 2024-09-04T17:48:22.715 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.715 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.715 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:22.715 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.830308+0000", 2024-09-04T17:48:22.716 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0927999999999999e-05 2024-09-04T17:48:22.716 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.716 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.716 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:22.716 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.830317+0000", 2024-09-04T17:48:22.716 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7280000000000001e-06 2024-09-04T17:48:22.716 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.716 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.716 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:22.716 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898892+0000", 2024-09-04T17:48:22.716 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.068574722000000005 2024-09-04T17:48:22.716 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.716 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.717 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:22.717 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898934+0000", 2024-09-04T17:48:22.717 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2249000000000002e-05 2024-09-04T17:48:22.717 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.717 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.717 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:22.717 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.899127+0000", 2024-09-04T17:48:22.717 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019298699999999999 2024-09-04T17:48:22.717 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.717 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:22.717 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.717 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.717 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.718 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61182 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 868352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:22.718 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839249+0000", 2024-09-04T17:48:22.718 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 68.820364830000003, 2024-09-04T17:48:22.718 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073680151999999999, 2024-09-04T17:48:22.718 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:22.718 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:22.718 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:22.718 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:22.718 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:22.718 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:22.718 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61182 2024-09-04T17:48:22.718 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.718 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:22.718 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.718 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:22.719 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839249+0000", 2024-09-04T17:48:22.719 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.719 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.719 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.719 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:22.719 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839249+0000", 2024-09-04T17:48:22.719 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.719 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.719 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.719 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:22.719 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839250+0000", 2024-09-04T17:48:22.719 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4999999999999998e-07 2024-09-04T17:48:22.719 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.720 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.720 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:22.720 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839257+0000", 2024-09-04T17:48:22.720 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7700000000000004e-06 2024-09-04T17:48:22.720 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.720 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.720 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:22.720 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839263+0000", 2024-09-04T17:48:22.720 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.246e-06 2024-09-04T17:48:22.720 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.720 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.720 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:22.720 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839268+0000", 2024-09-04T17:48:22.720 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.814e-06 2024-09-04T17:48:22.721 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.721 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.721 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:22.721 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.912831+0000", 2024-09-04T17:48:22.721 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073563294000000001 2024-09-04T17:48:22.721 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.721 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.721 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:22.721 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.912876+0000", 2024-09-04T17:48:22.721 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4742e-05 2024-09-04T17:48:22.721 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.721 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.721 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:22.721 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.912930+0000", 2024-09-04T17:48:22.722 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.3936e-05 2024-09-04T17:48:22.722 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.722 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:22.722 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.722 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.722 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.722 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61183 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:22.722 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839275+0000", 2024-09-04T17:48:22.722 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 68.820339637000004, 2024-09-04T17:48:22.722 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073781886000000005, 2024-09-04T17:48:22.722 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:22.722 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:22.722 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:22.722 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:22.723 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:22.723 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:22.723 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61183 2024-09-04T17:48:22.723 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.723 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:22.723 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.723 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:22.723 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839275+0000", 2024-09-04T17:48:22.723 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.723 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.723 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.723 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:22.723 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839275+0000", 2024-09-04T17:48:22.723 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.724 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.724 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.724 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:22.724 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839275+0000", 2024-09-04T17:48:22.724 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6899999999999997e-07 2024-09-04T17:48:22.724 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.724 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.724 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:22.724 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839277+0000", 2024-09-04T17:48:22.724 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9530000000000002e-06 2024-09-04T17:48:22.724 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.724 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.724 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:22.725 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839283+0000", 2024-09-04T17:48:22.725 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.9329999999999998e-06 2024-09-04T17:48:22.725 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.725 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.725 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:22.725 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839288+0000", 2024-09-04T17:48:22.725 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4100000000000001e-06 2024-09-04T17:48:22.725 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.725 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.725 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:22.725 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.912959+0000", 2024-09-04T17:48:22.725 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073671511999999995 2024-09-04T17:48:22.725 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.726 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.726 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:22.726 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913001+0000", 2024-09-04T17:48:22.726 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.1984999999999999e-05 2024-09-04T17:48:22.726 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.726 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.726 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:22.726 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913057+0000", 2024-09-04T17:48:22.726 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.5523999999999999e-05 2024-09-04T17:48:22.726 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.726 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:22.726 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.726 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.727 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.727 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61184 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1114112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:22.727 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839294+0000", 2024-09-04T17:48:22.727 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 68.820320425000006, 2024-09-04T17:48:22.727 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073995263000000006, 2024-09-04T17:48:22.727 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:22.727 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:22.727 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:22.727 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:22.727 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:22.727 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:22.727 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61184 2024-09-04T17:48:22.727 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.727 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:22.728 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.728 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:22.728 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839294+0000", 2024-09-04T17:48:22.728 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.728 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.728 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.728 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:22.728 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839294+0000", 2024-09-04T17:48:22.728 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.728 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.728 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.728 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:22.728 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839294+0000", 2024-09-04T17:48:22.729 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.77e-07 2024-09-04T17:48:22.729 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.729 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.729 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:22.729 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839296+0000", 2024-09-04T17:48:22.729 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9999999999999999e-06 2024-09-04T17:48:22.729 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.729 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.729 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:22.729 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839302+0000", 2024-09-04T17:48:22.729 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7230000000000001e-06 2024-09-04T17:48:22.729 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.729 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.729 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:22.730 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839314+0000", 2024-09-04T17:48:22.730 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1826e-05 2024-09-04T17:48:22.730 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.730 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.730 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:22.730 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913084+0000", 2024-09-04T17:48:22.730 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073770215 2024-09-04T17:48:22.730 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.730 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.730 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:22.730 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913127+0000", 2024-09-04T17:48:22.730 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2982999999999998e-05 2024-09-04T17:48:22.730 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.731 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.731 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:22.731 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913289+0000", 2024-09-04T17:48:22.731 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000162139 2024-09-04T17:48:22.731 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.731 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:22.731 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.731 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.731 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.731 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61185 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1220608~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:22.731 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839321+0000", 2024-09-04T17:48:22.731 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 68.820292957999996, 2024-09-04T17:48:22.731 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074206062000000003, 2024-09-04T17:48:22.732 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:22.732 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:22.732 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:22.732 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:22.732 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:22.732 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:22.732 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61185 2024-09-04T17:48:22.732 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.732 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:22.732 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.732 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:22.732 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839321+0000", 2024-09-04T17:48:22.732 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.732 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.733 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.733 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:22.733 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839321+0000", 2024-09-04T17:48:22.733 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.733 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.733 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.733 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:22.733 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839322+0000", 2024-09-04T17:48:22.733 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.5400000000000001e-07 2024-09-04T17:48:22.733 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.734 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.734 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:22.734 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839324+0000", 2024-09-04T17:48:22.734 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2539999999999999e-06 2024-09-04T17:48:22.734 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.734 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.734 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:22.734 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839331+0000", 2024-09-04T17:48:22.734 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2139999999999999e-06 2024-09-04T17:48:22.734 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.734 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.734 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:22.735 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839336+0000", 2024-09-04T17:48:22.735 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7029999999999997e-06 2024-09-04T17:48:22.735 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.735 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.735 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:22.735 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913320+0000", 2024-09-04T17:48:22.735 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073984016 2024-09-04T17:48:22.735 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.735 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.735 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:22.735 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913366+0000", 2024-09-04T17:48:22.735 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.5704000000000002e-05 2024-09-04T17:48:22.735 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.736 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.736 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:22.736 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913527+0000", 2024-09-04T17:48:22.736 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00016161699999999999 2024-09-04T17:48:22.736 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.736 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:22.736 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.736 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.736 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.736 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61186 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1359872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:22.736 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839344+0000", 2024-09-04T17:48:22.736 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 68.820270616000002, 2024-09-04T17:48:22.736 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074450673999999994, 2024-09-04T17:48:22.736 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:22.737 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:22.737 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:22.737 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:22.737 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:22.737 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:22.737 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61186 2024-09-04T17:48:22.737 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.737 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:22.737 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.737 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:22.737 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839344+0000", 2024-09-04T17:48:22.737 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.737 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.738 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.738 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:22.738 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839344+0000", 2024-09-04T17:48:22.738 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.738 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.738 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.738 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:22.738 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839344+0000", 2024-09-04T17:48:22.738 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4799999999999999e-07 2024-09-04T17:48:22.738 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.738 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.738 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:22.738 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839346+0000", 2024-09-04T17:48:22.739 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0889999999999998e-06 2024-09-04T17:48:22.739 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.739 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.739 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:22.739 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839352+0000", 2024-09-04T17:48:22.739 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6389999999999997e-06 2024-09-04T17:48:22.739 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.739 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.739 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:22.739 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839356+0000", 2024-09-04T17:48:22.739 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.233e-06 2024-09-04T17:48:22.739 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.739 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.740 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:22.740 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913557+0000", 2024-09-04T17:48:22.740 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074200944000000005 2024-09-04T17:48:22.740 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.740 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.740 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:22.740 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913601+0000", 2024-09-04T17:48:22.740 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4350000000000001e-05 2024-09-04T17:48:22.740 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.740 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.740 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:22.740 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913794+0000", 2024-09-04T17:48:22.740 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019297100000000001 2024-09-04T17:48:22.741 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.741 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:22.741 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.741 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.741 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.741 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61187 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1466368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:22.741 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839364+0000", 2024-09-04T17:48:22.741 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 68.820250767000005, 2024-09-04T17:48:22.741 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074692334999999999, 2024-09-04T17:48:22.741 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:22.741 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:22.741 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:22.741 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:22.742 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:22.742 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:22.742 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61187 2024-09-04T17:48:22.742 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.742 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:22.742 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.742 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:22.742 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839364+0000", 2024-09-04T17:48:22.742 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.742 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.742 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.742 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:22.742 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839364+0000", 2024-09-04T17:48:22.742 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.743 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.743 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.743 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:22.743 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839364+0000", 2024-09-04T17:48:22.743 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.75e-07 2024-09-04T17:48:22.743 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.743 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.743 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:22.743 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839366+0000", 2024-09-04T17:48:22.743 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0990000000000001e-06 2024-09-04T17:48:22.743 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.743 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.743 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:22.743 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839372+0000", 2024-09-04T17:48:22.744 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.2609999999999999e-06 2024-09-04T17:48:22.744 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.744 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.744 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:22.744 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839389+0000", 2024-09-04T17:48:22.744 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6489000000000001e-05 2024-09-04T17:48:22.744 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.744 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.744 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:22.744 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913824+0000", 2024-09-04T17:48:22.744 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074434811000000004 2024-09-04T17:48:22.744 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.745 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.745 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:22.745 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913868+0000", 2024-09-04T17:48:22.745 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4768000000000002e-05 2024-09-04T17:48:22.745 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.745 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.745 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:22.745 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914056+0000", 2024-09-04T17:48:22.745 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00018743199999999999 2024-09-04T17:48:22.745 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.745 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:22.746 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.746 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.746 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.746 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61188 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1605632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:22.746 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839399+0000", 2024-09-04T17:48:22.746 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 68.820214871000005, 2024-09-04T17:48:22.746 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074975095000000005, 2024-09-04T17:48:22.746 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:22.746 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:22.746 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:22.747 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:22.747 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:22.747 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:22.747 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61188 2024-09-04T17:48:22.747 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.747 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:22.747 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.747 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:22.747 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839399+0000", 2024-09-04T17:48:22.747 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.747 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.747 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.747 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:22.748 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839399+0000", 2024-09-04T17:48:22.748 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.748 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.748 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.748 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:22.748 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839400+0000", 2024-09-04T17:48:22.748 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0100000000000005e-07 2024-09-04T17:48:22.748 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.748 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.748 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:22.748 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839402+0000", 2024-09-04T17:48:22.748 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.491e-06 2024-09-04T17:48:22.748 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.749 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.749 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:22.749 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839409+0000", 2024-09-04T17:48:22.749 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.0450000000000003e-06 2024-09-04T17:48:22.749 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.749 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.749 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:22.749 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839414+0000", 2024-09-04T17:48:22.749 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.87e-06 2024-09-04T17:48:22.749 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.749 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.749 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:22.749 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914086+0000", 2024-09-04T17:48:22.749 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074671596000000007 2024-09-04T17:48:22.750 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.750 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.750 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:22.750 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914130+0000", 2024-09-04T17:48:22.750 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4579999999999997e-05 2024-09-04T17:48:22.750 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.750 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.750 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:22.750 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914375+0000", 2024-09-04T17:48:22.750 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024401199999999999 2024-09-04T17:48:22.750 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.750 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:22.750 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.751 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.751 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.751 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61189 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1712128~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:22.751 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839424+0000", 2024-09-04T17:48:22.751 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 68.820190670000002, 2024-09-04T17:48:22.751 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075247680999999997, 2024-09-04T17:48:22.751 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:22.751 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:22.751 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:22.751 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:22.751 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:22.751 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:22.751 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61189 2024-09-04T17:48:22.751 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.752 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:22.752 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.752 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:22.752 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839424+0000", 2024-09-04T17:48:22.752 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.752 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.752 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.752 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:22.752 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839424+0000", 2024-09-04T17:48:22.752 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:22.752 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.752 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.752 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:22.753 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.846320+0000", 2024-09-04T17:48:22.753 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0068965210000000001 2024-09-04T17:48:22.753 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.753 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.753 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:22.753 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.846328+0000", 2024-09-04T17:48:22.753 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.8189999999999992e-06 2024-09-04T17:48:22.753 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.753 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.753 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:22.753 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.846341+0000", 2024-09-04T17:48:22.753 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3338e-05 2024-09-04T17:48:22.753 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.753 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.754 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:22.754 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.846352+0000", 2024-09-04T17:48:22.754 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0479000000000001e-05 2024-09-04T17:48:22.754 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.754 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.754 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:22.754 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914405+0000", 2024-09-04T17:48:22.754 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.068053049000000004 2024-09-04T17:48:22.754 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.754 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.754 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:22.754 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914451+0000", 2024-09-04T17:48:22.754 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6545999999999998e-05 2024-09-04T17:48:22.755 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:22.755 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:22.755 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:22.755 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914671+0000", 2024-09-04T17:48:22.755 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000219929 2024-09-04T17:48:22.755 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.755 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:22.755 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.755 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:22.755 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:22.755 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:22.757 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-04T17:48:22.757 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:48:22.751+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:48:22.839 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is set 2024-09-04T17:48:22.857 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:48:22.854+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:48:22.902 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:22.903 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:48:22.903 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:48:22.903 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:22.921 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-04T17:48:22.959 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:22.954+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:23.059 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:23.054+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:23.092 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:23.092 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:48:23.092 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:48:23.093 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:48:23.093 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:23.107 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-04T17:48:23.160 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:23.155+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:23.227 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:23.228 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:48:23.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:48:23.228 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:48:23.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.228 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:27242 3.b 3.34414deb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-04T17:48:23.228 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:28.402347+0000", 2024-09-04T17:48:23.228 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 114.822611083, 2024-09-04T17:48:23.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.93265329299999999, 2024-09-04T17:48:23.228 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:23.228 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:23.228 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-04T17:48:23.228 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:23.228 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:23.229 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:23.229 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 27242 2024-09-04T17:48:23.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.229 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:23.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.229 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:23.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402347+0000", 2024-09-04T17:48:23.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:23.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.229 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:23.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402347+0000", 2024-09-04T17:48:23.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:23.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:23.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402349+0000", 2024-09-04T17:48:23.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.471e-06 2024-09-04T17:48:23.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:23.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402355+0000", 2024-09-04T17:48:23.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8399999999999997e-06 2024-09-04T17:48:23.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:23.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402364+0000", 2024-09-04T17:48:23.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1269999999999992e-06 2024-09-04T17:48:23.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:23.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402373+0000", 2024-09-04T17:48:23.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7909999999999993e-06 2024-09-04T17:48:23.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:23.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402460+0000", 2024-09-04T17:48:23.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6686000000000003e-05 2024-09-04T17:48:23.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-04T17:48:23.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402473+0000", 2024-09-04T17:48:23.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3281e-05 2024-09-04T17:48:23.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:23.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:29.334989+0000", 2024-09-04T17:48:23.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.93251549099999997 2024-09-04T17:48:23.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:23.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:29.335000+0000", 2024-09-04T17:48:23.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1606e-05 2024-09-04T17:48:23.233 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:23.233 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:23.233 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:23.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.233 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:33552 3.7 3:e79ad1db:::rbd_data.11342e8f3384.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-04T17:48:23.233 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:35.999748+0000", 2024-09-04T17:48:23.233 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 107.225210033, 2024-09-04T17:48:23.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045683823999999998, 2024-09-04T17:48:23.233 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:23.233 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:23.233 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:23.234 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:23.234 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:23.234 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:23.234 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 33552 2024-09-04T17:48:23.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.234 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:23.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:23.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999748+0000", 2024-09-04T17:48:23.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:23.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:23.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999748+0000", 2024-09-04T17:48:23.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:23.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:23.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999750+0000", 2024-09-04T17:48:23.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1299999999999999e-06 2024-09-04T17:48:23.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:23.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999759+0000", 2024-09-04T17:48:23.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4649999999999994e-06 2024-09-04T17:48:23.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:23.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999778+0000", 2024-09-04T17:48:23.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8833000000000001e-05 2024-09-04T17:48:23.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:23.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999786+0000", 2024-09-04T17:48:23.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7369999999999995e-06 2024-09-04T17:48:23.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:23.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999856+0000", 2024-09-04T17:48:23.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9247999999999998e-05 2024-09-04T17:48:23.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:23.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999973+0000", 2024-09-04T17:48:23.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000117921 2024-09-04T17:48:23.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 7", 2024-09-04T17:48:23.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.005955+0000", 2024-09-04T17:48:23.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059820560000000004 2024-09-04T17:48:23.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:23.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.045297+0000", 2024-09-04T17:48:23.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.039341650999999998 2024-09-04T17:48:23.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:23.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.045343+0000", 2024-09-04T17:48:23.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5899000000000002e-05 2024-09-04T17:48:23.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.239 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:23.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.045388+0000", 2024-09-04T17:48:23.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4601999999999997e-05 2024-09-04T17:48:23.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.239 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:23.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.045432+0000", 2024-09-04T17:48:23.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4282000000000001e-05 2024-09-04T17:48:23.239 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:23.239 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:23.239 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:23.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.239 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:37357 3.15 3:a897312a:::rbd_data.1134c5a43465.000000000000001f:head [stat,write 62464~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-04T17:48:23.239 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:39.616724+0000", 2024-09-04T17:48:23.240 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 103.60823462499999, 2024-09-04T17:48:23.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054965854000000001, 2024-09-04T17:48:23.240 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:23.240 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:23.240 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:23.240 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:23.240 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:23.240 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:23.240 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 37357 2024-09-04T17:48:23.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.240 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:23.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:23.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616724+0000", 2024-09-04T17:48:23.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:23.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:23.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616724+0000", 2024-09-04T17:48:23.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:23.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:23.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616725+0000", 2024-09-04T17:48:23.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.815e-06 2024-09-04T17:48:23.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:23.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616739+0000", 2024-09-04T17:48:23.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3723000000000001e-05 2024-09-04T17:48:23.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:23.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616755+0000", 2024-09-04T17:48:23.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5964000000000002e-05 2024-09-04T17:48:23.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:23.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616763+0000", 2024-09-04T17:48:23.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3869999999999995e-06 2024-09-04T17:48:23.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:23.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616825+0000", 2024-09-04T17:48:23.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1854999999999996e-05 2024-09-04T17:48:23.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:23.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616895+0000", 2024-09-04T17:48:23.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9597999999999993e-05 2024-09-04T17:48:23.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-04T17:48:23.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616991+0000", 2024-09-04T17:48:23.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5828000000000005e-05 2024-09-04T17:48:23.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.244 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:23.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.617659+0000", 2024-09-04T17:48:23.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00066821600000000001 2024-09-04T17:48:23.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.244 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:23.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.671482+0000", 2024-09-04T17:48:23.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053823533999999999 2024-09-04T17:48:23.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.245 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:23.245 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.671579+0000", 2024-09-04T17:48:23.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6657999999999995e-05 2024-09-04T17:48:23.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.245 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:23.245 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.671689+0000", 2024-09-04T17:48:23.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000110276 2024-09-04T17:48:23.245 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:23.245 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:23.245 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:23.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.246 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:43873 3.15 3:abd207ee:::rbd_data.113410ddd206.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:23.246 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:49.932940+0000", 2024-09-04T17:48:23.246 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 93.292018072000005, 2024-09-04T17:48:23.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042740514, 2024-09-04T17:48:23.246 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:23.246 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:23.246 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:23.246 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:23.246 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:23.246 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:23.246 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 43873 2024-09-04T17:48:23.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.247 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:23.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:23.247 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.932940+0000", 2024-09-04T17:48:23.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:23.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:23.247 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.932940+0000", 2024-09-04T17:48:23.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:23.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:23.247 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.932941+0000", 2024-09-04T17:48:23.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.404e-06 2024-09-04T17:48:23.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:23.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.933065+0000", 2024-09-04T17:48:23.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012361199999999999 2024-09-04T17:48:23.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:23.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.933078+0000", 2024-09-04T17:48:23.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3182e-05 2024-09-04T17:48:23.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:23.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.933085+0000", 2024-09-04T17:48:23.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4500000000000001e-06 2024-09-04T17:48:23.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:23.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.933134+0000", 2024-09-04T17:48:23.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9150999999999997e-05 2024-09-04T17:48:23.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:23.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.933242+0000", 2024-09-04T17:48:23.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010781 2024-09-04T17:48:23.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-04T17:48:23.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.938358+0000", 2024-09-04T17:48:23.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051160349999999997 2024-09-04T17:48:23.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:23.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.975343+0000", 2024-09-04T17:48:23.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036985663000000002 2024-09-04T17:48:23.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:23.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.975392+0000", 2024-09-04T17:48:23.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9044e-05 2024-09-04T17:48:23.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:23.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.975548+0000", 2024-09-04T17:48:23.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015572900000000001 2024-09-04T17:48:23.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:23.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.975681+0000", 2024-09-04T17:48:23.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013243400000000001 2024-09-04T17:48:23.251 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:23.251 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:23.252 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:23.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.252 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:46649 3.7 3:e0749ed5:::rbd_data.1134326b938e.000000000000014f:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-04T17:48:23.252 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:54.114730+0000", 2024-09-04T17:48:23.252 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 89.110227940000001, 2024-09-04T17:48:23.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032972972000000003, 2024-09-04T17:48:23.252 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:23.252 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:23.252 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:23.252 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:23.252 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:23.252 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:23.253 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 46649 2024-09-04T17:48:23.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.253 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:23.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:23.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114730+0000", 2024-09-04T17:48:23.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:23.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:23.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114730+0000", 2024-09-04T17:48:23.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:23.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:23.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114731+0000", 2024-09-04T17:48:23.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.147e-06 2024-09-04T17:48:23.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:23.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114737+0000", 2024-09-04T17:48:23.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6380000000000001e-06 2024-09-04T17:48:23.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:23.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114751+0000", 2024-09-04T17:48:23.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4012999999999999e-05 2024-09-04T17:48:23.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:23.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114758+0000", 2024-09-04T17:48:23.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1539999999999996e-06 2024-09-04T17:48:23.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:23.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114876+0000", 2024-09-04T17:48:23.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000117421 2024-09-04T17:48:23.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:23.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.115173+0000", 2024-09-04T17:48:23.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00029750799999999998 2024-09-04T17:48:23.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 7", 2024-09-04T17:48:23.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.120273+0000", 2024-09-04T17:48:23.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051002900000000004 2024-09-04T17:48:23.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:23.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.147571+0000", 2024-09-04T17:48:23.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027297906 2024-09-04T17:48:23.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:23.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.147588+0000", 2024-09-04T17:48:23.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6807999999999999e-05 2024-09-04T17:48:23.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:23.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.147637+0000", 2024-09-04T17:48:23.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8466e-05 2024-09-04T17:48:23.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:23.258 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.147703+0000", 2024-09-04T17:48:23.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6620999999999998e-05 2024-09-04T17:48:23.258 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:23.258 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:23.258 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:23.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.258 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49801 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 720896~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:48:23.258 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.075094+0000", 2024-09-04T17:48:23.258 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 80.149863952999993, 2024-09-04T17:48:23.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017824974, 2024-09-04T17:48:23.258 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:23.258 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:23.259 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:23.259 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:23.259 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:23.259 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:23.259 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49801 2024-09-04T17:48:23.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.259 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:23.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:23.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075094+0000", 2024-09-04T17:48:23.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:23.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:23.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075094+0000", 2024-09-04T17:48:23.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:23.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:23.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075095+0000", 2024-09-04T17:48:23.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.017e-06 2024-09-04T17:48:23.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:23.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075101+0000", 2024-09-04T17:48:23.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5029999999999997e-06 2024-09-04T17:48:23.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:23.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075119+0000", 2024-09-04T17:48:23.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8133000000000001e-05 2024-09-04T17:48:23.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:23.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075132+0000", 2024-09-04T17:48:23.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3305999999999999e-05 2024-09-04T17:48:23.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:23.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.092677+0000", 2024-09-04T17:48:23.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017544466000000002 2024-09-04T17:48:23.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:23.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.092782+0000", 2024-09-04T17:48:23.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000105284 2024-09-04T17:48:23.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:23.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.092919+0000", 2024-09-04T17:48:23.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013726499999999999 2024-09-04T17:48:23.263 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:23.263 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:23.263 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:23.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.263 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49809 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 163840~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:48:23.263 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077146+0000", 2024-09-04T17:48:23.263 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 80.147812509999994, 2024-09-04T17:48:23.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017984541999999999, 2024-09-04T17:48:23.263 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:23.263 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:23.263 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:23.264 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:23.264 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:23.264 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:23.264 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49809 2024-09-04T17:48:23.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.264 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:23.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:23.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077146+0000", 2024-09-04T17:48:23.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:23.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:23.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077146+0000", 2024-09-04T17:48:23.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:23.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:23.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077147+0000", 2024-09-04T17:48:23.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0330000000000001e-06 2024-09-04T17:48:23.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:23.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077164+0000", 2024-09-04T17:48:23.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7810000000000001e-05 2024-09-04T17:48:23.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:23.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077187+0000", 2024-09-04T17:48:23.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2246e-05 2024-09-04T17:48:23.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:23.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077203+0000", 2024-09-04T17:48:23.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6392000000000002e-05 2024-09-04T17:48:23.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:23.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.094885+0000", 2024-09-04T17:48:23.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017681589000000001 2024-09-04T17:48:23.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.267 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:23.267 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.094991+0000", 2024-09-04T17:48:23.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010668899999999999 2024-09-04T17:48:23.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.267 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:23.267 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095130+0000", 2024-09-04T17:48:23.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013878299999999999 2024-09-04T17:48:23.267 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:23.267 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:23.267 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:23.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.268 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49810 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 229376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:48:23.268 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077225+0000", 2024-09-04T17:48:23.268 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 80.147732669000007, 2024-09-04T17:48:23.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018218793, 2024-09-04T17:48:23.268 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:23.268 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:23.268 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:23.268 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:23.268 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:23.268 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:23.268 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49810 2024-09-04T17:48:23.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.268 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:23.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:23.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077225+0000", 2024-09-04T17:48:23.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:23.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:23.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077225+0000", 2024-09-04T17:48:23.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:23.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:23.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077227+0000", 2024-09-04T17:48:23.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1829999999999999e-06 2024-09-04T17:48:23.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:23.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077233+0000", 2024-09-04T17:48:23.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3899999999999998e-06 2024-09-04T17:48:23.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:23.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077253+0000", 2024-09-04T17:48:23.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0009000000000001e-05 2024-09-04T17:48:23.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:23.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077270+0000", 2024-09-04T17:48:23.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7336000000000001e-05 2024-09-04T17:48:23.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:23.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095177+0000", 2024-09-04T17:48:23.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017907101000000002 2024-09-04T17:48:23.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:23.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095299+0000", 2024-09-04T17:48:23.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000121749 2024-09-04T17:48:23.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:23.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095444+0000", 2024-09-04T17:48:23.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014502499999999999 2024-09-04T17:48:23.272 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:23.272 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:23.272 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:23.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.272 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49811 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 294912~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:48:23.272 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077288+0000", 2024-09-04T17:48:23.272 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 80.147670355000002, 2024-09-04T17:48:23.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018443620000000001, 2024-09-04T17:48:23.272 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:23.272 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:23.272 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:23.272 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:23.273 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:23.273 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:23.273 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49811 2024-09-04T17:48:23.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.273 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:23.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:23.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077288+0000", 2024-09-04T17:48:23.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:23.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:23.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077288+0000", 2024-09-04T17:48:23.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:23.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:23.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077289+0000", 2024-09-04T17:48:23.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6199999999999999e-06 2024-09-04T17:48:23.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:23.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077295+0000", 2024-09-04T17:48:23.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1499999999999998e-06 2024-09-04T17:48:23.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:23.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077312+0000", 2024-09-04T17:48:23.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7353000000000001e-05 2024-09-04T17:48:23.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:23.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077324+0000", 2024-09-04T17:48:23.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2593e-05 2024-09-04T17:48:23.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:23.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095490+0000", 2024-09-04T17:48:23.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018165976 2024-09-04T17:48:23.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:23.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095597+0000", 2024-09-04T17:48:23.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010629900000000001 2024-09-04T17:48:23.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:23.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095731+0000", 2024-09-04T17:48:23.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013462899999999999 2024-09-04T17:48:23.277 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:23.277 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:23.277 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:23.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.277 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49813 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 360448~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:48:23.277 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077349+0000", 2024-09-04T17:48:23.277 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 80.147609153000005, 2024-09-04T17:48:23.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018780305000000001, 2024-09-04T17:48:23.277 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:23.277 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:23.277 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:23.277 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:23.278 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:23.278 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:23.278 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49813 2024-09-04T17:48:23.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.278 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:23.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:23.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077349+0000", 2024-09-04T17:48:23.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:23.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:23.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077349+0000", 2024-09-04T17:48:23.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:23.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:23.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077350+0000", 2024-09-04T17:48:23.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0330000000000001e-06 2024-09-04T17:48:23.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:23.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077355+0000", 2024-09-04T17:48:23.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0699999999999997e-06 2024-09-04T17:48:23.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:23.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077376+0000", 2024-09-04T17:48:23.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0846e-05 2024-09-04T17:48:23.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:23.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077388+0000", 2024-09-04T17:48:23.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2353e-05 2024-09-04T17:48:23.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:23.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095819+0000", 2024-09-04T17:48:23.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01843057 2024-09-04T17:48:23.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:23.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095960+0000", 2024-09-04T17:48:23.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000140965 2024-09-04T17:48:23.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:23.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096129+0000", 2024-09-04T17:48:23.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016946800000000001 2024-09-04T17:48:23.281 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:23.281 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:23.281 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:23.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.282 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49814 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 425984~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:48:23.282 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077407+0000", 2024-09-04T17:48:23.282 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 80.147550652000007, 2024-09-04T17:48:23.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019041563000000001, 2024-09-04T17:48:23.282 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:23.282 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:23.282 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:23.282 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:23.282 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:23.282 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:23.282 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49814 2024-09-04T17:48:23.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.282 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:23.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:23.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077407+0000", 2024-09-04T17:48:23.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:23.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:23.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077407+0000", 2024-09-04T17:48:23.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:23.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:23.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077409+0000", 2024-09-04T17:48:23.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2330000000000001e-06 2024-09-04T17:48:23.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:23.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077415+0000", 2024-09-04T17:48:23.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2729999999999999e-06 2024-09-04T17:48:23.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:23.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077436+0000", 2024-09-04T17:48:23.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1209999999999999e-05 2024-09-04T17:48:23.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:23.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077448+0000", 2024-09-04T17:48:23.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2225999999999999e-05 2024-09-04T17:48:23.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:23.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096183+0000", 2024-09-04T17:48:23.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018735004 2024-09-04T17:48:23.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:23.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096297+0000", 2024-09-04T17:48:23.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000113962 2024-09-04T17:48:23.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:23.286 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096449+0000", 2024-09-04T17:48:23.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000151655 2024-09-04T17:48:23.286 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:23.286 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:23.286 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:23.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.286 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49815 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 491520~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:48:23.286 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077471+0000", 2024-09-04T17:48:23.286 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 80.147487413999997, 2024-09-04T17:48:23.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019266536000000001, 2024-09-04T17:48:23.287 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:23.287 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:23.287 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:23.287 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:23.287 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:23.287 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:23.287 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49815 2024-09-04T17:48:23.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.287 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:23.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:23.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077471+0000", 2024-09-04T17:48:23.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:23.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:23.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077471+0000", 2024-09-04T17:48:23.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:23.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:23.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077472+0000", 2024-09-04T17:48:23.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.17e-06 2024-09-04T17:48:23.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:23.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077478+0000", 2024-09-04T17:48:23.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0399999999999998e-06 2024-09-04T17:48:23.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:23.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077500+0000", 2024-09-04T17:48:23.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1959000000000001e-05 2024-09-04T17:48:23.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:23.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077513+0000", 2024-09-04T17:48:23.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2823000000000001e-05 2024-09-04T17:48:23.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:23.290 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096493+0000", 2024-09-04T17:48:23.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018980701999999999 2024-09-04T17:48:23.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:23.290 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096603+0000", 2024-09-04T17:48:23.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010942000000000001 2024-09-04T17:48:23.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:23.290 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096737+0000", 2024-09-04T17:48:23.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013442200000000001 2024-09-04T17:48:23.291 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:23.291 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:23.291 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:23.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.291 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49816 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 557056~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:48:23.291 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077532+0000", 2024-09-04T17:48:23.291 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 80.147425673000001, 2024-09-04T17:48:23.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019604331999999999, 2024-09-04T17:48:23.292 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:23.292 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:23.292 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:23.292 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:23.292 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:23.292 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:23.292 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49816 2024-09-04T17:48:23.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.292 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:23.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:23.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077532+0000", 2024-09-04T17:48:23.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:23.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:23.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077532+0000", 2024-09-04T17:48:23.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:23.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:23.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077534+0000", 2024-09-04T17:48:23.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1200000000000001e-06 2024-09-04T17:48:23.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:23.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077545+0000", 2024-09-04T17:48:23.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1673e-05 2024-09-04T17:48:23.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:23.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077569+0000", 2024-09-04T17:48:23.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3555999999999999e-05 2024-09-04T17:48:23.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:23.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077584+0000", 2024-09-04T17:48:23.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4883e-05 2024-09-04T17:48:23.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:23.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096825+0000", 2024-09-04T17:48:23.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019241734 2024-09-04T17:48:23.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:23.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096964+0000", 2024-09-04T17:48:23.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000138085 2024-09-04T17:48:23.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:23.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097137+0000", 2024-09-04T17:48:23.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017328100000000001 2024-09-04T17:48:23.295 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:23.295 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:23.296 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:23.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.296 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49817 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 622592~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:48:23.296 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077603+0000", 2024-09-04T17:48:23.296 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 80.147355114999996, 2024-09-04T17:48:23.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01986928, 2024-09-04T17:48:23.296 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:23.296 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:23.296 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:23.296 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:23.296 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:23.297 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:23.297 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49817 2024-09-04T17:48:23.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.297 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:23.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:23.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077603+0000", 2024-09-04T17:48:23.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:23.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:23.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077603+0000", 2024-09-04T17:48:23.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:23.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:23.298 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077604+0000", 2024-09-04T17:48:23.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7699999999999992e-07 2024-09-04T17:48:23.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:23.298 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077609+0000", 2024-09-04T17:48:23.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0830000000000003e-06 2024-09-04T17:48:23.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:23.298 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077630+0000", 2024-09-04T17:48:23.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0438999999999999e-05 2024-09-04T17:48:23.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:23.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077664+0000", 2024-09-04T17:48:23.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4579000000000002e-05 2024-09-04T17:48:23.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:23.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097193+0000", 2024-09-04T17:48:23.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019529174 2024-09-04T17:48:23.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:23.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097323+0000", 2024-09-04T17:48:23.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000129696 2024-09-04T17:48:23.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.300 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:23.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097472+0000", 2024-09-04T17:48:23.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014933200000000001 2024-09-04T17:48:23.300 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:23.300 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:23.300 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:23.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.300 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49818 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 688128~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:48:23.300 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077683+0000", 2024-09-04T17:48:23.300 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 80.147274750999998, 2024-09-04T17:48:23.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020086333000000001, 2024-09-04T17:48:23.301 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:23.301 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:23.301 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:23.301 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:23.301 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:23.301 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:23.301 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49818 2024-09-04T17:48:23.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.301 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:23.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:23.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077683+0000", 2024-09-04T17:48:23.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:23.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:23.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077683+0000", 2024-09-04T17:48:23.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:23.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:23.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077684+0000", 2024-09-04T17:48:23.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4e-07 2024-09-04T17:48:23.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.303 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:23.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077690+0000", 2024-09-04T17:48:23.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4729999999999999e-06 2024-09-04T17:48:23.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.303 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:23.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077710+0000", 2024-09-04T17:48:23.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0149999999999999e-05 2024-09-04T17:48:23.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.303 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:23.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077723+0000", 2024-09-04T17:48:23.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3271999999999999e-05 2024-09-04T17:48:23.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:23.304 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097521+0000", 2024-09-04T17:48:23.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019797981999999999 2024-09-04T17:48:23.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:23.304 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097628+0000", 2024-09-04T17:48:23.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010665700000000001 2024-09-04T17:48:23.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:23.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097770+0000", 2024-09-04T17:48:23.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014185900000000001 2024-09-04T17:48:23.305 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:23.305 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:23.305 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:23.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.305 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49820 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 753664~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:48:23.305 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077744+0000", 2024-09-04T17:48:23.305 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 80.147213820000005, 2024-09-04T17:48:23.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020364786999999999, 2024-09-04T17:48:23.305 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:23.305 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:23.306 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:23.306 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:23.306 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:23.306 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:23.306 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49820 2024-09-04T17:48:23.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.306 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:23.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:23.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077744+0000", 2024-09-04T17:48:23.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:23.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:23.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077744+0000", 2024-09-04T17:48:23.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:23.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:23.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077745+0000", 2024-09-04T17:48:23.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0270000000000001e-06 2024-09-04T17:48:23.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:23.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077751+0000", 2024-09-04T17:48:23.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.626e-06 2024-09-04T17:48:23.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:23.308 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077771+0000", 2024-09-04T17:48:23.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0469999999999999e-05 2024-09-04T17:48:23.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.308 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:23.308 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077785+0000", 2024-09-04T17:48:23.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3129000000000001e-05 2024-09-04T17:48:23.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.308 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:23.308 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097816+0000", 2024-09-04T17:48:23.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020031917999999999 2024-09-04T17:48:23.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.309 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:23.309 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097939+0000", 2024-09-04T17:48:23.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000122533 2024-09-04T17:48:23.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.309 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:23.309 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098109+0000", 2024-09-04T17:48:23.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017008400000000001 2024-09-04T17:48:23.309 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:23.309 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:23.309 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:23.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.310 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49821 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 819200~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:48:23.310 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077807+0000", 2024-09-04T17:48:23.310 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 80.147151608000001, 2024-09-04T17:48:23.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020702928999999998, 2024-09-04T17:48:23.310 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:23.310 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:23.310 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:23.310 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:23.310 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:23.310 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:23.310 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49821 2024-09-04T17:48:23.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.310 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:23.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:23.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077807+0000", 2024-09-04T17:48:23.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:23.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:23.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077807+0000", 2024-09-04T17:48:23.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:23.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:23.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077808+0000", 2024-09-04T17:48:23.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.463e-06 2024-09-04T17:48:23.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:23.312 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077813+0000", 2024-09-04T17:48:23.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7530000000000001e-06 2024-09-04T17:48:23.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.312 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:23.312 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077832+0000", 2024-09-04T17:48:23.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.946e-05 2024-09-04T17:48:23.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.312 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:23.312 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077846+0000", 2024-09-04T17:48:23.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3786e-05 2024-09-04T17:48:23.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:23.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098201+0000", 2024-09-04T17:48:23.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020354797000000001 2024-09-04T17:48:23.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:23.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098341+0000", 2024-09-04T17:48:23.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000140042 2024-09-04T17:48:23.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:23.314 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098509+0000", 2024-09-04T17:48:23.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016862800000000001 2024-09-04T17:48:23.314 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:23.314 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:23.314 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:23.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.314 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49823 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 884736~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:48:23.314 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077865+0000", 2024-09-04T17:48:23.314 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 80.147093163999998, 2024-09-04T17:48:23.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021008723, 2024-09-04T17:48:23.314 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:23.314 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:23.314 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:23.315 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:23.315 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:23.315 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:23.315 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49823 2024-09-04T17:48:23.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.315 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:23.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.315 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:23.315 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077865+0000", 2024-09-04T17:48:23.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:23.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.315 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:23.315 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077865+0000", 2024-09-04T17:48:23.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:23.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:23.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077866+0000", 2024-09-04T17:48:23.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.164e-06 2024-09-04T17:48:23.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:23.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077872+0000", 2024-09-04T17:48:23.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3959999999999997e-06 2024-09-04T17:48:23.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:23.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077892+0000", 2024-09-04T17:48:23.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0970000000000001e-05 2024-09-04T17:48:23.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:23.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077905+0000", 2024-09-04T17:48:23.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2312e-05 2024-09-04T17:48:23.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:23.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098579+0000", 2024-09-04T17:48:23.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020674486999999998 2024-09-04T17:48:23.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.318 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:23.318 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098705+0000", 2024-09-04T17:48:23.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012541900000000001 2024-09-04T17:48:23.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.318 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:23.318 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098874+0000", 2024-09-04T17:48:23.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000168975 2024-09-04T17:48:23.318 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:23.318 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:23.318 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:23.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.318 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49824 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 950272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:48:23.319 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077923+0000", 2024-09-04T17:48:23.319 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 80.147035471999999, 2024-09-04T17:48:23.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021246287999999999, 2024-09-04T17:48:23.319 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:23.319 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:23.319 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:23.319 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:23.319 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:23.319 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:23.319 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49824 2024-09-04T17:48:23.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.319 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:23.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:23.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077923+0000", 2024-09-04T17:48:23.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:23.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:23.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077923+0000", 2024-09-04T17:48:23.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:23.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:23.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077924+0000", 2024-09-04T17:48:23.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0100000000000001e-06 2024-09-04T17:48:23.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:23.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077944+0000", 2024-09-04T17:48:23.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0449e-05 2024-09-04T17:48:23.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:23.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077965+0000", 2024-09-04T17:48:23.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0479000000000002e-05 2024-09-04T17:48:23.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:23.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077990+0000", 2024-09-04T17:48:23.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5582999999999999e-05 2024-09-04T17:48:23.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:23.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098924+0000", 2024-09-04T17:48:23.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020933395 2024-09-04T17:48:23.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:23.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.099032+0000", 2024-09-04T17:48:23.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010846299999999999 2024-09-04T17:48:23.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.323 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:23.323 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.099169+0000", 2024-09-04T17:48:23.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000136909 2024-09-04T17:48:23.323 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:23.323 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:23.323 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:23.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.323 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49826 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 1015808~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:48:23.323 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.078010+0000", 2024-09-04T17:48:23.323 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 80.146947865000001, 2024-09-04T17:48:23.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021584564000000001, 2024-09-04T17:48:23.324 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:23.324 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:23.324 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:23.324 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:23.324 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:23.324 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:23.324 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49826 2024-09-04T17:48:23.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.324 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:23.324 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.324 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:23.324 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078010+0000", 2024-09-04T17:48:23.324 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:23.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:23.325 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078010+0000", 2024-09-04T17:48:23.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:23.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:23.325 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078011+0000", 2024-09-04T17:48:23.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1340000000000001e-06 2024-09-04T17:48:23.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:23.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078017+0000", 2024-09-04T17:48:23.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.682e-06 2024-09-04T17:48:23.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:23.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078036+0000", 2024-09-04T17:48:23.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9357000000000001e-05 2024-09-04T17:48:23.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:23.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078050+0000", 2024-09-04T17:48:23.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3122000000000001e-05 2024-09-04T17:48:23.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:23.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.099271+0000", 2024-09-04T17:48:23.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021221772999999999 2024-09-04T17:48:23.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:23.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.099410+0000", 2024-09-04T17:48:23.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000138415 2024-09-04T17:48:23.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:23.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:23.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:23.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.099595+0000", 2024-09-04T17:48:23.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000185081 2024-09-04T17:48:23.328 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:23.328 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:23.328 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:23.328 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:23.328 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:23.328 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:23.329 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-04T17:48:23.330 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:48:23.327+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:48:23.430 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:48:23.427+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:48:23.502 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:23.502 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:48:23.503 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:48:23.503 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:23.516 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-04T17:48:23.522 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-04T17:48:23.530 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:23.526+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:23.632 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:23.627+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:23.674 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:23.674 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:48:23.675 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:48:23.675 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:48:23.675 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:23.689 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-04T17:48:23.708 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:48:23.708 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:48:23.725 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:48:23.725 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:48:23.733 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:23.730+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:23.742 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-04T17:48:23.742 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-04T17:48:23.759 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-04T17:48:23.759 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-04T17:48:23.780 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:48:23.781 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:48:23.800 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:48:23.801 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:48:23.820 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-04T17:48:23.820 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-04T17:48:23.821 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:23.821 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:23.834 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:48:23.830+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:48:23.836 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-04T17:48:23.841 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:48:23.842 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:48:23.935 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:23.930+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:23.966 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:23.966 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:23.981 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-04T17:48:24.036 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:24.031+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:24.137 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:48:24.134+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:48:24.153 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:24.153 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:24.169 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-04T17:48:24.261 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:24.258+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:24.339 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:24.335+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:24.344 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:24.344 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:24.358 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-04T17:48:24.440 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:48:24.435+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:48:24.489 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:24.489 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:24.504 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-04T17:48:24.541 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:24.542+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:24.642 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:24.639+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:24.661 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:24.661 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:24.676 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-04T17:48:24.761 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:24.759+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:24.842 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:24.842 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:24.844 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:48:24.838+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:48:24.857 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-04T17:48:24.865 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-04T17:48:24.869 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 4, 5, 6, 7] out_osds: [3, 1] dead_osds: [] live_osds: [0, 1, 4, 5, 7, 3, 2, 6] 2024-09-04T17:48:24.869 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-04T17:48:24.869 INFO:tasks.thrashosds.thrasher:Adding osd 3 2024-09-04T17:48:24.869 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 3 2024-09-04T17:48:24.945 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:48:24.946+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:48:25.029 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:25.030 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:25.046 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:25.042+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:25.049 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-04T17:48:25.148 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:48:25.142+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:48:25.176 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:25.176 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:25.191 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-04T17:48:25.248 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:48:25.243+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:48:25.274 INFO:tasks.ceph.mon.a.smithi092.stderr:2024-09-04T17:48:25.271+0000 7f8f2f46f640 -1 mon.a@0(leader).osd e145 definitely_dead 0 2024-09-04T17:48:25.298 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:25.298 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:25.310 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-04T17:48:25.357 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:25.354+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:25.442 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:25.442 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:25.451 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:25.447+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:25.457 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-04T17:48:25.551 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:25.546+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:25.620 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:25.621 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:25.635 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-04T17:48:25.681 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:25.678+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:25.754 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:25.750+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:25.806 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:25.807 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:25.821 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-04T17:48:25.854 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:25.851+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:25.859 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-04T17:48:25.955 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:25.951+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:25.992 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:25.992 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:26.007 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-04T17:48:26.013 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:48:26.013 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:48:26.020 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is unset 2024-09-04T17:48:26.020 INFO:teuthology.orchestra.run.smithi092.stderr:marked in osd.3. 2024-09-04T17:48:26.036 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:48:26.036 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:48:26.045 INFO:tasks.thrashosds.thrasher:Added osd 3 2024-09-04T17:48:26.053 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-04T17:48:26.053 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-04T17:48:26.056 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:26.050+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:26.069 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-04T17:48:26.069 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-04T17:48:26.090 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:48:26.090 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:48:26.110 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:48:26.111 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:48:26.131 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-04T17:48:26.131 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-04T17:48:26.131 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:26.132 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:48:26.132 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:48:26.132 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:48:26.132 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:26.145 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-04T17:48:26.154 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:48:26.154 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:48:26.189 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:26.190+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:26.258 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:26.255+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:26.314 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:26.314 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:48:26.314 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:48:26.314 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:48:26.314 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.314 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8831 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2453504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:26.314 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:48:26.314 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 133.254432966, 2024-09-04T17:48:26.314 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14731111799999999, 2024-09-04T17:48:26.314 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:26.315 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:26.315 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:26.315 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:26.315 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:26.315 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:26.315 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8831 2024-09-04T17:48:26.315 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.315 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:26.315 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.315 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:26.315 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:48:26.315 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.315 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.315 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.316 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:26.316 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:48:26.316 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.316 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.316 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.316 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:26.316 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057811+0000", 2024-09-04T17:48:26.316 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.083e-06 2024-09-04T17:48:26.316 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.316 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.316 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:26.316 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057814+0000", 2024-09-04T17:48:26.316 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.407e-06 2024-09-04T17:48:26.317 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.317 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.317 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:26.317 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057825+0000", 2024-09-04T17:48:26.317 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.17e-05 2024-09-04T17:48:26.317 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.317 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.317 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:26.317 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057836+0000", 2024-09-04T17:48:26.317 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0283e-05 2024-09-04T17:48:26.317 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.317 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.317 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:26.318 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.204762+0000", 2024-09-04T17:48:26.318 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.146926205 2024-09-04T17:48:26.318 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.318 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.318 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:26.318 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.204895+0000", 2024-09-04T17:48:26.318 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013328999999999999 2024-09-04T17:48:26.318 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.318 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.318 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:26.318 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205121+0000", 2024-09-04T17:48:26.318 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00022614999999999999 2024-09-04T17:48:26.318 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.318 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:26.319 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.319 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.319 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.319 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8832 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2490368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:26.319 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:48:26.319 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 133.25439493299999, 2024-09-04T17:48:26.319 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14770012299999999, 2024-09-04T17:48:26.319 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:26.319 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:26.319 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:26.319 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:26.319 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:26.319 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:26.320 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8832 2024-09-04T17:48:26.320 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.320 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:26.320 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.320 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:26.320 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:48:26.320 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.320 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.320 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.321 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:26.321 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:48:26.321 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.321 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.321 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.321 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:26.321 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057849+0000", 2024-09-04T17:48:26.321 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6700000000000003e-07 2024-09-04T17:48:26.321 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.321 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.321 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:26.321 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057852+0000", 2024-09-04T17:48:26.321 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6359999999999998e-06 2024-09-04T17:48:26.322 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.322 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.322 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:26.322 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057860+0000", 2024-09-04T17:48:26.322 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8140000000000007e-06 2024-09-04T17:48:26.322 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.322 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.322 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:26.322 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057868+0000", 2024-09-04T17:48:26.323 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1999999999999997e-06 2024-09-04T17:48:26.323 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.323 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.323 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:26.323 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205208+0000", 2024-09-04T17:48:26.323 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14734067300000001 2024-09-04T17:48:26.323 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.323 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.323 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:26.323 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205347+0000", 2024-09-04T17:48:26.324 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013914900000000001 2024-09-04T17:48:26.324 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.324 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.324 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:26.324 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205548+0000", 2024-09-04T17:48:26.324 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000200984 2024-09-04T17:48:26.324 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.324 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:26.324 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.324 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.324 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.324 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8833 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2527232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:26.324 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:48:26.325 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 133.254363063, 2024-09-04T17:48:26.325 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148062572, 2024-09-04T17:48:26.325 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:26.325 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:26.325 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:26.325 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:26.325 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:26.325 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:26.325 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8833 2024-09-04T17:48:26.325 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.325 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:26.325 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.325 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:26.325 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:48:26.326 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.326 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.326 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.326 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:26.326 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:48:26.326 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.327 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.327 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.327 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:26.327 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057881+0000", 2024-09-04T17:48:26.327 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4300000000000002e-07 2024-09-04T17:48:26.327 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.327 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.327 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:26.328 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057893+0000", 2024-09-04T17:48:26.328 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1973999999999999e-05 2024-09-04T17:48:26.328 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.328 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.328 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:26.328 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057906+0000", 2024-09-04T17:48:26.328 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2716e-05 2024-09-04T17:48:26.328 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.328 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.328 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:26.328 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057913+0000", 2024-09-04T17:48:26.328 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3930000000000002e-06 2024-09-04T17:48:26.329 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.329 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.329 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:26.329 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205621+0000", 2024-09-04T17:48:26.329 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14770788800000001 2024-09-04T17:48:26.329 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.329 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.329 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:26.329 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205740+0000", 2024-09-04T17:48:26.329 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000118936 2024-09-04T17:48:26.329 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.329 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.329 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:26.329 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205943+0000", 2024-09-04T17:48:26.330 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020292199999999999 2024-09-04T17:48:26.330 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.330 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:26.330 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.330 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.330 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.330 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8834 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2564096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:26.330 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:48:26.330 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 133.25431978399999, 2024-09-04T17:48:26.330 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14846753200000001, 2024-09-04T17:48:26.330 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:26.330 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:26.331 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:26.331 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:26.331 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:26.331 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:26.331 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8834 2024-09-04T17:48:26.331 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.331 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:26.331 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.331 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:26.331 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:48:26.331 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.331 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.331 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.332 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:26.332 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:48:26.332 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.332 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.332 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.332 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:26.332 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057925+0000", 2024-09-04T17:48:26.332 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0270000000000001e-06 2024-09-04T17:48:26.332 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.332 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.332 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:26.332 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057936+0000", 2024-09-04T17:48:26.332 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.112e-05 2024-09-04T17:48:26.333 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.333 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.333 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:26.333 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057946+0000", 2024-09-04T17:48:26.333 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.009e-05 2024-09-04T17:48:26.333 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.333 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.333 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:26.333 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057953+0000", 2024-09-04T17:48:26.333 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.6229999999999997e-06 2024-09-04T17:48:26.333 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.333 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.333 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:26.333 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206027+0000", 2024-09-04T17:48:26.334 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148073591 2024-09-04T17:48:26.334 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.334 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.334 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:26.334 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206161+0000", 2024-09-04T17:48:26.334 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013411599999999999 2024-09-04T17:48:26.334 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.334 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.334 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:26.334 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206391+0000", 2024-09-04T17:48:26.334 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000229965 2024-09-04T17:48:26.334 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.334 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:26.334 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.335 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.335 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.335 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8835 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2600960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:26.335 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:48:26.335 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 133.254280134, 2024-09-04T17:48:26.335 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14882941599999999, 2024-09-04T17:48:26.335 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:26.335 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:26.335 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:26.335 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:26.335 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:26.335 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:26.335 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8835 2024-09-04T17:48:26.336 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.336 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:26.336 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.336 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:26.336 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:48:26.336 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.336 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.336 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.336 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:26.336 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:48:26.336 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.336 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.336 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.336 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:26.337 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058544+0000", 2024-09-04T17:48:26.337 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00058061899999999999 2024-09-04T17:48:26.337 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.337 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.337 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:26.337 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058550+0000", 2024-09-04T17:48:26.337 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.1129999999999997e-06 2024-09-04T17:48:26.337 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.337 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.337 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:26.337 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058566+0000", 2024-09-04T17:48:26.337 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5820000000000001e-05 2024-09-04T17:48:26.337 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.337 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.338 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:26.338 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058578+0000", 2024-09-04T17:48:26.338 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2537e-05 2024-09-04T17:48:26.338 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.338 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.338 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:26.338 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206470+0000", 2024-09-04T17:48:26.338 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14789153799999999 2024-09-04T17:48:26.338 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.338 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.338 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:26.338 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206589+0000", 2024-09-04T17:48:26.338 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000118837 2024-09-04T17:48:26.339 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.339 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.339 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:26.339 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206793+0000", 2024-09-04T17:48:26.339 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000203952 2024-09-04T17:48:26.339 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.339 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:26.339 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.339 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.339 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.339 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8836 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2654208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:26.339 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:48:26.339 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 133.253647446, 2024-09-04T17:48:26.340 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14862433999999999, 2024-09-04T17:48:26.340 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:26.340 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:26.340 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:26.340 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:26.340 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:26.340 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:26.340 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8836 2024-09-04T17:48:26.340 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.340 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:26.340 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.340 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:26.340 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:48:26.340 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.341 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.341 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.341 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:26.341 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:48:26.341 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.341 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.341 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.341 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:26.341 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062663+0000", 2024-09-04T17:48:26.341 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0040670619999999998 2024-09-04T17:48:26.341 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.341 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.341 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:26.341 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062668+0000", 2024-09-04T17:48:26.342 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4199999999999998e-06 2024-09-04T17:48:26.342 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.342 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.342 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:26.342 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062689+0000", 2024-09-04T17:48:26.342 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0356999999999998e-05 2024-09-04T17:48:26.342 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.342 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.342 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:26.342 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062726+0000", 2024-09-04T17:48:26.342 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7302999999999998e-05 2024-09-04T17:48:26.342 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.342 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.343 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:26.343 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206875+0000", 2024-09-04T17:48:26.343 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14414883100000001 2024-09-04T17:48:26.343 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.343 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.343 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:26.343 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.207016+0000", 2024-09-04T17:48:26.343 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000140822 2024-09-04T17:48:26.343 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.343 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.343 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:26.343 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.207220+0000", 2024-09-04T17:48:26.343 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020454499999999999 2024-09-04T17:48:26.343 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.344 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:26.344 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.344 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.344 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.344 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8844 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2928640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:26.344 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:48:26.344 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 133.24928336299999, 2024-09-04T17:48:26.344 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14752153100000001, 2024-09-04T17:48:26.344 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:26.344 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:26.344 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:26.344 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:26.344 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:26.345 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:26.345 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8844 2024-09-04T17:48:26.345 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.345 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:26.345 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.345 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:26.345 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:48:26.345 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.345 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.345 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.345 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:26.345 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:48:26.346 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.346 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.346 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.346 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:26.346 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062961+0000", 2024-09-04T17:48:26.346 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6700000000000003e-07 2024-09-04T17:48:26.346 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.346 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.346 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:26.346 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062973+0000", 2024-09-04T17:48:26.346 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2109999999999999e-05 2024-09-04T17:48:26.346 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.346 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.347 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:26.347 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062982+0000", 2024-09-04T17:48:26.347 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.6900000000000004e-06 2024-09-04T17:48:26.347 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.347 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.347 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:26.347 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062990+0000", 2024-09-04T17:48:26.347 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1899999999999998e-06 2024-09-04T17:48:26.347 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.348 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.348 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:26.348 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210090+0000", 2024-09-04T17:48:26.348 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14710026300000001 2024-09-04T17:48:26.348 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.348 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.348 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:26.348 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210225+0000", 2024-09-04T17:48:26.348 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013532400000000001 2024-09-04T17:48:26.348 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.348 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.348 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:26.348 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210481+0000", 2024-09-04T17:48:26.348 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00025628700000000001 2024-09-04T17:48:26.349 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.349 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:26.349 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.349 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.349 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.349 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8845 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2965504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:26.349 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:48:26.349 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 133.24924364, 2024-09-04T17:48:26.349 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147916819, 2024-09-04T17:48:26.349 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:26.349 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:26.349 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:26.349 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:26.349 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:26.350 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:26.350 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8845 2024-09-04T17:48:26.350 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.350 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:26.350 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.350 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:26.350 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:48:26.350 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.350 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.350 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.350 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:26.350 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:48:26.351 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.351 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.351 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.351 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:26.351 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:48:26.351 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.0399999999999995e-07 2024-09-04T17:48:26.351 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.351 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.351 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:26.351 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063003+0000", 2024-09-04T17:48:26.351 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6529999999999998e-06 2024-09-04T17:48:26.351 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.351 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.352 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:26.352 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063012+0000", 2024-09-04T17:48:26.352 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9500000000000007e-06 2024-09-04T17:48:26.352 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.352 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.352 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:26.352 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063027+0000", 2024-09-04T17:48:26.352 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5443000000000001e-05 2024-09-04T17:48:26.352 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.352 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.352 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:26.352 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210571+0000", 2024-09-04T17:48:26.352 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14754353100000001 2024-09-04T17:48:26.352 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.353 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.353 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:26.353 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210714+0000", 2024-09-04T17:48:26.353 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014295599999999999 2024-09-04T17:48:26.353 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.353 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.353 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:26.353 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210916+0000", 2024-09-04T17:48:26.353 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020258200000000001 2024-09-04T17:48:26.353 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.353 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:26.353 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.353 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.353 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.354 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8846 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3002368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:26.354 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:48:26.354 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 133.24920610300001, 2024-09-04T17:48:26.354 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14828011599999999, 2024-09-04T17:48:26.354 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:26.354 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:26.354 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:26.354 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:26.354 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:26.354 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:26.354 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8846 2024-09-04T17:48:26.354 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.354 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:26.354 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.355 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:26.355 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:48:26.355 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.355 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.355 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.355 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:26.355 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:48:26.355 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.355 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.355 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.355 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:26.355 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063038+0000", 2024-09-04T17:48:26.355 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.23e-07 2024-09-04T17:48:26.355 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.356 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.356 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:26.356 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063051+0000", 2024-09-04T17:48:26.356 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2757e-05 2024-09-04T17:48:26.356 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.356 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.356 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:26.356 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063070+0000", 2024-09-04T17:48:26.356 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9663000000000001e-05 2024-09-04T17:48:26.356 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.356 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.356 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:26.356 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063079+0000", 2024-09-04T17:48:26.356 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.0929999999999992e-06 2024-09-04T17:48:26.357 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.357 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.357 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:26.357 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210989+0000", 2024-09-04T17:48:26.357 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14790984500000001 2024-09-04T17:48:26.357 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.357 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.357 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:26.357 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211109+0000", 2024-09-04T17:48:26.357 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000120056 2024-09-04T17:48:26.357 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.357 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.357 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:26.357 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211317+0000", 2024-09-04T17:48:26.358 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020797900000000001 2024-09-04T17:48:26.358 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.358 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:26.358 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.358 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.358 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.358 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8847 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3039232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:26.358 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:48:26.358 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 133.24913167099999, 2024-09-04T17:48:26.358 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14859408900000001, 2024-09-04T17:48:26.358 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:26.358 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:26.358 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:26.358 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:26.359 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:26.359 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:26.359 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8847 2024-09-04T17:48:26.359 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.359 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:26.359 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.359 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:26.359 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:48:26.359 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.359 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.359 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.359 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:26.359 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:48:26.359 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.360 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.360 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.360 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:26.360 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063113+0000", 2024-09-04T17:48:26.360 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.204e-06 2024-09-04T17:48:26.360 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.360 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.360 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:26.360 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063116+0000", 2024-09-04T17:48:26.360 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.3330000000000001e-06 2024-09-04T17:48:26.360 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.360 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.360 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:26.361 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063126+0000", 2024-09-04T17:48:26.361 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0166999999999999e-05 2024-09-04T17:48:26.361 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.361 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.361 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:26.361 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063134+0000", 2024-09-04T17:48:26.361 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4329999999999998e-06 2024-09-04T17:48:26.361 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.361 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.361 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:26.361 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211392+0000", 2024-09-04T17:48:26.361 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148258109 2024-09-04T17:48:26.361 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.361 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.362 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:26.362 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211506+0000", 2024-09-04T17:48:26.362 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000113813 2024-09-04T17:48:26.362 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.362 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.362 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:26.362 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211706+0000", 2024-09-04T17:48:26.362 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020002999999999999 2024-09-04T17:48:26.362 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.362 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:26.362 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.362 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.362 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.362 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8848 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:26.363 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:48:26.363 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 133.24909933399999, 2024-09-04T17:48:26.363 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14907967599999999, 2024-09-04T17:48:26.363 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:26.363 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:26.363 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:26.363 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:26.363 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:26.363 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:26.363 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8848 2024-09-04T17:48:26.363 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.363 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:26.363 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.363 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:26.364 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:48:26.364 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.364 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.364 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.364 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:26.364 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:48:26.364 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.364 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.364 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.364 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:26.364 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070589+0000", 2024-09-04T17:48:26.364 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.007445386 2024-09-04T17:48:26.364 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.364 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.365 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:26.365 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070600+0000", 2024-09-04T17:48:26.365 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0346999999999999e-05 2024-09-04T17:48:26.365 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.365 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.365 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:26.365 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070622+0000", 2024-09-04T17:48:26.365 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2289999999999998e-05 2024-09-04T17:48:26.365 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.365 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.365 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:26.365 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070636+0000", 2024-09-04T17:48:26.365 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4093e-05 2024-09-04T17:48:26.365 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.366 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.366 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:26.366 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211815+0000", 2024-09-04T17:48:26.366 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14117874699999999 2024-09-04T17:48:26.366 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.366 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.366 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:26.366 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211983+0000", 2024-09-04T17:48:26.366 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00016852800000000001 2024-09-04T17:48:26.366 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.366 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.366 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:26.366 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.212224+0000", 2024-09-04T17:48:26.366 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024028499999999999 2024-09-04T17:48:26.367 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.367 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:26.367 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.367 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.367 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.367 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8862 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2019328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:26.367 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:48:26.367 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 133.241049866, 2024-09-04T17:48:26.367 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14755236399999999, 2024-09-04T17:48:26.367 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:26.367 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:26.367 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:26.367 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:26.367 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:26.368 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:26.368 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8862 2024-09-04T17:48:26.368 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.368 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:26.368 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.368 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:26.368 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:48:26.368 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.368 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.368 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.368 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:26.368 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:48:26.368 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.368 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.369 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.369 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:26.369 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071194+0000", 2024-09-04T17:48:26.369 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.5000000000000002e-07 2024-09-04T17:48:26.369 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.369 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.369 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:26.369 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071197+0000", 2024-09-04T17:48:26.369 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5900000000000002e-06 2024-09-04T17:48:26.369 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.369 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.369 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:26.369 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071206+0000", 2024-09-04T17:48:26.369 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9700000000000005e-06 2024-09-04T17:48:26.370 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.370 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.370 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:26.370 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071213+0000", 2024-09-04T17:48:26.370 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.9070000000000001e-06 2024-09-04T17:48:26.370 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.370 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.370 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:26.370 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218396+0000", 2024-09-04T17:48:26.370 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147183273 2024-09-04T17:48:26.370 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.370 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.370 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:26.370 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218538+0000", 2024-09-04T17:48:26.371 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014194199999999999 2024-09-04T17:48:26.371 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.371 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.371 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:26.371 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218746+0000", 2024-09-04T17:48:26.371 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000208032 2024-09-04T17:48:26.371 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.371 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:26.371 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.371 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.371 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.371 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8863 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2056192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:26.371 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:48:26.372 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 133.24102155599999, 2024-09-04T17:48:26.372 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14792281099999999, 2024-09-04T17:48:26.372 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:26.372 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:26.372 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:26.372 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:26.372 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:26.372 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:26.372 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8863 2024-09-04T17:48:26.372 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.372 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:26.372 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.372 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:26.373 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:48:26.373 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.373 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.373 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.373 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:26.373 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:48:26.373 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.373 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.373 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.373 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:26.373 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:48:26.373 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1299999999999999e-07 2024-09-04T17:48:26.373 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.373 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.374 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:26.374 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071225+0000", 2024-09-04T17:48:26.374 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6199999999999999e-06 2024-09-04T17:48:26.374 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.374 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.374 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:26.374 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071234+0000", 2024-09-04T17:48:26.374 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8769999999999999e-06 2024-09-04T17:48:26.374 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.374 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.374 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:26.374 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071251+0000", 2024-09-04T17:48:26.374 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.738e-05 2024-09-04T17:48:26.375 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.375 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.375 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:26.375 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218798+0000", 2024-09-04T17:48:26.375 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14754645 2024-09-04T17:48:26.375 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.375 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.375 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:26.375 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218926+0000", 2024-09-04T17:48:26.375 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000128446 2024-09-04T17:48:26.375 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.375 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.375 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:26.376 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219145+0000", 2024-09-04T17:48:26.376 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021832500000000001 2024-09-04T17:48:26.376 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.376 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:26.376 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.376 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.376 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.376 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8864 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2093056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:26.376 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:48:26.376 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 133.24097997999999, 2024-09-04T17:48:26.376 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148277309, 2024-09-04T17:48:26.376 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:26.376 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:26.376 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:26.377 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:26.377 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:26.377 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:26.377 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8864 2024-09-04T17:48:26.377 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.377 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:26.377 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.377 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:26.377 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:48:26.377 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.377 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.377 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.377 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:26.378 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:48:26.378 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.378 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.378 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.378 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:26.378 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071264+0000", 2024-09-04T17:48:26.378 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3e-07 2024-09-04T17:48:26.378 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.378 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.378 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:26.378 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071267+0000", 2024-09-04T17:48:26.378 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0139999999999999e-06 2024-09-04T17:48:26.378 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.379 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.379 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:26.379 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071276+0000", 2024-09-04T17:48:26.379 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1689999999999998e-06 2024-09-04T17:48:26.379 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.379 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.379 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:26.379 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071283+0000", 2024-09-04T17:48:26.379 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2269999999999996e-06 2024-09-04T17:48:26.379 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.379 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.379 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:26.379 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219195+0000", 2024-09-04T17:48:26.379 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147911868 2024-09-04T17:48:26.380 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.380 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.380 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:26.380 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219324+0000", 2024-09-04T17:48:26.380 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00012835900000000001 2024-09-04T17:48:26.380 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.380 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.380 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:26.380 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219541+0000", 2024-09-04T17:48:26.380 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000216942 2024-09-04T17:48:26.380 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.380 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:26.380 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.381 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.381 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.381 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8865 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2109440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:26.381 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:48:26.381 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 133.24095052000001, 2024-09-04T17:48:26.381 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14865747300000001, 2024-09-04T17:48:26.381 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:26.381 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:26.381 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:26.381 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:26.381 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:26.381 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:26.381 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8865 2024-09-04T17:48:26.381 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.382 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:26.382 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.382 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:26.382 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:48:26.382 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.382 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.382 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.382 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:26.382 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:48:26.382 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.382 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.382 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.382 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:26.382 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:48:26.383 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7999999999999995e-07 2024-09-04T17:48:26.383 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.383 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.383 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:26.383 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071296+0000", 2024-09-04T17:48:26.383 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6900000000000001e-06 2024-09-04T17:48:26.383 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.383 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.383 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:26.383 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071305+0000", 2024-09-04T17:48:26.383 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.5499999999999995e-06 2024-09-04T17:48:26.383 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.383 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.383 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:26.384 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071312+0000", 2024-09-04T17:48:26.384 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1559999999999998e-06 2024-09-04T17:48:26.384 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.384 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.384 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:26.384 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219591+0000", 2024-09-04T17:48:26.384 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14827944900000001 2024-09-04T17:48:26.384 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.384 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.384 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:26.384 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219731+0000", 2024-09-04T17:48:26.384 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013931600000000001 2024-09-04T17:48:26.384 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.385 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.385 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:26.385 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219950+0000", 2024-09-04T17:48:26.385 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021963199999999999 2024-09-04T17:48:26.385 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.385 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:26.385 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.385 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.385 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.385 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8866 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2146304~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:26.385 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:48:26.385 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 133.24092228699999, 2024-09-04T17:48:26.385 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14906117899999999, 2024-09-04T17:48:26.385 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:26.386 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:26.386 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:26.386 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:26.386 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:26.386 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:26.386 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8866 2024-09-04T17:48:26.386 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.386 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:26.386 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.386 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:26.386 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:48:26.386 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.386 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.386 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.386 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:26.387 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:48:26.387 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.387 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.387 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.387 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:26.387 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071322+0000", 2024-09-04T17:48:26.387 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7999999999999995e-07 2024-09-04T17:48:26.387 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.387 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.387 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:26.387 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071338+0000", 2024-09-04T17:48:26.387 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5872999999999999e-05 2024-09-04T17:48:26.387 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.387 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.388 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:26.388 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071347+0000", 2024-09-04T17:48:26.388 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4669999999999993e-06 2024-09-04T17:48:26.388 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.388 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.388 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:26.388 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071354+0000", 2024-09-04T17:48:26.388 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2099999999999996e-06 2024-09-04T17:48:26.388 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.388 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.388 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:26.388 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220008+0000", 2024-09-04T17:48:26.388 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148653642 2024-09-04T17:48:26.389 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.389 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.389 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:26.389 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220155+0000", 2024-09-04T17:48:26.389 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014726600000000001 2024-09-04T17:48:26.389 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.389 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.389 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:26.389 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220382+0000", 2024-09-04T17:48:26.389 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00022704100000000001 2024-09-04T17:48:26.389 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.389 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:26.389 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.390 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.390 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.390 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8867 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2183168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:26.390 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:48:26.390 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 133.24087893399999, 2024-09-04T17:48:26.390 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.149445831, 2024-09-04T17:48:26.390 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:26.390 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:26.390 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:26.390 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:26.390 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:26.390 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:26.390 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8867 2024-09-04T17:48:26.390 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.391 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:26.391 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.391 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:26.391 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:48:26.391 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.391 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.391 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.391 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:26.391 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:48:26.391 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.391 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.391 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.391 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:26.392 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078653+0000", 2024-09-04T17:48:26.392 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0072890430000000003 2024-09-04T17:48:26.392 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.392 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.392 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:26.392 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078664+0000", 2024-09-04T17:48:26.392 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0307e-05 2024-09-04T17:48:26.392 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.392 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.392 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:26.392 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078690+0000", 2024-09-04T17:48:26.392 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6780000000000001e-05 2024-09-04T17:48:26.392 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.393 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.393 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:26.393 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078706+0000", 2024-09-04T17:48:26.393 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5676999999999999e-05 2024-09-04T17:48:26.393 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.393 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.393 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:26.393 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220467+0000", 2024-09-04T17:48:26.393 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14176103500000001 2024-09-04T17:48:26.393 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.393 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.393 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:26.393 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220595+0000", 2024-09-04T17:48:26.393 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000127427 2024-09-04T17:48:26.394 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.394 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.394 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:26.394 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220810+0000", 2024-09-04T17:48:26.394 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021556200000000001 2024-09-04T17:48:26.394 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.394 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:26.394 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.394 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.394 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.394 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8881 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2695168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:26.395 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:48:26.395 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 133.23305686500001, 2024-09-04T17:48:26.395 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147543547, 2024-09-04T17:48:26.395 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:26.395 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:26.395 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:26.395 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:26.395 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:26.395 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:26.395 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8881 2024-09-04T17:48:26.395 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.395 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:26.395 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.396 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:26.396 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:48:26.396 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.396 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.396 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.396 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:26.396 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:48:26.396 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.396 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.396 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.396 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:26.396 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079187+0000", 2024-09-04T17:48:26.396 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7700000000000004e-07 2024-09-04T17:48:26.397 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.397 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.397 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:26.397 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079198+0000", 2024-09-04T17:48:26.397 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0849999999999999e-05 2024-09-04T17:48:26.397 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.397 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.397 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:26.397 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079207+0000", 2024-09-04T17:48:26.397 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5599999999999999e-06 2024-09-04T17:48:26.397 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.397 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.397 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:26.398 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079224+0000", 2024-09-04T17:48:26.398 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6387000000000001e-05 2024-09-04T17:48:26.398 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.398 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.398 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:26.398 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226414+0000", 2024-09-04T17:48:26.398 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14718975000000001 2024-09-04T17:48:26.398 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.398 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.398 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:26.398 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226534+0000", 2024-09-04T17:48:26.398 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000120288 2024-09-04T17:48:26.398 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.398 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.399 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:26.399 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226730+0000", 2024-09-04T17:48:26.399 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000196035 2024-09-04T17:48:26.399 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.399 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:26.399 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.399 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.399 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.399 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8882 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2732032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:26.399 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:48:26.399 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 133.23300865100001, 2024-09-04T17:48:26.399 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14795022399999999, 2024-09-04T17:48:26.399 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:26.399 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:26.400 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:26.400 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:26.400 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:26.400 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:26.400 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8882 2024-09-04T17:48:26.400 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.400 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:26.400 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.400 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:26.400 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:48:26.400 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.400 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.400 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.400 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:26.401 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:48:26.401 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.401 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.401 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.401 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:26.401 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086588+0000", 2024-09-04T17:48:26.401 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0073530959999999999 2024-09-04T17:48:26.401 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.401 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.401 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:26.401 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086598+0000", 2024-09-04T17:48:26.401 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0122999999999999e-05 2024-09-04T17:48:26.401 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.401 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.402 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:26.402 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086626+0000", 2024-09-04T17:48:26.402 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.7914000000000001e-05 2024-09-04T17:48:26.402 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.402 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.402 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:26.402 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086641+0000", 2024-09-04T17:48:26.402 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5270000000000001e-05 2024-09-04T17:48:26.402 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.402 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.402 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:26.402 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226801+0000", 2024-09-04T17:48:26.402 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.140160321 2024-09-04T17:48:26.403 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.403 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.403 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:26.403 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226951+0000", 2024-09-04T17:48:26.403 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014965200000000001 2024-09-04T17:48:26.403 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.403 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.403 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:26.403 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.227185+0000", 2024-09-04T17:48:26.403 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00023384799999999999 2024-09-04T17:48:26.403 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.403 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:26.403 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.404 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.404 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.404 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:93470 3.1f 3:f95bc674:::rbd_header.1134878eadad:head [watch reconnect cookie 140018821420464 gen 16] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-04T17:48:26.404 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:57.837081+0000", 2024-09-04T17:48:26.404 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 28.475162768000001, 2024-09-04T17:48:26.404 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.072023486, 2024-09-04T17:48:26.404 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:26.404 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:26.404 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:26.404 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:26.404 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:26.404 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:26.404 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 93470 2024-09-04T17:48:26.405 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.405 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:26.405 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.405 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:26.405 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:57.837081+0000", 2024-09-04T17:48:26.405 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.405 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.405 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.405 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:26.405 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:57.837081+0000", 2024-09-04T17:48:26.405 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.405 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.405 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.405 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:26.406 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:57.837084+0000", 2024-09-04T17:48:26.406 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.8229999999999997e-06 2024-09-04T17:48:26.406 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.406 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.406 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:26.406 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:57.837098+0000", 2024-09-04T17:48:26.406 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3436e-05 2024-09-04T17:48:26.406 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.406 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.406 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:26.406 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:57.837123+0000", 2024-09-04T17:48:26.406 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5079000000000002e-05 2024-09-04T17:48:26.406 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.407 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.407 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:26.407 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:57.837145+0000", 2024-09-04T17:48:26.407 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.228e-05 2024-09-04T17:48:26.407 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.407 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.407 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:26.407 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:57.837360+0000", 2024-09-04T17:48:26.407 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021464100000000001 2024-09-04T17:48:26.407 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.407 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.407 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for peered", 2024-09-04T17:48:26.407 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:57.837410+0000", 2024-09-04T17:48:26.408 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0235000000000003e-05 2024-09-04T17:48:26.408 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.408 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.408 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:26.408 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.900175+0000", 2024-09-04T17:48:26.408 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0627652830000001 2024-09-04T17:48:26.408 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.408 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.408 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for rw locks", 2024-09-04T17:48:26.408 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.900370+0000", 2024-09-04T17:48:26.408 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019497199999999999 2024-09-04T17:48:26.408 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.408 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.409 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:26.409 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.906957+0000", 2024-09-04T17:48:26.409 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0065866889999999997 2024-09-04T17:48:26.409 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.409 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.409 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:26.409 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.907088+0000", 2024-09-04T17:48:26.409 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013056100000000001 2024-09-04T17:48:26.409 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.409 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.409 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:26.409 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.909104+0000", 2024-09-04T17:48:26.409 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.002016487 2024-09-04T17:48:26.409 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.410 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:26.410 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.410 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.410 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:26.410 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:26.411 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-04T17:48:26.412 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:48:26.411+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:48:26.512 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:26.506+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:26.538 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:26.538 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:48:26.538 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:48:26.538 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:26.553 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-04T17:48:26.613 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:48:26.611+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:48:26.714 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:48:26.711+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:48:26.722 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:26.722 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:48:26.722 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:48:26.722 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:48:26.722 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:26.736 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-04T17:48:26.814 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:48:26.810+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:48:26.858 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:26.858 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:48:26.858 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:48:26.858 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:48:26.858 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.859 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60921 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 262144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:26.859 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822920+0000", 2024-09-04T17:48:26.859 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 73.033980956999997, 2024-09-04T17:48:26.859 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073904394999999998, 2024-09-04T17:48:26.859 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:26.859 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:26.859 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:26.859 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:26.859 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:26.859 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:26.859 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60921 2024-09-04T17:48:26.859 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.859 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:26.860 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.860 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:26.860 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822920+0000", 2024-09-04T17:48:26.860 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.860 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.860 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.860 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:26.860 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822920+0000", 2024-09-04T17:48:26.860 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.860 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.860 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.860 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:26.860 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822920+0000", 2024-09-04T17:48:26.860 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4999999999999999e-07 2024-09-04T17:48:26.861 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.861 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.861 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:26.861 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822921+0000", 2024-09-04T17:48:26.861 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.113e-06 2024-09-04T17:48:26.861 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.861 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.861 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:26.861 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822924+0000", 2024-09-04T17:48:26.861 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.354e-06 2024-09-04T17:48:26.861 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.861 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.861 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:26.861 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822927+0000", 2024-09-04T17:48:26.862 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6199999999999999e-06 2024-09-04T17:48:26.862 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.862 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.862 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:26.862 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896586+0000", 2024-09-04T17:48:26.862 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073658976000000001 2024-09-04T17:48:26.862 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.862 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.862 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:26.862 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896626+0000", 2024-09-04T17:48:26.862 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0484000000000001e-05 2024-09-04T17:48:26.862 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.862 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.863 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:26.863 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896824+0000", 2024-09-04T17:48:26.863 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000197598 2024-09-04T17:48:26.863 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.863 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:26.863 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.863 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.863 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.863 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60922 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:26.863 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822930+0000", 2024-09-04T17:48:26.863 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 73.033970382000007, 2024-09-04T17:48:26.863 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074178872000000007, 2024-09-04T17:48:26.863 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:26.864 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:26.864 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:26.864 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:26.864 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:26.864 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:26.864 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60922 2024-09-04T17:48:26.864 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.864 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:26.864 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.864 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:26.864 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822930+0000", 2024-09-04T17:48:26.864 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.864 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.865 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.865 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:26.865 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822930+0000", 2024-09-04T17:48:26.865 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.865 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.865 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.865 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:26.865 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822930+0000", 2024-09-04T17:48:26.865 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.36e-07 2024-09-04T17:48:26.865 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.865 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.865 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:26.865 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822932+0000", 2024-09-04T17:48:26.866 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.567e-06 2024-09-04T17:48:26.866 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.866 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.866 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:26.866 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822938+0000", 2024-09-04T17:48:26.866 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4840000000000003e-06 2024-09-04T17:48:26.866 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.866 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.866 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:26.866 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822942+0000", 2024-09-04T17:48:26.866 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2039999999999999e-06 2024-09-04T17:48:26.866 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.866 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.866 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:26.867 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896850+0000", 2024-09-04T17:48:26.867 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073907941000000005 2024-09-04T17:48:26.867 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.867 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.867 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:26.867 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896891+0000", 2024-09-04T17:48:26.867 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0964999999999998e-05 2024-09-04T17:48:26.867 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.867 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.867 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:26.867 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897109+0000", 2024-09-04T17:48:26.867 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021847500000000001 2024-09-04T17:48:26.867 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.868 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:26.868 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.868 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.868 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.868 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60923 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 475136~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:26.868 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822949+0000", 2024-09-04T17:48:26.868 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 73.033952092000007, 2024-09-04T17:48:26.868 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074426928000000003, 2024-09-04T17:48:26.868 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:26.868 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:26.868 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:26.868 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:26.868 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:26.868 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:26.869 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60923 2024-09-04T17:48:26.869 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.869 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:26.869 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.869 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:26.869 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822949+0000", 2024-09-04T17:48:26.869 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.869 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.869 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.869 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:26.869 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822949+0000", 2024-09-04T17:48:26.869 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.869 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.870 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.870 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:26.870 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822949+0000", 2024-09-04T17:48:26.870 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7399999999999999e-07 2024-09-04T17:48:26.870 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.870 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.870 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:26.870 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822951+0000", 2024-09-04T17:48:26.870 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1940000000000001e-06 2024-09-04T17:48:26.870 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.870 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.870 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:26.870 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822957+0000", 2024-09-04T17:48:26.871 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.0650000000000004e-06 2024-09-04T17:48:26.871 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.871 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.871 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:26.871 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822960+0000", 2024-09-04T17:48:26.871 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.8009999999999999e-06 2024-09-04T17:48:26.871 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.871 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.871 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:26.871 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897138+0000", 2024-09-04T17:48:26.871 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074177990999999999 2024-09-04T17:48:26.871 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.871 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.871 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:26.872 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897179+0000", 2024-09-04T17:48:26.872 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.1057999999999999e-05 2024-09-04T17:48:26.872 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.872 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.872 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:26.872 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897375+0000", 2024-09-04T17:48:26.872 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019644500000000001 2024-09-04T17:48:26.872 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.872 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:26.872 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.872 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.872 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.872 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60924 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:26.872 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822964+0000", 2024-09-04T17:48:26.873 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 73.033936906999998, 2024-09-04T17:48:26.873 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074531902999999997, 2024-09-04T17:48:26.873 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:26.873 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:26.873 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:26.873 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:26.873 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:26.873 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:26.873 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60924 2024-09-04T17:48:26.873 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.873 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:26.873 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.873 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:26.874 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822964+0000", 2024-09-04T17:48:26.874 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.874 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.874 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.874 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:26.874 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822964+0000", 2024-09-04T17:48:26.874 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.874 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.874 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.874 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:26.874 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822964+0000", 2024-09-04T17:48:26.874 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4999999999999999e-07 2024-09-04T17:48:26.874 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.874 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.875 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:26.875 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822966+0000", 2024-09-04T17:48:26.875 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.917e-06 2024-09-04T17:48:26.875 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.875 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.875 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:26.875 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822969+0000", 2024-09-04T17:48:26.875 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.315e-06 2024-09-04T17:48:26.875 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.875 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.875 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:26.875 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822972+0000", 2024-09-04T17:48:26.875 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.599e-06 2024-09-04T17:48:26.875 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.876 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.876 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:26.876 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897398+0000", 2024-09-04T17:48:26.876 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074426168000000001 2024-09-04T17:48:26.876 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.876 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.876 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:26.876 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897441+0000", 2024-09-04T17:48:26.876 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.3158000000000003e-05 2024-09-04T17:48:26.876 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.876 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.876 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:26.876 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897496+0000", 2024-09-04T17:48:26.877 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4496000000000001e-05 2024-09-04T17:48:26.877 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.877 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:26.877 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.877 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.877 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.877 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60925 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 720896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:26.877 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822975+0000", 2024-09-04T17:48:26.877 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 73.033925472000007, 2024-09-04T17:48:26.877 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074635272000000003, 2024-09-04T17:48:26.877 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:26.877 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:26.877 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:26.877 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:26.878 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:26.878 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:26.878 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60925 2024-09-04T17:48:26.878 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.878 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:26.878 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.878 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:26.878 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822975+0000", 2024-09-04T17:48:26.878 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.878 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.878 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.878 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:26.878 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822975+0000", 2024-09-04T17:48:26.879 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.879 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.879 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.879 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:26.879 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822975+0000", 2024-09-04T17:48:26.879 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.34e-07 2024-09-04T17:48:26.879 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.879 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.879 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:26.879 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822976+0000", 2024-09-04T17:48:26.879 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0359999999999999e-06 2024-09-04T17:48:26.879 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.879 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.879 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:26.880 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822980+0000", 2024-09-04T17:48:26.880 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.242e-06 2024-09-04T17:48:26.880 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.880 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.880 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:26.880 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822983+0000", 2024-09-04T17:48:26.880 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.8830000000000002e-06 2024-09-04T17:48:26.880 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.880 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.880 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:26.880 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897519+0000", 2024-09-04T17:48:26.880 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074536366000000007 2024-09-04T17:48:26.880 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.880 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.881 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:26.881 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897560+0000", 2024-09-04T17:48:26.881 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0677999999999999e-05 2024-09-04T17:48:26.881 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.881 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.881 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:26.881 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897610+0000", 2024-09-04T17:48:26.881 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0832999999999999e-05 2024-09-04T17:48:26.881 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.881 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:26.881 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.881 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.881 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.881 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60926 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 860160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:26.882 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822986+0000", 2024-09-04T17:48:26.882 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 73.033914874000004, 2024-09-04T17:48:26.882 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074736413000000002, 2024-09-04T17:48:26.882 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:26.882 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:26.882 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:26.882 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:26.882 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:26.882 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:26.882 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60926 2024-09-04T17:48:26.882 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.882 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:26.882 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.882 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:26.883 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822986+0000", 2024-09-04T17:48:26.883 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.883 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.883 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.883 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:26.883 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822986+0000", 2024-09-04T17:48:26.883 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.883 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.883 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.883 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:26.883 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822986+0000", 2024-09-04T17:48:26.883 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5600000000000002e-07 2024-09-04T17:48:26.883 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.883 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.884 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:26.884 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822987+0000", 2024-09-04T17:48:26.884 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.082e-06 2024-09-04T17:48:26.884 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.884 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.884 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:26.884 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822990+0000", 2024-09-04T17:48:26.884 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.2219999999999998e-06 2024-09-04T17:48:26.884 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.884 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.884 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:26.884 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822994+0000", 2024-09-04T17:48:26.884 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.19e-06 2024-09-04T17:48:26.885 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.885 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.885 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:26.885 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897632+0000", 2024-09-04T17:48:26.885 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074638205999999999 2024-09-04T17:48:26.885 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.885 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.885 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:26.885 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897672+0000", 2024-09-04T17:48:26.885 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.9873999999999998e-05 2024-09-04T17:48:26.885 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.885 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.885 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:26.885 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897722+0000", 2024-09-04T17:48:26.886 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0583e-05 2024-09-04T17:48:26.886 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.886 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:26.886 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.886 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.886 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.886 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60927 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 966656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:26.886 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823000+0000", 2024-09-04T17:48:26.886 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 73.033900692000003, 2024-09-04T17:48:26.886 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074834036000000007, 2024-09-04T17:48:26.886 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:26.886 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:26.886 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:26.886 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:26.887 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:26.887 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:26.887 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60927 2024-09-04T17:48:26.887 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.887 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:26.887 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.887 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:26.887 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823000+0000", 2024-09-04T17:48:26.887 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.887 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.887 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.887 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:26.887 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823000+0000", 2024-09-04T17:48:26.888 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.888 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.888 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.888 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:26.888 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823000+0000", 2024-09-04T17:48:26.888 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4099999999999999e-07 2024-09-04T17:48:26.888 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.888 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.888 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:26.888 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823003+0000", 2024-09-04T17:48:26.888 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0810000000000001e-06 2024-09-04T17:48:26.888 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.888 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.888 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:26.889 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823011+0000", 2024-09-04T17:48:26.889 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7090000000000008e-06 2024-09-04T17:48:26.889 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.889 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.889 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:26.889 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823017+0000", 2024-09-04T17:48:26.889 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.3890000000000004e-06 2024-09-04T17:48:26.889 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.889 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.889 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:26.889 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897745+0000", 2024-09-04T17:48:26.889 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074728252999999994 2024-09-04T17:48:26.889 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.890 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.890 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:26.890 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897785+0000", 2024-09-04T17:48:26.890 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.9830999999999999e-05 2024-09-04T17:48:26.890 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.890 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.890 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:26.890 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897834+0000", 2024-09-04T17:48:26.890 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9231999999999999e-05 2024-09-04T17:48:26.890 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.890 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:26.890 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.890 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.890 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.891 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60928 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:26.891 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823024+0000", 2024-09-04T17:48:26.891 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 73.033876761000002, 2024-09-04T17:48:26.891 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075067479000000006, 2024-09-04T17:48:26.891 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:26.891 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:26.891 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:26.891 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:26.891 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:26.891 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:26.891 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60928 2024-09-04T17:48:26.891 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.891 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:26.891 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.892 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:26.892 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823024+0000", 2024-09-04T17:48:26.892 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.892 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.892 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.892 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:26.892 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823024+0000", 2024-09-04T17:48:26.892 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.892 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.892 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.892 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:26.892 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823024+0000", 2024-09-04T17:48:26.892 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6499999999999999e-07 2024-09-04T17:48:26.892 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.893 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.893 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:26.893 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823027+0000", 2024-09-04T17:48:26.893 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.3099999999999999e-06 2024-09-04T17:48:26.893 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.893 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.893 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:26.893 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823033+0000", 2024-09-04T17:48:26.893 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7429999999999996e-06 2024-09-04T17:48:26.893 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.893 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.893 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:26.893 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823038+0000", 2024-09-04T17:48:26.893 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.561e-06 2024-09-04T17:48:26.894 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.894 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.894 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:26.894 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897855+0000", 2024-09-04T17:48:26.894 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074816740000000007 2024-09-04T17:48:26.894 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.894 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.894 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:26.894 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897894+0000", 2024-09-04T17:48:26.894 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.9085999999999998e-05 2024-09-04T17:48:26.894 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.894 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.894 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:26.894 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898091+0000", 2024-09-04T17:48:26.895 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000197574 2024-09-04T17:48:26.895 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.895 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:26.895 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.895 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.895 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.895 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60929 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1212416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:26.895 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823042+0000", 2024-09-04T17:48:26.895 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 73.033858319000004, 2024-09-04T17:48:26.895 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075301251, 2024-09-04T17:48:26.895 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:26.895 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:26.895 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:26.895 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:26.896 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:26.896 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:26.896 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60929 2024-09-04T17:48:26.896 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.896 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:26.896 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.896 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:26.896 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823042+0000", 2024-09-04T17:48:26.896 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.896 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.896 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.896 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:26.896 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823042+0000", 2024-09-04T17:48:26.896 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.897 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.897 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.897 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:26.897 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823043+0000", 2024-09-04T17:48:26.897 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.1100000000000002e-07 2024-09-04T17:48:26.897 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.897 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.897 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:26.897 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823045+0000", 2024-09-04T17:48:26.897 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.933e-06 2024-09-04T17:48:26.897 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.897 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.897 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:26.898 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823051+0000", 2024-09-04T17:48:26.898 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.9569999999999999e-06 2024-09-04T17:48:26.898 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.898 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.898 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:26.898 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823055+0000", 2024-09-04T17:48:26.898 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4479999999999996e-06 2024-09-04T17:48:26.898 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.898 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.898 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:26.898 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898117+0000", 2024-09-04T17:48:26.898 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075062344000000003 2024-09-04T17:48:26.898 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.898 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.899 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:26.899 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898157+0000", 2024-09-04T17:48:26.899 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0095999999999997e-05 2024-09-04T17:48:26.899 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.899 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.899 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:26.899 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898344+0000", 2024-09-04T17:48:26.899 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000186162 2024-09-04T17:48:26.899 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.899 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:26.899 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.899 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.899 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.899 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60930 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1351680~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:26.900 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823061+0000", 2024-09-04T17:48:26.900 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 73.033839439999994, 2024-09-04T17:48:26.900 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075545507999999997, 2024-09-04T17:48:26.900 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:26.900 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:26.900 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:26.900 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:26.900 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:26.900 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:26.900 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60930 2024-09-04T17:48:26.900 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.900 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:26.900 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.900 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:26.901 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823061+0000", 2024-09-04T17:48:26.901 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.901 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.901 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.901 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:26.901 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823061+0000", 2024-09-04T17:48:26.901 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.901 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.901 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.901 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:26.901 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823378+0000", 2024-09-04T17:48:26.901 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00031631299999999998 2024-09-04T17:48:26.901 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.902 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.902 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:26.902 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823381+0000", 2024-09-04T17:48:26.902 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4020000000000002e-06 2024-09-04T17:48:26.902 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.902 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.902 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:26.902 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823389+0000", 2024-09-04T17:48:26.902 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.1429999999999998e-06 2024-09-04T17:48:26.902 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.902 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.902 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:26.902 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823395+0000", 2024-09-04T17:48:26.902 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.0059999999999998e-06 2024-09-04T17:48:26.903 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.903 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.903 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:26.903 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898368+0000", 2024-09-04T17:48:26.903 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074973032999999994 2024-09-04T17:48:26.903 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.903 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.903 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:26.903 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898410+0000", 2024-09-04T17:48:26.903 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2243000000000002e-05 2024-09-04T17:48:26.903 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.903 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.903 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:26.903 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898607+0000", 2024-09-04T17:48:26.904 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019636799999999999 2024-09-04T17:48:26.904 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.904 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:26.904 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.904 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.904 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.904 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60931 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1458176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:26.904 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823407+0000", 2024-09-04T17:48:26.904 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 73.033493366000002, 2024-09-04T17:48:26.904 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075460334000000004, 2024-09-04T17:48:26.904 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:26.904 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:26.904 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:26.905 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:26.905 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:26.905 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:26.905 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60931 2024-09-04T17:48:26.905 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.905 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:26.905 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.905 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:26.905 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823407+0000", 2024-09-04T17:48:26.905 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.905 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.905 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.905 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:26.905 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823407+0000", 2024-09-04T17:48:26.906 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.906 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.906 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.906 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:26.906 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823408+0000", 2024-09-04T17:48:26.906 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9999999999999998e-07 2024-09-04T17:48:26.906 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.906 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.906 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:26.906 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823415+0000", 2024-09-04T17:48:26.906 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.5510000000000002e-06 2024-09-04T17:48:26.906 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.906 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.907 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:26.907 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823423+0000", 2024-09-04T17:48:26.907 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2760000000000003e-06 2024-09-04T17:48:26.907 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.907 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.907 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:26.907 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823426+0000", 2024-09-04T17:48:26.907 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.2839999999999998e-06 2024-09-04T17:48:26.907 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.907 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.907 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:26.907 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898631+0000", 2024-09-04T17:48:26.907 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075205045999999998 2024-09-04T17:48:26.907 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.908 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.908 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:26.908 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898672+0000", 2024-09-04T17:48:26.908 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0614000000000001e-05 2024-09-04T17:48:26.908 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.908 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.908 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:26.908 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898868+0000", 2024-09-04T17:48:26.908 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019606300000000001 2024-09-04T17:48:26.908 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.908 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:26.908 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.908 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.908 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.909 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60932 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1564672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:26.909 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823434+0000", 2024-09-04T17:48:26.909 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 73.033467067000004, 2024-09-04T17:48:26.909 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075693393999999997, 2024-09-04T17:48:26.909 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:26.909 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:26.909 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:26.909 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:26.909 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:26.909 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:26.909 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60932 2024-09-04T17:48:26.909 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.909 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:26.910 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.910 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:26.910 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823434+0000", 2024-09-04T17:48:26.910 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.910 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.910 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.910 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:26.910 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823434+0000", 2024-09-04T17:48:26.910 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.910 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.910 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.910 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:26.910 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.830290+0000", 2024-09-04T17:48:26.910 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0068563039999999997 2024-09-04T17:48:26.911 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.911 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.911 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:26.911 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.830297+0000", 2024-09-04T17:48:26.911 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4760000000000001e-06 2024-09-04T17:48:26.911 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.911 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.911 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:26.911 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.830308+0000", 2024-09-04T17:48:26.911 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0927999999999999e-05 2024-09-04T17:48:26.911 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.911 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.911 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:26.911 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.830317+0000", 2024-09-04T17:48:26.912 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7280000000000001e-06 2024-09-04T17:48:26.912 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.912 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.912 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:26.912 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898892+0000", 2024-09-04T17:48:26.912 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.068574722000000005 2024-09-04T17:48:26.912 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.912 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.912 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:26.912 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898934+0000", 2024-09-04T17:48:26.912 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2249000000000002e-05 2024-09-04T17:48:26.912 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.912 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.912 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:26.913 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.899127+0000", 2024-09-04T17:48:26.913 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019298699999999999 2024-09-04T17:48:26.913 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.913 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:26.913 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.913 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.913 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.913 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61182 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 868352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:26.913 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839249+0000", 2024-09-04T17:48:26.913 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 73.017651174999997, 2024-09-04T17:48:26.913 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073680151999999999, 2024-09-04T17:48:26.913 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:26.913 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:26.913 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:26.914 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:26.914 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:26.914 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:26.914 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61182 2024-09-04T17:48:26.914 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.914 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:26.914 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.914 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:26.914 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839249+0000", 2024-09-04T17:48:26.914 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.914 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.914 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.914 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:26.914 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839249+0000", 2024-09-04T17:48:26.915 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.915 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.915 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.915 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:26.915 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839250+0000", 2024-09-04T17:48:26.915 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4999999999999998e-07 2024-09-04T17:48:26.915 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.915 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.915 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:26.915 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839257+0000", 2024-09-04T17:48:26.915 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7700000000000004e-06 2024-09-04T17:48:26.915 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.915 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.916 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:26.916 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839263+0000", 2024-09-04T17:48:26.916 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.246e-06 2024-09-04T17:48:26.916 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.916 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.916 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:26.916 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839268+0000", 2024-09-04T17:48:26.916 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.814e-06 2024-09-04T17:48:26.916 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.916 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.916 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:26.916 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.912831+0000", 2024-09-04T17:48:26.916 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073563294000000001 2024-09-04T17:48:26.916 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.917 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.917 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:26.917 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.912876+0000", 2024-09-04T17:48:26.917 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4742e-05 2024-09-04T17:48:26.917 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.917 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.917 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:26.917 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.912930+0000", 2024-09-04T17:48:26.917 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.3936e-05 2024-09-04T17:48:26.917 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.917 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:26.917 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.917 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.917 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.918 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61183 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:26.918 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839275+0000", 2024-09-04T17:48:26.918 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 73.017625981999998, 2024-09-04T17:48:26.918 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073781886000000005, 2024-09-04T17:48:26.918 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:26.918 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:26.918 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:26.918 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:26.918 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:26.918 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:26.918 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61183 2024-09-04T17:48:26.918 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.918 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:26.919 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.919 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:26.919 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839275+0000", 2024-09-04T17:48:26.919 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.919 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.919 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.919 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:26.919 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839275+0000", 2024-09-04T17:48:26.919 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.919 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.919 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.919 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:26.919 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839275+0000", 2024-09-04T17:48:26.919 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6899999999999997e-07 2024-09-04T17:48:26.920 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.920 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.920 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:26.920 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839277+0000", 2024-09-04T17:48:26.920 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9530000000000002e-06 2024-09-04T17:48:26.920 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.920 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.920 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:26.920 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839283+0000", 2024-09-04T17:48:26.920 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.9329999999999998e-06 2024-09-04T17:48:26.920 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.920 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.921 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:26.921 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839288+0000", 2024-09-04T17:48:26.921 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4100000000000001e-06 2024-09-04T17:48:26.921 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.921 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.921 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:26.921 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.912959+0000", 2024-09-04T17:48:26.921 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073671511999999995 2024-09-04T17:48:26.921 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.921 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.922 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:26.922 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913001+0000", 2024-09-04T17:48:26.922 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.1984999999999999e-05 2024-09-04T17:48:26.922 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.922 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.922 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:26.922 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913057+0000", 2024-09-04T17:48:26.922 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.5523999999999999e-05 2024-09-04T17:48:26.922 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.922 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:26.922 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.922 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.922 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.922 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61184 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1114112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:26.923 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839294+0000", 2024-09-04T17:48:26.923 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 73.01760677, 2024-09-04T17:48:26.923 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073995263000000006, 2024-09-04T17:48:26.923 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:26.923 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:26.923 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:26.923 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:26.923 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:26.923 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:26.923 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61184 2024-09-04T17:48:26.923 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.923 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:26.923 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.923 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:26.924 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839294+0000", 2024-09-04T17:48:26.924 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.924 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.924 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.924 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:26.924 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839294+0000", 2024-09-04T17:48:26.924 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.924 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.924 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.924 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:26.924 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839294+0000", 2024-09-04T17:48:26.924 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.77e-07 2024-09-04T17:48:26.924 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.924 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.925 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:26.925 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839296+0000", 2024-09-04T17:48:26.925 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9999999999999999e-06 2024-09-04T17:48:26.925 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.925 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.925 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:26.925 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839302+0000", 2024-09-04T17:48:26.925 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7230000000000001e-06 2024-09-04T17:48:26.925 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.925 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.925 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:26.925 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839314+0000", 2024-09-04T17:48:26.925 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1826e-05 2024-09-04T17:48:26.925 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.926 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.926 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:26.926 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913084+0000", 2024-09-04T17:48:26.926 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073770215 2024-09-04T17:48:26.926 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.926 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.926 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:26.926 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913127+0000", 2024-09-04T17:48:26.926 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2982999999999998e-05 2024-09-04T17:48:26.926 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.926 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.926 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:26.926 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913289+0000", 2024-09-04T17:48:26.927 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000162139 2024-09-04T17:48:26.927 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.927 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:26.927 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.927 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.927 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.927 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61185 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1220608~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:26.927 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839321+0000", 2024-09-04T17:48:26.927 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 73.017579303000005, 2024-09-04T17:48:26.927 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074206062000000003, 2024-09-04T17:48:26.927 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:26.927 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:26.927 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:26.927 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:26.927 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:26.928 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:26.928 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61185 2024-09-04T17:48:26.928 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.928 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:26.928 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.928 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:26.928 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839321+0000", 2024-09-04T17:48:26.928 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.928 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.928 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.928 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:26.928 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839321+0000", 2024-09-04T17:48:26.928 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.928 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.929 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.929 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:26.929 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839322+0000", 2024-09-04T17:48:26.929 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.5400000000000001e-07 2024-09-04T17:48:26.929 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.929 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.929 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:26.929 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839324+0000", 2024-09-04T17:48:26.929 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2539999999999999e-06 2024-09-04T17:48:26.929 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.929 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.929 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:26.929 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839331+0000", 2024-09-04T17:48:26.930 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2139999999999999e-06 2024-09-04T17:48:26.930 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.930 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.930 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:26.930 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839336+0000", 2024-09-04T17:48:26.930 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7029999999999997e-06 2024-09-04T17:48:26.930 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.930 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.930 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:26.930 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913320+0000", 2024-09-04T17:48:26.930 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073984016 2024-09-04T17:48:26.930 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.930 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.930 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:26.931 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913366+0000", 2024-09-04T17:48:26.931 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.5704000000000002e-05 2024-09-04T17:48:26.931 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.931 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.931 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:26.931 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913527+0000", 2024-09-04T17:48:26.931 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00016161699999999999 2024-09-04T17:48:26.931 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.931 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:26.931 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.931 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.931 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.931 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61186 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1359872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:26.932 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839344+0000", 2024-09-04T17:48:26.932 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 73.017556960999997, 2024-09-04T17:48:26.932 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074450673999999994, 2024-09-04T17:48:26.932 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:26.932 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:26.932 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:26.932 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:26.932 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:26.932 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:26.932 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61186 2024-09-04T17:48:26.932 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.932 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:26.932 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.932 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:26.933 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839344+0000", 2024-09-04T17:48:26.933 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.933 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.933 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.933 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:26.933 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839344+0000", 2024-09-04T17:48:26.933 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.933 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.933 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.933 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:26.933 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839344+0000", 2024-09-04T17:48:26.933 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4799999999999999e-07 2024-09-04T17:48:26.933 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.933 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.934 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:26.934 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839346+0000", 2024-09-04T17:48:26.934 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0889999999999998e-06 2024-09-04T17:48:26.934 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.934 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.934 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:26.934 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839352+0000", 2024-09-04T17:48:26.934 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6389999999999997e-06 2024-09-04T17:48:26.934 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.934 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.934 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:26.934 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839356+0000", 2024-09-04T17:48:26.934 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.233e-06 2024-09-04T17:48:26.935 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.935 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.935 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:26.935 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913557+0000", 2024-09-04T17:48:26.935 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074200944000000005 2024-09-04T17:48:26.935 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.935 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.935 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:26.935 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913601+0000", 2024-09-04T17:48:26.935 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4350000000000001e-05 2024-09-04T17:48:26.935 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.935 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.935 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:26.935 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913794+0000", 2024-09-04T17:48:26.936 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019297100000000001 2024-09-04T17:48:26.936 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.936 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:26.936 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.936 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.936 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.936 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61187 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1466368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:26.936 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839364+0000", 2024-09-04T17:48:26.936 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 73.017537111999999, 2024-09-04T17:48:26.936 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074692334999999999, 2024-09-04T17:48:26.936 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:26.936 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:26.936 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:26.936 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:26.937 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:26.937 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:26.937 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61187 2024-09-04T17:48:26.937 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.937 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:26.937 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.937 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:26.937 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839364+0000", 2024-09-04T17:48:26.937 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.937 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.937 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.937 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:26.937 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839364+0000", 2024-09-04T17:48:26.937 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.938 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.938 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.938 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:26.938 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839364+0000", 2024-09-04T17:48:26.938 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.75e-07 2024-09-04T17:48:26.938 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.938 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.938 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:26.938 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839366+0000", 2024-09-04T17:48:26.938 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0990000000000001e-06 2024-09-04T17:48:26.938 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.938 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.938 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:26.938 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839372+0000", 2024-09-04T17:48:26.939 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.2609999999999999e-06 2024-09-04T17:48:26.939 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.939 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.939 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:26.939 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839389+0000", 2024-09-04T17:48:26.939 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6489000000000001e-05 2024-09-04T17:48:26.939 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.939 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.939 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:26.939 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913824+0000", 2024-09-04T17:48:26.939 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074434811000000004 2024-09-04T17:48:26.939 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.939 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.939 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:26.940 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913868+0000", 2024-09-04T17:48:26.940 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4768000000000002e-05 2024-09-04T17:48:26.940 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.940 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.940 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:26.940 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914056+0000", 2024-09-04T17:48:26.940 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00018743199999999999 2024-09-04T17:48:26.940 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.940 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:26.940 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.940 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.940 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.940 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61188 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1605632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:26.940 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839399+0000", 2024-09-04T17:48:26.941 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 73.017501215999999, 2024-09-04T17:48:26.941 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074975095000000005, 2024-09-04T17:48:26.941 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:26.941 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:26.941 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:26.941 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:26.941 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:26.941 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:26.941 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61188 2024-09-04T17:48:26.941 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.941 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:26.941 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.941 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:26.941 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839399+0000", 2024-09-04T17:48:26.942 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.942 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.942 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.942 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:26.942 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839399+0000", 2024-09-04T17:48:26.942 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.942 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.942 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.942 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:26.942 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839400+0000", 2024-09-04T17:48:26.942 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0100000000000005e-07 2024-09-04T17:48:26.942 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.942 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.942 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:26.943 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839402+0000", 2024-09-04T17:48:26.943 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.491e-06 2024-09-04T17:48:26.943 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.943 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.943 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:26.943 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839409+0000", 2024-09-04T17:48:26.943 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.0450000000000003e-06 2024-09-04T17:48:26.943 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.943 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.943 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:26.943 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839414+0000", 2024-09-04T17:48:26.943 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.87e-06 2024-09-04T17:48:26.943 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.943 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.944 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:26.944 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914086+0000", 2024-09-04T17:48:26.944 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074671596000000007 2024-09-04T17:48:26.944 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.944 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.944 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:26.944 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914130+0000", 2024-09-04T17:48:26.944 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4579999999999997e-05 2024-09-04T17:48:26.944 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.944 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.944 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:26.944 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914375+0000", 2024-09-04T17:48:26.944 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024401199999999999 2024-09-04T17:48:26.945 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.945 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:26.945 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.945 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.945 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.945 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61189 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1712128~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:26.945 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839424+0000", 2024-09-04T17:48:26.945 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 73.017477014999997, 2024-09-04T17:48:26.945 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075247680999999997, 2024-09-04T17:48:26.945 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:26.945 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:26.945 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:26.945 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:26.945 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:26.946 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:26.946 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61189 2024-09-04T17:48:26.946 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.946 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:26.946 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.946 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:26.946 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839424+0000", 2024-09-04T17:48:26.946 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.946 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.946 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.946 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:26.946 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839424+0000", 2024-09-04T17:48:26.946 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:26.946 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.947 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.947 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:26.947 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.846320+0000", 2024-09-04T17:48:26.947 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0068965210000000001 2024-09-04T17:48:26.947 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.947 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.947 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:26.947 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.846328+0000", 2024-09-04T17:48:26.947 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.8189999999999992e-06 2024-09-04T17:48:26.947 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.948 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.948 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:26.948 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.846341+0000", 2024-09-04T17:48:26.948 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3338e-05 2024-09-04T17:48:26.948 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.948 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.948 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:26.948 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.846352+0000", 2024-09-04T17:48:26.948 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0479000000000001e-05 2024-09-04T17:48:26.948 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.948 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.949 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:26.949 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914405+0000", 2024-09-04T17:48:26.949 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.068053049000000004 2024-09-04T17:48:26.949 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.949 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.949 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:26.949 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914451+0000", 2024-09-04T17:48:26.949 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6545999999999998e-05 2024-09-04T17:48:26.949 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:26.949 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:26.949 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:26.949 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914671+0000", 2024-09-04T17:48:26.949 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000219929 2024-09-04T17:48:26.950 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.950 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:26.950 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.950 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:26.950 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:26.950 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:26.951 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-04T17:48:26.952 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:26.951+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:27.052 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:48:27.050+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:48:27.090 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:27.090 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:48:27.090 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:48:27.090 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:27.105 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-04T17:48:27.153 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:27.154+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:27.226 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:27.227 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:48:27.227 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:48:27.227 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:48:27.227 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:27.241 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-04T17:48:27.254 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:27.254+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:27.355 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:27.354+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:27.367 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:27.368 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:48:27.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:48:27.368 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:48:27.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.368 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:27242 3.b 3.34414deb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-04T17:48:27.368 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:28.402347+0000", 2024-09-04T17:48:27.368 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 118.962680261, 2024-09-04T17:48:27.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.93265329299999999, 2024-09-04T17:48:27.368 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:27.368 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:27.368 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-04T17:48:27.368 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:27.369 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:27.369 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:27.369 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 27242 2024-09-04T17:48:27.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.369 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:27.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:27.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402347+0000", 2024-09-04T17:48:27.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:27.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:27.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402347+0000", 2024-09-04T17:48:27.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:27.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:27.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402349+0000", 2024-09-04T17:48:27.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.471e-06 2024-09-04T17:48:27.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:27.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402355+0000", 2024-09-04T17:48:27.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8399999999999997e-06 2024-09-04T17:48:27.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:27.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402364+0000", 2024-09-04T17:48:27.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1269999999999992e-06 2024-09-04T17:48:27.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:27.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402373+0000", 2024-09-04T17:48:27.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7909999999999993e-06 2024-09-04T17:48:27.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:27.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402460+0000", 2024-09-04T17:48:27.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6686000000000003e-05 2024-09-04T17:48:27.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-04T17:48:27.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402473+0000", 2024-09-04T17:48:27.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3281e-05 2024-09-04T17:48:27.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:27.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:29.334989+0000", 2024-09-04T17:48:27.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.93251549099999997 2024-09-04T17:48:27.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:27.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:29.335000+0000", 2024-09-04T17:48:27.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1606e-05 2024-09-04T17:48:27.373 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:27.373 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:27.374 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:27.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.374 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:33552 3.7 3:e79ad1db:::rbd_data.11342e8f3384.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-04T17:48:27.374 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:35.999748+0000", 2024-09-04T17:48:27.374 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 111.365279211, 2024-09-04T17:48:27.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045683823999999998, 2024-09-04T17:48:27.374 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:27.374 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:27.374 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:27.374 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:27.375 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:27.375 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:27.375 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 33552 2024-09-04T17:48:27.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.375 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:27.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:27.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999748+0000", 2024-09-04T17:48:27.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:27.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:27.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999748+0000", 2024-09-04T17:48:27.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:27.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:27.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999750+0000", 2024-09-04T17:48:27.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1299999999999999e-06 2024-09-04T17:48:27.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:27.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999759+0000", 2024-09-04T17:48:27.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4649999999999994e-06 2024-09-04T17:48:27.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:27.378 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999778+0000", 2024-09-04T17:48:27.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8833000000000001e-05 2024-09-04T17:48:27.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.378 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:27.378 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999786+0000", 2024-09-04T17:48:27.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7369999999999995e-06 2024-09-04T17:48:27.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.378 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:27.378 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999856+0000", 2024-09-04T17:48:27.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9247999999999998e-05 2024-09-04T17:48:27.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:27.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999973+0000", 2024-09-04T17:48:27.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000117921 2024-09-04T17:48:27.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 7", 2024-09-04T17:48:27.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.005955+0000", 2024-09-04T17:48:27.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059820560000000004 2024-09-04T17:48:27.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:27.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.045297+0000", 2024-09-04T17:48:27.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.039341650999999998 2024-09-04T17:48:27.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:27.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.045343+0000", 2024-09-04T17:48:27.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5899000000000002e-05 2024-09-04T17:48:27.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:27.382 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.045388+0000", 2024-09-04T17:48:27.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4601999999999997e-05 2024-09-04T17:48:27.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:27.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.045432+0000", 2024-09-04T17:48:27.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4282000000000001e-05 2024-09-04T17:48:27.383 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:27.383 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:27.383 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:27.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.384 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:37357 3.15 3:a897312a:::rbd_data.1134c5a43465.000000000000001f:head [stat,write 62464~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-04T17:48:27.384 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:39.616724+0000", 2024-09-04T17:48:27.384 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 107.748303803, 2024-09-04T17:48:27.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054965854000000001, 2024-09-04T17:48:27.384 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:27.385 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:27.385 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:27.385 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:27.385 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:27.385 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:27.385 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 37357 2024-09-04T17:48:27.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.386 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:27.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:27.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616724+0000", 2024-09-04T17:48:27.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:27.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:27.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616724+0000", 2024-09-04T17:48:27.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:27.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:27.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616725+0000", 2024-09-04T17:48:27.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.815e-06 2024-09-04T17:48:27.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:27.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616739+0000", 2024-09-04T17:48:27.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3723000000000001e-05 2024-09-04T17:48:27.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:27.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616755+0000", 2024-09-04T17:48:27.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5964000000000002e-05 2024-09-04T17:48:27.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:27.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616763+0000", 2024-09-04T17:48:27.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3869999999999995e-06 2024-09-04T17:48:27.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:27.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616825+0000", 2024-09-04T17:48:27.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1854999999999996e-05 2024-09-04T17:48:27.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:27.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616895+0000", 2024-09-04T17:48:27.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9597999999999993e-05 2024-09-04T17:48:27.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-04T17:48:27.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616991+0000", 2024-09-04T17:48:27.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5828000000000005e-05 2024-09-04T17:48:27.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:27.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.617659+0000", 2024-09-04T17:48:27.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00066821600000000001 2024-09-04T17:48:27.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:27.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.671482+0000", 2024-09-04T17:48:27.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053823533999999999 2024-09-04T17:48:27.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:27.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.671579+0000", 2024-09-04T17:48:27.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6657999999999995e-05 2024-09-04T17:48:27.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:27.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.671689+0000", 2024-09-04T17:48:27.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000110276 2024-09-04T17:48:27.392 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:27.392 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:27.392 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:27.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.392 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:43873 3.15 3:abd207ee:::rbd_data.113410ddd206.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:27.393 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:49.932940+0000", 2024-09-04T17:48:27.393 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 97.432087249999995, 2024-09-04T17:48:27.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042740514, 2024-09-04T17:48:27.393 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:27.393 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:27.393 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:27.393 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:27.393 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:27.393 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:27.393 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 43873 2024-09-04T17:48:27.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.394 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:27.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:27.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.932940+0000", 2024-09-04T17:48:27.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:27.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:27.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.932940+0000", 2024-09-04T17:48:27.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:27.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:27.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.932941+0000", 2024-09-04T17:48:27.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.404e-06 2024-09-04T17:48:27.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:27.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.933065+0000", 2024-09-04T17:48:27.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012361199999999999 2024-09-04T17:48:27.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:27.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.933078+0000", 2024-09-04T17:48:27.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3182e-05 2024-09-04T17:48:27.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:27.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.933085+0000", 2024-09-04T17:48:27.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4500000000000001e-06 2024-09-04T17:48:27.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:27.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.933134+0000", 2024-09-04T17:48:27.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9150999999999997e-05 2024-09-04T17:48:27.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:27.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.933242+0000", 2024-09-04T17:48:27.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010781 2024-09-04T17:48:27.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-04T17:48:27.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.938358+0000", 2024-09-04T17:48:27.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051160349999999997 2024-09-04T17:48:27.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:27.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.975343+0000", 2024-09-04T17:48:27.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036985663000000002 2024-09-04T17:48:27.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:27.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.975392+0000", 2024-09-04T17:48:27.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9044e-05 2024-09-04T17:48:27.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:27.401 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.975548+0000", 2024-09-04T17:48:27.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015572900000000001 2024-09-04T17:48:27.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:27.401 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.975681+0000", 2024-09-04T17:48:27.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013243400000000001 2024-09-04T17:48:27.401 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:27.401 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:27.402 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:27.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.402 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:46649 3.7 3:e0749ed5:::rbd_data.1134326b938e.000000000000014f:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-04T17:48:27.402 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:54.114730+0000", 2024-09-04T17:48:27.402 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 93.250297118000006, 2024-09-04T17:48:27.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032972972000000003, 2024-09-04T17:48:27.402 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:27.402 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:27.402 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:27.402 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:27.402 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:27.402 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:27.403 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 46649 2024-09-04T17:48:27.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.403 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:27.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:27.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114730+0000", 2024-09-04T17:48:27.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:27.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:27.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114730+0000", 2024-09-04T17:48:27.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:27.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:27.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114731+0000", 2024-09-04T17:48:27.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.147e-06 2024-09-04T17:48:27.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:27.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114737+0000", 2024-09-04T17:48:27.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6380000000000001e-06 2024-09-04T17:48:27.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:27.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114751+0000", 2024-09-04T17:48:27.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4012999999999999e-05 2024-09-04T17:48:27.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:27.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114758+0000", 2024-09-04T17:48:27.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1539999999999996e-06 2024-09-04T17:48:27.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:27.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114876+0000", 2024-09-04T17:48:27.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000117421 2024-09-04T17:48:27.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:27.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.115173+0000", 2024-09-04T17:48:27.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00029750799999999998 2024-09-04T17:48:27.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 7", 2024-09-04T17:48:27.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.120273+0000", 2024-09-04T17:48:27.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051002900000000004 2024-09-04T17:48:27.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:27.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.147571+0000", 2024-09-04T17:48:27.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027297906 2024-09-04T17:48:27.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:27.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.147588+0000", 2024-09-04T17:48:27.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6807999999999999e-05 2024-09-04T17:48:27.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:27.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.147637+0000", 2024-09-04T17:48:27.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8466e-05 2024-09-04T17:48:27.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:27.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.147703+0000", 2024-09-04T17:48:27.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6620999999999998e-05 2024-09-04T17:48:27.408 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:27.408 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:27.409 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:27.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.409 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49801 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 720896~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:48:27.409 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.075094+0000", 2024-09-04T17:48:27.409 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 84.289933130999998, 2024-09-04T17:48:27.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017824974, 2024-09-04T17:48:27.410 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:27.410 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:27.410 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:27.410 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:27.410 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:27.411 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:27.411 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49801 2024-09-04T17:48:27.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.411 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:27.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.411 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:27.411 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075094+0000", 2024-09-04T17:48:27.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:27.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:27.412 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075094+0000", 2024-09-04T17:48:27.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:27.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:27.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075095+0000", 2024-09-04T17:48:27.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.017e-06 2024-09-04T17:48:27.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:27.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075101+0000", 2024-09-04T17:48:27.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5029999999999997e-06 2024-09-04T17:48:27.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:27.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075119+0000", 2024-09-04T17:48:27.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8133000000000001e-05 2024-09-04T17:48:27.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:27.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075132+0000", 2024-09-04T17:48:27.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3305999999999999e-05 2024-09-04T17:48:27.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:27.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.092677+0000", 2024-09-04T17:48:27.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017544466000000002 2024-09-04T17:48:27.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:27.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.092782+0000", 2024-09-04T17:48:27.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000105284 2024-09-04T17:48:27.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:27.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.092919+0000", 2024-09-04T17:48:27.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013726499999999999 2024-09-04T17:48:27.418 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:27.418 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:27.418 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:27.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.419 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49809 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 163840~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:48:27.419 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077146+0000", 2024-09-04T17:48:27.419 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 84.287881687999999, 2024-09-04T17:48:27.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017984541999999999, 2024-09-04T17:48:27.419 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:27.419 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:27.419 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:27.419 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:27.419 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:27.419 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:27.420 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49809 2024-09-04T17:48:27.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.420 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:27.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:27.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077146+0000", 2024-09-04T17:48:27.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:27.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:27.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077146+0000", 2024-09-04T17:48:27.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:27.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:27.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077147+0000", 2024-09-04T17:48:27.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0330000000000001e-06 2024-09-04T17:48:27.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:27.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077164+0000", 2024-09-04T17:48:27.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7810000000000001e-05 2024-09-04T17:48:27.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:27.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077187+0000", 2024-09-04T17:48:27.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2246e-05 2024-09-04T17:48:27.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:27.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077203+0000", 2024-09-04T17:48:27.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6392000000000002e-05 2024-09-04T17:48:27.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:27.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.094885+0000", 2024-09-04T17:48:27.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017681589000000001 2024-09-04T17:48:27.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:27.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.094991+0000", 2024-09-04T17:48:27.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010668899999999999 2024-09-04T17:48:27.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:27.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095130+0000", 2024-09-04T17:48:27.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013878299999999999 2024-09-04T17:48:27.423 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:27.423 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:27.423 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:27.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.424 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49810 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 229376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:48:27.424 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077225+0000", 2024-09-04T17:48:27.424 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 84.287801846999997, 2024-09-04T17:48:27.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018218793, 2024-09-04T17:48:27.424 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:27.424 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:27.424 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:27.424 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:27.424 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:27.424 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:27.424 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49810 2024-09-04T17:48:27.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.424 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:27.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:27.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077225+0000", 2024-09-04T17:48:27.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:27.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:27.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077225+0000", 2024-09-04T17:48:27.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:27.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:27.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077227+0000", 2024-09-04T17:48:27.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1829999999999999e-06 2024-09-04T17:48:27.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:27.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077233+0000", 2024-09-04T17:48:27.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3899999999999998e-06 2024-09-04T17:48:27.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:27.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077253+0000", 2024-09-04T17:48:27.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0009000000000001e-05 2024-09-04T17:48:27.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:27.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077270+0000", 2024-09-04T17:48:27.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7336000000000001e-05 2024-09-04T17:48:27.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:27.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095177+0000", 2024-09-04T17:48:27.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017907101000000002 2024-09-04T17:48:27.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:27.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095299+0000", 2024-09-04T17:48:27.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000121749 2024-09-04T17:48:27.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:27.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095444+0000", 2024-09-04T17:48:27.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014502499999999999 2024-09-04T17:48:27.429 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:27.429 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:27.429 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:27.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.429 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49811 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 294912~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:48:27.429 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077288+0000", 2024-09-04T17:48:27.429 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 84.287739533000007, 2024-09-04T17:48:27.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018443620000000001, 2024-09-04T17:48:27.429 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:27.429 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:27.430 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:27.430 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:27.430 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:27.430 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:27.430 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49811 2024-09-04T17:48:27.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.430 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:27.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.430 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:27.430 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077288+0000", 2024-09-04T17:48:27.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:27.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:27.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077288+0000", 2024-09-04T17:48:27.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:27.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:27.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077289+0000", 2024-09-04T17:48:27.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6199999999999999e-06 2024-09-04T17:48:27.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:27.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077295+0000", 2024-09-04T17:48:27.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1499999999999998e-06 2024-09-04T17:48:27.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:27.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077312+0000", 2024-09-04T17:48:27.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7353000000000001e-05 2024-09-04T17:48:27.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:27.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077324+0000", 2024-09-04T17:48:27.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2593e-05 2024-09-04T17:48:27.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:27.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095490+0000", 2024-09-04T17:48:27.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018165976 2024-09-04T17:48:27.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:27.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095597+0000", 2024-09-04T17:48:27.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010629900000000001 2024-09-04T17:48:27.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:27.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095731+0000", 2024-09-04T17:48:27.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013462899999999999 2024-09-04T17:48:27.433 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:27.433 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:27.434 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:27.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.434 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49813 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 360448~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:48:27.434 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077349+0000", 2024-09-04T17:48:27.434 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 84.287678330999995, 2024-09-04T17:48:27.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018780305000000001, 2024-09-04T17:48:27.434 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:27.434 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:27.434 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:27.434 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:27.434 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:27.435 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:27.435 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49813 2024-09-04T17:48:27.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.435 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:27.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:27.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077349+0000", 2024-09-04T17:48:27.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:27.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:27.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077349+0000", 2024-09-04T17:48:27.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:27.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:27.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077350+0000", 2024-09-04T17:48:27.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0330000000000001e-06 2024-09-04T17:48:27.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:27.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077355+0000", 2024-09-04T17:48:27.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0699999999999997e-06 2024-09-04T17:48:27.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:27.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077376+0000", 2024-09-04T17:48:27.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0846e-05 2024-09-04T17:48:27.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:27.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077388+0000", 2024-09-04T17:48:27.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2353e-05 2024-09-04T17:48:27.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:27.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095819+0000", 2024-09-04T17:48:27.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01843057 2024-09-04T17:48:27.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:27.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095960+0000", 2024-09-04T17:48:27.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000140965 2024-09-04T17:48:27.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:27.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096129+0000", 2024-09-04T17:48:27.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016946800000000001 2024-09-04T17:48:27.438 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:27.438 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:27.438 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:27.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.438 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49814 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 425984~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:48:27.439 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077407+0000", 2024-09-04T17:48:27.439 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 84.287619829999997, 2024-09-04T17:48:27.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019041563000000001, 2024-09-04T17:48:27.439 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:27.439 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:27.439 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:27.439 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:27.439 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:27.439 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:27.439 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49814 2024-09-04T17:48:27.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.439 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:27.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:27.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077407+0000", 2024-09-04T17:48:27.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:27.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:27.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077407+0000", 2024-09-04T17:48:27.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:27.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:27.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077409+0000", 2024-09-04T17:48:27.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2330000000000001e-06 2024-09-04T17:48:27.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:27.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077415+0000", 2024-09-04T17:48:27.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2729999999999999e-06 2024-09-04T17:48:27.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:27.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077436+0000", 2024-09-04T17:48:27.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1209999999999999e-05 2024-09-04T17:48:27.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:27.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077448+0000", 2024-09-04T17:48:27.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2225999999999999e-05 2024-09-04T17:48:27.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:27.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096183+0000", 2024-09-04T17:48:27.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018735004 2024-09-04T17:48:27.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:27.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096297+0000", 2024-09-04T17:48:27.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000113962 2024-09-04T17:48:27.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:27.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096449+0000", 2024-09-04T17:48:27.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000151655 2024-09-04T17:48:27.443 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:27.443 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:27.443 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:27.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.443 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49815 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 491520~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:48:27.443 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077471+0000", 2024-09-04T17:48:27.443 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 84.287556592000001, 2024-09-04T17:48:27.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019266536000000001, 2024-09-04T17:48:27.443 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:27.444 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:27.444 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:27.444 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:27.444 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:27.444 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:27.444 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49815 2024-09-04T17:48:27.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.444 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:27.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:27.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077471+0000", 2024-09-04T17:48:27.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:27.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:27.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077471+0000", 2024-09-04T17:48:27.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:27.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:27.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077472+0000", 2024-09-04T17:48:27.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.17e-06 2024-09-04T17:48:27.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:27.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077478+0000", 2024-09-04T17:48:27.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0399999999999998e-06 2024-09-04T17:48:27.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:27.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077500+0000", 2024-09-04T17:48:27.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1959000000000001e-05 2024-09-04T17:48:27.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:27.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077513+0000", 2024-09-04T17:48:27.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2823000000000001e-05 2024-09-04T17:48:27.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:27.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096493+0000", 2024-09-04T17:48:27.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018980701999999999 2024-09-04T17:48:27.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:27.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096603+0000", 2024-09-04T17:48:27.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010942000000000001 2024-09-04T17:48:27.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:27.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096737+0000", 2024-09-04T17:48:27.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013442200000000001 2024-09-04T17:48:27.447 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:27.447 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:27.448 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:27.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.448 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49816 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 557056~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:48:27.448 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077532+0000", 2024-09-04T17:48:27.448 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 84.287494851000005, 2024-09-04T17:48:27.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019604331999999999, 2024-09-04T17:48:27.448 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:27.448 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:27.448 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:27.448 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:27.448 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:27.448 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:27.449 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49816 2024-09-04T17:48:27.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.449 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:27.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:27.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077532+0000", 2024-09-04T17:48:27.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:27.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:27.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077532+0000", 2024-09-04T17:48:27.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:27.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.450 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:27.450 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077534+0000", 2024-09-04T17:48:27.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1200000000000001e-06 2024-09-04T17:48:27.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.450 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:27.450 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077545+0000", 2024-09-04T17:48:27.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1673e-05 2024-09-04T17:48:27.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.450 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:27.450 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077569+0000", 2024-09-04T17:48:27.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3555999999999999e-05 2024-09-04T17:48:27.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.451 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:27.451 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077584+0000", 2024-09-04T17:48:27.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4883e-05 2024-09-04T17:48:27.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.451 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:27.451 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096825+0000", 2024-09-04T17:48:27.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019241734 2024-09-04T17:48:27.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.451 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:27.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096964+0000", 2024-09-04T17:48:27.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000138085 2024-09-04T17:48:27.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:27.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097137+0000", 2024-09-04T17:48:27.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017328100000000001 2024-09-04T17:48:27.452 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:27.452 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:27.452 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:27.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.453 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49817 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 622592~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:48:27.453 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077603+0000", 2024-09-04T17:48:27.453 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 84.287424293000001, 2024-09-04T17:48:27.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01986928, 2024-09-04T17:48:27.453 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:27.454 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:27.454 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:27.454 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:27.454 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:27.454 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:27.454 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49817 2024-09-04T17:48:27.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.454 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:27.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.454 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:27.454 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077603+0000", 2024-09-04T17:48:27.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:27.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:27.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077603+0000", 2024-09-04T17:48:27.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:27.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:27.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077604+0000", 2024-09-04T17:48:27.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7699999999999992e-07 2024-09-04T17:48:27.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:27.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077609+0000", 2024-09-04T17:48:27.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0830000000000003e-06 2024-09-04T17:48:27.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:27.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077630+0000", 2024-09-04T17:48:27.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0438999999999999e-05 2024-09-04T17:48:27.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:27.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077664+0000", 2024-09-04T17:48:27.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4579000000000002e-05 2024-09-04T17:48:27.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:27.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097193+0000", 2024-09-04T17:48:27.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019529174 2024-09-04T17:48:27.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:27.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097323+0000", 2024-09-04T17:48:27.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000129696 2024-09-04T17:48:27.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:27.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097472+0000", 2024-09-04T17:48:27.458 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014933200000000001 2024-09-04T17:48:27.458 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:27.458 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:27.458 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:27.458 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.458 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.458 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49818 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 688128~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:48:27.458 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077683+0000", 2024-09-04T17:48:27.458 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 84.287343929000002, 2024-09-04T17:48:27.458 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020086333000000001, 2024-09-04T17:48:27.458 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:27.458 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:27.458 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:27.459 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:27.459 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:27.459 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:27.459 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49818 2024-09-04T17:48:27.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.459 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:27.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:27.459 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077683+0000", 2024-09-04T17:48:27.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:27.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:27.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077683+0000", 2024-09-04T17:48:27.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:27.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:27.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077684+0000", 2024-09-04T17:48:27.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4e-07 2024-09-04T17:48:27.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:27.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077690+0000", 2024-09-04T17:48:27.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4729999999999999e-06 2024-09-04T17:48:27.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:27.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077710+0000", 2024-09-04T17:48:27.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0149999999999999e-05 2024-09-04T17:48:27.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:27.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077723+0000", 2024-09-04T17:48:27.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3271999999999999e-05 2024-09-04T17:48:27.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:27.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097521+0000", 2024-09-04T17:48:27.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019797981999999999 2024-09-04T17:48:27.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.462 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:27.462 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097628+0000", 2024-09-04T17:48:27.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010665700000000001 2024-09-04T17:48:27.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.462 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:27.462 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097770+0000", 2024-09-04T17:48:27.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014185900000000001 2024-09-04T17:48:27.462 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:27.462 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:27.462 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:27.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.463 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49820 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 753664~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:48:27.463 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077744+0000", 2024-09-04T17:48:27.463 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 84.287282997999995, 2024-09-04T17:48:27.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020364786999999999, 2024-09-04T17:48:27.463 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:27.463 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:27.463 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:27.463 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:27.463 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:27.463 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:27.464 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49820 2024-09-04T17:48:27.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.464 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:27.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.464 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:27.464 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077744+0000", 2024-09-04T17:48:27.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:27.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.464 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:27.464 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077744+0000", 2024-09-04T17:48:27.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:27.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:27.465 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077745+0000", 2024-09-04T17:48:27.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0270000000000001e-06 2024-09-04T17:48:27.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:27.465 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077751+0000", 2024-09-04T17:48:27.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.626e-06 2024-09-04T17:48:27.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:27.465 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077771+0000", 2024-09-04T17:48:27.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0469999999999999e-05 2024-09-04T17:48:27.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:27.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077785+0000", 2024-09-04T17:48:27.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3129000000000001e-05 2024-09-04T17:48:27.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:27.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097816+0000", 2024-09-04T17:48:27.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020031917999999999 2024-09-04T17:48:27.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.467 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:27.467 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097939+0000", 2024-09-04T17:48:27.467 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000122533 2024-09-04T17:48:27.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.467 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.467 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:27.467 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098109+0000", 2024-09-04T17:48:27.467 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017008400000000001 2024-09-04T17:48:27.467 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:27.467 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:27.467 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:27.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.467 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.468 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49821 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 819200~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:48:27.468 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077807+0000", 2024-09-04T17:48:27.468 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 84.287220786000006, 2024-09-04T17:48:27.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020702928999999998, 2024-09-04T17:48:27.468 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:27.468 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:27.468 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:27.468 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:27.468 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:27.468 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:27.468 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49821 2024-09-04T17:48:27.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.468 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:27.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.469 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:27.469 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077807+0000", 2024-09-04T17:48:27.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:27.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.469 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:27.469 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077807+0000", 2024-09-04T17:48:27.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:27.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.469 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:27.469 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077808+0000", 2024-09-04T17:48:27.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.463e-06 2024-09-04T17:48:27.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.470 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:27.470 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077813+0000", 2024-09-04T17:48:27.470 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7530000000000001e-06 2024-09-04T17:48:27.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.470 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:27.470 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077832+0000", 2024-09-04T17:48:27.470 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.946e-05 2024-09-04T17:48:27.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.470 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:27.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077846+0000", 2024-09-04T17:48:27.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3786e-05 2024-09-04T17:48:27.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:27.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098201+0000", 2024-09-04T17:48:27.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020354797000000001 2024-09-04T17:48:27.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:27.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098341+0000", 2024-09-04T17:48:27.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000140042 2024-09-04T17:48:27.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.472 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:27.472 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098509+0000", 2024-09-04T17:48:27.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016862800000000001 2024-09-04T17:48:27.472 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:27.472 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:27.472 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:27.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.472 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49823 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 884736~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:48:27.472 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077865+0000", 2024-09-04T17:48:27.472 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 84.287162342000002, 2024-09-04T17:48:27.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021008723, 2024-09-04T17:48:27.472 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:27.473 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:27.473 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:27.473 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:27.473 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:27.473 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:27.473 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49823 2024-09-04T17:48:27.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.473 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:27.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.473 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:27.473 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077865+0000", 2024-09-04T17:48:27.473 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:27.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.474 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:27.474 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077865+0000", 2024-09-04T17:48:27.474 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:27.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.474 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:27.474 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077866+0000", 2024-09-04T17:48:27.474 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.164e-06 2024-09-04T17:48:27.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.474 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:27.474 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077872+0000", 2024-09-04T17:48:27.475 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3959999999999997e-06 2024-09-04T17:48:27.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.475 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:27.475 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077892+0000", 2024-09-04T17:48:27.475 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0970000000000001e-05 2024-09-04T17:48:27.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.475 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:27.475 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077905+0000", 2024-09-04T17:48:27.475 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2312e-05 2024-09-04T17:48:27.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.475 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:27.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098579+0000", 2024-09-04T17:48:27.476 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020674486999999998 2024-09-04T17:48:27.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:27.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098705+0000", 2024-09-04T17:48:27.476 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012541900000000001 2024-09-04T17:48:27.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:27.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098874+0000", 2024-09-04T17:48:27.476 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000168975 2024-09-04T17:48:27.476 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:27.477 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:27.477 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:27.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.477 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49824 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 950272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:48:27.477 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077923+0000", 2024-09-04T17:48:27.477 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 84.287104650000003, 2024-09-04T17:48:27.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021246287999999999, 2024-09-04T17:48:27.477 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:27.477 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:27.477 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:27.477 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:27.477 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:27.478 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:27.478 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49824 2024-09-04T17:48:27.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.478 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:27.478 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.478 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:27.478 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077923+0000", 2024-09-04T17:48:27.478 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:27.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.478 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.478 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:27.478 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077923+0000", 2024-09-04T17:48:27.478 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:27.479 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.479 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:27.479 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077924+0000", 2024-09-04T17:48:27.479 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0100000000000001e-06 2024-09-04T17:48:27.479 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.479 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:27.479 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077944+0000", 2024-09-04T17:48:27.479 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0449e-05 2024-09-04T17:48:27.479 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.479 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:27.479 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077965+0000", 2024-09-04T17:48:27.480 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0479000000000002e-05 2024-09-04T17:48:27.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.480 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.480 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:27.480 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077990+0000", 2024-09-04T17:48:27.480 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5582999999999999e-05 2024-09-04T17:48:27.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.480 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.480 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:27.480 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098924+0000", 2024-09-04T17:48:27.480 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020933395 2024-09-04T17:48:27.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.481 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:27.481 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.099032+0000", 2024-09-04T17:48:27.481 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010846299999999999 2024-09-04T17:48:27.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.481 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:27.481 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.099169+0000", 2024-09-04T17:48:27.481 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000136909 2024-09-04T17:48:27.481 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:27.481 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:27.481 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:27.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.482 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49826 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 1015808~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:48:27.482 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.078010+0000", 2024-09-04T17:48:27.482 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 84.287017043000006, 2024-09-04T17:48:27.482 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021584564000000001, 2024-09-04T17:48:27.482 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:27.482 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:27.482 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:27.482 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:27.482 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:27.482 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:27.482 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49826 2024-09-04T17:48:27.482 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.482 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:27.483 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.483 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:27.483 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078010+0000", 2024-09-04T17:48:27.483 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:27.483 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.483 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.483 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:27.483 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078010+0000", 2024-09-04T17:48:27.483 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:27.483 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.483 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.483 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:27.483 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078011+0000", 2024-09-04T17:48:27.483 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1340000000000001e-06 2024-09-04T17:48:27.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.484 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.484 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:27.484 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078017+0000", 2024-09-04T17:48:27.484 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.682e-06 2024-09-04T17:48:27.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.484 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.484 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:27.484 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078036+0000", 2024-09-04T17:48:27.484 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9357000000000001e-05 2024-09-04T17:48:27.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.484 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.484 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:27.484 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078050+0000", 2024-09-04T17:48:27.485 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3122000000000001e-05 2024-09-04T17:48:27.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.485 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.485 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:27.485 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.099271+0000", 2024-09-04T17:48:27.485 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021221772999999999 2024-09-04T17:48:27.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.485 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.485 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:27.485 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.099410+0000", 2024-09-04T17:48:27.485 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000138415 2024-09-04T17:48:27.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:27.485 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:27.486 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:27.486 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.099595+0000", 2024-09-04T17:48:27.486 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000185081 2024-09-04T17:48:27.486 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:27.486 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:27.486 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:27.486 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:27.486 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:27.486 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:27.488 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-04T17:48:27.488 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:48:27.487+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:48:27.588 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:48:27.586+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:48:27.658 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:27.658 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:48:27.658 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:48:27.659 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:27.674 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-04T17:48:27.721 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:27.719+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:27.790 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:48:27.790+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:48:27.796 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:27.796 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:48:27.796 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:48:27.796 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:48:27.796 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:27.810 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-04T17:48:27.890 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:27.890+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:27.991 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:27.987+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:28.002 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:28.002 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:48:28.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:48:28.002 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:48:28.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.003 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:71638 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-04T17:48:28.003 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:44.479908+0000", 2024-09-04T17:48:28.003 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 43.521525783999998, 2024-09-04T17:48:28.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.469396488, 2024-09-04T17:48:28.003 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:28.003 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:28.003 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:28.003 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:28.003 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:28.003 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:28.003 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 71638 2024-09-04T17:48:28.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.004 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:28.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:28.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.479908+0000", 2024-09-04T17:48:28.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:28.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:28.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.479908+0000", 2024-09-04T17:48:28.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:28.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:28.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.479911+0000", 2024-09-04T17:48:28.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9160000000000001e-06 2024-09-04T17:48:28.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:28.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.479925+0000", 2024-09-04T17:48:28.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3896e-05 2024-09-04T17:48:28.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:28.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.479954+0000", 2024-09-04T17:48:28.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9249000000000001e-05 2024-09-04T17:48:28.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.006 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:28.006 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.479969+0000", 2024-09-04T17:48:28.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4683e-05 2024-09-04T17:48:28.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.006 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:28.006 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.480109+0000", 2024-09-04T17:48:28.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013995699999999999 2024-09-04T17:48:28.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.006 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-04T17:48:28.006 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.480125+0000", 2024-09-04T17:48:28.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6651999999999999e-05 2024-09-04T17:48:28.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:28.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.948598+0000", 2024-09-04T17:48:28.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.46847249600000002 2024-09-04T17:48:28.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:28.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.948889+0000", 2024-09-04T17:48:28.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000290947 2024-09-04T17:48:28.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:28.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.949304+0000", 2024-09-04T17:48:28.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000415692 2024-09-04T17:48:28.008 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:28.008 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:28.008 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:28.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.008 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:72267 3.1a 3:58dbd2e1:::rbd_data.11342ece613c.000000000000001e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-09-04T17:48:28.009 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:45.674662+0000", 2024-09-04T17:48:28.009 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.326771975, 2024-09-04T17:48:28.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032248339000000001, 2024-09-04T17:48:28.009 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:28.009 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:28.010 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:28.010 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:28.010 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:28.010 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:28.010 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 72267 2024-09-04T17:48:28.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.010 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:28.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:28.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.674662+0000", 2024-09-04T17:48:28.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:28.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:28.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.674662+0000", 2024-09-04T17:48:28.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:28.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:28.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.674664+0000", 2024-09-04T17:48:28.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4870000000000001e-06 2024-09-04T17:48:28.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:28.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.674689+0000", 2024-09-04T17:48:28.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4805000000000001e-05 2024-09-04T17:48:28.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:28.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.674715+0000", 2024-09-04T17:48:28.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5548999999999999e-05 2024-09-04T17:48:28.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:28.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.674731+0000", 2024-09-04T17:48:28.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6549999999999999e-05 2024-09-04T17:48:28.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:28.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.674843+0000", 2024-09-04T17:48:28.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000112001 2024-09-04T17:48:28.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:28.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.675176+0000", 2024-09-04T17:48:28.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00033337200000000002 2024-09-04T17:48:28.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-04T17:48:28.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.680121+0000", 2024-09-04T17:48:28.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0049445540000000003 2024-09-04T17:48:28.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:28.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.706702+0000", 2024-09-04T17:48:28.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026580840000000001 2024-09-04T17:48:28.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:28.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.706738+0000", 2024-09-04T17:48:28.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6158999999999997e-05 2024-09-04T17:48:28.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:28.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.706835+0000", 2024-09-04T17:48:28.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6909000000000003e-05 2024-09-04T17:48:28.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.016 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:28.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.706910+0000", 2024-09-04T17:48:28.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5112999999999997e-05 2024-09-04T17:48:28.016 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:28.016 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:28.016 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:28.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.017 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94568 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 1703936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:28.017 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.605863+0000", 2024-09-04T17:48:28.017 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.395571159999999, 2024-09-04T17:48:28.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027249301, 2024-09-04T17:48:28.017 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:28.017 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:28.017 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:28.017 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:28.017 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:28.018 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:28.018 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94568 2024-09-04T17:48:28.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.018 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:28.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:28.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605863+0000", 2024-09-04T17:48:28.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:28.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:28.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605863+0000", 2024-09-04T17:48:28.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:28.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:28.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605863+0000", 2024-09-04T17:48:28.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7999999999999996e-07 2024-09-04T17:48:28.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:28.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605866+0000", 2024-09-04T17:48:28.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6199999999999999e-06 2024-09-04T17:48:28.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:28.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605875+0000", 2024-09-04T17:48:28.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2499999999999995e-06 2024-09-04T17:48:28.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:28.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605881+0000", 2024-09-04T17:48:28.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.19e-06 2024-09-04T17:48:28.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.021 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:28.021 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.632629+0000", 2024-09-04T17:48:28.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026747567 2024-09-04T17:48:28.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.021 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:28.021 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.632874+0000", 2024-09-04T17:48:28.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024493600000000001 2024-09-04T17:48:28.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:28.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.633112+0000", 2024-09-04T17:48:28.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023825800000000001 2024-09-04T17:48:28.022 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:28.022 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:28.022 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:28.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.022 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94569 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:28.022 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.605888+0000", 2024-09-04T17:48:28.022 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.395545331000001, 2024-09-04T17:48:28.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027675620000000001, 2024-09-04T17:48:28.022 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:28.023 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:28.023 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:28.023 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:28.023 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:28.023 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:28.023 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94569 2024-09-04T17:48:28.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.023 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:28.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:28.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605888+0000", 2024-09-04T17:48:28.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:28.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:28.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605888+0000", 2024-09-04T17:48:28.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:28.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:28.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605889+0000", 2024-09-04T17:48:28.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4799999999999998e-07 2024-09-04T17:48:28.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:28.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605891+0000", 2024-09-04T17:48:28.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4880000000000001e-06 2024-09-04T17:48:28.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:28.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605901+0000", 2024-09-04T17:48:28.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4760000000000004e-06 2024-09-04T17:48:28.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:28.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605912+0000", 2024-09-04T17:48:28.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0698e-05 2024-09-04T17:48:28.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:28.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.633184+0000", 2024-09-04T17:48:28.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.02727253 2024-09-04T17:48:28.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:28.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.633330+0000", 2024-09-04T17:48:28.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014543800000000001 2024-09-04T17:48:28.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:28.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.633564+0000", 2024-09-04T17:48:28.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023444199999999999 2024-09-04T17:48:28.026 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:28.026 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:28.026 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:28.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.027 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94570 3.0 3.3f52c000 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:28.027 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.605932+0000", 2024-09-04T17:48:28.027 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.395501763999999, 2024-09-04T17:48:28.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027687021999999999, 2024-09-04T17:48:28.027 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:28.027 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:28.027 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-04T17:48:28.027 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:28.027 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:28.027 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:28.027 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94570 2024-09-04T17:48:28.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.028 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:28.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:28.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605932+0000", 2024-09-04T17:48:28.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:28.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:28.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605932+0000", 2024-09-04T17:48:28.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:28.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:28.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605933+0000", 2024-09-04T17:48:28.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5000000000000001e-07 2024-09-04T17:48:28.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:28.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605936+0000", 2024-09-04T17:48:28.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7769999999999998e-06 2024-09-04T17:48:28.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:28.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605943+0000", 2024-09-04T17:48:28.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7000000000000008e-06 2024-09-04T17:48:28.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:28.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605949+0000", 2024-09-04T17:48:28.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0170000000000002e-06 2024-09-04T17:48:28.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:28.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.633604+0000", 2024-09-04T17:48:28.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027654875999999998 2024-09-04T17:48:28.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:28.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.633619+0000", 2024-09-04T17:48:28.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4802e-05 2024-09-04T17:48:28.031 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:28.031 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:28.031 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:28.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.031 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94641 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 6094848~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:28.031 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.637288+0000", 2024-09-04T17:48:28.031 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.364145313000002, 2024-09-04T17:48:28.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026966783000000001, 2024-09-04T17:48:28.031 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:28.031 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:28.031 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:28.031 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:28.032 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:28.032 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:28.032 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94641 2024-09-04T17:48:28.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.032 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:28.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:28.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.637288+0000", 2024-09-04T17:48:28.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:28.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:28.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.637288+0000", 2024-09-04T17:48:28.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:28.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:28.033 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.637289+0000", 2024-09-04T17:48:28.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4600000000000005e-07 2024-09-04T17:48:28.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:28.033 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.637292+0000", 2024-09-04T17:48:28.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1049999999999999e-06 2024-09-04T17:48:28.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:28.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.637301+0000", 2024-09-04T17:48:28.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7509999999999997e-06 2024-09-04T17:48:28.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:28.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.637309+0000", 2024-09-04T17:48:28.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.67e-06 2024-09-04T17:48:28.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:28.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.663971+0000", 2024-09-04T17:48:28.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026661926999999998 2024-09-04T17:48:28.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:28.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.664062+0000", 2024-09-04T17:48:28.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0673000000000004e-05 2024-09-04T17:48:28.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:28.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.664255+0000", 2024-09-04T17:48:28.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019311099999999999 2024-09-04T17:48:28.035 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:28.035 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:28.035 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:28.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.036 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94642 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 6225920~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:28.036 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.637321+0000", 2024-09-04T17:48:28.036 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.364112366000001, 2024-09-04T17:48:28.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027131534999999998, 2024-09-04T17:48:28.036 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:28.036 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:28.036 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:28.036 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:28.036 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:28.036 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:28.036 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94642 2024-09-04T17:48:28.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.037 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:28.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.037 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:28.037 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.637321+0000", 2024-09-04T17:48:28.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:28.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.037 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:28.037 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.637321+0000", 2024-09-04T17:48:28.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:28.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.037 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:28.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.637322+0000", 2024-09-04T17:48:28.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6599999999999996e-07 2024-09-04T17:48:28.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:28.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.637325+0000", 2024-09-04T17:48:28.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1070000000000001e-06 2024-09-04T17:48:28.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:28.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.637337+0000", 2024-09-04T17:48:28.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1766000000000001e-05 2024-09-04T17:48:28.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:28.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.637350+0000", 2024-09-04T17:48:28.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3515e-05 2024-09-04T17:48:28.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:28.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.664273+0000", 2024-09-04T17:48:28.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026922643 2024-09-04T17:48:28.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:28.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.664337+0000", 2024-09-04T17:48:28.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4492999999999994e-05 2024-09-04T17:48:28.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:28.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.664453+0000", 2024-09-04T17:48:28.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000115345 2024-09-04T17:48:28.040 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:28.040 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:28.040 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:28.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.040 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94647 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 6684672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:28.040 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.638572+0000", 2024-09-04T17:48:28.040 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.362861479999999, 2024-09-04T17:48:28.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026910377999999999, 2024-09-04T17:48:28.041 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:28.041 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:28.041 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:28.041 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:28.041 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:28.041 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:28.041 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94647 2024-09-04T17:48:28.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.041 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:28.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:28.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638572+0000", 2024-09-04T17:48:28.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:28.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:28.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638572+0000", 2024-09-04T17:48:28.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:28.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:28.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638573+0000", 2024-09-04T17:48:28.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7599999999999997e-07 2024-09-04T17:48:28.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:28.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638575+0000", 2024-09-04T17:48:28.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2010000000000002e-06 2024-09-04T17:48:28.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:28.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638582+0000", 2024-09-04T17:48:28.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7399999999999998e-06 2024-09-04T17:48:28.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:28.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638588+0000", 2024-09-04T17:48:28.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8560000000000003e-06 2024-09-04T17:48:28.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:28.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.665294+0000", 2024-09-04T17:48:28.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026705893000000001 2024-09-04T17:48:28.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:28.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.665355+0000", 2024-09-04T17:48:28.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1253000000000005e-05 2024-09-04T17:48:28.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:28.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.665483+0000", 2024-09-04T17:48:28.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012775899999999999 2024-09-04T17:48:28.045 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:28.045 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:28.045 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:28.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.045 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94648 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 6815744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:28.045 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.638593+0000", 2024-09-04T17:48:28.045 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.362840391999999, 2024-09-04T17:48:28.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027163676000000001, 2024-09-04T17:48:28.045 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:28.045 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:28.045 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:28.045 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:28.046 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:28.046 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:28.046 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94648 2024-09-04T17:48:28.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.046 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:28.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:28.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638593+0000", 2024-09-04T17:48:28.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:28.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:28.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638593+0000", 2024-09-04T17:48:28.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:28.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:28.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638594+0000", 2024-09-04T17:48:28.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.53e-07 2024-09-04T17:48:28.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:28.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638596+0000", 2024-09-04T17:48:28.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8870000000000001e-06 2024-09-04T17:48:28.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:28.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638600+0000", 2024-09-04T17:48:28.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3290000000000004e-06 2024-09-04T17:48:28.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:28.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638603+0000", 2024-09-04T17:48:28.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4680000000000001e-06 2024-09-04T17:48:28.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:28.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.665524+0000", 2024-09-04T17:48:28.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.02692052 2024-09-04T17:48:28.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:28.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.665600+0000", 2024-09-04T17:48:28.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6569000000000005e-05 2024-09-04T17:48:28.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:28.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.665757+0000", 2024-09-04T17:48:28.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015655000000000001 2024-09-04T17:48:28.049 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:28.049 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:28.049 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:28.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.050 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94649 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 6946816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:28.050 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.638608+0000", 2024-09-04T17:48:28.050 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.362825961999999, 2024-09-04T17:48:28.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027355753999999999, 2024-09-04T17:48:28.050 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:28.050 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:28.050 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:28.050 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:28.050 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:28.050 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:28.050 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94649 2024-09-04T17:48:28.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.050 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:28.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:28.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638608+0000", 2024-09-04T17:48:28.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:28.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:28.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638608+0000", 2024-09-04T17:48:28.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:28.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:28.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638608+0000", 2024-09-04T17:48:28.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3099999999999998e-07 2024-09-04T17:48:28.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:28.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638610+0000", 2024-09-04T17:48:28.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.716e-06 2024-09-04T17:48:28.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:28.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638614+0000", 2024-09-04T17:48:28.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.941e-06 2024-09-04T17:48:28.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:28.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638618+0000", 2024-09-04T17:48:28.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5009999999999998e-06 2024-09-04T17:48:28.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:28.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.665795+0000", 2024-09-04T17:48:28.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027176404000000001 2024-09-04T17:48:28.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:28.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.665851+0000", 2024-09-04T17:48:28.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6530999999999998e-05 2024-09-04T17:48:28.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:28.054 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.665963+0000", 2024-09-04T17:48:28.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011213 2024-09-04T17:48:28.054 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:28.054 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:28.054 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:28.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.054 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94650 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 7077888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:28.054 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.638623+0000", 2024-09-04T17:48:28.054 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.362810666000001, 2024-09-04T17:48:28.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027665261999999999, 2024-09-04T17:48:28.055 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:28.055 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:28.055 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:28.055 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:28.055 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:28.055 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:28.055 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94650 2024-09-04T17:48:28.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.055 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:28.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:28.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638623+0000", 2024-09-04T17:48:28.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:28.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:28.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638623+0000", 2024-09-04T17:48:28.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:28.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:28.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638623+0000", 2024-09-04T17:48:28.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6300000000000001e-07 2024-09-04T17:48:28.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:28.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638625+0000", 2024-09-04T17:48:28.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7260000000000001e-06 2024-09-04T17:48:28.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:28.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638629+0000", 2024-09-04T17:48:28.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1309999999999999e-06 2024-09-04T17:48:28.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:28.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638634+0000", 2024-09-04T17:48:28.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2700000000000004e-06 2024-09-04T17:48:28.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:28.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.666028+0000", 2024-09-04T17:48:28.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027393303000000001 2024-09-04T17:48:28.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:28.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.666112+0000", 2024-09-04T17:48:28.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3844000000000006e-05 2024-09-04T17:48:28.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:28.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.666288+0000", 2024-09-04T17:48:28.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000176725 2024-09-04T17:48:28.059 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:28.059 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:28.059 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:28.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.059 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94651 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 7208960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:28.059 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.638639+0000", 2024-09-04T17:48:28.059 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.362794271999999, 2024-09-04T17:48:28.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027839592999999999, 2024-09-04T17:48:28.059 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:28.059 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:28.060 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:28.060 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:28.060 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:28.060 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:28.060 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94651 2024-09-04T17:48:28.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.060 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:28.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.060 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:28.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638639+0000", 2024-09-04T17:48:28.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:28.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:28.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638639+0000", 2024-09-04T17:48:28.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:28.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:28.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638640+0000", 2024-09-04T17:48:28.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8000000000000002e-07 2024-09-04T17:48:28.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:28.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638641+0000", 2024-09-04T17:48:28.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.77e-06 2024-09-04T17:48:28.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:28.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638646+0000", 2024-09-04T17:48:28.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1729999999999997e-06 2024-09-04T17:48:28.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:28.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638649+0000", 2024-09-04T17:48:28.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5260000000000002e-06 2024-09-04T17:48:28.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:28.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.666309+0000", 2024-09-04T17:48:28.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027660179 2024-09-04T17:48:28.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:28.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.666373+0000", 2024-09-04T17:48:28.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4152999999999994e-05 2024-09-04T17:48:28.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:28.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.666479+0000", 2024-09-04T17:48:28.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000105512 2024-09-04T17:48:28.063 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:28.063 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:28.063 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:28.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.064 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94657 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 7012352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:28.064 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.641793+0000", 2024-09-04T17:48:28.064 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.359640788, 2024-09-04T17:48:28.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026893672, 2024-09-04T17:48:28.064 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:28.064 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:28.064 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:28.064 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:28.064 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:28.064 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:28.065 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94657 2024-09-04T17:48:28.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.065 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:28.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:28.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641793+0000", 2024-09-04T17:48:28.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:28.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:28.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641793+0000", 2024-09-04T17:48:28.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:28.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:28.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641793+0000", 2024-09-04T17:48:28.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5000000000000003e-07 2024-09-04T17:48:28.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:28.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641797+0000", 2024-09-04T17:48:28.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2909999999999999e-06 2024-09-04T17:48:28.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:28.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641806+0000", 2024-09-04T17:48:28.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3270000000000007e-06 2024-09-04T17:48:28.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:28.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641812+0000", 2024-09-04T17:48:28.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2310000000000001e-06 2024-09-04T17:48:28.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:28.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.668412+0000", 2024-09-04T17:48:28.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026600104999999999 2024-09-04T17:48:28.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:28.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.668534+0000", 2024-09-04T17:48:28.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012115100000000001 2024-09-04T17:48:28.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:28.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.668687+0000", 2024-09-04T17:48:28.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000153017 2024-09-04T17:48:28.068 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:28.068 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:28.068 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:28.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.069 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94658 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 7143424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:28.069 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.641822+0000", 2024-09-04T17:48:28.069 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.359611857000001, 2024-09-04T17:48:28.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027395880000000001, 2024-09-04T17:48:28.069 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:28.069 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:28.069 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:28.069 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:28.069 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:28.069 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:28.069 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94658 2024-09-04T17:48:28.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.070 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:28.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:28.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641822+0000", 2024-09-04T17:48:28.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:28.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:28.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641822+0000", 2024-09-04T17:48:28.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:28.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:28.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641822+0000", 2024-09-04T17:48:28.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.58e-07 2024-09-04T17:48:28.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:28.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641825+0000", 2024-09-04T17:48:28.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5909999999999998e-06 2024-09-04T17:48:28.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:28.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641832+0000", 2024-09-04T17:48:28.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1799999999999999e-06 2024-09-04T17:48:28.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:28.072 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641838+0000", 2024-09-04T17:48:28.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5040000000000002e-06 2024-09-04T17:48:28.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:28.072 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.668748+0000", 2024-09-04T17:48:28.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026910119999999999 2024-09-04T17:48:28.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:28.072 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.668896+0000", 2024-09-04T17:48:28.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014804299999999999 2024-09-04T17:48:28.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:28.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.669218+0000", 2024-09-04T17:48:28.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00032198399999999999 2024-09-04T17:48:28.073 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:28.073 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:28.073 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:28.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.073 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94659 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 7274496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:28.073 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.641846+0000", 2024-09-04T17:48:28.073 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.359587786999999, 2024-09-04T17:48:28.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027757314000000002, 2024-09-04T17:48:28.074 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:28.074 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:28.074 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:28.074 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:28.074 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:28.074 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:28.074 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94659 2024-09-04T17:48:28.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.074 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:28.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:28.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641846+0000", 2024-09-04T17:48:28.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:28.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:28.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641846+0000", 2024-09-04T17:48:28.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:28.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:28.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641846+0000", 2024-09-04T17:48:28.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5400000000000002e-07 2024-09-04T17:48:28.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:28.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641849+0000", 2024-09-04T17:48:28.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0170000000000001e-06 2024-09-04T17:48:28.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:28.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641858+0000", 2024-09-04T17:48:28.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1499999999999999e-06 2024-09-04T17:48:28.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:28.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641864+0000", 2024-09-04T17:48:28.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2280000000000003e-06 2024-09-04T17:48:28.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.077 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:28.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.669278+0000", 2024-09-04T17:48:28.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027413843 2024-09-04T17:48:28.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.077 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:28.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.669392+0000", 2024-09-04T17:48:28.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000114654 2024-09-04T17:48:28.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.077 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:28.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.669603+0000", 2024-09-04T17:48:28.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021096799999999999 2024-09-04T17:48:28.078 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:28.078 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:28.078 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:28.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.078 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94664 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 7864320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:28.078 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.644474+0000", 2024-09-04T17:48:28.078 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.356960106999999, 2024-09-04T17:48:28.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026965042000000002, 2024-09-04T17:48:28.078 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:28.078 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:28.078 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:28.078 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:28.079 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:28.079 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:28.079 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94664 2024-09-04T17:48:28.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.079 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:28.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:28.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644474+0000", 2024-09-04T17:48:28.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:28.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:28.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644474+0000", 2024-09-04T17:48:28.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:28.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:28.080 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644474+0000", 2024-09-04T17:48:28.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.16e-07 2024-09-04T17:48:28.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:28.080 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644478+0000", 2024-09-04T17:48:28.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1389999999999999e-06 2024-09-04T17:48:28.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:28.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644494+0000", 2024-09-04T17:48:28.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6977e-05 2024-09-04T17:48:28.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:28.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644501+0000", 2024-09-04T17:48:28.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.438e-06 2024-09-04T17:48:28.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:28.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.671241+0000", 2024-09-04T17:48:28.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026740525000000001 2024-09-04T17:48:28.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:28.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.671325+0000", 2024-09-04T17:48:28.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3987999999999996e-05 2024-09-04T17:48:28.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:28.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.671439+0000", 2024-09-04T17:48:28.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011315900000000001 2024-09-04T17:48:28.082 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:28.082 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:28.082 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:28.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.082 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94665 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 7995392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:28.083 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.644511+0000", 2024-09-04T17:48:28.083 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.356923075000001, 2024-09-04T17:48:28.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027156390999999998, 2024-09-04T17:48:28.083 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:28.083 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:28.083 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:28.083 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:28.083 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:28.083 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:28.083 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94665 2024-09-04T17:48:28.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.083 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:28.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.083 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:28.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644511+0000", 2024-09-04T17:48:28.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:28.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:28.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644511+0000", 2024-09-04T17:48:28.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:28.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:28.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644511+0000", 2024-09-04T17:48:28.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2699999999999999e-07 2024-09-04T17:48:28.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:28.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644514+0000", 2024-09-04T17:48:28.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9890000000000001e-06 2024-09-04T17:48:28.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:28.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644521+0000", 2024-09-04T17:48:28.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5409999999999997e-06 2024-09-04T17:48:28.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:28.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644527+0000", 2024-09-04T17:48:28.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0340000000000002e-06 2024-09-04T17:48:28.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:28.086 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.671474+0000", 2024-09-04T17:48:28.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.02694767 2024-09-04T17:48:28.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:28.086 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.671529+0000", 2024-09-04T17:48:28.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4373000000000003e-05 2024-09-04T17:48:28.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.087 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:28.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.671667+0000", 2024-09-04T17:48:28.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000138257 2024-09-04T17:48:28.087 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:28.087 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:28.087 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:28.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.087 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94666 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 8126464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:28.087 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.644536+0000", 2024-09-04T17:48:28.087 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.356897936999999, 2024-09-04T17:48:28.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027360524000000001, 2024-09-04T17:48:28.087 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:28.088 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:28.088 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:28.088 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:28.088 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:28.088 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:28.088 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94666 2024-09-04T17:48:28.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.088 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:28.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:28.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644536+0000", 2024-09-04T17:48:28.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:28.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:28.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644536+0000", 2024-09-04T17:48:28.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:28.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:28.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644536+0000", 2024-09-04T17:48:28.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7000000000000004e-07 2024-09-04T17:48:28.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:28.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644540+0000", 2024-09-04T17:48:28.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1609999999999999e-06 2024-09-04T17:48:28.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:28.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644552+0000", 2024-09-04T17:48:28.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2794e-05 2024-09-04T17:48:28.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:28.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644559+0000", 2024-09-04T17:48:28.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5010000000000001e-06 2024-09-04T17:48:28.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:28.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.671703+0000", 2024-09-04T17:48:28.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027144492999999999 2024-09-04T17:48:28.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:28.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.671779+0000", 2024-09-04T17:48:28.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5578e-05 2024-09-04T17:48:28.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:28.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.671896+0000", 2024-09-04T17:48:28.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000117327 2024-09-04T17:48:28.091 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:28.091 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:28.091 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:28.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.092 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94667 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 8257536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:28.092 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.644570+0000", 2024-09-04T17:48:28.092 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.356863272999998, 2024-09-04T17:48:28.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027556085000000001, 2024-09-04T17:48:28.092 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:28.092 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:28.092 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:28.092 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:28.092 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:28.092 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:28.092 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94667 2024-09-04T17:48:28.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.092 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:28.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:28.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644570+0000", 2024-09-04T17:48:28.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:28.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:28.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644570+0000", 2024-09-04T17:48:28.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:28.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:28.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644571+0000", 2024-09-04T17:48:28.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5199999999999997e-07 2024-09-04T17:48:28.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:28.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644574+0000", 2024-09-04T17:48:28.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3340000000000002e-06 2024-09-04T17:48:28.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:28.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644581+0000", 2024-09-04T17:48:28.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6579999999999996e-06 2024-09-04T17:48:28.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:28.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644589+0000", 2024-09-04T17:48:28.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9699999999999999e-06 2024-09-04T17:48:28.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:28.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.671925+0000", 2024-09-04T17:48:28.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027335911000000001 2024-09-04T17:48:28.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:28.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.671987+0000", 2024-09-04T17:48:28.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2198000000000003e-05 2024-09-04T17:48:28.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:28.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.672126+0000", 2024-09-04T17:48:28.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000139462 2024-09-04T17:48:28.096 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:28.096 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:28.096 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:28.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.097 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:96164 3.18 3:19aa90a8:::rbd_data.1134c4e2d26.0000000000000062:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123)", 2024-09-04T17:48:28.097 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:48:02.862015+0000", 2024-09-04T17:48:28.097 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 25.139418751000001, 2024-09-04T17:48:28.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034031039999999999, 2024-09-04T17:48:28.097 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:28.097 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:28.097 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:28.097 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:28.097 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:28.097 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:28.097 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 96164 2024-09-04T17:48:28.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.098 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:28.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:28.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.862015+0000", 2024-09-04T17:48:28.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:28.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:28.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.862015+0000", 2024-09-04T17:48:28.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:28.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:28.099 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.862018+0000", 2024-09-04T17:48:28.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1200000000000002e-06 2024-09-04T17:48:28.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:28.099 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.862032+0000", 2024-09-04T17:48:28.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4083000000000001e-05 2024-09-04T17:48:28.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:28.099 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.862064+0000", 2024-09-04T17:48:28.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2095999999999999e-05 2024-09-04T17:48:28.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:28.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.862081+0000", 2024-09-04T17:48:28.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6625000000000001e-05 2024-09-04T17:48:28.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:28.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.862232+0000", 2024-09-04T17:48:28.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015144600000000001 2024-09-04T17:48:28.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:28.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.862588+0000", 2024-09-04T17:48:28.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00035525999999999998 2024-09-04T17:48:28.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 7", 2024-09-04T17:48:28.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.867988+0000", 2024-09-04T17:48:28.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054005479999999998 2024-09-04T17:48:28.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:28.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.895763+0000", 2024-09-04T17:48:28.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027774983 2024-09-04T17:48:28.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:28.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.895877+0000", 2024-09-04T17:48:28.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000113607 2024-09-04T17:48:28.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:28.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.895975+0000", 2024-09-04T17:48:28.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7876999999999998e-05 2024-09-04T17:48:28.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:28.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:28.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:28.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.896046+0000", 2024-09-04T17:48:28.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1395000000000003e-05 2024-09-04T17:48:28.103 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:28.103 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:28.103 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:28.103 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:28.103 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:28.103 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:28.103 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-04T17:48:28.105 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-04T17:48:28.106 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:28.099+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:28.174 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-04T17:48:28.206 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:28.206+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:28.238 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:28.238 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:48:28.238 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:48:28.238 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:28.254 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-04T17:48:28.345 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:48:28.346+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:48:28.365 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:48:28.365 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:48:28.385 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:48:28.386 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:48:28.403 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-04T17:48:28.404 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-04T17:48:28.407 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:48:28.406+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:48:28.421 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-04T17:48:28.421 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-04T17:48:28.424 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:28.424 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:48:28.424 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:48:28.424 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:48:28.424 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:28.439 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-04T17:48:28.453 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:48:28.454 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:48:28.486 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:48:28.486 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:48:28.505 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-04T17:48:28.505 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-04T17:48:28.507 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:48:28.503+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:48:28.527 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:48:28.527 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:48:28.608 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:28.603+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:28.611 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:28.612 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:28.627 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-04T17:48:28.710 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:48:28.710+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:48:28.751 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:28.751 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:28.767 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-04T17:48:28.810 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:48:28.810+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:48:28.890 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:28.891 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:28.906 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-04T17:48:28.911 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:28.910+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:29.012 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:48:29.007+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:48:29.033 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:29.033 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:29.035 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is unset 2024-09-04T17:48:29.050 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-04T17:48:29.113 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:29.114+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:29.214 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:29.214+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:29.228 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:29.228 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:29.242 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-04T17:48:29.315 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:29.314+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:29.359 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:29.360 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:29.373 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-04T17:48:29.416 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:48:29.414+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:48:29.495 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:29.495 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:29.509 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-04T17:48:29.516 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:48:29.511+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:48:29.617 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:29.615+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:29.654 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:29.654 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:29.670 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-04T17:48:29.718 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:48:29.715+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:48:29.788 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:29.788 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:29.803 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-04T17:48:29.818 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:29.815+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:29.919 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:48:29.915+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:48:29.925 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:29.926 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:29.940 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-04T17:48:30.020 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:30.015+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:30.075 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:30.076 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:30.091 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-04T17:48:30.137 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:48:30.138+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:48:30.223 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:30.219+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:30.223 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:30.223 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:30.239 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-04T17:48:30.333 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:48:30.334+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:48:30.417 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:30.417 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:30.425 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:48:30.426+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:48:30.432 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-04T17:48:30.525 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:30.526+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:30.546 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-04T17:48:30.594 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:30.594 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:30.609 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-04T17:48:30.626 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:30.623+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:30.728 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:30.726+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:30.776 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:30.777 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:30.791 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-04T17:48:30.833 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:30.834+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:30.881 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-04T17:48:30.879+0000 7f2b2530e640 0 --2- 172.21.15.92:0/3513702672 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2b200acac0 0x7f2b201d9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-04T17:48:30.909 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:48:30.909 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:48:30.929 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:48:30.930+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:48:30.933 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:48:30.933 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:48:30.955 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:30.955 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:30.958 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-04T17:48:30.959 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-04T17:48:30.969 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-04T17:48:30.980 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-04T17:48:30.981 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-04T17:48:31.003 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:48:31.003 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:48:31.026 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:48:31.026 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:48:31.031 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:31.030+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:31.046 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 4, 5, 6, 7, 3] out_osds: [1] dead_osds: [] live_osds: [0, 1, 4, 5, 7, 3, 2, 6] 2024-09-04T17:48:31.046 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-04T17:48:31.046 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-04T17:48:31.047 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-04T17:48:31.056 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-04T17:48:31.056 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-04T17:48:31.068 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-04T17:48:31.095 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:48:31.096 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:48:31.113 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:31.113 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:48:31.113 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:48:31.113 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:31.131 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-04T17:48:31.132 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:31.130+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:31.232 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:31.230+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:31.296 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:31.296 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:48:31.296 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:48:31.296 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:48:31.297 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:31.310 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-04T17:48:31.332 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:31.330+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:31.360 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:48:31.360 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:31.360 INFO:teuthology.orchestra.run.smithi092.stdout: "epoch": 151, 2024-09-04T17:48:31.360 INFO:teuthology.orchestra.run.smithi092.stdout: "fsid": "f36b6ce2-8c72-4c99-8607-5c53c38ef67b", 2024-09-04T17:48:31.360 INFO:teuthology.orchestra.run.smithi092.stdout: "created": "2024-09-04T17:45:28.072591+0000", 2024-09-04T17:48:31.360 INFO:teuthology.orchestra.run.smithi092.stdout: "modified": "2024-09-04T17:48:31.038086+0000", 2024-09-04T17:48:31.360 INFO:teuthology.orchestra.run.smithi092.stdout: "last_up_change": "2024-09-04T17:47:57.033539+0000", 2024-09-04T17:48:31.360 INFO:teuthology.orchestra.run.smithi092.stdout: "last_in_change": "2024-09-04T17:48:26.008743+0000", 2024-09-04T17:48:31.360 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-04T17:48:31.360 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_num": 5799936, 2024-09-04T17:48:31.361 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_set": [ 2024-09-04T17:48:31.361 INFO:teuthology.orchestra.run.smithi092.stdout: "pglog_hardlimit", 2024-09-04T17:48:31.361 INFO:teuthology.orchestra.run.smithi092.stdout: "purged_snapdirs", 2024-09-04T17:48:31.361 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_deletes", 2024-09-04T17:48:31.361 INFO:teuthology.orchestra.run.smithi092.stdout: "sortbitwise" 2024-09-04T17:48:31.361 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-04T17:48:31.361 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_version": 8, 2024-09-04T17:48:31.361 INFO:teuthology.orchestra.run.smithi092.stdout: "full_ratio": 0.94999998807907104, 2024-09-04T17:48:31.361 INFO:teuthology.orchestra.run.smithi092.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-04T17:48:31.361 INFO:teuthology.orchestra.run.smithi092.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-04T17:48:31.361 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_snapshot": "", 2024-09-04T17:48:31.361 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_max": 3, 2024-09-04T17:48:31.361 INFO:teuthology.orchestra.run.smithi092.stdout: "max_osd": 8, 2024-09-04T17:48:31.362 INFO:teuthology.orchestra.run.smithi092.stdout: "require_min_compat_client": "luminous", 2024-09-04T17:48:31.362 INFO:teuthology.orchestra.run.smithi092.stdout: "min_compat_client": "jewel", 2024-09-04T17:48:31.362 INFO:teuthology.orchestra.run.smithi092.stdout: "require_osd_release": "squid", 2024-09-04T17:48:31.362 INFO:teuthology.orchestra.run.smithi092.stdout: "allow_crimson": false, 2024-09-04T17:48:31.362 INFO:teuthology.orchestra.run.smithi092.stdout: "pools": [ 2024-09-04T17:48:31.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.362 INFO:teuthology.orchestra.run.smithi092.stdout: "pool": 1, 2024-09-04T17:48:31.362 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_name": ".mgr", 2024-09-04T17:48:31.362 INFO:teuthology.orchestra.run.smithi092.stdout: "create_time": "2024-09-04T17:45:41.111814+0000", 2024-09-04T17:48:31.362 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 1, 2024-09-04T17:48:31.362 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_names": "hashpspool", 2024-09-04T17:48:31.362 INFO:teuthology.orchestra.run.smithi092.stdout: "type": 1, 2024-09-04T17:48:31.362 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 2, 2024-09-04T17:48:31.363 INFO:teuthology.orchestra.run.smithi092.stdout: "min_size": 1, 2024-09-04T17:48:31.363 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_rule": 0, 2024-09-04T17:48:31.363 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_count": 0, 2024-09-04T17:48:31.363 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_target": 0, 2024-09-04T17:48:31.363 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_barrier": 0, 2024-09-04T17:48:31.363 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-04T17:48:31.363 INFO:teuthology.orchestra.run.smithi092.stdout: "is_stretch_pool": false, 2024-09-04T17:48:31.363 INFO:teuthology.orchestra.run.smithi092.stdout: "object_hash": 2, 2024-09-04T17:48:31.363 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscale_mode": "off", 2024-09-04T17:48:31.363 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num": 1, 2024-09-04T17:48:31.363 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num": 1, 2024-09-04T17:48:31.363 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num_target": 1, 2024-09-04T17:48:31.363 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_target": 1, 2024-09-04T17:48:31.364 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_pending": 1, 2024-09-04T17:48:31.364 INFO:teuthology.orchestra.run.smithi092.stdout: "last_pg_merge_meta": { 2024-09-04T17:48:31.364 INFO:teuthology.orchestra.run.smithi092.stdout: "source_pgid": "0.0", 2024-09-04T17:48:31.364 INFO:teuthology.orchestra.run.smithi092.stdout: "ready_epoch": 0, 2024-09-04T17:48:31.364 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_started": 0, 2024-09-04T17:48:31.364 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_clean": 0, 2024-09-04T17:48:31.364 INFO:teuthology.orchestra.run.smithi092.stdout: "source_version": "0'0", 2024-09-04T17:48:31.364 INFO:teuthology.orchestra.run.smithi092.stdout: "target_version": "0'0" 2024-09-04T17:48:31.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.364 INFO:teuthology.orchestra.run.smithi092.stdout: "last_change": "18", 2024-09-04T17:48:31.364 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend": "0", 2024-09-04T17:48:31.364 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-04T17:48:31.365 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_preluminous": "0", 2024-09-04T17:48:31.365 INFO:teuthology.orchestra.run.smithi092.stdout: "auid": 0, 2024-09-04T17:48:31.365 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_mode": "selfmanaged", 2024-09-04T17:48:31.365 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_seq": 0, 2024-09-04T17:48:31.365 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_epoch": 0, 2024-09-04T17:48:31.365 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_snaps": [], 2024-09-04T17:48:31.365 INFO:teuthology.orchestra.run.smithi092.stdout: "removed_snaps": "[]", 2024-09-04T17:48:31.365 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_bytes": 0, 2024-09-04T17:48:31.365 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_objects": 0, 2024-09-04T17:48:31.365 INFO:teuthology.orchestra.run.smithi092.stdout: "tiers": [], 2024-09-04T17:48:31.365 INFO:teuthology.orchestra.run.smithi092.stdout: "tier_of": -1, 2024-09-04T17:48:31.365 INFO:teuthology.orchestra.run.smithi092.stdout: "read_tier": -1, 2024-09-04T17:48:31.365 INFO:teuthology.orchestra.run.smithi092.stdout: "write_tier": -1, 2024-09-04T17:48:31.366 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_mode": "none", 2024-09-04T17:48:31.366 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_bytes": 0, 2024-09-04T17:48:31.366 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_objects": 0, 2024-09-04T17:48:31.366 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-04T17:48:31.366 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-04T17:48:31.366 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-04T17:48:31.366 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_flush_age": 0, 2024-09-04T17:48:31.366 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_evict_age": 0, 2024-09-04T17:48:31.366 INFO:teuthology.orchestra.run.smithi092.stdout: "erasure_code_profile": "", 2024-09-04T17:48:31.366 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_params": { 2024-09-04T17:48:31.366 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "none" 2024-09-04T17:48:31.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.366 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_period": 0, 2024-09-04T17:48:31.367 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_count": 0, 2024-09-04T17:48:31.367 INFO:teuthology.orchestra.run.smithi092.stdout: "use_gmt_hitset": true, 2024-09-04T17:48:31.367 INFO:teuthology.orchestra.run.smithi092.stdout: "min_read_recency_for_promote": 0, 2024-09-04T17:48:31.367 INFO:teuthology.orchestra.run.smithi092.stdout: "min_write_recency_for_promote": 0, 2024-09-04T17:48:31.367 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_grade_decay_rate": 0, 2024-09-04T17:48:31.367 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_search_last_n": 0, 2024-09-04T17:48:31.367 INFO:teuthology.orchestra.run.smithi092.stdout: "grade_table": [], 2024-09-04T17:48:31.367 INFO:teuthology.orchestra.run.smithi092.stdout: "stripe_width": 0, 2024-09-04T17:48:31.367 INFO:teuthology.orchestra.run.smithi092.stdout: "expected_num_objects": 0, 2024-09-04T17:48:31.367 INFO:teuthology.orchestra.run.smithi092.stdout: "fast_read": false, 2024-09-04T17:48:31.367 INFO:teuthology.orchestra.run.smithi092.stdout: "options": { 2024-09-04T17:48:31.367 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_max": 32, 2024-09-04T17:48:31.367 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_min": 1 2024-09-04T17:48:31.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.368 INFO:teuthology.orchestra.run.smithi092.stdout: "application_metadata": { 2024-09-04T17:48:31.368 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": {} 2024-09-04T17:48:31.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.368 INFO:teuthology.orchestra.run.smithi092.stdout: "read_balance": { 2024-09-04T17:48:31.368 INFO:teuthology.orchestra.run.smithi092.stdout: "score_type": "Fair distribution", 2024-09-04T17:48:31.368 INFO:teuthology.orchestra.run.smithi092.stdout: "score_acting": 6.9000000953674316, 2024-09-04T17:48:31.368 INFO:teuthology.orchestra.run.smithi092.stdout: "score_stable": 6.9000000953674316, 2024-09-04T17:48:31.368 INFO:teuthology.orchestra.run.smithi092.stdout: "optimal_score": 0.28999999165534973, 2024-09-04T17:48:31.368 INFO:teuthology.orchestra.run.smithi092.stdout: "raw_score_acting": 2, 2024-09-04T17:48:31.368 INFO:teuthology.orchestra.run.smithi092.stdout: "raw_score_stable": 2, 2024-09-04T17:48:31.368 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity_weighted": 0.875, 2024-09-04T17:48:31.368 INFO:teuthology.orchestra.run.smithi092.stdout: "average_primary_affinity": 1, 2024-09-04T17:48:31.369 INFO:teuthology.orchestra.run.smithi092.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-04T17:48:31.369 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.369 INFO:teuthology.orchestra.run.smithi092.stdout: "pool": 2, 2024-09-04T17:48:31.369 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_name": "rbd", 2024-09-04T17:48:31.369 INFO:teuthology.orchestra.run.smithi092.stdout: "create_time": "2024-09-04T17:45:46.265395+0000", 2024-09-04T17:48:31.369 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 8193, 2024-09-04T17:48:31.369 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-04T17:48:31.369 INFO:teuthology.orchestra.run.smithi092.stdout: "type": 1, 2024-09-04T17:48:31.369 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 2, 2024-09-04T17:48:31.369 INFO:teuthology.orchestra.run.smithi092.stdout: "min_size": 1, 2024-09-04T17:48:31.369 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_rule": 0, 2024-09-04T17:48:31.369 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_count": 0, 2024-09-04T17:48:31.370 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_target": 0, 2024-09-04T17:48:31.370 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_barrier": 0, 2024-09-04T17:48:31.370 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-04T17:48:31.370 INFO:teuthology.orchestra.run.smithi092.stdout: "is_stretch_pool": false, 2024-09-04T17:48:31.370 INFO:teuthology.orchestra.run.smithi092.stdout: "object_hash": 2, 2024-09-04T17:48:31.370 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscale_mode": "off", 2024-09-04T17:48:31.370 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num": 8, 2024-09-04T17:48:31.370 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num": 8, 2024-09-04T17:48:31.370 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num_target": 8, 2024-09-04T17:48:31.370 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_target": 8, 2024-09-04T17:48:31.370 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_pending": 8, 2024-09-04T17:48:31.370 INFO:teuthology.orchestra.run.smithi092.stdout: "last_pg_merge_meta": { 2024-09-04T17:48:31.370 INFO:teuthology.orchestra.run.smithi092.stdout: "source_pgid": "0.0", 2024-09-04T17:48:31.371 INFO:teuthology.orchestra.run.smithi092.stdout: "ready_epoch": 0, 2024-09-04T17:48:31.371 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_started": 0, 2024-09-04T17:48:31.371 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_clean": 0, 2024-09-04T17:48:31.371 INFO:teuthology.orchestra.run.smithi092.stdout: "source_version": "0'0", 2024-09-04T17:48:31.371 INFO:teuthology.orchestra.run.smithi092.stdout: "target_version": "0'0" 2024-09-04T17:48:31.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.371 INFO:teuthology.orchestra.run.smithi092.stdout: "last_change": "22", 2024-09-04T17:48:31.371 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend": "0", 2024-09-04T17:48:31.371 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-04T17:48:31.371 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_preluminous": "0", 2024-09-04T17:48:31.372 INFO:teuthology.orchestra.run.smithi092.stdout: "auid": 0, 2024-09-04T17:48:31.372 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_mode": "selfmanaged", 2024-09-04T17:48:31.372 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_seq": 2, 2024-09-04T17:48:31.372 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_epoch": 22, 2024-09-04T17:48:31.372 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_snaps": [], 2024-09-04T17:48:31.372 INFO:teuthology.orchestra.run.smithi092.stdout: "removed_snaps": "[]", 2024-09-04T17:48:31.372 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_bytes": 0, 2024-09-04T17:48:31.372 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_objects": 0, 2024-09-04T17:48:31.372 INFO:teuthology.orchestra.run.smithi092.stdout: "tiers": [], 2024-09-04T17:48:31.372 INFO:teuthology.orchestra.run.smithi092.stdout: "tier_of": -1, 2024-09-04T17:48:31.372 INFO:teuthology.orchestra.run.smithi092.stdout: "read_tier": -1, 2024-09-04T17:48:31.372 INFO:teuthology.orchestra.run.smithi092.stdout: "write_tier": -1, 2024-09-04T17:48:31.373 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_mode": "none", 2024-09-04T17:48:31.373 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_bytes": 0, 2024-09-04T17:48:31.373 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_objects": 0, 2024-09-04T17:48:31.373 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-04T17:48:31.373 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-04T17:48:31.373 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-04T17:48:31.373 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_flush_age": 0, 2024-09-04T17:48:31.373 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_evict_age": 0, 2024-09-04T17:48:31.373 INFO:teuthology.orchestra.run.smithi092.stdout: "erasure_code_profile": "", 2024-09-04T17:48:31.373 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_params": { 2024-09-04T17:48:31.373 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "none" 2024-09-04T17:48:31.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.374 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_period": 0, 2024-09-04T17:48:31.374 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_count": 0, 2024-09-04T17:48:31.374 INFO:teuthology.orchestra.run.smithi092.stdout: "use_gmt_hitset": true, 2024-09-04T17:48:31.374 INFO:teuthology.orchestra.run.smithi092.stdout: "min_read_recency_for_promote": 0, 2024-09-04T17:48:31.374 INFO:teuthology.orchestra.run.smithi092.stdout: "min_write_recency_for_promote": 0, 2024-09-04T17:48:31.374 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_grade_decay_rate": 0, 2024-09-04T17:48:31.374 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_search_last_n": 0, 2024-09-04T17:48:31.374 INFO:teuthology.orchestra.run.smithi092.stdout: "grade_table": [], 2024-09-04T17:48:31.374 INFO:teuthology.orchestra.run.smithi092.stdout: "stripe_width": 0, 2024-09-04T17:48:31.374 INFO:teuthology.orchestra.run.smithi092.stdout: "expected_num_objects": 0, 2024-09-04T17:48:31.374 INFO:teuthology.orchestra.run.smithi092.stdout: "fast_read": false, 2024-09-04T17:48:31.374 INFO:teuthology.orchestra.run.smithi092.stdout: "options": {}, 2024-09-04T17:48:31.374 INFO:teuthology.orchestra.run.smithi092.stdout: "application_metadata": { 2024-09-04T17:48:31.374 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd": {} 2024-09-04T17:48:31.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.375 INFO:teuthology.orchestra.run.smithi092.stdout: "read_balance": { 2024-09-04T17:48:31.375 INFO:teuthology.orchestra.run.smithi092.stdout: "score_type": "Fair distribution", 2024-09-04T17:48:31.375 INFO:teuthology.orchestra.run.smithi092.stdout: "score_acting": 1.75, 2024-09-04T17:48:31.375 INFO:teuthology.orchestra.run.smithi092.stdout: "score_stable": 1.75, 2024-09-04T17:48:31.375 INFO:teuthology.orchestra.run.smithi092.stdout: "optimal_score": 1, 2024-09-04T17:48:31.375 INFO:teuthology.orchestra.run.smithi092.stdout: "raw_score_acting": 1.75, 2024-09-04T17:48:31.375 INFO:teuthology.orchestra.run.smithi092.stdout: "raw_score_stable": 1.75, 2024-09-04T17:48:31.375 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity_weighted": 0.875, 2024-09-04T17:48:31.375 INFO:teuthology.orchestra.run.smithi092.stdout: "average_primary_affinity": 1, 2024-09-04T17:48:31.375 INFO:teuthology.orchestra.run.smithi092.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-04T17:48:31.375 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.376 INFO:teuthology.orchestra.run.smithi092.stdout: "pool": 3, 2024-09-04T17:48:31.376 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_name": "pool_client.0", 2024-09-04T17:48:31.376 INFO:teuthology.orchestra.run.smithi092.stdout: "create_time": "2024-09-04T17:45:54.393683+0000", 2024-09-04T17:48:31.376 INFO:teuthology.orchestra.run.smithi092.stdout: "flags": 8193, 2024-09-04T17:48:31.376 INFO:teuthology.orchestra.run.smithi092.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-04T17:48:31.376 INFO:teuthology.orchestra.run.smithi092.stdout: "type": 1, 2024-09-04T17:48:31.376 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 2, 2024-09-04T17:48:31.376 INFO:teuthology.orchestra.run.smithi092.stdout: "min_size": 1, 2024-09-04T17:48:31.376 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_rule": 0, 2024-09-04T17:48:31.376 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_count": 0, 2024-09-04T17:48:31.376 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_target": 0, 2024-09-04T17:48:31.376 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_barrier": 0, 2024-09-04T17:48:31.376 INFO:teuthology.orchestra.run.smithi092.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-04T17:48:31.377 INFO:teuthology.orchestra.run.smithi092.stdout: "is_stretch_pool": false, 2024-09-04T17:48:31.377 INFO:teuthology.orchestra.run.smithi092.stdout: "object_hash": 2, 2024-09-04T17:48:31.377 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_autoscale_mode": "off", 2024-09-04T17:48:31.377 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num": 32, 2024-09-04T17:48:31.377 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num": 32, 2024-09-04T17:48:31.377 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_placement_num_target": 32, 2024-09-04T17:48:31.377 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_target": 32, 2024-09-04T17:48:31.377 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_num_pending": 32, 2024-09-04T17:48:31.377 INFO:teuthology.orchestra.run.smithi092.stdout: "last_pg_merge_meta": { 2024-09-04T17:48:31.377 INFO:teuthology.orchestra.run.smithi092.stdout: "source_pgid": "0.0", 2024-09-04T17:48:31.377 INFO:teuthology.orchestra.run.smithi092.stdout: "ready_epoch": 0, 2024-09-04T17:48:31.377 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_started": 0, 2024-09-04T17:48:31.377 INFO:teuthology.orchestra.run.smithi092.stdout: "last_epoch_clean": 0, 2024-09-04T17:48:31.377 INFO:teuthology.orchestra.run.smithi092.stdout: "source_version": "0'0", 2024-09-04T17:48:31.378 INFO:teuthology.orchestra.run.smithi092.stdout: "target_version": "0'0" 2024-09-04T17:48:31.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.378 INFO:teuthology.orchestra.run.smithi092.stdout: "last_change": "149", 2024-09-04T17:48:31.378 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend": "0", 2024-09-04T17:48:31.378 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-04T17:48:31.378 INFO:teuthology.orchestra.run.smithi092.stdout: "last_force_op_resend_preluminous": "0", 2024-09-04T17:48:31.378 INFO:teuthology.orchestra.run.smithi092.stdout: "auid": 0, 2024-09-04T17:48:31.378 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_mode": "selfmanaged", 2024-09-04T17:48:31.378 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_seq": 45, 2024-09-04T17:48:31.378 INFO:teuthology.orchestra.run.smithi092.stdout: "snap_epoch": 149, 2024-09-04T17:48:31.378 INFO:teuthology.orchestra.run.smithi092.stdout: "pool_snaps": [], 2024-09-04T17:48:31.378 INFO:teuthology.orchestra.run.smithi092.stdout: "removed_snaps": "[]", 2024-09-04T17:48:31.378 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_bytes": 0, 2024-09-04T17:48:31.379 INFO:teuthology.orchestra.run.smithi092.stdout: "quota_max_objects": 0, 2024-09-04T17:48:31.379 INFO:teuthology.orchestra.run.smithi092.stdout: "tiers": [], 2024-09-04T17:48:31.379 INFO:teuthology.orchestra.run.smithi092.stdout: "tier_of": -1, 2024-09-04T17:48:31.379 INFO:teuthology.orchestra.run.smithi092.stdout: "read_tier": -1, 2024-09-04T17:48:31.379 INFO:teuthology.orchestra.run.smithi092.stdout: "write_tier": -1, 2024-09-04T17:48:31.379 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_mode": "none", 2024-09-04T17:48:31.379 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_bytes": 0, 2024-09-04T17:48:31.379 INFO:teuthology.orchestra.run.smithi092.stdout: "target_max_objects": 0, 2024-09-04T17:48:31.379 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-04T17:48:31.379 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-04T17:48:31.379 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-04T17:48:31.379 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_flush_age": 0, 2024-09-04T17:48:31.379 INFO:teuthology.orchestra.run.smithi092.stdout: "cache_min_evict_age": 0, 2024-09-04T17:48:31.380 INFO:teuthology.orchestra.run.smithi092.stdout: "erasure_code_profile": "", 2024-09-04T17:48:31.380 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_params": { 2024-09-04T17:48:31.380 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "none" 2024-09-04T17:48:31.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.380 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_period": 0, 2024-09-04T17:48:31.380 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_count": 0, 2024-09-04T17:48:31.380 INFO:teuthology.orchestra.run.smithi092.stdout: "use_gmt_hitset": true, 2024-09-04T17:48:31.380 INFO:teuthology.orchestra.run.smithi092.stdout: "min_read_recency_for_promote": 0, 2024-09-04T17:48:31.380 INFO:teuthology.orchestra.run.smithi092.stdout: "min_write_recency_for_promote": 0, 2024-09-04T17:48:31.380 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_grade_decay_rate": 0, 2024-09-04T17:48:31.380 INFO:teuthology.orchestra.run.smithi092.stdout: "hit_set_search_last_n": 0, 2024-09-04T17:48:31.380 INFO:teuthology.orchestra.run.smithi092.stdout: "grade_table": [], 2024-09-04T17:48:31.380 INFO:teuthology.orchestra.run.smithi092.stdout: "stripe_width": 0, 2024-09-04T17:48:31.381 INFO:teuthology.orchestra.run.smithi092.stdout: "expected_num_objects": 0, 2024-09-04T17:48:31.381 INFO:teuthology.orchestra.run.smithi092.stdout: "fast_read": false, 2024-09-04T17:48:31.381 INFO:teuthology.orchestra.run.smithi092.stdout: "options": {}, 2024-09-04T17:48:31.381 INFO:teuthology.orchestra.run.smithi092.stdout: "application_metadata": { 2024-09-04T17:48:31.381 INFO:teuthology.orchestra.run.smithi092.stdout: "rbd": {} 2024-09-04T17:48:31.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.381 INFO:teuthology.orchestra.run.smithi092.stdout: "read_balance": { 2024-09-04T17:48:31.381 INFO:teuthology.orchestra.run.smithi092.stdout: "score_type": "Fair distribution", 2024-09-04T17:48:31.381 INFO:teuthology.orchestra.run.smithi092.stdout: "score_acting": 1.5299999713897705, 2024-09-04T17:48:31.381 INFO:teuthology.orchestra.run.smithi092.stdout: "score_stable": 1.5299999713897705, 2024-09-04T17:48:31.381 INFO:teuthology.orchestra.run.smithi092.stdout: "optimal_score": 1, 2024-09-04T17:48:31.381 INFO:teuthology.orchestra.run.smithi092.stdout: "raw_score_acting": 1.5299999713897705, 2024-09-04T17:48:31.381 INFO:teuthology.orchestra.run.smithi092.stdout: "raw_score_stable": 1.5299999713897705, 2024-09-04T17:48:31.382 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity_weighted": 0.875, 2024-09-04T17:48:31.382 INFO:teuthology.orchestra.run.smithi092.stdout: "average_primary_affinity": 1, 2024-09-04T17:48:31.382 INFO:teuthology.orchestra.run.smithi092.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-04T17:48:31.382 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.382 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.382 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-04T17:48:31.382 INFO:teuthology.orchestra.run.smithi092.stdout: "osds": [ 2024-09-04T17:48:31.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.382 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 0, 2024-09-04T17:48:31.382 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "e58efca2-bc5a-4bc4-b3d7-4036ebd9dc63", 2024-09-04T17:48:31.382 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-09-04T17:48:31.382 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-09-04T17:48:31.382 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-09-04T17:48:31.383 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 1, 2024-09-04T17:48:31.383 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-09-04T17:48:31.383 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-09-04T17:48:31.383 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 15, 2024-09-04T17:48:31.383 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 150, 2024-09-04T17:48:31.383 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-09-04T17:48:31.383 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-09-04T17:48:31.383 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-09-04T17:48:31.383 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-04T17:48:31.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.383 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-04T17:48:31.383 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6808", 2024-09-04T17:48:31.383 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2135042466 2024-09-04T17:48:31.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.384 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-04T17:48:31.384 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6809", 2024-09-04T17:48:31.384 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2135042466 2024-09-04T17:48:31.384 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.384 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.384 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-09-04T17:48:31.384 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-04T17:48:31.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.384 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-04T17:48:31.384 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6810", 2024-09-04T17:48:31.385 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2135042466 2024-09-04T17:48:31.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.385 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-04T17:48:31.385 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6811", 2024-09-04T17:48:31.385 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2135042466 2024-09-04T17:48:31.385 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.385 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.385 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-09-04T17:48:31.385 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-04T17:48:31.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.385 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-04T17:48:31.385 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6814", 2024-09-04T17:48:31.386 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2135042466 2024-09-04T17:48:31.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.386 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-04T17:48:31.386 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6815", 2024-09-04T17:48:31.386 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2135042466 2024-09-04T17:48:31.386 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.386 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.386 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-09-04T17:48:31.386 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-04T17:48:31.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.386 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-04T17:48:31.387 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6812", 2024-09-04T17:48:31.387 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2135042466 2024-09-04T17:48:31.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.387 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-04T17:48:31.387 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6813", 2024-09-04T17:48:31.387 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2135042466 2024-09-04T17:48:31.387 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.387 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.387 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.92:6809/2135042466", 2024-09-04T17:48:31.387 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.92:6811/2135042466", 2024-09-04T17:48:31.387 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.92:6815/2135042466", 2024-09-04T17:48:31.388 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.92:6813/2135042466", 2024-09-04T17:48:31.388 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-09-04T17:48:31.388 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-09-04T17:48:31.388 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-09-04T17:48:31.388 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.388 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 1, 2024-09-04T17:48:31.388 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "36e03b64-2602-4143-bdd5-f7355de255fe", 2024-09-04T17:48:31.388 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-09-04T17:48:31.388 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 0, 2024-09-04T17:48:31.388 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 0, 2024-09-04T17:48:31.388 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 1, 2024-09-04T17:48:31.389 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-09-04T17:48:31.389 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-09-04T17:48:31.389 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 15, 2024-09-04T17:48:31.389 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 119, 2024-09-04T17:48:31.389 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-09-04T17:48:31.389 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-09-04T17:48:31.389 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-09-04T17:48:31.389 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-04T17:48:31.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.389 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-04T17:48:31.389 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6824", 2024-09-04T17:48:31.389 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1306159585 2024-09-04T17:48:31.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.390 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-04T17:48:31.390 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6825", 2024-09-04T17:48:31.390 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1306159585 2024-09-04T17:48:31.390 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.390 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.390 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-09-04T17:48:31.390 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-04T17:48:31.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.390 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-04T17:48:31.390 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6826", 2024-09-04T17:48:31.390 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1306159585 2024-09-04T17:48:31.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.391 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-04T17:48:31.391 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6827", 2024-09-04T17:48:31.391 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1306159585 2024-09-04T17:48:31.391 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.391 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.391 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-09-04T17:48:31.391 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-04T17:48:31.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.391 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-04T17:48:31.391 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6830", 2024-09-04T17:48:31.391 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1306159585 2024-09-04T17:48:31.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.392 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-04T17:48:31.392 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6831", 2024-09-04T17:48:31.392 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1306159585 2024-09-04T17:48:31.392 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.392 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.392 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-09-04T17:48:31.392 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-04T17:48:31.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.392 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-04T17:48:31.392 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6828", 2024-09-04T17:48:31.393 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1306159585 2024-09-04T17:48:31.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.393 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-04T17:48:31.393 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6829", 2024-09-04T17:48:31.393 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1306159585 2024-09-04T17:48:31.393 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.393 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.393 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.92:6825/1306159585", 2024-09-04T17:48:31.393 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.92:6827/1306159585", 2024-09-04T17:48:31.393 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.92:6831/1306159585", 2024-09-04T17:48:31.393 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.92:6829/1306159585", 2024-09-04T17:48:31.394 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-09-04T17:48:31.394 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-09-04T17:48:31.394 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-09-04T17:48:31.394 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.394 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 2, 2024-09-04T17:48:31.394 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "581ab71e-be04-4031-a0c4-f7e40b74b7c7", 2024-09-04T17:48:31.394 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-09-04T17:48:31.394 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-09-04T17:48:31.394 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-09-04T17:48:31.394 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 1, 2024-09-04T17:48:31.395 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 14, 2024-09-04T17:48:31.395 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 83, 2024-09-04T17:48:31.395 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 103, 2024-09-04T17:48:31.395 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 148, 2024-09-04T17:48:31.395 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 84, 2024-09-04T17:48:31.395 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-09-04T17:48:31.395 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-09-04T17:48:31.395 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-04T17:48:31.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.395 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-04T17:48:31.395 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6800", 2024-09-04T17:48:31.395 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2500708849 2024-09-04T17:48:31.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.396 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-04T17:48:31.396 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6801", 2024-09-04T17:48:31.396 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2500708849 2024-09-04T17:48:31.396 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.396 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.396 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-09-04T17:48:31.396 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-04T17:48:31.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.396 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-04T17:48:31.396 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6802", 2024-09-04T17:48:31.397 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2500708849 2024-09-04T17:48:31.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.397 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-04T17:48:31.397 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6803", 2024-09-04T17:48:31.397 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2500708849 2024-09-04T17:48:31.397 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.397 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.397 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-09-04T17:48:31.397 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-04T17:48:31.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.398 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-04T17:48:31.398 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6806", 2024-09-04T17:48:31.398 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2500708849 2024-09-04T17:48:31.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.398 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-04T17:48:31.398 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6807", 2024-09-04T17:48:31.398 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2500708849 2024-09-04T17:48:31.398 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.398 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.398 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-09-04T17:48:31.398 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-04T17:48:31.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.399 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-04T17:48:31.399 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6804", 2024-09-04T17:48:31.399 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2500708849 2024-09-04T17:48:31.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.399 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-04T17:48:31.399 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6805", 2024-09-04T17:48:31.399 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 2500708849 2024-09-04T17:48:31.399 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.399 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.399 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.92:6801/2500708849", 2024-09-04T17:48:31.399 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.92:6803/2500708849", 2024-09-04T17:48:31.399 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.92:6807/2500708849", 2024-09-04T17:48:31.400 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.92:6805/2500708849", 2024-09-04T17:48:31.400 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-09-04T17:48:31.400 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-09-04T17:48:31.400 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-09-04T17:48:31.400 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.400 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 3, 2024-09-04T17:48:31.400 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "5f5d096e-fab2-41fe-b8dd-943d1c40894c", 2024-09-04T17:48:31.400 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-09-04T17:48:31.400 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-09-04T17:48:31.400 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-09-04T17:48:31.401 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 1, 2024-09-04T17:48:31.401 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 15, 2024-09-04T17:48:31.401 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 23, 2024-09-04T17:48:31.401 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 48, 2024-09-04T17:48:31.401 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 150, 2024-09-04T17:48:31.401 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 24, 2024-09-04T17:48:31.401 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-09-04T17:48:31.401 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-09-04T17:48:31.401 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-04T17:48:31.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.401 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-04T17:48:31.401 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6816", 2024-09-04T17:48:31.401 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3390028311 2024-09-04T17:48:31.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.402 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-04T17:48:31.402 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6817", 2024-09-04T17:48:31.402 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3390028311 2024-09-04T17:48:31.402 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.402 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.402 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-09-04T17:48:31.402 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-04T17:48:31.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.402 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-04T17:48:31.402 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6818", 2024-09-04T17:48:31.403 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3390028311 2024-09-04T17:48:31.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.403 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-04T17:48:31.403 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6819", 2024-09-04T17:48:31.403 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3390028311 2024-09-04T17:48:31.403 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.403 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.403 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-09-04T17:48:31.403 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-04T17:48:31.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.403 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-04T17:48:31.404 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6822", 2024-09-04T17:48:31.404 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3390028311 2024-09-04T17:48:31.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.404 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-04T17:48:31.404 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6823", 2024-09-04T17:48:31.404 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3390028311 2024-09-04T17:48:31.404 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.404 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.404 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-09-04T17:48:31.404 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-04T17:48:31.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.405 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-04T17:48:31.405 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6820", 2024-09-04T17:48:31.405 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3390028311 2024-09-04T17:48:31.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.405 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-04T17:48:31.405 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:6821", 2024-09-04T17:48:31.405 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3390028311 2024-09-04T17:48:31.405 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.405 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.405 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.92:6817/3390028311", 2024-09-04T17:48:31.405 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.92:6819/3390028311", 2024-09-04T17:48:31.406 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.92:6823/3390028311", 2024-09-04T17:48:31.406 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.92:6821/3390028311", 2024-09-04T17:48:31.406 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-09-04T17:48:31.406 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-09-04T17:48:31.406 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-09-04T17:48:31.406 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.406 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 4, 2024-09-04T17:48:31.406 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "840951a5-e13b-462c-8c8e-2ca61a53fe3c", 2024-09-04T17:48:31.406 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-09-04T17:48:31.406 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-09-04T17:48:31.406 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-09-04T17:48:31.407 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 0, 2024-09-04T17:48:31.407 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-09-04T17:48:31.407 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-09-04T17:48:31.407 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 15, 2024-09-04T17:48:31.407 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 149, 2024-09-04T17:48:31.407 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-09-04T17:48:31.407 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-09-04T17:48:31.407 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-09-04T17:48:31.407 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-04T17:48:31.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.407 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-04T17:48:31.407 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.190:6816", 2024-09-04T17:48:31.407 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 4272750248 2024-09-04T17:48:31.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.408 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-04T17:48:31.408 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.190:6817", 2024-09-04T17:48:31.408 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 4272750248 2024-09-04T17:48:31.408 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.408 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.408 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-09-04T17:48:31.408 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-04T17:48:31.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.408 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-04T17:48:31.408 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.190:6818", 2024-09-04T17:48:31.408 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 4272750248 2024-09-04T17:48:31.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.409 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-04T17:48:31.409 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.190:6819", 2024-09-04T17:48:31.409 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 4272750248 2024-09-04T17:48:31.409 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.409 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.409 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-09-04T17:48:31.409 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-04T17:48:31.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.409 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-04T17:48:31.410 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.190:6822", 2024-09-04T17:48:31.410 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 4272750248 2024-09-04T17:48:31.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.410 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-04T17:48:31.410 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.190:6823", 2024-09-04T17:48:31.410 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 4272750248 2024-09-04T17:48:31.410 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.410 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.410 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-09-04T17:48:31.410 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-04T17:48:31.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.411 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-04T17:48:31.411 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.190:6820", 2024-09-04T17:48:31.411 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 4272750248 2024-09-04T17:48:31.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.411 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-04T17:48:31.411 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.190:6821", 2024-09-04T17:48:31.411 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 4272750248 2024-09-04T17:48:31.411 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.411 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.411 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.190:6817/4272750248", 2024-09-04T17:48:31.411 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.190:6819/4272750248", 2024-09-04T17:48:31.411 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.190:6823/4272750248", 2024-09-04T17:48:31.412 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.190:6821/4272750248", 2024-09-04T17:48:31.412 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-09-04T17:48:31.412 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-09-04T17:48:31.412 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-09-04T17:48:31.412 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.412 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 5, 2024-09-04T17:48:31.412 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "82e556e5-00c3-46fc-aab5-3544bba279d1", 2024-09-04T17:48:31.412 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-09-04T17:48:31.412 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-09-04T17:48:31.412 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-09-04T17:48:31.412 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 1, 2024-09-04T17:48:31.413 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-09-04T17:48:31.413 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-09-04T17:48:31.413 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 15, 2024-09-04T17:48:31.413 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 150, 2024-09-04T17:48:31.413 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-09-04T17:48:31.413 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-09-04T17:48:31.413 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-09-04T17:48:31.413 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-04T17:48:31.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.413 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-04T17:48:31.413 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.190:6808", 2024-09-04T17:48:31.413 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1688427632 2024-09-04T17:48:31.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.414 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-04T17:48:31.414 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.190:6809", 2024-09-04T17:48:31.414 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1688427632 2024-09-04T17:48:31.414 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.414 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.414 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-09-04T17:48:31.414 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-04T17:48:31.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.414 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-04T17:48:31.414 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.190:6810", 2024-09-04T17:48:31.414 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1688427632 2024-09-04T17:48:31.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.415 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-04T17:48:31.415 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.190:6811", 2024-09-04T17:48:31.415 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1688427632 2024-09-04T17:48:31.415 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.415 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.415 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-09-04T17:48:31.415 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-04T17:48:31.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.415 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-04T17:48:31.415 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.190:6814", 2024-09-04T17:48:31.416 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1688427632 2024-09-04T17:48:31.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.416 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-04T17:48:31.416 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.190:6815", 2024-09-04T17:48:31.416 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1688427632 2024-09-04T17:48:31.416 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.416 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.416 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-09-04T17:48:31.416 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-04T17:48:31.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.416 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-04T17:48:31.417 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.190:6812", 2024-09-04T17:48:31.417 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1688427632 2024-09-04T17:48:31.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.417 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-04T17:48:31.417 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.190:6813", 2024-09-04T17:48:31.417 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 1688427632 2024-09-04T17:48:31.417 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.417 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.417 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.190:6809/1688427632", 2024-09-04T17:48:31.417 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.190:6811/1688427632", 2024-09-04T17:48:31.417 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.190:6815/1688427632", 2024-09-04T17:48:31.418 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.190:6813/1688427632", 2024-09-04T17:48:31.418 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-09-04T17:48:31.418 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-09-04T17:48:31.418 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-09-04T17:48:31.418 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.418 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 6, 2024-09-04T17:48:31.418 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "4e1e3d67-3020-470d-98b7-891fdc966238", 2024-09-04T17:48:31.418 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-09-04T17:48:31.418 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-09-04T17:48:31.418 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-09-04T17:48:31.418 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 1, 2024-09-04T17:48:31.419 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 15, 2024-09-04T17:48:31.419 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 58, 2024-09-04T17:48:31.419 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 117, 2024-09-04T17:48:31.419 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 131, 2024-09-04T17:48:31.419 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 60, 2024-09-04T17:48:31.419 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-09-04T17:48:31.419 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-09-04T17:48:31.419 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-04T17:48:31.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.419 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-04T17:48:31.419 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.190:6824", 2024-09-04T17:48:31.419 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 466854783 2024-09-04T17:48:31.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.420 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-04T17:48:31.420 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.190:6825", 2024-09-04T17:48:31.420 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 466854783 2024-09-04T17:48:31.420 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.420 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.420 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-09-04T17:48:31.420 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-04T17:48:31.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.420 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-04T17:48:31.420 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.190:6826", 2024-09-04T17:48:31.421 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 466854783 2024-09-04T17:48:31.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.421 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-04T17:48:31.421 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.190:6827", 2024-09-04T17:48:31.421 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 466854783 2024-09-04T17:48:31.421 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.421 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.421 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-09-04T17:48:31.421 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-04T17:48:31.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.422 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-04T17:48:31.422 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.190:6830", 2024-09-04T17:48:31.422 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 466854783 2024-09-04T17:48:31.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.422 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-04T17:48:31.422 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.190:6831", 2024-09-04T17:48:31.422 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 466854783 2024-09-04T17:48:31.422 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.422 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.422 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-09-04T17:48:31.422 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-04T17:48:31.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.423 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-04T17:48:31.423 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.190:6828", 2024-09-04T17:48:31.423 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 466854783 2024-09-04T17:48:31.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.423 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-04T17:48:31.423 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.190:6829", 2024-09-04T17:48:31.423 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 466854783 2024-09-04T17:48:31.423 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.423 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.423 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.190:6825/466854783", 2024-09-04T17:48:31.423 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.190:6827/466854783", 2024-09-04T17:48:31.424 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.190:6831/466854783", 2024-09-04T17:48:31.424 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.190:6829/466854783", 2024-09-04T17:48:31.424 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-09-04T17:48:31.424 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-09-04T17:48:31.424 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-09-04T17:48:31.424 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.424 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 7, 2024-09-04T17:48:31.424 INFO:teuthology.orchestra.run.smithi092.stdout: "uuid": "bd9c9d3e-27db-4bc2-b998-7f446576a543", 2024-09-04T17:48:31.424 INFO:teuthology.orchestra.run.smithi092.stdout: "up": 1, 2024-09-04T17:48:31.424 INFO:teuthology.orchestra.run.smithi092.stdout: "in": 1, 2024-09-04T17:48:31.425 INFO:teuthology.orchestra.run.smithi092.stdout: "weight": 1, 2024-09-04T17:48:31.425 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_affinity": 1, 2024-09-04T17:48:31.425 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_begin": 0, 2024-09-04T17:48:31.425 INFO:teuthology.orchestra.run.smithi092.stdout: "last_clean_end": 0, 2024-09-04T17:48:31.425 INFO:teuthology.orchestra.run.smithi092.stdout: "up_from": 14, 2024-09-04T17:48:31.425 INFO:teuthology.orchestra.run.smithi092.stdout: "up_thru": 149, 2024-09-04T17:48:31.425 INFO:teuthology.orchestra.run.smithi092.stdout: "down_at": 0, 2024-09-04T17:48:31.425 INFO:teuthology.orchestra.run.smithi092.stdout: "lost_at": 0, 2024-09-04T17:48:31.425 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addrs": { 2024-09-04T17:48:31.425 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-04T17:48:31.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.425 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-04T17:48:31.426 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.190:6800", 2024-09-04T17:48:31.426 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 4006671087 2024-09-04T17:48:31.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.426 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-04T17:48:31.426 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.190:6801", 2024-09-04T17:48:31.426 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 4006671087 2024-09-04T17:48:31.426 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.426 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.426 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addrs": { 2024-09-04T17:48:31.426 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-04T17:48:31.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.426 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-04T17:48:31.427 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.190:6802", 2024-09-04T17:48:31.427 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 4006671087 2024-09-04T17:48:31.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.427 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-04T17:48:31.427 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.190:6803", 2024-09-04T17:48:31.427 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 4006671087 2024-09-04T17:48:31.427 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.427 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.427 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addrs": { 2024-09-04T17:48:31.427 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-04T17:48:31.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.428 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-04T17:48:31.428 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.190:6806", 2024-09-04T17:48:31.428 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 4006671087 2024-09-04T17:48:31.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.428 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-04T17:48:31.428 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.190:6807", 2024-09-04T17:48:31.428 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 4006671087 2024-09-04T17:48:31.428 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.428 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.428 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addrs": { 2024-09-04T17:48:31.429 INFO:teuthology.orchestra.run.smithi092.stdout: "addrvec": [ 2024-09-04T17:48:31.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.429 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v2", 2024-09-04T17:48:31.429 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.190:6804", 2024-09-04T17:48:31.429 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 4006671087 2024-09-04T17:48:31.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.429 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2024-09-04T17:48:31.429 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.190:6805", 2024-09-04T17:48:31.429 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 4006671087 2024-09-04T17:48:31.429 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.429 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.430 INFO:teuthology.orchestra.run.smithi092.stdout: "public_addr": "172.21.15.190:6801/4006671087", 2024-09-04T17:48:31.430 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_addr": "172.21.15.190:6803/4006671087", 2024-09-04T17:48:31.430 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_back_addr": "172.21.15.190:6807/4006671087", 2024-09-04T17:48:31.430 INFO:teuthology.orchestra.run.smithi092.stdout: "heartbeat_front_addr": "172.21.15.190:6805/4006671087", 2024-09-04T17:48:31.430 INFO:teuthology.orchestra.run.smithi092.stdout: "state": [ 2024-09-04T17:48:31.430 INFO:teuthology.orchestra.run.smithi092.stdout: "exists", 2024-09-04T17:48:31.430 INFO:teuthology.orchestra.run.smithi092.stdout: "up" 2024-09-04T17:48:31.430 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.430 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.430 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-04T17:48:31.430 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_xinfo": [ 2024-09-04T17:48:31.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.431 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 0, 2024-09-04T17:48:31.431 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-09-04T17:48:31.431 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-09-04T17:48:31.431 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-09-04T17:48:31.431 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540719139924082687, 2024-09-04T17:48:31.431 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-09-04T17:48:31.431 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-09-04T17:45:37.324122+0000", 2024-09-04T17:48:31.431 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-09-04T17:48:31.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.431 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 1, 2024-09-04T17:48:31.431 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-09-04T17:48:31.431 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-09-04T17:48:31.431 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-09-04T17:48:31.432 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540719139924082687, 2024-09-04T17:48:31.432 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 65536, 2024-09-04T17:48:31.432 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-09-04T17:45:37.856638+0000", 2024-09-04T17:48:31.432 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-09-04T17:48:31.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.432 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 2, 2024-09-04T17:48:31.432 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-09-04T17:48:31.432 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-09-04T17:48:31.432 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-09-04T17:48:31.432 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540719139924082687, 2024-09-04T17:48:31.432 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-09-04T17:48:31.433 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-09-04T17:45:36.966684+0000", 2024-09-04T17:48:31.433 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 84 2024-09-04T17:48:31.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.433 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 3, 2024-09-04T17:48:31.433 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-09-04T17:48:31.433 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-09-04T17:48:31.433 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-09-04T17:48:31.433 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540719139924082687, 2024-09-04T17:48:31.433 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-09-04T17:48:31.433 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-09-04T17:45:37.266271+0000", 2024-09-04T17:48:31.433 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 24 2024-09-04T17:48:31.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.434 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 4, 2024-09-04T17:48:31.434 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-09-04T17:48:31.434 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-09-04T17:48:31.434 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-09-04T17:48:31.434 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540719139924082687, 2024-09-04T17:48:31.434 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-09-04T17:48:31.434 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-09-04T17:45:37.627069+0000", 2024-09-04T17:48:31.434 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-09-04T17:48:31.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.434 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 5, 2024-09-04T17:48:31.434 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-09-04T17:48:31.434 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-09-04T17:48:31.435 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-09-04T17:48:31.435 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540719139924082687, 2024-09-04T17:48:31.435 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-09-04T17:48:31.435 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-09-04T17:45:37.241115+0000", 2024-09-04T17:48:31.435 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-09-04T17:48:31.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.435 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 6, 2024-09-04T17:48:31.435 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-09-04T17:48:31.435 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-09-04T17:48:31.435 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-09-04T17:48:31.435 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540719139924082687, 2024-09-04T17:48:31.435 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-09-04T17:48:31.436 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-09-04T17:45:37.397375+0000", 2024-09-04T17:48:31.436 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 60 2024-09-04T17:48:31.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.436 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": 7, 2024-09-04T17:48:31.436 INFO:teuthology.orchestra.run.smithi092.stdout: "down_stamp": "0.000000", 2024-09-04T17:48:31.436 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_probability": 0, 2024-09-04T17:48:31.436 INFO:teuthology.orchestra.run.smithi092.stdout: "laggy_interval": 0, 2024-09-04T17:48:31.436 INFO:teuthology.orchestra.run.smithi092.stdout: "features": 4540719139924082687, 2024-09-04T17:48:31.436 INFO:teuthology.orchestra.run.smithi092.stdout: "old_weight": 0, 2024-09-04T17:48:31.436 INFO:teuthology.orchestra.run.smithi092.stdout: "last_purged_snaps_scrub": "2024-09-04T17:45:36.619385+0000", 2024-09-04T17:48:31.436 INFO:teuthology.orchestra.run.smithi092.stdout: "dead_epoch": 0 2024-09-04T17:48:31.437 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.437 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-04T17:48:31.437 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_upmap": [], 2024-09-04T17:48:31.437 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_upmap_items": [], 2024-09-04T17:48:31.437 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_upmap_primaries": [], 2024-09-04T17:48:31.437 INFO:teuthology.orchestra.run.smithi092.stdout: "pg_temp": [], 2024-09-04T17:48:31.437 INFO:teuthology.orchestra.run.smithi092.stdout: "primary_temp": [], 2024-09-04T17:48:31.437 INFO:teuthology.orchestra.run.smithi092.stdout: "blocklist": {}, 2024-09-04T17:48:31.437 INFO:teuthology.orchestra.run.smithi092.stdout: "range_blocklist": {}, 2024-09-04T17:48:31.437 INFO:teuthology.orchestra.run.smithi092.stdout: "erasure_code_profiles": { 2024-09-04T17:48:31.437 INFO:teuthology.orchestra.run.smithi092.stdout: "default": { 2024-09-04T17:48:31.437 INFO:teuthology.orchestra.run.smithi092.stdout: "crush-failure-domain": "osd", 2024-09-04T17:48:31.437 INFO:teuthology.orchestra.run.smithi092.stdout: "k": "2", 2024-09-04T17:48:31.438 INFO:teuthology.orchestra.run.smithi092.stdout: "m": "1", 2024-09-04T17:48:31.438 INFO:teuthology.orchestra.run.smithi092.stdout: "plugin": "jerasure", 2024-09-04T17:48:31.438 INFO:teuthology.orchestra.run.smithi092.stdout: "technique": "reed_sol_van" 2024-09-04T17:48:31.438 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.438 INFO:teuthology.orchestra.run.smithi092.stdout: "removed_snaps_queue": [ 2024-09-04T17:48:31.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.438 INFO:teuthology.orchestra.run.smithi092.stdout: "pool": 3, 2024-09-04T17:48:31.438 INFO:teuthology.orchestra.run.smithi092.stdout: "snaps": [ 2024-09-04T17:48:31.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.438 INFO:teuthology.orchestra.run.smithi092.stdout: "begin": 27, 2024-09-04T17:48:31.438 INFO:teuthology.orchestra.run.smithi092.stdout: "length": 7 2024-09-04T17:48:31.438 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.438 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.439 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.439 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-04T17:48:31.439 INFO:teuthology.orchestra.run.smithi092.stdout: "new_removed_snaps": [], 2024-09-04T17:48:31.439 INFO:teuthology.orchestra.run.smithi092.stdout: "new_purged_snaps": [], 2024-09-04T17:48:31.439 INFO:teuthology.orchestra.run.smithi092.stdout: "crush_node_flags": {}, 2024-09-04T17:48:31.439 INFO:teuthology.orchestra.run.smithi092.stdout: "device_class_flags": {}, 2024-09-04T17:48:31.439 INFO:teuthology.orchestra.run.smithi092.stdout: "stretch_mode": { 2024-09-04T17:48:31.439 INFO:teuthology.orchestra.run.smithi092.stdout: "stretch_mode_enabled": false, 2024-09-04T17:48:31.439 INFO:teuthology.orchestra.run.smithi092.stdout: "stretch_bucket_count": 0, 2024-09-04T17:48:31.439 INFO:teuthology.orchestra.run.smithi092.stdout: "degraded_stretch_mode": 0, 2024-09-04T17:48:31.439 INFO:teuthology.orchestra.run.smithi092.stdout: "recovering_stretch_mode": 0, 2024-09-04T17:48:31.439 INFO:teuthology.orchestra.run.smithi092.stdout: "stretch_mode_bucket": 0 2024-09-04T17:48:31.439 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.439 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:31.442 INFO:tasks.thrashosds.thrasher:j is {'epoch': 151, 'fsid': 'f36b6ce2-8c72-4c99-8607-5c53c38ef67b', 'created': '2024-09-04T17:45:28.072591+0000', 'modified': '2024-09-04T17:48:31.038086+0000', 'last_up_change': '2024-09-04T17:47:57.033539+0000', 'last_in_change': '2024-09-04T17:48:26.008743+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 8, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-04T17:45:41.111814+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-04T17:45:46.265395+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-04T17:45:54.393683+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '149', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 45, 'snap_epoch': 149, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5299999713897705, 'score_stable': 1.5299999713897705, 'optimal_score': 1, 'raw_score_acting': 1.5299999713897705, 'raw_score_stable': 1.5299999713897705, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}], 'osds': [{'osd': 0, 'uuid': 'e58efca2-bc5a-4bc4-b3d7-4036ebd9dc63', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 150, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6808', 'nonce': 2135042466}, {'type': 'v1', 'addr': '172.21.15.92:6809', 'nonce': 2135042466}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6810', 'nonce': 2135042466}, {'type': 'v1', 'addr': '172.21.15.92:6811', 'nonce': 2135042466}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6814', 'nonce': 2135042466}, {'type': 'v1', 'addr': '172.21.15.92:6815', 'nonce': 2135042466}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6812', 'nonce': 2135042466}, {'type': 'v1', 'addr': '172.21.15.92:6813', 'nonce': 2135042466}]}, 'public_addr': '172.21.15.92:6809/2135042466', 'cluster_addr': '172.21.15.92:6811/2135042466', 'heartbeat_back_addr': '172.21.15.92:6815/2135042466', 'heartbeat_front_addr': '172.21.15.92:6813/2135042466', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '36e03b64-2602-4143-bdd5-f7355de255fe', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 119, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6824', 'nonce': 1306159585}, {'type': 'v1', 'addr': '172.21.15.92:6825', 'nonce': 1306159585}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6826', 'nonce': 1306159585}, {'type': 'v1', 'addr': '172.21.15.92:6827', 'nonce': 1306159585}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6830', 'nonce': 1306159585}, {'type': 'v1', 'addr': '172.21.15.92:6831', 'nonce': 1306159585}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6828', 'nonce': 1306159585}, {'type': 'v1', 'addr': '172.21.15.92:6829', 'nonce': 1306159585}]}, 'public_addr': '172.21.15.92:6825/1306159585', 'cluster_addr': '172.21.15.92:6827/1306159585', 'heartbeat_back_addr': '172.21.15.92:6831/1306159585', 'heartbeat_front_addr': '172.21.15.92:6829/1306159585', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '581ab71e-be04-4031-a0c4-f7e40b74b7c7', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 14, 'last_clean_end': 83, 'up_from': 103, 'up_thru': 148, 'down_at': 84, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6800', 'nonce': 2500708849}, {'type': 'v1', 'addr': '172.21.15.92:6801', 'nonce': 2500708849}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6802', 'nonce': 2500708849}, {'type': 'v1', 'addr': '172.21.15.92:6803', 'nonce': 2500708849}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6806', 'nonce': 2500708849}, {'type': 'v1', 'addr': '172.21.15.92:6807', 'nonce': 2500708849}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6804', 'nonce': 2500708849}, {'type': 'v1', 'addr': '172.21.15.92:6805', 'nonce': 2500708849}]}, 'public_addr': '172.21.15.92:6801/2500708849', 'cluster_addr': '172.21.15.92:6803/2500708849', 'heartbeat_back_addr': '172.21.15.92:6807/2500708849', 'heartbeat_front_addr': '172.21.15.92:6805/2500708849', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '5f5d096e-fab2-41fe-b8dd-943d1c40894c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 15, 'last_clean_end': 23, 'up_from': 48, 'up_thru': 150, 'down_at': 24, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6816', 'nonce': 3390028311}, {'type': 'v1', 'addr': '172.21.15.92:6817', 'nonce': 3390028311}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6818', 'nonce': 3390028311}, {'type': 'v1', 'addr': '172.21.15.92:6819', 'nonce': 3390028311}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6822', 'nonce': 3390028311}, {'type': 'v1', 'addr': '172.21.15.92:6823', 'nonce': 3390028311}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6820', 'nonce': 3390028311}, {'type': 'v1', 'addr': '172.21.15.92:6821', 'nonce': 3390028311}]}, 'public_addr': '172.21.15.92:6817/3390028311', 'cluster_addr': '172.21.15.92:6819/3390028311', 'heartbeat_back_addr': '172.21.15.92:6823/3390028311', 'heartbeat_front_addr': '172.21.15.92:6821/3390028311', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '840951a5-e13b-462c-8c8e-2ca61a53fe3c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 149, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6816', 'nonce': 4272750248}, {'type': 'v1', 'addr': '172.21.15.190:6817', 'nonce': 4272750248}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6818', 'nonce': 4272750248}, {'type': 'v1', 'addr': '172.21.15.190:6819', 'nonce': 4272750248}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6822', 'nonce': 4272750248}, {'type': 'v1', 'addr': '172.21.15.190:6823', 'nonce': 4272750248}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6820', 'nonce': 4272750248}, {'type': 'v1', 'addr': '172.21.15.190:6821', 'nonce': 4272750248}]}, 'public_addr': '172.21.15.190:6817/4272750248', 'cluster_addr': '172.21.15.190:6819/4272750248', 'heartbeat_back_addr': '172.21.15.190:6823/4272750248', 'heartbeat_front_addr': '172.21.15.190:6821/4272750248', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '82e556e5-00c3-46fc-aab5-3544bba279d1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 150, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6808', 'nonce': 1688427632}, {'type': 'v1', 'addr': '172.21.15.190:6809', 'nonce': 1688427632}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6810', 'nonce': 1688427632}, {'type': 'v1', 'addr': '172.21.15.190:6811', 'nonce': 1688427632}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6814', 'nonce': 1688427632}, {'type': 'v1', 'addr': '172.21.15.190:6815', 'nonce': 1688427632}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6812', 'nonce': 1688427632}, {'type': 'v1', 'addr': '172.21.15.190:6813', 'nonce': 1688427632}]}, 'public_addr': '172.21.15.190:6809/1688427632', 'cluster_addr': '172.21.15.190:6811/1688427632', 'heartbeat_back_addr': '172.21.15.190:6815/1688427632', 'heartbeat_front_addr': '172.21.15.190:6813/1688427632', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '4e1e3d67-3020-470d-98b7-891fdc966238', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 15, 'last_clean_end': 58, 'up_from': 117, 'up_thru': 131, 'down_at': 60, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6824', 'nonce': 466854783}, {'type': 'v1', 'addr': '172.21.15.190:6825', 'nonce': 466854783}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6826', 'nonce': 466854783}, {'type': 'v1', 'addr': '172.21.15.190:6827', 'nonce': 466854783}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6830', 'nonce': 466854783}, {'type': 'v1', 'addr': '172.21.15.190:6831', 'nonce': 466854783}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6828', 'nonce': 466854783}, {'type': 'v1', 'addr': '172.21.15.190:6829', 'nonce': 466854783}]}, 'public_addr': '172.21.15.190:6825/466854783', 'cluster_addr': '172.21.15.190:6827/466854783', 'heartbeat_back_addr': '172.21.15.190:6831/466854783', 'heartbeat_front_addr': '172.21.15.190:6829/466854783', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'bd9c9d3e-27db-4bc2-b998-7f446576a543', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 149, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6800', 'nonce': 4006671087}, {'type': 'v1', 'addr': '172.21.15.190:6801', 'nonce': 4006671087}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6802', 'nonce': 4006671087}, {'type': 'v1', 'addr': '172.21.15.190:6803', 'nonce': 4006671087}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6806', 'nonce': 4006671087}, {'type': 'v1', 'addr': '172.21.15.190:6807', 'nonce': 4006671087}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6804', 'nonce': 4006671087}, {'type': 'v1', 'addr': '172.21.15.190:6805', 'nonce': 4006671087}]}, 'public_addr': '172.21.15.190:6801/4006671087', 'cluster_addr': '172.21.15.190:6803/4006671087', 'heartbeat_back_addr': '172.21.15.190:6807/4006671087', 'heartbeat_front_addr': '172.21.15.190:6805/4006671087', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-04T17:45:37.324122+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-04T17:45:37.856638+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-04T17:45:36.966684+0000', 'dead_epoch': 84}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-04T17:45:37.266271+0000', 'dead_epoch': 24}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-04T17:45:37.627069+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-04T17:45:37.241115+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-04T17:45:37.397375+0000', 'dead_epoch': 60}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-04T17:45:36.619385+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 27, 'length': 7}]}], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-04T17:48:31.442 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-04T17:48:31.443 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:31.443 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:48:31.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:48:31.443 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:48:31.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.443 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:27242 3.b 3.34414deb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-04T17:48:31.443 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:28.402347+0000", 2024-09-04T17:48:31.443 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 123.024822241, 2024-09-04T17:48:31.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.93265329299999999, 2024-09-04T17:48:31.443 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:31.444 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:31.444 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-04T17:48:31.444 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:31.444 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:31.444 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:31.444 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 27242 2024-09-04T17:48:31.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.444 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:31.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:31.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402347+0000", 2024-09-04T17:48:31.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:31.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:31.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402347+0000", 2024-09-04T17:48:31.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:31.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:31.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402349+0000", 2024-09-04T17:48:31.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.471e-06 2024-09-04T17:48:31.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:31.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402355+0000", 2024-09-04T17:48:31.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8399999999999997e-06 2024-09-04T17:48:31.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:31.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402364+0000", 2024-09-04T17:48:31.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1269999999999992e-06 2024-09-04T17:48:31.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:31.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402373+0000", 2024-09-04T17:48:31.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7909999999999993e-06 2024-09-04T17:48:31.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:31.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402460+0000", 2024-09-04T17:48:31.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6686000000000003e-05 2024-09-04T17:48:31.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-04T17:48:31.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:28.402473+0000", 2024-09-04T17:48:31.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3281e-05 2024-09-04T17:48:31.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:31.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:29.334989+0000", 2024-09-04T17:48:31.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.93251549099999997 2024-09-04T17:48:31.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:31.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:29.335000+0000", 2024-09-04T17:48:31.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1606e-05 2024-09-04T17:48:31.448 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.448 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.448 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.448 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:33552 3.7 3:e79ad1db:::rbd_data.11342e8f3384.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-04T17:48:31.448 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:35.999748+0000", 2024-09-04T17:48:31.448 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 115.42742119099999, 2024-09-04T17:48:31.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045683823999999998, 2024-09-04T17:48:31.449 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:31.449 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:31.449 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:31.449 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:31.449 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:31.449 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:31.449 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 33552 2024-09-04T17:48:31.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.449 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:31.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:31.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999748+0000", 2024-09-04T17:48:31.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:31.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.450 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:31.450 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999748+0000", 2024-09-04T17:48:31.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:31.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.450 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:31.450 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999750+0000", 2024-09-04T17:48:31.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1299999999999999e-06 2024-09-04T17:48:31.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.451 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:31.451 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999759+0000", 2024-09-04T17:48:31.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4649999999999994e-06 2024-09-04T17:48:31.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.451 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:31.451 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999778+0000", 2024-09-04T17:48:31.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8833000000000001e-05 2024-09-04T17:48:31.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.451 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:31.451 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999786+0000", 2024-09-04T17:48:31.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7369999999999995e-06 2024-09-04T17:48:31.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:31.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999856+0000", 2024-09-04T17:48:31.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9247999999999998e-05 2024-09-04T17:48:31.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:31.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:35.999973+0000", 2024-09-04T17:48:31.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000117921 2024-09-04T17:48:31.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 7", 2024-09-04T17:48:31.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.005955+0000", 2024-09-04T17:48:31.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059820560000000004 2024-09-04T17:48:31.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.453 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:31.453 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.045297+0000", 2024-09-04T17:48:31.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.039341650999999998 2024-09-04T17:48:31.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.453 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:31.453 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.045343+0000", 2024-09-04T17:48:31.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5899000000000002e-05 2024-09-04T17:48:31.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.453 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:31.454 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.045388+0000", 2024-09-04T17:48:31.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4601999999999997e-05 2024-09-04T17:48:31.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.454 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:31.454 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:36.045432+0000", 2024-09-04T17:48:31.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4282000000000001e-05 2024-09-04T17:48:31.454 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.454 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.454 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.454 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:37357 3.15 3:a897312a:::rbd_data.1134c5a43465.000000000000001f:head [stat,write 62464~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-04T17:48:31.454 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:39.616724+0000", 2024-09-04T17:48:31.455 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 111.81044578300001, 2024-09-04T17:48:31.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054965854000000001, 2024-09-04T17:48:31.455 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:31.455 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:31.455 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:31.455 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:31.455 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:31.455 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:31.455 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 37357 2024-09-04T17:48:31.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.455 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:31.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:31.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616724+0000", 2024-09-04T17:48:31.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:31.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:31.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616724+0000", 2024-09-04T17:48:31.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:31.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:31.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616725+0000", 2024-09-04T17:48:31.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.815e-06 2024-09-04T17:48:31.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:31.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616739+0000", 2024-09-04T17:48:31.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3723000000000001e-05 2024-09-04T17:48:31.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:31.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616755+0000", 2024-09-04T17:48:31.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5964000000000002e-05 2024-09-04T17:48:31.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:31.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616763+0000", 2024-09-04T17:48:31.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3869999999999995e-06 2024-09-04T17:48:31.458 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.458 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.458 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:31.458 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616825+0000", 2024-09-04T17:48:31.458 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1854999999999996e-05 2024-09-04T17:48:31.458 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.458 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.458 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:31.458 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616895+0000", 2024-09-04T17:48:31.458 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9597999999999993e-05 2024-09-04T17:48:31.458 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.458 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-04T17:48:31.459 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.616991+0000", 2024-09-04T17:48:31.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5828000000000005e-05 2024-09-04T17:48:31.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:31.459 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.617659+0000", 2024-09-04T17:48:31.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00066821600000000001 2024-09-04T17:48:31.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:31.459 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.671482+0000", 2024-09-04T17:48:31.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053823533999999999 2024-09-04T17:48:31.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:31.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.671579+0000", 2024-09-04T17:48:31.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6657999999999995e-05 2024-09-04T17:48:31.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:31.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:39.671689+0000", 2024-09-04T17:48:31.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000110276 2024-09-04T17:48:31.460 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.460 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.460 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.461 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:43873 3.15 3:abd207ee:::rbd_data.113410ddd206.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:31.461 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:49.932940+0000", 2024-09-04T17:48:31.461 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 101.49422923, 2024-09-04T17:48:31.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042740514, 2024-09-04T17:48:31.461 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:31.461 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:31.461 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:31.461 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:31.461 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:31.461 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:31.461 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 43873 2024-09-04T17:48:31.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.462 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:31.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.462 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:31.462 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.932940+0000", 2024-09-04T17:48:31.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:31.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.462 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:31.462 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.932940+0000", 2024-09-04T17:48:31.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:31.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.462 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:31.463 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.932941+0000", 2024-09-04T17:48:31.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.404e-06 2024-09-04T17:48:31.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.463 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:31.463 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.933065+0000", 2024-09-04T17:48:31.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012361199999999999 2024-09-04T17:48:31.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.463 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:31.463 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.933078+0000", 2024-09-04T17:48:31.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3182e-05 2024-09-04T17:48:31.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.464 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:31.464 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.933085+0000", 2024-09-04T17:48:31.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4500000000000001e-06 2024-09-04T17:48:31.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.464 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:31.464 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.933134+0000", 2024-09-04T17:48:31.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9150999999999997e-05 2024-09-04T17:48:31.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.464 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:31.464 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.933242+0000", 2024-09-04T17:48:31.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010781 2024-09-04T17:48:31.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-04T17:48:31.465 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.938358+0000", 2024-09-04T17:48:31.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051160349999999997 2024-09-04T17:48:31.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:31.465 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.975343+0000", 2024-09-04T17:48:31.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036985663000000002 2024-09-04T17:48:31.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:31.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.975392+0000", 2024-09-04T17:48:31.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9044e-05 2024-09-04T17:48:31.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:31.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.975548+0000", 2024-09-04T17:48:31.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015572900000000001 2024-09-04T17:48:31.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:31.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:49.975681+0000", 2024-09-04T17:48:31.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013243400000000001 2024-09-04T17:48:31.466 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.467 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.467 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.467 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.467 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:46649 3.7 3:e0749ed5:::rbd_data.1134326b938e.000000000000014f:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-04T17:48:31.467 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:54.114730+0000", 2024-09-04T17:48:31.467 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 97.312439097999999, 2024-09-04T17:48:31.467 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032972972000000003, 2024-09-04T17:48:31.467 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:31.467 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:31.467 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:31.467 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:31.467 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:31.467 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:31.468 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 46649 2024-09-04T17:48:31.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.468 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:31.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.468 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:31.468 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114730+0000", 2024-09-04T17:48:31.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:31.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.468 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:31.468 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114730+0000", 2024-09-04T17:48:31.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:31.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.469 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:31.469 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114731+0000", 2024-09-04T17:48:31.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.147e-06 2024-09-04T17:48:31.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.469 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:31.469 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114737+0000", 2024-09-04T17:48:31.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6380000000000001e-06 2024-09-04T17:48:31.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.469 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:31.469 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114751+0000", 2024-09-04T17:48:31.470 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4012999999999999e-05 2024-09-04T17:48:31.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.470 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:31.470 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114758+0000", 2024-09-04T17:48:31.470 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1539999999999996e-06 2024-09-04T17:48:31.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.470 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:31.470 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.114876+0000", 2024-09-04T17:48:31.470 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000117421 2024-09-04T17:48:31.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:31.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.115173+0000", 2024-09-04T17:48:31.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00029750799999999998 2024-09-04T17:48:31.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 7", 2024-09-04T17:48:31.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.120273+0000", 2024-09-04T17:48:31.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051002900000000004 2024-09-04T17:48:31.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:31.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.147571+0000", 2024-09-04T17:48:31.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027297906 2024-09-04T17:48:31.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.472 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:31.472 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.147588+0000", 2024-09-04T17:48:31.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6807999999999999e-05 2024-09-04T17:48:31.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.472 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:31.472 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.147637+0000", 2024-09-04T17:48:31.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8466e-05 2024-09-04T17:48:31.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.473 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:31.473 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:54.147703+0000", 2024-09-04T17:48:31.473 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6620999999999998e-05 2024-09-04T17:48:31.473 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.473 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.473 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.473 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49801 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 720896~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:48:31.473 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.075094+0000", 2024-09-04T17:48:31.473 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 88.352075111000005, 2024-09-04T17:48:31.474 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017824974, 2024-09-04T17:48:31.474 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:31.474 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:31.474 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:31.474 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:31.474 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:31.474 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:31.474 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49801 2024-09-04T17:48:31.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.474 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:31.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.474 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:31.474 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075094+0000", 2024-09-04T17:48:31.474 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:31.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.475 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:31.475 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075094+0000", 2024-09-04T17:48:31.475 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:31.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.475 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:31.475 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075095+0000", 2024-09-04T17:48:31.475 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.017e-06 2024-09-04T17:48:31.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.475 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:31.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075101+0000", 2024-09-04T17:48:31.476 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5029999999999997e-06 2024-09-04T17:48:31.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:31.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075119+0000", 2024-09-04T17:48:31.476 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8133000000000001e-05 2024-09-04T17:48:31.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:31.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.075132+0000", 2024-09-04T17:48:31.476 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3305999999999999e-05 2024-09-04T17:48:31.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.477 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:31.477 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.092677+0000", 2024-09-04T17:48:31.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017544466000000002 2024-09-04T17:48:31.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.477 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:31.477 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.092782+0000", 2024-09-04T17:48:31.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000105284 2024-09-04T17:48:31.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.477 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:31.477 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.092919+0000", 2024-09-04T17:48:31.478 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013726499999999999 2024-09-04T17:48:31.478 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.478 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.478 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.478 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.478 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49809 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 163840~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:48:31.478 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077146+0000", 2024-09-04T17:48:31.478 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 88.350023668000006, 2024-09-04T17:48:31.478 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017984541999999999, 2024-09-04T17:48:31.478 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:31.478 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:31.478 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:31.478 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:31.479 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:31.479 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:31.479 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49809 2024-09-04T17:48:31.479 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.479 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:31.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.479 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:31.479 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077146+0000", 2024-09-04T17:48:31.479 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:31.479 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.479 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:31.479 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077146+0000", 2024-09-04T17:48:31.480 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:31.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.480 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.480 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:31.480 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077147+0000", 2024-09-04T17:48:31.480 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0330000000000001e-06 2024-09-04T17:48:31.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.480 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.480 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:31.480 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077164+0000", 2024-09-04T17:48:31.480 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7810000000000001e-05 2024-09-04T17:48:31.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.480 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.480 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:31.481 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077187+0000", 2024-09-04T17:48:31.481 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2246e-05 2024-09-04T17:48:31.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.481 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:31.481 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077203+0000", 2024-09-04T17:48:31.481 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6392000000000002e-05 2024-09-04T17:48:31.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.485 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.485 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:31.485 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.094885+0000", 2024-09-04T17:48:31.485 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017681589000000001 2024-09-04T17:48:31.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.485 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.485 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:31.485 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.094991+0000", 2024-09-04T17:48:31.485 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010668899999999999 2024-09-04T17:48:31.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.485 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.485 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:31.486 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095130+0000", 2024-09-04T17:48:31.486 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013878299999999999 2024-09-04T17:48:31.486 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.486 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.486 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.486 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.486 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.486 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49810 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 229376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:48:31.486 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077225+0000", 2024-09-04T17:48:31.486 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 88.349943827000004, 2024-09-04T17:48:31.486 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018218793, 2024-09-04T17:48:31.486 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:31.486 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:31.487 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:31.487 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:31.487 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:31.487 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:31.487 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49810 2024-09-04T17:48:31.487 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.487 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:31.487 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.487 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:31.487 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077225+0000", 2024-09-04T17:48:31.487 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:31.487 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.487 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.487 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:31.488 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077225+0000", 2024-09-04T17:48:31.488 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:31.488 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.488 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.488 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:31.488 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077227+0000", 2024-09-04T17:48:31.488 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1829999999999999e-06 2024-09-04T17:48:31.488 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.488 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.488 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:31.488 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077233+0000", 2024-09-04T17:48:31.488 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3899999999999998e-06 2024-09-04T17:48:31.488 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.489 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.489 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:31.489 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077253+0000", 2024-09-04T17:48:31.489 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0009000000000001e-05 2024-09-04T17:48:31.489 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.489 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.489 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:31.489 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077270+0000", 2024-09-04T17:48:31.489 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7336000000000001e-05 2024-09-04T17:48:31.489 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.489 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.489 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:31.489 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095177+0000", 2024-09-04T17:48:31.490 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.017907101000000002 2024-09-04T17:48:31.490 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.490 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.490 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:31.490 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095299+0000", 2024-09-04T17:48:31.490 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000121749 2024-09-04T17:48:31.490 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.490 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.490 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:31.490 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095444+0000", 2024-09-04T17:48:31.490 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014502499999999999 2024-09-04T17:48:31.490 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.490 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.491 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.491 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.491 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.491 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49811 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 294912~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:48:31.491 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077288+0000", 2024-09-04T17:48:31.491 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 88.349881513, 2024-09-04T17:48:31.491 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018443620000000001, 2024-09-04T17:48:31.491 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:31.491 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:31.491 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:31.491 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:31.491 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:31.491 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:31.491 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49811 2024-09-04T17:48:31.492 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.492 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:31.492 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.492 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:31.492 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077288+0000", 2024-09-04T17:48:31.492 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:31.492 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.492 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.492 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:31.492 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077288+0000", 2024-09-04T17:48:31.492 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:31.492 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.492 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.493 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:31.493 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077289+0000", 2024-09-04T17:48:31.493 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6199999999999999e-06 2024-09-04T17:48:31.493 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.493 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.493 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:31.493 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077295+0000", 2024-09-04T17:48:31.493 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1499999999999998e-06 2024-09-04T17:48:31.493 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.493 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.493 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:31.493 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077312+0000", 2024-09-04T17:48:31.493 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7353000000000001e-05 2024-09-04T17:48:31.493 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.494 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.494 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:31.494 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077324+0000", 2024-09-04T17:48:31.494 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2593e-05 2024-09-04T17:48:31.494 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.494 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.494 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:31.494 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095490+0000", 2024-09-04T17:48:31.494 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018165976 2024-09-04T17:48:31.494 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.494 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.494 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:31.494 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095597+0000", 2024-09-04T17:48:31.495 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010629900000000001 2024-09-04T17:48:31.495 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.495 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.495 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:31.495 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095731+0000", 2024-09-04T17:48:31.495 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013462899999999999 2024-09-04T17:48:31.495 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.495 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.495 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.495 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.495 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.495 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49813 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 360448~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:48:31.495 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077349+0000", 2024-09-04T17:48:31.496 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 88.349820311000002, 2024-09-04T17:48:31.496 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018780305000000001, 2024-09-04T17:48:31.496 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:31.496 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:31.496 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:31.496 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:31.496 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:31.496 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:31.496 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49813 2024-09-04T17:48:31.496 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.496 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:31.496 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.496 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:31.496 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077349+0000", 2024-09-04T17:48:31.497 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:31.497 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.497 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.497 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:31.497 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077349+0000", 2024-09-04T17:48:31.497 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:31.497 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.497 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.497 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:31.497 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077350+0000", 2024-09-04T17:48:31.497 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0330000000000001e-06 2024-09-04T17:48:31.497 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.497 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.498 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:31.498 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077355+0000", 2024-09-04T17:48:31.498 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0699999999999997e-06 2024-09-04T17:48:31.498 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.498 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.498 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:31.498 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077376+0000", 2024-09-04T17:48:31.498 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0846e-05 2024-09-04T17:48:31.498 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.498 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.498 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:31.498 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077388+0000", 2024-09-04T17:48:31.498 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2353e-05 2024-09-04T17:48:31.498 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.499 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.499 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:31.499 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095819+0000", 2024-09-04T17:48:31.499 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01843057 2024-09-04T17:48:31.499 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.499 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.499 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:31.499 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.095960+0000", 2024-09-04T17:48:31.499 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000140965 2024-09-04T17:48:31.499 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.499 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.499 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:31.499 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096129+0000", 2024-09-04T17:48:31.500 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016946800000000001 2024-09-04T17:48:31.500 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.500 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.500 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.500 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.500 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.500 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49814 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 425984~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:48:31.500 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077407+0000", 2024-09-04T17:48:31.500 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 88.349761810000004, 2024-09-04T17:48:31.500 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019041563000000001, 2024-09-04T17:48:31.500 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:31.500 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:31.500 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:31.501 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:31.501 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:31.501 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:31.501 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49814 2024-09-04T17:48:31.501 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.501 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:31.501 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.501 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:31.501 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077407+0000", 2024-09-04T17:48:31.501 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:31.501 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.501 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.501 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:31.501 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077407+0000", 2024-09-04T17:48:31.502 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:31.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.502 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:31.502 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077409+0000", 2024-09-04T17:48:31.502 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2330000000000001e-06 2024-09-04T17:48:31.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.502 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:31.502 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077415+0000", 2024-09-04T17:48:31.502 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2729999999999999e-06 2024-09-04T17:48:31.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.503 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:31.503 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077436+0000", 2024-09-04T17:48:31.503 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1209999999999999e-05 2024-09-04T17:48:31.503 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.503 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.503 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:31.503 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077448+0000", 2024-09-04T17:48:31.503 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2225999999999999e-05 2024-09-04T17:48:31.503 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.503 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.503 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:31.503 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096183+0000", 2024-09-04T17:48:31.503 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018735004 2024-09-04T17:48:31.503 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.504 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.504 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:31.504 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096297+0000", 2024-09-04T17:48:31.504 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000113962 2024-09-04T17:48:31.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.504 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.504 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:31.504 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096449+0000", 2024-09-04T17:48:31.504 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000151655 2024-09-04T17:48:31.504 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.504 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.504 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.505 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49815 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 491520~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:48:31.505 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077471+0000", 2024-09-04T17:48:31.505 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 88.349698571999994, 2024-09-04T17:48:31.505 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019266536000000001, 2024-09-04T17:48:31.505 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:31.505 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:31.505 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:31.505 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:31.505 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:31.505 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:31.505 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49815 2024-09-04T17:48:31.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.506 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:31.506 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.506 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:31.506 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077471+0000", 2024-09-04T17:48:31.506 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:31.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.506 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.506 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:31.506 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077471+0000", 2024-09-04T17:48:31.506 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:31.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.506 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.506 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:31.507 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077472+0000", 2024-09-04T17:48:31.507 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.17e-06 2024-09-04T17:48:31.507 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.507 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.507 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:31.507 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077478+0000", 2024-09-04T17:48:31.507 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0399999999999998e-06 2024-09-04T17:48:31.507 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.507 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.507 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:31.507 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077500+0000", 2024-09-04T17:48:31.507 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1959000000000001e-05 2024-09-04T17:48:31.507 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.508 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.508 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:31.508 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077513+0000", 2024-09-04T17:48:31.508 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2823000000000001e-05 2024-09-04T17:48:31.508 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.508 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.508 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:31.508 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096493+0000", 2024-09-04T17:48:31.508 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.018980701999999999 2024-09-04T17:48:31.508 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.508 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.508 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:31.508 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096603+0000", 2024-09-04T17:48:31.509 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010942000000000001 2024-09-04T17:48:31.509 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.509 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.509 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:31.509 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096737+0000", 2024-09-04T17:48:31.509 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013442200000000001 2024-09-04T17:48:31.509 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.509 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.509 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.509 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.509 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.509 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49816 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 557056~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:48:31.510 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077532+0000", 2024-09-04T17:48:31.510 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 88.349636830999998, 2024-09-04T17:48:31.510 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019604331999999999, 2024-09-04T17:48:31.510 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:31.510 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:31.510 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:31.510 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:31.510 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:31.510 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:31.510 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49816 2024-09-04T17:48:31.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.510 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:31.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.511 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:31.511 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077532+0000", 2024-09-04T17:48:31.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:31.511 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.511 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.511 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:31.511 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077532+0000", 2024-09-04T17:48:31.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:31.511 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.511 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.511 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:31.511 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077534+0000", 2024-09-04T17:48:31.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1200000000000001e-06 2024-09-04T17:48:31.512 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.512 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.512 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:31.512 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077545+0000", 2024-09-04T17:48:31.512 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1673e-05 2024-09-04T17:48:31.512 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.512 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.512 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:31.512 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077569+0000", 2024-09-04T17:48:31.512 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3555999999999999e-05 2024-09-04T17:48:31.512 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.512 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.512 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:31.513 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077584+0000", 2024-09-04T17:48:31.513 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4883e-05 2024-09-04T17:48:31.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.513 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.513 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:31.513 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096825+0000", 2024-09-04T17:48:31.513 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019241734 2024-09-04T17:48:31.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.513 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.513 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:31.513 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.096964+0000", 2024-09-04T17:48:31.513 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000138085 2024-09-04T17:48:31.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.514 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.514 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:31.514 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097137+0000", 2024-09-04T17:48:31.514 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017328100000000001 2024-09-04T17:48:31.514 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.514 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.514 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.514 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.514 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.514 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49817 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 622592~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:48:31.514 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077603+0000", 2024-09-04T17:48:31.514 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 88.349566272999994, 2024-09-04T17:48:31.514 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01986928, 2024-09-04T17:48:31.514 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:31.515 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:31.515 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:31.515 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:31.515 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:31.515 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:31.515 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49817 2024-09-04T17:48:31.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.515 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:31.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.515 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:31.515 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077603+0000", 2024-09-04T17:48:31.515 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:31.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.516 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.516 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:31.516 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077603+0000", 2024-09-04T17:48:31.516 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:31.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.516 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.516 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:31.516 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077604+0000", 2024-09-04T17:48:31.516 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7699999999999992e-07 2024-09-04T17:48:31.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.516 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.516 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:31.516 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077609+0000", 2024-09-04T17:48:31.517 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0830000000000003e-06 2024-09-04T17:48:31.517 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.517 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.517 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:31.517 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077630+0000", 2024-09-04T17:48:31.517 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0438999999999999e-05 2024-09-04T17:48:31.517 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.517 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.517 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:31.517 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077664+0000", 2024-09-04T17:48:31.517 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4579000000000002e-05 2024-09-04T17:48:31.517 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.517 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.518 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:31.518 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097193+0000", 2024-09-04T17:48:31.518 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019529174 2024-09-04T17:48:31.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.518 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.518 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:31.518 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097323+0000", 2024-09-04T17:48:31.518 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000129696 2024-09-04T17:48:31.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.518 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.518 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:31.518 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097472+0000", 2024-09-04T17:48:31.518 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014933200000000001 2024-09-04T17:48:31.518 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.519 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.519 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.519 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.519 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49818 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 688128~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:48:31.519 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077683+0000", 2024-09-04T17:48:31.519 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 88.349485908999995, 2024-09-04T17:48:31.519 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020086333000000001, 2024-09-04T17:48:31.519 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:31.519 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:31.519 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:31.519 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:31.520 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:31.520 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:31.520 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49818 2024-09-04T17:48:31.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.520 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:31.520 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.520 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:31.520 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077683+0000", 2024-09-04T17:48:31.520 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:31.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.520 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.520 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:31.520 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077683+0000", 2024-09-04T17:48:31.521 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:31.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.521 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.521 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:31.521 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077684+0000", 2024-09-04T17:48:31.521 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4e-07 2024-09-04T17:48:31.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.521 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.521 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:31.521 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077690+0000", 2024-09-04T17:48:31.521 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4729999999999999e-06 2024-09-04T17:48:31.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.521 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.522 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:31.522 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077710+0000", 2024-09-04T17:48:31.522 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0149999999999999e-05 2024-09-04T17:48:31.522 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.522 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.522 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:31.522 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077723+0000", 2024-09-04T17:48:31.522 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3271999999999999e-05 2024-09-04T17:48:31.522 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.522 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.522 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:31.522 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097521+0000", 2024-09-04T17:48:31.522 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.019797981999999999 2024-09-04T17:48:31.523 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.523 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.523 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:31.523 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097628+0000", 2024-09-04T17:48:31.523 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010665700000000001 2024-09-04T17:48:31.523 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.523 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.523 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:31.523 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097770+0000", 2024-09-04T17:48:31.523 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014185900000000001 2024-09-04T17:48:31.523 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.523 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.524 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.524 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.524 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49820 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 753664~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:48:31.524 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077744+0000", 2024-09-04T17:48:31.524 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 88.349424978000002, 2024-09-04T17:48:31.524 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020364786999999999, 2024-09-04T17:48:31.524 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:31.524 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:31.524 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:31.524 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:31.524 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:31.524 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:31.525 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49820 2024-09-04T17:48:31.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.525 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:31.525 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.525 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:31.525 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077744+0000", 2024-09-04T17:48:31.525 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:31.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.525 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.525 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:31.525 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077744+0000", 2024-09-04T17:48:31.525 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:31.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.525 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.526 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:31.526 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077745+0000", 2024-09-04T17:48:31.526 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0270000000000001e-06 2024-09-04T17:48:31.526 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.526 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.526 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:31.526 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077751+0000", 2024-09-04T17:48:31.526 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.626e-06 2024-09-04T17:48:31.526 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.526 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.526 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:31.526 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077771+0000", 2024-09-04T17:48:31.526 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0469999999999999e-05 2024-09-04T17:48:31.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.527 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.527 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:31.527 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077785+0000", 2024-09-04T17:48:31.527 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3129000000000001e-05 2024-09-04T17:48:31.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.527 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.527 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:31.527 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097816+0000", 2024-09-04T17:48:31.527 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020031917999999999 2024-09-04T17:48:31.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.527 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.528 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:31.528 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.097939+0000", 2024-09-04T17:48:31.528 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000122533 2024-09-04T17:48:31.528 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.528 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.528 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:31.528 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098109+0000", 2024-09-04T17:48:31.528 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017008400000000001 2024-09-04T17:48:31.528 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.528 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.528 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.528 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.529 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.529 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49821 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 819200~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:48:31.529 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077807+0000", 2024-09-04T17:48:31.529 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 88.349362765999999, 2024-09-04T17:48:31.529 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020702928999999998, 2024-09-04T17:48:31.529 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:31.529 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:31.529 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:31.529 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:31.529 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:31.529 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:31.529 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49821 2024-09-04T17:48:31.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.529 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:31.530 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.530 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:31.530 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077807+0000", 2024-09-04T17:48:31.530 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:31.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.530 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.530 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:31.530 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077807+0000", 2024-09-04T17:48:31.530 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:31.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.530 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.530 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:31.530 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077808+0000", 2024-09-04T17:48:31.531 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.463e-06 2024-09-04T17:48:31.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.531 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.531 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:31.531 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077813+0000", 2024-09-04T17:48:31.531 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7530000000000001e-06 2024-09-04T17:48:31.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.531 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.531 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:31.531 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077832+0000", 2024-09-04T17:48:31.531 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.946e-05 2024-09-04T17:48:31.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.531 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.532 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:31.532 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077846+0000", 2024-09-04T17:48:31.532 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3786e-05 2024-09-04T17:48:31.532 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.532 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.532 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:31.532 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098201+0000", 2024-09-04T17:48:31.532 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020354797000000001 2024-09-04T17:48:31.532 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.532 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.532 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:31.532 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098341+0000", 2024-09-04T17:48:31.532 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000140042 2024-09-04T17:48:31.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.533 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.533 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:31.533 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098509+0000", 2024-09-04T17:48:31.533 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016862800000000001 2024-09-04T17:48:31.533 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.533 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.533 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.533 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.533 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49823 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 884736~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:48:31.533 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077865+0000", 2024-09-04T17:48:31.533 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 88.349304321999995, 2024-09-04T17:48:31.534 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021008723, 2024-09-04T17:48:31.534 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:31.534 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:31.534 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:31.534 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:31.534 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:31.534 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:31.534 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49823 2024-09-04T17:48:31.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.534 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:31.534 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.534 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:31.534 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077865+0000", 2024-09-04T17:48:31.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:31.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:31.535 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077865+0000", 2024-09-04T17:48:31.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:31.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:31.535 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077866+0000", 2024-09-04T17:48:31.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.164e-06 2024-09-04T17:48:31.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.536 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:31.536 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077872+0000", 2024-09-04T17:48:31.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3959999999999997e-06 2024-09-04T17:48:31.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.536 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.536 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:31.536 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077892+0000", 2024-09-04T17:48:31.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0970000000000001e-05 2024-09-04T17:48:31.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.536 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.536 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:31.536 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077905+0000", 2024-09-04T17:48:31.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2312e-05 2024-09-04T17:48:31.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.537 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.537 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:31.537 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098579+0000", 2024-09-04T17:48:31.537 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020674486999999998 2024-09-04T17:48:31.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.537 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.537 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:31.537 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098705+0000", 2024-09-04T17:48:31.537 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012541900000000001 2024-09-04T17:48:31.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.537 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.537 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:31.538 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098874+0000", 2024-09-04T17:48:31.538 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000168975 2024-09-04T17:48:31.538 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.538 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.538 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.538 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.538 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.538 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49824 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 950272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:48:31.538 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.077923+0000", 2024-09-04T17:48:31.538 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 88.349246629999996, 2024-09-04T17:48:31.538 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021246287999999999, 2024-09-04T17:48:31.538 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:31.538 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:31.539 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:31.539 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:31.539 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:31.539 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:31.539 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49824 2024-09-04T17:48:31.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.539 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:31.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.539 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:31.539 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077923+0000", 2024-09-04T17:48:31.539 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:31.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:31.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077923+0000", 2024-09-04T17:48:31.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:31.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:31.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077924+0000", 2024-09-04T17:48:31.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0100000000000001e-06 2024-09-04T17:48:31.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:31.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077944+0000", 2024-09-04T17:48:31.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0449e-05 2024-09-04T17:48:31.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.541 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:31.541 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077965+0000", 2024-09-04T17:48:31.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0479000000000002e-05 2024-09-04T17:48:31.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.541 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:31.541 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.077990+0000", 2024-09-04T17:48:31.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5582999999999999e-05 2024-09-04T17:48:31.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.541 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:31.542 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.098924+0000", 2024-09-04T17:48:31.542 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.020933395 2024-09-04T17:48:31.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.542 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.542 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:31.542 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.099032+0000", 2024-09-04T17:48:31.542 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010846299999999999 2024-09-04T17:48:31.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.542 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.542 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:31.542 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.099169+0000", 2024-09-04T17:48:31.542 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000136909 2024-09-04T17:48:31.542 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.543 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.543 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.543 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.543 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:49826 3.b 3:d5942660:::rbd_data.1134968ccb73.000000000000000b:1a [read 1015808~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-04T17:48:31.543 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:03.078010+0000", 2024-09-04T17:48:31.543 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 88.349159022999999, 2024-09-04T17:48:31.543 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021584564000000001, 2024-09-04T17:48:31.543 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:31.543 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:31.543 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:31.543 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:31.543 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:31.544 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:31.544 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 49826 2024-09-04T17:48:31.544 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.544 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:31.544 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.544 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:31.544 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078010+0000", 2024-09-04T17:48:31.544 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:31.544 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.544 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.544 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:31.544 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078010+0000", 2024-09-04T17:48:31.544 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:31.544 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.545 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.545 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:31.545 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078011+0000", 2024-09-04T17:48:31.545 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1340000000000001e-06 2024-09-04T17:48:31.545 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.545 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.545 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:31.545 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078017+0000", 2024-09-04T17:48:31.545 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.682e-06 2024-09-04T17:48:31.545 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.545 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.545 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:31.545 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078036+0000", 2024-09-04T17:48:31.545 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9357000000000001e-05 2024-09-04T17:48:31.546 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.546 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.546 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:31.546 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.078050+0000", 2024-09-04T17:48:31.546 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3122000000000001e-05 2024-09-04T17:48:31.546 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.546 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.546 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:31.546 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.099271+0000", 2024-09-04T17:48:31.546 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.021221772999999999 2024-09-04T17:48:31.546 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.546 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.546 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:31.547 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.099410+0000", 2024-09-04T17:48:31.547 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000138415 2024-09-04T17:48:31.547 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.547 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.547 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:31.547 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:03.099595+0000", 2024-09-04T17:48:31.547 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000185081 2024-09-04T17:48:31.547 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.547 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.547 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.547 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.547 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.547 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:31.548 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-04T17:48:31.549 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:48:31.442+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:48:31.549 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:31.547+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:31.650 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:31.650+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:31.676 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:31.676 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:48:31.676 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:48:31.676 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:31.690 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-04T17:48:31.751 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:31.750+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:31.813 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:31.813 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:48:31.813 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:48:31.813 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:48:31.813 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:31.827 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-04T17:48:31.873 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:48:31.871+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:48:31.946 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:31.946 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:48:31.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:48:31.946 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:48:31.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.947 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:71638 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-04T17:48:31.947 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:44.479908+0000", 2024-09-04T17:48:31.947 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.464004367000001, 2024-09-04T17:48:31.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.469396488, 2024-09-04T17:48:31.947 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:31.947 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:31.947 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:31.947 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:31.947 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:31.947 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:31.947 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 71638 2024-09-04T17:48:31.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.948 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:31.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:31.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.479908+0000", 2024-09-04T17:48:31.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:31.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:31.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.479908+0000", 2024-09-04T17:48:31.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:31.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:31.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.479911+0000", 2024-09-04T17:48:31.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9160000000000001e-06 2024-09-04T17:48:31.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:31.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.479925+0000", 2024-09-04T17:48:31.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3896e-05 2024-09-04T17:48:31.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:31.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.479954+0000", 2024-09-04T17:48:31.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9249000000000001e-05 2024-09-04T17:48:31.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:31.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.479969+0000", 2024-09-04T17:48:31.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4683e-05 2024-09-04T17:48:31.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:31.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.480109+0000", 2024-09-04T17:48:31.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013995699999999999 2024-09-04T17:48:31.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-04T17:48:31.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.480125+0000", 2024-09-04T17:48:31.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6651999999999999e-05 2024-09-04T17:48:31.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:31.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.948598+0000", 2024-09-04T17:48:31.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.46847249600000002 2024-09-04T17:48:31.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:31.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.948889+0000", 2024-09-04T17:48:31.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000290947 2024-09-04T17:48:31.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:31.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.949304+0000", 2024-09-04T17:48:31.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000415692 2024-09-04T17:48:31.952 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.952 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.952 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.952 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:72267 3.1a 3:58dbd2e1:::rbd_data.11342ece613c.000000000000001e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-09-04T17:48:31.952 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:45.674662+0000", 2024-09-04T17:48:31.953 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 46.269250558000003, 2024-09-04T17:48:31.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032248339000000001, 2024-09-04T17:48:31.953 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:31.953 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:31.953 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:31.953 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:31.953 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:31.953 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:31.953 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 72267 2024-09-04T17:48:31.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.953 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:31.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.954 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:31.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.674662+0000", 2024-09-04T17:48:31.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:31.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.954 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:31.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.674662+0000", 2024-09-04T17:48:31.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:31.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.954 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:31.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.674664+0000", 2024-09-04T17:48:31.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4870000000000001e-06 2024-09-04T17:48:31.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:31.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.674689+0000", 2024-09-04T17:48:31.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4805000000000001e-05 2024-09-04T17:48:31.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:31.956 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.674715+0000", 2024-09-04T17:48:31.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5548999999999999e-05 2024-09-04T17:48:31.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.956 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:31.956 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.674731+0000", 2024-09-04T17:48:31.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6549999999999999e-05 2024-09-04T17:48:31.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.957 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:31.957 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.674843+0000", 2024-09-04T17:48:31.957 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000112001 2024-09-04T17:48:31.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.957 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:31.957 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.675176+0000", 2024-09-04T17:48:31.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00033337200000000002 2024-09-04T17:48:31.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-04T17:48:31.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.680121+0000", 2024-09-04T17:48:31.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0049445540000000003 2024-09-04T17:48:31.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.959 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:31.959 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.706702+0000", 2024-09-04T17:48:31.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026580840000000001 2024-09-04T17:48:31.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.959 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:31.959 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.706738+0000", 2024-09-04T17:48:31.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6158999999999997e-05 2024-09-04T17:48:31.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.960 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.960 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:31.960 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.706835+0000", 2024-09-04T17:48:31.960 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6909000000000003e-05 2024-09-04T17:48:31.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.960 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:31.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.706910+0000", 2024-09-04T17:48:31.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5112999999999997e-05 2024-09-04T17:48:31.961 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.961 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.961 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.962 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94568 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 1703936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:31.962 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.605863+0000", 2024-09-04T17:48:31.962 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.338049742999999, 2024-09-04T17:48:31.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027249301, 2024-09-04T17:48:31.962 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:31.962 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:31.962 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:31.962 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:31.963 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:31.963 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:31.963 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94568 2024-09-04T17:48:31.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.963 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:31.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:31.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605863+0000", 2024-09-04T17:48:31.964 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:31.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.964 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.964 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:31.964 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605863+0000", 2024-09-04T17:48:31.964 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:31.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.964 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.965 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:31.965 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605863+0000", 2024-09-04T17:48:31.965 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7999999999999996e-07 2024-09-04T17:48:31.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.965 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:31.965 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605866+0000", 2024-09-04T17:48:31.965 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6199999999999999e-06 2024-09-04T17:48:31.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:31.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605875+0000", 2024-09-04T17:48:31.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2499999999999995e-06 2024-09-04T17:48:31.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:31.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605881+0000", 2024-09-04T17:48:31.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.19e-06 2024-09-04T17:48:31.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:31.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.632629+0000", 2024-09-04T17:48:31.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026747567 2024-09-04T17:48:31.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.968 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:31.968 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.632874+0000", 2024-09-04T17:48:31.968 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024493600000000001 2024-09-04T17:48:31.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.968 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:31.968 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.633112+0000", 2024-09-04T17:48:31.968 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023825800000000001 2024-09-04T17:48:31.969 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.969 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.969 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.969 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94569 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:31.969 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.605888+0000", 2024-09-04T17:48:31.970 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.338023913999997, 2024-09-04T17:48:31.970 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027675620000000001, 2024-09-04T17:48:31.970 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:31.970 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:31.970 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:31.970 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:31.970 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:31.970 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:31.970 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94569 2024-09-04T17:48:31.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.970 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:31.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.970 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:31.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605888+0000", 2024-09-04T17:48:31.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:31.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.971 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:31.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605888+0000", 2024-09-04T17:48:31.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:31.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.971 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:31.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605889+0000", 2024-09-04T17:48:31.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4799999999999998e-07 2024-09-04T17:48:31.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:31.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605891+0000", 2024-09-04T17:48:31.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4880000000000001e-06 2024-09-04T17:48:31.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:31.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605901+0000", 2024-09-04T17:48:31.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4760000000000004e-06 2024-09-04T17:48:31.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.973 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.973 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:31.973 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605912+0000", 2024-09-04T17:48:31.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0698e-05 2024-09-04T17:48:31.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.973 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.973 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:31.973 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.633184+0000", 2024-09-04T17:48:31.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.02727253 2024-09-04T17:48:31.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.973 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.974 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:31.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.633330+0000", 2024-09-04T17:48:31.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014543800000000001 2024-09-04T17:48:31.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.974 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:31.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.633564+0000", 2024-09-04T17:48:31.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023444199999999999 2024-09-04T17:48:31.974 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.974 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.975 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.975 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94570 3.0 3.3f52c000 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:31.975 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.605932+0000", 2024-09-04T17:48:31.975 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.337980346999998, 2024-09-04T17:48:31.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027687021999999999, 2024-09-04T17:48:31.975 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:31.975 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:31.975 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-04T17:48:31.975 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:31.975 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:31.976 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:31.976 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94570 2024-09-04T17:48:31.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.976 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:31.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:31.976 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605932+0000", 2024-09-04T17:48:31.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:31.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:31.976 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605932+0000", 2024-09-04T17:48:31.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:31.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.977 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:31.977 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605933+0000", 2024-09-04T17:48:31.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5000000000000001e-07 2024-09-04T17:48:31.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.977 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:31.977 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605936+0000", 2024-09-04T17:48:31.978 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7769999999999998e-06 2024-09-04T17:48:31.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.978 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.978 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:31.978 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605943+0000", 2024-09-04T17:48:31.978 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7000000000000008e-06 2024-09-04T17:48:31.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.978 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.978 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:31.978 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605949+0000", 2024-09-04T17:48:31.978 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0170000000000002e-06 2024-09-04T17:48:31.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.978 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.978 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:31.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.633604+0000", 2024-09-04T17:48:31.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027654875999999998 2024-09-04T17:48:31.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:31.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.633619+0000", 2024-09-04T17:48:31.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4802e-05 2024-09-04T17:48:31.979 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.979 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.979 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.979 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94641 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 6094848~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:31.980 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.637288+0000", 2024-09-04T17:48:31.980 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.306623895999998, 2024-09-04T17:48:31.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026966783000000001, 2024-09-04T17:48:31.980 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:31.980 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:31.980 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:31.980 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:31.980 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:31.980 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:31.981 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94641 2024-09-04T17:48:31.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.981 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:31.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:31.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.637288+0000", 2024-09-04T17:48:31.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:31.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:31.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.637288+0000", 2024-09-04T17:48:31.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:31.982 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.982 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.982 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:31.982 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.637289+0000", 2024-09-04T17:48:31.982 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4600000000000005e-07 2024-09-04T17:48:31.982 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.982 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.982 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:31.982 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.637292+0000", 2024-09-04T17:48:31.982 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1049999999999999e-06 2024-09-04T17:48:31.982 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.982 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.982 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:31.983 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.637301+0000", 2024-09-04T17:48:31.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7509999999999997e-06 2024-09-04T17:48:31.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.983 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:31.983 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.637309+0000", 2024-09-04T17:48:31.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.67e-06 2024-09-04T17:48:31.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.983 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:31.983 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.663971+0000", 2024-09-04T17:48:31.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026661926999999998 2024-09-04T17:48:31.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:31.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.664062+0000", 2024-09-04T17:48:31.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0673000000000004e-05 2024-09-04T17:48:31.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:31.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.664255+0000", 2024-09-04T17:48:31.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019311099999999999 2024-09-04T17:48:31.984 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.984 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.984 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.985 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94642 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 6225920~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:31.985 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.637321+0000", 2024-09-04T17:48:31.985 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.306590948999997, 2024-09-04T17:48:31.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027131534999999998, 2024-09-04T17:48:31.985 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:31.985 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:31.985 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:31.985 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:31.985 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:31.985 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:31.985 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94642 2024-09-04T17:48:31.986 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.986 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:31.986 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.986 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:31.986 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.637321+0000", 2024-09-04T17:48:31.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:31.986 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.986 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.986 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:31.986 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.637321+0000", 2024-09-04T17:48:31.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:31.986 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.986 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.987 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:31.987 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.637322+0000", 2024-09-04T17:48:31.987 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6599999999999996e-07 2024-09-04T17:48:31.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.987 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.987 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:31.987 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.637325+0000", 2024-09-04T17:48:31.987 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1070000000000001e-06 2024-09-04T17:48:31.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.987 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.987 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:31.987 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.637337+0000", 2024-09-04T17:48:31.987 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1766000000000001e-05 2024-09-04T17:48:31.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.988 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:31.988 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.637350+0000", 2024-09-04T17:48:31.988 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3515e-05 2024-09-04T17:48:31.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.988 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:31.988 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.664273+0000", 2024-09-04T17:48:31.988 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026922643 2024-09-04T17:48:31.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.988 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:31.989 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.664337+0000", 2024-09-04T17:48:31.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4492999999999994e-05 2024-09-04T17:48:31.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.989 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.989 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:31.989 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.664453+0000", 2024-09-04T17:48:31.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000115345 2024-09-04T17:48:31.989 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.989 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.989 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.989 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.990 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94647 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 6684672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:31.990 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.638572+0000", 2024-09-04T17:48:31.990 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.305340063000003, 2024-09-04T17:48:31.990 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026910377999999999, 2024-09-04T17:48:31.990 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:31.990 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:31.990 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:31.990 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:31.990 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:31.990 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:31.990 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94647 2024-09-04T17:48:31.990 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.990 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:31.991 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.991 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:31.991 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638572+0000", 2024-09-04T17:48:31.991 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:31.991 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.991 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.991 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:31.991 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638572+0000", 2024-09-04T17:48:31.991 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:31.991 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.991 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.991 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:31.991 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638573+0000", 2024-09-04T17:48:31.991 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7599999999999997e-07 2024-09-04T17:48:31.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.992 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:31.992 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638575+0000", 2024-09-04T17:48:31.992 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2010000000000002e-06 2024-09-04T17:48:31.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.992 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:31.992 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638582+0000", 2024-09-04T17:48:31.992 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7399999999999998e-06 2024-09-04T17:48:31.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:31.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638588+0000", 2024-09-04T17:48:31.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8560000000000003e-06 2024-09-04T17:48:31.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:31.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.665294+0000", 2024-09-04T17:48:31.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026705893000000001 2024-09-04T17:48:31.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:31.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.665355+0000", 2024-09-04T17:48:31.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1253000000000005e-05 2024-09-04T17:48:31.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:31.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.665483+0000", 2024-09-04T17:48:31.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012775899999999999 2024-09-04T17:48:31.994 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.994 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:31.995 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:31.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.995 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94648 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 6815744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:31.995 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.638593+0000", 2024-09-04T17:48:31.995 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.305318974999999, 2024-09-04T17:48:31.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027163676000000001, 2024-09-04T17:48:31.995 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:31.995 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:31.995 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:31.995 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:31.995 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:31.995 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:31.996 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94648 2024-09-04T17:48:31.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.996 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:31.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.996 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:31.996 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638593+0000", 2024-09-04T17:48:31.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:31.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.996 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:31.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638593+0000", 2024-09-04T17:48:31.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:31.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:31.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638594+0000", 2024-09-04T17:48:31.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.53e-07 2024-09-04T17:48:31.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:31.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638596+0000", 2024-09-04T17:48:31.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8870000000000001e-06 2024-09-04T17:48:31.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:31.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638600+0000", 2024-09-04T17:48:31.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3290000000000004e-06 2024-09-04T17:48:31.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:31.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638603+0000", 2024-09-04T17:48:31.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4680000000000001e-06 2024-09-04T17:48:31.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:31.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:31.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:31.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.665524+0000", 2024-09-04T17:48:31.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.02692052 2024-09-04T17:48:32.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:32.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.665600+0000", 2024-09-04T17:48:32.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6569000000000005e-05 2024-09-04T17:48:32.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:32.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.665757+0000", 2024-09-04T17:48:32.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015655000000000001 2024-09-04T17:48:32.000 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:32.000 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:32.001 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:32.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.001 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94649 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 6946816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:32.001 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.638608+0000", 2024-09-04T17:48:32.001 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.305304544999998, 2024-09-04T17:48:32.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027355753999999999, 2024-09-04T17:48:32.001 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:32.001 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:32.001 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:32.001 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:32.001 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:32.001 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:32.002 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94649 2024-09-04T17:48:32.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.002 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:32.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:32.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638608+0000", 2024-09-04T17:48:32.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:32.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:32.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638608+0000", 2024-09-04T17:48:32.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:32.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:32.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638608+0000", 2024-09-04T17:48:32.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3099999999999998e-07 2024-09-04T17:48:32.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:32.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638610+0000", 2024-09-04T17:48:32.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.716e-06 2024-09-04T17:48:32.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:32.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638614+0000", 2024-09-04T17:48:32.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.941e-06 2024-09-04T17:48:32.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:32.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638618+0000", 2024-09-04T17:48:32.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5009999999999998e-06 2024-09-04T17:48:32.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:32.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.665795+0000", 2024-09-04T17:48:32.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027176404000000001 2024-09-04T17:48:32.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:32.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.665851+0000", 2024-09-04T17:48:32.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6530999999999998e-05 2024-09-04T17:48:32.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:32.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.665963+0000", 2024-09-04T17:48:32.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011213 2024-09-04T17:48:32.006 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:32.006 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:32.006 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:32.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.006 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94650 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 7077888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:32.006 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.638623+0000", 2024-09-04T17:48:32.006 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.305289248999998, 2024-09-04T17:48:32.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027665261999999999, 2024-09-04T17:48:32.006 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:32.006 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:32.006 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:32.006 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:32.007 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:32.007 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:32.007 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94650 2024-09-04T17:48:32.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.007 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:32.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:32.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638623+0000", 2024-09-04T17:48:32.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:32.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:32.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638623+0000", 2024-09-04T17:48:32.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:32.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:32.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638623+0000", 2024-09-04T17:48:32.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6300000000000001e-07 2024-09-04T17:48:32.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:32.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638625+0000", 2024-09-04T17:48:32.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7260000000000001e-06 2024-09-04T17:48:32.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:32.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638629+0000", 2024-09-04T17:48:32.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1309999999999999e-06 2024-09-04T17:48:32.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:32.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638634+0000", 2024-09-04T17:48:32.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2700000000000004e-06 2024-09-04T17:48:32.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:32.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.666028+0000", 2024-09-04T17:48:32.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027393303000000001 2024-09-04T17:48:32.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:32.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.666112+0000", 2024-09-04T17:48:32.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3844000000000006e-05 2024-09-04T17:48:32.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:32.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.666288+0000", 2024-09-04T17:48:32.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000176725 2024-09-04T17:48:32.011 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:32.011 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:32.011 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:32.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.011 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94651 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 7208960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:32.011 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.638639+0000", 2024-09-04T17:48:32.011 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.305272854999998, 2024-09-04T17:48:32.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027839592999999999, 2024-09-04T17:48:32.011 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:32.012 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:32.012 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:32.012 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:32.012 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:32.012 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:32.012 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94651 2024-09-04T17:48:32.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.012 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:32.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:32.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638639+0000", 2024-09-04T17:48:32.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:32.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:32.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638639+0000", 2024-09-04T17:48:32.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:32.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:32.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638640+0000", 2024-09-04T17:48:32.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8000000000000002e-07 2024-09-04T17:48:32.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:32.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638641+0000", 2024-09-04T17:48:32.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.77e-06 2024-09-04T17:48:32.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:32.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638646+0000", 2024-09-04T17:48:32.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1729999999999997e-06 2024-09-04T17:48:32.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:32.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638649+0000", 2024-09-04T17:48:32.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5260000000000002e-06 2024-09-04T17:48:32.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:32.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.666309+0000", 2024-09-04T17:48:32.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027660179 2024-09-04T17:48:32.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:32.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.666373+0000", 2024-09-04T17:48:32.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4152999999999994e-05 2024-09-04T17:48:32.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:32.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.666479+0000", 2024-09-04T17:48:32.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000105512 2024-09-04T17:48:32.015 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:32.015 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:32.016 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:32.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.016 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94657 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 7012352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:32.016 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.641793+0000", 2024-09-04T17:48:32.016 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.302119371000003, 2024-09-04T17:48:32.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026893672, 2024-09-04T17:48:32.016 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:32.016 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:32.016 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:32.016 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:32.016 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:32.017 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:32.017 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94657 2024-09-04T17:48:32.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.017 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:32.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:32.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641793+0000", 2024-09-04T17:48:32.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:32.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:32.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641793+0000", 2024-09-04T17:48:32.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:32.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:32.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641793+0000", 2024-09-04T17:48:32.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5000000000000003e-07 2024-09-04T17:48:32.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:32.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641797+0000", 2024-09-04T17:48:32.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2909999999999999e-06 2024-09-04T17:48:32.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:32.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641806+0000", 2024-09-04T17:48:32.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3270000000000007e-06 2024-09-04T17:48:32.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:32.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641812+0000", 2024-09-04T17:48:32.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2310000000000001e-06 2024-09-04T17:48:32.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:32.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.668412+0000", 2024-09-04T17:48:32.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026600104999999999 2024-09-04T17:48:32.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:32.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.668534+0000", 2024-09-04T17:48:32.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012115100000000001 2024-09-04T17:48:32.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:32.021 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.668687+0000", 2024-09-04T17:48:32.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000153017 2024-09-04T17:48:32.021 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:32.021 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:32.021 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:32.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.021 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94658 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 7143424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:32.021 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.641822+0000", 2024-09-04T17:48:32.021 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.302090440000001, 2024-09-04T17:48:32.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027395880000000001, 2024-09-04T17:48:32.021 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:32.022 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:32.022 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:32.022 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:32.022 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:32.022 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:32.022 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94658 2024-09-04T17:48:32.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.022 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:32.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:32.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641822+0000", 2024-09-04T17:48:32.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:32.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:32.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641822+0000", 2024-09-04T17:48:32.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:32.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:32.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641822+0000", 2024-09-04T17:48:32.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.58e-07 2024-09-04T17:48:32.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:32.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641825+0000", 2024-09-04T17:48:32.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5909999999999998e-06 2024-09-04T17:48:32.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:32.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641832+0000", 2024-09-04T17:48:32.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1799999999999999e-06 2024-09-04T17:48:32.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:32.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641838+0000", 2024-09-04T17:48:32.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5040000000000002e-06 2024-09-04T17:48:32.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:32.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.668748+0000", 2024-09-04T17:48:32.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026910119999999999 2024-09-04T17:48:32.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:32.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.668896+0000", 2024-09-04T17:48:32.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014804299999999999 2024-09-04T17:48:32.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:32.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.669218+0000", 2024-09-04T17:48:32.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00032198399999999999 2024-09-04T17:48:32.027 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:32.027 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:32.027 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:32.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.027 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94659 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 7274496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:32.027 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.641846+0000", 2024-09-04T17:48:32.027 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.302066369999999, 2024-09-04T17:48:32.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027757314000000002, 2024-09-04T17:48:32.027 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:32.027 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:32.027 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:32.027 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:32.028 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:32.028 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:32.028 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94659 2024-09-04T17:48:32.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.028 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:32.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:32.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641846+0000", 2024-09-04T17:48:32.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:32.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:32.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641846+0000", 2024-09-04T17:48:32.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:32.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:32.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641846+0000", 2024-09-04T17:48:32.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5400000000000002e-07 2024-09-04T17:48:32.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:32.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641849+0000", 2024-09-04T17:48:32.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0170000000000001e-06 2024-09-04T17:48:32.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:32.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641858+0000", 2024-09-04T17:48:32.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1499999999999999e-06 2024-09-04T17:48:32.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:32.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641864+0000", 2024-09-04T17:48:32.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2280000000000003e-06 2024-09-04T17:48:32.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:32.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.669278+0000", 2024-09-04T17:48:32.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027413843 2024-09-04T17:48:32.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:32.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.669392+0000", 2024-09-04T17:48:32.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000114654 2024-09-04T17:48:32.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:32.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.669603+0000", 2024-09-04T17:48:32.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021096799999999999 2024-09-04T17:48:32.032 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:32.032 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:32.032 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:32.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.032 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94664 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 7864320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:32.032 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.644474+0000", 2024-09-04T17:48:32.032 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.299438690000002, 2024-09-04T17:48:32.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026965042000000002, 2024-09-04T17:48:32.032 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:32.032 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:32.032 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:32.033 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:32.033 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:32.033 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:32.033 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94664 2024-09-04T17:48:32.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.033 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:32.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:32.033 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644474+0000", 2024-09-04T17:48:32.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:32.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:32.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644474+0000", 2024-09-04T17:48:32.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:32.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:32.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644474+0000", 2024-09-04T17:48:32.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.16e-07 2024-09-04T17:48:32.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:32.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644478+0000", 2024-09-04T17:48:32.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1389999999999999e-06 2024-09-04T17:48:32.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:32.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644494+0000", 2024-09-04T17:48:32.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6977e-05 2024-09-04T17:48:32.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:32.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644501+0000", 2024-09-04T17:48:32.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.438e-06 2024-09-04T17:48:32.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:32.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.671241+0000", 2024-09-04T17:48:32.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026740525000000001 2024-09-04T17:48:32.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:32.037 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.671325+0000", 2024-09-04T17:48:32.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3987999999999996e-05 2024-09-04T17:48:32.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.037 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:32.037 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.671439+0000", 2024-09-04T17:48:32.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011315900000000001 2024-09-04T17:48:32.037 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:32.037 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:32.037 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:32.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.038 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94665 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 7995392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:32.038 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.644511+0000", 2024-09-04T17:48:32.038 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.299401658000001, 2024-09-04T17:48:32.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027156390999999998, 2024-09-04T17:48:32.038 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:32.038 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:32.038 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:32.038 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:32.038 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:32.038 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:32.038 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94665 2024-09-04T17:48:32.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.039 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:32.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:32.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644511+0000", 2024-09-04T17:48:32.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:32.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:32.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644511+0000", 2024-09-04T17:48:32.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:32.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:32.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644511+0000", 2024-09-04T17:48:32.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2699999999999999e-07 2024-09-04T17:48:32.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:32.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644514+0000", 2024-09-04T17:48:32.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9890000000000001e-06 2024-09-04T17:48:32.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:32.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644521+0000", 2024-09-04T17:48:32.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5409999999999997e-06 2024-09-04T17:48:32.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:32.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644527+0000", 2024-09-04T17:48:32.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0340000000000002e-06 2024-09-04T17:48:32.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:32.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.671474+0000", 2024-09-04T17:48:32.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.02694767 2024-09-04T17:48:32.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:32.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.671529+0000", 2024-09-04T17:48:32.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4373000000000003e-05 2024-09-04T17:48:32.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:32.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.671667+0000", 2024-09-04T17:48:32.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000138257 2024-09-04T17:48:32.043 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:32.043 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:32.043 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:32.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.043 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94666 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 8126464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:32.043 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.644536+0000", 2024-09-04T17:48:32.043 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.299376520000003, 2024-09-04T17:48:32.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027360524000000001, 2024-09-04T17:48:32.043 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:32.044 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:32.044 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:32.044 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:32.044 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:32.044 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:32.044 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94666 2024-09-04T17:48:32.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.044 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:32.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:32.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644536+0000", 2024-09-04T17:48:32.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:32.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:32.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644536+0000", 2024-09-04T17:48:32.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:32.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:32.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644536+0000", 2024-09-04T17:48:32.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7000000000000004e-07 2024-09-04T17:48:32.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:32.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644540+0000", 2024-09-04T17:48:32.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1609999999999999e-06 2024-09-04T17:48:32.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:32.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644552+0000", 2024-09-04T17:48:32.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2794e-05 2024-09-04T17:48:32.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:32.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644559+0000", 2024-09-04T17:48:32.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5010000000000001e-06 2024-09-04T17:48:32.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:32.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.671703+0000", 2024-09-04T17:48:32.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027144492999999999 2024-09-04T17:48:32.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:32.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.671779+0000", 2024-09-04T17:48:32.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5578e-05 2024-09-04T17:48:32.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:32.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.671896+0000", 2024-09-04T17:48:32.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000117327 2024-09-04T17:48:32.048 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:32.048 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:32.048 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:32.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.048 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94667 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 8257536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:32.048 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.644570+0000", 2024-09-04T17:48:32.049 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.299341855999998, 2024-09-04T17:48:32.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027556085000000001, 2024-09-04T17:48:32.049 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:32.049 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:32.049 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:32.049 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:32.050 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:32.050 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:32.050 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94667 2024-09-04T17:48:32.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.050 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:32.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:32.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644570+0000", 2024-09-04T17:48:32.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:32.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:32.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644570+0000", 2024-09-04T17:48:32.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:32.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:32.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644571+0000", 2024-09-04T17:48:32.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5199999999999997e-07 2024-09-04T17:48:32.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:32.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644574+0000", 2024-09-04T17:48:32.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3340000000000002e-06 2024-09-04T17:48:32.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:32.054 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644581+0000", 2024-09-04T17:48:32.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6579999999999996e-06 2024-09-04T17:48:32.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:32.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644589+0000", 2024-09-04T17:48:32.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9699999999999999e-06 2024-09-04T17:48:32.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:32.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.671925+0000", 2024-09-04T17:48:32.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027335911000000001 2024-09-04T17:48:32.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:32.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.671987+0000", 2024-09-04T17:48:32.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2198000000000003e-05 2024-09-04T17:48:32.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:32.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.672126+0000", 2024-09-04T17:48:32.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000139462 2024-09-04T17:48:32.057 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:32.057 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:32.057 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:32.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.058 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:96164 3.18 3:19aa90a8:::rbd_data.1134c4e2d26.0000000000000062:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123)", 2024-09-04T17:48:32.058 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:48:02.862015+0000", 2024-09-04T17:48:32.058 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 29.081897334000001, 2024-09-04T17:48:32.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034031039999999999, 2024-09-04T17:48:32.058 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:32.058 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:32.058 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:32.058 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:32.059 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:32.059 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:32.059 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 96164 2024-09-04T17:48:32.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.059 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:32.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:32.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.862015+0000", 2024-09-04T17:48:32.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:32.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.060 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:32.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.862015+0000", 2024-09-04T17:48:32.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:32.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.060 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:32.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.862018+0000", 2024-09-04T17:48:32.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1200000000000002e-06 2024-09-04T17:48:32.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:32.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.862032+0000", 2024-09-04T17:48:32.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4083000000000001e-05 2024-09-04T17:48:32.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:32.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.862064+0000", 2024-09-04T17:48:32.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2095999999999999e-05 2024-09-04T17:48:32.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:32.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.862081+0000", 2024-09-04T17:48:32.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6625000000000001e-05 2024-09-04T17:48:32.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:32.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.862232+0000", 2024-09-04T17:48:32.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015144600000000001 2024-09-04T17:48:32.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:32.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.862588+0000", 2024-09-04T17:48:32.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00035525999999999998 2024-09-04T17:48:32.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 7", 2024-09-04T17:48:32.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.867988+0000", 2024-09-04T17:48:32.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054005479999999998 2024-09-04T17:48:32.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:32.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.895763+0000", 2024-09-04T17:48:32.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027774983 2024-09-04T17:48:32.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:32.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.895877+0000", 2024-09-04T17:48:32.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000113607 2024-09-04T17:48:32.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:32.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.895975+0000", 2024-09-04T17:48:32.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7876999999999998e-05 2024-09-04T17:48:32.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:32.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:32.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:32.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.896046+0000", 2024-09-04T17:48:32.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1395000000000003e-05 2024-09-04T17:48:32.067 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:32.067 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:32.067 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:32.067 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:32.067 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:32.067 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:32.070 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-04T17:48:32.070 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is set 2024-09-04T17:48:32.070 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:48:32.063+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:48:32.170 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:32.170+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:32.206 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:32.206 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:48:32.206 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:48:32.206 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:32.220 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-04T17:48:32.309 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:48:32.306+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:48:32.372 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:48:32.370+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:48:32.382 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:32.383 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:48:32.383 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:48:32.383 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:48:32.383 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:32.397 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-04T17:48:32.485 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:48:32.486+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:48:32.570 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:32.570 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:48:32.571 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:48:32.571 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:48:32.571 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.571 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "pg_update_log_missing(3.1f epoch 119/118 rep_tid 2838 entries 119'424 (0'0) error 3:f95bc674:::rbd_header.1134878eadad:head by client.4404.0:93468 0.000000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-04T17:48:32.571 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:58.898727+0000", 2024-09-04T17:48:32.571 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 33.669354507999998, 2024-09-04T17:48:32.571 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0072113999999999998, 2024-09-04T17:48:32.571 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:32.571 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:32.571 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:32.571 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:32.571 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.571 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:32.572 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.898727+0000", 2024-09-04T17:48:32.572 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:32.572 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.572 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.572 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:32.572 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.898727+0000", 2024-09-04T17:48:32.572 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:32.572 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.572 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.572 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:32.572 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.898728+0000", 2024-09-04T17:48:32.572 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.8200000000000001e-07 2024-09-04T17:48:32.573 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.573 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.573 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:32.573 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.898740+0000", 2024-09-04T17:48:32.573 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2186e-05 2024-09-04T17:48:32.573 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.573 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.573 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:32.573 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.898772+0000", 2024-09-04T17:48:32.573 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.2156000000000002e-05 2024-09-04T17:48:32.573 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.573 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.573 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:32.573 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.898796+0000", 2024-09-04T17:48:32.574 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4017999999999999e-05 2024-09-04T17:48:32.574 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.574 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.574 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:32.574 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.898966+0000", 2024-09-04T17:48:32.574 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000170185 2024-09-04T17:48:32.574 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.574 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.574 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:32.574 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.905939+0000", 2024-09-04T17:48:32.574 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0069724729999999999 2024-09-04T17:48:32.574 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:32.574 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:32.574 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:32.575 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.575 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.575 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:93469 3.1f e119/118 3:f95bc674:::rbd_header.1134878eadad:head v 119'425, mlcod=0'0)", 2024-09-04T17:48:32.575 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:58.900173+0000", 2024-09-04T17:48:32.575 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 33.66790872, 2024-09-04T17:48:32.575 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0059053930000000001, 2024-09-04T17:48:32.575 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:32.575 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:32.575 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:32.576 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:32.576 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.576 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:32.576 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.900173+0000", 2024-09-04T17:48:32.576 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:32.576 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.577 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.577 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:32.577 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.900173+0000", 2024-09-04T17:48:32.577 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:32.577 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.577 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.577 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:32.577 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.900174+0000", 2024-09-04T17:48:32.577 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.127e-06 2024-09-04T17:48:32.577 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.578 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.578 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:32.578 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.900181+0000", 2024-09-04T17:48:32.578 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.0099999999999998e-06 2024-09-04T17:48:32.578 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.578 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.578 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:32.578 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.900199+0000", 2024-09-04T17:48:32.578 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7876e-05 2024-09-04T17:48:32.578 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.578 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.578 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:32.579 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.900208+0000", 2024-09-04T17:48:32.579 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.4999999999999999e-06 2024-09-04T17:48:32.579 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.579 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.579 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:32.579 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.900310+0000", 2024-09-04T17:48:32.579 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00010202800000000001 2024-09-04T17:48:32.579 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.579 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.579 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:32.579 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.900353+0000", 2024-09-04T17:48:32.579 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.3213000000000001e-05 2024-09-04T17:48:32.579 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.579 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.580 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:32.580 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.905944+0000", 2024-09-04T17:48:32.580 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0055914900000000002 2024-09-04T17:48:32.580 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.580 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.580 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:32.580 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.906078+0000", 2024-09-04T17:48:32.580 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013414899999999999 2024-09-04T17:48:32.580 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:32.580 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:32.580 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:32.580 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.581 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.581 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "MOSDPGPush(3.19 119/118 [PushOp(3:9db25b11:::rbd_data.11341db94a06.0000000000000001:head, version: 78'254, data_included: [0~294912,458752~32768,1802240~65536,1966080~98304,3014656~294912], data_size: 786432, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:9db25b11:::rbd_data.11341db94a06.0000000000000001:head@78'254, size: 4096000, copy_subset: [(0, 4096000)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096000, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-04T17:48:32.581 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:59.229007+0000", 2024-09-04T17:48:32.581 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 33.339075225999999, 2024-09-04T17:48:32.581 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0049484100000000003, 2024-09-04T17:48:32.581 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:32.581 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:32.581 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:32.581 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:32.581 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.581 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:32.581 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.229007+0000", 2024-09-04T17:48:32.581 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:32.581 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.582 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.582 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:32.582 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.229007+0000", 2024-09-04T17:48:32.582 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:32.582 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.582 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.582 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:32.582 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.229008+0000", 2024-09-04T17:48:32.582 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.827e-06 2024-09-04T17:48:32.582 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.582 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.582 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:32.582 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.230334+0000", 2024-09-04T17:48:32.583 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0013259089999999999 2024-09-04T17:48:32.583 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.583 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.583 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:32.583 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.230382+0000", 2024-09-04T17:48:32.583 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.8168999999999999e-05 2024-09-04T17:48:32.583 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.583 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.583 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:32.583 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.230415+0000", 2024-09-04T17:48:32.583 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.2916000000000001e-05 2024-09-04T17:48:32.583 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.583 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.583 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:32.584 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.230676+0000", 2024-09-04T17:48:32.584 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00026107600000000001 2024-09-04T17:48:32.584 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.584 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.584 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:32.584 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.230719+0000", 2024-09-04T17:48:32.584 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2571999999999998e-05 2024-09-04T17:48:32.584 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.584 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.584 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:32.584 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.233955+0000", 2024-09-04T17:48:32.584 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.003235941 2024-09-04T17:48:32.584 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:32.585 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:32.585 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:32.585 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.585 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.585 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "MOSDPGPush(3.10 119/118 [PushOp(3:0c2d06e5:::rbd_data.113410ddd206.0000000000000000:18, version: 68'308, data_included: [0~884736,950272~65536,1081344~65536,1212416~32768], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:0c2d06e5:::rbd_data.113410ddd206.0000000000000000:18@68'308, size: 7471104, copy_subset: [(0, 7471104)], clone_subset: {}, snapset: 18=[]:{18: [18]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1245184, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-04T17:48:32.585 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:59.426517+0000", 2024-09-04T17:48:32.585 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 33.141564641999999, 2024-09-04T17:48:32.585 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0064985670000000002, 2024-09-04T17:48:32.585 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:32.585 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:32.585 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:32.585 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:32.586 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.586 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:32.586 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.426517+0000", 2024-09-04T17:48:32.586 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:32.586 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.586 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.586 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:32.586 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.426517+0000", 2024-09-04T17:48:32.586 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:32.586 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.586 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.586 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:32.586 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.426754+0000", 2024-09-04T17:48:32.586 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00023681799999999999 2024-09-04T17:48:32.587 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.587 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.587 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:32.587 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.429886+0000", 2024-09-04T17:48:32.587 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0031323789999999998 2024-09-04T17:48:32.587 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.587 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.587 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:32.587 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.429940+0000", 2024-09-04T17:48:32.587 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4104999999999998e-05 2024-09-04T17:48:32.587 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.587 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.587 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:32.588 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.429975+0000", 2024-09-04T17:48:32.588 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4289000000000003e-05 2024-09-04T17:48:32.588 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.588 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.588 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:32.588 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.430229+0000", 2024-09-04T17:48:32.588 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00025465900000000002 2024-09-04T17:48:32.588 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.588 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.588 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:32.588 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.430350+0000", 2024-09-04T17:48:32.588 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000120992 2024-09-04T17:48:32.588 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.589 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.589 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:32.589 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.433016+0000", 2024-09-04T17:48:32.589 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0026653250000000001 2024-09-04T17:48:32.589 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:32.589 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:32.589 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:32.589 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.589 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.589 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "MOSDPGPush(3.10 119/118 [PushOp(3:0c2d06e5:::rbd_data.113410ddd206.0000000000000000:18, version: 68'308, data_included: [1245184~32768,1343488~65536,1474560~65536,1605632~65536,1736704~65536,1867776~65536,1998848~65536,2129920~65536,2260992~65536,2392064~65536,2523136~65536,2654208~65536,2785280~65536,2916352~65536,3047424~65536,3154405~89627,3309568~8677], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:0c2d06e5:::rbd_data.113410ddd206.0000000000000000:18@68'308, size: 7471104, copy_subset: [(0, 7471104)], clone_subset: {}, snapset: 18=[]:{18: [18]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3318245, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1245184, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-04T17:48:32.589 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:59.434957+0000", 2024-09-04T17:48:32.589 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 33.133125018999998, 2024-09-04T17:48:32.589 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0061603969999999997, 2024-09-04T17:48:32.589 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:32.590 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:32.590 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:32.590 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:32.590 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.590 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:32.590 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.434957+0000", 2024-09-04T17:48:32.590 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:32.590 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.590 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.590 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:32.590 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.434957+0000", 2024-09-04T17:48:32.590 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:32.590 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.591 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.591 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:32.591 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.434958+0000", 2024-09-04T17:48:32.591 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4339999999999999e-06 2024-09-04T17:48:32.591 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.591 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.591 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:32.591 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.436921+0000", 2024-09-04T17:48:32.591 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0019629460000000001 2024-09-04T17:48:32.591 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.591 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.591 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:32.591 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.436973+0000", 2024-09-04T17:48:32.591 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1492000000000001e-05 2024-09-04T17:48:32.592 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.592 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.592 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:32.592 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.437041+0000", 2024-09-04T17:48:32.592 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.8275000000000006e-05 2024-09-04T17:48:32.592 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.592 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.592 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:32.592 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.437312+0000", 2024-09-04T17:48:32.592 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00027156800000000003 2024-09-04T17:48:32.592 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.592 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.592 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:32.593 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.437357+0000", 2024-09-04T17:48:32.593 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4079000000000003e-05 2024-09-04T17:48:32.593 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.593 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.593 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:32.593 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.441117+0000", 2024-09-04T17:48:32.593 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0037606029999999999 2024-09-04T17:48:32.593 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:32.593 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:32.593 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:32.593 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.593 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.593 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "MOSDPGPush(3.10 119/118 [PushOp(3:08a6a427:::rbd_data.11341db94a06.0000000000000002:head, version: 78'342, data_included: [0~271649,458752~32768,1802240~65536,1966080~98304,3014656~294912], data_size: 763169, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:08a6a427:::rbd_data.11341db94a06.0000000000000002:head@78'342, size: 4096000, copy_subset: [(0, 4096000)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096000, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-04T17:48:32.594 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:59.471247+0000", 2024-09-04T17:48:32.594 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 33.096834774999998, 2024-09-04T17:48:32.594 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0048034530000000001, 2024-09-04T17:48:32.594 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:32.594 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:32.594 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:32.594 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:32.594 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.594 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:32.594 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.471247+0000", 2024-09-04T17:48:32.594 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:32.594 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.594 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.594 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:32.595 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.471247+0000", 2024-09-04T17:48:32.595 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:32.595 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.595 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.595 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:32.595 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.471249+0000", 2024-09-04T17:48:32.595 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9e-06 2024-09-04T17:48:32.595 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.595 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.595 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:32.596 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.472560+0000", 2024-09-04T17:48:32.596 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0013114190000000001 2024-09-04T17:48:32.596 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.596 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.596 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:32.596 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.472610+0000", 2024-09-04T17:48:32.596 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9916000000000001e-05 2024-09-04T17:48:32.596 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.596 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.596 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:32.596 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.472692+0000", 2024-09-04T17:48:32.596 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.1384000000000005e-05 2024-09-04T17:48:32.597 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.597 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.597 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:32.597 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.472912+0000", 2024-09-04T17:48:32.597 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000220201 2024-09-04T17:48:32.597 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.597 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.597 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:32.597 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.472983+0000", 2024-09-04T17:48:32.597 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1137999999999996e-05 2024-09-04T17:48:32.597 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.597 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.597 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:32.598 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.476050+0000", 2024-09-04T17:48:32.598 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0030674949999999999 2024-09-04T17:48:32.598 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:32.598 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:32.598 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:32.598 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.598 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.598 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:96178 3.1e 3:7c9d4526:::rbd_data.1134c4e2d26.0000000000000026:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123)", 2024-09-04T17:48:32.598 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:02.985763+0000", 2024-09-04T17:48:32.598 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 29.582318870999998, 2024-09-04T17:48:32.598 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.020686758999999999, 2024-09-04T17:48:32.598 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:32.598 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:32.598 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:32.599 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:32.599 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:32.599 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:32.599 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 96178 2024-09-04T17:48:32.599 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.599 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:32.599 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.599 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:32.599 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:02.985763+0000", 2024-09-04T17:48:32.599 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:32.599 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.599 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.599 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:32.600 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:02.985763+0000", 2024-09-04T17:48:32.600 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:32.600 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.600 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.600 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:32.600 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:02.985766+0000", 2024-09-04T17:48:32.600 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.2990000000000001e-06 2024-09-04T17:48:32.600 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.600 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.600 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:32.600 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:02.985783+0000", 2024-09-04T17:48:32.600 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.645e-05 2024-09-04T17:48:32.600 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.601 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.601 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:32.601 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:02.985818+0000", 2024-09-04T17:48:32.601 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.5459000000000001e-05 2024-09-04T17:48:32.601 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.601 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.601 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:32.601 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:02.985835+0000", 2024-09-04T17:48:32.601 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.715e-05 2024-09-04T17:48:32.601 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.601 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.601 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:32.601 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:02.985953+0000", 2024-09-04T17:48:32.602 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000117351 2024-09-04T17:48:32.602 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.602 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.602 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:32.602 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:02.986207+0000", 2024-09-04T17:48:32.602 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00025483 2024-09-04T17:48:32.602 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.602 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.602 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 2", 2024-09-04T17:48:32.602 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:02.990784+0000", 2024-09-04T17:48:32.602 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0045763219999999999 2024-09-04T17:48:32.602 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.602 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.603 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:48:32.603 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.006139+0000", 2024-09-04T17:48:32.603 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.015355526 2024-09-04T17:48:32.603 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.603 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.603 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:32.603 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.006225+0000", 2024-09-04T17:48:32.603 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.6003999999999999e-05 2024-09-04T17:48:32.603 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.603 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.603 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:32.603 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.006371+0000", 2024-09-04T17:48:32.603 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000146052 2024-09-04T17:48:32.603 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.604 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.604 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:32.604 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.006450+0000", 2024-09-04T17:48:32.604 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.8316e-05 2024-09-04T17:48:32.604 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:32.604 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:32.604 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:32.604 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.604 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.604 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:96197 3.10 3:08a64171:::rbd_data.1134c4e2d26.00000000000000c3:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123)", 2024-09-04T17:48:32.604 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:03.049064+0000", 2024-09-04T17:48:32.604 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 29.519017900000001, 2024-09-04T17:48:32.604 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0053841139999999997, 2024-09-04T17:48:32.605 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:32.605 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:32.605 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:32.605 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:32.605 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:32.605 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:32.605 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 96197 2024-09-04T17:48:32.605 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.605 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:32.605 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.606 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:32.606 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.049064+0000", 2024-09-04T17:48:32.606 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:32.606 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.606 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.606 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:32.606 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.049064+0000", 2024-09-04T17:48:32.606 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:32.606 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.606 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.606 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:32.606 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.049066+0000", 2024-09-04T17:48:32.606 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.3360000000000002e-06 2024-09-04T17:48:32.606 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.607 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.607 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:32.607 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.049075+0000", 2024-09-04T17:48:32.607 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.2210000000000003e-06 2024-09-04T17:48:32.607 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.607 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.607 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:32.607 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.049097+0000", 2024-09-04T17:48:32.607 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1798e-05 2024-09-04T17:48:32.607 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.607 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.607 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:32.607 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.049107+0000", 2024-09-04T17:48:32.607 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0200999999999999e-05 2024-09-04T17:48:32.608 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.608 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.608 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:32.608 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.049183+0000", 2024-09-04T17:48:32.608 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.6079000000000001e-05 2024-09-04T17:48:32.608 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.608 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.608 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:32.608 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.049276+0000", 2024-09-04T17:48:32.608 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.2911000000000005e-05 2024-09-04T17:48:32.608 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.608 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.608 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 0", 2024-09-04T17:48:32.608 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.050313+0000", 2024-09-04T17:48:32.609 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.001036705 2024-09-04T17:48:32.609 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.609 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.609 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:48:32.609 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.054220+0000", 2024-09-04T17:48:32.609 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.003906779 2024-09-04T17:48:32.609 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.609 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.609 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:32.609 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.054297+0000", 2024-09-04T17:48:32.609 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.6816000000000005e-05 2024-09-04T17:48:32.609 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.609 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.610 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:32.610 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.054374+0000", 2024-09-04T17:48:32.610 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.7490000000000005e-05 2024-09-04T17:48:32.610 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.610 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.610 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:32.610 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.054448+0000", 2024-09-04T17:48:32.610 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3777999999999997e-05 2024-09-04T17:48:32.610 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:32.610 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:32.610 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:32.610 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.611 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.611 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:96206 3.10 3:08a64171:::rbd_data.1134c4e2d26.00000000000000c3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123)", 2024-09-04T17:48:32.611 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:03.092242+0000", 2024-09-04T17:48:32.611 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 29.475839281999999, 2024-09-04T17:48:32.611 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0045034109999999997, 2024-09-04T17:48:32.611 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:32.611 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:32.611 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:32.611 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:32.611 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:32.611 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:32.611 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 96206 2024-09-04T17:48:32.611 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.612 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:32.612 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.612 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:32.612 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.092242+0000", 2024-09-04T17:48:32.612 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:32.612 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.612 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.612 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:32.612 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.092242+0000", 2024-09-04T17:48:32.612 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:32.612 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.612 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.612 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:32.612 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.092245+0000", 2024-09-04T17:48:32.613 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.356e-06 2024-09-04T17:48:32.613 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.613 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.613 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:32.613 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.092253+0000", 2024-09-04T17:48:32.613 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.0199999999999994e-06 2024-09-04T17:48:32.613 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.613 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.613 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:32.613 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.092276+0000", 2024-09-04T17:48:32.613 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.3635999999999999e-05 2024-09-04T17:48:32.613 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.613 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.614 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:32.614 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.092291+0000", 2024-09-04T17:48:32.614 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4901e-05 2024-09-04T17:48:32.614 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.614 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.614 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:32.614 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.092433+0000", 2024-09-04T17:48:32.614 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014111500000000001 2024-09-04T17:48:32.614 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.614 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.614 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:32.614 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.092645+0000", 2024-09-04T17:48:32.614 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021203500000000001 2024-09-04T17:48:32.614 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.615 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.615 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 0", 2024-09-04T17:48:32.615 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.092917+0000", 2024-09-04T17:48:32.615 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00027224299999999998 2024-09-04T17:48:32.615 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.615 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.615 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:48:32.615 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.094780+0000", 2024-09-04T17:48:32.615 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0018633129999999999 2024-09-04T17:48:32.615 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.615 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.615 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:32.615 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.096439+0000", 2024-09-04T17:48:32.616 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0016588530000000001 2024-09-04T17:48:32.616 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.616 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.616 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:32.616 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.096631+0000", 2024-09-04T17:48:32.616 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000191905 2024-09-04T17:48:32.616 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.616 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.616 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:32.616 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.096746+0000", 2024-09-04T17:48:32.616 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000115034 2024-09-04T17:48:32.616 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:32.616 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:32.617 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:32.617 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.617 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.617 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:96267 3.1e 3:7a1dfe29:::rbd_header.1134c4e2d26:head [watch unwatch cookie 140018753450144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123)", 2024-09-04T17:48:32.617 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:03.482624+0000", 2024-09-04T17:48:32.617 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 29.085457568999999, 2024-09-04T17:48:32.617 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.006866853, 2024-09-04T17:48:32.617 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:32.617 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:32.617 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:32.617 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:32.617 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:32.617 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:32.618 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 96267 2024-09-04T17:48:32.618 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.618 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:32.618 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.618 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:32.618 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.482624+0000", 2024-09-04T17:48:32.618 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:32.618 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.618 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.618 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:32.618 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.482624+0000", 2024-09-04T17:48:32.618 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:32.618 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.619 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.619 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:32.619 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.482626+0000", 2024-09-04T17:48:32.619 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8360000000000001e-06 2024-09-04T17:48:32.619 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.619 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.619 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:32.619 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.482635+0000", 2024-09-04T17:48:32.619 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9099999999999994e-06 2024-09-04T17:48:32.619 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.619 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.619 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:32.619 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.482658+0000", 2024-09-04T17:48:32.620 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.3186000000000001e-05 2024-09-04T17:48:32.620 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.620 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.620 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:32.620 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.482677+0000", 2024-09-04T17:48:32.620 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8396000000000001e-05 2024-09-04T17:48:32.620 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.620 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.620 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:32.620 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.482849+0000", 2024-09-04T17:48:32.620 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000172936 2024-09-04T17:48:32.620 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.620 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.621 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:32.621 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.482987+0000", 2024-09-04T17:48:32.621 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013707699999999999 2024-09-04T17:48:32.621 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.621 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.621 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 2", 2024-09-04T17:48:32.621 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.483222+0000", 2024-09-04T17:48:32.621 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00023545 2024-09-04T17:48:32.621 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.621 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.621 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:48:32.621 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.484356+0000", 2024-09-04T17:48:32.621 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.001134049 2024-09-04T17:48:32.622 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.622 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.622 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:32.622 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.489303+0000", 2024-09-04T17:48:32.622 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0049472910000000004 2024-09-04T17:48:32.622 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.622 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.622 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:32.622 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.489396+0000", 2024-09-04T17:48:32.622 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.2638000000000003e-05 2024-09-04T17:48:32.622 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.622 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.622 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:32.622 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.489491+0000", 2024-09-04T17:48:32.623 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5084e-05 2024-09-04T17:48:32.623 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:32.623 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:32.623 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:32.623 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.623 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.623 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:96360 3.1e 3:7c9d4526:::rbd_data.1134c4e2d26.0000000000000026:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123)", 2024-09-04T17:48:32.623 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:03.603245+0000", 2024-09-04T17:48:32.623 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 28.964836465000001, 2024-09-04T17:48:32.623 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0074703160000000003, 2024-09-04T17:48:32.623 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:32.623 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:32.623 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:32.624 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:32.624 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:32.624 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:32.624 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 96360 2024-09-04T17:48:32.624 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.624 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:32.624 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.624 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:32.624 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.603245+0000", 2024-09-04T17:48:32.624 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:32.624 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.624 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.625 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:32.625 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.603245+0000", 2024-09-04T17:48:32.625 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:32.625 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.625 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.625 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:32.625 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.603248+0000", 2024-09-04T17:48:32.625 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.277e-06 2024-09-04T17:48:32.625 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.625 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.625 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:32.625 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.603259+0000", 2024-09-04T17:48:32.626 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1083e-05 2024-09-04T17:48:32.626 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.626 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.626 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:32.626 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.603295+0000", 2024-09-04T17:48:32.626 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.5892999999999997e-05 2024-09-04T17:48:32.626 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.626 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.626 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:32.626 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.603315+0000", 2024-09-04T17:48:32.626 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0953000000000001e-05 2024-09-04T17:48:32.626 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.626 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.626 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:32.627 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.603486+0000", 2024-09-04T17:48:32.627 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00017011199999999999 2024-09-04T17:48:32.627 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.627 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.627 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:32.627 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.604960+0000", 2024-09-04T17:48:32.627 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0014743340000000001 2024-09-04T17:48:32.627 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.627 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.627 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 2", 2024-09-04T17:48:32.627 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.605232+0000", 2024-09-04T17:48:32.627 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00027236300000000002 2024-09-04T17:48:32.627 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.628 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.628 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:48:32.628 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.610322+0000", 2024-09-04T17:48:32.628 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0050901849999999997 2024-09-04T17:48:32.628 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.628 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.628 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:32.628 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.610464+0000", 2024-09-04T17:48:32.628 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014186599999999999 2024-09-04T17:48:32.628 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.628 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.628 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:32.628 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.610607+0000", 2024-09-04T17:48:32.629 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000142423 2024-09-04T17:48:32.629 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.629 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.629 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:32.629 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.610716+0000", 2024-09-04T17:48:32.629 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00010882699999999999 2024-09-04T17:48:32.629 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:32.629 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:32.629 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:32.629 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.629 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.629 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:96520 3.2 e124/121 3:4549cdb4:::rbd_data.1134878eadad.0000000000000001:head v 124'509, mlcod=123'507)", 2024-09-04T17:48:32.629 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:04.014285+0000", 2024-09-04T17:48:32.630 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 28.553796546000001, 2024-09-04T17:48:32.630 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0066183220000000003, 2024-09-04T17:48:32.630 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:32.630 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:32.630 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:32.630 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:32.630 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.630 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:32.630 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:04.014285+0000", 2024-09-04T17:48:32.630 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:32.630 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.630 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.630 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:32.631 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:04.014285+0000", 2024-09-04T17:48:32.631 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:32.631 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.631 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.631 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:32.631 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:04.014287+0000", 2024-09-04T17:48:32.631 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1959999999999998e-06 2024-09-04T17:48:32.631 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.631 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.631 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:32.631 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:04.014332+0000", 2024-09-04T17:48:32.631 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4196000000000002e-05 2024-09-04T17:48:32.631 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.632 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.632 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:32.632 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:04.014374+0000", 2024-09-04T17:48:32.632 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2072e-05 2024-09-04T17:48:32.632 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.632 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.632 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:32.632 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:04.014394+0000", 2024-09-04T17:48:32.632 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0146999999999999e-05 2024-09-04T17:48:32.632 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.632 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.632 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:32.632 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:04.014575+0000", 2024-09-04T17:48:32.633 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000181132 2024-09-04T17:48:32.633 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.633 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.633 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:32.633 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:04.014611+0000", 2024-09-04T17:48:32.633 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.5815999999999997e-05 2024-09-04T17:48:32.633 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.633 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.633 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:32.633 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:04.020809+0000", 2024-09-04T17:48:32.633 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0061983159999999997 2024-09-04T17:48:32.633 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.633 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.634 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:32.634 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:04.020904+0000", 2024-09-04T17:48:32.634 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4446999999999999e-05 2024-09-04T17:48:32.634 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:32.634 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:32.634 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:32.634 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.634 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.634 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:97545 3.2 e134/121 3:4728633c:::rbd_data.1134579f27f7.0000000000000003:head v 134'551, mlcod=134'549)", 2024-09-04T17:48:32.634 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:14.061346+0000", 2024-09-04T17:48:32.634 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 18.506735261999999, 2024-09-04T17:48:32.634 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0045243619999999997, 2024-09-04T17:48:32.634 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:32.634 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:32.634 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:32.635 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:32.635 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.635 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:32.635 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:14.061346+0000", 2024-09-04T17:48:32.635 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:32.635 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.635 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.635 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:32.635 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:14.061346+0000", 2024-09-04T17:48:32.635 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:32.635 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.635 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.635 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:32.635 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:14.061348+0000", 2024-09-04T17:48:32.636 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9539999999999998e-06 2024-09-04T17:48:32.636 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.636 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.636 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:32.636 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:14.061362+0000", 2024-09-04T17:48:32.636 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.307e-05 2024-09-04T17:48:32.636 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.636 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.636 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:32.636 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:14.061399+0000", 2024-09-04T17:48:32.636 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7725000000000002e-05 2024-09-04T17:48:32.636 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.636 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.636 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:32.636 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:14.061415+0000", 2024-09-04T17:48:32.637 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5333999999999999e-05 2024-09-04T17:48:32.637 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.637 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.637 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:32.637 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:14.061534+0000", 2024-09-04T17:48:32.637 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000119494 2024-09-04T17:48:32.637 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.637 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.637 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:32.637 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:14.061571+0000", 2024-09-04T17:48:32.637 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.6943000000000002e-05 2024-09-04T17:48:32.637 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.638 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.638 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:32.638 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:14.065810+0000", 2024-09-04T17:48:32.638 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0042386120000000001 2024-09-04T17:48:32.638 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.638 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.638 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:32.638 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:14.065871+0000", 2024-09-04T17:48:32.638 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.1229999999999995e-05 2024-09-04T17:48:32.638 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:32.638 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:32.638 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:32.638 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.638 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.639 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(osd.2.0:369 3.0 e138/121 3:04fb2888:::rbd_data.11342e6808f0.0000000000000009:24 v 138'380, mlcod=135'377)", 2024-09-04T17:48:32.639 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:17.851779+0000", 2024-09-04T17:48:32.639 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 14.716302634, 2024-09-04T17:48:32.639 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0056329400000000003, 2024-09-04T17:48:32.639 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:32.639 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:32.639 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:32.639 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:32.639 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.639 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:32.639 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.851779+0000", 2024-09-04T17:48:32.639 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:32.639 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.640 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.640 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:32.640 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.851779+0000", 2024-09-04T17:48:32.640 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:32.640 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.640 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.640 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:32.640 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.851780+0000", 2024-09-04T17:48:32.640 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2399999999999998e-07 2024-09-04T17:48:32.640 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.640 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.640 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:32.640 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.851790+0000", 2024-09-04T17:48:32.641 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0399e-05 2024-09-04T17:48:32.641 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.641 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.641 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:32.641 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.851809+0000", 2024-09-04T17:48:32.641 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9228e-05 2024-09-04T17:48:32.641 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.641 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.641 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:32.641 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.851820+0000", 2024-09-04T17:48:32.641 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0814e-05 2024-09-04T17:48:32.641 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.642 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.642 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:32.642 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.851902+0000", 2024-09-04T17:48:32.642 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.2154e-05 2024-09-04T17:48:32.642 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.642 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.642 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:32.642 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.851930+0000", 2024-09-04T17:48:32.642 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.7577999999999999e-05 2024-09-04T17:48:32.642 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.642 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.642 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:32.642 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.857363+0000", 2024-09-04T17:48:32.642 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0054331989999999997 2024-09-04T17:48:32.643 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.643 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.643 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:32.643 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.857412+0000", 2024-09-04T17:48:32.643 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9044e-05 2024-09-04T17:48:32.643 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:32.643 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:32.643 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:32.643 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.643 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.643 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(osd.5.0:3167 3.2 e138/121 3:460b9dee:::rbd_data.11342e6808f0.000000000000000d:24 v 138'560, mlcod=135'557)", 2024-09-04T17:48:32.643 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:17.857279+0000", 2024-09-04T17:48:32.643 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 14.710803049999999, 2024-09-04T17:48:32.644 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0068482229999999996, 2024-09-04T17:48:32.644 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:32.644 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:32.644 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:32.644 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:32.644 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.644 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:32.644 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.857279+0000", 2024-09-04T17:48:32.644 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:32.644 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.644 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.644 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:32.644 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.857279+0000", 2024-09-04T17:48:32.644 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:32.645 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.645 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.645 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:32.645 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.857279+0000", 2024-09-04T17:48:32.645 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7300000000000001e-07 2024-09-04T17:48:32.645 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.645 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.645 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:32.645 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.857286+0000", 2024-09-04T17:48:32.645 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.4289999999999998e-06 2024-09-04T17:48:32.645 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.645 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.645 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:32.645 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.857300+0000", 2024-09-04T17:48:32.646 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4582e-05 2024-09-04T17:48:32.646 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.646 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.646 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:32.646 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.857315+0000", 2024-09-04T17:48:32.646 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5173e-05 2024-09-04T17:48:32.646 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.646 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.646 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:32.646 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.857388+0000", 2024-09-04T17:48:32.646 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2365000000000003e-05 2024-09-04T17:48:32.646 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.646 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.646 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:32.647 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.857414+0000", 2024-09-04T17:48:32.647 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6248999999999999e-05 2024-09-04T17:48:32.647 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.647 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.647 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:32.647 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.864076+0000", 2024-09-04T17:48:32.647 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0066619490000000003 2024-09-04T17:48:32.647 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.647 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.647 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:32.647 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.864127+0000", 2024-09-04T17:48:32.647 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1002999999999999e-05 2024-09-04T17:48:32.647 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:32.647 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:32.648 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:32.648 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.648 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.648 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:98345 3.a 3:55a1c81d:::rbd_header.113481c24fbf:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139)", 2024-09-04T17:48:32.648 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:18.971074+0000", 2024-09-04T17:48:32.648 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 13.597007770999999, 2024-09-04T17:48:32.648 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.017562632000000002, 2024-09-04T17:48:32.648 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:32.648 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:32.648 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:32.648 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:32.648 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:32.648 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:32.648 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 98345 2024-09-04T17:48:32.649 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.649 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:32.649 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.649 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:32.649 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.971074+0000", 2024-09-04T17:48:32.649 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:32.649 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.649 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.649 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:32.649 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.971074+0000", 2024-09-04T17:48:32.649 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:32.649 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.650 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.650 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:32.650 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.971078+0000", 2024-09-04T17:48:32.650 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.1969999999999998e-06 2024-09-04T17:48:32.650 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.650 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.650 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:32.650 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.971096+0000", 2024-09-04T17:48:32.650 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7898999999999999e-05 2024-09-04T17:48:32.650 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.650 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.650 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:32.650 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.971140+0000", 2024-09-04T17:48:32.650 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.3883e-05 2024-09-04T17:48:32.651 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.651 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.651 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:32.651 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.971165+0000", 2024-09-04T17:48:32.651 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5321999999999999e-05 2024-09-04T17:48:32.651 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.651 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.651 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:32.651 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.971346+0000", 2024-09-04T17:48:32.651 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000180784 2024-09-04T17:48:32.651 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.651 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.651 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op must wait for map", 2024-09-04T17:48:32.652 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.971369+0000", 2024-09-04T17:48:32.652 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.3286e-05 2024-09-04T17:48:32.652 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.652 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.652 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:32.652 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.977475+0000", 2024-09-04T17:48:32.652 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0061055459999999999 2024-09-04T17:48:32.652 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.652 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.652 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:32.652 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.977665+0000", 2024-09-04T17:48:32.652 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019043599999999999 2024-09-04T17:48:32.652 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.652 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.653 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 4", 2024-09-04T17:48:32.653 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.978586+0000", 2024-09-04T17:48:32.653 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000920597 2024-09-04T17:48:32.653 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.653 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.653 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:48:32.653 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.982469+0000", 2024-09-04T17:48:32.653 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0038834719999999998 2024-09-04T17:48:32.653 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.653 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.653 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:32.653 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.988390+0000", 2024-09-04T17:48:32.654 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0059206479999999997 2024-09-04T17:48:32.654 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.654 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.654 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:32.654 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.988546+0000", 2024-09-04T17:48:32.654 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00015548 2024-09-04T17:48:32.654 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.654 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.654 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:32.654 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.988637+0000", 2024-09-04T17:48:32.654 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1082000000000006e-05 2024-09-04T17:48:32.654 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:32.654 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:32.654 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:32.655 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.655 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.655 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:98349 3.a 3:55a1c81d:::rbd_header.113481c24fbf:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139)", 2024-09-04T17:48:32.655 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:18.998248+0000", 2024-09-04T17:48:32.655 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 13.569833406000001, 2024-09-04T17:48:32.655 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0044055120000000003, 2024-09-04T17:48:32.655 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:32.655 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:32.655 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:32.655 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:32.655 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:32.655 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:32.655 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 98349 2024-09-04T17:48:32.656 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.656 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:32.656 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.656 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:32.656 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.998248+0000", 2024-09-04T17:48:32.656 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:32.656 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.656 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.656 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:32.656 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.998248+0000", 2024-09-04T17:48:32.656 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:32.656 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.656 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.657 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:32.657 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.998251+0000", 2024-09-04T17:48:32.657 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.7099999999999999e-06 2024-09-04T17:48:32.657 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.657 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.657 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:32.657 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.998264+0000", 2024-09-04T17:48:32.657 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2999999999999999e-05 2024-09-04T17:48:32.657 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.657 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.657 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:32.657 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.998324+0000", 2024-09-04T17:48:32.657 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.9988999999999998e-05 2024-09-04T17:48:32.658 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.658 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.658 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:32.658 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.998347+0000", 2024-09-04T17:48:32.658 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2875999999999999e-05 2024-09-04T17:48:32.658 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.658 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.658 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:32.658 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.998509+0000", 2024-09-04T17:48:32.658 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00016241699999999999 2024-09-04T17:48:32.658 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.658 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.659 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:32.659 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.998804+0000", 2024-09-04T17:48:32.659 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00029443500000000001 2024-09-04T17:48:32.659 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.659 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.659 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 4", 2024-09-04T17:48:32.659 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.999363+0000", 2024-09-04T17:48:32.659 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00055894699999999996 2024-09-04T17:48:32.659 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.659 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.659 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:48:32.659 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:19.001742+0000", 2024-09-04T17:48:32.659 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0023789509999999998 2024-09-04T17:48:32.659 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.660 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.660 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:32.660 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:19.002457+0000", 2024-09-04T17:48:32.660 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00071553100000000004 2024-09-04T17:48:32.660 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.660 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.660 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:32.660 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:19.002543+0000", 2024-09-04T17:48:32.660 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.5940999999999996e-05 2024-09-04T17:48:32.660 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.660 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.660 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:32.660 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:19.002654+0000", 2024-09-04T17:48:32.661 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000110715 2024-09-04T17:48:32.661 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:32.661 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:32.661 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:32.661 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.661 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.661 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:98921 3.c e142/122 3:37678a48:::rbd_data.113460fdfacd.0000000000000006:head v 142'572, mlcod=137'569)", 2024-09-04T17:48:32.661 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:22.131110+0000", 2024-09-04T17:48:32.661 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 10.436971870000001, 2024-09-04T17:48:32.661 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0055872880000000002, 2024-09-04T17:48:32.661 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:32.661 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:32.661 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:32.662 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:32.662 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.662 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:32.662 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.131110+0000", 2024-09-04T17:48:32.662 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:32.662 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.662 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.662 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:32.662 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.131110+0000", 2024-09-04T17:48:32.662 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:32.662 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.662 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.662 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:32.663 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.131110+0000", 2024-09-04T17:48:32.663 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7e-07 2024-09-04T17:48:32.663 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.663 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.663 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:32.663 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.131120+0000", 2024-09-04T17:48:32.663 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.2499999999999995e-06 2024-09-04T17:48:32.663 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.663 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.663 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:32.663 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.131146+0000", 2024-09-04T17:48:32.663 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6461999999999998e-05 2024-09-04T17:48:32.664 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.664 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.664 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:32.664 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.131157+0000", 2024-09-04T17:48:32.664 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1522000000000001e-05 2024-09-04T17:48:32.664 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.664 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.664 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:32.664 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.131323+0000", 2024-09-04T17:48:32.664 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00016558499999999999 2024-09-04T17:48:32.664 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.664 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.664 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:32.664 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.131351+0000", 2024-09-04T17:48:32.665 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.7494000000000001e-05 2024-09-04T17:48:32.665 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.665 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.665 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:32.665 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.136648+0000", 2024-09-04T17:48:32.665 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0052975330000000001 2024-09-04T17:48:32.665 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.665 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.665 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:32.665 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.136697+0000", 2024-09-04T17:48:32.665 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9072e-05 2024-09-04T17:48:32.665 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:32.665 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:32.666 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:32.666 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.666 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.666 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:98922 3.0 e142/121 3:00fa19d2:::rbd_data.113460fdfacd.0000000000000007:head v 142'381, mlcod=135'378)", 2024-09-04T17:48:32.666 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:22.132316+0000", 2024-09-04T17:48:32.666 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 10.435766088999999, 2024-09-04T17:48:32.666 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0070776679999999996, 2024-09-04T17:48:32.666 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:32.666 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:32.666 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:32.666 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:32.666 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.666 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:32.666 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.132316+0000", 2024-09-04T17:48:32.667 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:32.667 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.667 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.667 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:32.667 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.132316+0000", 2024-09-04T17:48:32.667 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:32.667 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.667 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.667 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:32.667 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.132316+0000", 2024-09-04T17:48:32.667 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2300000000000002e-07 2024-09-04T17:48:32.667 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.667 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.667 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:32.668 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.132322+0000", 2024-09-04T17:48:32.668 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.48e-06 2024-09-04T17:48:32.668 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.668 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.668 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:32.668 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.132335+0000", 2024-09-04T17:48:32.668 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3040999999999999e-05 2024-09-04T17:48:32.668 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.668 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.668 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:32.668 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.132344+0000", 2024-09-04T17:48:32.668 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9989999999999997e-06 2024-09-04T17:48:32.668 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.669 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.669 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:32.669 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.132453+0000", 2024-09-04T17:48:32.669 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000109615 2024-09-04T17:48:32.669 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.669 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.669 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:32.669 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.132481+0000", 2024-09-04T17:48:32.669 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.777e-05 2024-09-04T17:48:32.669 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.669 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.669 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:32.670 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.139271+0000", 2024-09-04T17:48:32.670 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0067898180000000004 2024-09-04T17:48:32.670 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.670 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.670 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:32.670 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.139393+0000", 2024-09-04T17:48:32.670 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00012252199999999999 2024-09-04T17:48:32.670 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:32.670 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:32.670 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:32.670 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.670 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.670 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:99605 3.a 3:50de26c1:::rbd_data.11346a2b0bc9.0000000000000008:head [delete] snapc 1c=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-04T17:48:32.671 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:27.983388+0000", 2024-09-04T17:48:32.671 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.5846939300000003, 2024-09-04T17:48:32.671 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0073493309999999997, 2024-09-04T17:48:32.671 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:32.671 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:32.671 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:32.671 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:32.671 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:32.671 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:32.671 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 99605 2024-09-04T17:48:32.671 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.671 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:32.671 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.671 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:32.672 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:27.983388+0000", 2024-09-04T17:48:32.672 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:32.672 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.672 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.672 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:32.672 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:27.983388+0000", 2024-09-04T17:48:32.672 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:32.672 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.672 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.672 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:32.672 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:27.983392+0000", 2024-09-04T17:48:32.672 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.9770000000000002e-06 2024-09-04T17:48:32.672 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.673 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.673 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:32.673 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:27.983413+0000", 2024-09-04T17:48:32.673 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1075999999999999e-05 2024-09-04T17:48:32.673 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.673 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.673 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:32.673 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:27.983457+0000", 2024-09-04T17:48:32.673 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.3943999999999998e-05 2024-09-04T17:48:32.673 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.673 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.673 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:32.673 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:27.983482+0000", 2024-09-04T17:48:32.674 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5602999999999999e-05 2024-09-04T17:48:32.674 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.674 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.674 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:32.674 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:27.983671+0000", 2024-09-04T17:48:32.674 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00018823399999999999 2024-09-04T17:48:32.674 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.674 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.674 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:32.674 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:27.984069+0000", 2024-09-04T17:48:32.674 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000398639 2024-09-04T17:48:32.674 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.674 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.675 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 4", 2024-09-04T17:48:32.675 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:27.984378+0000", 2024-09-04T17:48:32.675 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000308392 2024-09-04T17:48:32.675 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.675 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.675 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:48:32.675 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:27.987068+0000", 2024-09-04T17:48:32.675 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.002690764 2024-09-04T17:48:32.675 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.675 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.675 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:32.675 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:27.990452+0000", 2024-09-04T17:48:32.675 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0033831080000000001 2024-09-04T17:48:32.675 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.675 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.675 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:32.676 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:27.990603+0000", 2024-09-04T17:48:32.676 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00015124 2024-09-04T17:48:32.676 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:32.676 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:32.676 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:32.676 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:27.990737+0000", 2024-09-04T17:48:32.676 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013435399999999999 2024-09-04T17:48:32.676 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:32.676 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:32.677 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:32.677 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:32.677 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:32.677 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:32.678 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-04T17:48:32.679 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:32.675+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:32.779 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:32.777+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:32.810 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:32.810 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:48:32.810 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:48:32.810 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:32.825 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-04T17:48:32.880 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:32.875+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:32.951 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:32.951 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:48:32.951 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:48:32.951 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:48:32.951 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:32.966 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-04T17:48:32.980 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:32.977+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:33.081 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:33.079+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:33.119 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-04T17:48:33.139 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:33.139 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:48:33.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:48:33.140 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:48:33.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.140 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:48444 3.12 3:4ff80fb5:::rbd_data.11346a2b0bc9.000000000000001d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-04T17:48:33.140 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:58.122601+0000", 2024-09-04T17:48:33.140 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 95.015313598000006, 2024-09-04T17:48:33.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057065713999999997, 2024-09-04T17:48:33.140 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:33.140 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:33.140 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:33.140 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:33.140 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:33.140 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:33.140 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 48444 2024-09-04T17:48:33.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.141 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:33.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:33.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122601+0000", 2024-09-04T17:48:33.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:33.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:33.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122601+0000", 2024-09-04T17:48:33.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:33.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:33.142 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122603+0000", 2024-09-04T17:48:33.142 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3499999999999999e-06 2024-09-04T17:48:33.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.142 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:33.142 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122614+0000", 2024-09-04T17:48:33.142 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.063e-05 2024-09-04T17:48:33.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.142 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:33.142 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122643+0000", 2024-09-04T17:48:33.142 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9434999999999999e-05 2024-09-04T17:48:33.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:33.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122657+0000", 2024-09-04T17:48:33.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3920000000000001e-05 2024-09-04T17:48:33.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:33.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122767+0000", 2024-09-04T17:48:33.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010981599999999999 2024-09-04T17:48:33.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:33.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122989+0000", 2024-09-04T17:48:33.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022158200000000001 2024-09-04T17:48:33.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 3", 2024-09-04T17:48:33.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.133349+0000", 2024-09-04T17:48:33.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010360729000000001 2024-09-04T17:48:33.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:33.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.179423+0000", 2024-09-04T17:48:33.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046073408000000003 2024-09-04T17:48:33.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:33.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.179453+0000", 2024-09-04T17:48:33.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0229000000000001e-05 2024-09-04T17:48:33.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:33.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.179545+0000", 2024-09-04T17:48:33.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1580000000000001e-05 2024-09-04T17:48:33.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:33.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.179667+0000", 2024-09-04T17:48:33.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012203500000000001 2024-09-04T17:48:33.145 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:33.145 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:33.145 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:33.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.146 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:55376 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-04T17:48:33.146 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:11.974053+0000", 2024-09-04T17:48:33.146 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 81.163861814000001, 2024-09-04T17:48:33.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060514722, 2024-09-04T17:48:33.146 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:33.146 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:33.146 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:33.146 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:33.146 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:33.146 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:33.146 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 55376 2024-09-04T17:48:33.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.146 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:33.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.147 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:33.147 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974053+0000", 2024-09-04T17:48:33.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:33.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.147 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:33.147 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974053+0000", 2024-09-04T17:48:33.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:33.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.147 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:33.147 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974054+0000", 2024-09-04T17:48:33.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.841e-06 2024-09-04T17:48:33.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.148 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:33.148 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974062+0000", 2024-09-04T17:48:33.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5460000000000003e-06 2024-09-04T17:48:33.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.148 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:33.148 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974077+0000", 2024-09-04T17:48:33.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5389000000000001e-05 2024-09-04T17:48:33.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.148 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:33.148 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974085+0000", 2024-09-04T17:48:33.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7659999999999998e-06 2024-09-04T17:48:33.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:33.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974188+0000", 2024-09-04T17:48:33.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000102521 2024-09-04T17:48:33.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:33.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974321+0000", 2024-09-04T17:48:33.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013324699999999999 2024-09-04T17:48:33.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-04T17:48:33.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.986481+0000", 2024-09-04T17:48:33.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012160450999999999 2024-09-04T17:48:33.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:33.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:12.034415+0000", 2024-09-04T17:48:33.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.047934065999999997 2024-09-04T17:48:33.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:33.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:12.034438+0000", 2024-09-04T17:48:33.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2923999999999999e-05 2024-09-04T17:48:33.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:33.151 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:12.034505+0000", 2024-09-04T17:48:33.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6711000000000006e-05 2024-09-04T17:48:33.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.151 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:33.151 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:12.034567+0000", 2024-09-04T17:48:33.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2260000000000004e-05 2024-09-04T17:48:33.151 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:33.151 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:33.151 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:33.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.151 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61141 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 2793472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:33.151 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819909+0000", 2024-09-04T17:48:33.151 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 79.318005822000003, 2024-09-04T17:48:33.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042786419999999999, 2024-09-04T17:48:33.152 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:33.152 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:33.152 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:33.152 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:33.152 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:33.152 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:33.152 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61141 2024-09-04T17:48:33.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.152 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:33.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.152 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:33.152 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819909+0000", 2024-09-04T17:48:33.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:33.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:33.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819909+0000", 2024-09-04T17:48:33.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:33.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:33.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819909+0000", 2024-09-04T17:48:33.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6300000000000005e-07 2024-09-04T17:48:33.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:33.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819913+0000", 2024-09-04T17:48:33.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4189999999999998e-06 2024-09-04T17:48:33.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:33.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819920+0000", 2024-09-04T17:48:33.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8619999999999997e-06 2024-09-04T17:48:33.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:33.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819926+0000", 2024-09-04T17:48:33.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9399999999999999e-06 2024-09-04T17:48:33.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:33.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862331+0000", 2024-09-04T17:48:33.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042405348000000002 2024-09-04T17:48:33.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:33.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862410+0000", 2024-09-04T17:48:33.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9078000000000004e-05 2024-09-04T17:48:33.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:33.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862695+0000", 2024-09-04T17:48:33.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028510999999999999 2024-09-04T17:48:33.155 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:33.155 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:33.155 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:33.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.156 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61142 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 2932736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:33.156 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819934+0000", 2024-09-04T17:48:33.156 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 79.317980203999994, 2024-09-04T17:48:33.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043101985000000002, 2024-09-04T17:48:33.156 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:33.156 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:33.156 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:33.156 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:33.156 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:33.156 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:33.156 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61142 2024-09-04T17:48:33.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.156 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:33.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:33.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819934+0000", 2024-09-04T17:48:33.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:33.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:33.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819934+0000", 2024-09-04T17:48:33.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:33.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:33.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819935+0000", 2024-09-04T17:48:33.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2024-09-04T17:48:33.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:33.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819937+0000", 2024-09-04T17:48:33.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5770000000000001e-06 2024-09-04T17:48:33.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:33.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819943+0000", 2024-09-04T17:48:33.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7950000000000004e-06 2024-09-04T17:48:33.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.159 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:33.159 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819948+0000", 2024-09-04T17:48:33.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6779999999999999e-06 2024-09-04T17:48:33.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.159 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:33.159 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862739+0000", 2024-09-04T17:48:33.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042791454999999999 2024-09-04T17:48:33.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.159 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:33.159 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862824+0000", 2024-09-04T17:48:33.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4790999999999995e-05 2024-09-04T17:48:33.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.160 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:33.160 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.863036+0000", 2024-09-04T17:48:33.160 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021224899999999999 2024-09-04T17:48:33.160 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:33.160 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:33.160 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:33.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.160 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61143 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 3039232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:33.160 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819954+0000", 2024-09-04T17:48:33.160 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 79.317960085999999, 2024-09-04T17:48:33.161 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043485134000000002, 2024-09-04T17:48:33.161 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:33.161 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:33.161 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:33.161 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:33.161 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:33.161 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:33.161 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61143 2024-09-04T17:48:33.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.161 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:33.161 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.161 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:33.161 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819954+0000", 2024-09-04T17:48:33.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:33.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:33.162 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819954+0000", 2024-09-04T17:48:33.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:33.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:33.162 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819955+0000", 2024-09-04T17:48:33.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7300000000000002e-07 2024-09-04T17:48:33.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.163 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:33.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819957+0000", 2024-09-04T17:48:33.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.024e-06 2024-09-04T17:48:33.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.163 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.163 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:33.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819963+0000", 2024-09-04T17:48:33.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8109999999999999e-06 2024-09-04T17:48:33.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.163 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.163 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:33.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819967+0000", 2024-09-04T17:48:33.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6149999999999999e-06 2024-09-04T17:48:33.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.163 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.164 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:33.164 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.863082+0000", 2024-09-04T17:48:33.164 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043115078000000001 2024-09-04T17:48:33.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.164 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:33.164 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.863172+0000", 2024-09-04T17:48:33.164 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0181999999999998e-05 2024-09-04T17:48:33.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.164 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:33.164 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.863440+0000", 2024-09-04T17:48:33.164 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026705099999999997 2024-09-04T17:48:33.164 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:33.164 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:33.165 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:33.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.165 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:70015 3.6 3:6012e780:::rbd_data.11341aa245a3.000000000000005b:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-04T17:48:33.165 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:26.155497+0000", 2024-09-04T17:48:33.165 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 66.982417029000004, 2024-09-04T17:48:33.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.049007124999999999, 2024-09-04T17:48:33.165 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:33.165 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:33.165 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:33.165 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:33.165 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:33.166 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:33.166 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 70015 2024-09-04T17:48:33.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.166 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:33.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.166 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:33.166 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155497+0000", 2024-09-04T17:48:33.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:33.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.166 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:33.166 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155497+0000", 2024-09-04T17:48:33.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:33.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:33.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155500+0000", 2024-09-04T17:48:33.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7070000000000001e-06 2024-09-04T17:48:33.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:33.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155513+0000", 2024-09-04T17:48:33.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2989e-05 2024-09-04T17:48:33.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:33.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155544+0000", 2024-09-04T17:48:33.168 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0747000000000002e-05 2024-09-04T17:48:33.168 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.168 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.168 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:33.168 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155560+0000", 2024-09-04T17:48:33.168 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5866e-05 2024-09-04T17:48:33.168 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.168 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.168 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:33.168 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155675+0000", 2024-09-04T17:48:33.168 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000114838 2024-09-04T17:48:33.168 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.168 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.169 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:33.169 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155803+0000", 2024-09-04T17:48:33.169 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012851799999999999 2024-09-04T17:48:33.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.169 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 1", 2024-09-04T17:48:33.169 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.163582+0000", 2024-09-04T17:48:33.169 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0077790960000000001 2024-09-04T17:48:33.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.169 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:33.169 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.204055+0000", 2024-09-04T17:48:33.169 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.040472517 2024-09-04T17:48:33.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.170 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:33.170 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.204195+0000", 2024-09-04T17:48:33.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014055200000000001 2024-09-04T17:48:33.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.170 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:33.170 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.204373+0000", 2024-09-04T17:48:33.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000177784 2024-09-04T17:48:33.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.170 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:33.170 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.204505+0000", 2024-09-04T17:48:33.171 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013151100000000001 2024-09-04T17:48:33.171 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:33.171 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:33.171 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:33.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.171 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:70434 3.12 3:4cd2342b:::rbd_data.11341aa245a3.0000000000000091:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-04T17:48:33.171 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:26.943124+0000", 2024-09-04T17:48:33.171 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 66.194790247, 2024-09-04T17:48:33.171 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055011831999999997, 2024-09-04T17:48:33.171 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:33.171 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:33.171 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:33.171 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:33.172 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:33.172 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:33.172 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 70434 2024-09-04T17:48:33.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.172 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:33.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.172 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:33.172 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943124+0000", 2024-09-04T17:48:33.172 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:33.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.172 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:33.172 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943124+0000", 2024-09-04T17:48:33.173 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:33.173 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.173 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.173 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:33.173 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943128+0000", 2024-09-04T17:48:33.173 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8460000000000001e-06 2024-09-04T17:48:33.173 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.173 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.173 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:33.173 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943142+0000", 2024-09-04T17:48:33.173 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4089e-05 2024-09-04T17:48:33.173 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.173 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.174 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:33.174 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943177+0000", 2024-09-04T17:48:33.174 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5167999999999999e-05 2024-09-04T17:48:33.174 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.174 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.174 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:33.174 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943194+0000", 2024-09-04T17:48:33.174 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6235e-05 2024-09-04T17:48:33.174 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.174 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.174 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:33.174 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943286+0000", 2024-09-04T17:48:33.175 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2793000000000004e-05 2024-09-04T17:48:33.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.175 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:33.175 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943526+0000", 2024-09-04T17:48:33.175 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023950999999999999 2024-09-04T17:48:33.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.175 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 1", 2024-09-04T17:48:33.175 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.952871+0000", 2024-09-04T17:48:33.175 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0093453070000000006 2024-09-04T17:48:33.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.175 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:33.175 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.997968+0000", 2024-09-04T17:48:33.176 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045096613000000001 2024-09-04T17:48:33.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.176 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:33.176 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.998006+0000", 2024-09-04T17:48:33.176 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8257000000000003e-05 2024-09-04T17:48:33.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.176 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:33.176 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.998073+0000", 2024-09-04T17:48:33.176 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6803000000000004e-05 2024-09-04T17:48:33.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.177 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:33.177 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.998136+0000", 2024-09-04T17:48:33.177 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3211000000000002e-05 2024-09-04T17:48:33.177 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:33.177 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:33.177 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:33.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.177 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.177 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93841 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 5111808~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:33.177 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386491+0000", 2024-09-04T17:48:33.177 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 33.75142305, 2024-09-04T17:48:33.177 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043069578999999997, 2024-09-04T17:48:33.177 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:33.178 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:33.178 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:33.178 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:33.178 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:33.178 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:33.178 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93841 2024-09-04T17:48:33.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.178 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:33.178 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.178 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:33.178 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386491+0000", 2024-09-04T17:48:33.178 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:33.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.179 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.179 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:33.179 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386491+0000", 2024-09-04T17:48:33.179 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:33.179 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.179 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.179 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:33.179 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386492+0000", 2024-09-04T17:48:33.179 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0299999999999999e-07 2024-09-04T17:48:33.179 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.179 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.179 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:33.180 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386499+0000", 2024-09-04T17:48:33.180 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6089999999999997e-06 2024-09-04T17:48:33.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.180 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.180 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:33.180 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386506+0000", 2024-09-04T17:48:33.180 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3599999999999998e-06 2024-09-04T17:48:33.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.180 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.180 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:33.180 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386514+0000", 2024-09-04T17:48:33.180 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9379999999999992e-06 2024-09-04T17:48:33.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.180 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.181 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:33.181 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.428957+0000", 2024-09-04T17:48:33.181 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042442831 2024-09-04T17:48:33.181 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.181 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.181 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:33.181 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.429080+0000", 2024-09-04T17:48:33.181 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012359299999999999 2024-09-04T17:48:33.181 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.181 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.181 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:33.181 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.429561+0000", 2024-09-04T17:48:33.182 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00048074499999999998 2024-09-04T17:48:33.182 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:33.182 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:33.182 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:33.182 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.182 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.182 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93842 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 4259840~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:33.182 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386521+0000", 2024-09-04T17:48:33.182 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 33.751393118000003, 2024-09-04T17:48:33.182 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043661933999999999, 2024-09-04T17:48:33.182 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:33.182 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:33.182 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:33.182 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:33.182 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:33.183 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:33.183 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93842 2024-09-04T17:48:33.183 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.183 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:33.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.183 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:33.183 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386521+0000", 2024-09-04T17:48:33.183 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:33.183 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.183 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:33.183 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386521+0000", 2024-09-04T17:48:33.183 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:33.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.184 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:33.184 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386522+0000", 2024-09-04T17:48:33.184 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4700000000000001e-07 2024-09-04T17:48:33.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.184 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:33.184 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386524+0000", 2024-09-04T17:48:33.184 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4559999999999999e-06 2024-09-04T17:48:33.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.184 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:33.185 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386534+0000", 2024-09-04T17:48:33.185 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2639999999999997e-06 2024-09-04T17:48:33.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.185 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.185 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:33.185 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386539+0000", 2024-09-04T17:48:33.185 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6130000000000003e-06 2024-09-04T17:48:33.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.185 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.185 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:33.185 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.429577+0000", 2024-09-04T17:48:33.185 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043037757000000003 2024-09-04T17:48:33.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.186 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:33.186 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.429638+0000", 2024-09-04T17:48:33.186 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1235999999999994e-05 2024-09-04T17:48:33.186 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.186 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:33.186 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.430183+0000", 2024-09-04T17:48:33.186 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000545061 2024-09-04T17:48:33.186 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:33.186 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:33.186 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:33.186 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.187 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93843 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 4390912~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:33.187 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386547+0000", 2024-09-04T17:48:33.187 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 33.751367881, 2024-09-04T17:48:33.187 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044260668000000003, 2024-09-04T17:48:33.187 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:33.187 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:33.187 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:33.187 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:33.187 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:33.187 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:33.187 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93843 2024-09-04T17:48:33.187 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.187 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:33.187 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.188 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:33.188 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386547+0000", 2024-09-04T17:48:33.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:33.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.188 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:33.188 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386547+0000", 2024-09-04T17:48:33.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:33.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.188 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:33.188 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386547+0000", 2024-09-04T17:48:33.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4199999999999996e-07 2024-09-04T17:48:33.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:33.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386550+0000", 2024-09-04T17:48:33.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6199999999999999e-06 2024-09-04T17:48:33.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:33.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386557+0000", 2024-09-04T17:48:33.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1919999999999999e-06 2024-09-04T17:48:33.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:33.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386562+0000", 2024-09-04T17:48:33.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2580000000000003e-06 2024-09-04T17:48:33.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:33.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.430226+0000", 2024-09-04T17:48:33.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043663484000000002 2024-09-04T17:48:33.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:33.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.430286+0000", 2024-09-04T17:48:33.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0455999999999999e-05 2024-09-04T17:48:33.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:33.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.430807+0000", 2024-09-04T17:48:33.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00052111599999999996 2024-09-04T17:48:33.191 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:33.191 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:33.191 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:33.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.191 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93844 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 4521984~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:33.191 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386603+0000", 2024-09-04T17:48:33.191 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 33.751311829999999, 2024-09-04T17:48:33.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044705350999999997, 2024-09-04T17:48:33.191 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:33.192 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:33.192 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:33.192 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:33.192 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:33.192 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:33.192 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93844 2024-09-04T17:48:33.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.192 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:33.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.192 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:33.192 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386603+0000", 2024-09-04T17:48:33.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:33.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:33.193 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386603+0000", 2024-09-04T17:48:33.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:33.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:33.193 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386603+0000", 2024-09-04T17:48:33.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.92e-07 2024-09-04T17:48:33.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:33.193 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386606+0000", 2024-09-04T17:48:33.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5500000000000001e-06 2024-09-04T17:48:33.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:33.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386614+0000", 2024-09-04T17:48:33.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9570000000000002e-06 2024-09-04T17:48:33.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:33.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386619+0000", 2024-09-04T17:48:33.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5119999999999999e-06 2024-09-04T17:48:33.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:33.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.430828+0000", 2024-09-04T17:48:33.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044208654 2024-09-04T17:48:33.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:33.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.430903+0000", 2024-09-04T17:48:33.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4629999999999995e-05 2024-09-04T17:48:33.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:33.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.431308+0000", 2024-09-04T17:48:33.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000405256 2024-09-04T17:48:33.195 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:33.195 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:33.195 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:33.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.196 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93845 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 4653056~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:33.196 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386627+0000", 2024-09-04T17:48:33.196 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 33.751287241, 2024-09-04T17:48:33.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045262927000000001, 2024-09-04T17:48:33.196 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:33.196 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:33.196 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:33.196 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:33.196 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:33.196 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:33.196 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93845 2024-09-04T17:48:33.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.197 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:33.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:33.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386627+0000", 2024-09-04T17:48:33.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:33.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:33.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386627+0000", 2024-09-04T17:48:33.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:33.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:33.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386628+0000", 2024-09-04T17:48:33.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0999999999999998e-07 2024-09-04T17:48:33.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:33.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386630+0000", 2024-09-04T17:48:33.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3300000000000001e-06 2024-09-04T17:48:33.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:33.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386645+0000", 2024-09-04T17:48:33.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5186e-05 2024-09-04T17:48:33.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:33.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386651+0000", 2024-09-04T17:48:33.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7710000000000003e-06 2024-09-04T17:48:33.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:33.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.431349+0000", 2024-09-04T17:48:33.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044697723000000002 2024-09-04T17:48:33.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:33.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.431410+0000", 2024-09-04T17:48:33.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0677999999999997e-05 2024-09-04T17:48:33.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.200 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:33.200 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.431890+0000", 2024-09-04T17:48:33.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00048052899999999999 2024-09-04T17:48:33.200 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:33.200 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:33.200 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:33.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.200 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93846 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 4784128~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:33.200 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386659+0000", 2024-09-04T17:48:33.201 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 33.751255571999998, 2024-09-04T17:48:33.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045784490999999997, 2024-09-04T17:48:33.201 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:33.201 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:33.201 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:33.201 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:33.201 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:33.201 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:33.201 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93846 2024-09-04T17:48:33.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.201 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:33.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:33.202 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386659+0000", 2024-09-04T17:48:33.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:33.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.202 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:33.202 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386659+0000", 2024-09-04T17:48:33.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:33.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.202 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:33.202 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386659+0000", 2024-09-04T17:48:33.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7000000000000005e-07 2024-09-04T17:48:33.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:33.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386662+0000", 2024-09-04T17:48:33.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3300000000000001e-06 2024-09-04T17:48:33.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:33.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386669+0000", 2024-09-04T17:48:33.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0589999999999997e-06 2024-09-04T17:48:33.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:33.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386674+0000", 2024-09-04T17:48:33.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3009999999999997e-06 2024-09-04T17:48:33.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.204 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:33.204 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.431934+0000", 2024-09-04T17:48:33.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045259570999999998 2024-09-04T17:48:33.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.204 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:33.204 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.431998+0000", 2024-09-04T17:48:33.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4775000000000002e-05 2024-09-04T17:48:33.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.204 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:33.204 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.432443+0000", 2024-09-04T17:48:33.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00044488500000000001 2024-09-04T17:48:33.205 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:33.205 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:33.205 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:33.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.205 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93847 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 4915200~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:33.205 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386681+0000", 2024-09-04T17:48:33.205 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 33.751233315, 2024-09-04T17:48:33.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046305101000000001, 2024-09-04T17:48:33.205 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:33.205 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:33.205 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:33.205 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:33.205 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:33.206 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:33.206 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93847 2024-09-04T17:48:33.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.206 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:33.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:33.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386681+0000", 2024-09-04T17:48:33.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:33.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:33.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386681+0000", 2024-09-04T17:48:33.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:33.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.207 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:33.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386682+0000", 2024-09-04T17:48:33.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1699999999999998e-07 2024-09-04T17:48:33.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.207 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:33.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386684+0000", 2024-09-04T17:48:33.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.153e-06 2024-09-04T17:48:33.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.207 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:33.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386693+0000", 2024-09-04T17:48:33.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9639999999999992e-06 2024-09-04T17:48:33.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.208 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:33.208 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386698+0000", 2024-09-04T17:48:33.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3709999999999999e-06 2024-09-04T17:48:33.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.208 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:33.208 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.432476+0000", 2024-09-04T17:48:33.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045778147999999998 2024-09-04T17:48:33.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.208 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:33.209 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.432529+0000", 2024-09-04T17:48:33.209 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3165999999999999e-05 2024-09-04T17:48:33.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.209 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.209 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:33.209 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.432986+0000", 2024-09-04T17:48:33.209 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00045678199999999998 2024-09-04T17:48:33.209 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:33.209 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:33.209 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:33.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.209 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.209 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93848 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 5046272~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:33.210 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386705+0000", 2024-09-04T17:48:33.210 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 33.751209213999999, 2024-09-04T17:48:33.210 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046825259000000001, 2024-09-04T17:48:33.210 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:33.210 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:33.210 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:33.210 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:33.210 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:33.210 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:33.210 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93848 2024-09-04T17:48:33.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.210 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:33.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:33.211 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386705+0000", 2024-09-04T17:48:33.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:33.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:33.211 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386705+0000", 2024-09-04T17:48:33.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:33.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:33.211 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386706+0000", 2024-09-04T17:48:33.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1500000000000005e-07 2024-09-04T17:48:33.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.212 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:33.212 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386708+0000", 2024-09-04T17:48:33.212 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2979999999999999e-06 2024-09-04T17:48:33.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.212 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:33.212 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386715+0000", 2024-09-04T17:48:33.212 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8669999999999996e-06 2024-09-04T17:48:33.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.212 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:33.212 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386720+0000", 2024-09-04T17:48:33.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.242e-06 2024-09-04T17:48:33.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.213 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:33.213 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.433019+0000", 2024-09-04T17:48:33.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046298359999999997 2024-09-04T17:48:33.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.213 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:33.213 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.433084+0000", 2024-09-04T17:48:33.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5395999999999995e-05 2024-09-04T17:48:33.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.214 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:33.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.433531+0000", 2024-09-04T17:48:33.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00044658099999999999 2024-09-04T17:48:33.214 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:33.214 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:33.214 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:33.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.214 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93849 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 5177344~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:33.214 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386727+0000", 2024-09-04T17:48:33.214 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 33.751187315999999, 2024-09-04T17:48:33.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.047267796000000001, 2024-09-04T17:48:33.214 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:33.214 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:33.215 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:33.215 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:33.215 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:33.215 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:33.215 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93849 2024-09-04T17:48:33.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.215 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:33.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:33.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386727+0000", 2024-09-04T17:48:33.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:33.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:33.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386727+0000", 2024-09-04T17:48:33.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:33.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:33.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386728+0000", 2024-09-04T17:48:33.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1699999999999998e-07 2024-09-04T17:48:33.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:33.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386730+0000", 2024-09-04T17:48:33.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2589999999999999e-06 2024-09-04T17:48:33.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.217 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:33.217 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386751+0000", 2024-09-04T17:48:33.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1424999999999999e-05 2024-09-04T17:48:33.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.217 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:33.217 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386759+0000", 2024-09-04T17:48:33.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9400000000000002e-06 2024-09-04T17:48:33.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.217 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:33.217 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.433559+0000", 2024-09-04T17:48:33.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046799573999999997 2024-09-04T17:48:33.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:33.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.433610+0000", 2024-09-04T17:48:33.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1134999999999997e-05 2024-09-04T17:48:33.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:33.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.433995+0000", 2024-09-04T17:48:33.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000384946 2024-09-04T17:48:33.218 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:33.218 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:33.218 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:33.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.219 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:96175 3.12 3:4d4471be:::rbd_data.1134c4e2d26.000000000000006e:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123)", 2024-09-04T17:48:33.219 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:48:02.919542+0000", 2024-09-04T17:48:33.219 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.218372072000001, 2024-09-04T17:48:33.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060180626000000001, 2024-09-04T17:48:33.219 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:33.219 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:33.219 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:33.219 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:33.219 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:33.219 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:33.219 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 96175 2024-09-04T17:48:33.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.219 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:33.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:33.220 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.919542+0000", 2024-09-04T17:48:33.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:33.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:33.220 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.919542+0000", 2024-09-04T17:48:33.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:33.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:33.220 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.919545+0000", 2024-09-04T17:48:33.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0199999999999999e-06 2024-09-04T17:48:33.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.221 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.221 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:33.221 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.919559+0000", 2024-09-04T17:48:33.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3293e-05 2024-09-04T17:48:33.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.221 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.221 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:33.221 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.919587+0000", 2024-09-04T17:48:33.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7922000000000001e-05 2024-09-04T17:48:33.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.221 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.221 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:33.222 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.919601+0000", 2024-09-04T17:48:33.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4657e-05 2024-09-04T17:48:33.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:33.222 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.919708+0000", 2024-09-04T17:48:33.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010624999999999999 2024-09-04T17:48:33.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:33.222 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.919929+0000", 2024-09-04T17:48:33.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022147399999999999 2024-09-04T17:48:33.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.223 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2024-09-04T17:48:33.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.929774+0000", 2024-09-04T17:48:33.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0098449839999999993 2024-09-04T17:48:33.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.223 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:33.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.979511+0000", 2024-09-04T17:48:33.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.049736537999999997 2024-09-04T17:48:33.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.223 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:33.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.979547+0000", 2024-09-04T17:48:33.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6069000000000003e-05 2024-09-04T17:48:33.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:33.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.979642+0000", 2024-09-04T17:48:33.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5526999999999995e-05 2024-09-04T17:48:33.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:33.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.979723+0000", 2024-09-04T17:48:33.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0891999999999997e-05 2024-09-04T17:48:33.224 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:33.224 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:33.224 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:33.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.225 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:97148 3.4 3:22dd2475:::rbd_id.image_client.0-clone39:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e128)", 2024-09-04T17:48:33.225 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:48:07.831881+0000", 2024-09-04T17:48:33.225 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 25.306033838000001, 2024-09-04T17:48:33.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.026553585, 2024-09-04T17:48:33.225 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:33.225 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:33.225 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:33.225 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:33.225 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:33.225 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:33.225 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 97148 2024-09-04T17:48:33.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.225 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:33.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:33.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:07.831881+0000", 2024-09-04T17:48:33.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:33.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:33.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:07.831881+0000", 2024-09-04T17:48:33.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:33.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:33.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:07.831883+0000", 2024-09-04T17:48:33.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3369999999999998e-06 2024-09-04T17:48:33.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:33.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:07.831890+0000", 2024-09-04T17:48:33.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7050000000000002e-06 2024-09-04T17:48:33.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:33.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:07.831905+0000", 2024-09-04T17:48:33.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5512999999999999e-05 2024-09-04T17:48:33.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:33.228 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:07.831918+0000", 2024-09-04T17:48:33.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3223e-05 2024-09-04T17:48:33.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.228 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:33.228 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:07.833441+0000", 2024-09-04T17:48:33.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0015228920000000001 2024-09-04T17:48:33.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.228 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-09-04T17:48:33.228 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:07.833451+0000", 2024-09-04T17:48:33.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6600000000000007e-06 2024-09-04T17:48:33.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.229 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:33.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:07.837721+0000", 2024-09-04T17:48:33.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0042702410000000001 2024-09-04T17:48:33.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.229 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-04T17:48:33.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:07.837732+0000", 2024-09-04T17:48:33.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.057e-05 2024-09-04T17:48:33.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.229 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:33.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:08.856291+0000", 2024-09-04T17:48:33.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0185595350000001 2024-09-04T17:48:33.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:33.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:08.856503+0000", 2024-09-04T17:48:33.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000212127 2024-09-04T17:48:33.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2,5", 2024-09-04T17:48:33.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:08.856664+0000", 2024-09-04T17:48:33.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016072800000000001 2024-09-04T17:48:33.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:33.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:08.857811+0000", 2024-09-04T17:48:33.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001146441 2024-09-04T17:48:33.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:33.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:08.858202+0000", 2024-09-04T17:48:33.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000391605 2024-09-04T17:48:33.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:33.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:08.858239+0000", 2024-09-04T17:48:33.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7200000000000003e-05 2024-09-04T17:48:33.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:33.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:08.858287+0000", 2024-09-04T17:48:33.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7515999999999997e-05 2024-09-04T17:48:33.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:33.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:08.858434+0000", 2024-09-04T17:48:33.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014729200000000001 2024-09-04T17:48:33.232 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:33.232 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:33.232 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:33.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.232 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:99510 3.18 3:1bf8b1d0:::rbd_header.11346a2b0bc9:head [call rbd.snapshot_remove in=18b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e147)", 2024-09-04T17:48:33.233 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:48:27.032522+0000", 2024-09-04T17:48:33.233 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.1053919749999999, 2024-09-04T17:48:33.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.84201522500000003, 2024-09-04T17:48:33.233 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:33.233 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:33.233 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:33.233 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:33.233 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:33.233 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:33.233 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 99510 2024-09-04T17:48:33.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.233 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:33.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:33.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032522+0000", 2024-09-04T17:48:33.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:33.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:33.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032522+0000", 2024-09-04T17:48:33.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:33.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:33.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032525+0000", 2024-09-04T17:48:33.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4789999999999999e-06 2024-09-04T17:48:33.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:33.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032536+0000", 2024-09-04T17:48:33.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0557000000000001e-05 2024-09-04T17:48:33.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:33.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032555+0000", 2024-09-04T17:48:33.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9782000000000001e-05 2024-09-04T17:48:33.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:33.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032569+0000", 2024-09-04T17:48:33.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4012000000000001e-05 2024-09-04T17:48:33.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:33.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032649+0000", 2024-09-04T17:48:33.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9782000000000003e-05 2024-09-04T17:48:33.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-09-04T17:48:33.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032657+0000", 2024-09-04T17:48:33.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7999999999999999e-06 2024-09-04T17:48:33.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:33.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.035521+0000", 2024-09-04T17:48:33.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028640850000000002 2024-09-04T17:48:33.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-04T17:48:33.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.035534+0000", 2024-09-04T17:48:33.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3157e-05 2024-09-04T17:48:33.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:33.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.869836+0000", 2024-09-04T17:48:33.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.83430168400000004 2024-09-04T17:48:33.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:33.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.870070+0000", 2024-09-04T17:48:33.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023443300000000001 2024-09-04T17:48:33.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 3", 2024-09-04T17:48:33.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.870423+0000", 2024-09-04T17:48:33.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00035290700000000001 2024-09-04T17:48:33.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:33.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.871254+0000", 2024-09-04T17:48:33.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00083113700000000002 2024-09-04T17:48:33.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.239 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:33.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.874234+0000", 2024-09-04T17:48:33.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029795189999999999 2024-09-04T17:48:33.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.239 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:33.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.874321+0000", 2024-09-04T17:48:33.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7637000000000001e-05 2024-09-04T17:48:33.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.239 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:33.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.874538+0000", 2024-09-04T17:48:33.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021625400000000001 2024-09-04T17:48:33.240 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:33.240 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:33.240 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:33.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.240 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:99511 3.18 3:1bf8b1d0:::rbd_header.11346a2b0bc9:head [watch reconnect cookie 140018821496000 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147)", 2024-09-04T17:48:33.240 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:48:27.032602+0000", 2024-09-04T17:48:33.240 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.1053121719999996, 2024-09-04T17:48:33.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.84736467999999998, 2024-09-04T17:48:33.240 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:33.240 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:33.240 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:33.240 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:33.241 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:33.241 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:33.241 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 99511 2024-09-04T17:48:33.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.241 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:33.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:33.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032602+0000", 2024-09-04T17:48:33.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:33.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:33.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032602+0000", 2024-09-04T17:48:33.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:33.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:33.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032603+0000", 2024-09-04T17:48:33.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3300000000000001e-07 2024-09-04T17:48:33.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:33.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032610+0000", 2024-09-04T17:48:33.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.754e-06 2024-09-04T17:48:33.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:33.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032621+0000", 2024-09-04T17:48:33.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1175999999999999e-05 2024-09-04T17:48:33.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:33.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032630+0000", 2024-09-04T17:48:33.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7900000000000005e-06 2024-09-04T17:48:33.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:33.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032900+0000", 2024-09-04T17:48:33.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026967499999999998 2024-09-04T17:48:33.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.244 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting_for_map not empty", 2024-09-04T17:48:33.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032907+0000", 2024-09-04T17:48:33.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6969999999999993e-06 2024-09-04T17:48:33.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.244 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:33.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.036096+0000", 2024-09-04T17:48:33.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031887090000000001 2024-09-04T17:48:33.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.244 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-04T17:48:33.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.036107+0000", 2024-09-04T17:48:33.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1361e-05 2024-09-04T17:48:33.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.245 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:33.245 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.870783+0000", 2024-09-04T17:48:33.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.834675265 2024-09-04T17:48:33.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.245 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-09-04T17:48:33.245 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.870943+0000", 2024-09-04T17:48:33.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000160364 2024-09-04T17:48:33.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:33.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.874588+0000", 2024-09-04T17:48:33.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0036446529999999999 2024-09-04T17:48:33.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:33.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.874731+0000", 2024-09-04T17:48:33.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014334499999999999 2024-09-04T17:48:33.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:33.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:33.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:33.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.879967+0000", 2024-09-04T17:48:33.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005236058 2024-09-04T17:48:33.246 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:33.247 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:33.247 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:33.247 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:33.247 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:33.247 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:33.248 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-04T17:48:33.249 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:48:33.243+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:48:33.258 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:48:33.259 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:48:33.275 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:48:33.276 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:48:33.292 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-04T17:48:33.293 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-04T17:48:33.308 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-04T17:48:33.309 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-04T17:48:33.330 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:48:33.330 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:48:33.349 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:33.347+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:33.350 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:48:33.351 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:48:33.371 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-04T17:48:33.372 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-04T17:48:33.373 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:33.373 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:33.388 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-04T17:48:33.393 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:48:33.393 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:48:33.457 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:33.455+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:33.514 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:33.515 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:33.529 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-04T17:48:33.551 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:48:33.549+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:48:33.653 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:33.647+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:33.692 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:33.692 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:33.706 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-04T17:48:33.753 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:48:33.753+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:48:33.838 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:33.838 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:33.854 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-04T17:48:33.857 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:48:33.851+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:48:33.955 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:48:33.953+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:48:34.036 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:34.036 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:34.051 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-04T17:48:34.074 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-04T17:48:34.093 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:34.093+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:34.156 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:48:34.151+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:48:34.216 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:34.216 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:34.230 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-04T17:48:34.277 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:34.277+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:34.347 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:34.347 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:34.362 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-04T17:48:34.363 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:34.359+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:34.463 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:34.459+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:34.531 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:34.531 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:34.545 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-04T17:48:34.593 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:48:34.593+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:48:34.663 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:34.664 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:34.667 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:34.665+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:34.678 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-04T17:48:34.768 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:48:34.767+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:48:34.847 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:34.847 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:34.849 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is set 2024-09-04T17:48:34.864 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-04T17:48:34.869 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:34.867+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:34.971 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:34.967+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:34.997 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:34.997 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:35.011 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-04T17:48:35.071 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:35.069+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:35.138 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:35.138 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:35.152 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-04T17:48:35.193 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:35.193+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:35.272 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:48:35.267+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:48:35.283 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:35.283 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:35.297 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-04T17:48:35.373 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:35.373+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:35.412 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-04T17:48:35.464 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:35.464 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:35.474 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:48:35.473+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:48:35.481 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-04T17:48:35.553 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:48:35.553 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:48:35.569 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:48:35.570 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:48:35.575 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:35.573+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:35.586 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-04T17:48:35.586 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-04T17:48:35.601 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:35.601 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:35.602 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-04T17:48:35.603 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-04T17:48:35.616 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-04T17:48:35.624 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:48:35.624 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:48:35.645 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:48:35.645 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:48:35.665 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-04T17:48:35.666 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-04T17:48:35.676 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:48:35.673+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:48:35.688 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:48:35.688 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:48:35.733 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:35.733 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:48:35.733 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:48:35.733 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:35.747 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-04T17:48:35.789 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:35.787+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:35.879 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:35.877+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:35.920 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:35.920 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:48:35.920 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:48:35.920 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:48:35.920 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:35.935 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-04T17:48:35.978 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:48:35.977+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:48:36.070 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:36.070 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:48:36.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:48:36.070 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:48:36.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.070 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:71638 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-04T17:48:36.070 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:44.479908+0000", 2024-09-04T17:48:36.070 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.588986818999999, 2024-09-04T17:48:36.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.469396488, 2024-09-04T17:48:36.070 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:36.070 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:36.071 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:36.071 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:36.071 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:36.071 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:36.071 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 71638 2024-09-04T17:48:36.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.071 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:36.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:36.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.479908+0000", 2024-09-04T17:48:36.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:36.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:36.072 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.479908+0000", 2024-09-04T17:48:36.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:36.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:36.072 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.479911+0000", 2024-09-04T17:48:36.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9160000000000001e-06 2024-09-04T17:48:36.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:36.072 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.479925+0000", 2024-09-04T17:48:36.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3896e-05 2024-09-04T17:48:36.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:36.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.479954+0000", 2024-09-04T17:48:36.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9249000000000001e-05 2024-09-04T17:48:36.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:36.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.479969+0000", 2024-09-04T17:48:36.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4683e-05 2024-09-04T17:48:36.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:36.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.480109+0000", 2024-09-04T17:48:36.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013995699999999999 2024-09-04T17:48:36.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-04T17:48:36.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.480125+0000", 2024-09-04T17:48:36.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6651999999999999e-05 2024-09-04T17:48:36.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:36.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.948598+0000", 2024-09-04T17:48:36.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.46847249600000002 2024-09-04T17:48:36.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:36.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.948889+0000", 2024-09-04T17:48:36.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000290947 2024-09-04T17:48:36.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:36.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.949304+0000", 2024-09-04T17:48:36.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000415692 2024-09-04T17:48:36.075 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:36.075 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:36.075 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:36.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.076 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:72267 3.1a 3:58dbd2e1:::rbd_data.11342ece613c.000000000000001e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-09-04T17:48:36.076 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:45.674662+0000", 2024-09-04T17:48:36.076 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.394233010000001, 2024-09-04T17:48:36.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032248339000000001, 2024-09-04T17:48:36.076 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:36.076 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:36.076 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:36.076 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:36.076 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:36.076 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:36.076 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 72267 2024-09-04T17:48:36.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.077 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:36.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.077 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:36.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.674662+0000", 2024-09-04T17:48:36.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:36.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.077 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:36.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.674662+0000", 2024-09-04T17:48:36.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:36.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:36.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.674664+0000", 2024-09-04T17:48:36.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4870000000000001e-06 2024-09-04T17:48:36.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:36.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.674689+0000", 2024-09-04T17:48:36.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4805000000000001e-05 2024-09-04T17:48:36.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:36.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.674715+0000", 2024-09-04T17:48:36.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5548999999999999e-05 2024-09-04T17:48:36.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:36.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.674731+0000", 2024-09-04T17:48:36.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6549999999999999e-05 2024-09-04T17:48:36.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:36.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.674843+0000", 2024-09-04T17:48:36.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000112001 2024-09-04T17:48:36.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:36.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.675176+0000", 2024-09-04T17:48:36.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00033337200000000002 2024-09-04T17:48:36.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-04T17:48:36.080 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.680121+0000", 2024-09-04T17:48:36.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0049445540000000003 2024-09-04T17:48:36.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:36.080 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.706702+0000", 2024-09-04T17:48:36.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026580840000000001 2024-09-04T17:48:36.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:36.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.706738+0000", 2024-09-04T17:48:36.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6158999999999997e-05 2024-09-04T17:48:36.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:36.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.706835+0000", 2024-09-04T17:48:36.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6909000000000003e-05 2024-09-04T17:48:36.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:36.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.706910+0000", 2024-09-04T17:48:36.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5112999999999997e-05 2024-09-04T17:48:36.082 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:36.082 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:36.082 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:36.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.082 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94568 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 1703936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:36.082 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.605863+0000", 2024-09-04T17:48:36.082 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.463032194999997, 2024-09-04T17:48:36.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027249301, 2024-09-04T17:48:36.082 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:36.082 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:36.083 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:36.083 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:36.083 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:36.083 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:36.083 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94568 2024-09-04T17:48:36.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.083 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:36.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.083 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:36.083 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605863+0000", 2024-09-04T17:48:36.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:36.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:36.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605863+0000", 2024-09-04T17:48:36.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:36.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:36.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605863+0000", 2024-09-04T17:48:36.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7999999999999996e-07 2024-09-04T17:48:36.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:36.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605866+0000", 2024-09-04T17:48:36.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6199999999999999e-06 2024-09-04T17:48:36.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:36.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605875+0000", 2024-09-04T17:48:36.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2499999999999995e-06 2024-09-04T17:48:36.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:36.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605881+0000", 2024-09-04T17:48:36.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.19e-06 2024-09-04T17:48:36.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:36.086 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.632629+0000", 2024-09-04T17:48:36.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026747567 2024-09-04T17:48:36.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:36.086 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.632874+0000", 2024-09-04T17:48:36.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024493600000000001 2024-09-04T17:48:36.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:36.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.633112+0000", 2024-09-04T17:48:36.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023825800000000001 2024-09-04T17:48:36.087 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:36.087 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:36.087 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:36.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.087 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94569 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:36.087 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.605888+0000", 2024-09-04T17:48:36.087 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.463006366000002, 2024-09-04T17:48:36.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027675620000000001, 2024-09-04T17:48:36.087 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:36.087 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:36.088 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:36.088 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:36.088 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:36.088 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:36.088 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94569 2024-09-04T17:48:36.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.088 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:36.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:36.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605888+0000", 2024-09-04T17:48:36.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:36.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:36.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605888+0000", 2024-09-04T17:48:36.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:36.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:36.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605889+0000", 2024-09-04T17:48:36.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4799999999999998e-07 2024-09-04T17:48:36.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:36.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605891+0000", 2024-09-04T17:48:36.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4880000000000001e-06 2024-09-04T17:48:36.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:36.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605901+0000", 2024-09-04T17:48:36.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4760000000000004e-06 2024-09-04T17:48:36.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:36.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605912+0000", 2024-09-04T17:48:36.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0698e-05 2024-09-04T17:48:36.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:36.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.633184+0000", 2024-09-04T17:48:36.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.02727253 2024-09-04T17:48:36.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:36.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.633330+0000", 2024-09-04T17:48:36.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014543800000000001 2024-09-04T17:48:36.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:36.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.633564+0000", 2024-09-04T17:48:36.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023444199999999999 2024-09-04T17:48:36.091 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:36.091 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:36.092 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:36.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.092 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94570 3.0 3.3f52c000 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:36.092 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.605932+0000", 2024-09-04T17:48:36.092 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.462962799000003, 2024-09-04T17:48:36.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027687021999999999, 2024-09-04T17:48:36.092 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:36.092 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:36.092 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-04T17:48:36.092 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:36.092 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:36.092 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:36.093 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94570 2024-09-04T17:48:36.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.093 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:36.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:36.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605932+0000", 2024-09-04T17:48:36.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:36.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:36.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605932+0000", 2024-09-04T17:48:36.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:36.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:36.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605933+0000", 2024-09-04T17:48:36.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5000000000000001e-07 2024-09-04T17:48:36.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:36.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605936+0000", 2024-09-04T17:48:36.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7769999999999998e-06 2024-09-04T17:48:36.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:36.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605943+0000", 2024-09-04T17:48:36.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7000000000000008e-06 2024-09-04T17:48:36.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:36.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605949+0000", 2024-09-04T17:48:36.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0170000000000002e-06 2024-09-04T17:48:36.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:36.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.633604+0000", 2024-09-04T17:48:36.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027654875999999998 2024-09-04T17:48:36.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:36.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.633619+0000", 2024-09-04T17:48:36.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4802e-05 2024-09-04T17:48:36.096 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:36.096 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:36.096 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:36.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.096 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94641 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 6094848~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:36.096 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.637288+0000", 2024-09-04T17:48:36.096 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.431606348000003, 2024-09-04T17:48:36.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026966783000000001, 2024-09-04T17:48:36.096 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:36.097 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:36.097 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:36.097 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:36.097 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:36.097 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:36.097 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94641 2024-09-04T17:48:36.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.097 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:36.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:36.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.637288+0000", 2024-09-04T17:48:36.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:36.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:36.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.637288+0000", 2024-09-04T17:48:36.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:36.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:36.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.637289+0000", 2024-09-04T17:48:36.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4600000000000005e-07 2024-09-04T17:48:36.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:36.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.637292+0000", 2024-09-04T17:48:36.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1049999999999999e-06 2024-09-04T17:48:36.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:36.099 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.637301+0000", 2024-09-04T17:48:36.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7509999999999997e-06 2024-09-04T17:48:36.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:36.099 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.637309+0000", 2024-09-04T17:48:36.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.67e-06 2024-09-04T17:48:36.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:36.099 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.663971+0000", 2024-09-04T17:48:36.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026661926999999998 2024-09-04T17:48:36.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:36.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.664062+0000", 2024-09-04T17:48:36.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0673000000000004e-05 2024-09-04T17:48:36.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:36.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.664255+0000", 2024-09-04T17:48:36.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019311099999999999 2024-09-04T17:48:36.100 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:36.101 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:36.101 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:36.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.101 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94642 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 6225920~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:36.101 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.637321+0000", 2024-09-04T17:48:36.101 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.431573401000001, 2024-09-04T17:48:36.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027131534999999998, 2024-09-04T17:48:36.101 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:36.101 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:36.101 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:36.101 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:36.101 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:36.102 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:36.102 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94642 2024-09-04T17:48:36.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.102 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:36.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:36.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.637321+0000", 2024-09-04T17:48:36.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:36.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:36.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.637321+0000", 2024-09-04T17:48:36.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:36.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.103 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:36.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.637322+0000", 2024-09-04T17:48:36.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6599999999999996e-07 2024-09-04T17:48:36.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.103 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:36.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.637325+0000", 2024-09-04T17:48:36.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1070000000000001e-06 2024-09-04T17:48:36.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.103 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:36.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.637337+0000", 2024-09-04T17:48:36.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1766000000000001e-05 2024-09-04T17:48:36.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.104 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:36.104 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.637350+0000", 2024-09-04T17:48:36.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3515e-05 2024-09-04T17:48:36.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.104 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:36.104 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.664273+0000", 2024-09-04T17:48:36.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026922643 2024-09-04T17:48:36.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.105 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:36.105 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.664337+0000", 2024-09-04T17:48:36.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4492999999999994e-05 2024-09-04T17:48:36.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.105 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:36.105 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.664453+0000", 2024-09-04T17:48:36.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000115345 2024-09-04T17:48:36.105 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:36.105 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:36.105 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:36.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.106 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94647 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 6684672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:36.106 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.638572+0000", 2024-09-04T17:48:36.106 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.430322515, 2024-09-04T17:48:36.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026910377999999999, 2024-09-04T17:48:36.106 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:36.106 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:36.106 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:36.106 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:36.106 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:36.106 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:36.106 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94647 2024-09-04T17:48:36.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.106 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:36.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:36.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638572+0000", 2024-09-04T17:48:36.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:36.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:36.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638572+0000", 2024-09-04T17:48:36.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:36.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:36.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638573+0000", 2024-09-04T17:48:36.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7599999999999997e-07 2024-09-04T17:48:36.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:36.108 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638575+0000", 2024-09-04T17:48:36.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2010000000000002e-06 2024-09-04T17:48:36.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:36.108 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638582+0000", 2024-09-04T17:48:36.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7399999999999998e-06 2024-09-04T17:48:36.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:36.109 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638588+0000", 2024-09-04T17:48:36.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8560000000000003e-06 2024-09-04T17:48:36.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.109 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:36.109 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.665294+0000", 2024-09-04T17:48:36.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026705893000000001 2024-09-04T17:48:36.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.109 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:36.109 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.665355+0000", 2024-09-04T17:48:36.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1253000000000005e-05 2024-09-04T17:48:36.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.110 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:36.110 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.665483+0000", 2024-09-04T17:48:36.110 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012775899999999999 2024-09-04T17:48:36.110 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:36.110 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:36.110 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:36.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.110 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.110 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94648 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 6815744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:36.110 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.638593+0000", 2024-09-04T17:48:36.110 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.430301427000003, 2024-09-04T17:48:36.110 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027163676000000001, 2024-09-04T17:48:36.111 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:36.111 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:36.111 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:36.111 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:36.111 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:36.111 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:36.111 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94648 2024-09-04T17:48:36.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.111 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:36.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.111 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:36.111 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638593+0000", 2024-09-04T17:48:36.111 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:36.112 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.112 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.112 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:36.112 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638593+0000", 2024-09-04T17:48:36.112 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:36.112 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.112 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.112 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:36.112 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638594+0000", 2024-09-04T17:48:36.112 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.53e-07 2024-09-04T17:48:36.112 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.112 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.112 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:36.113 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638596+0000", 2024-09-04T17:48:36.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8870000000000001e-06 2024-09-04T17:48:36.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:36.113 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638600+0000", 2024-09-04T17:48:36.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3290000000000004e-06 2024-09-04T17:48:36.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:36.113 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638603+0000", 2024-09-04T17:48:36.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4680000000000001e-06 2024-09-04T17:48:36.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.114 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.114 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:36.114 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.665524+0000", 2024-09-04T17:48:36.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.02692052 2024-09-04T17:48:36.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.114 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.114 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:36.114 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.665600+0000", 2024-09-04T17:48:36.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6569000000000005e-05 2024-09-04T17:48:36.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.114 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.114 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:36.114 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.665757+0000", 2024-09-04T17:48:36.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015655000000000001 2024-09-04T17:48:36.115 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:36.115 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:36.115 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:36.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.115 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94649 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 6946816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:36.115 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.638608+0000", 2024-09-04T17:48:36.115 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.430286997000003, 2024-09-04T17:48:36.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027355753999999999, 2024-09-04T17:48:36.115 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:36.115 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:36.115 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:36.115 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:36.116 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:36.116 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:36.116 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94649 2024-09-04T17:48:36.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.116 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:36.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:36.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638608+0000", 2024-09-04T17:48:36.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:36.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:36.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638608+0000", 2024-09-04T17:48:36.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:36.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:36.117 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638608+0000", 2024-09-04T17:48:36.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3099999999999998e-07 2024-09-04T17:48:36.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:36.117 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638610+0000", 2024-09-04T17:48:36.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.716e-06 2024-09-04T17:48:36.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.118 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:36.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638614+0000", 2024-09-04T17:48:36.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.941e-06 2024-09-04T17:48:36.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.118 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:36.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638618+0000", 2024-09-04T17:48:36.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5009999999999998e-06 2024-09-04T17:48:36.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.118 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:36.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.665795+0000", 2024-09-04T17:48:36.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027176404000000001 2024-09-04T17:48:36.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:36.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.665851+0000", 2024-09-04T17:48:36.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6530999999999998e-05 2024-09-04T17:48:36.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:36.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.665963+0000", 2024-09-04T17:48:36.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011213 2024-09-04T17:48:36.119 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:36.119 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:36.119 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:36.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.120 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94650 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 7077888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:36.120 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.638623+0000", 2024-09-04T17:48:36.120 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.430271701000002, 2024-09-04T17:48:36.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027665261999999999, 2024-09-04T17:48:36.120 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:36.120 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:36.120 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:36.120 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:36.120 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:36.120 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:36.120 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94650 2024-09-04T17:48:36.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.121 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:36.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.121 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:36.121 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638623+0000", 2024-09-04T17:48:36.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:36.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.121 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:36.121 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638623+0000", 2024-09-04T17:48:36.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:36.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.121 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:36.121 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638623+0000", 2024-09-04T17:48:36.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6300000000000001e-07 2024-09-04T17:48:36.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:36.122 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638625+0000", 2024-09-04T17:48:36.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7260000000000001e-06 2024-09-04T17:48:36.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:36.122 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638629+0000", 2024-09-04T17:48:36.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1309999999999999e-06 2024-09-04T17:48:36.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.123 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:36.123 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638634+0000", 2024-09-04T17:48:36.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2700000000000004e-06 2024-09-04T17:48:36.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.123 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:36.123 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.666028+0000", 2024-09-04T17:48:36.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027393303000000001 2024-09-04T17:48:36.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.123 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:36.123 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.666112+0000", 2024-09-04T17:48:36.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3844000000000006e-05 2024-09-04T17:48:36.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.124 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:36.124 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.666288+0000", 2024-09-04T17:48:36.124 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000176725 2024-09-04T17:48:36.124 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:36.124 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:36.124 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:36.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.124 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94651 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 7208960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:36.124 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.638639+0000", 2024-09-04T17:48:36.124 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.430255307000003, 2024-09-04T17:48:36.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027839592999999999, 2024-09-04T17:48:36.125 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:36.125 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:36.125 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:36.125 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:36.125 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:36.125 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:36.125 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94651 2024-09-04T17:48:36.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.125 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:36.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.125 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:36.125 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638639+0000", 2024-09-04T17:48:36.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:36.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.126 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:36.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638639+0000", 2024-09-04T17:48:36.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:36.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.126 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:36.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638640+0000", 2024-09-04T17:48:36.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8000000000000002e-07 2024-09-04T17:48:36.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.126 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:36.127 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638641+0000", 2024-09-04T17:48:36.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.77e-06 2024-09-04T17:48:36.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:36.127 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638646+0000", 2024-09-04T17:48:36.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1729999999999997e-06 2024-09-04T17:48:36.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:36.127 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638649+0000", 2024-09-04T17:48:36.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5260000000000002e-06 2024-09-04T17:48:36.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.128 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:36.128 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.666309+0000", 2024-09-04T17:48:36.128 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027660179 2024-09-04T17:48:36.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.128 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:36.128 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.666373+0000", 2024-09-04T17:48:36.128 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4152999999999994e-05 2024-09-04T17:48:36.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.128 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:36.128 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.666479+0000", 2024-09-04T17:48:36.129 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000105512 2024-09-04T17:48:36.129 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:36.129 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:36.129 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:36.129 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.129 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94657 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 7012352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:36.129 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.641793+0000", 2024-09-04T17:48:36.129 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.427101823000001, 2024-09-04T17:48:36.129 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026893672, 2024-09-04T17:48:36.129 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:36.129 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:36.130 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:36.130 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:36.130 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:36.130 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:36.130 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94657 2024-09-04T17:48:36.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.130 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:36.130 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.130 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:36.130 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641793+0000", 2024-09-04T17:48:36.130 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:36.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.130 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.130 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:36.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641793+0000", 2024-09-04T17:48:36.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:36.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:36.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641793+0000", 2024-09-04T17:48:36.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5000000000000003e-07 2024-09-04T17:48:36.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:36.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641797+0000", 2024-09-04T17:48:36.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2909999999999999e-06 2024-09-04T17:48:36.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.132 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:36.132 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641806+0000", 2024-09-04T17:48:36.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3270000000000007e-06 2024-09-04T17:48:36.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.132 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:36.132 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641812+0000", 2024-09-04T17:48:36.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2310000000000001e-06 2024-09-04T17:48:36.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.132 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:36.132 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.668412+0000", 2024-09-04T17:48:36.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026600104999999999 2024-09-04T17:48:36.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.133 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:36.133 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.668534+0000", 2024-09-04T17:48:36.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012115100000000001 2024-09-04T17:48:36.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.133 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:36.133 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.668687+0000", 2024-09-04T17:48:36.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000153017 2024-09-04T17:48:36.133 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:36.133 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:36.134 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:36.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.134 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94658 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 7143424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:36.134 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.641822+0000", 2024-09-04T17:48:36.134 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.427072891999998, 2024-09-04T17:48:36.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027395880000000001, 2024-09-04T17:48:36.134 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:36.134 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:36.134 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:36.134 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:36.134 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:36.134 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:36.134 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94658 2024-09-04T17:48:36.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.135 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:36.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.135 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:36.135 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641822+0000", 2024-09-04T17:48:36.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:36.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.135 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:36.135 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641822+0000", 2024-09-04T17:48:36.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:36.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:36.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641822+0000", 2024-09-04T17:48:36.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.58e-07 2024-09-04T17:48:36.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:36.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641825+0000", 2024-09-04T17:48:36.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5909999999999998e-06 2024-09-04T17:48:36.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:36.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641832+0000", 2024-09-04T17:48:36.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1799999999999999e-06 2024-09-04T17:48:36.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:36.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641838+0000", 2024-09-04T17:48:36.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5040000000000002e-06 2024-09-04T17:48:36.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:36.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.668748+0000", 2024-09-04T17:48:36.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026910119999999999 2024-09-04T17:48:36.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:36.138 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.668896+0000", 2024-09-04T17:48:36.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014804299999999999 2024-09-04T17:48:36.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:36.138 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.669218+0000", 2024-09-04T17:48:36.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00032198399999999999 2024-09-04T17:48:36.138 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:36.138 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:36.138 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:36.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.138 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94659 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 7274496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:36.138 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.641846+0000", 2024-09-04T17:48:36.139 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.427048822000003, 2024-09-04T17:48:36.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027757314000000002, 2024-09-04T17:48:36.139 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:36.139 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:36.139 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:36.139 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:36.139 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:36.139 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:36.139 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94659 2024-09-04T17:48:36.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.139 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:36.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.139 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:36.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641846+0000", 2024-09-04T17:48:36.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:36.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.140 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:36.140 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641846+0000", 2024-09-04T17:48:36.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:36.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.140 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:36.140 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641846+0000", 2024-09-04T17:48:36.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5400000000000002e-07 2024-09-04T17:48:36.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:36.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641849+0000", 2024-09-04T17:48:36.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0170000000000001e-06 2024-09-04T17:48:36.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:36.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641858+0000", 2024-09-04T17:48:36.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1499999999999999e-06 2024-09-04T17:48:36.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:36.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641864+0000", 2024-09-04T17:48:36.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2280000000000003e-06 2024-09-04T17:48:36.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.142 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:36.142 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.669278+0000", 2024-09-04T17:48:36.142 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027413843 2024-09-04T17:48:36.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.142 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:36.142 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.669392+0000", 2024-09-04T17:48:36.142 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000114654 2024-09-04T17:48:36.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.142 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:36.142 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.669603+0000", 2024-09-04T17:48:36.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021096799999999999 2024-09-04T17:48:36.143 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:36.143 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:36.143 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:36.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.143 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94664 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 7864320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:36.143 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.644474+0000", 2024-09-04T17:48:36.143 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.424421142, 2024-09-04T17:48:36.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026965042000000002, 2024-09-04T17:48:36.143 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:36.143 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:36.143 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:36.143 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:36.144 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:36.144 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:36.144 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94664 2024-09-04T17:48:36.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.144 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:36.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:36.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644474+0000", 2024-09-04T17:48:36.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:36.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:36.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644474+0000", 2024-09-04T17:48:36.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:36.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:36.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644474+0000", 2024-09-04T17:48:36.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.16e-07 2024-09-04T17:48:36.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.146 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:36.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644478+0000", 2024-09-04T17:48:36.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1389999999999999e-06 2024-09-04T17:48:36.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.146 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:36.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644494+0000", 2024-09-04T17:48:36.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6977e-05 2024-09-04T17:48:36.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.146 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:36.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644501+0000", 2024-09-04T17:48:36.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.438e-06 2024-09-04T17:48:36.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.147 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:36.147 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.671241+0000", 2024-09-04T17:48:36.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026740525000000001 2024-09-04T17:48:36.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.147 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:36.147 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.671325+0000", 2024-09-04T17:48:36.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3987999999999996e-05 2024-09-04T17:48:36.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.147 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:36.147 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.671439+0000", 2024-09-04T17:48:36.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011315900000000001 2024-09-04T17:48:36.148 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:36.148 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:36.148 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:36.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.148 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94665 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 7995392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:36.148 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.644511+0000", 2024-09-04T17:48:36.148 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.424384109999998, 2024-09-04T17:48:36.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027156390999999998, 2024-09-04T17:48:36.148 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:36.148 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:36.148 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:36.149 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:36.149 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:36.149 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:36.149 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94665 2024-09-04T17:48:36.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.149 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:36.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:36.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644511+0000", 2024-09-04T17:48:36.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:36.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:36.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644511+0000", 2024-09-04T17:48:36.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:36.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:36.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644511+0000", 2024-09-04T17:48:36.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2699999999999999e-07 2024-09-04T17:48:36.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:36.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644514+0000", 2024-09-04T17:48:36.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9890000000000001e-06 2024-09-04T17:48:36.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.151 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:36.151 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644521+0000", 2024-09-04T17:48:36.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5409999999999997e-06 2024-09-04T17:48:36.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.151 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:36.151 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644527+0000", 2024-09-04T17:48:36.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0340000000000002e-06 2024-09-04T17:48:36.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.151 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:36.151 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.671474+0000", 2024-09-04T17:48:36.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.02694767 2024-09-04T17:48:36.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.152 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:36.152 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.671529+0000", 2024-09-04T17:48:36.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4373000000000003e-05 2024-09-04T17:48:36.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.152 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:36.152 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.671667+0000", 2024-09-04T17:48:36.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000138257 2024-09-04T17:48:36.152 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:36.152 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:36.152 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:36.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.153 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94666 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 8126464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:36.153 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.644536+0000", 2024-09-04T17:48:36.153 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.424358972, 2024-09-04T17:48:36.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027360524000000001, 2024-09-04T17:48:36.153 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:36.153 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:36.153 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:36.153 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:36.153 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:36.153 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:36.153 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94666 2024-09-04T17:48:36.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.154 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:36.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:36.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644536+0000", 2024-09-04T17:48:36.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:36.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:36.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644536+0000", 2024-09-04T17:48:36.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:36.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:36.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644536+0000", 2024-09-04T17:48:36.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7000000000000004e-07 2024-09-04T17:48:36.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:36.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644540+0000", 2024-09-04T17:48:36.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1609999999999999e-06 2024-09-04T17:48:36.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:36.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644552+0000", 2024-09-04T17:48:36.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2794e-05 2024-09-04T17:48:36.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.156 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:36.156 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644559+0000", 2024-09-04T17:48:36.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5010000000000001e-06 2024-09-04T17:48:36.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.156 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:36.156 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.671703+0000", 2024-09-04T17:48:36.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027144492999999999 2024-09-04T17:48:36.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.156 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:36.156 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.671779+0000", 2024-09-04T17:48:36.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5578e-05 2024-09-04T17:48:36.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:36.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.671896+0000", 2024-09-04T17:48:36.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000117327 2024-09-04T17:48:36.157 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:36.157 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:36.157 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:36.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.157 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94667 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 8257536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:36.157 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.644570+0000", 2024-09-04T17:48:36.157 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.424324308000003, 2024-09-04T17:48:36.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027556085000000001, 2024-09-04T17:48:36.158 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:36.158 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:36.158 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:36.158 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:36.158 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:36.158 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:36.158 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94667 2024-09-04T17:48:36.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.158 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:36.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:36.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644570+0000", 2024-09-04T17:48:36.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:36.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.159 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:36.159 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644570+0000", 2024-09-04T17:48:36.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:36.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.159 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:36.159 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644571+0000", 2024-09-04T17:48:36.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5199999999999997e-07 2024-09-04T17:48:36.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.159 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:36.160 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644574+0000", 2024-09-04T17:48:36.160 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3340000000000002e-06 2024-09-04T17:48:36.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.160 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:36.160 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644581+0000", 2024-09-04T17:48:36.160 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6579999999999996e-06 2024-09-04T17:48:36.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.160 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:36.160 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644589+0000", 2024-09-04T17:48:36.160 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9699999999999999e-06 2024-09-04T17:48:36.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.161 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.161 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:36.161 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.671925+0000", 2024-09-04T17:48:36.161 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027335911000000001 2024-09-04T17:48:36.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.161 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.161 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:36.161 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.671987+0000", 2024-09-04T17:48:36.161 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2198000000000003e-05 2024-09-04T17:48:36.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.161 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.161 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:36.161 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.672126+0000", 2024-09-04T17:48:36.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000139462 2024-09-04T17:48:36.162 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:36.162 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:36.162 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:36.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.162 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:96164 3.18 3:19aa90a8:::rbd_data.1134c4e2d26.0000000000000062:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123)", 2024-09-04T17:48:36.162 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:48:02.862015+0000", 2024-09-04T17:48:36.162 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 33.206879786000002, 2024-09-04T17:48:36.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034031039999999999, 2024-09-04T17:48:36.162 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:36.162 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:36.162 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:36.162 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:36.163 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:36.163 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:36.163 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 96164 2024-09-04T17:48:36.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.163 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:36.163 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.163 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:36.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.862015+0000", 2024-09-04T17:48:36.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:36.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.163 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.163 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:36.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.862015+0000", 2024-09-04T17:48:36.164 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:36.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.164 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:36.164 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.862018+0000", 2024-09-04T17:48:36.164 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1200000000000002e-06 2024-09-04T17:48:36.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.164 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:36.164 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.862032+0000", 2024-09-04T17:48:36.164 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4083000000000001e-05 2024-09-04T17:48:36.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.164 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:36.165 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.862064+0000", 2024-09-04T17:48:36.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2095999999999999e-05 2024-09-04T17:48:36.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.165 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:36.165 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.862081+0000", 2024-09-04T17:48:36.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6625000000000001e-05 2024-09-04T17:48:36.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.165 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:36.165 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.862232+0000", 2024-09-04T17:48:36.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015144600000000001 2024-09-04T17:48:36.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.166 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:36.166 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.862588+0000", 2024-09-04T17:48:36.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00035525999999999998 2024-09-04T17:48:36.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.166 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 7", 2024-09-04T17:48:36.166 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.867988+0000", 2024-09-04T17:48:36.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054005479999999998 2024-09-04T17:48:36.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.166 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:36.166 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.895763+0000", 2024-09-04T17:48:36.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027774983 2024-09-04T17:48:36.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:36.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.895877+0000", 2024-09-04T17:48:36.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000113607 2024-09-04T17:48:36.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:36.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.895975+0000", 2024-09-04T17:48:36.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7876999999999998e-05 2024-09-04T17:48:36.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:36.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:36.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:36.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.896046+0000", 2024-09-04T17:48:36.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1395000000000003e-05 2024-09-04T17:48:36.167 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:36.168 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:36.168 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:36.168 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:36.168 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:36.168 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:36.170 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-04T17:48:36.170 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:48:36.167+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:48:36.270 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:36.269+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:36.303 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:36.303 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:48:36.304 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:48:36.304 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:36.317 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-04T17:48:36.409 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:48:36.409+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:48:36.443 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 4, 5, 6, 7, 3] out_osds: [1] dead_osds: [] live_osds: [0, 1, 4, 5, 7, 3, 2, 6] 2024-09-04T17:48:36.443 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-04T17:48:36.443 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-09-04T17:48:36.443 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-04T17:48:36.443 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-04T17:48:36.443 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set filestore_inject_stall 3 2024-09-04T17:48:36.472 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:36.469+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:36.483 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:36.483 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:48:36.483 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:48:36.483 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:48:36.483 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:36.497 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-04T17:48:36.572 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:36.567+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:36.574 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:36.574 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-04T17:48:36.574 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:36.670 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:36.670 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:48:36.670 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:48:36.670 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:48:36.670 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.670 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "pg_update_log_missing(3.1f epoch 119/118 rep_tid 2838 entries 119'424 (0'0) error 3:f95bc674:::rbd_header.1134878eadad:head by client.4404.0:93468 0.000000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-04T17:48:36.670 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:58.898727+0000", 2024-09-04T17:48:36.671 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 37.768914336000002, 2024-09-04T17:48:36.671 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0072113999999999998, 2024-09-04T17:48:36.671 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:36.671 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:36.671 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:36.671 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:36.671 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.671 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:36.671 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.898727+0000", 2024-09-04T17:48:36.671 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:36.671 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.671 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.671 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:36.672 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.898727+0000", 2024-09-04T17:48:36.672 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:36.672 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.672 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.672 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:36.672 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.898728+0000", 2024-09-04T17:48:36.672 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.8200000000000001e-07 2024-09-04T17:48:36.672 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.672 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.672 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:36.672 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.898740+0000", 2024-09-04T17:48:36.672 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2186e-05 2024-09-04T17:48:36.672 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.673 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.673 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:36.673 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.898772+0000", 2024-09-04T17:48:36.673 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.2156000000000002e-05 2024-09-04T17:48:36.673 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.673 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.673 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:36.673 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.898796+0000", 2024-09-04T17:48:36.673 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4017999999999999e-05 2024-09-04T17:48:36.673 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.673 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.673 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:36.673 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.898966+0000", 2024-09-04T17:48:36.673 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000170185 2024-09-04T17:48:36.674 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.674 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.674 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:36.674 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.905939+0000", 2024-09-04T17:48:36.674 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0069724729999999999 2024-09-04T17:48:36.674 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:36.674 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:36.674 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:36.674 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.674 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.674 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:93469 3.1f e119/118 3:f95bc674:::rbd_header.1134878eadad:head v 119'425, mlcod=0'0)", 2024-09-04T17:48:36.674 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:58.900173+0000", 2024-09-04T17:48:36.674 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 37.767468547999997, 2024-09-04T17:48:36.675 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0059053930000000001, 2024-09-04T17:48:36.675 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:36.675 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:36.675 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:36.675 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:36.675 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.675 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:36.675 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.900173+0000", 2024-09-04T17:48:36.675 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:36.675 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.675 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.675 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:36.675 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.900173+0000", 2024-09-04T17:48:36.676 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:36.676 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.676 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.676 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:36.676 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.900174+0000", 2024-09-04T17:48:36.676 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.127e-06 2024-09-04T17:48:36.676 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.676 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.677 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:36.677 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.900181+0000", 2024-09-04T17:48:36.677 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.0099999999999998e-06 2024-09-04T17:48:36.677 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.677 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.677 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:36.677 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.900199+0000", 2024-09-04T17:48:36.677 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7876e-05 2024-09-04T17:48:36.677 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.677 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.677 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:36.677 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.900208+0000", 2024-09-04T17:48:36.677 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.4999999999999999e-06 2024-09-04T17:48:36.678 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.678 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.678 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:36.678 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.900310+0000", 2024-09-04T17:48:36.678 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00010202800000000001 2024-09-04T17:48:36.678 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.678 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.678 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:36.678 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.900353+0000", 2024-09-04T17:48:36.678 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.3213000000000001e-05 2024-09-04T17:48:36.678 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.678 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.679 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:36.679 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.905944+0000", 2024-09-04T17:48:36.679 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0055914900000000002 2024-09-04T17:48:36.679 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.679 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.679 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:36.679 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.906078+0000", 2024-09-04T17:48:36.679 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013414899999999999 2024-09-04T17:48:36.679 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:36.679 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:36.679 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:36.679 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.679 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.679 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "MOSDPGPush(3.19 119/118 [PushOp(3:9db25b11:::rbd_data.11341db94a06.0000000000000001:head, version: 78'254, data_included: [0~294912,458752~32768,1802240~65536,1966080~98304,3014656~294912], data_size: 786432, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:9db25b11:::rbd_data.11341db94a06.0000000000000001:head@78'254, size: 4096000, copy_subset: [(0, 4096000)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096000, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-04T17:48:36.680 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:59.229007+0000", 2024-09-04T17:48:36.680 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 37.438635054000002, 2024-09-04T17:48:36.680 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0049484100000000003, 2024-09-04T17:48:36.680 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:36.680 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:36.680 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:36.680 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:36.680 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.680 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:36.680 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.229007+0000", 2024-09-04T17:48:36.680 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:36.680 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.680 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.681 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:36.681 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.229007+0000", 2024-09-04T17:48:36.681 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:36.681 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.681 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.681 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:36.681 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.229008+0000", 2024-09-04T17:48:36.681 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.827e-06 2024-09-04T17:48:36.681 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.681 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.681 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:36.681 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.230334+0000", 2024-09-04T17:48:36.681 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0013259089999999999 2024-09-04T17:48:36.681 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.682 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.682 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:36.682 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.230382+0000", 2024-09-04T17:48:36.682 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.8168999999999999e-05 2024-09-04T17:48:36.682 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.682 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.682 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:36.682 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.230415+0000", 2024-09-04T17:48:36.682 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.2916000000000001e-05 2024-09-04T17:48:36.682 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.683 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.683 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:36.683 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.230676+0000", 2024-09-04T17:48:36.683 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00026107600000000001 2024-09-04T17:48:36.683 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.683 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.683 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:36.683 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.230719+0000", 2024-09-04T17:48:36.683 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2571999999999998e-05 2024-09-04T17:48:36.683 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.683 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.683 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:36.683 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.233955+0000", 2024-09-04T17:48:36.683 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.003235941 2024-09-04T17:48:36.683 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:36.684 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:36.684 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:36.684 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.684 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.684 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "MOSDPGPush(3.10 119/118 [PushOp(3:0c2d06e5:::rbd_data.113410ddd206.0000000000000000:18, version: 68'308, data_included: [0~884736,950272~65536,1081344~65536,1212416~32768], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:0c2d06e5:::rbd_data.113410ddd206.0000000000000000:18@68'308, size: 7471104, copy_subset: [(0, 7471104)], clone_subset: {}, snapset: 18=[]:{18: [18]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1245184, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-04T17:48:36.684 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:59.426517+0000", 2024-09-04T17:48:36.684 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 37.241124470000003, 2024-09-04T17:48:36.684 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0064985670000000002, 2024-09-04T17:48:36.684 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:36.684 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:36.684 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:36.684 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:36.685 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.685 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:36.685 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.426517+0000", 2024-09-04T17:48:36.685 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:36.685 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.685 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.685 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:36.685 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.426517+0000", 2024-09-04T17:48:36.686 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:36.686 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.686 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.686 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:36.686 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.426754+0000", 2024-09-04T17:48:36.686 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00023681799999999999 2024-09-04T17:48:36.686 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.686 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.686 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:36.686 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.429886+0000", 2024-09-04T17:48:36.686 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0031323789999999998 2024-09-04T17:48:36.686 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.686 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.687 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:36.687 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.429940+0000", 2024-09-04T17:48:36.687 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4104999999999998e-05 2024-09-04T17:48:36.687 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.687 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.687 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:36.687 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.429975+0000", 2024-09-04T17:48:36.687 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4289000000000003e-05 2024-09-04T17:48:36.687 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.687 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.687 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:36.687 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.430229+0000", 2024-09-04T17:48:36.687 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00025465900000000002 2024-09-04T17:48:36.688 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.688 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.688 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:36.688 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.430350+0000", 2024-09-04T17:48:36.688 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000120992 2024-09-04T17:48:36.688 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.688 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.688 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:36.688 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.433016+0000", 2024-09-04T17:48:36.688 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0026653250000000001 2024-09-04T17:48:36.688 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:36.688 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:36.688 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:36.688 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.689 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.689 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "MOSDPGPush(3.10 119/118 [PushOp(3:0c2d06e5:::rbd_data.113410ddd206.0000000000000000:18, version: 68'308, data_included: [1245184~32768,1343488~65536,1474560~65536,1605632~65536,1736704~65536,1867776~65536,1998848~65536,2129920~65536,2260992~65536,2392064~65536,2523136~65536,2654208~65536,2785280~65536,2916352~65536,3047424~65536,3154405~89627,3309568~8677], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:0c2d06e5:::rbd_data.113410ddd206.0000000000000000:18@68'308, size: 7471104, copy_subset: [(0, 7471104)], clone_subset: {}, snapset: 18=[]:{18: [18]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3318245, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1245184, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-04T17:48:36.689 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:59.434957+0000", 2024-09-04T17:48:36.689 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 37.232684847000002, 2024-09-04T17:48:36.689 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0061603969999999997, 2024-09-04T17:48:36.689 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:36.689 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:36.689 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:36.689 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:36.689 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.689 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:36.689 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.434957+0000", 2024-09-04T17:48:36.690 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:36.690 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.690 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.690 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:36.690 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.434957+0000", 2024-09-04T17:48:36.690 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:36.690 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.690 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.690 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:36.690 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.434958+0000", 2024-09-04T17:48:36.690 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4339999999999999e-06 2024-09-04T17:48:36.690 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.690 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.691 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:36.691 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.436921+0000", 2024-09-04T17:48:36.691 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0019629460000000001 2024-09-04T17:48:36.691 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.691 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.691 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:36.691 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.436973+0000", 2024-09-04T17:48:36.691 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1492000000000001e-05 2024-09-04T17:48:36.691 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.691 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.691 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:36.691 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.437041+0000", 2024-09-04T17:48:36.691 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.8275000000000006e-05 2024-09-04T17:48:36.692 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.692 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.692 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:36.692 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.437312+0000", 2024-09-04T17:48:36.692 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00027156800000000003 2024-09-04T17:48:36.692 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.692 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.692 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:36.692 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.437357+0000", 2024-09-04T17:48:36.692 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4079000000000003e-05 2024-09-04T17:48:36.692 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.692 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.693 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:36.693 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.441117+0000", 2024-09-04T17:48:36.693 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0037606029999999999 2024-09-04T17:48:36.693 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:36.693 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:36.693 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:36.693 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.693 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.693 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "MOSDPGPush(3.10 119/118 [PushOp(3:08a6a427:::rbd_data.11341db94a06.0000000000000002:head, version: 78'342, data_included: [0~271649,458752~32768,1802240~65536,1966080~98304,3014656~294912], data_size: 763169, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:08a6a427:::rbd_data.11341db94a06.0000000000000002:head@78'342, size: 4096000, copy_subset: [(0, 4096000)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096000, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-04T17:48:36.693 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:59.471247+0000", 2024-09-04T17:48:36.693 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 37.196394603000002, 2024-09-04T17:48:36.693 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0048034530000000001, 2024-09-04T17:48:36.693 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:36.694 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:36.694 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:36.694 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:36.694 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.694 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:36.694 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.471247+0000", 2024-09-04T17:48:36.694 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:36.694 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.694 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.694 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:36.694 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.471247+0000", 2024-09-04T17:48:36.694 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:36.695 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.695 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.695 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:36.695 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.471249+0000", 2024-09-04T17:48:36.695 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9e-06 2024-09-04T17:48:36.695 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.695 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.695 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:36.695 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.472560+0000", 2024-09-04T17:48:36.695 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0013114190000000001 2024-09-04T17:48:36.695 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.695 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.695 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:36.696 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.472610+0000", 2024-09-04T17:48:36.696 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9916000000000001e-05 2024-09-04T17:48:36.696 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.696 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.696 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:36.696 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.472692+0000", 2024-09-04T17:48:36.696 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.1384000000000005e-05 2024-09-04T17:48:36.696 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.696 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.696 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:36.696 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.472912+0000", 2024-09-04T17:48:36.696 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000220201 2024-09-04T17:48:36.696 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.697 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.697 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:36.697 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.472983+0000", 2024-09-04T17:48:36.697 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1137999999999996e-05 2024-09-04T17:48:36.697 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.697 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.697 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:36.697 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.476050+0000", 2024-09-04T17:48:36.697 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0030674949999999999 2024-09-04T17:48:36.697 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:36.697 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:36.697 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:36.698 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.698 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.698 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:96178 3.1e 3:7c9d4526:::rbd_data.1134c4e2d26.0000000000000026:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123)", 2024-09-04T17:48:36.698 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:02.985763+0000", 2024-09-04T17:48:36.698 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 33.681878699000002, 2024-09-04T17:48:36.698 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.020686758999999999, 2024-09-04T17:48:36.698 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:36.698 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:36.698 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:36.698 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:36.698 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:36.698 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:36.698 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 96178 2024-09-04T17:48:36.699 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.699 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:36.699 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.699 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:36.699 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:02.985763+0000", 2024-09-04T17:48:36.699 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:36.699 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.699 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.699 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:36.699 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:02.985763+0000", 2024-09-04T17:48:36.699 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:36.700 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.700 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.700 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:36.700 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:02.985766+0000", 2024-09-04T17:48:36.700 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.2990000000000001e-06 2024-09-04T17:48:36.700 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.700 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.700 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:36.700 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:02.985783+0000", 2024-09-04T17:48:36.700 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.645e-05 2024-09-04T17:48:36.700 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.700 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.700 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:36.701 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:02.985818+0000", 2024-09-04T17:48:36.701 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.5459000000000001e-05 2024-09-04T17:48:36.701 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.701 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.701 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:36.701 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:02.985835+0000", 2024-09-04T17:48:36.701 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.715e-05 2024-09-04T17:48:36.701 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.701 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.701 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:36.701 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:02.985953+0000", 2024-09-04T17:48:36.701 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000117351 2024-09-04T17:48:36.702 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.702 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.702 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:36.702 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:02.986207+0000", 2024-09-04T17:48:36.702 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00025483 2024-09-04T17:48:36.702 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.702 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.702 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 2", 2024-09-04T17:48:36.702 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:02.990784+0000", 2024-09-04T17:48:36.702 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0045763219999999999 2024-09-04T17:48:36.702 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.703 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.703 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:48:36.703 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.006139+0000", 2024-09-04T17:48:36.703 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.015355526 2024-09-04T17:48:36.703 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.703 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.703 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:36.703 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.006225+0000", 2024-09-04T17:48:36.703 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.6003999999999999e-05 2024-09-04T17:48:36.703 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.703 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.704 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:36.704 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.006371+0000", 2024-09-04T17:48:36.704 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000146052 2024-09-04T17:48:36.704 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.704 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.704 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:36.704 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.006450+0000", 2024-09-04T17:48:36.704 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.8316e-05 2024-09-04T17:48:36.704 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:36.704 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:36.704 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:36.704 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.704 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.705 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:96197 3.10 3:08a64171:::rbd_data.1134c4e2d26.00000000000000c3:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123)", 2024-09-04T17:48:36.705 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:03.049064+0000", 2024-09-04T17:48:36.705 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 33.618577727999998, 2024-09-04T17:48:36.705 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0053841139999999997, 2024-09-04T17:48:36.705 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:36.705 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:36.705 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:36.705 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:36.705 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:36.705 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:36.705 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 96197 2024-09-04T17:48:36.705 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.705 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:36.706 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.706 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:36.706 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.049064+0000", 2024-09-04T17:48:36.706 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:36.706 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.706 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.706 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:36.706 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.049064+0000", 2024-09-04T17:48:36.706 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:36.706 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.706 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.706 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:36.706 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.049066+0000", 2024-09-04T17:48:36.707 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.3360000000000002e-06 2024-09-04T17:48:36.707 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.707 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.707 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:36.707 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.049075+0000", 2024-09-04T17:48:36.707 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.2210000000000003e-06 2024-09-04T17:48:36.707 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.707 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.707 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:36.707 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.049097+0000", 2024-09-04T17:48:36.707 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1798e-05 2024-09-04T17:48:36.707 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.707 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.707 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:36.708 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.049107+0000", 2024-09-04T17:48:36.708 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0200999999999999e-05 2024-09-04T17:48:36.708 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.708 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.708 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:36.708 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.049183+0000", 2024-09-04T17:48:36.708 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.6079000000000001e-05 2024-09-04T17:48:36.708 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.708 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.708 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:36.708 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.049276+0000", 2024-09-04T17:48:36.708 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.2911000000000005e-05 2024-09-04T17:48:36.709 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.709 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.709 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 0", 2024-09-04T17:48:36.709 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.050313+0000", 2024-09-04T17:48:36.709 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.001036705 2024-09-04T17:48:36.709 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.709 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.709 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:48:36.709 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.054220+0000", 2024-09-04T17:48:36.709 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.003906779 2024-09-04T17:48:36.709 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.710 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.710 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:36.710 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.054297+0000", 2024-09-04T17:48:36.710 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.6816000000000005e-05 2024-09-04T17:48:36.710 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.710 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.710 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:36.710 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.054374+0000", 2024-09-04T17:48:36.710 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.7490000000000005e-05 2024-09-04T17:48:36.710 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.710 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.710 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:36.710 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.054448+0000", 2024-09-04T17:48:36.711 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3777999999999997e-05 2024-09-04T17:48:36.711 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:36.711 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:36.711 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:36.711 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.711 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.711 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:96267 3.1e 3:7a1dfe29:::rbd_header.1134c4e2d26:head [watch unwatch cookie 140018753450144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123)", 2024-09-04T17:48:36.711 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:03.482624+0000", 2024-09-04T17:48:36.711 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 33.185017397000003, 2024-09-04T17:48:36.711 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.006866853, 2024-09-04T17:48:36.711 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:36.711 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:36.712 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:36.712 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:36.712 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:36.712 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:36.712 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 96267 2024-09-04T17:48:36.712 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.712 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:36.712 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.712 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:36.712 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.482624+0000", 2024-09-04T17:48:36.712 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:36.712 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.712 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.713 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:36.713 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.482624+0000", 2024-09-04T17:48:36.713 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:36.713 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.713 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.713 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:36.713 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.482626+0000", 2024-09-04T17:48:36.713 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8360000000000001e-06 2024-09-04T17:48:36.713 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.713 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.713 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:36.714 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.482635+0000", 2024-09-04T17:48:36.714 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9099999999999994e-06 2024-09-04T17:48:36.714 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.714 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.714 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:36.714 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.482658+0000", 2024-09-04T17:48:36.714 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.3186000000000001e-05 2024-09-04T17:48:36.714 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.714 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.714 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:36.714 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.482677+0000", 2024-09-04T17:48:36.714 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8396000000000001e-05 2024-09-04T17:48:36.714 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.715 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.715 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:36.715 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.482849+0000", 2024-09-04T17:48:36.715 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000172936 2024-09-04T17:48:36.715 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.715 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.715 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:36.715 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.482987+0000", 2024-09-04T17:48:36.715 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013707699999999999 2024-09-04T17:48:36.715 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.715 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.715 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 2", 2024-09-04T17:48:36.715 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.483222+0000", 2024-09-04T17:48:36.715 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00023545 2024-09-04T17:48:36.716 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.716 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.716 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:48:36.716 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.484356+0000", 2024-09-04T17:48:36.716 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.001134049 2024-09-04T17:48:36.716 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.716 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.716 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:36.716 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.489303+0000", 2024-09-04T17:48:36.716 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0049472910000000004 2024-09-04T17:48:36.716 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.716 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.716 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:36.717 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.489396+0000", 2024-09-04T17:48:36.717 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.2638000000000003e-05 2024-09-04T17:48:36.717 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.717 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.717 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:36.717 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.489491+0000", 2024-09-04T17:48:36.717 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5084e-05 2024-09-04T17:48:36.717 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:36.717 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:36.717 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:36.717 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.717 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.718 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:96360 3.1e 3:7c9d4526:::rbd_data.1134c4e2d26.0000000000000026:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123)", 2024-09-04T17:48:36.718 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:03.603245+0000", 2024-09-04T17:48:36.718 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 33.064396293000001, 2024-09-04T17:48:36.718 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0074703160000000003, 2024-09-04T17:48:36.718 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:36.718 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:36.718 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:36.718 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:36.718 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:36.718 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:36.718 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 96360 2024-09-04T17:48:36.718 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.718 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:36.719 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.719 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:36.719 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.603245+0000", 2024-09-04T17:48:36.719 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:36.719 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.719 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.719 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:36.719 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.603245+0000", 2024-09-04T17:48:36.719 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:36.719 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.719 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.719 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:36.719 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.603248+0000", 2024-09-04T17:48:36.719 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.277e-06 2024-09-04T17:48:36.720 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.720 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.720 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:36.720 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.603259+0000", 2024-09-04T17:48:36.720 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1083e-05 2024-09-04T17:48:36.720 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.720 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.720 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:36.720 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.603295+0000", 2024-09-04T17:48:36.720 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.5892999999999997e-05 2024-09-04T17:48:36.720 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.720 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.721 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:36.721 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.603315+0000", 2024-09-04T17:48:36.721 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0953000000000001e-05 2024-09-04T17:48:36.721 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.721 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.721 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:36.721 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.603486+0000", 2024-09-04T17:48:36.721 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00017011199999999999 2024-09-04T17:48:36.721 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.721 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.721 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:36.721 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.604960+0000", 2024-09-04T17:48:36.721 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0014743340000000001 2024-09-04T17:48:36.722 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.722 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.722 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 2", 2024-09-04T17:48:36.722 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.605232+0000", 2024-09-04T17:48:36.722 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00027236300000000002 2024-09-04T17:48:36.722 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.722 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.722 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:48:36.722 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.610322+0000", 2024-09-04T17:48:36.722 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0050901849999999997 2024-09-04T17:48:36.722 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.722 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.723 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:36.723 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.610464+0000", 2024-09-04T17:48:36.723 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014186599999999999 2024-09-04T17:48:36.723 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.723 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.723 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:36.723 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.610607+0000", 2024-09-04T17:48:36.723 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000142423 2024-09-04T17:48:36.723 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.723 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.723 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:36.723 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.610716+0000", 2024-09-04T17:48:36.723 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00010882699999999999 2024-09-04T17:48:36.724 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:36.724 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:36.724 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:36.724 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.724 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.724 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:96520 3.2 e124/121 3:4549cdb4:::rbd_data.1134878eadad.0000000000000001:head v 124'509, mlcod=123'507)", 2024-09-04T17:48:36.724 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:04.014285+0000", 2024-09-04T17:48:36.724 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 32.653356373999998, 2024-09-04T17:48:36.724 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0066183220000000003, 2024-09-04T17:48:36.724 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:36.724 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:36.724 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:36.724 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:36.725 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.725 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:36.725 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:04.014285+0000", 2024-09-04T17:48:36.725 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:36.725 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.725 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.725 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:36.725 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:04.014285+0000", 2024-09-04T17:48:36.725 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:36.725 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.725 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.725 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:36.725 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:04.014287+0000", 2024-09-04T17:48:36.726 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1959999999999998e-06 2024-09-04T17:48:36.726 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.726 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.726 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:36.726 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:04.014332+0000", 2024-09-04T17:48:36.726 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4196000000000002e-05 2024-09-04T17:48:36.726 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.726 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.726 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:36.726 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:04.014374+0000", 2024-09-04T17:48:36.726 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2072e-05 2024-09-04T17:48:36.726 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.726 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.727 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:36.727 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:04.014394+0000", 2024-09-04T17:48:36.727 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0146999999999999e-05 2024-09-04T17:48:36.727 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.727 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.727 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:36.727 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:04.014575+0000", 2024-09-04T17:48:36.727 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000181132 2024-09-04T17:48:36.727 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.727 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.727 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:36.727 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:04.014611+0000", 2024-09-04T17:48:36.727 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.5815999999999997e-05 2024-09-04T17:48:36.728 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.728 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.728 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:36.728 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:04.020809+0000", 2024-09-04T17:48:36.728 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0061983159999999997 2024-09-04T17:48:36.728 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.728 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.728 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:36.728 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:04.020904+0000", 2024-09-04T17:48:36.728 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4446999999999999e-05 2024-09-04T17:48:36.728 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:36.728 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:36.729 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:36.729 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.729 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.729 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(osd.2.0:369 3.0 e138/121 3:04fb2888:::rbd_data.11342e6808f0.0000000000000009:24 v 138'380, mlcod=135'377)", 2024-09-04T17:48:36.729 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:17.851779+0000", 2024-09-04T17:48:36.729 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 18.815862461999998, 2024-09-04T17:48:36.729 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0056329400000000003, 2024-09-04T17:48:36.729 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:36.729 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:36.729 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:36.729 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:36.729 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.730 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:36.730 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.851779+0000", 2024-09-04T17:48:36.730 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:36.730 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.730 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.730 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:36.730 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.851779+0000", 2024-09-04T17:48:36.730 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:36.730 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.730 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.730 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:36.730 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.851780+0000", 2024-09-04T17:48:36.730 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2399999999999998e-07 2024-09-04T17:48:36.731 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.731 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.731 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:36.731 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.851790+0000", 2024-09-04T17:48:36.731 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0399e-05 2024-09-04T17:48:36.731 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.731 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.731 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:36.731 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.851809+0000", 2024-09-04T17:48:36.731 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9228e-05 2024-09-04T17:48:36.731 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.731 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.731 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:36.732 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.851820+0000", 2024-09-04T17:48:36.732 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0814e-05 2024-09-04T17:48:36.732 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.732 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.732 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:36.732 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.851902+0000", 2024-09-04T17:48:36.732 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.2154e-05 2024-09-04T17:48:36.732 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.732 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.732 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:36.732 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.851930+0000", 2024-09-04T17:48:36.732 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.7577999999999999e-05 2024-09-04T17:48:36.732 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.732 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.733 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:36.733 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.857363+0000", 2024-09-04T17:48:36.733 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0054331989999999997 2024-09-04T17:48:36.733 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.733 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.733 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:36.733 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.857412+0000", 2024-09-04T17:48:36.733 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9044e-05 2024-09-04T17:48:36.733 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:36.733 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:36.733 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:36.733 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.733 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.734 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(osd.5.0:3167 3.2 e138/121 3:460b9dee:::rbd_data.11342e6808f0.000000000000000d:24 v 138'560, mlcod=135'557)", 2024-09-04T17:48:36.734 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:17.857279+0000", 2024-09-04T17:48:36.734 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 18.810362877999999, 2024-09-04T17:48:36.734 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0068482229999999996, 2024-09-04T17:48:36.734 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:36.734 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:36.734 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:36.734 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:36.734 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.734 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:36.734 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.857279+0000", 2024-09-04T17:48:36.734 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:36.734 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.734 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.735 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:36.735 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.857279+0000", 2024-09-04T17:48:36.735 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:36.735 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.735 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.735 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:36.735 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.857279+0000", 2024-09-04T17:48:36.735 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7300000000000001e-07 2024-09-04T17:48:36.735 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.735 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.735 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:36.735 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.857286+0000", 2024-09-04T17:48:36.736 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.4289999999999998e-06 2024-09-04T17:48:36.736 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.736 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.736 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:36.736 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.857300+0000", 2024-09-04T17:48:36.736 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4582e-05 2024-09-04T17:48:36.736 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.736 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.736 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:36.736 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.857315+0000", 2024-09-04T17:48:36.736 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5173e-05 2024-09-04T17:48:36.736 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.736 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.737 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:36.737 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.857388+0000", 2024-09-04T17:48:36.737 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2365000000000003e-05 2024-09-04T17:48:36.737 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.737 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.737 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:36.737 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.857414+0000", 2024-09-04T17:48:36.737 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6248999999999999e-05 2024-09-04T17:48:36.737 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.737 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.737 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:36.737 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.864076+0000", 2024-09-04T17:48:36.737 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0066619490000000003 2024-09-04T17:48:36.738 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.738 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.738 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:36.738 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.864127+0000", 2024-09-04T17:48:36.738 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1002999999999999e-05 2024-09-04T17:48:36.738 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:36.738 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:36.738 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:36.738 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.738 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.738 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:98345 3.a 3:55a1c81d:::rbd_header.113481c24fbf:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139)", 2024-09-04T17:48:36.738 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:18.971074+0000", 2024-09-04T17:48:36.738 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.696567599000002, 2024-09-04T17:48:36.739 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.017562632000000002, 2024-09-04T17:48:36.739 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:36.739 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:36.739 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:36.739 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:36.739 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:36.739 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:36.739 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 98345 2024-09-04T17:48:36.739 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.739 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:36.739 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.739 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:36.739 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.971074+0000", 2024-09-04T17:48:36.740 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:36.740 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.740 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.740 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:36.740 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.971074+0000", 2024-09-04T17:48:36.740 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:36.740 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.740 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.740 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:36.740 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.971078+0000", 2024-09-04T17:48:36.740 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.1969999999999998e-06 2024-09-04T17:48:36.740 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.740 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.740 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:36.741 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.971096+0000", 2024-09-04T17:48:36.741 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7898999999999999e-05 2024-09-04T17:48:36.741 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.741 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.741 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:36.741 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.971140+0000", 2024-09-04T17:48:36.741 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.3883e-05 2024-09-04T17:48:36.741 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.741 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.741 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:36.741 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.971165+0000", 2024-09-04T17:48:36.741 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5321999999999999e-05 2024-09-04T17:48:36.741 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.742 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.742 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:36.742 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.971346+0000", 2024-09-04T17:48:36.742 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000180784 2024-09-04T17:48:36.742 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.742 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.742 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op must wait for map", 2024-09-04T17:48:36.742 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.971369+0000", 2024-09-04T17:48:36.742 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.3286e-05 2024-09-04T17:48:36.742 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.742 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.742 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:36.742 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.977475+0000", 2024-09-04T17:48:36.743 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0061055459999999999 2024-09-04T17:48:36.743 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.743 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.743 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:36.743 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.977665+0000", 2024-09-04T17:48:36.743 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019043599999999999 2024-09-04T17:48:36.743 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.743 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.743 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 4", 2024-09-04T17:48:36.743 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.978586+0000", 2024-09-04T17:48:36.743 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000920597 2024-09-04T17:48:36.743 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.743 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.744 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:48:36.744 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.982469+0000", 2024-09-04T17:48:36.744 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0038834719999999998 2024-09-04T17:48:36.744 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.744 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.744 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:36.744 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.988390+0000", 2024-09-04T17:48:36.744 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0059206479999999997 2024-09-04T17:48:36.744 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.744 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.744 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:36.744 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.988546+0000", 2024-09-04T17:48:36.744 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00015548 2024-09-04T17:48:36.745 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.745 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.745 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:36.745 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.988637+0000", 2024-09-04T17:48:36.745 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1082000000000006e-05 2024-09-04T17:48:36.745 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:36.745 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:36.745 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:36.745 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.745 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.745 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:98921 3.c e142/122 3:37678a48:::rbd_data.113460fdfacd.0000000000000006:head v 142'572, mlcod=137'569)", 2024-09-04T17:48:36.745 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:22.131110+0000", 2024-09-04T17:48:36.745 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 14.536531697999999, 2024-09-04T17:48:36.746 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0055872880000000002, 2024-09-04T17:48:36.746 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:36.746 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:36.746 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:36.746 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:36.746 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.746 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:36.746 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.131110+0000", 2024-09-04T17:48:36.746 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:36.746 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.746 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.746 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:36.747 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.131110+0000", 2024-09-04T17:48:36.747 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:36.747 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.747 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.747 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:36.747 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.131110+0000", 2024-09-04T17:48:36.747 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7e-07 2024-09-04T17:48:36.747 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.747 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.747 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:36.747 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.131120+0000", 2024-09-04T17:48:36.747 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.2499999999999995e-06 2024-09-04T17:48:36.747 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.747 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.748 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:36.748 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.131146+0000", 2024-09-04T17:48:36.748 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6461999999999998e-05 2024-09-04T17:48:36.748 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.748 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.748 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:36.748 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.131157+0000", 2024-09-04T17:48:36.748 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1522000000000001e-05 2024-09-04T17:48:36.748 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.748 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.748 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:36.748 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.131323+0000", 2024-09-04T17:48:36.748 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00016558499999999999 2024-09-04T17:48:36.749 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.749 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.749 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:36.749 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.131351+0000", 2024-09-04T17:48:36.749 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.7494000000000001e-05 2024-09-04T17:48:36.749 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.749 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.749 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:36.749 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.136648+0000", 2024-09-04T17:48:36.749 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0052975330000000001 2024-09-04T17:48:36.749 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.749 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.750 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:36.750 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.136697+0000", 2024-09-04T17:48:36.750 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9072e-05 2024-09-04T17:48:36.750 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:36.750 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:36.750 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:36.750 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.750 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.750 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:98922 3.0 e142/121 3:00fa19d2:::rbd_data.113460fdfacd.0000000000000007:head v 142'381, mlcod=135'378)", 2024-09-04T17:48:36.750 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:22.132316+0000", 2024-09-04T17:48:36.750 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 14.535325917, 2024-09-04T17:48:36.750 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0070776679999999996, 2024-09-04T17:48:36.750 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:36.751 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:36.751 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:36.751 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:36.751 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.751 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:36.751 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.132316+0000", 2024-09-04T17:48:36.751 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:36.751 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.751 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.751 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:36.751 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.132316+0000", 2024-09-04T17:48:36.751 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:36.751 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.752 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.752 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:36.752 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.132316+0000", 2024-09-04T17:48:36.752 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2300000000000002e-07 2024-09-04T17:48:36.752 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.752 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.752 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:36.752 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.132322+0000", 2024-09-04T17:48:36.752 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.48e-06 2024-09-04T17:48:36.752 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.752 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.752 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:36.752 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.132335+0000", 2024-09-04T17:48:36.752 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3040999999999999e-05 2024-09-04T17:48:36.753 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.753 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.753 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:36.753 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.132344+0000", 2024-09-04T17:48:36.753 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9989999999999997e-06 2024-09-04T17:48:36.753 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.753 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.753 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:36.753 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.132453+0000", 2024-09-04T17:48:36.753 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000109615 2024-09-04T17:48:36.753 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.753 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.754 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:36.754 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.132481+0000", 2024-09-04T17:48:36.754 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.777e-05 2024-09-04T17:48:36.754 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.754 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.754 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:36.754 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.139271+0000", 2024-09-04T17:48:36.754 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0067898180000000004 2024-09-04T17:48:36.754 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.754 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.754 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:36.754 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.139393+0000", 2024-09-04T17:48:36.754 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00012252199999999999 2024-09-04T17:48:36.755 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:36.755 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:36.755 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:36.755 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.755 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.755 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:99605 3.a 3:50de26c1:::rbd_data.11346a2b0bc9.0000000000000008:head [delete] snapc 1c=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-04T17:48:36.755 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:27.983388+0000", 2024-09-04T17:48:36.755 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 8.6842537580000005, 2024-09-04T17:48:36.755 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0073493309999999997, 2024-09-04T17:48:36.755 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:36.755 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:36.755 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:36.755 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:36.756 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:36.756 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:36.756 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 99605 2024-09-04T17:48:36.756 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.756 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:36.756 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.756 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:36.756 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:27.983388+0000", 2024-09-04T17:48:36.756 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:36.756 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.756 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.756 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:36.757 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:27.983388+0000", 2024-09-04T17:48:36.757 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:36.757 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.757 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.757 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:36.757 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:27.983392+0000", 2024-09-04T17:48:36.757 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.9770000000000002e-06 2024-09-04T17:48:36.757 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.757 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.757 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:36.757 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:27.983413+0000", 2024-09-04T17:48:36.757 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1075999999999999e-05 2024-09-04T17:48:36.757 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.758 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.758 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:36.758 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:27.983457+0000", 2024-09-04T17:48:36.758 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.3943999999999998e-05 2024-09-04T17:48:36.758 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.758 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.758 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:36.758 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:27.983482+0000", 2024-09-04T17:48:36.758 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5602999999999999e-05 2024-09-04T17:48:36.758 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.758 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.758 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:36.758 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:27.983671+0000", 2024-09-04T17:48:36.758 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00018823399999999999 2024-09-04T17:48:36.759 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.759 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.759 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:36.759 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:27.984069+0000", 2024-09-04T17:48:36.759 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000398639 2024-09-04T17:48:36.759 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.759 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.759 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 4", 2024-09-04T17:48:36.759 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:27.984378+0000", 2024-09-04T17:48:36.759 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000308392 2024-09-04T17:48:36.759 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.759 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.759 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:48:36.760 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:27.987068+0000", 2024-09-04T17:48:36.760 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.002690764 2024-09-04T17:48:36.760 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.760 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.760 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:36.760 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:27.990452+0000", 2024-09-04T17:48:36.760 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0033831080000000001 2024-09-04T17:48:36.760 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.760 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.760 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:36.760 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:27.990603+0000", 2024-09-04T17:48:36.760 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00015124 2024-09-04T17:48:36.760 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.761 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.761 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:36.761 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:27.990737+0000", 2024-09-04T17:48:36.761 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013435399999999999 2024-09-04T17:48:36.761 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:36.761 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:36.761 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:36.761 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.761 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.761 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:99857 3.10 3:0f7166b0:::rbd_header.1134968ccb73:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e153)", 2024-09-04T17:48:36.761 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:32.841823+0000", 2024-09-04T17:48:36.761 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 3.825818376, 2024-09-04T17:48:36.761 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.014014146999999999, 2024-09-04T17:48:36.762 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:36.762 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:36.762 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:36.762 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:36.762 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:36.762 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:36.762 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 99857 2024-09-04T17:48:36.762 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.762 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:36.762 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.762 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:36.762 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.841823+0000", 2024-09-04T17:48:36.762 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:36.763 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.763 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.763 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:36.763 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.841823+0000", 2024-09-04T17:48:36.763 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:36.763 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.763 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.763 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:36.763 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.841827+0000", 2024-09-04T17:48:36.763 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.3699999999999999e-06 2024-09-04T17:48:36.763 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.763 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.763 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:36.764 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.841858+0000", 2024-09-04T17:48:36.764 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.1795999999999998e-05 2024-09-04T17:48:36.764 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.764 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.764 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:36.764 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.841906+0000", 2024-09-04T17:48:36.764 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7737000000000001e-05 2024-09-04T17:48:36.764 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.764 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.764 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:36.764 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.841930+0000", 2024-09-04T17:48:36.764 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4272000000000001e-05 2024-09-04T17:48:36.764 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.765 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.765 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:36.765 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.842177+0000", 2024-09-04T17:48:36.765 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024632300000000001 2024-09-04T17:48:36.765 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.765 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.765 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op must wait for map", 2024-09-04T17:48:36.765 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.842194+0000", 2024-09-04T17:48:36.765 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7606000000000001e-05 2024-09-04T17:48:36.765 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.765 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.765 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:36.765 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.850210+0000", 2024-09-04T17:48:36.765 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0080154179999999998 2024-09-04T17:48:36.766 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.766 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.766 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:36.766 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.850494+0000", 2024-09-04T17:48:36.766 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00028466100000000002 2024-09-04T17:48:36.766 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.766 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.766 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 0", 2024-09-04T17:48:36.766 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.851786+0000", 2024-09-04T17:48:36.766 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.001291468 2024-09-04T17:48:36.766 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.766 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.766 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:48:36.767 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.855374+0000", 2024-09-04T17:48:36.767 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0035884760000000002 2024-09-04T17:48:36.767 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.767 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.767 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:36.767 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.855504+0000", 2024-09-04T17:48:36.767 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00012924799999999999 2024-09-04T17:48:36.767 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.767 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.767 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:36.767 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.855704+0000", 2024-09-04T17:48:36.767 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000200764 2024-09-04T17:48:36.767 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.768 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.768 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:36.768 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.855837+0000", 2024-09-04T17:48:36.768 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000133008 2024-09-04T17:48:36.768 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:36.768 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:36.768 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:36.768 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.768 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.768 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:99861 3.10 3:0f7166b0:::rbd_header.1134968ccb73:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e153)", 2024-09-04T17:48:36.768 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:32.872359+0000", 2024-09-04T17:48:36.768 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 3.795282518, 2024-09-04T17:48:36.768 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.006863063, 2024-09-04T17:48:36.768 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:36.769 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:36.769 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:36.769 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:36.769 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:36.769 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:36.769 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 99861 2024-09-04T17:48:36.769 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.769 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:36.769 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.769 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:36.769 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.872359+0000", 2024-09-04T17:48:36.769 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:36.769 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.770 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.770 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:36.770 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.872359+0000", 2024-09-04T17:48:36.770 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:36.770 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.770 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.770 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:36.770 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.872363+0000", 2024-09-04T17:48:36.770 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.8260000000000003e-06 2024-09-04T17:48:36.770 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.770 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.770 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:36.770 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.872382+0000", 2024-09-04T17:48:36.770 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9318999999999999e-05 2024-09-04T17:48:36.771 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.771 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.771 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:36.771 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.872426+0000", 2024-09-04T17:48:36.771 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4038000000000001e-05 2024-09-04T17:48:36.771 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.771 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.771 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:36.771 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.872452+0000", 2024-09-04T17:48:36.771 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5479000000000001e-05 2024-09-04T17:48:36.771 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.771 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.771 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:36.772 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.872647+0000", 2024-09-04T17:48:36.772 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019568799999999999 2024-09-04T17:48:36.772 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.772 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.772 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:36.772 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.872894+0000", 2024-09-04T17:48:36.772 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000246226 2024-09-04T17:48:36.772 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.772 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.772 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 0", 2024-09-04T17:48:36.772 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.873410+0000", 2024-09-04T17:48:36.772 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00051657000000000005 2024-09-04T17:48:36.772 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.773 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.773 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:48:36.773 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.876401+0000", 2024-09-04T17:48:36.773 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.002990983 2024-09-04T17:48:36.773 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.773 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.773 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:36.773 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.878957+0000", 2024-09-04T17:48:36.773 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.002556039 2024-09-04T17:48:36.773 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.773 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.773 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:36.773 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.879109+0000", 2024-09-04T17:48:36.774 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00015156999999999999 2024-09-04T17:48:36.774 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.774 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.774 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:36.774 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.879222+0000", 2024-09-04T17:48:36.774 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000113325 2024-09-04T17:48:36.774 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:36.774 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:36.774 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:36.774 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.774 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.774 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:100488 3.1e e156/149 3:7e3581b1:::rbd_data.11343e517879.0000000000000008:head v 156'625, mlcod=151'621)", 2024-09-04T17:48:36.774 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:36.026379+0000", 2024-09-04T17:48:36.774 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.64126269199999997, 2024-09-04T17:48:36.775 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0076449719999999999, 2024-09-04T17:48:36.775 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:36.775 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:36.775 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:36.775 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:36.775 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.775 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:36.775 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:36.026379+0000", 2024-09-04T17:48:36.775 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:36.775 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.775 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.775 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:36.775 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:36.026379+0000", 2024-09-04T17:48:36.776 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:36.776 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.776 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.776 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:36.776 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:36.026381+0000", 2024-09-04T17:48:36.776 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.906e-06 2024-09-04T17:48:36.776 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.776 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.776 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:36.776 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:36.026395+0000", 2024-09-04T17:48:36.776 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4343e-05 2024-09-04T17:48:36.776 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.776 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.776 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:36.777 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:36.026427+0000", 2024-09-04T17:48:36.777 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.1569000000000002e-05 2024-09-04T17:48:36.777 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.777 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.777 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:36.777 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:36.026446+0000", 2024-09-04T17:48:36.777 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9459000000000001e-05 2024-09-04T17:48:36.777 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.777 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.777 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:36.777 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:36.026664+0000", 2024-09-04T17:48:36.777 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021771099999999999 2024-09-04T17:48:36.777 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.778 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.778 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:36.778 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:36.026720+0000", 2024-09-04T17:48:36.778 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.5943999999999998e-05 2024-09-04T17:48:36.778 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.778 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.778 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:36.778 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:36.033904+0000", 2024-09-04T17:48:36.778 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0071845219999999996 2024-09-04T17:48:36.778 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:36.778 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:36.778 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:36.778 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:36.034024+0000", 2024-09-04T17:48:36.779 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000119518 2024-09-04T17:48:36.779 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:36.779 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:36.779 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:36.779 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:36.779 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:36.779 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:36.780 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-04T17:48:36.781 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:48:36.779+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:48:36.882 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:48:36.879+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:48:36.883 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-04T17:48:36.927 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:36.927 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:48:36.927 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:48:36.927 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:36.945 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-04T17:48:36.985 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:36.983+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:37.083 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:37.081+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:37.109 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:37.109 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:48:37.109 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:48:37.109 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:48:37.109 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:37.123 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-04T17:48:37.184 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:37.179+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:37.240 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:37.240 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:48:37.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:48:37.241 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:48:37.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.241 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:48444 3.12 3:4ff80fb5:::rbd_data.11346a2b0bc9.000000000000001d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-04T17:48:37.241 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:58.122601+0000", 2024-09-04T17:48:37.241 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 99.116190896999996, 2024-09-04T17:48:37.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057065713999999997, 2024-09-04T17:48:37.241 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:37.241 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:37.241 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:37.241 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:37.241 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:37.241 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:37.242 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 48444 2024-09-04T17:48:37.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.242 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:37.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:37.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122601+0000", 2024-09-04T17:48:37.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:37.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122601+0000", 2024-09-04T17:48:37.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:37.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122603+0000", 2024-09-04T17:48:37.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3499999999999999e-06 2024-09-04T17:48:37.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:37.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122614+0000", 2024-09-04T17:48:37.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.063e-05 2024-09-04T17:48:37.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:37.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122643+0000", 2024-09-04T17:48:37.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9434999999999999e-05 2024-09-04T17:48:37.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.244 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:37.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122657+0000", 2024-09-04T17:48:37.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3920000000000001e-05 2024-09-04T17:48:37.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.244 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:37.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122767+0000", 2024-09-04T17:48:37.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010981599999999999 2024-09-04T17:48:37.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.244 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:37.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122989+0000", 2024-09-04T17:48:37.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022158200000000001 2024-09-04T17:48:37.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.245 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 3", 2024-09-04T17:48:37.245 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.133349+0000", 2024-09-04T17:48:37.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010360729000000001 2024-09-04T17:48:37.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.245 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:37.245 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.179423+0000", 2024-09-04T17:48:37.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046073408000000003 2024-09-04T17:48:37.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:37.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.179453+0000", 2024-09-04T17:48:37.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0229000000000001e-05 2024-09-04T17:48:37.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:37.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.179545+0000", 2024-09-04T17:48:37.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1580000000000001e-05 2024-09-04T17:48:37.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:37.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.179667+0000", 2024-09-04T17:48:37.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012203500000000001 2024-09-04T17:48:37.247 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.247 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:37.247 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.247 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:55376 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-04T17:48:37.247 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:11.974053+0000", 2024-09-04T17:48:37.247 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 85.264739113000005, 2024-09-04T17:48:37.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060514722, 2024-09-04T17:48:37.247 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:37.247 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:37.247 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:37.247 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:37.248 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:37.248 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:37.248 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 55376 2024-09-04T17:48:37.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.248 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:37.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:37.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974053+0000", 2024-09-04T17:48:37.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:37.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974053+0000", 2024-09-04T17:48:37.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:37.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974054+0000", 2024-09-04T17:48:37.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.841e-06 2024-09-04T17:48:37.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:37.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974062+0000", 2024-09-04T17:48:37.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5460000000000003e-06 2024-09-04T17:48:37.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:37.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974077+0000", 2024-09-04T17:48:37.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5389000000000001e-05 2024-09-04T17:48:37.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:37.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974085+0000", 2024-09-04T17:48:37.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7659999999999998e-06 2024-09-04T17:48:37.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:37.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974188+0000", 2024-09-04T17:48:37.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000102521 2024-09-04T17:48:37.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:37.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974321+0000", 2024-09-04T17:48:37.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013324699999999999 2024-09-04T17:48:37.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-04T17:48:37.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.986481+0000", 2024-09-04T17:48:37.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012160450999999999 2024-09-04T17:48:37.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:37.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:12.034415+0000", 2024-09-04T17:48:37.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.047934065999999997 2024-09-04T17:48:37.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:37.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:12.034438+0000", 2024-09-04T17:48:37.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2923999999999999e-05 2024-09-04T17:48:37.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:37.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:12.034505+0000", 2024-09-04T17:48:37.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6711000000000006e-05 2024-09-04T17:48:37.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:37.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:12.034567+0000", 2024-09-04T17:48:37.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2260000000000004e-05 2024-09-04T17:48:37.253 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.253 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:37.253 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.253 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61141 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 2793472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:37.253 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819909+0000", 2024-09-04T17:48:37.253 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 83.418883120999993, 2024-09-04T17:48:37.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042786419999999999, 2024-09-04T17:48:37.253 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:37.253 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:37.254 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:37.254 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:37.254 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:37.254 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:37.254 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61141 2024-09-04T17:48:37.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.254 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:37.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:37.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819909+0000", 2024-09-04T17:48:37.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:37.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819909+0000", 2024-09-04T17:48:37.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:37.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819909+0000", 2024-09-04T17:48:37.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6300000000000005e-07 2024-09-04T17:48:37.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:37.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819913+0000", 2024-09-04T17:48:37.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4189999999999998e-06 2024-09-04T17:48:37.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:37.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819920+0000", 2024-09-04T17:48:37.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8619999999999997e-06 2024-09-04T17:48:37.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:37.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819926+0000", 2024-09-04T17:48:37.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9399999999999999e-06 2024-09-04T17:48:37.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:37.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862331+0000", 2024-09-04T17:48:37.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042405348000000002 2024-09-04T17:48:37.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:37.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862410+0000", 2024-09-04T17:48:37.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9078000000000004e-05 2024-09-04T17:48:37.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:37.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862695+0000", 2024-09-04T17:48:37.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028510999999999999 2024-09-04T17:48:37.257 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.257 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:37.257 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.258 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61142 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 2932736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:37.258 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819934+0000", 2024-09-04T17:48:37.258 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 83.418857502999998, 2024-09-04T17:48:37.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043101985000000002, 2024-09-04T17:48:37.258 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:37.258 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:37.258 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:37.258 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:37.258 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:37.258 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:37.258 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61142 2024-09-04T17:48:37.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.258 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:37.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:37.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819934+0000", 2024-09-04T17:48:37.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:37.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819934+0000", 2024-09-04T17:48:37.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:37.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819935+0000", 2024-09-04T17:48:37.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2024-09-04T17:48:37.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:37.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819937+0000", 2024-09-04T17:48:37.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5770000000000001e-06 2024-09-04T17:48:37.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:37.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819943+0000", 2024-09-04T17:48:37.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7950000000000004e-06 2024-09-04T17:48:37.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:37.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819948+0000", 2024-09-04T17:48:37.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6779999999999999e-06 2024-09-04T17:48:37.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:37.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862739+0000", 2024-09-04T17:48:37.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042791454999999999 2024-09-04T17:48:37.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:37.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862824+0000", 2024-09-04T17:48:37.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4790999999999995e-05 2024-09-04T17:48:37.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:37.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.863036+0000", 2024-09-04T17:48:37.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021224899999999999 2024-09-04T17:48:37.262 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.262 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:37.262 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.262 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61143 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 3039232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:37.262 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819954+0000", 2024-09-04T17:48:37.262 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 83.418837385000003, 2024-09-04T17:48:37.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043485134000000002, 2024-09-04T17:48:37.262 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:37.262 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:37.262 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:37.262 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:37.263 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:37.263 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:37.263 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61143 2024-09-04T17:48:37.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.263 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:37.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:37.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819954+0000", 2024-09-04T17:48:37.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:37.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819954+0000", 2024-09-04T17:48:37.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:37.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819955+0000", 2024-09-04T17:48:37.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7300000000000002e-07 2024-09-04T17:48:37.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:37.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819957+0000", 2024-09-04T17:48:37.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.024e-06 2024-09-04T17:48:37.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:37.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819963+0000", 2024-09-04T17:48:37.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8109999999999999e-06 2024-09-04T17:48:37.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:37.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819967+0000", 2024-09-04T17:48:37.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6149999999999999e-06 2024-09-04T17:48:37.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:37.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.863082+0000", 2024-09-04T17:48:37.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043115078000000001 2024-09-04T17:48:37.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:37.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.863172+0000", 2024-09-04T17:48:37.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0181999999999998e-05 2024-09-04T17:48:37.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:37.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.863440+0000", 2024-09-04T17:48:37.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026705099999999997 2024-09-04T17:48:37.266 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.266 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:37.266 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.267 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:70015 3.6 3:6012e780:::rbd_data.11341aa245a3.000000000000005b:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-04T17:48:37.267 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:26.155497+0000", 2024-09-04T17:48:37.267 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 71.083294327999994, 2024-09-04T17:48:37.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.049007124999999999, 2024-09-04T17:48:37.267 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:37.267 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:37.267 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:37.267 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:37.267 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:37.267 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:37.268 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 70015 2024-09-04T17:48:37.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.268 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:37.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:37.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155497+0000", 2024-09-04T17:48:37.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:37.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155497+0000", 2024-09-04T17:48:37.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:37.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155500+0000", 2024-09-04T17:48:37.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7070000000000001e-06 2024-09-04T17:48:37.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:37.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155513+0000", 2024-09-04T17:48:37.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2989e-05 2024-09-04T17:48:37.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:37.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155544+0000", 2024-09-04T17:48:37.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0747000000000002e-05 2024-09-04T17:48:37.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:37.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155560+0000", 2024-09-04T17:48:37.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5866e-05 2024-09-04T17:48:37.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:37.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155675+0000", 2024-09-04T17:48:37.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000114838 2024-09-04T17:48:37.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:37.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155803+0000", 2024-09-04T17:48:37.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012851799999999999 2024-09-04T17:48:37.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 1", 2024-09-04T17:48:37.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.163582+0000", 2024-09-04T17:48:37.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0077790960000000001 2024-09-04T17:48:37.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:37.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.204055+0000", 2024-09-04T17:48:37.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.040472517 2024-09-04T17:48:37.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:37.272 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.204195+0000", 2024-09-04T17:48:37.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014055200000000001 2024-09-04T17:48:37.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:37.272 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.204373+0000", 2024-09-04T17:48:37.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000177784 2024-09-04T17:48:37.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:37.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.204505+0000", 2024-09-04T17:48:37.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013151100000000001 2024-09-04T17:48:37.273 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.273 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:37.273 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.273 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:70434 3.12 3:4cd2342b:::rbd_data.11341aa245a3.0000000000000091:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-04T17:48:37.273 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:26.943124+0000", 2024-09-04T17:48:37.273 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 70.295667546000004, 2024-09-04T17:48:37.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055011831999999997, 2024-09-04T17:48:37.273 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:37.273 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:37.273 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:37.274 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:37.274 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:37.274 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:37.274 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 70434 2024-09-04T17:48:37.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.274 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:37.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:37.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943124+0000", 2024-09-04T17:48:37.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:37.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943124+0000", 2024-09-04T17:48:37.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:37.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943128+0000", 2024-09-04T17:48:37.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8460000000000001e-06 2024-09-04T17:48:37.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:37.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943142+0000", 2024-09-04T17:48:37.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4089e-05 2024-09-04T17:48:37.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:37.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943177+0000", 2024-09-04T17:48:37.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5167999999999999e-05 2024-09-04T17:48:37.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:37.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943194+0000", 2024-09-04T17:48:37.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6235e-05 2024-09-04T17:48:37.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:37.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943286+0000", 2024-09-04T17:48:37.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2793000000000004e-05 2024-09-04T17:48:37.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:37.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943526+0000", 2024-09-04T17:48:37.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023950999999999999 2024-09-04T17:48:37.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 1", 2024-09-04T17:48:37.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.952871+0000", 2024-09-04T17:48:37.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0093453070000000006 2024-09-04T17:48:37.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:37.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.997968+0000", 2024-09-04T17:48:37.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045096613000000001 2024-09-04T17:48:37.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:37.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.998006+0000", 2024-09-04T17:48:37.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8257000000000003e-05 2024-09-04T17:48:37.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:37.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.998073+0000", 2024-09-04T17:48:37.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6803000000000004e-05 2024-09-04T17:48:37.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:37.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.998136+0000", 2024-09-04T17:48:37.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3211000000000002e-05 2024-09-04T17:48:37.279 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.279 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:37.279 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.279 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93841 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 5111808~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:37.279 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386491+0000", 2024-09-04T17:48:37.279 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.852300348999997, 2024-09-04T17:48:37.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043069578999999997, 2024-09-04T17:48:37.279 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:37.279 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:37.279 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:37.280 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:37.280 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:37.280 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:37.280 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93841 2024-09-04T17:48:37.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.280 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:37.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:37.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386491+0000", 2024-09-04T17:48:37.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:37.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386491+0000", 2024-09-04T17:48:37.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:37.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386492+0000", 2024-09-04T17:48:37.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0299999999999999e-07 2024-09-04T17:48:37.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:37.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386499+0000", 2024-09-04T17:48:37.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6089999999999997e-06 2024-09-04T17:48:37.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:37.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386506+0000", 2024-09-04T17:48:37.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3599999999999998e-06 2024-09-04T17:48:37.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:37.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386514+0000", 2024-09-04T17:48:37.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9379999999999992e-06 2024-09-04T17:48:37.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:37.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.428957+0000", 2024-09-04T17:48:37.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042442831 2024-09-04T17:48:37.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:37.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.429080+0000", 2024-09-04T17:48:37.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012359299999999999 2024-09-04T17:48:37.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:37.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.429561+0000", 2024-09-04T17:48:37.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00048074499999999998 2024-09-04T17:48:37.283 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.283 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:37.284 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.284 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93842 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 4259840~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:37.284 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386521+0000", 2024-09-04T17:48:37.284 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.852270417, 2024-09-04T17:48:37.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043661933999999999, 2024-09-04T17:48:37.284 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:37.284 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:37.284 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:37.284 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:37.284 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:37.284 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:37.284 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93842 2024-09-04T17:48:37.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.285 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:37.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:37.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386521+0000", 2024-09-04T17:48:37.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:37.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386521+0000", 2024-09-04T17:48:37.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:37.286 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386522+0000", 2024-09-04T17:48:37.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4700000000000001e-07 2024-09-04T17:48:37.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:37.286 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386524+0000", 2024-09-04T17:48:37.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4559999999999999e-06 2024-09-04T17:48:37.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:37.286 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386534+0000", 2024-09-04T17:48:37.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2639999999999997e-06 2024-09-04T17:48:37.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:37.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386539+0000", 2024-09-04T17:48:37.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6130000000000003e-06 2024-09-04T17:48:37.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:37.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.429577+0000", 2024-09-04T17:48:37.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043037757000000003 2024-09-04T17:48:37.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:37.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.429638+0000", 2024-09-04T17:48:37.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1235999999999994e-05 2024-09-04T17:48:37.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:37.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.430183+0000", 2024-09-04T17:48:37.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000545061 2024-09-04T17:48:37.288 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.288 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:37.288 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.288 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93843 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 4390912~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:37.289 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386547+0000", 2024-09-04T17:48:37.289 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.852245179999997, 2024-09-04T17:48:37.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044260668000000003, 2024-09-04T17:48:37.289 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:37.289 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:37.289 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:37.289 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:37.289 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:37.289 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:37.289 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93843 2024-09-04T17:48:37.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.289 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:37.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:37.290 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386547+0000", 2024-09-04T17:48:37.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:37.290 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386547+0000", 2024-09-04T17:48:37.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:37.290 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386547+0000", 2024-09-04T17:48:37.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4199999999999996e-07 2024-09-04T17:48:37.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:37.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386550+0000", 2024-09-04T17:48:37.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6199999999999999e-06 2024-09-04T17:48:37.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:37.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386557+0000", 2024-09-04T17:48:37.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1919999999999999e-06 2024-09-04T17:48:37.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:37.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386562+0000", 2024-09-04T17:48:37.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2580000000000003e-06 2024-09-04T17:48:37.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:37.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.430226+0000", 2024-09-04T17:48:37.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043663484000000002 2024-09-04T17:48:37.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:37.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.430286+0000", 2024-09-04T17:48:37.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0455999999999999e-05 2024-09-04T17:48:37.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:37.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.430807+0000", 2024-09-04T17:48:37.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00052111599999999996 2024-09-04T17:48:37.293 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.293 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:37.293 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.293 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93844 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 4521984~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:37.293 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386603+0000", 2024-09-04T17:48:37.293 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.852189129000003, 2024-09-04T17:48:37.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044705350999999997, 2024-09-04T17:48:37.293 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:37.293 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:37.293 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:37.294 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:37.294 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:37.294 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:37.294 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93844 2024-09-04T17:48:37.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.294 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:37.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:37.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386603+0000", 2024-09-04T17:48:37.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:37.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386603+0000", 2024-09-04T17:48:37.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:37.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386603+0000", 2024-09-04T17:48:37.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.92e-07 2024-09-04T17:48:37.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:37.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386606+0000", 2024-09-04T17:48:37.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5500000000000001e-06 2024-09-04T17:48:37.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.296 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:37.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386614+0000", 2024-09-04T17:48:37.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9570000000000002e-06 2024-09-04T17:48:37.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.296 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:37.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386619+0000", 2024-09-04T17:48:37.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5119999999999999e-06 2024-09-04T17:48:37.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:37.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.430828+0000", 2024-09-04T17:48:37.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044208654 2024-09-04T17:48:37.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:37.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.430903+0000", 2024-09-04T17:48:37.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4629999999999995e-05 2024-09-04T17:48:37.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:37.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.431308+0000", 2024-09-04T17:48:37.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000405256 2024-09-04T17:48:37.298 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.298 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:37.298 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.298 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93845 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 4653056~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:37.298 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386627+0000", 2024-09-04T17:48:37.298 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.852164539999997, 2024-09-04T17:48:37.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045262927000000001, 2024-09-04T17:48:37.298 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:37.298 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:37.298 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:37.299 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:37.299 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:37.299 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:37.299 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93845 2024-09-04T17:48:37.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.299 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:37.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:37.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386627+0000", 2024-09-04T17:48:37.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:37.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386627+0000", 2024-09-04T17:48:37.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.300 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:37.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386628+0000", 2024-09-04T17:48:37.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0999999999999998e-07 2024-09-04T17:48:37.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.300 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:37.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386630+0000", 2024-09-04T17:48:37.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3300000000000001e-06 2024-09-04T17:48:37.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:37.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386645+0000", 2024-09-04T17:48:37.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5186e-05 2024-09-04T17:48:37.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:37.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386651+0000", 2024-09-04T17:48:37.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7710000000000003e-06 2024-09-04T17:48:37.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:37.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.431349+0000", 2024-09-04T17:48:37.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044697723000000002 2024-09-04T17:48:37.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:37.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.431410+0000", 2024-09-04T17:48:37.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0677999999999997e-05 2024-09-04T17:48:37.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:37.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.431890+0000", 2024-09-04T17:48:37.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00048052899999999999 2024-09-04T17:48:37.302 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.302 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:37.302 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.303 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93846 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 4784128~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:37.303 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386659+0000", 2024-09-04T17:48:37.303 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.852132871000002, 2024-09-04T17:48:37.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045784490999999997, 2024-09-04T17:48:37.303 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:37.303 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:37.303 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:37.303 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:37.303 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:37.303 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:37.303 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93846 2024-09-04T17:48:37.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.304 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:37.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:37.304 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386659+0000", 2024-09-04T17:48:37.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:37.304 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386659+0000", 2024-09-04T17:48:37.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:37.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386659+0000", 2024-09-04T17:48:37.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7000000000000005e-07 2024-09-04T17:48:37.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:37.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386662+0000", 2024-09-04T17:48:37.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3300000000000001e-06 2024-09-04T17:48:37.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:37.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386669+0000", 2024-09-04T17:48:37.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0589999999999997e-06 2024-09-04T17:48:37.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:37.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386674+0000", 2024-09-04T17:48:37.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3009999999999997e-06 2024-09-04T17:48:37.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:37.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.431934+0000", 2024-09-04T17:48:37.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045259570999999998 2024-09-04T17:48:37.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:37.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.431998+0000", 2024-09-04T17:48:37.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4775000000000002e-05 2024-09-04T17:48:37.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:37.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.432443+0000", 2024-09-04T17:48:37.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00044488500000000001 2024-09-04T17:48:37.307 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.307 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:37.307 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.307 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93847 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 4915200~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:37.307 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386681+0000", 2024-09-04T17:48:37.307 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.852110613999997, 2024-09-04T17:48:37.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046305101000000001, 2024-09-04T17:48:37.308 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:37.308 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:37.308 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:37.308 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:37.308 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:37.308 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:37.308 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93847 2024-09-04T17:48:37.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.308 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:37.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.308 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:37.308 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386681+0000", 2024-09-04T17:48:37.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.309 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:37.309 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386681+0000", 2024-09-04T17:48:37.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.309 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:37.309 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386682+0000", 2024-09-04T17:48:37.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1699999999999998e-07 2024-09-04T17:48:37.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.309 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:37.309 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386684+0000", 2024-09-04T17:48:37.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.153e-06 2024-09-04T17:48:37.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:37.310 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386693+0000", 2024-09-04T17:48:37.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9639999999999992e-06 2024-09-04T17:48:37.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:37.310 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386698+0000", 2024-09-04T17:48:37.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3709999999999999e-06 2024-09-04T17:48:37.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:37.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.432476+0000", 2024-09-04T17:48:37.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045778147999999998 2024-09-04T17:48:37.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:37.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.432529+0000", 2024-09-04T17:48:37.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3165999999999999e-05 2024-09-04T17:48:37.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:37.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.432986+0000", 2024-09-04T17:48:37.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00045678199999999998 2024-09-04T17:48:37.311 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.312 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:37.312 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.312 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93848 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 5046272~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:37.312 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386705+0000", 2024-09-04T17:48:37.312 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.852086513000003, 2024-09-04T17:48:37.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046825259000000001, 2024-09-04T17:48:37.312 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:37.312 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:37.312 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:37.312 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:37.312 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:37.313 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:37.313 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93848 2024-09-04T17:48:37.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.313 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:37.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:37.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386705+0000", 2024-09-04T17:48:37.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:37.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386705+0000", 2024-09-04T17:48:37.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.314 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:37.314 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386706+0000", 2024-09-04T17:48:37.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1500000000000005e-07 2024-09-04T17:48:37.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.314 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:37.314 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386708+0000", 2024-09-04T17:48:37.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2979999999999999e-06 2024-09-04T17:48:37.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.314 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:37.315 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386715+0000", 2024-09-04T17:48:37.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8669999999999996e-06 2024-09-04T17:48:37.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.315 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:37.315 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386720+0000", 2024-09-04T17:48:37.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.242e-06 2024-09-04T17:48:37.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.315 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:37.315 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.433019+0000", 2024-09-04T17:48:37.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046298359999999997 2024-09-04T17:48:37.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:37.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.433084+0000", 2024-09-04T17:48:37.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5395999999999995e-05 2024-09-04T17:48:37.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:37.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.433531+0000", 2024-09-04T17:48:37.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00044658099999999999 2024-09-04T17:48:37.316 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.316 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:37.316 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.317 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93849 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 5177344~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:37.317 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386727+0000", 2024-09-04T17:48:37.317 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.852064615000003, 2024-09-04T17:48:37.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.047267796000000001, 2024-09-04T17:48:37.317 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:37.317 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:37.317 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:37.317 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:37.317 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:37.317 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:37.317 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93849 2024-09-04T17:48:37.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.318 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:37.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.318 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:37.318 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386727+0000", 2024-09-04T17:48:37.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.318 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:37.318 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386727+0000", 2024-09-04T17:48:37.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.318 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:37.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386728+0000", 2024-09-04T17:48:37.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1699999999999998e-07 2024-09-04T17:48:37.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:37.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386730+0000", 2024-09-04T17:48:37.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2589999999999999e-06 2024-09-04T17:48:37.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:37.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386751+0000", 2024-09-04T17:48:37.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1424999999999999e-05 2024-09-04T17:48:37.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:37.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386759+0000", 2024-09-04T17:48:37.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9400000000000002e-06 2024-09-04T17:48:37.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:37.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.433559+0000", 2024-09-04T17:48:37.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046799573999999997 2024-09-04T17:48:37.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:37.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.433610+0000", 2024-09-04T17:48:37.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1134999999999997e-05 2024-09-04T17:48:37.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:37.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.433995+0000", 2024-09-04T17:48:37.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000384946 2024-09-04T17:48:37.321 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.321 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:37.321 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.321 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:96175 3.12 3:4d4471be:::rbd_data.1134c4e2d26.000000000000006e:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123)", 2024-09-04T17:48:37.321 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:48:02.919542+0000", 2024-09-04T17:48:37.321 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 34.319249370999998, 2024-09-04T17:48:37.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060180626000000001, 2024-09-04T17:48:37.322 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:37.322 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:37.322 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:37.322 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:37.322 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:37.322 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:37.322 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 96175 2024-09-04T17:48:37.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.322 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:37.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:37.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.919542+0000", 2024-09-04T17:48:37.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.323 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:37.323 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.919542+0000", 2024-09-04T17:48:37.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.323 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:37.323 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.919545+0000", 2024-09-04T17:48:37.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0199999999999999e-06 2024-09-04T17:48:37.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.323 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:37.324 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.919559+0000", 2024-09-04T17:48:37.324 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3293e-05 2024-09-04T17:48:37.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.324 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.324 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:37.324 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.919587+0000", 2024-09-04T17:48:37.324 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7922000000000001e-05 2024-09-04T17:48:37.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.324 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.324 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:37.324 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.919601+0000", 2024-09-04T17:48:37.324 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4657e-05 2024-09-04T17:48:37.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:37.325 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.919708+0000", 2024-09-04T17:48:37.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010624999999999999 2024-09-04T17:48:37.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:37.325 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.919929+0000", 2024-09-04T17:48:37.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022147399999999999 2024-09-04T17:48:37.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2024-09-04T17:48:37.325 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.929774+0000", 2024-09-04T17:48:37.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0098449839999999993 2024-09-04T17:48:37.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:37.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.979511+0000", 2024-09-04T17:48:37.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.049736537999999997 2024-09-04T17:48:37.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:37.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.979547+0000", 2024-09-04T17:48:37.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6069000000000003e-05 2024-09-04T17:48:37.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:37.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.979642+0000", 2024-09-04T17:48:37.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5526999999999995e-05 2024-09-04T17:48:37.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:37.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.979723+0000", 2024-09-04T17:48:37.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0891999999999997e-05 2024-09-04T17:48:37.327 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.327 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:37.327 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.328 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:97148 3.4 3:22dd2475:::rbd_id.image_client.0-clone39:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e128)", 2024-09-04T17:48:37.328 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:48:07.831881+0000", 2024-09-04T17:48:37.328 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 29.406911137000002, 2024-09-04T17:48:37.328 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.026553585, 2024-09-04T17:48:37.328 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:37.328 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:37.328 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:37.328 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:37.328 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:37.328 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:37.328 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 97148 2024-09-04T17:48:37.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.328 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:37.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.329 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:37.329 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:07.831881+0000", 2024-09-04T17:48:37.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.329 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:37.329 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:07.831881+0000", 2024-09-04T17:48:37.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.329 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:37.329 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:07.831883+0000", 2024-09-04T17:48:37.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3369999999999998e-06 2024-09-04T17:48:37.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.330 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:37.330 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:07.831890+0000", 2024-09-04T17:48:37.330 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7050000000000002e-06 2024-09-04T17:48:37.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.330 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:37.330 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:07.831905+0000", 2024-09-04T17:48:37.330 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5512999999999999e-05 2024-09-04T17:48:37.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.331 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:37.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:07.831918+0000", 2024-09-04T17:48:37.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3223e-05 2024-09-04T17:48:37.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.331 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:37.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:07.833441+0000", 2024-09-04T17:48:37.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0015228920000000001 2024-09-04T17:48:37.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.331 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-09-04T17:48:37.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:07.833451+0000", 2024-09-04T17:48:37.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6600000000000007e-06 2024-09-04T17:48:37.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.332 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.332 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:37.332 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:07.837721+0000", 2024-09-04T17:48:37.332 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0042702410000000001 2024-09-04T17:48:37.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.332 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.332 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-04T17:48:37.332 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:07.837732+0000", 2024-09-04T17:48:37.332 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.057e-05 2024-09-04T17:48:37.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.332 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.333 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:37.333 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:08.856291+0000", 2024-09-04T17:48:37.333 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0185595350000001 2024-09-04T17:48:37.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.333 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.333 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:37.333 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:08.856503+0000", 2024-09-04T17:48:37.333 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000212127 2024-09-04T17:48:37.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.333 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.333 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2,5", 2024-09-04T17:48:37.333 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:08.856664+0000", 2024-09-04T17:48:37.333 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016072800000000001 2024-09-04T17:48:37.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.334 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:37.334 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:08.857811+0000", 2024-09-04T17:48:37.334 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001146441 2024-09-04T17:48:37.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.334 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:37.334 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:08.858202+0000", 2024-09-04T17:48:37.334 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000391605 2024-09-04T17:48:37.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.334 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:37.335 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:08.858239+0000", 2024-09-04T17:48:37.335 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7200000000000003e-05 2024-09-04T17:48:37.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.335 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:37.335 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:08.858287+0000", 2024-09-04T17:48:37.335 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7515999999999997e-05 2024-09-04T17:48:37.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.335 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:37.335 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:08.858434+0000", 2024-09-04T17:48:37.335 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014729200000000001 2024-09-04T17:48:37.335 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.336 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:37.336 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.336 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.336 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.336 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:99510 3.18 3:1bf8b1d0:::rbd_header.11346a2b0bc9:head [call rbd.snapshot_remove in=18b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e147)", 2024-09-04T17:48:37.336 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:48:27.032522+0000", 2024-09-04T17:48:37.336 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.206269274, 2024-09-04T17:48:37.336 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.84201522500000003, 2024-09-04T17:48:37.336 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:37.336 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:37.336 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:37.336 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:37.337 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:37.337 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:37.337 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 99510 2024-09-04T17:48:37.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.337 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:37.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.337 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:37.337 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032522+0000", 2024-09-04T17:48:37.337 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.337 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:37.337 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032522+0000", 2024-09-04T17:48:37.337 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.338 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:37.338 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032525+0000", 2024-09-04T17:48:37.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4789999999999999e-06 2024-09-04T17:48:37.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.338 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:37.338 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032536+0000", 2024-09-04T17:48:37.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0557000000000001e-05 2024-09-04T17:48:37.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.338 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:37.338 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032555+0000", 2024-09-04T17:48:37.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9782000000000001e-05 2024-09-04T17:48:37.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.339 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:37.339 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032569+0000", 2024-09-04T17:48:37.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4012000000000001e-05 2024-09-04T17:48:37.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.339 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:37.339 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032649+0000", 2024-09-04T17:48:37.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9782000000000003e-05 2024-09-04T17:48:37.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.339 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-09-04T17:48:37.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032657+0000", 2024-09-04T17:48:37.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7999999999999999e-06 2024-09-04T17:48:37.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.340 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:37.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.035521+0000", 2024-09-04T17:48:37.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028640850000000002 2024-09-04T17:48:37.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.340 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-04T17:48:37.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.035534+0000", 2024-09-04T17:48:37.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3157e-05 2024-09-04T17:48:37.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.341 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:37.341 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.869836+0000", 2024-09-04T17:48:37.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.83430168400000004 2024-09-04T17:48:37.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.341 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:37.341 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.870070+0000", 2024-09-04T17:48:37.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023443300000000001 2024-09-04T17:48:37.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.341 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 3", 2024-09-04T17:48:37.341 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.870423+0000", 2024-09-04T17:48:37.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00035290700000000001 2024-09-04T17:48:37.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.342 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:37.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.871254+0000", 2024-09-04T17:48:37.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00083113700000000002 2024-09-04T17:48:37.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.342 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:37.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.874234+0000", 2024-09-04T17:48:37.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029795189999999999 2024-09-04T17:48:37.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.342 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:37.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.874321+0000", 2024-09-04T17:48:37.343 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7637000000000001e-05 2024-09-04T17:48:37.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.343 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:37.343 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.874538+0000", 2024-09-04T17:48:37.343 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021625400000000001 2024-09-04T17:48:37.343 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.343 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:37.343 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.343 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:99511 3.18 3:1bf8b1d0:::rbd_header.11346a2b0bc9:head [watch reconnect cookie 140018821496000 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147)", 2024-09-04T17:48:37.343 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:48:27.032602+0000", 2024-09-04T17:48:37.343 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.206189471, 2024-09-04T17:48:37.344 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.84736467999999998, 2024-09-04T17:48:37.344 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:37.344 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:37.344 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:37.344 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:37.344 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:37.344 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:37.344 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 99511 2024-09-04T17:48:37.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.344 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:37.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.344 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:37.344 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032602+0000", 2024-09-04T17:48:37.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.345 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:37.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032602+0000", 2024-09-04T17:48:37.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.345 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:37.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032603+0000", 2024-09-04T17:48:37.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3300000000000001e-07 2024-09-04T17:48:37.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:37.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032610+0000", 2024-09-04T17:48:37.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.754e-06 2024-09-04T17:48:37.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:37.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032621+0000", 2024-09-04T17:48:37.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1175999999999999e-05 2024-09-04T17:48:37.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:37.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032630+0000", 2024-09-04T17:48:37.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7900000000000005e-06 2024-09-04T17:48:37.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:37.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032900+0000", 2024-09-04T17:48:37.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026967499999999998 2024-09-04T17:48:37.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting_for_map not empty", 2024-09-04T17:48:37.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032907+0000", 2024-09-04T17:48:37.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6969999999999993e-06 2024-09-04T17:48:37.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:37.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.036096+0000", 2024-09-04T17:48:37.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031887090000000001 2024-09-04T17:48:37.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-04T17:48:37.348 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.036107+0000", 2024-09-04T17:48:37.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1361e-05 2024-09-04T17:48:37.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:37.348 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.870783+0000", 2024-09-04T17:48:37.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.834675265 2024-09-04T17:48:37.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-09-04T17:48:37.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.870943+0000", 2024-09-04T17:48:37.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000160364 2024-09-04T17:48:37.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:37.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.874588+0000", 2024-09-04T17:48:37.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0036446529999999999 2024-09-04T17:48:37.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:37.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.874731+0000", 2024-09-04T17:48:37.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014334499999999999 2024-09-04T17:48:37.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:37.350 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.879967+0000", 2024-09-04T17:48:37.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005236058 2024-09-04T17:48:37.350 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.350 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:37.350 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.350 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.350 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:37.350 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:37.352 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-04T17:48:37.352 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:37.347+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:37.453 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:48:37.453+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:48:37.523 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:37.524 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:48:37.524 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:48:37.524 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:37.538 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-04T17:48:37.553 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:37.553+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:37.654 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:48:37.651+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:48:37.659 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:37.659 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:48:37.660 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:48:37.660 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:48:37.660 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:37.673 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-04T17:48:37.707 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-04T17:48:37.755 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:48:37.753+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:48:37.837 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:37.837 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:48:37.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:48:37.837 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:48:37.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.837 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59950 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5365760~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:48:37.837 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654917+0000", 2024-09-04T17:48:37.837 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 84.180588783000005, 2024-09-04T17:48:37.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066822450000000005, 2024-09-04T17:48:37.838 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:37.838 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:37.838 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:37.838 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:37.838 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:37.838 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:37.838 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59950 2024-09-04T17:48:37.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.838 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:37.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:37.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654917+0000", 2024-09-04T17:48:37.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:37.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654917+0000", 2024-09-04T17:48:37.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:37.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654918+0000", 2024-09-04T17:48:37.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.29e-07 2024-09-04T17:48:37.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:37.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654919+0000", 2024-09-04T17:48:37.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2589999999999999e-06 2024-09-04T17:48:37.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:37.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654922+0000", 2024-09-04T17:48:37.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1350000000000001e-06 2024-09-04T17:48:37.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:37.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654925+0000", 2024-09-04T17:48:37.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1250000000000001e-06 2024-09-04T17:48:37.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:37.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721466+0000", 2024-09-04T17:48:37.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066540959999999996 2024-09-04T17:48:37.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:37.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721574+0000", 2024-09-04T17:48:37.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010747399999999999 2024-09-04T17:48:37.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:37.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721740+0000", 2024-09-04T17:48:37.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016626799999999999 2024-09-04T17:48:37.842 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.842 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:37.842 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.842 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59951 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5472256~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:48:37.842 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654929+0000", 2024-09-04T17:48:37.843 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 84.180577597999999, 2024-09-04T17:48:37.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067110534999999999, 2024-09-04T17:48:37.843 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:37.843 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:37.843 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:37.843 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:37.843 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:37.843 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:37.843 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59951 2024-09-04T17:48:37.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.843 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:37.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:37.844 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654929+0000", 2024-09-04T17:48:37.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:37.844 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654929+0000", 2024-09-04T17:48:37.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:37.844 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654929+0000", 2024-09-04T17:48:37.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5100000000000001e-07 2024-09-04T17:48:37.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:37.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654930+0000", 2024-09-04T17:48:37.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2440000000000001e-06 2024-09-04T17:48:37.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:37.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654933+0000", 2024-09-04T17:48:37.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1599999999999998e-06 2024-09-04T17:48:37.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:37.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654936+0000", 2024-09-04T17:48:37.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6560000000000001e-06 2024-09-04T17:48:37.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:37.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721789+0000", 2024-09-04T17:48:37.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066853250000000003 2024-09-04T17:48:37.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:37.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721885+0000", 2024-09-04T17:48:37.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5616000000000001e-05 2024-09-04T17:48:37.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:37.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722039+0000", 2024-09-04T17:48:37.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000154258 2024-09-04T17:48:37.847 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.847 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:37.847 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.848 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59952 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5505024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:48:37.848 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654939+0000", 2024-09-04T17:48:37.848 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 84.180566721000005, 2024-09-04T17:48:37.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067433742000000005, 2024-09-04T17:48:37.848 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:37.848 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:37.848 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:37.848 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:37.848 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:37.849 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:37.849 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59952 2024-09-04T17:48:37.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.849 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:37.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:37.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654939+0000", 2024-09-04T17:48:37.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:37.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654939+0000", 2024-09-04T17:48:37.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:37.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654940+0000", 2024-09-04T17:48:37.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3300000000000001e-07 2024-09-04T17:48:37.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:37.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654941+0000", 2024-09-04T17:48:37.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1570000000000001e-06 2024-09-04T17:48:37.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:37.851 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654944+0000", 2024-09-04T17:48:37.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1319999999999998e-06 2024-09-04T17:48:37.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:37.851 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654947+0000", 2024-09-04T17:48:37.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5969999999999999e-06 2024-09-04T17:48:37.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:37.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722089+0000", 2024-09-04T17:48:37.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067142347000000005 2024-09-04T17:48:37.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:37.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722201+0000", 2024-09-04T17:48:37.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011235600000000001 2024-09-04T17:48:37.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:37.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722373+0000", 2024-09-04T17:48:37.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017191999999999999 2024-09-04T17:48:37.853 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.853 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:37.853 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.853 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59953 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5611520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:48:37.853 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654950+0000", 2024-09-04T17:48:37.853 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 84.180556081000006, 2024-09-04T17:48:37.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067601852000000004, 2024-09-04T17:48:37.854 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:37.854 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:37.854 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:37.854 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:37.854 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:37.854 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:37.854 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59953 2024-09-04T17:48:37.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.854 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:37.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.855 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:37.855 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654950+0000", 2024-09-04T17:48:37.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.855 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:37.855 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654950+0000", 2024-09-04T17:48:37.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:37.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654950+0000", 2024-09-04T17:48:37.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2499999999999999e-07 2024-09-04T17:48:37.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:37.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654954+0000", 2024-09-04T17:48:37.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.287e-06 2024-09-04T17:48:37.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.857 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:37.857 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654962+0000", 2024-09-04T17:48:37.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4230000000000005e-06 2024-09-04T17:48:37.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.857 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:37.857 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654966+0000", 2024-09-04T17:48:37.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6449999999999998e-06 2024-09-04T17:48:37.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.857 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:37.858 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722417+0000", 2024-09-04T17:48:37.858 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067451188999999995 2024-09-04T17:48:37.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.858 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.858 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:37.858 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722467+0000", 2024-09-04T17:48:37.858 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0556999999999997e-05 2024-09-04T17:48:37.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.858 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.858 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:37.858 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722552+0000", 2024-09-04T17:48:37.859 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4525999999999997e-05 2024-09-04T17:48:37.859 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.859 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:37.859 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.859 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.859 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59954 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5718016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:48:37.859 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654970+0000", 2024-09-04T17:48:37.859 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 84.180536320000002, 2024-09-04T17:48:37.859 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067874285000000006, 2024-09-04T17:48:37.859 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:37.859 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:37.859 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:37.860 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:37.860 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:37.860 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:37.860 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59954 2024-09-04T17:48:37.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.860 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:37.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.860 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:37.860 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654970+0000", 2024-09-04T17:48:37.860 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.861 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:37.861 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654970+0000", 2024-09-04T17:48:37.861 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.861 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.861 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.861 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:37.861 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654970+0000", 2024-09-04T17:48:37.861 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5899999999999998e-07 2024-09-04T17:48:37.861 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.861 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.861 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:37.861 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654971+0000", 2024-09-04T17:48:37.862 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2929999999999999e-06 2024-09-04T17:48:37.862 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.862 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.862 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:37.862 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654975+0000", 2024-09-04T17:48:37.862 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4699999999999998e-06 2024-09-04T17:48:37.862 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.862 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.862 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:37.862 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654978+0000", 2024-09-04T17:48:37.862 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7930000000000002e-06 2024-09-04T17:48:37.862 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.863 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:37.863 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722600+0000", 2024-09-04T17:48:37.863 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067622229000000006 2024-09-04T17:48:37.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.863 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:37.863 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722689+0000", 2024-09-04T17:48:37.863 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9511000000000004e-05 2024-09-04T17:48:37.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.863 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:37.863 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722844+0000", 2024-09-04T17:48:37.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015473 2024-09-04T17:48:37.864 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.864 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:37.864 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.864 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59955 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5857280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:48:37.864 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654981+0000", 2024-09-04T17:48:37.864 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 84.180524969999993, 2024-09-04T17:48:37.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068052360000000006, 2024-09-04T17:48:37.864 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:37.865 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:37.865 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:37.865 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:37.865 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:37.865 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:37.865 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59955 2024-09-04T17:48:37.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.865 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:37.865 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.865 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:37.865 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654981+0000", 2024-09-04T17:48:37.865 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.865 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.866 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:37.866 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654981+0000", 2024-09-04T17:48:37.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.866 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.866 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:37.866 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654981+0000", 2024-09-04T17:48:37.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2399999999999999e-07 2024-09-04T17:48:37.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.866 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.866 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:37.866 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654985+0000", 2024-09-04T17:48:37.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0980000000000004e-06 2024-09-04T17:48:37.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.866 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.867 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:37.867 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654990+0000", 2024-09-04T17:48:37.867 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3429999999999998e-06 2024-09-04T17:48:37.867 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.867 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.867 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:37.867 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654993+0000", 2024-09-04T17:48:37.867 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8700000000000001e-06 2024-09-04T17:48:37.867 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.867 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.867 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:37.867 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722888+0000", 2024-09-04T17:48:37.867 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067895239999999996 2024-09-04T17:48:37.867 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.868 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.868 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:37.868 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722948+0000", 2024-09-04T17:48:37.868 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9750999999999998e-05 2024-09-04T17:48:37.868 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.868 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.868 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:37.868 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.723034+0000", 2024-09-04T17:48:37.868 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5834000000000005e-05 2024-09-04T17:48:37.868 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.868 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:37.868 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.868 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.868 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.869 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59956 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5963776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:48:37.869 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654998+0000", 2024-09-04T17:48:37.869 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 84.180508441000001, 2024-09-04T17:48:37.869 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068359518999999994, 2024-09-04T17:48:37.869 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:37.869 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:37.869 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:37.869 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:37.869 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:37.869 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:37.869 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59956 2024-09-04T17:48:37.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.870 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:37.870 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.870 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:37.870 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654998+0000", 2024-09-04T17:48:37.870 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.870 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.870 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.870 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:37.870 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654998+0000", 2024-09-04T17:48:37.870 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.870 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.870 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.870 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:37.871 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654998+0000", 2024-09-04T17:48:37.871 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4299999999999999e-07 2024-09-04T17:48:37.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.871 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.871 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:37.871 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654999+0000", 2024-09-04T17:48:37.871 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1799999999999999e-06 2024-09-04T17:48:37.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.871 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.871 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:37.871 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.655002+0000", 2024-09-04T17:48:37.871 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3639999999999999e-06 2024-09-04T17:48:37.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.872 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.872 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:37.872 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.655005+0000", 2024-09-04T17:48:37.872 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7970000000000001e-06 2024-09-04T17:48:37.872 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.872 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.872 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:37.872 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.723085+0000", 2024-09-04T17:48:37.872 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068080229000000006 2024-09-04T17:48:37.872 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.872 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.872 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:37.873 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.723197+0000", 2024-09-04T17:48:37.873 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000111344 2024-09-04T17:48:37.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.873 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.873 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:37.873 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.723357+0000", 2024-09-04T17:48:37.873 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016036199999999999 2024-09-04T17:48:37.873 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.873 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:37.873 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.874 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61082 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 2654208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:37.874 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.822664+0000", 2024-09-04T17:48:37.874 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 84.012842054000004, 2024-09-04T17:48:37.874 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067072242000000004, 2024-09-04T17:48:37.874 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:37.874 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:37.874 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:37.874 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:37.874 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:37.874 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:37.874 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61082 2024-09-04T17:48:37.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.875 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:37.875 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.875 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:37.875 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822664+0000", 2024-09-04T17:48:37.875 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.875 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.875 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:37.875 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822664+0000", 2024-09-04T17:48:37.875 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.875 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.876 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:37.876 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822664+0000", 2024-09-04T17:48:37.876 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6600000000000003e-07 2024-09-04T17:48:37.876 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.876 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.876 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:37.876 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822666+0000", 2024-09-04T17:48:37.876 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.536e-06 2024-09-04T17:48:37.876 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.876 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.876 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:37.876 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822669+0000", 2024-09-04T17:48:37.877 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3759999999999999e-06 2024-09-04T17:48:37.877 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.877 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.877 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:37.877 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822672+0000", 2024-09-04T17:48:37.877 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8109999999999999e-06 2024-09-04T17:48:37.877 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.877 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.877 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:37.877 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889263+0000", 2024-09-04T17:48:37.877 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066591259999999999 2024-09-04T17:48:37.877 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.877 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.878 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:37.878 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889369+0000", 2024-09-04T17:48:37.878 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000105362 2024-09-04T17:48:37.878 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.878 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.878 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:37.878 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889736+0000", 2024-09-04T17:48:37.878 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00036763099999999999 2024-09-04T17:48:37.878 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.878 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:37.878 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.878 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.878 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.878 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61083 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 2760704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:37.879 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.822678+0000", 2024-09-04T17:48:37.879 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 84.012828319999997, 2024-09-04T17:48:37.879 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067475996999999996, 2024-09-04T17:48:37.879 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:37.879 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:37.879 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:37.879 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:37.879 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:37.879 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:37.879 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61083 2024-09-04T17:48:37.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.879 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:37.880 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.880 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:37.880 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822678+0000", 2024-09-04T17:48:37.880 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.880 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.880 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.880 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:37.880 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822678+0000", 2024-09-04T17:48:37.880 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.880 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.880 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.880 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:37.880 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822725+0000", 2024-09-04T17:48:37.881 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7429000000000003e-05 2024-09-04T17:48:37.881 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.881 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.881 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:37.881 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822728+0000", 2024-09-04T17:48:37.881 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2140000000000001e-06 2024-09-04T17:48:37.881 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.881 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.881 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:37.881 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822737+0000", 2024-09-04T17:48:37.881 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4370000000000007e-06 2024-09-04T17:48:37.881 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.882 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:37.882 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822743+0000", 2024-09-04T17:48:37.882 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7520000000000002e-06 2024-09-04T17:48:37.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.882 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:37.882 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889785+0000", 2024-09-04T17:48:37.882 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067042265000000004 2024-09-04T17:48:37.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.882 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:37.882 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889840+0000", 2024-09-04T17:48:37.882 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5254000000000003e-05 2024-09-04T17:48:37.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.883 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.883 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:37.883 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.890154+0000", 2024-09-04T17:48:37.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000313646 2024-09-04T17:48:37.883 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.883 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:37.883 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.883 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.883 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61084 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 2867200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:37.883 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.822753+0000", 2024-09-04T17:48:37.883 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 84.012753384000007, 2024-09-04T17:48:37.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068497161000000001, 2024-09-04T17:48:37.883 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:37.884 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:37.884 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:37.884 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:37.884 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:37.884 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:37.884 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61084 2024-09-04T17:48:37.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.884 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:37.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.884 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:37.884 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822753+0000", 2024-09-04T17:48:37.884 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.885 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:37.885 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822753+0000", 2024-09-04T17:48:37.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.885 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.885 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.885 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:37.885 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822753+0000", 2024-09-04T17:48:37.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9100000000000003e-07 2024-09-04T17:48:37.885 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.885 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.885 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:37.885 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822756+0000", 2024-09-04T17:48:37.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.678e-06 2024-09-04T17:48:37.886 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.886 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.886 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:37.886 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822763+0000", 2024-09-04T17:48:37.886 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8220000000000001e-06 2024-09-04T17:48:37.886 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.886 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.886 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:37.886 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822768+0000", 2024-09-04T17:48:37.886 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1930000000000001e-06 2024-09-04T17:48:37.886 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.886 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.886 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:37.886 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.890246+0000", 2024-09-04T17:48:37.887 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067477996999999998 2024-09-04T17:48:37.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.887 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:37.887 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.890555+0000", 2024-09-04T17:48:37.887 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000309143 2024-09-04T17:48:37.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.887 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:37.887 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.891250+0000", 2024-09-04T17:48:37.887 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00069463700000000001 2024-09-04T17:48:37.887 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.887 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:37.887 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.888 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.888 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.888 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61285 3.8 3:117cfeb9:::rbd_data.11341dfde0c5.0000000000000006:1f [read 2760704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:37.888 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835200+0000", 2024-09-04T17:48:37.888 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 84.000306316999996, 2024-09-04T17:48:37.888 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066899803999999993, 2024-09-04T17:48:37.888 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:37.888 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:37.888 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:37.888 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:37.888 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:37.889 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:37.889 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61285 2024-09-04T17:48:37.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.889 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:37.889 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.889 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:37.889 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835200+0000", 2024-09-04T17:48:37.889 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.889 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.889 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:37.889 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835200+0000", 2024-09-04T17:48:37.890 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.890 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.890 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:37.890 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835200+0000", 2024-09-04T17:48:37.890 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0899999999999997e-07 2024-09-04T17:48:37.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.890 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.890 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:37.890 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835206+0000", 2024-09-04T17:48:37.890 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3280000000000002e-06 2024-09-04T17:48:37.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.891 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.891 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:37.891 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835211+0000", 2024-09-04T17:48:37.891 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9230000000000001e-06 2024-09-04T17:48:37.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.891 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.891 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:37.891 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835216+0000", 2024-09-04T17:48:37.891 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9019999999999998e-06 2024-09-04T17:48:37.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.891 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.891 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:37.891 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.901719+0000", 2024-09-04T17:48:37.891 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066502981000000003 2024-09-04T17:48:37.892 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.892 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.892 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:37.892 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.901813+0000", 2024-09-04T17:48:37.892 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3905000000000003e-05 2024-09-04T17:48:37.892 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.892 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.892 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:37.892 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902100+0000", 2024-09-04T17:48:37.892 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028645599999999998 2024-09-04T17:48:37.892 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.892 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:37.892 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.893 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.893 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61286 3.8 3:117cfeb9:::rbd_data.11341dfde0c5.0000000000000006:1f [read 2867200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:37.893 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835244+0000", 2024-09-04T17:48:37.893 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 84.000261698000003, 2024-09-04T17:48:37.893 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067143215000000006, 2024-09-04T17:48:37.893 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:37.893 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:37.893 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:37.893 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:37.893 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:37.893 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:37.893 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61286 2024-09-04T17:48:37.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.894 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:37.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.894 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:37.894 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835244+0000", 2024-09-04T17:48:37.894 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.894 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:37.894 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835244+0000", 2024-09-04T17:48:37.894 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:37.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835245+0000", 2024-09-04T17:48:37.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8400000000000004e-07 2024-09-04T17:48:37.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:37.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835248+0000", 2024-09-04T17:48:37.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6850000000000001e-06 2024-09-04T17:48:37.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:37.896 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835254+0000", 2024-09-04T17:48:37.896 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.494e-06 2024-09-04T17:48:37.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.896 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:37.896 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835260+0000", 2024-09-04T17:48:37.896 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0580000000000003e-06 2024-09-04T17:48:37.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.896 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:37.896 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902119+0000", 2024-09-04T17:48:37.896 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066858318 2024-09-04T17:48:37.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.897 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:37.897 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902163+0000", 2024-09-04T17:48:37.897 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4753000000000003e-05 2024-09-04T17:48:37.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.897 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.897 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:37.897 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902388+0000", 2024-09-04T17:48:37.897 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022422300000000001 2024-09-04T17:48:37.897 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.897 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:37.897 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.897 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.898 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61315 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 98304~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:37.898 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835269+0000", 2024-09-04T17:48:37.898 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 84.000236905999998, 2024-09-04T17:48:37.898 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067329396, 2024-09-04T17:48:37.898 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:37.898 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:37.898 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:37.898 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:37.898 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:37.898 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:37.898 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61315 2024-09-04T17:48:37.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.898 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:37.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.899 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:37.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835269+0000", 2024-09-04T17:48:37.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.899 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:37.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835269+0000", 2024-09-04T17:48:37.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.899 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:37.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835270+0000", 2024-09-04T17:48:37.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6599999999999996e-07 2024-09-04T17:48:37.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.900 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:37.900 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835272+0000", 2024-09-04T17:48:37.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.261e-06 2024-09-04T17:48:37.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.900 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:37.900 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835278+0000", 2024-09-04T17:48:37.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.443e-06 2024-09-04T17:48:37.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:37.901 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835284+0000", 2024-09-04T17:48:37.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3290000000000001e-06 2024-09-04T17:48:37.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:37.901 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902147+0000", 2024-09-04T17:48:37.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066862695999999999 2024-09-04T17:48:37.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:37.901 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902248+0000", 2024-09-04T17:48:37.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000101501 2024-09-04T17:48:37.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.902 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:37.902 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902599+0000", 2024-09-04T17:48:37.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00035060000000000001 2024-09-04T17:48:37.902 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.902 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:37.902 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.902 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61316 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 204800~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:37.902 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835291+0000", 2024-09-04T17:48:37.902 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 84.000214886999998, 2024-09-04T17:48:37.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067632052999999998, 2024-09-04T17:48:37.903 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:37.903 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:37.903 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:37.903 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:37.903 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:37.903 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:37.903 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61316 2024-09-04T17:48:37.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.903 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:37.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.903 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:37.903 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835291+0000", 2024-09-04T17:48:37.903 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:37.904 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835291+0000", 2024-09-04T17:48:37.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:37.904 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835292+0000", 2024-09-04T17:48:37.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1899999999999998e-07 2024-09-04T17:48:37.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:37.905 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835294+0000", 2024-09-04T17:48:37.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0880000000000002e-06 2024-09-04T17:48:37.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.905 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:37.905 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835300+0000", 2024-09-04T17:48:37.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.224e-06 2024-09-04T17:48:37.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.905 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:37.905 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835305+0000", 2024-09-04T17:48:37.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0730000000000004e-06 2024-09-04T17:48:37.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.906 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:37.906 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902632+0000", 2024-09-04T17:48:37.906 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067327322999999994 2024-09-04T17:48:37.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.906 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.906 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:37.906 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902688+0000", 2024-09-04T17:48:37.906 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5177999999999999e-05 2024-09-04T17:48:37.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.906 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.906 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:37.906 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902923+0000", 2024-09-04T17:48:37.906 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023574800000000001 2024-09-04T17:48:37.907 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.907 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:37.907 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.907 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.907 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.907 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61317 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 344064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:37.907 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835312+0000", 2024-09-04T17:48:37.907 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 84.000193882999994, 2024-09-04T17:48:37.907 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067918511000000001, 2024-09-04T17:48:37.907 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:37.907 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:37.907 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:37.907 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:37.908 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:37.908 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:37.908 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61317 2024-09-04T17:48:37.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.908 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:37.908 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.908 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:37.908 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835312+0000", 2024-09-04T17:48:37.908 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.908 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.908 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:37.909 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835312+0000", 2024-09-04T17:48:37.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.909 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.909 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:37.909 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835313+0000", 2024-09-04T17:48:37.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9000000000000002e-07 2024-09-04T17:48:37.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.909 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.909 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:37.909 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835315+0000", 2024-09-04T17:48:37.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1689999999999999e-06 2024-09-04T17:48:37.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.910 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:37.910 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835321+0000", 2024-09-04T17:48:37.910 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0619999999999997e-06 2024-09-04T17:48:37.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.910 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:37.910 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835326+0000", 2024-09-04T17:48:37.910 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0390000000000004e-06 2024-09-04T17:48:37.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.910 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:37.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902967+0000", 2024-09-04T17:48:37.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067641520999999996 2024-09-04T17:48:37.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:37.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903067+0000", 2024-09-04T17:48:37.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9426999999999995e-05 2024-09-04T17:48:37.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:37.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903231+0000", 2024-09-04T17:48:37.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016390299999999999 2024-09-04T17:48:37.911 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.912 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:37.912 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.912 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.912 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61318 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 450560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:37.912 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835334+0000", 2024-09-04T17:48:37.912 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 84.000172366000001, 2024-09-04T17:48:37.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068091771999999995, 2024-09-04T17:48:37.912 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:37.912 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:37.912 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:37.912 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:37.912 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:37.913 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:37.913 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61318 2024-09-04T17:48:37.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.913 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:37.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.913 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:37.913 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835334+0000", 2024-09-04T17:48:37.913 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.913 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:37.913 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835334+0000", 2024-09-04T17:48:37.913 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.914 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:37.914 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835334+0000", 2024-09-04T17:48:37.914 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5699999999999998e-07 2024-09-04T17:48:37.914 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.914 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:37.914 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835337+0000", 2024-09-04T17:48:37.914 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3889999999999999e-06 2024-09-04T17:48:37.914 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.914 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:37.914 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835343+0000", 2024-09-04T17:48:37.914 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1809999999999998e-06 2024-09-04T17:48:37.914 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.915 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:37.915 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835348+0000", 2024-09-04T17:48:37.915 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3170000000000001e-06 2024-09-04T17:48:37.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.915 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:37.915 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903270+0000", 2024-09-04T17:48:37.915 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067922191000000007 2024-09-04T17:48:37.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.915 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:37.915 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903321+0000", 2024-09-04T17:48:37.915 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0889e-05 2024-09-04T17:48:37.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.916 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:37.916 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903426+0000", 2024-09-04T17:48:37.916 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104348 2024-09-04T17:48:37.916 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.916 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:37.916 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.916 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61319 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 589824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:37.916 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835357+0000", 2024-09-04T17:48:37.916 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 84.000149549, 2024-09-04T17:48:37.917 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068372535999999998, 2024-09-04T17:48:37.917 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:37.917 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:37.917 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:37.917 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:37.917 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:37.917 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:37.917 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61319 2024-09-04T17:48:37.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.917 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:37.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.917 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:37.918 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835357+0000", 2024-09-04T17:48:37.918 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.918 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.918 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.918 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:37.918 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835357+0000", 2024-09-04T17:48:37.918 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.918 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.918 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.918 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:37.918 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835357+0000", 2024-09-04T17:48:37.918 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.32e-07 2024-09-04T17:48:37.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.919 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.919 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:37.919 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835359+0000", 2024-09-04T17:48:37.919 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1660000000000001e-06 2024-09-04T17:48:37.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.919 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.919 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:37.919 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835366+0000", 2024-09-04T17:48:37.919 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3550000000000001e-06 2024-09-04T17:48:37.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.919 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.920 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:37.920 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835371+0000", 2024-09-04T17:48:37.920 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1050000000000003e-06 2024-09-04T17:48:37.920 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.920 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:37.920 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903471+0000", 2024-09-04T17:48:37.920 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068100451000000006 2024-09-04T17:48:37.920 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.920 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:37.920 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903573+0000", 2024-09-04T17:48:37.921 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000101691 2024-09-04T17:48:37.921 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.921 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.921 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:37.921 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903729+0000", 2024-09-04T17:48:37.921 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000156336 2024-09-04T17:48:37.921 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.921 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:37.921 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.921 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.921 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.921 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61320 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:37.921 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835380+0000", 2024-09-04T17:48:37.922 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 84.000125784000005, 2024-09-04T17:48:37.922 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068543058000000004, 2024-09-04T17:48:37.922 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:37.922 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:37.922 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:37.922 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:37.922 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:37.922 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:37.922 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61320 2024-09-04T17:48:37.922 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.922 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:37.922 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.922 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:37.922 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835380+0000", 2024-09-04T17:48:37.923 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.923 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.923 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:37.923 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835380+0000", 2024-09-04T17:48:37.923 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.923 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.923 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:37.923 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835381+0000", 2024-09-04T17:48:37.923 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9999999999999998e-07 2024-09-04T17:48:37.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.924 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:37.924 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835383+0000", 2024-09-04T17:48:37.924 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2230000000000001e-06 2024-09-04T17:48:37.924 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.924 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:37.924 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835390+0000", 2024-09-04T17:48:37.924 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.34e-06 2024-09-04T17:48:37.924 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.924 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:37.924 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835396+0000", 2024-09-04T17:48:37.924 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9830000000000001e-06 2024-09-04T17:48:37.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:37.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903765+0000", 2024-09-04T17:48:37.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068368441000000002 2024-09-04T17:48:37.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:37.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903835+0000", 2024-09-04T17:48:37.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0631999999999997e-05 2024-09-04T17:48:37.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:37.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903923+0000", 2024-09-04T17:48:37.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8039000000000002e-05 2024-09-04T17:48:37.926 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.926 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:37.926 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.926 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.926 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61321 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 835584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:37.926 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835404+0000", 2024-09-04T17:48:37.926 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 84.000101724999993, 2024-09-04T17:48:37.926 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068911043000000005, 2024-09-04T17:48:37.926 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:37.926 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:37.926 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:37.926 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:37.926 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:37.927 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:37.927 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61321 2024-09-04T17:48:37.927 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.927 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:37.927 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.927 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:37.927 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835404+0000", 2024-09-04T17:48:37.927 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.927 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.927 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.927 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:37.927 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835404+0000", 2024-09-04T17:48:37.927 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.927 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.928 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.928 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:37.928 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835405+0000", 2024-09-04T17:48:37.928 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.89e-07 2024-09-04T17:48:37.928 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.928 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.928 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:37.928 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835407+0000", 2024-09-04T17:48:37.928 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4439999999999998e-06 2024-09-04T17:48:37.928 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.928 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.928 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:37.929 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835414+0000", 2024-09-04T17:48:37.929 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4930000000000003e-06 2024-09-04T17:48:37.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.929 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:37.929 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835419+0000", 2024-09-04T17:48:37.929 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8919999999999999e-06 2024-09-04T17:48:37.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.929 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:37.929 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903957+0000", 2024-09-04T17:48:37.929 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068538500000000002 2024-09-04T17:48:37.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.930 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:37.930 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904019+0000", 2024-09-04T17:48:37.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1419000000000003e-05 2024-09-04T17:48:37.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.930 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:37.930 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904315+0000", 2024-09-04T17:48:37.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00029680599999999999 2024-09-04T17:48:37.930 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.930 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:37.930 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.931 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.931 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61322 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:37.931 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835427+0000", 2024-09-04T17:48:37.931 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 84.000079338999996, 2024-09-04T17:48:37.931 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.069550138999999997, 2024-09-04T17:48:37.931 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:37.931 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:37.931 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:37.931 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:37.931 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:37.931 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:37.931 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61322 2024-09-04T17:48:37.932 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.932 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:37.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.932 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:37.932 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835427+0000", 2024-09-04T17:48:37.932 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.932 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.932 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:37.932 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835427+0000", 2024-09-04T17:48:37.932 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:37.932 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.932 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:37.933 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.842157+0000", 2024-09-04T17:48:37.933 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067298419999999998 2024-09-04T17:48:37.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.933 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.933 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:37.933 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.842163+0000", 2024-09-04T17:48:37.933 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2029999999999997e-06 2024-09-04T17:48:37.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.933 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.933 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:37.933 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.842175+0000", 2024-09-04T17:48:37.933 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2072000000000001e-05 2024-09-04T17:48:37.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.933 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.934 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:37.934 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.842183+0000", 2024-09-04T17:48:37.934 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3850000000000002e-06 2024-09-04T17:48:37.934 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.934 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.934 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:37.934 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904371+0000", 2024-09-04T17:48:37.934 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.062187754999999997 2024-09-04T17:48:37.934 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.934 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.934 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:37.934 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904467+0000", 2024-09-04T17:48:37.934 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6117000000000001e-05 2024-09-04T17:48:37.934 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:37.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:37.935 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:37.935 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904977+0000", 2024-09-04T17:48:37.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00050976500000000002 2024-09-04T17:48:37.935 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.935 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:37.935 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.935 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:37.935 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:37.935 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:37.936 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is unset 2024-09-04T17:48:37.937 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-04T17:48:37.938 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:48:37.938 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:48:37.938 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:48:37.939 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:48:37.939 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-04T17:48:37.939 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-04T17:48:37.939 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-04T17:48:37.940 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-04T17:48:37.940 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:37.931+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:37.946 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:48:37.946 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:48:37.965 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:48:37.965 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:48:37.989 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-04T17:48:37.989 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-04T17:48:38.007 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:48:38.007 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:48:38.037 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:38.037+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:38.082 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:38.083 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:38.097 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-04T17:48:38.138 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:48:38.135+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:48:38.239 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:38.235+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:38.263 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:38.263 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:38.277 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-04T17:48:38.340 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:38.335+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:38.402 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:38.402 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:38.418 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-04T17:48:38.465 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:48:38.465+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:48:38.541 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:38.542 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:38.542 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:48:38.539+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:48:38.556 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-04T17:48:38.642 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:38.639+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:38.722 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:38.722 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:38.736 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-04T17:48:38.781 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:38.781+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:38.844 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:48:38.841+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:48:38.859 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:38.859 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:38.873 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-04T17:48:38.944 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:38.939+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:39.039 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:39.039 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:39.045 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:48:39.045+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:48:39.054 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-04T17:48:39.146 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:48:39.143+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:48:39.227 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:39.228 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:39.242 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-04T17:48:39.289 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:39.289+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:39.348 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:39.345+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:39.365 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:39.366 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:39.380 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-04T17:48:39.449 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:39.445+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:39.512 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:39.512 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:39.526 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-04T17:48:39.573 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:48:39.571+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:48:39.644 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:39.645 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:39.651 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:48:39.649+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:48:39.660 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-04T17:48:39.753 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:48:39.751+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:48:39.825 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:39.825 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:39.839 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-04T17:48:39.881 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:48:39.879+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:48:39.938 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-04T17:48:39.954 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:48:39.953+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:48:40.018 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:40.018 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:40.025 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-04T17:48:40.035 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-04T17:48:40.054 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:48:40.053+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:48:40.156 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:48:40.153+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:48:40.170 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:48:40.170 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:48:40.199 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:48:40.199 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:48:40.206 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:40.206 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:40.217 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-04T17:48:40.218 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-04T17:48:40.221 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-04T17:48:40.235 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-04T17:48:40.235 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-04T17:48:40.256 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:48:40.256 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:48:40.257 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:48:40.251+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:48:40.277 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:48:40.277 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:48:40.297 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-04T17:48:40.297 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-04T17:48:40.318 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:48:40.318 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:48:40.337 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:40.337 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:48:40.337 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:48:40.337 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:48:40.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.338 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:71638 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-04T17:48:40.338 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:44.479908+0000", 2024-09-04T17:48:40.338 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.856489768000003, 2024-09-04T17:48:40.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.469396488, 2024-09-04T17:48:40.338 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:40.338 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:40.338 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:40.338 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:40.338 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:40.338 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:40.338 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 71638 2024-09-04T17:48:40.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.339 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:40.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.339 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:40.339 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.479908+0000", 2024-09-04T17:48:40.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:40.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.339 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:40.339 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.479908+0000", 2024-09-04T17:48:40.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:40.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.340 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:40.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.479911+0000", 2024-09-04T17:48:40.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9160000000000001e-06 2024-09-04T17:48:40.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.340 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:40.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.479925+0000", 2024-09-04T17:48:40.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3896e-05 2024-09-04T17:48:40.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.340 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:40.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.479954+0000", 2024-09-04T17:48:40.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9249000000000001e-05 2024-09-04T17:48:40.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.341 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:40.341 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.479969+0000", 2024-09-04T17:48:40.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4683e-05 2024-09-04T17:48:40.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.341 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:40.341 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.480109+0000", 2024-09-04T17:48:40.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013995699999999999 2024-09-04T17:48:40.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.342 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-04T17:48:40.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.480125+0000", 2024-09-04T17:48:40.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6651999999999999e-05 2024-09-04T17:48:40.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.342 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:40.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.948598+0000", 2024-09-04T17:48:40.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.46847249600000002 2024-09-04T17:48:40.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.342 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:40.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.948889+0000", 2024-09-04T17:48:40.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000290947 2024-09-04T17:48:40.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.343 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:40.343 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:44.949304+0000", 2024-09-04T17:48:40.343 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000415692 2024-09-04T17:48:40.343 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:40.343 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:40.343 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:40.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.343 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:72267 3.1a 3:58dbd2e1:::rbd_data.11342ece613c.000000000000001e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-09-04T17:48:40.343 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:45.674662+0000", 2024-09-04T17:48:40.343 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 54.661735958999998, 2024-09-04T17:48:40.344 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.032248339000000001, 2024-09-04T17:48:40.344 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:40.344 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:40.344 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:40.344 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:40.344 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:40.344 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:40.344 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 72267 2024-09-04T17:48:40.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.344 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:40.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.344 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:40.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.674662+0000", 2024-09-04T17:48:40.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:40.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.345 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:40.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.674662+0000", 2024-09-04T17:48:40.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:40.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.345 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:40.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.674664+0000", 2024-09-04T17:48:40.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4870000000000001e-06 2024-09-04T17:48:40.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:40.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.674689+0000", 2024-09-04T17:48:40.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4805000000000001e-05 2024-09-04T17:48:40.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:40.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.674715+0000", 2024-09-04T17:48:40.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5548999999999999e-05 2024-09-04T17:48:40.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:40.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.674731+0000", 2024-09-04T17:48:40.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6549999999999999e-05 2024-09-04T17:48:40.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:40.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.674843+0000", 2024-09-04T17:48:40.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000112001 2024-09-04T17:48:40.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:40.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.675176+0000", 2024-09-04T17:48:40.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00033337200000000002 2024-09-04T17:48:40.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-04T17:48:40.348 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.680121+0000", 2024-09-04T17:48:40.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0049445540000000003 2024-09-04T17:48:40.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:40.348 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.706702+0000", 2024-09-04T17:48:40.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026580840000000001 2024-09-04T17:48:40.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:40.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.706738+0000", 2024-09-04T17:48:40.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6158999999999997e-05 2024-09-04T17:48:40.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:40.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.706835+0000", 2024-09-04T17:48:40.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6909000000000003e-05 2024-09-04T17:48:40.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:40.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:45.706910+0000", 2024-09-04T17:48:40.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5112999999999997e-05 2024-09-04T17:48:40.350 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:40.350 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:40.350 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:40.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.350 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94568 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 1703936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:40.350 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.605863+0000", 2024-09-04T17:48:40.350 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.730535144000001, 2024-09-04T17:48:40.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027249301, 2024-09-04T17:48:40.350 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:40.350 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:40.350 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:40.351 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:40.351 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:40.351 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:40.351 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94568 2024-09-04T17:48:40.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.351 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:40.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:40.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605863+0000", 2024-09-04T17:48:40.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:40.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:40.352 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605863+0000", 2024-09-04T17:48:40.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:40.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:40.352 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605863+0000", 2024-09-04T17:48:40.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7999999999999996e-07 2024-09-04T17:48:40.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:40.352 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605866+0000", 2024-09-04T17:48:40.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6199999999999999e-06 2024-09-04T17:48:40.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:40.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605875+0000", 2024-09-04T17:48:40.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2499999999999995e-06 2024-09-04T17:48:40.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:40.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605881+0000", 2024-09-04T17:48:40.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.19e-06 2024-09-04T17:48:40.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:40.354 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.632629+0000", 2024-09-04T17:48:40.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026747567 2024-09-04T17:48:40.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.354 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:40.354 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.632874+0000", 2024-09-04T17:48:40.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024493600000000001 2024-09-04T17:48:40.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.354 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:40.354 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.633112+0000", 2024-09-04T17:48:40.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023825800000000001 2024-09-04T17:48:40.354 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:40.355 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:40.355 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:40.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.355 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.355 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94569 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:40.355 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.605888+0000", 2024-09-04T17:48:40.355 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.730509314999999, 2024-09-04T17:48:40.355 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027675620000000001, 2024-09-04T17:48:40.355 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:40.355 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:40.355 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:40.355 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:40.355 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:40.356 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:40.356 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94569 2024-09-04T17:48:40.356 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.356 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:40.356 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.356 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:40.356 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605888+0000", 2024-09-04T17:48:40.356 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:40.356 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.356 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.356 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:40.356 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605888+0000", 2024-09-04T17:48:40.356 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:40.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.357 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:40.357 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605889+0000", 2024-09-04T17:48:40.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4799999999999998e-07 2024-09-04T17:48:40.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.357 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:40.357 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605891+0000", 2024-09-04T17:48:40.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4880000000000001e-06 2024-09-04T17:48:40.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.357 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:40.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605901+0000", 2024-09-04T17:48:40.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4760000000000004e-06 2024-09-04T17:48:40.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:40.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605912+0000", 2024-09-04T17:48:40.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0698e-05 2024-09-04T17:48:40.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:40.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.633184+0000", 2024-09-04T17:48:40.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.02727253 2024-09-04T17:48:40.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:40.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.633330+0000", 2024-09-04T17:48:40.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014543800000000001 2024-09-04T17:48:40.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:40.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.633564+0000", 2024-09-04T17:48:40.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023444199999999999 2024-09-04T17:48:40.359 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:40.359 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:40.359 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:40.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.360 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94570 3.0 3.3f52c000 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:40.360 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.605932+0000", 2024-09-04T17:48:40.360 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.730465748, 2024-09-04T17:48:40.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027687021999999999, 2024-09-04T17:48:40.360 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:40.360 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:40.360 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-04T17:48:40.360 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:40.360 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:40.360 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:40.360 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94570 2024-09-04T17:48:40.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.360 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:40.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:40.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605932+0000", 2024-09-04T17:48:40.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:40.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:40.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605932+0000", 2024-09-04T17:48:40.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:40.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:40.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605933+0000", 2024-09-04T17:48:40.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5000000000000001e-07 2024-09-04T17:48:40.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.362 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:40.362 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605936+0000", 2024-09-04T17:48:40.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7769999999999998e-06 2024-09-04T17:48:40.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.362 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:40.362 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605943+0000", 2024-09-04T17:48:40.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7000000000000008e-06 2024-09-04T17:48:40.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.362 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:40.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.605949+0000", 2024-09-04T17:48:40.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0170000000000002e-06 2024-09-04T17:48:40.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:40.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.633604+0000", 2024-09-04T17:48:40.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027654875999999998 2024-09-04T17:48:40.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:40.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.633619+0000", 2024-09-04T17:48:40.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4802e-05 2024-09-04T17:48:40.363 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:40.364 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:40.364 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:40.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.364 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94641 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 6094848~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:40.364 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.637288+0000", 2024-09-04T17:48:40.364 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.699109297, 2024-09-04T17:48:40.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026966783000000001, 2024-09-04T17:48:40.364 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:40.364 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:40.364 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:40.364 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:40.364 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:40.364 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:40.365 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94641 2024-09-04T17:48:40.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.365 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:40.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:40.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.637288+0000", 2024-09-04T17:48:40.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:40.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:40.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.637288+0000", 2024-09-04T17:48:40.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:40.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:40.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.637289+0000", 2024-09-04T17:48:40.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4600000000000005e-07 2024-09-04T17:48:40.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:40.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.637292+0000", 2024-09-04T17:48:40.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1049999999999999e-06 2024-09-04T17:48:40.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:40.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.637301+0000", 2024-09-04T17:48:40.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7509999999999997e-06 2024-09-04T17:48:40.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:40.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.637309+0000", 2024-09-04T17:48:40.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.67e-06 2024-09-04T17:48:40.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:40.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.663971+0000", 2024-09-04T17:48:40.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026661926999999998 2024-09-04T17:48:40.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:40.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.664062+0000", 2024-09-04T17:48:40.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0673000000000004e-05 2024-09-04T17:48:40.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:40.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.664255+0000", 2024-09-04T17:48:40.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019311099999999999 2024-09-04T17:48:40.368 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:40.368 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:40.368 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:40.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.368 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94642 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 6225920~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:40.368 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.637321+0000", 2024-09-04T17:48:40.368 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.699076349999999, 2024-09-04T17:48:40.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027131534999999998, 2024-09-04T17:48:40.369 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:40.369 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:40.369 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:40.369 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:40.369 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:40.369 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:40.369 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94642 2024-09-04T17:48:40.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.369 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:40.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:40.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.637321+0000", 2024-09-04T17:48:40.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:40.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:40.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.637321+0000", 2024-09-04T17:48:40.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:40.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:40.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.637322+0000", 2024-09-04T17:48:40.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6599999999999996e-07 2024-09-04T17:48:40.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:40.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.637325+0000", 2024-09-04T17:48:40.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1070000000000001e-06 2024-09-04T17:48:40.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:40.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.637337+0000", 2024-09-04T17:48:40.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1766000000000001e-05 2024-09-04T17:48:40.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:40.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.637350+0000", 2024-09-04T17:48:40.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3515e-05 2024-09-04T17:48:40.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:40.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.664273+0000", 2024-09-04T17:48:40.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026922643 2024-09-04T17:48:40.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:40.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.664337+0000", 2024-09-04T17:48:40.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4492999999999994e-05 2024-09-04T17:48:40.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:40.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.664453+0000", 2024-09-04T17:48:40.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000115345 2024-09-04T17:48:40.373 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:40.373 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:40.373 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:40.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.373 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94647 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 6684672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:40.373 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.638572+0000", 2024-09-04T17:48:40.373 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.697825463999997, 2024-09-04T17:48:40.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026910377999999999, 2024-09-04T17:48:40.373 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:40.373 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:40.374 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:40.374 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:40.374 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:40.374 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:40.374 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94647 2024-09-04T17:48:40.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.374 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:40.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:40.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638572+0000", 2024-09-04T17:48:40.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:40.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:40.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638572+0000", 2024-09-04T17:48:40.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:40.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:40.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638573+0000", 2024-09-04T17:48:40.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7599999999999997e-07 2024-09-04T17:48:40.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:40.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638575+0000", 2024-09-04T17:48:40.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2010000000000002e-06 2024-09-04T17:48:40.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:40.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638582+0000", 2024-09-04T17:48:40.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7399999999999998e-06 2024-09-04T17:48:40.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:40.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638588+0000", 2024-09-04T17:48:40.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8560000000000003e-06 2024-09-04T17:48:40.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:40.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.665294+0000", 2024-09-04T17:48:40.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026705893000000001 2024-09-04T17:48:40.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:40.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.665355+0000", 2024-09-04T17:48:40.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1253000000000005e-05 2024-09-04T17:48:40.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:40.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.665483+0000", 2024-09-04T17:48:40.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012775899999999999 2024-09-04T17:48:40.377 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:40.377 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:40.378 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:40.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.378 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94648 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 6815744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:40.378 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.638593+0000", 2024-09-04T17:48:40.378 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.697804376000001, 2024-09-04T17:48:40.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027163676000000001, 2024-09-04T17:48:40.378 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:40.378 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:40.378 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:40.378 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:40.378 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:40.378 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:40.379 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94648 2024-09-04T17:48:40.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.379 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:40.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:40.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638593+0000", 2024-09-04T17:48:40.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:40.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:40.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638593+0000", 2024-09-04T17:48:40.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:40.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:40.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638594+0000", 2024-09-04T17:48:40.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.53e-07 2024-09-04T17:48:40.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:40.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638596+0000", 2024-09-04T17:48:40.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8870000000000001e-06 2024-09-04T17:48:40.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:40.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638600+0000", 2024-09-04T17:48:40.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3290000000000004e-06 2024-09-04T17:48:40.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:40.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638603+0000", 2024-09-04T17:48:40.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4680000000000001e-06 2024-09-04T17:48:40.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:40.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.665524+0000", 2024-09-04T17:48:40.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.02692052 2024-09-04T17:48:40.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:40.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.665600+0000", 2024-09-04T17:48:40.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6569000000000005e-05 2024-09-04T17:48:40.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:40.382 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.665757+0000", 2024-09-04T17:48:40.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015655000000000001 2024-09-04T17:48:40.382 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:40.382 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:40.382 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:40.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.382 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94649 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 6946816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:40.382 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.638608+0000", 2024-09-04T17:48:40.382 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.697789946, 2024-09-04T17:48:40.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027355753999999999, 2024-09-04T17:48:40.383 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:40.383 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:40.383 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:40.383 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:40.383 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:40.383 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:40.383 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94649 2024-09-04T17:48:40.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.383 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:40.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:40.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638608+0000", 2024-09-04T17:48:40.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:40.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:40.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638608+0000", 2024-09-04T17:48:40.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:40.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:40.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638608+0000", 2024-09-04T17:48:40.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3099999999999998e-07 2024-09-04T17:48:40.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:40.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638610+0000", 2024-09-04T17:48:40.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.716e-06 2024-09-04T17:48:40.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:40.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638614+0000", 2024-09-04T17:48:40.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.941e-06 2024-09-04T17:48:40.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:40.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638618+0000", 2024-09-04T17:48:40.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5009999999999998e-06 2024-09-04T17:48:40.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:40.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.665795+0000", 2024-09-04T17:48:40.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027176404000000001 2024-09-04T17:48:40.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:40.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.665851+0000", 2024-09-04T17:48:40.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6530999999999998e-05 2024-09-04T17:48:40.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:40.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.665963+0000", 2024-09-04T17:48:40.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011213 2024-09-04T17:48:40.387 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:40.387 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:40.387 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:40.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.387 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94650 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 7077888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:40.387 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.638623+0000", 2024-09-04T17:48:40.387 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.697774649999999, 2024-09-04T17:48:40.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027665261999999999, 2024-09-04T17:48:40.387 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:40.387 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:40.387 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:40.387 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:40.387 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:40.388 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:40.388 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94650 2024-09-04T17:48:40.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.388 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:40.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:40.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638623+0000", 2024-09-04T17:48:40.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:40.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:40.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638623+0000", 2024-09-04T17:48:40.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:40.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:40.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638623+0000", 2024-09-04T17:48:40.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6300000000000001e-07 2024-09-04T17:48:40.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:40.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638625+0000", 2024-09-04T17:48:40.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7260000000000001e-06 2024-09-04T17:48:40.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:40.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638629+0000", 2024-09-04T17:48:40.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1309999999999999e-06 2024-09-04T17:48:40.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:40.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638634+0000", 2024-09-04T17:48:40.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2700000000000004e-06 2024-09-04T17:48:40.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:40.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.666028+0000", 2024-09-04T17:48:40.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027393303000000001 2024-09-04T17:48:40.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:40.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.666112+0000", 2024-09-04T17:48:40.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3844000000000006e-05 2024-09-04T17:48:40.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:40.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.666288+0000", 2024-09-04T17:48:40.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000176725 2024-09-04T17:48:40.391 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:40.391 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:40.391 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:40.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.392 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94651 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 7208960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:40.392 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.638639+0000", 2024-09-04T17:48:40.392 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.697758256, 2024-09-04T17:48:40.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027839592999999999, 2024-09-04T17:48:40.392 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:40.392 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:40.392 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:40.392 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:40.392 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:40.392 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:40.392 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94651 2024-09-04T17:48:40.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.393 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:40.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:40.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638639+0000", 2024-09-04T17:48:40.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:40.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:40.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638639+0000", 2024-09-04T17:48:40.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:40.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:40.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638640+0000", 2024-09-04T17:48:40.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8000000000000002e-07 2024-09-04T17:48:40.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:40.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638641+0000", 2024-09-04T17:48:40.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.77e-06 2024-09-04T17:48:40.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:40.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638646+0000", 2024-09-04T17:48:40.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1729999999999997e-06 2024-09-04T17:48:40.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:40.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.638649+0000", 2024-09-04T17:48:40.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5260000000000002e-06 2024-09-04T17:48:40.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:40.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.666309+0000", 2024-09-04T17:48:40.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027660179 2024-09-04T17:48:40.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:40.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.666373+0000", 2024-09-04T17:48:40.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4152999999999994e-05 2024-09-04T17:48:40.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:40.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.666479+0000", 2024-09-04T17:48:40.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000105512 2024-09-04T17:48:40.396 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:40.396 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:40.396 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:40.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.396 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94657 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 7012352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:40.396 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.641793+0000", 2024-09-04T17:48:40.397 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.694604771999998, 2024-09-04T17:48:40.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026893672, 2024-09-04T17:48:40.397 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:40.397 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:40.397 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:40.397 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:40.397 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:40.397 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:40.397 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94657 2024-09-04T17:48:40.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.397 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:40.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:40.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641793+0000", 2024-09-04T17:48:40.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:40.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:40.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641793+0000", 2024-09-04T17:48:40.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:40.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:40.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641793+0000", 2024-09-04T17:48:40.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5000000000000003e-07 2024-09-04T17:48:40.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:40.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641797+0000", 2024-09-04T17:48:40.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2909999999999999e-06 2024-09-04T17:48:40.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:40.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641806+0000", 2024-09-04T17:48:40.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3270000000000007e-06 2024-09-04T17:48:40.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:40.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641812+0000", 2024-09-04T17:48:40.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2310000000000001e-06 2024-09-04T17:48:40.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:40.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.668412+0000", 2024-09-04T17:48:40.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026600104999999999 2024-09-04T17:48:40.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:40.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.668534+0000", 2024-09-04T17:48:40.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012115100000000001 2024-09-04T17:48:40.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:40.401 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.668687+0000", 2024-09-04T17:48:40.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000153017 2024-09-04T17:48:40.401 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:40.401 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:40.401 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:40.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.401 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94658 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 7143424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:40.401 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.641822+0000", 2024-09-04T17:48:40.401 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.694575841000002, 2024-09-04T17:48:40.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027395880000000001, 2024-09-04T17:48:40.401 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:40.402 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:40.402 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:40.402 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:40.402 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:40.402 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:40.402 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94658 2024-09-04T17:48:40.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.402 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:40.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:40.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641822+0000", 2024-09-04T17:48:40.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:40.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:40.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641822+0000", 2024-09-04T17:48:40.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:40.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:40.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641822+0000", 2024-09-04T17:48:40.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.58e-07 2024-09-04T17:48:40.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:40.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641825+0000", 2024-09-04T17:48:40.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5909999999999998e-06 2024-09-04T17:48:40.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:40.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641832+0000", 2024-09-04T17:48:40.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1799999999999999e-06 2024-09-04T17:48:40.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:40.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641838+0000", 2024-09-04T17:48:40.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5040000000000002e-06 2024-09-04T17:48:40.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:40.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.668748+0000", 2024-09-04T17:48:40.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026910119999999999 2024-09-04T17:48:40.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:40.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.668896+0000", 2024-09-04T17:48:40.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014804299999999999 2024-09-04T17:48:40.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:40.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.669218+0000", 2024-09-04T17:48:40.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00032198399999999999 2024-09-04T17:48:40.405 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:40.405 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:40.406 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:40.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.406 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94659 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 7274496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:40.406 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.641846+0000", 2024-09-04T17:48:40.406 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.694551771, 2024-09-04T17:48:40.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027757314000000002, 2024-09-04T17:48:40.406 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:40.406 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:40.406 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:40.406 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:40.406 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:40.406 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:40.407 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94659 2024-09-04T17:48:40.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.407 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:40.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:40.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641846+0000", 2024-09-04T17:48:40.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:40.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:40.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641846+0000", 2024-09-04T17:48:40.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:40.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:40.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641846+0000", 2024-09-04T17:48:40.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5400000000000002e-07 2024-09-04T17:48:40.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:40.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641849+0000", 2024-09-04T17:48:40.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0170000000000001e-06 2024-09-04T17:48:40.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:40.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641858+0000", 2024-09-04T17:48:40.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1499999999999999e-06 2024-09-04T17:48:40.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:40.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.641864+0000", 2024-09-04T17:48:40.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2280000000000003e-06 2024-09-04T17:48:40.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:40.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.669278+0000", 2024-09-04T17:48:40.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027413843 2024-09-04T17:48:40.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:40.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.669392+0000", 2024-09-04T17:48:40.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000114654 2024-09-04T17:48:40.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:40.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.669603+0000", 2024-09-04T17:48:40.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021096799999999999 2024-09-04T17:48:40.410 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:40.410 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:40.410 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:40.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.410 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94664 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 7864320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:40.410 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.644474+0000", 2024-09-04T17:48:40.410 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.691924090999997, 2024-09-04T17:48:40.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026965042000000002, 2024-09-04T17:48:40.411 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:40.411 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:40.411 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:40.411 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:40.411 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:40.411 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:40.411 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94664 2024-09-04T17:48:40.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.411 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:40.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.411 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:40.411 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644474+0000", 2024-09-04T17:48:40.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:40.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:40.412 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644474+0000", 2024-09-04T17:48:40.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:40.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:40.412 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644474+0000", 2024-09-04T17:48:40.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.16e-07 2024-09-04T17:48:40.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:40.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644478+0000", 2024-09-04T17:48:40.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1389999999999999e-06 2024-09-04T17:48:40.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:40.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644494+0000", 2024-09-04T17:48:40.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6977e-05 2024-09-04T17:48:40.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:40.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644501+0000", 2024-09-04T17:48:40.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.438e-06 2024-09-04T17:48:40.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:40.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.671241+0000", 2024-09-04T17:48:40.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.026740525000000001 2024-09-04T17:48:40.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:40.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.671325+0000", 2024-09-04T17:48:40.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3987999999999996e-05 2024-09-04T17:48:40.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:40.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.671439+0000", 2024-09-04T17:48:40.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011315900000000001 2024-09-04T17:48:40.415 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:40.415 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:40.415 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:40.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.415 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94665 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 7995392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:40.415 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.644511+0000", 2024-09-04T17:48:40.415 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.691887059000003, 2024-09-04T17:48:40.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027156390999999998, 2024-09-04T17:48:40.415 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:40.415 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:40.416 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:40.416 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:40.416 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:40.416 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:40.416 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94665 2024-09-04T17:48:40.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.416 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:40.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:40.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644511+0000", 2024-09-04T17:48:40.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:40.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:40.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644511+0000", 2024-09-04T17:48:40.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:40.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:40.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644511+0000", 2024-09-04T17:48:40.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2699999999999999e-07 2024-09-04T17:48:40.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:40.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644514+0000", 2024-09-04T17:48:40.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9890000000000001e-06 2024-09-04T17:48:40.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:40.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644521+0000", 2024-09-04T17:48:40.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5409999999999997e-06 2024-09-04T17:48:40.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:40.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644527+0000", 2024-09-04T17:48:40.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0340000000000002e-06 2024-09-04T17:48:40.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:40.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.671474+0000", 2024-09-04T17:48:40.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.02694767 2024-09-04T17:48:40.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:40.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.671529+0000", 2024-09-04T17:48:40.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4373000000000003e-05 2024-09-04T17:48:40.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:40.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.671667+0000", 2024-09-04T17:48:40.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000138257 2024-09-04T17:48:40.419 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:40.419 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:40.420 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:40.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.420 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94666 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 8126464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:40.420 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.644536+0000", 2024-09-04T17:48:40.420 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.691861920999997, 2024-09-04T17:48:40.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027360524000000001, 2024-09-04T17:48:40.420 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:40.420 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:40.420 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:40.420 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:40.420 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:40.420 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:40.421 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94666 2024-09-04T17:48:40.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.421 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:40.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:40.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644536+0000", 2024-09-04T17:48:40.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:40.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:40.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644536+0000", 2024-09-04T17:48:40.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:40.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:40.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644536+0000", 2024-09-04T17:48:40.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7000000000000004e-07 2024-09-04T17:48:40.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:40.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644540+0000", 2024-09-04T17:48:40.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1609999999999999e-06 2024-09-04T17:48:40.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:40.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644552+0000", 2024-09-04T17:48:40.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2794e-05 2024-09-04T17:48:40.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:40.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644559+0000", 2024-09-04T17:48:40.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5010000000000001e-06 2024-09-04T17:48:40.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:40.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.671703+0000", 2024-09-04T17:48:40.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027144492999999999 2024-09-04T17:48:40.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:40.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.671779+0000", 2024-09-04T17:48:40.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5578e-05 2024-09-04T17:48:40.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:40.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.671896+0000", 2024-09-04T17:48:40.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000117327 2024-09-04T17:48:40.424 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:40.424 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:40.424 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:40.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.425 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:94667 3.0 3:00034afc:::rbd_data.1134878eadad.0000000000000006:2d [sparse-read 8257536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:40.425 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.644570+0000", 2024-09-04T17:48:40.425 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.691827257, 2024-09-04T17:48:40.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027556085000000001, 2024-09-04T17:48:40.425 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:40.425 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:40.425 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:40.425 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:40.425 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:40.425 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:40.425 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 94667 2024-09-04T17:48:40.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.426 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:40.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:40.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644570+0000", 2024-09-04T17:48:40.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:40.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:40.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644570+0000", 2024-09-04T17:48:40.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:40.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:40.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644571+0000", 2024-09-04T17:48:40.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5199999999999997e-07 2024-09-04T17:48:40.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:40.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644574+0000", 2024-09-04T17:48:40.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3340000000000002e-06 2024-09-04T17:48:40.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:40.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644581+0000", 2024-09-04T17:48:40.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6579999999999996e-06 2024-09-04T17:48:40.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:40.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.644589+0000", 2024-09-04T17:48:40.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9699999999999999e-06 2024-09-04T17:48:40.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:40.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.671925+0000", 2024-09-04T17:48:40.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027335911000000001 2024-09-04T17:48:40.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:40.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.671987+0000", 2024-09-04T17:48:40.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2198000000000003e-05 2024-09-04T17:48:40.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:40.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.672126+0000", 2024-09-04T17:48:40.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000139462 2024-09-04T17:48:40.429 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:40.429 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:40.429 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:40.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.430 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:96164 3.18 3:19aa90a8:::rbd_data.1134c4e2d26.0000000000000062:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123)", 2024-09-04T17:48:40.430 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:48:02.862015+0000", 2024-09-04T17:48:40.430 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.474382734999999, 2024-09-04T17:48:40.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.034031039999999999, 2024-09-04T17:48:40.430 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:40.430 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:40.430 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:40.430 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:40.430 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:40.430 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:40.430 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 96164 2024-09-04T17:48:40.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.430 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:40.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:40.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.862015+0000", 2024-09-04T17:48:40.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:40.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:40.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.862015+0000", 2024-09-04T17:48:40.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:40.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:40.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.862018+0000", 2024-09-04T17:48:40.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1200000000000002e-06 2024-09-04T17:48:40.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:40.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.862032+0000", 2024-09-04T17:48:40.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4083000000000001e-05 2024-09-04T17:48:40.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:40.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.862064+0000", 2024-09-04T17:48:40.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2095999999999999e-05 2024-09-04T17:48:40.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:40.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.862081+0000", 2024-09-04T17:48:40.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6625000000000001e-05 2024-09-04T17:48:40.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:40.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.862232+0000", 2024-09-04T17:48:40.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015144600000000001 2024-09-04T17:48:40.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:40.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.862588+0000", 2024-09-04T17:48:40.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00035525999999999998 2024-09-04T17:48:40.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.434 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 7", 2024-09-04T17:48:40.434 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.867988+0000", 2024-09-04T17:48:40.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054005479999999998 2024-09-04T17:48:40.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.434 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:40.434 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.895763+0000", 2024-09-04T17:48:40.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.027774983 2024-09-04T17:48:40.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.434 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:40.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.895877+0000", 2024-09-04T17:48:40.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000113607 2024-09-04T17:48:40.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:40.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.895975+0000", 2024-09-04T17:48:40.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7876999999999998e-05 2024-09-04T17:48:40.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:40.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:40.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:40.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.896046+0000", 2024-09-04T17:48:40.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1395000000000003e-05 2024-09-04T17:48:40.435 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:40.435 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:40.435 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:40.436 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:40.436 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:40.436 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:40.437 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-04T17:48:40.438 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:40.437+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:40.539 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:40.537+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:40.568 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:40.569 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:48:40.569 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:48:40.569 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:40.584 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-04T17:48:40.639 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:40.639+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:40.740 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:40.737+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:40.747 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:40.747 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:48:40.747 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:48:40.747 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:48:40.747 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:40.761 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-04T17:48:40.841 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:40.839+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:40.916 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is unset 2024-09-04T17:48:40.941 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:40.941+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:40.943 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:40.943 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:48:40.943 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:48:40.943 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:48:40.943 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.943 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "pg_update_log_missing(3.1f epoch 119/118 rep_tid 2838 entries 119'424 (0'0) error 3:f95bc674:::rbd_header.1134878eadad:head by client.4404.0:93468 0.000000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-04T17:48:40.943 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:58.898727+0000", 2024-09-04T17:48:40.943 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 42.041460039999997, 2024-09-04T17:48:40.943 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0072113999999999998, 2024-09-04T17:48:40.943 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:40.944 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:40.944 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:40.944 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:40.944 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.944 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:40.944 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.898727+0000", 2024-09-04T17:48:40.944 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:40.944 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.944 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.944 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:40.944 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.898727+0000", 2024-09-04T17:48:40.944 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:40.945 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.945 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.945 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:40.945 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.898728+0000", 2024-09-04T17:48:40.945 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.8200000000000001e-07 2024-09-04T17:48:40.945 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.945 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.945 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:40.945 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.898740+0000", 2024-09-04T17:48:40.945 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2186e-05 2024-09-04T17:48:40.945 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.945 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.945 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:40.946 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.898772+0000", 2024-09-04T17:48:40.946 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.2156000000000002e-05 2024-09-04T17:48:40.946 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.946 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.946 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:40.946 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.898796+0000", 2024-09-04T17:48:40.946 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4017999999999999e-05 2024-09-04T17:48:40.946 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.946 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.946 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:40.947 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.898966+0000", 2024-09-04T17:48:40.947 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000170185 2024-09-04T17:48:40.947 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.947 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.947 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:40.947 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.905939+0000", 2024-09-04T17:48:40.947 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0069724729999999999 2024-09-04T17:48:40.947 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:40.947 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:40.947 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:40.947 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.947 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.947 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:93469 3.1f e119/118 3:f95bc674:::rbd_header.1134878eadad:head v 119'425, mlcod=0'0)", 2024-09-04T17:48:40.948 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:58.900173+0000", 2024-09-04T17:48:40.948 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 42.040014251999999, 2024-09-04T17:48:40.948 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0059053930000000001, 2024-09-04T17:48:40.948 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:40.948 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:40.948 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:40.948 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:40.948 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.948 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:40.948 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.900173+0000", 2024-09-04T17:48:40.948 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:40.948 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.948 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.948 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:40.949 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.900173+0000", 2024-09-04T17:48:40.949 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:40.949 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.949 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.949 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:40.949 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.900174+0000", 2024-09-04T17:48:40.949 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.127e-06 2024-09-04T17:48:40.949 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.949 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.949 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:40.949 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.900181+0000", 2024-09-04T17:48:40.949 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.0099999999999998e-06 2024-09-04T17:48:40.949 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.950 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.950 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:40.950 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.900199+0000", 2024-09-04T17:48:40.950 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7876e-05 2024-09-04T17:48:40.950 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.950 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.950 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:40.950 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.900208+0000", 2024-09-04T17:48:40.950 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.4999999999999999e-06 2024-09-04T17:48:40.950 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.950 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.950 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:40.950 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.900310+0000", 2024-09-04T17:48:40.950 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00010202800000000001 2024-09-04T17:48:40.951 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.951 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.951 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:40.951 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.900353+0000", 2024-09-04T17:48:40.951 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.3213000000000001e-05 2024-09-04T17:48:40.951 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.951 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.951 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:40.951 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.905944+0000", 2024-09-04T17:48:40.951 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0055914900000000002 2024-09-04T17:48:40.951 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.951 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.951 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:40.951 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.906078+0000", 2024-09-04T17:48:40.952 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013414899999999999 2024-09-04T17:48:40.952 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:40.952 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:40.952 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:40.952 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.952 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.952 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "MOSDPGPush(3.10 119/118 [PushOp(3:0c2d06e5:::rbd_data.113410ddd206.0000000000000000:18, version: 68'308, data_included: [0~884736,950272~65536,1081344~65536,1212416~32768], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:0c2d06e5:::rbd_data.113410ddd206.0000000000000000:18@68'308, size: 7471104, copy_subset: [(0, 7471104)], clone_subset: {}, snapset: 18=[]:{18: [18]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1245184, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-04T17:48:40.952 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:59.426517+0000", 2024-09-04T17:48:40.952 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 41.513670173999998, 2024-09-04T17:48:40.952 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0064985670000000002, 2024-09-04T17:48:40.952 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:40.952 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:40.953 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:40.953 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:40.953 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.953 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:40.953 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.426517+0000", 2024-09-04T17:48:40.953 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:40.953 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.953 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.953 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:40.953 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.426517+0000", 2024-09-04T17:48:40.953 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:40.953 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.953 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.953 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:40.954 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.426754+0000", 2024-09-04T17:48:40.954 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00023681799999999999 2024-09-04T17:48:40.954 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.954 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.954 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:40.954 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.429886+0000", 2024-09-04T17:48:40.954 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0031323789999999998 2024-09-04T17:48:40.954 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.954 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.954 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:40.954 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.429940+0000", 2024-09-04T17:48:40.954 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4104999999999998e-05 2024-09-04T17:48:40.954 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.954 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.955 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:40.955 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.429975+0000", 2024-09-04T17:48:40.955 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4289000000000003e-05 2024-09-04T17:48:40.955 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.955 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.955 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:40.955 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.430229+0000", 2024-09-04T17:48:40.955 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00025465900000000002 2024-09-04T17:48:40.955 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.955 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.955 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:40.955 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.430350+0000", 2024-09-04T17:48:40.955 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000120992 2024-09-04T17:48:40.955 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.956 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.956 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:40.956 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.433016+0000", 2024-09-04T17:48:40.956 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0026653250000000001 2024-09-04T17:48:40.956 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:40.956 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:40.956 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:40.956 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.956 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.957 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "MOSDPGPush(3.10 119/118 [PushOp(3:0c2d06e5:::rbd_data.113410ddd206.0000000000000000:18, version: 68'308, data_included: [1245184~32768,1343488~65536,1474560~65536,1605632~65536,1736704~65536,1867776~65536,1998848~65536,2129920~65536,2260992~65536,2392064~65536,2523136~65536,2654208~65536,2785280~65536,2916352~65536,3047424~65536,3154405~89627,3309568~8677], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:0c2d06e5:::rbd_data.113410ddd206.0000000000000000:18@68'308, size: 7471104, copy_subset: [(0, 7471104)], clone_subset: {}, snapset: 18=[]:{18: [18]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3318245, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1245184, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-04T17:48:40.957 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:59.434957+0000", 2024-09-04T17:48:40.957 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 41.505230550999997, 2024-09-04T17:48:40.957 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0061603969999999997, 2024-09-04T17:48:40.957 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:40.957 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:40.957 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:40.957 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:40.957 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.957 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:40.958 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.434957+0000", 2024-09-04T17:48:40.958 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:40.958 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.958 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.958 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:40.958 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.434957+0000", 2024-09-04T17:48:40.958 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:40.958 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.958 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.958 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:40.958 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.434958+0000", 2024-09-04T17:48:40.958 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4339999999999999e-06 2024-09-04T17:48:40.958 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.958 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.959 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:40.959 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.436921+0000", 2024-09-04T17:48:40.959 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0019629460000000001 2024-09-04T17:48:40.959 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.959 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.959 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:40.959 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.436973+0000", 2024-09-04T17:48:40.959 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1492000000000001e-05 2024-09-04T17:48:40.959 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.959 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.959 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:40.959 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.437041+0000", 2024-09-04T17:48:40.959 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.8275000000000006e-05 2024-09-04T17:48:40.960 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.960 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.960 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:40.960 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.437312+0000", 2024-09-04T17:48:40.960 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00027156800000000003 2024-09-04T17:48:40.960 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.960 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.960 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:40.960 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.437357+0000", 2024-09-04T17:48:40.960 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4079000000000003e-05 2024-09-04T17:48:40.960 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.960 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.960 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:40.960 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.441117+0000", 2024-09-04T17:48:40.961 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0037606029999999999 2024-09-04T17:48:40.961 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:40.961 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:40.961 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:40.961 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.961 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.961 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:96178 3.1e 3:7c9d4526:::rbd_data.1134c4e2d26.0000000000000026:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123)", 2024-09-04T17:48:40.961 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:02.985763+0000", 2024-09-04T17:48:40.961 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 37.954424402999997, 2024-09-04T17:48:40.961 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.020686758999999999, 2024-09-04T17:48:40.961 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:40.961 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:40.961 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:40.962 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:40.962 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:40.962 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:40.962 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 96178 2024-09-04T17:48:40.962 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.962 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:40.962 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.962 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:40.962 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:02.985763+0000", 2024-09-04T17:48:40.962 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:40.962 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.962 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.962 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:40.962 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:02.985763+0000", 2024-09-04T17:48:40.963 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:40.963 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.963 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.963 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:40.963 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:02.985766+0000", 2024-09-04T17:48:40.963 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.2990000000000001e-06 2024-09-04T17:48:40.963 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.963 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.963 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:40.963 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:02.985783+0000", 2024-09-04T17:48:40.963 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.645e-05 2024-09-04T17:48:40.963 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.963 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.963 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:40.964 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:02.985818+0000", 2024-09-04T17:48:40.964 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.5459000000000001e-05 2024-09-04T17:48:40.964 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.964 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.964 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:40.964 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:02.985835+0000", 2024-09-04T17:48:40.964 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.715e-05 2024-09-04T17:48:40.964 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.964 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.964 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:40.964 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:02.985953+0000", 2024-09-04T17:48:40.964 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000117351 2024-09-04T17:48:40.964 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.964 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.965 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:40.965 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:02.986207+0000", 2024-09-04T17:48:40.965 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00025483 2024-09-04T17:48:40.965 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.965 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.965 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 2", 2024-09-04T17:48:40.965 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:02.990784+0000", 2024-09-04T17:48:40.965 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0045763219999999999 2024-09-04T17:48:40.965 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.965 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.966 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:48:40.966 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.006139+0000", 2024-09-04T17:48:40.966 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.015355526 2024-09-04T17:48:40.966 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.966 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.966 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:40.966 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.006225+0000", 2024-09-04T17:48:40.966 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.6003999999999999e-05 2024-09-04T17:48:40.966 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.966 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.966 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:40.967 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.006371+0000", 2024-09-04T17:48:40.967 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000146052 2024-09-04T17:48:40.967 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.967 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.967 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:40.967 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.006450+0000", 2024-09-04T17:48:40.967 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.8316e-05 2024-09-04T17:48:40.967 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:40.967 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:40.967 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:40.967 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.967 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.967 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:96267 3.1e 3:7a1dfe29:::rbd_header.1134c4e2d26:head [watch unwatch cookie 140018753450144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123)", 2024-09-04T17:48:40.967 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:03.482624+0000", 2024-09-04T17:48:40.968 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 37.457563100999998, 2024-09-04T17:48:40.968 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.006866853, 2024-09-04T17:48:40.968 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:40.968 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:40.968 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:40.968 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:40.968 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:40.968 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:40.968 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 96267 2024-09-04T17:48:40.968 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.968 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:40.968 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.968 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:40.969 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.482624+0000", 2024-09-04T17:48:40.969 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:40.969 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.969 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.969 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:40.969 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.482624+0000", 2024-09-04T17:48:40.969 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:40.969 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.969 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.969 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:40.969 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.482626+0000", 2024-09-04T17:48:40.969 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8360000000000001e-06 2024-09-04T17:48:40.969 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.969 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.970 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:40.970 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.482635+0000", 2024-09-04T17:48:40.970 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9099999999999994e-06 2024-09-04T17:48:40.970 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.970 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.970 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:40.970 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.482658+0000", 2024-09-04T17:48:40.970 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.3186000000000001e-05 2024-09-04T17:48:40.970 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.970 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.970 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:40.970 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.482677+0000", 2024-09-04T17:48:40.970 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8396000000000001e-05 2024-09-04T17:48:40.971 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.971 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.971 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:40.971 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.482849+0000", 2024-09-04T17:48:40.971 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000172936 2024-09-04T17:48:40.971 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.971 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.971 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:40.971 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.482987+0000", 2024-09-04T17:48:40.971 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013707699999999999 2024-09-04T17:48:40.971 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.971 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.971 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 2", 2024-09-04T17:48:40.972 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.483222+0000", 2024-09-04T17:48:40.972 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00023545 2024-09-04T17:48:40.972 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.972 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.972 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:48:40.972 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.484356+0000", 2024-09-04T17:48:40.972 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.001134049 2024-09-04T17:48:40.972 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.972 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.972 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:40.972 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.489303+0000", 2024-09-04T17:48:40.972 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0049472910000000004 2024-09-04T17:48:40.972 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.973 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.973 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:40.973 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.489396+0000", 2024-09-04T17:48:40.973 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.2638000000000003e-05 2024-09-04T17:48:40.973 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.973 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.973 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:40.973 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.489491+0000", 2024-09-04T17:48:40.973 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5084e-05 2024-09-04T17:48:40.973 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:40.973 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:40.974 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:40.974 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.974 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.974 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:96360 3.1e 3:7c9d4526:::rbd_data.1134c4e2d26.0000000000000026:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123)", 2024-09-04T17:48:40.974 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:03.603245+0000", 2024-09-04T17:48:40.974 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 37.336941996999997, 2024-09-04T17:48:40.974 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0074703160000000003, 2024-09-04T17:48:40.974 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:40.974 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:40.974 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:40.974 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:40.974 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:40.974 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:40.974 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 96360 2024-09-04T17:48:40.975 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.975 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:40.975 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.975 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:40.975 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.603245+0000", 2024-09-04T17:48:40.975 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:40.975 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.975 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.975 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:40.975 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.603245+0000", 2024-09-04T17:48:40.975 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:40.975 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.975 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.975 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:40.976 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.603248+0000", 2024-09-04T17:48:40.976 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.277e-06 2024-09-04T17:48:40.976 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.976 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.976 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:40.976 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.603259+0000", 2024-09-04T17:48:40.976 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1083e-05 2024-09-04T17:48:40.976 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.976 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.976 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:40.976 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.603295+0000", 2024-09-04T17:48:40.976 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.5892999999999997e-05 2024-09-04T17:48:40.976 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.977 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.977 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:40.977 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.603315+0000", 2024-09-04T17:48:40.977 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0953000000000001e-05 2024-09-04T17:48:40.977 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.977 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.977 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:40.977 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.603486+0000", 2024-09-04T17:48:40.977 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00017011199999999999 2024-09-04T17:48:40.977 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.977 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.977 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:40.977 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.604960+0000", 2024-09-04T17:48:40.978 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0014743340000000001 2024-09-04T17:48:40.978 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.978 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.978 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 2", 2024-09-04T17:48:40.978 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.605232+0000", 2024-09-04T17:48:40.978 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00027236300000000002 2024-09-04T17:48:40.978 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.978 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.978 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:48:40.978 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.610322+0000", 2024-09-04T17:48:40.978 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0050901849999999997 2024-09-04T17:48:40.978 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.978 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.978 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:40.979 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.610464+0000", 2024-09-04T17:48:40.979 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014186599999999999 2024-09-04T17:48:40.979 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.979 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.979 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:40.979 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.610607+0000", 2024-09-04T17:48:40.979 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000142423 2024-09-04T17:48:40.979 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.979 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.979 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:40.979 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.610716+0000", 2024-09-04T17:48:40.979 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00010882699999999999 2024-09-04T17:48:40.979 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:40.980 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:40.980 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:40.980 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.980 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.980 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:96520 3.2 e124/121 3:4549cdb4:::rbd_data.1134878eadad.0000000000000001:head v 124'509, mlcod=123'507)", 2024-09-04T17:48:40.980 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:04.014285+0000", 2024-09-04T17:48:40.980 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 36.925902078, 2024-09-04T17:48:40.980 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0066183220000000003, 2024-09-04T17:48:40.980 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:40.980 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:40.980 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:40.980 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:40.980 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.980 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:40.981 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:04.014285+0000", 2024-09-04T17:48:40.981 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:40.981 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.981 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.981 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:40.981 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:04.014285+0000", 2024-09-04T17:48:40.981 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:40.981 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.981 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.981 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:40.981 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:04.014287+0000", 2024-09-04T17:48:40.981 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1959999999999998e-06 2024-09-04T17:48:40.981 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.981 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.982 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:40.982 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:04.014332+0000", 2024-09-04T17:48:40.982 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4196000000000002e-05 2024-09-04T17:48:40.982 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.982 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.982 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:40.982 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:04.014374+0000", 2024-09-04T17:48:40.982 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2072e-05 2024-09-04T17:48:40.982 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.982 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.982 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:40.982 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:04.014394+0000", 2024-09-04T17:48:40.982 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0146999999999999e-05 2024-09-04T17:48:40.982 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.983 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.983 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:40.983 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:04.014575+0000", 2024-09-04T17:48:40.983 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000181132 2024-09-04T17:48:40.983 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.983 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.983 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:40.983 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:04.014611+0000", 2024-09-04T17:48:40.983 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.5815999999999997e-05 2024-09-04T17:48:40.983 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.983 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.983 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:40.983 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:04.020809+0000", 2024-09-04T17:48:40.983 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0061983159999999997 2024-09-04T17:48:40.984 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.984 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.984 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:40.984 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:04.020904+0000", 2024-09-04T17:48:40.984 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4446999999999999e-05 2024-09-04T17:48:40.984 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:40.984 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:40.984 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:40.984 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.984 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.984 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(osd.2.0:369 3.0 e138/121 3:04fb2888:::rbd_data.11342e6808f0.0000000000000009:24 v 138'380, mlcod=135'377)", 2024-09-04T17:48:40.984 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:17.851779+0000", 2024-09-04T17:48:40.985 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 23.088408166000001, 2024-09-04T17:48:40.985 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0056329400000000003, 2024-09-04T17:48:40.985 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:40.985 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:40.985 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:40.985 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:40.985 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.985 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:40.985 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.851779+0000", 2024-09-04T17:48:40.985 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:40.985 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.985 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.985 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:40.985 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.851779+0000", 2024-09-04T17:48:40.986 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:40.986 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.986 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.986 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:40.986 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.851780+0000", 2024-09-04T17:48:40.986 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2399999999999998e-07 2024-09-04T17:48:40.986 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.986 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.986 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:40.986 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.851790+0000", 2024-09-04T17:48:40.986 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0399e-05 2024-09-04T17:48:40.986 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.986 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.986 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:40.987 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.851809+0000", 2024-09-04T17:48:40.987 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9228e-05 2024-09-04T17:48:40.987 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.987 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.987 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:40.987 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.851820+0000", 2024-09-04T17:48:40.987 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0814e-05 2024-09-04T17:48:40.987 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.987 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.987 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:40.987 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.851902+0000", 2024-09-04T17:48:40.987 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.2154e-05 2024-09-04T17:48:40.987 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.988 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.988 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:40.988 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.851930+0000", 2024-09-04T17:48:40.988 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.7577999999999999e-05 2024-09-04T17:48:40.988 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.988 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.988 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:40.988 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.857363+0000", 2024-09-04T17:48:40.988 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0054331989999999997 2024-09-04T17:48:40.988 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.988 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.988 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:40.988 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.857412+0000", 2024-09-04T17:48:40.988 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9044e-05 2024-09-04T17:48:40.989 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:40.989 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:40.989 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:40.989 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.989 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.989 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(osd.5.0:3167 3.2 e138/121 3:460b9dee:::rbd_data.11342e6808f0.000000000000000d:24 v 138'560, mlcod=135'557)", 2024-09-04T17:48:40.989 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:17.857279+0000", 2024-09-04T17:48:40.989 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 23.082908582000002, 2024-09-04T17:48:40.989 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0068482229999999996, 2024-09-04T17:48:40.989 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:40.989 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:40.989 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:40.989 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:40.990 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.990 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:40.990 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.857279+0000", 2024-09-04T17:48:40.990 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:40.990 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.990 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.990 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:40.990 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.857279+0000", 2024-09-04T17:48:40.990 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:40.990 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.990 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.990 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:40.990 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.857279+0000", 2024-09-04T17:48:40.990 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7300000000000001e-07 2024-09-04T17:48:40.991 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.991 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.991 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:40.991 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.857286+0000", 2024-09-04T17:48:40.991 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.4289999999999998e-06 2024-09-04T17:48:40.991 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.991 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.991 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:40.991 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.857300+0000", 2024-09-04T17:48:40.991 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4582e-05 2024-09-04T17:48:40.991 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.991 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.991 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:40.992 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.857315+0000", 2024-09-04T17:48:40.992 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5173e-05 2024-09-04T17:48:40.992 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.992 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.992 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:40.992 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.857388+0000", 2024-09-04T17:48:40.992 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2365000000000003e-05 2024-09-04T17:48:40.992 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.992 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.992 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:40.992 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.857414+0000", 2024-09-04T17:48:40.992 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6248999999999999e-05 2024-09-04T17:48:40.992 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.992 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.993 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:40.993 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.864076+0000", 2024-09-04T17:48:40.993 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0066619490000000003 2024-09-04T17:48:40.993 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.993 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.993 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:40.993 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.864127+0000", 2024-09-04T17:48:40.993 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1002999999999999e-05 2024-09-04T17:48:40.993 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:40.993 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:40.993 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:40.993 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.993 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.993 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:98345 3.a 3:55a1c81d:::rbd_header.113481c24fbf:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139)", 2024-09-04T17:48:40.994 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:18.971074+0000", 2024-09-04T17:48:40.994 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 21.969113303, 2024-09-04T17:48:40.994 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.017562632000000002, 2024-09-04T17:48:40.994 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:40.994 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:40.994 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:40.994 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:40.994 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:40.994 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:40.994 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 98345 2024-09-04T17:48:40.994 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.994 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:40.994 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.995 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:40.995 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.971074+0000", 2024-09-04T17:48:40.995 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:40.995 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.995 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.995 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:40.995 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.971074+0000", 2024-09-04T17:48:40.995 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:40.995 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.995 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.995 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:40.995 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.971078+0000", 2024-09-04T17:48:40.995 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.1969999999999998e-06 2024-09-04T17:48:40.995 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.996 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.996 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:40.996 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.971096+0000", 2024-09-04T17:48:40.996 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7898999999999999e-05 2024-09-04T17:48:40.996 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.996 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.996 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:40.996 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.971140+0000", 2024-09-04T17:48:40.996 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.3883e-05 2024-09-04T17:48:40.996 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.996 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.996 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:40.996 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.971165+0000", 2024-09-04T17:48:40.997 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5321999999999999e-05 2024-09-04T17:48:40.997 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.997 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.997 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:40.997 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.971346+0000", 2024-09-04T17:48:40.997 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000180784 2024-09-04T17:48:40.997 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.997 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.997 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op must wait for map", 2024-09-04T17:48:40.997 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.971369+0000", 2024-09-04T17:48:40.997 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.3286e-05 2024-09-04T17:48:40.997 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.997 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.997 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:40.998 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.977475+0000", 2024-09-04T17:48:40.998 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0061055459999999999 2024-09-04T17:48:40.998 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.998 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.998 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:40.998 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.977665+0000", 2024-09-04T17:48:40.998 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019043599999999999 2024-09-04T17:48:40.998 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.998 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.998 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 4", 2024-09-04T17:48:40.998 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.978586+0000", 2024-09-04T17:48:40.998 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000920597 2024-09-04T17:48:40.998 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.999 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.999 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:48:40.999 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.982469+0000", 2024-09-04T17:48:40.999 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0038834719999999998 2024-09-04T17:48:40.999 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.999 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.999 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:40.999 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.988390+0000", 2024-09-04T17:48:40.999 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0059206479999999997 2024-09-04T17:48:40.999 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:40.999 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:40.999 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:40.999 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.988546+0000", 2024-09-04T17:48:41.000 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00015548 2024-09-04T17:48:41.000 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.000 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.000 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:41.000 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.988637+0000", 2024-09-04T17:48:41.000 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1082000000000006e-05 2024-09-04T17:48:41.000 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:41.000 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:41.000 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:41.000 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.000 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.000 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:98921 3.c e142/122 3:37678a48:::rbd_data.113460fdfacd.0000000000000006:head v 142'572, mlcod=137'569)", 2024-09-04T17:48:41.000 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:22.131110+0000", 2024-09-04T17:48:41.001 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 18.809077402, 2024-09-04T17:48:41.001 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0055872880000000002, 2024-09-04T17:48:41.001 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:41.001 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:41.001 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:41.001 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:41.001 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.001 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:41.001 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.131110+0000", 2024-09-04T17:48:41.001 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:41.001 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.001 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.001 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:41.002 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.131110+0000", 2024-09-04T17:48:41.002 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:41.002 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.002 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.002 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:41.002 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.131110+0000", 2024-09-04T17:48:41.002 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7e-07 2024-09-04T17:48:41.002 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.002 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.002 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:41.002 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.131120+0000", 2024-09-04T17:48:41.002 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.2499999999999995e-06 2024-09-04T17:48:41.002 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.002 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.003 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:41.003 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.131146+0000", 2024-09-04T17:48:41.003 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6461999999999998e-05 2024-09-04T17:48:41.003 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.003 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.003 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:41.003 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.131157+0000", 2024-09-04T17:48:41.003 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1522000000000001e-05 2024-09-04T17:48:41.003 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.003 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.003 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:41.003 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.131323+0000", 2024-09-04T17:48:41.003 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00016558499999999999 2024-09-04T17:48:41.003 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.004 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.004 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:41.004 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.131351+0000", 2024-09-04T17:48:41.004 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.7494000000000001e-05 2024-09-04T17:48:41.004 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.004 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.004 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:41.004 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.136648+0000", 2024-09-04T17:48:41.004 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0052975330000000001 2024-09-04T17:48:41.004 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.004 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.004 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:41.004 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.136697+0000", 2024-09-04T17:48:41.005 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9072e-05 2024-09-04T17:48:41.005 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:41.005 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:41.005 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:41.005 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.005 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.005 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:98922 3.0 e142/121 3:00fa19d2:::rbd_data.113460fdfacd.0000000000000007:head v 142'381, mlcod=135'378)", 2024-09-04T17:48:41.005 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:22.132316+0000", 2024-09-04T17:48:41.005 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 18.807871621, 2024-09-04T17:48:41.005 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0070776679999999996, 2024-09-04T17:48:41.005 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:41.005 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:41.005 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:41.006 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:41.006 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.006 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:41.006 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.132316+0000", 2024-09-04T17:48:41.006 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:41.006 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.006 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.006 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:41.006 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.132316+0000", 2024-09-04T17:48:41.006 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:41.006 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.006 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.006 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:41.006 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.132316+0000", 2024-09-04T17:48:41.007 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2300000000000002e-07 2024-09-04T17:48:41.007 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.007 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.007 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:41.007 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.132322+0000", 2024-09-04T17:48:41.007 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.48e-06 2024-09-04T17:48:41.007 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.007 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.007 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:41.007 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.132335+0000", 2024-09-04T17:48:41.007 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3040999999999999e-05 2024-09-04T17:48:41.007 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.007 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.008 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:41.008 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.132344+0000", 2024-09-04T17:48:41.008 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9989999999999997e-06 2024-09-04T17:48:41.008 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.008 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.008 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:41.008 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.132453+0000", 2024-09-04T17:48:41.008 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000109615 2024-09-04T17:48:41.008 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.008 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.008 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:41.008 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.132481+0000", 2024-09-04T17:48:41.008 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.777e-05 2024-09-04T17:48:41.008 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.009 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.009 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:41.009 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.139271+0000", 2024-09-04T17:48:41.009 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0067898180000000004 2024-09-04T17:48:41.009 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.009 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.009 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:41.009 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.139393+0000", 2024-09-04T17:48:41.009 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00012252199999999999 2024-09-04T17:48:41.009 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:41.009 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:41.009 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:41.009 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.010 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.010 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:99605 3.a 3:50de26c1:::rbd_data.11346a2b0bc9.0000000000000008:head [delete] snapc 1c=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-04T17:48:41.010 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:27.983388+0000", 2024-09-04T17:48:41.010 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 12.956799461999999, 2024-09-04T17:48:41.010 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0073493309999999997, 2024-09-04T17:48:41.010 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:41.010 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:41.010 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:41.010 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:41.010 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:41.010 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:41.010 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 99605 2024-09-04T17:48:41.010 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.010 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:41.011 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.011 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:41.011 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:27.983388+0000", 2024-09-04T17:48:41.011 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:41.011 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.011 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.011 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:41.011 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:27.983388+0000", 2024-09-04T17:48:41.011 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:41.011 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.011 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.011 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:41.011 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:27.983392+0000", 2024-09-04T17:48:41.012 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.9770000000000002e-06 2024-09-04T17:48:41.012 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.012 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.012 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:41.012 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:27.983413+0000", 2024-09-04T17:48:41.012 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1075999999999999e-05 2024-09-04T17:48:41.012 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.012 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.012 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:41.012 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:27.983457+0000", 2024-09-04T17:48:41.012 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.3943999999999998e-05 2024-09-04T17:48:41.012 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.012 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.013 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:41.013 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:27.983482+0000", 2024-09-04T17:48:41.013 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5602999999999999e-05 2024-09-04T17:48:41.013 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.013 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.013 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:41.013 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:27.983671+0000", 2024-09-04T17:48:41.013 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00018823399999999999 2024-09-04T17:48:41.013 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.013 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.013 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:41.013 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:27.984069+0000", 2024-09-04T17:48:41.013 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000398639 2024-09-04T17:48:41.013 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.014 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.014 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 4", 2024-09-04T17:48:41.014 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:27.984378+0000", 2024-09-04T17:48:41.014 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000308392 2024-09-04T17:48:41.014 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.014 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.014 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:48:41.014 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:27.987068+0000", 2024-09-04T17:48:41.014 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.002690764 2024-09-04T17:48:41.014 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.014 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.014 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:41.015 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:27.990452+0000", 2024-09-04T17:48:41.015 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0033831080000000001 2024-09-04T17:48:41.015 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.015 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.015 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:41.015 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:27.990603+0000", 2024-09-04T17:48:41.015 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00015124 2024-09-04T17:48:41.015 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.015 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.015 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:41.015 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:27.990737+0000", 2024-09-04T17:48:41.015 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013435399999999999 2024-09-04T17:48:41.015 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:41.016 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:41.016 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:41.016 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.016 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.016 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:99857 3.10 3:0f7166b0:::rbd_header.1134968ccb73:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e153)", 2024-09-04T17:48:41.016 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:32.841823+0000", 2024-09-04T17:48:41.016 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 8.0983640799999996, 2024-09-04T17:48:41.016 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.014014146999999999, 2024-09-04T17:48:41.016 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:41.016 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:41.016 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:41.016 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:41.016 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:41.017 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:41.017 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 99857 2024-09-04T17:48:41.017 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.017 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:41.017 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.017 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:41.017 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.841823+0000", 2024-09-04T17:48:41.017 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:41.017 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.017 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.017 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:41.017 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.841823+0000", 2024-09-04T17:48:41.017 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:41.018 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.018 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.018 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:41.018 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.841827+0000", 2024-09-04T17:48:41.018 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.3699999999999999e-06 2024-09-04T17:48:41.018 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.018 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.018 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:41.018 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.841858+0000", 2024-09-04T17:48:41.018 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.1795999999999998e-05 2024-09-04T17:48:41.018 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.018 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.018 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:41.019 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.841906+0000", 2024-09-04T17:48:41.019 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7737000000000001e-05 2024-09-04T17:48:41.019 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.019 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.019 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:41.019 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.841930+0000", 2024-09-04T17:48:41.019 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4272000000000001e-05 2024-09-04T17:48:41.019 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.019 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.019 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:41.019 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.842177+0000", 2024-09-04T17:48:41.019 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024632300000000001 2024-09-04T17:48:41.019 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.019 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.020 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op must wait for map", 2024-09-04T17:48:41.020 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.842194+0000", 2024-09-04T17:48:41.020 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7606000000000001e-05 2024-09-04T17:48:41.020 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.020 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.020 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:41.020 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.850210+0000", 2024-09-04T17:48:41.020 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0080154179999999998 2024-09-04T17:48:41.020 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.020 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.020 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:41.020 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.850494+0000", 2024-09-04T17:48:41.020 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00028466100000000002 2024-09-04T17:48:41.021 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.021 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.021 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 0", 2024-09-04T17:48:41.021 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.851786+0000", 2024-09-04T17:48:41.021 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.001291468 2024-09-04T17:48:41.021 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.021 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.021 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:48:41.021 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.855374+0000", 2024-09-04T17:48:41.021 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0035884760000000002 2024-09-04T17:48:41.021 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.021 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.021 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:41.021 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.855504+0000", 2024-09-04T17:48:41.022 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00012924799999999999 2024-09-04T17:48:41.022 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.022 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.022 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:41.022 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.855704+0000", 2024-09-04T17:48:41.022 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000200764 2024-09-04T17:48:41.022 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.022 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.022 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:41.022 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.855837+0000", 2024-09-04T17:48:41.022 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000133008 2024-09-04T17:48:41.022 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:41.023 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:41.023 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:41.023 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.023 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.023 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:99861 3.10 3:0f7166b0:::rbd_header.1134968ccb73:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e153)", 2024-09-04T17:48:41.023 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:32.872359+0000", 2024-09-04T17:48:41.023 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 8.0678282219999993, 2024-09-04T17:48:41.023 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.006863063, 2024-09-04T17:48:41.023 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:41.023 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:41.023 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:41.023 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:41.023 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:41.023 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:41.024 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 99861 2024-09-04T17:48:41.024 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.024 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:41.024 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.024 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:41.024 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.872359+0000", 2024-09-04T17:48:41.024 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:41.024 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.024 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.024 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:41.024 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.872359+0000", 2024-09-04T17:48:41.024 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:41.024 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.025 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.025 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:41.025 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.872363+0000", 2024-09-04T17:48:41.025 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.8260000000000003e-06 2024-09-04T17:48:41.025 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.025 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.025 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:41.025 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.872382+0000", 2024-09-04T17:48:41.025 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9318999999999999e-05 2024-09-04T17:48:41.025 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.025 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.025 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:41.025 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.872426+0000", 2024-09-04T17:48:41.025 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4038000000000001e-05 2024-09-04T17:48:41.026 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.026 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.026 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:41.026 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.872452+0000", 2024-09-04T17:48:41.026 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5479000000000001e-05 2024-09-04T17:48:41.026 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.026 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.026 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:41.026 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.872647+0000", 2024-09-04T17:48:41.026 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019568799999999999 2024-09-04T17:48:41.026 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.026 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.026 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:41.026 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.872894+0000", 2024-09-04T17:48:41.027 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000246226 2024-09-04T17:48:41.027 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.027 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.027 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 0", 2024-09-04T17:48:41.027 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.873410+0000", 2024-09-04T17:48:41.027 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00051657000000000005 2024-09-04T17:48:41.027 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.027 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.027 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:48:41.027 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.876401+0000", 2024-09-04T17:48:41.027 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.002990983 2024-09-04T17:48:41.027 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.027 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.028 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:41.028 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.878957+0000", 2024-09-04T17:48:41.028 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.002556039 2024-09-04T17:48:41.028 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.028 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.028 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:41.028 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.879109+0000", 2024-09-04T17:48:41.028 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00015156999999999999 2024-09-04T17:48:41.028 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.028 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.028 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:41.028 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.879222+0000", 2024-09-04T17:48:41.029 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000113325 2024-09-04T17:48:41.029 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:41.029 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:41.029 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:41.029 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.029 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.029 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:100488 3.1e e156/149 3:7e3581b1:::rbd_data.11343e517879.0000000000000008:head v 156'625, mlcod=151'621)", 2024-09-04T17:48:41.029 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:36.026379+0000", 2024-09-04T17:48:41.029 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.9138083960000003, 2024-09-04T17:48:41.029 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0076449719999999999, 2024-09-04T17:48:41.029 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:41.029 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:41.029 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:41.030 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:41.030 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.030 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:41.030 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:36.026379+0000", 2024-09-04T17:48:41.030 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:41.030 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.030 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.030 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:41.030 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:36.026379+0000", 2024-09-04T17:48:41.030 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:41.030 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.030 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.030 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:41.031 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:36.026381+0000", 2024-09-04T17:48:41.031 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.906e-06 2024-09-04T17:48:41.031 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.031 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.031 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:41.031 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:36.026395+0000", 2024-09-04T17:48:41.031 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4343e-05 2024-09-04T17:48:41.031 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.031 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.031 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:41.031 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:36.026427+0000", 2024-09-04T17:48:41.031 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.1569000000000002e-05 2024-09-04T17:48:41.031 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.031 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.032 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:41.032 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:36.026446+0000", 2024-09-04T17:48:41.032 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9459000000000001e-05 2024-09-04T17:48:41.032 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.032 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.032 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:41.032 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:36.026664+0000", 2024-09-04T17:48:41.032 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021771099999999999 2024-09-04T17:48:41.032 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.032 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.032 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:41.032 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:36.026720+0000", 2024-09-04T17:48:41.032 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.5943999999999998e-05 2024-09-04T17:48:41.032 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.033 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.033 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:41.033 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:36.033904+0000", 2024-09-04T17:48:41.033 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0071845219999999996 2024-09-04T17:48:41.033 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.033 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.033 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:41.033 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:36.034024+0000", 2024-09-04T17:48:41.033 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000119518 2024-09-04T17:48:41.033 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:41.033 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:41.033 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:41.033 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.034 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.034 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(osd.2.0:482 3.0 e157/121 3:05a6e060:::rbd_data.113418f5e708.0000000000000010:16 v 157'436, mlcod=155'433)", 2024-09-04T17:48:41.034 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:36.880429+0000", 2024-09-04T17:48:41.034 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.0597581849999997, 2024-09-04T17:48:41.034 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.005565909, 2024-09-04T17:48:41.034 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:41.034 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:41.034 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:41.034 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:41.034 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.034 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:41.034 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:36.880429+0000", 2024-09-04T17:48:41.034 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:41.035 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.035 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.035 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:41.035 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:36.880429+0000", 2024-09-04T17:48:41.035 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:41.035 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.035 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.035 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:41.035 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:36.880430+0000", 2024-09-04T17:48:41.035 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.8299999999999996e-07 2024-09-04T17:48:41.035 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.035 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.035 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:41.035 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:36.880438+0000", 2024-09-04T17:48:41.036 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.0560000000000005e-06 2024-09-04T17:48:41.036 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.036 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.036 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:41.036 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:36.880469+0000", 2024-09-04T17:48:41.036 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.1272000000000001e-05 2024-09-04T17:48:41.036 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.036 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.036 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:41.036 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:36.880488+0000", 2024-09-04T17:48:41.036 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8882999999999999e-05 2024-09-04T17:48:41.036 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.036 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.036 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:41.037 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:36.880665+0000", 2024-09-04T17:48:41.037 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000177209 2024-09-04T17:48:41.037 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.037 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.037 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:41.037 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:36.880726+0000", 2024-09-04T17:48:41.037 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.0594999999999998e-05 2024-09-04T17:48:41.037 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.037 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.037 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:41.037 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:36.885889+0000", 2024-09-04T17:48:41.037 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0051631949999999998 2024-09-04T17:48:41.037 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.038 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.038 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:41.038 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:36.885995+0000", 2024-09-04T17:48:41.038 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000106016 2024-09-04T17:48:41.038 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:41.038 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:41.038 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:41.038 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.038 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.038 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(osd.2.0:484 3.0 e157/121 3:070e9b06:::rbd_data.113418f5e708.000000000000002e:16 v 157'438, mlcod=155'433)", 2024-09-04T17:48:41.038 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:36.881063+0000", 2024-09-04T17:48:41.038 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 4.0591245999999996, 2024-09-04T17:48:41.038 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0094645140000000003, 2024-09-04T17:48:41.039 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:41.039 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:41.039 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:41.039 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:41.039 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.039 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:41.039 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:36.881063+0000", 2024-09-04T17:48:41.039 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:41.039 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.039 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.039 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:41.039 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:36.881063+0000", 2024-09-04T17:48:41.039 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:41.039 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.040 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.040 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:41.040 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:36.881065+0000", 2024-09-04T17:48:41.040 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8360000000000001e-06 2024-09-04T17:48:41.040 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.040 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.040 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:41.040 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:36.881080+0000", 2024-09-04T17:48:41.040 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5046e-05 2024-09-04T17:48:41.040 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.040 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.040 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:41.040 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:36.881109+0000", 2024-09-04T17:48:41.041 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.9246000000000001e-05 2024-09-04T17:48:41.041 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.041 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.041 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:41.041 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:36.881129+0000", 2024-09-04T17:48:41.041 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0018999999999999e-05 2024-09-04T17:48:41.041 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.041 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.041 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:41.041 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:36.882909+0000", 2024-09-04T17:48:41.041 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.001779912 2024-09-04T17:48:41.041 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.041 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.041 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:41.042 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:36.882973+0000", 2024-09-04T17:48:41.042 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.4177999999999994e-05 2024-09-04T17:48:41.042 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.042 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.042 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:41.042 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:36.890404+0000", 2024-09-04T17:48:41.042 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0074313119999999998 2024-09-04T17:48:41.042 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.042 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.042 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:41.042 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:36.890527+0000", 2024-09-04T17:48:41.042 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000122965 2024-09-04T17:48:41.042 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:41.043 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:41.043 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:41.043 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.043 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.043 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:100815 3.10 3:0883b82e:::rbd_data.1134c5a43465.0000000000000003:head [delete] snapc 15=[] ondisk+write+known_if_redirected+supports_pool_eio e158)", 2024-09-04T17:48:41.043 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:37.972704+0000", 2024-09-04T17:48:41.043 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 2.9674834309999998, 2024-09-04T17:48:41.043 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.006327404, 2024-09-04T17:48:41.043 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:41.043 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:41.043 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:41.043 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:41.043 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:41.043 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:41.044 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 100815 2024-09-04T17:48:41.044 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.044 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:41.044 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.044 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:41.044 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:37.972704+0000", 2024-09-04T17:48:41.044 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:41.044 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.044 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.044 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:41.044 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:37.972704+0000", 2024-09-04T17:48:41.044 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:41.045 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.045 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.045 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:41.045 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:37.972706+0000", 2024-09-04T17:48:41.045 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6780000000000001e-06 2024-09-04T17:48:41.045 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.045 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.045 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:41.045 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:37.972713+0000", 2024-09-04T17:48:41.045 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.6690000000000006e-06 2024-09-04T17:48:41.045 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.045 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.045 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:41.046 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:37.972731+0000", 2024-09-04T17:48:41.046 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8082999999999999e-05 2024-09-04T17:48:41.046 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.046 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.046 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:41.046 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:37.972741+0000", 2024-09-04T17:48:41.046 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.3230000000000004e-06 2024-09-04T17:48:41.046 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.046 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.046 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:41.046 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:37.972840+0000", 2024-09-04T17:48:41.046 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.9079000000000005e-05 2024-09-04T17:48:41.046 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.046 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.046 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op must wait for map", 2024-09-04T17:48:41.047 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:37.972850+0000", 2024-09-04T17:48:41.047 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.8490000000000001e-06 2024-09-04T17:48:41.047 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.047 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.047 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:41.047 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:37.976597+0000", 2024-09-04T17:48:41.047 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.003747289 2024-09-04T17:48:41.047 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.047 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.047 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:41.047 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:37.976972+0000", 2024-09-04T17:48:41.047 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00037561499999999999 2024-09-04T17:48:41.047 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.048 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.048 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 0", 2024-09-04T17:48:41.048 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:37.977275+0000", 2024-09-04T17:48:41.048 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00030304000000000003 2024-09-04T17:48:41.048 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.048 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.048 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:48:41.048 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:37.978722+0000", 2024-09-04T17:48:41.048 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.001446922 2024-09-04T17:48:41.048 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.048 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.048 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:41.049 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:37.978931+0000", 2024-09-04T17:48:41.049 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000208171 2024-09-04T17:48:41.049 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.049 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.049 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:41.049 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:37.978990+0000", 2024-09-04T17:48:41.049 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.9079999999999997e-05 2024-09-04T17:48:41.049 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:41.049 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:41.049 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:41.049 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:37.979031+0000", 2024-09-04T17:48:41.049 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.1606000000000001e-05 2024-09-04T17:48:41.049 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:41.049 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:41.050 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:41.050 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:41.050 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:41.050 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:41.051 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:41.049+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:41.051 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-04T17:48:41.152 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:48:41.149+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:48:41.191 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:41.191 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:48:41.191 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:48:41.191 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:41.205 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-04T17:48:41.253 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:41.253+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:41.321 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:41.322 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:48:41.322 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:48:41.322 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:48:41.322 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:41.336 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-04T17:48:41.385 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:48:41.383+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:48:41.454 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:48:41.453+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:48:41.463 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:41.463 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:48:41.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:48:41.463 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:48:41.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.463 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:48444 3.12 3:4ff80fb5:::rbd_data.11346a2b0bc9.000000000000001d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-04T17:48:41.463 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:58.122601+0000", 2024-09-04T17:48:41.463 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 103.339449745, 2024-09-04T17:48:41.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057065713999999997, 2024-09-04T17:48:41.463 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:41.464 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:41.464 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:41.464 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:41.464 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:41.464 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:41.464 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 48444 2024-09-04T17:48:41.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.465 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:41.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:41.465 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122601+0000", 2024-09-04T17:48:41.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:41.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:41.465 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122601+0000", 2024-09-04T17:48:41.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:41.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:41.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122603+0000", 2024-09-04T17:48:41.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3499999999999999e-06 2024-09-04T17:48:41.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:41.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122614+0000", 2024-09-04T17:48:41.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.063e-05 2024-09-04T17:48:41.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.467 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.467 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:41.467 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122643+0000", 2024-09-04T17:48:41.467 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9434999999999999e-05 2024-09-04T17:48:41.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.467 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.467 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:41.467 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122657+0000", 2024-09-04T17:48:41.467 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3920000000000001e-05 2024-09-04T17:48:41.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.467 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.467 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:41.467 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122767+0000", 2024-09-04T17:48:41.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010981599999999999 2024-09-04T17:48:41.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.468 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:41.468 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122989+0000", 2024-09-04T17:48:41.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022158200000000001 2024-09-04T17:48:41.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.468 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 3", 2024-09-04T17:48:41.468 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.133349+0000", 2024-09-04T17:48:41.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010360729000000001 2024-09-04T17:48:41.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.469 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:41.469 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.179423+0000", 2024-09-04T17:48:41.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046073408000000003 2024-09-04T17:48:41.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.469 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:41.469 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.179453+0000", 2024-09-04T17:48:41.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0229000000000001e-05 2024-09-04T17:48:41.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.469 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:41.469 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.179545+0000", 2024-09-04T17:48:41.470 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1580000000000001e-05 2024-09-04T17:48:41.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.470 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:41.470 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.179667+0000", 2024-09-04T17:48:41.470 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012203500000000001 2024-09-04T17:48:41.470 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:41.470 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:41.470 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:41.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.471 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:55376 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-04T17:48:41.471 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:11.974053+0000", 2024-09-04T17:48:41.471 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 89.487997961000005, 2024-09-04T17:48:41.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060514722, 2024-09-04T17:48:41.471 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:41.471 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:41.471 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:41.471 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:41.471 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:41.471 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:41.471 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 55376 2024-09-04T17:48:41.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.471 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:41.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.472 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:41.472 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974053+0000", 2024-09-04T17:48:41.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:41.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.472 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:41.472 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974053+0000", 2024-09-04T17:48:41.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:41.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.473 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:41.473 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974054+0000", 2024-09-04T17:48:41.473 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.841e-06 2024-09-04T17:48:41.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.473 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:41.473 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974062+0000", 2024-09-04T17:48:41.473 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5460000000000003e-06 2024-09-04T17:48:41.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.473 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:41.473 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974077+0000", 2024-09-04T17:48:41.474 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5389000000000001e-05 2024-09-04T17:48:41.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.474 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:41.474 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974085+0000", 2024-09-04T17:48:41.474 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7659999999999998e-06 2024-09-04T17:48:41.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.474 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:41.474 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974188+0000", 2024-09-04T17:48:41.475 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000102521 2024-09-04T17:48:41.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.475 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:41.475 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974321+0000", 2024-09-04T17:48:41.475 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013324699999999999 2024-09-04T17:48:41.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-04T17:48:41.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.986481+0000", 2024-09-04T17:48:41.476 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012160450999999999 2024-09-04T17:48:41.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:41.477 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:12.034415+0000", 2024-09-04T17:48:41.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.047934065999999997 2024-09-04T17:48:41.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.477 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:41.477 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:12.034438+0000", 2024-09-04T17:48:41.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2923999999999999e-05 2024-09-04T17:48:41.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.478 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:41.478 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:12.034505+0000", 2024-09-04T17:48:41.478 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6711000000000006e-05 2024-09-04T17:48:41.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.478 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.478 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:41.478 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:12.034567+0000", 2024-09-04T17:48:41.478 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2260000000000004e-05 2024-09-04T17:48:41.478 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:41.478 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:41.478 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:41.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.478 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.478 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61141 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 2793472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:41.479 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819909+0000", 2024-09-04T17:48:41.479 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 87.642141968999994, 2024-09-04T17:48:41.479 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042786419999999999, 2024-09-04T17:48:41.479 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:41.479 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:41.479 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:41.479 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:41.479 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:41.479 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:41.479 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61141 2024-09-04T17:48:41.479 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.479 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:41.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.480 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:41.480 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819909+0000", 2024-09-04T17:48:41.480 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:41.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.480 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.480 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:41.480 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819909+0000", 2024-09-04T17:48:41.480 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:41.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.480 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.481 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:41.482 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819909+0000", 2024-09-04T17:48:41.482 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6300000000000005e-07 2024-09-04T17:48:41.482 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.482 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.482 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:41.482 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819913+0000", 2024-09-04T17:48:41.482 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4189999999999998e-06 2024-09-04T17:48:41.482 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.483 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.483 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:41.483 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819920+0000", 2024-09-04T17:48:41.483 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8619999999999997e-06 2024-09-04T17:48:41.483 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.483 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.483 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:41.483 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819926+0000", 2024-09-04T17:48:41.483 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9399999999999999e-06 2024-09-04T17:48:41.483 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.483 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.483 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:41.483 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862331+0000", 2024-09-04T17:48:41.484 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042405348000000002 2024-09-04T17:48:41.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.484 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.484 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:41.484 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862410+0000", 2024-09-04T17:48:41.484 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9078000000000004e-05 2024-09-04T17:48:41.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.484 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.484 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:41.485 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862695+0000", 2024-09-04T17:48:41.485 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028510999999999999 2024-09-04T17:48:41.485 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:41.485 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:41.485 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:41.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.485 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.485 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61142 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 2932736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:41.485 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819934+0000", 2024-09-04T17:48:41.485 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 87.642116350999999, 2024-09-04T17:48:41.486 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043101985000000002, 2024-09-04T17:48:41.486 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:41.486 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:41.486 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:41.486 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:41.486 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:41.487 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:41.487 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61142 2024-09-04T17:48:41.487 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.487 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:41.487 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.487 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:41.488 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819934+0000", 2024-09-04T17:48:41.488 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:41.488 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.488 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.488 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:41.488 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819934+0000", 2024-09-04T17:48:41.488 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:41.488 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.488 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.488 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:41.488 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819935+0000", 2024-09-04T17:48:41.488 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2024-09-04T17:48:41.489 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.489 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.489 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:41.489 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819937+0000", 2024-09-04T17:48:41.489 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5770000000000001e-06 2024-09-04T17:48:41.489 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.489 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.489 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:41.489 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819943+0000", 2024-09-04T17:48:41.489 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7950000000000004e-06 2024-09-04T17:48:41.490 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.490 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.490 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:41.490 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819948+0000", 2024-09-04T17:48:41.490 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6779999999999999e-06 2024-09-04T17:48:41.490 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.490 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.490 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:41.490 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862739+0000", 2024-09-04T17:48:41.490 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042791454999999999 2024-09-04T17:48:41.490 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.490 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.490 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:41.491 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862824+0000", 2024-09-04T17:48:41.491 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4790999999999995e-05 2024-09-04T17:48:41.491 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.491 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.491 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:41.491 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.863036+0000", 2024-09-04T17:48:41.491 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021224899999999999 2024-09-04T17:48:41.491 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:41.491 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:41.491 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:41.492 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.492 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.492 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61143 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 3039232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:41.492 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819954+0000", 2024-09-04T17:48:41.492 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 87.642096233000004, 2024-09-04T17:48:41.492 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043485134000000002, 2024-09-04T17:48:41.492 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:41.492 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:41.492 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:41.492 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:41.493 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:41.493 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:41.493 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61143 2024-09-04T17:48:41.493 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.493 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:41.493 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.493 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:41.493 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819954+0000", 2024-09-04T17:48:41.493 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:41.493 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.493 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.493 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:41.493 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819954+0000", 2024-09-04T17:48:41.494 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:41.494 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.494 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.494 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:41.494 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819955+0000", 2024-09-04T17:48:41.494 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7300000000000002e-07 2024-09-04T17:48:41.494 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.494 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.494 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:41.494 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819957+0000", 2024-09-04T17:48:41.494 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.024e-06 2024-09-04T17:48:41.494 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.494 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.494 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:41.495 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819963+0000", 2024-09-04T17:48:41.495 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8109999999999999e-06 2024-09-04T17:48:41.495 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.495 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.495 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:41.495 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819967+0000", 2024-09-04T17:48:41.495 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6149999999999999e-06 2024-09-04T17:48:41.495 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.495 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.495 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:41.495 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.863082+0000", 2024-09-04T17:48:41.495 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043115078000000001 2024-09-04T17:48:41.495 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.496 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.496 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:41.496 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.863172+0000", 2024-09-04T17:48:41.496 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0181999999999998e-05 2024-09-04T17:48:41.496 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.496 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.496 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:41.496 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.863440+0000", 2024-09-04T17:48:41.496 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026705099999999997 2024-09-04T17:48:41.496 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:41.496 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:41.496 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:41.497 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.497 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.497 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:70015 3.6 3:6012e780:::rbd_data.11341aa245a3.000000000000005b:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-04T17:48:41.497 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:26.155497+0000", 2024-09-04T17:48:41.497 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 75.306553175999994, 2024-09-04T17:48:41.497 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.049007124999999999, 2024-09-04T17:48:41.497 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:41.497 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:41.497 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:41.497 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:41.497 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:41.497 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:41.497 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 70015 2024-09-04T17:48:41.498 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.498 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:41.498 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.498 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:41.498 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155497+0000", 2024-09-04T17:48:41.498 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:41.498 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.498 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.498 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:41.498 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155497+0000", 2024-09-04T17:48:41.498 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:41.499 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.499 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.499 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:41.499 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155500+0000", 2024-09-04T17:48:41.499 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7070000000000001e-06 2024-09-04T17:48:41.499 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.500 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.500 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:41.500 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155513+0000", 2024-09-04T17:48:41.500 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2989e-05 2024-09-04T17:48:41.500 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.500 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.500 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:41.500 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155544+0000", 2024-09-04T17:48:41.500 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0747000000000002e-05 2024-09-04T17:48:41.500 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.501 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.501 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:41.501 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155560+0000", 2024-09-04T17:48:41.501 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5866e-05 2024-09-04T17:48:41.501 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.501 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.501 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:41.501 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155675+0000", 2024-09-04T17:48:41.501 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000114838 2024-09-04T17:48:41.501 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.501 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.501 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:41.501 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155803+0000", 2024-09-04T17:48:41.502 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012851799999999999 2024-09-04T17:48:41.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.502 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 1", 2024-09-04T17:48:41.502 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.163582+0000", 2024-09-04T17:48:41.502 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0077790960000000001 2024-09-04T17:48:41.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.502 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:41.502 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.204055+0000", 2024-09-04T17:48:41.502 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.040472517 2024-09-04T17:48:41.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.503 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:41.503 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.204195+0000", 2024-09-04T17:48:41.503 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014055200000000001 2024-09-04T17:48:41.503 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.503 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.503 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:41.503 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.204373+0000", 2024-09-04T17:48:41.503 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000177784 2024-09-04T17:48:41.503 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.503 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.503 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:41.503 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.204505+0000", 2024-09-04T17:48:41.503 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013151100000000001 2024-09-04T17:48:41.504 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:41.504 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:41.504 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:41.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.504 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.504 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:70434 3.12 3:4cd2342b:::rbd_data.11341aa245a3.0000000000000091:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-04T17:48:41.504 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:26.943124+0000", 2024-09-04T17:48:41.504 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 74.518926394000005, 2024-09-04T17:48:41.504 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055011831999999997, 2024-09-04T17:48:41.504 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:41.504 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:41.504 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:41.504 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:41.505 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:41.505 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:41.505 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 70434 2024-09-04T17:48:41.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.505 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:41.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.505 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:41.505 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943124+0000", 2024-09-04T17:48:41.505 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:41.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.505 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:41.505 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943124+0000", 2024-09-04T17:48:41.506 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:41.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.506 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.506 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:41.506 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943128+0000", 2024-09-04T17:48:41.506 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8460000000000001e-06 2024-09-04T17:48:41.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.506 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.506 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:41.506 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943142+0000", 2024-09-04T17:48:41.507 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4089e-05 2024-09-04T17:48:41.507 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.507 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.507 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:41.507 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943177+0000", 2024-09-04T17:48:41.507 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5167999999999999e-05 2024-09-04T17:48:41.507 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.507 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.507 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:41.507 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943194+0000", 2024-09-04T17:48:41.507 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6235e-05 2024-09-04T17:48:41.507 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.507 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.507 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:41.508 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943286+0000", 2024-09-04T17:48:41.508 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2793000000000004e-05 2024-09-04T17:48:41.508 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.508 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.508 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:41.508 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943526+0000", 2024-09-04T17:48:41.509 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023950999999999999 2024-09-04T17:48:41.509 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.509 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.509 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 1", 2024-09-04T17:48:41.509 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.952871+0000", 2024-09-04T17:48:41.509 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0093453070000000006 2024-09-04T17:48:41.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.510 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:41.510 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.997968+0000", 2024-09-04T17:48:41.510 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045096613000000001 2024-09-04T17:48:41.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.510 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:41.511 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.998006+0000", 2024-09-04T17:48:41.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8257000000000003e-05 2024-09-04T17:48:41.511 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.511 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.511 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:41.511 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.998073+0000", 2024-09-04T17:48:41.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6803000000000004e-05 2024-09-04T17:48:41.512 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.512 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.512 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:41.512 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.998136+0000", 2024-09-04T17:48:41.512 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3211000000000002e-05 2024-09-04T17:48:41.512 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:41.512 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:41.513 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:41.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.513 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.513 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93841 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 5111808~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:41.513 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386491+0000", 2024-09-04T17:48:41.513 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.075559196999997, 2024-09-04T17:48:41.513 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043069578999999997, 2024-09-04T17:48:41.514 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:41.514 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:41.514 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:41.514 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:41.514 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:41.514 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:41.514 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93841 2024-09-04T17:48:41.514 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.515 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:41.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.515 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:41.515 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386491+0000", 2024-09-04T17:48:41.515 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:41.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.515 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:41.515 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386491+0000", 2024-09-04T17:48:41.515 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:41.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.516 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:41.516 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386492+0000", 2024-09-04T17:48:41.516 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0299999999999999e-07 2024-09-04T17:48:41.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.516 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.516 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:41.516 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386499+0000", 2024-09-04T17:48:41.516 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6089999999999997e-06 2024-09-04T17:48:41.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.516 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.516 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:41.516 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386506+0000", 2024-09-04T17:48:41.517 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3599999999999998e-06 2024-09-04T17:48:41.517 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.517 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.517 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:41.517 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386514+0000", 2024-09-04T17:48:41.517 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9379999999999992e-06 2024-09-04T17:48:41.517 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.517 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.517 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:41.517 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.428957+0000", 2024-09-04T17:48:41.517 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042442831 2024-09-04T17:48:41.517 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.518 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.518 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:41.518 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.429080+0000", 2024-09-04T17:48:41.518 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012359299999999999 2024-09-04T17:48:41.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.518 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.518 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:41.518 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.429561+0000", 2024-09-04T17:48:41.518 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00048074499999999998 2024-09-04T17:48:41.518 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:41.519 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:41.519 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:41.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.519 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.519 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93842 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 4259840~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:41.519 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386521+0000", 2024-09-04T17:48:41.519 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.075529265, 2024-09-04T17:48:41.519 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043661933999999999, 2024-09-04T17:48:41.519 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:41.519 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:41.519 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:41.519 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:41.519 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:41.520 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:41.520 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93842 2024-09-04T17:48:41.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.520 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:41.520 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.520 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:41.520 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386521+0000", 2024-09-04T17:48:41.520 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:41.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.520 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.520 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:41.520 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386521+0000", 2024-09-04T17:48:41.521 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:41.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.521 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.521 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:41.521 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386522+0000", 2024-09-04T17:48:41.521 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4700000000000001e-07 2024-09-04T17:48:41.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.521 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.521 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:41.521 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386524+0000", 2024-09-04T17:48:41.522 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4559999999999999e-06 2024-09-04T17:48:41.522 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.522 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.522 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:41.522 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386534+0000", 2024-09-04T17:48:41.522 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2639999999999997e-06 2024-09-04T17:48:41.522 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.522 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.522 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:41.522 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386539+0000", 2024-09-04T17:48:41.522 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6130000000000003e-06 2024-09-04T17:48:41.522 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.522 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.523 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:41.523 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.429577+0000", 2024-09-04T17:48:41.523 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043037757000000003 2024-09-04T17:48:41.523 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.523 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.523 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:41.523 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.429638+0000", 2024-09-04T17:48:41.523 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1235999999999994e-05 2024-09-04T17:48:41.523 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.523 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.523 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:41.523 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.430183+0000", 2024-09-04T17:48:41.523 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000545061 2024-09-04T17:48:41.524 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:41.524 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:41.524 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:41.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.524 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.524 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93843 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 4390912~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:41.524 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386547+0000", 2024-09-04T17:48:41.524 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.075504027999997, 2024-09-04T17:48:41.524 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044260668000000003, 2024-09-04T17:48:41.524 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:41.524 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:41.524 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:41.524 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:41.525 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:41.525 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:41.525 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93843 2024-09-04T17:48:41.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.525 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:41.525 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.525 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:41.525 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386547+0000", 2024-09-04T17:48:41.525 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:41.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.525 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.525 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:41.526 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386547+0000", 2024-09-04T17:48:41.526 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:41.526 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.526 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.526 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:41.526 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386547+0000", 2024-09-04T17:48:41.526 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4199999999999996e-07 2024-09-04T17:48:41.526 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.526 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.526 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:41.526 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386550+0000", 2024-09-04T17:48:41.526 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6199999999999999e-06 2024-09-04T17:48:41.526 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.527 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.527 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:41.527 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386557+0000", 2024-09-04T17:48:41.527 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1919999999999999e-06 2024-09-04T17:48:41.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.527 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.527 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:41.527 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386562+0000", 2024-09-04T17:48:41.527 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2580000000000003e-06 2024-09-04T17:48:41.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.527 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.527 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:41.527 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.430226+0000", 2024-09-04T17:48:41.527 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043663484000000002 2024-09-04T17:48:41.528 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.528 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.528 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:41.528 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.430286+0000", 2024-09-04T17:48:41.528 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0455999999999999e-05 2024-09-04T17:48:41.528 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.528 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.528 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:41.528 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.430807+0000", 2024-09-04T17:48:41.528 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00052111599999999996 2024-09-04T17:48:41.528 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:41.528 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:41.529 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:41.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.529 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.529 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93844 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 4521984~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:41.529 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386603+0000", 2024-09-04T17:48:41.529 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.075447977000003, 2024-09-04T17:48:41.529 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044705350999999997, 2024-09-04T17:48:41.529 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:41.529 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:41.529 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:41.529 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:41.529 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:41.530 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:41.530 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93844 2024-09-04T17:48:41.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.530 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:41.530 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.530 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:41.530 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386603+0000", 2024-09-04T17:48:41.530 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:41.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.530 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.530 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:41.530 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386603+0000", 2024-09-04T17:48:41.530 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:41.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.531 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.531 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:41.531 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386603+0000", 2024-09-04T17:48:41.531 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.92e-07 2024-09-04T17:48:41.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.531 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.531 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:41.531 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386606+0000", 2024-09-04T17:48:41.531 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5500000000000001e-06 2024-09-04T17:48:41.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.531 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.532 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:41.532 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386614+0000", 2024-09-04T17:48:41.532 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9570000000000002e-06 2024-09-04T17:48:41.532 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.532 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.532 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:41.532 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386619+0000", 2024-09-04T17:48:41.532 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5119999999999999e-06 2024-09-04T17:48:41.532 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.532 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.532 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:41.532 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.430828+0000", 2024-09-04T17:48:41.532 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044208654 2024-09-04T17:48:41.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.533 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.533 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:41.533 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.430903+0000", 2024-09-04T17:48:41.533 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4629999999999995e-05 2024-09-04T17:48:41.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.533 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.533 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:41.533 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.431308+0000", 2024-09-04T17:48:41.533 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000405256 2024-09-04T17:48:41.533 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:41.533 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:41.533 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:41.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.534 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.534 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93845 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 4653056~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:41.534 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386627+0000", 2024-09-04T17:48:41.534 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.075423387999997, 2024-09-04T17:48:41.534 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045262927000000001, 2024-09-04T17:48:41.534 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:41.534 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:41.534 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:41.534 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:41.534 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:41.534 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:41.535 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93845 2024-09-04T17:48:41.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.535 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:41.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:41.535 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386627+0000", 2024-09-04T17:48:41.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:41.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:41.535 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386627+0000", 2024-09-04T17:48:41.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:41.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.536 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.536 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:41.536 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386628+0000", 2024-09-04T17:48:41.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0999999999999998e-07 2024-09-04T17:48:41.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.536 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.536 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:41.536 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386630+0000", 2024-09-04T17:48:41.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3300000000000001e-06 2024-09-04T17:48:41.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.537 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.537 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:41.537 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386645+0000", 2024-09-04T17:48:41.537 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5186e-05 2024-09-04T17:48:41.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.537 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.537 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:41.537 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386651+0000", 2024-09-04T17:48:41.537 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7710000000000003e-06 2024-09-04T17:48:41.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.537 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.538 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:41.538 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.431349+0000", 2024-09-04T17:48:41.538 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044697723000000002 2024-09-04T17:48:41.538 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.538 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.538 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:41.538 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.431410+0000", 2024-09-04T17:48:41.538 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0677999999999997e-05 2024-09-04T17:48:41.538 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.538 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.538 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:41.539 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.431890+0000", 2024-09-04T17:48:41.539 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00048052899999999999 2024-09-04T17:48:41.539 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:41.539 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:41.539 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:41.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.539 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93846 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 4784128~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:41.539 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386659+0000", 2024-09-04T17:48:41.539 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.075391719000002, 2024-09-04T17:48:41.539 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045784490999999997, 2024-09-04T17:48:41.539 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:41.539 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:41.540 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:41.540 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:41.540 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:41.540 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:41.540 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93846 2024-09-04T17:48:41.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.540 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:41.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:41.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386659+0000", 2024-09-04T17:48:41.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:41.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.541 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:41.541 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386659+0000", 2024-09-04T17:48:41.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:41.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.541 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:41.541 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386659+0000", 2024-09-04T17:48:41.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7000000000000005e-07 2024-09-04T17:48:41.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.541 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:41.541 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386662+0000", 2024-09-04T17:48:41.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3300000000000001e-06 2024-09-04T17:48:41.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.542 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.542 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:41.542 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386669+0000", 2024-09-04T17:48:41.542 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0589999999999997e-06 2024-09-04T17:48:41.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.542 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.542 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:41.542 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386674+0000", 2024-09-04T17:48:41.542 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3009999999999997e-06 2024-09-04T17:48:41.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.542 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.542 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:41.542 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.431934+0000", 2024-09-04T17:48:41.543 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045259570999999998 2024-09-04T17:48:41.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.543 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.543 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:41.543 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.431998+0000", 2024-09-04T17:48:41.543 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4775000000000002e-05 2024-09-04T17:48:41.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.543 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.543 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:41.543 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.432443+0000", 2024-09-04T17:48:41.543 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00044488500000000001 2024-09-04T17:48:41.543 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:41.543 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:41.543 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:41.544 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.544 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.544 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93847 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 4915200~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:41.544 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386681+0000", 2024-09-04T17:48:41.544 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.075369461999998, 2024-09-04T17:48:41.544 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046305101000000001, 2024-09-04T17:48:41.544 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:41.544 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:41.544 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:41.544 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:41.544 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:41.544 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:41.544 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93847 2024-09-04T17:48:41.545 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.545 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:41.545 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.545 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:41.545 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386681+0000", 2024-09-04T17:48:41.545 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:41.545 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.545 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.545 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:41.545 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386681+0000", 2024-09-04T17:48:41.545 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:41.545 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.546 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.546 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:41.546 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386682+0000", 2024-09-04T17:48:41.546 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1699999999999998e-07 2024-09-04T17:48:41.546 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.546 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.546 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:41.546 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386684+0000", 2024-09-04T17:48:41.546 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.153e-06 2024-09-04T17:48:41.546 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.546 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.546 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:41.546 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386693+0000", 2024-09-04T17:48:41.546 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9639999999999992e-06 2024-09-04T17:48:41.547 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.547 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.547 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:41.547 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386698+0000", 2024-09-04T17:48:41.547 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3709999999999999e-06 2024-09-04T17:48:41.547 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.547 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.547 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:41.547 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.432476+0000", 2024-09-04T17:48:41.547 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045778147999999998 2024-09-04T17:48:41.547 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.547 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.547 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:41.547 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.432529+0000", 2024-09-04T17:48:41.548 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3165999999999999e-05 2024-09-04T17:48:41.548 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.548 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.548 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:41.548 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.432986+0000", 2024-09-04T17:48:41.548 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00045678199999999998 2024-09-04T17:48:41.548 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:41.548 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:41.548 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:41.548 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.548 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.548 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93848 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 5046272~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:41.549 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386705+0000", 2024-09-04T17:48:41.549 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.075345360999997, 2024-09-04T17:48:41.549 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046825259000000001, 2024-09-04T17:48:41.549 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:41.549 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:41.549 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:41.549 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:41.549 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:41.549 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:41.549 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93848 2024-09-04T17:48:41.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.549 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:41.549 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.550 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:41.550 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386705+0000", 2024-09-04T17:48:41.550 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:41.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.550 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.550 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:41.550 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386705+0000", 2024-09-04T17:48:41.550 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:41.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.550 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.550 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:41.550 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386706+0000", 2024-09-04T17:48:41.550 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1500000000000005e-07 2024-09-04T17:48:41.551 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.551 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.551 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:41.551 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386708+0000", 2024-09-04T17:48:41.551 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2979999999999999e-06 2024-09-04T17:48:41.551 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.551 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.551 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:41.551 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386715+0000", 2024-09-04T17:48:41.551 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8669999999999996e-06 2024-09-04T17:48:41.551 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.551 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.551 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:41.551 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386720+0000", 2024-09-04T17:48:41.552 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.242e-06 2024-09-04T17:48:41.552 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.552 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.552 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:41.552 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.433019+0000", 2024-09-04T17:48:41.552 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046298359999999997 2024-09-04T17:48:41.552 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.552 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.552 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:41.552 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.433084+0000", 2024-09-04T17:48:41.552 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5395999999999995e-05 2024-09-04T17:48:41.552 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.552 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.553 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:41.553 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.433531+0000", 2024-09-04T17:48:41.553 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00044658099999999999 2024-09-04T17:48:41.553 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:41.553 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:41.553 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:41.553 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.553 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.553 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93849 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 5177344~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:41.553 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386727+0000", 2024-09-04T17:48:41.553 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.075323462999997, 2024-09-04T17:48:41.553 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.047267796000000001, 2024-09-04T17:48:41.554 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:41.554 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:41.554 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:41.554 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:41.554 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:41.554 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:41.554 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93849 2024-09-04T17:48:41.554 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.554 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:41.554 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.554 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:41.554 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386727+0000", 2024-09-04T17:48:41.554 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:41.555 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.555 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.555 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:41.555 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386727+0000", 2024-09-04T17:48:41.555 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:41.555 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.555 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.555 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:41.555 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386728+0000", 2024-09-04T17:48:41.555 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1699999999999998e-07 2024-09-04T17:48:41.555 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.555 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.555 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:41.555 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386730+0000", 2024-09-04T17:48:41.556 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2589999999999999e-06 2024-09-04T17:48:41.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.556 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.556 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:41.556 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386751+0000", 2024-09-04T17:48:41.556 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1424999999999999e-05 2024-09-04T17:48:41.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.556 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.556 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:41.556 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386759+0000", 2024-09-04T17:48:41.556 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9400000000000002e-06 2024-09-04T17:48:41.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.556 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.557 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:41.557 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.433559+0000", 2024-09-04T17:48:41.557 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046799573999999997 2024-09-04T17:48:41.557 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.557 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.557 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:41.557 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.433610+0000", 2024-09-04T17:48:41.557 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1134999999999997e-05 2024-09-04T17:48:41.557 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.558 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:41.558 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.433995+0000", 2024-09-04T17:48:41.558 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000384946 2024-09-04T17:48:41.558 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:41.558 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:41.558 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:41.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.559 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:96175 3.12 3:4d4471be:::rbd_data.1134c4e2d26.000000000000006e:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123)", 2024-09-04T17:48:41.559 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:48:02.919542+0000", 2024-09-04T17:48:41.559 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 38.542508218999998, 2024-09-04T17:48:41.559 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060180626000000001, 2024-09-04T17:48:41.559 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:41.559 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:41.559 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:41.559 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:41.559 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:41.559 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:41.559 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 96175 2024-09-04T17:48:41.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.559 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:41.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.560 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:41.560 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.919542+0000", 2024-09-04T17:48:41.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:41.560 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.560 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:41.560 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.919542+0000", 2024-09-04T17:48:41.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:41.560 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.560 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:41.560 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.919545+0000", 2024-09-04T17:48:41.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0199999999999999e-06 2024-09-04T17:48:41.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.561 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:41.561 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.919559+0000", 2024-09-04T17:48:41.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3293e-05 2024-09-04T17:48:41.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.561 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:41.561 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.919587+0000", 2024-09-04T17:48:41.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7922000000000001e-05 2024-09-04T17:48:41.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.562 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:41.562 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.919601+0000", 2024-09-04T17:48:41.562 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4657e-05 2024-09-04T17:48:41.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.562 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.562 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:41.562 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.919708+0000", 2024-09-04T17:48:41.562 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010624999999999999 2024-09-04T17:48:41.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.562 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.562 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:41.562 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.919929+0000", 2024-09-04T17:48:41.562 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022147399999999999 2024-09-04T17:48:41.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.563 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2024-09-04T17:48:41.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.929774+0000", 2024-09-04T17:48:41.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0098449839999999993 2024-09-04T17:48:41.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.563 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:41.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.979511+0000", 2024-09-04T17:48:41.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.049736537999999997 2024-09-04T17:48:41.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.563 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:41.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.979547+0000", 2024-09-04T17:48:41.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6069000000000003e-05 2024-09-04T17:48:41.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:41.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.979642+0000", 2024-09-04T17:48:41.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5526999999999995e-05 2024-09-04T17:48:41.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:41.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.979723+0000", 2024-09-04T17:48:41.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0891999999999997e-05 2024-09-04T17:48:41.564 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:41.564 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:41.565 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:41.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.565 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:97148 3.4 3:22dd2475:::rbd_id.image_client.0-clone39:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e128)", 2024-09-04T17:48:41.565 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:48:07.831881+0000", 2024-09-04T17:48:41.565 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 33.630169985000002, 2024-09-04T17:48:41.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.026553585, 2024-09-04T17:48:41.565 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:41.565 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:41.565 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:41.565 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:41.565 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:41.565 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:41.566 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 97148 2024-09-04T17:48:41.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.566 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:41.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.566 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:41.566 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:07.831881+0000", 2024-09-04T17:48:41.566 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:41.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.566 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:41.566 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:07.831881+0000", 2024-09-04T17:48:41.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:41.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:41.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:07.831883+0000", 2024-09-04T17:48:41.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3369999999999998e-06 2024-09-04T17:48:41.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:41.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:07.831890+0000", 2024-09-04T17:48:41.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7050000000000002e-06 2024-09-04T17:48:41.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:41.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:07.831905+0000", 2024-09-04T17:48:41.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5512999999999999e-05 2024-09-04T17:48:41.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:41.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:07.831918+0000", 2024-09-04T17:48:41.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3223e-05 2024-09-04T17:48:41.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:41.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:07.833441+0000", 2024-09-04T17:48:41.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0015228920000000001 2024-09-04T17:48:41.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-09-04T17:48:41.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:07.833451+0000", 2024-09-04T17:48:41.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6600000000000007e-06 2024-09-04T17:48:41.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:41.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:07.837721+0000", 2024-09-04T17:48:41.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0042702410000000001 2024-09-04T17:48:41.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-04T17:48:41.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:07.837732+0000", 2024-09-04T17:48:41.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.057e-05 2024-09-04T17:48:41.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:41.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:08.856291+0000", 2024-09-04T17:48:41.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0185595350000001 2024-09-04T17:48:41.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:41.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:08.856503+0000", 2024-09-04T17:48:41.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000212127 2024-09-04T17:48:41.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2,5", 2024-09-04T17:48:41.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:08.856664+0000", 2024-09-04T17:48:41.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016072800000000001 2024-09-04T17:48:41.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:41.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:08.857811+0000", 2024-09-04T17:48:41.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001146441 2024-09-04T17:48:41.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.572 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:41.572 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:08.858202+0000", 2024-09-04T17:48:41.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000391605 2024-09-04T17:48:41.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.572 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.572 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:41.572 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:08.858239+0000", 2024-09-04T17:48:41.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7200000000000003e-05 2024-09-04T17:48:41.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.572 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.572 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:41.572 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:08.858287+0000", 2024-09-04T17:48:41.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7515999999999997e-05 2024-09-04T17:48:41.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.573 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:41.573 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:08.858434+0000", 2024-09-04T17:48:41.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014729200000000001 2024-09-04T17:48:41.573 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:41.573 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:41.573 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:41.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.574 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:99510 3.18 3:1bf8b1d0:::rbd_header.11346a2b0bc9:head [call rbd.snapshot_remove in=18b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e147)", 2024-09-04T17:48:41.574 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:48:27.032522+0000", 2024-09-04T17:48:41.574 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.429528122000001, 2024-09-04T17:48:41.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.84201522500000003, 2024-09-04T17:48:41.574 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:41.574 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:41.574 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:41.574 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:41.574 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:41.574 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:41.574 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 99510 2024-09-04T17:48:41.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.575 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:41.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:41.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032522+0000", 2024-09-04T17:48:41.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:41.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:41.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032522+0000", 2024-09-04T17:48:41.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:41.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.576 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:41.576 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032525+0000", 2024-09-04T17:48:41.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4789999999999999e-06 2024-09-04T17:48:41.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.576 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:41.576 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032536+0000", 2024-09-04T17:48:41.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0557000000000001e-05 2024-09-04T17:48:41.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:41.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032555+0000", 2024-09-04T17:48:41.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9782000000000001e-05 2024-09-04T17:48:41.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:41.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032569+0000", 2024-09-04T17:48:41.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4012000000000001e-05 2024-09-04T17:48:41.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:41.578 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032649+0000", 2024-09-04T17:48:41.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9782000000000003e-05 2024-09-04T17:48:41.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-09-04T17:48:41.578 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032657+0000", 2024-09-04T17:48:41.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7999999999999999e-06 2024-09-04T17:48:41.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:41.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.035521+0000", 2024-09-04T17:48:41.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028640850000000002 2024-09-04T17:48:41.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-04T17:48:41.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.035534+0000", 2024-09-04T17:48:41.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3157e-05 2024-09-04T17:48:41.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:41.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.869836+0000", 2024-09-04T17:48:41.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.83430168400000004 2024-09-04T17:48:41.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:41.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.870070+0000", 2024-09-04T17:48:41.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023443300000000001 2024-09-04T17:48:41.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 3", 2024-09-04T17:48:41.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.870423+0000", 2024-09-04T17:48:41.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00035290700000000001 2024-09-04T17:48:41.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:41.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.871254+0000", 2024-09-04T17:48:41.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00083113700000000002 2024-09-04T17:48:41.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.581 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:41.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.874234+0000", 2024-09-04T17:48:41.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029795189999999999 2024-09-04T17:48:41.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.581 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:41.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.874321+0000", 2024-09-04T17:48:41.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7637000000000001e-05 2024-09-04T17:48:41.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:41.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.874538+0000", 2024-09-04T17:48:41.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021625400000000001 2024-09-04T17:48:41.582 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:41.582 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:41.582 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:41.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.583 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:99511 3.18 3:1bf8b1d0:::rbd_header.11346a2b0bc9:head [watch reconnect cookie 140018821496000 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147)", 2024-09-04T17:48:41.583 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:48:27.032602+0000", 2024-09-04T17:48:41.583 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.429448319, 2024-09-04T17:48:41.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.84736467999999998, 2024-09-04T17:48:41.584 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:41.584 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:41.584 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:41.584 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:41.584 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:41.584 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:41.584 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 99511 2024-09-04T17:48:41.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.585 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:41.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:41.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032602+0000", 2024-09-04T17:48:41.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:41.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:41.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032602+0000", 2024-09-04T17:48:41.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:41.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:41.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032603+0000", 2024-09-04T17:48:41.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3300000000000001e-07 2024-09-04T17:48:41.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:41.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032610+0000", 2024-09-04T17:48:41.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.754e-06 2024-09-04T17:48:41.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:41.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032621+0000", 2024-09-04T17:48:41.587 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1175999999999999e-05 2024-09-04T17:48:41.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.587 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:41.587 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032630+0000", 2024-09-04T17:48:41.587 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7900000000000005e-06 2024-09-04T17:48:41.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.587 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:41.587 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032900+0000", 2024-09-04T17:48:41.587 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026967499999999998 2024-09-04T17:48:41.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting_for_map not empty", 2024-09-04T17:48:41.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032907+0000", 2024-09-04T17:48:41.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6969999999999993e-06 2024-09-04T17:48:41.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:41.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.036096+0000", 2024-09-04T17:48:41.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031887090000000001 2024-09-04T17:48:41.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-04T17:48:41.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.036107+0000", 2024-09-04T17:48:41.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1361e-05 2024-09-04T17:48:41.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:41.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.870783+0000", 2024-09-04T17:48:41.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.834675265 2024-09-04T17:48:41.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-09-04T17:48:41.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.870943+0000", 2024-09-04T17:48:41.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000160364 2024-09-04T17:48:41.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:41.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.874588+0000", 2024-09-04T17:48:41.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0036446529999999999 2024-09-04T17:48:41.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:41.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.874731+0000", 2024-09-04T17:48:41.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014334499999999999 2024-09-04T17:48:41.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:41.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:41.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:41.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.879967+0000", 2024-09-04T17:48:41.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005236058 2024-09-04T17:48:41.590 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:41.590 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:41.591 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:41.591 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:41.591 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:41.591 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:41.591 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 4, 5, 6, 7, 3] out_osds: [1] dead_osds: [] live_osds: [0, 1, 4, 5, 7, 3, 2, 6] 2024-09-04T17:48:41.591 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-04T17:48:41.592 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-09-04T17:48:41.592 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-04T17:48:41.592 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-04T17:48:41.592 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set filestore_inject_stall 3 2024-09-04T17:48:41.593 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:41.589+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:41.594 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-04T17:48:41.693 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:41.689+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:41.760 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:41.761 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:48:41.761 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:48:41.761 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:41.769 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:41.770 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "" 2024-09-04T17:48:41.770 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:41.778 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-04T17:48:41.793 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:41.793+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:41.894 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:41.891+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:41.896 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:41.896 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:48:41.896 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:48:41.896 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:48:41.896 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:41.913 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-04T17:48:41.995 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:48:41.993+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:48:42.085 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:42.085 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:48:42.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:48:42.085 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:48:42.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.085 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59950 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5365760~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:48:42.085 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654917+0000", 2024-09-04T17:48:42.085 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 88.428875246000004, 2024-09-04T17:48:42.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066822450000000005, 2024-09-04T17:48:42.086 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:42.086 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:42.086 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:42.086 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:42.086 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:42.086 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:42.086 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59950 2024-09-04T17:48:42.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.086 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:42.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:42.086 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654917+0000", 2024-09-04T17:48:42.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:42.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.087 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:42.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654917+0000", 2024-09-04T17:48:42.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:42.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.087 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:42.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654918+0000", 2024-09-04T17:48:42.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.29e-07 2024-09-04T17:48:42.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:42.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654919+0000", 2024-09-04T17:48:42.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2589999999999999e-06 2024-09-04T17:48:42.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:42.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654922+0000", 2024-09-04T17:48:42.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1350000000000001e-06 2024-09-04T17:48:42.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:42.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654925+0000", 2024-09-04T17:48:42.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1250000000000001e-06 2024-09-04T17:48:42.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:42.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721466+0000", 2024-09-04T17:48:42.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066540959999999996 2024-09-04T17:48:42.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:42.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721574+0000", 2024-09-04T17:48:42.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010747399999999999 2024-09-04T17:48:42.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:42.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721740+0000", 2024-09-04T17:48:42.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016626799999999999 2024-09-04T17:48:42.090 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:42.090 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:42.090 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:42.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.090 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59951 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5472256~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:48:42.090 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654929+0000", 2024-09-04T17:48:42.090 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 88.428864060999999, 2024-09-04T17:48:42.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067110534999999999, 2024-09-04T17:48:42.090 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:42.091 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:42.091 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:42.091 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:42.091 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:42.091 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:42.091 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59951 2024-09-04T17:48:42.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.091 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:42.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:42.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654929+0000", 2024-09-04T17:48:42.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:42.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:42.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654929+0000", 2024-09-04T17:48:42.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:42.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:42.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654929+0000", 2024-09-04T17:48:42.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5100000000000001e-07 2024-09-04T17:48:42.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:42.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654930+0000", 2024-09-04T17:48:42.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2440000000000001e-06 2024-09-04T17:48:42.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:42.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654933+0000", 2024-09-04T17:48:42.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1599999999999998e-06 2024-09-04T17:48:42.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:42.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654936+0000", 2024-09-04T17:48:42.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6560000000000001e-06 2024-09-04T17:48:42.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:42.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721789+0000", 2024-09-04T17:48:42.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066853250000000003 2024-09-04T17:48:42.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:42.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721885+0000", 2024-09-04T17:48:42.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5616000000000001e-05 2024-09-04T17:48:42.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:42.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722039+0000", 2024-09-04T17:48:42.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000154258 2024-09-04T17:48:42.094 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:42.095 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:42.095 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:42.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.095 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59952 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5505024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:48:42.095 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654939+0000", 2024-09-04T17:48:42.095 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 88.428853184000005, 2024-09-04T17:48:42.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067433742000000005, 2024-09-04T17:48:42.095 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:42.095 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:42.095 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:42.096 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:42.096 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:42.096 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:42.096 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59952 2024-09-04T17:48:42.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.096 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:42.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:42.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654939+0000", 2024-09-04T17:48:42.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:42.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:42.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654939+0000", 2024-09-04T17:48:42.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:42.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:42.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654940+0000", 2024-09-04T17:48:42.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3300000000000001e-07 2024-09-04T17:48:42.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:42.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654941+0000", 2024-09-04T17:48:42.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1570000000000001e-06 2024-09-04T17:48:42.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:42.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654944+0000", 2024-09-04T17:48:42.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1319999999999998e-06 2024-09-04T17:48:42.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:42.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654947+0000", 2024-09-04T17:48:42.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5969999999999999e-06 2024-09-04T17:48:42.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:42.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722089+0000", 2024-09-04T17:48:42.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067142347000000005 2024-09-04T17:48:42.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:42.099 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722201+0000", 2024-09-04T17:48:42.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011235600000000001 2024-09-04T17:48:42.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:42.099 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722373+0000", 2024-09-04T17:48:42.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017191999999999999 2024-09-04T17:48:42.099 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:42.099 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:42.099 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:42.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.100 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59953 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5611520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:48:42.100 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654950+0000", 2024-09-04T17:48:42.100 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 88.428842544000005, 2024-09-04T17:48:42.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067601852000000004, 2024-09-04T17:48:42.100 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:42.100 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:42.100 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:42.100 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:42.100 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:42.100 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:42.100 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59953 2024-09-04T17:48:42.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.101 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:42.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:42.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654950+0000", 2024-09-04T17:48:42.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:42.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:42.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654950+0000", 2024-09-04T17:48:42.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:42.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:42.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654950+0000", 2024-09-04T17:48:42.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2499999999999999e-07 2024-09-04T17:48:42.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:42.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654954+0000", 2024-09-04T17:48:42.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.287e-06 2024-09-04T17:48:42.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:42.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654962+0000", 2024-09-04T17:48:42.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4230000000000005e-06 2024-09-04T17:48:42.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.103 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:42.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654966+0000", 2024-09-04T17:48:42.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6449999999999998e-06 2024-09-04T17:48:42.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.103 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:42.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722417+0000", 2024-09-04T17:48:42.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067451188999999995 2024-09-04T17:48:42.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.103 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:42.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722467+0000", 2024-09-04T17:48:42.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0556999999999997e-05 2024-09-04T17:48:42.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.104 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:42.104 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722552+0000", 2024-09-04T17:48:42.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4525999999999997e-05 2024-09-04T17:48:42.104 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:42.104 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:42.104 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:42.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.104 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59954 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5718016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:48:42.104 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654970+0000", 2024-09-04T17:48:42.104 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 88.428822783000001, 2024-09-04T17:48:42.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067874285000000006, 2024-09-04T17:48:42.105 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:42.105 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:42.105 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:42.105 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:42.105 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:42.105 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:42.105 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59954 2024-09-04T17:48:42.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.105 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:42.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.105 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:42.105 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654970+0000", 2024-09-04T17:48:42.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:42.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:42.106 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654970+0000", 2024-09-04T17:48:42.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:42.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:42.106 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654970+0000", 2024-09-04T17:48:42.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5899999999999998e-07 2024-09-04T17:48:42.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:42.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654971+0000", 2024-09-04T17:48:42.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2929999999999999e-06 2024-09-04T17:48:42.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:42.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654975+0000", 2024-09-04T17:48:42.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4699999999999998e-06 2024-09-04T17:48:42.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:42.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654978+0000", 2024-09-04T17:48:42.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7930000000000002e-06 2024-09-04T17:48:42.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:42.108 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722600+0000", 2024-09-04T17:48:42.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067622229000000006 2024-09-04T17:48:42.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:42.108 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722689+0000", 2024-09-04T17:48:42.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9511000000000004e-05 2024-09-04T17:48:42.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:42.108 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722844+0000", 2024-09-04T17:48:42.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015473 2024-09-04T17:48:42.109 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:42.109 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:42.109 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:42.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.109 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59955 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5857280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:48:42.109 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654981+0000", 2024-09-04T17:48:42.109 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 88.428811433000007, 2024-09-04T17:48:42.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068052360000000006, 2024-09-04T17:48:42.109 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:42.109 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:42.109 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:42.110 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:42.110 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:42.110 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:42.110 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59955 2024-09-04T17:48:42.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.110 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:42.110 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.110 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:42.110 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654981+0000", 2024-09-04T17:48:42.110 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:42.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.110 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.110 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:42.111 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654981+0000", 2024-09-04T17:48:42.111 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:42.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.111 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:42.111 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654981+0000", 2024-09-04T17:48:42.111 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2399999999999999e-07 2024-09-04T17:48:42.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.111 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:42.111 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654985+0000", 2024-09-04T17:48:42.111 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0980000000000004e-06 2024-09-04T17:48:42.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.112 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.112 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:42.112 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654990+0000", 2024-09-04T17:48:42.112 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3429999999999998e-06 2024-09-04T17:48:42.112 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.112 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.112 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:42.112 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654993+0000", 2024-09-04T17:48:42.112 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8700000000000001e-06 2024-09-04T17:48:42.112 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.112 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.112 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:42.112 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722888+0000", 2024-09-04T17:48:42.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067895239999999996 2024-09-04T17:48:42.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:42.113 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722948+0000", 2024-09-04T17:48:42.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9750999999999998e-05 2024-09-04T17:48:42.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:42.113 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.723034+0000", 2024-09-04T17:48:42.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5834000000000005e-05 2024-09-04T17:48:42.113 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:42.113 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:42.114 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:42.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.114 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.114 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59956 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5963776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:48:42.114 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654998+0000", 2024-09-04T17:48:42.114 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 88.428794904, 2024-09-04T17:48:42.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068359518999999994, 2024-09-04T17:48:42.114 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:42.114 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:42.114 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:42.114 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:42.114 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:42.114 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:42.115 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59956 2024-09-04T17:48:42.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.115 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:42.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.115 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:42.115 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654998+0000", 2024-09-04T17:48:42.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:42.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.115 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:42.115 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654998+0000", 2024-09-04T17:48:42.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:42.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:42.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654998+0000", 2024-09-04T17:48:42.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4299999999999999e-07 2024-09-04T17:48:42.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:42.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654999+0000", 2024-09-04T17:48:42.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1799999999999999e-06 2024-09-04T17:48:42.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:42.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.655002+0000", 2024-09-04T17:48:42.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3639999999999999e-06 2024-09-04T17:48:42.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:42.117 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.655005+0000", 2024-09-04T17:48:42.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7970000000000001e-06 2024-09-04T17:48:42.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:42.117 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.723085+0000", 2024-09-04T17:48:42.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068080229000000006 2024-09-04T17:48:42.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:42.117 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.723197+0000", 2024-09-04T17:48:42.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000111344 2024-09-04T17:48:42.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.118 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:42.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.723357+0000", 2024-09-04T17:48:42.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016036199999999999 2024-09-04T17:48:42.118 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:42.118 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:42.118 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:42.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.118 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61082 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 2654208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:42.118 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.822664+0000", 2024-09-04T17:48:42.118 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 88.261128517000003, 2024-09-04T17:48:42.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067072242000000004, 2024-09-04T17:48:42.119 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:42.119 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:42.119 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:42.119 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:42.119 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:42.119 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:42.119 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61082 2024-09-04T17:48:42.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.119 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:42.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:42.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822664+0000", 2024-09-04T17:48:42.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:42.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.120 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:42.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822664+0000", 2024-09-04T17:48:42.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:42.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.120 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:42.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822664+0000", 2024-09-04T17:48:42.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6600000000000003e-07 2024-09-04T17:48:42.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.121 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:42.121 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822666+0000", 2024-09-04T17:48:42.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.536e-06 2024-09-04T17:48:42.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.121 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:42.121 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822669+0000", 2024-09-04T17:48:42.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3759999999999999e-06 2024-09-04T17:48:42.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.121 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:42.121 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822672+0000", 2024-09-04T17:48:42.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8109999999999999e-06 2024-09-04T17:48:42.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:42.122 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889263+0000", 2024-09-04T17:48:42.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066591259999999999 2024-09-04T17:48:42.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:42.122 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889369+0000", 2024-09-04T17:48:42.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000105362 2024-09-04T17:48:42.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:42.122 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889736+0000", 2024-09-04T17:48:42.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00036763099999999999 2024-09-04T17:48:42.123 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:42.123 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:42.123 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:42.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.123 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61083 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 2760704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:42.123 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.822678+0000", 2024-09-04T17:48:42.123 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 88.261114782999996, 2024-09-04T17:48:42.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067475996999999996, 2024-09-04T17:48:42.123 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:42.123 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:42.123 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:42.123 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:42.124 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:42.124 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:42.124 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61083 2024-09-04T17:48:42.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.124 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:42.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.124 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:42.124 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822678+0000", 2024-09-04T17:48:42.124 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:42.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.124 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:42.124 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822678+0000", 2024-09-04T17:48:42.124 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:42.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.125 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:42.125 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822725+0000", 2024-09-04T17:48:42.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7429000000000003e-05 2024-09-04T17:48:42.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.125 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:42.125 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822728+0000", 2024-09-04T17:48:42.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2140000000000001e-06 2024-09-04T17:48:42.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.125 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:42.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822737+0000", 2024-09-04T17:48:42.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4370000000000007e-06 2024-09-04T17:48:42.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.126 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:42.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822743+0000", 2024-09-04T17:48:42.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7520000000000002e-06 2024-09-04T17:48:42.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.126 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:42.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889785+0000", 2024-09-04T17:48:42.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067042265000000004 2024-09-04T17:48:42.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:42.127 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889840+0000", 2024-09-04T17:48:42.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5254000000000003e-05 2024-09-04T17:48:42.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:42.127 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.890154+0000", 2024-09-04T17:48:42.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000313646 2024-09-04T17:48:42.127 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:42.127 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:42.127 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:42.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.127 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61084 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 2867200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:42.128 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.822753+0000", 2024-09-04T17:48:42.128 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 88.261039847000006, 2024-09-04T17:48:42.128 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068497161000000001, 2024-09-04T17:48:42.128 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:42.128 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:42.128 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:42.128 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:42.128 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:42.128 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:42.128 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61084 2024-09-04T17:48:42.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.128 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:42.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.129 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:42.129 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822753+0000", 2024-09-04T17:48:42.129 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:42.129 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.129 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:42.129 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822753+0000", 2024-09-04T17:48:42.129 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:42.129 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.129 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:42.129 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822753+0000", 2024-09-04T17:48:42.130 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9100000000000003e-07 2024-09-04T17:48:42.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.130 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.130 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:42.130 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822756+0000", 2024-09-04T17:48:42.130 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.678e-06 2024-09-04T17:48:42.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.130 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.130 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:42.130 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822763+0000", 2024-09-04T17:48:42.130 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8220000000000001e-06 2024-09-04T17:48:42.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.130 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.130 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:42.130 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822768+0000", 2024-09-04T17:48:42.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1930000000000001e-06 2024-09-04T17:48:42.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:42.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.890246+0000", 2024-09-04T17:48:42.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067477996999999998 2024-09-04T17:48:42.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:42.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.890555+0000", 2024-09-04T17:48:42.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000309143 2024-09-04T17:48:42.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.132 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:42.132 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.891250+0000", 2024-09-04T17:48:42.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00069463700000000001 2024-09-04T17:48:42.132 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:42.132 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:42.132 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:42.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.132 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61285 3.8 3:117cfeb9:::rbd_data.11341dfde0c5.0000000000000006:1f [read 2760704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:42.132 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835200+0000", 2024-09-04T17:48:42.132 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 88.248592779999996, 2024-09-04T17:48:42.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066899803999999993, 2024-09-04T17:48:42.132 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:42.133 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:42.133 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:42.133 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:42.133 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:42.133 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:42.133 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61285 2024-09-04T17:48:42.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.133 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:42.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.133 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:42.133 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835200+0000", 2024-09-04T17:48:42.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:42.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.134 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:42.134 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835200+0000", 2024-09-04T17:48:42.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:42.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.134 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:42.134 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835200+0000", 2024-09-04T17:48:42.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0899999999999997e-07 2024-09-04T17:48:42.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.134 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:42.134 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835206+0000", 2024-09-04T17:48:42.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3280000000000002e-06 2024-09-04T17:48:42.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.135 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:42.135 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835211+0000", 2024-09-04T17:48:42.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9230000000000001e-06 2024-09-04T17:48:42.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.135 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:42.135 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835216+0000", 2024-09-04T17:48:42.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9019999999999998e-06 2024-09-04T17:48:42.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.135 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:42.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.901719+0000", 2024-09-04T17:48:42.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066502981000000003 2024-09-04T17:48:42.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:42.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.901813+0000", 2024-09-04T17:48:42.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3905000000000003e-05 2024-09-04T17:48:42.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:42.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902100+0000", 2024-09-04T17:48:42.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028645599999999998 2024-09-04T17:48:42.136 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:42.137 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:42.137 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:42.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.137 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61286 3.8 3:117cfeb9:::rbd_data.11341dfde0c5.0000000000000006:1f [read 2867200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:42.137 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835244+0000", 2024-09-04T17:48:42.137 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 88.248548161000002, 2024-09-04T17:48:42.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067143215000000006, 2024-09-04T17:48:42.137 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:42.137 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:42.137 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:42.137 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:42.137 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:42.137 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:42.138 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61286 2024-09-04T17:48:42.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.138 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:42.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:42.138 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835244+0000", 2024-09-04T17:48:42.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:42.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:42.138 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835244+0000", 2024-09-04T17:48:42.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:42.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.139 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:42.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835245+0000", 2024-09-04T17:48:42.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8400000000000004e-07 2024-09-04T17:48:42.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.139 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:42.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835248+0000", 2024-09-04T17:48:42.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6850000000000001e-06 2024-09-04T17:48:42.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.140 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:42.140 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835254+0000", 2024-09-04T17:48:42.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.494e-06 2024-09-04T17:48:42.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.140 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:42.140 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835260+0000", 2024-09-04T17:48:42.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0580000000000003e-06 2024-09-04T17:48:42.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.140 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:42.140 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902119+0000", 2024-09-04T17:48:42.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066858318 2024-09-04T17:48:42.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:42.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902163+0000", 2024-09-04T17:48:42.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4753000000000003e-05 2024-09-04T17:48:42.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:42.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902388+0000", 2024-09-04T17:48:42.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022422300000000001 2024-09-04T17:48:42.141 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:42.141 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:42.141 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:42.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.142 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61315 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 98304~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:42.142 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835269+0000", 2024-09-04T17:48:42.142 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 88.248523368999997, 2024-09-04T17:48:42.142 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067329396, 2024-09-04T17:48:42.142 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:42.142 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:42.142 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:42.142 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:42.142 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:42.142 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:42.142 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61315 2024-09-04T17:48:42.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.143 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:42.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:42.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835269+0000", 2024-09-04T17:48:42.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:42.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:42.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835269+0000", 2024-09-04T17:48:42.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:42.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:42.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835270+0000", 2024-09-04T17:48:42.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6599999999999996e-07 2024-09-04T17:48:42.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:42.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835272+0000", 2024-09-04T17:48:42.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.261e-06 2024-09-04T17:48:42.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:42.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835278+0000", 2024-09-04T17:48:42.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.443e-06 2024-09-04T17:48:42.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:42.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835284+0000", 2024-09-04T17:48:42.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3290000000000001e-06 2024-09-04T17:48:42.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:42.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902147+0000", 2024-09-04T17:48:42.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066862695999999999 2024-09-04T17:48:42.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:42.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902248+0000", 2024-09-04T17:48:42.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000101501 2024-09-04T17:48:42.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.146 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:42.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902599+0000", 2024-09-04T17:48:42.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00035060000000000001 2024-09-04T17:48:42.146 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:42.146 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:42.146 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:42.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.146 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61316 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 204800~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:42.146 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835291+0000", 2024-09-04T17:48:42.146 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 88.248501349999998, 2024-09-04T17:48:42.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067632052999999998, 2024-09-04T17:48:42.147 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:42.147 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:42.147 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:42.147 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:42.147 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:42.147 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:42.147 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61316 2024-09-04T17:48:42.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.147 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:42.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.147 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:42.147 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835291+0000", 2024-09-04T17:48:42.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:42.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.148 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:42.148 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835291+0000", 2024-09-04T17:48:42.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:42.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.148 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:42.148 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835292+0000", 2024-09-04T17:48:42.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1899999999999998e-07 2024-09-04T17:48:42.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:42.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835294+0000", 2024-09-04T17:48:42.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0880000000000002e-06 2024-09-04T17:48:42.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:42.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835300+0000", 2024-09-04T17:48:42.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.224e-06 2024-09-04T17:48:42.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:42.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835305+0000", 2024-09-04T17:48:42.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0730000000000004e-06 2024-09-04T17:48:42.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:42.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902632+0000", 2024-09-04T17:48:42.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067327322999999994 2024-09-04T17:48:42.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:42.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902688+0000", 2024-09-04T17:48:42.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5177999999999999e-05 2024-09-04T17:48:42.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:42.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902923+0000", 2024-09-04T17:48:42.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023574800000000001 2024-09-04T17:48:42.151 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:42.151 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:42.151 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:42.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.151 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61317 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 344064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:42.151 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835312+0000", 2024-09-04T17:48:42.151 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 88.248480345999994, 2024-09-04T17:48:42.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067918511000000001, 2024-09-04T17:48:42.151 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:42.151 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:42.151 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:42.151 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:42.152 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:42.152 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:42.152 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61317 2024-09-04T17:48:42.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.152 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:42.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.152 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:42.152 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835312+0000", 2024-09-04T17:48:42.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:42.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.152 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:42.152 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835312+0000", 2024-09-04T17:48:42.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:42.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:42.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835313+0000", 2024-09-04T17:48:42.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9000000000000002e-07 2024-09-04T17:48:42.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:42.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835315+0000", 2024-09-04T17:48:42.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1689999999999999e-06 2024-09-04T17:48:42.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:42.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835321+0000", 2024-09-04T17:48:42.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0619999999999997e-06 2024-09-04T17:48:42.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:42.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835326+0000", 2024-09-04T17:48:42.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0390000000000004e-06 2024-09-04T17:48:42.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:42.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902967+0000", 2024-09-04T17:48:42.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067641520999999996 2024-09-04T17:48:42.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:42.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903067+0000", 2024-09-04T17:48:42.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9426999999999995e-05 2024-09-04T17:48:42.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:42.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903231+0000", 2024-09-04T17:48:42.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016390299999999999 2024-09-04T17:48:42.155 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:42.155 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:42.155 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:42.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.155 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61318 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 450560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:42.156 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835334+0000", 2024-09-04T17:48:42.156 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 88.248458829, 2024-09-04T17:48:42.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068091771999999995, 2024-09-04T17:48:42.156 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:42.156 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:42.156 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:42.156 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:42.156 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:42.156 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:42.156 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61318 2024-09-04T17:48:42.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.156 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:42.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:42.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835334+0000", 2024-09-04T17:48:42.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:42.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:42.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835334+0000", 2024-09-04T17:48:42.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:42.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:42.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835334+0000", 2024-09-04T17:48:42.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5699999999999998e-07 2024-09-04T17:48:42.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:42.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835337+0000", 2024-09-04T17:48:42.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3889999999999999e-06 2024-09-04T17:48:42.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:42.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835343+0000", 2024-09-04T17:48:42.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1809999999999998e-06 2024-09-04T17:48:42.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:42.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835348+0000", 2024-09-04T17:48:42.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3170000000000001e-06 2024-09-04T17:48:42.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.159 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:42.159 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903270+0000", 2024-09-04T17:48:42.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067922191000000007 2024-09-04T17:48:42.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.159 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:42.159 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903321+0000", 2024-09-04T17:48:42.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0889e-05 2024-09-04T17:48:42.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.160 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:42.160 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903426+0000", 2024-09-04T17:48:42.160 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104348 2024-09-04T17:48:42.160 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:42.160 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:42.160 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:42.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.160 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61319 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 589824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:42.160 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835357+0000", 2024-09-04T17:48:42.160 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 88.248436011999999, 2024-09-04T17:48:42.160 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068372535999999998, 2024-09-04T17:48:42.160 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:42.160 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:42.161 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:42.161 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:42.161 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:42.161 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:42.161 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61319 2024-09-04T17:48:42.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.161 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:42.161 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.161 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:42.161 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835357+0000", 2024-09-04T17:48:42.161 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:42.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.161 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:42.162 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835357+0000", 2024-09-04T17:48:42.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:42.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:42.162 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835357+0000", 2024-09-04T17:48:42.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.32e-07 2024-09-04T17:48:42.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:42.162 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835359+0000", 2024-09-04T17:48:42.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1660000000000001e-06 2024-09-04T17:48:42.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.163 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.163 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:42.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835366+0000", 2024-09-04T17:48:42.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3550000000000001e-06 2024-09-04T17:48:42.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.165 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:42.165 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835371+0000", 2024-09-04T17:48:42.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1050000000000003e-06 2024-09-04T17:48:42.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.165 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:42.165 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903471+0000", 2024-09-04T17:48:42.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068100451000000006 2024-09-04T17:48:42.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.165 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:42.166 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903573+0000", 2024-09-04T17:48:42.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000101691 2024-09-04T17:48:42.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.166 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:42.166 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903729+0000", 2024-09-04T17:48:42.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000156336 2024-09-04T17:48:42.166 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:42.166 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:42.166 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:42.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.166 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61320 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:42.167 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835380+0000", 2024-09-04T17:48:42.167 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 88.248412247000005, 2024-09-04T17:48:42.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068543058000000004, 2024-09-04T17:48:42.167 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:42.167 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:42.167 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:42.167 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:42.167 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:42.167 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:42.167 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61320 2024-09-04T17:48:42.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.167 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:42.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:42.168 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835380+0000", 2024-09-04T17:48:42.168 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:42.168 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.168 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.168 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:42.168 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835380+0000", 2024-09-04T17:48:42.168 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:42.168 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.168 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.168 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:42.168 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835381+0000", 2024-09-04T17:48:42.168 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9999999999999998e-07 2024-09-04T17:48:42.168 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.169 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:42.169 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835383+0000", 2024-09-04T17:48:42.169 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2230000000000001e-06 2024-09-04T17:48:42.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.169 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:42.169 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835390+0000", 2024-09-04T17:48:42.169 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.34e-06 2024-09-04T17:48:42.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.169 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:42.169 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835396+0000", 2024-09-04T17:48:42.169 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9830000000000001e-06 2024-09-04T17:48:42.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.170 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:42.170 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903765+0000", 2024-09-04T17:48:42.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068368441000000002 2024-09-04T17:48:42.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.170 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:42.170 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903835+0000", 2024-09-04T17:48:42.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0631999999999997e-05 2024-09-04T17:48:42.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.170 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:42.170 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903923+0000", 2024-09-04T17:48:42.171 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8039000000000002e-05 2024-09-04T17:48:42.171 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:42.171 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:42.171 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:42.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.171 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61321 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 835584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:42.171 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835404+0000", 2024-09-04T17:48:42.171 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 88.248388188000007, 2024-09-04T17:48:42.171 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068911043000000005, 2024-09-04T17:48:42.171 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:42.171 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:42.171 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:42.172 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:42.172 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:42.172 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:42.172 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61321 2024-09-04T17:48:42.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.172 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:42.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.172 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:42.172 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835404+0000", 2024-09-04T17:48:42.172 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:42.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.173 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.173 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:42.173 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835404+0000", 2024-09-04T17:48:42.173 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:42.173 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.173 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.173 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:42.173 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835405+0000", 2024-09-04T17:48:42.173 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.89e-07 2024-09-04T17:48:42.173 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.173 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.173 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:42.174 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835407+0000", 2024-09-04T17:48:42.174 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4439999999999998e-06 2024-09-04T17:48:42.174 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.174 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.174 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:42.174 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835414+0000", 2024-09-04T17:48:42.174 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4930000000000003e-06 2024-09-04T17:48:42.174 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.174 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.174 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:42.174 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835419+0000", 2024-09-04T17:48:42.174 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8919999999999999e-06 2024-09-04T17:48:42.174 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.175 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:42.175 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903957+0000", 2024-09-04T17:48:42.175 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068538500000000002 2024-09-04T17:48:42.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.175 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:42.175 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904019+0000", 2024-09-04T17:48:42.175 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1419000000000003e-05 2024-09-04T17:48:42.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.175 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:42.175 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904315+0000", 2024-09-04T17:48:42.175 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00029680599999999999 2024-09-04T17:48:42.176 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:42.176 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:42.176 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:42.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.176 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61322 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:42.176 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835427+0000", 2024-09-04T17:48:42.176 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 88.248365801999995, 2024-09-04T17:48:42.176 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.069550138999999997, 2024-09-04T17:48:42.176 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:42.176 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:42.176 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:42.176 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:42.177 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:42.177 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:42.177 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61322 2024-09-04T17:48:42.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.177 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:42.177 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.177 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:42.177 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835427+0000", 2024-09-04T17:48:42.177 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:42.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.177 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.177 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:42.177 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835427+0000", 2024-09-04T17:48:42.178 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:42.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.178 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.178 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:42.178 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.842157+0000", 2024-09-04T17:48:42.178 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067298419999999998 2024-09-04T17:48:42.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.178 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.178 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:42.178 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.842163+0000", 2024-09-04T17:48:42.178 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2029999999999997e-06 2024-09-04T17:48:42.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.178 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.178 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:42.179 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.842175+0000", 2024-09-04T17:48:42.179 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2072000000000001e-05 2024-09-04T17:48:42.179 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.179 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.179 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:42.179 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.842183+0000", 2024-09-04T17:48:42.179 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3850000000000002e-06 2024-09-04T17:48:42.179 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.179 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.179 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:42.179 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904371+0000", 2024-09-04T17:48:42.179 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.062187754999999997 2024-09-04T17:48:42.179 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.180 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.180 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:42.180 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904467+0000", 2024-09-04T17:48:42.180 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6117000000000001e-05 2024-09-04T17:48:42.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:42.180 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:42.180 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:42.180 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904977+0000", 2024-09-04T17:48:42.180 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00050976500000000002 2024-09-04T17:48:42.180 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:42.180 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:42.180 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:42.180 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:42.180 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:42.181 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:42.182 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-04T17:48:42.182 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:42.181+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:42.283 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:48:42.279+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:48:42.317 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:42.317 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:48:42.317 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:48:42.317 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:42.332 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-04T17:48:42.336 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-04T17:48:42.384 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:48:42.379+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:48:42.457 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:42.457 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:48:42.457 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:48:42.457 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:48:42.457 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:42.472 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-04T17:48:42.494 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:48:42.494 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:48:42.511 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:48:42.511 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:48:42.517 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:48:42.517+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:48:42.527 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-04T17:48:42.527 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-04T17:48:42.544 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-04T17:48:42.545 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-04T17:48:42.564 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:48:42.564 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:48:42.585 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:48:42.585 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:48:42.586 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:42.585+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:42.595 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:42.596 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:42.606 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-04T17:48:42.606 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-04T17:48:42.610 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-04T17:48:42.627 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:48:42.627 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:48:42.701 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:42.701+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:42.773 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:42.773 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:42.787 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-04T17:48:42.829 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:42.829+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:42.889 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:42.887+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:42.941 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-04T17:48:42.968 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:42.969 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:42.985 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-04T17:48:43.029 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:43.029+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:43.091 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:48:43.087+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:48:43.108 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:43.109 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:43.123 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-04T17:48:43.209 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:48:43.209+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:48:43.289 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:43.289 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:43.293 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:48:43.291+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:48:43.303 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-04T17:48:43.394 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:43.391+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:43.473 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:43.473 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:43.487 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-04T17:48:43.533 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:43.533+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:43.597 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:43.591+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:43.604 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:43.604 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:43.618 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-04T17:48:43.698 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:43.695+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:43.748 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:43.748 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:43.762 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-04T17:48:43.799 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:48:43.797+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:48:43.900 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:43.895+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:43.926 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:43.927 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:43.943 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is set 2024-09-04T17:48:43.945 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-04T17:48:44.006 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:44.003+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:44.102 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:44.099+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:44.128 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:44.129 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:44.144 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-04T17:48:44.229 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:44.227+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:44.303 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:48:44.301+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:48:44.305 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:44.305 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:44.319 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-04T17:48:44.403 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:44.401+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:44.486 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:44.486 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:44.500 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-04T17:48:44.505 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:48:44.505+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:48:44.606 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:44.605+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:44.624 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:44.624 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:44.638 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-04T17:48:44.644 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-04T17:48:44.706 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:44.705+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:44.771 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:44.771 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:44.785 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-04T17:48:44.788 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:48:44.788 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:48:44.805 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:48:44.805 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:48:44.807 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:48:44.803+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:48:44.821 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-04T17:48:44.821 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-04T17:48:44.838 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-04T17:48:44.838 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-04T17:48:44.858 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:48:44.858 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:48:44.876 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:48:44.876 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:48:44.894 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-04T17:48:44.894 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-04T17:48:44.909 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:44.903+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:44.914 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:48:44.914 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:48:44.958 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:44.959 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:48:44.959 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:48:44.959 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:48:44.959 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:44.974 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-04T17:48:45.010 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:45.007+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:45.111 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:45.109+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:45.119 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:45.119 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:48:45.119 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:48:45.119 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:48:45.119 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.119 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "pg_update_log_missing(3.1f epoch 119/118 rep_tid 2838 entries 119'424 (0'0) error 3:f95bc674:::rbd_header.1134878eadad:head by client.4404.0:93468 0.000000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-04T17:48:45.119 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:58.898727+0000", 2024-09-04T17:48:45.120 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 46.219063783999999, 2024-09-04T17:48:45.120 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0072113999999999998, 2024-09-04T17:48:45.120 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:45.120 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:45.120 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:45.120 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:45.120 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.120 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:45.120 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.898727+0000", 2024-09-04T17:48:45.120 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:45.120 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.120 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.120 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:45.121 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.898727+0000", 2024-09-04T17:48:45.121 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:45.121 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.121 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.121 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:45.121 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.898728+0000", 2024-09-04T17:48:45.121 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.8200000000000001e-07 2024-09-04T17:48:45.121 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.121 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.121 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:45.121 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.898740+0000", 2024-09-04T17:48:45.121 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2186e-05 2024-09-04T17:48:45.121 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.122 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.122 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:45.122 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.898772+0000", 2024-09-04T17:48:45.122 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.2156000000000002e-05 2024-09-04T17:48:45.122 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.122 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.122 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:45.122 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.898796+0000", 2024-09-04T17:48:45.122 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4017999999999999e-05 2024-09-04T17:48:45.122 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.122 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.122 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:45.123 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.898966+0000", 2024-09-04T17:48:45.123 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000170185 2024-09-04T17:48:45.123 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.123 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.123 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:45.123 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.905939+0000", 2024-09-04T17:48:45.123 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0069724729999999999 2024-09-04T17:48:45.123 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:45.123 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:45.123 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:45.123 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.123 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.124 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:93469 3.1f e119/118 3:f95bc674:::rbd_header.1134878eadad:head v 119'425, mlcod=0'0)", 2024-09-04T17:48:45.124 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:58.900173+0000", 2024-09-04T17:48:45.124 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 46.217617996000001, 2024-09-04T17:48:45.124 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0059053930000000001, 2024-09-04T17:48:45.124 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:45.124 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:45.124 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:45.124 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:45.124 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.124 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:45.124 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.900173+0000", 2024-09-04T17:48:45.124 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:45.124 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.125 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.125 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:45.125 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.900173+0000", 2024-09-04T17:48:45.125 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:45.125 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.125 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.125 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:45.125 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.900174+0000", 2024-09-04T17:48:45.125 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.127e-06 2024-09-04T17:48:45.125 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.125 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.125 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:45.125 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.900181+0000", 2024-09-04T17:48:45.126 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.0099999999999998e-06 2024-09-04T17:48:45.126 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.126 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.126 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:45.126 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.900199+0000", 2024-09-04T17:48:45.126 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7876e-05 2024-09-04T17:48:45.126 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.126 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.126 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:45.126 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.900208+0000", 2024-09-04T17:48:45.126 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.4999999999999999e-06 2024-09-04T17:48:45.126 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.126 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.127 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:45.127 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.900310+0000", 2024-09-04T17:48:45.127 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00010202800000000001 2024-09-04T17:48:45.127 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.127 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.127 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:45.127 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.900353+0000", 2024-09-04T17:48:45.127 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.3213000000000001e-05 2024-09-04T17:48:45.127 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.127 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.127 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:45.127 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.905944+0000", 2024-09-04T17:48:45.127 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0055914900000000002 2024-09-04T17:48:45.128 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.128 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.128 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:45.128 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.906078+0000", 2024-09-04T17:48:45.128 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013414899999999999 2024-09-04T17:48:45.128 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:45.128 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:45.128 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:45.128 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.128 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.128 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "MOSDPGPush(3.10 119/118 [PushOp(3:0c2d06e5:::rbd_data.113410ddd206.0000000000000000:18, version: 68'308, data_included: [0~884736,950272~65536,1081344~65536,1212416~32768], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:0c2d06e5:::rbd_data.113410ddd206.0000000000000000:18@68'308, size: 7471104, copy_subset: [(0, 7471104)], clone_subset: {}, snapset: 18=[]:{18: [18]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1245184, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-04T17:48:45.128 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:59.426517+0000", 2024-09-04T17:48:45.129 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 45.691273918, 2024-09-04T17:48:45.129 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0064985670000000002, 2024-09-04T17:48:45.129 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:45.129 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:45.129 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:45.129 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:45.129 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.129 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:45.129 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.426517+0000", 2024-09-04T17:48:45.129 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:45.129 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.129 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.129 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:45.129 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.426517+0000", 2024-09-04T17:48:45.130 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:45.130 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.130 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.130 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:45.130 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.426754+0000", 2024-09-04T17:48:45.130 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00023681799999999999 2024-09-04T17:48:45.130 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.130 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.130 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:45.130 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.429886+0000", 2024-09-04T17:48:45.130 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0031323789999999998 2024-09-04T17:48:45.130 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.130 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.131 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:45.131 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.429940+0000", 2024-09-04T17:48:45.131 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4104999999999998e-05 2024-09-04T17:48:45.131 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.131 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.131 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:45.131 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.429975+0000", 2024-09-04T17:48:45.131 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4289000000000003e-05 2024-09-04T17:48:45.131 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.131 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.132 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:45.132 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.430229+0000", 2024-09-04T17:48:45.132 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00025465900000000002 2024-09-04T17:48:45.132 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.132 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.132 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:45.132 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.430350+0000", 2024-09-04T17:48:45.132 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000120992 2024-09-04T17:48:45.132 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.132 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.132 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:45.132 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.433016+0000", 2024-09-04T17:48:45.133 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0026653250000000001 2024-09-04T17:48:45.133 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:45.133 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:45.133 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:45.133 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.133 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.133 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "MOSDPGPush(3.10 119/118 [PushOp(3:0c2d06e5:::rbd_data.113410ddd206.0000000000000000:18, version: 68'308, data_included: [1245184~32768,1343488~65536,1474560~65536,1605632~65536,1736704~65536,1867776~65536,1998848~65536,2129920~65536,2260992~65536,2392064~65536,2523136~65536,2654208~65536,2785280~65536,2916352~65536,3047424~65536,3154405~89627,3309568~8677], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:0c2d06e5:::rbd_data.113410ddd206.0000000000000000:18@68'308, size: 7471104, copy_subset: [(0, 7471104)], clone_subset: {}, snapset: 18=[]:{18: [18]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3318245, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1245184, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-04T17:48:45.133 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:59.434957+0000", 2024-09-04T17:48:45.133 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 45.682834294999999, 2024-09-04T17:48:45.133 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0061603969999999997, 2024-09-04T17:48:45.133 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:45.133 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:45.134 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:45.134 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:45.134 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.134 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:45.134 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.434957+0000", 2024-09-04T17:48:45.134 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:45.134 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.134 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.134 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:45.134 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.434957+0000", 2024-09-04T17:48:45.134 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:45.134 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.134 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.134 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:45.135 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.434958+0000", 2024-09-04T17:48:45.135 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4339999999999999e-06 2024-09-04T17:48:45.135 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.135 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.135 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:45.135 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.436921+0000", 2024-09-04T17:48:45.135 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0019629460000000001 2024-09-04T17:48:45.135 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.135 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.135 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:45.135 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.436973+0000", 2024-09-04T17:48:45.135 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1492000000000001e-05 2024-09-04T17:48:45.135 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.136 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.136 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:45.136 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.437041+0000", 2024-09-04T17:48:45.136 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.8275000000000006e-05 2024-09-04T17:48:45.136 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.136 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.136 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:45.136 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.437312+0000", 2024-09-04T17:48:45.136 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00027156800000000003 2024-09-04T17:48:45.136 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.136 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.136 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:45.136 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.437357+0000", 2024-09-04T17:48:45.137 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4079000000000003e-05 2024-09-04T17:48:45.137 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.137 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.137 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:45.137 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:59.441117+0000", 2024-09-04T17:48:45.137 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0037606029999999999 2024-09-04T17:48:45.137 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:45.137 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:45.137 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:45.137 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.137 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.137 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:96178 3.1e 3:7c9d4526:::rbd_data.1134c4e2d26.0000000000000026:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123)", 2024-09-04T17:48:45.137 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:02.985763+0000", 2024-09-04T17:48:45.138 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 42.132028147, 2024-09-04T17:48:45.138 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.020686758999999999, 2024-09-04T17:48:45.138 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:45.138 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:45.138 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:45.138 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:45.138 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:45.138 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:45.138 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 96178 2024-09-04T17:48:45.138 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.138 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:45.138 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.138 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:45.138 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:02.985763+0000", 2024-09-04T17:48:45.139 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:45.139 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.139 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.139 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:45.139 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:02.985763+0000", 2024-09-04T17:48:45.139 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:45.139 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.139 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.139 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:45.139 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:02.985766+0000", 2024-09-04T17:48:45.139 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.2990000000000001e-06 2024-09-04T17:48:45.139 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.139 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.140 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:45.140 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:02.985783+0000", 2024-09-04T17:48:45.140 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.645e-05 2024-09-04T17:48:45.140 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.140 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.140 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:45.140 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:02.985818+0000", 2024-09-04T17:48:45.140 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.5459000000000001e-05 2024-09-04T17:48:45.140 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.140 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.140 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:45.140 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:02.985835+0000", 2024-09-04T17:48:45.140 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.715e-05 2024-09-04T17:48:45.141 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.141 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.141 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:45.141 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:02.985953+0000", 2024-09-04T17:48:45.141 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000117351 2024-09-04T17:48:45.141 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.141 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.141 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:45.141 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:02.986207+0000", 2024-09-04T17:48:45.141 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00025483 2024-09-04T17:48:45.141 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.141 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.141 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 2", 2024-09-04T17:48:45.142 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:02.990784+0000", 2024-09-04T17:48:45.142 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0045763219999999999 2024-09-04T17:48:45.142 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.142 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.142 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:48:45.142 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.006139+0000", 2024-09-04T17:48:45.142 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.015355526 2024-09-04T17:48:45.142 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.142 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.142 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:45.142 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.006225+0000", 2024-09-04T17:48:45.142 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.6003999999999999e-05 2024-09-04T17:48:45.142 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.143 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.143 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:45.143 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.006371+0000", 2024-09-04T17:48:45.143 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000146052 2024-09-04T17:48:45.143 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.143 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.143 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:45.143 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.006450+0000", 2024-09-04T17:48:45.143 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.8316e-05 2024-09-04T17:48:45.143 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:45.143 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:45.143 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:45.143 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.143 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.144 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:96267 3.1e 3:7a1dfe29:::rbd_header.1134c4e2d26:head [watch unwatch cookie 140018753450144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123)", 2024-09-04T17:48:45.144 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:03.482624+0000", 2024-09-04T17:48:45.144 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 41.635166845000001, 2024-09-04T17:48:45.144 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.006866853, 2024-09-04T17:48:45.144 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:45.144 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:45.144 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:45.144 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:45.144 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:45.144 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:45.144 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 96267 2024-09-04T17:48:45.144 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.145 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:45.145 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.145 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:45.145 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.482624+0000", 2024-09-04T17:48:45.145 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:45.145 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.145 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.145 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:45.145 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.482624+0000", 2024-09-04T17:48:45.145 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:45.145 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.145 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.145 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:45.145 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.482626+0000", 2024-09-04T17:48:45.146 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8360000000000001e-06 2024-09-04T17:48:45.146 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.146 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.146 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:45.146 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.482635+0000", 2024-09-04T17:48:45.146 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9099999999999994e-06 2024-09-04T17:48:45.146 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.146 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.146 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:45.146 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.482658+0000", 2024-09-04T17:48:45.146 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.3186000000000001e-05 2024-09-04T17:48:45.146 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.147 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.147 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:45.147 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.482677+0000", 2024-09-04T17:48:45.147 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8396000000000001e-05 2024-09-04T17:48:45.147 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.147 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.147 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:45.147 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.482849+0000", 2024-09-04T17:48:45.147 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000172936 2024-09-04T17:48:45.147 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.147 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.147 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:45.147 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.482987+0000", 2024-09-04T17:48:45.147 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013707699999999999 2024-09-04T17:48:45.148 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.148 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.148 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 2", 2024-09-04T17:48:45.148 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.483222+0000", 2024-09-04T17:48:45.148 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00023545 2024-09-04T17:48:45.148 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.148 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.148 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:48:45.148 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.484356+0000", 2024-09-04T17:48:45.148 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.001134049 2024-09-04T17:48:45.148 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.148 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.148 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:45.149 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.489303+0000", 2024-09-04T17:48:45.149 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0049472910000000004 2024-09-04T17:48:45.149 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.149 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.149 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:45.149 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.489396+0000", 2024-09-04T17:48:45.149 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.2638000000000003e-05 2024-09-04T17:48:45.149 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.149 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.149 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:45.149 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.489491+0000", 2024-09-04T17:48:45.149 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5084e-05 2024-09-04T17:48:45.149 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:45.150 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:45.150 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:45.150 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.150 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.150 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:96360 3.1e 3:7c9d4526:::rbd_data.1134c4e2d26.0000000000000026:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123)", 2024-09-04T17:48:45.150 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:03.603245+0000", 2024-09-04T17:48:45.150 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 41.514545740999999, 2024-09-04T17:48:45.150 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0074703160000000003, 2024-09-04T17:48:45.150 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:45.150 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:45.150 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:45.150 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:45.150 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:45.151 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:45.151 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 96360 2024-09-04T17:48:45.151 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.151 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:45.151 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.151 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:45.151 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.603245+0000", 2024-09-04T17:48:45.151 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:45.151 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.151 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.151 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:45.151 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.603245+0000", 2024-09-04T17:48:45.151 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:45.152 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.152 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.152 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:45.152 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.603248+0000", 2024-09-04T17:48:45.152 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.277e-06 2024-09-04T17:48:45.152 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.152 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.152 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:45.152 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.603259+0000", 2024-09-04T17:48:45.152 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1083e-05 2024-09-04T17:48:45.152 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.152 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.153 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:45.153 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.603295+0000", 2024-09-04T17:48:45.153 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.5892999999999997e-05 2024-09-04T17:48:45.153 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.153 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.153 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:45.153 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.603315+0000", 2024-09-04T17:48:45.153 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0953000000000001e-05 2024-09-04T17:48:45.153 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.153 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.153 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:45.153 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.603486+0000", 2024-09-04T17:48:45.153 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00017011199999999999 2024-09-04T17:48:45.154 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.154 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.154 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:45.154 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.604960+0000", 2024-09-04T17:48:45.154 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0014743340000000001 2024-09-04T17:48:45.154 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.154 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.154 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 2", 2024-09-04T17:48:45.154 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.605232+0000", 2024-09-04T17:48:45.154 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00027236300000000002 2024-09-04T17:48:45.154 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.154 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.154 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:48:45.155 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.610322+0000", 2024-09-04T17:48:45.155 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0050901849999999997 2024-09-04T17:48:45.155 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.155 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.155 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:45.155 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.610464+0000", 2024-09-04T17:48:45.155 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014186599999999999 2024-09-04T17:48:45.155 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.155 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.155 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:45.155 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.610607+0000", 2024-09-04T17:48:45.155 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000142423 2024-09-04T17:48:45.155 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.155 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.156 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:45.156 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:03.610716+0000", 2024-09-04T17:48:45.156 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00010882699999999999 2024-09-04T17:48:45.156 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:45.156 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:45.156 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:45.156 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.156 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.156 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:96520 3.2 e124/121 3:4549cdb4:::rbd_data.1134878eadad.0000000000000001:head v 124'509, mlcod=123'507)", 2024-09-04T17:48:45.156 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:04.014285+0000", 2024-09-04T17:48:45.156 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 41.103505822000002, 2024-09-04T17:48:45.156 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0066183220000000003, 2024-09-04T17:48:45.156 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:45.157 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:45.157 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:45.157 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:45.157 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.157 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:45.157 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:04.014285+0000", 2024-09-04T17:48:45.157 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:45.157 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.157 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.157 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:45.157 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:04.014285+0000", 2024-09-04T17:48:45.157 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:45.157 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.158 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.158 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:45.158 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:04.014287+0000", 2024-09-04T17:48:45.158 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1959999999999998e-06 2024-09-04T17:48:45.158 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.158 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.158 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:45.158 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:04.014332+0000", 2024-09-04T17:48:45.158 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4196000000000002e-05 2024-09-04T17:48:45.158 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.158 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.158 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:45.158 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:04.014374+0000", 2024-09-04T17:48:45.159 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2072e-05 2024-09-04T17:48:45.159 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.159 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.159 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:45.159 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:04.014394+0000", 2024-09-04T17:48:45.159 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0146999999999999e-05 2024-09-04T17:48:45.159 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.159 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.159 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:45.159 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:04.014575+0000", 2024-09-04T17:48:45.159 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000181132 2024-09-04T17:48:45.159 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.159 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.159 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:45.160 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:04.014611+0000", 2024-09-04T17:48:45.160 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.5815999999999997e-05 2024-09-04T17:48:45.160 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.160 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.160 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:45.160 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:04.020809+0000", 2024-09-04T17:48:45.160 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0061983159999999997 2024-09-04T17:48:45.160 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.160 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.160 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:45.160 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:04.020904+0000", 2024-09-04T17:48:45.160 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4446999999999999e-05 2024-09-04T17:48:45.160 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:45.161 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:45.161 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:45.161 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.161 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.161 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(osd.2.0:369 3.0 e138/121 3:04fb2888:::rbd_data.11342e6808f0.0000000000000009:24 v 138'380, mlcod=135'377)", 2024-09-04T17:48:45.161 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:17.851779+0000", 2024-09-04T17:48:45.161 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 27.26601191, 2024-09-04T17:48:45.161 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0056329400000000003, 2024-09-04T17:48:45.161 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:45.161 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:45.161 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:45.161 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:45.161 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.162 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:45.162 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.851779+0000", 2024-09-04T17:48:45.162 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:45.162 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.162 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.162 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:45.162 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.851779+0000", 2024-09-04T17:48:45.162 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:45.162 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.162 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.162 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:45.162 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.851780+0000", 2024-09-04T17:48:45.162 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2399999999999998e-07 2024-09-04T17:48:45.163 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.163 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.163 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:45.163 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.851790+0000", 2024-09-04T17:48:45.163 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0399e-05 2024-09-04T17:48:45.163 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.163 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.163 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:45.163 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.851809+0000", 2024-09-04T17:48:45.163 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9228e-05 2024-09-04T17:48:45.163 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.163 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.163 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:45.164 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.851820+0000", 2024-09-04T17:48:45.164 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0814e-05 2024-09-04T17:48:45.164 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.164 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.164 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:45.164 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.851902+0000", 2024-09-04T17:48:45.164 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.2154e-05 2024-09-04T17:48:45.164 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.164 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.164 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:45.164 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.851930+0000", 2024-09-04T17:48:45.164 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.7577999999999999e-05 2024-09-04T17:48:45.164 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.165 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.165 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:45.165 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.857363+0000", 2024-09-04T17:48:45.165 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0054331989999999997 2024-09-04T17:48:45.165 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.165 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.165 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:45.165 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.857412+0000", 2024-09-04T17:48:45.165 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9044e-05 2024-09-04T17:48:45.165 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:45.165 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:45.165 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:45.165 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.166 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.166 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(osd.5.0:3167 3.2 e138/121 3:460b9dee:::rbd_data.11342e6808f0.000000000000000d:24 v 138'560, mlcod=135'557)", 2024-09-04T17:48:45.166 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:17.857279+0000", 2024-09-04T17:48:45.166 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 27.260512326000001, 2024-09-04T17:48:45.166 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0068482229999999996, 2024-09-04T17:48:45.166 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:45.166 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:45.166 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:45.166 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:45.166 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.166 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:45.166 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.857279+0000", 2024-09-04T17:48:45.166 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:45.167 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.167 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.167 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:45.167 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.857279+0000", 2024-09-04T17:48:45.167 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:45.167 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.167 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.167 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:45.167 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.857279+0000", 2024-09-04T17:48:45.167 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7300000000000001e-07 2024-09-04T17:48:45.167 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.167 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.167 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:45.168 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.857286+0000", 2024-09-04T17:48:45.168 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.4289999999999998e-06 2024-09-04T17:48:45.168 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.168 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.168 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:45.168 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.857300+0000", 2024-09-04T17:48:45.168 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4582e-05 2024-09-04T17:48:45.168 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.168 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.168 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:45.168 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.857315+0000", 2024-09-04T17:48:45.168 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5173e-05 2024-09-04T17:48:45.168 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.168 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.169 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:45.169 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.857388+0000", 2024-09-04T17:48:45.169 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2365000000000003e-05 2024-09-04T17:48:45.169 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.169 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.169 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:45.169 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.857414+0000", 2024-09-04T17:48:45.169 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6248999999999999e-05 2024-09-04T17:48:45.169 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.169 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.169 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:45.169 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.864076+0000", 2024-09-04T17:48:45.169 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0066619490000000003 2024-09-04T17:48:45.170 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.170 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.170 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:45.170 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:17.864127+0000", 2024-09-04T17:48:45.170 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1002999999999999e-05 2024-09-04T17:48:45.170 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:45.170 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:45.170 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:45.170 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.170 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.170 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:98345 3.a 3:55a1c81d:::rbd_header.113481c24fbf:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139)", 2024-09-04T17:48:45.170 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:18.971074+0000", 2024-09-04T17:48:45.170 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 26.146717046999999, 2024-09-04T17:48:45.171 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.017562632000000002, 2024-09-04T17:48:45.171 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:45.171 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:45.171 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:45.171 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:45.171 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:45.171 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:45.171 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 98345 2024-09-04T17:48:45.171 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.171 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:45.171 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.171 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:45.171 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.971074+0000", 2024-09-04T17:48:45.172 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:45.172 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.172 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.172 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:45.172 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.971074+0000", 2024-09-04T17:48:45.172 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:45.172 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.172 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.172 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:45.172 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.971078+0000", 2024-09-04T17:48:45.172 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.1969999999999998e-06 2024-09-04T17:48:45.172 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.172 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.173 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:45.173 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.971096+0000", 2024-09-04T17:48:45.173 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7898999999999999e-05 2024-09-04T17:48:45.173 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.173 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.173 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:45.173 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.971140+0000", 2024-09-04T17:48:45.173 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.3883e-05 2024-09-04T17:48:45.173 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.173 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.173 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:45.173 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.971165+0000", 2024-09-04T17:48:45.173 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5321999999999999e-05 2024-09-04T17:48:45.173 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.174 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.174 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:45.174 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.971346+0000", 2024-09-04T17:48:45.174 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000180784 2024-09-04T17:48:45.174 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.174 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.174 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op must wait for map", 2024-09-04T17:48:45.174 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.971369+0000", 2024-09-04T17:48:45.174 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.3286e-05 2024-09-04T17:48:45.174 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.174 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.174 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:45.174 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.977475+0000", 2024-09-04T17:48:45.175 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0061055459999999999 2024-09-04T17:48:45.175 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.175 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.175 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:45.175 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.977665+0000", 2024-09-04T17:48:45.175 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019043599999999999 2024-09-04T17:48:45.175 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.175 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.175 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 4", 2024-09-04T17:48:45.175 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.978586+0000", 2024-09-04T17:48:45.175 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000920597 2024-09-04T17:48:45.175 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.175 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.175 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:48:45.176 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.982469+0000", 2024-09-04T17:48:45.176 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0038834719999999998 2024-09-04T17:48:45.176 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.176 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.176 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:45.176 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.988390+0000", 2024-09-04T17:48:45.176 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0059206479999999997 2024-09-04T17:48:45.176 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.176 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.176 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:45.176 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.988546+0000", 2024-09-04T17:48:45.176 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00015548 2024-09-04T17:48:45.176 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.177 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.177 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:45.177 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:18.988637+0000", 2024-09-04T17:48:45.177 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1082000000000006e-05 2024-09-04T17:48:45.177 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:45.177 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:45.177 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:45.177 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.177 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.177 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:98921 3.c e142/122 3:37678a48:::rbd_data.113460fdfacd.0000000000000006:head v 142'572, mlcod=137'569)", 2024-09-04T17:48:45.177 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:22.131110+0000", 2024-09-04T17:48:45.177 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.986681145999999, 2024-09-04T17:48:45.177 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0055872880000000002, 2024-09-04T17:48:45.178 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:45.178 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:45.178 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:45.178 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:45.178 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.178 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:45.178 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.131110+0000", 2024-09-04T17:48:45.178 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:45.178 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.178 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.178 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:45.178 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.131110+0000", 2024-09-04T17:48:45.178 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:45.179 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.179 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.179 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:45.179 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.131110+0000", 2024-09-04T17:48:45.179 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7e-07 2024-09-04T17:48:45.179 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.179 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.179 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:45.179 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.131120+0000", 2024-09-04T17:48:45.179 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.2499999999999995e-06 2024-09-04T17:48:45.179 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.179 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.179 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:45.179 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.131146+0000", 2024-09-04T17:48:45.180 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6461999999999998e-05 2024-09-04T17:48:45.180 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.180 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.180 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:45.180 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.131157+0000", 2024-09-04T17:48:45.180 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1522000000000001e-05 2024-09-04T17:48:45.180 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.180 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.180 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:45.180 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.131323+0000", 2024-09-04T17:48:45.180 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00016558499999999999 2024-09-04T17:48:45.180 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.180 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.181 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:45.181 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.131351+0000", 2024-09-04T17:48:45.181 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.7494000000000001e-05 2024-09-04T17:48:45.181 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.181 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.181 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:45.181 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.136648+0000", 2024-09-04T17:48:45.181 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0052975330000000001 2024-09-04T17:48:45.181 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.181 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.181 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:45.181 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.136697+0000", 2024-09-04T17:48:45.181 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9072e-05 2024-09-04T17:48:45.182 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:45.182 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:45.182 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:45.182 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.182 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.182 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:98922 3.0 e142/121 3:00fa19d2:::rbd_data.113460fdfacd.0000000000000007:head v 142'381, mlcod=135'378)", 2024-09-04T17:48:45.182 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:22.132316+0000", 2024-09-04T17:48:45.182 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 22.985475364999999, 2024-09-04T17:48:45.182 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0070776679999999996, 2024-09-04T17:48:45.182 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:45.182 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:45.182 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:45.182 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:45.183 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.183 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:45.183 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.132316+0000", 2024-09-04T17:48:45.183 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:45.183 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.183 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.183 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:45.183 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.132316+0000", 2024-09-04T17:48:45.183 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:45.183 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.183 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.183 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:45.183 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.132316+0000", 2024-09-04T17:48:45.183 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2300000000000002e-07 2024-09-04T17:48:45.184 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.184 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.184 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:45.184 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.132322+0000", 2024-09-04T17:48:45.184 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.48e-06 2024-09-04T17:48:45.184 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.184 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.184 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:45.184 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.132335+0000", 2024-09-04T17:48:45.184 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3040999999999999e-05 2024-09-04T17:48:45.184 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.184 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.184 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:45.185 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.132344+0000", 2024-09-04T17:48:45.185 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9989999999999997e-06 2024-09-04T17:48:45.185 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.185 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.185 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:45.185 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.132453+0000", 2024-09-04T17:48:45.185 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000109615 2024-09-04T17:48:45.185 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.185 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.185 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:45.185 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.132481+0000", 2024-09-04T17:48:45.185 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.777e-05 2024-09-04T17:48:45.185 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.186 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.186 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:45.186 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.139271+0000", 2024-09-04T17:48:45.186 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0067898180000000004 2024-09-04T17:48:45.186 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.186 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.186 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:45.186 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:22.139393+0000", 2024-09-04T17:48:45.186 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00012252199999999999 2024-09-04T17:48:45.186 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:45.186 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:45.186 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:45.186 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.186 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.187 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:99605 3.a 3:50de26c1:::rbd_data.11346a2b0bc9.0000000000000008:head [delete] snapc 1c=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-04T17:48:45.187 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:27.983388+0000", 2024-09-04T17:48:45.187 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 17.134403206000002, 2024-09-04T17:48:45.187 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0073493309999999997, 2024-09-04T17:48:45.187 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:45.187 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:45.187 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:45.187 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:45.187 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:45.187 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:45.187 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 99605 2024-09-04T17:48:45.187 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.187 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:45.187 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.188 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:45.188 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:27.983388+0000", 2024-09-04T17:48:45.188 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:45.188 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.188 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.188 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:45.188 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:27.983388+0000", 2024-09-04T17:48:45.188 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:45.188 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.188 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.188 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:45.188 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:27.983392+0000", 2024-09-04T17:48:45.188 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.9770000000000002e-06 2024-09-04T17:48:45.189 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.189 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.189 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:45.189 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:27.983413+0000", 2024-09-04T17:48:45.189 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1075999999999999e-05 2024-09-04T17:48:45.189 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.189 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.189 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:45.189 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:27.983457+0000", 2024-09-04T17:48:45.189 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.3943999999999998e-05 2024-09-04T17:48:45.189 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.189 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.189 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:45.190 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:27.983482+0000", 2024-09-04T17:48:45.190 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5602999999999999e-05 2024-09-04T17:48:45.190 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.190 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.190 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:45.190 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:27.983671+0000", 2024-09-04T17:48:45.190 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00018823399999999999 2024-09-04T17:48:45.190 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.190 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.190 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:45.190 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:27.984069+0000", 2024-09-04T17:48:45.190 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000398639 2024-09-04T17:48:45.190 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.191 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.191 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 4", 2024-09-04T17:48:45.191 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:27.984378+0000", 2024-09-04T17:48:45.191 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000308392 2024-09-04T17:48:45.191 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.191 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.191 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:48:45.191 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:27.987068+0000", 2024-09-04T17:48:45.191 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.002690764 2024-09-04T17:48:45.191 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.191 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.191 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:45.191 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:27.990452+0000", 2024-09-04T17:48:45.192 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0033831080000000001 2024-09-04T17:48:45.192 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.192 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.192 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:45.192 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:27.990603+0000", 2024-09-04T17:48:45.192 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00015124 2024-09-04T17:48:45.192 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.192 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.192 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:45.192 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:27.990737+0000", 2024-09-04T17:48:45.192 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013435399999999999 2024-09-04T17:48:45.192 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:45.192 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:45.193 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:45.193 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.193 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.193 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:99857 3.10 3:0f7166b0:::rbd_header.1134968ccb73:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e153)", 2024-09-04T17:48:45.193 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:32.841823+0000", 2024-09-04T17:48:45.193 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 12.275967824, 2024-09-04T17:48:45.193 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.014014146999999999, 2024-09-04T17:48:45.193 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:45.193 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:45.193 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:45.193 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:45.193 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:45.193 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:45.194 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 99857 2024-09-04T17:48:45.194 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.194 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:45.194 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.194 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:45.194 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.841823+0000", 2024-09-04T17:48:45.194 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:45.194 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.194 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.194 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:45.194 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.841823+0000", 2024-09-04T17:48:45.194 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:45.194 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.195 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.195 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:45.195 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.841827+0000", 2024-09-04T17:48:45.195 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.3699999999999999e-06 2024-09-04T17:48:45.195 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.195 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.195 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:45.195 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.841858+0000", 2024-09-04T17:48:45.195 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.1795999999999998e-05 2024-09-04T17:48:45.195 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.195 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.195 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:45.195 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.841906+0000", 2024-09-04T17:48:45.195 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7737000000000001e-05 2024-09-04T17:48:45.196 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.196 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.196 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:45.196 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.841930+0000", 2024-09-04T17:48:45.196 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4272000000000001e-05 2024-09-04T17:48:45.196 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.196 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.196 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:45.196 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.842177+0000", 2024-09-04T17:48:45.196 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024632300000000001 2024-09-04T17:48:45.196 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.196 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.196 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op must wait for map", 2024-09-04T17:48:45.197 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.842194+0000", 2024-09-04T17:48:45.197 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7606000000000001e-05 2024-09-04T17:48:45.197 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.197 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.197 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:45.197 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.850210+0000", 2024-09-04T17:48:45.197 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0080154179999999998 2024-09-04T17:48:45.197 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.197 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.197 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:45.197 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.850494+0000", 2024-09-04T17:48:45.197 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00028466100000000002 2024-09-04T17:48:45.197 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.198 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.198 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 0", 2024-09-04T17:48:45.198 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.851786+0000", 2024-09-04T17:48:45.198 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.001291468 2024-09-04T17:48:45.198 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.198 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.198 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:48:45.198 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.855374+0000", 2024-09-04T17:48:45.198 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0035884760000000002 2024-09-04T17:48:45.198 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.198 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.198 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:45.198 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.855504+0000", 2024-09-04T17:48:45.199 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00012924799999999999 2024-09-04T17:48:45.199 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.199 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.199 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:45.199 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.855704+0000", 2024-09-04T17:48:45.199 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000200764 2024-09-04T17:48:45.199 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.199 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.199 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:45.199 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.855837+0000", 2024-09-04T17:48:45.199 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000133008 2024-09-04T17:48:45.199 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:45.199 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:45.200 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:45.200 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.200 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.200 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:99861 3.10 3:0f7166b0:::rbd_header.1134968ccb73:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e153)", 2024-09-04T17:48:45.200 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:32.872359+0000", 2024-09-04T17:48:45.200 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 12.245431966, 2024-09-04T17:48:45.200 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.006863063, 2024-09-04T17:48:45.200 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:45.200 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:45.200 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:45.200 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:45.200 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:45.201 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:45.201 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 99861 2024-09-04T17:48:45.201 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.201 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:45.201 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.201 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:45.201 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.872359+0000", 2024-09-04T17:48:45.201 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:45.201 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.201 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.201 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:45.201 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.872359+0000", 2024-09-04T17:48:45.201 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:45.202 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.202 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.202 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:45.202 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.872363+0000", 2024-09-04T17:48:45.202 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.8260000000000003e-06 2024-09-04T17:48:45.202 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.202 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.202 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:45.202 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.872382+0000", 2024-09-04T17:48:45.202 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9318999999999999e-05 2024-09-04T17:48:45.202 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.202 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.202 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:45.202 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.872426+0000", 2024-09-04T17:48:45.203 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4038000000000001e-05 2024-09-04T17:48:45.203 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.203 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.203 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:45.203 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.872452+0000", 2024-09-04T17:48:45.203 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5479000000000001e-05 2024-09-04T17:48:45.203 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.203 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.203 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:45.203 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.872647+0000", 2024-09-04T17:48:45.203 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019568799999999999 2024-09-04T17:48:45.203 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.203 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.203 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:45.204 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.872894+0000", 2024-09-04T17:48:45.204 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000246226 2024-09-04T17:48:45.204 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.204 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.204 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 0", 2024-09-04T17:48:45.204 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.873410+0000", 2024-09-04T17:48:45.204 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00051657000000000005 2024-09-04T17:48:45.204 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.204 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.204 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:48:45.204 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.876401+0000", 2024-09-04T17:48:45.204 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.002990983 2024-09-04T17:48:45.204 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.205 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.205 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:45.205 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.878957+0000", 2024-09-04T17:48:45.205 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.002556039 2024-09-04T17:48:45.205 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.205 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.205 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:45.205 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.879109+0000", 2024-09-04T17:48:45.205 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00015156999999999999 2024-09-04T17:48:45.205 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.205 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.205 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:45.205 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:32.879222+0000", 2024-09-04T17:48:45.206 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000113325 2024-09-04T17:48:45.206 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:45.206 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:45.206 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:45.206 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.206 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.206 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:100488 3.1e e156/149 3:7e3581b1:::rbd_data.11343e517879.0000000000000008:head v 156'625, mlcod=151'621)", 2024-09-04T17:48:45.206 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:36.026379+0000", 2024-09-04T17:48:45.206 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 9.0914121399999992, 2024-09-04T17:48:45.206 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0076449719999999999, 2024-09-04T17:48:45.206 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:45.206 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:45.206 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:45.206 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:45.207 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.207 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:45.207 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:36.026379+0000", 2024-09-04T17:48:45.207 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:45.207 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.207 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.207 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:45.207 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:36.026379+0000", 2024-09-04T17:48:45.207 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:45.207 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.207 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.207 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:45.207 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:36.026381+0000", 2024-09-04T17:48:45.208 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.906e-06 2024-09-04T17:48:45.208 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.208 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.208 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:45.208 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:36.026395+0000", 2024-09-04T17:48:45.208 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4343e-05 2024-09-04T17:48:45.208 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.208 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.208 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:45.208 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:36.026427+0000", 2024-09-04T17:48:45.208 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.1569000000000002e-05 2024-09-04T17:48:45.208 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.208 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.209 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:45.209 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:36.026446+0000", 2024-09-04T17:48:45.209 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9459000000000001e-05 2024-09-04T17:48:45.209 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.209 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.209 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:45.209 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:36.026664+0000", 2024-09-04T17:48:45.209 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021771099999999999 2024-09-04T17:48:45.209 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.209 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.209 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:45.209 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:36.026720+0000", 2024-09-04T17:48:45.209 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.5943999999999998e-05 2024-09-04T17:48:45.210 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.210 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.210 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:45.210 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:36.033904+0000", 2024-09-04T17:48:45.210 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0071845219999999996 2024-09-04T17:48:45.210 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.210 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.210 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:45.210 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:36.034024+0000", 2024-09-04T17:48:45.210 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000119518 2024-09-04T17:48:45.210 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:45.210 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:45.210 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:45.210 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.211 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.211 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(osd.2.0:484 3.0 e157/121 3:070e9b06:::rbd_data.113418f5e708.000000000000002e:16 v 157'438, mlcod=155'433)", 2024-09-04T17:48:45.211 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:36.881063+0000", 2024-09-04T17:48:45.211 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 8.2367283439999994, 2024-09-04T17:48:45.211 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0094645140000000003, 2024-09-04T17:48:45.211 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:45.211 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:45.211 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:45.211 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:45.211 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.211 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:45.211 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:36.881063+0000", 2024-09-04T17:48:45.211 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:45.212 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.212 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.212 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:45.212 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:36.881063+0000", 2024-09-04T17:48:45.212 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:45.212 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.212 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.212 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:45.212 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:36.881065+0000", 2024-09-04T17:48:45.212 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8360000000000001e-06 2024-09-04T17:48:45.212 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.212 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.212 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:45.213 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:36.881080+0000", 2024-09-04T17:48:45.213 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5046e-05 2024-09-04T17:48:45.213 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.213 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.213 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:45.213 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:36.881109+0000", 2024-09-04T17:48:45.213 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.9246000000000001e-05 2024-09-04T17:48:45.213 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.213 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.213 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:45.213 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:36.881129+0000", 2024-09-04T17:48:45.213 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0018999999999999e-05 2024-09-04T17:48:45.213 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.214 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.214 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:45.214 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:36.882909+0000", 2024-09-04T17:48:45.214 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.001779912 2024-09-04T17:48:45.214 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.214 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.214 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:45.214 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:36.882973+0000", 2024-09-04T17:48:45.214 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.4177999999999994e-05 2024-09-04T17:48:45.214 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.214 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.214 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:45.214 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:36.890404+0000", 2024-09-04T17:48:45.215 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0074313119999999998 2024-09-04T17:48:45.215 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.215 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.215 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:45.215 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:36.890527+0000", 2024-09-04T17:48:45.215 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000122965 2024-09-04T17:48:45.215 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:45.215 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:45.215 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:45.215 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.215 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.215 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:100815 3.10 3:0883b82e:::rbd_data.1134c5a43465.0000000000000003:head [delete] snapc 15=[] ondisk+write+known_if_redirected+supports_pool_eio e158)", 2024-09-04T17:48:45.215 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:37.972704+0000", 2024-09-04T17:48:45.216 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 7.1450871750000005, 2024-09-04T17:48:45.216 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.006327404, 2024-09-04T17:48:45.216 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:45.216 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:45.216 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:45.216 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:45.216 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:45.216 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:45.216 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 100815 2024-09-04T17:48:45.216 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.216 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:45.216 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.216 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:45.217 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:37.972704+0000", 2024-09-04T17:48:45.217 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:45.217 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.217 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.217 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:45.217 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:37.972704+0000", 2024-09-04T17:48:45.217 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:45.217 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.217 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.217 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:45.217 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:37.972706+0000", 2024-09-04T17:48:45.217 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6780000000000001e-06 2024-09-04T17:48:45.217 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.218 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.218 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:45.218 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:37.972713+0000", 2024-09-04T17:48:45.218 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.6690000000000006e-06 2024-09-04T17:48:45.218 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.218 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.218 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:45.218 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:37.972731+0000", 2024-09-04T17:48:45.218 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8082999999999999e-05 2024-09-04T17:48:45.218 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.218 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.218 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:45.218 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:37.972741+0000", 2024-09-04T17:48:45.219 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.3230000000000004e-06 2024-09-04T17:48:45.219 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.219 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.219 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:45.219 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:37.972840+0000", 2024-09-04T17:48:45.219 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.9079000000000005e-05 2024-09-04T17:48:45.219 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.219 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.219 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op must wait for map", 2024-09-04T17:48:45.219 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:37.972850+0000", 2024-09-04T17:48:45.219 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.8490000000000001e-06 2024-09-04T17:48:45.219 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.219 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.219 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:45.220 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:37.976597+0000", 2024-09-04T17:48:45.220 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.003747289 2024-09-04T17:48:45.220 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.220 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.220 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:45.220 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:37.976972+0000", 2024-09-04T17:48:45.220 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00037561499999999999 2024-09-04T17:48:45.220 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.220 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.220 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 0", 2024-09-04T17:48:45.220 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:37.977275+0000", 2024-09-04T17:48:45.220 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00030304000000000003 2024-09-04T17:48:45.220 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.221 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.221 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:48:45.221 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:37.978722+0000", 2024-09-04T17:48:45.221 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.001446922 2024-09-04T17:48:45.221 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.221 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.221 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:45.221 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:37.978931+0000", 2024-09-04T17:48:45.221 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000208171 2024-09-04T17:48:45.221 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.221 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.221 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:45.221 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:37.978990+0000", 2024-09-04T17:48:45.222 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.9079999999999997e-05 2024-09-04T17:48:45.222 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.222 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.222 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:45.222 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:37.979031+0000", 2024-09-04T17:48:45.222 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.1606000000000001e-05 2024-09-04T17:48:45.222 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:45.222 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:45.222 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:45.222 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.222 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.222 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:101584 3.a 3:535385f0:::rbd_children:head [call rbd.remove_child in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e162)", 2024-09-04T17:48:45.222 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:42.101123+0000", 2024-09-04T17:48:45.223 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 3.016668036, 2024-09-04T17:48:45.223 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0082904399999999996, 2024-09-04T17:48:45.223 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:45.223 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:45.223 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:45.223 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:45.223 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:45.223 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:45.223 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 101584 2024-09-04T17:48:45.223 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.223 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:45.223 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.223 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:45.224 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:42.101123+0000", 2024-09-04T17:48:45.224 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:45.224 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.224 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.224 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:45.224 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:42.101123+0000", 2024-09-04T17:48:45.224 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:45.224 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.224 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.224 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:45.224 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:42.101127+0000", 2024-09-04T17:48:45.224 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.687e-06 2024-09-04T17:48:45.224 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.225 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.225 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:45.225 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:42.101142+0000", 2024-09-04T17:48:45.225 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5489e-05 2024-09-04T17:48:45.225 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.225 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.225 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:45.225 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:42.101179+0000", 2024-09-04T17:48:45.225 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.6532000000000002e-05 2024-09-04T17:48:45.225 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.225 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.225 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:45.225 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:42.101196+0000", 2024-09-04T17:48:45.226 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7396000000000001e-05 2024-09-04T17:48:45.226 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.226 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.226 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:45.226 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:42.101357+0000", 2024-09-04T17:48:45.226 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00016047199999999999 2024-09-04T17:48:45.226 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.226 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.226 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op must wait for map", 2024-09-04T17:48:45.226 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:42.101368+0000", 2024-09-04T17:48:45.226 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1816e-05 2024-09-04T17:48:45.226 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.226 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.227 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:45.227 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:42.105091+0000", 2024-09-04T17:48:45.227 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0037228740000000001 2024-09-04T17:48:45.227 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.227 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.227 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:45.227 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:42.105240+0000", 2024-09-04T17:48:45.227 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014837800000000001 2024-09-04T17:48:45.227 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.227 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.227 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 4", 2024-09-04T17:48:45.227 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:42.105598+0000", 2024-09-04T17:48:45.227 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00035803099999999997 2024-09-04T17:48:45.227 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.228 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.228 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:48:45.228 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:42.109057+0000", 2024-09-04T17:48:45.228 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0034588409999999998 2024-09-04T17:48:45.228 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.228 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.228 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:45.228 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:42.109140+0000", 2024-09-04T17:48:45.228 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.3823e-05 2024-09-04T17:48:45.228 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.228 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.228 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:45.228 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:42.109231+0000", 2024-09-04T17:48:45.228 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.0884000000000006e-05 2024-09-04T17:48:45.229 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:45.229 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:45.229 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:45.229 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:42.109413+0000", 2024-09-04T17:48:45.229 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00018221700000000001 2024-09-04T17:48:45.229 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:45.229 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:45.229 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:45.229 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:45.229 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:45.229 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:45.231 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-04T17:48:45.231 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:45.227+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:45.332 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:45.329+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:45.361 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:45.361 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:48:45.361 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:48:45.361 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:45.375 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-04T17:48:45.433 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:48:45.433+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:48:45.493 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:45.493 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:48:45.494 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:48:45.494 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:48:45.494 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:45.508 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-04T17:48:45.553 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:45.551+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:45.634 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:48:45.631+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:48:45.636 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:45.636 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:48:45.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:48:45.636 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:48:45.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.636 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:48444 3.12 3:4ff80fb5:::rbd_data.11346a2b0bc9.000000000000001d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-04T17:48:45.637 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:58.122601+0000", 2024-09-04T17:48:45.637 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 107.510627027, 2024-09-04T17:48:45.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057065713999999997, 2024-09-04T17:48:45.637 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:45.637 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:45.637 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:45.637 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:45.637 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:45.637 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:45.637 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 48444 2024-09-04T17:48:45.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.637 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:45.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:45.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122601+0000", 2024-09-04T17:48:45.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:45.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:45.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122601+0000", 2024-09-04T17:48:45.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:45.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:45.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122603+0000", 2024-09-04T17:48:45.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3499999999999999e-06 2024-09-04T17:48:45.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:45.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122614+0000", 2024-09-04T17:48:45.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.063e-05 2024-09-04T17:48:45.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:45.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122643+0000", 2024-09-04T17:48:45.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9434999999999999e-05 2024-09-04T17:48:45.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:45.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122657+0000", 2024-09-04T17:48:45.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3920000000000001e-05 2024-09-04T17:48:45.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:45.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122767+0000", 2024-09-04T17:48:45.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010981599999999999 2024-09-04T17:48:45.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:45.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122989+0000", 2024-09-04T17:48:45.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022158200000000001 2024-09-04T17:48:45.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 3", 2024-09-04T17:48:45.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.133349+0000", 2024-09-04T17:48:45.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010360729000000001 2024-09-04T17:48:45.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:45.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.179423+0000", 2024-09-04T17:48:45.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046073408000000003 2024-09-04T17:48:45.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:45.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.179453+0000", 2024-09-04T17:48:45.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0229000000000001e-05 2024-09-04T17:48:45.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:45.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.179545+0000", 2024-09-04T17:48:45.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1580000000000001e-05 2024-09-04T17:48:45.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:45.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.179667+0000", 2024-09-04T17:48:45.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012203500000000001 2024-09-04T17:48:45.643 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:45.643 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:45.643 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:45.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.644 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:55376 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-04T17:48:45.644 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:11.974053+0000", 2024-09-04T17:48:45.644 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 93.659175242999993, 2024-09-04T17:48:45.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060514722, 2024-09-04T17:48:45.644 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:45.644 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:45.644 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:45.644 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:45.644 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:45.644 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:45.644 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 55376 2024-09-04T17:48:45.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.645 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:45.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:45.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974053+0000", 2024-09-04T17:48:45.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:45.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:45.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974053+0000", 2024-09-04T17:48:45.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:45.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:45.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974054+0000", 2024-09-04T17:48:45.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.841e-06 2024-09-04T17:48:45.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:45.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974062+0000", 2024-09-04T17:48:45.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5460000000000003e-06 2024-09-04T17:48:45.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:45.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974077+0000", 2024-09-04T17:48:45.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5389000000000001e-05 2024-09-04T17:48:45.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.647 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:45.647 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974085+0000", 2024-09-04T17:48:45.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7659999999999998e-06 2024-09-04T17:48:45.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.647 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:45.647 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974188+0000", 2024-09-04T17:48:45.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000102521 2024-09-04T17:48:45.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.647 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:45.647 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974321+0000", 2024-09-04T17:48:45.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013324699999999999 2024-09-04T17:48:45.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.648 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-04T17:48:45.648 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.986481+0000", 2024-09-04T17:48:45.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012160450999999999 2024-09-04T17:48:45.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.648 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:45.648 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:12.034415+0000", 2024-09-04T17:48:45.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.047934065999999997 2024-09-04T17:48:45.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.649 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:45.649 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:12.034438+0000", 2024-09-04T17:48:45.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2923999999999999e-05 2024-09-04T17:48:45.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.649 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:45.649 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:12.034505+0000", 2024-09-04T17:48:45.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6711000000000006e-05 2024-09-04T17:48:45.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.649 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:45.649 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:12.034567+0000", 2024-09-04T17:48:45.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2260000000000004e-05 2024-09-04T17:48:45.649 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:45.650 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:45.650 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:45.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.650 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61141 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 2793472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:45.650 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819909+0000", 2024-09-04T17:48:45.650 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 91.813319250999996, 2024-09-04T17:48:45.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042786419999999999, 2024-09-04T17:48:45.652 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:45.652 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:45.652 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:45.652 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:45.652 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:45.652 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:45.652 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61141 2024-09-04T17:48:45.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.652 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:45.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.652 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:45.652 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819909+0000", 2024-09-04T17:48:45.652 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:45.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:45.653 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819909+0000", 2024-09-04T17:48:45.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:45.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:45.653 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819909+0000", 2024-09-04T17:48:45.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6300000000000005e-07 2024-09-04T17:48:45.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:45.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819913+0000", 2024-09-04T17:48:45.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4189999999999998e-06 2024-09-04T17:48:45.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:45.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819920+0000", 2024-09-04T17:48:45.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8619999999999997e-06 2024-09-04T17:48:45.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:45.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819926+0000", 2024-09-04T17:48:45.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9399999999999999e-06 2024-09-04T17:48:45.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:45.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862331+0000", 2024-09-04T17:48:45.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042405348000000002 2024-09-04T17:48:45.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:45.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862410+0000", 2024-09-04T17:48:45.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9078000000000004e-05 2024-09-04T17:48:45.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:45.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862695+0000", 2024-09-04T17:48:45.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028510999999999999 2024-09-04T17:48:45.656 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:45.656 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:45.656 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:45.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.656 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61142 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 2932736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:45.656 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819934+0000", 2024-09-04T17:48:45.656 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 91.813293633000001, 2024-09-04T17:48:45.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043101985000000002, 2024-09-04T17:48:45.656 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:45.656 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:45.657 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:45.657 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:45.657 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:45.657 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:45.657 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61142 2024-09-04T17:48:45.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.657 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:45.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.657 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:45.657 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819934+0000", 2024-09-04T17:48:45.657 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:45.658 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.658 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.658 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:45.658 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819934+0000", 2024-09-04T17:48:45.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:45.658 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.658 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.658 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:45.658 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819935+0000", 2024-09-04T17:48:45.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2024-09-04T17:48:45.658 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.658 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.659 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:45.659 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819937+0000", 2024-09-04T17:48:45.659 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5770000000000001e-06 2024-09-04T17:48:45.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.659 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.659 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:45.659 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819943+0000", 2024-09-04T17:48:45.659 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7950000000000004e-06 2024-09-04T17:48:45.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.659 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.659 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:45.660 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819948+0000", 2024-09-04T17:48:45.660 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6779999999999999e-06 2024-09-04T17:48:45.660 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.660 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:45.660 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862739+0000", 2024-09-04T17:48:45.660 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042791454999999999 2024-09-04T17:48:45.660 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.660 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:45.660 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862824+0000", 2024-09-04T17:48:45.661 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4790999999999995e-05 2024-09-04T17:48:45.661 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.661 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.661 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:45.661 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.863036+0000", 2024-09-04T17:48:45.661 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021224899999999999 2024-09-04T17:48:45.661 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:45.661 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:45.661 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:45.661 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.661 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.661 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61143 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 3039232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:45.661 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819954+0000", 2024-09-04T17:48:45.662 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 91.813273515000006, 2024-09-04T17:48:45.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043485134000000002, 2024-09-04T17:48:45.662 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:45.662 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:45.662 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:45.662 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:45.662 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:45.662 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:45.662 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61143 2024-09-04T17:48:45.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.662 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:45.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.662 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:45.662 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819954+0000", 2024-09-04T17:48:45.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:45.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:45.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819954+0000", 2024-09-04T17:48:45.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:45.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:45.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819955+0000", 2024-09-04T17:48:45.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7300000000000002e-07 2024-09-04T17:48:45.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:45.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819957+0000", 2024-09-04T17:48:45.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.024e-06 2024-09-04T17:48:45.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:45.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819963+0000", 2024-09-04T17:48:45.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8109999999999999e-06 2024-09-04T17:48:45.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:45.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819967+0000", 2024-09-04T17:48:45.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6149999999999999e-06 2024-09-04T17:48:45.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.665 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:45.665 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.863082+0000", 2024-09-04T17:48:45.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043115078000000001 2024-09-04T17:48:45.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.665 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:45.665 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.863172+0000", 2024-09-04T17:48:45.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0181999999999998e-05 2024-09-04T17:48:45.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.665 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:45.666 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.863440+0000", 2024-09-04T17:48:45.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026705099999999997 2024-09-04T17:48:45.666 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:45.666 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:45.666 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:45.666 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.666 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:70015 3.6 3:6012e780:::rbd_data.11341aa245a3.000000000000005b:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-04T17:48:45.666 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:26.155497+0000", 2024-09-04T17:48:45.666 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 79.477730457999996, 2024-09-04T17:48:45.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.049007124999999999, 2024-09-04T17:48:45.666 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:45.666 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:45.667 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:45.667 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:45.667 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:45.667 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:45.667 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 70015 2024-09-04T17:48:45.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.667 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:45.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.667 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:45.667 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155497+0000", 2024-09-04T17:48:45.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:45.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.668 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:45.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155497+0000", 2024-09-04T17:48:45.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:45.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.668 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:45.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155500+0000", 2024-09-04T17:48:45.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7070000000000001e-06 2024-09-04T17:48:45.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.668 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:45.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155513+0000", 2024-09-04T17:48:45.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2989e-05 2024-09-04T17:48:45.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.669 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:45.669 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155544+0000", 2024-09-04T17:48:45.669 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0747000000000002e-05 2024-09-04T17:48:45.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.669 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:45.669 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155560+0000", 2024-09-04T17:48:45.669 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5866e-05 2024-09-04T17:48:45.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.669 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:45.670 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155675+0000", 2024-09-04T17:48:45.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000114838 2024-09-04T17:48:45.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.670 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:45.670 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155803+0000", 2024-09-04T17:48:45.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012851799999999999 2024-09-04T17:48:45.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.670 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 1", 2024-09-04T17:48:45.670 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.163582+0000", 2024-09-04T17:48:45.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0077790960000000001 2024-09-04T17:48:45.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:45.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.204055+0000", 2024-09-04T17:48:45.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.040472517 2024-09-04T17:48:45.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:45.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.204195+0000", 2024-09-04T17:48:45.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014055200000000001 2024-09-04T17:48:45.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.672 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:45.672 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.204373+0000", 2024-09-04T17:48:45.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000177784 2024-09-04T17:48:45.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.672 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:45.673 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.204505+0000", 2024-09-04T17:48:45.673 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013151100000000001 2024-09-04T17:48:45.673 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:45.673 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:45.673 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:45.673 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.673 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.673 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:70434 3.12 3:4cd2342b:::rbd_data.11341aa245a3.0000000000000091:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-04T17:48:45.673 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:26.943124+0000", 2024-09-04T17:48:45.673 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 78.690103676000007, 2024-09-04T17:48:45.673 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055011831999999997, 2024-09-04T17:48:45.673 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:45.673 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:45.674 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:45.674 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:45.674 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:45.674 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:45.674 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 70434 2024-09-04T17:48:45.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.674 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:45.674 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.674 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:45.674 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943124+0000", 2024-09-04T17:48:45.674 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:45.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.674 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.675 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:45.675 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943124+0000", 2024-09-04T17:48:45.675 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:45.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.675 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.675 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:45.675 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943128+0000", 2024-09-04T17:48:45.675 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8460000000000001e-06 2024-09-04T17:48:45.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.675 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.675 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:45.675 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943142+0000", 2024-09-04T17:48:45.675 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4089e-05 2024-09-04T17:48:45.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.676 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:45.676 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943177+0000", 2024-09-04T17:48:45.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5167999999999999e-05 2024-09-04T17:48:45.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.676 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:45.676 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943194+0000", 2024-09-04T17:48:45.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6235e-05 2024-09-04T17:48:45.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.676 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:45.677 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943286+0000", 2024-09-04T17:48:45.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2793000000000004e-05 2024-09-04T17:48:45.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:45.677 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943526+0000", 2024-09-04T17:48:45.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023950999999999999 2024-09-04T17:48:45.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 1", 2024-09-04T17:48:45.677 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.952871+0000", 2024-09-04T17:48:45.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0093453070000000006 2024-09-04T17:48:45.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.678 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:45.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.997968+0000", 2024-09-04T17:48:45.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045096613000000001 2024-09-04T17:48:45.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.678 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:45.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.998006+0000", 2024-09-04T17:48:45.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8257000000000003e-05 2024-09-04T17:48:45.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.678 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:45.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.998073+0000", 2024-09-04T17:48:45.679 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6803000000000004e-05 2024-09-04T17:48:45.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.679 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:45.679 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.998136+0000", 2024-09-04T17:48:45.679 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3211000000000002e-05 2024-09-04T17:48:45.679 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:45.679 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:45.679 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:45.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.679 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93841 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 5111808~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:45.679 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386491+0000", 2024-09-04T17:48:45.679 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 46.246736478999999, 2024-09-04T17:48:45.680 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043069578999999997, 2024-09-04T17:48:45.680 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:45.680 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:45.680 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:45.680 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:45.680 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:45.680 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:45.680 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93841 2024-09-04T17:48:45.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.680 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:45.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.680 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:45.680 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386491+0000", 2024-09-04T17:48:45.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:45.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:45.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386491+0000", 2024-09-04T17:48:45.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:45.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:45.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386492+0000", 2024-09-04T17:48:45.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0299999999999999e-07 2024-09-04T17:48:45.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:45.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386499+0000", 2024-09-04T17:48:45.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6089999999999997e-06 2024-09-04T17:48:45.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:45.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386506+0000", 2024-09-04T17:48:45.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3599999999999998e-06 2024-09-04T17:48:45.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:45.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386514+0000", 2024-09-04T17:48:45.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9379999999999992e-06 2024-09-04T17:48:45.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.683 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:45.683 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.428957+0000", 2024-09-04T17:48:45.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042442831 2024-09-04T17:48:45.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.683 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:45.683 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.429080+0000", 2024-09-04T17:48:45.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012359299999999999 2024-09-04T17:48:45.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.684 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:45.684 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.429561+0000", 2024-09-04T17:48:45.684 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00048074499999999998 2024-09-04T17:48:45.684 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:45.684 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:45.684 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:45.684 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.684 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.684 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93842 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 4259840~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:45.684 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386521+0000", 2024-09-04T17:48:45.684 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 46.246706547000002, 2024-09-04T17:48:45.684 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043661933999999999, 2024-09-04T17:48:45.684 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:45.685 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:45.685 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:45.685 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:45.685 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:45.685 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:45.685 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93842 2024-09-04T17:48:45.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.685 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:45.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.685 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:45.685 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386521+0000", 2024-09-04T17:48:45.685 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:45.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:45.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386521+0000", 2024-09-04T17:48:45.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:45.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:45.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386522+0000", 2024-09-04T17:48:45.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4700000000000001e-07 2024-09-04T17:48:45.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:45.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386524+0000", 2024-09-04T17:48:45.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4559999999999999e-06 2024-09-04T17:48:45.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:45.687 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386534+0000", 2024-09-04T17:48:45.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2639999999999997e-06 2024-09-04T17:48:45.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:45.687 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386539+0000", 2024-09-04T17:48:45.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6130000000000003e-06 2024-09-04T17:48:45.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.688 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:45.688 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.429577+0000", 2024-09-04T17:48:45.688 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043037757000000003 2024-09-04T17:48:45.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.688 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.688 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:45.688 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.429638+0000", 2024-09-04T17:48:45.688 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1235999999999994e-05 2024-09-04T17:48:45.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.688 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.688 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:45.688 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.430183+0000", 2024-09-04T17:48:45.688 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000545061 2024-09-04T17:48:45.689 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:45.689 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:45.689 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:45.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.689 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93843 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 4390912~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:45.689 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386547+0000", 2024-09-04T17:48:45.689 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 46.24668131, 2024-09-04T17:48:45.689 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044260668000000003, 2024-09-04T17:48:45.689 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:45.689 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:45.689 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:45.689 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:45.690 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:45.690 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:45.690 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93843 2024-09-04T17:48:45.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.690 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:45.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.690 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:45.690 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386547+0000", 2024-09-04T17:48:45.690 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:45.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.690 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:45.690 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386547+0000", 2024-09-04T17:48:45.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:45.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.691 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:45.691 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386547+0000", 2024-09-04T17:48:45.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4199999999999996e-07 2024-09-04T17:48:45.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.691 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:45.691 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386550+0000", 2024-09-04T17:48:45.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6199999999999999e-06 2024-09-04T17:48:45.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.692 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:45.692 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386557+0000", 2024-09-04T17:48:45.692 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1919999999999999e-06 2024-09-04T17:48:45.692 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.692 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.692 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:45.692 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386562+0000", 2024-09-04T17:48:45.692 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2580000000000003e-06 2024-09-04T17:48:45.692 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.692 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.692 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:45.692 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.430226+0000", 2024-09-04T17:48:45.693 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043663484000000002 2024-09-04T17:48:45.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.693 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:45.693 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.430286+0000", 2024-09-04T17:48:45.693 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0455999999999999e-05 2024-09-04T17:48:45.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.693 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:45.693 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.430807+0000", 2024-09-04T17:48:45.693 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00052111599999999996 2024-09-04T17:48:45.693 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:45.693 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:45.694 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:45.694 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.694 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.694 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93844 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 4521984~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:45.694 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386603+0000", 2024-09-04T17:48:45.694 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 46.246625258999998, 2024-09-04T17:48:45.694 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044705350999999997, 2024-09-04T17:48:45.694 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:45.694 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:45.694 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:45.694 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:45.694 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:45.694 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:45.695 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93844 2024-09-04T17:48:45.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.695 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:45.695 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.695 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:45.695 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386603+0000", 2024-09-04T17:48:45.695 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:45.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.695 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.695 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:45.695 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386603+0000", 2024-09-04T17:48:45.695 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:45.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.695 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.696 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:45.696 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386603+0000", 2024-09-04T17:48:45.696 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.92e-07 2024-09-04T17:48:45.696 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.696 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.696 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:45.696 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386606+0000", 2024-09-04T17:48:45.696 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5500000000000001e-06 2024-09-04T17:48:45.696 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.696 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.696 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:45.696 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386614+0000", 2024-09-04T17:48:45.696 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9570000000000002e-06 2024-09-04T17:48:45.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.697 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:45.697 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386619+0000", 2024-09-04T17:48:45.697 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5119999999999999e-06 2024-09-04T17:48:45.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.697 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:45.697 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.430828+0000", 2024-09-04T17:48:45.697 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044208654 2024-09-04T17:48:45.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.697 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:45.698 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.430903+0000", 2024-09-04T17:48:45.698 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4629999999999995e-05 2024-09-04T17:48:45.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.698 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.698 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:45.698 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.431308+0000", 2024-09-04T17:48:45.698 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000405256 2024-09-04T17:48:45.698 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:45.698 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:45.698 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:45.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.698 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.699 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93845 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 4653056~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:45.699 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386627+0000", 2024-09-04T17:48:45.699 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 46.246600669999999, 2024-09-04T17:48:45.699 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045262927000000001, 2024-09-04T17:48:45.699 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:45.699 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:45.699 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:45.699 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:45.699 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:45.699 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:45.699 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93845 2024-09-04T17:48:45.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.700 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:45.700 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.700 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:45.700 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386627+0000", 2024-09-04T17:48:45.700 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:45.700 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.700 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.700 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:45.700 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386627+0000", 2024-09-04T17:48:45.700 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:45.700 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.700 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.700 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:45.701 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386628+0000", 2024-09-04T17:48:45.701 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0999999999999998e-07 2024-09-04T17:48:45.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.701 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.701 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:45.701 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386630+0000", 2024-09-04T17:48:45.701 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3300000000000001e-06 2024-09-04T17:48:45.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.701 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.701 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:45.701 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386645+0000", 2024-09-04T17:48:45.701 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5186e-05 2024-09-04T17:48:45.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:45.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386651+0000", 2024-09-04T17:48:45.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7710000000000003e-06 2024-09-04T17:48:45.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:45.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.431349+0000", 2024-09-04T17:48:45.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044697723000000002 2024-09-04T17:48:45.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:45.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.431410+0000", 2024-09-04T17:48:45.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0677999999999997e-05 2024-09-04T17:48:45.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.703 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:45.703 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.431890+0000", 2024-09-04T17:48:45.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00048052899999999999 2024-09-04T17:48:45.703 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:45.703 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:45.703 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:45.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.703 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93846 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 4784128~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:45.703 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386659+0000", 2024-09-04T17:48:45.703 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 46.246569000999997, 2024-09-04T17:48:45.704 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045784490999999997, 2024-09-04T17:48:45.704 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:45.704 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:45.704 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:45.704 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:45.704 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:45.704 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:45.704 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93846 2024-09-04T17:48:45.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.704 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:45.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.704 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:45.705 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386659+0000", 2024-09-04T17:48:45.705 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:45.705 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.705 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.705 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:45.705 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386659+0000", 2024-09-04T17:48:45.705 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:45.705 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.705 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.705 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:45.705 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386659+0000", 2024-09-04T17:48:45.705 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7000000000000005e-07 2024-09-04T17:48:45.705 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.706 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.706 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:45.706 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386662+0000", 2024-09-04T17:48:45.706 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3300000000000001e-06 2024-09-04T17:48:45.706 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.706 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.706 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:45.706 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386669+0000", 2024-09-04T17:48:45.706 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0589999999999997e-06 2024-09-04T17:48:45.706 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.706 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.706 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:45.707 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386674+0000", 2024-09-04T17:48:45.707 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3009999999999997e-06 2024-09-04T17:48:45.707 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.707 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:45.707 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.431934+0000", 2024-09-04T17:48:45.707 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045259570999999998 2024-09-04T17:48:45.707 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.707 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:45.707 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.431998+0000", 2024-09-04T17:48:45.707 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4775000000000002e-05 2024-09-04T17:48:45.707 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.708 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:45.708 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.432443+0000", 2024-09-04T17:48:45.708 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00044488500000000001 2024-09-04T17:48:45.708 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:45.708 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:45.708 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:45.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.708 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.708 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93847 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 4915200~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:45.708 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386681+0000", 2024-09-04T17:48:45.708 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 46.246546744, 2024-09-04T17:48:45.708 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046305101000000001, 2024-09-04T17:48:45.709 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:45.709 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:45.709 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:45.709 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:45.709 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:45.709 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:45.709 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93847 2024-09-04T17:48:45.709 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.709 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:45.709 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.709 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:45.709 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386681+0000", 2024-09-04T17:48:45.709 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:45.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.710 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.710 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:45.710 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386681+0000", 2024-09-04T17:48:45.710 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:45.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.710 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.710 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:45.710 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386682+0000", 2024-09-04T17:48:45.710 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1699999999999998e-07 2024-09-04T17:48:45.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.710 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.710 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:45.711 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386684+0000", 2024-09-04T17:48:45.711 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.153e-06 2024-09-04T17:48:45.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.711 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:45.711 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386693+0000", 2024-09-04T17:48:45.711 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9639999999999992e-06 2024-09-04T17:48:45.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.711 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:45.712 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386698+0000", 2024-09-04T17:48:45.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3709999999999999e-06 2024-09-04T17:48:45.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.712 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:45.712 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.432476+0000", 2024-09-04T17:48:45.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045778147999999998 2024-09-04T17:48:45.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.713 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:45.713 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.432529+0000", 2024-09-04T17:48:45.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3165999999999999e-05 2024-09-04T17:48:45.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.713 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:45.713 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.432986+0000", 2024-09-04T17:48:45.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00045678199999999998 2024-09-04T17:48:45.713 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:45.713 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:45.713 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:45.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.714 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93848 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 5046272~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:45.714 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386705+0000", 2024-09-04T17:48:45.714 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 46.246522642999999, 2024-09-04T17:48:45.714 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046825259000000001, 2024-09-04T17:48:45.714 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:45.714 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:45.714 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:45.714 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:45.714 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:45.715 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:45.715 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93848 2024-09-04T17:48:45.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.715 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:45.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.715 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:45.715 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386705+0000", 2024-09-04T17:48:45.715 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:45.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.715 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:45.716 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386705+0000", 2024-09-04T17:48:45.716 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:45.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.716 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.716 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:45.716 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386706+0000", 2024-09-04T17:48:45.716 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1500000000000005e-07 2024-09-04T17:48:45.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.717 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:45.717 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386708+0000", 2024-09-04T17:48:45.717 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2979999999999999e-06 2024-09-04T17:48:45.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.717 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:45.717 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386715+0000", 2024-09-04T17:48:45.717 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8669999999999996e-06 2024-09-04T17:48:45.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.717 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:45.717 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386720+0000", 2024-09-04T17:48:45.718 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.242e-06 2024-09-04T17:48:45.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.718 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:45.718 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.433019+0000", 2024-09-04T17:48:45.718 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046298359999999997 2024-09-04T17:48:45.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.718 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:45.718 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.433084+0000", 2024-09-04T17:48:45.718 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5395999999999995e-05 2024-09-04T17:48:45.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.718 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:45.719 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.433531+0000", 2024-09-04T17:48:45.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00044658099999999999 2024-09-04T17:48:45.719 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:45.719 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:45.719 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:45.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.719 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.719 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93849 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 5177344~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:45.719 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386727+0000", 2024-09-04T17:48:45.719 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 46.246500744999999, 2024-09-04T17:48:45.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.047267796000000001, 2024-09-04T17:48:45.719 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:45.719 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:45.720 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:45.720 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:45.720 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:45.720 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:45.720 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93849 2024-09-04T17:48:45.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.720 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:45.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.720 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:45.720 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386727+0000", 2024-09-04T17:48:45.720 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:45.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.721 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:45.721 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386727+0000", 2024-09-04T17:48:45.721 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:45.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.721 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.721 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:45.721 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386728+0000", 2024-09-04T17:48:45.721 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1699999999999998e-07 2024-09-04T17:48:45.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.721 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.721 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:45.721 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386730+0000", 2024-09-04T17:48:45.722 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2589999999999999e-06 2024-09-04T17:48:45.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.722 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:45.722 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386751+0000", 2024-09-04T17:48:45.722 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1424999999999999e-05 2024-09-04T17:48:45.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.722 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:45.722 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386759+0000", 2024-09-04T17:48:45.722 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9400000000000002e-06 2024-09-04T17:48:45.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:45.723 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.433559+0000", 2024-09-04T17:48:45.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046799573999999997 2024-09-04T17:48:45.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:45.723 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.433610+0000", 2024-09-04T17:48:45.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1134999999999997e-05 2024-09-04T17:48:45.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:45.723 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.433995+0000", 2024-09-04T17:48:45.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000384946 2024-09-04T17:48:45.724 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:45.724 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:45.724 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:45.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.724 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:96175 3.12 3:4d4471be:::rbd_data.1134c4e2d26.000000000000006e:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123)", 2024-09-04T17:48:45.724 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:48:02.919542+0000", 2024-09-04T17:48:45.724 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.713685501, 2024-09-04T17:48:45.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060180626000000001, 2024-09-04T17:48:45.724 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:45.724 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:45.724 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:45.724 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:45.725 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:45.725 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:45.725 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 96175 2024-09-04T17:48:45.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.725 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:45.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:45.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.919542+0000", 2024-09-04T17:48:45.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:45.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:45.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.919542+0000", 2024-09-04T17:48:45.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:45.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.726 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:45.726 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.919545+0000", 2024-09-04T17:48:45.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0199999999999999e-06 2024-09-04T17:48:45.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.726 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:45.726 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.919559+0000", 2024-09-04T17:48:45.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3293e-05 2024-09-04T17:48:45.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.727 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:45.727 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.919587+0000", 2024-09-04T17:48:45.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7922000000000001e-05 2024-09-04T17:48:45.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.727 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.727 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:45.727 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.919601+0000", 2024-09-04T17:48:45.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4657e-05 2024-09-04T17:48:45.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.727 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.727 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:45.727 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.919708+0000", 2024-09-04T17:48:45.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010624999999999999 2024-09-04T17:48:45.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:45.728 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.919929+0000", 2024-09-04T17:48:45.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022147399999999999 2024-09-04T17:48:45.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2024-09-04T17:48:45.728 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.929774+0000", 2024-09-04T17:48:45.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0098449839999999993 2024-09-04T17:48:45.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:45.729 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.979511+0000", 2024-09-04T17:48:45.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.049736537999999997 2024-09-04T17:48:45.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.729 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:45.729 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.979547+0000", 2024-09-04T17:48:45.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6069000000000003e-05 2024-09-04T17:48:45.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.729 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:45.729 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.979642+0000", 2024-09-04T17:48:45.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5526999999999995e-05 2024-09-04T17:48:45.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.730 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:45.730 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.979723+0000", 2024-09-04T17:48:45.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0891999999999997e-05 2024-09-04T17:48:45.730 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:45.730 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:45.730 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:45.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.730 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:97148 3.4 3:22dd2475:::rbd_id.image_client.0-clone39:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e128)", 2024-09-04T17:48:45.730 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:48:07.831881+0000", 2024-09-04T17:48:45.730 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.801347266999997, 2024-09-04T17:48:45.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.026553585, 2024-09-04T17:48:45.731 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:45.731 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:45.731 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:45.731 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:45.731 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:45.731 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:45.731 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 97148 2024-09-04T17:48:45.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.731 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:45.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.731 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:45.731 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:07.831881+0000", 2024-09-04T17:48:45.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:45.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:45.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:07.831881+0000", 2024-09-04T17:48:45.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:45.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:45.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:07.831883+0000", 2024-09-04T17:48:45.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3369999999999998e-06 2024-09-04T17:48:45.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:45.733 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:07.831890+0000", 2024-09-04T17:48:45.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7050000000000002e-06 2024-09-04T17:48:45.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:45.733 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:07.831905+0000", 2024-09-04T17:48:45.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5512999999999999e-05 2024-09-04T17:48:45.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:45.733 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:07.831918+0000", 2024-09-04T17:48:45.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3223e-05 2024-09-04T17:48:45.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.734 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:45.734 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:07.833441+0000", 2024-09-04T17:48:45.734 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0015228920000000001 2024-09-04T17:48:45.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.734 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-09-04T17:48:45.734 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:07.833451+0000", 2024-09-04T17:48:45.734 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6600000000000007e-06 2024-09-04T17:48:45.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.734 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:45.735 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:07.837721+0000", 2024-09-04T17:48:45.735 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0042702410000000001 2024-09-04T17:48:45.735 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.735 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.735 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-04T17:48:45.735 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:07.837732+0000", 2024-09-04T17:48:45.735 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.057e-05 2024-09-04T17:48:45.735 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.735 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.735 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:45.735 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:08.856291+0000", 2024-09-04T17:48:45.735 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0185595350000001 2024-09-04T17:48:45.735 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.735 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.736 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:45.736 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:08.856503+0000", 2024-09-04T17:48:45.736 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000212127 2024-09-04T17:48:45.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.736 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2,5", 2024-09-04T17:48:45.736 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:08.856664+0000", 2024-09-04T17:48:45.736 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016072800000000001 2024-09-04T17:48:45.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.736 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:45.736 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:08.857811+0000", 2024-09-04T17:48:45.736 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001146441 2024-09-04T17:48:45.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:45.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:08.858202+0000", 2024-09-04T17:48:45.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000391605 2024-09-04T17:48:45.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:45.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:08.858239+0000", 2024-09-04T17:48:45.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7200000000000003e-05 2024-09-04T17:48:45.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:45.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:08.858287+0000", 2024-09-04T17:48:45.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7515999999999997e-05 2024-09-04T17:48:45.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:45.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:08.858434+0000", 2024-09-04T17:48:45.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014729200000000001 2024-09-04T17:48:45.738 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:45.738 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:45.738 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:45.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.738 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:99510 3.18 3:1bf8b1d0:::rbd_header.11346a2b0bc9:head [call rbd.snapshot_remove in=18b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e147)", 2024-09-04T17:48:45.739 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:48:27.032522+0000", 2024-09-04T17:48:45.739 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.600705403999999, 2024-09-04T17:48:45.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.84201522500000003, 2024-09-04T17:48:45.739 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:45.739 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:45.739 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:45.739 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:45.739 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:45.739 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:45.739 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 99510 2024-09-04T17:48:45.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.739 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:45.739 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.740 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:45.740 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032522+0000", 2024-09-04T17:48:45.740 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:45.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.740 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.740 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:45.740 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032522+0000", 2024-09-04T17:48:45.740 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:45.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.740 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.740 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:45.740 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032525+0000", 2024-09-04T17:48:45.740 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4789999999999999e-06 2024-09-04T17:48:45.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:45.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032536+0000", 2024-09-04T17:48:45.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0557000000000001e-05 2024-09-04T17:48:45.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:45.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032555+0000", 2024-09-04T17:48:45.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9782000000000001e-05 2024-09-04T17:48:45.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:45.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032569+0000", 2024-09-04T17:48:45.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4012000000000001e-05 2024-09-04T17:48:45.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:45.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032649+0000", 2024-09-04T17:48:45.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9782000000000003e-05 2024-09-04T17:48:45.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-09-04T17:48:45.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032657+0000", 2024-09-04T17:48:45.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7999999999999999e-06 2024-09-04T17:48:45.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.743 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:45.743 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.035521+0000", 2024-09-04T17:48:45.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028640850000000002 2024-09-04T17:48:45.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.743 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-04T17:48:45.743 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.035534+0000", 2024-09-04T17:48:45.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3157e-05 2024-09-04T17:48:45.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.743 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:45.743 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.869836+0000", 2024-09-04T17:48:45.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.83430168400000004 2024-09-04T17:48:45.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.744 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:45.744 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.870070+0000", 2024-09-04T17:48:45.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023443300000000001 2024-09-04T17:48:45.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.744 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 3", 2024-09-04T17:48:45.744 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.870423+0000", 2024-09-04T17:48:45.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00035290700000000001 2024-09-04T17:48:45.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:45.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.871254+0000", 2024-09-04T17:48:45.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00083113700000000002 2024-09-04T17:48:45.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:45.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.874234+0000", 2024-09-04T17:48:45.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029795189999999999 2024-09-04T17:48:45.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:45.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.874321+0000", 2024-09-04T17:48:45.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7637000000000001e-05 2024-09-04T17:48:45.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:45.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.874538+0000", 2024-09-04T17:48:45.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021625400000000001 2024-09-04T17:48:45.746 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:45.746 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:45.746 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:45.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.746 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:99511 3.18 3:1bf8b1d0:::rbd_header.11346a2b0bc9:head [watch reconnect cookie 140018821496000 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147)", 2024-09-04T17:48:45.746 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:48:27.032602+0000", 2024-09-04T17:48:45.746 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.600625601000001, 2024-09-04T17:48:45.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.84736467999999998, 2024-09-04T17:48:45.747 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:45.747 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:45.747 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:45.747 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:45.747 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:45.747 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:45.747 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 99511 2024-09-04T17:48:45.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.747 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:45.747 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.747 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:45.747 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032602+0000", 2024-09-04T17:48:45.747 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:45.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.748 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.748 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:45.748 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032602+0000", 2024-09-04T17:48:45.748 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:45.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.748 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.748 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:45.748 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032603+0000", 2024-09-04T17:48:45.748 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3300000000000001e-07 2024-09-04T17:48:45.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.748 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.748 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:45.749 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032610+0000", 2024-09-04T17:48:45.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.754e-06 2024-09-04T17:48:45.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.749 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:45.749 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032621+0000", 2024-09-04T17:48:45.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1175999999999999e-05 2024-09-04T17:48:45.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.749 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:45.749 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032630+0000", 2024-09-04T17:48:45.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7900000000000005e-06 2024-09-04T17:48:45.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:45.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032900+0000", 2024-09-04T17:48:45.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026967499999999998 2024-09-04T17:48:45.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting_for_map not empty", 2024-09-04T17:48:45.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032907+0000", 2024-09-04T17:48:45.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6969999999999993e-06 2024-09-04T17:48:45.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:45.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.036096+0000", 2024-09-04T17:48:45.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031887090000000001 2024-09-04T17:48:45.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-04T17:48:45.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.036107+0000", 2024-09-04T17:48:45.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1361e-05 2024-09-04T17:48:45.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:45.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.870783+0000", 2024-09-04T17:48:45.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.834675265 2024-09-04T17:48:45.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-09-04T17:48:45.752 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.870943+0000", 2024-09-04T17:48:45.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000160364 2024-09-04T17:48:45.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.752 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:45.752 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.874588+0000", 2024-09-04T17:48:45.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0036446529999999999 2024-09-04T17:48:45.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.752 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:45.752 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.874731+0000", 2024-09-04T17:48:45.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014334499999999999 2024-09-04T17:48:45.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:45.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:45.753 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:45.753 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.879967+0000", 2024-09-04T17:48:45.753 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005236058 2024-09-04T17:48:45.753 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:45.753 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:45.753 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:45.753 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:45.753 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:45.753 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:45.754 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-04T17:48:45.755 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:45.753+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:45.857 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:48:45.851+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:48:45.925 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:45.925 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:48:45.925 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:48:45.925 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:45.939 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-04T17:48:45.965 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-04T17:48:45.982 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:45.979+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:46.059 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:48:46.057+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:48:46.119 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:46.119 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:48:46.119 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:48:46.120 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:48:46.120 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:46.135 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-04T17:48:46.159 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:46.157+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:46.252 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:46.252 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:48:46.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:48:46.252 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:48:46.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.252 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59950 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5365760~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:48:46.253 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654917+0000", 2024-09-04T17:48:46.253 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 92.596194745999995, 2024-09-04T17:48:46.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066822450000000005, 2024-09-04T17:48:46.253 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:46.253 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:46.253 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:46.253 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:46.253 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:46.253 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:46.253 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59950 2024-09-04T17:48:46.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.253 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:46.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:46.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654917+0000", 2024-09-04T17:48:46.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:46.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:46.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654917+0000", 2024-09-04T17:48:46.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:46.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:46.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654918+0000", 2024-09-04T17:48:46.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.29e-07 2024-09-04T17:48:46.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:46.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654919+0000", 2024-09-04T17:48:46.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2589999999999999e-06 2024-09-04T17:48:46.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:46.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654922+0000", 2024-09-04T17:48:46.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1350000000000001e-06 2024-09-04T17:48:46.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:46.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654925+0000", 2024-09-04T17:48:46.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1250000000000001e-06 2024-09-04T17:48:46.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:46.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721466+0000", 2024-09-04T17:48:46.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066540959999999996 2024-09-04T17:48:46.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:46.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721574+0000", 2024-09-04T17:48:46.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010747399999999999 2024-09-04T17:48:46.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:46.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721740+0000", 2024-09-04T17:48:46.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016626799999999999 2024-09-04T17:48:46.257 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:46.257 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:46.257 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:46.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.258 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59951 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5472256~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:48:46.258 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654929+0000", 2024-09-04T17:48:46.258 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 92.596183561000004, 2024-09-04T17:48:46.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067110534999999999, 2024-09-04T17:48:46.258 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:46.258 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:46.258 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:46.258 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:46.258 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:46.258 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:46.258 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59951 2024-09-04T17:48:46.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.258 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:46.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:46.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654929+0000", 2024-09-04T17:48:46.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:46.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:46.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654929+0000", 2024-09-04T17:48:46.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:46.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:46.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654929+0000", 2024-09-04T17:48:46.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5100000000000001e-07 2024-09-04T17:48:46.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:46.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654930+0000", 2024-09-04T17:48:46.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2440000000000001e-06 2024-09-04T17:48:46.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:46.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654933+0000", 2024-09-04T17:48:46.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1599999999999998e-06 2024-09-04T17:48:46.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:46.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654936+0000", 2024-09-04T17:48:46.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6560000000000001e-06 2024-09-04T17:48:46.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:46.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721789+0000", 2024-09-04T17:48:46.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066853250000000003 2024-09-04T17:48:46.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:46.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721885+0000", 2024-09-04T17:48:46.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5616000000000001e-05 2024-09-04T17:48:46.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:46.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722039+0000", 2024-09-04T17:48:46.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000154258 2024-09-04T17:48:46.262 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:46.262 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:46.262 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:46.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.262 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59952 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5505024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:48:46.263 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654939+0000", 2024-09-04T17:48:46.263 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 92.596172683999995, 2024-09-04T17:48:46.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067433742000000005, 2024-09-04T17:48:46.263 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:46.263 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:46.263 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:46.263 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:46.263 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:46.263 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:46.263 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59952 2024-09-04T17:48:46.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.263 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:46.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:46.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654939+0000", 2024-09-04T17:48:46.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:46.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:46.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654939+0000", 2024-09-04T17:48:46.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:46.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:46.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654940+0000", 2024-09-04T17:48:46.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3300000000000001e-07 2024-09-04T17:48:46.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:46.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654941+0000", 2024-09-04T17:48:46.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1570000000000001e-06 2024-09-04T17:48:46.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:46.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654944+0000", 2024-09-04T17:48:46.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1319999999999998e-06 2024-09-04T17:48:46.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:46.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654947+0000", 2024-09-04T17:48:46.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5969999999999999e-06 2024-09-04T17:48:46.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:46.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722089+0000", 2024-09-04T17:48:46.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067142347000000005 2024-09-04T17:48:46.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:46.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722201+0000", 2024-09-04T17:48:46.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011235600000000001 2024-09-04T17:48:46.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:46.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722373+0000", 2024-09-04T17:48:46.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017191999999999999 2024-09-04T17:48:46.269 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:46.269 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:46.269 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:46.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.270 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59953 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5611520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:48:46.270 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654950+0000", 2024-09-04T17:48:46.270 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 92.596162043999996, 2024-09-04T17:48:46.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067601852000000004, 2024-09-04T17:48:46.270 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:46.270 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:46.270 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:46.270 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:46.270 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:46.270 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:46.270 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59953 2024-09-04T17:48:46.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.271 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:46.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:46.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654950+0000", 2024-09-04T17:48:46.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:46.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:46.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654950+0000", 2024-09-04T17:48:46.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:46.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:46.272 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654950+0000", 2024-09-04T17:48:46.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2499999999999999e-07 2024-09-04T17:48:46.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:46.272 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654954+0000", 2024-09-04T17:48:46.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.287e-06 2024-09-04T17:48:46.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:46.272 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654962+0000", 2024-09-04T17:48:46.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4230000000000005e-06 2024-09-04T17:48:46.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:46.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654966+0000", 2024-09-04T17:48:46.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6449999999999998e-06 2024-09-04T17:48:46.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:46.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722417+0000", 2024-09-04T17:48:46.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067451188999999995 2024-09-04T17:48:46.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:46.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722467+0000", 2024-09-04T17:48:46.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0556999999999997e-05 2024-09-04T17:48:46.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:46.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722552+0000", 2024-09-04T17:48:46.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4525999999999997e-05 2024-09-04T17:48:46.274 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:46.274 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:46.274 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:46.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.274 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59954 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5718016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:48:46.275 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654970+0000", 2024-09-04T17:48:46.275 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 92.596142283000006, 2024-09-04T17:48:46.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067874285000000006, 2024-09-04T17:48:46.275 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:46.275 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:46.275 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:46.275 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:46.275 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:46.275 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:46.275 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59954 2024-09-04T17:48:46.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.275 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:46.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:46.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654970+0000", 2024-09-04T17:48:46.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:46.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:46.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654970+0000", 2024-09-04T17:48:46.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:46.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:46.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654970+0000", 2024-09-04T17:48:46.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5899999999999998e-07 2024-09-04T17:48:46.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:46.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654971+0000", 2024-09-04T17:48:46.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2929999999999999e-06 2024-09-04T17:48:46.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:46.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654975+0000", 2024-09-04T17:48:46.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4699999999999998e-06 2024-09-04T17:48:46.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:46.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654978+0000", 2024-09-04T17:48:46.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7930000000000002e-06 2024-09-04T17:48:46.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:46.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722600+0000", 2024-09-04T17:48:46.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067622229000000006 2024-09-04T17:48:46.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:46.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722689+0000", 2024-09-04T17:48:46.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9511000000000004e-05 2024-09-04T17:48:46.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:46.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722844+0000", 2024-09-04T17:48:46.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015473 2024-09-04T17:48:46.279 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:46.279 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:46.279 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:46.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.280 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59955 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5857280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:48:46.280 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654981+0000", 2024-09-04T17:48:46.280 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 92.596130932999998, 2024-09-04T17:48:46.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068052360000000006, 2024-09-04T17:48:46.280 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:46.280 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:46.280 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:46.280 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:46.280 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:46.280 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:46.280 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59955 2024-09-04T17:48:46.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.281 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:46.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:46.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654981+0000", 2024-09-04T17:48:46.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:46.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:46.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654981+0000", 2024-09-04T17:48:46.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:46.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:46.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654981+0000", 2024-09-04T17:48:46.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2399999999999999e-07 2024-09-04T17:48:46.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:46.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654985+0000", 2024-09-04T17:48:46.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0980000000000004e-06 2024-09-04T17:48:46.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:46.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654990+0000", 2024-09-04T17:48:46.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3429999999999998e-06 2024-09-04T17:48:46.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:46.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654993+0000", 2024-09-04T17:48:46.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8700000000000001e-06 2024-09-04T17:48:46.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:46.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722888+0000", 2024-09-04T17:48:46.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067895239999999996 2024-09-04T17:48:46.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:46.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722948+0000", 2024-09-04T17:48:46.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9750999999999998e-05 2024-09-04T17:48:46.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:46.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.723034+0000", 2024-09-04T17:48:46.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5834000000000005e-05 2024-09-04T17:48:46.284 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:46.284 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:46.284 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:46.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.284 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59956 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5963776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:48:46.284 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654998+0000", 2024-09-04T17:48:46.284 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 92.596114404000005, 2024-09-04T17:48:46.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068359518999999994, 2024-09-04T17:48:46.285 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:46.285 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:46.285 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:46.285 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:46.285 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:46.285 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:46.285 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59956 2024-09-04T17:48:46.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.285 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:46.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:46.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654998+0000", 2024-09-04T17:48:46.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:46.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:46.286 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654998+0000", 2024-09-04T17:48:46.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:46.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:46.286 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654998+0000", 2024-09-04T17:48:46.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4299999999999999e-07 2024-09-04T17:48:46.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:46.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654999+0000", 2024-09-04T17:48:46.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1799999999999999e-06 2024-09-04T17:48:46.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:46.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.655002+0000", 2024-09-04T17:48:46.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3639999999999999e-06 2024-09-04T17:48:46.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:46.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.655005+0000", 2024-09-04T17:48:46.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7970000000000001e-06 2024-09-04T17:48:46.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:46.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.723085+0000", 2024-09-04T17:48:46.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068080229000000006 2024-09-04T17:48:46.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:46.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.723197+0000", 2024-09-04T17:48:46.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000111344 2024-09-04T17:48:46.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:46.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.723357+0000", 2024-09-04T17:48:46.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016036199999999999 2024-09-04T17:48:46.289 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:46.289 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:46.289 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:46.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.289 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61082 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 2654208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:46.289 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.822664+0000", 2024-09-04T17:48:46.289 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 92.428448016999994, 2024-09-04T17:48:46.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067072242000000004, 2024-09-04T17:48:46.289 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:46.289 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:46.289 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:46.289 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:46.290 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:46.290 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:46.290 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61082 2024-09-04T17:48:46.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.290 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:46.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:46.290 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822664+0000", 2024-09-04T17:48:46.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:46.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:46.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822664+0000", 2024-09-04T17:48:46.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:46.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:46.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822664+0000", 2024-09-04T17:48:46.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6600000000000003e-07 2024-09-04T17:48:46.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:46.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822666+0000", 2024-09-04T17:48:46.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.536e-06 2024-09-04T17:48:46.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:46.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822669+0000", 2024-09-04T17:48:46.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3759999999999999e-06 2024-09-04T17:48:46.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:46.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822672+0000", 2024-09-04T17:48:46.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8109999999999999e-06 2024-09-04T17:48:46.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:46.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889263+0000", 2024-09-04T17:48:46.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066591259999999999 2024-09-04T17:48:46.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:46.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889369+0000", 2024-09-04T17:48:46.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000105362 2024-09-04T17:48:46.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:46.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889736+0000", 2024-09-04T17:48:46.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00036763099999999999 2024-09-04T17:48:46.294 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:46.294 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:46.294 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:46.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.294 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61083 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 2760704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:46.294 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.822678+0000", 2024-09-04T17:48:46.295 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 92.428434283000001, 2024-09-04T17:48:46.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067475996999999996, 2024-09-04T17:48:46.295 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:46.295 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:46.295 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:46.295 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:46.295 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:46.295 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:46.295 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61083 2024-09-04T17:48:46.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.295 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:46.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:46.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822678+0000", 2024-09-04T17:48:46.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:46.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.296 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:46.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822678+0000", 2024-09-04T17:48:46.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:46.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.296 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:46.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822725+0000", 2024-09-04T17:48:46.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7429000000000003e-05 2024-09-04T17:48:46.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:46.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822728+0000", 2024-09-04T17:48:46.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2140000000000001e-06 2024-09-04T17:48:46.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:46.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822737+0000", 2024-09-04T17:48:46.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4370000000000007e-06 2024-09-04T17:48:46.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:46.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822743+0000", 2024-09-04T17:48:46.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7520000000000002e-06 2024-09-04T17:48:46.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:46.298 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889785+0000", 2024-09-04T17:48:46.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067042265000000004 2024-09-04T17:48:46.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:46.298 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889840+0000", 2024-09-04T17:48:46.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5254000000000003e-05 2024-09-04T17:48:46.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:46.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.890154+0000", 2024-09-04T17:48:46.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000313646 2024-09-04T17:48:46.299 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:46.299 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:46.299 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:46.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.299 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61084 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 2867200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:46.299 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.822753+0000", 2024-09-04T17:48:46.299 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 92.428359346999997, 2024-09-04T17:48:46.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068497161000000001, 2024-09-04T17:48:46.299 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:46.299 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:46.300 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:46.300 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:46.300 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:46.300 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:46.300 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61084 2024-09-04T17:48:46.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.300 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:46.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.300 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:46.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822753+0000", 2024-09-04T17:48:46.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:46.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:46.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822753+0000", 2024-09-04T17:48:46.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:46.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:46.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822753+0000", 2024-09-04T17:48:46.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9100000000000003e-07 2024-09-04T17:48:46.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:46.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822756+0000", 2024-09-04T17:48:46.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.678e-06 2024-09-04T17:48:46.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:46.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822763+0000", 2024-09-04T17:48:46.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8220000000000001e-06 2024-09-04T17:48:46.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:46.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822768+0000", 2024-09-04T17:48:46.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1930000000000001e-06 2024-09-04T17:48:46.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:46.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.890246+0000", 2024-09-04T17:48:46.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067477996999999998 2024-09-04T17:48:46.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.303 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:46.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.890555+0000", 2024-09-04T17:48:46.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000309143 2024-09-04T17:48:46.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.303 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:46.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.891250+0000", 2024-09-04T17:48:46.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00069463700000000001 2024-09-04T17:48:46.303 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:46.304 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:46.304 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:46.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.304 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61285 3.8 3:117cfeb9:::rbd_data.11341dfde0c5.0000000000000006:1f [read 2760704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:46.304 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835200+0000", 2024-09-04T17:48:46.304 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 92.415912280000001, 2024-09-04T17:48:46.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066899803999999993, 2024-09-04T17:48:46.304 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:46.304 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:46.304 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:46.304 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:46.304 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:46.305 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:46.305 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61285 2024-09-04T17:48:46.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.305 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:46.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:46.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835200+0000", 2024-09-04T17:48:46.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:46.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:46.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835200+0000", 2024-09-04T17:48:46.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:46.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:46.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835200+0000", 2024-09-04T17:48:46.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0899999999999997e-07 2024-09-04T17:48:46.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:46.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835206+0000", 2024-09-04T17:48:46.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3280000000000002e-06 2024-09-04T17:48:46.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:46.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835211+0000", 2024-09-04T17:48:46.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9230000000000001e-06 2024-09-04T17:48:46.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:46.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835216+0000", 2024-09-04T17:48:46.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9019999999999998e-06 2024-09-04T17:48:46.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:46.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.901719+0000", 2024-09-04T17:48:46.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066502981000000003 2024-09-04T17:48:46.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:46.308 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.901813+0000", 2024-09-04T17:48:46.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3905000000000003e-05 2024-09-04T17:48:46.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.308 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:46.308 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902100+0000", 2024-09-04T17:48:46.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028645599999999998 2024-09-04T17:48:46.308 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:46.309 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:46.309 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:46.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.309 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61286 3.8 3:117cfeb9:::rbd_data.11341dfde0c5.0000000000000006:1f [read 2867200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:46.309 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835244+0000", 2024-09-04T17:48:46.309 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 92.415867660999993, 2024-09-04T17:48:46.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067143215000000006, 2024-09-04T17:48:46.309 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:46.309 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:46.309 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:46.309 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:46.310 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:46.310 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:46.310 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61286 2024-09-04T17:48:46.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.310 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:46.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:46.310 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835244+0000", 2024-09-04T17:48:46.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:46.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:46.310 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835244+0000", 2024-09-04T17:48:46.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:46.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:46.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835245+0000", 2024-09-04T17:48:46.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8400000000000004e-07 2024-09-04T17:48:46.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:46.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835248+0000", 2024-09-04T17:48:46.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6850000000000001e-06 2024-09-04T17:48:46.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:46.312 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835254+0000", 2024-09-04T17:48:46.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.494e-06 2024-09-04T17:48:46.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.312 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:46.312 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835260+0000", 2024-09-04T17:48:46.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0580000000000003e-06 2024-09-04T17:48:46.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.312 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:46.312 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902119+0000", 2024-09-04T17:48:46.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066858318 2024-09-04T17:48:46.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:46.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902163+0000", 2024-09-04T17:48:46.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4753000000000003e-05 2024-09-04T17:48:46.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:46.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902388+0000", 2024-09-04T17:48:46.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022422300000000001 2024-09-04T17:48:46.313 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:46.313 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:46.313 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:46.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.314 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61315 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 98304~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:46.314 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835269+0000", 2024-09-04T17:48:46.314 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 92.415842869000002, 2024-09-04T17:48:46.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067329396, 2024-09-04T17:48:46.314 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:46.314 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:46.314 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:46.314 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:46.314 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:46.314 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:46.314 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61315 2024-09-04T17:48:46.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.315 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:46.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.315 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:46.315 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835269+0000", 2024-09-04T17:48:46.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:46.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.315 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:46.315 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835269+0000", 2024-09-04T17:48:46.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:46.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.315 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:46.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835270+0000", 2024-09-04T17:48:46.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6599999999999996e-07 2024-09-04T17:48:46.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:46.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835272+0000", 2024-09-04T17:48:46.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.261e-06 2024-09-04T17:48:46.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:46.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835278+0000", 2024-09-04T17:48:46.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.443e-06 2024-09-04T17:48:46.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:46.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835284+0000", 2024-09-04T17:48:46.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3290000000000001e-06 2024-09-04T17:48:46.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:46.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902147+0000", 2024-09-04T17:48:46.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066862695999999999 2024-09-04T17:48:46.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:46.318 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902248+0000", 2024-09-04T17:48:46.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000101501 2024-09-04T17:48:46.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.318 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:46.318 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902599+0000", 2024-09-04T17:48:46.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00035060000000000001 2024-09-04T17:48:46.318 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:46.318 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:46.318 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:46.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.318 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61316 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 204800~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:46.318 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835291+0000", 2024-09-04T17:48:46.319 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 92.415820850000003, 2024-09-04T17:48:46.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067632052999999998, 2024-09-04T17:48:46.319 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:46.319 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:46.319 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:46.319 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:46.319 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:46.319 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:46.319 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61316 2024-09-04T17:48:46.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.319 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:46.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:46.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835291+0000", 2024-09-04T17:48:46.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:46.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:46.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835291+0000", 2024-09-04T17:48:46.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:46.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:46.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835292+0000", 2024-09-04T17:48:46.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1899999999999998e-07 2024-09-04T17:48:46.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:46.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835294+0000", 2024-09-04T17:48:46.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0880000000000002e-06 2024-09-04T17:48:46.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:46.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835300+0000", 2024-09-04T17:48:46.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.224e-06 2024-09-04T17:48:46.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:46.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835305+0000", 2024-09-04T17:48:46.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0730000000000004e-06 2024-09-04T17:48:46.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:46.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902632+0000", 2024-09-04T17:48:46.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067327322999999994 2024-09-04T17:48:46.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:46.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902688+0000", 2024-09-04T17:48:46.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5177999999999999e-05 2024-09-04T17:48:46.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.323 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:46.323 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902923+0000", 2024-09-04T17:48:46.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023574800000000001 2024-09-04T17:48:46.323 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:46.323 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:46.323 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:46.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.323 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61317 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 344064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:46.323 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835312+0000", 2024-09-04T17:48:46.323 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 92.415799845999999, 2024-09-04T17:48:46.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067918511000000001, 2024-09-04T17:48:46.323 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:46.324 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:46.324 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:46.324 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:46.324 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:46.324 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:46.324 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61317 2024-09-04T17:48:46.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.324 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:46.324 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.324 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:46.324 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835312+0000", 2024-09-04T17:48:46.324 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:46.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:46.325 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835312+0000", 2024-09-04T17:48:46.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:46.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:46.325 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835313+0000", 2024-09-04T17:48:46.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9000000000000002e-07 2024-09-04T17:48:46.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:46.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835315+0000", 2024-09-04T17:48:46.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1689999999999999e-06 2024-09-04T17:48:46.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:46.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835321+0000", 2024-09-04T17:48:46.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0619999999999997e-06 2024-09-04T17:48:46.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:46.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835326+0000", 2024-09-04T17:48:46.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0390000000000004e-06 2024-09-04T17:48:46.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:46.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902967+0000", 2024-09-04T17:48:46.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067641520999999996 2024-09-04T17:48:46.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:46.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903067+0000", 2024-09-04T17:48:46.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9426999999999995e-05 2024-09-04T17:48:46.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.328 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.328 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:46.328 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903231+0000", 2024-09-04T17:48:46.328 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016390299999999999 2024-09-04T17:48:46.328 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:46.328 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:46.328 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:46.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.328 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.328 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61318 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 450560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:46.328 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835334+0000", 2024-09-04T17:48:46.328 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 92.415778329000005, 2024-09-04T17:48:46.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068091771999999995, 2024-09-04T17:48:46.329 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:46.329 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:46.329 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:46.329 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:46.329 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:46.329 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:46.329 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61318 2024-09-04T17:48:46.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.329 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:46.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.329 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:46.329 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835334+0000", 2024-09-04T17:48:46.330 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:46.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.330 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:46.330 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835334+0000", 2024-09-04T17:48:46.330 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:46.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.330 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:46.330 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835334+0000", 2024-09-04T17:48:46.330 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5699999999999998e-07 2024-09-04T17:48:46.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.330 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:46.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835337+0000", 2024-09-04T17:48:46.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3889999999999999e-06 2024-09-04T17:48:46.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.331 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:46.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835343+0000", 2024-09-04T17:48:46.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1809999999999998e-06 2024-09-04T17:48:46.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.331 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:46.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835348+0000", 2024-09-04T17:48:46.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3170000000000001e-06 2024-09-04T17:48:46.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.332 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.332 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:46.332 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903270+0000", 2024-09-04T17:48:46.332 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067922191000000007 2024-09-04T17:48:46.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.332 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.332 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:46.332 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903321+0000", 2024-09-04T17:48:46.332 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0889e-05 2024-09-04T17:48:46.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.332 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.332 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:46.333 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903426+0000", 2024-09-04T17:48:46.333 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104348 2024-09-04T17:48:46.333 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:46.333 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:46.333 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:46.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.333 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.333 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61319 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 589824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:46.333 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835357+0000", 2024-09-04T17:48:46.333 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 92.415755512000004, 2024-09-04T17:48:46.333 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068372535999999998, 2024-09-04T17:48:46.333 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:46.333 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:46.334 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:46.334 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:46.334 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:46.334 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:46.334 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61319 2024-09-04T17:48:46.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.334 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:46.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.334 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:46.334 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835357+0000", 2024-09-04T17:48:46.334 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:46.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.335 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:46.335 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835357+0000", 2024-09-04T17:48:46.335 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:46.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.335 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:46.335 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835357+0000", 2024-09-04T17:48:46.335 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.32e-07 2024-09-04T17:48:46.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.336 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.336 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:46.336 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835359+0000", 2024-09-04T17:48:46.336 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1660000000000001e-06 2024-09-04T17:48:46.336 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.336 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.336 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:46.336 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835366+0000", 2024-09-04T17:48:46.336 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3550000000000001e-06 2024-09-04T17:48:46.336 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.336 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.336 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:46.336 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835371+0000", 2024-09-04T17:48:46.337 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1050000000000003e-06 2024-09-04T17:48:46.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.337 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:46.337 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903471+0000", 2024-09-04T17:48:46.337 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068100451000000006 2024-09-04T17:48:46.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.337 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:46.337 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903573+0000", 2024-09-04T17:48:46.337 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000101691 2024-09-04T17:48:46.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.338 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:46.338 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903729+0000", 2024-09-04T17:48:46.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000156336 2024-09-04T17:48:46.338 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:46.338 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:46.338 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:46.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.338 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61320 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:46.338 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835380+0000", 2024-09-04T17:48:46.338 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 92.415731746999995, 2024-09-04T17:48:46.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068543058000000004, 2024-09-04T17:48:46.339 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:46.339 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:46.339 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:46.339 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:46.339 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:46.339 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:46.339 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61320 2024-09-04T17:48:46.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.339 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:46.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.339 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:46.339 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835380+0000", 2024-09-04T17:48:46.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:46.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.340 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:46.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835380+0000", 2024-09-04T17:48:46.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:46.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.340 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:46.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835381+0000", 2024-09-04T17:48:46.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9999999999999998e-07 2024-09-04T17:48:46.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.340 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:46.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835383+0000", 2024-09-04T17:48:46.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2230000000000001e-06 2024-09-04T17:48:46.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.341 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:46.341 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835390+0000", 2024-09-04T17:48:46.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.34e-06 2024-09-04T17:48:46.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.341 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:46.341 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835396+0000", 2024-09-04T17:48:46.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9830000000000001e-06 2024-09-04T17:48:46.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.341 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:46.341 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903765+0000", 2024-09-04T17:48:46.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068368441000000002 2024-09-04T17:48:46.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.342 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:46.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903835+0000", 2024-09-04T17:48:46.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0631999999999997e-05 2024-09-04T17:48:46.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.342 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:46.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903923+0000", 2024-09-04T17:48:46.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8039000000000002e-05 2024-09-04T17:48:46.342 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:46.342 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:46.342 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:46.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.343 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61321 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 835584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:46.343 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835404+0000", 2024-09-04T17:48:46.343 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 92.415707687999998, 2024-09-04T17:48:46.343 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068911043000000005, 2024-09-04T17:48:46.343 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:46.343 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:46.343 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:46.343 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:46.343 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:46.343 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:46.343 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61321 2024-09-04T17:48:46.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.344 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:46.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.344 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:46.344 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835404+0000", 2024-09-04T17:48:46.344 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:46.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.344 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:46.344 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835404+0000", 2024-09-04T17:48:46.344 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:46.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.344 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:46.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835405+0000", 2024-09-04T17:48:46.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.89e-07 2024-09-04T17:48:46.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.345 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:46.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835407+0000", 2024-09-04T17:48:46.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4439999999999998e-06 2024-09-04T17:48:46.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.345 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:46.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835414+0000", 2024-09-04T17:48:46.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4930000000000003e-06 2024-09-04T17:48:46.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:46.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835419+0000", 2024-09-04T17:48:46.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8919999999999999e-06 2024-09-04T17:48:46.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:46.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903957+0000", 2024-09-04T17:48:46.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068538500000000002 2024-09-04T17:48:46.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:46.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904019+0000", 2024-09-04T17:48:46.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1419000000000003e-05 2024-09-04T17:48:46.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:46.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904315+0000", 2024-09-04T17:48:46.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00029680599999999999 2024-09-04T17:48:46.347 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:46.347 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:46.347 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:46.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.347 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61322 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:46.348 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835427+0000", 2024-09-04T17:48:46.348 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 92.415685302, 2024-09-04T17:48:46.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.069550138999999997, 2024-09-04T17:48:46.348 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:46.348 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:46.348 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:46.348 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:46.348 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:46.348 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:46.348 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61322 2024-09-04T17:48:46.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.348 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:46.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:46.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835427+0000", 2024-09-04T17:48:46.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:46.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:46.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835427+0000", 2024-09-04T17:48:46.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:46.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:46.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.842157+0000", 2024-09-04T17:48:46.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067298419999999998 2024-09-04T17:48:46.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:46.350 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.842163+0000", 2024-09-04T17:48:46.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2029999999999997e-06 2024-09-04T17:48:46.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:46.350 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.842175+0000", 2024-09-04T17:48:46.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2072000000000001e-05 2024-09-04T17:48:46.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:46.350 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.842183+0000", 2024-09-04T17:48:46.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3850000000000002e-06 2024-09-04T17:48:46.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:46.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904371+0000", 2024-09-04T17:48:46.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.062187754999999997 2024-09-04T17:48:46.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:46.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904467+0000", 2024-09-04T17:48:46.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6117000000000001e-05 2024-09-04T17:48:46.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:46.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:46.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:46.352 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904977+0000", 2024-09-04T17:48:46.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00050976500000000002 2024-09-04T17:48:46.352 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:46.352 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:46.352 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:46.352 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:46.352 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:46.352 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:46.354 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-04T17:48:46.354 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:48:46.351+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:48:46.455 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:48:46.453+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:48:46.485 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:46.485 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:48:46.486 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:48:46.486 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:46.500 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-04T17:48:46.555 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:48:46.551+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:48:46.656 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:46.655+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:46.664 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:46.664 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:48:46.664 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:48:46.664 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:48:46.664 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:46.678 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-04T17:48:46.757 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:46.755+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:46.785 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 4, 5, 6, 7, 3] out_osds: [1] dead_osds: [] live_osds: [0, 1, 4, 5, 7, 3, 2, 6] 2024-09-04T17:48:46.785 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-04T17:48:46.799 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:46.800 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:48:46.800 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:48:46.800 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:48:46.800 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.800 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:12722 3.1b 3:d8152af1:::rbd_data.1134ef3a5a7d.00000000000000bd:head [delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-04T17:48:46.800 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:16.421965+0000", 2024-09-04T17:48:46.800 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 150.37581335300001, 2024-09-04T17:48:46.800 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.890002811, 2024-09-04T17:48:46.800 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:46.800 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:46.800 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:46.800 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:46.800 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:46.801 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:46.801 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 12722 2024-09-04T17:48:46.801 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.801 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:46.801 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.801 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:46.801 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421965+0000", 2024-09-04T17:48:46.801 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:46.801 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.801 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.801 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:46.801 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421965+0000", 2024-09-04T17:48:46.801 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:46.801 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.802 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.802 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:46.802 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421966+0000", 2024-09-04T17:48:46.802 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.102e-06 2024-09-04T17:48:46.802 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.802 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.802 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:46.802 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421971+0000", 2024-09-04T17:48:46.802 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9989999999999999e-06 2024-09-04T17:48:46.802 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.802 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.802 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:46.802 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421981+0000", 2024-09-04T17:48:46.803 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.7380000000000006e-06 2024-09-04T17:48:46.803 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.803 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.803 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:46.803 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421989+0000", 2024-09-04T17:48:46.803 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.1890000000000007e-06 2024-09-04T17:48:46.803 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.803 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.803 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:46.803 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.422050+0000", 2024-09-04T17:48:46.803 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.0702000000000003e-05 2024-09-04T17:48:46.803 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.803 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.803 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for peered", 2024-09-04T17:48:46.804 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.422059+0000", 2024-09-04T17:48:46.804 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5440000000000004e-06 2024-09-04T17:48:46.804 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.804 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.804 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:46.804 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.308156+0000", 2024-09-04T17:48:46.804 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.88609676199999998 2024-09-04T17:48:46.804 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.804 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.804 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:46.804 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.308398+0000", 2024-09-04T17:48:46.804 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024215400000000001 2024-09-04T17:48:46.804 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.805 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.805 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 7", 2024-09-04T17:48:46.805 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.308795+0000", 2024-09-04T17:48:46.805 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000396704 2024-09-04T17:48:46.805 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.805 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.805 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:48:46.805 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.310831+0000", 2024-09-04T17:48:46.805 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0020364200000000002 2024-09-04T17:48:46.805 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.805 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.805 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:46.805 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.311818+0000", 2024-09-04T17:48:46.805 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00098649799999999993 2024-09-04T17:48:46.806 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.806 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.806 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:46.806 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.311905+0000", 2024-09-04T17:48:46.806 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7070999999999994e-05 2024-09-04T17:48:46.806 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.806 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.806 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:46.806 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.311968+0000", 2024-09-04T17:48:46.806 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.2928000000000005e-05 2024-09-04T17:48:46.806 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:46.806 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:46.806 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:46.806 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.807 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.807 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37359 3.f e58/24 3:f29e50a5:::rbd_data.1134c5a43465.0000000000000021:head v 58'135, mlcod=58'133)", 2024-09-04T17:48:46.807 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:48:46.807 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 127.18048274900001, 2024-09-04T17:48:46.807 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053483197000000003, 2024-09-04T17:48:46.807 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:46.807 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:46.807 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:46.807 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:46.807 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.807 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:46.807 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:48:46.807 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:46.808 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.808 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.808 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:46.808 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:48:46.808 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:46.808 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.808 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.808 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:46.808 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:48:46.808 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1699999999999997e-07 2024-09-04T17:48:46.808 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.808 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.808 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:46.808 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617338+0000", 2024-09-04T17:48:46.809 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.1168999999999998e-05 2024-09-04T17:48:46.809 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.809 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.809 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:46.809 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617351+0000", 2024-09-04T17:48:46.809 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3094e-05 2024-09-04T17:48:46.809 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.809 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.809 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:46.809 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617359+0000", 2024-09-04T17:48:46.809 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7970000000000006e-06 2024-09-04T17:48:46.809 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.809 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.809 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:46.810 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617417+0000", 2024-09-04T17:48:46.810 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7079e-05 2024-09-04T17:48:46.810 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.810 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.810 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:46.810 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617438+0000", 2024-09-04T17:48:46.810 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1654999999999998e-05 2024-09-04T17:48:46.810 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.810 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.810 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:46.810 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670738+0000", 2024-09-04T17:48:46.810 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053300142000000002 2024-09-04T17:48:46.810 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.811 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.811 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:46.811 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670779+0000", 2024-09-04T17:48:46.811 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0343999999999999e-05 2024-09-04T17:48:46.811 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:46.811 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:46.811 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:46.811 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.811 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.811 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37358 3.1d e58/24 3:bd3215f3:::rbd_data.1134c5a43465.0000000000000020:head v 58'91, mlcod=58'89)", 2024-09-04T17:48:46.812 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617502+0000", 2024-09-04T17:48:46.812 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 127.18027603100001, 2024-09-04T17:48:46.812 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053505641999999999, 2024-09-04T17:48:46.812 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:46.812 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:46.812 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:46.812 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:46.812 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.812 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:46.812 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617502+0000", 2024-09-04T17:48:46.812 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:46.812 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.812 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.812 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:46.813 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617502+0000", 2024-09-04T17:48:46.813 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:46.813 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.813 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.813 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:46.813 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617503+0000", 2024-09-04T17:48:46.813 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.184e-06 2024-09-04T17:48:46.813 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.813 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.813 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:46.813 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617526+0000", 2024-09-04T17:48:46.813 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2099999999999998e-05 2024-09-04T17:48:46.813 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.813 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.814 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:46.814 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617540+0000", 2024-09-04T17:48:46.814 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4741e-05 2024-09-04T17:48:46.814 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.814 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.814 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:46.814 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617550+0000", 2024-09-04T17:48:46.814 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4259999999999992e-06 2024-09-04T17:48:46.815 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.815 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.815 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:46.815 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617613+0000", 2024-09-04T17:48:46.815 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.3625000000000002e-05 2024-09-04T17:48:46.815 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.815 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.816 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:46.816 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617637+0000", 2024-09-04T17:48:46.816 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.3604999999999998e-05 2024-09-04T17:48:46.816 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.816 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.816 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:46.816 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670967+0000", 2024-09-04T17:48:46.816 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053329881000000003 2024-09-04T17:48:46.816 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.817 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.817 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:46.817 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671008+0000", 2024-09-04T17:48:46.817 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.108e-05 2024-09-04T17:48:46.817 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:46.817 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:46.817 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:46.817 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.817 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.817 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37361 3.1c e58/24 3:3c7f8976:::rbd_data.1134c5a43465.0000000000000023:head v 58'159, mlcod=58'157)", 2024-09-04T17:48:46.817 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617578+0000", 2024-09-04T17:48:46.817 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 127.180200181, 2024-09-04T17:48:46.817 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053990367999999997, 2024-09-04T17:48:46.818 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:46.818 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:46.818 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:46.818 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:46.818 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.818 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:46.818 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617578+0000", 2024-09-04T17:48:46.818 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:46.818 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.818 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.818 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:46.818 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617578+0000", 2024-09-04T17:48:46.818 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:46.819 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.819 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.819 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:46.819 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617579+0000", 2024-09-04T17:48:46.819 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7299999999999996e-07 2024-09-04T17:48:46.819 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.819 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.819 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:46.819 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617610+0000", 2024-09-04T17:48:46.819 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0951000000000002e-05 2024-09-04T17:48:46.819 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.819 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.819 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:46.820 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:48:46.820 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0002e-05 2024-09-04T17:48:46.820 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.820 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.820 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:46.820 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617626+0000", 2024-09-04T17:48:46.820 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6660000000000002e-06 2024-09-04T17:48:46.820 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.820 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.820 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:46.820 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670548+0000", 2024-09-04T17:48:46.820 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.052921943999999999 2024-09-04T17:48:46.820 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.821 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.821 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:46.821 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670574+0000", 2024-09-04T17:48:46.821 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5405000000000002e-05 2024-09-04T17:48:46.821 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.821 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.821 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:46.821 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671522+0000", 2024-09-04T17:48:46.821 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00094815299999999997 2024-09-04T17:48:46.821 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.821 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.821 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:46.821 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671568+0000", 2024-09-04T17:48:46.822 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6674000000000001e-05 2024-09-04T17:48:46.822 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:46.822 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:46.822 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:46.822 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.822 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.822 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37357 3.15 e58/49 3:a897312a:::rbd_data.1134c5a43465.000000000000001f:head v 58'117, mlcod=58'115)", 2024-09-04T17:48:46.822 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:48:46.822 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 127.180157843, 2024-09-04T17:48:46.822 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053734297, 2024-09-04T17:48:46.822 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:46.822 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:46.822 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:46.822 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:46.823 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.823 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:46.823 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:48:46.823 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:46.823 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.823 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.823 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:46.823 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:48:46.823 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:46.823 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.823 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.823 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:46.823 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617621+0000", 2024-09-04T17:48:46.824 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.7300000000000004e-07 2024-09-04T17:48:46.824 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.824 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.824 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:46.824 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617642+0000", 2024-09-04T17:48:46.824 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0185000000000001e-05 2024-09-04T17:48:46.824 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.824 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.824 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:46.824 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617654+0000", 2024-09-04T17:48:46.824 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2745000000000001e-05 2024-09-04T17:48:46.824 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.824 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.825 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:46.825 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617662+0000", 2024-09-04T17:48:46.825 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.0499999999999992e-06 2024-09-04T17:48:46.825 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.825 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.825 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:46.825 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617727+0000", 2024-09-04T17:48:46.825 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.4923000000000002e-05 2024-09-04T17:48:46.825 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.825 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.825 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:46.825 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617753+0000", 2024-09-04T17:48:46.825 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5899999999999999e-05 2024-09-04T17:48:46.826 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.826 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.826 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:46.826 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671298+0000", 2024-09-04T17:48:46.826 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053545276000000003 2024-09-04T17:48:46.826 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.826 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.826 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:46.826 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671355+0000", 2024-09-04T17:48:46.826 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6245000000000001e-05 2024-09-04T17:48:46.826 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:46.826 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:46.827 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:46.827 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.827 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.827 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44687 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:46.827 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253012+0000", 2024-09-04T17:48:46.827 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 116.54476577600001, 2024-09-04T17:48:46.827 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051128777, 2024-09-04T17:48:46.827 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:46.827 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:46.827 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:46.827 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:46.827 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:46.827 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:46.828 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44687 2024-09-04T17:48:46.828 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.828 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:46.828 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.828 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:46.828 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253012+0000", 2024-09-04T17:48:46.828 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:46.828 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.828 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.828 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:46.828 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253012+0000", 2024-09-04T17:48:46.828 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:46.828 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.829 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.829 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:46.829 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253014+0000", 2024-09-04T17:48:46.829 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.24e-06 2024-09-04T17:48:46.829 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.829 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.829 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:46.829 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253019+0000", 2024-09-04T17:48:46.829 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4569999999999996e-06 2024-09-04T17:48:46.829 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.830 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.830 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:46.830 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253035+0000", 2024-09-04T17:48:46.830 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6291999999999999e-05 2024-09-04T17:48:46.830 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.830 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.830 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:46.830 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253050+0000", 2024-09-04T17:48:46.830 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4423000000000001e-05 2024-09-04T17:48:46.830 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.830 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.830 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:46.831 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304115+0000", 2024-09-04T17:48:46.831 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051064778999999998 2024-09-04T17:48:46.831 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.831 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.831 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:46.831 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304141+0000", 2024-09-04T17:48:46.831 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6585999999999999e-05 2024-09-04T17:48:46.831 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:46.831 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:46.831 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:46.831 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.831 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.831 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44688 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:46.832 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253067+0000", 2024-09-04T17:48:46.832 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 116.544711224, 2024-09-04T17:48:46.832 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051116979, 2024-09-04T17:48:46.832 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:46.832 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:46.832 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:46.832 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:46.832 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:46.832 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:46.832 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44688 2024-09-04T17:48:46.832 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.832 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:46.832 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.833 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:46.833 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253067+0000", 2024-09-04T17:48:46.833 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:46.833 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.833 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.833 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:46.833 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253067+0000", 2024-09-04T17:48:46.833 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:46.833 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.833 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.833 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:46.833 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253068+0000", 2024-09-04T17:48:46.833 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0729999999999999e-06 2024-09-04T17:48:46.833 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.834 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.834 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:46.834 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253073+0000", 2024-09-04T17:48:46.834 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2259999999999996e-06 2024-09-04T17:48:46.834 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.834 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.834 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:46.834 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253091+0000", 2024-09-04T17:48:46.834 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7255999999999999e-05 2024-09-04T17:48:46.834 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.834 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.834 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:46.834 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253103+0000", 2024-09-04T17:48:46.835 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2632999999999999e-05 2024-09-04T17:48:46.835 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.835 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.835 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:46.835 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304173+0000", 2024-09-04T17:48:46.835 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051069537999999998 2024-09-04T17:48:46.835 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.835 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.835 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:46.835 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304184+0000", 2024-09-04T17:48:46.835 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1253000000000001e-05 2024-09-04T17:48:46.835 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:46.835 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:46.835 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:46.836 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.836 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.836 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44689 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:46.836 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253121+0000", 2024-09-04T17:48:46.836 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 116.544657087, 2024-09-04T17:48:46.836 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051192595, 2024-09-04T17:48:46.836 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:46.836 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:46.836 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:46.836 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:46.837 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:46.837 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:46.837 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44689 2024-09-04T17:48:46.837 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.837 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:46.837 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.837 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:46.837 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253121+0000", 2024-09-04T17:48:46.837 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:46.837 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.837 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.837 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:46.837 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253121+0000", 2024-09-04T17:48:46.838 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:46.838 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.838 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.838 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:46.838 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253122+0000", 2024-09-04T17:48:46.838 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.15e-06 2024-09-04T17:48:46.838 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.838 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.838 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:46.838 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253128+0000", 2024-09-04T17:48:46.838 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2569999999999998e-06 2024-09-04T17:48:46.838 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.838 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.838 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:46.839 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253145+0000", 2024-09-04T17:48:46.839 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7176e-05 2024-09-04T17:48:46.839 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.839 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.839 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:46.839 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253158+0000", 2024-09-04T17:48:46.839 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3139000000000001e-05 2024-09-04T17:48:46.839 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.839 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.839 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:46.839 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304287+0000", 2024-09-04T17:48:46.839 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051129149999999998 2024-09-04T17:48:46.839 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.839 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.840 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:46.840 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304314+0000", 2024-09-04T17:48:46.840 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6723000000000001e-05 2024-09-04T17:48:46.840 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:46.840 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:46.840 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:46.840 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.840 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.840 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44690 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:46.840 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253177+0000", 2024-09-04T17:48:46.840 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 116.544600962, 2024-09-04T17:48:46.840 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051169263, 2024-09-04T17:48:46.840 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:46.840 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:46.841 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:46.841 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:46.841 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:46.841 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:46.841 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44690 2024-09-04T17:48:46.841 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.841 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:46.841 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.841 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:46.841 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253177+0000", 2024-09-04T17:48:46.842 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:46.842 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.842 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.842 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:46.842 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253177+0000", 2024-09-04T17:48:46.842 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:46.842 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.842 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.842 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:46.842 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253178+0000", 2024-09-04T17:48:46.842 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1000000000000001e-06 2024-09-04T17:48:46.842 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.842 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.843 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:46.843 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253184+0000", 2024-09-04T17:48:46.843 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2089999999999996e-06 2024-09-04T17:48:46.843 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.843 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.843 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:46.843 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253200+0000", 2024-09-04T17:48:46.843 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6087e-05 2024-09-04T17:48:46.843 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.843 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.843 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:46.843 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253231+0000", 2024-09-04T17:48:46.843 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0874999999999998e-05 2024-09-04T17:48:46.843 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.844 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.844 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:46.844 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304337+0000", 2024-09-04T17:48:46.844 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051106430000000001 2024-09-04T17:48:46.844 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.844 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.844 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:46.844 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304347+0000", 2024-09-04T17:48:46.844 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5619999999999993e-06 2024-09-04T17:48:46.844 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:46.844 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:46.844 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:46.844 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.844 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.845 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44691 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:46.845 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253249+0000", 2024-09-04T17:48:46.845 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 116.54452892800001, 2024-09-04T17:48:46.845 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051195630999999998, 2024-09-04T17:48:46.845 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:46.845 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:46.845 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:46.845 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:46.845 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:46.845 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:46.845 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44691 2024-09-04T17:48:46.845 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.846 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:46.846 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.846 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:46.846 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253249+0000", 2024-09-04T17:48:46.846 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:46.846 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.846 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.846 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:46.846 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253249+0000", 2024-09-04T17:48:46.846 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:46.847 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.847 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.847 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:46.847 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253251+0000", 2024-09-04T17:48:46.847 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2330000000000001e-06 2024-09-04T17:48:46.847 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.847 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.847 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:46.847 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253256+0000", 2024-09-04T17:48:46.847 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6099999999999997e-06 2024-09-04T17:48:46.847 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.847 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.847 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:46.847 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253274+0000", 2024-09-04T17:48:46.848 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7952999999999999e-05 2024-09-04T17:48:46.848 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.848 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.848 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:46.848 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253287+0000", 2024-09-04T17:48:46.848 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2839e-05 2024-09-04T17:48:46.848 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.848 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.849 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:46.849 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304396+0000", 2024-09-04T17:48:46.849 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051108990999999999 2024-09-04T17:48:46.849 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.849 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.849 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:46.849 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304445+0000", 2024-09-04T17:48:46.849 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9005000000000003e-05 2024-09-04T17:48:46.849 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:46.849 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:46.849 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:46.849 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.849 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.850 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44692 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:46.850 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253337+0000", 2024-09-04T17:48:46.850 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 116.544441355, 2024-09-04T17:48:46.850 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051138909000000003, 2024-09-04T17:48:46.850 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:46.850 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:46.850 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:46.850 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:46.850 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:46.850 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:46.850 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44692 2024-09-04T17:48:46.850 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.850 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:46.851 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.851 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:46.851 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253337+0000", 2024-09-04T17:48:46.851 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:46.851 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.851 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.851 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:46.851 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253337+0000", 2024-09-04T17:48:46.851 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:46.851 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.851 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.851 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:46.851 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253338+0000", 2024-09-04T17:48:46.851 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.463e-06 2024-09-04T17:48:46.852 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.852 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.852 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:46.852 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253357+0000", 2024-09-04T17:48:46.852 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9043e-05 2024-09-04T17:48:46.852 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.852 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.852 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:46.852 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253379+0000", 2024-09-04T17:48:46.852 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1243e-05 2024-09-04T17:48:46.852 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.852 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.852 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:46.853 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253396+0000", 2024-09-04T17:48:46.853 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7059e-05 2024-09-04T17:48:46.853 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.853 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.853 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:46.853 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304466+0000", 2024-09-04T17:48:46.853 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051070624000000002 2024-09-04T17:48:46.853 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.853 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.853 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:46.853 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304476+0000", 2024-09-04T17:48:46.853 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4769999999999992e-06 2024-09-04T17:48:46.853 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:46.854 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:46.854 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:46.854 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.854 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.854 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44693 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:46.854 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253415+0000", 2024-09-04T17:48:46.854 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 116.544362952, 2024-09-04T17:48:46.854 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051141949999999999, 2024-09-04T17:48:46.854 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:46.854 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:46.854 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:46.854 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:46.854 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:46.855 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:46.855 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44693 2024-09-04T17:48:46.855 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.855 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:46.855 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.855 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:46.855 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253415+0000", 2024-09-04T17:48:46.855 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:46.855 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.855 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.855 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:46.855 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253415+0000", 2024-09-04T17:48:46.855 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:46.855 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.856 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.856 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:46.856 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253416+0000", 2024-09-04T17:48:46.856 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1799999999999999e-06 2024-09-04T17:48:46.856 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.856 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.856 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:46.856 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253422+0000", 2024-09-04T17:48:46.856 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.5940000000000001e-06 2024-09-04T17:48:46.856 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.856 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.856 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:46.856 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253438+0000", 2024-09-04T17:48:46.857 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6008999999999999e-05 2024-09-04T17:48:46.857 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.857 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.857 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:46.857 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253452+0000", 2024-09-04T17:48:46.857 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3543000000000001e-05 2024-09-04T17:48:46.857 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.857 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.857 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:46.857 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304533+0000", 2024-09-04T17:48:46.857 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051081609 2024-09-04T17:48:46.857 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.857 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.857 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:46.858 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304557+0000", 2024-09-04T17:48:46.858 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4015e-05 2024-09-04T17:48:46.858 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:46.858 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:46.858 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:46.858 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.858 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.858 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44694 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:46.858 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253469+0000", 2024-09-04T17:48:46.858 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 116.54430882699999, 2024-09-04T17:48:46.858 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051121486000000001, 2024-09-04T17:48:46.858 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:46.858 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:46.858 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:46.859 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:46.859 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:46.859 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:46.859 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44694 2024-09-04T17:48:46.859 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.859 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:46.859 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.859 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:46.859 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253469+0000", 2024-09-04T17:48:46.859 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:46.859 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.859 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.859 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:46.859 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253469+0000", 2024-09-04T17:48:46.860 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:46.860 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.860 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.860 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:46.860 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253471+0000", 2024-09-04T17:48:46.860 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1400000000000001e-06 2024-09-04T17:48:46.860 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.860 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.860 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:46.860 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253476+0000", 2024-09-04T17:48:46.860 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7370000000000003e-06 2024-09-04T17:48:46.860 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.860 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.860 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:46.861 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253493+0000", 2024-09-04T17:48:46.861 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6549e-05 2024-09-04T17:48:46.861 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.861 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.861 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:46.861 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253506+0000", 2024-09-04T17:48:46.861 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3406e-05 2024-09-04T17:48:46.861 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.861 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.861 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:46.861 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304582+0000", 2024-09-04T17:48:46.861 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051075534999999998 2024-09-04T17:48:46.861 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.861 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.862 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:46.862 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304591+0000", 2024-09-04T17:48:46.862 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1190000000000003e-06 2024-09-04T17:48:46.862 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:46.862 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:46.862 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:46.862 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.862 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.862 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44695 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:46.862 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253552+0000", 2024-09-04T17:48:46.862 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 116.544226073, 2024-09-04T17:48:46.862 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051068872000000001, 2024-09-04T17:48:46.862 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:46.863 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:46.863 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:46.863 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:46.863 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:46.863 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:46.863 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44695 2024-09-04T17:48:46.863 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.863 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:46.863 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.863 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:46.863 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253552+0000", 2024-09-04T17:48:46.863 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:46.863 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.863 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.864 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:46.864 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253552+0000", 2024-09-04T17:48:46.864 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:46.864 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.864 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.864 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:46.864 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253553+0000", 2024-09-04T17:48:46.864 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1960000000000001e-06 2024-09-04T17:48:46.864 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.864 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.865 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:46.865 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253560+0000", 2024-09-04T17:48:46.865 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.2040000000000002e-06 2024-09-04T17:48:46.865 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.865 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.865 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:46.865 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253578+0000", 2024-09-04T17:48:46.865 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8241999999999999e-05 2024-09-04T17:48:46.865 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.865 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.865 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:46.865 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253591+0000", 2024-09-04T17:48:46.865 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3453e-05 2024-09-04T17:48:46.866 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.866 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.866 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:46.866 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304614+0000", 2024-09-04T17:48:46.866 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051022456000000001 2024-09-04T17:48:46.866 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.866 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.866 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:46.866 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304621+0000", 2024-09-04T17:48:46.866 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3209999999999999e-06 2024-09-04T17:48:46.866 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:46.866 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:46.866 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:46.866 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.867 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.867 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44696 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:46.867 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253609+0000", 2024-09-04T17:48:46.867 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 116.544169572, 2024-09-04T17:48:46.867 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051115351000000003, 2024-09-04T17:48:46.867 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:46.867 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:46.867 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:46.867 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:46.867 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:46.867 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:46.867 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44696 2024-09-04T17:48:46.867 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.868 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:46.868 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.868 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:46.868 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253609+0000", 2024-09-04T17:48:46.868 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:46.868 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.868 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.868 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:46.868 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253609+0000", 2024-09-04T17:48:46.868 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:46.868 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.868 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.868 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:46.869 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253610+0000", 2024-09-04T17:48:46.869 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2300000000000001e-06 2024-09-04T17:48:46.869 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.869 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.869 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:46.869 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253615+0000", 2024-09-04T17:48:46.869 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7260000000000002e-06 2024-09-04T17:48:46.869 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.869 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.869 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:46.869 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253632+0000", 2024-09-04T17:48:46.869 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7039999999999999e-05 2024-09-04T17:48:46.869 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.870 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.870 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:46.870 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253666+0000", 2024-09-04T17:48:46.870 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4662000000000001e-05 2024-09-04T17:48:46.870 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.870 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.870 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:46.870 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304677+0000", 2024-09-04T17:48:46.870 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051011017999999998 2024-09-04T17:48:46.870 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.870 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.870 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:46.870 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304724+0000", 2024-09-04T17:48:46.871 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6675000000000003e-05 2024-09-04T17:48:46.871 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:46.871 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:46.871 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:46.871 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.871 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.871 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44697 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:46.871 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253684+0000", 2024-09-04T17:48:46.871 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 116.544094078, 2024-09-04T17:48:46.871 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051073107, 2024-09-04T17:48:46.871 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:46.871 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:46.871 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:46.871 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:46.872 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:46.872 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:46.872 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44697 2024-09-04T17:48:46.872 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.872 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:46.872 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.872 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:46.872 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253684+0000", 2024-09-04T17:48:46.872 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:46.872 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.872 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.872 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:46.872 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253684+0000", 2024-09-04T17:48:46.873 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:46.873 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.873 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.873 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:46.873 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253685+0000", 2024-09-04T17:48:46.873 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0699999999999999e-06 2024-09-04T17:48:46.873 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.873 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.873 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:46.873 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253690+0000", 2024-09-04T17:48:46.873 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1959999999999999e-06 2024-09-04T17:48:46.873 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.873 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.873 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:46.874 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253707+0000", 2024-09-04T17:48:46.874 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6946e-05 2024-09-04T17:48:46.874 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.874 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.874 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:46.874 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253720+0000", 2024-09-04T17:48:46.874 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3076e-05 2024-09-04T17:48:46.874 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.874 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.874 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:46.874 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304745+0000", 2024-09-04T17:48:46.874 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051024760000000002 2024-09-04T17:48:46.874 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.874 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.875 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:46.875 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304757+0000", 2024-09-04T17:48:46.875 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2058999999999999e-05 2024-09-04T17:48:46.875 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:46.875 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:46.875 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:46.875 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.875 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.875 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44698 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:46.875 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253737+0000", 2024-09-04T17:48:46.875 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 116.54404172700001, 2024-09-04T17:48:46.875 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051176078, 2024-09-04T17:48:46.875 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:46.876 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:46.876 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:46.876 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:46.876 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:46.876 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:46.876 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44698 2024-09-04T17:48:46.876 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.876 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:46.876 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.876 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:46.876 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253737+0000", 2024-09-04T17:48:46.876 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:46.876 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.877 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.877 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:46.877 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253737+0000", 2024-09-04T17:48:46.877 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:46.877 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.877 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.877 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:46.877 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253738+0000", 2024-09-04T17:48:46.877 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0300000000000001e-06 2024-09-04T17:48:46.877 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.877 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.877 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:46.877 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253752+0000", 2024-09-04T17:48:46.878 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4358999999999999e-05 2024-09-04T17:48:46.878 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.878 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.878 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:46.878 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253771+0000", 2024-09-04T17:48:46.878 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8879999999999999e-05 2024-09-04T17:48:46.878 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.878 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.880 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:46.880 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253785+0000", 2024-09-04T17:48:46.880 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3852000000000001e-05 2024-09-04T17:48:46.880 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.880 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.880 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:46.880 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304876+0000", 2024-09-04T17:48:46.880 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051091105999999997 2024-09-04T17:48:46.880 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.880 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.881 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:46.881 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304913+0000", 2024-09-04T17:48:46.881 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.6850999999999997e-05 2024-09-04T17:48:46.881 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:46.881 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:46.881 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:46.881 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.881 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.881 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44699 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:46.881 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253804+0000", 2024-09-04T17:48:46.881 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 116.543974203, 2024-09-04T17:48:46.881 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051172250000000002, 2024-09-04T17:48:46.881 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:46.882 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:46.882 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:46.882 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:46.882 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:46.882 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:46.882 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44699 2024-09-04T17:48:46.882 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.882 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:46.882 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.882 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:46.882 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253804+0000", 2024-09-04T17:48:46.882 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:46.882 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.883 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.883 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:46.883 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253804+0000", 2024-09-04T17:48:46.883 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:46.883 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.883 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.883 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:46.883 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253805+0000", 2024-09-04T17:48:46.883 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.08e-06 2024-09-04T17:48:46.883 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.883 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.883 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:46.883 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253810+0000", 2024-09-04T17:48:46.883 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1830000000000002e-06 2024-09-04T17:48:46.884 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.884 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.884 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:46.884 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253828+0000", 2024-09-04T17:48:46.884 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7662000000000001e-05 2024-09-04T17:48:46.884 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.884 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.884 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:46.884 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253841+0000", 2024-09-04T17:48:46.884 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3203999999999999e-05 2024-09-04T17:48:46.884 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.884 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.884 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:46.885 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304966+0000", 2024-09-04T17:48:46.885 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051125273999999998 2024-09-04T17:48:46.885 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.885 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.885 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:46.885 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304976+0000", 2024-09-04T17:48:46.885 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.8470000000000008e-06 2024-09-04T17:48:46.885 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:46.885 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:46.885 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:46.885 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.885 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.886 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44700 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:46.886 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253858+0000", 2024-09-04T17:48:46.886 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 116.54392054900001, 2024-09-04T17:48:46.886 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051218957000000002, 2024-09-04T17:48:46.886 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:46.886 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:46.886 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:46.886 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:46.886 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:46.886 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:46.886 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44700 2024-09-04T17:48:46.886 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.886 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:46.886 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.887 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:46.887 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253858+0000", 2024-09-04T17:48:46.887 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:46.887 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.887 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.887 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:46.887 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253858+0000", 2024-09-04T17:48:46.887 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:46.887 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.887 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.887 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:46.887 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253859+0000", 2024-09-04T17:48:46.887 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.217e-06 2024-09-04T17:48:46.887 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.888 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.888 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:46.888 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253864+0000", 2024-09-04T17:48:46.888 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7600000000000002e-06 2024-09-04T17:48:46.888 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.888 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.888 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:46.888 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253880+0000", 2024-09-04T17:48:46.888 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6501999999999999e-05 2024-09-04T17:48:46.888 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.888 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.888 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:46.888 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253899+0000", 2024-09-04T17:48:46.889 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8907e-05 2024-09-04T17:48:46.889 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.889 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.889 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:46.889 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305052+0000", 2024-09-04T17:48:46.889 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051152989000000003 2024-09-04T17:48:46.889 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.889 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.889 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:46.889 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305077+0000", 2024-09-04T17:48:46.889 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4581999999999999e-05 2024-09-04T17:48:46.889 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:46.889 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:46.890 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:46.890 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.890 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.890 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44701 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:46.890 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253917+0000", 2024-09-04T17:48:46.890 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 116.543861197, 2024-09-04T17:48:46.890 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051231113000000002, 2024-09-04T17:48:46.890 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:46.890 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:46.890 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:46.890 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:46.890 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:46.890 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:46.890 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44701 2024-09-04T17:48:46.891 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.891 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:46.891 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.891 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:46.891 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253917+0000", 2024-09-04T17:48:46.891 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:46.891 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.891 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.891 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:46.891 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253917+0000", 2024-09-04T17:48:46.891 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:46.891 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.891 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.891 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:46.892 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253918+0000", 2024-09-04T17:48:46.892 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.136e-06 2024-09-04T17:48:46.892 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.892 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.892 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:46.892 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253924+0000", 2024-09-04T17:48:46.892 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.3070000000000002e-06 2024-09-04T17:48:46.892 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.892 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.892 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:46.892 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253940+0000", 2024-09-04T17:48:46.892 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6612e-05 2024-09-04T17:48:46.892 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.893 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.893 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:46.893 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253954+0000", 2024-09-04T17:48:46.893 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3709999999999999e-05 2024-09-04T17:48:46.893 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.893 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.893 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:46.893 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305136+0000", 2024-09-04T17:48:46.893 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051181935999999997 2024-09-04T17:48:46.893 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:46.893 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:46.893 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:46.893 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305148+0000", 2024-09-04T17:48:46.893 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2412000000000001e-05 2024-09-04T17:48:46.894 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:46.894 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:46.894 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:46.894 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:46.894 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:46.894 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:46.895 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-04T17:48:46.896 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:46.893+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:46.932 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-04T17:48:46.957 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is set 2024-09-04T17:48:47.029 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:47.027+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:47.079 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:48:47.080 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:48:47.081 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:47.081 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:48:47.081 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:47.081 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:102516 3.5 3.f641be25 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e167)", 2024-09-04T17:48:47.081 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:48:47.081935+0000", 2024-09-04T17:48:47.081 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 0.00012005100000000001, 2024-09-04T17:48:47.082 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013476199999999999, 2024-09-04T17:48:47.082 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:47.082 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:47.082 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:47.082 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:47.082 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:47.082 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:47.082 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 102516 2024-09-04T17:48:47.082 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:47.082 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:47.082 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:47.082 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:47.082 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:47.081935+0000", 2024-09-04T17:48:47.082 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:47.082 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:47.083 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:47.083 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:47.083 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:47.081935+0000", 2024-09-04T17:48:47.083 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:47.083 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:47.083 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:47.083 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:47.083 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:47.081935+0000", 2024-09-04T17:48:47.083 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.3399999999999999e-07 2024-09-04T17:48:47.083 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:47.083 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:47.083 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:47.083 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:47.081938+0000", 2024-09-04T17:48:47.083 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.3949999999999999e-06 2024-09-04T17:48:47.084 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:47.084 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:47.084 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:47.084 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:47.081945+0000", 2024-09-04T17:48:47.084 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.985e-06 2024-09-04T17:48:47.084 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:47.084 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:47.084 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:47.084 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:47.081949+0000", 2024-09-04T17:48:47.084 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.968e-06 2024-09-04T17:48:47.084 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:47.084 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:47.084 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:47.084 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:48:47.082042+0000", 2024-09-04T17:48:47.085 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.3137e-05 2024-09-04T17:48:47.085 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:47.085 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:47.085 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:47.085 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:47.085 INFO:teuthology.orchestra.run.smithi190.stdout: ], 2024-09-04T17:48:47.085 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 1 2024-09-04T17:48:47.085 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:47.096 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:48:47.096 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:48:47.097 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-04T17:48:47.098 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:48:47.097+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:48:47.112 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-04T17:48:47.112 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-04T17:48:47.131 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-04T17:48:47.131 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-04T17:48:47.148 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:48:47.148 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:48:47.164 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:48:47.165 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:48:47.181 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-04T17:48:47.181 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-04T17:48:47.198 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:48:47.198 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:48:47.198 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:48:47.195+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:48:47.268 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:47.268 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:47.282 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-04T17:48:47.300 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:47.295+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:47.401 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:48:47.397+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:48:47.406 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:47.407 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:47.421 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-04T17:48:47.509 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:48:47.509+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:48:47.588 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:47.588 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:47.603 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-04T17:48:47.603 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:48:47.601+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:48:47.704 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:47.702+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:47.773 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:47.774 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:47.787 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-04T17:48:47.804 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:48:47.802+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:48:47.854 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.0 to 1 (10000) 2024-09-04T17:48:47.886 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-04T17:48:47.905 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:47.905+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:47.925 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:47.925 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:47.940 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-04T17:48:48.006 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:48.005+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:48.096 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:48.096 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:48.107 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:48.105+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:48.110 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-04T17:48:48.208 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:48:48.205+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:48:48.227 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:48.227 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:48.241 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-04T17:48:48.309 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:48:48.306+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:48:48.358 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:48.358 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:48.372 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-04T17:48:48.414 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:48.410+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:48.511 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:48:48.509+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:48:48.536 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:48.536 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:48.550 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-04T17:48:48.612 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:48:48.606+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:48:48.673 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:48.673 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:48.687 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-04T17:48:48.730 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:48:48.726+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:48:48.814 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:48:48.813+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:48:48.854 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:48.854 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:48.868 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-04T17:48:48.915 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:48:48.910+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:48:48.968 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.2 to 1 (10000) 2024-09-04T17:48:48.982 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-04T17:48:48.998 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-04T17:48:49.010 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:49.010 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:49.029 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-04T17:48:49.041 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:49.038+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:49.117 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:49.117+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:49.157 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:49.157 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:49.172 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-04T17:48:49.214 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-04T17:48:49.257 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:48:49.257+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:48:49.319 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:49.317+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:49.340 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:49.340 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:49.353 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:48:49.353 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:48:49.354 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-04T17:48:49.370 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:48:49.370 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:48:49.387 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-04T17:48:49.387 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-04T17:48:49.404 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-04T17:48:49.404 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-04T17:48:49.420 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:49.417+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:49.425 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:48:49.425 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:48:49.445 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:48:49.445 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:48:49.465 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-04T17:48:49.466 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-04T17:48:49.470 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:49.470 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:48:49.470 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:48:49.470 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:49.484 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-04T17:48:49.486 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:48:49.486 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:48:49.520 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:49.517+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:49.605 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:49.605 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:48:49.606 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:48:49.606 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:48:49.606 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:49.619 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-04T17:48:49.621 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:48:49.621+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:48:49.723 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:49.721+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:49.739 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:49.739 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:48:49.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:48:49.739 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:48:49.739 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.739 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:48444 3.12 3:4ff80fb5:::rbd_data.11346a2b0bc9.000000000000001d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-04T17:48:49.740 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:46:58.122601+0000", 2024-09-04T17:48:49.740 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 111.614487884, 2024-09-04T17:48:49.740 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057065713999999997, 2024-09-04T17:48:49.740 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:49.740 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:49.740 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:49.740 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:49.740 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:49.740 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:49.740 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 48444 2024-09-04T17:48:49.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.740 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:49.740 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:49.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122601+0000", 2024-09-04T17:48:49.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:49.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:49.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122601+0000", 2024-09-04T17:48:49.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:49.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:49.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122603+0000", 2024-09-04T17:48:49.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3499999999999999e-06 2024-09-04T17:48:49.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:49.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122614+0000", 2024-09-04T17:48:49.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.063e-05 2024-09-04T17:48:49.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:49.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122643+0000", 2024-09-04T17:48:49.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9434999999999999e-05 2024-09-04T17:48:49.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:49.743 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122657+0000", 2024-09-04T17:48:49.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3920000000000001e-05 2024-09-04T17:48:49.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.743 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:49.743 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122767+0000", 2024-09-04T17:48:49.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010981599999999999 2024-09-04T17:48:49.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.743 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:49.743 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.122989+0000", 2024-09-04T17:48:49.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022158200000000001 2024-09-04T17:48:49.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.744 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 3", 2024-09-04T17:48:49.744 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.133349+0000", 2024-09-04T17:48:49.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010360729000000001 2024-09-04T17:48:49.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.744 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:49.744 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.179423+0000", 2024-09-04T17:48:49.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046073408000000003 2024-09-04T17:48:49.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.744 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:49.744 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.179453+0000", 2024-09-04T17:48:49.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0229000000000001e-05 2024-09-04T17:48:49.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:49.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.179545+0000", 2024-09-04T17:48:49.748 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1580000000000001e-05 2024-09-04T17:48:49.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.748 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.748 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:49.748 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:46:58.179667+0000", 2024-09-04T17:48:49.748 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012203500000000001 2024-09-04T17:48:49.749 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:49.749 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:49.749 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:49.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.749 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:55376 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-04T17:48:49.749 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:11.974053+0000", 2024-09-04T17:48:49.749 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 97.763036099999994, 2024-09-04T17:48:49.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060514722, 2024-09-04T17:48:49.749 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:49.749 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:49.749 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:49.749 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:49.750 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:49.750 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:49.750 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 55376 2024-09-04T17:48:49.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.750 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:49.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:49.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974053+0000", 2024-09-04T17:48:49.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:49.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:49.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974053+0000", 2024-09-04T17:48:49.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:49.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.752 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:49.752 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974054+0000", 2024-09-04T17:48:49.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.841e-06 2024-09-04T17:48:49.753 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.753 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:49.753 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974062+0000", 2024-09-04T17:48:49.754 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5460000000000003e-06 2024-09-04T17:48:49.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:49.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974077+0000", 2024-09-04T17:48:49.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5389000000000001e-05 2024-09-04T17:48:49.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.756 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:49.756 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974085+0000", 2024-09-04T17:48:49.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7659999999999998e-06 2024-09-04T17:48:49.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.756 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:49.757 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974188+0000", 2024-09-04T17:48:49.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000102521 2024-09-04T17:48:49.757 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.757 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:49.757 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.974321+0000", 2024-09-04T17:48:49.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013324699999999999 2024-09-04T17:48:49.757 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.757 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 4", 2024-09-04T17:48:49.757 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:11.986481+0000", 2024-09-04T17:48:49.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012160450999999999 2024-09-04T17:48:49.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.758 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:49.758 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:12.034415+0000", 2024-09-04T17:48:49.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.047934065999999997 2024-09-04T17:48:49.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.758 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:49.758 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:12.034438+0000", 2024-09-04T17:48:49.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2923999999999999e-05 2024-09-04T17:48:49.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:49.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:12.034505+0000", 2024-09-04T17:48:49.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6711000000000006e-05 2024-09-04T17:48:49.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:49.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:12.034567+0000", 2024-09-04T17:48:49.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2260000000000004e-05 2024-09-04T17:48:49.759 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:49.759 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:49.759 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:49.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.760 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61141 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 2793472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:49.760 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819909+0000", 2024-09-04T17:48:49.760 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 95.917180107999997, 2024-09-04T17:48:49.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042786419999999999, 2024-09-04T17:48:49.760 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:49.761 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:49.761 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:49.761 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:49.761 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:49.761 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:49.761 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61141 2024-09-04T17:48:49.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.761 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:49.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:49.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819909+0000", 2024-09-04T17:48:49.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:49.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.762 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:49.762 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819909+0000", 2024-09-04T17:48:49.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:49.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.763 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:49.763 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819909+0000", 2024-09-04T17:48:49.763 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6300000000000005e-07 2024-09-04T17:48:49.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.763 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:49.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819913+0000", 2024-09-04T17:48:49.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4189999999999998e-06 2024-09-04T17:48:49.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:49.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819920+0000", 2024-09-04T17:48:49.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8619999999999997e-06 2024-09-04T17:48:49.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:49.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819926+0000", 2024-09-04T17:48:49.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9399999999999999e-06 2024-09-04T17:48:49.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.766 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:49.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862331+0000", 2024-09-04T17:48:49.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042405348000000002 2024-09-04T17:48:49.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.767 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:49.767 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862410+0000", 2024-09-04T17:48:49.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9078000000000004e-05 2024-09-04T17:48:49.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.767 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:49.768 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862695+0000", 2024-09-04T17:48:49.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028510999999999999 2024-09-04T17:48:49.768 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:49.768 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:49.768 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:49.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.769 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61142 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 2932736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:49.769 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819934+0000", 2024-09-04T17:48:49.769 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 95.917154490000001, 2024-09-04T17:48:49.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043101985000000002, 2024-09-04T17:48:49.769 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:49.769 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:49.770 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:49.770 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:49.770 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:49.770 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:49.770 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61142 2024-09-04T17:48:49.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.770 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:49.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:49.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819934+0000", 2024-09-04T17:48:49.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:49.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.772 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:49.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819934+0000", 2024-09-04T17:48:49.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:49.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.772 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:49.773 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819935+0000", 2024-09-04T17:48:49.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2024-09-04T17:48:49.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.773 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:49.773 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819937+0000", 2024-09-04T17:48:49.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5770000000000001e-06 2024-09-04T17:48:49.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:49.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819943+0000", 2024-09-04T17:48:49.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7950000000000004e-06 2024-09-04T17:48:49.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:49.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819948+0000", 2024-09-04T17:48:49.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6779999999999999e-06 2024-09-04T17:48:49.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:49.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862739+0000", 2024-09-04T17:48:49.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042791454999999999 2024-09-04T17:48:49.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:49.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.862824+0000", 2024-09-04T17:48:49.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4790999999999995e-05 2024-09-04T17:48:49.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:49.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.863036+0000", 2024-09-04T17:48:49.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021224899999999999 2024-09-04T17:48:49.779 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:49.779 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:49.779 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:49.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.780 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61143 3.3 3:c55189f7:::rbd_data.11341dfde0c5.0000000000000001:1f [read 3039232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:49.780 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.819954+0000", 2024-09-04T17:48:49.780 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 95.917134372000007, 2024-09-04T17:48:49.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043485134000000002, 2024-09-04T17:48:49.780 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:49.780 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:49.780 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:49.780 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:49.780 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:49.780 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:49.780 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61143 2024-09-04T17:48:49.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.781 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:49.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:49.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819954+0000", 2024-09-04T17:48:49.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:49.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:49.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819954+0000", 2024-09-04T17:48:49.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:49.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:49.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819955+0000", 2024-09-04T17:48:49.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7300000000000002e-07 2024-09-04T17:48:49.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:49.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819957+0000", 2024-09-04T17:48:49.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.024e-06 2024-09-04T17:48:49.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:49.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819963+0000", 2024-09-04T17:48:49.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8109999999999999e-06 2024-09-04T17:48:49.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:49.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.819967+0000", 2024-09-04T17:48:49.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6149999999999999e-06 2024-09-04T17:48:49.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.785 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:49.785 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.863082+0000", 2024-09-04T17:48:49.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043115078000000001 2024-09-04T17:48:49.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.785 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:49.785 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.863172+0000", 2024-09-04T17:48:49.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0181999999999998e-05 2024-09-04T17:48:49.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:49.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.863440+0000", 2024-09-04T17:48:49.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026705099999999997 2024-09-04T17:48:49.786 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:49.786 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:49.786 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:49.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.786 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:70015 3.6 3:6012e780:::rbd_data.11341aa245a3.000000000000005b:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-04T17:48:49.787 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:26.155497+0000", 2024-09-04T17:48:49.787 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 83.581591314999997, 2024-09-04T17:48:49.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.049007124999999999, 2024-09-04T17:48:49.787 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:49.787 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:49.787 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:49.787 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:49.787 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:49.787 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:49.787 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 70015 2024-09-04T17:48:49.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.788 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:49.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:49.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155497+0000", 2024-09-04T17:48:49.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:49.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:49.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155497+0000", 2024-09-04T17:48:49.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:49.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:49.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155500+0000", 2024-09-04T17:48:49.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7070000000000001e-06 2024-09-04T17:48:49.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:49.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155513+0000", 2024-09-04T17:48:49.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2989e-05 2024-09-04T17:48:49.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.790 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:49.790 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155544+0000", 2024-09-04T17:48:49.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0747000000000002e-05 2024-09-04T17:48:49.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.790 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:49.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155560+0000", 2024-09-04T17:48:49.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5866e-05 2024-09-04T17:48:49.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:49.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155675+0000", 2024-09-04T17:48:49.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000114838 2024-09-04T17:48:49.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:49.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.155803+0000", 2024-09-04T17:48:49.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012851799999999999 2024-09-04T17:48:49.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 1", 2024-09-04T17:48:49.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.163582+0000", 2024-09-04T17:48:49.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0077790960000000001 2024-09-04T17:48:49.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:49.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.204055+0000", 2024-09-04T17:48:49.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.040472517 2024-09-04T17:48:49.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:49.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.204195+0000", 2024-09-04T17:48:49.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014055200000000001 2024-09-04T17:48:49.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:49.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.204373+0000", 2024-09-04T17:48:49.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000177784 2024-09-04T17:48:49.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:49.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.204505+0000", 2024-09-04T17:48:49.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013151100000000001 2024-09-04T17:48:49.794 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:49.794 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:49.795 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:49.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.795 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:70434 3.12 3:4cd2342b:::rbd_data.11341aa245a3.0000000000000091:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-04T17:48:49.795 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:26.943124+0000", 2024-09-04T17:48:49.795 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 82.793964532999993, 2024-09-04T17:48:49.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055011831999999997, 2024-09-04T17:48:49.795 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:49.795 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:49.795 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:49.795 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:49.795 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:49.796 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:49.796 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 70434 2024-09-04T17:48:49.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.796 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:49.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:49.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943124+0000", 2024-09-04T17:48:49.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:49.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:49.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943124+0000", 2024-09-04T17:48:49.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:49.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:49.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943128+0000", 2024-09-04T17:48:49.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8460000000000001e-06 2024-09-04T17:48:49.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:49.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943142+0000", 2024-09-04T17:48:49.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4089e-05 2024-09-04T17:48:49.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:49.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943177+0000", 2024-09-04T17:48:49.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5167999999999999e-05 2024-09-04T17:48:49.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:49.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943194+0000", 2024-09-04T17:48:49.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6235e-05 2024-09-04T17:48:49.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:49.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943286+0000", 2024-09-04T17:48:49.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2793000000000004e-05 2024-09-04T17:48:49.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:49.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.943526+0000", 2024-09-04T17:48:49.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023950999999999999 2024-09-04T17:48:49.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 1", 2024-09-04T17:48:49.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.952871+0000", 2024-09-04T17:48:49.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0093453070000000006 2024-09-04T17:48:49.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:49.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.997968+0000", 2024-09-04T17:48:49.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045096613000000001 2024-09-04T17:48:49.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:49.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.998006+0000", 2024-09-04T17:48:49.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8257000000000003e-05 2024-09-04T17:48:49.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:49.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.998073+0000", 2024-09-04T17:48:49.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6803000000000004e-05 2024-09-04T17:48:49.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:49.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:26.998136+0000", 2024-09-04T17:48:49.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3211000000000002e-05 2024-09-04T17:48:49.801 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:49.801 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:49.801 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:49.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.801 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93841 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 5111808~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:49.801 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386491+0000", 2024-09-04T17:48:49.801 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.350597336, 2024-09-04T17:48:49.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043069578999999997, 2024-09-04T17:48:49.802 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:49.802 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:49.802 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:49.802 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:49.802 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:49.802 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:49.802 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93841 2024-09-04T17:48:49.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.802 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:49.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:49.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386491+0000", 2024-09-04T17:48:49.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:49.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:49.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386491+0000", 2024-09-04T17:48:49.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:49.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:49.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386492+0000", 2024-09-04T17:48:49.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0299999999999999e-07 2024-09-04T17:48:49.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:49.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386499+0000", 2024-09-04T17:48:49.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6089999999999997e-06 2024-09-04T17:48:49.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:49.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386506+0000", 2024-09-04T17:48:49.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3599999999999998e-06 2024-09-04T17:48:49.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:49.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386514+0000", 2024-09-04T17:48:49.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9379999999999992e-06 2024-09-04T17:48:49.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:49.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.428957+0000", 2024-09-04T17:48:49.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042442831 2024-09-04T17:48:49.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:49.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.429080+0000", 2024-09-04T17:48:49.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012359299999999999 2024-09-04T17:48:49.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:49.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.429561+0000", 2024-09-04T17:48:49.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00048074499999999998 2024-09-04T17:48:49.806 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:49.806 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:49.806 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:49.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.806 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93842 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 4259840~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:49.806 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386521+0000", 2024-09-04T17:48:49.806 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.350567404000003, 2024-09-04T17:48:49.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043661933999999999, 2024-09-04T17:48:49.806 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:49.807 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:49.807 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:49.807 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:49.807 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:49.807 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:49.807 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93842 2024-09-04T17:48:49.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.807 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:49.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:49.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386521+0000", 2024-09-04T17:48:49.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:49.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:49.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386521+0000", 2024-09-04T17:48:49.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:49.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:49.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386522+0000", 2024-09-04T17:48:49.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4700000000000001e-07 2024-09-04T17:48:49.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:49.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386524+0000", 2024-09-04T17:48:49.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4559999999999999e-06 2024-09-04T17:48:49.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:49.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386534+0000", 2024-09-04T17:48:49.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2639999999999997e-06 2024-09-04T17:48:49.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:49.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386539+0000", 2024-09-04T17:48:49.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6130000000000003e-06 2024-09-04T17:48:49.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:49.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.429577+0000", 2024-09-04T17:48:49.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043037757000000003 2024-09-04T17:48:49.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:49.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.429638+0000", 2024-09-04T17:48:49.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1235999999999994e-05 2024-09-04T17:48:49.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:49.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.430183+0000", 2024-09-04T17:48:49.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000545061 2024-09-04T17:48:49.812 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:49.812 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:49.812 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:49.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.812 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93843 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 4390912~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:49.812 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386547+0000", 2024-09-04T17:48:49.812 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.350542167, 2024-09-04T17:48:49.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044260668000000003, 2024-09-04T17:48:49.812 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:49.812 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:49.813 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:49.813 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:49.813 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:49.813 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:49.813 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93843 2024-09-04T17:48:49.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.813 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:49.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:49.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386547+0000", 2024-09-04T17:48:49.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:49.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:49.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386547+0000", 2024-09-04T17:48:49.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:49.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:49.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386547+0000", 2024-09-04T17:48:49.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4199999999999996e-07 2024-09-04T17:48:49.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:49.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386550+0000", 2024-09-04T17:48:49.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6199999999999999e-06 2024-09-04T17:48:49.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:49.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386557+0000", 2024-09-04T17:48:49.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1919999999999999e-06 2024-09-04T17:48:49.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:49.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386562+0000", 2024-09-04T17:48:49.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2580000000000003e-06 2024-09-04T17:48:49.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:49.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.430226+0000", 2024-09-04T17:48:49.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.043663484000000002 2024-09-04T17:48:49.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:49.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.430286+0000", 2024-09-04T17:48:49.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0455999999999999e-05 2024-09-04T17:48:49.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:49.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.430807+0000", 2024-09-04T17:48:49.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00052111599999999996 2024-09-04T17:48:49.818 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:49.818 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:49.818 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:49.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.819 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93844 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 4521984~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:49.819 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386603+0000", 2024-09-04T17:48:49.819 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.350486115999999, 2024-09-04T17:48:49.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044705350999999997, 2024-09-04T17:48:49.819 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:49.819 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:49.819 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:49.819 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:49.819 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:49.819 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:49.820 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93844 2024-09-04T17:48:49.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.820 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:49.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:49.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386603+0000", 2024-09-04T17:48:49.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:49.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:49.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386603+0000", 2024-09-04T17:48:49.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:49.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:49.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386603+0000", 2024-09-04T17:48:49.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.92e-07 2024-09-04T17:48:49.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:49.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386606+0000", 2024-09-04T17:48:49.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5500000000000001e-06 2024-09-04T17:48:49.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:49.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386614+0000", 2024-09-04T17:48:49.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9570000000000002e-06 2024-09-04T17:48:49.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:49.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386619+0000", 2024-09-04T17:48:49.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5119999999999999e-06 2024-09-04T17:48:49.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:49.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.430828+0000", 2024-09-04T17:48:49.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044208654 2024-09-04T17:48:49.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:49.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.430903+0000", 2024-09-04T17:48:49.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4629999999999995e-05 2024-09-04T17:48:49.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:49.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.431308+0000", 2024-09-04T17:48:49.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000405256 2024-09-04T17:48:49.825 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:49.825 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:49.825 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:49.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.825 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93845 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 4653056~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:49.825 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386627+0000", 2024-09-04T17:48:49.825 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.350461527, 2024-09-04T17:48:49.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045262927000000001, 2024-09-04T17:48:49.826 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:49.826 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:49.826 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:49.826 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:49.826 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:49.826 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:49.826 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93845 2024-09-04T17:48:49.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.826 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:49.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:49.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386627+0000", 2024-09-04T17:48:49.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:49.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:49.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386627+0000", 2024-09-04T17:48:49.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:49.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:49.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386628+0000", 2024-09-04T17:48:49.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0999999999999998e-07 2024-09-04T17:48:49.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:49.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386630+0000", 2024-09-04T17:48:49.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3300000000000001e-06 2024-09-04T17:48:49.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:49.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386645+0000", 2024-09-04T17:48:49.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5186e-05 2024-09-04T17:48:49.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:49.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386651+0000", 2024-09-04T17:48:49.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7710000000000003e-06 2024-09-04T17:48:49.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:49.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.431349+0000", 2024-09-04T17:48:49.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.044697723000000002 2024-09-04T17:48:49.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:49.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.431410+0000", 2024-09-04T17:48:49.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0677999999999997e-05 2024-09-04T17:48:49.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:49.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.431890+0000", 2024-09-04T17:48:49.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00048052899999999999 2024-09-04T17:48:49.830 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:49.830 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:49.830 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:49.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.830 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93846 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 4784128~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:49.830 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386659+0000", 2024-09-04T17:48:49.830 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.350429857999998, 2024-09-04T17:48:49.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045784490999999997, 2024-09-04T17:48:49.830 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:49.830 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:49.831 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:49.831 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:49.831 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:49.831 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:49.831 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93846 2024-09-04T17:48:49.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.831 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:49.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:49.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386659+0000", 2024-09-04T17:48:49.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:49.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:49.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386659+0000", 2024-09-04T17:48:49.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:49.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:49.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386659+0000", 2024-09-04T17:48:49.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7000000000000005e-07 2024-09-04T17:48:49.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:49.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386662+0000", 2024-09-04T17:48:49.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3300000000000001e-06 2024-09-04T17:48:49.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:49.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386669+0000", 2024-09-04T17:48:49.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0589999999999997e-06 2024-09-04T17:48:49.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:49.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386674+0000", 2024-09-04T17:48:49.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3009999999999997e-06 2024-09-04T17:48:49.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:49.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.431934+0000", 2024-09-04T17:48:49.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045259570999999998 2024-09-04T17:48:49.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:49.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.431998+0000", 2024-09-04T17:48:49.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4775000000000002e-05 2024-09-04T17:48:49.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.835 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:49.835 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.432443+0000", 2024-09-04T17:48:49.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00044488500000000001 2024-09-04T17:48:49.835 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:49.835 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:49.835 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:49.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.835 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93847 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 4915200~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:49.835 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386681+0000", 2024-09-04T17:48:49.835 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.350407601000001, 2024-09-04T17:48:49.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046305101000000001, 2024-09-04T17:48:49.836 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:49.836 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:49.836 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:49.836 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:49.836 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:49.836 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:49.836 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93847 2024-09-04T17:48:49.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.836 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:49.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:49.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386681+0000", 2024-09-04T17:48:49.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:49.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.837 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:49.837 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386681+0000", 2024-09-04T17:48:49.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:49.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.837 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:49.837 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386682+0000", 2024-09-04T17:48:49.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1699999999999998e-07 2024-09-04T17:48:49.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:49.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386684+0000", 2024-09-04T17:48:49.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.153e-06 2024-09-04T17:48:49.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:49.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386693+0000", 2024-09-04T17:48:49.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9639999999999992e-06 2024-09-04T17:48:49.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:49.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386698+0000", 2024-09-04T17:48:49.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3709999999999999e-06 2024-09-04T17:48:49.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:49.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.432476+0000", 2024-09-04T17:48:49.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.045778147999999998 2024-09-04T17:48:49.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:49.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.432529+0000", 2024-09-04T17:48:49.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3165999999999999e-05 2024-09-04T17:48:49.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:49.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.432986+0000", 2024-09-04T17:48:49.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00045678199999999998 2024-09-04T17:48:49.841 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:49.841 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:49.841 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:49.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.841 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93848 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 5046272~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:49.841 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386705+0000", 2024-09-04T17:48:49.841 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.3503835, 2024-09-04T17:48:49.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046825259000000001, 2024-09-04T17:48:49.842 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:49.842 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:49.842 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:49.842 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:49.842 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:49.842 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:49.842 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93848 2024-09-04T17:48:49.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.842 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:49.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:49.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386705+0000", 2024-09-04T17:48:49.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:49.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:49.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386705+0000", 2024-09-04T17:48:49.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:49.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:49.844 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386706+0000", 2024-09-04T17:48:49.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1500000000000005e-07 2024-09-04T17:48:49.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:49.844 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386708+0000", 2024-09-04T17:48:49.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2979999999999999e-06 2024-09-04T17:48:49.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:49.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386715+0000", 2024-09-04T17:48:49.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8669999999999996e-06 2024-09-04T17:48:49.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:49.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386720+0000", 2024-09-04T17:48:49.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.242e-06 2024-09-04T17:48:49.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:49.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.433019+0000", 2024-09-04T17:48:49.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046298359999999997 2024-09-04T17:48:49.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:49.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.433084+0000", 2024-09-04T17:48:49.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5395999999999995e-05 2024-09-04T17:48:49.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:49.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.433531+0000", 2024-09-04T17:48:49.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00044658099999999999 2024-09-04T17:48:49.847 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:49.847 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:49.847 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:49.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.847 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:93849 3.1 3:85412eeb:::rbd_data.1134878eadad.0000000000000000:2d [sparse-read 5177344~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e119)", 2024-09-04T17:48:49.847 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:59.386727+0000", 2024-09-04T17:48:49.848 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.350361602, 2024-09-04T17:48:49.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.047267796000000001, 2024-09-04T17:48:49.848 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:49.848 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:49.848 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:49.848 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:49.848 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:49.848 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:49.848 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 93849 2024-09-04T17:48:49.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.849 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:49.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:49.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386727+0000", 2024-09-04T17:48:49.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:49.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:49.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386727+0000", 2024-09-04T17:48:49.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:49.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:49.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386728+0000", 2024-09-04T17:48:49.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1699999999999998e-07 2024-09-04T17:48:49.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:49.851 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386730+0000", 2024-09-04T17:48:49.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2589999999999999e-06 2024-09-04T17:48:49.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:49.851 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386751+0000", 2024-09-04T17:48:49.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1424999999999999e-05 2024-09-04T17:48:49.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:49.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.386759+0000", 2024-09-04T17:48:49.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9400000000000002e-06 2024-09-04T17:48:49.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:49.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.433559+0000", 2024-09-04T17:48:49.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046799573999999997 2024-09-04T17:48:49.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:49.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.433610+0000", 2024-09-04T17:48:49.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1134999999999997e-05 2024-09-04T17:48:49.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:49.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:59.433995+0000", 2024-09-04T17:48:49.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000384946 2024-09-04T17:48:49.854 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:49.854 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:49.854 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:49.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.854 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:96175 3.12 3:4d4471be:::rbd_data.1134c4e2d26.000000000000006e:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123)", 2024-09-04T17:48:49.854 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:48:02.919542+0000", 2024-09-04T17:48:49.854 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 46.817546358000001, 2024-09-04T17:48:49.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060180626000000001, 2024-09-04T17:48:49.854 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:49.855 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:49.855 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:49.855 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:49.855 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:49.855 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:49.855 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 96175 2024-09-04T17:48:49.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.855 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:49.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.855 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:49.855 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.919542+0000", 2024-09-04T17:48:49.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:49.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:49.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.919542+0000", 2024-09-04T17:48:49.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:49.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:49.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.919545+0000", 2024-09-04T17:48:49.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0199999999999999e-06 2024-09-04T17:48:49.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.857 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:49.857 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.919559+0000", 2024-09-04T17:48:49.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3293e-05 2024-09-04T17:48:49.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.857 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:49.857 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.919587+0000", 2024-09-04T17:48:49.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7922000000000001e-05 2024-09-04T17:48:49.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.857 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:49.857 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.919601+0000", 2024-09-04T17:48:49.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4657e-05 2024-09-04T17:48:49.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.858 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.858 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:49.858 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.919708+0000", 2024-09-04T17:48:49.858 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010624999999999999 2024-09-04T17:48:49.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.858 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.858 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:49.858 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.919929+0000", 2024-09-04T17:48:49.858 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022147399999999999 2024-09-04T17:48:49.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.858 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.858 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2024-09-04T17:48:49.858 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.929774+0000", 2024-09-04T17:48:49.858 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0098449839999999993 2024-09-04T17:48:49.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.859 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.859 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:49.859 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.979511+0000", 2024-09-04T17:48:49.859 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.049736537999999997 2024-09-04T17:48:49.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.859 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.859 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:49.859 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.979547+0000", 2024-09-04T17:48:49.859 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6069000000000003e-05 2024-09-04T17:48:49.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.859 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.859 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:49.859 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.979642+0000", 2024-09-04T17:48:49.860 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5526999999999995e-05 2024-09-04T17:48:49.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.860 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:49.860 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:02.979723+0000", 2024-09-04T17:48:49.860 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0891999999999997e-05 2024-09-04T17:48:49.860 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:49.860 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:49.860 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:49.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.860 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:97148 3.4 3:22dd2475:::rbd_id.image_client.0-clone39:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e128)", 2024-09-04T17:48:49.861 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:48:07.831881+0000", 2024-09-04T17:48:49.861 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 41.905208123999998, 2024-09-04T17:48:49.861 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.026553585, 2024-09-04T17:48:49.861 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:49.861 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:49.861 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:49.861 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:49.861 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:49.861 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:49.861 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 97148 2024-09-04T17:48:49.861 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.861 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:49.861 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.862 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:49.862 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:07.831881+0000", 2024-09-04T17:48:49.862 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:49.862 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.862 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.862 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:49.862 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:07.831881+0000", 2024-09-04T17:48:49.862 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:49.862 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.862 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.862 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:49.862 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:07.831883+0000", 2024-09-04T17:48:49.862 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3369999999999998e-06 2024-09-04T17:48:49.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.863 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:49.863 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:07.831890+0000", 2024-09-04T17:48:49.863 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7050000000000002e-06 2024-09-04T17:48:49.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.863 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:49.863 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:07.831905+0000", 2024-09-04T17:48:49.863 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5512999999999999e-05 2024-09-04T17:48:49.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.863 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:49.864 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:07.831918+0000", 2024-09-04T17:48:49.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3223e-05 2024-09-04T17:48:49.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.864 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:49.864 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:07.833441+0000", 2024-09-04T17:48:49.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0015228920000000001 2024-09-04T17:48:49.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.864 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-09-04T17:48:49.864 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:07.833451+0000", 2024-09-04T17:48:49.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6600000000000007e-06 2024-09-04T17:48:49.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.865 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.865 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:49.865 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:07.837721+0000", 2024-09-04T17:48:49.865 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0042702410000000001 2024-09-04T17:48:49.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.865 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.865 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-04T17:48:49.865 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:07.837732+0000", 2024-09-04T17:48:49.865 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.057e-05 2024-09-04T17:48:49.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.865 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.865 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:49.865 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:08.856291+0000", 2024-09-04T17:48:49.865 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0185595350000001 2024-09-04T17:48:49.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.866 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.866 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:49.866 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:08.856503+0000", 2024-09-04T17:48:49.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000212127 2024-09-04T17:48:49.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.866 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.866 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2,5", 2024-09-04T17:48:49.866 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:08.856664+0000", 2024-09-04T17:48:49.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016072800000000001 2024-09-04T17:48:49.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.867 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.867 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:49.867 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:08.857811+0000", 2024-09-04T17:48:49.867 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001146441 2024-09-04T17:48:49.867 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.867 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.867 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:49.867 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:08.858202+0000", 2024-09-04T17:48:49.867 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000391605 2024-09-04T17:48:49.868 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.868 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.868 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:49.868 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:08.858239+0000", 2024-09-04T17:48:49.868 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7200000000000003e-05 2024-09-04T17:48:49.868 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.868 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.868 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:49.868 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:08.858287+0000", 2024-09-04T17:48:49.868 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7515999999999997e-05 2024-09-04T17:48:49.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.869 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.869 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:49.869 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:08.858434+0000", 2024-09-04T17:48:49.869 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014729200000000001 2024-09-04T17:48:49.869 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:49.869 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:49.869 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:49.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.869 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.870 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:99510 3.18 3:1bf8b1d0:::rbd_header.11346a2b0bc9:head [call rbd.snapshot_remove in=18b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e147)", 2024-09-04T17:48:49.870 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:48:27.032522+0000", 2024-09-04T17:48:49.870 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.704566261, 2024-09-04T17:48:49.870 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.84201522500000003, 2024-09-04T17:48:49.870 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:49.870 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:49.870 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:49.870 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:49.870 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:49.870 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:49.870 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 99510 2024-09-04T17:48:49.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.871 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:49.871 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.871 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:49.871 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032522+0000", 2024-09-04T17:48:49.871 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:49.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.871 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.871 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:49.871 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032522+0000", 2024-09-04T17:48:49.871 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:49.872 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.872 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.872 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:49.872 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032525+0000", 2024-09-04T17:48:49.872 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4789999999999999e-06 2024-09-04T17:48:49.872 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.872 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.872 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:49.872 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032536+0000", 2024-09-04T17:48:49.872 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0557000000000001e-05 2024-09-04T17:48:49.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.873 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.873 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:49.873 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032555+0000", 2024-09-04T17:48:49.873 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9782000000000001e-05 2024-09-04T17:48:49.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.873 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.873 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:49.873 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032569+0000", 2024-09-04T17:48:49.873 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4012000000000001e-05 2024-09-04T17:48:49.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.874 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:49.874 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032649+0000", 2024-09-04T17:48:49.874 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9782000000000003e-05 2024-09-04T17:48:49.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.874 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-09-04T17:48:49.874 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032657+0000", 2024-09-04T17:48:49.874 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7999999999999999e-06 2024-09-04T17:48:49.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.875 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.875 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:49.875 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.035521+0000", 2024-09-04T17:48:49.875 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028640850000000002 2024-09-04T17:48:49.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.875 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.875 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-04T17:48:49.875 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.035534+0000", 2024-09-04T17:48:49.875 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3157e-05 2024-09-04T17:48:49.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.876 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.876 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:49.876 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.869836+0000", 2024-09-04T17:48:49.876 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.83430168400000004 2024-09-04T17:48:49.876 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.876 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.876 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:49.876 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.870070+0000", 2024-09-04T17:48:49.876 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023443300000000001 2024-09-04T17:48:49.876 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.877 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.877 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 3", 2024-09-04T17:48:49.877 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.870423+0000", 2024-09-04T17:48:49.877 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00035290700000000001 2024-09-04T17:48:49.877 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.877 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.877 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2024-09-04T17:48:49.877 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.871254+0000", 2024-09-04T17:48:49.877 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00083113700000000002 2024-09-04T17:48:49.878 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.878 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.878 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:49.878 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.874234+0000", 2024-09-04T17:48:49.878 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029795189999999999 2024-09-04T17:48:49.878 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.878 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.879 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-04T17:48:49.879 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.874321+0000", 2024-09-04T17:48:49.879 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7637000000000001e-05 2024-09-04T17:48:49.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.879 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.879 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:49.879 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.874538+0000", 2024-09-04T17:48:49.879 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021625400000000001 2024-09-04T17:48:49.879 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:49.879 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:49.879 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:49.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.880 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.880 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:99511 3.18 3:1bf8b1d0:::rbd_header.11346a2b0bc9:head [watch reconnect cookie 140018821496000 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147)", 2024-09-04T17:48:49.880 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:48:27.032602+0000", 2024-09-04T17:48:49.880 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.704486458000002, 2024-09-04T17:48:49.880 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.84736467999999998, 2024-09-04T17:48:49.880 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:49.880 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:49.880 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:49.880 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:49.880 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:49.880 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:49.881 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 99511 2024-09-04T17:48:49.881 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.881 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:49.881 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.881 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:49.881 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032602+0000", 2024-09-04T17:48:49.881 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:49.881 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.881 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.881 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:49.882 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032602+0000", 2024-09-04T17:48:49.882 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:49.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.882 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:49.882 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032603+0000", 2024-09-04T17:48:49.882 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3300000000000001e-07 2024-09-04T17:48:49.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.883 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.883 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:49.883 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032610+0000", 2024-09-04T17:48:49.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.754e-06 2024-09-04T17:48:49.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.883 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.883 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:49.883 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032621+0000", 2024-09-04T17:48:49.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1175999999999999e-05 2024-09-04T17:48:49.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.884 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:49.884 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032630+0000", 2024-09-04T17:48:49.884 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7900000000000005e-06 2024-09-04T17:48:49.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.884 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:49.884 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032900+0000", 2024-09-04T17:48:49.884 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026967499999999998 2024-09-04T17:48:49.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.885 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting_for_map not empty", 2024-09-04T17:48:49.885 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.032907+0000", 2024-09-04T17:48:49.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6969999999999993e-06 2024-09-04T17:48:49.885 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.885 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.885 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:49.885 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.036096+0000", 2024-09-04T17:48:49.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031887090000000001 2024-09-04T17:48:49.885 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.885 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.885 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for peered", 2024-09-04T17:48:49.885 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.036107+0000", 2024-09-04T17:48:49.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1361e-05 2024-09-04T17:48:49.886 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.886 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.886 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:49.886 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.870783+0000", 2024-09-04T17:48:49.886 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.834675265 2024-09-04T17:48:49.886 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.886 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.886 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-09-04T17:48:49.886 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.870943+0000", 2024-09-04T17:48:49.886 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000160364 2024-09-04T17:48:49.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.887 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:49.887 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.874588+0000", 2024-09-04T17:48:49.887 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0036446529999999999 2024-09-04T17:48:49.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.888 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.888 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:49.888 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.874731+0000", 2024-09-04T17:48:49.888 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014334499999999999 2024-09-04T17:48:49.888 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:49.888 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:49.889 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:49.889 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:48:27.879967+0000", 2024-09-04T17:48:49.889 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005236058 2024-09-04T17:48:49.889 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:49.889 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:49.889 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:49.890 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:49.890 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:49.890 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:49.892 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-04T17:48:49.892 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:48:49.886+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:48:49.987 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is unset 2024-09-04T17:48:49.992 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:48:49.989+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:48:49.993 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.4 to 1 (10000) 2024-09-04T17:48:50.029 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-04T17:48:50.080 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:50.081 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:48:50.081 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:48:50.081 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:50.093 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:50.090+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:50.098 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-04T17:48:50.194 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:50.190+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:50.266 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:50.267 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:48:50.267 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:48:50.267 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:48:50.267 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:50.279 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-04T17:48:50.295 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:50.293+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:50.396 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:48:50.390+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:48:50.433 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:50.433 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:48:50.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:48:50.433 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:48:50.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.433 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59950 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5365760~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:48:50.433 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654917+0000", 2024-09-04T17:48:50.434 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 96.777212227000007, 2024-09-04T17:48:50.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066822450000000005, 2024-09-04T17:48:50.434 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:50.434 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:50.434 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:50.434 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:50.434 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:50.434 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:50.434 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59950 2024-09-04T17:48:50.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.434 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:50.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.434 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:50.434 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654917+0000", 2024-09-04T17:48:50.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:50.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:50.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654917+0000", 2024-09-04T17:48:50.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:50.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:50.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654918+0000", 2024-09-04T17:48:50.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.29e-07 2024-09-04T17:48:50.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:50.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654919+0000", 2024-09-04T17:48:50.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2589999999999999e-06 2024-09-04T17:48:50.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:50.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654922+0000", 2024-09-04T17:48:50.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1350000000000001e-06 2024-09-04T17:48:50.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:50.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654925+0000", 2024-09-04T17:48:50.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1250000000000001e-06 2024-09-04T17:48:50.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:50.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721466+0000", 2024-09-04T17:48:50.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066540959999999996 2024-09-04T17:48:50.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:50.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721574+0000", 2024-09-04T17:48:50.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010747399999999999 2024-09-04T17:48:50.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:50.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721740+0000", 2024-09-04T17:48:50.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016626799999999999 2024-09-04T17:48:50.438 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:50.438 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:50.438 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:50.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.438 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59951 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5472256~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:48:50.438 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654929+0000", 2024-09-04T17:48:50.438 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 96.777201042000002, 2024-09-04T17:48:50.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067110534999999999, 2024-09-04T17:48:50.438 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:50.438 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:50.438 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:50.438 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:50.439 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:50.439 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:50.439 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59951 2024-09-04T17:48:50.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.439 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:50.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.439 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:50.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654929+0000", 2024-09-04T17:48:50.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:50.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.439 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:50.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654929+0000", 2024-09-04T17:48:50.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:50.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:50.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654929+0000", 2024-09-04T17:48:50.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5100000000000001e-07 2024-09-04T17:48:50.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:50.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654930+0000", 2024-09-04T17:48:50.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2440000000000001e-06 2024-09-04T17:48:50.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:50.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654933+0000", 2024-09-04T17:48:50.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1599999999999998e-06 2024-09-04T17:48:50.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:50.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654936+0000", 2024-09-04T17:48:50.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6560000000000001e-06 2024-09-04T17:48:50.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:50.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721789+0000", 2024-09-04T17:48:50.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066853250000000003 2024-09-04T17:48:50.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:50.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721885+0000", 2024-09-04T17:48:50.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5616000000000001e-05 2024-09-04T17:48:50.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:50.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722039+0000", 2024-09-04T17:48:50.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000154258 2024-09-04T17:48:50.442 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:50.442 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:50.442 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:50.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.443 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59952 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5505024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:48:50.443 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654939+0000", 2024-09-04T17:48:50.443 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 96.777190164999993, 2024-09-04T17:48:50.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067433742000000005, 2024-09-04T17:48:50.443 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:50.443 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:50.443 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:50.443 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:50.443 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:50.443 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:50.443 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59952 2024-09-04T17:48:50.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.443 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:50.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:50.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654939+0000", 2024-09-04T17:48:50.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:50.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:50.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654939+0000", 2024-09-04T17:48:50.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:50.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:50.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654940+0000", 2024-09-04T17:48:50.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3300000000000001e-07 2024-09-04T17:48:50.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:50.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654941+0000", 2024-09-04T17:48:50.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1570000000000001e-06 2024-09-04T17:48:50.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:50.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654944+0000", 2024-09-04T17:48:50.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1319999999999998e-06 2024-09-04T17:48:50.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:50.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654947+0000", 2024-09-04T17:48:50.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5969999999999999e-06 2024-09-04T17:48:50.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:50.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722089+0000", 2024-09-04T17:48:50.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067142347000000005 2024-09-04T17:48:50.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:50.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722201+0000", 2024-09-04T17:48:50.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011235600000000001 2024-09-04T17:48:50.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:50.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722373+0000", 2024-09-04T17:48:50.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017191999999999999 2024-09-04T17:48:50.447 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:50.447 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:50.447 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:50.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.447 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59953 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5611520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:48:50.447 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654950+0000", 2024-09-04T17:48:50.447 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 96.777179524999994, 2024-09-04T17:48:50.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067601852000000004, 2024-09-04T17:48:50.447 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:50.447 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:50.448 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:50.448 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:50.448 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:50.448 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:50.448 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59953 2024-09-04T17:48:50.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.448 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:50.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:50.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654950+0000", 2024-09-04T17:48:50.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:50.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:50.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654950+0000", 2024-09-04T17:48:50.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:50.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:50.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654950+0000", 2024-09-04T17:48:50.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2499999999999999e-07 2024-09-04T17:48:50.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.450 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:50.450 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654954+0000", 2024-09-04T17:48:50.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.287e-06 2024-09-04T17:48:50.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.450 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:50.450 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654962+0000", 2024-09-04T17:48:50.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4230000000000005e-06 2024-09-04T17:48:50.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.451 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:50.451 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654966+0000", 2024-09-04T17:48:50.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6449999999999998e-06 2024-09-04T17:48:50.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.451 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:50.451 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722417+0000", 2024-09-04T17:48:50.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067451188999999995 2024-09-04T17:48:50.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:50.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722467+0000", 2024-09-04T17:48:50.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0556999999999997e-05 2024-09-04T17:48:50.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:50.453 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722552+0000", 2024-09-04T17:48:50.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4525999999999997e-05 2024-09-04T17:48:50.453 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:50.453 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:50.453 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:50.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.454 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59954 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5718016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:48:50.454 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654970+0000", 2024-09-04T17:48:50.454 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 96.777159764000004, 2024-09-04T17:48:50.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067874285000000006, 2024-09-04T17:48:50.454 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:50.454 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:50.454 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:50.454 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:50.454 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:50.455 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:50.455 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59954 2024-09-04T17:48:50.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.455 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:50.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:50.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654970+0000", 2024-09-04T17:48:50.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:50.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:50.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654970+0000", 2024-09-04T17:48:50.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:50.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:50.458 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654970+0000", 2024-09-04T17:48:50.458 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5899999999999998e-07 2024-09-04T17:48:50.458 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.458 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.458 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:50.459 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654971+0000", 2024-09-04T17:48:50.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2929999999999999e-06 2024-09-04T17:48:50.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:50.459 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654975+0000", 2024-09-04T17:48:50.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4699999999999998e-06 2024-09-04T17:48:50.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:50.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654978+0000", 2024-09-04T17:48:50.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7930000000000002e-06 2024-09-04T17:48:50.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:50.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722600+0000", 2024-09-04T17:48:50.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067622229000000006 2024-09-04T17:48:50.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.462 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:50.462 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722689+0000", 2024-09-04T17:48:50.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9511000000000004e-05 2024-09-04T17:48:50.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.462 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:50.463 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722844+0000", 2024-09-04T17:48:50.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015473 2024-09-04T17:48:50.463 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:50.463 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:50.463 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:50.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.463 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59955 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5857280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:48:50.464 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654981+0000", 2024-09-04T17:48:50.464 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 96.777148413999996, 2024-09-04T17:48:50.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068052360000000006, 2024-09-04T17:48:50.464 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:50.464 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:50.464 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:50.464 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:50.464 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:50.464 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:50.464 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59955 2024-09-04T17:48:50.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.465 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:50.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:50.465 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654981+0000", 2024-09-04T17:48:50.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:50.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:50.465 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654981+0000", 2024-09-04T17:48:50.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:50.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:50.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654981+0000", 2024-09-04T17:48:50.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2399999999999999e-07 2024-09-04T17:48:50.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:50.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654985+0000", 2024-09-04T17:48:50.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0980000000000004e-06 2024-09-04T17:48:50.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:50.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654990+0000", 2024-09-04T17:48:50.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3429999999999998e-06 2024-09-04T17:48:50.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.467 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.467 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:50.467 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654993+0000", 2024-09-04T17:48:50.467 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8700000000000001e-06 2024-09-04T17:48:50.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.467 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.467 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:50.467 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722888+0000", 2024-09-04T17:48:50.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067895239999999996 2024-09-04T17:48:50.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.468 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:50.468 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722948+0000", 2024-09-04T17:48:50.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9750999999999998e-05 2024-09-04T17:48:50.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.468 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:50.468 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.723034+0000", 2024-09-04T17:48:50.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5834000000000005e-05 2024-09-04T17:48:50.468 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:50.468 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:50.468 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:50.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.469 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59956 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5963776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:48:50.469 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654998+0000", 2024-09-04T17:48:50.469 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 96.777131885000003, 2024-09-04T17:48:50.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068359518999999994, 2024-09-04T17:48:50.469 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:50.469 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:50.469 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:50.469 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:50.469 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:50.470 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:50.470 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59956 2024-09-04T17:48:50.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.470 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:50.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.470 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:50.470 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654998+0000", 2024-09-04T17:48:50.470 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:50.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.470 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:50.470 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654998+0000", 2024-09-04T17:48:50.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:50.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:50.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654998+0000", 2024-09-04T17:48:50.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4299999999999999e-07 2024-09-04T17:48:50.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:50.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654999+0000", 2024-09-04T17:48:50.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1799999999999999e-06 2024-09-04T17:48:50.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.472 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:50.472 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.655002+0000", 2024-09-04T17:48:50.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3639999999999999e-06 2024-09-04T17:48:50.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.472 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:50.472 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.655005+0000", 2024-09-04T17:48:50.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7970000000000001e-06 2024-09-04T17:48:50.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.473 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:50.473 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.723085+0000", 2024-09-04T17:48:50.473 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068080229000000006 2024-09-04T17:48:50.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.473 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:50.473 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.723197+0000", 2024-09-04T17:48:50.474 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000111344 2024-09-04T17:48:50.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.474 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:50.474 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.723357+0000", 2024-09-04T17:48:50.474 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016036199999999999 2024-09-04T17:48:50.474 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:50.474 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:50.474 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:50.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.475 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61082 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 2654208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:50.475 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.822664+0000", 2024-09-04T17:48:50.475 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 96.609465498000006, 2024-09-04T17:48:50.475 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067072242000000004, 2024-09-04T17:48:50.475 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:50.475 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:50.475 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:50.475 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:50.475 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:50.476 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:50.476 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61082 2024-09-04T17:48:50.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.476 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:50.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:50.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822664+0000", 2024-09-04T17:48:50.476 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:50.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.477 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:50.477 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822664+0000", 2024-09-04T17:48:50.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:50.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.477 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:50.477 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822664+0000", 2024-09-04T17:48:50.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6600000000000003e-07 2024-09-04T17:48:50.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.478 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.478 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:50.478 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822666+0000", 2024-09-04T17:48:50.478 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.536e-06 2024-09-04T17:48:50.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.478 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.478 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:50.478 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822669+0000", 2024-09-04T17:48:50.478 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3759999999999999e-06 2024-09-04T17:48:50.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.479 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:50.479 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822672+0000", 2024-09-04T17:48:50.479 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8109999999999999e-06 2024-09-04T17:48:50.479 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.479 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:50.479 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889263+0000", 2024-09-04T17:48:50.479 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066591259999999999 2024-09-04T17:48:50.479 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.480 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.480 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:50.480 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889369+0000", 2024-09-04T17:48:50.480 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000105362 2024-09-04T17:48:50.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.480 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.480 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:50.480 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889736+0000", 2024-09-04T17:48:50.481 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00036763099999999999 2024-09-04T17:48:50.481 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:50.481 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:50.481 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:50.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.481 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61083 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 2760704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:50.482 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.822678+0000", 2024-09-04T17:48:50.482 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 96.609451763999999, 2024-09-04T17:48:50.482 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067475996999999996, 2024-09-04T17:48:50.482 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:50.482 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:50.482 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:50.482 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:50.483 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:50.483 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:50.483 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61083 2024-09-04T17:48:50.483 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.483 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:50.483 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.483 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:50.483 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822678+0000", 2024-09-04T17:48:50.483 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:50.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.484 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.484 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:50.484 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822678+0000", 2024-09-04T17:48:50.484 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:50.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.484 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.484 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:50.485 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822725+0000", 2024-09-04T17:48:50.485 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7429000000000003e-05 2024-09-04T17:48:50.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.485 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.485 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:50.486 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822728+0000", 2024-09-04T17:48:50.486 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2140000000000001e-06 2024-09-04T17:48:50.486 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.486 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.486 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:50.487 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822737+0000", 2024-09-04T17:48:50.487 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4370000000000007e-06 2024-09-04T17:48:50.487 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.487 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.487 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:50.487 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822743+0000", 2024-09-04T17:48:50.488 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7520000000000002e-06 2024-09-04T17:48:50.488 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.488 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.488 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:50.488 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889785+0000", 2024-09-04T17:48:50.488 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067042265000000004 2024-09-04T17:48:50.489 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.489 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.489 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:50.489 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889840+0000", 2024-09-04T17:48:50.489 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5254000000000003e-05 2024-09-04T17:48:50.489 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.490 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.490 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:50.490 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.890154+0000", 2024-09-04T17:48:50.490 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000313646 2024-09-04T17:48:50.490 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:50.490 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:50.490 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:50.491 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.491 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.491 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61084 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 2867200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:50.491 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.822753+0000", 2024-09-04T17:48:50.491 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 96.609376827999995, 2024-09-04T17:48:50.491 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068497161000000001, 2024-09-04T17:48:50.492 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:50.492 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:50.492 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:50.492 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:50.492 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:50.493 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:50.493 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61084 2024-09-04T17:48:50.493 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.493 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:50.493 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.493 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:50.493 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822753+0000", 2024-09-04T17:48:50.493 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:50.493 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.493 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.494 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:50.494 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822753+0000", 2024-09-04T17:48:50.494 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:50.494 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.494 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.494 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:50.494 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822753+0000", 2024-09-04T17:48:50.494 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9100000000000003e-07 2024-09-04T17:48:50.494 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.494 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.494 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:50.494 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822756+0000", 2024-09-04T17:48:50.495 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.678e-06 2024-09-04T17:48:50.495 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.495 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.495 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:50.495 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822763+0000", 2024-09-04T17:48:50.495 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8220000000000001e-06 2024-09-04T17:48:50.495 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.495 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.495 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:50.495 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822768+0000", 2024-09-04T17:48:50.495 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1930000000000001e-06 2024-09-04T17:48:50.495 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.495 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.496 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:50.496 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.890246+0000", 2024-09-04T17:48:50.496 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067477996999999998 2024-09-04T17:48:50.496 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.496 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.496 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:50.496 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.890555+0000", 2024-09-04T17:48:50.496 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000309143 2024-09-04T17:48:50.496 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.496 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.496 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:50.496 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.891250+0000", 2024-09-04T17:48:50.497 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00069463700000000001 2024-09-04T17:48:50.497 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:50.497 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:50.497 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:50.497 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.497 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.497 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61285 3.8 3:117cfeb9:::rbd_data.11341dfde0c5.0000000000000006:1f [read 2760704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:50.497 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835200+0000", 2024-09-04T17:48:50.497 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 96.596929760999998, 2024-09-04T17:48:50.497 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066899803999999993, 2024-09-04T17:48:50.497 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:50.497 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:50.498 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:50.498 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:50.498 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:50.498 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:50.498 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61285 2024-09-04T17:48:50.498 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.498 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:50.498 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.498 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:50.498 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835200+0000", 2024-09-04T17:48:50.498 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:50.498 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.498 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.498 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:50.499 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835200+0000", 2024-09-04T17:48:50.499 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:50.499 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.499 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.499 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:50.499 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835200+0000", 2024-09-04T17:48:50.499 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0899999999999997e-07 2024-09-04T17:48:50.499 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.499 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.499 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:50.499 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835206+0000", 2024-09-04T17:48:50.499 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3280000000000002e-06 2024-09-04T17:48:50.500 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.500 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.500 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:50.500 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835211+0000", 2024-09-04T17:48:50.500 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9230000000000001e-06 2024-09-04T17:48:50.500 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.500 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.500 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:50.500 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835216+0000", 2024-09-04T17:48:50.500 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9019999999999998e-06 2024-09-04T17:48:50.500 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.500 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.501 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:50.501 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.901719+0000", 2024-09-04T17:48:50.501 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066502981000000003 2024-09-04T17:48:50.501 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.501 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.501 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:50.501 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.901813+0000", 2024-09-04T17:48:50.501 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3905000000000003e-05 2024-09-04T17:48:50.501 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.501 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.502 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:50.502 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902100+0000", 2024-09-04T17:48:50.502 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028645599999999998 2024-09-04T17:48:50.502 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:50.502 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:50.502 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:50.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.502 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61286 3.8 3:117cfeb9:::rbd_data.11341dfde0c5.0000000000000006:1f [read 2867200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:50.502 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835244+0000", 2024-09-04T17:48:50.502 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 96.596885142000005, 2024-09-04T17:48:50.502 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067143215000000006, 2024-09-04T17:48:50.503 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:50.503 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:50.503 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:50.503 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:50.503 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:50.503 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:50.503 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61286 2024-09-04T17:48:50.503 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.503 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:50.503 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.503 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:50.503 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835244+0000", 2024-09-04T17:48:50.503 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:50.503 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.504 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.504 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:50.504 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835244+0000", 2024-09-04T17:48:50.504 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:50.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.504 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.504 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:50.504 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835245+0000", 2024-09-04T17:48:50.504 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8400000000000004e-07 2024-09-04T17:48:50.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.504 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.504 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:50.505 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835248+0000", 2024-09-04T17:48:50.505 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6850000000000001e-06 2024-09-04T17:48:50.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.505 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:50.505 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835254+0000", 2024-09-04T17:48:50.505 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.494e-06 2024-09-04T17:48:50.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.505 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:50.505 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835260+0000", 2024-09-04T17:48:50.505 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0580000000000003e-06 2024-09-04T17:48:50.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.506 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.506 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:50.506 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902119+0000", 2024-09-04T17:48:50.506 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066858318 2024-09-04T17:48:50.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.506 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.506 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:50.506 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902163+0000", 2024-09-04T17:48:50.506 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4753000000000003e-05 2024-09-04T17:48:50.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.506 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.506 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:50.506 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902388+0000", 2024-09-04T17:48:50.507 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022422300000000001 2024-09-04T17:48:50.507 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:50.507 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:50.507 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:50.507 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.507 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.507 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61315 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 98304~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:50.507 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835269+0000", 2024-09-04T17:48:50.507 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 96.59686035, 2024-09-04T17:48:50.507 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067329396, 2024-09-04T17:48:50.507 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:50.507 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:50.507 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:50.508 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:50.508 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:50.508 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:50.508 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61315 2024-09-04T17:48:50.508 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.508 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:50.508 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.508 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:50.508 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835269+0000", 2024-09-04T17:48:50.508 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:50.508 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.508 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.508 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:50.509 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835269+0000", 2024-09-04T17:48:50.509 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:50.509 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.509 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.509 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:50.509 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835270+0000", 2024-09-04T17:48:50.509 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6599999999999996e-07 2024-09-04T17:48:50.509 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.509 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.509 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:50.509 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835272+0000", 2024-09-04T17:48:50.509 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.261e-06 2024-09-04T17:48:50.509 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.510 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:50.510 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835278+0000", 2024-09-04T17:48:50.510 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.443e-06 2024-09-04T17:48:50.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.510 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:50.510 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835284+0000", 2024-09-04T17:48:50.510 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3290000000000001e-06 2024-09-04T17:48:50.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.510 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:50.510 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902147+0000", 2024-09-04T17:48:50.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066862695999999999 2024-09-04T17:48:50.511 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.511 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.511 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:50.511 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902248+0000", 2024-09-04T17:48:50.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000101501 2024-09-04T17:48:50.511 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.511 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.511 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:50.511 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902599+0000", 2024-09-04T17:48:50.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00035060000000000001 2024-09-04T17:48:50.511 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:50.512 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:50.512 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:50.512 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.512 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.512 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61316 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 204800~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:50.512 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835291+0000", 2024-09-04T17:48:50.512 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 96.596838331000001, 2024-09-04T17:48:50.512 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067632052999999998, 2024-09-04T17:48:50.512 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:50.512 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:50.512 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:50.513 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:50.513 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:50.513 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:50.513 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61316 2024-09-04T17:48:50.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.513 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:50.513 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.513 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:50.513 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835291+0000", 2024-09-04T17:48:50.513 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:50.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.514 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.514 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:50.514 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835291+0000", 2024-09-04T17:48:50.514 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:50.514 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.514 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.514 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:50.514 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835292+0000", 2024-09-04T17:48:50.514 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1899999999999998e-07 2024-09-04T17:48:50.514 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.514 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.515 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:50.515 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835294+0000", 2024-09-04T17:48:50.515 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0880000000000002e-06 2024-09-04T17:48:50.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.515 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:50.515 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835300+0000", 2024-09-04T17:48:50.515 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.224e-06 2024-09-04T17:48:50.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.515 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:50.516 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835305+0000", 2024-09-04T17:48:50.516 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0730000000000004e-06 2024-09-04T17:48:50.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.516 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.516 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:50.516 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902632+0000", 2024-09-04T17:48:50.516 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067327322999999994 2024-09-04T17:48:50.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.516 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.516 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:50.516 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902688+0000", 2024-09-04T17:48:50.517 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5177999999999999e-05 2024-09-04T17:48:50.517 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.517 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.517 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:50.517 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902923+0000", 2024-09-04T17:48:50.517 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023574800000000001 2024-09-04T17:48:50.517 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:50.517 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:50.518 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:50.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.518 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.518 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61317 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 344064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:50.518 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835312+0000", 2024-09-04T17:48:50.518 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 96.596817326999997, 2024-09-04T17:48:50.518 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067918511000000001, 2024-09-04T17:48:50.518 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:50.518 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:50.519 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:50.519 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:50.519 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:50.519 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:50.519 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61317 2024-09-04T17:48:50.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.520 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:50.520 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.520 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:50.520 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835312+0000", 2024-09-04T17:48:50.520 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:50.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.521 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.521 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:50.521 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835312+0000", 2024-09-04T17:48:50.522 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:50.522 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.522 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.522 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:50.522 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835313+0000", 2024-09-04T17:48:50.523 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9000000000000002e-07 2024-09-04T17:48:50.523 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.523 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.523 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:50.524 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835315+0000", 2024-09-04T17:48:50.524 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1689999999999999e-06 2024-09-04T17:48:50.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.524 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.524 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:50.525 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835321+0000", 2024-09-04T17:48:50.525 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0619999999999997e-06 2024-09-04T17:48:50.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.525 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.526 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:50.526 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835326+0000", 2024-09-04T17:48:50.526 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0390000000000004e-06 2024-09-04T17:48:50.526 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.526 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.527 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:50.527 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902967+0000", 2024-09-04T17:48:50.527 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067641520999999996 2024-09-04T17:48:50.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.528 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.528 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:50.528 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903067+0000", 2024-09-04T17:48:50.528 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9426999999999995e-05 2024-09-04T17:48:50.528 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.529 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.529 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:50.529 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903231+0000", 2024-09-04T17:48:50.529 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016390299999999999 2024-09-04T17:48:50.530 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:50.530 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:50.530 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:50.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.530 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.531 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61318 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 450560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:50.531 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835334+0000", 2024-09-04T17:48:50.531 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 96.596795810000003, 2024-09-04T17:48:50.531 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068091771999999995, 2024-09-04T17:48:50.532 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:50.532 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:50.532 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:50.532 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:50.532 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:50.533 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:50.533 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61318 2024-09-04T17:48:50.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.533 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:50.533 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.534 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:50.534 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835334+0000", 2024-09-04T17:48:50.534 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:50.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:50.535 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835334+0000", 2024-09-04T17:48:50.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:50.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.536 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:50.536 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835334+0000", 2024-09-04T17:48:50.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5699999999999998e-07 2024-09-04T17:48:50.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.536 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.537 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:50.537 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835337+0000", 2024-09-04T17:48:50.537 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3889999999999999e-06 2024-09-04T17:48:50.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.538 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.538 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:50.538 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835343+0000", 2024-09-04T17:48:50.538 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1809999999999998e-06 2024-09-04T17:48:50.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.539 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:50.539 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835348+0000", 2024-09-04T17:48:50.539 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3170000000000001e-06 2024-09-04T17:48:50.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:50.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903270+0000", 2024-09-04T17:48:50.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067922191000000007 2024-09-04T17:48:50.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.542 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:50.542 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903321+0000", 2024-09-04T17:48:50.542 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0889e-05 2024-09-04T17:48:50.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.543 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.543 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:50.543 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903426+0000", 2024-09-04T17:48:50.543 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104348 2024-09-04T17:48:50.544 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:50.544 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:50.544 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:50.544 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.545 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.545 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61319 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 589824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:50.545 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835357+0000", 2024-09-04T17:48:50.545 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 96.596772993000002, 2024-09-04T17:48:50.545 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068372535999999998, 2024-09-04T17:48:50.546 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:50.546 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:50.546 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:50.546 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:50.547 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:50.547 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:50.547 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61319 2024-09-04T17:48:50.547 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.547 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:50.548 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.548 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:50.548 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835357+0000", 2024-09-04T17:48:50.548 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:50.548 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.549 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.549 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:50.549 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835357+0000", 2024-09-04T17:48:50.549 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:50.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.550 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.550 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:50.550 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835357+0000", 2024-09-04T17:48:50.550 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.32e-07 2024-09-04T17:48:50.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.551 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.551 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:50.551 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835359+0000", 2024-09-04T17:48:50.551 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1660000000000001e-06 2024-09-04T17:48:50.551 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.552 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.552 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:50.552 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835366+0000", 2024-09-04T17:48:50.552 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3550000000000001e-06 2024-09-04T17:48:50.552 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.553 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.553 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:50.553 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835371+0000", 2024-09-04T17:48:50.553 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1050000000000003e-06 2024-09-04T17:48:50.553 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.553 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.554 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:50.554 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903471+0000", 2024-09-04T17:48:50.554 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068100451000000006 2024-09-04T17:48:50.554 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.555 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.555 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:50.555 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903573+0000", 2024-09-04T17:48:50.555 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000101691 2024-09-04T17:48:50.555 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.555 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.556 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:50.556 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903729+0000", 2024-09-04T17:48:50.556 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000156336 2024-09-04T17:48:50.556 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:50.556 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:50.557 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:50.557 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.557 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.557 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61320 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:50.557 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835380+0000", 2024-09-04T17:48:50.557 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 96.596749227999993, 2024-09-04T17:48:50.557 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068543058000000004, 2024-09-04T17:48:50.558 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:50.558 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:50.558 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:50.558 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:50.558 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:50.558 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:50.558 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61320 2024-09-04T17:48:50.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.558 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:50.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.558 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:50.558 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835380+0000", 2024-09-04T17:48:50.559 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:50.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.559 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.559 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:50.559 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835380+0000", 2024-09-04T17:48:50.559 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:50.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.559 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.559 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:50.559 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835381+0000", 2024-09-04T17:48:50.559 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9999999999999998e-07 2024-09-04T17:48:50.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.559 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.559 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:50.560 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835383+0000", 2024-09-04T17:48:50.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2230000000000001e-06 2024-09-04T17:48:50.560 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.560 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:50.560 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835390+0000", 2024-09-04T17:48:50.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.34e-06 2024-09-04T17:48:50.560 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.560 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:50.560 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835396+0000", 2024-09-04T17:48:50.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9830000000000001e-06 2024-09-04T17:48:50.560 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.561 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:50.561 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903765+0000", 2024-09-04T17:48:50.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068368441000000002 2024-09-04T17:48:50.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.561 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:50.561 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903835+0000", 2024-09-04T17:48:50.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0631999999999997e-05 2024-09-04T17:48:50.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.561 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:50.561 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903923+0000", 2024-09-04T17:48:50.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8039000000000002e-05 2024-09-04T17:48:50.562 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:50.562 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:50.562 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:50.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.562 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.562 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61321 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 835584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:50.562 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835404+0000", 2024-09-04T17:48:50.562 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 96.596725168999996, 2024-09-04T17:48:50.562 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068911043000000005, 2024-09-04T17:48:50.562 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:50.562 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:50.562 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:50.562 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:50.562 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:50.563 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:50.563 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61321 2024-09-04T17:48:50.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.563 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:50.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.563 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:50.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835404+0000", 2024-09-04T17:48:50.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:50.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.563 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:50.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835404+0000", 2024-09-04T17:48:50.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:50.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:50.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835405+0000", 2024-09-04T17:48:50.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.89e-07 2024-09-04T17:48:50.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:50.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835407+0000", 2024-09-04T17:48:50.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4439999999999998e-06 2024-09-04T17:48:50.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:50.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835414+0000", 2024-09-04T17:48:50.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4930000000000003e-06 2024-09-04T17:48:50.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.565 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:50.565 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835419+0000", 2024-09-04T17:48:50.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8919999999999999e-06 2024-09-04T17:48:50.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.565 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:50.565 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903957+0000", 2024-09-04T17:48:50.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068538500000000002 2024-09-04T17:48:50.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.566 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:50.566 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904019+0000", 2024-09-04T17:48:50.566 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1419000000000003e-05 2024-09-04T17:48:50.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.566 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:50.566 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904315+0000", 2024-09-04T17:48:50.566 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00029680599999999999 2024-09-04T17:48:50.566 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:50.566 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:50.566 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:50.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.567 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61322 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:50.567 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835427+0000", 2024-09-04T17:48:50.567 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 96.596702782999998, 2024-09-04T17:48:50.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.069550138999999997, 2024-09-04T17:48:50.567 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:50.567 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:50.567 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:50.567 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:50.567 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:50.567 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:50.567 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61322 2024-09-04T17:48:50.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.567 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:50.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:50.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835427+0000", 2024-09-04T17:48:50.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:50.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:50.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835427+0000", 2024-09-04T17:48:50.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:50.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:50.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.842157+0000", 2024-09-04T17:48:50.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067298419999999998 2024-09-04T17:48:50.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:50.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.842163+0000", 2024-09-04T17:48:50.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2029999999999997e-06 2024-09-04T17:48:50.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:50.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.842175+0000", 2024-09-04T17:48:50.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2072000000000001e-05 2024-09-04T17:48:50.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:50.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.842183+0000", 2024-09-04T17:48:50.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3850000000000002e-06 2024-09-04T17:48:50.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:50.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904371+0000", 2024-09-04T17:48:50.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.062187754999999997 2024-09-04T17:48:50.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:50.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904467+0000", 2024-09-04T17:48:50.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6117000000000001e-05 2024-09-04T17:48:50.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:50.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:50.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:50.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904977+0000", 2024-09-04T17:48:50.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00050976500000000002 2024-09-04T17:48:50.571 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:50.571 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:50.571 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:50.572 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:50.572 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:50.572 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:50.573 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-04T17:48:50.574 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:48:50.570+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:48:50.674 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:48:50.670+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:48:50.696 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:50.696 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:48:50.696 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:48:50.697 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:50.710 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-04T17:48:50.775 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:48:50.773+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:48:50.841 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:50.841 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:48:50.841 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:48:50.841 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:48:50.841 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:50.857 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-04T17:48:50.901 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:50.901+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:50.977 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:48:50.970+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:48:51.005 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.5 to 1 (10000) 2024-09-04T17:48:51.031 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-04T17:48:51.035 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:51.035 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:48:51.035 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:48:51.035 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:48:51.035 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.035 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:12722 3.1b 3:d8152af1:::rbd_data.1134ef3a5a7d.00000000000000bd:head [delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-04T17:48:51.036 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:16.421965+0000", 2024-09-04T17:48:51.036 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 154.610665525, 2024-09-04T17:48:51.036 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.890002811, 2024-09-04T17:48:51.036 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:51.036 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:51.036 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:51.036 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:51.036 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:51.036 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:51.036 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 12722 2024-09-04T17:48:51.036 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.036 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:51.037 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.037 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:51.037 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421965+0000", 2024-09-04T17:48:51.037 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.037 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.037 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.037 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:51.037 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421965+0000", 2024-09-04T17:48:51.037 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.037 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.037 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.038 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:51.038 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421966+0000", 2024-09-04T17:48:51.038 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.102e-06 2024-09-04T17:48:51.038 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.038 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.038 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:51.038 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421971+0000", 2024-09-04T17:48:51.038 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9989999999999999e-06 2024-09-04T17:48:51.038 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.038 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.038 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:51.038 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421981+0000", 2024-09-04T17:48:51.039 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.7380000000000006e-06 2024-09-04T17:48:51.039 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.039 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.039 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:51.039 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421989+0000", 2024-09-04T17:48:51.039 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.1890000000000007e-06 2024-09-04T17:48:51.039 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.039 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.039 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:51.039 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.422050+0000", 2024-09-04T17:48:51.039 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.0702000000000003e-05 2024-09-04T17:48:51.039 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.040 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.040 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for peered", 2024-09-04T17:48:51.040 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.422059+0000", 2024-09-04T17:48:51.040 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5440000000000004e-06 2024-09-04T17:48:51.040 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.040 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.040 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:51.040 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.308156+0000", 2024-09-04T17:48:51.040 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.88609676199999998 2024-09-04T17:48:51.040 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.040 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.040 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:51.041 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.308398+0000", 2024-09-04T17:48:51.041 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024215400000000001 2024-09-04T17:48:51.041 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.041 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.041 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 7", 2024-09-04T17:48:51.041 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.308795+0000", 2024-09-04T17:48:51.041 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000396704 2024-09-04T17:48:51.041 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.041 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.041 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:48:51.041 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.310831+0000", 2024-09-04T17:48:51.041 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0020364200000000002 2024-09-04T17:48:51.041 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.042 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.042 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:51.042 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.311818+0000", 2024-09-04T17:48:51.042 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00098649799999999993 2024-09-04T17:48:51.042 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.042 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.042 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:51.042 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.311905+0000", 2024-09-04T17:48:51.042 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7070999999999994e-05 2024-09-04T17:48:51.042 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.042 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.042 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:51.042 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.311968+0000", 2024-09-04T17:48:51.043 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.2928000000000005e-05 2024-09-04T17:48:51.043 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.043 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:51.043 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.043 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.043 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.043 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37359 3.f e58/24 3:f29e50a5:::rbd_data.1134c5a43465.0000000000000021:head v 58'135, mlcod=58'133)", 2024-09-04T17:48:51.043 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:48:51.043 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 131.41533492100001, 2024-09-04T17:48:51.043 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053483197000000003, 2024-09-04T17:48:51.043 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:51.043 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:51.044 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:51.044 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:51.044 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.044 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:51.044 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:48:51.044 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.044 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.044 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.045 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:51.045 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:48:51.045 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.045 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.045 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.045 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:51.045 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:48:51.045 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1699999999999997e-07 2024-09-04T17:48:51.045 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.045 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.045 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:51.045 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617338+0000", 2024-09-04T17:48:51.046 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.1168999999999998e-05 2024-09-04T17:48:51.046 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.046 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.046 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:51.046 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617351+0000", 2024-09-04T17:48:51.046 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3094e-05 2024-09-04T17:48:51.046 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.046 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.046 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:51.046 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617359+0000", 2024-09-04T17:48:51.046 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7970000000000006e-06 2024-09-04T17:48:51.046 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.047 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.047 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:51.047 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617417+0000", 2024-09-04T17:48:51.047 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7079e-05 2024-09-04T17:48:51.047 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.047 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.047 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:51.047 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617438+0000", 2024-09-04T17:48:51.047 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1654999999999998e-05 2024-09-04T17:48:51.047 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.047 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.047 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:51.048 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670738+0000", 2024-09-04T17:48:51.048 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053300142000000002 2024-09-04T17:48:51.048 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.048 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.048 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:51.048 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670779+0000", 2024-09-04T17:48:51.048 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0343999999999999e-05 2024-09-04T17:48:51.048 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.048 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:51.048 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.048 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.048 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.049 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37358 3.1d e58/24 3:bd3215f3:::rbd_data.1134c5a43465.0000000000000020:head v 58'91, mlcod=58'89)", 2024-09-04T17:48:51.049 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617502+0000", 2024-09-04T17:48:51.049 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 131.41512820299999, 2024-09-04T17:48:51.049 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053505641999999999, 2024-09-04T17:48:51.049 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:51.049 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:51.049 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:51.049 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:51.049 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.049 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:51.049 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617502+0000", 2024-09-04T17:48:51.050 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.050 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.050 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.050 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:51.050 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617502+0000", 2024-09-04T17:48:51.050 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.050 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.050 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.050 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:51.050 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617503+0000", 2024-09-04T17:48:51.050 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.184e-06 2024-09-04T17:48:51.050 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.051 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.051 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:51.051 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617526+0000", 2024-09-04T17:48:51.051 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2099999999999998e-05 2024-09-04T17:48:51.051 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.051 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.051 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:51.051 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617540+0000", 2024-09-04T17:48:51.051 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4741e-05 2024-09-04T17:48:51.051 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.051 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.051 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:51.052 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617550+0000", 2024-09-04T17:48:51.052 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4259999999999992e-06 2024-09-04T17:48:51.052 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.052 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.052 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:51.052 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617613+0000", 2024-09-04T17:48:51.052 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.3625000000000002e-05 2024-09-04T17:48:51.052 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.052 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.052 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:51.052 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617637+0000", 2024-09-04T17:48:51.053 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.3604999999999998e-05 2024-09-04T17:48:51.053 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.053 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.053 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:51.053 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670967+0000", 2024-09-04T17:48:51.053 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053329881000000003 2024-09-04T17:48:51.053 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.053 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.053 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:51.053 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671008+0000", 2024-09-04T17:48:51.053 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.108e-05 2024-09-04T17:48:51.053 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.054 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:51.054 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.054 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.054 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.054 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37361 3.1c e58/24 3:3c7f8976:::rbd_data.1134c5a43465.0000000000000023:head v 58'159, mlcod=58'157)", 2024-09-04T17:48:51.054 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617578+0000", 2024-09-04T17:48:51.054 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 131.41505235299999, 2024-09-04T17:48:51.054 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053990367999999997, 2024-09-04T17:48:51.054 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:51.054 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:51.054 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:51.054 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:51.054 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.055 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:51.055 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617578+0000", 2024-09-04T17:48:51.055 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.055 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.055 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.055 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:51.055 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617578+0000", 2024-09-04T17:48:51.055 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.055 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.055 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.055 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:51.055 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617579+0000", 2024-09-04T17:48:51.055 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7299999999999996e-07 2024-09-04T17:48:51.056 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.056 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.056 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:51.056 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617610+0000", 2024-09-04T17:48:51.056 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0951000000000002e-05 2024-09-04T17:48:51.056 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.056 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.056 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:51.056 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:48:51.056 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0002e-05 2024-09-04T17:48:51.056 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.056 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.056 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:51.057 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617626+0000", 2024-09-04T17:48:51.057 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6660000000000002e-06 2024-09-04T17:48:51.057 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.057 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.057 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:51.057 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670548+0000", 2024-09-04T17:48:51.057 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.052921943999999999 2024-09-04T17:48:51.057 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.057 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.057 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:51.057 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670574+0000", 2024-09-04T17:48:51.057 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5405000000000002e-05 2024-09-04T17:48:51.057 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.058 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.058 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:51.058 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671522+0000", 2024-09-04T17:48:51.058 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00094815299999999997 2024-09-04T17:48:51.058 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.058 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.058 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:51.058 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671568+0000", 2024-09-04T17:48:51.058 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6674000000000001e-05 2024-09-04T17:48:51.058 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.058 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:51.058 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.059 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.059 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.059 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37357 3.15 e58/49 3:a897312a:::rbd_data.1134c5a43465.000000000000001f:head v 58'117, mlcod=58'115)", 2024-09-04T17:48:51.059 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:48:51.059 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 131.41501001500001, 2024-09-04T17:48:51.059 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053734297, 2024-09-04T17:48:51.059 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:51.059 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:51.059 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:51.059 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:51.059 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.059 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:51.059 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:48:51.059 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.060 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.060 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.060 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:51.060 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:48:51.060 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.060 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.060 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.060 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:51.060 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617621+0000", 2024-09-04T17:48:51.060 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.7300000000000004e-07 2024-09-04T17:48:51.060 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.060 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.060 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:51.061 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617642+0000", 2024-09-04T17:48:51.061 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0185000000000001e-05 2024-09-04T17:48:51.061 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.061 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.061 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:51.061 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617654+0000", 2024-09-04T17:48:51.061 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2745000000000001e-05 2024-09-04T17:48:51.061 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.061 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.061 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:51.061 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617662+0000", 2024-09-04T17:48:51.061 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.0499999999999992e-06 2024-09-04T17:48:51.061 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.062 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.062 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:51.062 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617727+0000", 2024-09-04T17:48:51.062 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.4923000000000002e-05 2024-09-04T17:48:51.062 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.062 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.062 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:51.062 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617753+0000", 2024-09-04T17:48:51.062 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5899999999999999e-05 2024-09-04T17:48:51.062 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.062 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.062 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:51.062 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671298+0000", 2024-09-04T17:48:51.063 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053545276000000003 2024-09-04T17:48:51.063 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.063 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.063 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:51.063 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671355+0000", 2024-09-04T17:48:51.063 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6245000000000001e-05 2024-09-04T17:48:51.063 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.063 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:51.063 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.063 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.063 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.063 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44687 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:51.064 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253012+0000", 2024-09-04T17:48:51.064 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 120.77961794799999, 2024-09-04T17:48:51.064 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051128777, 2024-09-04T17:48:51.064 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:51.064 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:51.064 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:51.064 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:51.064 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:51.064 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:51.064 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44687 2024-09-04T17:48:51.064 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.064 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:51.065 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.065 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:51.065 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253012+0000", 2024-09-04T17:48:51.065 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.065 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.065 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.065 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:51.065 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253012+0000", 2024-09-04T17:48:51.065 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.065 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.065 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.065 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:51.066 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253014+0000", 2024-09-04T17:48:51.066 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.24e-06 2024-09-04T17:48:51.066 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.066 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.066 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:51.066 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253019+0000", 2024-09-04T17:48:51.066 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4569999999999996e-06 2024-09-04T17:48:51.066 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.066 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.066 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:51.066 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253035+0000", 2024-09-04T17:48:51.066 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6291999999999999e-05 2024-09-04T17:48:51.066 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.067 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.067 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:51.067 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253050+0000", 2024-09-04T17:48:51.067 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4423000000000001e-05 2024-09-04T17:48:51.067 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.067 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.067 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:51.067 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304115+0000", 2024-09-04T17:48:51.067 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051064778999999998 2024-09-04T17:48:51.067 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.067 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.068 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:51.068 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304141+0000", 2024-09-04T17:48:51.068 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6585999999999999e-05 2024-09-04T17:48:51.068 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.068 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:51.068 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.068 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.068 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.068 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44688 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:51.068 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253067+0000", 2024-09-04T17:48:51.068 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 120.779563396, 2024-09-04T17:48:51.069 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051116979, 2024-09-04T17:48:51.069 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:51.069 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:51.069 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:51.069 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:51.069 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:51.069 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:51.069 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44688 2024-09-04T17:48:51.069 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.069 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:51.069 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.069 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:51.070 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253067+0000", 2024-09-04T17:48:51.070 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.070 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.070 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.070 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:51.070 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253067+0000", 2024-09-04T17:48:51.070 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.070 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.070 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.070 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:51.070 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253068+0000", 2024-09-04T17:48:51.070 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0729999999999999e-06 2024-09-04T17:48:51.071 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.071 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.071 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:51.071 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253073+0000", 2024-09-04T17:48:51.071 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2259999999999996e-06 2024-09-04T17:48:51.071 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.071 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.071 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:51.071 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253091+0000", 2024-09-04T17:48:51.071 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7255999999999999e-05 2024-09-04T17:48:51.071 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.071 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.071 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:51.071 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253103+0000", 2024-09-04T17:48:51.072 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2632999999999999e-05 2024-09-04T17:48:51.072 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.072 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.072 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:51.072 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304173+0000", 2024-09-04T17:48:51.072 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051069537999999998 2024-09-04T17:48:51.072 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.072 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.072 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:51.072 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304184+0000", 2024-09-04T17:48:51.072 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1253000000000001e-05 2024-09-04T17:48:51.072 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.073 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:51.073 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.073 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.073 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.073 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44689 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:51.073 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253121+0000", 2024-09-04T17:48:51.073 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 120.77950925899999, 2024-09-04T17:48:51.073 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051192595, 2024-09-04T17:48:51.073 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:51.073 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:51.073 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:51.073 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:51.073 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:51.074 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:51.074 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44689 2024-09-04T17:48:51.074 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.074 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:51.074 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.074 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:51.074 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253121+0000", 2024-09-04T17:48:51.074 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.074 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.074 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.074 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:51.074 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253121+0000", 2024-09-04T17:48:51.074 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.074 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.075 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.075 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:51.075 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253122+0000", 2024-09-04T17:48:51.075 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.15e-06 2024-09-04T17:48:51.075 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.075 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.075 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:51.075 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253128+0000", 2024-09-04T17:48:51.075 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2569999999999998e-06 2024-09-04T17:48:51.075 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.075 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.075 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:51.075 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253145+0000", 2024-09-04T17:48:51.076 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7176e-05 2024-09-04T17:48:51.076 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.076 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.076 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:51.076 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253158+0000", 2024-09-04T17:48:51.076 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3139000000000001e-05 2024-09-04T17:48:51.076 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.076 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.076 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:51.076 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304287+0000", 2024-09-04T17:48:51.076 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051129149999999998 2024-09-04T17:48:51.076 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.076 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.077 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:51.077 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304314+0000", 2024-09-04T17:48:51.077 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6723000000000001e-05 2024-09-04T17:48:51.077 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.077 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:51.077 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.077 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.077 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.077 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44690 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:51.077 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253177+0000", 2024-09-04T17:48:51.077 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 120.77945313399999, 2024-09-04T17:48:51.077 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051169263, 2024-09-04T17:48:51.077 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:51.077 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:51.078 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:51.078 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:51.078 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:51.078 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:51.078 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44690 2024-09-04T17:48:51.078 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.078 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:51.078 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.078 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:51.078 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253177+0000", 2024-09-04T17:48:51.078 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.078 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.078 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.078 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:51.079 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253177+0000", 2024-09-04T17:48:51.079 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.079 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.079 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.079 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:51.079 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253178+0000", 2024-09-04T17:48:51.079 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1000000000000001e-06 2024-09-04T17:48:51.079 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.079 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.079 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:51.079 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253184+0000", 2024-09-04T17:48:51.079 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2089999999999996e-06 2024-09-04T17:48:51.079 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.080 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.080 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:51.080 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253200+0000", 2024-09-04T17:48:51.080 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6087e-05 2024-09-04T17:48:51.080 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.080 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.080 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:51.080 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253231+0000", 2024-09-04T17:48:51.080 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0874999999999998e-05 2024-09-04T17:48:51.080 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.080 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.080 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:51.080 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304337+0000", 2024-09-04T17:48:51.081 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051106430000000001 2024-09-04T17:48:51.081 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.081 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.081 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:51.081 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304347+0000", 2024-09-04T17:48:51.081 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5619999999999993e-06 2024-09-04T17:48:51.081 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.081 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:51.081 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.081 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.081 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.081 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44691 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:51.081 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253249+0000", 2024-09-04T17:48:51.082 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 120.77938109999999, 2024-09-04T17:48:51.082 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051195630999999998, 2024-09-04T17:48:51.082 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:51.082 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:51.082 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:51.082 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:51.082 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:51.082 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:51.082 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44691 2024-09-04T17:48:51.082 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.082 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:51.082 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.082 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:51.082 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253249+0000", 2024-09-04T17:48:51.083 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.083 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.083 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.083 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:51.083 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253249+0000", 2024-09-04T17:48:51.083 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.083 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.083 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.083 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:51.083 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253251+0000", 2024-09-04T17:48:51.083 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2330000000000001e-06 2024-09-04T17:48:51.083 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.083 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.084 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:51.084 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253256+0000", 2024-09-04T17:48:51.084 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6099999999999997e-06 2024-09-04T17:48:51.084 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.084 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.084 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:51.084 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253274+0000", 2024-09-04T17:48:51.084 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7952999999999999e-05 2024-09-04T17:48:51.084 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.084 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.084 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:51.084 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253287+0000", 2024-09-04T17:48:51.084 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2839e-05 2024-09-04T17:48:51.085 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.085 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.085 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:51.085 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304396+0000", 2024-09-04T17:48:51.085 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051108990999999999 2024-09-04T17:48:51.085 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.085 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.085 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:51.085 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304445+0000", 2024-09-04T17:48:51.085 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9005000000000003e-05 2024-09-04T17:48:51.085 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.085 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:51.086 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.086 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.086 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.086 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44692 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:51.086 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253337+0000", 2024-09-04T17:48:51.086 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 120.77929352699999, 2024-09-04T17:48:51.086 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051138909000000003, 2024-09-04T17:48:51.086 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:51.086 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:51.086 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:51.086 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:51.086 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:51.086 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:51.087 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44692 2024-09-04T17:48:51.087 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.087 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:51.087 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.087 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:51.087 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253337+0000", 2024-09-04T17:48:51.087 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.087 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.087 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.087 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:51.087 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253337+0000", 2024-09-04T17:48:51.087 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.088 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.088 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.088 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:51.088 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253338+0000", 2024-09-04T17:48:51.088 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.463e-06 2024-09-04T17:48:51.088 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.088 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.088 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:51.088 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253357+0000", 2024-09-04T17:48:51.088 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9043e-05 2024-09-04T17:48:51.088 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.088 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.088 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:51.089 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253379+0000", 2024-09-04T17:48:51.089 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1243e-05 2024-09-04T17:48:51.089 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.089 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.089 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:51.089 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253396+0000", 2024-09-04T17:48:51.089 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7059e-05 2024-09-04T17:48:51.089 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.089 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.089 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:51.089 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304466+0000", 2024-09-04T17:48:51.089 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051070624000000002 2024-09-04T17:48:51.090 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.090 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.090 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:51.090 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304476+0000", 2024-09-04T17:48:51.090 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4769999999999992e-06 2024-09-04T17:48:51.090 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.090 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:51.090 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.090 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.090 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.090 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44693 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:51.090 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253415+0000", 2024-09-04T17:48:51.091 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 120.779215124, 2024-09-04T17:48:51.091 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051141949999999999, 2024-09-04T17:48:51.091 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:51.091 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:51.091 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:51.091 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:51.091 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:51.091 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:51.091 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44693 2024-09-04T17:48:51.091 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.091 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:51.091 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.091 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:51.091 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253415+0000", 2024-09-04T17:48:51.092 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.092 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.092 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.092 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:51.092 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253415+0000", 2024-09-04T17:48:51.092 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.092 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.092 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.092 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:51.092 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253416+0000", 2024-09-04T17:48:51.092 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1799999999999999e-06 2024-09-04T17:48:51.092 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.092 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.093 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:51.093 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253422+0000", 2024-09-04T17:48:51.093 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.5940000000000001e-06 2024-09-04T17:48:51.093 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.093 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.093 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:51.093 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253438+0000", 2024-09-04T17:48:51.093 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6008999999999999e-05 2024-09-04T17:48:51.093 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.093 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.093 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:51.093 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253452+0000", 2024-09-04T17:48:51.094 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3543000000000001e-05 2024-09-04T17:48:51.094 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.094 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.094 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:51.094 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304533+0000", 2024-09-04T17:48:51.094 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051081609 2024-09-04T17:48:51.094 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.094 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.094 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:51.094 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304557+0000", 2024-09-04T17:48:51.095 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4015e-05 2024-09-04T17:48:51.095 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.095 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:51.095 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.095 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.095 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.095 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44694 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:51.095 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253469+0000", 2024-09-04T17:48:51.095 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 120.779160999, 2024-09-04T17:48:51.095 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051121486000000001, 2024-09-04T17:48:51.095 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:51.095 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:51.095 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:51.096 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:51.096 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:51.096 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:51.096 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44694 2024-09-04T17:48:51.096 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.096 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:51.096 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.096 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:51.096 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253469+0000", 2024-09-04T17:48:51.096 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.096 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.096 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.097 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:51.097 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253469+0000", 2024-09-04T17:48:51.097 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.097 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.097 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.097 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:51.097 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253471+0000", 2024-09-04T17:48:51.097 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1400000000000001e-06 2024-09-04T17:48:51.097 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.097 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.097 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:51.097 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253476+0000", 2024-09-04T17:48:51.098 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7370000000000003e-06 2024-09-04T17:48:51.098 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.098 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.098 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:51.098 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253493+0000", 2024-09-04T17:48:51.098 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6549e-05 2024-09-04T17:48:51.098 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.098 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.098 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:51.098 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253506+0000", 2024-09-04T17:48:51.098 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3406e-05 2024-09-04T17:48:51.098 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.099 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.099 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:51.099 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304582+0000", 2024-09-04T17:48:51.099 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051075534999999998 2024-09-04T17:48:51.099 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.099 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.099 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:51.099 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304591+0000", 2024-09-04T17:48:51.099 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1190000000000003e-06 2024-09-04T17:48:51.099 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.099 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:51.099 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.100 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.100 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.100 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44695 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:51.100 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253552+0000", 2024-09-04T17:48:51.100 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 120.77907824499999, 2024-09-04T17:48:51.100 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051068872000000001, 2024-09-04T17:48:51.100 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:51.100 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:51.100 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:51.100 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:51.100 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:51.100 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:51.100 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44695 2024-09-04T17:48:51.101 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.101 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:51.101 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.101 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:51.101 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253552+0000", 2024-09-04T17:48:51.101 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.101 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.101 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.101 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:51.101 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253552+0000", 2024-09-04T17:48:51.101 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.101 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.101 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.102 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:51.102 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253553+0000", 2024-09-04T17:48:51.102 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1960000000000001e-06 2024-09-04T17:48:51.102 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.102 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.102 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:51.102 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253560+0000", 2024-09-04T17:48:51.102 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.2040000000000002e-06 2024-09-04T17:48:51.102 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.102 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.102 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:51.102 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253578+0000", 2024-09-04T17:48:51.102 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8241999999999999e-05 2024-09-04T17:48:51.103 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.103 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.103 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:51.103 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253591+0000", 2024-09-04T17:48:51.103 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3453e-05 2024-09-04T17:48:51.103 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.103 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.103 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:51.103 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304614+0000", 2024-09-04T17:48:51.103 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051022456000000001 2024-09-04T17:48:51.103 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.103 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.103 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:51.104 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304621+0000", 2024-09-04T17:48:51.104 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3209999999999999e-06 2024-09-04T17:48:51.104 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.104 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:51.104 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.104 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.104 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.104 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44696 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:51.104 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253609+0000", 2024-09-04T17:48:51.104 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 120.779021744, 2024-09-04T17:48:51.104 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051115351000000003, 2024-09-04T17:48:51.104 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:51.104 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:51.105 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:51.105 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:51.105 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:51.105 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:51.105 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44696 2024-09-04T17:48:51.105 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.105 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:51.105 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.105 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:51.105 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253609+0000", 2024-09-04T17:48:51.105 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.105 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.105 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.106 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:51.106 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253609+0000", 2024-09-04T17:48:51.106 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.106 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.106 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.106 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:51.106 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253610+0000", 2024-09-04T17:48:51.106 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2300000000000001e-06 2024-09-04T17:48:51.106 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.106 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.106 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:51.106 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253615+0000", 2024-09-04T17:48:51.106 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7260000000000002e-06 2024-09-04T17:48:51.107 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.107 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.107 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:51.107 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253632+0000", 2024-09-04T17:48:51.107 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7039999999999999e-05 2024-09-04T17:48:51.107 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.107 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.107 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:51.107 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253666+0000", 2024-09-04T17:48:51.107 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4662000000000001e-05 2024-09-04T17:48:51.107 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.107 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.108 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:51.108 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304677+0000", 2024-09-04T17:48:51.108 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051011017999999998 2024-09-04T17:48:51.108 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.108 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.108 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:51.108 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304724+0000", 2024-09-04T17:48:51.108 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6675000000000003e-05 2024-09-04T17:48:51.108 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.108 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:51.108 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.108 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.108 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.109 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44697 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:51.109 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253684+0000", 2024-09-04T17:48:51.109 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 120.77894625, 2024-09-04T17:48:51.109 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051073107, 2024-09-04T17:48:51.109 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:51.109 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:51.109 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:51.109 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:51.109 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:51.109 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:51.109 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44697 2024-09-04T17:48:51.109 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.109 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:51.110 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.110 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:51.110 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253684+0000", 2024-09-04T17:48:51.110 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.110 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.110 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.110 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:51.110 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253684+0000", 2024-09-04T17:48:51.110 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.110 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.110 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.110 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:51.110 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253685+0000", 2024-09-04T17:48:51.110 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0699999999999999e-06 2024-09-04T17:48:51.111 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.111 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.111 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:51.111 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253690+0000", 2024-09-04T17:48:51.111 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1959999999999999e-06 2024-09-04T17:48:51.111 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.111 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.111 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:51.111 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253707+0000", 2024-09-04T17:48:51.111 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6946e-05 2024-09-04T17:48:51.111 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.111 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.111 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:51.112 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253720+0000", 2024-09-04T17:48:51.112 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3076e-05 2024-09-04T17:48:51.112 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.112 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.112 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:51.112 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304745+0000", 2024-09-04T17:48:51.112 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051024760000000002 2024-09-04T17:48:51.112 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.112 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.112 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:51.112 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304757+0000", 2024-09-04T17:48:51.112 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2058999999999999e-05 2024-09-04T17:48:51.112 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.113 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:51.113 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.113 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.113 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.113 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44698 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:51.113 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253737+0000", 2024-09-04T17:48:51.113 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 120.778893899, 2024-09-04T17:48:51.113 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051176078, 2024-09-04T17:48:51.113 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:51.113 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:51.113 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:51.113 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:51.114 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:51.114 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:51.114 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44698 2024-09-04T17:48:51.114 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.114 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:51.114 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.114 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:51.114 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253737+0000", 2024-09-04T17:48:51.114 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.114 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.114 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.114 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:51.114 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253737+0000", 2024-09-04T17:48:51.114 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.115 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.115 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.115 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:51.115 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253738+0000", 2024-09-04T17:48:51.115 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0300000000000001e-06 2024-09-04T17:48:51.115 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.115 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.115 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:51.115 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253752+0000", 2024-09-04T17:48:51.115 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4358999999999999e-05 2024-09-04T17:48:51.115 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.115 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.115 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:51.116 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253771+0000", 2024-09-04T17:48:51.116 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8879999999999999e-05 2024-09-04T17:48:51.116 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.116 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.116 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:51.116 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253785+0000", 2024-09-04T17:48:51.116 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3852000000000001e-05 2024-09-04T17:48:51.116 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.116 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.116 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:51.116 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304876+0000", 2024-09-04T17:48:51.116 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051091105999999997 2024-09-04T17:48:51.116 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.117 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.117 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:51.117 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304913+0000", 2024-09-04T17:48:51.117 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.6850999999999997e-05 2024-09-04T17:48:51.117 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.117 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:51.117 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.117 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.117 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.117 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44699 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:51.117 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253804+0000", 2024-09-04T17:48:51.117 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 120.77882637499999, 2024-09-04T17:48:51.118 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051172250000000002, 2024-09-04T17:48:51.118 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:51.118 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:51.118 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:51.118 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:51.118 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:51.118 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:51.118 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44699 2024-09-04T17:48:51.118 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.118 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:51.118 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.118 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:51.118 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253804+0000", 2024-09-04T17:48:51.118 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.119 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.119 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.119 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:51.119 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253804+0000", 2024-09-04T17:48:51.119 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.119 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.119 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.119 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:51.119 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253805+0000", 2024-09-04T17:48:51.119 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.08e-06 2024-09-04T17:48:51.119 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.119 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.119 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:51.120 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253810+0000", 2024-09-04T17:48:51.120 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1830000000000002e-06 2024-09-04T17:48:51.120 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.120 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.120 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:51.120 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253828+0000", 2024-09-04T17:48:51.120 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7662000000000001e-05 2024-09-04T17:48:51.120 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.120 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.120 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:51.120 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253841+0000", 2024-09-04T17:48:51.120 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3203999999999999e-05 2024-09-04T17:48:51.120 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.121 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.121 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:51.121 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304966+0000", 2024-09-04T17:48:51.121 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051125273999999998 2024-09-04T17:48:51.121 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.121 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.121 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:51.121 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304976+0000", 2024-09-04T17:48:51.121 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.8470000000000008e-06 2024-09-04T17:48:51.121 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.121 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:51.121 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.122 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.122 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.122 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44700 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:51.122 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253858+0000", 2024-09-04T17:48:51.122 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 120.778772721, 2024-09-04T17:48:51.122 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051218957000000002, 2024-09-04T17:48:51.122 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:51.122 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:51.122 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:51.122 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:51.122 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:51.122 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:51.122 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44700 2024-09-04T17:48:51.123 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.123 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:51.123 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.123 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:51.123 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253858+0000", 2024-09-04T17:48:51.123 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.123 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.123 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.123 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:51.123 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253858+0000", 2024-09-04T17:48:51.123 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.123 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.123 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.124 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:51.124 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253859+0000", 2024-09-04T17:48:51.124 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.217e-06 2024-09-04T17:48:51.124 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.124 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.124 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:51.124 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253864+0000", 2024-09-04T17:48:51.124 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7600000000000002e-06 2024-09-04T17:48:51.124 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.124 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.124 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:51.124 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253880+0000", 2024-09-04T17:48:51.124 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6501999999999999e-05 2024-09-04T17:48:51.125 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.125 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.125 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:51.125 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253899+0000", 2024-09-04T17:48:51.125 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8907e-05 2024-09-04T17:48:51.125 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.125 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.125 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:51.125 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305052+0000", 2024-09-04T17:48:51.125 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051152989000000003 2024-09-04T17:48:51.125 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.125 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.125 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:51.126 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305077+0000", 2024-09-04T17:48:51.126 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4581999999999999e-05 2024-09-04T17:48:51.126 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.126 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:51.126 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.126 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.126 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.126 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44701 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:51.126 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253917+0000", 2024-09-04T17:48:51.126 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 120.778713369, 2024-09-04T17:48:51.126 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051231113000000002, 2024-09-04T17:48:51.126 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:51.127 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:51.127 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:51.127 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:51.127 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:51.127 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:51.127 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44701 2024-09-04T17:48:51.127 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.127 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:51.127 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.127 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:51.127 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253917+0000", 2024-09-04T17:48:51.127 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.128 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.128 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.128 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:51.128 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253917+0000", 2024-09-04T17:48:51.128 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.128 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.128 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.128 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:51.128 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253918+0000", 2024-09-04T17:48:51.128 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.136e-06 2024-09-04T17:48:51.128 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.128 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.129 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:51.129 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253924+0000", 2024-09-04T17:48:51.129 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.3070000000000002e-06 2024-09-04T17:48:51.129 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.129 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.129 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:51.129 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253940+0000", 2024-09-04T17:48:51.129 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6612e-05 2024-09-04T17:48:51.129 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.129 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.129 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:51.129 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253954+0000", 2024-09-04T17:48:51.130 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3709999999999999e-05 2024-09-04T17:48:51.130 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.130 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.130 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:51.130 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305136+0000", 2024-09-04T17:48:51.130 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051181935999999997 2024-09-04T17:48:51.130 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.130 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.130 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:51.130 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305148+0000", 2024-09-04T17:48:51.130 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2412000000000001e-05 2024-09-04T17:48:51.130 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.130 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:51.131 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.131 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.131 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:51.131 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:51.132 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-04T17:48:51.133 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:48:51.129+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:48:51.233 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:51.230+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:51.305 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:51.305 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:48:51.305 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:48:51.305 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:51.319 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-04T17:48:51.365 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:48:51.365+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:48:51.435 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:51.433+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:51.442 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:51.443 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:48:51.443 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:48:51.443 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:48:51.443 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:51.458 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-04T17:48:51.505 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-04T17:48:51.549 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:48:51.546+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:48:51.625 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:51.625 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:48:51.625 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:48:51.625 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:48:51.625 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.625 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8831 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2453504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:51.625 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:48:51.625 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 158.565198286, 2024-09-04T17:48:51.625 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14731111799999999, 2024-09-04T17:48:51.625 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:51.626 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:51.626 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:51.626 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:51.626 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:51.626 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:51.626 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8831 2024-09-04T17:48:51.626 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.626 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:51.626 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.626 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:51.626 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:48:51.626 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.626 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.627 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.627 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:51.627 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:48:51.627 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.627 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.627 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.627 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:51.627 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057811+0000", 2024-09-04T17:48:51.627 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.083e-06 2024-09-04T17:48:51.627 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.627 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.627 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:51.628 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057814+0000", 2024-09-04T17:48:51.628 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.407e-06 2024-09-04T17:48:51.628 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.628 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.628 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:51.628 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057825+0000", 2024-09-04T17:48:51.628 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.17e-05 2024-09-04T17:48:51.628 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.628 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.628 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:51.628 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057836+0000", 2024-09-04T17:48:51.628 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0283e-05 2024-09-04T17:48:51.628 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.629 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.629 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:51.629 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.204762+0000", 2024-09-04T17:48:51.629 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.146926205 2024-09-04T17:48:51.629 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.629 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.629 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:51.629 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.204895+0000", 2024-09-04T17:48:51.629 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013328999999999999 2024-09-04T17:48:51.629 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.629 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.629 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:51.629 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205121+0000", 2024-09-04T17:48:51.630 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00022614999999999999 2024-09-04T17:48:51.630 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.630 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:51.630 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.630 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.630 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.630 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8832 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2490368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:51.630 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:48:51.630 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 158.56516025299999, 2024-09-04T17:48:51.630 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14770012299999999, 2024-09-04T17:48:51.630 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:51.630 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:51.630 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:51.630 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:51.631 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:51.631 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:51.631 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8832 2024-09-04T17:48:51.631 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.631 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:51.631 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.631 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:51.631 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:48:51.631 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.631 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.631 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.631 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:51.632 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:48:51.632 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.632 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.632 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.632 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:51.632 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057849+0000", 2024-09-04T17:48:51.632 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6700000000000003e-07 2024-09-04T17:48:51.632 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.632 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.632 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:51.632 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057852+0000", 2024-09-04T17:48:51.632 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6359999999999998e-06 2024-09-04T17:48:51.633 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.633 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.633 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:51.633 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057860+0000", 2024-09-04T17:48:51.633 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8140000000000007e-06 2024-09-04T17:48:51.633 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.633 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.633 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:51.633 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057868+0000", 2024-09-04T17:48:51.633 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1999999999999997e-06 2024-09-04T17:48:51.633 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.633 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.633 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:51.634 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205208+0000", 2024-09-04T17:48:51.634 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14734067300000001 2024-09-04T17:48:51.634 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.634 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.634 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:51.634 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205347+0000", 2024-09-04T17:48:51.634 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013914900000000001 2024-09-04T17:48:51.634 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.634 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.634 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:51.634 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205548+0000", 2024-09-04T17:48:51.634 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000200984 2024-09-04T17:48:51.634 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.635 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:51.635 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.635 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.635 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.635 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8833 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2527232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:51.635 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:48:51.635 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 158.565128383, 2024-09-04T17:48:51.635 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148062572, 2024-09-04T17:48:51.635 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:51.635 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:51.635 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:51.635 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:51.635 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:51.636 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:51.636 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8833 2024-09-04T17:48:51.636 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.636 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:51.636 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.636 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:51.636 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:48:51.636 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.636 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.636 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.636 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:51.636 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:48:51.637 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.637 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.637 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.637 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:51.637 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057881+0000", 2024-09-04T17:48:51.637 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4300000000000002e-07 2024-09-04T17:48:51.637 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.637 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.637 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:51.637 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057893+0000", 2024-09-04T17:48:51.637 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1973999999999999e-05 2024-09-04T17:48:51.637 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.637 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.638 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:51.638 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057906+0000", 2024-09-04T17:48:51.638 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2716e-05 2024-09-04T17:48:51.638 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.638 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.638 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:51.638 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057913+0000", 2024-09-04T17:48:51.638 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3930000000000002e-06 2024-09-04T17:48:51.638 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.638 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.638 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:51.638 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205621+0000", 2024-09-04T17:48:51.638 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14770788800000001 2024-09-04T17:48:51.639 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.639 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.639 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:51.639 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205740+0000", 2024-09-04T17:48:51.639 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000118936 2024-09-04T17:48:51.639 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.639 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.639 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:51.639 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205943+0000", 2024-09-04T17:48:51.639 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020292199999999999 2024-09-04T17:48:51.639 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.639 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:51.639 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.640 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.640 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.640 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8834 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2564096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:51.640 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:48:51.640 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 158.56508510399999, 2024-09-04T17:48:51.640 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14846753200000001, 2024-09-04T17:48:51.640 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:51.640 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:51.640 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:51.640 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:51.640 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:51.640 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:51.641 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8834 2024-09-04T17:48:51.641 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.641 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:51.641 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.641 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:51.641 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:48:51.641 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.641 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.641 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.641 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:51.641 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:48:51.641 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.641 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.642 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.642 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:51.642 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057925+0000", 2024-09-04T17:48:51.642 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0270000000000001e-06 2024-09-04T17:48:51.642 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.642 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.642 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:51.642 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057936+0000", 2024-09-04T17:48:51.642 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.112e-05 2024-09-04T17:48:51.642 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.642 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.642 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:51.642 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057946+0000", 2024-09-04T17:48:51.643 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.009e-05 2024-09-04T17:48:51.643 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.643 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.643 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:51.643 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057953+0000", 2024-09-04T17:48:51.643 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.6229999999999997e-06 2024-09-04T17:48:51.643 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.643 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.643 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:51.643 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206027+0000", 2024-09-04T17:48:51.643 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148073591 2024-09-04T17:48:51.643 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.644 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.644 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:51.644 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206161+0000", 2024-09-04T17:48:51.644 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013411599999999999 2024-09-04T17:48:51.644 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.644 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.644 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:51.644 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206391+0000", 2024-09-04T17:48:51.644 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000229965 2024-09-04T17:48:51.644 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.644 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:51.644 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.644 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.645 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.645 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8835 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2600960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:51.645 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:48:51.645 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 158.565045454, 2024-09-04T17:48:51.645 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14882941599999999, 2024-09-04T17:48:51.645 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:51.645 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:51.645 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:51.645 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:51.645 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:51.645 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:51.645 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8835 2024-09-04T17:48:51.645 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.646 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:51.646 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.700 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:51.700 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:48:51.700 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.700 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.700 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.700 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:51.700 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:48:51.700 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.700 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.701 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.701 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:51.701 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058544+0000", 2024-09-04T17:48:51.701 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00058061899999999999 2024-09-04T17:48:51.701 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.701 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.701 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:51.701 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058550+0000", 2024-09-04T17:48:51.701 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.1129999999999997e-06 2024-09-04T17:48:51.701 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.701 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.701 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:51.701 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058566+0000", 2024-09-04T17:48:51.702 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5820000000000001e-05 2024-09-04T17:48:51.702 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.702 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.702 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:51.702 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058578+0000", 2024-09-04T17:48:51.702 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2537e-05 2024-09-04T17:48:51.702 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.702 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.702 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:51.702 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206470+0000", 2024-09-04T17:48:51.702 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14789153799999999 2024-09-04T17:48:51.702 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.703 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.703 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:51.703 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206589+0000", 2024-09-04T17:48:51.703 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000118837 2024-09-04T17:48:51.703 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.703 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.703 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:51.703 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206793+0000", 2024-09-04T17:48:51.703 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000203952 2024-09-04T17:48:51.703 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.703 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:51.703 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.703 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.703 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.704 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8836 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2654208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:51.704 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:48:51.704 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 158.564412766, 2024-09-04T17:48:51.704 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14862433999999999, 2024-09-04T17:48:51.704 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:51.704 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:51.704 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:51.704 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:51.704 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:51.704 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:51.704 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8836 2024-09-04T17:48:51.704 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.704 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:51.705 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.705 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:51.705 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:48:51.705 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.705 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.705 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.705 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:51.705 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:48:51.705 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.705 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.705 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.705 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:51.705 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062663+0000", 2024-09-04T17:48:51.706 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0040670619999999998 2024-09-04T17:48:51.706 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.706 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.706 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:51.706 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062668+0000", 2024-09-04T17:48:51.706 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4199999999999998e-06 2024-09-04T17:48:51.706 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.706 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.706 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:51.706 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062689+0000", 2024-09-04T17:48:51.706 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0356999999999998e-05 2024-09-04T17:48:51.706 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.706 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.707 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:51.707 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062726+0000", 2024-09-04T17:48:51.707 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7302999999999998e-05 2024-09-04T17:48:51.707 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.707 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.707 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:51.707 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206875+0000", 2024-09-04T17:48:51.707 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14414883100000001 2024-09-04T17:48:51.707 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.707 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.707 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:51.707 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.207016+0000", 2024-09-04T17:48:51.707 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000140822 2024-09-04T17:48:51.708 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.708 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.708 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:51.708 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.207220+0000", 2024-09-04T17:48:51.708 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020454499999999999 2024-09-04T17:48:51.708 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.708 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:51.708 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.708 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.708 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.708 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8844 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2928640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:51.708 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:48:51.708 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 158.56004868299999, 2024-09-04T17:48:51.709 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14752153100000001, 2024-09-04T17:48:51.709 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:51.709 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:51.709 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:51.709 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:51.709 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:51.709 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:51.709 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8844 2024-09-04T17:48:51.709 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.709 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:51.709 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.709 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:51.709 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:48:51.710 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.710 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.710 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.710 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:51.710 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:48:51.710 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.710 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.710 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.710 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:51.710 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062961+0000", 2024-09-04T17:48:51.710 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6700000000000003e-07 2024-09-04T17:48:51.710 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.711 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.711 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:51.711 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062973+0000", 2024-09-04T17:48:51.711 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2109999999999999e-05 2024-09-04T17:48:51.711 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.711 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.711 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:51.711 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062982+0000", 2024-09-04T17:48:51.711 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.6900000000000004e-06 2024-09-04T17:48:51.711 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.711 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.711 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:51.711 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062990+0000", 2024-09-04T17:48:51.712 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1899999999999998e-06 2024-09-04T17:48:51.712 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.712 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.712 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:51.712 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210090+0000", 2024-09-04T17:48:51.712 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14710026300000001 2024-09-04T17:48:51.712 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.712 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.712 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:51.712 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210225+0000", 2024-09-04T17:48:51.712 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013532400000000001 2024-09-04T17:48:51.712 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.713 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.713 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:51.713 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210481+0000", 2024-09-04T17:48:51.713 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00025628700000000001 2024-09-04T17:48:51.713 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.713 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:51.713 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.713 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.713 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.713 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8845 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2965504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:51.713 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:48:51.713 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 158.56000896, 2024-09-04T17:48:51.713 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147916819, 2024-09-04T17:48:51.714 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:51.714 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:51.714 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:51.714 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:51.714 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:51.714 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:51.714 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8845 2024-09-04T17:48:51.714 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.714 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:51.714 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.714 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:51.714 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:48:51.714 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.715 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.715 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.715 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:51.715 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:48:51.715 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.715 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.715 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.715 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:51.715 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:48:51.715 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.0399999999999995e-07 2024-09-04T17:48:51.715 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.715 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.715 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:51.716 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063003+0000", 2024-09-04T17:48:51.716 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6529999999999998e-06 2024-09-04T17:48:51.716 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.716 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.716 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:51.716 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063012+0000", 2024-09-04T17:48:51.716 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9500000000000007e-06 2024-09-04T17:48:51.716 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.716 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.716 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:51.716 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063027+0000", 2024-09-04T17:48:51.716 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5443000000000001e-05 2024-09-04T17:48:51.717 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.717 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.717 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:51.717 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210571+0000", 2024-09-04T17:48:51.717 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14754353100000001 2024-09-04T17:48:51.717 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.717 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.717 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:51.717 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210714+0000", 2024-09-04T17:48:51.717 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014295599999999999 2024-09-04T17:48:51.717 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.717 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.717 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:51.718 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210916+0000", 2024-09-04T17:48:51.718 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020258200000000001 2024-09-04T17:48:51.718 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.718 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:51.718 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.718 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.718 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.718 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8846 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3002368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:51.718 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:48:51.718 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 158.55997142300001, 2024-09-04T17:48:51.718 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14828011599999999, 2024-09-04T17:48:51.718 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:51.719 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:51.719 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:51.719 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:51.719 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:51.719 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:51.719 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8846 2024-09-04T17:48:51.719 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.719 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:51.719 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.719 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:51.719 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:48:51.719 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.719 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.720 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.720 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:51.720 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:48:51.720 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.720 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.720 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.720 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:51.720 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063038+0000", 2024-09-04T17:48:51.720 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.23e-07 2024-09-04T17:48:51.720 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.720 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.720 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:51.720 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063051+0000", 2024-09-04T17:48:51.721 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2757e-05 2024-09-04T17:48:51.721 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.721 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.721 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:51.721 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063070+0000", 2024-09-04T17:48:51.721 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9663000000000001e-05 2024-09-04T17:48:51.721 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.721 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.721 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:51.721 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063079+0000", 2024-09-04T17:48:51.721 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.0929999999999992e-06 2024-09-04T17:48:51.721 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.721 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.722 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:51.722 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210989+0000", 2024-09-04T17:48:51.722 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14790984500000001 2024-09-04T17:48:51.722 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.722 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.722 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:51.722 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211109+0000", 2024-09-04T17:48:51.722 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000120056 2024-09-04T17:48:51.722 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.722 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.722 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:51.722 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211317+0000", 2024-09-04T17:48:51.722 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020797900000000001 2024-09-04T17:48:51.723 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.723 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:51.723 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.723 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.723 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.723 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8847 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3039232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:51.723 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:48:51.723 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 158.55989699099999, 2024-09-04T17:48:51.723 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14859408900000001, 2024-09-04T17:48:51.723 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:51.723 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:51.723 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:51.723 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:51.724 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:51.724 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:51.724 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8847 2024-09-04T17:48:51.724 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.724 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:51.724 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.724 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:51.724 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:48:51.724 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.724 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.724 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.724 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:51.724 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:48:51.725 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.725 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.725 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.725 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:51.725 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063113+0000", 2024-09-04T17:48:51.725 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.204e-06 2024-09-04T17:48:51.725 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.725 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.725 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:51.725 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063116+0000", 2024-09-04T17:48:51.725 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.3330000000000001e-06 2024-09-04T17:48:51.725 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.726 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.726 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:51.726 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063126+0000", 2024-09-04T17:48:51.726 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0166999999999999e-05 2024-09-04T17:48:51.726 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.726 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.726 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:51.726 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063134+0000", 2024-09-04T17:48:51.726 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4329999999999998e-06 2024-09-04T17:48:51.726 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.726 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.726 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:51.727 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211392+0000", 2024-09-04T17:48:51.727 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148258109 2024-09-04T17:48:51.727 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.727 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.727 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:51.727 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211506+0000", 2024-09-04T17:48:51.727 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000113813 2024-09-04T17:48:51.727 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.727 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.727 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:51.727 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211706+0000", 2024-09-04T17:48:51.727 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020002999999999999 2024-09-04T17:48:51.727 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.728 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:51.728 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.728 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.728 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.728 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8848 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:51.728 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:48:51.728 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 158.55986465399999, 2024-09-04T17:48:51.728 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14907967599999999, 2024-09-04T17:48:51.728 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:51.728 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:51.728 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:51.728 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:51.729 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:51.729 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:51.729 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8848 2024-09-04T17:48:51.729 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.729 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:51.729 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.729 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:51.729 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:48:51.729 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.729 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.729 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.729 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:51.729 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:48:51.729 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.730 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.730 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.730 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:51.730 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070589+0000", 2024-09-04T17:48:51.730 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.007445386 2024-09-04T17:48:51.730 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.730 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.730 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:51.730 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070600+0000", 2024-09-04T17:48:51.730 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0346999999999999e-05 2024-09-04T17:48:51.730 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.730 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.731 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:51.731 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070622+0000", 2024-09-04T17:48:51.731 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2289999999999998e-05 2024-09-04T17:48:51.731 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.731 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.731 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:51.731 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070636+0000", 2024-09-04T17:48:51.731 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4093e-05 2024-09-04T17:48:51.731 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.731 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.731 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:51.731 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211815+0000", 2024-09-04T17:48:51.731 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14117874699999999 2024-09-04T17:48:51.732 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.732 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.732 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:51.732 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211983+0000", 2024-09-04T17:48:51.732 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00016852800000000001 2024-09-04T17:48:51.732 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.732 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.732 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:51.732 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.212224+0000", 2024-09-04T17:48:51.732 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024028499999999999 2024-09-04T17:48:51.732 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.732 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:51.733 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.733 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.733 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.733 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8862 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2019328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:51.733 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:48:51.733 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 158.551815186, 2024-09-04T17:48:51.733 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14755236399999999, 2024-09-04T17:48:51.733 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:51.733 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:51.733 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:51.733 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:51.733 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:51.733 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:51.734 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8862 2024-09-04T17:48:51.734 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.734 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:51.734 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.734 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:51.734 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:48:51.734 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.734 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.734 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.734 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:51.734 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:48:51.734 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.734 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.735 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.735 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:51.735 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071194+0000", 2024-09-04T17:48:51.735 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.5000000000000002e-07 2024-09-04T17:48:51.735 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.735 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.735 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:51.735 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071197+0000", 2024-09-04T17:48:51.735 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5900000000000002e-06 2024-09-04T17:48:51.735 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.735 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.735 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:51.735 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071206+0000", 2024-09-04T17:48:51.735 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9700000000000005e-06 2024-09-04T17:48:51.736 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.736 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.736 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:51.736 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071213+0000", 2024-09-04T17:48:51.736 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.9070000000000001e-06 2024-09-04T17:48:51.736 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.736 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.736 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:51.736 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218396+0000", 2024-09-04T17:48:51.736 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147183273 2024-09-04T17:48:51.736 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.736 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.736 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:51.737 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218538+0000", 2024-09-04T17:48:51.737 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014194199999999999 2024-09-04T17:48:51.737 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.737 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.737 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:51.737 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218746+0000", 2024-09-04T17:48:51.737 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000208032 2024-09-04T17:48:51.737 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.737 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:51.737 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.737 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.737 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.737 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8863 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2056192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:51.737 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:48:51.738 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 158.55178687599999, 2024-09-04T17:48:51.738 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14792281099999999, 2024-09-04T17:48:51.738 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:51.738 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:51.738 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:51.738 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:51.738 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:51.738 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:51.738 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8863 2024-09-04T17:48:51.738 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.738 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:51.738 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.738 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:51.739 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:48:51.739 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.739 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.739 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.739 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:51.739 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:48:51.739 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.739 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.739 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.739 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:51.739 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:48:51.739 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1299999999999999e-07 2024-09-04T17:48:51.739 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.739 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.740 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:51.740 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071225+0000", 2024-09-04T17:48:51.740 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6199999999999999e-06 2024-09-04T17:48:51.740 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.740 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.740 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:51.740 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071234+0000", 2024-09-04T17:48:51.740 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8769999999999999e-06 2024-09-04T17:48:51.740 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.740 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.740 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:51.740 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071251+0000", 2024-09-04T17:48:51.741 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.738e-05 2024-09-04T17:48:51.741 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.741 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.741 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:51.741 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218798+0000", 2024-09-04T17:48:51.741 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14754645 2024-09-04T17:48:51.741 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.741 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.741 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:51.741 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218926+0000", 2024-09-04T17:48:51.741 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000128446 2024-09-04T17:48:51.741 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.741 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.742 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:51.742 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219145+0000", 2024-09-04T17:48:51.742 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021832500000000001 2024-09-04T17:48:51.742 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.742 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:51.742 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.742 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.742 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.742 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8864 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2093056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:51.742 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:48:51.742 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 158.55174529999999, 2024-09-04T17:48:51.742 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148277309, 2024-09-04T17:48:51.742 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:51.743 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:51.743 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:51.743 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:51.743 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:51.743 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:51.743 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8864 2024-09-04T17:48:51.743 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.743 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:51.743 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.743 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:51.743 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:48:51.743 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.743 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.744 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.744 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:51.744 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:48:51.744 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.744 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.744 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.744 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:51.744 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071264+0000", 2024-09-04T17:48:51.744 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3e-07 2024-09-04T17:48:51.744 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.744 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.744 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:51.745 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071267+0000", 2024-09-04T17:48:51.745 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0139999999999999e-06 2024-09-04T17:48:51.745 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.745 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.745 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:51.745 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071276+0000", 2024-09-04T17:48:51.745 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1689999999999998e-06 2024-09-04T17:48:51.745 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.745 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.745 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:51.745 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071283+0000", 2024-09-04T17:48:51.745 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2269999999999996e-06 2024-09-04T17:48:51.745 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.745 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.746 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:51.746 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219195+0000", 2024-09-04T17:48:51.746 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147911868 2024-09-04T17:48:51.746 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.746 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.746 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:51.746 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219324+0000", 2024-09-04T17:48:51.746 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00012835900000000001 2024-09-04T17:48:51.746 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.746 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.746 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:51.746 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219541+0000", 2024-09-04T17:48:51.746 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000216942 2024-09-04T17:48:51.747 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.747 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:51.747 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.747 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.747 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.747 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8865 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2109440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:51.747 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:48:51.747 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 158.55171583999999, 2024-09-04T17:48:51.747 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14865747300000001, 2024-09-04T17:48:51.747 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:51.747 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:51.747 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:51.748 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:51.748 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:51.748 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:51.748 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8865 2024-09-04T17:48:51.748 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.748 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:51.748 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.748 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:51.748 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:48:51.748 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.748 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.748 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.748 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:51.748 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:48:51.749 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.749 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.749 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.749 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:51.749 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:48:51.749 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7999999999999995e-07 2024-09-04T17:48:51.749 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.749 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.749 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:51.749 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071296+0000", 2024-09-04T17:48:51.749 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6900000000000001e-06 2024-09-04T17:48:51.749 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.749 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.750 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:51.750 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071305+0000", 2024-09-04T17:48:51.750 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.5499999999999995e-06 2024-09-04T17:48:51.750 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.750 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.750 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:51.750 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071312+0000", 2024-09-04T17:48:51.750 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1559999999999998e-06 2024-09-04T17:48:51.750 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.750 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.750 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:51.750 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219591+0000", 2024-09-04T17:48:51.750 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14827944900000001 2024-09-04T17:48:51.750 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.751 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.751 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:51.751 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219731+0000", 2024-09-04T17:48:51.751 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013931600000000001 2024-09-04T17:48:51.751 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.751 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.751 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:51.751 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219950+0000", 2024-09-04T17:48:51.751 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021963199999999999 2024-09-04T17:48:51.751 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.751 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:51.751 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.751 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.751 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.752 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8866 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2146304~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:51.752 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:48:51.752 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 158.55168760699999, 2024-09-04T17:48:51.752 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14906117899999999, 2024-09-04T17:48:51.752 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:51.752 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:51.752 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:51.752 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:51.752 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:51.752 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:51.752 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8866 2024-09-04T17:48:51.752 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.752 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:51.753 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.753 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:51.753 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:48:51.753 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.753 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.753 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.753 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:51.753 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:48:51.753 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.753 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.753 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.753 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:51.753 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071322+0000", 2024-09-04T17:48:51.754 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7999999999999995e-07 2024-09-04T17:48:51.754 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.754 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.754 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:51.754 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071338+0000", 2024-09-04T17:48:51.754 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5872999999999999e-05 2024-09-04T17:48:51.754 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.754 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.754 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:51.754 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071347+0000", 2024-09-04T17:48:51.754 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4669999999999993e-06 2024-09-04T17:48:51.754 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.754 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.755 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:51.755 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071354+0000", 2024-09-04T17:48:51.755 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2099999999999996e-06 2024-09-04T17:48:51.755 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.755 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.755 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:51.755 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220008+0000", 2024-09-04T17:48:51.755 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148653642 2024-09-04T17:48:51.755 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.755 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.755 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:51.755 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220155+0000", 2024-09-04T17:48:51.756 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014726600000000001 2024-09-04T17:48:51.756 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.756 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.756 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:51.756 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220382+0000", 2024-09-04T17:48:51.756 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00022704100000000001 2024-09-04T17:48:51.756 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.756 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:51.756 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.756 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.756 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.756 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8867 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2183168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:51.756 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:48:51.757 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 158.551644254, 2024-09-04T17:48:51.757 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.149445831, 2024-09-04T17:48:51.757 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:51.757 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:51.757 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:51.757 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:51.757 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:51.757 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:51.757 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8867 2024-09-04T17:48:51.757 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.757 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:51.757 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.757 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:51.758 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:48:51.758 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.758 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.758 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.758 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:51.758 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:48:51.758 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.758 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.758 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.758 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:51.758 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078653+0000", 2024-09-04T17:48:51.758 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0072890430000000003 2024-09-04T17:48:51.758 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.759 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.759 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:51.759 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078664+0000", 2024-09-04T17:48:51.759 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0307e-05 2024-09-04T17:48:51.759 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.759 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.759 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:51.759 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078690+0000", 2024-09-04T17:48:51.759 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6780000000000001e-05 2024-09-04T17:48:51.759 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.759 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.759 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:51.759 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078706+0000", 2024-09-04T17:48:51.759 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5676999999999999e-05 2024-09-04T17:48:51.760 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.760 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.760 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:51.760 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220467+0000", 2024-09-04T17:48:51.760 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14176103500000001 2024-09-04T17:48:51.760 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.760 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.760 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:51.760 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220595+0000", 2024-09-04T17:48:51.760 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000127427 2024-09-04T17:48:51.760 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.760 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.760 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:51.761 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220810+0000", 2024-09-04T17:48:51.761 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021556200000000001 2024-09-04T17:48:51.761 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.761 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:51.761 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.761 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.761 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.761 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8881 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2695168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:51.761 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:48:51.761 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 158.54382218500001, 2024-09-04T17:48:51.761 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147543547, 2024-09-04T17:48:51.761 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:51.761 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:51.762 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:51.762 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:51.762 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:51.762 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:51.762 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8881 2024-09-04T17:48:51.762 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.762 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:51.762 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.762 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:51.762 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:48:51.762 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.762 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.762 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.763 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:51.763 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:48:51.763 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.763 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.763 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.763 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:51.763 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079187+0000", 2024-09-04T17:48:51.763 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7700000000000004e-07 2024-09-04T17:48:51.763 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.763 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.763 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:51.763 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079198+0000", 2024-09-04T17:48:51.763 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0849999999999999e-05 2024-09-04T17:48:51.764 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.764 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.764 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:51.764 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079207+0000", 2024-09-04T17:48:51.764 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5599999999999999e-06 2024-09-04T17:48:51.764 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.764 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.764 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:51.764 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079224+0000", 2024-09-04T17:48:51.764 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6387000000000001e-05 2024-09-04T17:48:51.764 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.764 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.764 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:51.765 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226414+0000", 2024-09-04T17:48:51.765 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14718975000000001 2024-09-04T17:48:51.765 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.765 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.765 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:51.765 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226534+0000", 2024-09-04T17:48:51.765 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000120288 2024-09-04T17:48:51.765 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.765 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.765 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:51.765 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226730+0000", 2024-09-04T17:48:51.765 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000196035 2024-09-04T17:48:51.765 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.766 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:51.766 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.766 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.766 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.766 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8882 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2732032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:51.766 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:48:51.766 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 158.54377397100001, 2024-09-04T17:48:51.766 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14795022399999999, 2024-09-04T17:48:51.766 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:51.766 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:51.766 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:51.766 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:51.767 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:51.767 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:51.767 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8882 2024-09-04T17:48:51.767 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.767 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:51.767 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.767 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:51.767 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:48:51.767 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.767 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.767 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.767 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:51.767 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:48:51.767 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.768 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.768 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.768 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:51.768 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086588+0000", 2024-09-04T17:48:51.768 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0073530959999999999 2024-09-04T17:48:51.768 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.768 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.768 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:51.768 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086598+0000", 2024-09-04T17:48:51.768 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0122999999999999e-05 2024-09-04T17:48:51.768 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.768 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.768 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:51.769 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086626+0000", 2024-09-04T17:48:51.769 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.7914000000000001e-05 2024-09-04T17:48:51.769 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.769 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.769 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:51.769 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086641+0000", 2024-09-04T17:48:51.769 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5270000000000001e-05 2024-09-04T17:48:51.769 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.769 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.769 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:51.769 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226801+0000", 2024-09-04T17:48:51.769 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.140160321 2024-09-04T17:48:51.770 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.770 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.770 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:51.770 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226951+0000", 2024-09-04T17:48:51.770 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014965200000000001 2024-09-04T17:48:51.770 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.770 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.770 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:51.770 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.227185+0000", 2024-09-04T17:48:51.770 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00023384799999999999 2024-09-04T17:48:51.770 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.770 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:51.770 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.771 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.771 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.771 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:93470 3.1f 3:f95bc674:::rbd_header.1134878eadad:head [watch reconnect cookie 140018821420464 gen 16] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-04T17:48:51.771 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:57.837081+0000", 2024-09-04T17:48:51.771 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 53.785928087999999, 2024-09-04T17:48:51.771 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.072023486, 2024-09-04T17:48:51.771 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:51.771 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:51.771 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:51.771 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:51.771 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:51.771 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:51.771 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 93470 2024-09-04T17:48:51.772 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.772 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:51.772 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.772 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:51.772 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:57.837081+0000", 2024-09-04T17:48:51.772 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.772 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.772 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.772 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:51.772 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:57.837081+0000", 2024-09-04T17:48:51.772 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:51.772 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.773 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.773 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:51.773 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:57.837084+0000", 2024-09-04T17:48:51.773 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.8229999999999997e-06 2024-09-04T17:48:51.773 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.773 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.773 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:51.773 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:57.837098+0000", 2024-09-04T17:48:51.773 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3436e-05 2024-09-04T17:48:51.773 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.773 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.773 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:51.773 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:57.837123+0000", 2024-09-04T17:48:51.774 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5079000000000002e-05 2024-09-04T17:48:51.774 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.774 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.774 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:51.774 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:57.837145+0000", 2024-09-04T17:48:51.774 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.228e-05 2024-09-04T17:48:51.774 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.774 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.774 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:51.774 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:57.837360+0000", 2024-09-04T17:48:51.774 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021464100000000001 2024-09-04T17:48:51.774 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.774 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.774 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for peered", 2024-09-04T17:48:51.775 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:57.837410+0000", 2024-09-04T17:48:51.775 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0235000000000003e-05 2024-09-04T17:48:51.775 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.775 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.775 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:51.775 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.900175+0000", 2024-09-04T17:48:51.775 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0627652830000001 2024-09-04T17:48:51.775 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.775 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.775 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for rw locks", 2024-09-04T17:48:51.775 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.900370+0000", 2024-09-04T17:48:51.775 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019497199999999999 2024-09-04T17:48:51.775 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.776 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.776 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:51.776 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.906957+0000", 2024-09-04T17:48:51.776 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0065866889999999997 2024-09-04T17:48:51.776 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.776 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.776 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:51.776 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.907088+0000", 2024-09-04T17:48:51.776 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013056100000000001 2024-09-04T17:48:51.776 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:51.776 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:51.776 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:51.776 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.909104+0000", 2024-09-04T17:48:51.777 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.002016487 2024-09-04T17:48:51.777 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.777 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:51.777 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.777 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:51.777 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:51.777 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:51.778 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-04T17:48:51.779 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:51.777+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:51.863 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:48:51.864 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:48:51.879 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:48:51.877+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:48:51.895 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:48:51.895 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:48:51.927 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-04T17:48:51.928 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-04T17:48:51.952 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:51.952 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:48:51.952 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:48:51.953 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:51.954 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-04T17:48:51.954 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-04T17:48:51.967 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-04T17:48:51.983 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:48:51.983 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:48:52.005 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.6 to 1 (10000) 2024-09-04T17:48:52.009 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:52.009+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:52.015 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:48:52.016 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:48:52.020 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-04T17:48:52.045 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-04T17:48:52.050 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-04T17:48:52.050 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-04T17:48:52.073 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:48:52.074 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:48:52.080 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:52.074+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:52.141 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:52.141 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:52.156 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-04T17:48:52.201 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:52.201+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:52.278 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:52.279 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:52.282 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:52.278+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:52.293 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-04T17:48:52.383 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:48:52.378+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:48:52.412 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:52.412 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:52.426 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-04T17:48:52.513 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:52.510+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:52.585 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:48:52.582+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:48:52.589 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:52.589 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:52.605 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-04T17:48:52.697 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:52.694+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:52.771 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:52.772 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:52.784 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-04T17:48:52.825 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:52.822+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:52.843 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is unset 2024-09-04T17:48:52.851 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.7 to 1 (10000) 2024-09-04T17:48:52.883 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-04T17:48:52.925 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:48:52.922+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:48:52.956 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:52.956 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:52.971 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-04T17:48:53.013 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:48:53.010+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:48:53.089 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:48:53.086+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:48:53.125 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:53.125 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:53.138 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-04T17:48:53.225 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:48:53.222+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:48:53.292 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:48:53.289+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:48:53.307 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:53.307 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:53.321 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-04T17:48:53.393 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:48:53.386+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:48:53.455 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:53.455 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:53.469 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-04T17:48:53.493 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:53.490+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:53.586 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:53.587 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:53.594 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:48:53.594+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:48:53.601 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-04T17:48:53.695 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:48:53.693+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:48:53.718 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:53.719 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:53.732 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-04T17:48:53.796 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:53.793+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:53.859 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:53.859 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:53.873 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-04T17:48:53.921 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:53.918+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:53.996 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:53.996 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:53.998 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:53.994+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:54.012 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-04T17:48:54.026 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.3 to 1 (10000) 2024-09-04T17:48:54.046 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-04T17:48:54.046 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-04T17:48:54.097 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-04T17:48:54.133 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:54.130+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:54.147 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:54.147 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:54.164 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-04T17:48:54.209 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:48:54.206+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:48:54.282 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:48:54.282 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:48:54.299 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:48:54.299 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:48:54.300 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:54.297+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:54.315 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-04T17:48:54.316 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-04T17:48:54.332 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-04T17:48:54.332 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-04T17:48:54.343 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:54.343 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:48:54.343 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:48:54.343 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:54.353 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:48:54.353 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:48:54.358 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-04T17:48:54.374 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:48:54.374 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:48:54.395 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-04T17:48:54.395 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-04T17:48:54.400 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:54.394+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:54.416 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:48:54.416 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:48:54.418 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:48:54.419 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-09-04T17:48:54.441 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":264,"stamp":"2024-09-04T17:48:54.018925+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":177824216,"num_objects":1103,"num_object_clones":40,"num_object_copies":2206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1103,"num_whiteouts":960,"num_read":140902,"num_read_kb":875569,"num_write":20107,"num_write_kb":405833,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4671,"num_bytes_recovered":238943375,"num_keys_recovered":777,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17883,"ondisk_log_size":17883,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":479072,"kb_used_data":159096,"kb_used_omap":45,"kb_used_meta":319442,"kb_avail":660123808,"statfs":{"total":676457349120,"available":675966779392,"internally_reserved":0,"allocated":162914304,"data_stored":186315761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":46351,"internal_metadata":327109361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-165325659,"num_objects":-159,"num_object_clones":-55,"num_object_copies":-318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-159,"num_whiteouts":-40,"num_read":1064,"num_read_kb":909,"num_write":215,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.030164"},"pg_stats":[{"pgid":"3.1f","version":"171'497","reported_seq":3048,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-04T17:48:51.056660+0000","last_change":"2024-09-04T17:48:51.057439+0000","last_active":"2024-09-04T17:48:51.056660+0000","last_peered":"2024-09-04T17:48:51.056660+0000","last_clean":"2024-09-04T17:48:51.056660+0000","last_became_active":"2024-09-04T17:48:09.845199+0000","last_became_peered":"2024-09-04T17:48:09.845199+0000","last_unstale":"2024-09-04T17:48:51.056660+0000","last_undegraded":"2024-09-04T17:48:51.056660+0000","last_fullsized":"2024-09-04T17:48:51.056660+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":497,"log_dups_size":0,"ondisk_log_size":497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T17:49:02.934920+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0034712879999999999,"stat_sum":{"num_bytes":2490384,"num_objects":39,"num_object_clones":1,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":36,"num_read":2980,"num_read_kb":27189,"num_write":522,"num_write_kb":10445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":24433786,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"8","length":"b"}]},{"pgid":"3.1e","version":"169'639","reported_seq":3904,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-04T17:48:52.023850+0000","last_change":"2024-09-04T17:48:52.024108+0000","last_active":"2024-09-04T17:48:52.023850+0000","last_peered":"2024-09-04T17:48:52.023850+0000","last_clean":"2024-09-04T17:48:52.023850+0000","last_became_active":"2024-09-04T17:48:30.050904+0000","last_became_peered":"2024-09-04T17:48:30.050904+0000","last_unstale":"2024-09-04T17:48:52.023850+0000","last_undegraded":"2024-09-04T17:48:52.023850+0000","last_fullsized":"2024-09-04T17:48:52.023850+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":639,"log_dups_size":0,"ondisk_log_size":639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:15:06.068015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017791739999999999,"stat_sum":{"num_bytes":14614509,"num_objects":28,"num_object_clones":2,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":23,"num_read":4726,"num_read_kb":30478,"num_write":799,"num_write_kb":10568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":230,"num_bytes_recovered":15412196,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"7","length":"c"}]},{"pgid":"3.1d","version":"170'541","reported_seq":2469,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-04T17:48:51.029548+0000","last_change":"2024-09-04T17:48:51.029974+0000","last_active":"2024-09-04T17:48:51.029548+0000","last_peered":"2024-09-04T17:48:51.029548+0000","last_clean":"2024-09-04T17:48:51.029548+0000","last_became_active":"2024-09-04T17:45:55.955213+0000","last_became_peered":"2024-09-04T17:45:55.955213+0000","last_unstale":"2024-09-04T17:48:51.029548+0000","last_undegraded":"2024-09-04T17:48:51.029548+0000","last_fullsized":"2024-09-04T17:48:51.029548+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":541,"log_dups_size":0,"ondisk_log_size":541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:18:29.193686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024144710000000001,"stat_sum":{"num_bytes":18178765,"num_objects":33,"num_object_clones":2,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":28,"num_read":2820,"num_read_kb":20284,"num_write":620,"num_write_kb":9401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"b"}]},{"pgid":"3.1c","version":"171'751","reported_seq":6724,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-04T17:48:51.170471+0000","last_change":"2024-09-04T17:48:51.033957+0000","last_active":"2024-09-04T17:48:51.170471+0000","last_peered":"2024-09-04T17:48:51.170471+0000","last_clean":"2024-09-04T17:48:51.170471+0000","last_became_active":"2024-09-04T17:45:55.951880+0000","last_became_peered":"2024-09-04T17:45:55.951880+0000","last_unstale":"2024-09-04T17:48:51.170471+0000","last_undegraded":"2024-09-04T17:48:51.170471+0000","last_fullsized":"2024-09-04T17:48:51.170471+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":751,"log_dups_size":0,"ondisk_log_size":751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T23:55:10.451122+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0062691759999999996,"stat_sum":{"num_bytes":917504,"num_objects":38,"num_object_clones":1,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":36,"num_read":7446,"num_read_kb":47991,"num_write":864,"num_write_kb":11556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"b"}]},{"pgid":"3.1b","version":"170'573","reported_seq":2234,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-04T17:48:51.055423+0000","last_change":"2024-09-04T17:48:51.055423+0000","last_active":"2024-09-04T17:48:51.055423+0000","last_peered":"2024-09-04T17:48:51.055423+0000","last_clean":"2024-09-04T17:48:51.055423+0000","last_became_active":"2024-09-04T17:48:09.853793+0000","last_became_peered":"2024-09-04T17:48:09.853793+0000","last_unstale":"2024-09-04T17:48:51.055423+0000","last_undegraded":"2024-09-04T17:48:51.055423+0000","last_fullsized":"2024-09-04T17:48:51.055423+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":573,"log_dups_size":0,"ondisk_log_size":573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:27:41.733656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001528373,"stat_sum":{"num_bytes":262259,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":33,"num_read":3545,"num_read_kb":8003,"num_write":628,"num_write_kb":4594,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":1993914,"num_keys_recovered":48,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"8","length":"b"}]},{"pgid":"3.1a","version":"168'419","reported_seq":6337,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-09-04T17:48:49.116376+0000","last_change":"2024-09-04T17:48:48.978137+0000","last_active":"2024-09-04T17:48:49.116376+0000","last_peered":"2024-09-04T17:48:49.116376+0000","last_clean":"2024-09-04T17:48:49.116376+0000","last_became_active":"2024-09-04T17:47:44.946366+0000","last_became_peered":"2024-09-04T17:47:44.946366+0000","last_unstale":"2024-09-04T17:48:49.116376+0000","last_undegraded":"2024-09-04T17:48:49.116376+0000","last_fullsized":"2024-09-04T17:48:49.116376+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:05:54.083403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030946229999999999,"stat_sum":{"num_bytes":1411895,"num_objects":31,"num_object_clones":1,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":26,"num_read":5950,"num_read_kb":45812,"num_write":373,"num_write_kb":15447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":5992020,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"a","length":"9"}]},{"pgid":"3.19","version":"171'417","reported_seq":2451,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-04T17:48:52.022528+0000","last_change":"2024-09-04T17:48:52.022873+0000","last_active":"2024-09-04T17:48:52.022528+0000","last_peered":"2024-09-04T17:48:52.022528+0000","last_clean":"2024-09-04T17:48:52.022528+0000","last_became_active":"2024-09-04T17:48:30.052691+0000","last_became_peered":"2024-09-04T17:48:30.052691+0000","last_unstale":"2024-09-04T17:48:52.022528+0000","last_undegraded":"2024-09-04T17:48:52.022528+0000","last_fullsized":"2024-09-04T17:48:52.022528+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":417,"log_dups_size":0,"ondisk_log_size":417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:01:08.252224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092401799999999997,"stat_sum":{"num_bytes":917504,"num_objects":21,"num_object_clones":1,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":19,"num_read":2092,"num_read_kb":16058,"num_write":443,"num_write_kb":4611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":4911830,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"7","length":"c"}]},{"pgid":"3.18","version":"170'505","reported_seq":2273,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-04T17:48:52.022144+0000","last_change":"2024-09-04T17:48:52.022878+0000","last_active":"2024-09-04T17:48:52.022144+0000","last_peered":"2024-09-04T17:48:52.022144+0000","last_clean":"2024-09-04T17:48:52.022144+0000","last_became_active":"2024-09-04T17:48:30.051421+0000","last_became_peered":"2024-09-04T17:48:30.051421+0000","last_unstale":"2024-09-04T17:48:52.022144+0000","last_undegraded":"2024-09-04T17:48:52.022144+0000","last_fullsized":"2024-09-04T17:48:52.022144+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":505,"log_dups_size":0,"ondisk_log_size":505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T21:57:46.314512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001005061,"stat_sum":{"num_bytes":3281039,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":28,"num_read":8892,"num_read_kb":35489,"num_write":440,"num_write_kb":14429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":391,"num_bytes_recovered":28241520,"num_keys_recovered":141,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"7","length":"c"}]},{"pgid":"3.17","version":"170'319","reported_seq":2298,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-04T17:48:51.057914+0000","last_change":"2024-09-04T17:48:51.058565+0000","last_active":"2024-09-04T17:48:51.057914+0000","last_peered":"2024-09-04T17:48:51.057914+0000","last_clean":"2024-09-04T17:48:51.057914+0000","last_became_active":"2024-09-04T17:45:55.958625+0000","last_became_peered":"2024-09-04T17:45:55.958625+0000","last_unstale":"2024-09-04T17:48:51.057914+0000","last_undegraded":"2024-09-04T17:48:51.057914+0000","last_fullsized":"2024-09-04T17:48:51.057914+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:25:43.927202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039964520000000002,"stat_sum":{"num_bytes":14310102,"num_objects":31,"num_object_clones":3,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":22,"num_read":1608,"num_read_kb":20937,"num_write":310,"num_write_kb":18406,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"8","length":"b"}]},{"pgid":"3.16","version":"170'359","reported_seq":1969,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-04T17:48:51.027642+0000","last_change":"2024-09-04T17:48:51.029349+0000","last_active":"2024-09-04T17:48:51.027642+0000","last_peered":"2024-09-04T17:48:51.027642+0000","last_clean":"2024-09-04T17:48:51.027642+0000","last_became_active":"2024-09-04T17:45:55.953254+0000","last_became_peered":"2024-09-04T17:45:55.953254+0000","last_unstale":"2024-09-04T17:48:51.027642+0000","last_undegraded":"2024-09-04T17:48:51.027642+0000","last_fullsized":"2024-09-04T17:48:51.027642+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T18:01:30.670072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001924565,"stat_sum":{"num_bytes":3432819,"num_objects":29,"num_object_clones":2,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":26,"num_read":1729,"num_read_kb":10785,"num_write":395,"num_write_kb":7391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"b"}]},{"pgid":"3.15","version":"168'506","reported_seq":2587,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-04T17:48:50.124297+0000","last_change":"2024-09-04T17:48:50.128895+0000","last_active":"2024-09-04T17:48:50.124297+0000","last_peered":"2024-09-04T17:48:50.124297+0000","last_clean":"2024-09-04T17:48:50.124297+0000","last_became_active":"2024-09-04T17:48:30.053758+0000","last_became_peered":"2024-09-04T17:48:30.053758+0000","last_unstale":"2024-09-04T17:48:50.124297+0000","last_undegraded":"2024-09-04T17:48:50.124297+0000","last_fullsized":"2024-09-04T17:48:50.124297+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T19:59:47.863082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0052204620000000004,"stat_sum":{"num_bytes":6291456,"num_objects":27,"num_object_clones":2,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":25,"num_read":4020,"num_read_kb":16975,"num_write":525,"num_write_kb":4439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":12989343,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.14","version":"167'767","reported_seq":2445,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-04T17:48:50.129543+0000","last_change":"2024-09-04T17:48:50.130470+0000","last_active":"2024-09-04T17:48:50.129543+0000","last_peered":"2024-09-04T17:48:50.129543+0000","last_clean":"2024-09-04T17:48:50.129543+0000","last_became_active":"2024-09-04T17:46:53.539886+0000","last_became_peered":"2024-09-04T17:46:53.539886+0000","last_unstale":"2024-09-04T17:48:50.129543+0000","last_undegraded":"2024-09-04T17:48:50.129543+0000","last_fullsized":"2024-09-04T17:48:50.129543+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":767,"log_dups_size":0,"ondisk_log_size":767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:35:07.260316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039357239999999998,"stat_sum":{"num_bytes":14743423,"num_objects":36,"num_object_clones":2,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":30,"num_read":4006,"num_read_kb":29861,"num_write":997,"num_write_kb":35909,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"2.7","version":"0'0","reported_seq":182,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-04T17:48:51.054058+0000","last_change":"2024-09-04T17:48:11.850550+0000","last_active":"2024-09-04T17:48:51.054058+0000","last_peered":"2024-09-04T17:48:51.054058+0000","last_clean":"2024-09-04T17:48:51.054058+0000","last_became_active":"2024-09-04T17:48:11.849710+0000","last_became_peered":"2024-09-04T17:48:11.849710+0000","last_unstale":"2024-09-04T17:48:51.054058+0000","last_undegraded":"2024-09-04T17:48:51.054058+0000","last_fullsized":"2024-09-04T17:48:51.054058+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:37:18.791459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001226074,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"171'387","reported_seq":3117,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-04T17:48:51.057732+0000","last_change":"2024-09-04T17:48:51.057732+0000","last_active":"2024-09-04T17:48:51.057732+0000","last_peered":"2024-09-04T17:48:51.057732+0000","last_clean":"2024-09-04T17:48:51.057732+0000","last_became_active":"2024-09-04T17:47:32.848310+0000","last_became_peered":"2024-09-04T17:47:32.848310+0000","last_unstale":"2024-09-04T17:48:51.057732+0000","last_undegraded":"2024-09-04T17:48:51.057732+0000","last_fullsized":"2024-09-04T17:48:51.057732+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:15:15.636862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034750010000000001,"stat_sum":{"num_bytes":2024970,"num_objects":31,"num_object_clones":1,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":26,"num_read":2454,"num_read_kb":14526,"num_write":394,"num_write_kb":5524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":919873,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"8","length":"b"}]},{"pgid":"2.6","version":"0'0","reported_seq":117,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-04T17:48:51.054064+0000","last_change":"2024-09-04T17:47:58.862844+0000","last_active":"2024-09-04T17:48:51.054064+0000","last_peered":"2024-09-04T17:48:51.054064+0000","last_clean":"2024-09-04T17:48:51.054064+0000","last_became_active":"2024-09-04T17:47:58.859941+0000","last_became_peered":"2024-09-04T17:47:58.859941+0000","last_unstale":"2024-09-04T17:48:51.054064+0000","last_undegraded":"2024-09-04T17:48:51.054064+0000","last_fullsized":"2024-09-04T17:48:51.054064+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:08:53.409036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011364179999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"171'468","reported_seq":2942,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-04T17:48:52.023583+0000","last_change":"2024-09-04T17:48:52.023786+0000","last_active":"2024-09-04T17:48:52.023583+0000","last_peered":"2024-09-04T17:48:52.023583+0000","last_clean":"2024-09-04T17:48:52.023583+0000","last_became_active":"2024-09-04T17:48:32.053253+0000","last_became_peered":"2024-09-04T17:48:32.053253+0000","last_unstale":"2024-09-04T17:48:52.023583+0000","last_undegraded":"2024-09-04T17:48:52.023583+0000","last_fullsized":"2024-09-04T17:48:52.023583+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:45:34.358654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014545039999999999,"stat_sum":{"num_bytes":297887,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":32,"num_read":2953,"num_read_kb":23389,"num_write":574,"num_write_kb":21569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":1588111,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"7","length":"c"}]},{"pgid":"2.5","version":"0'0","reported_seq":316,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-04T17:48:50.125074+0000","last_change":"2024-09-04T17:45:49.476607+0000","last_active":"2024-09-04T17:48:50.125074+0000","last_peered":"2024-09-04T17:48:50.125074+0000","last_clean":"2024-09-04T17:48:50.125074+0000","last_became_active":"2024-09-04T17:45:47.369308+0000","last_became_peered":"2024-09-04T17:45:47.369308+0000","last_unstale":"2024-09-04T17:48:50.125074+0000","last_undegraded":"2024-09-04T17:48:50.125074+0000","last_fullsized":"2024-09-04T17:48:50.125074+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:17:13.378553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001029826,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"169'380","reported_seq":2671,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-09-04T17:48:49.081869+0000","last_change":"2024-09-04T17:48:48.977622+0000","last_active":"2024-09-04T17:48:49.081869+0000","last_peered":"2024-09-04T17:48:49.081869+0000","last_clean":"2024-09-04T17:48:49.081869+0000","last_became_active":"2024-09-04T17:48:10.845756+0000","last_became_peered":"2024-09-04T17:48:10.845756+0000","last_unstale":"2024-09-04T17:48:49.081869+0000","last_undegraded":"2024-09-04T17:48:49.081869+0000","last_fullsized":"2024-09-04T17:48:49.081869+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":380,"log_dups_size":0,"ondisk_log_size":380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:41:58.975677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029618859999999999,"stat_sum":{"num_bytes":4039247,"num_objects":39,"num_object_clones":2,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":33,"num_read":2434,"num_read_kb":18577,"num_write":416,"num_write_kb":9850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":237,"num_bytes_recovered":13375622,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"a","length":"9"}]},{"pgid":"2.4","version":"0'0","reported_seq":162,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-04T17:48:51.053771+0000","last_change":"2024-09-04T17:47:29.803682+0000","last_active":"2024-09-04T17:48:51.053771+0000","last_peered":"2024-09-04T17:48:51.053771+0000","last_clean":"2024-09-04T17:48:51.053771+0000","last_became_active":"2024-09-04T17:47:29.803256+0000","last_became_peered":"2024-09-04T17:47:29.803256+0000","last_unstale":"2024-09-04T17:48:51.053771+0000","last_undegraded":"2024-09-04T17:48:51.053771+0000","last_fullsized":"2024-09-04T17:48:51.053771+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:32:26.523162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"170'353","reported_seq":2530,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-04T17:48:51.030126+0000","last_change":"2024-09-04T17:48:51.031451+0000","last_active":"2024-09-04T17:48:51.030126+0000","last_peered":"2024-09-04T17:48:51.030126+0000","last_clean":"2024-09-04T17:48:51.030126+0000","last_became_active":"2024-09-04T17:48:30.051194+0000","last_became_peered":"2024-09-04T17:48:30.051194+0000","last_unstale":"2024-09-04T17:48:51.030126+0000","last_undegraded":"2024-09-04T17:48:51.030126+0000","last_fullsized":"2024-09-04T17:48:51.030126+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:03:10.995220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038200370000000001,"stat_sum":{"num_bytes":4785515,"num_objects":31,"num_object_clones":1,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":27,"num_read":2552,"num_read_kb":7638,"num_write":302,"num_write_kb":7011,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":15596571,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"b"}]},{"pgid":"3.3","version":"171'865","reported_seq":3446,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-04T17:48:51.155138+0000","last_change":"2024-09-04T17:48:51.058156+0000","last_active":"2024-09-04T17:48:51.155138+0000","last_peered":"2024-09-04T17:48:51.155138+0000","last_clean":"2024-09-04T17:48:51.155138+0000","last_became_active":"2024-09-04T17:46:53.542889+0000","last_became_peered":"2024-09-04T17:46:53.542889+0000","last_unstale":"2024-09-04T17:48:51.155138+0000","last_undegraded":"2024-09-04T17:48:51.155138+0000","last_fullsized":"2024-09-04T17:48:51.155138+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T19:24:58.820902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038337470000000002,"stat_sum":{"num_bytes":6660903,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":29,"num_read":4643,"num_read_kb":26531,"num_write":1078,"num_write_kb":14207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"8","length":"b"}]},{"pgid":"2.2","version":"22'2","reported_seq":335,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-04T17:48:51.026966+0000","last_change":"2024-09-04T17:47:59.598241+0000","last_active":"2024-09-04T17:48:51.026966+0000","last_peered":"2024-09-04T17:48:51.026966+0000","last_clean":"2024-09-04T17:48:51.026966+0000","last_became_active":"2024-09-04T17:47:58.884634+0000","last_became_peered":"2024-09-04T17:47:58.884634+0000","last_unstale":"2024-09-04T17:48:51.026966+0000","last_undegraded":"2024-09-04T17:48:51.026966+0000","last_fullsized":"2024-09-04T17:48:51.026966+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T22:02:31.692385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026316069999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"168'449","reported_seq":3340,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-09-04T17:48:48.974647+0000","last_change":"2024-09-04T17:48:48.975209+0000","last_active":"2024-09-04T17:48:48.974647+0000","last_peered":"2024-09-04T17:48:48.974647+0000","last_clean":"2024-09-04T17:48:48.974647+0000","last_became_active":"2024-09-04T17:48:01.849363+0000","last_became_peered":"2024-09-04T17:48:01.849363+0000","last_unstale":"2024-09-04T17:48:48.974647+0000","last_undegraded":"2024-09-04T17:48:48.974647+0000","last_fullsized":"2024-09-04T17:48:48.974647+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T21:53:49.070677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089001200000000003,"stat_sum":{"num_bytes":531127,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":36,"num_read":3563,"num_read_kb":22520,"num_write":361,"num_write_kb":4893,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":5692590,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"a","length":"9"}]},{"pgid":"2.1","version":"0'0","reported_seq":300,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-09-04T17:48:48.974299+0000","last_change":"2024-09-04T17:48:27.034831+0000","last_active":"2024-09-04T17:48:48.974299+0000","last_peered":"2024-09-04T17:48:48.974299+0000","last_clean":"2024-09-04T17:48:48.974299+0000","last_became_active":"2024-09-04T17:48:27.034383+0000","last_became_peered":"2024-09-04T17:48:27.034383+0000","last_unstale":"2024-09-04T17:48:48.974299+0000","last_undegraded":"2024-09-04T17:48:48.974299+0000","last_fullsized":"2024-09-04T17:48:48.974299+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T23:15:59.538283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001025195,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"171'619","reported_seq":3274,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-04T17:48:51.061337+0000","last_change":"2024-09-04T17:48:51.061706+0000","last_active":"2024-09-04T17:48:51.061337+0000","last_peered":"2024-09-04T17:48:51.061337+0000","last_clean":"2024-09-04T17:48:51.061337+0000","last_became_active":"2024-09-04T17:48:11.845510+0000","last_became_peered":"2024-09-04T17:48:11.845510+0000","last_unstale":"2024-09-04T17:48:51.061337+0000","last_undegraded":"2024-09-04T17:48:51.061337+0000","last_fullsized":"2024-09-04T17:48:51.061337+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":619,"log_dups_size":0,"ondisk_log_size":619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:08:34.472021+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0063845020000000002,"stat_sum":{"num_bytes":4643589,"num_objects":34,"num_object_clones":1,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":28,"num_read":3285,"num_read_kb":40522,"num_write":655,"num_write_kb":23355,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":240,"num_bytes_recovered":21297737,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"8","length":"b"}]},{"pgid":"2.0","version":"0'0","reported_seq":322,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-04T17:48:50.122800+0000","last_change":"2024-09-04T17:47:29.804429+0000","last_active":"2024-09-04T17:48:50.122800+0000","last_peered":"2024-09-04T17:48:50.122800+0000","last_clean":"2024-09-04T17:48:50.122800+0000","last_became_active":"2024-09-04T17:47:29.803992+0000","last_became_peered":"2024-09-04T17:47:29.803992+0000","last_unstale":"2024-09-04T17:48:50.122800+0000","last_undegraded":"2024-09-04T17:48:50.122800+0000","last_fullsized":"2024-09-04T17:48:50.122800+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:50:32.563345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010553590000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"172'620","reported_seq":2781,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-04T17:48:52.027487+0000","last_change":"2024-09-04T17:48:52.027880+0000","last_active":"2024-09-04T17:48:52.027487+0000","last_peered":"2024-09-04T17:48:52.027487+0000","last_clean":"2024-09-04T17:48:52.027487+0000","last_became_active":"2024-09-04T17:48:31.052384+0000","last_became_peered":"2024-09-04T17:48:31.052384+0000","last_unstale":"2024-09-04T17:48:52.027487+0000","last_undegraded":"2024-09-04T17:48:52.027487+0000","last_fullsized":"2024-09-04T17:48:52.027487+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":620,"log_dups_size":0,"ondisk_log_size":620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T18:35:30.514410+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0055418890000000004,"stat_sum":{"num_bytes":293590,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":28,"num_read":5711,"num_read_kb":30159,"num_write":738,"num_write_kb":13818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":16173004,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"7","length":"c"}]},{"pgid":"2.3","version":"20'1","reported_seq":329,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-04T17:48:51.026927+0000","last_change":"2024-09-04T17:47:41.919544+0000","last_active":"2024-09-04T17:48:51.026927+0000","last_peered":"2024-09-04T17:48:51.026927+0000","last_clean":"2024-09-04T17:48:51.026927+0000","last_became_active":"2024-09-04T17:47:41.919027+0000","last_became_peered":"2024-09-04T17:47:41.919027+0000","last_unstale":"2024-09-04T17:48:51.026927+0000","last_undegraded":"2024-09-04T17:48:51.026927+0000","last_fullsized":"2024-09-04T17:48:51.026927+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:42:55.688562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086618299999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":407,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-04T17:48:50.122912+0000","last_change":"2024-09-04T17:45:42.341149+0000","last_active":"2024-09-04T17:48:50.122912+0000","last_peered":"2024-09-04T17:48:50.122912+0000","last_clean":"2024-09-04T17:48:50.122912+0000","last_became_active":"2024-09-04T17:45:42.339286+0000","last_became_peered":"2024-09-04T17:45:42.339286+0000","last_unstale":"2024-09-04T17:48:50.122912+0000","last_undegraded":"2024-09-04T17:48:50.122912+0000","last_fullsized":"2024-09-04T17:48:50.122912+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:41.305128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:41.305128+0000","last_clean_scrub_stamp":"2024-09-04T17:45:41.305128+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:52:24.860049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"172'544","reported_seq":2999,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-04T17:48:52.023503+0000","last_change":"2024-09-04T17:48:52.023503+0000","last_active":"2024-09-04T17:48:52.023503+0000","last_peered":"2024-09-04T17:48:52.023503+0000","last_clean":"2024-09-04T17:48:52.023503+0000","last_became_active":"2024-09-04T17:48:31.051869+0000","last_became_peered":"2024-09-04T17:48:31.051869+0000","last_unstale":"2024-09-04T17:48:52.023503+0000","last_undegraded":"2024-09-04T17:48:52.023503+0000","last_fullsized":"2024-09-04T17:48:52.023503+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T22:01:34.207704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014354140000000001,"stat_sum":{"num_bytes":3359411,"num_objects":34,"num_object_clones":1,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":31,"num_read":2788,"num_read_kb":17264,"num_write":575,"num_write_kb":29373,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":319,"num_bytes_recovered":17940277,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"7","length":"c"}]},{"pgid":"3.9","version":"168'672","reported_seq":4533,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-09-04T17:48:48.974926+0000","last_change":"2024-09-04T17:48:48.975831+0000","last_active":"2024-09-04T17:48:48.974926+0000","last_peered":"2024-09-04T17:48:48.974926+0000","last_clean":"2024-09-04T17:48:48.974926+0000","last_became_active":"2024-09-04T17:47:44.949630+0000","last_became_peered":"2024-09-04T17:47:44.949630+0000","last_unstale":"2024-09-04T17:48:48.974926+0000","last_undegraded":"2024-09-04T17:48:48.974926+0000","last_fullsized":"2024-09-04T17:48:48.974926+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":672,"log_dups_size":0,"ondisk_log_size":672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T21:58:53.185242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00131708,"stat_sum":{"num_bytes":8421425,"num_objects":44,"num_object_clones":4,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":36,"num_read":5999,"num_read_kb":37328,"num_write":739,"num_write_kb":18642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":4534479,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"a","length":"9"}]},{"pgid":"3.a","version":"170'640","reported_seq":3547,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-04T17:48:51.054571+0000","last_change":"2024-09-04T17:48:51.055000+0000","last_active":"2024-09-04T17:48:51.054571+0000","last_peered":"2024-09-04T17:48:51.054571+0000","last_clean":"2024-09-04T17:48:51.054571+0000","last_became_active":"2024-09-04T17:48:00.841070+0000","last_became_peered":"2024-09-04T17:48:00.841070+0000","last_unstale":"2024-09-04T17:48:51.054571+0000","last_undegraded":"2024-09-04T17:48:51.054571+0000","last_fullsized":"2024-09-04T17:48:51.054571+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":640,"log_dups_size":0,"ondisk_log_size":640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T19:35:01.271446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077206699999999998,"stat_sum":{"num_bytes":641130,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":31,"num_read":5676,"num_read_kb":16456,"num_write":602,"num_write_kb":2753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":1572433,"num_keys_recovered":66,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"8","length":"b"}]},{"pgid":"3.b","version":"170'741","reported_seq":2558,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-04T17:48:52.023143+0000","last_change":"2024-09-04T17:48:52.023882+0000","last_active":"2024-09-04T17:48:52.023143+0000","last_peered":"2024-09-04T17:48:52.023143+0000","last_clean":"2024-09-04T17:48:52.023143+0000","last_became_active":"2024-09-04T17:48:31.049842+0000","last_became_peered":"2024-09-04T17:48:31.049842+0000","last_unstale":"2024-09-04T17:48:52.023143+0000","last_undegraded":"2024-09-04T17:48:52.023143+0000","last_fullsized":"2024-09-04T17:48:52.023143+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":741,"log_dups_size":0,"ondisk_log_size":741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:28:07.850605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018877830000000001,"stat_sum":{"num_bytes":4832707,"num_objects":33,"num_object_clones":1,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":28,"num_read":4059,"num_read_kb":14632,"num_write":895,"num_write_kb":7921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":5211501,"num_keys_recovered":33,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"7","length":"c"}]},{"pgid":"3.c","version":"171'640","reported_seq":8550,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-04T17:48:51.030409+0000","last_change":"2024-09-04T17:48:51.031474+0000","last_active":"2024-09-04T17:48:51.030409+0000","last_peered":"2024-09-04T17:48:51.030409+0000","last_clean":"2024-09-04T17:48:51.030409+0000","last_became_active":"2024-09-04T17:48:32.054696+0000","last_became_peered":"2024-09-04T17:48:32.054696+0000","last_unstale":"2024-09-04T17:48:51.030409+0000","last_undegraded":"2024-09-04T17:48:51.030409+0000","last_fullsized":"2024-09-04T17:48:51.030409+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":640,"log_dups_size":0,"ondisk_log_size":640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T21:00:26.563001+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0039887289999999999,"stat_sum":{"num_bytes":8460475,"num_objects":41,"num_object_clones":2,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":36,"num_read":9052,"num_read_kb":71439,"num_write":751,"num_write_kb":16863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":281,"num_bytes_recovered":8281724,"num_keys_recovered":57,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"b"}]},{"pgid":"3.d","version":"169'1351","reported_seq":4494,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-04T17:48:50.129047+0000","last_change":"2024-09-04T17:48:50.129396+0000","last_active":"2024-09-04T17:48:50.129047+0000","last_peered":"2024-09-04T17:48:50.129047+0000","last_clean":"2024-09-04T17:48:50.129047+0000","last_became_active":"2024-09-04T17:48:12.850381+0000","last_became_peered":"2024-09-04T17:48:12.850381+0000","last_unstale":"2024-09-04T17:48:50.129047+0000","last_undegraded":"2024-09-04T17:48:50.129047+0000","last_fullsized":"2024-09-04T17:48:50.129047+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":1351,"log_dups_size":0,"ondisk_log_size":1351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T18:09:18.281356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049047689999999998,"stat_sum":{"num_bytes":12260025,"num_objects":37,"num_object_clones":2,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":32,"num_read":8863,"num_read_kb":26532,"num_write":2251,"num_write_kb":13198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":179,"num_bytes_recovered":18877339,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.e","version":"170'538","reported_seq":5989,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-04T17:48:50.125352+0000","last_change":"2024-09-04T17:48:50.125352+0000","last_active":"2024-09-04T17:48:50.125352+0000","last_peered":"2024-09-04T17:48:50.125352+0000","last_clean":"2024-09-04T17:48:50.125352+0000","last_became_active":"2024-09-04T17:48:11.849605+0000","last_became_peered":"2024-09-04T17:48:11.849605+0000","last_unstale":"2024-09-04T17:48:50.125352+0000","last_undegraded":"2024-09-04T17:48:50.125352+0000","last_fullsized":"2024-09-04T17:48:50.125352+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":538,"log_dups_size":0,"ondisk_log_size":538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T19:08:13.023058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015976580000000001,"stat_sum":{"num_bytes":5723405,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":23,"num_read":6678,"num_read_kb":28511,"num_write":520,"num_write_kb":11228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":7806862,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.f","version":"168'336","reported_seq":3136,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-04T17:48:50.129270+0000","last_change":"2024-09-04T17:48:50.129954+0000","last_active":"2024-09-04T17:48:50.129270+0000","last_peered":"2024-09-04T17:48:50.129270+0000","last_clean":"2024-09-04T17:48:50.129270+0000","last_became_active":"2024-09-04T17:45:55.956734+0000","last_became_peered":"2024-09-04T17:45:55.956734+0000","last_unstale":"2024-09-04T17:48:50.129270+0000","last_undegraded":"2024-09-04T17:48:50.129270+0000","last_fullsized":"2024-09-04T17:48:50.129270+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":336,"log_dups_size":0,"ondisk_log_size":336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:27:46.715773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048776569999999997,"stat_sum":{"num_bytes":17021332,"num_objects":36,"num_object_clones":3,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":27,"num_read":2605,"num_read_kb":44969,"num_write":324,"num_write_kb":13427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.10","version":"170'603","reported_seq":3673,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-04T17:48:51.055439+0000","last_change":"2024-09-04T17:48:51.055990+0000","last_active":"2024-09-04T17:48:51.055439+0000","last_peered":"2024-09-04T17:48:51.055439+0000","last_clean":"2024-09-04T17:48:51.055439+0000","last_became_active":"2024-09-04T17:48:00.843136+0000","last_became_peered":"2024-09-04T17:48:00.843136+0000","last_unstale":"2024-09-04T17:48:51.055439+0000","last_undegraded":"2024-09-04T17:48:51.055439+0000","last_fullsized":"2024-09-04T17:48:51.055439+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T23:38:14.448304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011819700000000001,"stat_sum":{"num_bytes":3184544,"num_objects":43,"num_object_clones":2,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":39,"num_read":4584,"num_read_kb":18785,"num_write":655,"num_write_kb":12848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":3377599,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"8","length":"b"}]},{"pgid":"3.11","version":"169'581","reported_seq":6234,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-04T17:48:50.123903+0000","last_change":"2024-09-04T17:48:50.124316+0000","last_active":"2024-09-04T17:48:50.123903+0000","last_peered":"2024-09-04T17:48:50.123903+0000","last_clean":"2024-09-04T17:48:50.123903+0000","last_became_active":"2024-09-04T17:45:55.951870+0000","last_became_peered":"2024-09-04T17:45:55.951870+0000","last_unstale":"2024-09-04T17:48:50.123903+0000","last_undegraded":"2024-09-04T17:48:50.123903+0000","last_fullsized":"2024-09-04T17:48:50.123903+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":581,"log_dups_size":0,"ondisk_log_size":581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T21:31:17.027357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010857200000000001,"stat_sum":{"num_bytes":2228378,"num_objects":45,"num_object_clones":2,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":40,"num_read":7130,"num_read_kb":61414,"num_write":569,"num_write_kb":17572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.12","version":"171'421","reported_seq":4140,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-04T17:48:51.057325+0000","last_change":"2024-09-04T17:48:51.057828+0000","last_active":"2024-09-04T17:48:51.057325+0000","last_peered":"2024-09-04T17:48:51.057325+0000","last_clean":"2024-09-04T17:48:51.057325+0000","last_became_active":"2024-09-04T17:48:14.943778+0000","last_became_peered":"2024-09-04T17:48:14.943778+0000","last_unstale":"2024-09-04T17:48:51.057325+0000","last_undegraded":"2024-09-04T17:48:51.057325+0000","last_fullsized":"2024-09-04T17:48:51.057325+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:06:04.625801+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0037784440000000002,"stat_sum":{"num_bytes":1336414,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":31,"num_read":3573,"num_read_kb":21717,"num_write":443,"num_write_kb":8575,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":290,"num_bytes_recovered":2723025,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"8","length":"b"}]},{"pgid":"3.13","version":"168'307","reported_seq":2603,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-04T17:48:50.125561+0000","last_change":"2024-09-04T17:48:50.128944+0000","last_active":"2024-09-04T17:48:50.125561+0000","last_peered":"2024-09-04T17:48:50.125561+0000","last_clean":"2024-09-04T17:48:50.125561+0000","last_became_active":"2024-09-04T17:45:55.954544+0000","last_became_peered":"2024-09-04T17:45:55.954544+0000","last_unstale":"2024-09-04T17:48:50.125561+0000","last_undegraded":"2024-09-04T17:48:50.125561+0000","last_fullsized":"2024-09-04T17:48:50.125561+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":307,"log_dups_size":0,"ondisk_log_size":307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:57:58.642719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045456990000000003,"stat_sum":{"num_bytes":5636096,"num_objects":40,"num_object_clones":1,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":35,"num_read":2420,"num_read_kb":22743,"num_write":246,"num_write_kb":4380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"9","length":"a"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":177233829,"num_objects":1099,"num_object_clones":40,"num_object_copies":2198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1099,"num_whiteouts":960,"num_read":140836,"num_read_kb":875514,"num_write":20004,"num_write_kb":404203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4670,"num_bytes_recovered":238943356,"num_keys_recovered":777,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":144166912,"data_stored":172911879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21395,"internal_metadata":0},"log_size":17808,"ondisk_log_size":17808,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":117,"seq":502511173644,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60408,"kb_used_data":22500,"kb_used_omap":22,"kb_used_meta":37865,"kb_avail":94311432,"statfs":{"total":96636764160,"available":96574906368,"internally_reserved":0,"allocated":23040000,"data_stored":21949565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22742,"internal_metadata":38774570},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509481,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81304,"kb_used_data":32064,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94290536,"statfs":{"total":96636764160,"available":96553508864,"internally_reserved":0,"allocated":32833536,"data_stored":39997901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509482,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79832,"kb_used_data":30600,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94292008,"statfs":{"total":96636764160,"available":96555016192,"internally_reserved":0,"allocated":31334400,"data_stored":39075236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":48,"seq":206158430238,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45252,"kb_used_data":13104,"kb_used_omap":4,"kb_used_meta":32123,"kb_avail":94326588,"statfs":{"total":96636764160,"available":96590426112,"internally_reserved":0,"allocated":13418496,"data_stored":14341975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4430,"internal_metadata":32894642},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":0,"seq":64424509481,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509481,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72668,"kb_used_data":19008,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94299172,"statfs":{"total":96636764160,"available":96562352128,"internally_reserved":0,"allocated":19464192,"data_stored":21407686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542185,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":89476,"kb_used_data":31840,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94282364,"statfs":{"total":96636764160,"available":96545140736,"internally_reserved":0,"allocated":32604160,"data_stored":38842952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":103,"seq":442381631503,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50132,"kb_used_data":9980,"kb_used_omap":12,"kb_used_meta":40115,"kb_avail":94321708,"statfs":{"total":96636764160,"available":96585428992,"internally_reserved":0,"allocated":10219520,"data_stored":10700446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12804,"internal_metadata":41078268},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16711680,"data_stored":19720305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2035712,"data_stored":2023494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192000,"data_stored":9718476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6121,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":7086080,"data_stored":7009490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":29192192,"data_stored":38005652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":30531584,"data_stored":38760181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20996096,"data_stored":20961341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15274,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":29421568,"data_stored":36712940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-04T17:48:54.444 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-04T17:48:54.477 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:54.477 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:48:54.477 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-04T17:48:54.477 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-04T17:48:54.477 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:54.492 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-04T17:48:54.538 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:48:54.534+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:48:54.602 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:48:54.601+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:48:54.662 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:54.662 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-04T17:48:54.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-04T17:48:54.662 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-04T17:48:54.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.662 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59950 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5365760~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:48:54.662 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654917+0000", 2024-09-04T17:48:54.662 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 101.00442932, 2024-09-04T17:48:54.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066822450000000005, 2024-09-04T17:48:54.662 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:54.663 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:54.663 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:54.663 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:54.663 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:54.663 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:54.663 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59950 2024-09-04T17:48:54.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.663 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:54.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:54.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654917+0000", 2024-09-04T17:48:54.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:54.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:54.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654917+0000", 2024-09-04T17:48:54.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:54.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:54.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654918+0000", 2024-09-04T17:48:54.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.29e-07 2024-09-04T17:48:54.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:54.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654919+0000", 2024-09-04T17:48:54.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2589999999999999e-06 2024-09-04T17:48:54.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.665 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:54.665 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654922+0000", 2024-09-04T17:48:54.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1350000000000001e-06 2024-09-04T17:48:54.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.665 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:54.665 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654925+0000", 2024-09-04T17:48:54.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1250000000000001e-06 2024-09-04T17:48:54.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.666 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:54.666 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721466+0000", 2024-09-04T17:48:54.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066540959999999996 2024-09-04T17:48:54.666 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.666 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:54.666 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721574+0000", 2024-09-04T17:48:54.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010747399999999999 2024-09-04T17:48:54.666 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.666 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:54.666 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721740+0000", 2024-09-04T17:48:54.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016626799999999999 2024-09-04T17:48:54.667 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:54.667 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:54.667 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:54.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.667 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59951 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5472256~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:48:54.667 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654929+0000", 2024-09-04T17:48:54.667 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 101.00441813499999, 2024-09-04T17:48:54.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067110534999999999, 2024-09-04T17:48:54.667 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:54.667 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:54.667 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:54.667 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:54.668 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:54.668 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:54.668 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59951 2024-09-04T17:48:54.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.668 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:54.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.668 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:54.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654929+0000", 2024-09-04T17:48:54.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:54.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.669 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:54.669 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654929+0000", 2024-09-04T17:48:54.669 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:54.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.669 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:54.669 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654929+0000", 2024-09-04T17:48:54.669 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5100000000000001e-07 2024-09-04T17:48:54.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.669 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:54.670 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654930+0000", 2024-09-04T17:48:54.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2440000000000001e-06 2024-09-04T17:48:54.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.670 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:54.670 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654933+0000", 2024-09-04T17:48:54.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1599999999999998e-06 2024-09-04T17:48:54.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.670 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:54.670 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654936+0000", 2024-09-04T17:48:54.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6560000000000001e-06 2024-09-04T17:48:54.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:54.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721789+0000", 2024-09-04T17:48:54.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066853250000000003 2024-09-04T17:48:54.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:54.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.721885+0000", 2024-09-04T17:48:54.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5616000000000001e-05 2024-09-04T17:48:54.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:54.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722039+0000", 2024-09-04T17:48:54.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000154258 2024-09-04T17:48:54.672 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:54.672 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:54.672 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:54.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.672 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59952 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5505024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:48:54.672 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654939+0000", 2024-09-04T17:48:54.672 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 101.004407258, 2024-09-04T17:48:54.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067433742000000005, 2024-09-04T17:48:54.672 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:54.672 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:54.672 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:54.672 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:54.673 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:54.673 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:54.673 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59952 2024-09-04T17:48:54.673 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.673 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:54.673 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.673 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:54.673 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654939+0000", 2024-09-04T17:48:54.673 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:54.673 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.673 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.673 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:54.673 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654939+0000", 2024-09-04T17:48:54.674 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:54.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.674 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.674 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:54.674 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654940+0000", 2024-09-04T17:48:54.674 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3300000000000001e-07 2024-09-04T17:48:54.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.674 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.674 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:54.674 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654941+0000", 2024-09-04T17:48:54.674 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1570000000000001e-06 2024-09-04T17:48:54.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.674 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.675 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:54.675 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654944+0000", 2024-09-04T17:48:54.675 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1319999999999998e-06 2024-09-04T17:48:54.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.675 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.675 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:54.675 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654947+0000", 2024-09-04T17:48:54.675 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5969999999999999e-06 2024-09-04T17:48:54.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.675 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.675 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:54.675 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722089+0000", 2024-09-04T17:48:54.675 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067142347000000005 2024-09-04T17:48:54.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.676 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:54.676 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722201+0000", 2024-09-04T17:48:54.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011235600000000001 2024-09-04T17:48:54.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.676 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:54.676 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722373+0000", 2024-09-04T17:48:54.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017191999999999999 2024-09-04T17:48:54.676 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:54.676 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:54.676 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:54.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.677 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59953 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5611520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:48:54.677 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654950+0000", 2024-09-04T17:48:54.677 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 101.004396618, 2024-09-04T17:48:54.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067601852000000004, 2024-09-04T17:48:54.677 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:54.677 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:54.677 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:54.677 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:54.677 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:54.677 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:54.677 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59953 2024-09-04T17:48:54.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.678 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:54.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.678 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:54.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654950+0000", 2024-09-04T17:48:54.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:54.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.678 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:54.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654950+0000", 2024-09-04T17:48:54.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:54.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.678 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:54.679 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654950+0000", 2024-09-04T17:48:54.679 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2499999999999999e-07 2024-09-04T17:48:54.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.679 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:54.679 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654954+0000", 2024-09-04T17:48:54.679 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.287e-06 2024-09-04T17:48:54.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.679 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:54.679 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654962+0000", 2024-09-04T17:48:54.679 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4230000000000005e-06 2024-09-04T17:48:54.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.680 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:54.680 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654966+0000", 2024-09-04T17:48:54.680 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6449999999999998e-06 2024-09-04T17:48:54.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.680 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:54.680 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722417+0000", 2024-09-04T17:48:54.680 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067451188999999995 2024-09-04T17:48:54.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:54.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722467+0000", 2024-09-04T17:48:54.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0556999999999997e-05 2024-09-04T17:48:54.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:54.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722552+0000", 2024-09-04T17:48:54.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4525999999999997e-05 2024-09-04T17:48:54.681 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:54.681 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:54.681 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:54.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.682 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59954 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5718016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:48:54.682 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654970+0000", 2024-09-04T17:48:54.682 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 101.004376857, 2024-09-04T17:48:54.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067874285000000006, 2024-09-04T17:48:54.682 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:54.682 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:54.682 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:54.682 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:54.682 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:54.682 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:54.682 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59954 2024-09-04T17:48:54.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.683 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:54.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.683 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:54.683 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654970+0000", 2024-09-04T17:48:54.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:54.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.683 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:54.683 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654970+0000", 2024-09-04T17:48:54.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:54.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.684 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:54.684 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654970+0000", 2024-09-04T17:48:54.684 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5899999999999998e-07 2024-09-04T17:48:54.684 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.684 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.684 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:54.684 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654971+0000", 2024-09-04T17:48:54.684 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2929999999999999e-06 2024-09-04T17:48:54.684 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.684 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.684 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:54.684 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654975+0000", 2024-09-04T17:48:54.685 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4699999999999998e-06 2024-09-04T17:48:54.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.685 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:54.685 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654978+0000", 2024-09-04T17:48:54.685 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7930000000000002e-06 2024-09-04T17:48:54.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.685 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:54.685 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722600+0000", 2024-09-04T17:48:54.685 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067622229000000006 2024-09-04T17:48:54.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:54.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722689+0000", 2024-09-04T17:48:54.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9511000000000004e-05 2024-09-04T17:48:54.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:54.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722844+0000", 2024-09-04T17:48:54.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015473 2024-09-04T17:48:54.686 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:54.686 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:54.686 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:54.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.686 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59955 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5857280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:48:54.687 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654981+0000", 2024-09-04T17:48:54.687 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 101.004365507, 2024-09-04T17:48:54.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068052360000000006, 2024-09-04T17:48:54.687 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:54.687 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:54.687 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:54.687 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:54.687 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:54.687 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:54.687 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59955 2024-09-04T17:48:54.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.687 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:54.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.688 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:54.688 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654981+0000", 2024-09-04T17:48:54.688 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:54.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.688 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.688 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:54.688 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654981+0000", 2024-09-04T17:48:54.688 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:54.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.688 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.688 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:54.688 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654981+0000", 2024-09-04T17:48:54.688 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2399999999999999e-07 2024-09-04T17:48:54.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.689 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:54.689 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654985+0000", 2024-09-04T17:48:54.689 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0980000000000004e-06 2024-09-04T17:48:54.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.689 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:54.689 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654990+0000", 2024-09-04T17:48:54.689 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3429999999999998e-06 2024-09-04T17:48:54.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.689 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:54.690 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654993+0000", 2024-09-04T17:48:54.690 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8700000000000001e-06 2024-09-04T17:48:54.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.690 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:54.690 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722888+0000", 2024-09-04T17:48:54.690 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067895239999999996 2024-09-04T17:48:54.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.690 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:54.690 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.722948+0000", 2024-09-04T17:48:54.690 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9750999999999998e-05 2024-09-04T17:48:54.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.691 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:54.691 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.723034+0000", 2024-09-04T17:48:54.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5834000000000005e-05 2024-09-04T17:48:54.691 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:54.691 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:54.691 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:54.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.691 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:59956 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 5963776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-04T17:48:54.691 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.654998+0000", 2024-09-04T17:48:54.691 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 101.004348978, 2024-09-04T17:48:54.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068359518999999994, 2024-09-04T17:48:54.691 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:54.692 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:54.692 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:54.692 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:54.692 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:54.692 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:54.692 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 59956 2024-09-04T17:48:54.692 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.692 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:54.692 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.692 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:54.692 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654998+0000", 2024-09-04T17:48:54.692 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:54.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.693 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:54.693 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654998+0000", 2024-09-04T17:48:54.693 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:54.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.693 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:54.693 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654998+0000", 2024-09-04T17:48:54.693 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4299999999999999e-07 2024-09-04T17:48:54.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.693 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:54.694 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.654999+0000", 2024-09-04T17:48:54.694 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1799999999999999e-06 2024-09-04T17:48:54.694 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.694 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.694 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:54.694 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.655002+0000", 2024-09-04T17:48:54.694 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3639999999999999e-06 2024-09-04T17:48:54.694 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.694 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.694 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:54.694 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.655005+0000", 2024-09-04T17:48:54.694 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7970000000000001e-06 2024-09-04T17:48:54.694 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.694 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.695 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:54.695 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.723085+0000", 2024-09-04T17:48:54.695 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068080229000000006 2024-09-04T17:48:54.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.695 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.695 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:54.695 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.723197+0000", 2024-09-04T17:48:54.695 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000111344 2024-09-04T17:48:54.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.695 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.695 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:54.695 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.723357+0000", 2024-09-04T17:48:54.695 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016036199999999999 2024-09-04T17:48:54.696 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:54.696 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:54.696 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:54.696 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.696 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.696 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61082 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 2654208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:54.696 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.822664+0000", 2024-09-04T17:48:54.696 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 100.836682591, 2024-09-04T17:48:54.696 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067072242000000004, 2024-09-04T17:48:54.696 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:54.696 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:54.696 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:54.696 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:54.696 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:54.697 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:54.697 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61082 2024-09-04T17:48:54.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.697 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:54.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.697 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:54.697 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822664+0000", 2024-09-04T17:48:54.697 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:54.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.697 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:54.697 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822664+0000", 2024-09-04T17:48:54.698 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:54.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.698 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.698 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:54.698 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822664+0000", 2024-09-04T17:48:54.698 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6600000000000003e-07 2024-09-04T17:48:54.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.698 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.698 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:54.698 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822666+0000", 2024-09-04T17:48:54.698 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.536e-06 2024-09-04T17:48:54.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.698 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.698 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:54.699 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822669+0000", 2024-09-04T17:48:54.699 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3759999999999999e-06 2024-09-04T17:48:54.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.699 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.699 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:54.699 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822672+0000", 2024-09-04T17:48:54.699 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8109999999999999e-06 2024-09-04T17:48:54.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.699 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.699 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:54.699 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889263+0000", 2024-09-04T17:48:54.699 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066591259999999999 2024-09-04T17:48:54.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.700 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.700 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:54.700 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889369+0000", 2024-09-04T17:48:54.700 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000105362 2024-09-04T17:48:54.700 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.700 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.700 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:54.700 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889736+0000", 2024-09-04T17:48:54.700 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00036763099999999999 2024-09-04T17:48:54.700 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:54.700 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:54.700 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:54.700 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.701 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.701 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61083 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 2760704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:54.701 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.822678+0000", 2024-09-04T17:48:54.701 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 100.83666885700001, 2024-09-04T17:48:54.701 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067475996999999996, 2024-09-04T17:48:54.701 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:54.701 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:54.701 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:54.701 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:54.701 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:54.701 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:54.701 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61083 2024-09-04T17:48:54.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.701 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:54.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:54.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822678+0000", 2024-09-04T17:48:54.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:54.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:54.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822678+0000", 2024-09-04T17:48:54.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:54.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:54.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822725+0000", 2024-09-04T17:48:54.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7429000000000003e-05 2024-09-04T17:48:54.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.703 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:54.703 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822728+0000", 2024-09-04T17:48:54.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2140000000000001e-06 2024-09-04T17:48:54.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.703 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:54.703 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822737+0000", 2024-09-04T17:48:54.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4370000000000007e-06 2024-09-04T17:48:54.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.703 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:54.704 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822743+0000", 2024-09-04T17:48:54.704 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7520000000000002e-06 2024-09-04T17:48:54.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.704 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:54.704 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889785+0000", 2024-09-04T17:48:54.704 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067042265000000004 2024-09-04T17:48:54.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.704 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:54.704 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.889840+0000", 2024-09-04T17:48:54.704 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5254000000000003e-05 2024-09-04T17:48:54.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.705 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.705 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:54.705 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.890154+0000", 2024-09-04T17:48:54.705 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000313646 2024-09-04T17:48:54.705 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:54.705 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:54.705 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:54.705 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.705 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.705 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61084 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 2867200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:54.705 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.822753+0000", 2024-09-04T17:48:54.705 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 100.836593921, 2024-09-04T17:48:54.705 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068497161000000001, 2024-09-04T17:48:54.706 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:54.706 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:54.706 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:54.706 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:54.706 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:54.706 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:54.706 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61084 2024-09-04T17:48:54.706 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.706 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:54.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.707 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:54.707 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822753+0000", 2024-09-04T17:48:54.707 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:54.707 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.707 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:54.707 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822753+0000", 2024-09-04T17:48:54.707 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:54.707 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.707 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:54.708 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822753+0000", 2024-09-04T17:48:54.708 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9100000000000003e-07 2024-09-04T17:48:54.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.708 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.708 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:54.708 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822756+0000", 2024-09-04T17:48:54.708 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.678e-06 2024-09-04T17:48:54.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.708 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.708 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:54.708 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822763+0000", 2024-09-04T17:48:54.708 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8220000000000001e-06 2024-09-04T17:48:54.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.709 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.709 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:54.709 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.822768+0000", 2024-09-04T17:48:54.709 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1930000000000001e-06 2024-09-04T17:48:54.709 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.709 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.709 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:54.709 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.890246+0000", 2024-09-04T17:48:54.709 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067477996999999998 2024-09-04T17:48:54.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.710 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.710 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:54.710 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.890555+0000", 2024-09-04T17:48:54.710 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000309143 2024-09-04T17:48:54.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.710 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.710 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:54.710 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.891250+0000", 2024-09-04T17:48:54.710 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00069463700000000001 2024-09-04T17:48:54.710 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:54.710 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:54.710 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:54.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.711 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61285 3.8 3:117cfeb9:::rbd_data.11341dfde0c5.0000000000000006:1f [read 2760704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:54.711 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835200+0000", 2024-09-04T17:48:54.711 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 100.82414685400001, 2024-09-04T17:48:54.711 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066899803999999993, 2024-09-04T17:48:54.711 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:54.711 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:54.711 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:54.711 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:54.711 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:54.711 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:54.711 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61285 2024-09-04T17:48:54.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.712 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:54.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.712 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:54.712 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835200+0000", 2024-09-04T17:48:54.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:54.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.712 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:54.712 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835200+0000", 2024-09-04T17:48:54.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:54.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.712 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:54.713 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835200+0000", 2024-09-04T17:48:54.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0899999999999997e-07 2024-09-04T17:48:54.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.713 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:54.713 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835206+0000", 2024-09-04T17:48:54.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3280000000000002e-06 2024-09-04T17:48:54.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.714 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:54.714 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835211+0000", 2024-09-04T17:48:54.714 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9230000000000001e-06 2024-09-04T17:48:54.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.714 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.714 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:54.714 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835216+0000", 2024-09-04T17:48:54.714 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9019999999999998e-06 2024-09-04T17:48:54.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.714 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.715 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:54.715 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.901719+0000", 2024-09-04T17:48:54.715 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066502981000000003 2024-09-04T17:48:54.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.715 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:54.715 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.901813+0000", 2024-09-04T17:48:54.715 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.3905000000000003e-05 2024-09-04T17:48:54.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.716 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:54.716 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902100+0000", 2024-09-04T17:48:54.716 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028645599999999998 2024-09-04T17:48:54.716 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:54.716 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:54.716 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:54.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.716 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.716 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61286 3.8 3:117cfeb9:::rbd_data.11341dfde0c5.0000000000000006:1f [read 2867200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:54.716 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835244+0000", 2024-09-04T17:48:54.716 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 100.824102235, 2024-09-04T17:48:54.716 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067143215000000006, 2024-09-04T17:48:54.716 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:54.717 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:54.717 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:54.717 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:54.717 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:54.717 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:54.717 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61286 2024-09-04T17:48:54.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.717 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:54.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.717 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:54.717 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835244+0000", 2024-09-04T17:48:54.717 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:54.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.718 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:54.718 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835244+0000", 2024-09-04T17:48:54.718 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:54.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.718 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:54.718 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835245+0000", 2024-09-04T17:48:54.718 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8400000000000004e-07 2024-09-04T17:48:54.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.718 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:54.718 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835248+0000", 2024-09-04T17:48:54.718 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6850000000000001e-06 2024-09-04T17:48:54.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.719 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.719 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:54.719 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835254+0000", 2024-09-04T17:48:54.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.494e-06 2024-09-04T17:48:54.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.719 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.719 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:54.719 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835260+0000", 2024-09-04T17:48:54.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0580000000000003e-06 2024-09-04T17:48:54.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.719 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.719 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:54.719 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902119+0000", 2024-09-04T17:48:54.720 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066858318 2024-09-04T17:48:54.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.720 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:54.720 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902163+0000", 2024-09-04T17:48:54.720 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4753000000000003e-05 2024-09-04T17:48:54.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.720 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:54.720 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902388+0000", 2024-09-04T17:48:54.720 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022422300000000001 2024-09-04T17:48:54.720 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:54.720 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:54.721 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:54.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.721 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.721 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61315 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 98304~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:54.721 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835269+0000", 2024-09-04T17:48:54.721 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 100.82407744299999, 2024-09-04T17:48:54.721 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067329396, 2024-09-04T17:48:54.721 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:54.721 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:54.721 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:54.721 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:54.721 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:54.721 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:54.721 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61315 2024-09-04T17:48:54.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.722 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:54.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.722 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:54.722 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835269+0000", 2024-09-04T17:48:54.722 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:54.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.722 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:54.722 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835269+0000", 2024-09-04T17:48:54.722 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:54.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:54.723 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835270+0000", 2024-09-04T17:48:54.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6599999999999996e-07 2024-09-04T17:48:54.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:54.723 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835272+0000", 2024-09-04T17:48:54.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.261e-06 2024-09-04T17:48:54.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:54.723 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835278+0000", 2024-09-04T17:48:54.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.443e-06 2024-09-04T17:48:54.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.724 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:54.724 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835284+0000", 2024-09-04T17:48:54.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3290000000000001e-06 2024-09-04T17:48:54.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.724 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:54.724 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902147+0000", 2024-09-04T17:48:54.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066862695999999999 2024-09-04T17:48:54.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.724 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:54.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902248+0000", 2024-09-04T17:48:54.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000101501 2024-09-04T17:48:54.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:54.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902599+0000", 2024-09-04T17:48:54.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00035060000000000001 2024-09-04T17:48:54.725 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:54.725 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:54.725 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:54.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.725 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61316 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 204800~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:54.725 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835291+0000", 2024-09-04T17:48:54.726 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 100.82405542399999, 2024-09-04T17:48:54.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067632052999999998, 2024-09-04T17:48:54.726 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:54.726 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:54.726 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:54.726 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:54.726 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:54.726 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:54.726 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61316 2024-09-04T17:48:54.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.726 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:54.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.726 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:54.727 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835291+0000", 2024-09-04T17:48:54.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:54.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.727 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.727 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:54.727 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835291+0000", 2024-09-04T17:48:54.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:54.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.727 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.727 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:54.727 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835292+0000", 2024-09-04T17:48:54.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1899999999999998e-07 2024-09-04T17:48:54.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:54.728 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835294+0000", 2024-09-04T17:48:54.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0880000000000002e-06 2024-09-04T17:48:54.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:54.728 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835300+0000", 2024-09-04T17:48:54.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.224e-06 2024-09-04T17:48:54.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:54.728 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835305+0000", 2024-09-04T17:48:54.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0730000000000004e-06 2024-09-04T17:48:54.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.729 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:54.729 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902632+0000", 2024-09-04T17:48:54.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067327322999999994 2024-09-04T17:48:54.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.729 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:54.729 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902688+0000", 2024-09-04T17:48:54.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5177999999999999e-05 2024-09-04T17:48:54.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.730 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:54.730 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902923+0000", 2024-09-04T17:48:54.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023574800000000001 2024-09-04T17:48:54.730 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:54.730 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:54.730 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:54.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.730 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61317 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 344064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:54.730 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835312+0000", 2024-09-04T17:48:54.730 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 100.82403442, 2024-09-04T17:48:54.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067918511000000001, 2024-09-04T17:48:54.730 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:54.731 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:54.731 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:54.731 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:54.731 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:54.731 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:54.731 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61317 2024-09-04T17:48:54.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.731 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:54.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.731 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:54.731 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835312+0000", 2024-09-04T17:48:54.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:54.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:54.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835312+0000", 2024-09-04T17:48:54.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:54.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:54.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835313+0000", 2024-09-04T17:48:54.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9000000000000002e-07 2024-09-04T17:48:54.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:54.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835315+0000", 2024-09-04T17:48:54.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1689999999999999e-06 2024-09-04T17:48:54.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:54.733 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835321+0000", 2024-09-04T17:48:54.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0619999999999997e-06 2024-09-04T17:48:54.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:54.733 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835326+0000", 2024-09-04T17:48:54.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0390000000000004e-06 2024-09-04T17:48:54.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.734 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:54.734 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.902967+0000", 2024-09-04T17:48:54.734 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067641520999999996 2024-09-04T17:48:54.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.734 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:54.734 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903067+0000", 2024-09-04T17:48:54.734 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9426999999999995e-05 2024-09-04T17:48:54.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.734 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:54.734 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903231+0000", 2024-09-04T17:48:54.734 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016390299999999999 2024-09-04T17:48:54.735 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:54.735 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:54.735 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:54.735 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.735 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.735 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61318 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 450560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:54.735 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835334+0000", 2024-09-04T17:48:54.735 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 100.824012903, 2024-09-04T17:48:54.735 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068091771999999995, 2024-09-04T17:48:54.735 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:54.735 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:54.735 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:54.735 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:54.735 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:54.736 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:54.736 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61318 2024-09-04T17:48:54.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.736 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:54.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.736 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:54.736 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835334+0000", 2024-09-04T17:48:54.736 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:54.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.736 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:54.736 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835334+0000", 2024-09-04T17:48:54.736 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:54.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:54.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835334+0000", 2024-09-04T17:48:54.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5699999999999998e-07 2024-09-04T17:48:54.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:54.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835337+0000", 2024-09-04T17:48:54.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3889999999999999e-06 2024-09-04T17:48:54.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:54.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835343+0000", 2024-09-04T17:48:54.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1809999999999998e-06 2024-09-04T17:48:54.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:54.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835348+0000", 2024-09-04T17:48:54.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3170000000000001e-06 2024-09-04T17:48:54.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:54.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903270+0000", 2024-09-04T17:48:54.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067922191000000007 2024-09-04T17:48:54.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.739 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.739 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:54.739 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903321+0000", 2024-09-04T17:48:54.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0889e-05 2024-09-04T17:48:54.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.739 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.739 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:54.739 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903426+0000", 2024-09-04T17:48:54.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104348 2024-09-04T17:48:54.739 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:54.740 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:54.740 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:54.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.740 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.740 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61319 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 589824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:54.740 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835357+0000", 2024-09-04T17:48:54.740 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 100.82399008599999, 2024-09-04T17:48:54.740 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068372535999999998, 2024-09-04T17:48:54.740 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:54.740 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:54.740 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:54.740 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:54.740 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:54.741 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:54.741 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61319 2024-09-04T17:48:54.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.741 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:54.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:54.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835357+0000", 2024-09-04T17:48:54.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:54.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:54.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835357+0000", 2024-09-04T17:48:54.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:54.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:54.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835357+0000", 2024-09-04T17:48:54.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.32e-07 2024-09-04T17:48:54.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:54.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835359+0000", 2024-09-04T17:48:54.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1660000000000001e-06 2024-09-04T17:48:54.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.743 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:54.743 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835366+0000", 2024-09-04T17:48:54.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3550000000000001e-06 2024-09-04T17:48:54.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.743 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:54.743 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835371+0000", 2024-09-04T17:48:54.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1050000000000003e-06 2024-09-04T17:48:54.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.743 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:54.743 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903471+0000", 2024-09-04T17:48:54.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068100451000000006 2024-09-04T17:48:54.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.744 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:54.744 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903573+0000", 2024-09-04T17:48:54.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000101691 2024-09-04T17:48:54.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.744 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:54.744 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903729+0000", 2024-09-04T17:48:54.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000156336 2024-09-04T17:48:54.744 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:54.744 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:54.744 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:54.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.745 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61320 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:54.745 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835380+0000", 2024-09-04T17:48:54.745 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 100.823966321, 2024-09-04T17:48:54.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068543058000000004, 2024-09-04T17:48:54.745 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:54.745 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:54.745 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:54.745 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:54.745 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:54.745 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:54.745 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61320 2024-09-04T17:48:54.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.746 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:54.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:54.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835380+0000", 2024-09-04T17:48:54.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:54.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:54.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835380+0000", 2024-09-04T17:48:54.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:54.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:54.747 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835381+0000", 2024-09-04T17:48:54.747 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9999999999999998e-07 2024-09-04T17:48:54.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.747 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.747 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:54.747 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835383+0000", 2024-09-04T17:48:54.747 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2230000000000001e-06 2024-09-04T17:48:54.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.747 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.747 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:54.747 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835390+0000", 2024-09-04T17:48:54.747 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.34e-06 2024-09-04T17:48:54.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.748 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.748 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:54.748 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835396+0000", 2024-09-04T17:48:54.748 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9830000000000001e-06 2024-09-04T17:48:54.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.748 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.748 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:54.748 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903765+0000", 2024-09-04T17:48:54.748 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068368441000000002 2024-09-04T17:48:54.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.748 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.748 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:54.748 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903835+0000", 2024-09-04T17:48:54.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0631999999999997e-05 2024-09-04T17:48:54.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.749 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:54.749 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903923+0000", 2024-09-04T17:48:54.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8039000000000002e-05 2024-09-04T17:48:54.749 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:54.749 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:54.749 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:54.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.749 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61321 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 835584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:54.749 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835404+0000", 2024-09-04T17:48:54.750 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 100.823942262, 2024-09-04T17:48:54.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068911043000000005, 2024-09-04T17:48:54.750 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:54.750 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:54.750 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:54.750 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:54.750 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:54.750 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:54.750 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61321 2024-09-04T17:48:54.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.750 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:54.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:54.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835404+0000", 2024-09-04T17:48:54.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:54.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:54.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835404+0000", 2024-09-04T17:48:54.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:54.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:54.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835405+0000", 2024-09-04T17:48:54.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.89e-07 2024-09-04T17:48:54.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.752 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:54.752 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835407+0000", 2024-09-04T17:48:54.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4439999999999998e-06 2024-09-04T17:48:54.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.752 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:54.752 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835414+0000", 2024-09-04T17:48:54.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4930000000000003e-06 2024-09-04T17:48:54.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.752 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:54.753 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835419+0000", 2024-09-04T17:48:54.753 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8919999999999999e-06 2024-09-04T17:48:54.753 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.753 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:54.753 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.903957+0000", 2024-09-04T17:48:54.753 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.068538500000000002 2024-09-04T17:48:54.753 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.753 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:54.753 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904019+0000", 2024-09-04T17:48:54.753 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1419000000000003e-05 2024-09-04T17:48:54.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:54.754 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904315+0000", 2024-09-04T17:48:54.754 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00029680599999999999 2024-09-04T17:48:54.754 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:54.754 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:54.754 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:54.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.754 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4404.0:61322 3.18 3:1999c2d7:::rbd_data.11341dfde0c5.0000000000000008:1f [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:54.754 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-04T17:47:13.835427+0000", 2024-09-04T17:48:54.755 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 100.82391987600001, 2024-09-04T17:48:54.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.069550138999999997, 2024-09-04T17:48:54.755 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-04T17:48:54.755 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-04T17:48:54.755 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-04T17:48:54.755 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-04T17:48:54.755 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4404", 2024-09-04T17:48:54.755 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:54.755 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 61322 2024-09-04T17:48:54.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.755 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-04T17:48:54.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-04T17:48:54.756 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835427+0000", 2024-09-04T17:48:54.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:54.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.756 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-04T17:48:54.756 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.835427+0000", 2024-09-04T17:48:54.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-04T17:48:54.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.756 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-04T17:48:54.756 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.842157+0000", 2024-09-04T17:48:54.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067298419999999998 2024-09-04T17:48:54.757 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.757 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-04T17:48:54.757 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.842163+0000", 2024-09-04T17:48:54.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2029999999999997e-06 2024-09-04T17:48:54.757 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.757 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-04T17:48:54.757 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.842175+0000", 2024-09-04T17:48:54.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2072000000000001e-05 2024-09-04T17:48:54.757 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.757 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-04T17:48:54.758 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.842183+0000", 2024-09-04T17:48:54.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3850000000000002e-06 2024-09-04T17:48:54.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.758 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-04T17:48:54.758 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904371+0000", 2024-09-04T17:48:54.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.062187754999999997 2024-09-04T17:48:54.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.758 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-04T17:48:54.758 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904467+0000", 2024-09-04T17:48:54.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6117000000000001e-05 2024-09-04T17:48:54.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-04T17:48:54.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-04T17:48:54.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-04T17:48:54.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-04T17:47:13.904977+0000", 2024-09-04T17:48:54.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00050976500000000002 2024-09-04T17:48:54.759 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:54.759 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:54.759 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:54.759 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-04T17:48:54.759 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-04T17:48:54.759 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:54.761 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-04T17:48:54.761 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:48:54.754+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:48:54.780 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:48:54.781 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-09-04T17:48:54.799 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":264,"stamp":"2024-09-04T17:48:54.018925+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":177824216,"num_objects":1103,"num_object_clones":40,"num_object_copies":2206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1103,"num_whiteouts":960,"num_read":140902,"num_read_kb":875569,"num_write":20107,"num_write_kb":405833,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4671,"num_bytes_recovered":238943375,"num_keys_recovered":777,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17883,"ondisk_log_size":17883,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":479072,"kb_used_data":159096,"kb_used_omap":45,"kb_used_meta":319442,"kb_avail":660123808,"statfs":{"total":676457349120,"available":675966779392,"internally_reserved":0,"allocated":162914304,"data_stored":186315761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":46351,"internal_metadata":327109361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-165325659,"num_objects":-159,"num_object_clones":-55,"num_object_copies":-318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-159,"num_whiteouts":-40,"num_read":1064,"num_read_kb":909,"num_write":215,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.030164"},"pg_stats":[{"pgid":"3.1f","version":"171'497","reported_seq":3048,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-04T17:48:51.056660+0000","last_change":"2024-09-04T17:48:51.057439+0000","last_active":"2024-09-04T17:48:51.056660+0000","last_peered":"2024-09-04T17:48:51.056660+0000","last_clean":"2024-09-04T17:48:51.056660+0000","last_became_active":"2024-09-04T17:48:09.845199+0000","last_became_peered":"2024-09-04T17:48:09.845199+0000","last_unstale":"2024-09-04T17:48:51.056660+0000","last_undegraded":"2024-09-04T17:48:51.056660+0000","last_fullsized":"2024-09-04T17:48:51.056660+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":497,"log_dups_size":0,"ondisk_log_size":497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T17:49:02.934920+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0034712879999999999,"stat_sum":{"num_bytes":2490384,"num_objects":39,"num_object_clones":1,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":36,"num_read":2980,"num_read_kb":27189,"num_write":522,"num_write_kb":10445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":24433786,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"8","length":"b"}]},{"pgid":"3.1e","version":"169'639","reported_seq":3904,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-04T17:48:52.023850+0000","last_change":"2024-09-04T17:48:52.024108+0000","last_active":"2024-09-04T17:48:52.023850+0000","last_peered":"2024-09-04T17:48:52.023850+0000","last_clean":"2024-09-04T17:48:52.023850+0000","last_became_active":"2024-09-04T17:48:30.050904+0000","last_became_peered":"2024-09-04T17:48:30.050904+0000","last_unstale":"2024-09-04T17:48:52.023850+0000","last_undegraded":"2024-09-04T17:48:52.023850+0000","last_fullsized":"2024-09-04T17:48:52.023850+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":639,"log_dups_size":0,"ondisk_log_size":639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:15:06.068015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017791739999999999,"stat_sum":{"num_bytes":14614509,"num_objects":28,"num_object_clones":2,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":23,"num_read":4726,"num_read_kb":30478,"num_write":799,"num_write_kb":10568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":230,"num_bytes_recovered":15412196,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"7","length":"c"}]},{"pgid":"3.1d","version":"170'541","reported_seq":2469,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-04T17:48:51.029548+0000","last_change":"2024-09-04T17:48:51.029974+0000","last_active":"2024-09-04T17:48:51.029548+0000","last_peered":"2024-09-04T17:48:51.029548+0000","last_clean":"2024-09-04T17:48:51.029548+0000","last_became_active":"2024-09-04T17:45:55.955213+0000","last_became_peered":"2024-09-04T17:45:55.955213+0000","last_unstale":"2024-09-04T17:48:51.029548+0000","last_undegraded":"2024-09-04T17:48:51.029548+0000","last_fullsized":"2024-09-04T17:48:51.029548+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":541,"log_dups_size":0,"ondisk_log_size":541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:18:29.193686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024144710000000001,"stat_sum":{"num_bytes":18178765,"num_objects":33,"num_object_clones":2,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":28,"num_read":2820,"num_read_kb":20284,"num_write":620,"num_write_kb":9401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"b"}]},{"pgid":"3.1c","version":"171'751","reported_seq":6724,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-04T17:48:51.170471+0000","last_change":"2024-09-04T17:48:51.033957+0000","last_active":"2024-09-04T17:48:51.170471+0000","last_peered":"2024-09-04T17:48:51.170471+0000","last_clean":"2024-09-04T17:48:51.170471+0000","last_became_active":"2024-09-04T17:45:55.951880+0000","last_became_peered":"2024-09-04T17:45:55.951880+0000","last_unstale":"2024-09-04T17:48:51.170471+0000","last_undegraded":"2024-09-04T17:48:51.170471+0000","last_fullsized":"2024-09-04T17:48:51.170471+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":751,"log_dups_size":0,"ondisk_log_size":751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T23:55:10.451122+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0062691759999999996,"stat_sum":{"num_bytes":917504,"num_objects":38,"num_object_clones":1,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":36,"num_read":7446,"num_read_kb":47991,"num_write":864,"num_write_kb":11556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"b"}]},{"pgid":"3.1b","version":"170'573","reported_seq":2234,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-04T17:48:51.055423+0000","last_change":"2024-09-04T17:48:51.055423+0000","last_active":"2024-09-04T17:48:51.055423+0000","last_peered":"2024-09-04T17:48:51.055423+0000","last_clean":"2024-09-04T17:48:51.055423+0000","last_became_active":"2024-09-04T17:48:09.853793+0000","last_became_peered":"2024-09-04T17:48:09.853793+0000","last_unstale":"2024-09-04T17:48:51.055423+0000","last_undegraded":"2024-09-04T17:48:51.055423+0000","last_fullsized":"2024-09-04T17:48:51.055423+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":573,"log_dups_size":0,"ondisk_log_size":573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:27:41.733656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001528373,"stat_sum":{"num_bytes":262259,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":33,"num_read":3545,"num_read_kb":8003,"num_write":628,"num_write_kb":4594,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":1993914,"num_keys_recovered":48,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"8","length":"b"}]},{"pgid":"3.1a","version":"168'419","reported_seq":6337,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-09-04T17:48:49.116376+0000","last_change":"2024-09-04T17:48:48.978137+0000","last_active":"2024-09-04T17:48:49.116376+0000","last_peered":"2024-09-04T17:48:49.116376+0000","last_clean":"2024-09-04T17:48:49.116376+0000","last_became_active":"2024-09-04T17:47:44.946366+0000","last_became_peered":"2024-09-04T17:47:44.946366+0000","last_unstale":"2024-09-04T17:48:49.116376+0000","last_undegraded":"2024-09-04T17:48:49.116376+0000","last_fullsized":"2024-09-04T17:48:49.116376+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:05:54.083403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030946229999999999,"stat_sum":{"num_bytes":1411895,"num_objects":31,"num_object_clones":1,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":26,"num_read":5950,"num_read_kb":45812,"num_write":373,"num_write_kb":15447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":5992020,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"a","length":"9"}]},{"pgid":"3.19","version":"171'417","reported_seq":2451,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-04T17:48:52.022528+0000","last_change":"2024-09-04T17:48:52.022873+0000","last_active":"2024-09-04T17:48:52.022528+0000","last_peered":"2024-09-04T17:48:52.022528+0000","last_clean":"2024-09-04T17:48:52.022528+0000","last_became_active":"2024-09-04T17:48:30.052691+0000","last_became_peered":"2024-09-04T17:48:30.052691+0000","last_unstale":"2024-09-04T17:48:52.022528+0000","last_undegraded":"2024-09-04T17:48:52.022528+0000","last_fullsized":"2024-09-04T17:48:52.022528+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":417,"log_dups_size":0,"ondisk_log_size":417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:01:08.252224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092401799999999997,"stat_sum":{"num_bytes":917504,"num_objects":21,"num_object_clones":1,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":19,"num_read":2092,"num_read_kb":16058,"num_write":443,"num_write_kb":4611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":4911830,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"7","length":"c"}]},{"pgid":"3.18","version":"170'505","reported_seq":2273,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-04T17:48:52.022144+0000","last_change":"2024-09-04T17:48:52.022878+0000","last_active":"2024-09-04T17:48:52.022144+0000","last_peered":"2024-09-04T17:48:52.022144+0000","last_clean":"2024-09-04T17:48:52.022144+0000","last_became_active":"2024-09-04T17:48:30.051421+0000","last_became_peered":"2024-09-04T17:48:30.051421+0000","last_unstale":"2024-09-04T17:48:52.022144+0000","last_undegraded":"2024-09-04T17:48:52.022144+0000","last_fullsized":"2024-09-04T17:48:52.022144+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":505,"log_dups_size":0,"ondisk_log_size":505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T21:57:46.314512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001005061,"stat_sum":{"num_bytes":3281039,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":28,"num_read":8892,"num_read_kb":35489,"num_write":440,"num_write_kb":14429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":391,"num_bytes_recovered":28241520,"num_keys_recovered":141,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"7","length":"c"}]},{"pgid":"3.17","version":"170'319","reported_seq":2298,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-04T17:48:51.057914+0000","last_change":"2024-09-04T17:48:51.058565+0000","last_active":"2024-09-04T17:48:51.057914+0000","last_peered":"2024-09-04T17:48:51.057914+0000","last_clean":"2024-09-04T17:48:51.057914+0000","last_became_active":"2024-09-04T17:45:55.958625+0000","last_became_peered":"2024-09-04T17:45:55.958625+0000","last_unstale":"2024-09-04T17:48:51.057914+0000","last_undegraded":"2024-09-04T17:48:51.057914+0000","last_fullsized":"2024-09-04T17:48:51.057914+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:25:43.927202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039964520000000002,"stat_sum":{"num_bytes":14310102,"num_objects":31,"num_object_clones":3,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":22,"num_read":1608,"num_read_kb":20937,"num_write":310,"num_write_kb":18406,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"8","length":"b"}]},{"pgid":"3.16","version":"170'359","reported_seq":1969,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-04T17:48:51.027642+0000","last_change":"2024-09-04T17:48:51.029349+0000","last_active":"2024-09-04T17:48:51.027642+0000","last_peered":"2024-09-04T17:48:51.027642+0000","last_clean":"2024-09-04T17:48:51.027642+0000","last_became_active":"2024-09-04T17:45:55.953254+0000","last_became_peered":"2024-09-04T17:45:55.953254+0000","last_unstale":"2024-09-04T17:48:51.027642+0000","last_undegraded":"2024-09-04T17:48:51.027642+0000","last_fullsized":"2024-09-04T17:48:51.027642+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T18:01:30.670072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001924565,"stat_sum":{"num_bytes":3432819,"num_objects":29,"num_object_clones":2,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":26,"num_read":1729,"num_read_kb":10785,"num_write":395,"num_write_kb":7391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"b"}]},{"pgid":"3.15","version":"168'506","reported_seq":2587,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-04T17:48:50.124297+0000","last_change":"2024-09-04T17:48:50.128895+0000","last_active":"2024-09-04T17:48:50.124297+0000","last_peered":"2024-09-04T17:48:50.124297+0000","last_clean":"2024-09-04T17:48:50.124297+0000","last_became_active":"2024-09-04T17:48:30.053758+0000","last_became_peered":"2024-09-04T17:48:30.053758+0000","last_unstale":"2024-09-04T17:48:50.124297+0000","last_undegraded":"2024-09-04T17:48:50.124297+0000","last_fullsized":"2024-09-04T17:48:50.124297+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T19:59:47.863082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0052204620000000004,"stat_sum":{"num_bytes":6291456,"num_objects":27,"num_object_clones":2,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":25,"num_read":4020,"num_read_kb":16975,"num_write":525,"num_write_kb":4439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":12989343,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.14","version":"167'767","reported_seq":2445,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-04T17:48:50.129543+0000","last_change":"2024-09-04T17:48:50.130470+0000","last_active":"2024-09-04T17:48:50.129543+0000","last_peered":"2024-09-04T17:48:50.129543+0000","last_clean":"2024-09-04T17:48:50.129543+0000","last_became_active":"2024-09-04T17:46:53.539886+0000","last_became_peered":"2024-09-04T17:46:53.539886+0000","last_unstale":"2024-09-04T17:48:50.129543+0000","last_undegraded":"2024-09-04T17:48:50.129543+0000","last_fullsized":"2024-09-04T17:48:50.129543+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":767,"log_dups_size":0,"ondisk_log_size":767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:35:07.260316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039357239999999998,"stat_sum":{"num_bytes":14743423,"num_objects":36,"num_object_clones":2,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":30,"num_read":4006,"num_read_kb":29861,"num_write":997,"num_write_kb":35909,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"2.7","version":"0'0","reported_seq":182,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-04T17:48:51.054058+0000","last_change":"2024-09-04T17:48:11.850550+0000","last_active":"2024-09-04T17:48:51.054058+0000","last_peered":"2024-09-04T17:48:51.054058+0000","last_clean":"2024-09-04T17:48:51.054058+0000","last_became_active":"2024-09-04T17:48:11.849710+0000","last_became_peered":"2024-09-04T17:48:11.849710+0000","last_unstale":"2024-09-04T17:48:51.054058+0000","last_undegraded":"2024-09-04T17:48:51.054058+0000","last_fullsized":"2024-09-04T17:48:51.054058+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:37:18.791459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001226074,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"171'387","reported_seq":3117,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-04T17:48:51.057732+0000","last_change":"2024-09-04T17:48:51.057732+0000","last_active":"2024-09-04T17:48:51.057732+0000","last_peered":"2024-09-04T17:48:51.057732+0000","last_clean":"2024-09-04T17:48:51.057732+0000","last_became_active":"2024-09-04T17:47:32.848310+0000","last_became_peered":"2024-09-04T17:47:32.848310+0000","last_unstale":"2024-09-04T17:48:51.057732+0000","last_undegraded":"2024-09-04T17:48:51.057732+0000","last_fullsized":"2024-09-04T17:48:51.057732+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:15:15.636862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034750010000000001,"stat_sum":{"num_bytes":2024970,"num_objects":31,"num_object_clones":1,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":26,"num_read":2454,"num_read_kb":14526,"num_write":394,"num_write_kb":5524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":919873,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"8","length":"b"}]},{"pgid":"2.6","version":"0'0","reported_seq":117,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-04T17:48:51.054064+0000","last_change":"2024-09-04T17:47:58.862844+0000","last_active":"2024-09-04T17:48:51.054064+0000","last_peered":"2024-09-04T17:48:51.054064+0000","last_clean":"2024-09-04T17:48:51.054064+0000","last_became_active":"2024-09-04T17:47:58.859941+0000","last_became_peered":"2024-09-04T17:47:58.859941+0000","last_unstale":"2024-09-04T17:48:51.054064+0000","last_undegraded":"2024-09-04T17:48:51.054064+0000","last_fullsized":"2024-09-04T17:48:51.054064+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:08:53.409036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011364179999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"171'468","reported_seq":2942,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-04T17:48:52.023583+0000","last_change":"2024-09-04T17:48:52.023786+0000","last_active":"2024-09-04T17:48:52.023583+0000","last_peered":"2024-09-04T17:48:52.023583+0000","last_clean":"2024-09-04T17:48:52.023583+0000","last_became_active":"2024-09-04T17:48:32.053253+0000","last_became_peered":"2024-09-04T17:48:32.053253+0000","last_unstale":"2024-09-04T17:48:52.023583+0000","last_undegraded":"2024-09-04T17:48:52.023583+0000","last_fullsized":"2024-09-04T17:48:52.023583+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:45:34.358654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014545039999999999,"stat_sum":{"num_bytes":297887,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":32,"num_read":2953,"num_read_kb":23389,"num_write":574,"num_write_kb":21569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":1588111,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"7","length":"c"}]},{"pgid":"2.5","version":"0'0","reported_seq":316,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-04T17:48:50.125074+0000","last_change":"2024-09-04T17:45:49.476607+0000","last_active":"2024-09-04T17:48:50.125074+0000","last_peered":"2024-09-04T17:48:50.125074+0000","last_clean":"2024-09-04T17:48:50.125074+0000","last_became_active":"2024-09-04T17:45:47.369308+0000","last_became_peered":"2024-09-04T17:45:47.369308+0000","last_unstale":"2024-09-04T17:48:50.125074+0000","last_undegraded":"2024-09-04T17:48:50.125074+0000","last_fullsized":"2024-09-04T17:48:50.125074+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:17:13.378553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001029826,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"169'380","reported_seq":2671,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-09-04T17:48:49.081869+0000","last_change":"2024-09-04T17:48:48.977622+0000","last_active":"2024-09-04T17:48:49.081869+0000","last_peered":"2024-09-04T17:48:49.081869+0000","last_clean":"2024-09-04T17:48:49.081869+0000","last_became_active":"2024-09-04T17:48:10.845756+0000","last_became_peered":"2024-09-04T17:48:10.845756+0000","last_unstale":"2024-09-04T17:48:49.081869+0000","last_undegraded":"2024-09-04T17:48:49.081869+0000","last_fullsized":"2024-09-04T17:48:49.081869+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":380,"log_dups_size":0,"ondisk_log_size":380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:41:58.975677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029618859999999999,"stat_sum":{"num_bytes":4039247,"num_objects":39,"num_object_clones":2,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":33,"num_read":2434,"num_read_kb":18577,"num_write":416,"num_write_kb":9850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":237,"num_bytes_recovered":13375622,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"a","length":"9"}]},{"pgid":"2.4","version":"0'0","reported_seq":162,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-04T17:48:51.053771+0000","last_change":"2024-09-04T17:47:29.803682+0000","last_active":"2024-09-04T17:48:51.053771+0000","last_peered":"2024-09-04T17:48:51.053771+0000","last_clean":"2024-09-04T17:48:51.053771+0000","last_became_active":"2024-09-04T17:47:29.803256+0000","last_became_peered":"2024-09-04T17:47:29.803256+0000","last_unstale":"2024-09-04T17:48:51.053771+0000","last_undegraded":"2024-09-04T17:48:51.053771+0000","last_fullsized":"2024-09-04T17:48:51.053771+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:32:26.523162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"170'353","reported_seq":2530,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-04T17:48:51.030126+0000","last_change":"2024-09-04T17:48:51.031451+0000","last_active":"2024-09-04T17:48:51.030126+0000","last_peered":"2024-09-04T17:48:51.030126+0000","last_clean":"2024-09-04T17:48:51.030126+0000","last_became_active":"2024-09-04T17:48:30.051194+0000","last_became_peered":"2024-09-04T17:48:30.051194+0000","last_unstale":"2024-09-04T17:48:51.030126+0000","last_undegraded":"2024-09-04T17:48:51.030126+0000","last_fullsized":"2024-09-04T17:48:51.030126+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:03:10.995220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038200370000000001,"stat_sum":{"num_bytes":4785515,"num_objects":31,"num_object_clones":1,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":27,"num_read":2552,"num_read_kb":7638,"num_write":302,"num_write_kb":7011,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":15596571,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"b"}]},{"pgid":"3.3","version":"171'865","reported_seq":3446,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-04T17:48:51.155138+0000","last_change":"2024-09-04T17:48:51.058156+0000","last_active":"2024-09-04T17:48:51.155138+0000","last_peered":"2024-09-04T17:48:51.155138+0000","last_clean":"2024-09-04T17:48:51.155138+0000","last_became_active":"2024-09-04T17:46:53.542889+0000","last_became_peered":"2024-09-04T17:46:53.542889+0000","last_unstale":"2024-09-04T17:48:51.155138+0000","last_undegraded":"2024-09-04T17:48:51.155138+0000","last_fullsized":"2024-09-04T17:48:51.155138+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T19:24:58.820902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038337470000000002,"stat_sum":{"num_bytes":6660903,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":29,"num_read":4643,"num_read_kb":26531,"num_write":1078,"num_write_kb":14207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"8","length":"b"}]},{"pgid":"2.2","version":"22'2","reported_seq":335,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-04T17:48:51.026966+0000","last_change":"2024-09-04T17:47:59.598241+0000","last_active":"2024-09-04T17:48:51.026966+0000","last_peered":"2024-09-04T17:48:51.026966+0000","last_clean":"2024-09-04T17:48:51.026966+0000","last_became_active":"2024-09-04T17:47:58.884634+0000","last_became_peered":"2024-09-04T17:47:58.884634+0000","last_unstale":"2024-09-04T17:48:51.026966+0000","last_undegraded":"2024-09-04T17:48:51.026966+0000","last_fullsized":"2024-09-04T17:48:51.026966+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T22:02:31.692385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026316069999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"168'449","reported_seq":3340,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-09-04T17:48:48.974647+0000","last_change":"2024-09-04T17:48:48.975209+0000","last_active":"2024-09-04T17:48:48.974647+0000","last_peered":"2024-09-04T17:48:48.974647+0000","last_clean":"2024-09-04T17:48:48.974647+0000","last_became_active":"2024-09-04T17:48:01.849363+0000","last_became_peered":"2024-09-04T17:48:01.849363+0000","last_unstale":"2024-09-04T17:48:48.974647+0000","last_undegraded":"2024-09-04T17:48:48.974647+0000","last_fullsized":"2024-09-04T17:48:48.974647+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T21:53:49.070677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089001200000000003,"stat_sum":{"num_bytes":531127,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":36,"num_read":3563,"num_read_kb":22520,"num_write":361,"num_write_kb":4893,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":5692590,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"a","length":"9"}]},{"pgid":"2.1","version":"0'0","reported_seq":300,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-09-04T17:48:48.974299+0000","last_change":"2024-09-04T17:48:27.034831+0000","last_active":"2024-09-04T17:48:48.974299+0000","last_peered":"2024-09-04T17:48:48.974299+0000","last_clean":"2024-09-04T17:48:48.974299+0000","last_became_active":"2024-09-04T17:48:27.034383+0000","last_became_peered":"2024-09-04T17:48:27.034383+0000","last_unstale":"2024-09-04T17:48:48.974299+0000","last_undegraded":"2024-09-04T17:48:48.974299+0000","last_fullsized":"2024-09-04T17:48:48.974299+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T23:15:59.538283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001025195,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"171'619","reported_seq":3274,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-04T17:48:51.061337+0000","last_change":"2024-09-04T17:48:51.061706+0000","last_active":"2024-09-04T17:48:51.061337+0000","last_peered":"2024-09-04T17:48:51.061337+0000","last_clean":"2024-09-04T17:48:51.061337+0000","last_became_active":"2024-09-04T17:48:11.845510+0000","last_became_peered":"2024-09-04T17:48:11.845510+0000","last_unstale":"2024-09-04T17:48:51.061337+0000","last_undegraded":"2024-09-04T17:48:51.061337+0000","last_fullsized":"2024-09-04T17:48:51.061337+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":619,"log_dups_size":0,"ondisk_log_size":619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:08:34.472021+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0063845020000000002,"stat_sum":{"num_bytes":4643589,"num_objects":34,"num_object_clones":1,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":28,"num_read":3285,"num_read_kb":40522,"num_write":655,"num_write_kb":23355,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":240,"num_bytes_recovered":21297737,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"8","length":"b"}]},{"pgid":"2.0","version":"0'0","reported_seq":322,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-04T17:48:50.122800+0000","last_change":"2024-09-04T17:47:29.804429+0000","last_active":"2024-09-04T17:48:50.122800+0000","last_peered":"2024-09-04T17:48:50.122800+0000","last_clean":"2024-09-04T17:48:50.122800+0000","last_became_active":"2024-09-04T17:47:29.803992+0000","last_became_peered":"2024-09-04T17:47:29.803992+0000","last_unstale":"2024-09-04T17:48:50.122800+0000","last_undegraded":"2024-09-04T17:48:50.122800+0000","last_fullsized":"2024-09-04T17:48:50.122800+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:50:32.563345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010553590000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"172'620","reported_seq":2781,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-04T17:48:52.027487+0000","last_change":"2024-09-04T17:48:52.027880+0000","last_active":"2024-09-04T17:48:52.027487+0000","last_peered":"2024-09-04T17:48:52.027487+0000","last_clean":"2024-09-04T17:48:52.027487+0000","last_became_active":"2024-09-04T17:48:31.052384+0000","last_became_peered":"2024-09-04T17:48:31.052384+0000","last_unstale":"2024-09-04T17:48:52.027487+0000","last_undegraded":"2024-09-04T17:48:52.027487+0000","last_fullsized":"2024-09-04T17:48:52.027487+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":620,"log_dups_size":0,"ondisk_log_size":620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T18:35:30.514410+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0055418890000000004,"stat_sum":{"num_bytes":293590,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":28,"num_read":5711,"num_read_kb":30159,"num_write":738,"num_write_kb":13818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":16173004,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"7","length":"c"}]},{"pgid":"2.3","version":"20'1","reported_seq":329,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-04T17:48:51.026927+0000","last_change":"2024-09-04T17:47:41.919544+0000","last_active":"2024-09-04T17:48:51.026927+0000","last_peered":"2024-09-04T17:48:51.026927+0000","last_clean":"2024-09-04T17:48:51.026927+0000","last_became_active":"2024-09-04T17:47:41.919027+0000","last_became_peered":"2024-09-04T17:47:41.919027+0000","last_unstale":"2024-09-04T17:48:51.026927+0000","last_undegraded":"2024-09-04T17:48:51.026927+0000","last_fullsized":"2024-09-04T17:48:51.026927+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:42:55.688562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086618299999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":407,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-04T17:48:50.122912+0000","last_change":"2024-09-04T17:45:42.341149+0000","last_active":"2024-09-04T17:48:50.122912+0000","last_peered":"2024-09-04T17:48:50.122912+0000","last_clean":"2024-09-04T17:48:50.122912+0000","last_became_active":"2024-09-04T17:45:42.339286+0000","last_became_peered":"2024-09-04T17:45:42.339286+0000","last_unstale":"2024-09-04T17:48:50.122912+0000","last_undegraded":"2024-09-04T17:48:50.122912+0000","last_fullsized":"2024-09-04T17:48:50.122912+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:41.305128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:41.305128+0000","last_clean_scrub_stamp":"2024-09-04T17:45:41.305128+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:52:24.860049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"172'544","reported_seq":2999,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-04T17:48:52.023503+0000","last_change":"2024-09-04T17:48:52.023503+0000","last_active":"2024-09-04T17:48:52.023503+0000","last_peered":"2024-09-04T17:48:52.023503+0000","last_clean":"2024-09-04T17:48:52.023503+0000","last_became_active":"2024-09-04T17:48:31.051869+0000","last_became_peered":"2024-09-04T17:48:31.051869+0000","last_unstale":"2024-09-04T17:48:52.023503+0000","last_undegraded":"2024-09-04T17:48:52.023503+0000","last_fullsized":"2024-09-04T17:48:52.023503+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T22:01:34.207704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014354140000000001,"stat_sum":{"num_bytes":3359411,"num_objects":34,"num_object_clones":1,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":31,"num_read":2788,"num_read_kb":17264,"num_write":575,"num_write_kb":29373,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":319,"num_bytes_recovered":17940277,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"7","length":"c"}]},{"pgid":"3.9","version":"168'672","reported_seq":4533,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-09-04T17:48:48.974926+0000","last_change":"2024-09-04T17:48:48.975831+0000","last_active":"2024-09-04T17:48:48.974926+0000","last_peered":"2024-09-04T17:48:48.974926+0000","last_clean":"2024-09-04T17:48:48.974926+0000","last_became_active":"2024-09-04T17:47:44.949630+0000","last_became_peered":"2024-09-04T17:47:44.949630+0000","last_unstale":"2024-09-04T17:48:48.974926+0000","last_undegraded":"2024-09-04T17:48:48.974926+0000","last_fullsized":"2024-09-04T17:48:48.974926+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":672,"log_dups_size":0,"ondisk_log_size":672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T21:58:53.185242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00131708,"stat_sum":{"num_bytes":8421425,"num_objects":44,"num_object_clones":4,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":36,"num_read":5999,"num_read_kb":37328,"num_write":739,"num_write_kb":18642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":4534479,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"a","length":"9"}]},{"pgid":"3.a","version":"170'640","reported_seq":3547,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-04T17:48:51.054571+0000","last_change":"2024-09-04T17:48:51.055000+0000","last_active":"2024-09-04T17:48:51.054571+0000","last_peered":"2024-09-04T17:48:51.054571+0000","last_clean":"2024-09-04T17:48:51.054571+0000","last_became_active":"2024-09-04T17:48:00.841070+0000","last_became_peered":"2024-09-04T17:48:00.841070+0000","last_unstale":"2024-09-04T17:48:51.054571+0000","last_undegraded":"2024-09-04T17:48:51.054571+0000","last_fullsized":"2024-09-04T17:48:51.054571+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":640,"log_dups_size":0,"ondisk_log_size":640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T19:35:01.271446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077206699999999998,"stat_sum":{"num_bytes":641130,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":31,"num_read":5676,"num_read_kb":16456,"num_write":602,"num_write_kb":2753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":1572433,"num_keys_recovered":66,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"8","length":"b"}]},{"pgid":"3.b","version":"170'741","reported_seq":2558,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-04T17:48:52.023143+0000","last_change":"2024-09-04T17:48:52.023882+0000","last_active":"2024-09-04T17:48:52.023143+0000","last_peered":"2024-09-04T17:48:52.023143+0000","last_clean":"2024-09-04T17:48:52.023143+0000","last_became_active":"2024-09-04T17:48:31.049842+0000","last_became_peered":"2024-09-04T17:48:31.049842+0000","last_unstale":"2024-09-04T17:48:52.023143+0000","last_undegraded":"2024-09-04T17:48:52.023143+0000","last_fullsized":"2024-09-04T17:48:52.023143+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":741,"log_dups_size":0,"ondisk_log_size":741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:28:07.850605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018877830000000001,"stat_sum":{"num_bytes":4832707,"num_objects":33,"num_object_clones":1,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":28,"num_read":4059,"num_read_kb":14632,"num_write":895,"num_write_kb":7921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":5211501,"num_keys_recovered":33,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"7","length":"c"}]},{"pgid":"3.c","version":"171'640","reported_seq":8550,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-04T17:48:51.030409+0000","last_change":"2024-09-04T17:48:51.031474+0000","last_active":"2024-09-04T17:48:51.030409+0000","last_peered":"2024-09-04T17:48:51.030409+0000","last_clean":"2024-09-04T17:48:51.030409+0000","last_became_active":"2024-09-04T17:48:32.054696+0000","last_became_peered":"2024-09-04T17:48:32.054696+0000","last_unstale":"2024-09-04T17:48:51.030409+0000","last_undegraded":"2024-09-04T17:48:51.030409+0000","last_fullsized":"2024-09-04T17:48:51.030409+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":640,"log_dups_size":0,"ondisk_log_size":640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T21:00:26.563001+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0039887289999999999,"stat_sum":{"num_bytes":8460475,"num_objects":41,"num_object_clones":2,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":36,"num_read":9052,"num_read_kb":71439,"num_write":751,"num_write_kb":16863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":281,"num_bytes_recovered":8281724,"num_keys_recovered":57,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"b"}]},{"pgid":"3.d","version":"169'1351","reported_seq":4494,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-04T17:48:50.129047+0000","last_change":"2024-09-04T17:48:50.129396+0000","last_active":"2024-09-04T17:48:50.129047+0000","last_peered":"2024-09-04T17:48:50.129047+0000","last_clean":"2024-09-04T17:48:50.129047+0000","last_became_active":"2024-09-04T17:48:12.850381+0000","last_became_peered":"2024-09-04T17:48:12.850381+0000","last_unstale":"2024-09-04T17:48:50.129047+0000","last_undegraded":"2024-09-04T17:48:50.129047+0000","last_fullsized":"2024-09-04T17:48:50.129047+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":1351,"log_dups_size":0,"ondisk_log_size":1351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T18:09:18.281356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049047689999999998,"stat_sum":{"num_bytes":12260025,"num_objects":37,"num_object_clones":2,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":32,"num_read":8863,"num_read_kb":26532,"num_write":2251,"num_write_kb":13198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":179,"num_bytes_recovered":18877339,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.e","version":"170'538","reported_seq":5989,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-04T17:48:50.125352+0000","last_change":"2024-09-04T17:48:50.125352+0000","last_active":"2024-09-04T17:48:50.125352+0000","last_peered":"2024-09-04T17:48:50.125352+0000","last_clean":"2024-09-04T17:48:50.125352+0000","last_became_active":"2024-09-04T17:48:11.849605+0000","last_became_peered":"2024-09-04T17:48:11.849605+0000","last_unstale":"2024-09-04T17:48:50.125352+0000","last_undegraded":"2024-09-04T17:48:50.125352+0000","last_fullsized":"2024-09-04T17:48:50.125352+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":538,"log_dups_size":0,"ondisk_log_size":538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T19:08:13.023058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015976580000000001,"stat_sum":{"num_bytes":5723405,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":23,"num_read":6678,"num_read_kb":28511,"num_write":520,"num_write_kb":11228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":7806862,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.f","version":"168'336","reported_seq":3136,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-04T17:48:50.129270+0000","last_change":"2024-09-04T17:48:50.129954+0000","last_active":"2024-09-04T17:48:50.129270+0000","last_peered":"2024-09-04T17:48:50.129270+0000","last_clean":"2024-09-04T17:48:50.129270+0000","last_became_active":"2024-09-04T17:45:55.956734+0000","last_became_peered":"2024-09-04T17:45:55.956734+0000","last_unstale":"2024-09-04T17:48:50.129270+0000","last_undegraded":"2024-09-04T17:48:50.129270+0000","last_fullsized":"2024-09-04T17:48:50.129270+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":336,"log_dups_size":0,"ondisk_log_size":336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:27:46.715773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048776569999999997,"stat_sum":{"num_bytes":17021332,"num_objects":36,"num_object_clones":3,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":27,"num_read":2605,"num_read_kb":44969,"num_write":324,"num_write_kb":13427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.10","version":"170'603","reported_seq":3673,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-04T17:48:51.055439+0000","last_change":"2024-09-04T17:48:51.055990+0000","last_active":"2024-09-04T17:48:51.055439+0000","last_peered":"2024-09-04T17:48:51.055439+0000","last_clean":"2024-09-04T17:48:51.055439+0000","last_became_active":"2024-09-04T17:48:00.843136+0000","last_became_peered":"2024-09-04T17:48:00.843136+0000","last_unstale":"2024-09-04T17:48:51.055439+0000","last_undegraded":"2024-09-04T17:48:51.055439+0000","last_fullsized":"2024-09-04T17:48:51.055439+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T23:38:14.448304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011819700000000001,"stat_sum":{"num_bytes":3184544,"num_objects":43,"num_object_clones":2,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":39,"num_read":4584,"num_read_kb":18785,"num_write":655,"num_write_kb":12848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":3377599,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"8","length":"b"}]},{"pgid":"3.11","version":"169'581","reported_seq":6234,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-04T17:48:50.123903+0000","last_change":"2024-09-04T17:48:50.124316+0000","last_active":"2024-09-04T17:48:50.123903+0000","last_peered":"2024-09-04T17:48:50.123903+0000","last_clean":"2024-09-04T17:48:50.123903+0000","last_became_active":"2024-09-04T17:45:55.951870+0000","last_became_peered":"2024-09-04T17:45:55.951870+0000","last_unstale":"2024-09-04T17:48:50.123903+0000","last_undegraded":"2024-09-04T17:48:50.123903+0000","last_fullsized":"2024-09-04T17:48:50.123903+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":581,"log_dups_size":0,"ondisk_log_size":581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T21:31:17.027357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010857200000000001,"stat_sum":{"num_bytes":2228378,"num_objects":45,"num_object_clones":2,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":40,"num_read":7130,"num_read_kb":61414,"num_write":569,"num_write_kb":17572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.12","version":"171'421","reported_seq":4140,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-04T17:48:51.057325+0000","last_change":"2024-09-04T17:48:51.057828+0000","last_active":"2024-09-04T17:48:51.057325+0000","last_peered":"2024-09-04T17:48:51.057325+0000","last_clean":"2024-09-04T17:48:51.057325+0000","last_became_active":"2024-09-04T17:48:14.943778+0000","last_became_peered":"2024-09-04T17:48:14.943778+0000","last_unstale":"2024-09-04T17:48:51.057325+0000","last_undegraded":"2024-09-04T17:48:51.057325+0000","last_fullsized":"2024-09-04T17:48:51.057325+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:06:04.625801+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0037784440000000002,"stat_sum":{"num_bytes":1336414,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":31,"num_read":3573,"num_read_kb":21717,"num_write":443,"num_write_kb":8575,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":290,"num_bytes_recovered":2723025,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"8","length":"b"}]},{"pgid":"3.13","version":"168'307","reported_seq":2603,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-04T17:48:50.125561+0000","last_change":"2024-09-04T17:48:50.128944+0000","last_active":"2024-09-04T17:48:50.125561+0000","last_peered":"2024-09-04T17:48:50.125561+0000","last_clean":"2024-09-04T17:48:50.125561+0000","last_became_active":"2024-09-04T17:45:55.954544+0000","last_became_peered":"2024-09-04T17:45:55.954544+0000","last_unstale":"2024-09-04T17:48:50.125561+0000","last_undegraded":"2024-09-04T17:48:50.125561+0000","last_fullsized":"2024-09-04T17:48:50.125561+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":307,"log_dups_size":0,"ondisk_log_size":307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:57:58.642719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045456990000000003,"stat_sum":{"num_bytes":5636096,"num_objects":40,"num_object_clones":1,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":35,"num_read":2420,"num_read_kb":22743,"num_write":246,"num_write_kb":4380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"9","length":"a"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":177233829,"num_objects":1099,"num_object_clones":40,"num_object_copies":2198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1099,"num_whiteouts":960,"num_read":140836,"num_read_kb":875514,"num_write":20004,"num_write_kb":404203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4670,"num_bytes_recovered":238943356,"num_keys_recovered":777,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":144166912,"data_stored":172911879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21395,"internal_metadata":0},"log_size":17808,"ondisk_log_size":17808,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":117,"seq":502511173644,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60408,"kb_used_data":22500,"kb_used_omap":22,"kb_used_meta":37865,"kb_avail":94311432,"statfs":{"total":96636764160,"available":96574906368,"internally_reserved":0,"allocated":23040000,"data_stored":21949565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22742,"internal_metadata":38774570},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509481,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81304,"kb_used_data":32064,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94290536,"statfs":{"total":96636764160,"available":96553508864,"internally_reserved":0,"allocated":32833536,"data_stored":39997901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509482,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79832,"kb_used_data":30600,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94292008,"statfs":{"total":96636764160,"available":96555016192,"internally_reserved":0,"allocated":31334400,"data_stored":39075236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":48,"seq":206158430238,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45252,"kb_used_data":13104,"kb_used_omap":4,"kb_used_meta":32123,"kb_avail":94326588,"statfs":{"total":96636764160,"available":96590426112,"internally_reserved":0,"allocated":13418496,"data_stored":14341975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4430,"internal_metadata":32894642},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":0,"seq":64424509481,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509481,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72668,"kb_used_data":19008,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94299172,"statfs":{"total":96636764160,"available":96562352128,"internally_reserved":0,"allocated":19464192,"data_stored":21407686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542185,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":89476,"kb_used_data":31840,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94282364,"statfs":{"total":96636764160,"available":96545140736,"internally_reserved":0,"allocated":32604160,"data_stored":38842952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":103,"seq":442381631503,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50132,"kb_used_data":9980,"kb_used_omap":12,"kb_used_meta":40115,"kb_avail":94321708,"statfs":{"total":96636764160,"available":96585428992,"internally_reserved":0,"allocated":10219520,"data_stored":10700446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12804,"internal_metadata":41078268},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16711680,"data_stored":19720305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2035712,"data_stored":2023494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192000,"data_stored":9718476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6121,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":7086080,"data_stored":7009490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":29192192,"data_stored":38005652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":30531584,"data_stored":38760181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20996096,"data_stored":20961341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15274,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":29421568,"data_stored":36712940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-04T17:48:54.801 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-04T17:48:54.802 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-04T17:48:54.802 INFO:tasks.thrashosds.thrasher:Removing osd 3, in_osds are: [0, 2, 4, 5, 6, 7, 3] 2024-09-04T17:48:54.802 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 3 2024-09-04T17:48:54.861 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:54.861+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:54.885 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-04T17:48:54.894 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:54.895 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:48:54.895 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:48:54.895 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:54.909 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-04T17:48:54.997 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:54.997+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:55.063 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:55.058+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:55.082 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:55.082 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:48:55.082 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:48:55.082 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:48:55.083 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:55.098 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-04T17:48:55.164 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:55.158+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:55.199 INFO:tasks.ceph.mon.a.smithi092.stderr:2024-09-04T17:48:55.194+0000 7f8f2f46f640 -1 mon.a@0(leader).osd e175 definitely_dead 0 2024-09-04T17:48:55.264 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:48:55.261+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:48:55.266 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:55.266 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:48:55.266 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:48:55.266 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:48:55.266 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.267 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:12722 3.1b 3:d8152af1:::rbd_data.1134ef3a5a7d.00000000000000bd:head [delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-04T17:48:55.267 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:16.421965+0000", 2024-09-04T17:48:55.267 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 158.842599186, 2024-09-04T17:48:55.267 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.890002811, 2024-09-04T17:48:55.267 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:55.267 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:55.267 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:55.267 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:55.267 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:55.267 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:55.267 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 12722 2024-09-04T17:48:55.267 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.268 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:55.268 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.268 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:55.268 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421965+0000", 2024-09-04T17:48:55.268 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.268 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.268 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.268 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:55.268 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421965+0000", 2024-09-04T17:48:55.268 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.268 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.268 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.269 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:55.269 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421966+0000", 2024-09-04T17:48:55.269 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.102e-06 2024-09-04T17:48:55.269 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.269 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.269 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:55.269 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421971+0000", 2024-09-04T17:48:55.269 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9989999999999999e-06 2024-09-04T17:48:55.269 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.269 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.269 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:55.269 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421981+0000", 2024-09-04T17:48:55.269 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.7380000000000006e-06 2024-09-04T17:48:55.270 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.270 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.270 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:55.270 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.421989+0000", 2024-09-04T17:48:55.270 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.1890000000000007e-06 2024-09-04T17:48:55.270 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.270 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.270 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:55.270 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.422050+0000", 2024-09-04T17:48:55.270 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.0702000000000003e-05 2024-09-04T17:48:55.270 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.270 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.270 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for peered", 2024-09-04T17:48:55.270 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:16.422059+0000", 2024-09-04T17:48:55.271 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5440000000000004e-06 2024-09-04T17:48:55.271 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.271 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.271 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:55.271 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.308156+0000", 2024-09-04T17:48:55.271 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.88609676199999998 2024-09-04T17:48:55.271 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.271 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.271 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:55.271 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.308398+0000", 2024-09-04T17:48:55.271 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024215400000000001 2024-09-04T17:48:55.271 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.272 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.272 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for subops from 7", 2024-09-04T17:48:55.272 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.308795+0000", 2024-09-04T17:48:55.272 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000396704 2024-09-04T17:48:55.272 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.272 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.272 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "op_commit", 2024-09-04T17:48:55.272 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.310831+0000", 2024-09-04T17:48:55.272 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0020364200000000002 2024-09-04T17:48:55.272 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.272 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.272 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "sub_op_commit_rec", 2024-09-04T17:48:55.272 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.311818+0000", 2024-09-04T17:48:55.273 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00098649799999999993 2024-09-04T17:48:55.273 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.273 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.273 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:55.273 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.311905+0000", 2024-09-04T17:48:55.273 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7070999999999994e-05 2024-09-04T17:48:55.273 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.273 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.273 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:55.273 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:17.311968+0000", 2024-09-04T17:48:55.273 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.2928000000000005e-05 2024-09-04T17:48:55.273 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.273 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:55.274 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.274 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.274 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.274 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37359 3.f e58/24 3:f29e50a5:::rbd_data.1134c5a43465.0000000000000021:head v 58'135, mlcod=58'133)", 2024-09-04T17:48:55.274 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:48:55.274 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 135.64726858200001, 2024-09-04T17:48:55.274 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053483197000000003, 2024-09-04T17:48:55.274 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:55.274 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:55.274 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:55.274 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:55.274 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.274 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:55.274 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:48:55.275 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.275 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.275 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.275 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:55.275 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:48:55.275 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.275 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.275 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.275 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:55.275 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617296+0000", 2024-09-04T17:48:55.275 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1699999999999997e-07 2024-09-04T17:48:55.275 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.275 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.276 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:55.276 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617338+0000", 2024-09-04T17:48:55.276 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.1168999999999998e-05 2024-09-04T17:48:55.276 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.276 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.276 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:55.276 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617351+0000", 2024-09-04T17:48:55.276 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3094e-05 2024-09-04T17:48:55.276 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.276 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.276 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:55.276 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617359+0000", 2024-09-04T17:48:55.276 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7970000000000006e-06 2024-09-04T17:48:55.277 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.277 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.277 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:55.277 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617417+0000", 2024-09-04T17:48:55.277 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7079e-05 2024-09-04T17:48:55.277 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.277 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.277 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:55.277 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617438+0000", 2024-09-04T17:48:55.277 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1654999999999998e-05 2024-09-04T17:48:55.277 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.277 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.277 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:55.278 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670738+0000", 2024-09-04T17:48:55.278 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053300142000000002 2024-09-04T17:48:55.278 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.278 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.278 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:55.278 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670779+0000", 2024-09-04T17:48:55.278 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0343999999999999e-05 2024-09-04T17:48:55.278 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.278 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:55.278 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.278 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.278 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.278 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37358 3.1d e58/24 3:bd3215f3:::rbd_data.1134c5a43465.0000000000000020:head v 58'91, mlcod=58'89)", 2024-09-04T17:48:55.279 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617502+0000", 2024-09-04T17:48:55.279 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 135.64706186399999, 2024-09-04T17:48:55.279 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053505641999999999, 2024-09-04T17:48:55.279 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:55.279 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:55.279 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:55.279 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:55.279 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.279 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:55.279 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617502+0000", 2024-09-04T17:48:55.279 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.279 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.279 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.279 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:55.280 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617502+0000", 2024-09-04T17:48:55.280 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.280 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.280 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.280 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:55.280 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617503+0000", 2024-09-04T17:48:55.280 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.184e-06 2024-09-04T17:48:55.280 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.280 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.280 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:55.280 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617526+0000", 2024-09-04T17:48:55.280 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2099999999999998e-05 2024-09-04T17:48:55.280 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.281 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.281 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:55.281 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617540+0000", 2024-09-04T17:48:55.281 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4741e-05 2024-09-04T17:48:55.281 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.281 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.281 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:55.281 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617550+0000", 2024-09-04T17:48:55.281 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4259999999999992e-06 2024-09-04T17:48:55.281 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.281 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.281 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:55.282 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617613+0000", 2024-09-04T17:48:55.282 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.3625000000000002e-05 2024-09-04T17:48:55.282 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.282 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.282 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:55.282 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617637+0000", 2024-09-04T17:48:55.282 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.3604999999999998e-05 2024-09-04T17:48:55.282 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.282 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.282 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:55.282 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670967+0000", 2024-09-04T17:48:55.282 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053329881000000003 2024-09-04T17:48:55.283 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.283 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.283 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:55.283 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671008+0000", 2024-09-04T17:48:55.283 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.108e-05 2024-09-04T17:48:55.283 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.283 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:55.283 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.283 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.283 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.283 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37361 3.1c e58/24 3:3c7f8976:::rbd_data.1134c5a43465.0000000000000023:head v 58'159, mlcod=58'157)", 2024-09-04T17:48:55.283 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617578+0000", 2024-09-04T17:48:55.283 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 135.64698601399999, 2024-09-04T17:48:55.284 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053990367999999997, 2024-09-04T17:48:55.284 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:55.284 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:55.284 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:55.284 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:55.284 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.284 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:55.284 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617578+0000", 2024-09-04T17:48:55.284 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.284 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.284 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.284 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:55.284 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617578+0000", 2024-09-04T17:48:55.285 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.285 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.285 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.285 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:55.285 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617579+0000", 2024-09-04T17:48:55.285 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7299999999999996e-07 2024-09-04T17:48:55.285 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.285 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.285 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:55.285 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617610+0000", 2024-09-04T17:48:55.285 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0951000000000002e-05 2024-09-04T17:48:55.285 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.286 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.286 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:55.286 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:48:55.286 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0002e-05 2024-09-04T17:48:55.286 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.286 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.286 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:55.286 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617626+0000", 2024-09-04T17:48:55.286 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6660000000000002e-06 2024-09-04T17:48:55.286 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.286 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.286 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:55.286 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670548+0000", 2024-09-04T17:48:55.286 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.052921943999999999 2024-09-04T17:48:55.287 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.287 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.287 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:55.287 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.670574+0000", 2024-09-04T17:48:55.287 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5405000000000002e-05 2024-09-04T17:48:55.287 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.287 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.287 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:55.287 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671522+0000", 2024-09-04T17:48:55.287 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00094815299999999997 2024-09-04T17:48:55.287 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.287 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.287 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:55.288 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671568+0000", 2024-09-04T17:48:55.288 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6674000000000001e-05 2024-09-04T17:48:55.288 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.288 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:55.288 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.288 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.288 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.288 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_repop(client.4404.0:37357 3.15 e58/49 3:a897312a:::rbd_data.1134c5a43465.000000000000001f:head v 58'117, mlcod=58'115)", 2024-09-04T17:48:55.288 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:48:55.288 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 135.64694367600001, 2024-09-04T17:48:55.288 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053734297, 2024-09-04T17:48:55.288 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:55.289 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:55.289 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-04T17:48:55.289 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:55.289 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.289 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:55.289 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:48:55.289 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.289 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.289 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.289 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:55.289 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617620+0000", 2024-09-04T17:48:55.289 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.289 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.290 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.290 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:55.290 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617621+0000", 2024-09-04T17:48:55.290 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.7300000000000004e-07 2024-09-04T17:48:55.290 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.290 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.290 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:55.290 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617642+0000", 2024-09-04T17:48:55.290 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0185000000000001e-05 2024-09-04T17:48:55.290 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.290 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.290 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:55.290 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617654+0000", 2024-09-04T17:48:55.290 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2745000000000001e-05 2024-09-04T17:48:55.291 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.291 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.291 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:55.291 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617662+0000", 2024-09-04T17:48:55.291 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.0499999999999992e-06 2024-09-04T17:48:55.291 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.291 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.291 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:55.291 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617727+0000", 2024-09-04T17:48:55.291 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.4923000000000002e-05 2024-09-04T17:48:55.291 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.291 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.292 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:55.292 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.617753+0000", 2024-09-04T17:48:55.292 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5899999999999999e-05 2024-09-04T17:48:55.292 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.292 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.292 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "commit_sent", 2024-09-04T17:48:55.292 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671298+0000", 2024-09-04T17:48:55.292 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.053545276000000003 2024-09-04T17:48:55.292 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.292 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.293 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:55.293 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:39.671355+0000", 2024-09-04T17:48:55.293 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6245000000000001e-05 2024-09-04T17:48:55.293 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.293 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:55.293 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.293 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.293 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.293 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44687 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:55.293 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253012+0000", 2024-09-04T17:48:55.294 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 125.01155160899999, 2024-09-04T17:48:55.294 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051128777, 2024-09-04T17:48:55.294 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:55.294 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:55.294 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:55.294 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:55.294 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:55.294 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:55.294 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44687 2024-09-04T17:48:55.294 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.294 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:55.294 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.294 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:55.294 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253012+0000", 2024-09-04T17:48:55.295 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.295 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.295 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.295 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:55.295 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253012+0000", 2024-09-04T17:48:55.295 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.295 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.295 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.295 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:55.295 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253014+0000", 2024-09-04T17:48:55.295 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.24e-06 2024-09-04T17:48:55.295 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.295 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.295 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:55.296 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253019+0000", 2024-09-04T17:48:55.296 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4569999999999996e-06 2024-09-04T17:48:55.296 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.296 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.296 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:55.296 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253035+0000", 2024-09-04T17:48:55.296 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6291999999999999e-05 2024-09-04T17:48:55.296 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.296 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.296 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:55.296 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253050+0000", 2024-09-04T17:48:55.296 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4423000000000001e-05 2024-09-04T17:48:55.296 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.297 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.297 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:55.297 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304115+0000", 2024-09-04T17:48:55.297 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051064778999999998 2024-09-04T17:48:55.297 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.297 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.297 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:55.297 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304141+0000", 2024-09-04T17:48:55.297 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6585999999999999e-05 2024-09-04T17:48:55.297 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.297 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:55.297 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.297 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.298 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.298 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44688 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:55.298 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253067+0000", 2024-09-04T17:48:55.298 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 125.011497057, 2024-09-04T17:48:55.298 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051116979, 2024-09-04T17:48:55.298 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:55.298 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:55.298 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:55.298 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:55.298 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:55.298 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:55.298 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44688 2024-09-04T17:48:55.298 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.298 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:55.299 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.299 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:55.299 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253067+0000", 2024-09-04T17:48:55.299 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.299 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.299 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.299 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:55.299 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253067+0000", 2024-09-04T17:48:55.299 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.299 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.299 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.299 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:55.299 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253068+0000", 2024-09-04T17:48:55.299 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0729999999999999e-06 2024-09-04T17:48:55.300 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.300 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.300 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:55.300 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253073+0000", 2024-09-04T17:48:55.300 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2259999999999996e-06 2024-09-04T17:48:55.300 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.300 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.300 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:55.300 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253091+0000", 2024-09-04T17:48:55.300 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7255999999999999e-05 2024-09-04T17:48:55.300 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.300 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.300 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:55.301 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253103+0000", 2024-09-04T17:48:55.301 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2632999999999999e-05 2024-09-04T17:48:55.301 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.301 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.301 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:55.301 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304173+0000", 2024-09-04T17:48:55.301 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051069537999999998 2024-09-04T17:48:55.301 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.301 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.301 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:55.301 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304184+0000", 2024-09-04T17:48:55.301 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1253000000000001e-05 2024-09-04T17:48:55.301 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.302 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:55.302 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.302 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.302 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.302 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44689 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:55.302 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253121+0000", 2024-09-04T17:48:55.302 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 125.01144291999999, 2024-09-04T17:48:55.302 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051192595, 2024-09-04T17:48:55.302 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:55.302 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:55.302 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:55.302 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:55.302 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:55.303 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:55.303 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44689 2024-09-04T17:48:55.303 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.303 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:55.303 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.303 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:55.303 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253121+0000", 2024-09-04T17:48:55.303 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.303 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.303 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.303 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:55.303 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253121+0000", 2024-09-04T17:48:55.303 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.303 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.304 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.304 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:55.304 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253122+0000", 2024-09-04T17:48:55.304 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.15e-06 2024-09-04T17:48:55.304 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.304 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.304 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:55.304 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253128+0000", 2024-09-04T17:48:55.304 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2569999999999998e-06 2024-09-04T17:48:55.304 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.304 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.304 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:55.304 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253145+0000", 2024-09-04T17:48:55.305 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7176e-05 2024-09-04T17:48:55.305 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.305 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.305 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:55.305 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253158+0000", 2024-09-04T17:48:55.305 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3139000000000001e-05 2024-09-04T17:48:55.305 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.305 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.305 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:55.305 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304287+0000", 2024-09-04T17:48:55.305 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051129149999999998 2024-09-04T17:48:55.305 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.305 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.305 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:55.306 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304314+0000", 2024-09-04T17:48:55.306 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6723000000000001e-05 2024-09-04T17:48:55.306 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.306 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:55.306 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.306 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.306 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.306 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44690 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:55.306 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253177+0000", 2024-09-04T17:48:55.306 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 125.01138679500001, 2024-09-04T17:48:55.306 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051169263, 2024-09-04T17:48:55.306 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:55.306 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:55.307 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:55.307 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:55.307 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:55.307 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:55.307 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44690 2024-09-04T17:48:55.307 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.307 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:55.307 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.307 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:55.307 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253177+0000", 2024-09-04T17:48:55.307 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.307 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.307 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.307 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:55.308 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253177+0000", 2024-09-04T17:48:55.308 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.308 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.308 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.308 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:55.308 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253178+0000", 2024-09-04T17:48:55.308 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1000000000000001e-06 2024-09-04T17:48:55.308 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.308 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.308 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:55.308 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253184+0000", 2024-09-04T17:48:55.308 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.2089999999999996e-06 2024-09-04T17:48:55.308 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.309 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.309 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:55.309 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253200+0000", 2024-09-04T17:48:55.309 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6087e-05 2024-09-04T17:48:55.309 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.309 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.309 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:55.309 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253231+0000", 2024-09-04T17:48:55.309 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0874999999999998e-05 2024-09-04T17:48:55.309 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.309 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.309 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:55.309 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304337+0000", 2024-09-04T17:48:55.310 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051106430000000001 2024-09-04T17:48:55.310 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.310 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.310 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:55.310 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304347+0000", 2024-09-04T17:48:55.310 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5619999999999993e-06 2024-09-04T17:48:55.310 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.310 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:55.310 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.310 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.310 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.310 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44691 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:55.310 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253249+0000", 2024-09-04T17:48:55.310 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 125.01131476099999, 2024-09-04T17:48:55.311 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051195630999999998, 2024-09-04T17:48:55.311 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:55.311 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:55.311 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:55.311 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:55.311 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:55.311 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:55.311 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44691 2024-09-04T17:48:55.311 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.311 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:55.311 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.311 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:55.311 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253249+0000", 2024-09-04T17:48:55.311 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.312 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.312 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.312 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:55.312 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253249+0000", 2024-09-04T17:48:55.312 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.312 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.312 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.312 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:55.312 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253251+0000", 2024-09-04T17:48:55.312 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2330000000000001e-06 2024-09-04T17:48:55.312 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.312 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.313 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:55.313 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253256+0000", 2024-09-04T17:48:55.313 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6099999999999997e-06 2024-09-04T17:48:55.313 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.313 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.313 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:55.313 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253274+0000", 2024-09-04T17:48:55.313 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7952999999999999e-05 2024-09-04T17:48:55.313 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.313 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.313 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:55.313 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253287+0000", 2024-09-04T17:48:55.313 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2839e-05 2024-09-04T17:48:55.313 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.314 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.314 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:55.314 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304396+0000", 2024-09-04T17:48:55.314 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051108990999999999 2024-09-04T17:48:55.314 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.314 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.314 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:55.314 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304445+0000", 2024-09-04T17:48:55.314 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9005000000000003e-05 2024-09-04T17:48:55.314 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.314 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:55.314 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.314 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.315 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.315 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44692 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:55.315 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253337+0000", 2024-09-04T17:48:55.315 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 125.01122718800001, 2024-09-04T17:48:55.315 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051138909000000003, 2024-09-04T17:48:55.315 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:55.315 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:55.315 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:55.315 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:55.315 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:55.315 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:55.315 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44692 2024-09-04T17:48:55.315 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.315 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:55.316 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.316 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:55.316 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253337+0000", 2024-09-04T17:48:55.316 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.316 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.316 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.316 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:55.316 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253337+0000", 2024-09-04T17:48:55.316 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.316 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.316 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.316 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:55.316 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253338+0000", 2024-09-04T17:48:55.317 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.463e-06 2024-09-04T17:48:55.317 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.317 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.317 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:55.317 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253357+0000", 2024-09-04T17:48:55.317 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9043e-05 2024-09-04T17:48:55.317 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.317 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.317 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:55.317 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253379+0000", 2024-09-04T17:48:55.317 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1243e-05 2024-09-04T17:48:55.317 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.317 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.318 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:55.318 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253396+0000", 2024-09-04T17:48:55.318 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7059e-05 2024-09-04T17:48:55.318 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.318 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.318 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:55.318 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304466+0000", 2024-09-04T17:48:55.318 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051070624000000002 2024-09-04T17:48:55.318 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.318 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.318 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:55.318 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304476+0000", 2024-09-04T17:48:55.318 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4769999999999992e-06 2024-09-04T17:48:55.318 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.319 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:55.319 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.319 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.319 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.319 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44693 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:55.319 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253415+0000", 2024-09-04T17:48:55.319 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 125.011148785, 2024-09-04T17:48:55.319 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051141949999999999, 2024-09-04T17:48:55.319 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:55.319 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:55.319 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:55.319 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:55.319 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:55.320 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:55.320 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44693 2024-09-04T17:48:55.320 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.320 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:55.320 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.320 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:55.320 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253415+0000", 2024-09-04T17:48:55.320 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.320 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.320 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.320 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:55.320 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253415+0000", 2024-09-04T17:48:55.320 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.320 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.321 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.321 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:55.321 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253416+0000", 2024-09-04T17:48:55.321 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1799999999999999e-06 2024-09-04T17:48:55.321 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.321 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.321 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:55.321 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253422+0000", 2024-09-04T17:48:55.321 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.5940000000000001e-06 2024-09-04T17:48:55.321 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.321 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.321 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:55.321 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253438+0000", 2024-09-04T17:48:55.321 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6008999999999999e-05 2024-09-04T17:48:55.322 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.322 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.322 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:55.322 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253452+0000", 2024-09-04T17:48:55.322 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3543000000000001e-05 2024-09-04T17:48:55.322 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.322 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.322 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:55.322 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304533+0000", 2024-09-04T17:48:55.322 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051081609 2024-09-04T17:48:55.322 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.322 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.322 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:55.323 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304557+0000", 2024-09-04T17:48:55.323 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4015e-05 2024-09-04T17:48:55.323 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.323 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:55.323 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.323 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.323 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.323 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44694 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:55.323 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253469+0000", 2024-09-04T17:48:55.323 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 125.01109466, 2024-09-04T17:48:55.323 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051121486000000001, 2024-09-04T17:48:55.323 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:55.323 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:55.323 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:55.324 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:55.324 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:55.324 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:55.324 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44694 2024-09-04T17:48:55.324 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.324 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:55.324 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.324 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:55.324 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253469+0000", 2024-09-04T17:48:55.324 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.324 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.324 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.324 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:55.324 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253469+0000", 2024-09-04T17:48:55.325 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.325 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.325 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.325 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:55.325 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253471+0000", 2024-09-04T17:48:55.325 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1400000000000001e-06 2024-09-04T17:48:55.325 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.325 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.325 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:55.325 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253476+0000", 2024-09-04T17:48:55.325 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7370000000000003e-06 2024-09-04T17:48:55.326 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.326 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.326 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:55.326 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253493+0000", 2024-09-04T17:48:55.326 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6549e-05 2024-09-04T17:48:55.326 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.326 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.326 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:55.326 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253506+0000", 2024-09-04T17:48:55.326 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3406e-05 2024-09-04T17:48:55.326 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.326 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.327 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:55.327 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304582+0000", 2024-09-04T17:48:55.327 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051075534999999998 2024-09-04T17:48:55.327 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.327 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.327 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:55.327 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304591+0000", 2024-09-04T17:48:55.327 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1190000000000003e-06 2024-09-04T17:48:55.327 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.327 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:55.327 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.328 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.328 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.328 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44695 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:55.328 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253552+0000", 2024-09-04T17:48:55.328 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 125.01101190599999, 2024-09-04T17:48:55.328 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051068872000000001, 2024-09-04T17:48:55.328 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:55.328 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:55.328 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:55.328 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:55.328 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:55.328 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:55.328 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44695 2024-09-04T17:48:55.329 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.329 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:55.329 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.329 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:55.329 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253552+0000", 2024-09-04T17:48:55.329 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.329 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.329 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.329 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:55.329 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253552+0000", 2024-09-04T17:48:55.329 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.329 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.329 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.329 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:55.330 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253553+0000", 2024-09-04T17:48:55.330 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1960000000000001e-06 2024-09-04T17:48:55.330 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.330 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.330 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:55.330 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253560+0000", 2024-09-04T17:48:55.330 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.2040000000000002e-06 2024-09-04T17:48:55.330 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.330 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.330 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:55.330 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253578+0000", 2024-09-04T17:48:55.330 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8241999999999999e-05 2024-09-04T17:48:55.330 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.331 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.331 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:55.331 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253591+0000", 2024-09-04T17:48:55.331 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3453e-05 2024-09-04T17:48:55.331 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.331 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.331 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:55.331 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304614+0000", 2024-09-04T17:48:55.331 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051022456000000001 2024-09-04T17:48:55.331 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.331 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.331 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:55.331 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304621+0000", 2024-09-04T17:48:55.332 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3209999999999999e-06 2024-09-04T17:48:55.332 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.332 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:55.332 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.332 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.332 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.332 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44696 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:55.332 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253609+0000", 2024-09-04T17:48:55.332 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 125.010955405, 2024-09-04T17:48:55.332 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051115351000000003, 2024-09-04T17:48:55.332 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:55.332 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:55.332 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:55.333 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:55.333 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:55.333 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:55.333 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44696 2024-09-04T17:48:55.333 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.333 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:55.333 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.333 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:55.333 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253609+0000", 2024-09-04T17:48:55.333 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.333 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.333 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.333 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:55.333 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253609+0000", 2024-09-04T17:48:55.334 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.334 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.334 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.334 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:55.334 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253610+0000", 2024-09-04T17:48:55.334 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2300000000000001e-06 2024-09-04T17:48:55.334 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.334 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.334 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:55.334 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253615+0000", 2024-09-04T17:48:55.334 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7260000000000002e-06 2024-09-04T17:48:55.334 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.335 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.335 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:55.335 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253632+0000", 2024-09-04T17:48:55.335 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7039999999999999e-05 2024-09-04T17:48:55.335 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.335 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.335 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:55.335 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253666+0000", 2024-09-04T17:48:55.335 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4662000000000001e-05 2024-09-04T17:48:55.335 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.335 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.335 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:55.335 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304677+0000", 2024-09-04T17:48:55.335 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051011017999999998 2024-09-04T17:48:55.336 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.336 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.336 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:55.336 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304724+0000", 2024-09-04T17:48:55.336 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6675000000000003e-05 2024-09-04T17:48:55.336 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.336 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:55.336 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.336 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.336 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.336 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44697 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:55.336 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253684+0000", 2024-09-04T17:48:55.336 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 125.010879911, 2024-09-04T17:48:55.337 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051073107, 2024-09-04T17:48:55.337 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:55.337 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:55.337 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:55.337 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:55.337 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:55.337 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:55.337 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44697 2024-09-04T17:48:55.337 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.337 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:55.337 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.337 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:55.337 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253684+0000", 2024-09-04T17:48:55.337 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.338 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.338 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.338 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:55.338 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253684+0000", 2024-09-04T17:48:55.338 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.338 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.338 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.338 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:55.339 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253685+0000", 2024-09-04T17:48:55.339 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0699999999999999e-06 2024-09-04T17:48:55.339 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.339 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.339 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:55.339 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253690+0000", 2024-09-04T17:48:55.339 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1959999999999999e-06 2024-09-04T17:48:55.339 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.339 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.339 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:55.340 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253707+0000", 2024-09-04T17:48:55.340 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6946e-05 2024-09-04T17:48:55.340 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.340 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.340 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:55.340 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253720+0000", 2024-09-04T17:48:55.340 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3076e-05 2024-09-04T17:48:55.340 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.340 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.340 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:55.340 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304745+0000", 2024-09-04T17:48:55.341 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051024760000000002 2024-09-04T17:48:55.341 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.341 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.341 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:55.341 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304757+0000", 2024-09-04T17:48:55.341 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2058999999999999e-05 2024-09-04T17:48:55.341 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.341 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:55.341 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.341 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.341 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.341 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44698 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:55.341 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253737+0000", 2024-09-04T17:48:55.342 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 125.01082756, 2024-09-04T17:48:55.342 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051176078, 2024-09-04T17:48:55.342 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:55.342 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:55.342 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:55.342 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:55.342 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:55.342 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:55.342 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44698 2024-09-04T17:48:55.342 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.342 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:55.342 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.342 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:55.343 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253737+0000", 2024-09-04T17:48:55.343 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.343 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.343 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.343 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:55.343 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253737+0000", 2024-09-04T17:48:55.343 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.343 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.343 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.343 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:55.344 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253738+0000", 2024-09-04T17:48:55.344 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0300000000000001e-06 2024-09-04T17:48:55.344 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.344 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.344 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:55.344 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253752+0000", 2024-09-04T17:48:55.344 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4358999999999999e-05 2024-09-04T17:48:55.344 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.344 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.344 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:55.345 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253771+0000", 2024-09-04T17:48:55.345 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8879999999999999e-05 2024-09-04T17:48:55.345 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.345 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.345 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:55.345 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253785+0000", 2024-09-04T17:48:55.345 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3852000000000001e-05 2024-09-04T17:48:55.345 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.345 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.345 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:55.345 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304876+0000", 2024-09-04T17:48:55.345 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051091105999999997 2024-09-04T17:48:55.345 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.346 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.346 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:55.346 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304913+0000", 2024-09-04T17:48:55.346 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.6850999999999997e-05 2024-09-04T17:48:55.346 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.346 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:55.346 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.346 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.346 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.346 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44699 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:55.346 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253804+0000", 2024-09-04T17:48:55.346 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 125.01076003599999, 2024-09-04T17:48:55.346 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051172250000000002, 2024-09-04T17:48:55.346 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:55.347 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:55.347 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:55.347 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:55.347 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:55.347 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:55.347 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44699 2024-09-04T17:48:55.347 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.347 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:55.347 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.347 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:55.347 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253804+0000", 2024-09-04T17:48:55.347 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.347 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.348 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.348 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:55.348 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253804+0000", 2024-09-04T17:48:55.348 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.348 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.348 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.348 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:55.348 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253805+0000", 2024-09-04T17:48:55.348 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.08e-06 2024-09-04T17:48:55.348 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.348 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.348 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:55.348 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253810+0000", 2024-09-04T17:48:55.348 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.1830000000000002e-06 2024-09-04T17:48:55.349 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.349 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.349 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:55.349 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253828+0000", 2024-09-04T17:48:55.349 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.7662000000000001e-05 2024-09-04T17:48:55.349 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.349 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.349 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:55.349 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253841+0000", 2024-09-04T17:48:55.349 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3203999999999999e-05 2024-09-04T17:48:55.349 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.350 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.350 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:55.350 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304966+0000", 2024-09-04T17:48:55.350 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051125273999999998 2024-09-04T17:48:55.350 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.350 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.350 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:55.350 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.304976+0000", 2024-09-04T17:48:55.350 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.8470000000000008e-06 2024-09-04T17:48:55.350 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.350 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:55.350 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.351 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.351 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.351 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44700 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:55.351 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253858+0000", 2024-09-04T17:48:55.351 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 125.010706382, 2024-09-04T17:48:55.351 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051218957000000002, 2024-09-04T17:48:55.351 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:55.351 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:55.351 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:55.351 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:55.352 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:55.352 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:55.352 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44700 2024-09-04T17:48:55.352 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.352 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:55.352 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.352 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:55.352 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253858+0000", 2024-09-04T17:48:55.352 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.352 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.352 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.352 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:55.352 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253858+0000", 2024-09-04T17:48:55.353 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.353 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.353 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.353 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:55.353 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253859+0000", 2024-09-04T17:48:55.353 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.217e-06 2024-09-04T17:48:55.353 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.353 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.353 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:55.353 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253864+0000", 2024-09-04T17:48:55.353 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7600000000000002e-06 2024-09-04T17:48:55.353 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.353 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.353 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:55.354 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253880+0000", 2024-09-04T17:48:55.354 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6501999999999999e-05 2024-09-04T17:48:55.354 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.354 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.354 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:55.354 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253899+0000", 2024-09-04T17:48:55.354 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.8907e-05 2024-09-04T17:48:55.354 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.354 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.354 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:55.355 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305052+0000", 2024-09-04T17:48:55.355 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051152989000000003 2024-09-04T17:48:55.355 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.355 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.355 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:55.355 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305077+0000", 2024-09-04T17:48:55.355 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4581999999999999e-05 2024-09-04T17:48:55.355 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.355 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:55.355 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.355 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.355 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.355 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:44701 3.3 3.bcc4a2c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-04T17:48:55.356 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:50.253917+0000", 2024-09-04T17:48:55.356 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 125.01064703, 2024-09-04T17:48:55.356 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051231113000000002, 2024-09-04T17:48:55.356 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:55.356 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:55.356 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "reached pg", 2024-09-04T17:48:55.356 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:55.356 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:55.356 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:55.356 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 44701 2024-09-04T17:48:55.356 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.356 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:55.356 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.356 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:55.357 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253917+0000", 2024-09-04T17:48:55.357 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.357 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.357 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.357 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:55.357 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253917+0000", 2024-09-04T17:48:55.357 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.357 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.357 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.357 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:55.357 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253918+0000", 2024-09-04T17:48:55.357 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.136e-06 2024-09-04T17:48:55.357 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.357 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.358 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:55.358 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253924+0000", 2024-09-04T17:48:55.358 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.3070000000000002e-06 2024-09-04T17:48:55.358 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.358 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.358 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:55.358 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253940+0000", 2024-09-04T17:48:55.358 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6612e-05 2024-09-04T17:48:55.358 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.358 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.359 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:55.359 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.253954+0000", 2024-09-04T17:48:55.359 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3709999999999999e-05 2024-09-04T17:48:55.359 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.359 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.359 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:55.359 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305136+0000", 2024-09-04T17:48:55.359 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.051181935999999997 2024-09-04T17:48:55.359 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.359 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.359 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:55.359 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:50.305148+0000", 2024-09-04T17:48:55.359 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2412000000000001e-05 2024-09-04T17:48:55.360 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.360 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:55.360 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.360 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.360 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:55.360 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:55.361 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-04T17:48:55.366 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:55.362+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:55.466 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:55.466+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:55.535 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:55.535 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:48:55.535 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:48:55.535 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:55.549 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-04T17:48:55.567 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:55.566+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:55.668 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:55.662+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:55.672 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:55.672 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:48:55.672 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:48:55.672 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:48:55.672 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:55.686 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-04T17:48:55.769 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:48:55.766+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:48:55.813 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:55.813 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:48:55.813 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:48:55.813 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:48:55.813 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.813 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8831 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2453504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:55.813 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:48:55.813 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 162.75298709399999, 2024-09-04T17:48:55.813 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14731111799999999, 2024-09-04T17:48:55.813 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:55.814 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:55.814 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:55.814 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:55.814 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:55.814 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:55.814 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8831 2024-09-04T17:48:55.814 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.814 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:55.814 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.814 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:55.814 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:48:55.814 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.814 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.815 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.815 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:55.815 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057810+0000", 2024-09-04T17:48:55.815 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.815 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.815 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.815 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:55.815 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057811+0000", 2024-09-04T17:48:55.815 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.083e-06 2024-09-04T17:48:55.815 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.815 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.815 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:55.815 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057814+0000", 2024-09-04T17:48:55.816 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.407e-06 2024-09-04T17:48:55.816 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.816 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.816 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:55.816 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057825+0000", 2024-09-04T17:48:55.816 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.17e-05 2024-09-04T17:48:55.816 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.816 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.816 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:55.816 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057836+0000", 2024-09-04T17:48:55.816 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0283e-05 2024-09-04T17:48:55.816 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.816 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.817 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:55.817 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.204762+0000", 2024-09-04T17:48:55.817 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.146926205 2024-09-04T17:48:55.817 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.817 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.818 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:55.818 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.204895+0000", 2024-09-04T17:48:55.818 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013328999999999999 2024-09-04T17:48:55.818 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.818 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.818 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:55.818 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205121+0000", 2024-09-04T17:48:55.818 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00022614999999999999 2024-09-04T17:48:55.818 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.818 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:55.818 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.819 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.819 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.819 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8832 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2490368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:55.819 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:48:55.819 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 162.75294906100001, 2024-09-04T17:48:55.819 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14770012299999999, 2024-09-04T17:48:55.820 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:55.820 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:55.820 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:55.820 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:55.820 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:55.820 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:55.820 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8832 2024-09-04T17:48:55.820 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.821 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:55.821 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.821 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:55.821 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:48:55.821 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.821 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.821 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.821 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:55.821 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057848+0000", 2024-09-04T17:48:55.821 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.821 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.821 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.821 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:55.822 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057849+0000", 2024-09-04T17:48:55.822 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6700000000000003e-07 2024-09-04T17:48:55.822 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.822 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.822 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:55.822 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057852+0000", 2024-09-04T17:48:55.822 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6359999999999998e-06 2024-09-04T17:48:55.822 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.822 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.822 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:55.822 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057860+0000", 2024-09-04T17:48:55.822 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8140000000000007e-06 2024-09-04T17:48:55.822 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.823 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.823 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:55.823 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057868+0000", 2024-09-04T17:48:55.823 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1999999999999997e-06 2024-09-04T17:48:55.823 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.823 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.823 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:55.823 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205208+0000", 2024-09-04T17:48:55.823 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14734067300000001 2024-09-04T17:48:55.823 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.823 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.823 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:55.823 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205347+0000", 2024-09-04T17:48:55.824 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013914900000000001 2024-09-04T17:48:55.824 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.824 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.824 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:55.824 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205548+0000", 2024-09-04T17:48:55.824 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000200984 2024-09-04T17:48:55.824 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.824 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:55.824 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.824 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.824 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.824 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8833 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2527232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:55.824 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:48:55.825 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 162.75291719099999, 2024-09-04T17:48:55.825 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148062572, 2024-09-04T17:48:55.825 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:55.825 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:55.825 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:55.825 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:55.825 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:55.825 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:55.825 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8833 2024-09-04T17:48:55.825 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.825 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:55.825 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.825 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:55.826 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:48:55.826 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.826 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.826 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.826 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:55.826 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057880+0000", 2024-09-04T17:48:55.826 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.826 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.826 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.826 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:55.826 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057881+0000", 2024-09-04T17:48:55.826 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4300000000000002e-07 2024-09-04T17:48:55.826 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.827 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.827 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:55.827 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057893+0000", 2024-09-04T17:48:55.827 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1973999999999999e-05 2024-09-04T17:48:55.827 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.827 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.827 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:55.827 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057906+0000", 2024-09-04T17:48:55.827 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2716e-05 2024-09-04T17:48:55.827 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.827 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.827 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:55.827 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057913+0000", 2024-09-04T17:48:55.828 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3930000000000002e-06 2024-09-04T17:48:55.828 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.828 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.828 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:55.828 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205621+0000", 2024-09-04T17:48:55.828 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14770788800000001 2024-09-04T17:48:55.828 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.828 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.828 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:55.828 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205740+0000", 2024-09-04T17:48:55.828 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000118936 2024-09-04T17:48:55.828 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.829 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.829 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:55.829 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.205943+0000", 2024-09-04T17:48:55.829 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020292199999999999 2024-09-04T17:48:55.829 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.830 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:55.830 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.830 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.830 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.830 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8834 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2564096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:55.830 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:48:55.830 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 162.75287391200001, 2024-09-04T17:48:55.830 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14846753200000001, 2024-09-04T17:48:55.830 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:55.830 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:55.830 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:55.830 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:55.830 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:55.831 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:55.831 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8834 2024-09-04T17:48:55.831 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.831 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:55.831 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.831 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:55.831 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:48:55.831 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.831 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.831 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.831 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:55.831 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057923+0000", 2024-09-04T17:48:55.831 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.832 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.832 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.832 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:55.832 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057925+0000", 2024-09-04T17:48:55.832 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0270000000000001e-06 2024-09-04T17:48:55.832 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.832 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.832 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:55.832 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057936+0000", 2024-09-04T17:48:55.832 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.112e-05 2024-09-04T17:48:55.833 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.833 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.833 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:55.833 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057946+0000", 2024-09-04T17:48:55.833 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.009e-05 2024-09-04T17:48:55.833 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.833 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.833 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:55.833 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057953+0000", 2024-09-04T17:48:55.833 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.6229999999999997e-06 2024-09-04T17:48:55.833 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.833 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.833 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:55.834 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206027+0000", 2024-09-04T17:48:55.834 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148073591 2024-09-04T17:48:55.834 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.834 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.834 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:55.834 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206161+0000", 2024-09-04T17:48:55.834 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013411599999999999 2024-09-04T17:48:55.834 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.834 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.834 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:55.834 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206391+0000", 2024-09-04T17:48:55.834 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000229965 2024-09-04T17:48:55.834 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.835 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:55.835 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.835 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.835 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.835 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8835 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2600960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:55.835 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:48:55.835 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 162.75283426199999, 2024-09-04T17:48:55.835 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14882941599999999, 2024-09-04T17:48:55.835 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:55.835 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:55.835 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:55.835 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:55.835 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:55.836 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:55.836 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8835 2024-09-04T17:48:55.836 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.836 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:55.836 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.836 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:55.836 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:48:55.836 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.836 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.836 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.836 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:55.836 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.057963+0000", 2024-09-04T17:48:55.836 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.837 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.837 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.837 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:55.837 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058544+0000", 2024-09-04T17:48:55.837 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00058061899999999999 2024-09-04T17:48:55.837 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.837 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.837 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:55.837 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058550+0000", 2024-09-04T17:48:55.837 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.1129999999999997e-06 2024-09-04T17:48:55.837 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.837 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.837 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:55.838 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058566+0000", 2024-09-04T17:48:55.838 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5820000000000001e-05 2024-09-04T17:48:55.838 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.838 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.838 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:55.838 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058578+0000", 2024-09-04T17:48:55.838 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2537e-05 2024-09-04T17:48:55.838 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.838 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.838 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:55.838 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206470+0000", 2024-09-04T17:48:55.838 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14789153799999999 2024-09-04T17:48:55.838 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.838 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.839 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:55.839 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206589+0000", 2024-09-04T17:48:55.839 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000118837 2024-09-04T17:48:55.839 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.839 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.839 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:55.839 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206793+0000", 2024-09-04T17:48:55.839 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000203952 2024-09-04T17:48:55.839 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.839 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:55.839 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.839 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.840 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.840 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8836 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2654208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:55.840 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:48:55.840 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 162.752201574, 2024-09-04T17:48:55.840 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14862433999999999, 2024-09-04T17:48:55.840 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:55.840 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:55.840 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:55.840 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:55.840 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:55.840 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:55.840 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8836 2024-09-04T17:48:55.840 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.841 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:55.841 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.841 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:55.841 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:48:55.841 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.841 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.841 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.841 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:55.841 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.058596+0000", 2024-09-04T17:48:55.841 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.841 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.841 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.841 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:55.842 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062663+0000", 2024-09-04T17:48:55.842 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0040670619999999998 2024-09-04T17:48:55.842 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.842 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.842 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:55.842 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062668+0000", 2024-09-04T17:48:55.842 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4199999999999998e-06 2024-09-04T17:48:55.842 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.842 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.842 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:55.842 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062689+0000", 2024-09-04T17:48:55.842 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0356999999999998e-05 2024-09-04T17:48:55.842 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.843 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.843 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:55.843 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062726+0000", 2024-09-04T17:48:55.843 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7302999999999998e-05 2024-09-04T17:48:55.843 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.843 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.843 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:55.843 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.206875+0000", 2024-09-04T17:48:55.843 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14414883100000001 2024-09-04T17:48:55.843 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.843 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.843 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:55.843 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.207016+0000", 2024-09-04T17:48:55.843 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000140822 2024-09-04T17:48:55.844 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.844 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.844 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:55.844 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.207220+0000", 2024-09-04T17:48:55.844 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020454499999999999 2024-09-04T17:48:55.844 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.844 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:55.844 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.844 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.844 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.844 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8844 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2928640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:55.844 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:48:55.844 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 162.74783749100001, 2024-09-04T17:48:55.845 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14752153100000001, 2024-09-04T17:48:55.845 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:55.845 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:55.845 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:55.845 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:55.845 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:55.845 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:55.845 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8844 2024-09-04T17:48:55.845 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.845 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:55.845 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.845 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:55.846 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:48:55.846 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.846 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.846 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.846 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:55.846 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062960+0000", 2024-09-04T17:48:55.846 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.846 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.846 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.846 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:55.846 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062961+0000", 2024-09-04T17:48:55.846 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.6700000000000003e-07 2024-09-04T17:48:55.846 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.847 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.847 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:55.847 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062973+0000", 2024-09-04T17:48:55.847 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2109999999999999e-05 2024-09-04T17:48:55.847 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.847 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.847 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:55.847 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062982+0000", 2024-09-04T17:48:55.847 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.6900000000000004e-06 2024-09-04T17:48:55.847 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.847 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.847 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:55.847 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.062990+0000", 2024-09-04T17:48:55.848 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1899999999999998e-06 2024-09-04T17:48:55.848 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.848 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.848 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:55.848 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210090+0000", 2024-09-04T17:48:55.848 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14710026300000001 2024-09-04T17:48:55.848 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.848 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.848 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:55.848 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210225+0000", 2024-09-04T17:48:55.848 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013532400000000001 2024-09-04T17:48:55.848 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.848 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.849 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:55.849 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210481+0000", 2024-09-04T17:48:55.849 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00025628700000000001 2024-09-04T17:48:55.849 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.849 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:55.849 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.849 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.849 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.849 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8845 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2965504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:55.849 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:48:55.849 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 162.747797768, 2024-09-04T17:48:55.849 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147916819, 2024-09-04T17:48:55.849 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:55.850 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:55.850 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:55.850 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:55.850 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:55.850 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:55.850 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8845 2024-09-04T17:48:55.850 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.850 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:55.850 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.850 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:55.850 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:48:55.850 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.850 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.851 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.851 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:55.851 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:48:55.851 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.851 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.851 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.851 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:55.851 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063000+0000", 2024-09-04T17:48:55.851 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.0399999999999995e-07 2024-09-04T17:48:55.851 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.851 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.851 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:55.851 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063003+0000", 2024-09-04T17:48:55.852 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6529999999999998e-06 2024-09-04T17:48:55.852 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.852 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.852 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:55.852 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063012+0000", 2024-09-04T17:48:55.852 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9500000000000007e-06 2024-09-04T17:48:55.852 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.852 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.852 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:55.852 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063027+0000", 2024-09-04T17:48:55.852 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5443000000000001e-05 2024-09-04T17:48:55.852 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.852 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.853 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:55.853 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210571+0000", 2024-09-04T17:48:55.853 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14754353100000001 2024-09-04T17:48:55.853 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.853 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.853 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:55.853 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210714+0000", 2024-09-04T17:48:55.853 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014295599999999999 2024-09-04T17:48:55.853 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.853 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.853 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:55.853 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210916+0000", 2024-09-04T17:48:55.854 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020258200000000001 2024-09-04T17:48:55.854 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.854 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:55.854 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.854 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.854 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.854 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8846 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3002368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:55.854 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:48:55.854 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 162.747760231, 2024-09-04T17:48:55.854 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14828011599999999, 2024-09-04T17:48:55.854 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:55.854 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:55.854 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:55.855 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:55.855 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:55.855 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:55.855 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8846 2024-09-04T17:48:55.855 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.855 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:55.855 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.855 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:55.855 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:48:55.855 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.855 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.855 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.855 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:55.855 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063037+0000", 2024-09-04T17:48:55.856 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.856 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.856 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.856 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:55.856 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063038+0000", 2024-09-04T17:48:55.856 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.23e-07 2024-09-04T17:48:55.856 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.856 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.856 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:55.856 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063051+0000", 2024-09-04T17:48:55.856 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.2757e-05 2024-09-04T17:48:55.856 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.856 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.857 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:55.857 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063070+0000", 2024-09-04T17:48:55.857 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9663000000000001e-05 2024-09-04T17:48:55.857 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.857 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.857 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:55.857 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063079+0000", 2024-09-04T17:48:55.857 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.0929999999999992e-06 2024-09-04T17:48:55.857 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.857 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.857 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:55.857 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.210989+0000", 2024-09-04T17:48:55.857 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14790984500000001 2024-09-04T17:48:55.858 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.858 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.858 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:55.858 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211109+0000", 2024-09-04T17:48:55.858 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000120056 2024-09-04T17:48:55.858 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.858 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.858 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:55.858 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211317+0000", 2024-09-04T17:48:55.858 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020797900000000001 2024-09-04T17:48:55.858 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.858 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:55.858 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.858 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.859 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.859 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8847 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3039232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:55.859 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:48:55.859 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 162.74768579900001, 2024-09-04T17:48:55.859 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14859408900000001, 2024-09-04T17:48:55.859 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:55.859 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:55.859 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:55.859 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:55.859 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:55.859 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:55.860 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8847 2024-09-04T17:48:55.860 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.860 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:55.860 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.860 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:55.860 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:48:55.860 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.860 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.860 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.860 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:55.860 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063112+0000", 2024-09-04T17:48:55.860 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.860 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.861 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.861 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:55.861 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063113+0000", 2024-09-04T17:48:55.861 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.204e-06 2024-09-04T17:48:55.861 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.861 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.861 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:55.861 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063116+0000", 2024-09-04T17:48:55.861 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.3330000000000001e-06 2024-09-04T17:48:55.861 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.861 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.862 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:55.862 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063126+0000", 2024-09-04T17:48:55.862 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0166999999999999e-05 2024-09-04T17:48:55.862 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.862 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.862 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:55.862 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063134+0000", 2024-09-04T17:48:55.862 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4329999999999998e-06 2024-09-04T17:48:55.862 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.862 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.862 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:55.862 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211392+0000", 2024-09-04T17:48:55.862 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148258109 2024-09-04T17:48:55.863 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.863 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.863 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:55.863 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211506+0000", 2024-09-04T17:48:55.863 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000113813 2024-09-04T17:48:55.863 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.863 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.863 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:55.863 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211706+0000", 2024-09-04T17:48:55.863 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00020002999999999999 2024-09-04T17:48:55.863 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.863 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:55.863 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.864 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.864 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.864 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8848 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:55.864 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:48:55.864 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 162.74765346199999, 2024-09-04T17:48:55.864 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14907967599999999, 2024-09-04T17:48:55.864 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:55.864 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:55.864 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:55.864 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:55.864 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:55.864 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:55.864 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8848 2024-09-04T17:48:55.865 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.865 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:55.865 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.865 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:55.865 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:48:55.865 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.865 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.865 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.865 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:55.865 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.063144+0000", 2024-09-04T17:48:55.865 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.865 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.865 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.866 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:55.866 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070589+0000", 2024-09-04T17:48:55.866 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.007445386 2024-09-04T17:48:55.866 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.866 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.866 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:55.866 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070600+0000", 2024-09-04T17:48:55.868 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0346999999999999e-05 2024-09-04T17:48:55.868 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.868 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.868 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:55.868 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070622+0000", 2024-09-04T17:48:55.868 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2289999999999998e-05 2024-09-04T17:48:55.868 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.868 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.868 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:55.868 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.070636+0000", 2024-09-04T17:48:55.869 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.4093e-05 2024-09-04T17:48:55.869 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.869 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.869 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:55.869 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211815+0000", 2024-09-04T17:48:55.869 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14117874699999999 2024-09-04T17:48:55.869 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.869 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.869 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:55.869 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.211983+0000", 2024-09-04T17:48:55.869 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00016852800000000001 2024-09-04T17:48:55.869 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.869 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.870 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:55.870 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.212224+0000", 2024-09-04T17:48:55.870 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024028499999999999 2024-09-04T17:48:55.870 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.870 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:55.870 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.870 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.870 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.870 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8862 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2019328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:55.870 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:48:55.870 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 162.73960399399999, 2024-09-04T17:48:55.870 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14755236399999999, 2024-09-04T17:48:55.870 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:55.871 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:55.871 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:55.871 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:55.871 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:55.871 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:55.871 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8862 2024-09-04T17:48:55.871 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.871 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:55.871 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.871 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:55.871 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:48:55.871 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.871 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.872 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.872 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:55.872 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071193+0000", 2024-09-04T17:48:55.872 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.872 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.872 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.872 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:55.872 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071194+0000", 2024-09-04T17:48:55.872 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.5000000000000002e-07 2024-09-04T17:48:55.872 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.872 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.872 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:55.872 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071197+0000", 2024-09-04T17:48:55.873 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5900000000000002e-06 2024-09-04T17:48:55.873 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.873 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.873 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:55.873 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071206+0000", 2024-09-04T17:48:55.873 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.9700000000000005e-06 2024-09-04T17:48:55.873 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.873 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.873 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:55.873 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071213+0000", 2024-09-04T17:48:55.873 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.9070000000000001e-06 2024-09-04T17:48:55.873 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.873 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.874 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:55.874 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218396+0000", 2024-09-04T17:48:55.874 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147183273 2024-09-04T17:48:55.874 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.874 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.874 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:55.874 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218538+0000", 2024-09-04T17:48:55.874 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014194199999999999 2024-09-04T17:48:55.874 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.874 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.874 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:55.874 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218746+0000", 2024-09-04T17:48:55.874 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000208032 2024-09-04T17:48:55.875 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.875 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:55.875 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.875 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.875 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.875 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8863 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2056192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:55.875 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:48:55.875 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 162.73957568399999, 2024-09-04T17:48:55.875 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14792281099999999, 2024-09-04T17:48:55.875 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:55.875 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:55.875 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:55.875 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:55.876 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:55.876 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:55.876 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8863 2024-09-04T17:48:55.876 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.876 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:55.876 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.876 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:55.876 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:48:55.876 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.876 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.876 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.876 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:55.876 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:48:55.877 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.877 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.877 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.877 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:55.877 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071222+0000", 2024-09-04T17:48:55.877 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1299999999999999e-07 2024-09-04T17:48:55.877 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.877 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.877 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:55.877 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071225+0000", 2024-09-04T17:48:55.877 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6199999999999999e-06 2024-09-04T17:48:55.877 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.878 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.878 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:55.878 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071234+0000", 2024-09-04T17:48:55.878 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.8769999999999999e-06 2024-09-04T17:48:55.878 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.878 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.878 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:55.878 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071251+0000", 2024-09-04T17:48:55.878 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.738e-05 2024-09-04T17:48:55.878 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.878 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.878 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:55.878 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218798+0000", 2024-09-04T17:48:55.879 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14754645 2024-09-04T17:48:55.879 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.879 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.879 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:55.879 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.218926+0000", 2024-09-04T17:48:55.879 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000128446 2024-09-04T17:48:55.879 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.879 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.879 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:55.879 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219145+0000", 2024-09-04T17:48:55.879 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021832500000000001 2024-09-04T17:48:55.879 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.879 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:55.880 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.880 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.880 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.880 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8864 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2093056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:55.880 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:48:55.880 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 162.73953410799999, 2024-09-04T17:48:55.880 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148277309, 2024-09-04T17:48:55.880 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:55.880 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:55.880 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:55.880 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:55.880 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:55.880 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:55.881 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8864 2024-09-04T17:48:55.881 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.881 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:55.881 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.881 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:55.881 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:48:55.881 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.881 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.881 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.881 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:55.881 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071263+0000", 2024-09-04T17:48:55.881 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.881 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.882 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.882 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:55.882 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071264+0000", 2024-09-04T17:48:55.882 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.3e-07 2024-09-04T17:48:55.882 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.882 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.882 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:55.882 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071267+0000", 2024-09-04T17:48:55.882 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.0139999999999999e-06 2024-09-04T17:48:55.882 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.882 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.882 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:55.882 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071276+0000", 2024-09-04T17:48:55.882 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.1689999999999998e-06 2024-09-04T17:48:55.883 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.883 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.883 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:55.883 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071283+0000", 2024-09-04T17:48:55.883 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2269999999999996e-06 2024-09-04T17:48:55.883 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.883 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.883 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:55.883 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219195+0000", 2024-09-04T17:48:55.883 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147911868 2024-09-04T17:48:55.883 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.883 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.883 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:55.884 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219324+0000", 2024-09-04T17:48:55.884 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00012835900000000001 2024-09-04T17:48:55.884 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.884 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.884 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:55.884 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219541+0000", 2024-09-04T17:48:55.884 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000216942 2024-09-04T17:48:55.884 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.884 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:55.884 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.884 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.884 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.884 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8865 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2109440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:55.885 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:48:55.885 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 162.73950464800001, 2024-09-04T17:48:55.885 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14865747300000001, 2024-09-04T17:48:55.885 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:55.885 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:55.885 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:55.885 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:55.885 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:55.885 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:55.885 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8865 2024-09-04T17:48:55.885 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.885 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:55.885 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.886 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:55.886 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:48:55.886 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.886 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.886 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.886 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:55.886 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:48:55.886 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.886 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.886 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.886 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:55.886 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071293+0000", 2024-09-04T17:48:55.886 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7999999999999995e-07 2024-09-04T17:48:55.887 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.887 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.887 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:55.887 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071296+0000", 2024-09-04T17:48:55.887 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6900000000000001e-06 2024-09-04T17:48:55.887 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.887 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.887 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:55.887 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071305+0000", 2024-09-04T17:48:55.887 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.5499999999999995e-06 2024-09-04T17:48:55.887 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.887 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.887 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:55.887 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071312+0000", 2024-09-04T17:48:55.888 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.1559999999999998e-06 2024-09-04T17:48:55.888 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.888 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.888 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:55.888 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219591+0000", 2024-09-04T17:48:55.888 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14827944900000001 2024-09-04T17:48:55.888 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.888 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.888 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:55.888 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219731+0000", 2024-09-04T17:48:55.888 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013931600000000001 2024-09-04T17:48:55.888 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.888 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.889 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:55.889 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.219950+0000", 2024-09-04T17:48:55.889 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021963199999999999 2024-09-04T17:48:55.889 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.889 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:55.889 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.889 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.889 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.889 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8866 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2146304~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:55.889 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:48:55.889 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 162.73947641500001, 2024-09-04T17:48:55.889 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14906117899999999, 2024-09-04T17:48:55.889 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:55.890 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:55.890 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:55.890 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:55.890 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:55.890 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:55.890 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8866 2024-09-04T17:48:55.890 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.890 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:55.890 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.890 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:55.890 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:48:55.890 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.890 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.890 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.891 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:55.891 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071321+0000", 2024-09-04T17:48:55.891 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.891 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.891 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.891 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:55.891 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071322+0000", 2024-09-04T17:48:55.891 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7999999999999995e-07 2024-09-04T17:48:55.891 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.891 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.891 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:55.891 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071338+0000", 2024-09-04T17:48:55.891 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5872999999999999e-05 2024-09-04T17:48:55.892 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.892 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.892 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:55.892 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071347+0000", 2024-09-04T17:48:55.892 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.4669999999999993e-06 2024-09-04T17:48:55.892 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.892 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.892 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:55.892 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071354+0000", 2024-09-04T17:48:55.892 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2099999999999996e-06 2024-09-04T17:48:55.892 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.892 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.892 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:55.893 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220008+0000", 2024-09-04T17:48:55.893 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.148653642 2024-09-04T17:48:55.893 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.893 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.893 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:55.893 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220155+0000", 2024-09-04T17:48:55.893 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014726600000000001 2024-09-04T17:48:55.893 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.893 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.893 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:55.893 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220382+0000", 2024-09-04T17:48:55.893 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00022704100000000001 2024-09-04T17:48:55.894 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.894 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:55.894 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.894 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.894 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.894 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8867 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2183168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:55.894 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:48:55.894 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 162.73943306199999, 2024-09-04T17:48:55.894 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.149445831, 2024-09-04T17:48:55.894 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:55.895 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:55.895 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:55.895 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:55.895 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:55.895 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:55.895 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8867 2024-09-04T17:48:55.895 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.895 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:55.895 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.895 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:55.895 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:48:55.895 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.895 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.895 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.896 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:55.896 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.071364+0000", 2024-09-04T17:48:55.896 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.896 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.896 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.896 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:55.896 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078653+0000", 2024-09-04T17:48:55.896 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0072890430000000003 2024-09-04T17:48:55.896 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.896 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.896 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:55.896 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078664+0000", 2024-09-04T17:48:55.896 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0307e-05 2024-09-04T17:48:55.897 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.897 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.897 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:55.897 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078690+0000", 2024-09-04T17:48:55.897 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6780000000000001e-05 2024-09-04T17:48:55.897 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.897 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.897 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:55.898 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.078706+0000", 2024-09-04T17:48:55.898 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5676999999999999e-05 2024-09-04T17:48:55.898 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.898 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.898 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:55.898 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220467+0000", 2024-09-04T17:48:55.898 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14176103500000001 2024-09-04T17:48:55.898 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.898 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.898 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:55.898 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220595+0000", 2024-09-04T17:48:55.898 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000127427 2024-09-04T17:48:55.899 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.899 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.899 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:55.899 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.220810+0000", 2024-09-04T17:48:55.899 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021556200000000001 2024-09-04T17:48:55.899 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.899 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:55.899 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.899 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.899 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.899 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8881 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2695168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:55.899 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:48:55.899 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 162.731610993, 2024-09-04T17:48:55.900 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.147543547, 2024-09-04T17:48:55.900 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:55.900 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:55.900 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:55.900 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:55.900 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:55.900 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:55.900 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8881 2024-09-04T17:48:55.900 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.900 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:55.900 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.900 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:55.900 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:48:55.901 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.901 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.901 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.901 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:55.901 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079186+0000", 2024-09-04T17:48:55.901 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.901 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.901 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.901 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:55.901 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079187+0000", 2024-09-04T17:48:55.901 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7700000000000004e-07 2024-09-04T17:48:55.901 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.901 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.902 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:55.902 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079198+0000", 2024-09-04T17:48:55.902 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0849999999999999e-05 2024-09-04T17:48:55.902 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.902 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.902 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:55.902 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079207+0000", 2024-09-04T17:48:55.902 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 9.5599999999999999e-06 2024-09-04T17:48:55.902 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.902 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.902 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:55.902 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079224+0000", 2024-09-04T17:48:55.902 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6387000000000001e-05 2024-09-04T17:48:55.902 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.903 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.903 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:55.903 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226414+0000", 2024-09-04T17:48:55.903 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14718975000000001 2024-09-04T17:48:55.903 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.903 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.903 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:55.903 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226534+0000", 2024-09-04T17:48:55.903 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000120288 2024-09-04T17:48:55.903 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.903 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.903 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:55.903 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226730+0000", 2024-09-04T17:48:55.904 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000196035 2024-09-04T17:48:55.904 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.904 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:55.904 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.904 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.904 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.904 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:8882 3.c 3:37c2e06a:::rbd_data.11345bd5898b.0000000000000013:a [read 2732032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-04T17:48:55.904 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:48:55.904 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 162.731562779, 2024-09-04T17:48:55.904 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.14795022399999999, 2024-09-04T17:48:55.904 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:55.904 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:55.904 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:55.905 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:55.905 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:55.905 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:55.905 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 8882 2024-09-04T17:48:55.905 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.905 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:55.905 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.905 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:55.905 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:48:55.905 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.905 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.905 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.905 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:55.905 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.079235+0000", 2024-09-04T17:48:55.906 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.906 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.906 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.906 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:55.906 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086588+0000", 2024-09-04T17:48:55.906 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0073530959999999999 2024-09-04T17:48:55.906 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.906 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.906 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:55.906 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086598+0000", 2024-09-04T17:48:55.906 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0122999999999999e-05 2024-09-04T17:48:55.906 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.906 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.907 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:55.907 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086626+0000", 2024-09-04T17:48:55.907 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.7914000000000001e-05 2024-09-04T17:48:55.907 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.907 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.907 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:55.907 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.086641+0000", 2024-09-04T17:48:55.907 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.5270000000000001e-05 2024-09-04T17:48:55.907 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.907 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.907 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:55.907 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226801+0000", 2024-09-04T17:48:55.907 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.140160321 2024-09-04T17:48:55.907 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.908 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.908 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:55.908 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.226951+0000", 2024-09-04T17:48:55.908 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00014965200000000001 2024-09-04T17:48:55.908 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.908 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.908 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:55.908 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:46:13.227185+0000", 2024-09-04T17:48:55.908 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00023384799999999999 2024-09-04T17:48:55.908 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.908 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:55.908 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.908 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.909 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.909 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:93470 3.1f 3:f95bc674:::rbd_header.1134878eadad:head [watch reconnect cookie 140018821420464 gen 16] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-04T17:48:55.909 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:57.837081+0000", 2024-09-04T17:48:55.909 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 57.973716895999999, 2024-09-04T17:48:55.909 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.072023486, 2024-09-04T17:48:55.909 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:55.909 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:55.909 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:55.909 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:55.909 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:55.909 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:55.909 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 93470 2024-09-04T17:48:55.909 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.910 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:55.910 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.910 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:55.910 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:57.837081+0000", 2024-09-04T17:48:55.910 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.910 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.910 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.910 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:55.910 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:57.837081+0000", 2024-09-04T17:48:55.910 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:55.910 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.910 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.910 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:55.911 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:57.837084+0000", 2024-09-04T17:48:55.911 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.8229999999999997e-06 2024-09-04T17:48:55.911 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.911 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.911 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:55.911 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:57.837098+0000", 2024-09-04T17:48:55.911 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3436e-05 2024-09-04T17:48:55.911 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.911 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.911 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:55.911 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:57.837123+0000", 2024-09-04T17:48:55.911 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5079000000000002e-05 2024-09-04T17:48:55.911 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.911 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.912 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:55.912 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:57.837145+0000", 2024-09-04T17:48:55.912 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.228e-05 2024-09-04T17:48:55.912 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.912 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.912 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:55.912 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:57.837360+0000", 2024-09-04T17:48:55.912 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021464100000000001 2024-09-04T17:48:55.912 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.912 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.912 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for peered", 2024-09-04T17:48:55.912 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:57.837410+0000", 2024-09-04T17:48:55.912 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0235000000000003e-05 2024-09-04T17:48:55.913 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.913 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.913 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:55.913 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.900175+0000", 2024-09-04T17:48:55.913 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0627652830000001 2024-09-04T17:48:55.913 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.913 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.913 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "waiting for rw locks", 2024-09-04T17:48:55.913 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.900370+0000", 2024-09-04T17:48:55.913 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019497199999999999 2024-09-04T17:48:55.913 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.913 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.913 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:55.914 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.906957+0000", 2024-09-04T17:48:55.914 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0065866889999999997 2024-09-04T17:48:55.914 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.914 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.914 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:55.914 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.907088+0000", 2024-09-04T17:48:55.914 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00013056100000000001 2024-09-04T17:48:55.914 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:55.914 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:55.914 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:55.914 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:58.909104+0000", 2024-09-04T17:48:55.914 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.002016487 2024-09-04T17:48:55.914 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.914 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:55.915 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.915 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:55.915 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:55.915 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:55.917 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-04T17:48:55.917 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:55.913+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:56.018 INFO:tasks.ceph.osd.0.smithi092.stderr:2024-09-04T17:48:56.014+0000 7ff58a897640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26421) UID: 0 2024-09-04T17:48:56.041 INFO:teuthology.orchestra.run.smithi092.stderr:marked out osd.3. 2024-09-04T17:48:56.056 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is set 2024-09-04T17:48:56.099 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:56.099 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:48:56.099 INFO:teuthology.orchestra.run.smithi190.stdout: "num_ops": 0 2024-09-04T17:48:56.099 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:56.114 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-04T17:48:56.161 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:48:56.161+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:48:56.220 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:56.214+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:56.233 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:56.233 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [], 2024-09-04T17:48:56.233 INFO:teuthology.orchestra.run.smithi190.stdout: "complaint_time": 30, 2024-09-04T17:48:56.233 INFO:teuthology.orchestra.run.smithi190.stdout: "num_blocked_ops": 0 2024-09-04T17:48:56.233 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:56.247 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-04T17:48:56.333 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:56.333+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:56.417 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-09-04T17:48:56.417 INFO:teuthology.orchestra.run.smithi190.stdout: "size": 20, 2024-09-04T17:48:56.417 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 600, 2024-09-04T17:48:56.417 INFO:teuthology.orchestra.run.smithi190.stdout: "ops": [ 2024-09-04T17:48:56.417 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.417 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60921 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 262144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:56.418 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822920+0000", 2024-09-04T17:48:56.418 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 102.59238073900001, 2024-09-04T17:48:56.418 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073904394999999998, 2024-09-04T17:48:56.418 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:56.418 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:56.418 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:56.418 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:56.418 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:56.418 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:56.418 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60921 2024-09-04T17:48:56.418 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.418 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:56.418 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.419 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:56.419 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822920+0000", 2024-09-04T17:48:56.419 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:56.419 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.419 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.419 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:56.419 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822920+0000", 2024-09-04T17:48:56.419 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:56.419 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.419 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.419 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:56.419 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822920+0000", 2024-09-04T17:48:56.420 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4999999999999999e-07 2024-09-04T17:48:56.420 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.420 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.420 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:56.420 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822921+0000", 2024-09-04T17:48:56.420 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.113e-06 2024-09-04T17:48:56.420 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.420 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.420 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:56.421 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822924+0000", 2024-09-04T17:48:56.421 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.354e-06 2024-09-04T17:48:56.421 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.421 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.421 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:56.421 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822927+0000", 2024-09-04T17:48:56.421 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.6199999999999999e-06 2024-09-04T17:48:56.421 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.421 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.421 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:56.421 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896586+0000", 2024-09-04T17:48:56.421 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073658976000000001 2024-09-04T17:48:56.421 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.422 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.422 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:56.422 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896626+0000", 2024-09-04T17:48:56.422 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0484000000000001e-05 2024-09-04T17:48:56.422 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.422 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.422 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:56.422 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896824+0000", 2024-09-04T17:48:56.422 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000197598 2024-09-04T17:48:56.422 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:56.422 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:56.422 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:56.422 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.423 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.423 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60922 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:56.423 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822930+0000", 2024-09-04T17:48:56.423 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 102.592370164, 2024-09-04T17:48:56.423 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074178872000000007, 2024-09-04T17:48:56.423 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:56.423 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:56.423 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:56.423 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:56.424 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:56.424 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:56.424 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60922 2024-09-04T17:48:56.424 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.424 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:56.424 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.424 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:56.424 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822930+0000", 2024-09-04T17:48:56.424 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:56.424 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.424 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.424 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:56.424 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822930+0000", 2024-09-04T17:48:56.425 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:56.425 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.425 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.425 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:56.425 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822930+0000", 2024-09-04T17:48:56.425 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.36e-07 2024-09-04T17:48:56.425 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.425 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.425 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:56.425 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822932+0000", 2024-09-04T17:48:56.425 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.567e-06 2024-09-04T17:48:56.426 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.426 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.426 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:56.426 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822938+0000", 2024-09-04T17:48:56.426 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4840000000000003e-06 2024-09-04T17:48:56.426 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.426 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.426 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:56.426 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822942+0000", 2024-09-04T17:48:56.426 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2039999999999999e-06 2024-09-04T17:48:56.426 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.426 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.426 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:56.427 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896850+0000", 2024-09-04T17:48:56.427 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073907941000000005 2024-09-04T17:48:56.427 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.427 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.427 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:56.427 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.896891+0000", 2024-09-04T17:48:56.427 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0964999999999998e-05 2024-09-04T17:48:56.427 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.427 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.427 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:56.427 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897109+0000", 2024-09-04T17:48:56.427 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00021847500000000001 2024-09-04T17:48:56.427 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:56.428 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:56.428 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:56.428 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.428 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.428 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60923 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 475136~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:56.428 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822949+0000", 2024-09-04T17:48:56.428 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 102.592351874, 2024-09-04T17:48:56.428 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074426928000000003, 2024-09-04T17:48:56.428 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:56.428 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:56.428 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:56.428 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:56.428 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:56.429 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:56.429 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60923 2024-09-04T17:48:56.429 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.429 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:56.429 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.429 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:56.429 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822949+0000", 2024-09-04T17:48:56.429 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:56.429 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.429 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.429 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:56.429 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822949+0000", 2024-09-04T17:48:56.430 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:56.430 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.430 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.430 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:56.430 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822949+0000", 2024-09-04T17:48:56.430 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.7399999999999999e-07 2024-09-04T17:48:56.430 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.430 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.430 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:56.430 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822951+0000", 2024-09-04T17:48:56.430 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.1940000000000001e-06 2024-09-04T17:48:56.430 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.431 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.431 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:56.431 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822957+0000", 2024-09-04T17:48:56.431 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.0650000000000004e-06 2024-09-04T17:48:56.431 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.431 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.431 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:56.431 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822960+0000", 2024-09-04T17:48:56.431 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.8009999999999999e-06 2024-09-04T17:48:56.431 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.431 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.431 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:56.431 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897138+0000", 2024-09-04T17:48:56.431 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074177990999999999 2024-09-04T17:48:56.432 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.432 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.432 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:56.432 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897179+0000", 2024-09-04T17:48:56.432 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.1057999999999999e-05 2024-09-04T17:48:56.432 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.432 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.432 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:56.432 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897375+0000", 2024-09-04T17:48:56.432 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019644500000000001 2024-09-04T17:48:56.432 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:56.432 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:56.432 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:56.433 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.433 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.433 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60924 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:56.433 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822964+0000", 2024-09-04T17:48:56.433 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 102.59233668900001, 2024-09-04T17:48:56.433 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074531902999999997, 2024-09-04T17:48:56.433 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:56.433 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:56.433 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:56.433 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:56.433 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:56.433 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:56.433 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60924 2024-09-04T17:48:56.434 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.434 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:56.434 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.434 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:56.434 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822964+0000", 2024-09-04T17:48:56.434 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:56.434 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.434 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.434 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:56.434 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822964+0000", 2024-09-04T17:48:56.434 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:56.434 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.434 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.435 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:56.435 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822964+0000", 2024-09-04T17:48:56.435 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.4999999999999999e-07 2024-09-04T17:48:56.435 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.435 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.435 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:56.435 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822966+0000", 2024-09-04T17:48:56.435 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.917e-06 2024-09-04T17:48:56.435 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.435 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.435 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:56.435 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822969+0000", 2024-09-04T17:48:56.435 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.315e-06 2024-09-04T17:48:56.436 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.436 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.436 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:56.436 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822972+0000", 2024-09-04T17:48:56.436 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.599e-06 2024-09-04T17:48:56.436 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.436 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.436 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:56.436 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897398+0000", 2024-09-04T17:48:56.436 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074426168000000001 2024-09-04T17:48:56.436 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.436 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.436 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:56.437 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897441+0000", 2024-09-04T17:48:56.437 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.3158000000000003e-05 2024-09-04T17:48:56.437 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.437 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.437 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:56.437 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897496+0000", 2024-09-04T17:48:56.437 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4496000000000001e-05 2024-09-04T17:48:56.437 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:56.437 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:56.437 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:56.437 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.437 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.437 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60925 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 720896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:56.438 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822975+0000", 2024-09-04T17:48:56.438 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 102.592325254, 2024-09-04T17:48:56.438 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074635272000000003, 2024-09-04T17:48:56.438 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:56.438 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:56.438 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:56.438 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:56.438 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:56.438 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:56.438 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60925 2024-09-04T17:48:56.438 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.438 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:56.438 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.438 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:56.439 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822975+0000", 2024-09-04T17:48:56.439 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:56.439 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.439 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.439 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:56.439 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822975+0000", 2024-09-04T17:48:56.439 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:56.439 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.439 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.439 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:56.439 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822975+0000", 2024-09-04T17:48:56.439 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.34e-07 2024-09-04T17:48:56.439 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.440 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.440 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:56.440 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822976+0000", 2024-09-04T17:48:56.440 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0359999999999999e-06 2024-09-04T17:48:56.440 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.440 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.440 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:56.440 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822980+0000", 2024-09-04T17:48:56.440 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.242e-06 2024-09-04T17:48:56.440 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.440 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.440 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:56.440 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822983+0000", 2024-09-04T17:48:56.441 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.8830000000000002e-06 2024-09-04T17:48:56.441 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.441 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.441 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:56.441 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897519+0000", 2024-09-04T17:48:56.441 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074536366000000007 2024-09-04T17:48:56.441 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.441 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.441 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:56.441 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897560+0000", 2024-09-04T17:48:56.441 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0677999999999999e-05 2024-09-04T17:48:56.441 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.441 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.442 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:56.442 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897610+0000", 2024-09-04T17:48:56.442 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0832999999999999e-05 2024-09-04T17:48:56.442 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:56.442 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:56.442 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:56.442 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.442 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.442 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60926 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 860160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:56.442 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.822986+0000", 2024-09-04T17:48:56.442 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 102.592314656, 2024-09-04T17:48:56.442 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074736413000000002, 2024-09-04T17:48:56.442 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:56.443 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:56.443 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:56.443 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:56.443 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:56.443 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:56.443 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60926 2024-09-04T17:48:56.443 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.443 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:56.443 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.443 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:56.443 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822986+0000", 2024-09-04T17:48:56.443 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:56.443 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.444 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.444 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:56.444 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822986+0000", 2024-09-04T17:48:56.444 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:56.444 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.444 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.444 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:56.444 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822986+0000", 2024-09-04T17:48:56.444 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.5600000000000002e-07 2024-09-04T17:48:56.444 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.444 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.444 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:56.444 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822987+0000", 2024-09-04T17:48:56.445 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.082e-06 2024-09-04T17:48:56.445 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.445 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.445 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:56.445 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822990+0000", 2024-09-04T17:48:56.445 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.2219999999999998e-06 2024-09-04T17:48:56.445 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.445 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.445 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:56.445 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.822994+0000", 2024-09-04T17:48:56.445 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.19e-06 2024-09-04T17:48:56.445 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.445 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.445 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:56.446 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897632+0000", 2024-09-04T17:48:56.446 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074638205999999999 2024-09-04T17:48:56.446 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.446 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.446 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:56.446 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897672+0000", 2024-09-04T17:48:56.446 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.9873999999999998e-05 2024-09-04T17:48:56.447 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.447 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.447 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:56.447 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897722+0000", 2024-09-04T17:48:56.447 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0583e-05 2024-09-04T17:48:56.447 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:56.447 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:56.447 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:56.447 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.447 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.447 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60927 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 966656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:56.448 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823000+0000", 2024-09-04T17:48:56.448 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 102.592300474, 2024-09-04T17:48:56.448 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074834036000000007, 2024-09-04T17:48:56.448 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:56.448 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:56.448 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:56.448 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:56.448 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:56.448 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:56.448 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60927 2024-09-04T17:48:56.448 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.448 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:56.448 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.449 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:56.449 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823000+0000", 2024-09-04T17:48:56.449 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:56.449 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.449 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.449 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:56.449 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823000+0000", 2024-09-04T17:48:56.449 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:56.449 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.449 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.450 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:56.450 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823000+0000", 2024-09-04T17:48:56.450 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.4099999999999999e-07 2024-09-04T17:48:56.450 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.450 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.450 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:56.450 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823003+0000", 2024-09-04T17:48:56.450 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0810000000000001e-06 2024-09-04T17:48:56.450 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.450 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.451 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:56.451 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823011+0000", 2024-09-04T17:48:56.451 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7090000000000008e-06 2024-09-04T17:48:56.451 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.451 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.451 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:56.451 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823017+0000", 2024-09-04T17:48:56.451 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.3890000000000004e-06 2024-09-04T17:48:56.451 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.451 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.451 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:56.451 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897745+0000", 2024-09-04T17:48:56.451 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074728252999999994 2024-09-04T17:48:56.452 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.452 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.452 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:56.452 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897785+0000", 2024-09-04T17:48:56.452 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.9830999999999999e-05 2024-09-04T17:48:56.452 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.452 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.452 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:56.452 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897834+0000", 2024-09-04T17:48:56.452 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9231999999999999e-05 2024-09-04T17:48:56.452 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:56.452 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:56.452 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:56.453 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.453 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.453 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60928 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:56.453 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823024+0000", 2024-09-04T17:48:56.453 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 102.592276543, 2024-09-04T17:48:56.453 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075067479000000006, 2024-09-04T17:48:56.453 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:56.453 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:56.453 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:56.453 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:56.453 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:56.453 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:56.453 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60928 2024-09-04T17:48:56.454 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.454 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:56.454 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.454 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:56.454 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823024+0000", 2024-09-04T17:48:56.454 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:56.454 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.454 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.454 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:56.454 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823024+0000", 2024-09-04T17:48:56.454 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:56.454 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.454 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.454 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:56.455 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823024+0000", 2024-09-04T17:48:56.455 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6499999999999999e-07 2024-09-04T17:48:56.455 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.455 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.455 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:56.455 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823027+0000", 2024-09-04T17:48:56.455 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.3099999999999999e-06 2024-09-04T17:48:56.455 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.455 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.455 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:56.455 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823033+0000", 2024-09-04T17:48:56.455 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7429999999999996e-06 2024-09-04T17:48:56.455 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.455 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.456 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:56.456 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823038+0000", 2024-09-04T17:48:56.456 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.561e-06 2024-09-04T17:48:56.456 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.456 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.456 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:56.456 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897855+0000", 2024-09-04T17:48:56.456 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074816740000000007 2024-09-04T17:48:56.456 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.456 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.456 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:56.456 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.897894+0000", 2024-09-04T17:48:56.456 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.9085999999999998e-05 2024-09-04T17:48:56.457 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.457 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.457 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:56.457 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898091+0000", 2024-09-04T17:48:56.457 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000197574 2024-09-04T17:48:56.457 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:56.457 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:56.457 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:56.457 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.457 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.457 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60929 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1212416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:56.457 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823042+0000", 2024-09-04T17:48:56.457 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 102.592258101, 2024-09-04T17:48:56.458 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075301251, 2024-09-04T17:48:56.458 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:56.458 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:56.458 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:56.458 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:56.458 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:56.458 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:56.458 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60929 2024-09-04T17:48:56.458 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.458 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:56.458 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.458 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:56.458 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823042+0000", 2024-09-04T17:48:56.459 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:56.459 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.459 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.459 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:56.459 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823042+0000", 2024-09-04T17:48:56.459 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:56.459 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.459 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.459 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:56.459 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823043+0000", 2024-09-04T17:48:56.459 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.1100000000000002e-07 2024-09-04T17:48:56.459 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.459 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.459 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:56.460 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823045+0000", 2024-09-04T17:48:56.460 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.933e-06 2024-09-04T17:48:56.460 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.460 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.460 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:56.460 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823051+0000", 2024-09-04T17:48:56.460 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.9569999999999999e-06 2024-09-04T17:48:56.460 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.460 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.460 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:56.460 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823055+0000", 2024-09-04T17:48:56.460 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4479999999999996e-06 2024-09-04T17:48:56.460 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.461 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.461 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:56.461 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898117+0000", 2024-09-04T17:48:56.461 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075062344000000003 2024-09-04T17:48:56.461 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.461 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.461 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:56.461 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898157+0000", 2024-09-04T17:48:56.461 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0095999999999997e-05 2024-09-04T17:48:56.461 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.461 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.461 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:56.462 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898344+0000", 2024-09-04T17:48:56.462 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000186162 2024-09-04T17:48:56.462 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:56.462 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:56.462 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:56.462 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.462 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.462 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60930 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1351680~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:56.462 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823061+0000", 2024-09-04T17:48:56.462 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 102.592239222, 2024-09-04T17:48:56.462 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075545507999999997, 2024-09-04T17:48:56.462 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:56.462 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:56.462 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:56.463 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:56.463 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:56.463 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:56.463 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60930 2024-09-04T17:48:56.463 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.463 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:56.463 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.463 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:56.463 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823061+0000", 2024-09-04T17:48:56.463 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:56.463 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.463 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.463 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:56.464 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823061+0000", 2024-09-04T17:48:56.464 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:56.464 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.464 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.464 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:56.464 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823378+0000", 2024-09-04T17:48:56.464 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00031631299999999998 2024-09-04T17:48:56.464 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.465 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.465 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:56.465 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823381+0000", 2024-09-04T17:48:56.465 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4020000000000002e-06 2024-09-04T17:48:56.465 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.465 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.465 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:56.465 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823389+0000", 2024-09-04T17:48:56.465 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.1429999999999998e-06 2024-09-04T17:48:56.465 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.465 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.465 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:56.466 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823395+0000", 2024-09-04T17:48:56.466 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.0059999999999998e-06 2024-09-04T17:48:56.466 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.466 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.466 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:56.466 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898368+0000", 2024-09-04T17:48:56.466 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074973032999999994 2024-09-04T17:48:56.466 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.466 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.466 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:56.466 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898410+0000", 2024-09-04T17:48:56.467 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2243000000000002e-05 2024-09-04T17:48:56.467 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.467 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.467 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:56.467 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898607+0000", 2024-09-04T17:48:56.467 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019636799999999999 2024-09-04T17:48:56.467 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:56.467 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:56.467 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:56.467 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.467 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.467 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60931 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1458176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:56.467 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823407+0000", 2024-09-04T17:48:56.468 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 102.591893148, 2024-09-04T17:48:56.468 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075460334000000004, 2024-09-04T17:48:56.468 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:56.468 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:56.468 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:56.468 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:56.468 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:56.468 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:56.468 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60931 2024-09-04T17:48:56.468 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.468 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:56.468 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.468 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:56.469 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823407+0000", 2024-09-04T17:48:56.469 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:56.469 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.469 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.469 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:56.469 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823407+0000", 2024-09-04T17:48:56.469 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:56.469 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.469 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.469 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:56.469 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823408+0000", 2024-09-04T17:48:56.469 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.9999999999999998e-07 2024-09-04T17:48:56.469 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.470 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.470 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:56.470 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823415+0000", 2024-09-04T17:48:56.470 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.5510000000000002e-06 2024-09-04T17:48:56.470 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.470 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.470 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:56.470 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823423+0000", 2024-09-04T17:48:56.470 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2760000000000003e-06 2024-09-04T17:48:56.470 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.470 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.470 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:56.470 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823426+0000", 2024-09-04T17:48:56.471 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.2839999999999998e-06 2024-09-04T17:48:56.471 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.471 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.471 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:56.471 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898631+0000", 2024-09-04T17:48:56.471 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075205045999999998 2024-09-04T17:48:56.471 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.471 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.471 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:56.471 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898672+0000", 2024-09-04T17:48:56.471 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.0614000000000001e-05 2024-09-04T17:48:56.471 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.471 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.472 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:56.472 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898868+0000", 2024-09-04T17:48:56.472 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019606300000000001 2024-09-04T17:48:56.472 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:56.472 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:56.472 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:56.472 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.472 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.472 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:60932 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1564672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:56.472 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.823434+0000", 2024-09-04T17:48:56.472 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 102.591866849, 2024-09-04T17:48:56.472 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075693393999999997, 2024-09-04T17:48:56.472 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:56.472 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:56.473 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:56.473 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:56.473 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:56.473 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:56.473 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 60932 2024-09-04T17:48:56.473 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.473 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:56.473 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.473 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:56.473 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823434+0000", 2024-09-04T17:48:56.473 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:56.473 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.474 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.474 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:56.474 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.823434+0000", 2024-09-04T17:48:56.474 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:56.474 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.474 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.474 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:56.474 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.830290+0000", 2024-09-04T17:48:56.474 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0068563039999999997 2024-09-04T17:48:56.474 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.474 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.474 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:56.474 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.830297+0000", 2024-09-04T17:48:56.474 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.4760000000000001e-06 2024-09-04T17:48:56.475 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.475 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.475 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:56.475 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.830308+0000", 2024-09-04T17:48:56.475 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0927999999999999e-05 2024-09-04T17:48:56.475 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.475 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.475 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:56.475 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.830317+0000", 2024-09-04T17:48:56.475 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 8.7280000000000001e-06 2024-09-04T17:48:56.475 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.475 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.475 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:56.476 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898892+0000", 2024-09-04T17:48:56.476 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.068574722000000005 2024-09-04T17:48:56.476 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.476 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.476 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:56.476 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.898934+0000", 2024-09-04T17:48:56.476 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2249000000000002e-05 2024-09-04T17:48:56.476 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.476 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.476 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:56.476 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.899127+0000", 2024-09-04T17:48:56.476 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019298699999999999 2024-09-04T17:48:56.476 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:56.477 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:56.477 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:56.480 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.480 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.480 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61182 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 868352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:56.480 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839249+0000", 2024-09-04T17:48:56.480 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 102.57605095700001, 2024-09-04T17:48:56.480 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073680151999999999, 2024-09-04T17:48:56.480 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:56.480 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:56.480 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:56.480 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:56.481 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:56.481 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:56.481 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61182 2024-09-04T17:48:56.481 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.481 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:56.481 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.481 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:56.481 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839249+0000", 2024-09-04T17:48:56.481 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:56.481 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.481 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.481 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:56.481 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839249+0000", 2024-09-04T17:48:56.482 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:56.482 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.482 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.482 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:56.482 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839250+0000", 2024-09-04T17:48:56.482 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.4999999999999998e-07 2024-09-04T17:48:56.482 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.482 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.482 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:56.482 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839257+0000", 2024-09-04T17:48:56.482 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.7700000000000004e-06 2024-09-04T17:48:56.482 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.483 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.483 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:56.483 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839263+0000", 2024-09-04T17:48:56.483 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.246e-06 2024-09-04T17:48:56.483 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.483 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.483 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:56.483 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839268+0000", 2024-09-04T17:48:56.483 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.814e-06 2024-09-04T17:48:56.483 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.483 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.483 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:56.483 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.912831+0000", 2024-09-04T17:48:56.483 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073563294000000001 2024-09-04T17:48:56.484 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.484 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.484 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:56.484 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.912876+0000", 2024-09-04T17:48:56.484 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4742e-05 2024-09-04T17:48:56.484 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.484 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.484 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:56.484 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.912930+0000", 2024-09-04T17:48:56.484 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.3936e-05 2024-09-04T17:48:56.484 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:56.484 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:56.485 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:56.485 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.485 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.485 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61183 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:56.485 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839275+0000", 2024-09-04T17:48:56.485 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 102.57602576399999, 2024-09-04T17:48:56.485 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073781886000000005, 2024-09-04T17:48:56.485 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:56.485 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:56.485 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:56.485 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:56.485 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:56.485 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:56.486 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61183 2024-09-04T17:48:56.486 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.486 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:56.486 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.486 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:56.486 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839275+0000", 2024-09-04T17:48:56.486 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:56.486 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.486 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.487 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:56.487 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839275+0000", 2024-09-04T17:48:56.487 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:56.487 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.487 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.487 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:56.487 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839275+0000", 2024-09-04T17:48:56.487 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6899999999999997e-07 2024-09-04T17:48:56.487 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.487 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.487 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:56.487 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839277+0000", 2024-09-04T17:48:56.487 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9530000000000002e-06 2024-09-04T17:48:56.488 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.488 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.488 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:56.488 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839283+0000", 2024-09-04T17:48:56.488 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.9329999999999998e-06 2024-09-04T17:48:56.488 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.488 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.488 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:56.488 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839288+0000", 2024-09-04T17:48:56.488 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4100000000000001e-06 2024-09-04T17:48:56.488 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.488 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.489 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:56.489 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.912959+0000", 2024-09-04T17:48:56.489 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073671511999999995 2024-09-04T17:48:56.489 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.489 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.489 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:56.489 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913001+0000", 2024-09-04T17:48:56.489 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.1984999999999999e-05 2024-09-04T17:48:56.489 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.489 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.489 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:56.489 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913057+0000", 2024-09-04T17:48:56.490 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.5523999999999999e-05 2024-09-04T17:48:56.490 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:56.490 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:56.490 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:56.490 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.490 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.490 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61184 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1114112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:56.490 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839294+0000", 2024-09-04T17:48:56.490 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 102.576006552, 2024-09-04T17:48:56.490 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073995263000000006, 2024-09-04T17:48:56.490 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:56.490 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:56.490 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:56.491 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:56.491 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:56.491 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:56.491 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61184 2024-09-04T17:48:56.491 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.491 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:56.491 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.491 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:56.491 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839294+0000", 2024-09-04T17:48:56.491 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:56.491 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.491 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.492 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:56.492 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839294+0000", 2024-09-04T17:48:56.492 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:56.492 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.492 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.492 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:56.492 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839294+0000", 2024-09-04T17:48:56.492 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 3.77e-07 2024-09-04T17:48:56.492 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.492 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.492 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:56.492 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839296+0000", 2024-09-04T17:48:56.492 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.9999999999999999e-06 2024-09-04T17:48:56.493 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.493 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.493 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:56.493 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839302+0000", 2024-09-04T17:48:56.493 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.7230000000000001e-06 2024-09-04T17:48:56.493 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.493 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.493 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:56.493 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839314+0000", 2024-09-04T17:48:56.493 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.1826e-05 2024-09-04T17:48:56.493 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.493 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.494 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:56.494 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913084+0000", 2024-09-04T17:48:56.494 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073770215 2024-09-04T17:48:56.494 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.494 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.494 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:56.494 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913127+0000", 2024-09-04T17:48:56.494 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.2982999999999998e-05 2024-09-04T17:48:56.494 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.494 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.495 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:56.495 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913289+0000", 2024-09-04T17:48:56.495 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000162139 2024-09-04T17:48:56.495 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:56.495 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:56.495 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:56.495 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.495 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.495 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61185 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1220608~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:56.495 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839321+0000", 2024-09-04T17:48:56.495 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 102.575979085, 2024-09-04T17:48:56.495 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074206062000000003, 2024-09-04T17:48:56.495 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:56.496 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:56.496 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:56.496 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:56.496 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:56.496 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:56.496 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61185 2024-09-04T17:48:56.496 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.496 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:56.496 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.496 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:56.496 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839321+0000", 2024-09-04T17:48:56.496 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:56.496 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.497 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.497 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:56.497 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839321+0000", 2024-09-04T17:48:56.497 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:56.497 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.497 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.497 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:56.497 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839322+0000", 2024-09-04T17:48:56.497 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.5400000000000001e-07 2024-09-04T17:48:56.497 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.497 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.497 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:56.497 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839324+0000", 2024-09-04T17:48:56.497 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.2539999999999999e-06 2024-09-04T17:48:56.498 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.498 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.498 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:56.498 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839331+0000", 2024-09-04T17:48:56.498 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.2139999999999999e-06 2024-09-04T17:48:56.498 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.498 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.498 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:56.498 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839336+0000", 2024-09-04T17:48:56.498 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.7029999999999997e-06 2024-09-04T17:48:56.498 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.498 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.498 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:56.499 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913320+0000", 2024-09-04T17:48:56.499 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.073984016 2024-09-04T17:48:56.499 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.499 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.499 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:56.499 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913366+0000", 2024-09-04T17:48:56.499 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.5704000000000002e-05 2024-09-04T17:48:56.499 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.499 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.499 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:56.499 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913527+0000", 2024-09-04T17:48:56.499 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00016161699999999999 2024-09-04T17:48:56.500 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:56.500 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:56.500 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:56.500 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.500 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.500 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61186 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1359872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:56.500 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839344+0000", 2024-09-04T17:48:56.500 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 102.57595674300001, 2024-09-04T17:48:56.500 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074450673999999994, 2024-09-04T17:48:56.500 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:56.500 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:56.500 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:56.500 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:56.501 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:56.501 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:56.501 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61186 2024-09-04T17:48:56.501 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.501 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:56.501 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.501 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:56.501 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839344+0000", 2024-09-04T17:48:56.501 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:56.501 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.501 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.501 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:56.501 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839344+0000", 2024-09-04T17:48:56.502 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:56.502 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.502 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.502 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:56.502 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839344+0000", 2024-09-04T17:48:56.502 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4799999999999999e-07 2024-09-04T17:48:56.502 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.502 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.502 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:56.502 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839346+0000", 2024-09-04T17:48:56.502 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0889999999999998e-06 2024-09-04T17:48:56.502 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.502 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.503 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:56.503 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839352+0000", 2024-09-04T17:48:56.503 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.6389999999999997e-06 2024-09-04T17:48:56.503 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.503 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.503 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:56.503 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839356+0000", 2024-09-04T17:48:56.503 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.233e-06 2024-09-04T17:48:56.503 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.503 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.503 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:56.503 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913557+0000", 2024-09-04T17:48:56.503 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074200944000000005 2024-09-04T17:48:56.503 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.504 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.504 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:56.504 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913601+0000", 2024-09-04T17:48:56.504 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4350000000000001e-05 2024-09-04T17:48:56.504 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.504 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.504 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:56.504 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913794+0000", 2024-09-04T17:48:56.504 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00019297100000000001 2024-09-04T17:48:56.504 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:56.504 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:56.505 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:56.505 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.505 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.505 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61187 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1466368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:56.505 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839364+0000", 2024-09-04T17:48:56.505 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 102.57593689399999, 2024-09-04T17:48:56.505 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074692334999999999, 2024-09-04T17:48:56.505 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:56.505 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:56.505 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:56.505 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:56.505 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:56.506 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:56.506 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61187 2024-09-04T17:48:56.506 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.506 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:56.506 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.506 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:56.506 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839364+0000", 2024-09-04T17:48:56.506 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:56.506 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.506 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.506 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:56.506 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839364+0000", 2024-09-04T17:48:56.506 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:56.507 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.507 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.507 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:56.507 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839364+0000", 2024-09-04T17:48:56.507 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.75e-07 2024-09-04T17:48:56.507 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.507 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.507 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:56.507 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839366+0000", 2024-09-04T17:48:56.507 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.0990000000000001e-06 2024-09-04T17:48:56.507 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.507 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.508 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:56.508 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839372+0000", 2024-09-04T17:48:56.508 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 6.2609999999999999e-06 2024-09-04T17:48:56.508 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.508 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.508 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:56.508 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839389+0000", 2024-09-04T17:48:56.508 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.6489000000000001e-05 2024-09-04T17:48:56.508 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.508 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.508 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:56.508 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913824+0000", 2024-09-04T17:48:56.509 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074434811000000004 2024-09-04T17:48:56.509 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.509 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.509 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:56.509 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.913868+0000", 2024-09-04T17:48:56.509 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4768000000000002e-05 2024-09-04T17:48:56.509 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.509 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.509 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:56.509 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914056+0000", 2024-09-04T17:48:56.509 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00018743199999999999 2024-09-04T17:48:56.509 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:56.510 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:56.510 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:56.510 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.510 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.510 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61188 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1605632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:56.510 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839399+0000", 2024-09-04T17:48:56.510 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 102.57590099799999, 2024-09-04T17:48:56.510 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074975095000000005, 2024-09-04T17:48:56.510 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:56.510 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:56.510 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:56.510 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:56.510 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:56.511 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:56.511 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61188 2024-09-04T17:48:56.511 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.511 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:56.511 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.511 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:56.511 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839399+0000", 2024-09-04T17:48:56.511 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:56.511 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.511 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.511 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:56.511 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839399+0000", 2024-09-04T17:48:56.512 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:56.512 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.512 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.512 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:56.512 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839400+0000", 2024-09-04T17:48:56.512 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 5.0100000000000005e-07 2024-09-04T17:48:56.512 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.512 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.512 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:56.512 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839402+0000", 2024-09-04T17:48:56.512 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 2.491e-06 2024-09-04T17:48:56.512 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.513 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.513 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:56.513 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839409+0000", 2024-09-04T17:48:56.513 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.0450000000000003e-06 2024-09-04T17:48:56.513 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.513 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.513 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:56.513 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839414+0000", 2024-09-04T17:48:56.513 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.87e-06 2024-09-04T17:48:56.513 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.513 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.513 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:56.513 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914086+0000", 2024-09-04T17:48:56.514 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.074671596000000007 2024-09-04T17:48:56.514 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.514 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.514 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:56.514 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914130+0000", 2024-09-04T17:48:56.514 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.4579999999999997e-05 2024-09-04T17:48:56.514 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.514 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.514 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:56.514 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914375+0000", 2024-09-04T17:48:56.514 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.00024401199999999999 2024-09-04T17:48:56.514 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:56.515 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:56.515 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:56.515 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.515 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.515 INFO:teuthology.orchestra.run.smithi190.stdout: "description": "osd_op(client.4404.0:61189 3.7 3:e26978f0:::rbd_data.11341dfde0c5.0000000000000003:1f [read 1712128~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-04T17:48:56.515 INFO:teuthology.orchestra.run.smithi190.stdout: "initiated_at": "2024-09-04T17:47:13.839424+0000", 2024-09-04T17:48:56.515 INFO:teuthology.orchestra.run.smithi190.stdout: "age": 102.57587679700001, 2024-09-04T17:48:56.515 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.075247680999999997, 2024-09-04T17:48:56.515 INFO:teuthology.orchestra.run.smithi190.stdout: "continuous": false, 2024-09-04T17:48:56.515 INFO:teuthology.orchestra.run.smithi190.stdout: "type_data": { 2024-09-04T17:48:56.515 INFO:teuthology.orchestra.run.smithi190.stdout: "flag_point": "started", 2024-09-04T17:48:56.515 INFO:teuthology.orchestra.run.smithi190.stdout: "client_info": { 2024-09-04T17:48:56.515 INFO:teuthology.orchestra.run.smithi190.stdout: "client": "client.4404", 2024-09-04T17:48:56.515 INFO:teuthology.orchestra.run.smithi190.stdout: "client_addr": "172.21.15.92:0/1008268690", 2024-09-04T17:48:56.516 INFO:teuthology.orchestra.run.smithi190.stdout: "tid": 61189 2024-09-04T17:48:56.516 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.516 INFO:teuthology.orchestra.run.smithi190.stdout: "events": [ 2024-09-04T17:48:56.516 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.516 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "initiated", 2024-09-04T17:48:56.516 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839424+0000", 2024-09-04T17:48:56.516 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:56.516 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.516 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.516 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "header_read", 2024-09-04T17:48:56.516 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.839424+0000", 2024-09-04T17:48:56.516 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0 2024-09-04T17:48:56.516 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.517 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.517 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "throttled", 2024-09-04T17:48:56.517 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.846320+0000", 2024-09-04T17:48:56.517 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.0068965210000000001 2024-09-04T17:48:56.517 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.517 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.517 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "all_read", 2024-09-04T17:48:56.517 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.846328+0000", 2024-09-04T17:48:56.517 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 7.8189999999999992e-06 2024-09-04T17:48:56.517 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.517 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.517 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "dispatched", 2024-09-04T17:48:56.518 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.846341+0000", 2024-09-04T17:48:56.518 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.3338e-05 2024-09-04T17:48:56.518 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.518 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.518 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "queued_for_pg", 2024-09-04T17:48:56.518 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.846352+0000", 2024-09-04T17:48:56.518 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 1.0479000000000001e-05 2024-09-04T17:48:56.518 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.518 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.518 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "reached_pg", 2024-09-04T17:48:56.518 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914405+0000", 2024-09-04T17:48:56.518 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.068053049000000004 2024-09-04T17:48:56.518 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.518 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.519 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "started", 2024-09-04T17:48:56.519 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914451+0000", 2024-09-04T17:48:56.519 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 4.6545999999999998e-05 2024-09-04T17:48:56.519 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2024-09-04T17:48:56.519 INFO:teuthology.orchestra.run.smithi190.stdout: { 2024-09-04T17:48:56.519 INFO:teuthology.orchestra.run.smithi190.stdout: "event": "done", 2024-09-04T17:48:56.519 INFO:teuthology.orchestra.run.smithi190.stdout: "time": "2024-09-04T17:47:13.914671+0000", 2024-09-04T17:48:56.519 INFO:teuthology.orchestra.run.smithi190.stdout: "duration": 0.000219929 2024-09-04T17:48:56.519 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:56.519 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:56.519 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:56.519 INFO:teuthology.orchestra.run.smithi190.stdout: } 2024-09-04T17:48:56.519 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-09-04T17:48:56.520 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-09-04T17:48:56.520 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-04T17:48:56.522 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-04T17:48:56.522 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:56.521+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:56.622 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:56.621+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:56.654 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:48:56.655 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-04T17:48:56.655 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-04T17:48:56.655 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:48:56.669 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-04T17:48:56.677 INFO:teuthology.orchestra.run.smithi092.stdout:osd.0: {} 2024-09-04T17:48:56.678 INFO:teuthology.orchestra.run.smithi092.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-04T17:48:56.694 INFO:teuthology.orchestra.run.smithi092.stdout:osd.1: {} 2024-09-04T17:48:56.694 INFO:teuthology.orchestra.run.smithi092.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-04T17:48:56.711 INFO:teuthology.orchestra.run.smithi092.stdout:osd.2: {} 2024-09-04T17:48:56.711 INFO:teuthology.orchestra.run.smithi092.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-04T17:48:56.723 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:56.718+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:56.727 INFO:teuthology.orchestra.run.smithi092.stdout:osd.3: {} 2024-09-04T17:48:56.727 INFO:teuthology.orchestra.run.smithi092.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-04T17:48:56.748 INFO:teuthology.orchestra.run.smithi092.stdout:osd.4: {} 2024-09-04T17:48:56.748 INFO:teuthology.orchestra.run.smithi092.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-04T17:48:56.768 INFO:teuthology.orchestra.run.smithi092.stdout:osd.5: {} 2024-09-04T17:48:56.768 INFO:teuthology.orchestra.run.smithi092.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-04T17:48:56.782 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:56.782 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:56.789 INFO:teuthology.orchestra.run.smithi092.stdout:osd.6: {} 2024-09-04T17:48:56.789 INFO:teuthology.orchestra.run.smithi092.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-04T17:48:56.796 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-04T17:48:56.810 INFO:teuthology.orchestra.run.smithi092.stdout:osd.7: {} 2024-09-04T17:48:56.810 INFO:teuthology.orchestra.run.smithi092.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-04T17:48:56.824 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:56.821+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:56.925 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:56.918+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:56.940 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:56.941 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:56.956 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-04T17:48:57.025 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:57.025+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:57.126 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:57.126 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:57.127 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:48:57.125+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:48:57.141 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-04T17:48:57.228 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:48:57.225+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:48:57.264 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:57.264 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:57.278 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-04T17:48:57.328 INFO:tasks.ceph.osd.1.smithi092.stderr:2024-09-04T17:48:57.322+0000 7f31064ee640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26424) UID: 0 2024-09-04T17:48:57.399 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:57.400 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:57.414 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-04T17:48:57.461 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-04T17:48:57.461+0000 7f2351511640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26443) UID: 0 2024-09-04T17:48:57.530 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:57.529+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:57.546 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:57.546 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:57.560 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-04T17:48:57.645 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-04T17:48:57.645+0000 7fb8fb203640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26449) UID: 0 2024-09-04T17:48:57.718 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:57.718 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:57.732 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-04T17:48:57.733 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-04T17:48:57.729+0000 7f57931d9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36735) UID: 0 2024-09-04T17:48:57.833 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-04T17:48:57.833+0000 7feca124f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26444) UID: 0 2024-09-04T17:48:57.922 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:57.922 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:57.933 INFO:tasks.ceph.osd.3.smithi092.stderr:2024-09-04T17:48:57.930+0000 7f4ae4332640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35380) UID: 0 2024-09-04T17:48:57.939 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-04T17:48:58.034 INFO:tasks.ceph.osd.2.smithi092.stderr:2024-09-04T17:48:58.030+0000 7fdf4d1ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 39406) UID: 0 2024-09-04T17:48:58.072 INFO:teuthology.orchestra.run.smithi092.stdout:All operations completed A-OK! 2024-09-04T17:48:58.080 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-04T17:48:58.110 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:58.112 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:58.124 DEBUG:teuthology.parallel:result is None 2024-09-04T17:48:58.124 DEBUG:teuthology.run_tasks:Unwinding manager rbd_fsx 2024-09-04T17:48:58.133 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-04T17:48:58.141 INFO:tasks.thrashosds:joining thrashosds 2024-09-04T17:48:58.142 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-04T17:48:58.316 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:58.316 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:58.332 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-04T17:48:58.461 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:58.461 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:58.475 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-04T17:48:58.588 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:58.588 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:58.601 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-04T17:48:58.721 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:58.721 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:58.735 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-04T17:48:58.852 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:58.853 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:58.867 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-04T17:48:58.999 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:58.999 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:59.014 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-04T17:48:59.064 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is set 2024-09-04T17:48:59.140 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:59.141 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:59.155 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-04T17:48:59.279 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:59.279 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:59.294 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-04T17:48:59.416 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:59.417 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:59.431 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-04T17:48:59.554 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:59.554 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:59.567 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-04T17:48:59.689 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:59.690 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:59.704 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-04T17:48:59.827 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:59.827 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:59.841 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-04T17:48:59.960 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:48:59.960 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:48:59.975 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-04T17:49:00.109 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:49:00.109 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:49:00.125 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-04T17:49:00.247 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:49:00.248 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:49:00.261 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-04T17:49:00.383 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:49:00.383 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:49:00.398 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-04T17:49:00.522 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:49:00.522 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:49:00.536 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-04T17:49:00.660 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:49:00.660 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:49:00.674 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-04T17:49:00.797 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:49:00.797 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:49:00.812 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-04T17:49:00.935 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-04T17:49:00.935 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:49:00.949 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-04T17:49:01.075 INFO:tasks.thrashosds.thrasher:inning osd 2024-09-04T17:49:01.076 INFO:tasks.thrashosds.thrasher:Adding osd 3 2024-09-04T17:49:01.076 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 3 2024-09-04T17:49:01.083 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:49:01.083 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:49:01.092 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-04T17:49:01.098 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-04T17:49:01.222 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:49:01.222 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:49:01.236 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-04T17:49:01.358 INFO:teuthology.orchestra.run.smithi190.stdout:ERROR: (22) Invalid argument 2024-09-04T17:49:01.358 INFO:teuthology.orchestra.run.smithi190.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-04T17:49:01.433 INFO:tasks.ceph.mon.a.smithi092.stderr:2024-09-04T17:49:01.430+0000 7f8f2f46f640 -1 mon.a@0(leader).osd e180 definitely_dead 0 2024-09-04T17:49:02.084 INFO:teuthology.orchestra.run.smithi092.stderr:marked in osd.3. 2024-09-04T17:49:02.085 INFO:teuthology.orchestra.run.smithi092.stderr:noscrub is unset 2024-09-04T17:49:02.117 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-04T17:49:02.119 INFO:tasks.thrashosds.thrasher:Added osd 3 2024-09-04T17:49:02.119 INFO:tasks.thrashosds.thrasher:inning osd 2024-09-04T17:49:02.119 INFO:tasks.thrashosds.thrasher:Adding osd 1 2024-09-04T17:49:02.120 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 1 2024-09-04T17:49:02.477 INFO:tasks.ceph.mon.a.smithi092.stderr:2024-09-04T17:49:02.470+0000 7f8f2f46f640 -1 mon.a@0(leader).osd e181 definitely_dead 0 2024-09-04T17:49:02.849 INFO:teuthology.orchestra.run.smithi092.stderr:nodeep-scrub is unset 2024-09-04T17:49:02.849 INFO:teuthology.orchestra.run.smithi092.stderr:marked in osd.1. 2024-09-04T17:49:02.884 INFO:tasks.thrashosds.thrasher:Added osd 1 2024-09-04T17:49:02.884 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-04T17:49:03.013 INFO:teuthology.orchestra.run.smithi092.stdout:mon.a: {} 2024-09-04T17:49:03.013 INFO:teuthology.orchestra.run.smithi092.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-04T17:49:03.026 INFO:teuthology.orchestra.run.smithi092.stdout:mon.b: {} 2024-09-04T17:49:03.027 INFO:teuthology.orchestra.run.smithi092.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-04T17:49:03.036 INFO:teuthology.orchestra.run.smithi092.stdout:mon.c: {} 2024-09-04T17:49:03.036 INFO:teuthology.orchestra.run.smithi092.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-04T17:49:03.052 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-04T17:49:03.866 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.0 to 1 (10000) 2024-09-04T17:49:03.915 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-04T17:49:04.861 INFO:teuthology.orchestra.run.smithi092.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-04T17:49:04.917 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-04T17:49:05.858 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.1 to 1 (10000) 2024-09-04T17:49:05.890 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-04T17:49:06.876 INFO:teuthology.orchestra.run.smithi092.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-04T17:49:06.904 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-04T17:49:07.870 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.4 to 1 (10000) 2024-09-04T17:49:07.900 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-04T17:49:09.130 INFO:teuthology.orchestra.run.smithi092.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-04T17:49:09.153 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-04T17:49:10.131 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.5 to 1 (10000) 2024-09-04T17:49:10.157 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-04T17:49:11.145 INFO:teuthology.orchestra.run.smithi092.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-04T17:49:11.169 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-04T17:49:12.154 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.7 to 1 (10000) 2024-09-04T17:49:12.178 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-09-04T17:49:12.852 INFO:teuthology.orchestra.run.smithi092.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-04T17:49:12.877 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-04T17:49:14.168 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.3 to 1 (10000) 2024-09-04T17:49:14.191 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-04T17:49:15.178 INFO:teuthology.orchestra.run.smithi092.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-04T17:49:15.202 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-04T17:49:16.196 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.2 to 1 (10000) 2024-09-04T17:49:16.220 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-04T17:49:17.201 INFO:teuthology.orchestra.run.smithi092.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-04T17:49:17.228 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-04T17:49:18.217 INFO:teuthology.orchestra.run.smithi092.stderr:reweighted osd.6 to 1 (10000) 2024-09-04T17:49:18.244 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-09-04T17:49:19.233 INFO:teuthology.orchestra.run.smithi092.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-04T17:49:19.256 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-04T17:49:19.257 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-04T17:49:19.257 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-04T17:49:19.257 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-04T17:49:19.257 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-04T17:49:19.257 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-04T17:49:19.561 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:49:19.561 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":198,"fsid":"f36b6ce2-8c72-4c99-8607-5c53c38ef67b","created":"2024-09-04T17:45:28.072591+0000","modified":"2024-09-04T17:49:19.222562+0000","last_up_change":"2024-09-04T17:47:57.033539+0000","last_in_change":"2024-09-04T17:49:02.845732+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-04T17:45:41.111814+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-04T17:45:46.265395+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-04T17:45:54.393683+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"176","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":45,"snap_epoch":176,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58efca2-bc5a-4bc4-b3d7-4036ebd9dc63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2135042466},{"type":"v1","addr":"172.21.15.92:6809","nonce":2135042466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":2135042466},{"type":"v1","addr":"172.21.15.92:6811","nonce":2135042466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":2135042466},{"type":"v1","addr":"172.21.15.92:6815","nonce":2135042466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":2135042466},{"type":"v1","addr":"172.21.15.92:6813","nonce":2135042466}]},"public_addr":"172.21.15.92:6809/2135042466","cluster_addr":"172.21.15.92:6811/2135042466","heartbeat_back_addr":"172.21.15.92:6815/2135042466","heartbeat_front_addr":"172.21.15.92:6813/2135042466","state":["exists","up"]},{"osd":1,"uuid":"36e03b64-2602-4143-bdd5-f7355de255fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":1306159585},{"type":"v1","addr":"172.21.15.92:6825","nonce":1306159585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1306159585},{"type":"v1","addr":"172.21.15.92:6827","nonce":1306159585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1306159585},{"type":"v1","addr":"172.21.15.92:6831","nonce":1306159585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1306159585},{"type":"v1","addr":"172.21.15.92:6829","nonce":1306159585}]},"public_addr":"172.21.15.92:6825/1306159585","cluster_addr":"172.21.15.92:6827/1306159585","heartbeat_back_addr":"172.21.15.92:6831/1306159585","heartbeat_front_addr":"172.21.15.92:6829/1306159585","state":["exists","up"]},{"osd":2,"uuid":"581ab71e-be04-4031-a0c4-f7e40b74b7c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":83,"up_from":103,"up_thru":185,"down_at":84,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":2500708849},{"type":"v1","addr":"172.21.15.92:6801","nonce":2500708849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":2500708849},{"type":"v1","addr":"172.21.15.92:6803","nonce":2500708849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":2500708849},{"type":"v1","addr":"172.21.15.92:6807","nonce":2500708849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":2500708849},{"type":"v1","addr":"172.21.15.92:6805","nonce":2500708849}]},"public_addr":"172.21.15.92:6801/2500708849","cluster_addr":"172.21.15.92:6803/2500708849","heartbeat_back_addr":"172.21.15.92:6807/2500708849","heartbeat_front_addr":"172.21.15.92:6805/2500708849","state":["exists","up"]},{"osd":3,"uuid":"5f5d096e-fab2-41fe-b8dd-943d1c40894c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":23,"up_from":48,"up_thru":186,"down_at":24,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":3390028311},{"type":"v1","addr":"172.21.15.92:6817","nonce":3390028311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":3390028311},{"type":"v1","addr":"172.21.15.92:6819","nonce":3390028311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":3390028311},{"type":"v1","addr":"172.21.15.92:6823","nonce":3390028311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":3390028311},{"type":"v1","addr":"172.21.15.92:6821","nonce":3390028311}]},"public_addr":"172.21.15.92:6817/3390028311","cluster_addr":"172.21.15.92:6819/3390028311","heartbeat_back_addr":"172.21.15.92:6823/3390028311","heartbeat_front_addr":"172.21.15.92:6821/3390028311","state":["exists","up"]},{"osd":4,"uuid":"840951a5-e13b-462c-8c8e-2ca61a53fe3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4272750248},{"type":"v1","addr":"172.21.15.190:6817","nonce":4272750248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4272750248},{"type":"v1","addr":"172.21.15.190:6819","nonce":4272750248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4272750248},{"type":"v1","addr":"172.21.15.190:6823","nonce":4272750248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":4272750248},{"type":"v1","addr":"172.21.15.190:6821","nonce":4272750248}]},"public_addr":"172.21.15.190:6817/4272750248","cluster_addr":"172.21.15.190:6819/4272750248","heartbeat_back_addr":"172.21.15.190:6823/4272750248","heartbeat_front_addr":"172.21.15.190:6821/4272750248","state":["exists","up"]},{"osd":5,"uuid":"82e556e5-00c3-46fc-aab5-3544bba279d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":1688427632},{"type":"v1","addr":"172.21.15.190:6809","nonce":1688427632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":1688427632},{"type":"v1","addr":"172.21.15.190:6811","nonce":1688427632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":1688427632},{"type":"v1","addr":"172.21.15.190:6815","nonce":1688427632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1688427632},{"type":"v1","addr":"172.21.15.190:6813","nonce":1688427632}]},"public_addr":"172.21.15.190:6809/1688427632","cluster_addr":"172.21.15.190:6811/1688427632","heartbeat_back_addr":"172.21.15.190:6815/1688427632","heartbeat_front_addr":"172.21.15.190:6813/1688427632","state":["exists","up"]},{"osd":6,"uuid":"4e1e3d67-3020-470d-98b7-891fdc966238","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":58,"up_from":117,"up_thru":179,"down_at":60,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":466854783},{"type":"v1","addr":"172.21.15.190:6825","nonce":466854783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":466854783},{"type":"v1","addr":"172.21.15.190:6827","nonce":466854783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":466854783},{"type":"v1","addr":"172.21.15.190:6831","nonce":466854783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":466854783},{"type":"v1","addr":"172.21.15.190:6829","nonce":466854783}]},"public_addr":"172.21.15.190:6825/466854783","cluster_addr":"172.21.15.190:6827/466854783","heartbeat_back_addr":"172.21.15.190:6831/466854783","heartbeat_front_addr":"172.21.15.190:6829/466854783","state":["exists","up"]},{"osd":7,"uuid":"bd9c9d3e-27db-4bc2-b998-7f446576a543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":4006671087},{"type":"v1","addr":"172.21.15.190:6801","nonce":4006671087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":4006671087},{"type":"v1","addr":"172.21.15.190:6803","nonce":4006671087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":4006671087},{"type":"v1","addr":"172.21.15.190:6807","nonce":4006671087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":4006671087},{"type":"v1","addr":"172.21.15.190:6805","nonce":4006671087}]},"public_addr":"172.21.15.190:6801/4006671087","cluster_addr":"172.21.15.190:6803/4006671087","heartbeat_back_addr":"172.21.15.190:6807/4006671087","heartbeat_front_addr":"172.21.15.190:6805/4006671087","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-04T17:45:37.324122+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-04T17:45:37.856638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-04T17:45:36.966684+0000","dead_epoch":84},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-04T17:45:37.266271+0000","dead_epoch":24},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-04T17:45:37.627069+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-04T17:45:37.241115+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-04T17:45:37.397375+0000","dead_epoch":60},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-04T17:45:36.619385+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-04T17:49:19.580 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-04T17:49:19.580 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-04T17:49:19.887 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:49:19.887 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":198,"fsid":"f36b6ce2-8c72-4c99-8607-5c53c38ef67b","created":"2024-09-04T17:45:28.072591+0000","modified":"2024-09-04T17:49:19.222562+0000","last_up_change":"2024-09-04T17:47:57.033539+0000","last_in_change":"2024-09-04T17:49:02.845732+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-04T17:45:41.111814+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-04T17:45:46.265395+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-04T17:45:54.393683+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"176","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":45,"snap_epoch":176,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58efca2-bc5a-4bc4-b3d7-4036ebd9dc63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2135042466},{"type":"v1","addr":"172.21.15.92:6809","nonce":2135042466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":2135042466},{"type":"v1","addr":"172.21.15.92:6811","nonce":2135042466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":2135042466},{"type":"v1","addr":"172.21.15.92:6815","nonce":2135042466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":2135042466},{"type":"v1","addr":"172.21.15.92:6813","nonce":2135042466}]},"public_addr":"172.21.15.92:6809/2135042466","cluster_addr":"172.21.15.92:6811/2135042466","heartbeat_back_addr":"172.21.15.92:6815/2135042466","heartbeat_front_addr":"172.21.15.92:6813/2135042466","state":["exists","up"]},{"osd":1,"uuid":"36e03b64-2602-4143-bdd5-f7355de255fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":1306159585},{"type":"v1","addr":"172.21.15.92:6825","nonce":1306159585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1306159585},{"type":"v1","addr":"172.21.15.92:6827","nonce":1306159585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1306159585},{"type":"v1","addr":"172.21.15.92:6831","nonce":1306159585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1306159585},{"type":"v1","addr":"172.21.15.92:6829","nonce":1306159585}]},"public_addr":"172.21.15.92:6825/1306159585","cluster_addr":"172.21.15.92:6827/1306159585","heartbeat_back_addr":"172.21.15.92:6831/1306159585","heartbeat_front_addr":"172.21.15.92:6829/1306159585","state":["exists","up"]},{"osd":2,"uuid":"581ab71e-be04-4031-a0c4-f7e40b74b7c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":83,"up_from":103,"up_thru":185,"down_at":84,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":2500708849},{"type":"v1","addr":"172.21.15.92:6801","nonce":2500708849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":2500708849},{"type":"v1","addr":"172.21.15.92:6803","nonce":2500708849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":2500708849},{"type":"v1","addr":"172.21.15.92:6807","nonce":2500708849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":2500708849},{"type":"v1","addr":"172.21.15.92:6805","nonce":2500708849}]},"public_addr":"172.21.15.92:6801/2500708849","cluster_addr":"172.21.15.92:6803/2500708849","heartbeat_back_addr":"172.21.15.92:6807/2500708849","heartbeat_front_addr":"172.21.15.92:6805/2500708849","state":["exists","up"]},{"osd":3,"uuid":"5f5d096e-fab2-41fe-b8dd-943d1c40894c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":23,"up_from":48,"up_thru":186,"down_at":24,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":3390028311},{"type":"v1","addr":"172.21.15.92:6817","nonce":3390028311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":3390028311},{"type":"v1","addr":"172.21.15.92:6819","nonce":3390028311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":3390028311},{"type":"v1","addr":"172.21.15.92:6823","nonce":3390028311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":3390028311},{"type":"v1","addr":"172.21.15.92:6821","nonce":3390028311}]},"public_addr":"172.21.15.92:6817/3390028311","cluster_addr":"172.21.15.92:6819/3390028311","heartbeat_back_addr":"172.21.15.92:6823/3390028311","heartbeat_front_addr":"172.21.15.92:6821/3390028311","state":["exists","up"]},{"osd":4,"uuid":"840951a5-e13b-462c-8c8e-2ca61a53fe3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4272750248},{"type":"v1","addr":"172.21.15.190:6817","nonce":4272750248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4272750248},{"type":"v1","addr":"172.21.15.190:6819","nonce":4272750248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4272750248},{"type":"v1","addr":"172.21.15.190:6823","nonce":4272750248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":4272750248},{"type":"v1","addr":"172.21.15.190:6821","nonce":4272750248}]},"public_addr":"172.21.15.190:6817/4272750248","cluster_addr":"172.21.15.190:6819/4272750248","heartbeat_back_addr":"172.21.15.190:6823/4272750248","heartbeat_front_addr":"172.21.15.190:6821/4272750248","state":["exists","up"]},{"osd":5,"uuid":"82e556e5-00c3-46fc-aab5-3544bba279d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":1688427632},{"type":"v1","addr":"172.21.15.190:6809","nonce":1688427632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":1688427632},{"type":"v1","addr":"172.21.15.190:6811","nonce":1688427632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":1688427632},{"type":"v1","addr":"172.21.15.190:6815","nonce":1688427632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1688427632},{"type":"v1","addr":"172.21.15.190:6813","nonce":1688427632}]},"public_addr":"172.21.15.190:6809/1688427632","cluster_addr":"172.21.15.190:6811/1688427632","heartbeat_back_addr":"172.21.15.190:6815/1688427632","heartbeat_front_addr":"172.21.15.190:6813/1688427632","state":["exists","up"]},{"osd":6,"uuid":"4e1e3d67-3020-470d-98b7-891fdc966238","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":58,"up_from":117,"up_thru":179,"down_at":60,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":466854783},{"type":"v1","addr":"172.21.15.190:6825","nonce":466854783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":466854783},{"type":"v1","addr":"172.21.15.190:6827","nonce":466854783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":466854783},{"type":"v1","addr":"172.21.15.190:6831","nonce":466854783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":466854783},{"type":"v1","addr":"172.21.15.190:6829","nonce":466854783}]},"public_addr":"172.21.15.190:6825/466854783","cluster_addr":"172.21.15.190:6827/466854783","heartbeat_back_addr":"172.21.15.190:6831/466854783","heartbeat_front_addr":"172.21.15.190:6829/466854783","state":["exists","up"]},{"osd":7,"uuid":"bd9c9d3e-27db-4bc2-b998-7f446576a543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":4006671087},{"type":"v1","addr":"172.21.15.190:6801","nonce":4006671087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":4006671087},{"type":"v1","addr":"172.21.15.190:6803","nonce":4006671087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":4006671087},{"type":"v1","addr":"172.21.15.190:6807","nonce":4006671087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":4006671087},{"type":"v1","addr":"172.21.15.190:6805","nonce":4006671087}]},"public_addr":"172.21.15.190:6801/4006671087","cluster_addr":"172.21.15.190:6803/4006671087","heartbeat_back_addr":"172.21.15.190:6807/4006671087","heartbeat_front_addr":"172.21.15.190:6805/4006671087","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-04T17:45:37.324122+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-04T17:45:37.856638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-04T17:45:36.966684+0000","dead_epoch":84},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-04T17:45:37.266271+0000","dead_epoch":24},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-04T17:45:37.627069+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-04T17:45:37.241115+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-04T17:45:37.397375+0000","dead_epoch":60},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-04T17:45:36.619385+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-04T17:49:19.906 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-04T17:49:19.906 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-04T17:49:19.907 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-04T17:49:19.907 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-04T17:49:19.908 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-04T17:49:19.908 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-04T17:49:19.908 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-04T17:49:19.909 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-04T17:49:20.159 INFO:teuthology.orchestra.run.smithi092.stdout:502511173650 2024-09-04T17:49:20.159 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-04T17:49:20.174 INFO:teuthology.orchestra.run.smithi092.stdout:442381631510 2024-09-04T17:49:20.175 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-04T17:49:20.175 INFO:teuthology.orchestra.run.smithi092.stdout:64424509488 2024-09-04T17:49:20.176 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-04T17:49:20.177 INFO:teuthology.orchestra.run.smithi092.stdout:64424509487 2024-09-04T17:49:20.177 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-04T17:49:20.178 INFO:teuthology.orchestra.run.smithi092.stdout:206158430244 2024-09-04T17:49:20.178 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-04T17:49:20.182 INFO:teuthology.orchestra.run.smithi092.stdout:64424509487 2024-09-04T17:49:20.182 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-04T17:49:20.189 INFO:teuthology.orchestra.run.smithi092.stdout:64424509487 2024-09-04T17:49:20.189 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-04T17:49:20.199 INFO:teuthology.orchestra.run.smithi092.stdout:60129542192 2024-09-04T17:49:20.200 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-04T17:49:20.670 INFO:teuthology.orchestra.run.smithi092.stdout:64424509487 2024-09-04T17:49:20.678 INFO:teuthology.orchestra.run.smithi092.stdout:502511173649 2024-09-04T17:49:20.699 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509488 got 64424509487 for osd.4 2024-09-04T17:49:20.705 INFO:tasks.ceph.ceph_manager.ceph:need seq 502511173650 got 502511173649 for osd.6 2024-09-04T17:49:20.705 INFO:teuthology.orchestra.run.smithi092.stdout:442381631508 2024-09-04T17:49:20.727 INFO:teuthology.orchestra.run.smithi092.stdout:206158430243 2024-09-04T17:49:20.728 INFO:teuthology.orchestra.run.smithi092.stdout:64424509486 2024-09-04T17:49:20.729 INFO:tasks.ceph.ceph_manager.ceph:need seq 442381631510 got 442381631508 for osd.2 2024-09-04T17:49:20.734 INFO:teuthology.orchestra.run.smithi092.stdout:64424509486 2024-09-04T17:49:20.751 INFO:tasks.ceph.ceph_manager.ceph:need seq 206158430244 got 206158430243 for osd.3 2024-09-04T17:49:20.751 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509487 got 64424509486 for osd.1 2024-09-04T17:49:20.763 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509487 got 64424509486 for osd.0 2024-09-04T17:49:20.764 INFO:teuthology.orchestra.run.smithi092.stdout:60129542190 2024-09-04T17:49:20.766 INFO:teuthology.orchestra.run.smithi092.stdout:64424509486 2024-09-04T17:49:20.784 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542192 got 60129542190 for osd.7 2024-09-04T17:49:20.789 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509487 got 64424509486 for osd.5 2024-09-04T17:49:21.700 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-04T17:49:21.705 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-04T17:49:21.730 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-04T17:49:21.751 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-04T17:49:21.752 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-04T17:49:21.763 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-04T17:49:21.785 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-04T17:49:21.789 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-04T17:49:22.208 INFO:teuthology.orchestra.run.smithi092.stdout:64424509488 2024-09-04T17:49:22.231 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509488 got 64424509488 for osd.4 2024-09-04T17:49:22.231 DEBUG:teuthology.parallel:result is None 2024-09-04T17:49:22.280 INFO:teuthology.orchestra.run.smithi092.stdout:64424509487 2024-09-04T17:49:22.287 INFO:teuthology.orchestra.run.smithi092.stdout:502511173651 2024-09-04T17:49:22.289 INFO:teuthology.orchestra.run.smithi092.stdout:442381631510 2024-09-04T17:49:22.301 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509487 got 64424509487 for osd.1 2024-09-04T17:49:22.301 DEBUG:teuthology.parallel:result is None 2024-09-04T17:49:22.312 INFO:teuthology.orchestra.run.smithi092.stdout:60129542192 2024-09-04T17:49:22.314 INFO:tasks.ceph.ceph_manager.ceph:need seq 442381631510 got 442381631510 for osd.2 2024-09-04T17:49:22.314 DEBUG:teuthology.parallel:result is None 2024-09-04T17:49:22.318 INFO:tasks.ceph.ceph_manager.ceph:need seq 502511173650 got 502511173651 for osd.6 2024-09-04T17:49:22.318 DEBUG:teuthology.parallel:result is None 2024-09-04T17:49:22.330 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542192 got 60129542192 for osd.7 2024-09-04T17:49:22.330 DEBUG:teuthology.parallel:result is None 2024-09-04T17:49:22.342 INFO:teuthology.orchestra.run.smithi092.stdout:64424509487 2024-09-04T17:49:22.342 INFO:teuthology.orchestra.run.smithi092.stdout:206158430244 2024-09-04T17:49:22.352 INFO:teuthology.orchestra.run.smithi092.stdout:64424509487 2024-09-04T17:49:22.370 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509487 got 64424509487 for osd.0 2024-09-04T17:49:22.709 DEBUG:teuthology.parallel:result is None 2024-09-04T17:49:22.710 INFO:tasks.ceph.ceph_manager.ceph:need seq 206158430244 got 206158430244 for osd.3 2024-09-04T17:49:22.711 DEBUG:teuthology.parallel:result is None 2024-09-04T17:49:22.711 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509487 got 64424509487 for osd.5 2024-09-04T17:49:22.711 DEBUG:teuthology.parallel:result is None 2024-09-04T17:49:22.711 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-04T17:49:22.712 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-04T17:49:23.054 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:49:23.055 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-09-04T17:49:23.071 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":302,"stamp":"2024-09-04T17:49:21.130141+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":939,"num_object_clones":0,"num_object_copies":1878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":939,"num_whiteouts":931,"num_read":142346,"num_read_kb":876804,"num_write":20402,"num_write_kb":405906,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6763,"num_bytes_recovered":238961871,"num_keys_recovered":777,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18286,"ondisk_log_size":18286,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":388664,"kb_used_data":19752,"kb_used_omap":46,"kb_used_meta":368657,"kb_avail":754586056,"statfs":{"total":773094113280,"available":772696121344,"internally_reserved":0,"allocated":20226048,"data_stored":10421720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":47946,"internal_metadata":377504950},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.938656"},"pg_stats":[{"pgid":"3.1f","version":"178'501","reported_seq":3131,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-04T17:49:18.214784+0000","last_change":"2024-09-04T17:48:56.059852+0000","last_active":"2024-09-04T17:49:18.214784+0000","last_peered":"2024-09-04T17:49:18.214784+0000","last_clean":"2024-09-04T17:49:18.214784+0000","last_became_active":"2024-09-04T17:48:09.845199+0000","last_became_peered":"2024-09-04T17:48:09.845199+0000","last_unstale":"2024-09-04T17:49:18.214784+0000","last_undegraded":"2024-09-04T17:49:18.214784+0000","last_fullsized":"2024-09-04T17:49:18.214784+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T21:36:09.244561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003726699,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":2983,"num_read_kb":27190,"num_write":524,"num_write_kb":10445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":24433786,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"175'657","reported_seq":3987,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.935494+0000","last_change":"2024-09-04T17:49:05.876073+0000","last_active":"2024-09-04T17:49:19.935494+0000","last_peered":"2024-09-04T17:49:19.935494+0000","last_clean":"2024-09-04T17:49:19.935494+0000","last_became_active":"2024-09-04T17:49:05.874586+0000","last_became_peered":"2024-09-04T17:49:05.874586+0000","last_unstale":"2024-09-04T17:49:19.935494+0000","last_undegraded":"2024-09-04T17:49:19.935494+0000","last_fullsized":"2024-09-04T17:49:19.935494+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:25:15.903921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001926142,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4757,"num_read_kb":30502,"num_write":815,"num_write_kb":10573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":322,"num_bytes_recovered":15412196,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"175'548","reported_seq":2552,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-04T17:49:18.352808+0000","last_change":"2024-09-04T17:48:56.060983+0000","last_active":"2024-09-04T17:49:18.352808+0000","last_peered":"2024-09-04T17:49:18.352808+0000","last_clean":"2024-09-04T17:49:18.352808+0000","last_became_active":"2024-09-04T17:45:55.955213+0000","last_became_peered":"2024-09-04T17:45:55.955213+0000","last_unstale":"2024-09-04T17:49:18.352808+0000","last_undegraded":"2024-09-04T17:49:18.352808+0000","last_fullsized":"2024-09-04T17:49:18.352808+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":548,"log_dups_size":0,"ondisk_log_size":548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T19:29:02.548970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022810360000000002,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":2820,"num_read_kb":20284,"num_write":623,"num_write_kb":9401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"178'758","reported_seq":6824,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-04T17:49:18.352727+0000","last_change":"2024-09-04T17:48:56.055219+0000","last_active":"2024-09-04T17:49:18.352727+0000","last_peered":"2024-09-04T17:49:18.352727+0000","last_clean":"2024-09-04T17:49:18.352727+0000","last_became_active":"2024-09-04T17:45:55.951880+0000","last_became_peered":"2024-09-04T17:45:55.951880+0000","last_unstale":"2024-09-04T17:49:18.352727+0000","last_undegraded":"2024-09-04T17:49:18.352727+0000","last_fullsized":"2024-09-04T17:49:18.352727+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":758,"log_dups_size":0,"ondisk_log_size":758,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:58:30.591123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025730409999999999,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":35,"num_read":7473,"num_read_kb":48018,"num_write":874,"num_write_kb":11556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"175'597","reported_seq":2408,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-04T17:49:18.214870+0000","last_change":"2024-09-04T17:48:56.058271+0000","last_active":"2024-09-04T17:49:18.214870+0000","last_peered":"2024-09-04T17:49:18.214870+0000","last_clean":"2024-09-04T17:49:18.214870+0000","last_became_active":"2024-09-04T17:48:09.853793+0000","last_became_peered":"2024-09-04T17:48:09.853793+0000","last_unstale":"2024-09-04T17:49:18.214870+0000","last_undegraded":"2024-09-04T17:49:18.214870+0000","last_fullsized":"2024-09-04T17:49:18.214870+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":597,"log_dups_size":0,"ondisk_log_size":597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:56:08.051891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026463849999999998,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":3715,"num_read_kb":8148,"num_write":647,"num_write_kb":4600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":1993914,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"175'424","reported_seq":6455,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.240117+0000","last_change":"2024-09-04T17:49:10.140837+0000","last_active":"2024-09-04T17:49:19.240117+0000","last_peered":"2024-09-04T17:49:19.240117+0000","last_clean":"2024-09-04T17:49:19.240117+0000","last_became_active":"2024-09-04T17:49:10.137472+0000","last_became_peered":"2024-09-04T17:49:10.137472+0000","last_unstale":"2024-09-04T17:49:19.240117+0000","last_undegraded":"2024-09-04T17:49:19.240117+0000","last_fullsized":"2024-09-04T17:49:19.240117+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":424,"log_dups_size":0,"ondisk_log_size":424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:51:59.854799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000885438,"stat_sum":{"num_bytes":19,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":25,"num_read":5992,"num_read_kb":45852,"num_write":376,"num_write_kb":15447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":5992020,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"178'442","reported_seq":2720,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.237063+0000","last_change":"2024-09-04T17:49:05.878999+0000","last_active":"2024-09-04T17:49:19.237063+0000","last_peered":"2024-09-04T17:49:19.237063+0000","last_clean":"2024-09-04T17:49:19.237063+0000","last_became_active":"2024-09-04T17:49:05.876985+0000","last_became_peered":"2024-09-04T17:49:05.876985+0000","last_unstale":"2024-09-04T17:49:19.237063+0000","last_undegraded":"2024-09-04T17:49:19.237063+0000","last_fullsized":"2024-09-04T17:49:19.237063+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:42:50.926456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001467744,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":2248,"num_read_kb":16196,"num_write":460,"num_write_kb":4617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":494,"num_bytes_recovered":4911830,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"178'508","reported_seq":2560,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.935295+0000","last_change":"2024-09-04T17:49:07.880474+0000","last_active":"2024-09-04T17:49:19.935295+0000","last_peered":"2024-09-04T17:49:19.935295+0000","last_clean":"2024-09-04T17:49:19.935295+0000","last_became_active":"2024-09-04T17:49:07.878360+0000","last_became_peered":"2024-09-04T17:49:07.878360+0000","last_unstale":"2024-09-04T17:49:19.935295+0000","last_undegraded":"2024-09-04T17:49:19.935295+0000","last_fullsized":"2024-09-04T17:49:19.935295+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:25:31.312054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017798989999999999,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":8892,"num_read_kb":35489,"num_write":443,"num_write_kb":14429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":597,"num_bytes_recovered":28259997,"num_keys_recovered":141,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"178'331","reported_seq":2388,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-04T17:49:18.214976+0000","last_change":"2024-09-04T17:48:56.056526+0000","last_active":"2024-09-04T17:49:18.214976+0000","last_peered":"2024-09-04T17:49:18.214976+0000","last_clean":"2024-09-04T17:49:18.214976+0000","last_became_active":"2024-09-04T17:45:55.958625+0000","last_became_peered":"2024-09-04T17:45:55.958625+0000","last_unstale":"2024-09-04T17:49:18.214976+0000","last_undegraded":"2024-09-04T17:49:18.214976+0000","last_fullsized":"2024-09-04T17:49:18.214976+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":331,"log_dups_size":0,"ondisk_log_size":331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T22:39:37.983710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001574039,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":1612,"num_read_kb":20939,"num_write":316,"num_write_kb":18406,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"175'364","reported_seq":2049,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-04T17:49:18.352973+0000","last_change":"2024-09-04T17:48:56.060132+0000","last_active":"2024-09-04T17:49:18.352973+0000","last_peered":"2024-09-04T17:49:18.352973+0000","last_clean":"2024-09-04T17:49:18.352973+0000","last_became_active":"2024-09-04T17:45:55.953254+0000","last_became_peered":"2024-09-04T17:45:55.953254+0000","last_unstale":"2024-09-04T17:49:18.352973+0000","last_undegraded":"2024-09-04T17:49:18.352973+0000","last_fullsized":"2024-09-04T17:49:18.352973+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":364,"log_dups_size":0,"ondisk_log_size":364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T21:59:17.936385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0044959860000000004,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":1729,"num_read_kb":10785,"num_write":396,"num_write_kb":7391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"174'510","reported_seq":3054,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-04T17:49:18.308234+0000","last_change":"2024-09-04T17:49:06.889185+0000","last_active":"2024-09-04T17:49:18.308234+0000","last_peered":"2024-09-04T17:49:18.308234+0000","last_clean":"2024-09-04T17:49:18.308234+0000","last_became_active":"2024-09-04T17:49:06.884929+0000","last_became_peered":"2024-09-04T17:49:06.884929+0000","last_unstale":"2024-09-04T17:49:18.308234+0000","last_undegraded":"2024-09-04T17:49:18.308234+0000","last_fullsized":"2024-09-04T17:49:18.308234+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":510,"log_dups_size":0,"ondisk_log_size":510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:02:50.836583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001003051,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":4020,"num_read_kb":16975,"num_write":525,"num_write_kb":4439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":446,"num_bytes_recovered":12989343,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"178'775","reported_seq":2528,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.240494+0000","last_change":"2024-09-04T17:49:10.140905+0000","last_active":"2024-09-04T17:49:19.240494+0000","last_peered":"2024-09-04T17:49:19.240494+0000","last_clean":"2024-09-04T17:49:19.240494+0000","last_became_active":"2024-09-04T17:49:10.137573+0000","last_became_peered":"2024-09-04T17:49:10.137573+0000","last_unstale":"2024-09-04T17:49:19.240494+0000","last_undegraded":"2024-09-04T17:49:19.240494+0000","last_fullsized":"2024-09-04T17:49:19.240494+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":775,"log_dups_size":0,"ondisk_log_size":775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:56:50.281112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027107059999999998,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":29,"num_read":4009,"num_read_kb":29862,"num_write":1001,"num_write_kb":35909,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":235,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.080313+0000","last_change":"2024-09-04T17:48:11.850550+0000","last_active":"2024-09-04T17:49:20.080313+0000","last_peered":"2024-09-04T17:49:20.080313+0000","last_clean":"2024-09-04T17:49:20.080313+0000","last_became_active":"2024-09-04T17:48:11.849710+0000","last_became_peered":"2024-09-04T17:48:11.849710+0000","last_unstale":"2024-09-04T17:49:20.080313+0000","last_undegraded":"2024-09-04T17:49:20.080313+0000","last_fullsized":"2024-09-04T17:49:20.080313+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:37:18.791459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001226074,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"178'409","reported_seq":3428,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-04T17:49:18.215458+0000","last_change":"2024-09-04T17:49:07.876177+0000","last_active":"2024-09-04T17:49:18.215458+0000","last_peered":"2024-09-04T17:49:18.215458+0000","last_clean":"2024-09-04T17:49:18.215458+0000","last_became_active":"2024-09-04T17:49:07.874501+0000","last_became_peered":"2024-09-04T17:49:07.874501+0000","last_unstale":"2024-09-04T17:49:18.215458+0000","last_undegraded":"2024-09-04T17:49:18.215458+0000","last_fullsized":"2024-09-04T17:49:18.215458+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":409,"log_dups_size":0,"ondisk_log_size":409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:52:42.831486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001494427,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":2485,"num_read_kb":14550,"num_write":410,"num_write_kb":5528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":919873,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":40,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.237456+0000","last_change":"2024-09-04T17:49:03.886949+0000","last_active":"2024-09-04T17:49:19.237456+0000","last_peered":"2024-09-04T17:49:19.237456+0000","last_clean":"2024-09-04T17:49:19.237456+0000","last_became_active":"2024-09-04T17:49:03.886525+0000","last_became_peered":"2024-09-04T17:49:03.886525+0000","last_unstale":"2024-09-04T17:49:19.237456+0000","last_undegraded":"2024-09-04T17:49:19.237456+0000","last_fullsized":"2024-09-04T17:49:19.237456+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:24:57.789704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011364179999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"175'469","reported_seq":3004,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.935427+0000","last_change":"2024-09-04T17:49:05.877106+0000","last_active":"2024-09-04T17:49:19.935427+0000","last_peered":"2024-09-04T17:49:19.935427+0000","last_clean":"2024-09-04T17:49:19.935427+0000","last_became_active":"2024-09-04T17:49:05.874579+0000","last_became_peered":"2024-09-04T17:49:05.874579+0000","last_unstale":"2024-09-04T17:49:19.935427+0000","last_undegraded":"2024-09-04T17:49:19.935427+0000","last_fullsized":"2024-09-04T17:49:19.935427+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":469,"log_dups_size":0,"ondisk_log_size":469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:32:31.466355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019775819999999999,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":2953,"num_read_kb":23389,"num_write":575,"num_write_kb":21569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":313,"num_bytes_recovered":1588111,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":370,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-04T17:49:18.308514+0000","last_change":"2024-09-04T17:45:49.476607+0000","last_active":"2024-09-04T17:49:18.308514+0000","last_peered":"2024-09-04T17:49:18.308514+0000","last_clean":"2024-09-04T17:49:18.308514+0000","last_became_active":"2024-09-04T17:45:47.369308+0000","last_became_peered":"2024-09-04T17:45:47.369308+0000","last_unstale":"2024-09-04T17:49:18.308514+0000","last_undegraded":"2024-09-04T17:49:18.308514+0000","last_fullsized":"2024-09-04T17:49:18.308514+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:17:13.378553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001029826,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"175'410","reported_seq":2886,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.237416+0000","last_change":"2024-09-04T17:49:07.879430+0000","last_active":"2024-09-04T17:49:19.237416+0000","last_peered":"2024-09-04T17:49:19.237416+0000","last_clean":"2024-09-04T17:49:19.237416+0000","last_became_active":"2024-09-04T17:49:07.877184+0000","last_became_peered":"2024-09-04T17:49:07.877184+0000","last_unstale":"2024-09-04T17:49:19.237416+0000","last_undegraded":"2024-09-04T17:49:19.237416+0000","last_fullsized":"2024-09-04T17:49:19.237416+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T21:09:45.025593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029821800000000001,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":31,"num_read":2672,"num_read_kb":18782,"num_write":436,"num_write_kb":9856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":13375622,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.237128+0000","last_change":"2024-09-04T17:49:03.886170+0000","last_active":"2024-09-04T17:49:19.237128+0000","last_peered":"2024-09-04T17:49:19.237128+0000","last_clean":"2024-09-04T17:49:19.237128+0000","last_became_active":"2024-09-04T17:49:03.885776+0000","last_became_peered":"2024-09-04T17:49:03.885776+0000","last_unstale":"2024-09-04T17:49:19.237128+0000","last_undegraded":"2024-09-04T17:49:19.237128+0000","last_fullsized":"2024-09-04T17:49:19.237128+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T19:30:49.636681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"175'358","reported_seq":3040,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-04T17:49:18.352982+0000","last_change":"2024-09-04T17:49:06.887617+0000","last_active":"2024-09-04T17:49:18.352982+0000","last_peered":"2024-09-04T17:49:18.352982+0000","last_clean":"2024-09-04T17:49:18.352982+0000","last_became_active":"2024-09-04T17:49:06.885686+0000","last_became_peered":"2024-09-04T17:49:06.885686+0000","last_unstale":"2024-09-04T17:49:18.352982+0000","last_undegraded":"2024-09-04T17:49:18.352982+0000","last_fullsized":"2024-09-04T17:49:18.352982+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T22:44:47.142525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019787199999999998,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":2555,"num_read_kb":7639,"num_write":305,"num_write_kb":7011,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":391,"num_bytes_recovered":15596571,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"178'887","reported_seq":3553,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.240376+0000","last_change":"2024-09-04T17:49:10.141003+0000","last_active":"2024-09-04T17:49:19.240376+0000","last_peered":"2024-09-04T17:49:19.240376+0000","last_clean":"2024-09-04T17:49:19.240376+0000","last_became_active":"2024-09-04T17:49:10.137895+0000","last_became_peered":"2024-09-04T17:49:10.137895+0000","last_unstale":"2024-09-04T17:49:19.240376+0000","last_undegraded":"2024-09-04T17:49:19.240376+0000","last_fullsized":"2024-09-04T17:49:19.240376+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":887,"log_dups_size":0,"ondisk_log_size":887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:24:10.064606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032310310000000001,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":29,"num_read":4673,"num_read_kb":26561,"num_write":1100,"num_write_kb":14222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":396,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-04T17:49:18.352627+0000","last_change":"2024-09-04T17:49:03.892940+0000","last_active":"2024-09-04T17:49:18.352627+0000","last_peered":"2024-09-04T17:49:18.352627+0000","last_clean":"2024-09-04T17:49:18.352627+0000","last_became_active":"2024-09-04T17:49:03.887053+0000","last_became_peered":"2024-09-04T17:49:03.887053+0000","last_unstale":"2024-09-04T17:49:18.352627+0000","last_undegraded":"2024-09-04T17:49:18.352627+0000","last_fullsized":"2024-09-04T17:49:18.352627+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:30:44.144337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026316069999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"175'453","reported_seq":3440,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.237166+0000","last_change":"2024-09-04T17:49:07.878051+0000","last_active":"2024-09-04T17:49:19.237166+0000","last_peered":"2024-09-04T17:49:19.237166+0000","last_clean":"2024-09-04T17:49:19.237166+0000","last_became_active":"2024-09-04T17:49:07.876339+0000","last_became_peered":"2024-09-04T17:49:07.876339+0000","last_unstale":"2024-09-04T17:49:19.237166+0000","last_undegraded":"2024-09-04T17:49:19.237166+0000","last_fullsized":"2024-09-04T17:49:19.237166+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":453,"log_dups_size":0,"ondisk_log_size":453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T22:43:32.300162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00120166,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":36,"num_read":3566,"num_read_kb":22521,"num_write":365,"num_write_kb":4893,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":261,"num_bytes_recovered":5692590,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":370,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.236829+0000","last_change":"2024-09-04T17:49:02.864996+0000","last_active":"2024-09-04T17:49:19.236829+0000","last_peered":"2024-09-04T17:49:19.236829+0000","last_clean":"2024-09-04T17:49:19.236829+0000","last_became_active":"2024-09-04T17:49:02.864647+0000","last_became_peered":"2024-09-04T17:49:02.864647+0000","last_unstale":"2024-09-04T17:49:19.236829+0000","last_undegraded":"2024-09-04T17:49:19.236829+0000","last_fullsized":"2024-09-04T17:49:19.236829+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:41:35.190910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001025195,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"178'637","reported_seq":3373,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-04T17:49:18.214603+0000","last_change":"2024-09-04T17:48:56.058939+0000","last_active":"2024-09-04T17:49:18.214603+0000","last_peered":"2024-09-04T17:49:18.214603+0000","last_clean":"2024-09-04T17:49:18.214603+0000","last_became_active":"2024-09-04T17:48:11.845510+0000","last_became_peered":"2024-09-04T17:48:11.845510+0000","last_unstale":"2024-09-04T17:49:18.214603+0000","last_undegraded":"2024-09-04T17:49:18.214603+0000","last_fullsized":"2024-09-04T17:49:18.214603+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":637,"log_dups_size":0,"ondisk_log_size":637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:14:18.552668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031660479999999999,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":3316,"num_read_kb":40546,"num_write":672,"num_write_kb":23359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":240,"num_bytes_recovered":21297737,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":382,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-04T17:49:18.308065+0000","last_change":"2024-09-04T17:49:03.874392+0000","last_active":"2024-09-04T17:49:18.308065+0000","last_peered":"2024-09-04T17:49:18.308065+0000","last_clean":"2024-09-04T17:49:18.308065+0000","last_became_active":"2024-09-04T17:49:03.870678+0000","last_became_peered":"2024-09-04T17:49:03.870678+0000","last_unstale":"2024-09-04T17:49:18.308065+0000","last_undegraded":"2024-09-04T17:49:18.308065+0000","last_fullsized":"2024-09-04T17:49:18.308065+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:49:38.360230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010553590000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"175'622","reported_seq":3059,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.935307+0000","last_change":"2024-09-04T17:49:07.879558+0000","last_active":"2024-09-04T17:49:19.935307+0000","last_peered":"2024-09-04T17:49:19.935307+0000","last_clean":"2024-09-04T17:49:19.935307+0000","last_became_active":"2024-09-04T17:49:07.876340+0000","last_became_peered":"2024-09-04T17:49:07.876340+0000","last_unstale":"2024-09-04T17:49:19.935307+0000","last_undegraded":"2024-09-04T17:49:19.935307+0000","last_fullsized":"2024-09-04T17:49:19.935307+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":622,"log_dups_size":0,"ondisk_log_size":622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:20:26.042540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021278730000000002,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":5714,"num_read_kb":30160,"num_write":740,"num_write_kb":13818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":405,"num_bytes_recovered":16173004,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":380,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-04T17:49:18.352652+0000","last_change":"2024-09-04T17:47:41.919544+0000","last_active":"2024-09-04T17:49:18.352652+0000","last_peered":"2024-09-04T17:49:18.352652+0000","last_clean":"2024-09-04T17:49:18.352652+0000","last_became_active":"2024-09-04T17:47:41.919027+0000","last_became_peered":"2024-09-04T17:47:41.919027+0000","last_unstale":"2024-09-04T17:49:18.352652+0000","last_undegraded":"2024-09-04T17:49:18.352652+0000","last_fullsized":"2024-09-04T17:49:18.352652+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:42:55.688562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086618299999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":461,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-04T17:49:18.308163+0000","last_change":"2024-09-04T17:45:42.341149+0000","last_active":"2024-09-04T17:49:18.308163+0000","last_peered":"2024-09-04T17:49:18.308163+0000","last_clean":"2024-09-04T17:49:18.308163+0000","last_became_active":"2024-09-04T17:45:42.339286+0000","last_became_peered":"2024-09-04T17:45:42.339286+0000","last_unstale":"2024-09-04T17:49:18.308163+0000","last_undegraded":"2024-09-04T17:49:18.308163+0000","last_fullsized":"2024-09-04T17:49:18.308163+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:41.305128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:41.305128+0000","last_clean_scrub_stamp":"2024-09-04T17:45:41.305128+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:52:24.860049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"178'548","reported_seq":3065,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.935375+0000","last_change":"2024-09-04T17:49:07.883398+0000","last_active":"2024-09-04T17:49:19.935375+0000","last_peered":"2024-09-04T17:49:19.935375+0000","last_clean":"2024-09-04T17:49:19.935375+0000","last_became_active":"2024-09-04T17:49:07.879442+0000","last_became_peered":"2024-09-04T17:49:07.879442+0000","last_unstale":"2024-09-04T17:49:19.935375+0000","last_undegraded":"2024-09-04T17:49:19.935375+0000","last_fullsized":"2024-09-04T17:49:19.935375+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":548,"log_dups_size":0,"ondisk_log_size":548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:55:18.249867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011779189999999999,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":2788,"num_read_kb":17264,"num_write":577,"num_write_kb":29373,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":445,"num_bytes_recovered":17940277,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"178'716","reported_seq":4731,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.240231+0000","last_change":"2024-09-04T17:49:10.137619+0000","last_active":"2024-09-04T17:49:19.240231+0000","last_peered":"2024-09-04T17:49:19.240231+0000","last_clean":"2024-09-04T17:49:19.240231+0000","last_became_active":"2024-09-04T17:49:10.133837+0000","last_became_peered":"2024-09-04T17:49:10.133837+0000","last_unstale":"2024-09-04T17:49:19.240231+0000","last_undegraded":"2024-09-04T17:49:19.240231+0000","last_fullsized":"2024-09-04T17:49:19.240231+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":716,"log_dups_size":0,"ondisk_log_size":716,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T23:43:19.357232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003529346,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":6198,"num_read_kb":37498,"num_write":772,"num_write_kb":18653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":4534479,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"175'656","reported_seq":3645,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.080234+0000","last_change":"2024-09-04T17:48:56.054512+0000","last_active":"2024-09-04T17:49:20.080234+0000","last_peered":"2024-09-04T17:49:20.080234+0000","last_clean":"2024-09-04T17:49:20.080234+0000","last_became_active":"2024-09-04T17:48:00.841070+0000","last_became_peered":"2024-09-04T17:48:00.841070+0000","last_unstale":"2024-09-04T17:49:20.080234+0000","last_undegraded":"2024-09-04T17:49:20.080234+0000","last_fullsized":"2024-09-04T17:49:20.080234+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T19:18:47.901288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003342214,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":31,"num_read":5699,"num_read_kb":16474,"num_write":617,"num_write_kb":2753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":1572433,"num_keys_recovered":66,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"178'770","reported_seq":2984,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.935456+0000","last_change":"2024-09-04T17:49:06.885887+0000","last_active":"2024-09-04T17:49:19.935456+0000","last_peered":"2024-09-04T17:49:19.935456+0000","last_clean":"2024-09-04T17:49:19.935456+0000","last_became_active":"2024-09-04T17:49:06.883685+0000","last_became_peered":"2024-09-04T17:49:06.883685+0000","last_unstale":"2024-09-04T17:49:19.935456+0000","last_undegraded":"2024-09-04T17:49:19.935456+0000","last_fullsized":"2024-09-04T17:49:19.935456+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":770,"log_dups_size":0,"ondisk_log_size":770,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T17:55:41.907650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002090967,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":4267,"num_read_kb":14813,"num_write":915,"num_write_kb":7927,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":459,"num_bytes_recovered":5211501,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"175'647","reported_seq":9164,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-04T17:49:18.352774+0000","last_change":"2024-09-04T17:49:07.884097+0000","last_active":"2024-09-04T17:49:18.352774+0000","last_peered":"2024-09-04T17:49:18.352774+0000","last_clean":"2024-09-04T17:49:18.352774+0000","last_became_active":"2024-09-04T17:49:07.880558+0000","last_became_peered":"2024-09-04T17:49:07.880558+0000","last_unstale":"2024-09-04T17:49:18.352774+0000","last_undegraded":"2024-09-04T17:49:18.352774+0000","last_fullsized":"2024-09-04T17:49:18.352774+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":647,"log_dups_size":0,"ondisk_log_size":647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:43:25.854985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096708700000000005,"stat_sum":{"num_bytes":0,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":34,"num_read":9052,"num_read_kb":71439,"num_write":754,"num_write_kb":16863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":537,"num_bytes_recovered":8281724,"num_keys_recovered":57,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"178'1358","reported_seq":4585,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-04T17:49:18.308408+0000","last_change":"2024-09-04T17:48:56.066067+0000","last_active":"2024-09-04T17:49:18.308408+0000","last_peered":"2024-09-04T17:49:18.308408+0000","last_clean":"2024-09-04T17:49:18.308408+0000","last_became_active":"2024-09-04T17:48:12.850381+0000","last_became_peered":"2024-09-04T17:48:12.850381+0000","last_unstale":"2024-09-04T17:49:18.308408+0000","last_undegraded":"2024-09-04T17:49:18.308408+0000","last_fullsized":"2024-09-04T17:49:18.308408+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":1358,"log_dups_size":0,"ondisk_log_size":1358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T18:53:56.503643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002193296,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":31,"num_read":8863,"num_read_kb":26532,"num_write":2254,"num_write_kb":13198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":179,"num_bytes_recovered":18877339,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"178'541","reported_seq":6076,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-04T17:49:18.308579+0000","last_change":"2024-09-04T17:48:56.056521+0000","last_active":"2024-09-04T17:49:18.308579+0000","last_peered":"2024-09-04T17:49:18.308579+0000","last_clean":"2024-09-04T17:49:18.308579+0000","last_became_active":"2024-09-04T17:48:11.849605+0000","last_became_peered":"2024-09-04T17:48:11.849605+0000","last_unstale":"2024-09-04T17:49:18.308579+0000","last_undegraded":"2024-09-04T17:49:18.308579+0000","last_fullsized":"2024-09-04T17:49:18.308579+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":541,"log_dups_size":0,"ondisk_log_size":541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:57:37.250572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001652911,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":6678,"num_read_kb":28511,"num_write":523,"num_write_kb":11228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":7806862,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"178'348","reported_seq":3231,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-04T17:49:18.308740+0000","last_change":"2024-09-04T17:48:56.063351+0000","last_active":"2024-09-04T17:49:18.308740+0000","last_peered":"2024-09-04T17:49:18.308740+0000","last_clean":"2024-09-04T17:49:18.308740+0000","last_became_active":"2024-09-04T17:45:55.956734+0000","last_became_peered":"2024-09-04T17:45:55.956734+0000","last_unstale":"2024-09-04T17:49:18.308740+0000","last_undegraded":"2024-09-04T17:49:18.308740+0000","last_fullsized":"2024-09-04T17:49:18.308740+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T23:04:26.785513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0065435850000000002,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":2605,"num_read_kb":44969,"num_write":330,"num_write_kb":13427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"175'630","reported_seq":3858,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.080256+0000","last_change":"2024-09-04T17:48:56.053981+0000","last_active":"2024-09-04T17:49:20.080256+0000","last_peered":"2024-09-04T17:49:20.080256+0000","last_clean":"2024-09-04T17:49:20.080256+0000","last_became_active":"2024-09-04T17:48:00.843136+0000","last_became_peered":"2024-09-04T17:48:00.843136+0000","last_unstale":"2024-09-04T17:49:20.080256+0000","last_undegraded":"2024-09-04T17:49:20.080256+0000","last_fullsized":"2024-09-04T17:49:20.080256+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":630,"log_dups_size":0,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T23:36:00.802962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025156979999999998,"stat_sum":{"num_bytes":0,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":37,"num_read":4786,"num_read_kb":18960,"num_write":673,"num_write_kb":12854,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":3377599,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"175'599","reported_seq":6340,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-04T17:49:18.308074+0000","last_change":"2024-09-04T17:48:56.055370+0000","last_active":"2024-09-04T17:49:18.308074+0000","last_peered":"2024-09-04T17:49:18.308074+0000","last_clean":"2024-09-04T17:49:18.308074+0000","last_became_active":"2024-09-04T17:45:55.951870+0000","last_became_peered":"2024-09-04T17:45:55.951870+0000","last_unstale":"2024-09-04T17:49:18.308074+0000","last_undegraded":"2024-09-04T17:49:18.308074+0000","last_fullsized":"2024-09-04T17:49:18.308074+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:32:28.517472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012658840000000001,"stat_sum":{"num_bytes":0,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":38,"num_read":7163,"num_read_kb":61439,"num_write":584,"num_write_kb":17576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"175'425","reported_seq":4223,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-04T17:49:18.215290+0000","last_change":"2024-09-04T17:48:56.055907+0000","last_active":"2024-09-04T17:49:18.215290+0000","last_peered":"2024-09-04T17:49:18.215290+0000","last_clean":"2024-09-04T17:49:18.215290+0000","last_became_active":"2024-09-04T17:48:14.943778+0000","last_became_peered":"2024-09-04T17:48:14.943778+0000","last_unstale":"2024-09-04T17:49:18.215290+0000","last_undegraded":"2024-09-04T17:49:18.215290+0000","last_fullsized":"2024-09-04T17:49:18.215290+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":425,"log_dups_size":0,"ondisk_log_size":425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:28:33.747314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011847660000000001,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":31,"num_read":3577,"num_read_kb":21719,"num_write":447,"num_write_kb":8575,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":290,"num_bytes_recovered":2723025,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"175'313","reported_seq":2693,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-04T17:49:18.308158+0000","last_change":"2024-09-04T17:48:56.055871+0000","last_active":"2024-09-04T17:49:18.308158+0000","last_peered":"2024-09-04T17:49:18.308158+0000","last_clean":"2024-09-04T17:49:18.308158+0000","last_became_active":"2024-09-04T17:45:55.954544+0000","last_became_peered":"2024-09-04T17:45:55.954544+0000","last_unstale":"2024-09-04T17:49:18.308158+0000","last_undegraded":"2024-09-04T17:49:18.308158+0000","last_fullsized":"2024-09-04T17:49:18.308158+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:20:57.091315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001451911,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":2420,"num_read_kb":22743,"num_write":250,"num_write_kb":4380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":935,"num_object_clones":0,"num_object_copies":1870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":935,"num_whiteouts":931,"num_read":142280,"num_read_kb":876749,"num_write":20299,"num_write_kb":404276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6761,"num_bytes_recovered":238961833,"num_keys_recovered":777,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21395,"internal_metadata":0},"log_size":18211,"ondisk_log_size":18211,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":117,"seq":502511173651,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44384,"kb_used_data":2316,"kb_used_omap":22,"kb_used_meta":42025,"kb_avail":94327456,"statfs":{"total":96636764160,"available":96591314944,"internally_reserved":0,"allocated":2371584,"data_stored":1152207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22742,"internal_metadata":43034410},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509487,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51568,"kb_used_data":2320,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320272,"statfs":{"total":96636764160,"available":96583958528,"internally_reserved":0,"allocated":2375680,"data_stored":1152226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509488,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51572,"kb_used_data":2332,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320268,"statfs":{"total":96636764160,"available":96583954432,"internally_reserved":0,"allocated":2387968,"data_stored":1158039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":48,"seq":206158430244,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60848,"kb_used_data":2328,"kb_used_omap":4,"kb_used_meta":58491,"kb_avail":94310992,"statfs":{"total":96636764160,"available":96574455808,"internally_reserved":0,"allocated":2383872,"data_stored":1158020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4430,"internal_metadata":59895474},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509487,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29368,"kb_used_data":2336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342472,"statfs":{"total":96636764160,"available":96606691328,"internally_reserved":0,"allocated":2392064,"data_stored":1158058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509487,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60464,"kb_used_data":2896,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311376,"statfs":{"total":96636764160,"available":96574849024,"internally_reserved":0,"allocated":2965504,"data_stored":1742575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542192,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60460,"kb_used_data":2896,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311380,"statfs":{"total":96636764160,"available":96574853120,"internally_reserved":0,"allocated":2965504,"data_stored":1742575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":103,"seq":442381631510,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30000,"kb_used_data":2328,"kb_used_omap":12,"kb_used_meta":27635,"kb_avail":94341840,"statfs":{"total":96636764160,"available":96606044160,"internally_reserved":0,"allocated":2383872,"data_stored":1158020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12804,"internal_metadata":28298748},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6121,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15274,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-04T17:49:23.074 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-04T17:49:23.366 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:49:23.367 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-09-04T17:49:23.383 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":303,"stamp":"2024-09-04T17:49:23.130883+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":939,"num_object_clones":0,"num_object_copies":1878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":939,"num_whiteouts":931,"num_read":142346,"num_read_kb":876804,"num_write":20402,"num_write_kb":405906,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6763,"num_bytes_recovered":238961871,"num_keys_recovered":777,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18286,"ondisk_log_size":18286,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":388664,"kb_used_data":19764,"kb_used_omap":46,"kb_used_meta":368657,"kb_avail":754586056,"statfs":{"total":773094113280,"available":772696121344,"internally_reserved":0,"allocated":20238336,"data_stored":10427533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":47946,"internal_metadata":377504950},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001829"},"pg_stats":[{"pgid":"3.1f","version":"178'501","reported_seq":3133,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.470279+0000","last_change":"2024-09-04T17:48:56.059852+0000","last_active":"2024-09-04T17:49:20.470279+0000","last_peered":"2024-09-04T17:49:20.470279+0000","last_clean":"2024-09-04T17:49:20.470279+0000","last_became_active":"2024-09-04T17:48:09.845199+0000","last_became_peered":"2024-09-04T17:48:09.845199+0000","last_unstale":"2024-09-04T17:49:20.470279+0000","last_undegraded":"2024-09-04T17:49:20.470279+0000","last_fullsized":"2024-09-04T17:49:20.470279+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T21:36:09.244561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003726699,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":2983,"num_read_kb":27190,"num_write":524,"num_write_kb":10445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":24433786,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"175'657","reported_seq":3987,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.935494+0000","last_change":"2024-09-04T17:49:05.876073+0000","last_active":"2024-09-04T17:49:19.935494+0000","last_peered":"2024-09-04T17:49:19.935494+0000","last_clean":"2024-09-04T17:49:19.935494+0000","last_became_active":"2024-09-04T17:49:05.874586+0000","last_became_peered":"2024-09-04T17:49:05.874586+0000","last_unstale":"2024-09-04T17:49:19.935494+0000","last_undegraded":"2024-09-04T17:49:19.935494+0000","last_fullsized":"2024-09-04T17:49:19.935494+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:25:15.903921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001926142,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4757,"num_read_kb":30502,"num_write":815,"num_write_kb":10573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":322,"num_bytes_recovered":15412196,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"175'548","reported_seq":2554,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.187160+0000","last_change":"2024-09-04T17:48:56.060983+0000","last_active":"2024-09-04T17:49:20.187160+0000","last_peered":"2024-09-04T17:49:20.187160+0000","last_clean":"2024-09-04T17:49:20.187160+0000","last_became_active":"2024-09-04T17:45:55.955213+0000","last_became_peered":"2024-09-04T17:45:55.955213+0000","last_unstale":"2024-09-04T17:49:20.187160+0000","last_undegraded":"2024-09-04T17:49:20.187160+0000","last_fullsized":"2024-09-04T17:49:20.187160+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":548,"log_dups_size":0,"ondisk_log_size":548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T19:29:02.548970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022810360000000002,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":2820,"num_read_kb":20284,"num_write":623,"num_write_kb":9401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"178'758","reported_seq":6826,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.187131+0000","last_change":"2024-09-04T17:48:56.055219+0000","last_active":"2024-09-04T17:49:20.187131+0000","last_peered":"2024-09-04T17:49:20.187131+0000","last_clean":"2024-09-04T17:49:20.187131+0000","last_became_active":"2024-09-04T17:45:55.951880+0000","last_became_peered":"2024-09-04T17:45:55.951880+0000","last_unstale":"2024-09-04T17:49:20.187131+0000","last_undegraded":"2024-09-04T17:49:20.187131+0000","last_fullsized":"2024-09-04T17:49:20.187131+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":758,"log_dups_size":0,"ondisk_log_size":758,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:58:30.591123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025730409999999999,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":35,"num_read":7473,"num_read_kb":48018,"num_write":874,"num_write_kb":11556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"175'597","reported_seq":2410,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.469937+0000","last_change":"2024-09-04T17:48:56.058271+0000","last_active":"2024-09-04T17:49:20.469937+0000","last_peered":"2024-09-04T17:49:20.469937+0000","last_clean":"2024-09-04T17:49:20.469937+0000","last_became_active":"2024-09-04T17:48:09.853793+0000","last_became_peered":"2024-09-04T17:48:09.853793+0000","last_unstale":"2024-09-04T17:49:20.469937+0000","last_undegraded":"2024-09-04T17:49:20.469937+0000","last_fullsized":"2024-09-04T17:49:20.469937+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":597,"log_dups_size":0,"ondisk_log_size":597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:56:08.051891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026463849999999998,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":3715,"num_read_kb":8148,"num_write":647,"num_write_kb":4600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":1993914,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"175'424","reported_seq":6455,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.240117+0000","last_change":"2024-09-04T17:49:10.140837+0000","last_active":"2024-09-04T17:49:19.240117+0000","last_peered":"2024-09-04T17:49:19.240117+0000","last_clean":"2024-09-04T17:49:19.240117+0000","last_became_active":"2024-09-04T17:49:10.137472+0000","last_became_peered":"2024-09-04T17:49:10.137472+0000","last_unstale":"2024-09-04T17:49:19.240117+0000","last_undegraded":"2024-09-04T17:49:19.240117+0000","last_fullsized":"2024-09-04T17:49:19.240117+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":424,"log_dups_size":0,"ondisk_log_size":424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:51:59.854799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000885438,"stat_sum":{"num_bytes":19,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":25,"num_read":5992,"num_read_kb":45852,"num_write":376,"num_write_kb":15447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":5992020,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"178'442","reported_seq":2720,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.237063+0000","last_change":"2024-09-04T17:49:05.878999+0000","last_active":"2024-09-04T17:49:19.237063+0000","last_peered":"2024-09-04T17:49:19.237063+0000","last_clean":"2024-09-04T17:49:19.237063+0000","last_became_active":"2024-09-04T17:49:05.876985+0000","last_became_peered":"2024-09-04T17:49:05.876985+0000","last_unstale":"2024-09-04T17:49:19.237063+0000","last_undegraded":"2024-09-04T17:49:19.237063+0000","last_fullsized":"2024-09-04T17:49:19.237063+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:42:50.926456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001467744,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":2248,"num_read_kb":16196,"num_write":460,"num_write_kb":4617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":494,"num_bytes_recovered":4911830,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"178'508","reported_seq":2560,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.935295+0000","last_change":"2024-09-04T17:49:07.880474+0000","last_active":"2024-09-04T17:49:19.935295+0000","last_peered":"2024-09-04T17:49:19.935295+0000","last_clean":"2024-09-04T17:49:19.935295+0000","last_became_active":"2024-09-04T17:49:07.878360+0000","last_became_peered":"2024-09-04T17:49:07.878360+0000","last_unstale":"2024-09-04T17:49:19.935295+0000","last_undegraded":"2024-09-04T17:49:19.935295+0000","last_fullsized":"2024-09-04T17:49:19.935295+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:25:31.312054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017798989999999999,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":8892,"num_read_kb":35489,"num_write":443,"num_write_kb":14429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":597,"num_bytes_recovered":28259997,"num_keys_recovered":141,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"178'331","reported_seq":2390,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.470375+0000","last_change":"2024-09-04T17:48:56.056526+0000","last_active":"2024-09-04T17:49:20.470375+0000","last_peered":"2024-09-04T17:49:20.470375+0000","last_clean":"2024-09-04T17:49:20.470375+0000","last_became_active":"2024-09-04T17:45:55.958625+0000","last_became_peered":"2024-09-04T17:45:55.958625+0000","last_unstale":"2024-09-04T17:49:20.470375+0000","last_undegraded":"2024-09-04T17:49:20.470375+0000","last_fullsized":"2024-09-04T17:49:20.470375+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":331,"log_dups_size":0,"ondisk_log_size":331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T22:39:37.983710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001574039,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":1612,"num_read_kb":20939,"num_write":316,"num_write_kb":18406,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"175'364","reported_seq":2051,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.187266+0000","last_change":"2024-09-04T17:48:56.060132+0000","last_active":"2024-09-04T17:49:20.187266+0000","last_peered":"2024-09-04T17:49:20.187266+0000","last_clean":"2024-09-04T17:49:20.187266+0000","last_became_active":"2024-09-04T17:45:55.953254+0000","last_became_peered":"2024-09-04T17:45:55.953254+0000","last_unstale":"2024-09-04T17:49:20.187266+0000","last_undegraded":"2024-09-04T17:49:20.187266+0000","last_fullsized":"2024-09-04T17:49:20.187266+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":364,"log_dups_size":0,"ondisk_log_size":364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T21:59:17.936385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0044959860000000004,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":1729,"num_read_kb":10785,"num_write":396,"num_write_kb":7391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"174'510","reported_seq":3054,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-04T17:49:18.308234+0000","last_change":"2024-09-04T17:49:06.889185+0000","last_active":"2024-09-04T17:49:18.308234+0000","last_peered":"2024-09-04T17:49:18.308234+0000","last_clean":"2024-09-04T17:49:18.308234+0000","last_became_active":"2024-09-04T17:49:06.884929+0000","last_became_peered":"2024-09-04T17:49:06.884929+0000","last_unstale":"2024-09-04T17:49:18.308234+0000","last_undegraded":"2024-09-04T17:49:18.308234+0000","last_fullsized":"2024-09-04T17:49:18.308234+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":510,"log_dups_size":0,"ondisk_log_size":510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:02:50.836583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001003051,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":4020,"num_read_kb":16975,"num_write":525,"num_write_kb":4439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":446,"num_bytes_recovered":12989343,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"178'775","reported_seq":2528,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.240494+0000","last_change":"2024-09-04T17:49:10.140905+0000","last_active":"2024-09-04T17:49:19.240494+0000","last_peered":"2024-09-04T17:49:19.240494+0000","last_clean":"2024-09-04T17:49:19.240494+0000","last_became_active":"2024-09-04T17:49:10.137573+0000","last_became_peered":"2024-09-04T17:49:10.137573+0000","last_unstale":"2024-09-04T17:49:19.240494+0000","last_undegraded":"2024-09-04T17:49:19.240494+0000","last_fullsized":"2024-09-04T17:49:19.240494+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":775,"log_dups_size":0,"ondisk_log_size":775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:56:50.281112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027107059999999998,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":29,"num_read":4009,"num_read_kb":29862,"num_write":1001,"num_write_kb":35909,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":235,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.080313+0000","last_change":"2024-09-04T17:48:11.850550+0000","last_active":"2024-09-04T17:49:20.080313+0000","last_peered":"2024-09-04T17:49:20.080313+0000","last_clean":"2024-09-04T17:49:20.080313+0000","last_became_active":"2024-09-04T17:48:11.849710+0000","last_became_peered":"2024-09-04T17:48:11.849710+0000","last_unstale":"2024-09-04T17:49:20.080313+0000","last_undegraded":"2024-09-04T17:49:20.080313+0000","last_fullsized":"2024-09-04T17:49:20.080313+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:37:18.791459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001226074,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"178'409","reported_seq":3430,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.470300+0000","last_change":"2024-09-04T17:49:07.876177+0000","last_active":"2024-09-04T17:49:20.470300+0000","last_peered":"2024-09-04T17:49:20.470300+0000","last_clean":"2024-09-04T17:49:20.470300+0000","last_became_active":"2024-09-04T17:49:07.874501+0000","last_became_peered":"2024-09-04T17:49:07.874501+0000","last_unstale":"2024-09-04T17:49:20.470300+0000","last_undegraded":"2024-09-04T17:49:20.470300+0000","last_fullsized":"2024-09-04T17:49:20.470300+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":409,"log_dups_size":0,"ondisk_log_size":409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:52:42.831486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001494427,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":2485,"num_read_kb":14550,"num_write":410,"num_write_kb":5528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":919873,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":40,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.237456+0000","last_change":"2024-09-04T17:49:03.886949+0000","last_active":"2024-09-04T17:49:19.237456+0000","last_peered":"2024-09-04T17:49:19.237456+0000","last_clean":"2024-09-04T17:49:19.237456+0000","last_became_active":"2024-09-04T17:49:03.886525+0000","last_became_peered":"2024-09-04T17:49:03.886525+0000","last_unstale":"2024-09-04T17:49:19.237456+0000","last_undegraded":"2024-09-04T17:49:19.237456+0000","last_fullsized":"2024-09-04T17:49:19.237456+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:24:57.789704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011364179999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"175'469","reported_seq":3004,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.935427+0000","last_change":"2024-09-04T17:49:05.877106+0000","last_active":"2024-09-04T17:49:19.935427+0000","last_peered":"2024-09-04T17:49:19.935427+0000","last_clean":"2024-09-04T17:49:19.935427+0000","last_became_active":"2024-09-04T17:49:05.874579+0000","last_became_peered":"2024-09-04T17:49:05.874579+0000","last_unstale":"2024-09-04T17:49:19.935427+0000","last_undegraded":"2024-09-04T17:49:19.935427+0000","last_fullsized":"2024-09-04T17:49:19.935427+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":469,"log_dups_size":0,"ondisk_log_size":469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:32:31.466355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019775819999999999,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":2953,"num_read_kb":23389,"num_write":575,"num_write_kb":21569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":313,"num_bytes_recovered":1588111,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":370,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-04T17:49:18.308514+0000","last_change":"2024-09-04T17:45:49.476607+0000","last_active":"2024-09-04T17:49:18.308514+0000","last_peered":"2024-09-04T17:49:18.308514+0000","last_clean":"2024-09-04T17:49:18.308514+0000","last_became_active":"2024-09-04T17:45:47.369308+0000","last_became_peered":"2024-09-04T17:45:47.369308+0000","last_unstale":"2024-09-04T17:49:18.308514+0000","last_undegraded":"2024-09-04T17:49:18.308514+0000","last_fullsized":"2024-09-04T17:49:18.308514+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:17:13.378553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001029826,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"175'410","reported_seq":2886,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.237416+0000","last_change":"2024-09-04T17:49:07.879430+0000","last_active":"2024-09-04T17:49:19.237416+0000","last_peered":"2024-09-04T17:49:19.237416+0000","last_clean":"2024-09-04T17:49:19.237416+0000","last_became_active":"2024-09-04T17:49:07.877184+0000","last_became_peered":"2024-09-04T17:49:07.877184+0000","last_unstale":"2024-09-04T17:49:19.237416+0000","last_undegraded":"2024-09-04T17:49:19.237416+0000","last_fullsized":"2024-09-04T17:49:19.237416+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T21:09:45.025593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029821800000000001,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":31,"num_read":2672,"num_read_kb":18782,"num_write":436,"num_write_kb":9856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":13375622,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.237128+0000","last_change":"2024-09-04T17:49:03.886170+0000","last_active":"2024-09-04T17:49:19.237128+0000","last_peered":"2024-09-04T17:49:19.237128+0000","last_clean":"2024-09-04T17:49:19.237128+0000","last_became_active":"2024-09-04T17:49:03.885776+0000","last_became_peered":"2024-09-04T17:49:03.885776+0000","last_unstale":"2024-09-04T17:49:19.237128+0000","last_undegraded":"2024-09-04T17:49:19.237128+0000","last_fullsized":"2024-09-04T17:49:19.237128+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T19:30:49.636681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"175'358","reported_seq":3042,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.187305+0000","last_change":"2024-09-04T17:49:06.887617+0000","last_active":"2024-09-04T17:49:20.187305+0000","last_peered":"2024-09-04T17:49:20.187305+0000","last_clean":"2024-09-04T17:49:20.187305+0000","last_became_active":"2024-09-04T17:49:06.885686+0000","last_became_peered":"2024-09-04T17:49:06.885686+0000","last_unstale":"2024-09-04T17:49:20.187305+0000","last_undegraded":"2024-09-04T17:49:20.187305+0000","last_fullsized":"2024-09-04T17:49:20.187305+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T22:44:47.142525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019787199999999998,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":2555,"num_read_kb":7639,"num_write":305,"num_write_kb":7011,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":391,"num_bytes_recovered":15596571,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"178'887","reported_seq":3553,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.240376+0000","last_change":"2024-09-04T17:49:10.141003+0000","last_active":"2024-09-04T17:49:19.240376+0000","last_peered":"2024-09-04T17:49:19.240376+0000","last_clean":"2024-09-04T17:49:19.240376+0000","last_became_active":"2024-09-04T17:49:10.137895+0000","last_became_peered":"2024-09-04T17:49:10.137895+0000","last_unstale":"2024-09-04T17:49:19.240376+0000","last_undegraded":"2024-09-04T17:49:19.240376+0000","last_fullsized":"2024-09-04T17:49:19.240376+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":887,"log_dups_size":0,"ondisk_log_size":887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:24:10.064606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032310310000000001,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":29,"num_read":4673,"num_read_kb":26561,"num_write":1100,"num_write_kb":14222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":398,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.186827+0000","last_change":"2024-09-04T17:49:03.892940+0000","last_active":"2024-09-04T17:49:20.186827+0000","last_peered":"2024-09-04T17:49:20.186827+0000","last_clean":"2024-09-04T17:49:20.186827+0000","last_became_active":"2024-09-04T17:49:03.887053+0000","last_became_peered":"2024-09-04T17:49:03.887053+0000","last_unstale":"2024-09-04T17:49:20.186827+0000","last_undegraded":"2024-09-04T17:49:20.186827+0000","last_fullsized":"2024-09-04T17:49:20.186827+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:30:44.144337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026316069999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"175'453","reported_seq":3440,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.237166+0000","last_change":"2024-09-04T17:49:07.878051+0000","last_active":"2024-09-04T17:49:19.237166+0000","last_peered":"2024-09-04T17:49:19.237166+0000","last_clean":"2024-09-04T17:49:19.237166+0000","last_became_active":"2024-09-04T17:49:07.876339+0000","last_became_peered":"2024-09-04T17:49:07.876339+0000","last_unstale":"2024-09-04T17:49:19.237166+0000","last_undegraded":"2024-09-04T17:49:19.237166+0000","last_fullsized":"2024-09-04T17:49:19.237166+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":453,"log_dups_size":0,"ondisk_log_size":453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T22:43:32.300162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00120166,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":36,"num_read":3566,"num_read_kb":22521,"num_write":365,"num_write_kb":4893,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":261,"num_bytes_recovered":5692590,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":370,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.236829+0000","last_change":"2024-09-04T17:49:02.864996+0000","last_active":"2024-09-04T17:49:19.236829+0000","last_peered":"2024-09-04T17:49:19.236829+0000","last_clean":"2024-09-04T17:49:19.236829+0000","last_became_active":"2024-09-04T17:49:02.864647+0000","last_became_peered":"2024-09-04T17:49:02.864647+0000","last_unstale":"2024-09-04T17:49:19.236829+0000","last_undegraded":"2024-09-04T17:49:19.236829+0000","last_fullsized":"2024-09-04T17:49:19.236829+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:41:35.190910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001025195,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"178'637","reported_seq":3375,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.469830+0000","last_change":"2024-09-04T17:48:56.058939+0000","last_active":"2024-09-04T17:49:20.469830+0000","last_peered":"2024-09-04T17:49:20.469830+0000","last_clean":"2024-09-04T17:49:20.469830+0000","last_became_active":"2024-09-04T17:48:11.845510+0000","last_became_peered":"2024-09-04T17:48:11.845510+0000","last_unstale":"2024-09-04T17:49:20.469830+0000","last_undegraded":"2024-09-04T17:49:20.469830+0000","last_fullsized":"2024-09-04T17:49:20.469830+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":637,"log_dups_size":0,"ondisk_log_size":637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:14:18.552668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031660479999999999,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":3316,"num_read_kb":40546,"num_write":672,"num_write_kb":23359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":240,"num_bytes_recovered":21297737,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":382,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-04T17:49:18.308065+0000","last_change":"2024-09-04T17:49:03.874392+0000","last_active":"2024-09-04T17:49:18.308065+0000","last_peered":"2024-09-04T17:49:18.308065+0000","last_clean":"2024-09-04T17:49:18.308065+0000","last_became_active":"2024-09-04T17:49:03.870678+0000","last_became_peered":"2024-09-04T17:49:03.870678+0000","last_unstale":"2024-09-04T17:49:18.308065+0000","last_undegraded":"2024-09-04T17:49:18.308065+0000","last_fullsized":"2024-09-04T17:49:18.308065+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:49:38.360230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010553590000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"175'622","reported_seq":3059,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.935307+0000","last_change":"2024-09-04T17:49:07.879558+0000","last_active":"2024-09-04T17:49:19.935307+0000","last_peered":"2024-09-04T17:49:19.935307+0000","last_clean":"2024-09-04T17:49:19.935307+0000","last_became_active":"2024-09-04T17:49:07.876340+0000","last_became_peered":"2024-09-04T17:49:07.876340+0000","last_unstale":"2024-09-04T17:49:19.935307+0000","last_undegraded":"2024-09-04T17:49:19.935307+0000","last_fullsized":"2024-09-04T17:49:19.935307+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":622,"log_dups_size":0,"ondisk_log_size":622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:20:26.042540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021278730000000002,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":5714,"num_read_kb":30160,"num_write":740,"num_write_kb":13818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":405,"num_bytes_recovered":16173004,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":382,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.186955+0000","last_change":"2024-09-04T17:47:41.919544+0000","last_active":"2024-09-04T17:49:20.186955+0000","last_peered":"2024-09-04T17:49:20.186955+0000","last_clean":"2024-09-04T17:49:20.186955+0000","last_became_active":"2024-09-04T17:47:41.919027+0000","last_became_peered":"2024-09-04T17:47:41.919027+0000","last_unstale":"2024-09-04T17:49:20.186955+0000","last_undegraded":"2024-09-04T17:49:20.186955+0000","last_fullsized":"2024-09-04T17:49:20.186955+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:42:55.688562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086618299999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":461,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-04T17:49:18.308163+0000","last_change":"2024-09-04T17:45:42.341149+0000","last_active":"2024-09-04T17:49:18.308163+0000","last_peered":"2024-09-04T17:49:18.308163+0000","last_clean":"2024-09-04T17:49:18.308163+0000","last_became_active":"2024-09-04T17:45:42.339286+0000","last_became_peered":"2024-09-04T17:45:42.339286+0000","last_unstale":"2024-09-04T17:49:18.308163+0000","last_undegraded":"2024-09-04T17:49:18.308163+0000","last_fullsized":"2024-09-04T17:49:18.308163+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:41.305128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:41.305128+0000","last_clean_scrub_stamp":"2024-09-04T17:45:41.305128+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:52:24.860049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"178'548","reported_seq":3065,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.935375+0000","last_change":"2024-09-04T17:49:07.883398+0000","last_active":"2024-09-04T17:49:19.935375+0000","last_peered":"2024-09-04T17:49:19.935375+0000","last_clean":"2024-09-04T17:49:19.935375+0000","last_became_active":"2024-09-04T17:49:07.879442+0000","last_became_peered":"2024-09-04T17:49:07.879442+0000","last_unstale":"2024-09-04T17:49:19.935375+0000","last_undegraded":"2024-09-04T17:49:19.935375+0000","last_fullsized":"2024-09-04T17:49:19.935375+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":548,"log_dups_size":0,"ondisk_log_size":548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:55:18.249867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011779189999999999,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":2788,"num_read_kb":17264,"num_write":577,"num_write_kb":29373,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":445,"num_bytes_recovered":17940277,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"178'716","reported_seq":4731,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.240231+0000","last_change":"2024-09-04T17:49:10.137619+0000","last_active":"2024-09-04T17:49:19.240231+0000","last_peered":"2024-09-04T17:49:19.240231+0000","last_clean":"2024-09-04T17:49:19.240231+0000","last_became_active":"2024-09-04T17:49:10.133837+0000","last_became_peered":"2024-09-04T17:49:10.133837+0000","last_unstale":"2024-09-04T17:49:19.240231+0000","last_undegraded":"2024-09-04T17:49:19.240231+0000","last_fullsized":"2024-09-04T17:49:19.240231+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":716,"log_dups_size":0,"ondisk_log_size":716,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T23:43:19.357232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003529346,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":6198,"num_read_kb":37498,"num_write":772,"num_write_kb":18653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":4534479,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"175'656","reported_seq":3645,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.080234+0000","last_change":"2024-09-04T17:48:56.054512+0000","last_active":"2024-09-04T17:49:20.080234+0000","last_peered":"2024-09-04T17:49:20.080234+0000","last_clean":"2024-09-04T17:49:20.080234+0000","last_became_active":"2024-09-04T17:48:00.841070+0000","last_became_peered":"2024-09-04T17:48:00.841070+0000","last_unstale":"2024-09-04T17:49:20.080234+0000","last_undegraded":"2024-09-04T17:49:20.080234+0000","last_fullsized":"2024-09-04T17:49:20.080234+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T19:18:47.901288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003342214,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":31,"num_read":5699,"num_read_kb":16474,"num_write":617,"num_write_kb":2753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":1572433,"num_keys_recovered":66,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"178'770","reported_seq":2984,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.935456+0000","last_change":"2024-09-04T17:49:06.885887+0000","last_active":"2024-09-04T17:49:19.935456+0000","last_peered":"2024-09-04T17:49:19.935456+0000","last_clean":"2024-09-04T17:49:19.935456+0000","last_became_active":"2024-09-04T17:49:06.883685+0000","last_became_peered":"2024-09-04T17:49:06.883685+0000","last_unstale":"2024-09-04T17:49:19.935456+0000","last_undegraded":"2024-09-04T17:49:19.935456+0000","last_fullsized":"2024-09-04T17:49:19.935456+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":770,"log_dups_size":0,"ondisk_log_size":770,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T17:55:41.907650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002090967,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":4267,"num_read_kb":14813,"num_write":915,"num_write_kb":7927,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":459,"num_bytes_recovered":5211501,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"175'647","reported_seq":9166,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.187139+0000","last_change":"2024-09-04T17:49:07.884097+0000","last_active":"2024-09-04T17:49:20.187139+0000","last_peered":"2024-09-04T17:49:20.187139+0000","last_clean":"2024-09-04T17:49:20.187139+0000","last_became_active":"2024-09-04T17:49:07.880558+0000","last_became_peered":"2024-09-04T17:49:07.880558+0000","last_unstale":"2024-09-04T17:49:20.187139+0000","last_undegraded":"2024-09-04T17:49:20.187139+0000","last_fullsized":"2024-09-04T17:49:20.187139+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":647,"log_dups_size":0,"ondisk_log_size":647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:43:25.854985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096708700000000005,"stat_sum":{"num_bytes":0,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":34,"num_read":9052,"num_read_kb":71439,"num_write":754,"num_write_kb":16863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":537,"num_bytes_recovered":8281724,"num_keys_recovered":57,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"178'1358","reported_seq":4585,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-04T17:49:18.308408+0000","last_change":"2024-09-04T17:48:56.066067+0000","last_active":"2024-09-04T17:49:18.308408+0000","last_peered":"2024-09-04T17:49:18.308408+0000","last_clean":"2024-09-04T17:49:18.308408+0000","last_became_active":"2024-09-04T17:48:12.850381+0000","last_became_peered":"2024-09-04T17:48:12.850381+0000","last_unstale":"2024-09-04T17:49:18.308408+0000","last_undegraded":"2024-09-04T17:49:18.308408+0000","last_fullsized":"2024-09-04T17:49:18.308408+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":1358,"log_dups_size":0,"ondisk_log_size":1358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T18:53:56.503643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002193296,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":31,"num_read":8863,"num_read_kb":26532,"num_write":2254,"num_write_kb":13198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":179,"num_bytes_recovered":18877339,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"178'541","reported_seq":6076,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-04T17:49:18.308579+0000","last_change":"2024-09-04T17:48:56.056521+0000","last_active":"2024-09-04T17:49:18.308579+0000","last_peered":"2024-09-04T17:49:18.308579+0000","last_clean":"2024-09-04T17:49:18.308579+0000","last_became_active":"2024-09-04T17:48:11.849605+0000","last_became_peered":"2024-09-04T17:48:11.849605+0000","last_unstale":"2024-09-04T17:49:18.308579+0000","last_undegraded":"2024-09-04T17:49:18.308579+0000","last_fullsized":"2024-09-04T17:49:18.308579+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":541,"log_dups_size":0,"ondisk_log_size":541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:57:37.250572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001652911,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":6678,"num_read_kb":28511,"num_write":523,"num_write_kb":11228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":7806862,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"178'348","reported_seq":3231,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-04T17:49:18.308740+0000","last_change":"2024-09-04T17:48:56.063351+0000","last_active":"2024-09-04T17:49:18.308740+0000","last_peered":"2024-09-04T17:49:18.308740+0000","last_clean":"2024-09-04T17:49:18.308740+0000","last_became_active":"2024-09-04T17:45:55.956734+0000","last_became_peered":"2024-09-04T17:45:55.956734+0000","last_unstale":"2024-09-04T17:49:18.308740+0000","last_undegraded":"2024-09-04T17:49:18.308740+0000","last_fullsized":"2024-09-04T17:49:18.308740+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T23:04:26.785513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0065435850000000002,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":2605,"num_read_kb":44969,"num_write":330,"num_write_kb":13427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"175'630","reported_seq":3858,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.080256+0000","last_change":"2024-09-04T17:48:56.053981+0000","last_active":"2024-09-04T17:49:20.080256+0000","last_peered":"2024-09-04T17:49:20.080256+0000","last_clean":"2024-09-04T17:49:20.080256+0000","last_became_active":"2024-09-04T17:48:00.843136+0000","last_became_peered":"2024-09-04T17:48:00.843136+0000","last_unstale":"2024-09-04T17:49:20.080256+0000","last_undegraded":"2024-09-04T17:49:20.080256+0000","last_fullsized":"2024-09-04T17:49:20.080256+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":630,"log_dups_size":0,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T23:36:00.802962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025156979999999998,"stat_sum":{"num_bytes":0,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":37,"num_read":4786,"num_read_kb":18960,"num_write":673,"num_write_kb":12854,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":3377599,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"175'599","reported_seq":6340,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-04T17:49:18.308074+0000","last_change":"2024-09-04T17:48:56.055370+0000","last_active":"2024-09-04T17:49:18.308074+0000","last_peered":"2024-09-04T17:49:18.308074+0000","last_clean":"2024-09-04T17:49:18.308074+0000","last_became_active":"2024-09-04T17:45:55.951870+0000","last_became_peered":"2024-09-04T17:45:55.951870+0000","last_unstale":"2024-09-04T17:49:18.308074+0000","last_undegraded":"2024-09-04T17:49:18.308074+0000","last_fullsized":"2024-09-04T17:49:18.308074+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:32:28.517472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012658840000000001,"stat_sum":{"num_bytes":0,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":38,"num_read":7163,"num_read_kb":61439,"num_write":584,"num_write_kb":17576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"175'425","reported_seq":4225,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.469949+0000","last_change":"2024-09-04T17:48:56.055907+0000","last_active":"2024-09-04T17:49:20.469949+0000","last_peered":"2024-09-04T17:49:20.469949+0000","last_clean":"2024-09-04T17:49:20.469949+0000","last_became_active":"2024-09-04T17:48:14.943778+0000","last_became_peered":"2024-09-04T17:48:14.943778+0000","last_unstale":"2024-09-04T17:49:20.469949+0000","last_undegraded":"2024-09-04T17:49:20.469949+0000","last_fullsized":"2024-09-04T17:49:20.469949+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":425,"log_dups_size":0,"ondisk_log_size":425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:28:33.747314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011847660000000001,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":31,"num_read":3577,"num_read_kb":21719,"num_write":447,"num_write_kb":8575,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":290,"num_bytes_recovered":2723025,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"175'313","reported_seq":2693,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-04T17:49:18.308158+0000","last_change":"2024-09-04T17:48:56.055871+0000","last_active":"2024-09-04T17:49:18.308158+0000","last_peered":"2024-09-04T17:49:18.308158+0000","last_clean":"2024-09-04T17:49:18.308158+0000","last_became_active":"2024-09-04T17:45:55.954544+0000","last_became_peered":"2024-09-04T17:45:55.954544+0000","last_unstale":"2024-09-04T17:49:18.308158+0000","last_undegraded":"2024-09-04T17:49:18.308158+0000","last_fullsized":"2024-09-04T17:49:18.308158+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:20:57.091315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001451911,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":2420,"num_read_kb":22743,"num_write":250,"num_write_kb":4380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":935,"num_object_clones":0,"num_object_copies":1870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":935,"num_whiteouts":931,"num_read":142280,"num_read_kb":876749,"num_write":20299,"num_write_kb":404276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6761,"num_bytes_recovered":238961833,"num_keys_recovered":777,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21395,"internal_metadata":0},"log_size":18211,"ondisk_log_size":18211,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":117,"seq":502511173651,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44384,"kb_used_data":2316,"kb_used_omap":22,"kb_used_meta":42025,"kb_avail":94327456,"statfs":{"total":96636764160,"available":96591314944,"internally_reserved":0,"allocated":2371584,"data_stored":1152207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22742,"internal_metadata":43034410},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509488,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51568,"kb_used_data":2332,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320272,"statfs":{"total":96636764160,"available":96583958528,"internally_reserved":0,"allocated":2387968,"data_stored":1158039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509489,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51572,"kb_used_data":2332,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320268,"statfs":{"total":96636764160,"available":96583954432,"internally_reserved":0,"allocated":2387968,"data_stored":1158039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":48,"seq":206158430245,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60848,"kb_used_data":2328,"kb_used_omap":4,"kb_used_meta":58491,"kb_avail":94310992,"statfs":{"total":96636764160,"available":96574455808,"internally_reserved":0,"allocated":2383872,"data_stored":1158020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4430,"internal_metadata":59895474},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509488,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29368,"kb_used_data":2336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342472,"statfs":{"total":96636764160,"available":96606691328,"internally_reserved":0,"allocated":2392064,"data_stored":1158058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509488,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60464,"kb_used_data":2896,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311376,"statfs":{"total":96636764160,"available":96574849024,"internally_reserved":0,"allocated":2965504,"data_stored":1742575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542192,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60460,"kb_used_data":2896,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311380,"statfs":{"total":96636764160,"available":96574853120,"internally_reserved":0,"allocated":2965504,"data_stored":1742575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":103,"seq":442381631510,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30000,"kb_used_data":2328,"kb_used_omap":12,"kb_used_meta":27635,"kb_avail":94341840,"statfs":{"total":96636764160,"available":96606044160,"internally_reserved":0,"allocated":2383872,"data_stored":1158020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12804,"internal_metadata":28298748},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6121,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15274,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-04T17:49:23.385 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-04T17:49:23.386 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-04T17:49:23.688 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-04T17:49:23.697 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-04T17:49:23.697 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-04T17:49:23.985 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:49:23.986 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-09-04T17:49:24.002 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":303,"stamp":"2024-09-04T17:49:23.130883+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":939,"num_object_clones":0,"num_object_copies":1878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":939,"num_whiteouts":931,"num_read":142346,"num_read_kb":876804,"num_write":20402,"num_write_kb":405906,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6763,"num_bytes_recovered":238961871,"num_keys_recovered":777,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18286,"ondisk_log_size":18286,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":388664,"kb_used_data":19764,"kb_used_omap":46,"kb_used_meta":368657,"kb_avail":754586056,"statfs":{"total":773094113280,"available":772696121344,"internally_reserved":0,"allocated":20238336,"data_stored":10427533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":47946,"internal_metadata":377504950},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001829"},"pg_stats":[{"pgid":"3.1f","version":"178'501","reported_seq":3133,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.470279+0000","last_change":"2024-09-04T17:48:56.059852+0000","last_active":"2024-09-04T17:49:20.470279+0000","last_peered":"2024-09-04T17:49:20.470279+0000","last_clean":"2024-09-04T17:49:20.470279+0000","last_became_active":"2024-09-04T17:48:09.845199+0000","last_became_peered":"2024-09-04T17:48:09.845199+0000","last_unstale":"2024-09-04T17:49:20.470279+0000","last_undegraded":"2024-09-04T17:49:20.470279+0000","last_fullsized":"2024-09-04T17:49:20.470279+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T21:36:09.244561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003726699,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":2983,"num_read_kb":27190,"num_write":524,"num_write_kb":10445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":24433786,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"175'657","reported_seq":3987,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.935494+0000","last_change":"2024-09-04T17:49:05.876073+0000","last_active":"2024-09-04T17:49:19.935494+0000","last_peered":"2024-09-04T17:49:19.935494+0000","last_clean":"2024-09-04T17:49:19.935494+0000","last_became_active":"2024-09-04T17:49:05.874586+0000","last_became_peered":"2024-09-04T17:49:05.874586+0000","last_unstale":"2024-09-04T17:49:19.935494+0000","last_undegraded":"2024-09-04T17:49:19.935494+0000","last_fullsized":"2024-09-04T17:49:19.935494+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:25:15.903921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001926142,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4757,"num_read_kb":30502,"num_write":815,"num_write_kb":10573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":322,"num_bytes_recovered":15412196,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"175'548","reported_seq":2554,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.187160+0000","last_change":"2024-09-04T17:48:56.060983+0000","last_active":"2024-09-04T17:49:20.187160+0000","last_peered":"2024-09-04T17:49:20.187160+0000","last_clean":"2024-09-04T17:49:20.187160+0000","last_became_active":"2024-09-04T17:45:55.955213+0000","last_became_peered":"2024-09-04T17:45:55.955213+0000","last_unstale":"2024-09-04T17:49:20.187160+0000","last_undegraded":"2024-09-04T17:49:20.187160+0000","last_fullsized":"2024-09-04T17:49:20.187160+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":548,"log_dups_size":0,"ondisk_log_size":548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T19:29:02.548970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022810360000000002,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":2820,"num_read_kb":20284,"num_write":623,"num_write_kb":9401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"178'758","reported_seq":6826,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.187131+0000","last_change":"2024-09-04T17:48:56.055219+0000","last_active":"2024-09-04T17:49:20.187131+0000","last_peered":"2024-09-04T17:49:20.187131+0000","last_clean":"2024-09-04T17:49:20.187131+0000","last_became_active":"2024-09-04T17:45:55.951880+0000","last_became_peered":"2024-09-04T17:45:55.951880+0000","last_unstale":"2024-09-04T17:49:20.187131+0000","last_undegraded":"2024-09-04T17:49:20.187131+0000","last_fullsized":"2024-09-04T17:49:20.187131+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":758,"log_dups_size":0,"ondisk_log_size":758,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:58:30.591123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025730409999999999,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":35,"num_read":7473,"num_read_kb":48018,"num_write":874,"num_write_kb":11556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"175'597","reported_seq":2410,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.469937+0000","last_change":"2024-09-04T17:48:56.058271+0000","last_active":"2024-09-04T17:49:20.469937+0000","last_peered":"2024-09-04T17:49:20.469937+0000","last_clean":"2024-09-04T17:49:20.469937+0000","last_became_active":"2024-09-04T17:48:09.853793+0000","last_became_peered":"2024-09-04T17:48:09.853793+0000","last_unstale":"2024-09-04T17:49:20.469937+0000","last_undegraded":"2024-09-04T17:49:20.469937+0000","last_fullsized":"2024-09-04T17:49:20.469937+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":597,"log_dups_size":0,"ondisk_log_size":597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:56:08.051891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026463849999999998,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":3715,"num_read_kb":8148,"num_write":647,"num_write_kb":4600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":1993914,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"175'424","reported_seq":6455,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.240117+0000","last_change":"2024-09-04T17:49:10.140837+0000","last_active":"2024-09-04T17:49:19.240117+0000","last_peered":"2024-09-04T17:49:19.240117+0000","last_clean":"2024-09-04T17:49:19.240117+0000","last_became_active":"2024-09-04T17:49:10.137472+0000","last_became_peered":"2024-09-04T17:49:10.137472+0000","last_unstale":"2024-09-04T17:49:19.240117+0000","last_undegraded":"2024-09-04T17:49:19.240117+0000","last_fullsized":"2024-09-04T17:49:19.240117+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":424,"log_dups_size":0,"ondisk_log_size":424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:51:59.854799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000885438,"stat_sum":{"num_bytes":19,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":25,"num_read":5992,"num_read_kb":45852,"num_write":376,"num_write_kb":15447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":5992020,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"178'442","reported_seq":2720,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.237063+0000","last_change":"2024-09-04T17:49:05.878999+0000","last_active":"2024-09-04T17:49:19.237063+0000","last_peered":"2024-09-04T17:49:19.237063+0000","last_clean":"2024-09-04T17:49:19.237063+0000","last_became_active":"2024-09-04T17:49:05.876985+0000","last_became_peered":"2024-09-04T17:49:05.876985+0000","last_unstale":"2024-09-04T17:49:19.237063+0000","last_undegraded":"2024-09-04T17:49:19.237063+0000","last_fullsized":"2024-09-04T17:49:19.237063+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:42:50.926456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001467744,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":2248,"num_read_kb":16196,"num_write":460,"num_write_kb":4617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":494,"num_bytes_recovered":4911830,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"178'508","reported_seq":2560,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.935295+0000","last_change":"2024-09-04T17:49:07.880474+0000","last_active":"2024-09-04T17:49:19.935295+0000","last_peered":"2024-09-04T17:49:19.935295+0000","last_clean":"2024-09-04T17:49:19.935295+0000","last_became_active":"2024-09-04T17:49:07.878360+0000","last_became_peered":"2024-09-04T17:49:07.878360+0000","last_unstale":"2024-09-04T17:49:19.935295+0000","last_undegraded":"2024-09-04T17:49:19.935295+0000","last_fullsized":"2024-09-04T17:49:19.935295+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:25:31.312054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017798989999999999,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":8892,"num_read_kb":35489,"num_write":443,"num_write_kb":14429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":597,"num_bytes_recovered":28259997,"num_keys_recovered":141,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"178'331","reported_seq":2390,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.470375+0000","last_change":"2024-09-04T17:48:56.056526+0000","last_active":"2024-09-04T17:49:20.470375+0000","last_peered":"2024-09-04T17:49:20.470375+0000","last_clean":"2024-09-04T17:49:20.470375+0000","last_became_active":"2024-09-04T17:45:55.958625+0000","last_became_peered":"2024-09-04T17:45:55.958625+0000","last_unstale":"2024-09-04T17:49:20.470375+0000","last_undegraded":"2024-09-04T17:49:20.470375+0000","last_fullsized":"2024-09-04T17:49:20.470375+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":331,"log_dups_size":0,"ondisk_log_size":331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T22:39:37.983710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001574039,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":1612,"num_read_kb":20939,"num_write":316,"num_write_kb":18406,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"175'364","reported_seq":2051,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.187266+0000","last_change":"2024-09-04T17:48:56.060132+0000","last_active":"2024-09-04T17:49:20.187266+0000","last_peered":"2024-09-04T17:49:20.187266+0000","last_clean":"2024-09-04T17:49:20.187266+0000","last_became_active":"2024-09-04T17:45:55.953254+0000","last_became_peered":"2024-09-04T17:45:55.953254+0000","last_unstale":"2024-09-04T17:49:20.187266+0000","last_undegraded":"2024-09-04T17:49:20.187266+0000","last_fullsized":"2024-09-04T17:49:20.187266+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":364,"log_dups_size":0,"ondisk_log_size":364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T21:59:17.936385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0044959860000000004,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":1729,"num_read_kb":10785,"num_write":396,"num_write_kb":7391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"174'510","reported_seq":3054,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-04T17:49:18.308234+0000","last_change":"2024-09-04T17:49:06.889185+0000","last_active":"2024-09-04T17:49:18.308234+0000","last_peered":"2024-09-04T17:49:18.308234+0000","last_clean":"2024-09-04T17:49:18.308234+0000","last_became_active":"2024-09-04T17:49:06.884929+0000","last_became_peered":"2024-09-04T17:49:06.884929+0000","last_unstale":"2024-09-04T17:49:18.308234+0000","last_undegraded":"2024-09-04T17:49:18.308234+0000","last_fullsized":"2024-09-04T17:49:18.308234+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":510,"log_dups_size":0,"ondisk_log_size":510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:02:50.836583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001003051,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":4020,"num_read_kb":16975,"num_write":525,"num_write_kb":4439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":446,"num_bytes_recovered":12989343,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"178'775","reported_seq":2528,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.240494+0000","last_change":"2024-09-04T17:49:10.140905+0000","last_active":"2024-09-04T17:49:19.240494+0000","last_peered":"2024-09-04T17:49:19.240494+0000","last_clean":"2024-09-04T17:49:19.240494+0000","last_became_active":"2024-09-04T17:49:10.137573+0000","last_became_peered":"2024-09-04T17:49:10.137573+0000","last_unstale":"2024-09-04T17:49:19.240494+0000","last_undegraded":"2024-09-04T17:49:19.240494+0000","last_fullsized":"2024-09-04T17:49:19.240494+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":775,"log_dups_size":0,"ondisk_log_size":775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:56:50.281112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027107059999999998,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":29,"num_read":4009,"num_read_kb":29862,"num_write":1001,"num_write_kb":35909,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":235,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.080313+0000","last_change":"2024-09-04T17:48:11.850550+0000","last_active":"2024-09-04T17:49:20.080313+0000","last_peered":"2024-09-04T17:49:20.080313+0000","last_clean":"2024-09-04T17:49:20.080313+0000","last_became_active":"2024-09-04T17:48:11.849710+0000","last_became_peered":"2024-09-04T17:48:11.849710+0000","last_unstale":"2024-09-04T17:49:20.080313+0000","last_undegraded":"2024-09-04T17:49:20.080313+0000","last_fullsized":"2024-09-04T17:49:20.080313+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:37:18.791459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001226074,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"178'409","reported_seq":3430,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.470300+0000","last_change":"2024-09-04T17:49:07.876177+0000","last_active":"2024-09-04T17:49:20.470300+0000","last_peered":"2024-09-04T17:49:20.470300+0000","last_clean":"2024-09-04T17:49:20.470300+0000","last_became_active":"2024-09-04T17:49:07.874501+0000","last_became_peered":"2024-09-04T17:49:07.874501+0000","last_unstale":"2024-09-04T17:49:20.470300+0000","last_undegraded":"2024-09-04T17:49:20.470300+0000","last_fullsized":"2024-09-04T17:49:20.470300+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":409,"log_dups_size":0,"ondisk_log_size":409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:52:42.831486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001494427,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":2485,"num_read_kb":14550,"num_write":410,"num_write_kb":5528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":919873,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":40,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.237456+0000","last_change":"2024-09-04T17:49:03.886949+0000","last_active":"2024-09-04T17:49:19.237456+0000","last_peered":"2024-09-04T17:49:19.237456+0000","last_clean":"2024-09-04T17:49:19.237456+0000","last_became_active":"2024-09-04T17:49:03.886525+0000","last_became_peered":"2024-09-04T17:49:03.886525+0000","last_unstale":"2024-09-04T17:49:19.237456+0000","last_undegraded":"2024-09-04T17:49:19.237456+0000","last_fullsized":"2024-09-04T17:49:19.237456+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:24:57.789704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011364179999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"175'469","reported_seq":3004,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.935427+0000","last_change":"2024-09-04T17:49:05.877106+0000","last_active":"2024-09-04T17:49:19.935427+0000","last_peered":"2024-09-04T17:49:19.935427+0000","last_clean":"2024-09-04T17:49:19.935427+0000","last_became_active":"2024-09-04T17:49:05.874579+0000","last_became_peered":"2024-09-04T17:49:05.874579+0000","last_unstale":"2024-09-04T17:49:19.935427+0000","last_undegraded":"2024-09-04T17:49:19.935427+0000","last_fullsized":"2024-09-04T17:49:19.935427+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":469,"log_dups_size":0,"ondisk_log_size":469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:32:31.466355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019775819999999999,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":2953,"num_read_kb":23389,"num_write":575,"num_write_kb":21569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":313,"num_bytes_recovered":1588111,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":370,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-04T17:49:18.308514+0000","last_change":"2024-09-04T17:45:49.476607+0000","last_active":"2024-09-04T17:49:18.308514+0000","last_peered":"2024-09-04T17:49:18.308514+0000","last_clean":"2024-09-04T17:49:18.308514+0000","last_became_active":"2024-09-04T17:45:47.369308+0000","last_became_peered":"2024-09-04T17:45:47.369308+0000","last_unstale":"2024-09-04T17:49:18.308514+0000","last_undegraded":"2024-09-04T17:49:18.308514+0000","last_fullsized":"2024-09-04T17:49:18.308514+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:17:13.378553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001029826,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"175'410","reported_seq":2886,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.237416+0000","last_change":"2024-09-04T17:49:07.879430+0000","last_active":"2024-09-04T17:49:19.237416+0000","last_peered":"2024-09-04T17:49:19.237416+0000","last_clean":"2024-09-04T17:49:19.237416+0000","last_became_active":"2024-09-04T17:49:07.877184+0000","last_became_peered":"2024-09-04T17:49:07.877184+0000","last_unstale":"2024-09-04T17:49:19.237416+0000","last_undegraded":"2024-09-04T17:49:19.237416+0000","last_fullsized":"2024-09-04T17:49:19.237416+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T21:09:45.025593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029821800000000001,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":31,"num_read":2672,"num_read_kb":18782,"num_write":436,"num_write_kb":9856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":13375622,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.237128+0000","last_change":"2024-09-04T17:49:03.886170+0000","last_active":"2024-09-04T17:49:19.237128+0000","last_peered":"2024-09-04T17:49:19.237128+0000","last_clean":"2024-09-04T17:49:19.237128+0000","last_became_active":"2024-09-04T17:49:03.885776+0000","last_became_peered":"2024-09-04T17:49:03.885776+0000","last_unstale":"2024-09-04T17:49:19.237128+0000","last_undegraded":"2024-09-04T17:49:19.237128+0000","last_fullsized":"2024-09-04T17:49:19.237128+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T19:30:49.636681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"175'358","reported_seq":3042,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.187305+0000","last_change":"2024-09-04T17:49:06.887617+0000","last_active":"2024-09-04T17:49:20.187305+0000","last_peered":"2024-09-04T17:49:20.187305+0000","last_clean":"2024-09-04T17:49:20.187305+0000","last_became_active":"2024-09-04T17:49:06.885686+0000","last_became_peered":"2024-09-04T17:49:06.885686+0000","last_unstale":"2024-09-04T17:49:20.187305+0000","last_undegraded":"2024-09-04T17:49:20.187305+0000","last_fullsized":"2024-09-04T17:49:20.187305+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T22:44:47.142525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019787199999999998,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":2555,"num_read_kb":7639,"num_write":305,"num_write_kb":7011,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":391,"num_bytes_recovered":15596571,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"178'887","reported_seq":3553,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.240376+0000","last_change":"2024-09-04T17:49:10.141003+0000","last_active":"2024-09-04T17:49:19.240376+0000","last_peered":"2024-09-04T17:49:19.240376+0000","last_clean":"2024-09-04T17:49:19.240376+0000","last_became_active":"2024-09-04T17:49:10.137895+0000","last_became_peered":"2024-09-04T17:49:10.137895+0000","last_unstale":"2024-09-04T17:49:19.240376+0000","last_undegraded":"2024-09-04T17:49:19.240376+0000","last_fullsized":"2024-09-04T17:49:19.240376+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":887,"log_dups_size":0,"ondisk_log_size":887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:24:10.064606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032310310000000001,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":29,"num_read":4673,"num_read_kb":26561,"num_write":1100,"num_write_kb":14222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":398,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.186827+0000","last_change":"2024-09-04T17:49:03.892940+0000","last_active":"2024-09-04T17:49:20.186827+0000","last_peered":"2024-09-04T17:49:20.186827+0000","last_clean":"2024-09-04T17:49:20.186827+0000","last_became_active":"2024-09-04T17:49:03.887053+0000","last_became_peered":"2024-09-04T17:49:03.887053+0000","last_unstale":"2024-09-04T17:49:20.186827+0000","last_undegraded":"2024-09-04T17:49:20.186827+0000","last_fullsized":"2024-09-04T17:49:20.186827+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:30:44.144337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026316069999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"175'453","reported_seq":3440,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.237166+0000","last_change":"2024-09-04T17:49:07.878051+0000","last_active":"2024-09-04T17:49:19.237166+0000","last_peered":"2024-09-04T17:49:19.237166+0000","last_clean":"2024-09-04T17:49:19.237166+0000","last_became_active":"2024-09-04T17:49:07.876339+0000","last_became_peered":"2024-09-04T17:49:07.876339+0000","last_unstale":"2024-09-04T17:49:19.237166+0000","last_undegraded":"2024-09-04T17:49:19.237166+0000","last_fullsized":"2024-09-04T17:49:19.237166+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":453,"log_dups_size":0,"ondisk_log_size":453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T22:43:32.300162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00120166,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":36,"num_read":3566,"num_read_kb":22521,"num_write":365,"num_write_kb":4893,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":261,"num_bytes_recovered":5692590,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":370,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.236829+0000","last_change":"2024-09-04T17:49:02.864996+0000","last_active":"2024-09-04T17:49:19.236829+0000","last_peered":"2024-09-04T17:49:19.236829+0000","last_clean":"2024-09-04T17:49:19.236829+0000","last_became_active":"2024-09-04T17:49:02.864647+0000","last_became_peered":"2024-09-04T17:49:02.864647+0000","last_unstale":"2024-09-04T17:49:19.236829+0000","last_undegraded":"2024-09-04T17:49:19.236829+0000","last_fullsized":"2024-09-04T17:49:19.236829+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:41:35.190910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001025195,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"178'637","reported_seq":3375,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.469830+0000","last_change":"2024-09-04T17:48:56.058939+0000","last_active":"2024-09-04T17:49:20.469830+0000","last_peered":"2024-09-04T17:49:20.469830+0000","last_clean":"2024-09-04T17:49:20.469830+0000","last_became_active":"2024-09-04T17:48:11.845510+0000","last_became_peered":"2024-09-04T17:48:11.845510+0000","last_unstale":"2024-09-04T17:49:20.469830+0000","last_undegraded":"2024-09-04T17:49:20.469830+0000","last_fullsized":"2024-09-04T17:49:20.469830+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":637,"log_dups_size":0,"ondisk_log_size":637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:14:18.552668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031660479999999999,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":3316,"num_read_kb":40546,"num_write":672,"num_write_kb":23359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":240,"num_bytes_recovered":21297737,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":382,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-04T17:49:18.308065+0000","last_change":"2024-09-04T17:49:03.874392+0000","last_active":"2024-09-04T17:49:18.308065+0000","last_peered":"2024-09-04T17:49:18.308065+0000","last_clean":"2024-09-04T17:49:18.308065+0000","last_became_active":"2024-09-04T17:49:03.870678+0000","last_became_peered":"2024-09-04T17:49:03.870678+0000","last_unstale":"2024-09-04T17:49:18.308065+0000","last_undegraded":"2024-09-04T17:49:18.308065+0000","last_fullsized":"2024-09-04T17:49:18.308065+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:49:38.360230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010553590000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"175'622","reported_seq":3059,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.935307+0000","last_change":"2024-09-04T17:49:07.879558+0000","last_active":"2024-09-04T17:49:19.935307+0000","last_peered":"2024-09-04T17:49:19.935307+0000","last_clean":"2024-09-04T17:49:19.935307+0000","last_became_active":"2024-09-04T17:49:07.876340+0000","last_became_peered":"2024-09-04T17:49:07.876340+0000","last_unstale":"2024-09-04T17:49:19.935307+0000","last_undegraded":"2024-09-04T17:49:19.935307+0000","last_fullsized":"2024-09-04T17:49:19.935307+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":622,"log_dups_size":0,"ondisk_log_size":622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:20:26.042540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021278730000000002,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":5714,"num_read_kb":30160,"num_write":740,"num_write_kb":13818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":405,"num_bytes_recovered":16173004,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":382,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.186955+0000","last_change":"2024-09-04T17:47:41.919544+0000","last_active":"2024-09-04T17:49:20.186955+0000","last_peered":"2024-09-04T17:49:20.186955+0000","last_clean":"2024-09-04T17:49:20.186955+0000","last_became_active":"2024-09-04T17:47:41.919027+0000","last_became_peered":"2024-09-04T17:47:41.919027+0000","last_unstale":"2024-09-04T17:49:20.186955+0000","last_undegraded":"2024-09-04T17:49:20.186955+0000","last_fullsized":"2024-09-04T17:49:20.186955+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:42:55.688562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086618299999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":461,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-04T17:49:18.308163+0000","last_change":"2024-09-04T17:45:42.341149+0000","last_active":"2024-09-04T17:49:18.308163+0000","last_peered":"2024-09-04T17:49:18.308163+0000","last_clean":"2024-09-04T17:49:18.308163+0000","last_became_active":"2024-09-04T17:45:42.339286+0000","last_became_peered":"2024-09-04T17:45:42.339286+0000","last_unstale":"2024-09-04T17:49:18.308163+0000","last_undegraded":"2024-09-04T17:49:18.308163+0000","last_fullsized":"2024-09-04T17:49:18.308163+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:41.305128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:41.305128+0000","last_clean_scrub_stamp":"2024-09-04T17:45:41.305128+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:52:24.860049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"178'548","reported_seq":3065,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.935375+0000","last_change":"2024-09-04T17:49:07.883398+0000","last_active":"2024-09-04T17:49:19.935375+0000","last_peered":"2024-09-04T17:49:19.935375+0000","last_clean":"2024-09-04T17:49:19.935375+0000","last_became_active":"2024-09-04T17:49:07.879442+0000","last_became_peered":"2024-09-04T17:49:07.879442+0000","last_unstale":"2024-09-04T17:49:19.935375+0000","last_undegraded":"2024-09-04T17:49:19.935375+0000","last_fullsized":"2024-09-04T17:49:19.935375+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":548,"log_dups_size":0,"ondisk_log_size":548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:55:18.249867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011779189999999999,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":2788,"num_read_kb":17264,"num_write":577,"num_write_kb":29373,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":445,"num_bytes_recovered":17940277,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"178'716","reported_seq":4731,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.240231+0000","last_change":"2024-09-04T17:49:10.137619+0000","last_active":"2024-09-04T17:49:19.240231+0000","last_peered":"2024-09-04T17:49:19.240231+0000","last_clean":"2024-09-04T17:49:19.240231+0000","last_became_active":"2024-09-04T17:49:10.133837+0000","last_became_peered":"2024-09-04T17:49:10.133837+0000","last_unstale":"2024-09-04T17:49:19.240231+0000","last_undegraded":"2024-09-04T17:49:19.240231+0000","last_fullsized":"2024-09-04T17:49:19.240231+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":716,"log_dups_size":0,"ondisk_log_size":716,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T23:43:19.357232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003529346,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":6198,"num_read_kb":37498,"num_write":772,"num_write_kb":18653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":4534479,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"175'656","reported_seq":3645,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.080234+0000","last_change":"2024-09-04T17:48:56.054512+0000","last_active":"2024-09-04T17:49:20.080234+0000","last_peered":"2024-09-04T17:49:20.080234+0000","last_clean":"2024-09-04T17:49:20.080234+0000","last_became_active":"2024-09-04T17:48:00.841070+0000","last_became_peered":"2024-09-04T17:48:00.841070+0000","last_unstale":"2024-09-04T17:49:20.080234+0000","last_undegraded":"2024-09-04T17:49:20.080234+0000","last_fullsized":"2024-09-04T17:49:20.080234+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T19:18:47.901288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003342214,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":31,"num_read":5699,"num_read_kb":16474,"num_write":617,"num_write_kb":2753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":1572433,"num_keys_recovered":66,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"178'770","reported_seq":2984,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.935456+0000","last_change":"2024-09-04T17:49:06.885887+0000","last_active":"2024-09-04T17:49:19.935456+0000","last_peered":"2024-09-04T17:49:19.935456+0000","last_clean":"2024-09-04T17:49:19.935456+0000","last_became_active":"2024-09-04T17:49:06.883685+0000","last_became_peered":"2024-09-04T17:49:06.883685+0000","last_unstale":"2024-09-04T17:49:19.935456+0000","last_undegraded":"2024-09-04T17:49:19.935456+0000","last_fullsized":"2024-09-04T17:49:19.935456+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":770,"log_dups_size":0,"ondisk_log_size":770,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T17:55:41.907650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002090967,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":4267,"num_read_kb":14813,"num_write":915,"num_write_kb":7927,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":459,"num_bytes_recovered":5211501,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"175'647","reported_seq":9166,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.187139+0000","last_change":"2024-09-04T17:49:07.884097+0000","last_active":"2024-09-04T17:49:20.187139+0000","last_peered":"2024-09-04T17:49:20.187139+0000","last_clean":"2024-09-04T17:49:20.187139+0000","last_became_active":"2024-09-04T17:49:07.880558+0000","last_became_peered":"2024-09-04T17:49:07.880558+0000","last_unstale":"2024-09-04T17:49:20.187139+0000","last_undegraded":"2024-09-04T17:49:20.187139+0000","last_fullsized":"2024-09-04T17:49:20.187139+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":647,"log_dups_size":0,"ondisk_log_size":647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:43:25.854985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096708700000000005,"stat_sum":{"num_bytes":0,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":34,"num_read":9052,"num_read_kb":71439,"num_write":754,"num_write_kb":16863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":537,"num_bytes_recovered":8281724,"num_keys_recovered":57,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"178'1358","reported_seq":4585,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-04T17:49:18.308408+0000","last_change":"2024-09-04T17:48:56.066067+0000","last_active":"2024-09-04T17:49:18.308408+0000","last_peered":"2024-09-04T17:49:18.308408+0000","last_clean":"2024-09-04T17:49:18.308408+0000","last_became_active":"2024-09-04T17:48:12.850381+0000","last_became_peered":"2024-09-04T17:48:12.850381+0000","last_unstale":"2024-09-04T17:49:18.308408+0000","last_undegraded":"2024-09-04T17:49:18.308408+0000","last_fullsized":"2024-09-04T17:49:18.308408+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":1358,"log_dups_size":0,"ondisk_log_size":1358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T18:53:56.503643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002193296,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":31,"num_read":8863,"num_read_kb":26532,"num_write":2254,"num_write_kb":13198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":179,"num_bytes_recovered":18877339,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"178'541","reported_seq":6076,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-04T17:49:18.308579+0000","last_change":"2024-09-04T17:48:56.056521+0000","last_active":"2024-09-04T17:49:18.308579+0000","last_peered":"2024-09-04T17:49:18.308579+0000","last_clean":"2024-09-04T17:49:18.308579+0000","last_became_active":"2024-09-04T17:48:11.849605+0000","last_became_peered":"2024-09-04T17:48:11.849605+0000","last_unstale":"2024-09-04T17:49:18.308579+0000","last_undegraded":"2024-09-04T17:49:18.308579+0000","last_fullsized":"2024-09-04T17:49:18.308579+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":541,"log_dups_size":0,"ondisk_log_size":541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:57:37.250572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001652911,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":6678,"num_read_kb":28511,"num_write":523,"num_write_kb":11228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":7806862,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"178'348","reported_seq":3231,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-04T17:49:18.308740+0000","last_change":"2024-09-04T17:48:56.063351+0000","last_active":"2024-09-04T17:49:18.308740+0000","last_peered":"2024-09-04T17:49:18.308740+0000","last_clean":"2024-09-04T17:49:18.308740+0000","last_became_active":"2024-09-04T17:45:55.956734+0000","last_became_peered":"2024-09-04T17:45:55.956734+0000","last_unstale":"2024-09-04T17:49:18.308740+0000","last_undegraded":"2024-09-04T17:49:18.308740+0000","last_fullsized":"2024-09-04T17:49:18.308740+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T23:04:26.785513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0065435850000000002,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":2605,"num_read_kb":44969,"num_write":330,"num_write_kb":13427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"175'630","reported_seq":3858,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.080256+0000","last_change":"2024-09-04T17:48:56.053981+0000","last_active":"2024-09-04T17:49:20.080256+0000","last_peered":"2024-09-04T17:49:20.080256+0000","last_clean":"2024-09-04T17:49:20.080256+0000","last_became_active":"2024-09-04T17:48:00.843136+0000","last_became_peered":"2024-09-04T17:48:00.843136+0000","last_unstale":"2024-09-04T17:49:20.080256+0000","last_undegraded":"2024-09-04T17:49:20.080256+0000","last_fullsized":"2024-09-04T17:49:20.080256+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":630,"log_dups_size":0,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T23:36:00.802962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025156979999999998,"stat_sum":{"num_bytes":0,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":37,"num_read":4786,"num_read_kb":18960,"num_write":673,"num_write_kb":12854,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":3377599,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"175'599","reported_seq":6340,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-04T17:49:18.308074+0000","last_change":"2024-09-04T17:48:56.055370+0000","last_active":"2024-09-04T17:49:18.308074+0000","last_peered":"2024-09-04T17:49:18.308074+0000","last_clean":"2024-09-04T17:49:18.308074+0000","last_became_active":"2024-09-04T17:45:55.951870+0000","last_became_peered":"2024-09-04T17:45:55.951870+0000","last_unstale":"2024-09-04T17:49:18.308074+0000","last_undegraded":"2024-09-04T17:49:18.308074+0000","last_fullsized":"2024-09-04T17:49:18.308074+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:32:28.517472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012658840000000001,"stat_sum":{"num_bytes":0,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":38,"num_read":7163,"num_read_kb":61439,"num_write":584,"num_write_kb":17576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"175'425","reported_seq":4225,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.469949+0000","last_change":"2024-09-04T17:48:56.055907+0000","last_active":"2024-09-04T17:49:20.469949+0000","last_peered":"2024-09-04T17:49:20.469949+0000","last_clean":"2024-09-04T17:49:20.469949+0000","last_became_active":"2024-09-04T17:48:14.943778+0000","last_became_peered":"2024-09-04T17:48:14.943778+0000","last_unstale":"2024-09-04T17:49:20.469949+0000","last_undegraded":"2024-09-04T17:49:20.469949+0000","last_fullsized":"2024-09-04T17:49:20.469949+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":425,"log_dups_size":0,"ondisk_log_size":425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:28:33.747314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011847660000000001,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":31,"num_read":3577,"num_read_kb":21719,"num_write":447,"num_write_kb":8575,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":290,"num_bytes_recovered":2723025,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"175'313","reported_seq":2693,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-04T17:49:18.308158+0000","last_change":"2024-09-04T17:48:56.055871+0000","last_active":"2024-09-04T17:49:18.308158+0000","last_peered":"2024-09-04T17:49:18.308158+0000","last_clean":"2024-09-04T17:49:18.308158+0000","last_became_active":"2024-09-04T17:45:55.954544+0000","last_became_peered":"2024-09-04T17:45:55.954544+0000","last_unstale":"2024-09-04T17:49:18.308158+0000","last_undegraded":"2024-09-04T17:49:18.308158+0000","last_fullsized":"2024-09-04T17:49:18.308158+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:20:57.091315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001451911,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":2420,"num_read_kb":22743,"num_write":250,"num_write_kb":4380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":935,"num_object_clones":0,"num_object_copies":1870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":935,"num_whiteouts":931,"num_read":142280,"num_read_kb":876749,"num_write":20299,"num_write_kb":404276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6761,"num_bytes_recovered":238961833,"num_keys_recovered":777,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21395,"internal_metadata":0},"log_size":18211,"ondisk_log_size":18211,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":117,"seq":502511173651,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44384,"kb_used_data":2316,"kb_used_omap":22,"kb_used_meta":42025,"kb_avail":94327456,"statfs":{"total":96636764160,"available":96591314944,"internally_reserved":0,"allocated":2371584,"data_stored":1152207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22742,"internal_metadata":43034410},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509488,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51568,"kb_used_data":2332,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320272,"statfs":{"total":96636764160,"available":96583958528,"internally_reserved":0,"allocated":2387968,"data_stored":1158039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509489,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51572,"kb_used_data":2332,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320268,"statfs":{"total":96636764160,"available":96583954432,"internally_reserved":0,"allocated":2387968,"data_stored":1158039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":48,"seq":206158430245,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60848,"kb_used_data":2328,"kb_used_omap":4,"kb_used_meta":58491,"kb_avail":94310992,"statfs":{"total":96636764160,"available":96574455808,"internally_reserved":0,"allocated":2383872,"data_stored":1158020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4430,"internal_metadata":59895474},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509488,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29368,"kb_used_data":2336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342472,"statfs":{"total":96636764160,"available":96606691328,"internally_reserved":0,"allocated":2392064,"data_stored":1158058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509488,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60464,"kb_used_data":2896,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311376,"statfs":{"total":96636764160,"available":96574849024,"internally_reserved":0,"allocated":2965504,"data_stored":1742575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542192,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60460,"kb_used_data":2896,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311380,"statfs":{"total":96636764160,"available":96574853120,"internally_reserved":0,"allocated":2965504,"data_stored":1742575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":103,"seq":442381631510,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30000,"kb_used_data":2328,"kb_used_omap":12,"kb_used_meta":27635,"kb_avail":94341840,"statfs":{"total":96636764160,"available":96606044160,"internally_reserved":0,"allocated":2383872,"data_stored":1158020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12804,"internal_metadata":28298748},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6121,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15274,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-04T17:49:24.004 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-04T17:49:24.297 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:49:24.298 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-09-04T17:49:24.313 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":303,"stamp":"2024-09-04T17:49:23.130883+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":939,"num_object_clones":0,"num_object_copies":1878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":939,"num_whiteouts":931,"num_read":142346,"num_read_kb":876804,"num_write":20402,"num_write_kb":405906,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6763,"num_bytes_recovered":238961871,"num_keys_recovered":777,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18286,"ondisk_log_size":18286,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":388664,"kb_used_data":19764,"kb_used_omap":46,"kb_used_meta":368657,"kb_avail":754586056,"statfs":{"total":773094113280,"available":772696121344,"internally_reserved":0,"allocated":20238336,"data_stored":10427533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":47946,"internal_metadata":377504950},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001829"},"pg_stats":[{"pgid":"3.1f","version":"178'501","reported_seq":3133,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.470279+0000","last_change":"2024-09-04T17:48:56.059852+0000","last_active":"2024-09-04T17:49:20.470279+0000","last_peered":"2024-09-04T17:49:20.470279+0000","last_clean":"2024-09-04T17:49:20.470279+0000","last_became_active":"2024-09-04T17:48:09.845199+0000","last_became_peered":"2024-09-04T17:48:09.845199+0000","last_unstale":"2024-09-04T17:49:20.470279+0000","last_undegraded":"2024-09-04T17:49:20.470279+0000","last_fullsized":"2024-09-04T17:49:20.470279+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T21:36:09.244561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003726699,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":2983,"num_read_kb":27190,"num_write":524,"num_write_kb":10445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":24433786,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"175'657","reported_seq":3987,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.935494+0000","last_change":"2024-09-04T17:49:05.876073+0000","last_active":"2024-09-04T17:49:19.935494+0000","last_peered":"2024-09-04T17:49:19.935494+0000","last_clean":"2024-09-04T17:49:19.935494+0000","last_became_active":"2024-09-04T17:49:05.874586+0000","last_became_peered":"2024-09-04T17:49:05.874586+0000","last_unstale":"2024-09-04T17:49:19.935494+0000","last_undegraded":"2024-09-04T17:49:19.935494+0000","last_fullsized":"2024-09-04T17:49:19.935494+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:25:15.903921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001926142,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4757,"num_read_kb":30502,"num_write":815,"num_write_kb":10573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":322,"num_bytes_recovered":15412196,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"175'548","reported_seq":2554,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.187160+0000","last_change":"2024-09-04T17:48:56.060983+0000","last_active":"2024-09-04T17:49:20.187160+0000","last_peered":"2024-09-04T17:49:20.187160+0000","last_clean":"2024-09-04T17:49:20.187160+0000","last_became_active":"2024-09-04T17:45:55.955213+0000","last_became_peered":"2024-09-04T17:45:55.955213+0000","last_unstale":"2024-09-04T17:49:20.187160+0000","last_undegraded":"2024-09-04T17:49:20.187160+0000","last_fullsized":"2024-09-04T17:49:20.187160+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":548,"log_dups_size":0,"ondisk_log_size":548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T19:29:02.548970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022810360000000002,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":2820,"num_read_kb":20284,"num_write":623,"num_write_kb":9401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"178'758","reported_seq":6826,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.187131+0000","last_change":"2024-09-04T17:48:56.055219+0000","last_active":"2024-09-04T17:49:20.187131+0000","last_peered":"2024-09-04T17:49:20.187131+0000","last_clean":"2024-09-04T17:49:20.187131+0000","last_became_active":"2024-09-04T17:45:55.951880+0000","last_became_peered":"2024-09-04T17:45:55.951880+0000","last_unstale":"2024-09-04T17:49:20.187131+0000","last_undegraded":"2024-09-04T17:49:20.187131+0000","last_fullsized":"2024-09-04T17:49:20.187131+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":758,"log_dups_size":0,"ondisk_log_size":758,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:58:30.591123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025730409999999999,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":35,"num_read":7473,"num_read_kb":48018,"num_write":874,"num_write_kb":11556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"175'597","reported_seq":2410,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.469937+0000","last_change":"2024-09-04T17:48:56.058271+0000","last_active":"2024-09-04T17:49:20.469937+0000","last_peered":"2024-09-04T17:49:20.469937+0000","last_clean":"2024-09-04T17:49:20.469937+0000","last_became_active":"2024-09-04T17:48:09.853793+0000","last_became_peered":"2024-09-04T17:48:09.853793+0000","last_unstale":"2024-09-04T17:49:20.469937+0000","last_undegraded":"2024-09-04T17:49:20.469937+0000","last_fullsized":"2024-09-04T17:49:20.469937+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":597,"log_dups_size":0,"ondisk_log_size":597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:56:08.051891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026463849999999998,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":3715,"num_read_kb":8148,"num_write":647,"num_write_kb":4600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":1993914,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"175'424","reported_seq":6455,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.240117+0000","last_change":"2024-09-04T17:49:10.140837+0000","last_active":"2024-09-04T17:49:19.240117+0000","last_peered":"2024-09-04T17:49:19.240117+0000","last_clean":"2024-09-04T17:49:19.240117+0000","last_became_active":"2024-09-04T17:49:10.137472+0000","last_became_peered":"2024-09-04T17:49:10.137472+0000","last_unstale":"2024-09-04T17:49:19.240117+0000","last_undegraded":"2024-09-04T17:49:19.240117+0000","last_fullsized":"2024-09-04T17:49:19.240117+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":424,"log_dups_size":0,"ondisk_log_size":424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:51:59.854799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000885438,"stat_sum":{"num_bytes":19,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":25,"num_read":5992,"num_read_kb":45852,"num_write":376,"num_write_kb":15447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":5992020,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"178'442","reported_seq":2720,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.237063+0000","last_change":"2024-09-04T17:49:05.878999+0000","last_active":"2024-09-04T17:49:19.237063+0000","last_peered":"2024-09-04T17:49:19.237063+0000","last_clean":"2024-09-04T17:49:19.237063+0000","last_became_active":"2024-09-04T17:49:05.876985+0000","last_became_peered":"2024-09-04T17:49:05.876985+0000","last_unstale":"2024-09-04T17:49:19.237063+0000","last_undegraded":"2024-09-04T17:49:19.237063+0000","last_fullsized":"2024-09-04T17:49:19.237063+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:42:50.926456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001467744,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":2248,"num_read_kb":16196,"num_write":460,"num_write_kb":4617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":494,"num_bytes_recovered":4911830,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"178'508","reported_seq":2560,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.935295+0000","last_change":"2024-09-04T17:49:07.880474+0000","last_active":"2024-09-04T17:49:19.935295+0000","last_peered":"2024-09-04T17:49:19.935295+0000","last_clean":"2024-09-04T17:49:19.935295+0000","last_became_active":"2024-09-04T17:49:07.878360+0000","last_became_peered":"2024-09-04T17:49:07.878360+0000","last_unstale":"2024-09-04T17:49:19.935295+0000","last_undegraded":"2024-09-04T17:49:19.935295+0000","last_fullsized":"2024-09-04T17:49:19.935295+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:25:31.312054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017798989999999999,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":8892,"num_read_kb":35489,"num_write":443,"num_write_kb":14429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":597,"num_bytes_recovered":28259997,"num_keys_recovered":141,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"178'331","reported_seq":2390,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.470375+0000","last_change":"2024-09-04T17:48:56.056526+0000","last_active":"2024-09-04T17:49:20.470375+0000","last_peered":"2024-09-04T17:49:20.470375+0000","last_clean":"2024-09-04T17:49:20.470375+0000","last_became_active":"2024-09-04T17:45:55.958625+0000","last_became_peered":"2024-09-04T17:45:55.958625+0000","last_unstale":"2024-09-04T17:49:20.470375+0000","last_undegraded":"2024-09-04T17:49:20.470375+0000","last_fullsized":"2024-09-04T17:49:20.470375+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":331,"log_dups_size":0,"ondisk_log_size":331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T22:39:37.983710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001574039,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":1612,"num_read_kb":20939,"num_write":316,"num_write_kb":18406,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"175'364","reported_seq":2051,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.187266+0000","last_change":"2024-09-04T17:48:56.060132+0000","last_active":"2024-09-04T17:49:20.187266+0000","last_peered":"2024-09-04T17:49:20.187266+0000","last_clean":"2024-09-04T17:49:20.187266+0000","last_became_active":"2024-09-04T17:45:55.953254+0000","last_became_peered":"2024-09-04T17:45:55.953254+0000","last_unstale":"2024-09-04T17:49:20.187266+0000","last_undegraded":"2024-09-04T17:49:20.187266+0000","last_fullsized":"2024-09-04T17:49:20.187266+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":364,"log_dups_size":0,"ondisk_log_size":364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T21:59:17.936385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0044959860000000004,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":1729,"num_read_kb":10785,"num_write":396,"num_write_kb":7391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"174'510","reported_seq":3054,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-04T17:49:18.308234+0000","last_change":"2024-09-04T17:49:06.889185+0000","last_active":"2024-09-04T17:49:18.308234+0000","last_peered":"2024-09-04T17:49:18.308234+0000","last_clean":"2024-09-04T17:49:18.308234+0000","last_became_active":"2024-09-04T17:49:06.884929+0000","last_became_peered":"2024-09-04T17:49:06.884929+0000","last_unstale":"2024-09-04T17:49:18.308234+0000","last_undegraded":"2024-09-04T17:49:18.308234+0000","last_fullsized":"2024-09-04T17:49:18.308234+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":510,"log_dups_size":0,"ondisk_log_size":510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:02:50.836583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001003051,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":4020,"num_read_kb":16975,"num_write":525,"num_write_kb":4439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":446,"num_bytes_recovered":12989343,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"178'775","reported_seq":2528,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.240494+0000","last_change":"2024-09-04T17:49:10.140905+0000","last_active":"2024-09-04T17:49:19.240494+0000","last_peered":"2024-09-04T17:49:19.240494+0000","last_clean":"2024-09-04T17:49:19.240494+0000","last_became_active":"2024-09-04T17:49:10.137573+0000","last_became_peered":"2024-09-04T17:49:10.137573+0000","last_unstale":"2024-09-04T17:49:19.240494+0000","last_undegraded":"2024-09-04T17:49:19.240494+0000","last_fullsized":"2024-09-04T17:49:19.240494+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":775,"log_dups_size":0,"ondisk_log_size":775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:56:50.281112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027107059999999998,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":29,"num_read":4009,"num_read_kb":29862,"num_write":1001,"num_write_kb":35909,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":235,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.080313+0000","last_change":"2024-09-04T17:48:11.850550+0000","last_active":"2024-09-04T17:49:20.080313+0000","last_peered":"2024-09-04T17:49:20.080313+0000","last_clean":"2024-09-04T17:49:20.080313+0000","last_became_active":"2024-09-04T17:48:11.849710+0000","last_became_peered":"2024-09-04T17:48:11.849710+0000","last_unstale":"2024-09-04T17:49:20.080313+0000","last_undegraded":"2024-09-04T17:49:20.080313+0000","last_fullsized":"2024-09-04T17:49:20.080313+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:37:18.791459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001226074,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"178'409","reported_seq":3430,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.470300+0000","last_change":"2024-09-04T17:49:07.876177+0000","last_active":"2024-09-04T17:49:20.470300+0000","last_peered":"2024-09-04T17:49:20.470300+0000","last_clean":"2024-09-04T17:49:20.470300+0000","last_became_active":"2024-09-04T17:49:07.874501+0000","last_became_peered":"2024-09-04T17:49:07.874501+0000","last_unstale":"2024-09-04T17:49:20.470300+0000","last_undegraded":"2024-09-04T17:49:20.470300+0000","last_fullsized":"2024-09-04T17:49:20.470300+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":409,"log_dups_size":0,"ondisk_log_size":409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:52:42.831486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001494427,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":2485,"num_read_kb":14550,"num_write":410,"num_write_kb":5528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":919873,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":40,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.237456+0000","last_change":"2024-09-04T17:49:03.886949+0000","last_active":"2024-09-04T17:49:19.237456+0000","last_peered":"2024-09-04T17:49:19.237456+0000","last_clean":"2024-09-04T17:49:19.237456+0000","last_became_active":"2024-09-04T17:49:03.886525+0000","last_became_peered":"2024-09-04T17:49:03.886525+0000","last_unstale":"2024-09-04T17:49:19.237456+0000","last_undegraded":"2024-09-04T17:49:19.237456+0000","last_fullsized":"2024-09-04T17:49:19.237456+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:24:57.789704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011364179999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"175'469","reported_seq":3004,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.935427+0000","last_change":"2024-09-04T17:49:05.877106+0000","last_active":"2024-09-04T17:49:19.935427+0000","last_peered":"2024-09-04T17:49:19.935427+0000","last_clean":"2024-09-04T17:49:19.935427+0000","last_became_active":"2024-09-04T17:49:05.874579+0000","last_became_peered":"2024-09-04T17:49:05.874579+0000","last_unstale":"2024-09-04T17:49:19.935427+0000","last_undegraded":"2024-09-04T17:49:19.935427+0000","last_fullsized":"2024-09-04T17:49:19.935427+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":469,"log_dups_size":0,"ondisk_log_size":469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:32:31.466355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019775819999999999,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":2953,"num_read_kb":23389,"num_write":575,"num_write_kb":21569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":313,"num_bytes_recovered":1588111,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":370,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-04T17:49:18.308514+0000","last_change":"2024-09-04T17:45:49.476607+0000","last_active":"2024-09-04T17:49:18.308514+0000","last_peered":"2024-09-04T17:49:18.308514+0000","last_clean":"2024-09-04T17:49:18.308514+0000","last_became_active":"2024-09-04T17:45:47.369308+0000","last_became_peered":"2024-09-04T17:45:47.369308+0000","last_unstale":"2024-09-04T17:49:18.308514+0000","last_undegraded":"2024-09-04T17:49:18.308514+0000","last_fullsized":"2024-09-04T17:49:18.308514+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:17:13.378553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001029826,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"175'410","reported_seq":2886,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.237416+0000","last_change":"2024-09-04T17:49:07.879430+0000","last_active":"2024-09-04T17:49:19.237416+0000","last_peered":"2024-09-04T17:49:19.237416+0000","last_clean":"2024-09-04T17:49:19.237416+0000","last_became_active":"2024-09-04T17:49:07.877184+0000","last_became_peered":"2024-09-04T17:49:07.877184+0000","last_unstale":"2024-09-04T17:49:19.237416+0000","last_undegraded":"2024-09-04T17:49:19.237416+0000","last_fullsized":"2024-09-04T17:49:19.237416+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T21:09:45.025593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029821800000000001,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":31,"num_read":2672,"num_read_kb":18782,"num_write":436,"num_write_kb":9856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":13375622,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.237128+0000","last_change":"2024-09-04T17:49:03.886170+0000","last_active":"2024-09-04T17:49:19.237128+0000","last_peered":"2024-09-04T17:49:19.237128+0000","last_clean":"2024-09-04T17:49:19.237128+0000","last_became_active":"2024-09-04T17:49:03.885776+0000","last_became_peered":"2024-09-04T17:49:03.885776+0000","last_unstale":"2024-09-04T17:49:19.237128+0000","last_undegraded":"2024-09-04T17:49:19.237128+0000","last_fullsized":"2024-09-04T17:49:19.237128+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T19:30:49.636681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"175'358","reported_seq":3042,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.187305+0000","last_change":"2024-09-04T17:49:06.887617+0000","last_active":"2024-09-04T17:49:20.187305+0000","last_peered":"2024-09-04T17:49:20.187305+0000","last_clean":"2024-09-04T17:49:20.187305+0000","last_became_active":"2024-09-04T17:49:06.885686+0000","last_became_peered":"2024-09-04T17:49:06.885686+0000","last_unstale":"2024-09-04T17:49:20.187305+0000","last_undegraded":"2024-09-04T17:49:20.187305+0000","last_fullsized":"2024-09-04T17:49:20.187305+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T22:44:47.142525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019787199999999998,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":2555,"num_read_kb":7639,"num_write":305,"num_write_kb":7011,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":391,"num_bytes_recovered":15596571,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"178'887","reported_seq":3553,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.240376+0000","last_change":"2024-09-04T17:49:10.141003+0000","last_active":"2024-09-04T17:49:19.240376+0000","last_peered":"2024-09-04T17:49:19.240376+0000","last_clean":"2024-09-04T17:49:19.240376+0000","last_became_active":"2024-09-04T17:49:10.137895+0000","last_became_peered":"2024-09-04T17:49:10.137895+0000","last_unstale":"2024-09-04T17:49:19.240376+0000","last_undegraded":"2024-09-04T17:49:19.240376+0000","last_fullsized":"2024-09-04T17:49:19.240376+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":887,"log_dups_size":0,"ondisk_log_size":887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:24:10.064606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032310310000000001,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":29,"num_read":4673,"num_read_kb":26561,"num_write":1100,"num_write_kb":14222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":398,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.186827+0000","last_change":"2024-09-04T17:49:03.892940+0000","last_active":"2024-09-04T17:49:20.186827+0000","last_peered":"2024-09-04T17:49:20.186827+0000","last_clean":"2024-09-04T17:49:20.186827+0000","last_became_active":"2024-09-04T17:49:03.887053+0000","last_became_peered":"2024-09-04T17:49:03.887053+0000","last_unstale":"2024-09-04T17:49:20.186827+0000","last_undegraded":"2024-09-04T17:49:20.186827+0000","last_fullsized":"2024-09-04T17:49:20.186827+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:30:44.144337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026316069999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"175'453","reported_seq":3440,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.237166+0000","last_change":"2024-09-04T17:49:07.878051+0000","last_active":"2024-09-04T17:49:19.237166+0000","last_peered":"2024-09-04T17:49:19.237166+0000","last_clean":"2024-09-04T17:49:19.237166+0000","last_became_active":"2024-09-04T17:49:07.876339+0000","last_became_peered":"2024-09-04T17:49:07.876339+0000","last_unstale":"2024-09-04T17:49:19.237166+0000","last_undegraded":"2024-09-04T17:49:19.237166+0000","last_fullsized":"2024-09-04T17:49:19.237166+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":453,"log_dups_size":0,"ondisk_log_size":453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T22:43:32.300162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00120166,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":36,"num_read":3566,"num_read_kb":22521,"num_write":365,"num_write_kb":4893,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":261,"num_bytes_recovered":5692590,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":370,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.236829+0000","last_change":"2024-09-04T17:49:02.864996+0000","last_active":"2024-09-04T17:49:19.236829+0000","last_peered":"2024-09-04T17:49:19.236829+0000","last_clean":"2024-09-04T17:49:19.236829+0000","last_became_active":"2024-09-04T17:49:02.864647+0000","last_became_peered":"2024-09-04T17:49:02.864647+0000","last_unstale":"2024-09-04T17:49:19.236829+0000","last_undegraded":"2024-09-04T17:49:19.236829+0000","last_fullsized":"2024-09-04T17:49:19.236829+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:41:35.190910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001025195,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"178'637","reported_seq":3375,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.469830+0000","last_change":"2024-09-04T17:48:56.058939+0000","last_active":"2024-09-04T17:49:20.469830+0000","last_peered":"2024-09-04T17:49:20.469830+0000","last_clean":"2024-09-04T17:49:20.469830+0000","last_became_active":"2024-09-04T17:48:11.845510+0000","last_became_peered":"2024-09-04T17:48:11.845510+0000","last_unstale":"2024-09-04T17:49:20.469830+0000","last_undegraded":"2024-09-04T17:49:20.469830+0000","last_fullsized":"2024-09-04T17:49:20.469830+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":637,"log_dups_size":0,"ondisk_log_size":637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:14:18.552668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031660479999999999,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":3316,"num_read_kb":40546,"num_write":672,"num_write_kb":23359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":240,"num_bytes_recovered":21297737,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":382,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-04T17:49:18.308065+0000","last_change":"2024-09-04T17:49:03.874392+0000","last_active":"2024-09-04T17:49:18.308065+0000","last_peered":"2024-09-04T17:49:18.308065+0000","last_clean":"2024-09-04T17:49:18.308065+0000","last_became_active":"2024-09-04T17:49:03.870678+0000","last_became_peered":"2024-09-04T17:49:03.870678+0000","last_unstale":"2024-09-04T17:49:18.308065+0000","last_undegraded":"2024-09-04T17:49:18.308065+0000","last_fullsized":"2024-09-04T17:49:18.308065+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:49:38.360230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010553590000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"175'622","reported_seq":3059,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.935307+0000","last_change":"2024-09-04T17:49:07.879558+0000","last_active":"2024-09-04T17:49:19.935307+0000","last_peered":"2024-09-04T17:49:19.935307+0000","last_clean":"2024-09-04T17:49:19.935307+0000","last_became_active":"2024-09-04T17:49:07.876340+0000","last_became_peered":"2024-09-04T17:49:07.876340+0000","last_unstale":"2024-09-04T17:49:19.935307+0000","last_undegraded":"2024-09-04T17:49:19.935307+0000","last_fullsized":"2024-09-04T17:49:19.935307+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":622,"log_dups_size":0,"ondisk_log_size":622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:20:26.042540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021278730000000002,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":5714,"num_read_kb":30160,"num_write":740,"num_write_kb":13818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":405,"num_bytes_recovered":16173004,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":382,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.186955+0000","last_change":"2024-09-04T17:47:41.919544+0000","last_active":"2024-09-04T17:49:20.186955+0000","last_peered":"2024-09-04T17:49:20.186955+0000","last_clean":"2024-09-04T17:49:20.186955+0000","last_became_active":"2024-09-04T17:47:41.919027+0000","last_became_peered":"2024-09-04T17:47:41.919027+0000","last_unstale":"2024-09-04T17:49:20.186955+0000","last_undegraded":"2024-09-04T17:49:20.186955+0000","last_fullsized":"2024-09-04T17:49:20.186955+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:42:55.688562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086618299999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":461,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-04T17:49:18.308163+0000","last_change":"2024-09-04T17:45:42.341149+0000","last_active":"2024-09-04T17:49:18.308163+0000","last_peered":"2024-09-04T17:49:18.308163+0000","last_clean":"2024-09-04T17:49:18.308163+0000","last_became_active":"2024-09-04T17:45:42.339286+0000","last_became_peered":"2024-09-04T17:45:42.339286+0000","last_unstale":"2024-09-04T17:49:18.308163+0000","last_undegraded":"2024-09-04T17:49:18.308163+0000","last_fullsized":"2024-09-04T17:49:18.308163+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:41.305128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:41.305128+0000","last_clean_scrub_stamp":"2024-09-04T17:45:41.305128+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:52:24.860049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"178'548","reported_seq":3065,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.935375+0000","last_change":"2024-09-04T17:49:07.883398+0000","last_active":"2024-09-04T17:49:19.935375+0000","last_peered":"2024-09-04T17:49:19.935375+0000","last_clean":"2024-09-04T17:49:19.935375+0000","last_became_active":"2024-09-04T17:49:07.879442+0000","last_became_peered":"2024-09-04T17:49:07.879442+0000","last_unstale":"2024-09-04T17:49:19.935375+0000","last_undegraded":"2024-09-04T17:49:19.935375+0000","last_fullsized":"2024-09-04T17:49:19.935375+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":548,"log_dups_size":0,"ondisk_log_size":548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:55:18.249867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011779189999999999,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":2788,"num_read_kb":17264,"num_write":577,"num_write_kb":29373,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":445,"num_bytes_recovered":17940277,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"178'716","reported_seq":4731,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.240231+0000","last_change":"2024-09-04T17:49:10.137619+0000","last_active":"2024-09-04T17:49:19.240231+0000","last_peered":"2024-09-04T17:49:19.240231+0000","last_clean":"2024-09-04T17:49:19.240231+0000","last_became_active":"2024-09-04T17:49:10.133837+0000","last_became_peered":"2024-09-04T17:49:10.133837+0000","last_unstale":"2024-09-04T17:49:19.240231+0000","last_undegraded":"2024-09-04T17:49:19.240231+0000","last_fullsized":"2024-09-04T17:49:19.240231+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":716,"log_dups_size":0,"ondisk_log_size":716,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T23:43:19.357232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003529346,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":6198,"num_read_kb":37498,"num_write":772,"num_write_kb":18653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":4534479,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"175'656","reported_seq":3645,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.080234+0000","last_change":"2024-09-04T17:48:56.054512+0000","last_active":"2024-09-04T17:49:20.080234+0000","last_peered":"2024-09-04T17:49:20.080234+0000","last_clean":"2024-09-04T17:49:20.080234+0000","last_became_active":"2024-09-04T17:48:00.841070+0000","last_became_peered":"2024-09-04T17:48:00.841070+0000","last_unstale":"2024-09-04T17:49:20.080234+0000","last_undegraded":"2024-09-04T17:49:20.080234+0000","last_fullsized":"2024-09-04T17:49:20.080234+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T19:18:47.901288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003342214,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":31,"num_read":5699,"num_read_kb":16474,"num_write":617,"num_write_kb":2753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":1572433,"num_keys_recovered":66,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"178'770","reported_seq":2984,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.935456+0000","last_change":"2024-09-04T17:49:06.885887+0000","last_active":"2024-09-04T17:49:19.935456+0000","last_peered":"2024-09-04T17:49:19.935456+0000","last_clean":"2024-09-04T17:49:19.935456+0000","last_became_active":"2024-09-04T17:49:06.883685+0000","last_became_peered":"2024-09-04T17:49:06.883685+0000","last_unstale":"2024-09-04T17:49:19.935456+0000","last_undegraded":"2024-09-04T17:49:19.935456+0000","last_fullsized":"2024-09-04T17:49:19.935456+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":770,"log_dups_size":0,"ondisk_log_size":770,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T17:55:41.907650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002090967,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":4267,"num_read_kb":14813,"num_write":915,"num_write_kb":7927,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":459,"num_bytes_recovered":5211501,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"175'647","reported_seq":9166,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.187139+0000","last_change":"2024-09-04T17:49:07.884097+0000","last_active":"2024-09-04T17:49:20.187139+0000","last_peered":"2024-09-04T17:49:20.187139+0000","last_clean":"2024-09-04T17:49:20.187139+0000","last_became_active":"2024-09-04T17:49:07.880558+0000","last_became_peered":"2024-09-04T17:49:07.880558+0000","last_unstale":"2024-09-04T17:49:20.187139+0000","last_undegraded":"2024-09-04T17:49:20.187139+0000","last_fullsized":"2024-09-04T17:49:20.187139+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":647,"log_dups_size":0,"ondisk_log_size":647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:43:25.854985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096708700000000005,"stat_sum":{"num_bytes":0,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":34,"num_read":9052,"num_read_kb":71439,"num_write":754,"num_write_kb":16863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":537,"num_bytes_recovered":8281724,"num_keys_recovered":57,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"178'1358","reported_seq":4585,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-04T17:49:18.308408+0000","last_change":"2024-09-04T17:48:56.066067+0000","last_active":"2024-09-04T17:49:18.308408+0000","last_peered":"2024-09-04T17:49:18.308408+0000","last_clean":"2024-09-04T17:49:18.308408+0000","last_became_active":"2024-09-04T17:48:12.850381+0000","last_became_peered":"2024-09-04T17:48:12.850381+0000","last_unstale":"2024-09-04T17:49:18.308408+0000","last_undegraded":"2024-09-04T17:49:18.308408+0000","last_fullsized":"2024-09-04T17:49:18.308408+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":1358,"log_dups_size":0,"ondisk_log_size":1358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T18:53:56.503643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002193296,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":31,"num_read":8863,"num_read_kb":26532,"num_write":2254,"num_write_kb":13198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":179,"num_bytes_recovered":18877339,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"178'541","reported_seq":6076,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-04T17:49:18.308579+0000","last_change":"2024-09-04T17:48:56.056521+0000","last_active":"2024-09-04T17:49:18.308579+0000","last_peered":"2024-09-04T17:49:18.308579+0000","last_clean":"2024-09-04T17:49:18.308579+0000","last_became_active":"2024-09-04T17:48:11.849605+0000","last_became_peered":"2024-09-04T17:48:11.849605+0000","last_unstale":"2024-09-04T17:49:18.308579+0000","last_undegraded":"2024-09-04T17:49:18.308579+0000","last_fullsized":"2024-09-04T17:49:18.308579+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":541,"log_dups_size":0,"ondisk_log_size":541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:57:37.250572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001652911,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":6678,"num_read_kb":28511,"num_write":523,"num_write_kb":11228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":7806862,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"178'348","reported_seq":3231,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-04T17:49:18.308740+0000","last_change":"2024-09-04T17:48:56.063351+0000","last_active":"2024-09-04T17:49:18.308740+0000","last_peered":"2024-09-04T17:49:18.308740+0000","last_clean":"2024-09-04T17:49:18.308740+0000","last_became_active":"2024-09-04T17:45:55.956734+0000","last_became_peered":"2024-09-04T17:45:55.956734+0000","last_unstale":"2024-09-04T17:49:18.308740+0000","last_undegraded":"2024-09-04T17:49:18.308740+0000","last_fullsized":"2024-09-04T17:49:18.308740+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T23:04:26.785513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0065435850000000002,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":2605,"num_read_kb":44969,"num_write":330,"num_write_kb":13427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"175'630","reported_seq":3858,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.080256+0000","last_change":"2024-09-04T17:48:56.053981+0000","last_active":"2024-09-04T17:49:20.080256+0000","last_peered":"2024-09-04T17:49:20.080256+0000","last_clean":"2024-09-04T17:49:20.080256+0000","last_became_active":"2024-09-04T17:48:00.843136+0000","last_became_peered":"2024-09-04T17:48:00.843136+0000","last_unstale":"2024-09-04T17:49:20.080256+0000","last_undegraded":"2024-09-04T17:49:20.080256+0000","last_fullsized":"2024-09-04T17:49:20.080256+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":630,"log_dups_size":0,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T23:36:00.802962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025156979999999998,"stat_sum":{"num_bytes":0,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":37,"num_read":4786,"num_read_kb":18960,"num_write":673,"num_write_kb":12854,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":3377599,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"175'599","reported_seq":6340,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-04T17:49:18.308074+0000","last_change":"2024-09-04T17:48:56.055370+0000","last_active":"2024-09-04T17:49:18.308074+0000","last_peered":"2024-09-04T17:49:18.308074+0000","last_clean":"2024-09-04T17:49:18.308074+0000","last_became_active":"2024-09-04T17:45:55.951870+0000","last_became_peered":"2024-09-04T17:45:55.951870+0000","last_unstale":"2024-09-04T17:49:18.308074+0000","last_undegraded":"2024-09-04T17:49:18.308074+0000","last_fullsized":"2024-09-04T17:49:18.308074+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:32:28.517472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012658840000000001,"stat_sum":{"num_bytes":0,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":38,"num_read":7163,"num_read_kb":61439,"num_write":584,"num_write_kb":17576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"175'425","reported_seq":4225,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.469949+0000","last_change":"2024-09-04T17:48:56.055907+0000","last_active":"2024-09-04T17:49:20.469949+0000","last_peered":"2024-09-04T17:49:20.469949+0000","last_clean":"2024-09-04T17:49:20.469949+0000","last_became_active":"2024-09-04T17:48:14.943778+0000","last_became_peered":"2024-09-04T17:48:14.943778+0000","last_unstale":"2024-09-04T17:49:20.469949+0000","last_undegraded":"2024-09-04T17:49:20.469949+0000","last_fullsized":"2024-09-04T17:49:20.469949+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":425,"log_dups_size":0,"ondisk_log_size":425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:28:33.747314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011847660000000001,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":31,"num_read":3577,"num_read_kb":21719,"num_write":447,"num_write_kb":8575,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":290,"num_bytes_recovered":2723025,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"175'313","reported_seq":2693,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-04T17:49:18.308158+0000","last_change":"2024-09-04T17:48:56.055871+0000","last_active":"2024-09-04T17:49:18.308158+0000","last_peered":"2024-09-04T17:49:18.308158+0000","last_clean":"2024-09-04T17:49:18.308158+0000","last_became_active":"2024-09-04T17:45:55.954544+0000","last_became_peered":"2024-09-04T17:45:55.954544+0000","last_unstale":"2024-09-04T17:49:18.308158+0000","last_undegraded":"2024-09-04T17:49:18.308158+0000","last_fullsized":"2024-09-04T17:49:18.308158+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:20:57.091315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001451911,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":2420,"num_read_kb":22743,"num_write":250,"num_write_kb":4380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":935,"num_object_clones":0,"num_object_copies":1870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":935,"num_whiteouts":931,"num_read":142280,"num_read_kb":876749,"num_write":20299,"num_write_kb":404276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6761,"num_bytes_recovered":238961833,"num_keys_recovered":777,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21395,"internal_metadata":0},"log_size":18211,"ondisk_log_size":18211,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":117,"seq":502511173651,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44384,"kb_used_data":2316,"kb_used_omap":22,"kb_used_meta":42025,"kb_avail":94327456,"statfs":{"total":96636764160,"available":96591314944,"internally_reserved":0,"allocated":2371584,"data_stored":1152207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22742,"internal_metadata":43034410},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509488,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51568,"kb_used_data":2332,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320272,"statfs":{"total":96636764160,"available":96583958528,"internally_reserved":0,"allocated":2387968,"data_stored":1158039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509489,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51572,"kb_used_data":2332,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320268,"statfs":{"total":96636764160,"available":96583954432,"internally_reserved":0,"allocated":2387968,"data_stored":1158039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":48,"seq":206158430245,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60848,"kb_used_data":2328,"kb_used_omap":4,"kb_used_meta":58491,"kb_avail":94310992,"statfs":{"total":96636764160,"available":96574455808,"internally_reserved":0,"allocated":2383872,"data_stored":1158020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4430,"internal_metadata":59895474},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509488,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29368,"kb_used_data":2336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342472,"statfs":{"total":96636764160,"available":96606691328,"internally_reserved":0,"allocated":2392064,"data_stored":1158058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509488,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60464,"kb_used_data":2896,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311376,"statfs":{"total":96636764160,"available":96574849024,"internally_reserved":0,"allocated":2965504,"data_stored":1742575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542192,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60460,"kb_used_data":2896,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311380,"statfs":{"total":96636764160,"available":96574853120,"internally_reserved":0,"allocated":2965504,"data_stored":1742575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":103,"seq":442381631510,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30000,"kb_used_data":2328,"kb_used_omap":12,"kb_used_meta":27635,"kb_avail":94341840,"statfs":{"total":96636764160,"available":96606044160,"internally_reserved":0,"allocated":2383872,"data_stored":1158020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12804,"internal_metadata":28298748},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6121,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15274,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-04T17:49:24.315 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-04T17:49:24.316 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-04T17:49:24.608 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:49:24.609 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-09-04T17:49:24.625 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":303,"stamp":"2024-09-04T17:49:23.130883+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":939,"num_object_clones":0,"num_object_copies":1878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":939,"num_whiteouts":931,"num_read":142346,"num_read_kb":876804,"num_write":20402,"num_write_kb":405906,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6763,"num_bytes_recovered":238961871,"num_keys_recovered":777,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18286,"ondisk_log_size":18286,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":388664,"kb_used_data":19764,"kb_used_omap":46,"kb_used_meta":368657,"kb_avail":754586056,"statfs":{"total":773094113280,"available":772696121344,"internally_reserved":0,"allocated":20238336,"data_stored":10427533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":47946,"internal_metadata":377504950},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001829"},"pg_stats":[{"pgid":"3.1f","version":"178'501","reported_seq":3133,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.470279+0000","last_change":"2024-09-04T17:48:56.059852+0000","last_active":"2024-09-04T17:49:20.470279+0000","last_peered":"2024-09-04T17:49:20.470279+0000","last_clean":"2024-09-04T17:49:20.470279+0000","last_became_active":"2024-09-04T17:48:09.845199+0000","last_became_peered":"2024-09-04T17:48:09.845199+0000","last_unstale":"2024-09-04T17:49:20.470279+0000","last_undegraded":"2024-09-04T17:49:20.470279+0000","last_fullsized":"2024-09-04T17:49:20.470279+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T21:36:09.244561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003726699,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":2983,"num_read_kb":27190,"num_write":524,"num_write_kb":10445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":24433786,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"175'657","reported_seq":3987,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.935494+0000","last_change":"2024-09-04T17:49:05.876073+0000","last_active":"2024-09-04T17:49:19.935494+0000","last_peered":"2024-09-04T17:49:19.935494+0000","last_clean":"2024-09-04T17:49:19.935494+0000","last_became_active":"2024-09-04T17:49:05.874586+0000","last_became_peered":"2024-09-04T17:49:05.874586+0000","last_unstale":"2024-09-04T17:49:19.935494+0000","last_undegraded":"2024-09-04T17:49:19.935494+0000","last_fullsized":"2024-09-04T17:49:19.935494+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:25:15.903921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001926142,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4757,"num_read_kb":30502,"num_write":815,"num_write_kb":10573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":322,"num_bytes_recovered":15412196,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"175'548","reported_seq":2554,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.187160+0000","last_change":"2024-09-04T17:48:56.060983+0000","last_active":"2024-09-04T17:49:20.187160+0000","last_peered":"2024-09-04T17:49:20.187160+0000","last_clean":"2024-09-04T17:49:20.187160+0000","last_became_active":"2024-09-04T17:45:55.955213+0000","last_became_peered":"2024-09-04T17:45:55.955213+0000","last_unstale":"2024-09-04T17:49:20.187160+0000","last_undegraded":"2024-09-04T17:49:20.187160+0000","last_fullsized":"2024-09-04T17:49:20.187160+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":548,"log_dups_size":0,"ondisk_log_size":548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T19:29:02.548970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022810360000000002,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":2820,"num_read_kb":20284,"num_write":623,"num_write_kb":9401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"178'758","reported_seq":6826,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.187131+0000","last_change":"2024-09-04T17:48:56.055219+0000","last_active":"2024-09-04T17:49:20.187131+0000","last_peered":"2024-09-04T17:49:20.187131+0000","last_clean":"2024-09-04T17:49:20.187131+0000","last_became_active":"2024-09-04T17:45:55.951880+0000","last_became_peered":"2024-09-04T17:45:55.951880+0000","last_unstale":"2024-09-04T17:49:20.187131+0000","last_undegraded":"2024-09-04T17:49:20.187131+0000","last_fullsized":"2024-09-04T17:49:20.187131+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":758,"log_dups_size":0,"ondisk_log_size":758,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:58:30.591123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025730409999999999,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":35,"num_read":7473,"num_read_kb":48018,"num_write":874,"num_write_kb":11556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"175'597","reported_seq":2410,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.469937+0000","last_change":"2024-09-04T17:48:56.058271+0000","last_active":"2024-09-04T17:49:20.469937+0000","last_peered":"2024-09-04T17:49:20.469937+0000","last_clean":"2024-09-04T17:49:20.469937+0000","last_became_active":"2024-09-04T17:48:09.853793+0000","last_became_peered":"2024-09-04T17:48:09.853793+0000","last_unstale":"2024-09-04T17:49:20.469937+0000","last_undegraded":"2024-09-04T17:49:20.469937+0000","last_fullsized":"2024-09-04T17:49:20.469937+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":597,"log_dups_size":0,"ondisk_log_size":597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:56:08.051891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026463849999999998,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":3715,"num_read_kb":8148,"num_write":647,"num_write_kb":4600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":1993914,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"175'424","reported_seq":6455,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.240117+0000","last_change":"2024-09-04T17:49:10.140837+0000","last_active":"2024-09-04T17:49:19.240117+0000","last_peered":"2024-09-04T17:49:19.240117+0000","last_clean":"2024-09-04T17:49:19.240117+0000","last_became_active":"2024-09-04T17:49:10.137472+0000","last_became_peered":"2024-09-04T17:49:10.137472+0000","last_unstale":"2024-09-04T17:49:19.240117+0000","last_undegraded":"2024-09-04T17:49:19.240117+0000","last_fullsized":"2024-09-04T17:49:19.240117+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":424,"log_dups_size":0,"ondisk_log_size":424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:51:59.854799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000885438,"stat_sum":{"num_bytes":19,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":25,"num_read":5992,"num_read_kb":45852,"num_write":376,"num_write_kb":15447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":5992020,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"178'442","reported_seq":2720,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.237063+0000","last_change":"2024-09-04T17:49:05.878999+0000","last_active":"2024-09-04T17:49:19.237063+0000","last_peered":"2024-09-04T17:49:19.237063+0000","last_clean":"2024-09-04T17:49:19.237063+0000","last_became_active":"2024-09-04T17:49:05.876985+0000","last_became_peered":"2024-09-04T17:49:05.876985+0000","last_unstale":"2024-09-04T17:49:19.237063+0000","last_undegraded":"2024-09-04T17:49:19.237063+0000","last_fullsized":"2024-09-04T17:49:19.237063+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:42:50.926456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001467744,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":2248,"num_read_kb":16196,"num_write":460,"num_write_kb":4617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":494,"num_bytes_recovered":4911830,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"178'508","reported_seq":2560,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.935295+0000","last_change":"2024-09-04T17:49:07.880474+0000","last_active":"2024-09-04T17:49:19.935295+0000","last_peered":"2024-09-04T17:49:19.935295+0000","last_clean":"2024-09-04T17:49:19.935295+0000","last_became_active":"2024-09-04T17:49:07.878360+0000","last_became_peered":"2024-09-04T17:49:07.878360+0000","last_unstale":"2024-09-04T17:49:19.935295+0000","last_undegraded":"2024-09-04T17:49:19.935295+0000","last_fullsized":"2024-09-04T17:49:19.935295+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:25:31.312054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017798989999999999,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":8892,"num_read_kb":35489,"num_write":443,"num_write_kb":14429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":597,"num_bytes_recovered":28259997,"num_keys_recovered":141,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"178'331","reported_seq":2390,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.470375+0000","last_change":"2024-09-04T17:48:56.056526+0000","last_active":"2024-09-04T17:49:20.470375+0000","last_peered":"2024-09-04T17:49:20.470375+0000","last_clean":"2024-09-04T17:49:20.470375+0000","last_became_active":"2024-09-04T17:45:55.958625+0000","last_became_peered":"2024-09-04T17:45:55.958625+0000","last_unstale":"2024-09-04T17:49:20.470375+0000","last_undegraded":"2024-09-04T17:49:20.470375+0000","last_fullsized":"2024-09-04T17:49:20.470375+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":331,"log_dups_size":0,"ondisk_log_size":331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T22:39:37.983710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001574039,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":1612,"num_read_kb":20939,"num_write":316,"num_write_kb":18406,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"175'364","reported_seq":2051,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.187266+0000","last_change":"2024-09-04T17:48:56.060132+0000","last_active":"2024-09-04T17:49:20.187266+0000","last_peered":"2024-09-04T17:49:20.187266+0000","last_clean":"2024-09-04T17:49:20.187266+0000","last_became_active":"2024-09-04T17:45:55.953254+0000","last_became_peered":"2024-09-04T17:45:55.953254+0000","last_unstale":"2024-09-04T17:49:20.187266+0000","last_undegraded":"2024-09-04T17:49:20.187266+0000","last_fullsized":"2024-09-04T17:49:20.187266+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":364,"log_dups_size":0,"ondisk_log_size":364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T21:59:17.936385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0044959860000000004,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":1729,"num_read_kb":10785,"num_write":396,"num_write_kb":7391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"174'510","reported_seq":3054,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-04T17:49:18.308234+0000","last_change":"2024-09-04T17:49:06.889185+0000","last_active":"2024-09-04T17:49:18.308234+0000","last_peered":"2024-09-04T17:49:18.308234+0000","last_clean":"2024-09-04T17:49:18.308234+0000","last_became_active":"2024-09-04T17:49:06.884929+0000","last_became_peered":"2024-09-04T17:49:06.884929+0000","last_unstale":"2024-09-04T17:49:18.308234+0000","last_undegraded":"2024-09-04T17:49:18.308234+0000","last_fullsized":"2024-09-04T17:49:18.308234+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":510,"log_dups_size":0,"ondisk_log_size":510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:02:50.836583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001003051,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":4020,"num_read_kb":16975,"num_write":525,"num_write_kb":4439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":446,"num_bytes_recovered":12989343,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"178'775","reported_seq":2528,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.240494+0000","last_change":"2024-09-04T17:49:10.140905+0000","last_active":"2024-09-04T17:49:19.240494+0000","last_peered":"2024-09-04T17:49:19.240494+0000","last_clean":"2024-09-04T17:49:19.240494+0000","last_became_active":"2024-09-04T17:49:10.137573+0000","last_became_peered":"2024-09-04T17:49:10.137573+0000","last_unstale":"2024-09-04T17:49:19.240494+0000","last_undegraded":"2024-09-04T17:49:19.240494+0000","last_fullsized":"2024-09-04T17:49:19.240494+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":775,"log_dups_size":0,"ondisk_log_size":775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:56:50.281112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027107059999999998,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":29,"num_read":4009,"num_read_kb":29862,"num_write":1001,"num_write_kb":35909,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":235,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.080313+0000","last_change":"2024-09-04T17:48:11.850550+0000","last_active":"2024-09-04T17:49:20.080313+0000","last_peered":"2024-09-04T17:49:20.080313+0000","last_clean":"2024-09-04T17:49:20.080313+0000","last_became_active":"2024-09-04T17:48:11.849710+0000","last_became_peered":"2024-09-04T17:48:11.849710+0000","last_unstale":"2024-09-04T17:49:20.080313+0000","last_undegraded":"2024-09-04T17:49:20.080313+0000","last_fullsized":"2024-09-04T17:49:20.080313+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:37:18.791459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001226074,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"178'409","reported_seq":3430,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.470300+0000","last_change":"2024-09-04T17:49:07.876177+0000","last_active":"2024-09-04T17:49:20.470300+0000","last_peered":"2024-09-04T17:49:20.470300+0000","last_clean":"2024-09-04T17:49:20.470300+0000","last_became_active":"2024-09-04T17:49:07.874501+0000","last_became_peered":"2024-09-04T17:49:07.874501+0000","last_unstale":"2024-09-04T17:49:20.470300+0000","last_undegraded":"2024-09-04T17:49:20.470300+0000","last_fullsized":"2024-09-04T17:49:20.470300+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":409,"log_dups_size":0,"ondisk_log_size":409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:52:42.831486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001494427,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":2485,"num_read_kb":14550,"num_write":410,"num_write_kb":5528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":919873,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":40,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.237456+0000","last_change":"2024-09-04T17:49:03.886949+0000","last_active":"2024-09-04T17:49:19.237456+0000","last_peered":"2024-09-04T17:49:19.237456+0000","last_clean":"2024-09-04T17:49:19.237456+0000","last_became_active":"2024-09-04T17:49:03.886525+0000","last_became_peered":"2024-09-04T17:49:03.886525+0000","last_unstale":"2024-09-04T17:49:19.237456+0000","last_undegraded":"2024-09-04T17:49:19.237456+0000","last_fullsized":"2024-09-04T17:49:19.237456+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:24:57.789704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011364179999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"175'469","reported_seq":3004,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.935427+0000","last_change":"2024-09-04T17:49:05.877106+0000","last_active":"2024-09-04T17:49:19.935427+0000","last_peered":"2024-09-04T17:49:19.935427+0000","last_clean":"2024-09-04T17:49:19.935427+0000","last_became_active":"2024-09-04T17:49:05.874579+0000","last_became_peered":"2024-09-04T17:49:05.874579+0000","last_unstale":"2024-09-04T17:49:19.935427+0000","last_undegraded":"2024-09-04T17:49:19.935427+0000","last_fullsized":"2024-09-04T17:49:19.935427+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":469,"log_dups_size":0,"ondisk_log_size":469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:32:31.466355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019775819999999999,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":2953,"num_read_kb":23389,"num_write":575,"num_write_kb":21569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":313,"num_bytes_recovered":1588111,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":370,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-04T17:49:18.308514+0000","last_change":"2024-09-04T17:45:49.476607+0000","last_active":"2024-09-04T17:49:18.308514+0000","last_peered":"2024-09-04T17:49:18.308514+0000","last_clean":"2024-09-04T17:49:18.308514+0000","last_became_active":"2024-09-04T17:45:47.369308+0000","last_became_peered":"2024-09-04T17:45:47.369308+0000","last_unstale":"2024-09-04T17:49:18.308514+0000","last_undegraded":"2024-09-04T17:49:18.308514+0000","last_fullsized":"2024-09-04T17:49:18.308514+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:17:13.378553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001029826,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"175'410","reported_seq":2886,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.237416+0000","last_change":"2024-09-04T17:49:07.879430+0000","last_active":"2024-09-04T17:49:19.237416+0000","last_peered":"2024-09-04T17:49:19.237416+0000","last_clean":"2024-09-04T17:49:19.237416+0000","last_became_active":"2024-09-04T17:49:07.877184+0000","last_became_peered":"2024-09-04T17:49:07.877184+0000","last_unstale":"2024-09-04T17:49:19.237416+0000","last_undegraded":"2024-09-04T17:49:19.237416+0000","last_fullsized":"2024-09-04T17:49:19.237416+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T21:09:45.025593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029821800000000001,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":31,"num_read":2672,"num_read_kb":18782,"num_write":436,"num_write_kb":9856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":13375622,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.237128+0000","last_change":"2024-09-04T17:49:03.886170+0000","last_active":"2024-09-04T17:49:19.237128+0000","last_peered":"2024-09-04T17:49:19.237128+0000","last_clean":"2024-09-04T17:49:19.237128+0000","last_became_active":"2024-09-04T17:49:03.885776+0000","last_became_peered":"2024-09-04T17:49:03.885776+0000","last_unstale":"2024-09-04T17:49:19.237128+0000","last_undegraded":"2024-09-04T17:49:19.237128+0000","last_fullsized":"2024-09-04T17:49:19.237128+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T19:30:49.636681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"175'358","reported_seq":3042,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.187305+0000","last_change":"2024-09-04T17:49:06.887617+0000","last_active":"2024-09-04T17:49:20.187305+0000","last_peered":"2024-09-04T17:49:20.187305+0000","last_clean":"2024-09-04T17:49:20.187305+0000","last_became_active":"2024-09-04T17:49:06.885686+0000","last_became_peered":"2024-09-04T17:49:06.885686+0000","last_unstale":"2024-09-04T17:49:20.187305+0000","last_undegraded":"2024-09-04T17:49:20.187305+0000","last_fullsized":"2024-09-04T17:49:20.187305+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T22:44:47.142525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019787199999999998,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":2555,"num_read_kb":7639,"num_write":305,"num_write_kb":7011,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":391,"num_bytes_recovered":15596571,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"178'887","reported_seq":3553,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.240376+0000","last_change":"2024-09-04T17:49:10.141003+0000","last_active":"2024-09-04T17:49:19.240376+0000","last_peered":"2024-09-04T17:49:19.240376+0000","last_clean":"2024-09-04T17:49:19.240376+0000","last_became_active":"2024-09-04T17:49:10.137895+0000","last_became_peered":"2024-09-04T17:49:10.137895+0000","last_unstale":"2024-09-04T17:49:19.240376+0000","last_undegraded":"2024-09-04T17:49:19.240376+0000","last_fullsized":"2024-09-04T17:49:19.240376+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":887,"log_dups_size":0,"ondisk_log_size":887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:24:10.064606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032310310000000001,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":29,"num_read":4673,"num_read_kb":26561,"num_write":1100,"num_write_kb":14222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":398,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.186827+0000","last_change":"2024-09-04T17:49:03.892940+0000","last_active":"2024-09-04T17:49:20.186827+0000","last_peered":"2024-09-04T17:49:20.186827+0000","last_clean":"2024-09-04T17:49:20.186827+0000","last_became_active":"2024-09-04T17:49:03.887053+0000","last_became_peered":"2024-09-04T17:49:03.887053+0000","last_unstale":"2024-09-04T17:49:20.186827+0000","last_undegraded":"2024-09-04T17:49:20.186827+0000","last_fullsized":"2024-09-04T17:49:20.186827+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:30:44.144337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026316069999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"175'453","reported_seq":3440,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.237166+0000","last_change":"2024-09-04T17:49:07.878051+0000","last_active":"2024-09-04T17:49:19.237166+0000","last_peered":"2024-09-04T17:49:19.237166+0000","last_clean":"2024-09-04T17:49:19.237166+0000","last_became_active":"2024-09-04T17:49:07.876339+0000","last_became_peered":"2024-09-04T17:49:07.876339+0000","last_unstale":"2024-09-04T17:49:19.237166+0000","last_undegraded":"2024-09-04T17:49:19.237166+0000","last_fullsized":"2024-09-04T17:49:19.237166+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":453,"log_dups_size":0,"ondisk_log_size":453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T22:43:32.300162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00120166,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":36,"num_read":3566,"num_read_kb":22521,"num_write":365,"num_write_kb":4893,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":261,"num_bytes_recovered":5692590,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":370,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.236829+0000","last_change":"2024-09-04T17:49:02.864996+0000","last_active":"2024-09-04T17:49:19.236829+0000","last_peered":"2024-09-04T17:49:19.236829+0000","last_clean":"2024-09-04T17:49:19.236829+0000","last_became_active":"2024-09-04T17:49:02.864647+0000","last_became_peered":"2024-09-04T17:49:02.864647+0000","last_unstale":"2024-09-04T17:49:19.236829+0000","last_undegraded":"2024-09-04T17:49:19.236829+0000","last_fullsized":"2024-09-04T17:49:19.236829+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:41:35.190910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001025195,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"178'637","reported_seq":3375,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.469830+0000","last_change":"2024-09-04T17:48:56.058939+0000","last_active":"2024-09-04T17:49:20.469830+0000","last_peered":"2024-09-04T17:49:20.469830+0000","last_clean":"2024-09-04T17:49:20.469830+0000","last_became_active":"2024-09-04T17:48:11.845510+0000","last_became_peered":"2024-09-04T17:48:11.845510+0000","last_unstale":"2024-09-04T17:49:20.469830+0000","last_undegraded":"2024-09-04T17:49:20.469830+0000","last_fullsized":"2024-09-04T17:49:20.469830+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":637,"log_dups_size":0,"ondisk_log_size":637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:14:18.552668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031660479999999999,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":3316,"num_read_kb":40546,"num_write":672,"num_write_kb":23359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":240,"num_bytes_recovered":21297737,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":382,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-04T17:49:18.308065+0000","last_change":"2024-09-04T17:49:03.874392+0000","last_active":"2024-09-04T17:49:18.308065+0000","last_peered":"2024-09-04T17:49:18.308065+0000","last_clean":"2024-09-04T17:49:18.308065+0000","last_became_active":"2024-09-04T17:49:03.870678+0000","last_became_peered":"2024-09-04T17:49:03.870678+0000","last_unstale":"2024-09-04T17:49:18.308065+0000","last_undegraded":"2024-09-04T17:49:18.308065+0000","last_fullsized":"2024-09-04T17:49:18.308065+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:49:38.360230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010553590000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"175'622","reported_seq":3059,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.935307+0000","last_change":"2024-09-04T17:49:07.879558+0000","last_active":"2024-09-04T17:49:19.935307+0000","last_peered":"2024-09-04T17:49:19.935307+0000","last_clean":"2024-09-04T17:49:19.935307+0000","last_became_active":"2024-09-04T17:49:07.876340+0000","last_became_peered":"2024-09-04T17:49:07.876340+0000","last_unstale":"2024-09-04T17:49:19.935307+0000","last_undegraded":"2024-09-04T17:49:19.935307+0000","last_fullsized":"2024-09-04T17:49:19.935307+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":622,"log_dups_size":0,"ondisk_log_size":622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:20:26.042540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021278730000000002,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":5714,"num_read_kb":30160,"num_write":740,"num_write_kb":13818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":405,"num_bytes_recovered":16173004,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":382,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.186955+0000","last_change":"2024-09-04T17:47:41.919544+0000","last_active":"2024-09-04T17:49:20.186955+0000","last_peered":"2024-09-04T17:49:20.186955+0000","last_clean":"2024-09-04T17:49:20.186955+0000","last_became_active":"2024-09-04T17:47:41.919027+0000","last_became_peered":"2024-09-04T17:47:41.919027+0000","last_unstale":"2024-09-04T17:49:20.186955+0000","last_undegraded":"2024-09-04T17:49:20.186955+0000","last_fullsized":"2024-09-04T17:49:20.186955+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:42:55.688562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086618299999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":461,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-04T17:49:18.308163+0000","last_change":"2024-09-04T17:45:42.341149+0000","last_active":"2024-09-04T17:49:18.308163+0000","last_peered":"2024-09-04T17:49:18.308163+0000","last_clean":"2024-09-04T17:49:18.308163+0000","last_became_active":"2024-09-04T17:45:42.339286+0000","last_became_peered":"2024-09-04T17:45:42.339286+0000","last_unstale":"2024-09-04T17:49:18.308163+0000","last_undegraded":"2024-09-04T17:49:18.308163+0000","last_fullsized":"2024-09-04T17:49:18.308163+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:41.305128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:41.305128+0000","last_clean_scrub_stamp":"2024-09-04T17:45:41.305128+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:52:24.860049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"178'548","reported_seq":3065,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.935375+0000","last_change":"2024-09-04T17:49:07.883398+0000","last_active":"2024-09-04T17:49:19.935375+0000","last_peered":"2024-09-04T17:49:19.935375+0000","last_clean":"2024-09-04T17:49:19.935375+0000","last_became_active":"2024-09-04T17:49:07.879442+0000","last_became_peered":"2024-09-04T17:49:07.879442+0000","last_unstale":"2024-09-04T17:49:19.935375+0000","last_undegraded":"2024-09-04T17:49:19.935375+0000","last_fullsized":"2024-09-04T17:49:19.935375+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":548,"log_dups_size":0,"ondisk_log_size":548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:55:18.249867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011779189999999999,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":2788,"num_read_kb":17264,"num_write":577,"num_write_kb":29373,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":445,"num_bytes_recovered":17940277,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"178'716","reported_seq":4731,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.240231+0000","last_change":"2024-09-04T17:49:10.137619+0000","last_active":"2024-09-04T17:49:19.240231+0000","last_peered":"2024-09-04T17:49:19.240231+0000","last_clean":"2024-09-04T17:49:19.240231+0000","last_became_active":"2024-09-04T17:49:10.133837+0000","last_became_peered":"2024-09-04T17:49:10.133837+0000","last_unstale":"2024-09-04T17:49:19.240231+0000","last_undegraded":"2024-09-04T17:49:19.240231+0000","last_fullsized":"2024-09-04T17:49:19.240231+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":716,"log_dups_size":0,"ondisk_log_size":716,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T23:43:19.357232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003529346,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":6198,"num_read_kb":37498,"num_write":772,"num_write_kb":18653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":4534479,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"175'656","reported_seq":3645,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.080234+0000","last_change":"2024-09-04T17:48:56.054512+0000","last_active":"2024-09-04T17:49:20.080234+0000","last_peered":"2024-09-04T17:49:20.080234+0000","last_clean":"2024-09-04T17:49:20.080234+0000","last_became_active":"2024-09-04T17:48:00.841070+0000","last_became_peered":"2024-09-04T17:48:00.841070+0000","last_unstale":"2024-09-04T17:49:20.080234+0000","last_undegraded":"2024-09-04T17:49:20.080234+0000","last_fullsized":"2024-09-04T17:49:20.080234+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T19:18:47.901288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003342214,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":31,"num_read":5699,"num_read_kb":16474,"num_write":617,"num_write_kb":2753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":1572433,"num_keys_recovered":66,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"178'770","reported_seq":2984,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.935456+0000","last_change":"2024-09-04T17:49:06.885887+0000","last_active":"2024-09-04T17:49:19.935456+0000","last_peered":"2024-09-04T17:49:19.935456+0000","last_clean":"2024-09-04T17:49:19.935456+0000","last_became_active":"2024-09-04T17:49:06.883685+0000","last_became_peered":"2024-09-04T17:49:06.883685+0000","last_unstale":"2024-09-04T17:49:19.935456+0000","last_undegraded":"2024-09-04T17:49:19.935456+0000","last_fullsized":"2024-09-04T17:49:19.935456+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":770,"log_dups_size":0,"ondisk_log_size":770,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T17:55:41.907650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002090967,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":4267,"num_read_kb":14813,"num_write":915,"num_write_kb":7927,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":459,"num_bytes_recovered":5211501,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"175'647","reported_seq":9166,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.187139+0000","last_change":"2024-09-04T17:49:07.884097+0000","last_active":"2024-09-04T17:49:20.187139+0000","last_peered":"2024-09-04T17:49:20.187139+0000","last_clean":"2024-09-04T17:49:20.187139+0000","last_became_active":"2024-09-04T17:49:07.880558+0000","last_became_peered":"2024-09-04T17:49:07.880558+0000","last_unstale":"2024-09-04T17:49:20.187139+0000","last_undegraded":"2024-09-04T17:49:20.187139+0000","last_fullsized":"2024-09-04T17:49:20.187139+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":647,"log_dups_size":0,"ondisk_log_size":647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:43:25.854985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096708700000000005,"stat_sum":{"num_bytes":0,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":34,"num_read":9052,"num_read_kb":71439,"num_write":754,"num_write_kb":16863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":537,"num_bytes_recovered":8281724,"num_keys_recovered":57,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"178'1358","reported_seq":4585,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-04T17:49:18.308408+0000","last_change":"2024-09-04T17:48:56.066067+0000","last_active":"2024-09-04T17:49:18.308408+0000","last_peered":"2024-09-04T17:49:18.308408+0000","last_clean":"2024-09-04T17:49:18.308408+0000","last_became_active":"2024-09-04T17:48:12.850381+0000","last_became_peered":"2024-09-04T17:48:12.850381+0000","last_unstale":"2024-09-04T17:49:18.308408+0000","last_undegraded":"2024-09-04T17:49:18.308408+0000","last_fullsized":"2024-09-04T17:49:18.308408+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":1358,"log_dups_size":0,"ondisk_log_size":1358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T18:53:56.503643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002193296,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":31,"num_read":8863,"num_read_kb":26532,"num_write":2254,"num_write_kb":13198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":179,"num_bytes_recovered":18877339,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"178'541","reported_seq":6076,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-04T17:49:18.308579+0000","last_change":"2024-09-04T17:48:56.056521+0000","last_active":"2024-09-04T17:49:18.308579+0000","last_peered":"2024-09-04T17:49:18.308579+0000","last_clean":"2024-09-04T17:49:18.308579+0000","last_became_active":"2024-09-04T17:48:11.849605+0000","last_became_peered":"2024-09-04T17:48:11.849605+0000","last_unstale":"2024-09-04T17:49:18.308579+0000","last_undegraded":"2024-09-04T17:49:18.308579+0000","last_fullsized":"2024-09-04T17:49:18.308579+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":541,"log_dups_size":0,"ondisk_log_size":541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:57:37.250572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001652911,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":6678,"num_read_kb":28511,"num_write":523,"num_write_kb":11228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":7806862,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"178'348","reported_seq":3231,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-04T17:49:18.308740+0000","last_change":"2024-09-04T17:48:56.063351+0000","last_active":"2024-09-04T17:49:18.308740+0000","last_peered":"2024-09-04T17:49:18.308740+0000","last_clean":"2024-09-04T17:49:18.308740+0000","last_became_active":"2024-09-04T17:45:55.956734+0000","last_became_peered":"2024-09-04T17:45:55.956734+0000","last_unstale":"2024-09-04T17:49:18.308740+0000","last_undegraded":"2024-09-04T17:49:18.308740+0000","last_fullsized":"2024-09-04T17:49:18.308740+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T23:04:26.785513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0065435850000000002,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":2605,"num_read_kb":44969,"num_write":330,"num_write_kb":13427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"175'630","reported_seq":3858,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.080256+0000","last_change":"2024-09-04T17:48:56.053981+0000","last_active":"2024-09-04T17:49:20.080256+0000","last_peered":"2024-09-04T17:49:20.080256+0000","last_clean":"2024-09-04T17:49:20.080256+0000","last_became_active":"2024-09-04T17:48:00.843136+0000","last_became_peered":"2024-09-04T17:48:00.843136+0000","last_unstale":"2024-09-04T17:49:20.080256+0000","last_undegraded":"2024-09-04T17:49:20.080256+0000","last_fullsized":"2024-09-04T17:49:20.080256+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":630,"log_dups_size":0,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T23:36:00.802962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025156979999999998,"stat_sum":{"num_bytes":0,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":37,"num_read":4786,"num_read_kb":18960,"num_write":673,"num_write_kb":12854,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":3377599,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"175'599","reported_seq":6340,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-04T17:49:18.308074+0000","last_change":"2024-09-04T17:48:56.055370+0000","last_active":"2024-09-04T17:49:18.308074+0000","last_peered":"2024-09-04T17:49:18.308074+0000","last_clean":"2024-09-04T17:49:18.308074+0000","last_became_active":"2024-09-04T17:45:55.951870+0000","last_became_peered":"2024-09-04T17:45:55.951870+0000","last_unstale":"2024-09-04T17:49:18.308074+0000","last_undegraded":"2024-09-04T17:49:18.308074+0000","last_fullsized":"2024-09-04T17:49:18.308074+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:32:28.517472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012658840000000001,"stat_sum":{"num_bytes":0,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":38,"num_read":7163,"num_read_kb":61439,"num_write":584,"num_write_kb":17576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"175'425","reported_seq":4225,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.469949+0000","last_change":"2024-09-04T17:48:56.055907+0000","last_active":"2024-09-04T17:49:20.469949+0000","last_peered":"2024-09-04T17:49:20.469949+0000","last_clean":"2024-09-04T17:49:20.469949+0000","last_became_active":"2024-09-04T17:48:14.943778+0000","last_became_peered":"2024-09-04T17:48:14.943778+0000","last_unstale":"2024-09-04T17:49:20.469949+0000","last_undegraded":"2024-09-04T17:49:20.469949+0000","last_fullsized":"2024-09-04T17:49:20.469949+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":425,"log_dups_size":0,"ondisk_log_size":425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:28:33.747314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011847660000000001,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":31,"num_read":3577,"num_read_kb":21719,"num_write":447,"num_write_kb":8575,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":290,"num_bytes_recovered":2723025,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"175'313","reported_seq":2693,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-04T17:49:18.308158+0000","last_change":"2024-09-04T17:48:56.055871+0000","last_active":"2024-09-04T17:49:18.308158+0000","last_peered":"2024-09-04T17:49:18.308158+0000","last_clean":"2024-09-04T17:49:18.308158+0000","last_became_active":"2024-09-04T17:45:55.954544+0000","last_became_peered":"2024-09-04T17:45:55.954544+0000","last_unstale":"2024-09-04T17:49:18.308158+0000","last_undegraded":"2024-09-04T17:49:18.308158+0000","last_fullsized":"2024-09-04T17:49:18.308158+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:20:57.091315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001451911,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":2420,"num_read_kb":22743,"num_write":250,"num_write_kb":4380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":935,"num_object_clones":0,"num_object_copies":1870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":935,"num_whiteouts":931,"num_read":142280,"num_read_kb":876749,"num_write":20299,"num_write_kb":404276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6761,"num_bytes_recovered":238961833,"num_keys_recovered":777,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21395,"internal_metadata":0},"log_size":18211,"ondisk_log_size":18211,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":117,"seq":502511173651,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44384,"kb_used_data":2316,"kb_used_omap":22,"kb_used_meta":42025,"kb_avail":94327456,"statfs":{"total":96636764160,"available":96591314944,"internally_reserved":0,"allocated":2371584,"data_stored":1152207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22742,"internal_metadata":43034410},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509488,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51568,"kb_used_data":2332,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320272,"statfs":{"total":96636764160,"available":96583958528,"internally_reserved":0,"allocated":2387968,"data_stored":1158039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509489,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51572,"kb_used_data":2332,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320268,"statfs":{"total":96636764160,"available":96583954432,"internally_reserved":0,"allocated":2387968,"data_stored":1158039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":48,"seq":206158430245,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60848,"kb_used_data":2328,"kb_used_omap":4,"kb_used_meta":58491,"kb_avail":94310992,"statfs":{"total":96636764160,"available":96574455808,"internally_reserved":0,"allocated":2383872,"data_stored":1158020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4430,"internal_metadata":59895474},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509488,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29368,"kb_used_data":2336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342472,"statfs":{"total":96636764160,"available":96606691328,"internally_reserved":0,"allocated":2392064,"data_stored":1158058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509488,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60464,"kb_used_data":2896,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311376,"statfs":{"total":96636764160,"available":96574849024,"internally_reserved":0,"allocated":2965504,"data_stored":1742575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542192,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60460,"kb_used_data":2896,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311380,"statfs":{"total":96636764160,"available":96574853120,"internally_reserved":0,"allocated":2965504,"data_stored":1742575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":103,"seq":442381631510,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30000,"kb_used_data":2328,"kb_used_omap":12,"kb_used_meta":27635,"kb_avail":94341840,"statfs":{"total":96636764160,"available":96606044160,"internally_reserved":0,"allocated":2383872,"data_stored":1158020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12804,"internal_metadata":28298748},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6121,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15274,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-04T17:49:24.627 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-04T17:49:24.917 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:49:24.917 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":198,"fsid":"f36b6ce2-8c72-4c99-8607-5c53c38ef67b","created":"2024-09-04T17:45:28.072591+0000","modified":"2024-09-04T17:49:19.222562+0000","last_up_change":"2024-09-04T17:47:57.033539+0000","last_in_change":"2024-09-04T17:49:02.845732+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-04T17:45:41.111814+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-04T17:45:46.265395+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-04T17:45:54.393683+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"176","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":45,"snap_epoch":176,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58efca2-bc5a-4bc4-b3d7-4036ebd9dc63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2135042466},{"type":"v1","addr":"172.21.15.92:6809","nonce":2135042466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":2135042466},{"type":"v1","addr":"172.21.15.92:6811","nonce":2135042466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":2135042466},{"type":"v1","addr":"172.21.15.92:6815","nonce":2135042466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":2135042466},{"type":"v1","addr":"172.21.15.92:6813","nonce":2135042466}]},"public_addr":"172.21.15.92:6809/2135042466","cluster_addr":"172.21.15.92:6811/2135042466","heartbeat_back_addr":"172.21.15.92:6815/2135042466","heartbeat_front_addr":"172.21.15.92:6813/2135042466","state":["exists","up"]},{"osd":1,"uuid":"36e03b64-2602-4143-bdd5-f7355de255fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":1306159585},{"type":"v1","addr":"172.21.15.92:6825","nonce":1306159585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":1306159585},{"type":"v1","addr":"172.21.15.92:6827","nonce":1306159585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":1306159585},{"type":"v1","addr":"172.21.15.92:6831","nonce":1306159585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":1306159585},{"type":"v1","addr":"172.21.15.92:6829","nonce":1306159585}]},"public_addr":"172.21.15.92:6825/1306159585","cluster_addr":"172.21.15.92:6827/1306159585","heartbeat_back_addr":"172.21.15.92:6831/1306159585","heartbeat_front_addr":"172.21.15.92:6829/1306159585","state":["exists","up"]},{"osd":2,"uuid":"581ab71e-be04-4031-a0c4-f7e40b74b7c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":83,"up_from":103,"up_thru":185,"down_at":84,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":2500708849},{"type":"v1","addr":"172.21.15.92:6801","nonce":2500708849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":2500708849},{"type":"v1","addr":"172.21.15.92:6803","nonce":2500708849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":2500708849},{"type":"v1","addr":"172.21.15.92:6807","nonce":2500708849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":2500708849},{"type":"v1","addr":"172.21.15.92:6805","nonce":2500708849}]},"public_addr":"172.21.15.92:6801/2500708849","cluster_addr":"172.21.15.92:6803/2500708849","heartbeat_back_addr":"172.21.15.92:6807/2500708849","heartbeat_front_addr":"172.21.15.92:6805/2500708849","state":["exists","up"]},{"osd":3,"uuid":"5f5d096e-fab2-41fe-b8dd-943d1c40894c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":23,"up_from":48,"up_thru":186,"down_at":24,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":3390028311},{"type":"v1","addr":"172.21.15.92:6817","nonce":3390028311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":3390028311},{"type":"v1","addr":"172.21.15.92:6819","nonce":3390028311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":3390028311},{"type":"v1","addr":"172.21.15.92:6823","nonce":3390028311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":3390028311},{"type":"v1","addr":"172.21.15.92:6821","nonce":3390028311}]},"public_addr":"172.21.15.92:6817/3390028311","cluster_addr":"172.21.15.92:6819/3390028311","heartbeat_back_addr":"172.21.15.92:6823/3390028311","heartbeat_front_addr":"172.21.15.92:6821/3390028311","state":["exists","up"]},{"osd":4,"uuid":"840951a5-e13b-462c-8c8e-2ca61a53fe3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4272750248},{"type":"v1","addr":"172.21.15.190:6817","nonce":4272750248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4272750248},{"type":"v1","addr":"172.21.15.190:6819","nonce":4272750248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4272750248},{"type":"v1","addr":"172.21.15.190:6823","nonce":4272750248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":4272750248},{"type":"v1","addr":"172.21.15.190:6821","nonce":4272750248}]},"public_addr":"172.21.15.190:6817/4272750248","cluster_addr":"172.21.15.190:6819/4272750248","heartbeat_back_addr":"172.21.15.190:6823/4272750248","heartbeat_front_addr":"172.21.15.190:6821/4272750248","state":["exists","up"]},{"osd":5,"uuid":"82e556e5-00c3-46fc-aab5-3544bba279d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":1688427632},{"type":"v1","addr":"172.21.15.190:6809","nonce":1688427632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":1688427632},{"type":"v1","addr":"172.21.15.190:6811","nonce":1688427632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":1688427632},{"type":"v1","addr":"172.21.15.190:6815","nonce":1688427632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1688427632},{"type":"v1","addr":"172.21.15.190:6813","nonce":1688427632}]},"public_addr":"172.21.15.190:6809/1688427632","cluster_addr":"172.21.15.190:6811/1688427632","heartbeat_back_addr":"172.21.15.190:6815/1688427632","heartbeat_front_addr":"172.21.15.190:6813/1688427632","state":["exists","up"]},{"osd":6,"uuid":"4e1e3d67-3020-470d-98b7-891fdc966238","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":58,"up_from":117,"up_thru":179,"down_at":60,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":466854783},{"type":"v1","addr":"172.21.15.190:6825","nonce":466854783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":466854783},{"type":"v1","addr":"172.21.15.190:6827","nonce":466854783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":466854783},{"type":"v1","addr":"172.21.15.190:6831","nonce":466854783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":466854783},{"type":"v1","addr":"172.21.15.190:6829","nonce":466854783}]},"public_addr":"172.21.15.190:6825/466854783","cluster_addr":"172.21.15.190:6827/466854783","heartbeat_back_addr":"172.21.15.190:6831/466854783","heartbeat_front_addr":"172.21.15.190:6829/466854783","state":["exists","up"]},{"osd":7,"uuid":"bd9c9d3e-27db-4bc2-b998-7f446576a543","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":4006671087},{"type":"v1","addr":"172.21.15.190:6801","nonce":4006671087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":4006671087},{"type":"v1","addr":"172.21.15.190:6803","nonce":4006671087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":4006671087},{"type":"v1","addr":"172.21.15.190:6807","nonce":4006671087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":4006671087},{"type":"v1","addr":"172.21.15.190:6805","nonce":4006671087}]},"public_addr":"172.21.15.190:6801/4006671087","cluster_addr":"172.21.15.190:6803/4006671087","heartbeat_back_addr":"172.21.15.190:6807/4006671087","heartbeat_front_addr":"172.21.15.190:6805/4006671087","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-04T17:45:37.324122+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-04T17:45:37.856638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-04T17:45:36.966684+0000","dead_epoch":84},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-04T17:45:37.266271+0000","dead_epoch":24},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-04T17:45:37.627069+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-04T17:45:37.241115+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-04T17:45:37.397375+0000","dead_epoch":60},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-04T17:45:36.619385+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-04T17:49:25.936 INFO:tasks.ceph:Scrubbing osd.0 2024-09-04T17:49:25.936 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-04T17:49:26.098 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:49:26.099 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-04T17:49:26.099 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:49:26.115 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-04T17:49:26.409 INFO:teuthology.orchestra.run.smithi092.stderr:instructed osd(s) 0 to deep-scrub 2024-09-04T17:49:26.427 INFO:tasks.ceph:Scrubbing osd.1 2024-09-04T17:49:26.427 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-04T17:49:26.581 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:49:26.581 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-04T17:49:26.581 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:49:26.596 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-04T17:49:26.881 INFO:teuthology.orchestra.run.smithi092.stderr:instructed osd(s) 1 to deep-scrub 2024-09-04T17:49:26.899 INFO:tasks.ceph:Scrubbing osd.2 2024-09-04T17:49:26.899 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-04T17:49:27.042 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:49:27.042 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-04T17:49:27.042 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:49:27.057 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-04T17:49:27.350 INFO:teuthology.orchestra.run.smithi092.stderr:instructed osd(s) 2 to deep-scrub 2024-09-04T17:49:27.369 INFO:tasks.ceph:Scrubbing osd.3 2024-09-04T17:49:27.369 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-04T17:49:27.513 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:49:27.513 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-04T17:49:27.513 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:49:27.528 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-04T17:49:27.806 INFO:teuthology.orchestra.run.smithi092.stderr:instructed osd(s) 3 to deep-scrub 2024-09-04T17:49:27.824 INFO:tasks.ceph:Scrubbing osd.4 2024-09-04T17:49:27.824 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-04T17:49:27.972 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:49:27.972 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-04T17:49:27.972 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:49:27.986 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-04T17:49:28.290 INFO:teuthology.orchestra.run.smithi092.stderr:instructed osd(s) 4 to deep-scrub 2024-09-04T17:49:28.310 INFO:tasks.ceph:Scrubbing osd.5 2024-09-04T17:49:28.310 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-04T17:49:28.463 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:49:28.463 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-04T17:49:28.463 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:49:28.479 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-04T17:49:28.772 INFO:teuthology.orchestra.run.smithi092.stderr:instructed osd(s) 5 to deep-scrub 2024-09-04T17:49:28.789 INFO:tasks.ceph:Scrubbing osd.6 2024-09-04T17:49:28.790 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-04T17:49:28.934 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:49:28.935 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-04T17:49:28.935 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:49:28.951 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-04T17:49:29.249 INFO:teuthology.orchestra.run.smithi092.stderr:instructed osd(s) 6 to deep-scrub 2024-09-04T17:49:29.268 INFO:tasks.ceph:Scrubbing osd.7 2024-09-04T17:49:29.268 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-04T17:49:29.417 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-04T17:49:29.417 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-04T17:49:29.417 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-04T17:49:29.433 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-04T17:49:29.723 INFO:teuthology.orchestra.run.smithi092.stderr:instructed osd(s) 7 to deep-scrub 2024-09-04T17:49:29.741 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-04T17:49:30.042 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:49:30.043 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-09-04T17:49:30.058 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":306,"stamp":"2024-09-04T17:49:29.132760+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":939,"num_object_clones":0,"num_object_copies":1878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":939,"num_whiteouts":931,"num_read":142346,"num_read_kb":876804,"num_write":20402,"num_write_kb":405906,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6763,"num_bytes_recovered":238961871,"num_keys_recovered":777,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18286,"ondisk_log_size":18286,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":388684,"kb_used_data":19800,"kb_used_omap":46,"kb_used_meta":368657,"kb_avail":754586036,"statfs":{"total":773094113280,"available":772696100864,"internally_reserved":0,"allocated":20275200,"data_stored":10444972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":47946,"internal_metadata":377504950},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003163"},"pg_stats":[{"pgid":"3.1f","version":"178'501","reported_seq":3133,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.470279+0000","last_change":"2024-09-04T17:48:56.059852+0000","last_active":"2024-09-04T17:49:20.470279+0000","last_peered":"2024-09-04T17:49:20.470279+0000","last_clean":"2024-09-04T17:49:20.470279+0000","last_became_active":"2024-09-04T17:48:09.845199+0000","last_became_peered":"2024-09-04T17:48:09.845199+0000","last_unstale":"2024-09-04T17:49:20.470279+0000","last_undegraded":"2024-09-04T17:49:20.470279+0000","last_fullsized":"2024-09-04T17:49:20.470279+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T21:36:09.244561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003726699,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":2983,"num_read_kb":27190,"num_write":524,"num_write_kb":10445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":24433786,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"175'657","reported_seq":3987,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.935494+0000","last_change":"2024-09-04T17:49:05.876073+0000","last_active":"2024-09-04T17:49:19.935494+0000","last_peered":"2024-09-04T17:49:19.935494+0000","last_clean":"2024-09-04T17:49:19.935494+0000","last_became_active":"2024-09-04T17:49:05.874586+0000","last_became_peered":"2024-09-04T17:49:05.874586+0000","last_unstale":"2024-09-04T17:49:19.935494+0000","last_undegraded":"2024-09-04T17:49:19.935494+0000","last_fullsized":"2024-09-04T17:49:19.935494+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:25:15.903921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001926142,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4757,"num_read_kb":30502,"num_write":815,"num_write_kb":10573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":322,"num_bytes_recovered":15412196,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"175'548","reported_seq":2554,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.187160+0000","last_change":"2024-09-04T17:48:56.060983+0000","last_active":"2024-09-04T17:49:20.187160+0000","last_peered":"2024-09-04T17:49:20.187160+0000","last_clean":"2024-09-04T17:49:20.187160+0000","last_became_active":"2024-09-04T17:45:55.955213+0000","last_became_peered":"2024-09-04T17:45:55.955213+0000","last_unstale":"2024-09-04T17:49:20.187160+0000","last_undegraded":"2024-09-04T17:49:20.187160+0000","last_fullsized":"2024-09-04T17:49:20.187160+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":548,"log_dups_size":0,"ondisk_log_size":548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T19:29:02.548970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022810360000000002,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":2820,"num_read_kb":20284,"num_write":623,"num_write_kb":9401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"178'758","reported_seq":6826,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.187131+0000","last_change":"2024-09-04T17:48:56.055219+0000","last_active":"2024-09-04T17:49:20.187131+0000","last_peered":"2024-09-04T17:49:20.187131+0000","last_clean":"2024-09-04T17:49:20.187131+0000","last_became_active":"2024-09-04T17:45:55.951880+0000","last_became_peered":"2024-09-04T17:45:55.951880+0000","last_unstale":"2024-09-04T17:49:20.187131+0000","last_undegraded":"2024-09-04T17:49:20.187131+0000","last_fullsized":"2024-09-04T17:49:20.187131+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":758,"log_dups_size":0,"ondisk_log_size":758,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:58:30.591123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025730409999999999,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":35,"num_read":7473,"num_read_kb":48018,"num_write":874,"num_write_kb":11556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"175'597","reported_seq":2410,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.469937+0000","last_change":"2024-09-04T17:48:56.058271+0000","last_active":"2024-09-04T17:49:20.469937+0000","last_peered":"2024-09-04T17:49:20.469937+0000","last_clean":"2024-09-04T17:49:20.469937+0000","last_became_active":"2024-09-04T17:48:09.853793+0000","last_became_peered":"2024-09-04T17:48:09.853793+0000","last_unstale":"2024-09-04T17:49:20.469937+0000","last_undegraded":"2024-09-04T17:49:20.469937+0000","last_fullsized":"2024-09-04T17:49:20.469937+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":597,"log_dups_size":0,"ondisk_log_size":597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:56:08.051891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026463849999999998,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":3715,"num_read_kb":8148,"num_write":647,"num_write_kb":4600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":1993914,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"175'424","reported_seq":6455,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.240117+0000","last_change":"2024-09-04T17:49:10.140837+0000","last_active":"2024-09-04T17:49:19.240117+0000","last_peered":"2024-09-04T17:49:19.240117+0000","last_clean":"2024-09-04T17:49:19.240117+0000","last_became_active":"2024-09-04T17:49:10.137472+0000","last_became_peered":"2024-09-04T17:49:10.137472+0000","last_unstale":"2024-09-04T17:49:19.240117+0000","last_undegraded":"2024-09-04T17:49:19.240117+0000","last_fullsized":"2024-09-04T17:49:19.240117+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":424,"log_dups_size":0,"ondisk_log_size":424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:51:59.854799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000885438,"stat_sum":{"num_bytes":19,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":25,"num_read":5992,"num_read_kb":45852,"num_write":376,"num_write_kb":15447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":5992020,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"178'442","reported_seq":2720,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.237063+0000","last_change":"2024-09-04T17:49:05.878999+0000","last_active":"2024-09-04T17:49:19.237063+0000","last_peered":"2024-09-04T17:49:19.237063+0000","last_clean":"2024-09-04T17:49:19.237063+0000","last_became_active":"2024-09-04T17:49:05.876985+0000","last_became_peered":"2024-09-04T17:49:05.876985+0000","last_unstale":"2024-09-04T17:49:19.237063+0000","last_undegraded":"2024-09-04T17:49:19.237063+0000","last_fullsized":"2024-09-04T17:49:19.237063+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:42:50.926456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001467744,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":2248,"num_read_kb":16196,"num_write":460,"num_write_kb":4617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":494,"num_bytes_recovered":4911830,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"178'508","reported_seq":2560,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.935295+0000","last_change":"2024-09-04T17:49:07.880474+0000","last_active":"2024-09-04T17:49:19.935295+0000","last_peered":"2024-09-04T17:49:19.935295+0000","last_clean":"2024-09-04T17:49:19.935295+0000","last_became_active":"2024-09-04T17:49:07.878360+0000","last_became_peered":"2024-09-04T17:49:07.878360+0000","last_unstale":"2024-09-04T17:49:19.935295+0000","last_undegraded":"2024-09-04T17:49:19.935295+0000","last_fullsized":"2024-09-04T17:49:19.935295+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:25:31.312054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017798989999999999,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":8892,"num_read_kb":35489,"num_write":443,"num_write_kb":14429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":597,"num_bytes_recovered":28259997,"num_keys_recovered":141,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"178'331","reported_seq":2390,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.470375+0000","last_change":"2024-09-04T17:48:56.056526+0000","last_active":"2024-09-04T17:49:20.470375+0000","last_peered":"2024-09-04T17:49:20.470375+0000","last_clean":"2024-09-04T17:49:20.470375+0000","last_became_active":"2024-09-04T17:45:55.958625+0000","last_became_peered":"2024-09-04T17:45:55.958625+0000","last_unstale":"2024-09-04T17:49:20.470375+0000","last_undegraded":"2024-09-04T17:49:20.470375+0000","last_fullsized":"2024-09-04T17:49:20.470375+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":331,"log_dups_size":0,"ondisk_log_size":331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T22:39:37.983710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001574039,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":1612,"num_read_kb":20939,"num_write":316,"num_write_kb":18406,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"175'364","reported_seq":2051,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.187266+0000","last_change":"2024-09-04T17:48:56.060132+0000","last_active":"2024-09-04T17:49:20.187266+0000","last_peered":"2024-09-04T17:49:20.187266+0000","last_clean":"2024-09-04T17:49:20.187266+0000","last_became_active":"2024-09-04T17:45:55.953254+0000","last_became_peered":"2024-09-04T17:45:55.953254+0000","last_unstale":"2024-09-04T17:49:20.187266+0000","last_undegraded":"2024-09-04T17:49:20.187266+0000","last_fullsized":"2024-09-04T17:49:20.187266+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":364,"log_dups_size":0,"ondisk_log_size":364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T21:59:17.936385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0044959860000000004,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":1729,"num_read_kb":10785,"num_write":396,"num_write_kb":7391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"174'510","reported_seq":3056,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.568547+0000","last_change":"2024-09-04T17:49:06.889185+0000","last_active":"2024-09-04T17:49:20.568547+0000","last_peered":"2024-09-04T17:49:20.568547+0000","last_clean":"2024-09-04T17:49:20.568547+0000","last_became_active":"2024-09-04T17:49:06.884929+0000","last_became_peered":"2024-09-04T17:49:06.884929+0000","last_unstale":"2024-09-04T17:49:20.568547+0000","last_undegraded":"2024-09-04T17:49:20.568547+0000","last_fullsized":"2024-09-04T17:49:20.568547+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":510,"log_dups_size":0,"ondisk_log_size":510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:02:50.836583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001003051,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":4020,"num_read_kb":16975,"num_write":525,"num_write_kb":4439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":446,"num_bytes_recovered":12989343,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"178'775","reported_seq":2528,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.240494+0000","last_change":"2024-09-04T17:49:10.140905+0000","last_active":"2024-09-04T17:49:19.240494+0000","last_peered":"2024-09-04T17:49:19.240494+0000","last_clean":"2024-09-04T17:49:19.240494+0000","last_became_active":"2024-09-04T17:49:10.137573+0000","last_became_peered":"2024-09-04T17:49:10.137573+0000","last_unstale":"2024-09-04T17:49:19.240494+0000","last_undegraded":"2024-09-04T17:49:19.240494+0000","last_fullsized":"2024-09-04T17:49:19.240494+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":775,"log_dups_size":0,"ondisk_log_size":775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:56:50.281112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027107059999999998,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":29,"num_read":4009,"num_read_kb":29862,"num_write":1001,"num_write_kb":35909,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":235,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.080313+0000","last_change":"2024-09-04T17:48:11.850550+0000","last_active":"2024-09-04T17:49:20.080313+0000","last_peered":"2024-09-04T17:49:20.080313+0000","last_clean":"2024-09-04T17:49:20.080313+0000","last_became_active":"2024-09-04T17:48:11.849710+0000","last_became_peered":"2024-09-04T17:48:11.849710+0000","last_unstale":"2024-09-04T17:49:20.080313+0000","last_undegraded":"2024-09-04T17:49:20.080313+0000","last_fullsized":"2024-09-04T17:49:20.080313+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:37:18.791459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001226074,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"178'409","reported_seq":3430,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.470300+0000","last_change":"2024-09-04T17:49:07.876177+0000","last_active":"2024-09-04T17:49:20.470300+0000","last_peered":"2024-09-04T17:49:20.470300+0000","last_clean":"2024-09-04T17:49:20.470300+0000","last_became_active":"2024-09-04T17:49:07.874501+0000","last_became_peered":"2024-09-04T17:49:07.874501+0000","last_unstale":"2024-09-04T17:49:20.470300+0000","last_undegraded":"2024-09-04T17:49:20.470300+0000","last_fullsized":"2024-09-04T17:49:20.470300+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":409,"log_dups_size":0,"ondisk_log_size":409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:52:42.831486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001494427,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":2485,"num_read_kb":14550,"num_write":410,"num_write_kb":5528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":919873,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":40,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.237456+0000","last_change":"2024-09-04T17:49:03.886949+0000","last_active":"2024-09-04T17:49:19.237456+0000","last_peered":"2024-09-04T17:49:19.237456+0000","last_clean":"2024-09-04T17:49:19.237456+0000","last_became_active":"2024-09-04T17:49:03.886525+0000","last_became_peered":"2024-09-04T17:49:03.886525+0000","last_unstale":"2024-09-04T17:49:19.237456+0000","last_undegraded":"2024-09-04T17:49:19.237456+0000","last_fullsized":"2024-09-04T17:49:19.237456+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:24:57.789704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011364179999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"175'469","reported_seq":3004,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.935427+0000","last_change":"2024-09-04T17:49:05.877106+0000","last_active":"2024-09-04T17:49:19.935427+0000","last_peered":"2024-09-04T17:49:19.935427+0000","last_clean":"2024-09-04T17:49:19.935427+0000","last_became_active":"2024-09-04T17:49:05.874579+0000","last_became_peered":"2024-09-04T17:49:05.874579+0000","last_unstale":"2024-09-04T17:49:19.935427+0000","last_undegraded":"2024-09-04T17:49:19.935427+0000","last_fullsized":"2024-09-04T17:49:19.935427+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":469,"log_dups_size":0,"ondisk_log_size":469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:32:31.466355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019775819999999999,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":2953,"num_read_kb":23389,"num_write":575,"num_write_kb":21569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":313,"num_bytes_recovered":1588111,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":372,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.568849+0000","last_change":"2024-09-04T17:45:49.476607+0000","last_active":"2024-09-04T17:49:20.568849+0000","last_peered":"2024-09-04T17:49:20.568849+0000","last_clean":"2024-09-04T17:49:20.568849+0000","last_became_active":"2024-09-04T17:45:47.369308+0000","last_became_peered":"2024-09-04T17:45:47.369308+0000","last_unstale":"2024-09-04T17:49:20.568849+0000","last_undegraded":"2024-09-04T17:49:20.568849+0000","last_fullsized":"2024-09-04T17:49:20.568849+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:17:13.378553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001029826,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"175'410","reported_seq":2886,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.237416+0000","last_change":"2024-09-04T17:49:07.879430+0000","last_active":"2024-09-04T17:49:19.237416+0000","last_peered":"2024-09-04T17:49:19.237416+0000","last_clean":"2024-09-04T17:49:19.237416+0000","last_became_active":"2024-09-04T17:49:07.877184+0000","last_became_peered":"2024-09-04T17:49:07.877184+0000","last_unstale":"2024-09-04T17:49:19.237416+0000","last_undegraded":"2024-09-04T17:49:19.237416+0000","last_fullsized":"2024-09-04T17:49:19.237416+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T21:09:45.025593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029821800000000001,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":31,"num_read":2672,"num_read_kb":18782,"num_write":436,"num_write_kb":9856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":13375622,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.237128+0000","last_change":"2024-09-04T17:49:03.886170+0000","last_active":"2024-09-04T17:49:19.237128+0000","last_peered":"2024-09-04T17:49:19.237128+0000","last_clean":"2024-09-04T17:49:19.237128+0000","last_became_active":"2024-09-04T17:49:03.885776+0000","last_became_peered":"2024-09-04T17:49:03.885776+0000","last_unstale":"2024-09-04T17:49:19.237128+0000","last_undegraded":"2024-09-04T17:49:19.237128+0000","last_fullsized":"2024-09-04T17:49:19.237128+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T19:30:49.636681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"175'358","reported_seq":3042,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.187305+0000","last_change":"2024-09-04T17:49:06.887617+0000","last_active":"2024-09-04T17:49:20.187305+0000","last_peered":"2024-09-04T17:49:20.187305+0000","last_clean":"2024-09-04T17:49:20.187305+0000","last_became_active":"2024-09-04T17:49:06.885686+0000","last_became_peered":"2024-09-04T17:49:06.885686+0000","last_unstale":"2024-09-04T17:49:20.187305+0000","last_undegraded":"2024-09-04T17:49:20.187305+0000","last_fullsized":"2024-09-04T17:49:20.187305+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T22:44:47.142525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019787199999999998,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":2555,"num_read_kb":7639,"num_write":305,"num_write_kb":7011,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":391,"num_bytes_recovered":15596571,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"178'887","reported_seq":3553,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.240376+0000","last_change":"2024-09-04T17:49:10.141003+0000","last_active":"2024-09-04T17:49:19.240376+0000","last_peered":"2024-09-04T17:49:19.240376+0000","last_clean":"2024-09-04T17:49:19.240376+0000","last_became_active":"2024-09-04T17:49:10.137895+0000","last_became_peered":"2024-09-04T17:49:10.137895+0000","last_unstale":"2024-09-04T17:49:19.240376+0000","last_undegraded":"2024-09-04T17:49:19.240376+0000","last_fullsized":"2024-09-04T17:49:19.240376+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":887,"log_dups_size":0,"ondisk_log_size":887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:24:10.064606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032310310000000001,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":29,"num_read":4673,"num_read_kb":26561,"num_write":1100,"num_write_kb":14222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":398,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.186827+0000","last_change":"2024-09-04T17:49:03.892940+0000","last_active":"2024-09-04T17:49:20.186827+0000","last_peered":"2024-09-04T17:49:20.186827+0000","last_clean":"2024-09-04T17:49:20.186827+0000","last_became_active":"2024-09-04T17:49:03.887053+0000","last_became_peered":"2024-09-04T17:49:03.887053+0000","last_unstale":"2024-09-04T17:49:20.186827+0000","last_undegraded":"2024-09-04T17:49:20.186827+0000","last_fullsized":"2024-09-04T17:49:20.186827+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:30:44.144337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026316069999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"175'453","reported_seq":3440,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.237166+0000","last_change":"2024-09-04T17:49:07.878051+0000","last_active":"2024-09-04T17:49:19.237166+0000","last_peered":"2024-09-04T17:49:19.237166+0000","last_clean":"2024-09-04T17:49:19.237166+0000","last_became_active":"2024-09-04T17:49:07.876339+0000","last_became_peered":"2024-09-04T17:49:07.876339+0000","last_unstale":"2024-09-04T17:49:19.237166+0000","last_undegraded":"2024-09-04T17:49:19.237166+0000","last_fullsized":"2024-09-04T17:49:19.237166+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":453,"log_dups_size":0,"ondisk_log_size":453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T22:43:32.300162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00120166,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":36,"num_read":3566,"num_read_kb":22521,"num_write":365,"num_write_kb":4893,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":261,"num_bytes_recovered":5692590,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":370,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.236829+0000","last_change":"2024-09-04T17:49:02.864996+0000","last_active":"2024-09-04T17:49:19.236829+0000","last_peered":"2024-09-04T17:49:19.236829+0000","last_clean":"2024-09-04T17:49:19.236829+0000","last_became_active":"2024-09-04T17:49:02.864647+0000","last_became_peered":"2024-09-04T17:49:02.864647+0000","last_unstale":"2024-09-04T17:49:19.236829+0000","last_undegraded":"2024-09-04T17:49:19.236829+0000","last_fullsized":"2024-09-04T17:49:19.236829+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:41:35.190910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001025195,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"178'637","reported_seq":3375,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.469830+0000","last_change":"2024-09-04T17:48:56.058939+0000","last_active":"2024-09-04T17:49:20.469830+0000","last_peered":"2024-09-04T17:49:20.469830+0000","last_clean":"2024-09-04T17:49:20.469830+0000","last_became_active":"2024-09-04T17:48:11.845510+0000","last_became_peered":"2024-09-04T17:48:11.845510+0000","last_unstale":"2024-09-04T17:49:20.469830+0000","last_undegraded":"2024-09-04T17:49:20.469830+0000","last_fullsized":"2024-09-04T17:49:20.469830+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":637,"log_dups_size":0,"ondisk_log_size":637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:14:18.552668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031660479999999999,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":3316,"num_read_kb":40546,"num_write":672,"num_write_kb":23359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":240,"num_bytes_recovered":21297737,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":384,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.568376+0000","last_change":"2024-09-04T17:49:03.874392+0000","last_active":"2024-09-04T17:49:20.568376+0000","last_peered":"2024-09-04T17:49:20.568376+0000","last_clean":"2024-09-04T17:49:20.568376+0000","last_became_active":"2024-09-04T17:49:03.870678+0000","last_became_peered":"2024-09-04T17:49:03.870678+0000","last_unstale":"2024-09-04T17:49:20.568376+0000","last_undegraded":"2024-09-04T17:49:20.568376+0000","last_fullsized":"2024-09-04T17:49:20.568376+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:49:38.360230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010553590000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"175'622","reported_seq":3059,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.935307+0000","last_change":"2024-09-04T17:49:07.879558+0000","last_active":"2024-09-04T17:49:19.935307+0000","last_peered":"2024-09-04T17:49:19.935307+0000","last_clean":"2024-09-04T17:49:19.935307+0000","last_became_active":"2024-09-04T17:49:07.876340+0000","last_became_peered":"2024-09-04T17:49:07.876340+0000","last_unstale":"2024-09-04T17:49:19.935307+0000","last_undegraded":"2024-09-04T17:49:19.935307+0000","last_fullsized":"2024-09-04T17:49:19.935307+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":622,"log_dups_size":0,"ondisk_log_size":622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:20:26.042540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021278730000000002,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":5714,"num_read_kb":30160,"num_write":740,"num_write_kb":13818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":405,"num_bytes_recovered":16173004,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":382,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.186955+0000","last_change":"2024-09-04T17:47:41.919544+0000","last_active":"2024-09-04T17:49:20.186955+0000","last_peered":"2024-09-04T17:49:20.186955+0000","last_clean":"2024-09-04T17:49:20.186955+0000","last_became_active":"2024-09-04T17:47:41.919027+0000","last_became_peered":"2024-09-04T17:47:41.919027+0000","last_unstale":"2024-09-04T17:49:20.186955+0000","last_undegraded":"2024-09-04T17:49:20.186955+0000","last_fullsized":"2024-09-04T17:49:20.186955+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:46.343951+0000","last_clean_scrub_stamp":"2024-09-04T17:45:46.343951+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:42:55.688562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086618299999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":463,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.568735+0000","last_change":"2024-09-04T17:45:42.341149+0000","last_active":"2024-09-04T17:49:20.568735+0000","last_peered":"2024-09-04T17:49:20.568735+0000","last_clean":"2024-09-04T17:49:20.568735+0000","last_became_active":"2024-09-04T17:45:42.339286+0000","last_became_peered":"2024-09-04T17:45:42.339286+0000","last_unstale":"2024-09-04T17:49:20.568735+0000","last_undegraded":"2024-09-04T17:49:20.568735+0000","last_fullsized":"2024-09-04T17:49:20.568735+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:41.305128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:41.305128+0000","last_clean_scrub_stamp":"2024-09-04T17:45:41.305128+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:52:24.860049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"178'548","reported_seq":3065,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.935375+0000","last_change":"2024-09-04T17:49:07.883398+0000","last_active":"2024-09-04T17:49:19.935375+0000","last_peered":"2024-09-04T17:49:19.935375+0000","last_clean":"2024-09-04T17:49:19.935375+0000","last_became_active":"2024-09-04T17:49:07.879442+0000","last_became_peered":"2024-09-04T17:49:07.879442+0000","last_unstale":"2024-09-04T17:49:19.935375+0000","last_undegraded":"2024-09-04T17:49:19.935375+0000","last_fullsized":"2024-09-04T17:49:19.935375+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":548,"log_dups_size":0,"ondisk_log_size":548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:55:18.249867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011779189999999999,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":2788,"num_read_kb":17264,"num_write":577,"num_write_kb":29373,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":445,"num_bytes_recovered":17940277,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"178'716","reported_seq":4731,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.240231+0000","last_change":"2024-09-04T17:49:10.137619+0000","last_active":"2024-09-04T17:49:19.240231+0000","last_peered":"2024-09-04T17:49:19.240231+0000","last_clean":"2024-09-04T17:49:19.240231+0000","last_became_active":"2024-09-04T17:49:10.133837+0000","last_became_peered":"2024-09-04T17:49:10.133837+0000","last_unstale":"2024-09-04T17:49:19.240231+0000","last_undegraded":"2024-09-04T17:49:19.240231+0000","last_fullsized":"2024-09-04T17:49:19.240231+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":716,"log_dups_size":0,"ondisk_log_size":716,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T23:43:19.357232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003529346,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":6198,"num_read_kb":37498,"num_write":772,"num_write_kb":18653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":4534479,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"175'656","reported_seq":3645,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.080234+0000","last_change":"2024-09-04T17:48:56.054512+0000","last_active":"2024-09-04T17:49:20.080234+0000","last_peered":"2024-09-04T17:49:20.080234+0000","last_clean":"2024-09-04T17:49:20.080234+0000","last_became_active":"2024-09-04T17:48:00.841070+0000","last_became_peered":"2024-09-04T17:48:00.841070+0000","last_unstale":"2024-09-04T17:49:20.080234+0000","last_undegraded":"2024-09-04T17:49:20.080234+0000","last_fullsized":"2024-09-04T17:49:20.080234+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T19:18:47.901288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003342214,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":31,"num_read":5699,"num_read_kb":16474,"num_write":617,"num_write_kb":2753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":1572433,"num_keys_recovered":66,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"178'770","reported_seq":2984,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:19.935456+0000","last_change":"2024-09-04T17:49:06.885887+0000","last_active":"2024-09-04T17:49:19.935456+0000","last_peered":"2024-09-04T17:49:19.935456+0000","last_clean":"2024-09-04T17:49:19.935456+0000","last_became_active":"2024-09-04T17:49:06.883685+0000","last_became_peered":"2024-09-04T17:49:06.883685+0000","last_unstale":"2024-09-04T17:49:19.935456+0000","last_undegraded":"2024-09-04T17:49:19.935456+0000","last_fullsized":"2024-09-04T17:49:19.935456+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":770,"log_dups_size":0,"ondisk_log_size":770,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T17:55:41.907650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002090967,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":4267,"num_read_kb":14813,"num_write":915,"num_write_kb":7927,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":459,"num_bytes_recovered":5211501,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"175'647","reported_seq":9166,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.187139+0000","last_change":"2024-09-04T17:49:07.884097+0000","last_active":"2024-09-04T17:49:20.187139+0000","last_peered":"2024-09-04T17:49:20.187139+0000","last_clean":"2024-09-04T17:49:20.187139+0000","last_became_active":"2024-09-04T17:49:07.880558+0000","last_became_peered":"2024-09-04T17:49:07.880558+0000","last_unstale":"2024-09-04T17:49:20.187139+0000","last_undegraded":"2024-09-04T17:49:20.187139+0000","last_fullsized":"2024-09-04T17:49:20.187139+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":647,"log_dups_size":0,"ondisk_log_size":647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:43:25.854985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096708700000000005,"stat_sum":{"num_bytes":0,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":34,"num_read":9052,"num_read_kb":71439,"num_write":754,"num_write_kb":16863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":537,"num_bytes_recovered":8281724,"num_keys_recovered":57,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"178'1358","reported_seq":4587,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.568736+0000","last_change":"2024-09-04T17:48:56.066067+0000","last_active":"2024-09-04T17:49:20.568736+0000","last_peered":"2024-09-04T17:49:20.568736+0000","last_clean":"2024-09-04T17:49:20.568736+0000","last_became_active":"2024-09-04T17:48:12.850381+0000","last_became_peered":"2024-09-04T17:48:12.850381+0000","last_unstale":"2024-09-04T17:49:20.568736+0000","last_undegraded":"2024-09-04T17:49:20.568736+0000","last_fullsized":"2024-09-04T17:49:20.568736+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":1358,"log_dups_size":0,"ondisk_log_size":1358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T18:53:56.503643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002193296,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":31,"num_read":8863,"num_read_kb":26532,"num_write":2254,"num_write_kb":13198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":179,"num_bytes_recovered":18877339,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"178'541","reported_seq":6078,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.568736+0000","last_change":"2024-09-04T17:48:56.056521+0000","last_active":"2024-09-04T17:49:20.568736+0000","last_peered":"2024-09-04T17:49:20.568736+0000","last_clean":"2024-09-04T17:49:20.568736+0000","last_became_active":"2024-09-04T17:48:11.849605+0000","last_became_peered":"2024-09-04T17:48:11.849605+0000","last_unstale":"2024-09-04T17:49:20.568736+0000","last_undegraded":"2024-09-04T17:49:20.568736+0000","last_fullsized":"2024-09-04T17:49:20.568736+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":541,"log_dups_size":0,"ondisk_log_size":541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:57:37.250572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001652911,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":6678,"num_read_kb":28511,"num_write":523,"num_write_kb":11228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":7806862,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"178'348","reported_seq":3233,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.568931+0000","last_change":"2024-09-04T17:48:56.063351+0000","last_active":"2024-09-04T17:49:20.568931+0000","last_peered":"2024-09-04T17:49:20.568931+0000","last_clean":"2024-09-04T17:49:20.568931+0000","last_became_active":"2024-09-04T17:45:55.956734+0000","last_became_peered":"2024-09-04T17:45:55.956734+0000","last_unstale":"2024-09-04T17:49:20.568931+0000","last_undegraded":"2024-09-04T17:49:20.568931+0000","last_fullsized":"2024-09-04T17:49:20.568931+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T23:04:26.785513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0065435850000000002,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":2605,"num_read_kb":44969,"num_write":330,"num_write_kb":13427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"175'630","reported_seq":3858,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.080256+0000","last_change":"2024-09-04T17:48:56.053981+0000","last_active":"2024-09-04T17:49:20.080256+0000","last_peered":"2024-09-04T17:49:20.080256+0000","last_clean":"2024-09-04T17:49:20.080256+0000","last_became_active":"2024-09-04T17:48:00.843136+0000","last_became_peered":"2024-09-04T17:48:00.843136+0000","last_unstale":"2024-09-04T17:49:20.080256+0000","last_undegraded":"2024-09-04T17:49:20.080256+0000","last_fullsized":"2024-09-04T17:49:20.080256+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":630,"log_dups_size":0,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T23:36:00.802962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025156979999999998,"stat_sum":{"num_bytes":0,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":37,"num_read":4786,"num_read_kb":18960,"num_write":673,"num_write_kb":12854,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":3377599,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"175'599","reported_seq":6342,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.568435+0000","last_change":"2024-09-04T17:48:56.055370+0000","last_active":"2024-09-04T17:49:20.568435+0000","last_peered":"2024-09-04T17:49:20.568435+0000","last_clean":"2024-09-04T17:49:20.568435+0000","last_became_active":"2024-09-04T17:45:55.951870+0000","last_became_peered":"2024-09-04T17:45:55.951870+0000","last_unstale":"2024-09-04T17:49:20.568435+0000","last_undegraded":"2024-09-04T17:49:20.568435+0000","last_fullsized":"2024-09-04T17:49:20.568435+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:32:28.517472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012658840000000001,"stat_sum":{"num_bytes":0,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":38,"num_read":7163,"num_read_kb":61439,"num_write":584,"num_write_kb":17576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"175'425","reported_seq":4225,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.469949+0000","last_change":"2024-09-04T17:48:56.055907+0000","last_active":"2024-09-04T17:49:20.469949+0000","last_peered":"2024-09-04T17:49:20.469949+0000","last_clean":"2024-09-04T17:49:20.469949+0000","last_became_active":"2024-09-04T17:48:14.943778+0000","last_became_peered":"2024-09-04T17:48:14.943778+0000","last_unstale":"2024-09-04T17:49:20.469949+0000","last_undegraded":"2024-09-04T17:49:20.469949+0000","last_fullsized":"2024-09-04T17:49:20.469949+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":425,"log_dups_size":0,"ondisk_log_size":425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:28:33.747314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011847660000000001,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":31,"num_read":3577,"num_read_kb":21719,"num_write":447,"num_write_kb":8575,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":290,"num_bytes_recovered":2723025,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"175'313","reported_seq":2695,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:20.568491+0000","last_change":"2024-09-04T17:48:56.055871+0000","last_active":"2024-09-04T17:49:20.568491+0000","last_peered":"2024-09-04T17:49:20.568491+0000","last_clean":"2024-09-04T17:49:20.568491+0000","last_became_active":"2024-09-04T17:45:55.954544+0000","last_became_peered":"2024-09-04T17:45:55.954544+0000","last_unstale":"2024-09-04T17:49:20.568491+0000","last_undegraded":"2024-09-04T17:49:20.568491+0000","last_fullsized":"2024-09-04T17:49:20.568491+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:45:54.925016+0000","last_clean_scrub_stamp":"2024-09-04T17:45:54.925016+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:20:57.091315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001451911,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":2420,"num_read_kb":22743,"num_write":250,"num_write_kb":4380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":935,"num_object_clones":0,"num_object_copies":1870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":935,"num_whiteouts":931,"num_read":142280,"num_read_kb":876749,"num_write":20299,"num_write_kb":404276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6761,"num_bytes_recovered":238961833,"num_keys_recovered":777,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21395,"internal_metadata":0},"log_size":18211,"ondisk_log_size":18211,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":117,"seq":502511173652,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44400,"kb_used_data":2328,"kb_used_omap":22,"kb_used_meta":42025,"kb_avail":94327440,"statfs":{"total":96636764160,"available":96591298560,"internally_reserved":0,"allocated":2383872,"data_stored":1158020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22742,"internal_metadata":43034410},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509489,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51568,"kb_used_data":2332,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320272,"statfs":{"total":96636764160,"available":96583958528,"internally_reserved":0,"allocated":2387968,"data_stored":1158039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509490,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51572,"kb_used_data":2332,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320268,"statfs":{"total":96636764160,"available":96583954432,"internally_reserved":0,"allocated":2387968,"data_stored":1158039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":48,"seq":206158430246,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60848,"kb_used_data":2328,"kb_used_omap":4,"kb_used_meta":58491,"kb_avail":94310992,"statfs":{"total":96636764160,"available":96574455808,"internally_reserved":0,"allocated":2383872,"data_stored":1158020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4430,"internal_metadata":59895474},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509489,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29368,"kb_used_data":2336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342472,"statfs":{"total":96636764160,"available":96606691328,"internally_reserved":0,"allocated":2392064,"data_stored":1158058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509489,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60464,"kb_used_data":2908,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311376,"statfs":{"total":96636764160,"available":96574849024,"internally_reserved":0,"allocated":2977792,"data_stored":1748388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542193,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60464,"kb_used_data":2908,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311376,"statfs":{"total":96636764160,"available":96574849024,"internally_reserved":0,"allocated":2977792,"data_stored":1748388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":103,"seq":442381631511,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30000,"kb_used_data":2328,"kb_used_omap":12,"kb_used_meta":27635,"kb_avail":94341840,"statfs":{"total":96636764160,"available":96606044160,"internally_reserved":0,"allocated":2383872,"data_stored":1158020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12804,"internal_metadata":28298748},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6121,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15274,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-04T17:49:30.065 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-04T17:45:54.925016+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=45, tm_sec=54, tm_wday=2, tm_yday=248, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=49, tm_sec=24, tm_wday=2, tm_yday=248, tm_isdst=0) 2024-09-04T17:49:30.065 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-04T17:45:54.925016+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=45, tm_sec=54, tm_wday=2, tm_yday=248, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=49, tm_sec=24, tm_wday=2, tm_yday=248, tm_isdst=0) 2024-09-04T17:49:30.065 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-04T17:45:54.925016+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=45, tm_sec=54, tm_wday=2, tm_yday=248, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=49, tm_sec=24, tm_wday=2, tm_yday=248, tm_isdst=0) 2024-09-04T17:49:30.065 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-04T17:45:54.925016+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=45, tm_sec=54, tm_wday=2, tm_yday=248, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=49, tm_sec=24, tm_wday=2, tm_yday=248, tm_isdst=0) 2024-09-04T17:49:30.065 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-04T17:45:54.925016+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=45, tm_sec=54, tm_wday=2, tm_yday=248, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=49, tm_sec=24, tm_wday=2, tm_yday=248, tm_isdst=0) 2024-09-04T17:49:30.065 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-04T17:45:54.925016+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=45, tm_sec=54, tm_wday=2, tm_yday=248, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=49, tm_sec=24, tm_wday=2, tm_yday=248, tm_isdst=0) 2024-09-04T17:49:30.066 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-04T17:45:54.925016+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=45, tm_sec=54, tm_wday=2, tm_yday=248, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=49, tm_sec=24, tm_wday=2, tm_yday=248, tm_isdst=0) 2024-09-04T17:49:30.066 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-04T17:45:54.925016+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=45, tm_sec=54, tm_wday=2, tm_yday=248, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=49, tm_sec=24, tm_wday=2, tm_yday=248, tm_isdst=0) 2024-09-04T17:49:30.066 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-09-04T17:45:54.925016+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=45, tm_sec=54, tm_wday=2, tm_yday=248, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=49, tm_sec=24, tm_wday=2, tm_yday=248, tm_isdst=0) 2024-09-04T17:49:30.066 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-04T17:45:54.925016+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=45, tm_sec=54, tm_wday=2, tm_yday=248, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=49, tm_sec=24, tm_wday=2, tm_yday=248, tm_isdst=0) 2024-09-04T17:49:30.066 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-04T17:45:54.925016+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=45, tm_sec=54, tm_wday=2, tm_yday=248, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=49, tm_sec=24, tm_wday=2, tm_yday=248, tm_isdst=0) 2024-09-04T17:49:30.066 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-04T17:45:54.925016+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=45, tm_sec=54, tm_wday=2, tm_yday=248, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=49, tm_sec=24, tm_wday=2, tm_yday=248, tm_isdst=0) 2024-09-04T17:49:30.066 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-04T17:45:46.343951+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=45, tm_sec=46, tm_wday=2, tm_yday=248, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=49, tm_sec=24, tm_wday=2, tm_yday=248, tm_isdst=0) 2024-09-04T17:49:30.066 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-09-04T17:45:54.925016+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=45, tm_sec=54, tm_wday=2, tm_yday=248, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=49, tm_sec=24, tm_wday=2, tm_yday=248, tm_isdst=0) 2024-09-04T17:49:30.066 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-04T17:45:46.343951+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=45, tm_sec=46, tm_wday=2, tm_yday=248, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=49, tm_sec=24, tm_wday=2, tm_yday=248, tm_isdst=0) 2024-09-04T17:49:30.067 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-04T17:45:54.925016+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=45, tm_sec=54, tm_wday=2, tm_yday=248, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=49, tm_sec=24, tm_wday=2, tm_yday=248, tm_isdst=0) 2024-09-04T17:49:30.067 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-04T17:45:46.343951+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=45, tm_sec=46, tm_wday=2, tm_yday=248, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=49, tm_sec=24, tm_wday=2, tm_yday=248, tm_isdst=0) 2024-09-04T17:49:30.067 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-04T17:45:54.925016+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=45, tm_sec=54, tm_wday=2, tm_yday=248, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=49, tm_sec=24, tm_wday=2, tm_yday=248, tm_isdst=0) 2024-09-04T17:49:30.067 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-04T17:45:46.343951+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=45, tm_sec=46, tm_wday=2, tm_yday=248, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=49, tm_sec=24, tm_wday=2, tm_yday=248, tm_isdst=0) 2024-09-04T17:49:30.067 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-09-04T17:45:54.925016+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=45, tm_sec=54, tm_wday=2, tm_yday=248, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=49, tm_sec=24, tm_wday=2, tm_yday=248, tm_isdst=0) 2024-09-04T17:49:30.067 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-04T17:45:54.925016+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=45, tm_sec=54, tm_wday=2, tm_yday=248, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=49, tm_sec=24, tm_wday=2, tm_yday=248, tm_isdst=0) 2024-09-04T17:49:30.067 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-04T17:45:46.343951+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=45, tm_sec=46, tm_wday=2, tm_yday=248, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=49, tm_sec=24, tm_wday=2, tm_yday=248, tm_isdst=0) 2024-09-04T17:49:30.067 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-04T17:45:54.925016+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=45, tm_sec=54, tm_wday=2, tm_yday=248, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=49, tm_sec=24, tm_wday=2, tm_yday=248, tm_isdst=0) 2024-09-04T17:49:30.067 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-04T17:45:46.343951+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=45, tm_sec=46, tm_wday=2, tm_yday=248, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=49, tm_sec=24, tm_wday=2, tm_yday=248, tm_isdst=0) 2024-09-04T17:49:30.068 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-09-04T17:45:54.925016+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=45, tm_sec=54, tm_wday=2, tm_yday=248, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=49, tm_sec=24, tm_wday=2, tm_yday=248, tm_isdst=0) 2024-09-04T17:49:30.068 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-04T17:45:46.343951+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=45, tm_sec=46, tm_wday=2, tm_yday=248, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=49, tm_sec=24, tm_wday=2, tm_yday=248, tm_isdst=0) 2024-09-04T17:49:30.068 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-04T17:45:54.925016+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=45, tm_sec=54, tm_wday=2, tm_yday=248, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=49, tm_sec=24, tm_wday=2, tm_yday=248, tm_isdst=0) 2024-09-04T17:49:30.068 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-04T17:45:46.343951+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=45, tm_sec=46, tm_wday=2, tm_yday=248, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=49, tm_sec=24, tm_wday=2, tm_yday=248, tm_isdst=0) 2024-09-04T17:49:30.068 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-04T17:45:41.305128+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=45, tm_sec=41, tm_wday=2, tm_yday=248, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=49, tm_sec=24, tm_wday=2, tm_yday=248, tm_isdst=0) 2024-09-04T17:49:30.068 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-04T17:45:54.925016+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=45, tm_sec=54, tm_wday=2, tm_yday=248, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=49, tm_sec=24, tm_wday=2, tm_yday=248, tm_isdst=0) 2024-09-04T17:49:30.068 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-04T17:45:54.925016+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=45, tm_sec=54, tm_wday=2, tm_yday=248, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=49, tm_sec=24, tm_wday=2, tm_yday=248, tm_isdst=0) 2024-09-04T17:49:30.069 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-04T17:45:54.925016+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=45, tm_sec=54, tm_wday=2, tm_yday=248, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=49, tm_sec=24, tm_wday=2, tm_yday=248, tm_isdst=0) 2024-09-04T17:49:30.069 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-04T17:45:54.925016+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=45, tm_sec=54, tm_wday=2, tm_yday=248, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=49, tm_sec=24, tm_wday=2, tm_yday=248, tm_isdst=0) 2024-09-04T17:49:30.069 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-04T17:45:54.925016+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=45, tm_sec=54, tm_wday=2, tm_yday=248, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=49, tm_sec=24, tm_wday=2, tm_yday=248, tm_isdst=0) 2024-09-04T17:49:30.069 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-04T17:45:54.925016+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=45, tm_sec=54, tm_wday=2, tm_yday=248, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=49, tm_sec=24, tm_wday=2, tm_yday=248, tm_isdst=0) 2024-09-04T17:49:30.069 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-04T17:45:54.925016+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=45, tm_sec=54, tm_wday=2, tm_yday=248, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=49, tm_sec=24, tm_wday=2, tm_yday=248, tm_isdst=0) 2024-09-04T17:49:30.069 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-04T17:45:54.925016+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=45, tm_sec=54, tm_wday=2, tm_yday=248, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=49, tm_sec=24, tm_wday=2, tm_yday=248, tm_isdst=0) 2024-09-04T17:49:30.069 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-04T17:45:54.925016+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=45, tm_sec=54, tm_wday=2, tm_yday=248, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=49, tm_sec=24, tm_wday=2, tm_yday=248, tm_isdst=0) 2024-09-04T17:49:30.070 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-04T17:45:54.925016+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=45, tm_sec=54, tm_wday=2, tm_yday=248, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=49, tm_sec=24, tm_wday=2, tm_yday=248, tm_isdst=0) 2024-09-04T17:49:30.070 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-04T17:45:54.925016+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=45, tm_sec=54, tm_wday=2, tm_yday=248, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=49, tm_sec=24, tm_wday=2, tm_yday=248, tm_isdst=0) 2024-09-04T17:49:30.070 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-04T17:45:54.925016+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=45, tm_sec=54, tm_wday=2, tm_yday=248, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=4, tm_hour=17, tm_min=49, tm_sec=24, tm_wday=2, tm_yday=248, tm_isdst=0) 2024-09-04T17:49:30.070 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-04T17:49:50.071 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-04T17:49:50.387 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-04T17:49:50.388 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2024-09-04T17:49:50.404 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":316,"stamp":"2024-09-04T17:49:49.139280+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":939,"num_object_clones":0,"num_object_copies":1878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":939,"num_whiteouts":931,"num_read":142346,"num_read_kb":876804,"num_write":20402,"num_write_kb":405906,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6763,"num_bytes_recovered":238961871,"num_keys_recovered":777,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19225,"ondisk_log_size":19225,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":396952,"kb_used_data":19800,"kb_used_omap":46,"kb_used_meta":376977,"kb_avail":754577768,"statfs":{"total":773094113280,"available":772687634432,"internally_reserved":0,"allocated":20275200,"data_stored":10444972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":47946,"internal_metadata":386024630},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003975"},"pg_stats":[{"pgid":"3.1f","version":"198'536","reported_seq":3177,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:29.537293+0000","last_change":"2024-09-04T17:49:29.537293+0000","last_active":"2024-09-04T17:49:29.537293+0000","last_peered":"2024-09-04T17:49:29.537293+0000","last_clean":"2024-09-04T17:49:29.537293+0000","last_became_active":"2024-09-04T17:48:09.845199+0000","last_became_peered":"2024-09-04T17:48:09.845199+0000","last_unstale":"2024-09-04T17:49:29.537293+0000","last_undegraded":"2024-09-04T17:49:29.537293+0000","last_fullsized":"2024-09-04T17:49:29.537293+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'536","last_scrub_stamp":"2024-09-04T17:49:29.537214+0000","last_deep_scrub":"198'536","last_deep_scrub_stamp":"2024-09-04T17:49:29.537214+0000","last_clean_scrub_stamp":"2024-09-04T17:49:29.537214+0000","objects_scrubbed":35,"log_size":536,"log_dups_size":0,"ondisk_log_size":536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T19:16:08.876547+0000","scrub_duration":43,"objects_trimmed":0,"snaptrim_duration":0.003726699,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":2983,"num_read_kb":27190,"num_write":524,"num_write_kb":10445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":24433786,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"198'679","reported_seq":4017,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:32.230178+0000","last_change":"2024-09-04T17:49:32.230178+0000","last_active":"2024-09-04T17:49:32.230178+0000","last_peered":"2024-09-04T17:49:32.230178+0000","last_clean":"2024-09-04T17:49:32.230178+0000","last_became_active":"2024-09-04T17:49:05.874586+0000","last_became_peered":"2024-09-04T17:49:05.874586+0000","last_unstale":"2024-09-04T17:49:32.230178+0000","last_undegraded":"2024-09-04T17:49:32.230178+0000","last_fullsized":"2024-09-04T17:49:32.230178+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'679","last_scrub_stamp":"2024-09-04T17:49:32.229935+0000","last_deep_scrub":"198'679","last_deep_scrub_stamp":"2024-09-04T17:49:32.229935+0000","last_clean_scrub_stamp":"2024-09-04T17:49:32.229935+0000","objects_scrubbed":22,"log_size":679,"log_dups_size":0,"ondisk_log_size":679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:15:44.490753+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0.001926142,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4757,"num_read_kb":30502,"num_write":815,"num_write_kb":10573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":322,"num_bytes_recovered":15412196,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"198'575","reported_seq":2590,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:33.191087+0000","last_change":"2024-09-04T17:49:33.191087+0000","last_active":"2024-09-04T17:49:33.191087+0000","last_peered":"2024-09-04T17:49:33.191087+0000","last_clean":"2024-09-04T17:49:33.191087+0000","last_became_active":"2024-09-04T17:45:55.955213+0000","last_became_peered":"2024-09-04T17:45:55.955213+0000","last_unstale":"2024-09-04T17:49:33.191087+0000","last_undegraded":"2024-09-04T17:49:33.191087+0000","last_fullsized":"2024-09-04T17:49:33.191087+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'575","last_scrub_stamp":"2024-09-04T17:49:33.190939+0000","last_deep_scrub":"198'575","last_deep_scrub_stamp":"2024-09-04T17:49:33.190939+0000","last_clean_scrub_stamp":"2024-09-04T17:49:33.190939+0000","objects_scrubbed":27,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:18:04.759937+0000","scrub_duration":55,"objects_trimmed":0,"snaptrim_duration":0.0022810360000000002,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":2820,"num_read_kb":20284,"num_write":623,"num_write_kb":9401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"198'794","reported_seq":6871,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:35.199679+0000","last_change":"2024-09-04T17:49:35.199679+0000","last_active":"2024-09-04T17:49:35.199679+0000","last_peered":"2024-09-04T17:49:35.199679+0000","last_clean":"2024-09-04T17:49:35.199679+0000","last_became_active":"2024-09-04T17:45:55.951880+0000","last_became_peered":"2024-09-04T17:45:55.951880+0000","last_unstale":"2024-09-04T17:49:35.199679+0000","last_undegraded":"2024-09-04T17:49:35.199679+0000","last_fullsized":"2024-09-04T17:49:35.199679+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'794","last_scrub_stamp":"2024-09-04T17:49:35.199418+0000","last_deep_scrub":"198'794","last_deep_scrub_stamp":"2024-09-04T17:49:35.199418+0000","last_clean_scrub_stamp":"2024-09-04T17:49:35.199418+0000","objects_scrubbed":36,"log_size":794,"log_dups_size":0,"ondisk_log_size":794,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T18:10:13.115074+0000","scrub_duration":67,"objects_trimmed":0,"snaptrim_duration":0.0025730409999999999,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":35,"num_read":7473,"num_read_kb":48018,"num_write":874,"num_write_kb":11556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"198'630","reported_seq":2452,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:31.603871+0000","last_change":"2024-09-04T17:49:31.603871+0000","last_active":"2024-09-04T17:49:31.603871+0000","last_peered":"2024-09-04T17:49:31.603871+0000","last_clean":"2024-09-04T17:49:31.603871+0000","last_became_active":"2024-09-04T17:48:09.853793+0000","last_became_peered":"2024-09-04T17:48:09.853793+0000","last_unstale":"2024-09-04T17:49:31.603871+0000","last_undegraded":"2024-09-04T17:49:31.603871+0000","last_fullsized":"2024-09-04T17:49:31.603871+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'630","last_scrub_stamp":"2024-09-04T17:49:31.603676+0000","last_deep_scrub":"198'630","last_deep_scrub_stamp":"2024-09-04T17:49:31.603676+0000","last_clean_scrub_stamp":"2024-09-04T17:49:31.603676+0000","objects_scrubbed":33,"log_size":630,"log_dups_size":0,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:23:28.415331+0000","scrub_duration":63,"objects_trimmed":0,"snaptrim_duration":0.0026463849999999998,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":3715,"num_read_kb":8148,"num_write":647,"num_write_kb":4600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":1993914,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"198'450","reported_seq":6490,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:30.975851+0000","last_change":"2024-09-04T17:49:30.975851+0000","last_active":"2024-09-04T17:49:30.975851+0000","last_peered":"2024-09-04T17:49:30.975851+0000","last_clean":"2024-09-04T17:49:30.975851+0000","last_became_active":"2024-09-04T17:49:10.137472+0000","last_became_peered":"2024-09-04T17:49:10.137472+0000","last_unstale":"2024-09-04T17:49:30.975851+0000","last_undegraded":"2024-09-04T17:49:30.975851+0000","last_fullsized":"2024-09-04T17:49:30.975851+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'450","last_scrub_stamp":"2024-09-04T17:49:30.975670+0000","last_deep_scrub":"198'450","last_deep_scrub_stamp":"2024-09-04T17:49:30.975670+0000","last_clean_scrub_stamp":"2024-09-04T17:49:30.975670+0000","objects_scrubbed":26,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T23:52:43.709940+0000","scrub_duration":47,"objects_trimmed":0,"snaptrim_duration":0.000885438,"stat_sum":{"num_bytes":19,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":25,"num_read":5992,"num_read_kb":45852,"num_write":376,"num_write_kb":15447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":5992020,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"198'460","reported_seq":2746,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:28.367503+0000","last_change":"2024-09-04T17:49:28.367503+0000","last_active":"2024-09-04T17:49:28.367503+0000","last_peered":"2024-09-04T17:49:28.367503+0000","last_clean":"2024-09-04T17:49:28.367503+0000","last_became_active":"2024-09-04T17:49:05.876985+0000","last_became_peered":"2024-09-04T17:49:05.876985+0000","last_unstale":"2024-09-04T17:49:28.367503+0000","last_undegraded":"2024-09-04T17:49:28.367503+0000","last_fullsized":"2024-09-04T17:49:28.367503+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'460","last_scrub_stamp":"2024-09-04T17:49:28.367341+0000","last_deep_scrub":"198'460","last_deep_scrub_stamp":"2024-09-04T17:49:28.367341+0000","last_clean_scrub_stamp":"2024-09-04T17:49:28.367341+0000","objects_scrubbed":18,"log_size":460,"log_dups_size":0,"ondisk_log_size":460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T18:43:34.904229+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.001467744,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":2248,"num_read_kb":16196,"num_write":460,"num_write_kb":4617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":494,"num_bytes_recovered":4911830,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"198'536","reported_seq":2597,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:31.271095+0000","last_change":"2024-09-04T17:49:31.271095+0000","last_active":"2024-09-04T17:49:31.271095+0000","last_peered":"2024-09-04T17:49:31.271095+0000","last_clean":"2024-09-04T17:49:31.271095+0000","last_became_active":"2024-09-04T17:49:07.878360+0000","last_became_peered":"2024-09-04T17:49:07.878360+0000","last_unstale":"2024-09-04T17:49:31.271095+0000","last_undegraded":"2024-09-04T17:49:31.271095+0000","last_fullsized":"2024-09-04T17:49:31.271095+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'536","last_scrub_stamp":"2024-09-04T17:49:31.270972+0000","last_deep_scrub":"198'536","last_deep_scrub_stamp":"2024-09-04T17:49:31.270972+0000","last_clean_scrub_stamp":"2024-09-04T17:49:31.270972+0000","objects_scrubbed":28,"log_size":536,"log_dups_size":0,"ondisk_log_size":536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T17:55:27.642063+0000","scrub_duration":47,"objects_trimmed":0,"snaptrim_duration":0.0017798989999999999,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":8892,"num_read_kb":35489,"num_write":443,"num_write_kb":14429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":597,"num_bytes_recovered":28259997,"num_keys_recovered":141,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"198'352","reported_seq":2419,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:28.552179+0000","last_change":"2024-09-04T17:49:28.552179+0000","last_active":"2024-09-04T17:49:28.552179+0000","last_peered":"2024-09-04T17:49:28.552179+0000","last_clean":"2024-09-04T17:49:28.552179+0000","last_became_active":"2024-09-04T17:45:55.958625+0000","last_became_peered":"2024-09-04T17:45:55.958625+0000","last_unstale":"2024-09-04T17:49:28.552179+0000","last_undegraded":"2024-09-04T17:49:28.552179+0000","last_fullsized":"2024-09-04T17:49:28.552179+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'352","last_scrub_stamp":"2024-09-04T17:49:28.551997+0000","last_deep_scrub":"198'352","last_deep_scrub_stamp":"2024-09-04T17:49:28.551997+0000","last_clean_scrub_stamp":"2024-09-04T17:49:28.551997+0000","objects_scrubbed":21,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T18:31:40.372122+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.001574039,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":1612,"num_read_kb":20939,"num_write":316,"num_write_kb":18406,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"198'388","reported_seq":2083,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:34.138896+0000","last_change":"2024-09-04T17:49:34.138896+0000","last_active":"2024-09-04T17:49:34.138896+0000","last_peered":"2024-09-04T17:49:34.138896+0000","last_clean":"2024-09-04T17:49:34.138896+0000","last_became_active":"2024-09-04T17:45:55.953254+0000","last_became_peered":"2024-09-04T17:45:55.953254+0000","last_unstale":"2024-09-04T17:49:34.138896+0000","last_undegraded":"2024-09-04T17:49:34.138896+0000","last_fullsized":"2024-09-04T17:49:34.138896+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'388","last_scrub_stamp":"2024-09-04T17:49:34.138567+0000","last_deep_scrub":"198'388","last_deep_scrub_stamp":"2024-09-04T17:49:34.138567+0000","last_clean_scrub_stamp":"2024-09-04T17:49:34.138567+0000","objects_scrubbed":24,"log_size":388,"log_dups_size":0,"ondisk_log_size":388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:40:57.174437+0000","scrub_duration":43,"objects_trimmed":0,"snaptrim_duration":0.0044959860000000004,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":1729,"num_read_kb":10785,"num_write":396,"num_write_kb":7391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"198'533","reported_seq":3087,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:36.526299+0000","last_change":"2024-09-04T17:49:36.526299+0000","last_active":"2024-09-04T17:49:36.526299+0000","last_peered":"2024-09-04T17:49:36.526299+0000","last_clean":"2024-09-04T17:49:36.526299+0000","last_became_active":"2024-09-04T17:49:06.884929+0000","last_became_peered":"2024-09-04T17:49:06.884929+0000","last_unstale":"2024-09-04T17:49:36.526299+0000","last_undegraded":"2024-09-04T17:49:36.526299+0000","last_fullsized":"2024-09-04T17:49:36.526299+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'533","last_scrub_stamp":"2024-09-04T17:49:36.526070+0000","last_deep_scrub":"198'533","last_deep_scrub_stamp":"2024-09-04T17:49:36.526070+0000","last_clean_scrub_stamp":"2024-09-04T17:49:36.526070+0000","objects_scrubbed":23,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:19:31.117237+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0.001003051,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":4020,"num_read_kb":16975,"num_write":525,"num_write_kb":4439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":446,"num_bytes_recovered":12989343,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"198'804","reported_seq":2566,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:32.027838+0000","last_change":"2024-09-04T17:49:32.027838+0000","last_active":"2024-09-04T17:49:32.027838+0000","last_peered":"2024-09-04T17:49:32.027838+0000","last_clean":"2024-09-04T17:49:32.027838+0000","last_became_active":"2024-09-04T17:49:10.137573+0000","last_became_peered":"2024-09-04T17:49:10.137573+0000","last_unstale":"2024-09-04T17:49:32.027838+0000","last_undegraded":"2024-09-04T17:49:32.027838+0000","last_fullsized":"2024-09-04T17:49:32.027838+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'804","last_scrub_stamp":"2024-09-04T17:49:32.027595+0000","last_deep_scrub":"198'804","last_deep_scrub_stamp":"2024-09-04T17:49:32.027595+0000","last_clean_scrub_stamp":"2024-09-04T17:49:32.027595+0000","objects_scrubbed":29,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:24:01.997414+0000","scrub_duration":59,"objects_trimmed":0,"snaptrim_duration":0.0027107059999999998,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":29,"num_read":4009,"num_read_kb":29862,"num_write":1001,"num_write_kb":35909,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":243,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:30.084663+0000","last_change":"2024-09-04T17:49:30.084663+0000","last_active":"2024-09-04T17:49:30.084663+0000","last_peered":"2024-09-04T17:49:30.084663+0000","last_clean":"2024-09-04T17:49:30.084663+0000","last_became_active":"2024-09-04T17:48:11.849710+0000","last_became_peered":"2024-09-04T17:48:11.849710+0000","last_unstale":"2024-09-04T17:49:30.084663+0000","last_undegraded":"2024-09-04T17:49:30.084663+0000","last_fullsized":"2024-09-04T17:49:30.084663+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:49:30.084484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:49:30.084484+0000","last_clean_scrub_stamp":"2024-09-04T17:49:30.084484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T18:11:31.117090+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.001226074,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"198'434","reported_seq":3464,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:26.510303+0000","last_change":"2024-09-04T17:49:26.510303+0000","last_active":"2024-09-04T17:49:26.510303+0000","last_peered":"2024-09-04T17:49:26.510303+0000","last_clean":"2024-09-04T17:49:26.510303+0000","last_became_active":"2024-09-04T17:49:07.874501+0000","last_became_peered":"2024-09-04T17:49:07.874501+0000","last_unstale":"2024-09-04T17:49:26.510303+0000","last_undegraded":"2024-09-04T17:49:26.510303+0000","last_fullsized":"2024-09-04T17:49:26.510303+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'434","last_scrub_stamp":"2024-09-04T17:49:26.510238+0000","last_deep_scrub":"198'434","last_deep_scrub_stamp":"2024-09-04T17:49:26.510238+0000","last_clean_scrub_stamp":"2024-09-04T17:49:26.510238+0000","objects_scrubbed":25,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T21:30:48.997523+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0.001494427,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":2485,"num_read_kb":14550,"num_write":410,"num_write_kb":5528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":919873,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":48,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:31.442716+0000","last_change":"2024-09-04T17:49:31.442716+0000","last_active":"2024-09-04T17:49:31.442716+0000","last_peered":"2024-09-04T17:49:31.442716+0000","last_clean":"2024-09-04T17:49:31.442716+0000","last_became_active":"2024-09-04T17:49:03.886525+0000","last_became_peered":"2024-09-04T17:49:03.886525+0000","last_unstale":"2024-09-04T17:49:31.442716+0000","last_undegraded":"2024-09-04T17:49:31.442716+0000","last_fullsized":"2024-09-04T17:49:31.442716+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:49:31.442532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:49:31.442532+0000","last_clean_scrub_stamp":"2024-09-04T17:49:31.442532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:24:36.662185+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0011364179999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"198'501","reported_seq":3045,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:30.259294+0000","last_change":"2024-09-04T17:49:30.259294+0000","last_active":"2024-09-04T17:49:30.259294+0000","last_peered":"2024-09-04T17:49:30.259294+0000","last_clean":"2024-09-04T17:49:30.259294+0000","last_became_active":"2024-09-04T17:49:05.874579+0000","last_became_peered":"2024-09-04T17:49:05.874579+0000","last_unstale":"2024-09-04T17:49:30.259294+0000","last_undegraded":"2024-09-04T17:49:30.259294+0000","last_fullsized":"2024-09-04T17:49:30.259294+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'501","last_scrub_stamp":"2024-09-04T17:49:30.259107+0000","last_deep_scrub":"198'501","last_deep_scrub_stamp":"2024-09-04T17:49:30.259107+0000","last_clean_scrub_stamp":"2024-09-04T17:49:30.259107+0000","objects_scrubbed":32,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T18:35:39.800493+0000","scrub_duration":43,"objects_trimmed":0,"snaptrim_duration":0.0019775819999999999,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":32,"num_read":2953,"num_read_kb":23389,"num_write":575,"num_write_kb":21569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":313,"num_bytes_recovered":1588111,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":380,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:30.564515+0000","last_change":"2024-09-04T17:49:30.564515+0000","last_active":"2024-09-04T17:49:30.564515+0000","last_peered":"2024-09-04T17:49:30.564515+0000","last_clean":"2024-09-04T17:49:30.564515+0000","last_became_active":"2024-09-04T17:45:47.369308+0000","last_became_peered":"2024-09-04T17:45:47.369308+0000","last_unstale":"2024-09-04T17:49:30.564515+0000","last_undegraded":"2024-09-04T17:49:30.564515+0000","last_fullsized":"2024-09-04T17:49:30.564515+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:49:30.564344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:49:30.564344+0000","last_clean_scrub_stamp":"2024-09-04T17:49:30.564344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T23:38:03.224328+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.001029826,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"198'441","reported_seq":2926,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:29.408378+0000","last_change":"2024-09-04T17:49:29.408378+0000","last_active":"2024-09-04T17:49:29.408378+0000","last_peered":"2024-09-04T17:49:29.408378+0000","last_clean":"2024-09-04T17:49:29.408378+0000","last_became_active":"2024-09-04T17:49:07.877184+0000","last_became_peered":"2024-09-04T17:49:07.877184+0000","last_unstale":"2024-09-04T17:49:29.408378+0000","last_undegraded":"2024-09-04T17:49:29.408378+0000","last_fullsized":"2024-09-04T17:49:29.408378+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'441","last_scrub_stamp":"2024-09-04T17:49:29.408313+0000","last_deep_scrub":"198'441","last_deep_scrub_stamp":"2024-09-04T17:49:29.408313+0000","last_clean_scrub_stamp":"2024-09-04T17:49:29.408313+0000","objects_scrubbed":31,"log_size":441,"log_dups_size":0,"ondisk_log_size":441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T22:51:58.560473+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.0029821800000000001,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":31,"num_read":2672,"num_read_kb":18782,"num_write":436,"num_write_kb":9856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":13375622,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":48,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:30.419298+0000","last_change":"2024-09-04T17:49:30.419298+0000","last_active":"2024-09-04T17:49:30.419298+0000","last_peered":"2024-09-04T17:49:30.419298+0000","last_clean":"2024-09-04T17:49:30.419298+0000","last_became_active":"2024-09-04T17:49:03.885776+0000","last_became_peered":"2024-09-04T17:49:03.885776+0000","last_unstale":"2024-09-04T17:49:30.419298+0000","last_undegraded":"2024-09-04T17:49:30.419298+0000","last_fullsized":"2024-09-04T17:49:30.419298+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:49:30.419112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:49:30.419112+0000","last_clean_scrub_stamp":"2024-09-04T17:49:30.419112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T20:27:05.349949+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"198'385","reported_seq":3078,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:32.209122+0000","last_change":"2024-09-04T17:49:32.209122+0000","last_active":"2024-09-04T17:49:32.209122+0000","last_peered":"2024-09-04T17:49:32.209122+0000","last_clean":"2024-09-04T17:49:32.209122+0000","last_became_active":"2024-09-04T17:49:06.885686+0000","last_became_peered":"2024-09-04T17:49:06.885686+0000","last_unstale":"2024-09-04T17:49:32.209122+0000","last_undegraded":"2024-09-04T17:49:32.209122+0000","last_fullsized":"2024-09-04T17:49:32.209122+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'385","last_scrub_stamp":"2024-09-04T17:49:32.209005+0000","last_deep_scrub":"198'385","last_deep_scrub_stamp":"2024-09-04T17:49:32.209005+0000","last_clean_scrub_stamp":"2024-09-04T17:49:32.209005+0000","objects_scrubbed":27,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:07:20.588074+0000","scrub_duration":51,"objects_trimmed":0,"snaptrim_duration":0.0019787199999999998,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":2555,"num_read_kb":7639,"num_write":305,"num_write_kb":7011,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":391,"num_bytes_recovered":15596571,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"198'917","reported_seq":3592,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:28.949805+0000","last_change":"2024-09-04T17:49:28.949805+0000","last_active":"2024-09-04T17:49:28.949805+0000","last_peered":"2024-09-04T17:49:28.949805+0000","last_clean":"2024-09-04T17:49:28.949805+0000","last_became_active":"2024-09-04T17:49:10.137895+0000","last_became_peered":"2024-09-04T17:49:10.137895+0000","last_unstale":"2024-09-04T17:49:28.949805+0000","last_undegraded":"2024-09-04T17:49:28.949805+0000","last_fullsized":"2024-09-04T17:49:28.949805+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'917","last_scrub_stamp":"2024-09-04T17:49:28.949626+0000","last_deep_scrub":"198'917","last_deep_scrub_stamp":"2024-09-04T17:49:28.949626+0000","last_clean_scrub_stamp":"2024-09-04T17:49:28.949626+0000","objects_scrubbed":30,"log_size":917,"log_dups_size":0,"ondisk_log_size":917,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:24:49.804305+0000","scrub_duration":51,"objects_trimmed":0,"snaptrim_duration":0.0032310310000000001,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":29,"num_read":4673,"num_read_kb":26561,"num_write":1100,"num_write_kb":14222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"198'3","reported_seq":407,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:29.158626+0000","last_change":"2024-09-04T17:49:29.158626+0000","last_active":"2024-09-04T17:49:29.158626+0000","last_peered":"2024-09-04T17:49:29.158626+0000","last_clean":"2024-09-04T17:49:29.158626+0000","last_became_active":"2024-09-04T17:49:03.887053+0000","last_became_peered":"2024-09-04T17:49:03.887053+0000","last_unstale":"2024-09-04T17:49:29.158626+0000","last_undegraded":"2024-09-04T17:49:29.158626+0000","last_fullsized":"2024-09-04T17:49:29.158626+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'3","last_scrub_stamp":"2024-09-04T17:49:29.158428+0000","last_deep_scrub":"198'3","last_deep_scrub_stamp":"2024-09-04T17:49:29.158428+0000","last_clean_scrub_stamp":"2024-09-04T17:49:29.158428+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T19:15:14.378320+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.0026316069999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"198'489","reported_seq":3485,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:27.405482+0000","last_change":"2024-09-04T17:49:27.405482+0000","last_active":"2024-09-04T17:49:27.405482+0000","last_peered":"2024-09-04T17:49:27.405482+0000","last_clean":"2024-09-04T17:49:27.405482+0000","last_became_active":"2024-09-04T17:49:07.876339+0000","last_became_peered":"2024-09-04T17:49:07.876339+0000","last_unstale":"2024-09-04T17:49:27.405482+0000","last_undegraded":"2024-09-04T17:49:27.405482+0000","last_fullsized":"2024-09-04T17:49:27.405482+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'489","last_scrub_stamp":"2024-09-04T17:49:27.405378+0000","last_deep_scrub":"198'489","last_deep_scrub_stamp":"2024-09-04T17:49:27.405378+0000","last_clean_scrub_stamp":"2024-09-04T17:49:27.405378+0000","objects_scrubbed":36,"log_size":489,"log_dups_size":0,"ondisk_log_size":489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T18:34:58.855239+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0.00120166,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":36,"num_read":3566,"num_read_kb":22521,"num_write":365,"num_write_kb":4893,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":261,"num_bytes_recovered":5692590,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":379,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:27.806594+0000","last_change":"2024-09-04T17:49:27.806468+0000","last_active":"2024-09-04T17:49:27.806594+0000","last_peered":"2024-09-04T17:49:27.806594+0000","last_clean":"2024-09-04T17:49:27.806594+0000","last_became_active":"2024-09-04T17:49:02.864647+0000","last_became_peered":"2024-09-04T17:49:02.864647+0000","last_unstale":"2024-09-04T17:49:27.806594+0000","last_undegraded":"2024-09-04T17:49:27.806594+0000","last_fullsized":"2024-09-04T17:49:27.806594+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:49:27.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:49:27.806294+0000","last_clean_scrub_stamp":"2024-09-04T17:49:27.806294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T22:46:23.640387+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.001025195,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"198'664","reported_seq":3411,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:27.588325+0000","last_change":"2024-09-04T17:49:27.588325+0000","last_active":"2024-09-04T17:49:27.588325+0000","last_peered":"2024-09-04T17:49:27.588325+0000","last_clean":"2024-09-04T17:49:27.588325+0000","last_became_active":"2024-09-04T17:48:11.845510+0000","last_became_peered":"2024-09-04T17:48:11.845510+0000","last_unstale":"2024-09-04T17:49:27.588325+0000","last_undegraded":"2024-09-04T17:49:27.588325+0000","last_fullsized":"2024-09-04T17:49:27.588325+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'664","last_scrub_stamp":"2024-09-04T17:49:27.588233+0000","last_deep_scrub":"198'664","last_deep_scrub_stamp":"2024-09-04T17:49:27.588233+0000","last_clean_scrub_stamp":"2024-09-04T17:49:27.588233+0000","objects_scrubbed":27,"log_size":664,"log_dups_size":0,"ondisk_log_size":664,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:52:08.806665+0000","scrub_duration":103,"objects_trimmed":0,"snaptrim_duration":0.0031660479999999999,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":3316,"num_read_kb":40546,"num_write":672,"num_write_kb":23359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":240,"num_bytes_recovered":21297737,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":392,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:31.532091+0000","last_change":"2024-09-04T17:49:31.532091+0000","last_active":"2024-09-04T17:49:31.532091+0000","last_peered":"2024-09-04T17:49:31.532091+0000","last_clean":"2024-09-04T17:49:31.532091+0000","last_became_active":"2024-09-04T17:49:03.870678+0000","last_became_peered":"2024-09-04T17:49:03.870678+0000","last_unstale":"2024-09-04T17:49:31.532091+0000","last_undegraded":"2024-09-04T17:49:31.532091+0000","last_fullsized":"2024-09-04T17:49:31.532091+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-04T17:49:31.531912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-04T17:49:31.531912+0000","last_clean_scrub_stamp":"2024-09-04T17:49:31.531912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T18:29:42.042435+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0010553590000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"198'650","reported_seq":3096,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:29.214863+0000","last_change":"2024-09-04T17:49:29.214863+0000","last_active":"2024-09-04T17:49:29.214863+0000","last_peered":"2024-09-04T17:49:29.214863+0000","last_clean":"2024-09-04T17:49:29.214863+0000","last_became_active":"2024-09-04T17:49:07.876340+0000","last_became_peered":"2024-09-04T17:49:07.876340+0000","last_unstale":"2024-09-04T17:49:29.214863+0000","last_undegraded":"2024-09-04T17:49:29.214863+0000","last_fullsized":"2024-09-04T17:49:29.214863+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'650","last_scrub_stamp":"2024-09-04T17:49:29.214797+0000","last_deep_scrub":"198'650","last_deep_scrub_stamp":"2024-09-04T17:49:29.214797+0000","last_clean_scrub_stamp":"2024-09-04T17:49:29.214797+0000","objects_scrubbed":28,"log_size":650,"log_dups_size":0,"ondisk_log_size":650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:19:33.458279+0000","scrub_duration":43,"objects_trimmed":0,"snaptrim_duration":0.0021278730000000002,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":5714,"num_read_kb":30160,"num_write":740,"num_write_kb":13818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":405,"num_bytes_recovered":16173004,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"198'2","reported_seq":391,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:30.189260+0000","last_change":"2024-09-04T17:49:30.189260+0000","last_active":"2024-09-04T17:49:30.189260+0000","last_peered":"2024-09-04T17:49:30.189260+0000","last_clean":"2024-09-04T17:49:30.189260+0000","last_became_active":"2024-09-04T17:47:41.919027+0000","last_became_peered":"2024-09-04T17:47:41.919027+0000","last_unstale":"2024-09-04T17:49:30.189260+0000","last_undegraded":"2024-09-04T17:49:30.189260+0000","last_fullsized":"2024-09-04T17:49:30.189260+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'2","last_scrub_stamp":"2024-09-04T17:49:30.189074+0000","last_deep_scrub":"198'2","last_deep_scrub_stamp":"2024-09-04T17:49:30.189074+0000","last_clean_scrub_stamp":"2024-09-04T17:49:30.189074+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:14:21.359733+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.00086618299999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"198'74","reported_seq":474,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:32.508140+0000","last_change":"2024-09-04T17:49:32.507985+0000","last_active":"2024-09-04T17:49:32.508140+0000","last_peered":"2024-09-04T17:49:32.508140+0000","last_clean":"2024-09-04T17:49:32.508140+0000","last_became_active":"2024-09-04T17:45:42.339286+0000","last_became_peered":"2024-09-04T17:45:42.339286+0000","last_unstale":"2024-09-04T17:49:32.508140+0000","last_undegraded":"2024-09-04T17:49:32.508140+0000","last_fullsized":"2024-09-04T17:49:32.508140+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'74","last_scrub_stamp":"2024-09-04T17:49:32.507799+0000","last_deep_scrub":"198'74","last_deep_scrub_stamp":"2024-09-04T17:49:32.507799+0000","last_clean_scrub_stamp":"2024-09-04T17:49:32.507799+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:48:04.902375+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"198'578","reported_seq":3104,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:28.159244+0000","last_change":"2024-09-04T17:49:28.159244+0000","last_active":"2024-09-04T17:49:28.159244+0000","last_peered":"2024-09-04T17:49:28.159244+0000","last_clean":"2024-09-04T17:49:28.159244+0000","last_became_active":"2024-09-04T17:49:07.879442+0000","last_became_peered":"2024-09-04T17:49:07.879442+0000","last_unstale":"2024-09-04T17:49:28.159244+0000","last_undegraded":"2024-09-04T17:49:28.159244+0000","last_fullsized":"2024-09-04T17:49:28.159244+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'578","last_scrub_stamp":"2024-09-04T17:49:28.159175+0000","last_deep_scrub":"198'578","last_deep_scrub_stamp":"2024-09-04T17:49:28.159175+0000","last_clean_scrub_stamp":"2024-09-04T17:49:28.159175+0000","objects_scrubbed":30,"log_size":578,"log_dups_size":0,"ondisk_log_size":578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T19:23:39.290787+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0.0011779189999999999,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":2788,"num_read_kb":17264,"num_write":577,"num_write_kb":29373,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":445,"num_bytes_recovered":17940277,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"198'749","reported_seq":4773,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:29.987453+0000","last_change":"2024-09-04T17:49:29.987453+0000","last_active":"2024-09-04T17:49:29.987453+0000","last_peered":"2024-09-04T17:49:29.987453+0000","last_clean":"2024-09-04T17:49:29.987453+0000","last_became_active":"2024-09-04T17:49:10.133837+0000","last_became_peered":"2024-09-04T17:49:10.133837+0000","last_unstale":"2024-09-04T17:49:29.987453+0000","last_undegraded":"2024-09-04T17:49:29.987453+0000","last_fullsized":"2024-09-04T17:49:29.987453+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'749","last_scrub_stamp":"2024-09-04T17:49:29.987257+0000","last_deep_scrub":"198'749","last_deep_scrub_stamp":"2024-09-04T17:49:29.987257+0000","last_clean_scrub_stamp":"2024-09-04T17:49:29.987257+0000","objects_scrubbed":33,"log_size":749,"log_dups_size":0,"ondisk_log_size":749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:16:24.533382+0000","scrub_duration":55,"objects_trimmed":0,"snaptrim_duration":0.003529346,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":6198,"num_read_kb":37498,"num_write":772,"num_write_kb":18653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":4534479,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"198'688","reported_seq":3686,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:32.200505+0000","last_change":"2024-09-04T17:49:32.200505+0000","last_active":"2024-09-04T17:49:32.200505+0000","last_peered":"2024-09-04T17:49:32.200505+0000","last_clean":"2024-09-04T17:49:32.200505+0000","last_became_active":"2024-09-04T17:48:00.841070+0000","last_became_peered":"2024-09-04T17:48:00.841070+0000","last_unstale":"2024-09-04T17:49:32.200505+0000","last_undegraded":"2024-09-04T17:49:32.200505+0000","last_fullsized":"2024-09-04T17:49:32.200505+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'688","last_scrub_stamp":"2024-09-04T17:49:32.200303+0000","last_deep_scrub":"198'688","last_deep_scrub_stamp":"2024-09-04T17:49:32.200303+0000","last_clean_scrub_stamp":"2024-09-04T17:49:32.200303+0000","objects_scrubbed":32,"log_size":688,"log_dups_size":0,"ondisk_log_size":688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:40:38.298692+0000","scrub_duration":63,"objects_trimmed":0,"snaptrim_duration":0.003342214,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":31,"num_read":5699,"num_read_kb":16474,"num_write":617,"num_write_kb":2753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":1572433,"num_keys_recovered":66,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"198'797","reported_seq":3020,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:33.199993+0000","last_change":"2024-09-04T17:49:33.199993+0000","last_active":"2024-09-04T17:49:33.199993+0000","last_peered":"2024-09-04T17:49:33.199993+0000","last_clean":"2024-09-04T17:49:33.199993+0000","last_became_active":"2024-09-04T17:49:06.883685+0000","last_became_peered":"2024-09-04T17:49:06.883685+0000","last_unstale":"2024-09-04T17:49:33.199993+0000","last_undegraded":"2024-09-04T17:49:33.199993+0000","last_fullsized":"2024-09-04T17:49:33.199993+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'797","last_scrub_stamp":"2024-09-04T17:49:33.199811+0000","last_deep_scrub":"198'797","last_deep_scrub_stamp":"2024-09-04T17:49:33.199811+0000","last_clean_scrub_stamp":"2024-09-04T17:49:33.199811+0000","objects_scrubbed":27,"log_size":797,"log_dups_size":0,"ondisk_log_size":797,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:35:45.811861+0000","scrub_duration":51,"objects_trimmed":0,"snaptrim_duration":0.002090967,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":4267,"num_read_kb":14813,"num_write":915,"num_write_kb":7927,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":459,"num_bytes_recovered":5211501,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"198'681","reported_seq":9209,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:31.254676+0000","last_change":"2024-09-04T17:49:31.254676+0000","last_active":"2024-09-04T17:49:31.254676+0000","last_peered":"2024-09-04T17:49:31.254676+0000","last_clean":"2024-09-04T17:49:31.254676+0000","last_became_active":"2024-09-04T17:49:07.880558+0000","last_became_peered":"2024-09-04T17:49:07.880558+0000","last_unstale":"2024-09-04T17:49:31.254676+0000","last_undegraded":"2024-09-04T17:49:31.254676+0000","last_fullsized":"2024-09-04T17:49:31.254676+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'681","last_scrub_stamp":"2024-09-04T17:49:31.254494+0000","last_deep_scrub":"198'681","last_deep_scrub_stamp":"2024-09-04T17:49:31.254494+0000","last_clean_scrub_stamp":"2024-09-04T17:49:31.254494+0000","objects_scrubbed":34,"log_size":681,"log_dups_size":0,"ondisk_log_size":681,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T18:29:13.407115+0000","scrub_duration":59,"objects_trimmed":0,"snaptrim_duration":0.00096708700000000005,"stat_sum":{"num_bytes":0,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":34,"num_read":9052,"num_read_kb":71439,"num_write":754,"num_write_kb":16863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":537,"num_bytes_recovered":8281724,"num_keys_recovered":57,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"198'1389","reported_seq":4627,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:33.580337+0000","last_change":"2024-09-04T17:49:33.580337+0000","last_active":"2024-09-04T17:49:33.580337+0000","last_peered":"2024-09-04T17:49:33.580337+0000","last_clean":"2024-09-04T17:49:33.580337+0000","last_became_active":"2024-09-04T17:48:12.850381+0000","last_became_peered":"2024-09-04T17:48:12.850381+0000","last_unstale":"2024-09-04T17:49:33.580337+0000","last_undegraded":"2024-09-04T17:49:33.580337+0000","last_fullsized":"2024-09-04T17:49:33.580337+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'1389","last_scrub_stamp":"2024-09-04T17:49:33.580141+0000","last_deep_scrub":"198'1389","last_deep_scrub_stamp":"2024-09-04T17:49:33.580141+0000","last_clean_scrub_stamp":"2024-09-04T17:49:33.580141+0000","objects_scrubbed":31,"log_size":1389,"log_dups_size":0,"ondisk_log_size":1389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:03:25.302710+0000","scrub_duration":67,"objects_trimmed":0,"snaptrim_duration":0.002193296,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":31,"num_read":8863,"num_read_kb":26532,"num_write":2254,"num_write_kb":13198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":179,"num_bytes_recovered":18877339,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"198'564","reported_seq":6109,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:34.559006+0000","last_change":"2024-09-04T17:49:34.559006+0000","last_active":"2024-09-04T17:49:34.559006+0000","last_peered":"2024-09-04T17:49:34.559006+0000","last_clean":"2024-09-04T17:49:34.559006+0000","last_became_active":"2024-09-04T17:48:11.849605+0000","last_became_peered":"2024-09-04T17:48:11.849605+0000","last_unstale":"2024-09-04T17:49:34.559006+0000","last_undegraded":"2024-09-04T17:49:34.559006+0000","last_fullsized":"2024-09-04T17:49:34.559006+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'564","last_scrub_stamp":"2024-09-04T17:49:34.558646+0000","last_deep_scrub":"198'564","last_deep_scrub_stamp":"2024-09-04T17:49:34.558646+0000","last_clean_scrub_stamp":"2024-09-04T17:49:34.558646+0000","objects_scrubbed":23,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T19:50:39.818317+0000","scrub_duration":47,"objects_trimmed":0,"snaptrim_duration":0.001652911,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":6678,"num_read_kb":28511,"num_write":523,"num_write_kb":11228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":7806862,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"198'374","reported_seq":3268,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:37.571965+0000","last_change":"2024-09-04T17:49:37.571965+0000","last_active":"2024-09-04T17:49:37.571965+0000","last_peered":"2024-09-04T17:49:37.571965+0000","last_clean":"2024-09-04T17:49:37.571965+0000","last_became_active":"2024-09-04T17:45:55.956734+0000","last_became_peered":"2024-09-04T17:45:55.956734+0000","last_unstale":"2024-09-04T17:49:37.571965+0000","last_undegraded":"2024-09-04T17:49:37.571965+0000","last_fullsized":"2024-09-04T17:49:37.571965+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'374","last_scrub_stamp":"2024-09-04T17:49:37.571764+0000","last_deep_scrub":"198'374","last_deep_scrub_stamp":"2024-09-04T17:49:37.571764+0000","last_clean_scrub_stamp":"2024-09-04T17:49:37.571764+0000","objects_scrubbed":26,"log_size":374,"log_dups_size":0,"ondisk_log_size":374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-05T22:59:54.149109+0000","scrub_duration":43,"objects_trimmed":0,"snaptrim_duration":0.0065435850000000002,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":2605,"num_read_kb":44969,"num_write":330,"num_write_kb":13427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"198'667","reported_seq":3904,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:31.185770+0000","last_change":"2024-09-04T17:49:31.185770+0000","last_active":"2024-09-04T17:49:31.185770+0000","last_peered":"2024-09-04T17:49:31.185770+0000","last_clean":"2024-09-04T17:49:31.185770+0000","last_became_active":"2024-09-04T17:48:00.843136+0000","last_became_peered":"2024-09-04T17:48:00.843136+0000","last_unstale":"2024-09-04T17:49:31.185770+0000","last_undegraded":"2024-09-04T17:49:31.185770+0000","last_fullsized":"2024-09-04T17:49:31.185770+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'667","last_scrub_stamp":"2024-09-04T17:49:31.185569+0000","last_deep_scrub":"198'667","last_deep_scrub_stamp":"2024-09-04T17:49:31.185569+0000","last_clean_scrub_stamp":"2024-09-04T17:49:31.185569+0000","objects_scrubbed":37,"log_size":667,"log_dups_size":0,"ondisk_log_size":667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:15:27.987295+0000","scrub_duration":51,"objects_trimmed":0,"snaptrim_duration":0.0025156979999999998,"stat_sum":{"num_bytes":0,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":37,"num_read":4786,"num_read_kb":18960,"num_write":673,"num_write_kb":12854,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":3377599,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"198'637","reported_seq":6389,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:35.599944+0000","last_change":"2024-09-04T17:49:35.599944+0000","last_active":"2024-09-04T17:49:35.599944+0000","last_peered":"2024-09-04T17:49:35.599944+0000","last_clean":"2024-09-04T17:49:35.599944+0000","last_became_active":"2024-09-04T17:45:55.951870+0000","last_became_peered":"2024-09-04T17:45:55.951870+0000","last_unstale":"2024-09-04T17:49:35.599944+0000","last_undegraded":"2024-09-04T17:49:35.599944+0000","last_fullsized":"2024-09-04T17:49:35.599944+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'637","last_scrub_stamp":"2024-09-04T17:49:35.599699+0000","last_deep_scrub":"198'637","last_deep_scrub_stamp":"2024-09-04T17:49:35.599699+0000","last_clean_scrub_stamp":"2024-09-04T17:49:35.599699+0000","objects_scrubbed":38,"log_size":637,"log_dups_size":0,"ondisk_log_size":637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T00:32:18.941429+0000","scrub_duration":67,"objects_trimmed":0,"snaptrim_duration":0.0012658840000000001,"stat_sum":{"num_bytes":0,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":38,"num_read":7163,"num_read_kb":61439,"num_write":584,"num_write_kb":17576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"198'456","reported_seq":4265,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:30.561927+0000","last_change":"2024-09-04T17:49:30.561927+0000","last_active":"2024-09-04T17:49:30.561927+0000","last_peered":"2024-09-04T17:49:30.561927+0000","last_clean":"2024-09-04T17:49:30.561927+0000","last_became_active":"2024-09-04T17:48:14.943778+0000","last_became_peered":"2024-09-04T17:48:14.943778+0000","last_unstale":"2024-09-04T17:49:30.561927+0000","last_undegraded":"2024-09-04T17:49:30.561927+0000","last_fullsized":"2024-09-04T17:49:30.561927+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'456","last_scrub_stamp":"2024-09-04T17:49:30.561673+0000","last_deep_scrub":"198'456","last_deep_scrub_stamp":"2024-09-04T17:49:30.561673+0000","last_clean_scrub_stamp":"2024-09-04T17:49:30.561673+0000","objects_scrubbed":31,"log_size":456,"log_dups_size":0,"ondisk_log_size":456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:00:36.767820+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0.0011847660000000001,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":31,"num_read":3577,"num_read_kb":21719,"num_write":447,"num_write_kb":8575,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":290,"num_bytes_recovered":2723025,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"198'348","reported_seq":2739,"reported_epoch":198,"state":"active+clean","last_fresh":"2024-09-04T17:49:38.570476+0000","last_change":"2024-09-04T17:49:38.570476+0000","last_active":"2024-09-04T17:49:38.570476+0000","last_peered":"2024-09-04T17:49:38.570476+0000","last_clean":"2024-09-04T17:49:38.570476+0000","last_became_active":"2024-09-04T17:45:55.954544+0000","last_became_peered":"2024-09-04T17:45:55.954544+0000","last_unstale":"2024-09-04T17:49:38.570476+0000","last_undegraded":"2024-09-04T17:49:38.570476+0000","last_fullsized":"2024-09-04T17:49:38.570476+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'348","last_scrub_stamp":"2024-09-04T17:49:38.570241+0000","last_deep_scrub":"198'348","last_deep_scrub_stamp":"2024-09-04T17:49:38.570241+0000","last_clean_scrub_stamp":"2024-09-04T17:49:38.570241+0000","objects_scrubbed":35,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:28:14.689048+0000","scrub_duration":55,"objects_trimmed":0,"snaptrim_duration":0.001451911,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":2420,"num_read_kb":22743,"num_write":250,"num_write_kb":4380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":935,"num_object_clones":0,"num_object_copies":1870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":935,"num_whiteouts":931,"num_read":142280,"num_read_kb":876749,"num_write":20299,"num_write_kb":404276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6761,"num_bytes_recovered":238961833,"num_keys_recovered":777,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21395,"internal_metadata":0},"log_size":19146,"ondisk_log_size":19146,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":117,"seq":502511173656,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44392,"kb_used_data":2328,"kb_used_omap":22,"kb_used_meta":42025,"kb_avail":94327448,"statfs":{"total":96636764160,"available":96591306752,"internally_reserved":0,"allocated":2383872,"data_stored":1158020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22742,"internal_metadata":43034410},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509493,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55724,"kb_used_data":2332,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316116,"statfs":{"total":96636764160,"available":96579702784,"internally_reserved":0,"allocated":2387968,"data_stored":1158039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509494,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55724,"kb_used_data":2332,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316116,"statfs":{"total":96636764160,"available":96579702784,"internally_reserved":0,"allocated":2387968,"data_stored":1158039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":48,"seq":206158430250,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60840,"kb_used_data":2328,"kb_used_omap":4,"kb_used_meta":58491,"kb_avail":94311000,"statfs":{"total":96636764160,"available":96574464000,"internally_reserved":0,"allocated":2383872,"data_stored":1158020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4430,"internal_metadata":59895474},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509493,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29360,"kb_used_data":2336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342480,"statfs":{"total":96636764160,"available":96606699520,"internally_reserved":0,"allocated":2392064,"data_stored":1158058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509493,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60460,"kb_used_data":2908,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311380,"statfs":{"total":96636764160,"available":96574853120,"internally_reserved":0,"allocated":2977792,"data_stored":1748388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542197,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60460,"kb_used_data":2908,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311380,"statfs":{"total":96636764160,"available":96574853120,"internally_reserved":0,"allocated":2977792,"data_stored":1748388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":103,"seq":442381631515,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29992,"kb_used_data":2328,"kb_used_omap":12,"kb_used_meta":27635,"kb_avail":94341848,"statfs":{"total":96636764160,"available":96606052352,"internally_reserved":0,"allocated":2383872,"data_stored":1158020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12804,"internal_metadata":28298748},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6121,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15274,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-04T17:49:50.408 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-04T17:49:50.708 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-04T17:49:50.708 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-04T17:49:50.708 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-04T17:49:50.708 INFO:teuthology.orchestra.run:waiting for 300 2024-09-04T17:49:50.776 INFO:tasks.ceph.osd.0:Stopped 2024-09-04T17:49:50.776 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-04T17:49:50.777 INFO:teuthology.orchestra.run:waiting for 300 2024-09-04T17:49:56.712 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-09-04T17:49:56.878 INFO:tasks.ceph.osd.1:Stopped 2024-09-04T17:49:56.879 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-04T17:49:56.879 INFO:teuthology.orchestra.run:waiting for 300 2024-09-04T17:49:56.974 INFO:tasks.ceph.osd.2:Stopped 2024-09-04T17:49:56.974 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-04T17:49:56.974 INFO:teuthology.orchestra.run:waiting for 300 2024-09-04T17:49:57.063 INFO:tasks.ceph.osd.3:Stopped 2024-09-04T17:49:57.063 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-04T17:49:57.063 INFO:teuthology.orchestra.run:waiting for 300 2024-09-04T17:49:57.157 INFO:tasks.ceph.osd.4:Stopped 2024-09-04T17:49:57.157 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-04T17:49:57.157 INFO:teuthology.orchestra.run:waiting for 300 2024-09-04T17:49:57.250 INFO:tasks.ceph.osd.5:Stopped 2024-09-04T17:49:57.250 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-04T17:49:57.251 INFO:teuthology.orchestra.run:waiting for 300 2024-09-04T17:49:57.333 INFO:tasks.ceph.osd.6:Stopped 2024-09-04T17:49:57.333 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-04T17:49:57.333 INFO:teuthology.orchestra.run:waiting for 300 2024-09-04T17:49:57.429 INFO:tasks.ceph.osd.7:Stopped 2024-09-04T17:49:57.429 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-04T17:49:57.430 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-04T17:49:57.430 INFO:teuthology.orchestra.run:waiting for 300 2024-09-04T17:49:57.458 INFO:tasks.ceph.mgr.y:Stopped 2024-09-04T17:49:57.458 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-04T17:49:57.458 INFO:teuthology.orchestra.run:waiting for 300 2024-09-04T17:49:57.489 INFO:tasks.ceph.mgr.x:Stopped 2024-09-04T17:49:57.489 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-04T17:49:57.489 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-04T17:49:57.489 INFO:teuthology.orchestra.run:waiting for 300 2024-09-04T17:49:57.521 INFO:tasks.ceph.mon.a:Stopped 2024-09-04T17:49:57.521 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-04T17:49:57.521 INFO:teuthology.orchestra.run:waiting for 300 2024-09-04T17:49:57.591 INFO:tasks.ceph.mon.c:Stopped 2024-09-04T17:49:57.592 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-04T17:49:57.592 INFO:teuthology.orchestra.run:waiting for 300 2024-09-04T17:49:57.612 INFO:tasks.ceph.mon.b:Stopped 2024-09-04T17:49:57.612 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-04T17:49:57.613 DEBUG:teuthology.orchestra.run.smithi092:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v SLOW_OPS | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | egrep -v noscrub | egrep -v nodeep-scrub | head -n 1 2024-09-04T17:49:57.650 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi092.front.sepia.ceph.com 2024-09-04T17:49:57.651 DEBUG:teuthology.orchestra.run.smithi092:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-04T17:49:58.079 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi092.front.sepia.ceph.com 2024-09-04T17:49:58.079 DEBUG:teuthology.orchestra.run.smithi092:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-04T17:49:58.312 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi092.front.sepia.ceph.com 2024-09-04T17:49:58.312 DEBUG:teuthology.orchestra.run.smithi092:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-04T17:49:58.529 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi092.front.sepia.ceph.com 2024-09-04T17:49:58.529 DEBUG:teuthology.orchestra.run.smithi092:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-04T17:49:58.721 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi190.front.sepia.ceph.com 2024-09-04T17:49:58.722 DEBUG:teuthology.orchestra.run.smithi190:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-04T17:49:58.992 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi190.front.sepia.ceph.com 2024-09-04T17:49:58.992 DEBUG:teuthology.orchestra.run.smithi190:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-04T17:49:59.280 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi190.front.sepia.ceph.com 2024-09-04T17:49:59.280 DEBUG:teuthology.orchestra.run.smithi190:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-04T17:49:59.501 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi190.front.sepia.ceph.com 2024-09-04T17:49:59.501 DEBUG:teuthology.orchestra.run.smithi190:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-04T17:49:59.727 INFO:tasks.ceph:Archiving mon data... 2024-09-04T17:49:59.728 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/dis-2024-09-04_17:03:08-rbd-wip-dis-testing-distro-default-smithi/7889281/data/mon.a.tgz 2024-09-04T17:49:59.729 DEBUG:teuthology.orchestra.run.smithi092:> mktemp 2024-09-04T17:49:59.734 INFO:teuthology.orchestra.run.smithi092.stdout:/tmp/tmp.zupjnVJoUY 2024-09-04T17:49:59.735 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.zupjnVJoUY 2024-09-04T17:49:59.889 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0666 /tmp/tmp.zupjnVJoUY 2024-09-04T17:49:59.952 DEBUG:teuthology.orchestra.remote:smithi092:/tmp/tmp.zupjnVJoUY is 464KB 2024-09-04T17:50:00.015 DEBUG:teuthology.orchestra.run.smithi092:> rm -fr /tmp/tmp.zupjnVJoUY 2024-09-04T17:50:00.021 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/dis-2024-09-04_17:03:08-rbd-wip-dis-testing-distro-default-smithi/7889281/data/mon.c.tgz 2024-09-04T17:50:00.022 DEBUG:teuthology.orchestra.run.smithi092:> mktemp 2024-09-04T17:50:00.069 INFO:teuthology.orchestra.run.smithi092.stdout:/tmp/tmp.PJvAHsw0Nj 2024-09-04T17:50:00.070 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.PJvAHsw0Nj 2024-09-04T17:50:00.261 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0666 /tmp/tmp.PJvAHsw0Nj 2024-09-04T17:50:00.325 DEBUG:teuthology.orchestra.remote:smithi092:/tmp/tmp.PJvAHsw0Nj is 638KB 2024-09-04T17:50:00.387 DEBUG:teuthology.orchestra.run.smithi092:> rm -fr /tmp/tmp.PJvAHsw0Nj 2024-09-04T17:50:00.392 DEBUG:teuthology.misc:Transferring archived files from smithi190:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/dis-2024-09-04_17:03:08-rbd-wip-dis-testing-distro-default-smithi/7889281/data/mon.b.tgz 2024-09-04T17:50:00.393 DEBUG:teuthology.orchestra.run.smithi190:> mktemp 2024-09-04T17:50:00.398 INFO:teuthology.orchestra.run.smithi190.stdout:/tmp/tmp.7ZtwvgeQ7w 2024-09-04T17:50:00.399 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.7ZtwvgeQ7w 2024-09-04T17:50:00.592 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 0666 /tmp/tmp.7ZtwvgeQ7w 2024-09-04T17:50:00.657 DEBUG:teuthology.orchestra.remote:smithi190:/tmp/tmp.7ZtwvgeQ7w is 638KB 2024-09-04T17:50:00.719 DEBUG:teuthology.orchestra.run.smithi190:> rm -fr /tmp/tmp.7ZtwvgeQ7w 2024-09-04T17:50:00.724 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-04T17:50:00.724 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-04T17:50:00.727 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-04T17:50:00.787 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-04T17:50:00.829 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-04T17:50:00.829 INFO:tasks.ceph:Archiving crash dumps... 2024-09-04T17:50:00.831 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/lib/ceph/crash to /home/teuthworker/archive/dis-2024-09-04_17:03:08-rbd-wip-dis-testing-distro-default-smithi/7889281/remote/smithi092/crash 2024-09-04T17:50:00.832 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-04T17:50:00.845 DEBUG:teuthology.misc:Transferring archived files from smithi190:/var/lib/ceph/crash to /home/teuthworker/archive/dis-2024-09-04_17:03:08-rbd-wip-dis-testing-distro-default-smithi/7889281/remote/smithi190/crash 2024-09-04T17:50:00.846 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-04T17:50:00.885 INFO:tasks.ceph:Compressing logs... 2024-09-04T17:50:00.885 DEBUG:teuthology.orchestra.run.smithi092:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-04T17:50:00.890 DEBUG:teuthology.orchestra.run.smithi190:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-04T17:50:00.918 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38945.log 2024-09-04T17:50:00.919 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36942.log 2024-09-04T17:50:00.919 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.38945.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38945.log.gz 2024-09-04T17:50:00.919 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43141.log 2024-09-04T17:50:00.919 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.36942.log: gzip -5 --verbose 0.0% -- /var/log/ceph/ceph-client.admin.43817.log -- replaced with /var/log/ceph/ceph-client.admin.36942.log.gz 2024-09-04T17:50:00.919 INFO:teuthology.orchestra.run.smithi092.stderr: 2024-09-04T17:50:00.919 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.43141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37406.log 2024-09-04T17:50:00.919 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43141.log.gz 2024-09-04T17:50:00.919 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.43817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44023.log 2024-09-04T17:50:00.919 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43817.log.gz 2024-09-04T17:50:00.919 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.37406.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37406.log.gz 2024-09-04T17:50:00.919 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39311.log 2024-09-04T17:50:00.920 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.44023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43972.log 2024-09-04T17:50:00.920 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44023.log.gz 2024-09-04T17:50:00.920 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.39311.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39311.log.gz 2024-09-04T17:50:00.920 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39253.log 2024-09-04T17:50:00.926 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.43972.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43972.log.gz 2024-09-04T17:50:00.926 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45386.log 2024-09-04T17:50:00.926 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.39253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44580.log 2024-09-04T17:50:00.926 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39253.log.gz 2024-09-04T17:50:00.926 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.45386.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45386.log.gz 2024-09-04T17:50:00.926 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38799.log 2024-09-04T17:50:00.926 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.44580.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44580.log.gz 2024-09-04T17:50:00.926 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-04T17:50:00.926 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.38799.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.33988.log 2024-09-04T17:50:00.926 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38799.log.gz 2024-09-04T17:50:00.926 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44528.log 2024-09-04T17:50:00.926 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/objectstore_tool.33988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37068.log 2024-09-04T17:50:00.927 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.44528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36711.log 2024-09-04T17:50:00.927 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44528.log.gz 2024-09-04T17:50:00.927 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.37068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36983.log 2024-09-04T17:50:00.927 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37068.log.gz 2024-09-04T17:50:00.927 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.36711.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36711.log.gz 2024-09-04T17:50:00.927 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39087.log 2024-09-04T17:50:00.927 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.36983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34945.log 2024-09-04T17:50:00.927 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36983.log.gz 2024-09-04T17:50:00.927 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.39087.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39087.log.gz 2024-09-04T17:50:00.927 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43137.log 2024-09-04T17:50:00.927 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.34945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44785.log 2024-09-04T17:50:00.927 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34945.log.gz 2024-09-04T17:50:00.927 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.43137.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43137.log.gz -5 2024-09-04T17:50:00.928 INFO:teuthology.orchestra.run.smithi092.stderr: --verbose -- /var/log/ceph/ceph-client.admin.38287.log 2024-09-04T17:50:00.928 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.44785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37621.log 2024-09-04T17:50:00.928 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44785.log.gz 2024-09-04T17:50:00.928 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.38287.log: 93.4% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-04T17:50:00.928 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.41536.log 2024-09-04T17:50:00.928 INFO:teuthology.orchestra.run.smithi092.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38287.log.gz 2024-09-04T17:50:00.928 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.37621.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.32908.log 2024-09-04T17:50:00.928 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37621.log.gz 2024-09-04T17:50:00.936 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.41536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39429.log 2024-09-04T17:50:00.936 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41536.log.gz 2024-09-04T17:50:00.936 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/objectstore_tool.32908.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-04T17:50:00.936 INFO:teuthology.orchestra.run.smithi092.stderr: 95.2% -- replaced with /var/log/ceph/objectstore_tool.33988.log.gz 2024-09-04T17:50:00.936 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.39429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36634.log 2024-09-04T17:50:00.936 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39429.log.gz 2024-09-04T17:50:00.936 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41449.log 2024-09-04T17:50:00.936 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.36634.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36634.log.gz 2024-09-04T17:50:00.937 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36462.log 2024-09-04T17:50:00.937 INFO:teuthology.orchestra.run.smithi092.stderr: 94.8% -- replaced with /var/log/ceph/objectstore_tool.32908.log.gz 2024-09-04T17:50:00.937 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45880.log 2024-09-04T17:50:00.937 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.41449.log: /var/log/ceph/ceph-client.admin.36462.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41449.log.gz 2024-09-04T17:50:00.937 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31895.log 0.0% 2024-09-04T17:50:00.937 INFO:teuthology.orchestra.run.smithi092.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36462.log.gz 2024-09-04T17:50:00.937 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.45880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43079.log 2024-09-04T17:50:00.937 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45880.log.gz 2024-09-04T17:50:00.937 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44263.log 2024-09-04T17:50:00.937 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.31895.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31895.log.gz 2024-09-04T17:50:00.937 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.43079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42595.log 2024-09-04T17:50:00.937 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43079.log.gz 2024-09-04T17:50:00.937 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.44263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44987.log 2024-09-04T17:50:00.937 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44263.log.gz 2024-09-04T17:50:00.938 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35622.log 2024-09-04T17:50:00.938 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.42595.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42595.log.gz 2024-09-04T17:50:00.938 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.44987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32469.log 2024-09-04T17:50:00.938 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44987.log.gz 2024-09-04T17:50:00.938 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.35622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45438.log 2024-09-04T17:50:00.938 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35622.log.gz 2024-09-04T17:50:00.938 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39059.log 2024-09-04T17:50:00.938 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.32469.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32469.log.gz 2024-09-04T17:50:00.938 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.45438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41270.log 2024-09-04T17:50:00.938 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45438.log.gz 2024-09-04T17:50:00.938 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.39059.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.33157.log 2024-09-04T17:50:00.938 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39059.log.gz 2024-09-04T17:50:00.938 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41841.log 2024-09-04T17:50:00.938 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.41270.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41270.log.gz 2024-09-04T17:50:00.939 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/objectstore_tool.33157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43489.log 2024-09-04T17:50:00.939 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.41841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43281.log 2024-09-04T17:50:00.939 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41841.log.gz 2024-09-04T17:50:00.939 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.43489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45568.log 2024-09-04T17:50:00.939 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43489.log.gz 2024-09-04T17:50:00.939 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.43281.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43281.log.gz 2024-09-04T17:50:00.939 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36822.log 2024-09-04T17:50:00.939 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.45568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44784.log 2024-09-04T17:50:00.939 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45568.log.gz 2024-09-04T17:50:00.939 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32335.log 2024-09-04T17:50:00.939 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.36822.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36822.log.gz 2024-09-04T17:50:00.939 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.44784.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.44784.log.gz -5 2024-09-04T17:50:00.939 INFO:teuthology.orchestra.run.smithi092.stderr: --verbose -- /var/log/ceph/ceph-client.admin.34587.log 2024-09-04T17:50:00.939 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.32335.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/ceph-client.admin.32335.log.gz -- 2024-09-04T17:50:00.939 INFO:teuthology.orchestra.run.smithi092.stderr: /var/log/ceph/ceph-client.admin.44159.log 2024-09-04T17:50:00.940 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43867.log 2024-09-04T17:50:00.952 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.34587.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34587.log.gz 2024-09-04T17:50:00.952 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.44159.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44159.log.gz 2024-09-04T17:50:00.952 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45802.log 2024-09-04T17:50:00.952 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.43867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31893.log 2024-09-04T17:50:00.952 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43867.log.gz 2024-09-04T17:50:00.953 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.45802.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.45802.log.gz 2024-09-04T17:50:00.953 INFO:teuthology.orchestra.run.smithi092.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.44990.log 2024-09-04T17:50:00.953 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.31893.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31893.log.gz 2024-09-04T17:50:00.953 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38011.log 2024-09-04T17:50:00.953 INFO:teuthology.orchestra.run.smithi092.stderr: 95.2% -- replaced with /var/log/ceph/objectstore_tool.33157.log.gz 2024-09-04T17:50:00.953 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.44990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45542.log 2024-09-04T17:50:00.953 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44990.log.gz 2024-09-04T17:50:00.953 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.38011.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-04T17:50:00.953 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38011.log.gz 2024-09-04T17:50:00.953 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.45542.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45542.log.gz 2024-09-04T17:50:00.953 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44788.log 2024-09-04T17:50:00.953 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40696.log 2024-09-04T17:50:00.954 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.44788.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44788.log.gz 2024-09-04T17:50:00.954 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38999.log 2024-09-04T17:50:00.954 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.40696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.32501.log 2024-09-04T17:50:00.954 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40696.log.gz 2024-09-04T17:50:00.954 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.38999.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38999.log.gz 2024-09-04T17:50:00.954 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39579.log 2024-09-04T17:50:00.954 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.0.32501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37172.log 2024-09-04T17:50:00.954 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.39579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41697.log 2024-09-04T17:50:00.954 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39579.log.gz 2024-09-04T17:50:00.954 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.37172.log: 95.0% -- replaced with /var/log/ceph/ceph-client.0.32501.log.gz 2024-09-04T17:50:00.954 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.37172.log.gz 2024-09-04T17:50:00.954 INFO:teuthology.orchestra.run.smithi092.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.39142.log 2024-09-04T17:50:00.954 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.41697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31889.log 2024-09-04T17:50:00.954 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41697.log.gz 2024-09-04T17:50:00.955 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.39142.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/ceph-client.admin.39142.log.gz 2024-09-04T17:50:00.955 INFO:teuthology.orchestra.run.smithi092.stderr: -- /var/log/ceph/ceph-client.admin.42133.log 2024-09-04T17:50:00.955 INFO:teuthology.orchestra.run.smithi092.stderr:gzip/var/log/ceph/ceph-client.admin.31889.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.44995.log 2024-09-04T17:50:00.955 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31889.log.gz 2024-09-04T17:50:00.955 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.42133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35051.log 2024-09-04T17:50:00.955 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42133.log.gz 2024-09-04T17:50:00.955 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.44995.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44995.log.gz 2024-09-04T17:50:00.955 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41163.log 2024-09-04T17:50:00.955 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37716.log 2024-09-04T17:50:00.955 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.35051.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35051.log.gz 2024-09-04T17:50:00.955 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.41163.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41163.log.gz 2024-09-04T17:50:00.955 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45206.log 2024-09-04T17:50:00.955 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.37716.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-04T17:50:00.956 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37716.log.gz 2024-09-04T17:50:00.956 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.45206.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45206.log.gz 2024-09-04T17:50:00.956 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38209.log 2024-09-04T17:50:00.956 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44710.log 2024-09-04T17:50:00.956 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.38209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45672.log 2024-09-04T17:50:00.956 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38209.log.gz 2024-09-04T17:50:00.956 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.44710.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44710.log.gz 2024-09-04T17:50:00.956 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35405.log 2024-09-04T17:50:00.956 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42101.log 2024-09-04T17:50:00.956 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.45672.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45672.log.gz 2024-09-04T17:50:00.956 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.35405.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35405.log.gz 2024-09-04T17:50:00.956 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46010.log 2024-09-04T17:50:00.956 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44163.log 2024-09-04T17:50:00.956 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.42101.log: /var/log/ceph/ceph-client.admin.46010.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42101.log.gz 2024-09-04T17:50:00.957 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46010.log.gz 2024-09-04T17:50:00.957 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37537.log 2024-09-04T17:50:00.957 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41411.log 2024-09-04T17:50:00.957 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.44163.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44163.log.gz 2024-09-04T17:50:00.957 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.37537.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37537.log.gz 2024-09-04T17:50:00.957 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37144.log 2024-09-04T17:50:00.957 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42334.log 2024-09-04T17:50:00.957 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.41411.log: /var/log/ceph/ceph-client.admin.37144.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41411.log.gz 2024-09-04T17:50:00.957 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37144.log.gz 2024-09-04T17:50:00.957 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40469.log 2024-09-04T17:50:00.957 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44980.log 2024-09-04T17:50:00.957 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.42334.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42334.log.gz 2024-09-04T17:50:00.957 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.40469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42969.log 2024-09-04T17:50:00.958 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40469.log.gz 2024-09-04T17:50:00.958 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.44980.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44980.log.gz 2024-09-04T17:50:00.958 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44398.log 2024-09-04T17:50:00.958 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45750.log 2024-09-04T17:50:00.958 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.42969.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42969.log.gz 2024-09-04T17:50:00.958 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.44398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43648.log 2024-09-04T17:50:00.958 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44398.log.gz 2024-09-04T17:50:00.958 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.45750.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45750.log.gz 2024-09-04T17:50:00.958 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32283.log 2024-09-04T17:50:00.958 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32614.log 2024-09-04T17:50:00.973 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.32457.log 2024-09-04T17:50:00.973 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26392.log 2024-09-04T17:50:00.973 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/objectstore_tool.32457.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-04T17:50:00.973 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.26392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26296.log 2024-09-04T17:50:00.973 INFO:teuthology.orchestra.run.smithi190.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26392.log.gz 2024-09-04T17:50:00.973 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-04T17:50:00.973 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.26296.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26296.log.gz 2024-09-04T17:50:00.973 INFO:teuthology.orchestra.run.smithi190.stderr: 94.8% -- replaced with /var/log/ceph/objectstore_tool.32457.log.gz 2024-09-04T17:50:00.973 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.34411.log 2024-09-04T17:50:00.973 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26320.log 2024-09-04T17:50:00.974 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/objectstore_tool.34411.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-04T17:50:00.974 INFO:teuthology.orchestra.run.smithi190.stderr:gzip/var/log/ceph/ceph-client.admin.26320.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.26368.log 2024-09-04T17:50:00.974 INFO:teuthology.orchestra.run.smithi190.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26320.log.gz 2024-09-04T17:50:00.974 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-04T17:50:00.974 INFO:teuthology.orchestra.run.smithi190.stderr: 94.5% -- replaced with /var/log/ceph/objectstore_tool.34411.log.gz 2024-09-04T17:50:00.974 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.26368.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-04T17:50:00.974 INFO:teuthology.orchestra.run.smithi190.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26368.log.gz 2024-09-04T17:50:00.974 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.33433.log 2024-09-04T17:50:00.974 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.34821.log 2024-09-04T17:50:00.974 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/objectstore_tool.33433.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.32653.log 2024-09-04T17:50:00.974 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/objectstore_tool.34821.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-04T17:50:00.974 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/objectstore_tool.32653.log: 85.9% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-04T17:50:00.974 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.32261.log 2024-09-04T17:50:00.984 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph.audit.log: 89.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-04T17:50:00.984 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26416.log 2024-09-04T17:50:00.984 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/objectstore_tool.32261.log: 95.0% -- replaced with /var/log/ceph/objectstore_tool.33433.log.gz 2024-09-04T17:50:00.984 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.34623.log 2024-09-04T17:50:00.984 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.26416.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26416.log.gz 2024-09-04T17:50:00.985 INFO:teuthology.orchestra.run.smithi190.stderr: 95.0% -- replaced with /var/log/ceph/objectstore_tool.32653.log.gzgzip -5 --verbose -- /var/log/ceph/objectstore_tool.35601.log 2024-09-04T17:50:00.985 INFO:teuthology.orchestra.run.smithi190.stderr: 94.8% -- replaced with /var/log/ceph/objectstore_tool.32261.log.gz 2024-09-04T17:50:00.985 INFO:teuthology.orchestra.run.smithi190.stderr: 2024-09-04T17:50:00.985 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/objectstore_tool.34623.log: 95.1% -- replaced with /var/log/ceph/objectstore_tool.34821.log.gz 2024-09-04T17:50:00.985 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-04T17:50:00.985 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/objectstore_tool.35601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26344.log 2024-09-04T17:50:00.985 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-04T17:50:00.985 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.26344.log: 94.5% -- replaced with /var/log/ceph/objectstore_tool.34623.log.gz 2024-09-04T17:50:00.985 INFO:teuthology.orchestra.run.smithi190.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26344.log.gz 2024-09-04T17:50:00.985 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26248.log 2024-09-04T17:50:00.985 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26272.log 2024-09-04T17:50:00.985 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.26248.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26248.log.gz 2024-09-04T17:50:00.985 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.26272.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26272.log.gz 2024-09-04T17:50:00.990 INFO:teuthology.orchestra.run.smithi190.stderr: 95.0% -- replaced with /var/log/ceph/objectstore_tool.35601.log.gz 2024-09-04T17:50:01.002 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.43648.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43648.log.gz 2024-09-04T17:50:01.002 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.32283.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32283.log.gz 2024-09-04T17:50:01.002 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32078.log 2024-09-04T17:50:01.002 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42882.log 2024-09-04T17:50:01.002 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.32614.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32614.log.gz 2024-09-04T17:50:01.002 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.32078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36604.log 2024-09-04T17:50:01.002 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32078.log.gz 2024-09-04T17:50:01.003 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.42882.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.42882.log.gz 2024-09-04T17:50:01.003 INFO:teuthology.orchestra.run.smithi092.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.43938.log 2024-09-04T17:50:01.003 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26286.log 2024-09-04T17:50:01.003 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.36604.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36604.log.gz 2024-09-04T17:50:01.003 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.43938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37872.log 2024-09-04T17:50:01.003 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43938.log.gz 2024-09-04T17:50:01.003 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.26286.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26286.log.gz 2024-09-04T17:50:01.003 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32510.log 2024-09-04T17:50:01.003 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32102.log 2024-09-04T17:50:01.003 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.37872.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37872.log.gz 2024-09-04T17:50:01.003 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.32510.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32510.log.gz 2024-09-04T17:50:01.003 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43968.log 2024-09-04T17:50:01.003 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44606.log 2024-09-04T17:50:01.003 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.32102.log: 0.0%/var/log/ceph/ceph-client.admin.43968.log: -- replaced with /var/log/ceph/ceph-client.admin.32102.log.gz 2024-09-04T17:50:01.004 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43968.log.gz 2024-09-04T17:50:01.004 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.32711.log 2024-09-04T17:50:01.004 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40607.log 2024-09-04T17:50:01.004 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.44606.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44606.log.gz 2024-09-04T17:50:01.004 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/objectstore_tool.32711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44320.log 2024-09-04T17:50:01.004 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.40607.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40607.log.gz -5 2024-09-04T17:50:01.004 INFO:teuthology.orchestra.run.smithi092.stderr: --verbose -- /var/log/ceph/ceph-client.admin.31763.log 2024-09-04T17:50:01.004 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.44320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38941.log 2024-09-04T17:50:01.004 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44320.log.gz 2024-09-04T17:50:01.004 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.31763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37262.log 2024-09-04T17:50:01.004 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31763.log.gz 2024-09-04T17:50:01.004 INFO:teuthology.orchestra.run.smithi092.stderr: 94.8% -- replaced with /var/log/ceph/objectstore_tool.32711.log.gz 2024-09-04T17:50:01.005 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.38941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32387.log 2024-09-04T17:50:01.005 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38941.log.gz 2024-09-04T17:50:01.005 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31693.log 2024-09-04T17:50:01.005 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.37262.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37262.log.gz 2024-09-04T17:50:01.005 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.32387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43392.log 2024-09-04T17:50:01.005 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32387.log.gz 2024-09-04T17:50:01.005 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.31693.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31693.log.gz 2024-09-04T17:50:01.005 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43421.log 2024-09-04T17:50:01.005 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.43392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35125.log 2024-09-04T17:50:01.005 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43392.log.gz 2024-09-04T17:50:01.005 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.43421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41025.log 2024-09-04T17:50:01.005 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43421.log.gz 2024-09-04T17:50:01.006 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.35125.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35125.log.gz 2024-09-04T17:50:01.006 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-04T17:50:01.006 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41021.log 2024-09-04T17:50:01.006 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.41025.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41025.log.gz 2024-09-04T17:50:01.006 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41483.log 2024-09-04T17:50:01.006 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.41021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38843.log 2024-09-04T17:50:01.006 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41021.log.gz 2024-09-04T17:50:01.006 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.41483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38007.log 2024-09-04T17:50:01.006 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41483.log.gz 2024-09-04T17:50:01.006 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.38843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42662.log 2024-09-04T17:50:01.006 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38843.log.gz 2024-09-04T17:50:01.006 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.38007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44985.log 2024-09-04T17:50:01.006 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38007.log.gz 2024-09-04T17:50:01.007 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.42662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42430.log 2024-09-04T17:50:01.007 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42662.log.gz 2024-09-04T17:50:01.007 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.44985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36744.log 2024-09-04T17:50:01.007 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44985.log.gz 2024-09-04T17:50:01.007 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.42430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45490.log 2024-09-04T17:50:01.007 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42430.log.gz 2024-09-04T17:50:01.007 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.36744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45932.log 2024-09-04T17:50:01.007 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36744.log.gz 2024-09-04T17:50:01.007 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.45490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32439.log 2024-09-04T17:50:01.007 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45490.log.gz 2024-09-04T17:50:01.007 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.45932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35215.log 2024-09-04T17:50:01.007 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45932.log.gz 2024-09-04T17:50:01.007 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.32439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44215.log 2024-09-04T17:50:01.007 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32439.log.gz 2024-09-04T17:50:01.008 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45205.log 2024-09-04T17:50:01.008 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.35215.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35215.log.gz 2024-09-04T17:50:01.008 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.44215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40015.log 2024-09-04T17:50:01.008 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44215.log.gz 2024-09-04T17:50:01.008 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45698.log 2024-09-04T17:50:01.008 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.45205.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45205.log.gz 2024-09-04T17:50:01.008 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.40015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42541.log 2024-09-04T17:50:01.008 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40015.log.gz 2024-09-04T17:50:01.008 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.45698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44658.log 2024-09-04T17:50:01.008 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45698.log.gz 2024-09-04T17:50:01.008 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.42541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38261.log 2024-09-04T17:50:01.008 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42541.log.gz 2024-09-04T17:50:01.009 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32091.log 2024-09-04T17:50:01.009 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.44658.log: /var/log/ceph/ceph-client.admin.38261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41871.log 2024-09-04T17:50:01.009 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44658.log.gz 2024-09-04T17:50:01.009 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38261.log.gz 2024-09-04T17:50:01.009 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.32091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35571.log 2024-09-04T17:50:01.009 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32091.log.gz 2024-09-04T17:50:01.009 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5/var/log/ceph/ceph-client.admin.41871.log: --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-04T17:50:01.009 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41871.log.gz 2024-09-04T17:50:01.009 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.35571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45984.log 2024-09-04T17:50:01.009 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35571.log.gz 2024-09-04T17:50:01.009 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41221.log 2024-09-04T17:50:01.009 INFO:teuthology.orchestra.run.smithi092.stderr:gzip/var/log/ceph/ceph-client.admin.45984.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.34583.log 2024-09-04T17:50:01.009 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45984.log.gz 2024-09-04T17:50:01.010 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.41221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37512.log 2024-09-04T17:50:01.010 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41221.log.gz 2024-09-04T17:50:01.010 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45828.log 2024-09-04T17:50:01.010 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.34583.log: /var/log/ceph/ceph-client.admin.37512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31789.log 2024-09-04T17:50:01.010 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34583.log.gz 2024-09-04T17:50:01.010 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0%/var/log/ceph/ceph-client.admin.45828.log: -- replaced with /var/log/ceph/ceph-client.admin.37512.log.gz 2024-09-04T17:50:01.010 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45828.log.gz 2024-09-04T17:50:01.010 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32687.log 2024-09-04T17:50:01.010 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37829.log 2024-09-04T17:50:01.010 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.31789.log: /var/log/ceph/ceph-client.admin.32687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35008.log 2024-09-04T17:50:01.010 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31789.log.gz 2024-09-04T17:50:01.010 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0%/var/log/ceph/ceph-client.admin.37829.log: -- replaced with /var/log/ceph/ceph-client.admin.32687.log.gz 2024-09-04T17:50:01.010 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37829.log.gz 2024-09-04T17:50:01.011 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.38312.log 2024-09-04T17:50:01.011 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.35008.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35008.log.gz 2024-09-04T17:50:01.011 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41378.log 2024-09-04T17:50:01.011 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41529.log 2024-09-04T17:50:01.011 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/objectstore_tool.38312.log: /var/log/ceph/ceph-client.admin.41378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40961.log 2024-09-04T17:50:01.011 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41378.log.gz 2024-09-04T17:50:01.011 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.41529.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41529.log.gz 2024-09-04T17:50:01.011 INFO:teuthology.orchestra.run.smithi092.stderr: 94.8% -- replaced with /var/log/ceph/objectstore_tool.38312.log.gz 2024-09-04T17:50:01.011 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35052.log 2024-09-04T17:50:01.011 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45776.log 2024-09-04T17:50:01.011 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.40961.log: /var/log/ceph/ceph-client.admin.35052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31044.log 2024-09-04T17:50:01.011 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40961.log.gz 2024-09-04T17:50:01.011 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35052.log.gz/var/log/ceph/ceph-client.admin.45776.log: 2024-09-04T17:50:01.012 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45776.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37373.log 2024-09-04T17:50:01.012 INFO:teuthology.orchestra.run.smithi092.stderr: 2024-09-04T17:50:01.012 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40823.log 2024-09-04T17:50:01.012 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.31044.log: /var/log/ceph/ceph-client.admin.37373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43707.log 2024-09-04T17:50:01.012 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31044.log.gz 2024-09-04T17:50:01.012 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37373.log.gz 2024-09-04T17:50:01.012 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.40823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40566.log 2024-09-04T17:50:01.012 INFO:teuthology.orchestra.run.smithi092.stderr: 27.5% -- replaced with /var/log/ceph/ceph-client.admin.40823.log.gz 2024-09-04T17:50:01.012 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.43707.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43707.log.gz 2024-09-04T17:50:01.012 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38112.log 2024-09-04T17:50:01.012 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31891.log 2024-09-04T17:50:01.012 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.40566.log: /var/log/ceph/ceph-client.admin.38112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43543.log 2024-09-04T17:50:01.012 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40566.log.gz 2024-09-04T17:50:01.012 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38112.log.gz 2024-09-04T17:50:01.012 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.31891.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31891.log.gz 2024-09-04T17:50:01.013 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31886.log 2024-09-04T17:50:01.013 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35469.log 2024-09-04T17:50:01.013 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.43543.log: /var/log/ceph/ceph-client.admin.31886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34844.log 2024-09-04T17:50:01.013 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43543.log.gz 2024-09-04T17:50:01.013 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31886.log.gz 2024-09-04T17:50:01.013 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.35469.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.35469.log.gz 2024-09-04T17:50:01.013 INFO:teuthology.orchestra.run.smithi092.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.36458.log 2024-09-04T17:50:01.013 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38733.log 2024-09-04T17:50:01.013 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.34844.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34844.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42475.log 2024-09-04T17:50:01.013 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.36458.log: 2024-09-04T17:50:01.013 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36458.log.gz 2024-09-04T17:50:01.013 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.38733.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38733.log.gz 2024-09-04T17:50:01.013 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44372.log 2024-09-04T17:50:01.014 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.42475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32087.log 2024-09-04T17:50:01.014 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42475.log.gz 2024-09-04T17:50:01.014 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.44372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41743.log 2024-09-04T17:50:01.014 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44372.log.gz 2024-09-04T17:50:01.014 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.32087.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32087.log.gz 2024-09-04T17:50:01.014 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41815.log 2024-09-04T17:50:01.014 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45906.log 2024-09-04T17:50:01.014 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.41743.log: /var/log/ceph/ceph-client.admin.41815.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41903.log 2024-09-04T17:50:01.014 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41743.log.gz 2024-09-04T17:50:01.014 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41815.log.gz 2024-09-04T17:50:01.014 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.45906.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45906.log.gz 2024-09-04T17:50:01.014 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32095.log 2024-09-04T17:50:01.014 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44782.log 2024-09-04T17:50:01.014 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.41903.log: /var/log/ceph/ceph-client.admin.32095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44424.log 2024-09-04T17:50:01.015 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41903.log.gz 2024-09-04T17:50:01.015 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32095.log.gz 2024-09-04T17:50:01.015 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.44782.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44782.log.gz 2024-09-04T17:50:01.015 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42539.log 2024-09-04T17:50:01.015 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43682.log 2024-09-04T17:50:01.015 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.44424.log: /var/log/ceph/ceph-client.admin.42539.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-04T17:50:01.015 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44424.log.gz 2024-09-04T17:50:01.015 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42539.log.gz 2024-09-04T17:50:01.015 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.43682.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43682.log.gz 2024-09-04T17:50:01.015 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45464.log 2024-09-04T17:50:01.015 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42283.log 2024-09-04T17:50:01.015 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.45464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40526.log 2024-09-04T17:50:01.015 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45464.log.gz 2024-09-04T17:50:01.016 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.42283.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42283.log.gz 2024-09-04T17:50:01.016 INFO:teuthology.orchestra.run.smithi092.stderr: 92.1% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-04T17:50:01.016 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34984.log 2024-09-04T17:50:01.016 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38695.log 2024-09-04T17:50:01.016 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.40526.log: /var/log/ceph/ceph-client.admin.34984.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34984.log.gz -5 2024-09-04T17:50:01.016 INFO:teuthology.orchestra.run.smithi092.stderr: --verbose -- /var/log/ceph/ceph-client.admin.38159.log 2024-09-04T17:50:01.016 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40526.log.gz 2024-09-04T17:50:01.016 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.38695.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38695.log.gz 2024-09-04T17:50:01.016 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37938.log 2024-09-04T17:50:01.016 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39222.log 2024-09-04T17:50:01.016 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.38159.log: /var/log/ceph/ceph-client.admin.37938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45646.log 2024-09-04T17:50:01.016 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38159.log.gz 2024-09-04T17:50:01.016 INFO:teuthology.orchestra.run.smithi092.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37938.log.gz 2024-09-04T17:50:01.017 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.39222.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39222.log.gz 2024-09-04T17:50:01.017 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37229.log 2024-09-04T17:50:01.017 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33556.log 2024-09-04T17:50:01.017 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.45646.log: /var/log/ceph/ceph-client.admin.37229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38868.log 2024-09-04T17:50:01.017 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45646.log.gz 2024-09-04T17:50:01.017 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37229.log.gz 2024-09-04T17:50:01.017 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.33556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34891.log 2024-09-04T17:50:01.017 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33556.log.gz 2024-09-04T17:50:01.017 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31892.log 2024-09-04T17:50:01.017 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.38868.log: /var/log/ceph/ceph-client.admin.34891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45204.log 2024-09-04T17:50:01.017 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38868.log.gz 2024-09-04T17:50:01.017 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34891.log.gz 2024-09-04T17:50:01.017 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.31892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45185.log 2024-09-04T17:50:01.017 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31892.log.gz 2024-09-04T17:50:01.018 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-04T17:50:01.018 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.45204.log: /var/log/ceph/ceph-client.admin.45185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38916.log 2024-09-04T17:50:01.018 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45204.log.gz 2024-09-04T17:50:01.018 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45185.log.gz 2024-09-04T17:50:01.018 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43331.log 2024-09-04T17:50:01.031 INFO:teuthology.orchestra.run.smithi190.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-04T17:50:01.039 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42075.log 2024-09-04T17:50:01.039 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.38916.log: /var/log/ceph/ceph-client.admin.43331.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38916.log.gz 2024-09-04T17:50:01.039 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43331.log.gz 2024-09-04T17:50:01.039 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42666.log 2024-09-04T17:50:01.039 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36559.log 2024-09-04T17:50:01.039 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.42075.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42075.log.gz 2024-09-04T17:50:01.040 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.42666.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42666.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35239.log 2024-09-04T17:50:01.040 INFO:teuthology.orchestra.run.smithi092.stderr: 2024-09-04T17:50:01.040 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.36559.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36559.log.gz 2024-09-04T17:50:01.040 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40671.log 2024-09-04T17:50:01.040 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.35239.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35239.log.gz 2024-09-04T17:50:01.040 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31894.log 2024-09-04T17:50:01.040 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.40671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45958.log 2024-09-04T17:50:01.040 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40671.log.gz 2024-09-04T17:50:01.040 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.31894.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31894.log.gz 2024-09-04T17:50:01.040 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36794.log 2024-09-04T17:50:01.040 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45620.log 2024-09-04T17:50:01.040 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.45958.log: /var/log/ceph/ceph-client.admin.36794.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45958.log.gz 2024-09-04T17:50:01.040 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36794.log.gz 2024-09-04T17:50:01.041 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38557.log 2024-09-04T17:50:01.041 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32084.log 2024-09-04T17:50:01.041 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.38557.log: /var/log/ceph/ceph-client.admin.45620.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45620.log.gz 2024-09-04T17:50:01.041 INFO:teuthology.orchestra.run.smithi092.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38557.log.gz 2024-09-04T17:50:01.041 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35312.log 2024-09-04T17:50:01.041 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44450.log 2024-09-04T17:50:01.041 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.32084.log: /var/log/ceph/ceph-client.admin.35312.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32084.log.gz 2024-09-04T17:50:01.041 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35312.log.gz 2024-09-04T17:50:01.041 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44998.log 2024-09-04T17:50:01.041 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43034.log 2024-09-04T17:50:01.041 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.44450.log: /var/log/ceph/ceph-client.admin.44998.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44450.log.gz 2024-09-04T17:50:01.041 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44998.log.gz 2024-09-04T17:50:01.041 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44214.log 2024-09-04T17:50:01.042 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44789.log 2024-09-04T17:50:01.042 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.43034.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43034.log.gz 2024-09-04T17:50:01.042 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.44214.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44214.log.gzgzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-04T17:50:01.042 INFO:teuthology.orchestra.run.smithi092.stderr: 2024-09-04T17:50:01.042 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.44789.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44789.log.gz 2024-09-04T17:50:01.042 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37613.log 2024-09-04T17:50:01.042 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43751.log 2024-09-04T17:50:01.042 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-client.admin.37613.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37613.log.gz 2024-09-04T17:50:01.042 INFO:teuthology.orchestra.run.smithi092.stderr: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-04T17:50:01.042 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45412.log 2024-09-04T17:50:01.042 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42796.log 2024-09-04T17:50:01.042 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.43751.log: /var/log/ceph/ceph-client.admin.45412.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43751.log.gz 2024-09-04T17:50:01.042 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45412.log.gz 2024-09-04T17:50:01.043 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32309.log 2024-09-04T17:50:01.043 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42389.log 2024-09-04T17:50:01.043 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.42796.log: /var/log/ceph/ceph-client.admin.32309.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42796.log.gz 2024-09-04T17:50:01.043 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32309.log.gz 2024-09-04T17:50:01.043 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38581.log 2024-09-04T17:50:01.043 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43445.log 2024-09-04T17:50:01.043 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.42389.log: /var/log/ceph/ceph-client.admin.38581.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42389.log.gz 2024-09-04T17:50:01.043 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38581.log.gz 2024-09-04T17:50:01.043 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38796.log 2024-09-04T17:50:01.043 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.43445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43587.log 2024-09-04T17:50:01.043 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0%/var/log/ceph/ceph-client.admin.38796.log: -- replaced with /var/log/ceph/ceph-client.admin.43445.log.gz 2024-09-04T17:50:01.043 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38796.log.gz 2024-09-04T17:50:01.043 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37646.log 2024-09-04T17:50:01.043 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.43587.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43587.log.gz 2024-09-04T17:50:01.044 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35518.log 2024-09-04T17:50:01.044 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42198.log 2024-09-04T17:50:01.044 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.37646.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37646.log.gz 2024-09-04T17:50:01.044 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.35518.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35518.log.gz 2024-09-04T17:50:01.044 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40932.log 2024-09-04T17:50:01.044 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.42198.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42198.log.gz 2024-09-04T17:50:01.044 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43539.log 2024-09-04T17:50:01.044 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43240.log 2024-09-04T17:50:01.044 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.40932.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40932.log.gz 2024-09-04T17:50:01.044 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.43539.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43539.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44554.log 2024-09-04T17:50:01.044 INFO:teuthology.orchestra.run.smithi092.stderr: 2024-09-04T17:50:01.044 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.43240.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43240.log.gz 2024-09-04T17:50:01.044 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37449.log 2024-09-04T17:50:01.045 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45854.log 2024-09-04T17:50:01.045 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.44554.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44554.log.gz 2024-09-04T17:50:01.045 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.37449.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37449.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35348.log 2024-09-04T17:50:01.045 INFO:teuthology.orchestra.run.smithi092.stderr: 2024-09-04T17:50:01.045 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.45854.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45854.log.gz 2024-09-04T17:50:01.045 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44790.log 2024-09-04T17:50:01.045 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45201.log 2024-09-04T17:50:01.045 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.35348.log: /var/log/ceph/ceph-client.admin.44790.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35348.log.gz 2024-09-04T17:50:01.045 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44790.log.gz 2024-09-04T17:50:01.045 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40780.log 2024-09-04T17:50:01.045 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44736.log 2024-09-04T17:50:01.045 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.45201.log: /var/log/ceph/ceph-client.admin.40780.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40780.log.gz -- replaced with /var/log/ceph/ceph-client.admin.45201.log.gz 2024-09-04T17:50:01.045 INFO:teuthology.orchestra.run.smithi092.stderr: 2024-09-04T17:50:01.045 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44476.log 2024-09-04T17:50:01.045 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37334.log 2024-09-04T17:50:01.046 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.44736.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44736.log.gz 2024-09-04T17:50:01.046 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.44476.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44476.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31815.log 2024-09-04T17:50:01.046 INFO:teuthology.orchestra.run.smithi092.stderr: 2024-09-04T17:50:01.046 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.37334.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37334.log.gz 2024-09-04T17:50:01.046 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39532.log 2024-09-04T17:50:01.046 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26528.log 2024-09-04T17:50:01.046 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.31815.log: /var/log/ceph/ceph-client.admin.39532.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31815.log.gz 2024-09-04T17:50:01.046 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39532.log.gz 2024-09-04T17:50:01.046 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40857.log 2024-09-04T17:50:01.046 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32939.log 2024-09-04T17:50:01.046 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.26528.log: /var/log/ceph/ceph-client.admin.40857.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26528.log.gz 2024-09-04T17:50:01.046 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40857.log.gz 2024-09-04T17:50:01.046 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44970.log 2024-09-04T17:50:01.047 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45178.log 2024-09-04T17:50:01.047 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.32939.log: /var/log/ceph/ceph-client.admin.44970.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32939.log.gz 2024-09-04T17:50:01.047 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44970.log.gz 2024-09-04T17:50:01.047 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44502.log 2024-09-04T17:50:01.047 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39353.log 2024-09-04T17:50:01.047 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.45178.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45178.log.gz 2024-09-04T17:50:01.047 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.44502.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44502.log.gz 2024-09-04T17:50:01.047 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39465.log 2024-09-04T17:50:01.047 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.39353.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39353.log.gz 2024-09-04T17:50:01.047 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42753.log 2024-09-04T17:50:01.047 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44781.log 2024-09-04T17:50:01.047 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.39465.log: /var/log/ceph/ceph-client.admin.42753.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39465.log.gz 2024-09-04T17:50:01.047 INFO:teuthology.orchestra.run.smithi092.stderr: 28.5% -- replaced with /var/log/ceph/ceph-client.admin.42753.log.gz 2024-09-04T17:50:01.047 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31883.log 2024-09-04T17:50:01.048 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.44781.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44781.log.gz 2024-09-04T17:50:01.048 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37775.log 2024-09-04T17:50:01.060 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32361.log 2024-09-04T17:50:01.060 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.31883.log: /var/log/ceph/ceph-client.admin.37775.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31883.log.gz 2024-09-04T17:50:01.061 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37775.log.gz 2024-09-04T17:50:01.061 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45594.log 2024-09-04T17:50:01.061 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45516.log 2024-09-04T17:50:01.061 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.32361.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32361.log.gz 2024-09-04T17:50:01.061 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.45594.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37876.log 2024-09-04T17:50:01.061 INFO:teuthology.orchestra.run.smithi092.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45594.log.gz 2024-09-04T17:50:01.061 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.45516.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45516.log.gz 2024-09-04T17:50:01.061 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31841.log 2024-09-04T17:50:01.061 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31718.log 2024-09-04T17:50:01.061 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.37876.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37876.log.gz 2024-09-04T17:50:01.061 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.31841.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31841.log.gzgzip 2024-09-04T17:50:01.061 INFO:teuthology.orchestra.run.smithi092.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.44977.log 2024-09-04T17:50:01.061 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.31718.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31718.log.gz 2024-09-04T17:50:01.062 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44786.log 2024-09-04T17:50:01.062 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42935.log 2024-09-04T17:50:01.062 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.44977.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44977.log.gz 2024-09-04T17:50:01.062 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.44786.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44786.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44684.log 2024-09-04T17:50:01.062 INFO:teuthology.orchestra.run.smithi092.stderr: 2024-09-04T17:50:01.062 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.42935.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42935.log.gz 2024-09-04T17:50:01.062 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38649.log 2024-09-04T17:50:01.062 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41638.log 2024-09-04T17:50:01.062 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.44684.log: /var/log/ceph/ceph-client.admin.38649.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44684.log.gz 2024-09-04T17:50:01.062 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38649.log.gz 2024-09-04T17:50:01.062 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26433.log 2024-09-04T17:50:01.062 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45724.log 2024-09-04T17:50:01.062 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.41638.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41638.log.gz 2024-09-04T17:50:01.062 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.26433.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26433.log.gz 2024-09-04T17:50:01.063 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41984.log 2024-09-04T17:50:01.063 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.45724.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45724.log.gz 2024-09-04T17:50:01.063 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37043.log 2024-09-04T17:50:01.063 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43745.log 2024-09-04T17:50:01.063 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.41984.log: /var/log/ceph/ceph-client.admin.37043.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41984.log.gz 2024-09-04T17:50:01.063 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37043.log.gz 2024-09-04T17:50:01.063 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41672.log 2024-09-04T17:50:01.063 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41740.log 2024-09-04T17:50:01.063 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.43745.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43745.log.gz 2024-09-04T17:50:01.063 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.41672.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41672.log.gz 2024-09-04T17:50:01.063 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42252.log 2024-09-04T17:50:01.063 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.41740.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41740.log.gz 2024-09-04T17:50:01.063 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41114.log 2024-09-04T17:50:01.064 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35159.log 2024-09-04T17:50:01.064 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.42252.log: /var/log/ceph/ceph-client.admin.41114.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41114.log.gz -- replaced with /var/log/ceph/ceph-client.admin.42252.log.gz 2024-09-04T17:50:01.064 INFO:teuthology.orchestra.run.smithi092.stderr: 2024-09-04T17:50:01.064 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42163.log 2024-09-04T17:50:01.064 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42010.log 2024-09-04T17:50:01.064 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.35159.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35159.log.gz 2024-09-04T17:50:01.064 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.42163.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42163.log.gz 2024-09-04T17:50:01.064 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41308.log 2024-09-04T17:50:01.064 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.42010.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42010.log.gz 2024-09-04T17:50:01.064 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44346.log 2024-09-04T17:50:01.064 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46034.log 2024-09-04T17:50:01.064 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.41308.log: /var/log/ceph/ceph-client.admin.44346.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41308.log.gz 2024-09-04T17:50:01.064 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44346.log.gz 2024-09-04T17:50:01.065 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44294.log 2024-09-04T17:50:01.065 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32413.log 2024-09-04T17:50:01.065 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.46034.log: /var/log/ceph/ceph-client.admin.44294.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46034.log.gz 2024-09-04T17:50:01.065 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44294.log.gz 2024-09-04T17:50:01.065 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36901.log 2024-09-04T17:50:01.065 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32103.log 2024-09-04T17:50:01.065 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.32413.log: /var/log/ceph/ceph-client.admin.36901.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32413.log.gz 2024-09-04T17:50:01.065 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36901.log.gz 2024-09-04T17:50:01.065 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42800.log 2024-09-04T17:50:01.065 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44632.log 2024-09-04T17:50:01.065 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.32103.log: /var/log/ceph/ceph-client.admin.42800.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32103.log.gz 2024-09-04T17:50:01.065 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42800.log.gz 2024-09-04T17:50:01.065 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43863.log 2024-09-04T17:50:01.065 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32511.log 2024-09-04T17:50:01.066 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.44632.log: /var/log/ceph/ceph-client.admin.43863.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44632.log.gz 2024-09-04T17:50:01.066 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43863.log.gz 2024-09-04T17:50:01.066 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35189.log 2024-09-04T17:50:01.070 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41936.log 2024-09-04T17:50:01.070 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.32511.log: /var/log/ceph/ceph-client.admin.35189.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35189.log.gz -- replaced with /var/log/ceph/ceph-client.admin.32511.log.gz 2024-09-04T17:50:01.070 INFO:teuthology.orchestra.run.smithi092.stderr: 2024-09-04T17:50:01.070 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45186.log 2024-09-04T17:50:01.070 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32088.log 2024-09-04T17:50:01.071 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.41936.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41936.log.gz 2024-09-04T17:50:01.071 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.45186.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45186.log.gz 2024-09-04T17:50:01.071 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31743.log 2024-09-04T17:50:01.071 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.32088.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32088.log.gz 2024-09-04T17:50:01.071 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45203.log 2024-09-04T17:50:01.071 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44089.log 2024-09-04T17:50:01.071 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.31743.log: /var/log/ceph/ceph-client.admin.45203.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31743.log.gz 2024-09-04T17:50:01.071 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45203.log.gz 2024-09-04T17:50:01.071 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33950.log 2024-09-04T17:50:01.071 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38892.log 2024-09-04T17:50:01.071 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.44089.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44089.log.gz 2024-09-04T17:50:01.071 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.33950.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33950.log.gzgzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.21491.log 2024-09-04T17:50:01.071 INFO:teuthology.orchestra.run.smithi092.stderr: 2024-09-04T17:50:01.072 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.38892.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38892.log.gz 2024-09-04T17:50:01.072 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39189.log 2024-09-04T17:50:01.072 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41537.log 2024-09-04T17:50:01.072 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph.tmp-client.admin.21491.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.21491.log.gz 2024-09-04T17:50:01.072 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.39189.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39189.log.gz 2024-09-04T17:50:01.072 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.41537.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41537.log.gz 2024-09-04T17:50:01.141 INFO:teuthology.orchestra.run.smithi190.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-04T17:50:01.181 INFO:teuthology.orchestra.run.smithi092.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-04T17:50:01.289 INFO:teuthology.orchestra.run.smithi092.stderr: 90.6% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-04T17:50:01.712 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-09-04T17:50:02.056 INFO:teuthology.orchestra.run.smithi190.stderr: 95.0% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-04T17:50:02.384 INFO:teuthology.orchestra.run.smithi092.stderr: 95.0% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-04T17:50:02.406 INFO:teuthology.orchestra.run.smithi092.stderr: 94.6% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-04T17:50:02.927 INFO:teuthology.orchestra.run.smithi092.stderr: 95.7% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-04T17:50:03.070 INFO:teuthology.orchestra.run.smithi190.stderr: 94.8% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-04T17:50:03.894 INFO:teuthology.orchestra.run.smithi092.stderr: 94.8% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-04T17:50:03.895 INFO:teuthology.orchestra.run.smithi092.stderr: 2024-09-04T17:50:03.895 INFO:teuthology.orchestra.run.smithi092.stderr:real 0m3.003s 2024-09-04T17:50:03.895 INFO:teuthology.orchestra.run.smithi092.stderr:user 0m8.273s 2024-09-04T17:50:03.895 INFO:teuthology.orchestra.run.smithi092.stderr:sys 0m0.401s 2024-09-04T17:50:04.406 INFO:teuthology.orchestra.run.smithi190.stderr: 95.3% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-04T17:50:04.442 INFO:teuthology.orchestra.run.smithi190.stderr: 95.4% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-04T17:50:04.444 INFO:teuthology.orchestra.run.smithi190.stderr: 2024-09-04T17:50:04.444 INFO:teuthology.orchestra.run.smithi190.stderr:real 0m3.506s 2024-09-04T17:50:04.444 INFO:teuthology.orchestra.run.smithi190.stderr:user 0m10.004s 2024-09-04T17:50:04.444 INFO:teuthology.orchestra.run.smithi190.stderr:sys 0m0.415s 2024-09-04T17:50:04.445 INFO:tasks.ceph:Archiving logs... 2024-09-04T17:50:04.445 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/log/ceph to /home/teuthworker/archive/dis-2024-09-04_17:03:08-rbd-wip-dis-testing-distro-default-smithi/7889281/remote/smithi092/log 2024-09-04T17:50:04.446 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-04T17:50:05.562 DEBUG:teuthology.misc:Transferring archived files from smithi190:/var/log/ceph to /home/teuthworker/archive/dis-2024-09-04_17:03:08-rbd-wip-dis-testing-distro-default-smithi/7889281/remote/smithi190/log 2024-09-04T17:50:05.563 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-04T17:50:06.697 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-04T17:50:06.706 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-04T17:50:06.707 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-04T17:50:06.710 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-04T17:50:06.785 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-04T17:50:06.794 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-04T17:50:06.794 DEBUG:teuthology.orchestra.run.smithi092:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-04T17:50:06.797 DEBUG:teuthology.orchestra.run.smithi190:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-04T17:50:06.805 INFO:teuthology.orchestra.run.smithi092.stdout: remote refid st t when poll reach delay offset jitter 2024-09-04T17:50:06.805 INFO:teuthology.orchestra.run.smithi092.stdout:============================================================================== 2024-09-04T17:50:06.806 INFO:teuthology.orchestra.run.smithi092.stdout:*hv01.front.sepi 67.205.162.81 3 u 35 64 177 0.099 -1.192 1.267 2024-09-04T17:50:06.806 INFO:teuthology.orchestra.run.smithi092.stdout:+hv02.front.sepi 96.245.170.99 3 u 31 64 177 0.090 -0.230 1.232 2024-09-04T17:50:06.806 INFO:teuthology.orchestra.run.smithi092.stdout:+hv03.front.sepi 74.6.168.72 3 u 33 64 177 0.104 -1.588 1.331 2024-09-04T17:50:06.806 INFO:teuthology.orchestra.run.smithi092.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-04T17:50:06.809 INFO:teuthology.orchestra.run.smithi190.stdout: remote refid st t when poll reach delay offset jitter 2024-09-04T17:50:06.809 INFO:teuthology.orchestra.run.smithi190.stdout:============================================================================== 2024-09-04T17:50:06.810 INFO:teuthology.orchestra.run.smithi190.stdout:*hv01.front.sepi 67.205.162.81 3 u 33 64 177 0.130 -1.558 0.907 2024-09-04T17:50:06.810 INFO:teuthology.orchestra.run.smithi190.stdout:+hv02.front.sepi 96.245.170.99 3 u 41 64 177 0.125 -0.457 0.959 2024-09-04T17:50:06.810 INFO:teuthology.orchestra.run.smithi190.stdout:+hv03.front.sepi 74.6.168.72 3 u 34 64 177 0.146 -1.918 0.910 2024-09-04T17:50:06.810 INFO:teuthology.orchestra.run.smithi190.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-04T17:50:06.811 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-04T17:50:06.819 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-04T17:50:06.821 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-04T17:50:06.830 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-04T17:50:06.899 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-04T17:50:06.941 INFO:teuthology.task.internal:Duration was 883.495991 seconds 2024-09-04T17:50:06.941 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-04T17:50:06.953 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-04T17:50:06.953 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-04T17:50:06.956 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-04T17:50:06.993 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-04T17:50:06.993 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi092.front.sepia.ceph.com 2024-09-04T17:50:06.993 DEBUG:teuthology.orchestra.run.smithi092:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-04T17:50:07.042 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi190.front.sepia.ceph.com 2024-09-04T17:50:07.042 DEBUG:teuthology.orchestra.run.smithi190:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-04T17:50:07.052 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-04T17:50:07.052 DEBUG:teuthology.orchestra.run.smithi092:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-04T17:50:07.086 DEBUG:teuthology.orchestra.run.smithi190:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-04T17:50:07.122 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-04T17:50:07.122 DEBUG:teuthology.orchestra.run.smithi092:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-04T17:50:07.158 DEBUG:teuthology.orchestra.run.smithi190:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-04T17:50:07.232 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-04T17:50:07.245 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-04T17:50:07.245 DEBUG:teuthology.orchestra.run.smithi092:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-04T17:50:07.257 DEBUG:teuthology.orchestra.run.smithi190:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-04T17:50:07.270 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-04T17:50:07.280 DEBUG:teuthology.orchestra.run.smithi092:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-04T17:50:07.302 DEBUG:teuthology.orchestra.run.smithi190:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-04T17:50:07.312 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern = core 2024-09-04T17:50:07.323 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern = core 2024-09-04T17:50:07.414 DEBUG:teuthology.orchestra.run.smithi092:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-04T17:50:07.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-04T17:50:07.441 DEBUG:teuthology.orchestra.run.smithi190:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-04T17:50:07.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-04T17:50:07.461 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-04T17:50:07.474 INFO:teuthology.task.internal:Transferring archived files... 2024-09-04T17:50:07.475 DEBUG:teuthology.misc:Transferring archived files from smithi092:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/dis-2024-09-04_17:03:08-rbd-wip-dis-testing-distro-default-smithi/7889281/remote/smithi092 2024-09-04T17:50:07.475 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-04T17:50:07.512 DEBUG:teuthology.misc:Transferring archived files from smithi190:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/dis-2024-09-04_17:03:08-rbd-wip-dis-testing-distro-default-smithi/7889281/remote/smithi190 2024-09-04T17:50:07.512 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-04T17:50:07.533 INFO:teuthology.task.internal:Removing archive directory... 2024-09-04T17:50:07.534 DEBUG:teuthology.orchestra.run.smithi092:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-04T17:50:07.550 DEBUG:teuthology.orchestra.run.smithi190:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-04T17:50:07.578 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-04T17:50:07.588 INFO:teuthology.task.internal:Not uploading archives. 2024-09-04T17:50:07.588 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-04T17:50:07.597 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-04T17:50:07.597 DEBUG:teuthology.orchestra.run.smithi092:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-04T17:50:07.600 DEBUG:teuthology.orchestra.run.smithi190:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-04T17:50:07.604 INFO:teuthology.orchestra.run.smithi092.stdout: 658623 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 4 17:50 /home/ubuntu/cephtest 2024-09-04T17:50:07.625 INFO:teuthology.orchestra.run.smithi190.stdout: 658462 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 4 17:50 /home/ubuntu/cephtest 2024-09-04T17:50:07.627 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-04T17:50:07.637 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-04T17:50:07.660 INFO:teuthology.run:Summary data: description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_fsx_rate_limit} duration: 883.495991230011 flavor: default owner: scheduled_dis@teuthology success: true 2024-09-04T17:50:07.660 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-04T17:50:07.744 INFO:teuthology.run:pass